Black Performance as Social Protest
All Projects
Virtual ProductionLED Volume & Media Server OperationTechnical Art2022-2025

Black
Performance
as Social Protest

School of Music Theater and Dance - UM, Georgetown University
2022-2025
Virtual Production Technical Artist & Developer
18 months

Work Type

Collaborative Academic Virtual Production, University of Michigan Center for Academic Innovation, in partnership with Georgetown University

Duration

Roughly 3 years overall (early 2022 to early 2025), including an 18 month hold while the university's own LED volume was being built. Active production and post-production spanned about 14 months.

The Purpose

Louise Toppin, Professor of Music at the University of Michigan, and Dr. Anita Gonzalez, Professor of Performing Arts and Black Studies at Georgetown University, had previously collaborated on an online Coursera course and wanted to add a 10 minute video exploring Black performance as a form of social protest. The film traces how song, spoken word, and resilience have served as tools of resistance and catalysts for change throughout American history, from slavery through emancipation to more recent movements such as Black Lives Matter. It was produced using virtual production methods on an LED volume at the University of Michigan.

Behind the Scenes - the opening scene in the Field

My Role & Contributions

I worked across the production as a technical artist, animator, developer, media server support, and LED stage operator. My responsibilities included scanning actors and building MetaHumans in their likeness, dressing them in period-accurate virtual costumes, optimizing them for real-time crowd performance, retargeting and cleaning up motion capture animation, and building the systems that let those crowds perform alongside live actors on stage. I was also one of two people on the team qualified to operate Disguise, the media server software running the LED volume, and served as the Disguise and Unreal Engine operator during rehearsals and all production days.

The team

Design Philosophy & Approach

This was a project where historical accuracy, artistic intent, and hard real-time performance constraints all had to hold together at once. Every digital human needed to convincingly represent a specific era (the early 1800s, the 1920s, and the 2010s) while running at a stable 72 frames per second on the LED wall, the frame rate needed to prevent visible tearing as live cameras moved during filming. Every pipeline decision, from costume construction to crowd optimization, was made in service of that balance.

Technologies Used

Unreal Engine (MetaHuman Creator, Mass AI, Mass Entity, ZoneGraph, Animation Blueprints, Sequencer, Movie Render Queue)

Disguise Designer media server

Blender

CLO 3D

Substance Painter

Rokoko AI motion capture

Mixamo

Adobe Premiere Pro

Scanning Actors & Building MetaHumans

In early 2022, while Unreal's MetaHuman Creator was still in beta and web-only, I helped run a photogrammetry-based scanning process, lighting the studio evenly and photographing actors from all angles. Those photos were used to generate 3D facial meshes, which were then used to produce a MetaHuman using the Mesh to MetaHuman feature. This process was used to build digital doubles of the real actors later in the production.

Digital Doubles with Metahumans
Digital Doubles using Metahumans

Period-Accurate Costuming Pipeline

I built a custom cross-software workflow to dress MetaHumans in historically accurate clothing across the three time periods, working from digital garments built in CLO 3D by a costume artist. I applied and adjusted textures and UVs in CLO to ensure correct placement. I also prepared some textures in Substance Painter. Then brought the garments from CLO into Blender, since no direct MetaHuman-to-garment workflow existed yet at the time. In Blender, I decimated the clothing to reduce polygon count, removed internal double-layered and non-front-facing geometry, and transferred UV weighting from the MetaHuman body to the costume. I repeated this process across three body sizes (small, medium, large) for both genders, then exported the finished costumes and MetaHumans into Unreal as matching skeletal meshes. Costumes were added to each MetaHuman blueprint as an attached skeletal mesh, weight-painted onto the body, with any body geometry hidden beneath clothing removed.

Preparing MetaHumans with period-accurate Costumes · CLO → Substance Painter → Blender → Unreal

Optimizing MetaHumans for Real-Time Crowds

MetaHuman crowds were needed in three scenes: seated directly behind live actors in an early 1800s church scene, walking behind live actors on the Edmund Pettus Bridge, and as a fully digital aerial crowd on Black Lives Matter Plaza in Washington, D.C. To hit the 72fps target needed for tear-free real-time playback on stage, I removed the facial skeletal mesh from background MetaHumans, replaced real-time cloth simulation with weight-painted clothing, removed hidden body geometry, instanced blueprints, set up LODs, and reduced draw calls by combining meshes and keeping every costume on a small set of shared materials with material instances for variation.

Motion Capture & Animation

I worked with a digital artist who captured motion data with a webcam through Rokoko's AI-based 2D motion capture software and exported the results as FBX. I imported these into Unreal, retargeted them from the Mixamo skeleton onto each MetaHuman body type, and cleaned up issues like root drift. I built animation blueprints with triggerable variables so crowd behavior could be controlled live: idle states like sitting, reading, fanning, and clapping for the church scene, and spline-based looping walk cycles for the bridge scene, with walking speed controlled by a real-time variable so the crowd's pace could be matched to the live actors during filming.

Crowd Simulation with Mass AI

For the fully digital Black Lives Matter Plaza sequence, I built a large-scale simulated protest crowd using Unreal's Mass AI and Mass Entity systems, populated with Mixamo characters rather than MetaHumans given the scale involved. I configured crowd behaviors through the Mass Entity configuration asset for collision avoidance and set up ZoneGraph for pathfinding and navigation. When rendering this wide aerial shot through Movie Render Queue, character shadows were initially culling out at low LOD levels, which I resolved by adjusting distance scaling, shadow culling parameters, and other render settings.

Crowd on BLM Plaza  - Rendered Image
Rendered Image of Crowd on the BLM Plaza.
Rendered Video - Crowd on the BLM Plaza.

Rendering & Post-Production

I rendered the fully virtual sequences using Sequencer, animating cine camera moves in direct collaboration with the director to plan framing and shot composition. Crowd animations were finalized with seamless loops, rendered as high-resolution image sequences through Movie Render Queue, then encoded into ProRes and handed off to the editor, matched carefully to the lighting captured during live production. In some sequences, such as a window-opening shot, a MetaHuman built to resemble the real actor was animated to seamlessly continue into footage of that same actor shot live.

Setting up animations to render out the field scene
Rendered Video - People working in the fields.
Rendered Video - People working in the fields- bird's eye view.
Rendered Video - Crowd crossing the Edmund Pettus Bridge.
Animated and rendered women opening the window
Rendered video transitioning into the real footage
Rendered video transitioning into the real footage

On-Set Production Support

As Disguise and Unreal Engine operator, I loaded the correct scene for each setup, matched virtual lighting to set lighting, and set virtual camera angles and focus values on direction from the director and virtual production producer. I also made real-time adjustments on stage, including repositioning MetaHumans, triggering crowd animations, and adjusting crowd walking speed to stay in sync with the live actors during filming.

First Iteration (2022)

Work began with sound recording, storyboarding, and scripting, alongside the start of actor scanning and MetaHuman creation. Production paused when the originally planned LED stage became unavailable, ahead of the university's own volume being completed at the Center for Academic Innovation.

Second Iteration (2024 to 2025)

Once the University of Michigan's own LED volume was ready, the team, myself included, was trained on real-time 3D content delivery and Disguise Designer operation, and production resumed. Because the final MetaHuman crowds were still being built and optimized, I provided the actors and directors with earlier previz shots and faster, lower-fidelity versions of the digital crowd, using already-optimized Mixamo skeletons, so they had something concrete to rehearse against on stage early on. This let the cast and crew get comfortable performing alongside virtual humans well before the final crowds were production-ready, while I continued refining the MetaHuman crowds and environments in parallel. Full costume pipelines, optimized crowds, live rehearsals, and three days of filming followed, and the video was released as an updated Coursera course module in early 2025.

Early iteration of the Crowd to help actors rehearse on stage
Early iteration of the Crowd to help actors rehearse on stage
Final scene with accurately dressed digital crowd and actors
Final scene with accurately dressed digital crowd and actors

Project Challenges & Solutions

Challenge: No existing pipeline to bring period-accurate garments onto MetaHumans across multiple body types.

Solution: Built a custom CLO 3D to Blender to Unreal workflow covering three body sizes and both genders.

Challenge: Meeting a 72fps real-time target with dozens of high-fidelity digital humans on stage.

Solution: Systematically optimized geometry, materials, and draw calls without sacrificing costume and character fidelity.

Challenge: Actors and directors needed to rehearse with virtual crowds before the final MetaHuman crowds were finished.

Solution: Delivered early previz and faster Mixamo-based crowd iterations so the cast could rehearse and build comfort with the technology while the final crowds were completed in parallel.

Challenge: Keeping digital crowd performance in sync with live actors during filming.

Solution: Built live-adjustable animation blueprints with real-time speed and trigger controls operable from the stage.

Skills Demonstrated

Technical art and character pipeline development, real-time performance optimization, crowd simulation and AI behavior systems, virtual production and LED volume operation, motion capture cleanup and retargeting, real-time cinematography, iterative prototyping under production deadlines, live on-set technical operation, and cross-disciplinary collaboration across a large production team.

Outcomes & Impact

The finished 10 minute film was released as an updated module in an existing Coursera course, reaching a global learner audience with a virtual production built around African American history and protest. To my knowledge, this was among the first academic virtual production projects to run MetaHuman crowds at this scale in real time on an LED volume, roughly 20 unique digital humans in the church scene and 40 on the Edmund Pettus Bridge sequence, each with individualized costume variation by design and color.

Recognition & Usage

Produced through a formal partnership between the University of Michigan and Georgetown University and distributed as official content within an existing Coursera course. The workflows and pipelines developed on this project also helped establish real-time virtual production capability at the University of Michigan Center for Academic Innovation for future use.

Next Project
Ford Nuclear Reactor VR
→