Disney Animation’s fall 2022 release is Strange World, which is the studio’s 61st animated feature, and third original story in as many years. Strange World takes us to the land of Avalonia, a realm surrounded by impenetrable mountains and home to a society that blends elements of early 20th century pulp fiction, steampunk, and environmental solarpunk. The core story of Strange World revolves around father-son relationships and is exactly the type of story that Disney Animation excels at: something personal and relatable but set in a fantastic world we’ve never seen before. That “never seen before” aspect made my two years of working on rendering technology for Strange World an interesting time indeed!

In my writeups about our films, two recurring themes always are: 1. with each film, we build upon advancements made and lessons learned from the previous film, and 2. one of the greatest advantages that having in-house tools gives us is the ability to customize and build exactly what each film’s story and art direction requires. Strange World’s production exemplifies both of these themes; so much of what we had to do on Strange World builds upon things we learned from and developed for previous films that I’m not entirely sure we would have been able to make Strange World a few years ago, and much of what we learned we have only been able to apply as effectively as we have because we have the ability to extend and improve our own tools.

As an example: a large part of Strange World takes place on the airship Venture, which from a production pipeline perspective has to function as both a set/environment in which characters move and interact, and as a sort of character of its own as it moves around in the larger surrounding environment. In CG production pipelines, various pipeline optimizations are often built around the reasonable assumption that sets are relatively static; sets typically don’t need complex animation rigs and can be used as a stationary frame of reference for all kinds of different things. The Venture, of course, breaks all of these expectations. Disney Animation had to deal with this type of scenario before on Moana, where much of the movie is set on a canoe out at sea, so handling the “sets as characters” challenge wasn’t new. Instead of having to solve this problem from scratch, our artists and TDs were able to build on top of what they had learned before to enable the Venture to be a far more complex “set as a character” than anything we had done before. In fact, the Venture isn’t the only case of this type of challenge in Strange World! Huge parts of Strange World follow this “set as a character” pattern; entire chunks of terrain get up and walk around in this movie! All of these complex sets were made possible by advancements [Vo et al. 2023] in our USD based pipeline, which in turn built upon all of the lessons learned [Miller et al. 2022] from our previous pipeline.

Things got even more complex once crowds were brought into the mix too. Strange World has some of the most massive crowd simulation ever made by Disney Animation [Devlin et al. 2023], and these huge crowds had to interact with the Venture and complex terrain. One of the main tools our crowds team used to guide giant swarms of creatures traveling through Strange World’s massive environments and around the Venture originated as a tool made for a single character on Frozen 2, and had to be turbocharged to massive scales to go from handling the requirements for one character to handling thousands upon thousands of creatures [Lin et al. 2023]. Challenges involving simulating collisions in huge crowds like this, along with similar challenges in hair simulation, helped inspire further research work [Zhang et al. 2023] for future films as well.

Once the story moves into the subterranean world, the environment of the film ratchets up in production complexity on multiple different axes. Essentially every single surface in the subterranean world has significant subsurface scattering since everything is made up of organic gummy materials, and of course all of the giant crowds also all have subsurface scattering. Many of our previous films already were beginning to push the use of subsurface scattering in environments for things like plants and plastics and other materials, all thanks to the work that the rendering team put into making path traced subsurface scattering efficient and controllable enough for large-scale production usage [Chiang et al. 2016], but Strange World saw the widest usage of subsurface scattering in environments yet, by far.

Everything we’ve learned about controlling subsurface scattering also proved to be extremely important for creating the look for the Splat character, who is essentially a giant immune cell. Splat wound up requiring a unique custom one-off shader with custom functionality in Disney’s Hyperion Renderer combining subsurface scattering, a custom faux volumetric emission technique, our multiple-scattering sheen solution [Zeltner et al. 2022], and more in order to achieve the target art-direction in a single render pass [Litaker et al. 2023]. Splat’s challenges weren’t limited to just rendering though; rigging and animating Splat also required novel solutions in order to handle how varied and multi-purpose Splat’s limbs are [Black and Pederson 2023]. Splat’s rig was only made possible through a combination of new novel techniques and a decade of experience and continuous improvement in Disney Animation’s DRig modular rig building system [Smith et al. 2012].

Splat wasn’t the only character that provided interesting technical challenges though; in fact, our entire character asset workflow got an upgrade on Strange World. Our standard character asset workflow saw three major improvements on Strange World: eyes, skin, and curves.

Strange World’s character art direction called for eyes to use a bit of a different look from Disney Animation’s usual style; eyes on Strange World have more of an oblong oval shape. Over the past several shows, we introduced a new eye shading model that incorporates manifold next event estimation for physically accurate iris caustics and limbal arcs [Chiang et al. 2018]; one of my smaller projects on Strange World was to help work out the minor modifications to this system that were required to support Strange World’s eye shapes.

For skin shading, Strange World uses the same fully path traced subsurface scattering approach [Chiang et al. 2016] (as opposed to older diffusion-based approaches [Burley 2015]) that we have now used for all of our movies over the past few years. However, Strange World has one of the most diverse casts of any of our recent films in terms of skin tones, and our lighting and look dev artists took special care to make sure all of the different skin types were depicted accurately and beautifully. Doing so required rebuilding our entire skin material from the ground up and radically rethinking our entire approach to lighting characters to better handle contrasting skin tones and high specularity skin [Khoo et al. 2023].

Previously on Encanto, our look artists started to replace triangle mesh-based geometric representations for cloth with curve-based fiber level representations [Velasquez et al. 2022]. This authoring approach was pushed to new limits on Strange World, where curve-based garments were extended to incorporate custom weave patterns and widely varying fiber thicknesses ranging from fine threads to thick yarns [Lipson and Velasquez 2023]. Humans weren’t the only type of characters on Strange World to see upgraded curve geometry though; the Clade family’s lovable dog, Legend, also required upgraded curve grooming techniques to produce one of the most complex animal grooms the studio has ever made [Chun et al. 2023]. Of course all of these improved authoring techniques meant increased curve rendering complexity, but interestingly, we didn’t actually need to improve anything in the renderer to handle the increased curve rendering demand. After having spent many prior shows improving Hyperion’s ability to chew through vast geometric complexity, on Strange World we found that Hyperion was able to just handle all of the meshes and curves that we threw at it!

The hardest rendering challenge I worked on for Strange World was volume rendering. Strange World’s environments have some of the largest scale and most ambitious use of volumes in any of our films to date. Strange World extensively utilizes mist and atmospherics and low cloud cover to help convey a sense of mystery and to sell the sheer scale of the environments. Frozen 2 was the first movie that really extensively leveraged Hyperion’s modern volume rendering system (which we rewrote essentially from scratch during the early production of Ralph Breaks the Internet) and the first movie that introduced our modern volumes authoring workflow. This workflow, which is heavily based around quickly set-dressing atmospherics and clouds around environments by kitbashing together volumes from a large pre-made in-house library of VDBs, was further fleshed out on Raya and the Last Dragon and saw its largest and most complex usage yet on Strange World. Strange World also further extended our volume workflows with an evolved version [Navarro 2023] of the neural volume stylization tech we first introduced on Raya and the Last Dragon [Navarro and Rice 2021].

During Raya and the Last Dragon we consolidated various different experiments and techniques in our volume rendering system into a single unified volume integrator [Huang et al. 2022] that can efficiently handle every imaginable type of volume effect, so the challenge presented by Strange World’s volumes wasn’t so much light transport as it was simply a problem of efficiency at scale. When volumes are simultaneously highly detailed but also span kilometers of world space, massive memory usage becomes challenging, even with instancing. Also, super large and detailed volumes coverage means that average path length in volumes can get very long, exposing any potential performance issues in the volume integrator. A huge part of my time on Strange World was spent optimizing our volume integrator. There were no clever shortcuts or brilliant solutions here, just tons of profiling and careful analysis of the existing system architecture and hard low level optimization work.

We also noticed during Strange World that artists sometimes had to overauthor volume details in areas as a way to work around the lack of true procedural volumes evaluation support in our renderer. While Hyperion does support authoring procedural volumes, these procedural volumes are not actually evaluated at render time but instead are pre-evaluated and baked into a required underlying VDB grid at renderer startup. The reason for this limitation is fundamental to null collision-based volume rendering theory [Novák 2018]; null collision approaches only work if the bounding majorant (AKA max density) for all volumes in a region of space is known upfront. In theory we could just require artists to input a max density value that we would clamp all higher values down to, but such a value isn’t easy for artists to estimate in practice; too low of a value clamps away detail, while too high of value results in an overly loose bounding majorant, which in null collision theory-based volume rendering can result in significantly slower performance. Inspired by what we were seeing on Strange World, we kicked off a research project in collaboration with the Visual Computing Lab at Dartmouth College to solve this problem, with promising results [Misso et al. 2023]!

As usual, I’ve only written about the parts of making Strange World that I know a bit more about; hundreds of artists, TDs, and engineers worked to craft every frame of this movie and solve many many more problems. For the entire history of Disney Animation, one of the studio’s primary driving purposes has been to push the limits of animation as an art form, and Strange World is no exception to this rule. Strange World is the latest example of how each of our films builds upon what we’ve learned on previous films to push our filmmaking process forward, and as always, getting to be a part of this process is a lot of work but also a lot of fun!

Below are some frames from the strange but gorgeous world of Strange World, pulled from the Blu-ray and presented in semi-randomized order to prevent giving away too much of the story. Go see Strange World on the biggest screen you can find!

Here is the credits frame for the Hyperion team, which is listed as part of the larger Rendering & Visualization group at Disney Animation. In addition to the Hyperion team, this group also includes our sister render translation pipeline and interactive visualization teams:

All images in this post are courtesy of and the property of Walt Disney Animation Studios.

References

Cameron Black and Christoffer Pedersen. 2023. The Versatile Rigging of Splat in ‘Strange World’. In ACM SIGGRAPH 2023 Talks. Article 29.

Brent Burley. 2015. Extending the Disney BRDF to a BSDF with Integrated Subsurface Scattering. In ACM SIGGRAPH 2015 Course Notes: Physically Based Shading in Theory and Practice.

Matt Jen-Yuan Chiang, Peter Kutz, and Brent Burley. 2016. Practical and Controllable Subsurface Scattering for Production Path Tracing. In ACM SIGGRAPH 2016 Talks. Article 49.

Matt Jen-Yuan Chiang and Brent Burley. 2018. Plausible Iris Caustics and Limbal Arc Rendering. In ACM SIGGRAPH 2018 Talks. Article 15.

Courtney Chun, Jose Velasquez, and Haixiang Liu. 2023. Creating the Art-directed Groom for Legend in Disney’s Strange World. In ACM SIGGRAPH 2023 Talks. Article 7.

Nathan Devlin, Yasser Hamed, Alberto J Luceño Ros, Jeff Sullivan, and D’Lun Wong. 2023. Creating Creature Chaos: The Methods That Brought Crowds to the Forefront on Disney’s ‘Strange World’. In ACM SIGGRAPH 2023 Talks. Article 34.

Wei-Feng Wayne Huang, Peter Kutz, Yining Karl Li, and Matt Jen-Yuan Chiang. 2021. Unbiased Emission and Scattering Importance Sampling for Heterogeneous Volumes. In ACM SIGGRAPH 2021 Talks. Article 3.

Mason Khoo, Dan Lipson, and Jose Velasquez. 2023. Lighting and Look Dev for Skin Tones in Disney’s “Strange World”. In Proc. of Digital Production Symposium (DigiPro 2023). Article 5.

Andy Lin, Hannah Swan, Justin Walker, Cathy Lam, and Ricky Arietta. 2023. Swoop: Animating Characters Along a Path. In ACM SIGGRAPH 2023 Talks. Article 45.

Dan Lipson and Jose Velasquez. 2023. Creating Curve-based Garments With Custom Weave Patterns. In ACM SIGGRAPH 2023 Talks. Article 18.

Kendall Litaker, Brent Burley, Dan Lipson, and Mason Khoo. 2023. Splat: Developing a ‘Strange’ Shader. In ACM SIGGRAPH 2023 Talks. Article 28.

Tad Miller, Harmony M. Li, Neelima Karanam, Nadim Sinno, and Todd Scopio. 2022. Making Encanto with USD: Rebuilding a Production Pipeline Working from Home. In ACM SIGGRAPH 2022 Talks. Article 12.

Zackary Misso, Yining Karl Li, Brent Burley, Daniel Teece, and Wojciech Jarosz. 2023. Progressive Null-tracking for Volumetric Rendering. In Proc. of SIGGRAPH (SIGGRAPH 2023). Article 31.

Mike Navarro and Jacob Rice. 2021. Stylizing Volumes with Neural Networks. In ACM SIGGRAPH 2021 Talks. Article 54.

Mike Navarro. 2023. Diving Deeper Into Volume Style Transfer. In ACM SIGGRAPH 2023 Talks. Article 39.

Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo Methods for Volumetric Light Transport Simulation. Computer Graphics Forum (Proc. of Eurographics) 37, 2 (May 2018), 551-576.

Greg Smith, Mark McLaughlin, Andy Lin, Evan Goldberg, and Frank Hanner. 2012. DRig: An Artist-Friendly, Object-Oriented Approach to Rig Building. In ACM SIGGRAPH 2012 Talks. Article 18.

Jose Velasquez, Alexander Alvarado, Ying Liu, and Maryann Simmons. 2022. Embroidery and Cloth Fiber Workflows on Disney’s “Encanto”. In ACM SIGGRAPH 2022 Talks. Article 22.

Emily Vo, George Rieckenberg, and Ernest Petti. 2023. Honing USD: Lessons Learned and Workflow Enhancements at Walt Disney Animation Studios. In ACM SIGGRAPH 2023 Talks. Article 13.

Tizian Zeltner, Brent Burley, and Matt Jen-Yuan Chiang. 2022. Practical Multiple-Scattering Sheen Using Linearly Transformed Cosines. In ACM SIGGRAPH 2022 Talks. Article 7.

Paul Zhang, Zoë Marschner, Justin Solomon, and Rasmus Tamstorf. 2023. Sum-of-squares Collision Detection for Curved Shapes and Paths. In Proc. of SIGGRAPH (SIGGRAPH 2023). Article 76.

This year at SIGGRAPH 2022, Corey Butler, Brent Burley, Wei-Feng Wayne Huang, Benjamin Huang, and I have a talk that presents the technical and artistic challenges and solutions that went into creating the holographic look for Bruno’s visions in Encanto. In Encanto, Bruno is a character who has a magical gift of being able to see into the future, and the visions he sees of the future get crystalized into a sort of glassy emerald tablet with the vision embedded in the glassy surface with a holographic effect. Coming up with this unique look and an efficient and robust authoring workflow required a tight collaboration between visual development, lookdev, lighting, and the Hyperion rendering team to develop a custom solution in Disney’s Hyperion Renderer. On the artist side, Corey was the main lighter and Benjamin was the main lookdev artist for this project, while on the rendering team side, Wayne and I worked closely together to develop a series of prototype shaders that were instrumental in defining how the effect should look and then Brent came up with the implementation approach for the final production version of the shader. This project was a lot of fun to be a part of and in my opinion really demonstrates the benefits of having an in-house rendering team that works closely with and embedded within a production context.

Here is the paper abstract:

In Walt Disney Animation Studios’ “Encanto”, Mirabel discovers the remnants of her Uncle Bruno’s mysterious visions of the future. Developing the look and lighting for the emerald shards required close collaboration between our Visual Development, Look Development, Lighting, and Technology departments to create a holographic effect. With an innovative new teleporting holographic shader, we were able to bring a unique and unusual effect to the screen.

The paper and related materials can be found at:

• Project Page (Author’s Version and Supplementary Material)
• Official Print Version (ACM Library)

When Corey first came to the rendering team with the request for a more efficient way to create the hologram effect that lighting had prototyped using camera mapping, our initial instinct actually wasn’t to develop a new shader at all. Hyperion has an existing “hologram” shader that was developed for use on Big Hero 6 [Joseph et al. 2014], and our initial instinct was to tell Corey that they should use the hologram shader. The way the Big Hero 6 era hologram shader works is: upon hitting a surface that has the hologram shader applied, the ray is moved into a virtual space containing a bunch of imaginary parallel planes, with each plane textured with a 2D slice of a 3D interior. In some ways the hologram shader can be thought of as raymarching through a sparse volumetric representation of a 3D interior, but the sparse volumetric interior really is just a stack of 2D slices. This technique works really well for things like building interiors seen through glass windows. However, our artists… really dislike using the hologram shader, to put things lightly. The problem with the hologram shader is that setting up the 2D slices that are inputs to the shader is an incredibly annoying and difficult process, and since the 2D slice baker has to be run as an offline process before the shader can be authored and rendered, making changes and iterating on the contents of the hologram shader is a slow process. Furthermore, if the inside of the hologram shader has to be animated, the slice baker needs to be run for every frame. We were told in no uncertain terms that the hologram shader was likely more work to set up and iterate on than the already painful manual camera mapping approach that the artists had prototyped the effect with. This request also came to us fairly late in Encanto’s production schedule, so easy setup and fast iteration times along with an extremely accelerated development timeline were hard requirements for whatever approach we took.

Upon receiving this feedback, Wayne and I set out to prototype a version of the teleportation shader that Pixar came up with for the portals in Incredibles 2 [Coleman et al. 2014]. This process was a lot of fun; Wayne and I spent a few days rapidly iterating on several different ideas for both how to implement ray teleportation in Hyperion and on how the artist workflow and interface for this new teleportation system should work. At the same time that we were prototyping, we started giving test builds of our latest prototypes to Corey to try out, which produced a feedback loop where Corey would use our prototypes to further iterate on how the final effect would look and go back and forth with the movie’s production designer and we would use Corey’s feedback to further improve the prototype. One example of where our prototype directly informed the final look was in how the prophecies fade away towards the edges of the emerald tablet- Wayne and I threw in a feature where artists could use a map to paint in the ratio of teleportation effect versus normal surface BSDF that would be applied at each surface point, and this feature wound up driving the faded edges.

The key thing that made our new approach work better than the old hologram shader was in simplicity of setup. Instead of having to run a pre-bake process and then wire up a whole bunch of texture slices into the renderer, our new approach was designed so that all an artist had to do was set up the 3D geometry that they wanted to put inside of the hologram in a target space hidden somewhere in the overall scene (typically below the ground plane in a black box or something), and then select the geometry in the main scene that they wanted to act as the “entrance” portal, select the geometry in the target space that they wanted to act as the “exit” portal, and link the two using the teleportation shader. The renderer then did all of the rest of the work of figuring out how each point on the entrance portal corresponded to the surface of the exit portal, how transforms needed to be calculated, and so on and so forth. Multiple portal pairs could be set up in a single scene too, and the contents of a world seen through a portal could contain more portals, all of which was important because in the movie, Mirabel initially finds Bruno’s prophecy broken into shards, which had to be set up as a separate entrance portal per shard all into the same interior world. Since all of this just piggy-backed off of the normal way artists set up scenes, things like animation just worked out-of-the-box with no additional code or effort.

The last piece of the puzzle fell into place when Wayne and I discussed our progress with Brent. One of the big remaining challenges for us was that tracking correspondences between entrance and exit geometry and transforms was prone to easy breakage if input geometry wasn’t set up exactly the way we expected. At the time Brent was working on a new fracture-aware tessellation system for subdivision surfaces in Hyperion [Burley and Rodriguez 2022], and Brent quickly realized that the approach we were using for figuring out the transform from the entrance to the exit portal could be replaced with something he had already developed for the fracture-aware tessellation system. Specifically, the fracture-aware tessellation system has to be able to calculate correspondences between undeformed unfractured reference points and corresponding points in a deformed fractured fragment space; this is done using a best-fit process to find orthonormal transforms [Horn et al. 1998]. Brent realized that the problem we were trying to solve was actually the same problem he that he had already solved in the fracture system, so he took our latest prototype and reworked the internals to use the same best-fit orthonormal transform solution as in the fracturing system. With Brent’s improvements, we arrived at the final production version of the teleportation shader used on Encanto.

Going from the start of brainstorming and prototyping to delivering the final production version of the shader took us a little over a week, which anyone who has worked in an animation/VFX production setting before will know is very fast for a large new rendering feature. Working tightly with Corey and Benjamin to simultaneously iterate on the art and the software and inform each other was key to this project’s fast development time and key to achieving an amazing looking effect in the film. At Disney Animation, we have a mantra that goes “art challenges technology and technology inspires the art”- this project was a case that exemplifies how we carry out that mantra in real-world filmmaking and demonstrates the amazing results that come out of such a process. Bruno’s visions in Encanto are every bit a case where the artistic vision challenged us to develop new technology, and the process of iterating on the new technology between engineers and artists in turn informed the final artwork that made it into the movie; for me, projects like these are one of the things that makes Disney Animation such a fun and amazing place to be.

References

Brent Burley, David Adler, Matt Jen-Yuan Chiang, Hank Driskill, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. 2018. The Design and Evolution of Disney’s Hyperion Renderer. ACM Transactions on Graphics 37, 3 (Jul. 2018), Article 33.

Brent Burley and Francisco Rodriguez. 2022. Fracture-Aware Tessellation of Subdivision Surfaces. In ACM SIGGRAPH 2022 Talks. Article 10.

Patrick Coleman, Darwyn Peachey, Tom Nettleship, Ryusuke Villemin, and Tobin Jones. 2018. Into the Voyd: Teleportation of Light Transport in Incredibles 2. In Proc. of Digital Production Symposium (DigiPro 2018). Article 12.

Berthold K. P. Horn, Hugh M. Hilden, and Shahriar Negahdaripour. 1988. Close-Form Solution of Absolute Orientation using Orthonormal Matrices. Journal of the Optical Society of America A 5, 7 (Jul. 1988), 1127–1135.

Norman Moses Joseph, Brett Achorn, Sean D. Jenkins, and Hank Driskill. Visualizing Building Interiors Using Virtual Windows. In ACM SIGGRAPH Asia 2014 Technical Briefs. Article 18.

Along with Encanto’s release last fall, Disney Animation also released Far From The Tree, which is the studio’s newest short film. Far From The Tree was made in parallel with our last short, Us Again, but the two shorts have completely different visual styles. Both shorts were rendered using Disney’s Hyperion Renderer, but visually they stand as almost polar opposites. From a computer graphics perspective, Us Again is a showcase of the studio’s cutting-edge modern physically based rendering capabilities, while Far From The Tree is a showcase of the studio’s hand-drawn¹ inspired stylized rendering capabilities. The two shorts weren’t set up this way intentionally; in both cases, the visual style was chosen based entirely off of what was right for the story, but as a rendering engineer, these two shorts going into production at the same time was also a serendipitous opportunity to see how far we could push our visual filmmaking capabilities in two very different directions.

Normally in these posts, I write that the project was rendered entirely using Disney’s Hyperion Renderer, but that’s not the whole story here, and writing that would be a disservice to how this short was actually made. There is quite a lot of 3D CG in this short; all of the character animation was made using our normal 3D animation process, and all of the 3D stuff is in fact rendered entirely using Hyperion. However, the final look of the short involves extensive additional 2D work done on top of the base 3D renders. Far From The Tree’s look and production process is an extension of the hand-drawn inspired hybrid 2D-3D approach that Disney Animation previously used on Paperman , Feast, and several Short Circuit shorts. The base 3D renders are essentially unlit flat surface-color passes; in fact, if you look at just the raw beauty passes out of the renderer, they’re completely black! The real look of the film comes from extensive work done on top of the flat surface-color passes and a ton of AOVs output from the renderer. Much like on Paperman [Kahrs et al. 2012 ,Whited et al. 2012], Far From The Tree uses extensive linework drawn by hand using Meander on top of the rendered layers, and much like on Feast [Osborne and Staub 2014], final 2D lighting was created entirely in the composite in Nuke. The backgrounds are a combination of similar 2D-3D hybrid work to the foreground along with a lot of pure matte paintings, and much of the textural detail across the entire frame is similarly painted and projected in 2D, all of which were evolved at the studio on the Short Circuit experimental shorts program [Newfield and Staub 2020]. To further help enhance the 2D-3D hybrid look, much of the short is animated on twos and threes, to help match the motion of traditional hand-drawn animation.

On the Hyperion front, even though there’s no meaningful path tracing taking place, the renderer was still doing quite a lot on Far From The Tree! By the very nature of where they exist in production pipelines, production renderers tend to function as the final “source of truth” for what all 3D data in the production pipeline actually looks like, and this is definitely true of Hyperion. In our pipeline, Hyperion doesn’t just serve as the final frame renderer; it also acts as a powerful data visualization tool that is used in all departments upstream of lighting to generate authoritative visualizations of what our 3D data actually looks like. To help serve this function, Hyperion has incredibly extensive custom AOV capabilities. As far as I’m aware, I think Hyperion’s custom AOV capabilities are a fair bit more extensive than in even a lot of commercial renderers. Commercial renderers usually break down light transport into a bunch of individual components and expose these as AOVs (so, things like specular vs diffuse, direct vs indirect, individual BXDF lobes, etc), and also expose some basic geometric information as AOVs (things like position, normals, object IDs, cryptomattes, etc). Hyperion provides all of these as AOVs as one would expect, but beyond that, Hyperion allows essentially any signal or snippet of shader code from inside of the pattern generation part of the shading system to be routed into a custom AOV. Hyperion also goes even further and allows for custom AOVs to be driven directly using SeExpr [Disney Animation 2011]; on the first hit from the camera, user-authored SeExpr programs can be run and the result directly splatted to specified custom AOVs. For stylized rendering projects such as Far From The Tree’s hybrid 2D-3D, this capability is really powerful since it allows for artists to drive custom signals from 3D animated geometry, funnel those signals into 2D layers, and then use the result to drive any kind of effect they want in compositing. This system means that even without needing to run full light transport, on highly stylized projects such as Far From The Tree, Hyperion still plays a big role. Actually, on Far From The Tree, Hyperion is doing almost no light transport, but not exactly zero. There is one specific small detail that did make use of light transport: the full eye shader [Chiang & Burley 2018] was used to get enough input to drive the final flatter look of the stylized eyes.

The final result in Far From The Tree is a wonderful combination of Disney Animation’s modern state-of-the-art 3D CG capabilities and rich hand-drawn 2D legacy. Any randomly chosen frame from Far From The Tree basically looks exactly like the concept art used to art-direct the short, and that is both a very cool and technically astonishing feat; it’s a really beautiful film. I also just love the character design and character animation in this short; raccoons are an endless source of interesting animation, and the derpy birds are a fantastically fun piece of cartoon design.

Here are some frames from Far From The Tree from the Blu-ray, presented in no particular order. You can get Far From The Tree with a copy of Encanto on Blu-ray or digital, or watch it on Disney+; as always I recommend watching it on the biggest screen you can!

All images in this post are courtesy of and the property of Walt Disney Animation Studios.

References

Matt Jen-Yuan Chiang and Brent Burley. 2018. Plausible Iris Caustics and Limbal Arc Rendering. In ACM SIGGRAPH 2018 Talks. Article 15.

John Kahrs, Patrick Osborne, Amol Sathe, Jeff Turley, Brian Whited, and Darrin Butters. 2012. The Art and Science Behind Walt Disney Animation Studios’ “Paperman”. In ACM SIGGRAPH 2012 Production Sessions.

Jennifer Newfield and Josh Staub. 2020. How Short Circuit Experiments: Experimental Filmmaking at Walt Disney Animation Studios. In ACM SIGGRAPH 2020 Talks. Article 72.

Patrick Osborne and Josh Staub. 2014. Feast – A Look at Walt Disney Animation Studios’ Newest Short. In ACM SIGGRAPH 2014 Production Sessions.

Brian Whited, Eric Daniels, Michael Kaschalk, Patrick Osborne, and Kyle Odermatt. 2012. Computer-Assisted Animation of Line and Paint in Disney’s Paperman. In ACM SIGGRAPH 2012 Talks. Article 19.

Walt Disney Animation Studios. 2011. SeExpr.

Footnotes

¹ A lot of people use “2D animation” to describe Disney Animation’s work before the CG era, but at Disney Animation we prefer the term “hand-drawn animation”. I think the distinction is really important; a lot of modern 2D animation is made entirely digitally using rigged digital models/puppets similar to what we do for 3D animation. This is totally fine! However, Disney Animation’s previous traditional animation work was distinguished not just by being visually 2D, but really by the fact that everything was drawn by hand, either using pencil on paper or using a stylus and tablet when digital. keyboard_return

For the first time since 2016, Walt Disney Animation Studios is releasing not just one animated feature in a year, but two! The second Disney Animation release of 2021 is Encanto, which marks a major milestone as Disney Animation’s 60th animated feature film. Encanto is a musical set in Colombia about a girl named Mirabel and her family: the amazing, fantastical, magical Madrigals. I’m proud of every Disney Animation project that I’ve had the privilege to work on, but I have to admit that this year was something different and something very special to me, because this year we completed both Raya and the Last Dragon and Encanto, which are together two of my favorite Disney Animation projects so far. Earlier this year, I wrote about the amazing work that went into Raya and the Last Dragon and why I loved working on that project; with Encanto now in theaters, I now get to share why I’ve loved working on Encanto so much as well!

Disney Animation feature films take many years and hundreds of people to make, and often the film’s story can remain in a state of flux for much of the film’s production. All of the above isn’t unusual; large-scale creative endeavors like filmmaking often entail an extremely complex and challenging process. More often than not, a film requires time and many iterations to really find its voice and gain that spark that makes it a great film. Encanto, however, is a film that a lot of my coworkers and I realized was going to be really special very early on in production. Now obviously, that hunch didn’t mean that making Encanto was easy by any means; every film requires tons of hard work from the most amazing, inspiring, talented artists and engineers that I know. But, I think in the end, that initial hunch about Encanto was proven correct: the finished Encanto has a story that is bursting with warmth and meaning, has one of Disney Animation’s best main characters to date with a huge cast of charming supporting characters, has the most beautiful, magical animation and visuals we’ve ever done, and sets all of the above to a wonderful soundtrack with a bunch of catchy, really cleverly written new songs. Both the production process and final film for Encanto were a strong reminder for me of why I love working on Disney Animation films in the first place.

From a technical perspective, Encanto also represents something very special in the history of Disney Animation’s continual advancements in animation technology. To understand why this is, a very brief history review about Disney Animation’s modern production pipeline and toolset is helpful. In retrospect, Disney Animation’s 50th animated feature film, Tangled, was probably one of the most important films the studio has ever made from a technical perspective, because the production of Tangled required a near-total ground-up rebuild of the studio’s production pipeline and tools that wound up laying the technical foundations for Disney Animation’s modern era. While every film we’ve made since Tangled has seen us make enormous technical strides in a variety of eras, the starting point of the production pipeline we’ve used and evolved for every CG film up until Encanto were put into place during Tangled. The fact that Encanto is Disney Animation’s 60th animated feature film is therefore fitting; Encanto is the first film made using the USD-based successor [Miller et al. 2022] to the production pipeline that was first built for Tangled, and just like how Tangled laid the technical foundations for the subsequent ten films that followed, Encanto lays the technical foundations for many more future films to come! As presented in the USD Birds of a Feather session at SIGGRAPH 2021, this new production pipeline is built on the open-source Universal Scene Description project and brings massive upgrades to almost every piece of software and every custom tool that our artists use. Already Encanto’s upgraded production pipeline has enabled cool new tools that would have been much harder to create previously, such as a new command center tool that gives a bird’s eye unified overview of stats and data across the entire movie [Tennant et al. 2022]. An absolutely monumental amount of work was put into building a new USD-based world at Disney Animation, but I think the effort was extremely worthwhile: thanks to the work done on Encanto, Disney Animation is now well set up for another decade of technical innovation and another decade of pushing animation as a medium forward.

Moving to a new production pipeline meant also moving Disney’s Hyperion Renderer to work in the new production pipeline. To me, one of the biggest advantages of an in-house production renderer is the ability for the renderer development team to work extremely closely with other teams in the studio in an integrated fashion, and moving Hyperion to work well in the new USD-based world exemplifies just how important this collaboration is. We couldn’t have pulled off this effort without the huge amount of amazing work that engineers and TDs and artists from many other departments pitched in. However, having to move an existing renderer to a new pipeline isn’t the only impact on rendering that the new USD-based world has had. One of the most exciting things about the new pipeline is all of the new possibilities and capabilities that USD and Hydra unlocks; one of the biggest projects our rendering team worked on during Encanto’s production was a new, very exciting next-generation rendering project. I can’t talk too much about this project yet; all I can say is that we see it as a major step towards the future of rendering at Disney Animation, and that even in its initial deployment on Encanto, we’ve already seen huge fundamental improvements to how our lighters work every day. Hopefully we’ll be able to reveal more soon!

Of course, just because Encanto saw huge foundational changes to how we make movies doesn’t mean that there weren’t the usual fun and interesting show-specific challenges as well. Encanto presented many new, weird, fun problems for the rendering team to think about. Geometry fracturing was a major effect used extensively throughout Encanto, and in order to author and render fractured geometry as efficiently as possible, the rendering team had to devise some really clever new geometry-processing features in Hyperion [Burley and Rodriguez 2021]. Encanto’s cinematography direction called for a beautiful, really colorful look [Robinson 2022] that required pushing artistic controllability in our lighting capabilities even further, and to that end our team developed a bunch of cool new artistic control enhancements in Hyperion’s volume rendering and light shaping systems. One of my favorite show-specific challenges that I got to work on for Encanto was for the holographic effect in Bruno’s emerald crystal prophecies [Butler et al. 2021]. For a variety of reasons, the artists wanted this effect done completely in-render; coming up with an in-render solution required many iterations and prototypes and experiments carried out over several months through a close collaboration between a number of artists and TDs and the rendering team.

Encanto also saw continued advancements to Hyperion’s state-of-the-art deep-learning denoiser and stereo rendering solutions and saw continued advancements in Hyperion’s shading models and traversal system. A particularly notable advancement in our shading model is the addition of a new physically accurate practical multiple-scattering sheen lobe [Zeltner et al. 2022] to the Disney BSDF [Burley 2015]; I think this new sheen model is going to catch on widely in industry due to its combination of accuracy, ease of implementation, and performance, all of which improves greatly over previously existing sheen models [Conty and Kulla 2017]. These advancements helped us tackle many of the interesting complexity and scaling challenges that Encanto presented; effects like Isabella’s flowers and the glowing magical particles associated with the Madrigal family’s miracle pushed instancing counts to incredible new record levels [Finley et al. 2022], and for the first time ever on a Disney Animation film, we actually rendered some of the gorgeous costumes in the movie not as displaced triangle meshes with fuzz on top, but as actual woven curves at the thread-level [Velasquez et al. 2022]. The latter proved crucial to creating the chiffon and tulle in Isabella’s outfit and was a huge part in creating the look of Mirabel’s characteristic custom-embroidered skirt. My mind was thoroughly blown when I saw those renders for the first time; on every film, I’m constantly amazed and impressed by what our artists can do with the tools we provide them with.

Encanto also saw rendering features that we first developed for previous films pushed even further and used in interesting new ways. We first deployed a path guiding implementation [Müller et al. 2017] in Hyperion back on Frozen 2 [Müller 2019], but path guiding wound up not seeing too much use on Raya and the Last Dragon since Raya’s setting was mostly outdoors, and path guiding doesn’t help as much in direct-lighting dominant scenarios such as outdoor scenes. However, since a huge part of Encanto takes place inside of the magical Madrigal casita, indoor indirect illumination was a huge component of Encanto’s lighting. We found that path guiding provided enormous benefits to render times in many indoor scenes, and especially in settings like the Madrigal family’s kitchen at night, where lighting was almost entirely provided by outdoor light sources coming in through windows and from candles and stuff. I think this case was a great example of how we benefit from how closely our lighting artists and our rendering engineers work together on many shows over time; because we had all worked together on similar problems before, we all had shared experiences with past solutions that we were able to draw on together to quickly arrive at a common understanding of the new challenges on Encanto. Another good example of how this collaboration continues to pay dividends over time is in the choices of lens and bokeh effects that were used on Encanto. For Raya and the Last Dragon, we learned a lot about creating non-uniform bokeh and interesting lensing effects, and what we learned on Raya in turn helped further inform early cinematography and lensing experiments on Encanto. One more great example can be found in how eyes are shaded in Encanto- over our last few shows, we’ve been steadily moving our eye shading approach over to a next-generation shading model with advanced, physically accurate iris caustics [Chiang and Burley 2018] sampled using manifold next event estimation, and Encanto is the first show to use this new eye shading model on 100% of characters. The way we push technology further and further on each film isn’t limited to just rendering either; I mostly write about only lighting/shading/rendering topics here because that’s my home domain, but there are countless other examples in things like rigging, animation, simulation, procedural authoring, interactive visualization, and more about how we each film tech advances on top of the previous film. A great example published at SIGGRAPH 2022 is the new hair simulation technique that was developed for Mirabel’s bouncy curly hair [Liu 2022]; ever since Tangled, Disney Animation has been great at hair, but with each movie we still keep advancing what we can do!

In addition to all of the cool renderer development work that I usually do, I also got to take part in something a little bit different on Encanto. Every year, the lighting department brings on a handful of trainees, who are put through several months of in-studio “lighting school” to learn our tools and pipeline and approach to lighting before lighting real shots on the film itself. This year, I got to join in with the lighting trainees while they were going through lighting training; this experience wound up being one of my favorites from the past year. I think that having to sit down and actually learn and use software the same way that the users have to is an extraordinarily valuable experience for any software engineer that is building tools for users. Even though I’ve been working at Disney Animation for six years now, and even though I know the internals of how our renderer works extensively, I still learned a ton from having to actually use Hyperion to light shots and address notes from lighting supervisors and stuff! Encanto’s lighting style required really leaning on the tools that we have for art-directing and pushing and modifying fully physical lighting, which really changed my perspective on some of these tools. For most rendering engineers and researchers, features that allow for breaking purely physical light transport are often seen as annoying and difficult to implement but necessary concessions to the artists. Having now used these features in order to hit artistic notes on short time frames though, I now have a better understanding of just how critical a component these features can be in an artist’s toolbox. I owe a huge amount of thanks to Disney Animation’s technology department leadership and to the lighting department for having made this experience possible and for having strongly supported this entire “exchange program”; I’d strongly recommend that every rendering engineer should go try lighting some shots sometime!

Finally, here are some stills from the movie pulled from the Blu-ray, 100% created using Disney’s Hyperion Renderer by our amazing artists. I’ve ordered the frames randomly, to try to prevent spoiling anything important. These frames showcase just how gorgeous Encanto looks, but they only represent a small fraction of how breathtakingly beautiful and colorful the total film is. I highly recommend seeing Encanto on the biggest screen you can; if you are a computer graphics enthusiast, go see it twice: the first time for the wonderful, magical story and the second time for the incredible artistry that went into every single shot and every single frame! I love working on Disney Animation films because Disney Animation is a place where some of the most amazing artists and engineers in the world work together to simultaneously advance animation as a storytelling medium, as a visual medium, and as a technology goal. Art being inspired by technology and technology being challenged by art is a legacy that is deeply baked into the very DNA of Disney Animation, and that approach is exemplified by every single frame in Encanto:

Here is the credits frame for Disney Animation’s rendering and visualization teams! These two teams collectively are responsible for generating all of the pixels at Disney Animation, be it final frames from Hyperion, or interactive viewports using our internal realtime rasterizer:

All images in this post are courtesy of and the property of Walt Disney Animation Studios.

Also, be sure to catch our new short, Far From the Tree, which is accompanying Encanto in theaters. Far From the Tree deserves its own discussion later; all I’ll write here is that I’m sure it’s going to be fascinating for rendering and computer graphics enthusiasts to see! Far From the Tree tells the story of a parent and child raccoon as they explore a beach; the short has a beautiful hand-drawn watercolor look that is actually CG rendered out of Disney’s Hyperion Renderer and extensively augmented with hand-crafted elements. Be sure to see Far From the Tree in theaters with Encanto!

References

Brent Burley. 2015. Extending the Disney BRDF to a BSDF with Integrated Subsurface Scattering. In ACM SIGGRAPH 2015 Course Notes: Physically Based Shading in Theory and Practice.

Brent Burley and Francisco Rodriguez. 2022. Fracture-Aware Tessellation of Subdivision Surfaces. In ACM SIGGRAPH 2022 Talks. Article 10.

Corey Butler, Brent Burley, Wei-Feng Wayne Huang, Yining Karl Li, and Benjamin Huang. 2022. “Encanto” - Let’s Talk About Bruno’s Visions. In ACM SIGGRAPH 2022 Talks. Article 8.

Matt Jen-Yuan Chiang and Brent Burley. 2018. Plausible Iris Caustics and Limbal Arc Rendering. In ACM SIGGRAPH 2018 Talks. Article 15.

Alejandro Conty and Christopher Kulla. 2017. Production Friendly Microfacet Sheen BRDF. In ACM SIGGRAPH 2017 Course Notes: Physically Based Shading in Theory and Practice.

Henrik Dahlberg, David Adler, and Jeremy Newlin. 2019. Machine-Learning Denoising in Feature Film Production. In ACM SIGGRAPH 2019 Talks. Article 21.

Andrew Finley, Jesse Erickson, Peter De Mund, and Ying Liu. 2022. Modeling Animated Jumbo Floral Display on Disney’s “Encanto”. In ACM SIGGRAPH 2022 Talks. Article 43.

Haixiang Liu. 2022. Gravity Preloading for Maintaining Hair Shape Using the Simulator as a Closed-box Function. In ACM SIGGRAPH 2022 Talks. Article 40.

Tad Miller, Harmony M. Li, Neelima Karanam, Nadim Sinno, and Todd Scopio. 2022. Making Encanto with USD: Rebuilding a Production Pipeline Working from Home. In ACM SIGGRAPH 2022 Talks. Article 12.

Thomas Müller. 2019. Practical Path Guiding in Production. In ACM SIGGRAPH 2019 Course Notes: Path Guiding in Production. 37-50.

Thomas Müller, Markus Gross, and Jan Novák. 2017. Practical Path Guiding for Efficient Light-Transport Simulation. Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering) 36, 4 (Jun. 2017), 91-100.

Michelle Robinson, Michael Woodside, Daniel Rice, Tad Miller, Scott Kersavage, and Tyler Kupferer. 2022. We Don’t Talk About Bruno - An Encanto Musical Sequence Unveiled. In ACM SIGGRAPH 2022 Production Sessions. Article 2.

Justin Tennant, Mitch Counsell, Far Jangtrakool, Salina Ortega, Rajesh Sharma, Tad Miller, and Scott Kersavage. 2022. Visualizing the Production Process of “Encanto” with the Command Center. In ACM SIGGRAPH 2022 Talks. Article 11.

Jose Velasquez, Alexander Alvarado, Ying Liu, and Maryann Simmons. 2022. Embroidery and Cloth Fiber Workflows on Disney’s “Encanto”. In ACM SIGGRAPH 2022 Talks. Article 22.

Tizian Zeltner, Brent Burley, and Matt Jen-Yuan Chiang. 2022. Practical Multiple-Scattering Sheen Using Linearly Transformed Cosines. In ACM SIGGRAPH 2022 Talks. Article 7.