The second of Disney Animation’s two short-form television series in 2022 is Zootopia+, which was released in November of last year. Zootopia+ is comprised of six episodes set during Zootopia (2016), showing us events that are interwoven with the events of the original movie. Much like how Baymax! was made, Zootopia+ was produced entirely in-house at Disney Animation using the same tech and pipeline, and by the same artists, as our feature films. Furthermore, because Zootopia+ is interwoven with the original Zootopia, it had to be an exact match in terms of visual quality, even though at the time of release Zootopia was the most advanced and challenging film the studio had ever made. Getting to work on a project where we revisited Zootopia from some new angles was great fun!

Zootopia+ was made using essentially the same pipeline, toolset, and version of Disney’s Hyperion Renderer as Encanto. Notably since Encanto’s pipeline represented a massive jump forward from old proprietary data formats to an all-native-USD pipeline [Miller et al. 2022], Zootopia+ had to make the same jump. However, instead of starting with a clean slate of characters and assets as Encanto did, Zootopia+ had to port all of the familiar characters and assets from the original Zootopia forward from the studio’s legacy proprietary data formats into our new USD world. Interestingly, while much of the rest of the pipeline required a large porting effort to bring everything forward, on the rendering front, there was not actually much for us to do at all. Zootopia in a lot of senses was the first truly modern Hyperion show, where every rendering and shading feature used by the characters and assets back during the original production is still supported in the renderer today.

Big Hero 6 was the first show to make use of Hyperion, but Big Hero 6 represented something of a bridge period between the old REYES-based RenderMan world and the modern Hyperion world, and between Big Hero 6 and Zootopia, a lot of modernization happened that became the foundation of what we have today [Burley et al. 2017, Burley et al. 2018]. For example, Big Hero 6 is the only Hyperion show to not have used our modern fur/hair shading model, which was invented for Zootopia, and the old Tangled-era shading model [Sadeghi et al. 2010] was removed entirely from the renderer after the modern shading model [Chiang et al. 2016a] was put in place. We have since made many advancements on top of the features that Zootopia used, but all of the Zootopia-era shading features still exist and are supported in the renderer, so assets ported from Zootopia into our modern USD pipeline basically render more or less correctly out of the box without additional lookdev work. From a porting perspective, one useful aspect of the Zootopia world is that most characters are covered in fur, which means that even though characters ported from Zootopia are using our older normalized diffusion subsurface scattering model [Burley 2015] instead of the modern path traced subsurface scattering model [Chiang et al. 2016b], it doesn’t actually matter too much since bare skin is rarely seen anyway! One aspect of the renderer that has changed enormously since Zootopia and where the Zootopia-era system no longer exists is volume rendering; our modern volume rendering system completely replaced the old system [Kutz et al. 2017, Huang et al. 2021]. However, because volumes are typically used for shot-specific effects work, this wasn’t a problem for Zootopia+. Shot-specific effects didn’t need to be ported over from Zootopia because any volumetric effects needed for Zootopia+ would need to be newly made for the new shots anyway.

Much like on Once Upon A Snowman, because Zootopia+ has to interweave with the original Zootopia, the overall look of Zootopia+ has to match the look of the original Zootopia and look recognizable as being the same world at the same time. Unlike Once Upon A Snowman though, there wasn’t nearly as much of a visual gap to cross despite the 7 year gap, since Zootopia was also a modern Hyperion show whereas Once Upon A Snowman had to bridge the gap between a rasterized world without multi-bounce global illumination and the modern path traced world. Really I think what this speaks to is how successful the jump to path traced global illumination has been at Disney Animation and across the animation and VFX industry broadly. At the time of Zootopia, we were just barely past the point where path traced global illumination at scale was practical, but as a field we’ve now gotten incredibly good at it and now most of our research and development has moved away from trying to make path tracing practical at all and instead building better and more efficient features and workflows on top of path tracing. For the most part Zootopia+ matches Zootopia visually exactly, but a few episodes of Zootopia+ diverge and do something slightly different in order to serve the episode’s story. One of the ways this manifests is in varying aspect ratios; most of Zootopia+ uses the studio’s house-standard 2.39:1 Cinemascope widescreen aspect ratio, but a few episodes use a more widescreen TV styled 16:9 (or 1.78:1) aspect ratio, and one episode uses a throwback 1.85:1 old-school theatrical aspect ratio.

One area unrelated to rendering where Zootopia+ made an enormous technical jump is in rigging and animation. Historically Disney Animation has been a Maya shop when it comes to rigging and character animation, but Zootopia+ is the first project at Disney Animation to make use of Presto for rigging and character animation instead of Maya. Presto was originally Pixar’s proprietary in-house rigging and animation package [ElKoura 2014], first used on Brave, but going forward, Disney Animation is planning on using Presto as well and is moving all rigging and character animation over to Presto. To support this adoption effort, Presto is now co-developed by the two studios. So, even though porting characters from Zootopia to Zootopia+ required very little to no additional work on the shading and rendering side of things, some characters had to be completely re-rigged from scratch in Presto. Tools such as Disney Animation’s procedural rig authoring system, dRig [Smith et al. 2012], also had to be ported to work with Presto. Animating part of Zootopia+ in Presto and the rest in Maya is a great example of the type of workflows and capabilities that a fully native USD pipeline unlocks; previously a lot of our workflows were tied to specific DCCs because of proprietary data formats, but now in our pipeline, any DCC can be used once it has been customized to work with our internal flavor of USD.

Zootopia+ is available for streaming on Disney+. My suggestion is always to watch on the largest screen you can, and Zootopia+ is no exception; the final animation and visuals are every bit feature film quality, and watching Zootopia+ back to back with Zootopia is a fun way to see how the two interleave. Here is a selection of stills from Disney+, presented in no particular order:

Here is the credits frame for the Hyperion team, which on Zootopia+ is presented together with the rest of the entire production technology department in a single credits block:

All images in this post are courtesy of and the property of Walt Disney Animation Studios.

References

Brent Burley. 2015. Extending the Disney BRDF to a BSDF with Integrated Subsurface Scattering. In ACM SIGGRAPH 2015 Course Notes: Physically Based Shading in Theory and Practice.

Brent Burley, David Adler, Matt Jen-Yuan Chiang, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. 2017. Recent Advancements in Disney’s Hyperion Renderer. In ACM SIGGRAPH 2017 Course Notes: Path Tracing in Production Part 1. 26-34.

Brent Burley, David Adler, Matt Jen-Yuan Chiang, Hank Driskill, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. 2018. The Design and Evolution of Disney’s Hyperion Renderer. ACM Transactions on Graphics 37, 3 (Jul. 2018), Article 33.

Matt Jen-Yuan Chiang, Benedikt Bitterli, Chuck Tappan, and Brent Burley. 2016. A Practical and Controllable Hair and Fur Model for Production Path Tracing. Computer Graphics Forum (Proc. of Eurographics) 35, 2 (May 2016), 275-283.

Matt Jen-Yuan Chiang, Peter Kutz, and Brent Burley. 2016. Practical and Controllable Subsurface Scattering for Production Path Tracing. In ACM SIGGRAPH 2016 Talks. Article 49.

Wei-Feng Wayne Huang, Peter Kutz, Yining Karl Li, and Matt Jen-Yuan Chiang. 2021. Unbiased Emission and Scattering Importance Sampling for Heterogeneous Volumes. In ACM SIGGRAPH 2021 Talks. Article 3.

Peter Kutz, Ralf Habel, Yining Karl Li, and Jan Novák. 2017. Spectral and Decomposition Tracking for Rendering Heterogeneous Volumes. ACM Transactions on Graphics (Proc. of SIGGRAPH) 36, 4 (Aug. 2017), Article 111.

Tad Miller, Harmony M. Li, Neelima Karanam, Nadim Sinno, and Todd Scopio. 2022. Making Encanto with USD: Rebuilding a Production Pipeline Working from Home. In ACM SIGGRAPH 2022 Talks. Article 12.

Iman Sadeghi, Heather Pritchett, Henrik Wann Jensen, and Rasmus Tamstorf. 2010. An Artist Friendly Hair Shading System. ACM Transactions on Graphics (Proc. of SIGGRAPH) 29, 4 (Jul. 2010), Article 56.

Greg Smith, Mark McLaughlin, Andy Lin, Evan Goldberg, and Frank Hanner. 2012. dRig: An Artist-Friendly, Object-Oriented Approach to Rig Building. In ACM SIGGRAPH 2012 Talks. Article 18.

George ElKoura. 2014. The Presto Execution System: Designing for Multithreading. In Multithreading for Visual Effects. 47-71.

Disney Animation’s fall 2022 release is Strange World, which is the studio’s 61st animated feature, and third original story in as many years. Strange World takes us to the land of Avalonia, a realm surrounded by impenetrable mountains and home to a society that blends elements of early 20th century pulp fiction, steampunk, and environmental solarpunk. The core story of Strange World revolves around father-son relationships and is exactly the type of story that Disney Animation excels at: something personal and relatable but set in a fantastic world we’ve never seen before. That “never seen before” aspect made my two years of working on rendering technology for Strange World an interesting time indeed!

In my writeups about our films, two recurring themes always are: 1. with each film, we build upon advancements made and lessons learned from the previous film, and 2. one of the greatest advantages that having in-house tools gives us is the ability to customize and build exactly what each film’s story and art direction requires. Strange World’s production exemplifies both of these themes; so much of what we had to do on Strange World builds upon things we learned from and developed for previous films that I’m not entirely sure we would have been able to make Strange World a few years ago, and much of what we learned we have only been able to apply as effectively as we have because we have the ability to extend and improve our own tools.

As an example: a large part of Strange World takes place on the airship Venture, which from a production pipeline perspective has to function as both a set/environment in which characters move and interact, and as a sort of character of its own as it moves around in the larger surrounding environment. In CG production pipelines, various pipeline optimizations are often built around the reasonable assumption that sets are relatively static; sets typically don’t need complex animation rigs and can be used as a stationary frame of reference for all kinds of different things. The Venture, of course, breaks all of these expectations. Disney Animation had to deal with this type of scenario before on Moana, where much of the movie is set on a canoe out at sea, so handling the “sets as characters” challenge wasn’t new. Instead of having to solve this problem from scratch, our artists and TDs were able to build on top of what they had learned before to enable the Venture to be a far more complex “set as a character” than anything we had done before. In fact, the Venture isn’t the only case of this type of challenge in Strange World! Huge parts of Strange World follow this “set as a character” pattern; entire chunks of terrain get up and walk around in this movie! All of these complex sets were made possible by advancements [Vo et al. 2023] in our USD based pipeline, which in turn built upon all of the lessons learned [Miller et al. 2022] from our previous pipeline.

Things got even more complex once crowds were brought into the mix too. Strange World has some of the most massive crowd simulation ever made by Disney Animation [Devlin et al. 2023], and these huge crowds had to interact with the Venture and complex terrain. One of the main tools our crowds team used to guide giant swarms of creatures traveling through Strange World’s massive environments and around the Venture originated as a tool made for a single character on Frozen 2, and had to be turbocharged to massive scales to go from handling the requirements for one character to handling thousands upon thousands of creatures [Lin et al. 2023]. Challenges involving simulating collisions in huge crowds like this, along with similar challenges in hair simulation, helped inspire further research work [Zhang et al. 2023] for future films as well.

Once the story moves into the subterranean world, the environment of the film ratchets up in production complexity on multiple different axes. Essentially every single surface in the subterranean world has significant subsurface scattering since everything is made up of organic gummy materials, and of course all of the giant crowds also all have subsurface scattering. Many of our previous films already were beginning to push the use of subsurface scattering in environments for things like plants and plastics and other materials, all thanks to the work that the rendering team put into making path traced subsurface scattering efficient and controllable enough for large-scale production usage [Chiang et al. 2016], but Strange World saw the widest usage of subsurface scattering in environments yet, by far.

Everything we’ve learned about controlling subsurface scattering also proved to be extremely important for creating the look for the Splat character, who is essentially a giant immune cell. Splat wound up requiring a unique custom one-off shader with custom functionality in Disney’s Hyperion Renderer combining subsurface scattering, a custom faux volumetric emission technique, our multiple-scattering sheen solution [Zeltner et al. 2022], and more in order to achieve the target art-direction in a single render pass [Litaker et al. 2023]. Splat’s challenges weren’t limited to just rendering though; rigging and animating Splat also required novel solutions in order to handle how varied and multi-purpose Splat’s limbs are [Black and Pederson 2023]. Splat’s rig was only made possible through a combination of new novel techniques and a decade of experience and continuous improvement in Disney Animation’s DRig modular rig building system [Smith et al. 2012].

Splat wasn’t the only character that provided interesting technical challenges though; in fact, our entire character asset workflow got an upgrade on Strange World. Our standard character asset workflow saw three major improvements on Strange World: eyes, skin, and curves.

Strange World’s character art direction called for eyes to use a bit of a different look from Disney Animation’s usual style; eyes on Strange World have more of an oblong oval shape. Over the past several shows, we introduced a new eye shading model that incorporates manifold next event estimation for physically accurate iris caustics and limbal arcs [Chiang et al. 2018]; one of my smaller projects on Strange World was to help work out the minor modifications to this system that were required to support Strange World’s eye shapes.

For skin shading, Strange World uses the same fully path traced subsurface scattering approach [Chiang et al. 2016] (as opposed to older diffusion-based approaches [Burley 2015]) that we have now used for all of our movies over the past few years. However, Strange World has one of the most diverse casts of any of our recent films in terms of skin tones, and our lighting and look dev artists took special care to make sure all of the different skin types were depicted accurately and beautifully. Doing so required rebuilding our entire skin material from the ground up and radically rethinking our entire approach to lighting characters to better handle contrasting skin tones and high specularity skin [Khoo et al. 2023].

Previously on Encanto, our look artists started to replace triangle mesh-based geometric representations for cloth with curve-based fiber level representations [Velasquez et al. 2022]. This authoring approach was pushed to new limits on Strange World, where curve-based garments were extended to incorporate custom weave patterns and widely varying fiber thicknesses ranging from fine threads to thick yarns [Lipson and Velasquez 2023]. Humans weren’t the only type of characters on Strange World to see upgraded curve geometry though; the Clade family’s lovable dog, Legend, also required upgraded curve grooming techniques to produce one of the most complex animal grooms the studio has ever made [Chun et al. 2023]. Of course all of these improved authoring techniques meant increased curve rendering complexity, but interestingly, we didn’t actually need to improve anything in the renderer to handle the increased curve rendering demand. After having spent many prior shows improving Hyperion’s ability to chew through vast geometric complexity, on Strange World we found that Hyperion was able to just handle all of the meshes and curves that we threw at it!

The hardest rendering challenge I worked on for Strange World was volume rendering. Strange World’s environments have some of the largest scale and most ambitious use of volumes in any of our films to date. Strange World extensively utilizes mist and atmospherics and low cloud cover to help convey a sense of mystery and to sell the sheer scale of the environments. Frozen 2 was the first movie that really extensively leveraged Hyperion’s modern volume rendering system (which we rewrote essentially from scratch during the early production of Ralph Breaks the Internet) and the first movie that introduced our modern volumes authoring workflow. This workflow, which is heavily based around quickly set-dressing atmospherics and clouds around environments by kitbashing together volumes from a large pre-made in-house library of VDBs, was further fleshed out on Raya and the Last Dragon and saw its largest and most complex usage yet on Strange World. Strange World also further extended our volume workflows with an evolved version [Navarro 2023] of the neural volume stylization tech we first introduced on Raya and the Last Dragon [Navarro and Rice 2021].

During Raya and the Last Dragon we consolidated various different experiments and techniques in our volume rendering system into a single unified volume integrator [Huang et al. 2022] that can efficiently handle every imaginable type of volume effect, so the challenge presented by Strange World’s volumes wasn’t so much light transport as it was simply a problem of efficiency at scale. When volumes are simultaneously highly detailed but also span kilometers of world space, massive memory usage becomes challenging, even with instancing. Also, super large and detailed volumes coverage means that average path length in volumes can get very long, exposing any potential performance issues in the volume integrator. A huge part of my time on Strange World was spent optimizing our volume integrator. There were no clever shortcuts or brilliant solutions here, just tons of profiling and careful analysis of the existing system architecture and hard low level optimization work.

We also noticed during Strange World that artists sometimes had to overauthor volume details in areas as a way to work around the lack of true procedural volumes evaluation support in our renderer. While Hyperion does support authoring procedural volumes, these procedural volumes are not actually evaluated at render time but instead are pre-evaluated and baked into a required underlying VDB grid at renderer startup. The reason for this limitation is fundamental to null collision-based volume rendering theory [Novák 2018]; null collision approaches only work if the bounding majorant (AKA max density) for all volumes in a region of space is known upfront. In theory we could just require artists to input a max density value that we would clamp all higher values down to, but such a value isn’t easy for artists to estimate in practice; too low of a value clamps away detail, while too high of value results in an overly loose bounding majorant, which in null collision theory-based volume rendering can result in significantly slower performance. Inspired by what we were seeing on Strange World, we kicked off a research project in collaboration with the Visual Computing Lab at Dartmouth College to solve this problem, with promising results [Misso et al. 2023]!

As usual, I’ve only written about the parts of making Strange World that I know a bit more about; hundreds of artists, TDs, and engineers worked to craft every frame of this movie and solve many many more problems. For the entire history of Disney Animation, one of the studio’s primary driving purposes has been to push the limits of animation as an art form, and Strange World is no exception to this rule. Strange World is the latest example of how each of our films builds upon what we’ve learned on previous films to push our filmmaking process forward, and as always, getting to be a part of this process is a lot of work but also a lot of fun!

Below are some frames from the strange but gorgeous world of Strange World, pulled from the Blu-ray and presented in semi-randomized order to prevent giving away too much of the story. Go see Strange World on the biggest screen you can find!

Here is the credits frame for the Hyperion team, which is listed as part of the larger Rendering & Visualization group at Disney Animation. In addition to the Hyperion team, this group also includes our sister render translation pipeline and interactive visualization teams:

All images in this post are courtesy of and the property of Walt Disney Animation Studios.

References

Cameron Black and Christoffer Pedersen. 2023. The Versatile Rigging of Splat in ‘Strange World’. In ACM SIGGRAPH 2023 Talks. Article 29.

Brent Burley. 2015. Extending the Disney BRDF to a BSDF with Integrated Subsurface Scattering. In ACM SIGGRAPH 2015 Course Notes: Physically Based Shading in Theory and Practice.

Matt Jen-Yuan Chiang, Peter Kutz, and Brent Burley. 2016. Practical and Controllable Subsurface Scattering for Production Path Tracing. In ACM SIGGRAPH 2016 Talks. Article 49.

Matt Jen-Yuan Chiang and Brent Burley. 2018. Plausible Iris Caustics and Limbal Arc Rendering. In ACM SIGGRAPH 2018 Talks. Article 15.

Courtney Chun, Jose Velasquez, and Haixiang Liu. 2023. Creating the Art-directed Groom for Legend in Disney’s Strange World. In ACM SIGGRAPH 2023 Talks. Article 7.

Nathan Devlin, Yasser Hamed, Alberto J Luceño Ros, Jeff Sullivan, and D’Lun Wong. 2023. Creating Creature Chaos: The Methods That Brought Crowds to the Forefront on Disney’s ‘Strange World’. In ACM SIGGRAPH 2023 Talks. Article 34.

Wei-Feng Wayne Huang, Peter Kutz, Yining Karl Li, and Matt Jen-Yuan Chiang. 2021. Unbiased Emission and Scattering Importance Sampling for Heterogeneous Volumes. In ACM SIGGRAPH 2021 Talks. Article 3.

Mason Khoo, Dan Lipson, and Jose Velasquez. 2023. Lighting and Look Dev for Skin Tones in Disney’s “Strange World”. In Proc. of Digital Production Symposium (DigiPro 2023). Article 5.

Andy Lin, Hannah Swan, Justin Walker, Cathy Lam, and Ricky Arietta. 2023. Swoop: Animating Characters Along a Path. In ACM SIGGRAPH 2023 Talks. Article 45.

Dan Lipson and Jose Velasquez. 2023. Creating Curve-based Garments With Custom Weave Patterns. In ACM SIGGRAPH 2023 Talks. Article 18.

Kendall Litaker, Brent Burley, Dan Lipson, and Mason Khoo. 2023. Splat: Developing a ‘Strange’ Shader. In ACM SIGGRAPH 2023 Talks. Article 28.

Tad Miller, Harmony M. Li, Neelima Karanam, Nadim Sinno, and Todd Scopio. 2022. Making Encanto with USD: Rebuilding a Production Pipeline Working from Home. In ACM SIGGRAPH 2022 Talks. Article 12.

Zackary Misso, Yining Karl Li, Brent Burley, Daniel Teece, and Wojciech Jarosz. 2023. Progressive Null-tracking for Volumetric Rendering. In Proc. of SIGGRAPH (SIGGRAPH 2023). Article 31.

Mike Navarro and Jacob Rice. 2021. Stylizing Volumes with Neural Networks. In ACM SIGGRAPH 2021 Talks. Article 54.

Mike Navarro. 2023. Diving Deeper Into Volume Style Transfer. In ACM SIGGRAPH 2023 Talks. Article 39.

Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo Methods for Volumetric Light Transport Simulation. Computer Graphics Forum (Proc. of Eurographics) 37, 2 (May 2018), 551-576.

Greg Smith, Mark McLaughlin, Andy Lin, Evan Goldberg, and Frank Hanner. 2012. DRig: An Artist-Friendly, Object-Oriented Approach to Rig Building. In ACM SIGGRAPH 2012 Talks. Article 18.

Jose Velasquez, Alexander Alvarado, Ying Liu, and Maryann Simmons. 2022. Embroidery and Cloth Fiber Workflows on Disney’s “Encanto”. In ACM SIGGRAPH 2022 Talks. Article 22.

Emily Vo, George Rieckenberg, and Ernest Petti. 2023. Honing USD: Lessons Learned and Workflow Enhancements at Walt Disney Animation Studios. In ACM SIGGRAPH 2023 Talks. Article 13.

Tizian Zeltner, Brent Burley, and Matt Jen-Yuan Chiang. 2022. Practical Multiple-Scattering Sheen Using Linearly Transformed Cosines. In ACM SIGGRAPH 2022 Talks. Article 7.

Paul Zhang, Zoë Marschner, Justin Solomon, and Rasmus Tamstorf. 2023. Sum-of-squares Collision Detection for Curved Shapes and Paths. In Proc. of SIGGRAPH (SIGGRAPH 2023). Article 76.

This year at SIGGRAPH 2022, Corey Butler, Brent Burley, Wei-Feng Wayne Huang, Benjamin Huang, and I have a talk that presents the technical and artistic challenges and solutions that went into creating the holographic look for Bruno’s visions in Encanto. In Encanto, Bruno is a character who has a magical gift of being able to see into the future, and the visions he sees of the future get crystalized into a sort of glassy emerald tablet with the vision embedded in the glassy surface with a holographic effect. Coming up with this unique look and an efficient and robust authoring workflow required a tight collaboration between visual development, lookdev, lighting, and the Hyperion rendering team to develop a custom solution in Disney’s Hyperion Renderer. On the artist side, Corey was the main lighter and Benjamin was the main lookdev artist for this project, while on the rendering team side, Wayne and I worked closely together to develop a series of prototype shaders that were instrumental in defining how the effect should look and then Brent came up with the implementation approach for the final production version of the shader. This project was a lot of fun to be a part of and in my opinion really demonstrates the benefits of having an in-house rendering team that works closely with and embedded within a production context.

Here is the paper abstract:

In Walt Disney Animation Studios’ “Encanto”, Mirabel discovers the remnants of her Uncle Bruno’s mysterious visions of the future. Developing the look and lighting for the emerald shards required close collaboration between our Visual Development, Look Development, Lighting, and Technology departments to create a holographic effect. With an innovative new teleporting holographic shader, we were able to bring a unique and unusual effect to the screen.

The paper and related materials can be found at:

• Project Page (Author’s Version and Supplementary Material)
• Official Print Version (ACM Library)

When Corey first came to the rendering team with the request for a more efficient way to create the hologram effect that lighting had prototyped using camera mapping, our initial instinct actually wasn’t to develop a new shader at all. Hyperion has an existing “hologram” shader that was developed for use on Big Hero 6 [Joseph et al. 2014], and our initial instinct was to tell Corey that they should use the hologram shader. The way the Big Hero 6 era hologram shader works is: upon hitting a surface that has the hologram shader applied, the ray is moved into a virtual space containing a bunch of imaginary parallel planes, with each plane textured with a 2D slice of a 3D interior. In some ways the hologram shader can be thought of as raymarching through a sparse volumetric representation of a 3D interior, but the sparse volumetric interior really is just a stack of 2D slices. This technique works really well for things like building interiors seen through glass windows. However, our artists… really dislike using the hologram shader, to put things lightly. The problem with the hologram shader is that setting up the 2D slices that are inputs to the shader is an incredibly annoying and difficult process, and since the 2D slice baker has to be run as an offline process before the shader can be authored and rendered, making changes and iterating on the contents of the hologram shader is a slow process. Furthermore, if the inside of the hologram shader has to be animated, the slice baker needs to be run for every frame. We were told in no uncertain terms that the hologram shader was likely more work to set up and iterate on than the already painful manual camera mapping approach that the artists had prototyped the effect with. This request also came to us fairly late in Encanto’s production schedule, so easy setup and fast iteration times along with an extremely accelerated development timeline were hard requirements for whatever approach we took.

Upon receiving this feedback, Wayne and I set out to prototype a version of the teleportation shader that Pixar came up with for the portals in Incredibles 2 [Coleman et al. 2014]. This process was a lot of fun; Wayne and I spent a few days rapidly iterating on several different ideas for both how to implement ray teleportation in Hyperion and on how the artist workflow and interface for this new teleportation system should work. At the same time that we were prototyping, we started giving test builds of our latest prototypes to Corey to try out, which produced a feedback loop where Corey would use our prototypes to further iterate on how the final effect would look and go back and forth with the movie’s production designer and we would use Corey’s feedback to further improve the prototype. One example of where our prototype directly informed the final look was in how the prophecies fade away towards the edges of the emerald tablet- Wayne and I threw in a feature where artists could use a map to paint in the ratio of teleportation effect versus normal surface BSDF that would be applied at each surface point, and this feature wound up driving the faded edges.

The key thing that made our new approach work better than the old hologram shader was in simplicity of setup. Instead of having to run a pre-bake process and then wire up a whole bunch of texture slices into the renderer, our new approach was designed so that all an artist had to do was set up the 3D geometry that they wanted to put inside of the hologram in a target space hidden somewhere in the overall scene (typically below the ground plane in a black box or something), and then select the geometry in the main scene that they wanted to act as the “entrance” portal, select the geometry in the target space that they wanted to act as the “exit” portal, and link the two using the teleportation shader. The renderer then did all of the rest of the work of figuring out how each point on the entrance portal corresponded to the surface of the exit portal, how transforms needed to be calculated, and so on and so forth. Multiple portal pairs could be set up in a single scene too, and the contents of a world seen through a portal could contain more portals, all of which was important because in the movie, Mirabel initially finds Bruno’s prophecy broken into shards, which had to be set up as a separate entrance portal per shard all into the same interior world. Since all of this just piggy-backed off of the normal way artists set up scenes, things like animation just worked out-of-the-box with no additional code or effort.

The last piece of the puzzle fell into place when Wayne and I discussed our progress with Brent. One of the big remaining challenges for us was that tracking correspondences between entrance and exit geometry and transforms was prone to easy breakage if input geometry wasn’t set up exactly the way we expected. At the time Brent was working on a new fracture-aware tessellation system for subdivision surfaces in Hyperion [Burley and Rodriguez 2022], and Brent quickly realized that the approach we were using for figuring out the transform from the entrance to the exit portal could be replaced with something he had already developed for the fracture-aware tessellation system. Specifically, the fracture-aware tessellation system has to be able to calculate correspondences between undeformed unfractured reference points and corresponding points in a deformed fractured fragment space; this is done using a best-fit process to find orthonormal transforms [Horn et al. 1998]. Brent realized that the problem we were trying to solve was actually the same problem he that he had already solved in the fracture system, so he took our latest prototype and reworked the internals to use the same best-fit orthonormal transform solution as in the fracturing system. With Brent’s improvements, we arrived at the final production version of the teleportation shader used on Encanto.

Going from the start of brainstorming and prototyping to delivering the final production version of the shader took us a little over a week, which anyone who has worked in an animation/VFX production setting before will know is very fast for a large new rendering feature. Working tightly with Corey and Benjamin to simultaneously iterate on the art and the software and inform each other was key to this project’s fast development time and key to achieving an amazing looking effect in the film. At Disney Animation, we have a mantra that goes “art challenges technology and technology inspires the art”- this project was a case that exemplifies how we carry out that mantra in real-world filmmaking and demonstrates the amazing results that come out of such a process. Bruno’s visions in Encanto are every bit a case where the artistic vision challenged us to develop new technology, and the process of iterating on the new technology between engineers and artists in turn informed the final artwork that made it into the movie; for me, projects like these are one of the things that makes Disney Animation such a fun and amazing place to be.

References

Brent Burley, David Adler, Matt Jen-Yuan Chiang, Hank Driskill, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. 2018. The Design and Evolution of Disney’s Hyperion Renderer. ACM Transactions on Graphics 37, 3 (Jul. 2018), Article 33.

Brent Burley and Francisco Rodriguez. 2022. Fracture-Aware Tessellation of Subdivision Surfaces. In ACM SIGGRAPH 2022 Talks. Article 10.

Patrick Coleman, Darwyn Peachey, Tom Nettleship, Ryusuke Villemin, and Tobin Jones. 2018. Into the Voyd: Teleportation of Light Transport in Incredibles 2. In Proc. of Digital Production Symposium (DigiPro 2018). Article 12.

Berthold K. P. Horn, Hugh M. Hilden, and Shahriar Negahdaripour. 1988. Close-Form Solution of Absolute Orientation using Orthonormal Matrices. Journal of the Optical Society of America A 5, 7 (Jul. 1988), 1127–1135.

Norman Moses Joseph, Brett Achorn, Sean D. Jenkins, and Hank Driskill. Visualizing Building Interiors Using Virtual Windows. In ACM SIGGRAPH Asia 2014 Technical Briefs. Article 18.

Disney Animation has two short-form television series lined up for release in 2022 on Disney+, and the first of the two series is Baymax!, which is comprised of six episodes set after the events of Big Hero 6 and follows Baymax around San Fransokyo as he helps various people. Baymax! notably is the first time Disney Animation has ever produced a television series completely in-house; because Baymax! was made in-house, it was made using the same tech and pipeline, by the same artists, and to the same standards that we make our feature-length films using. Big Hero 6 was the first project at Disney Animation ever rendered using Disney’s Hyperion Renderer (and to a large extent the first version of Hyperion was specifically developed for Big Hero 6), so getting to revisit the world of San Fransokyo in many ways represented a sort of homecoming for Hyperion. However, I joined the Hyperion development team shortly after the completion of Big Hero 6, so for me personally, getting to work a bit for the first time on a project tied to the show where Hyperion began was a very cool experience.

Baymax! was made using a slightly evolved version of the same pipeline and toolset that was used for Raya and the Last Dragon. Since production overlapped with Encanto, some improvements developed for Encanto made their way over to Baymax! as well, which in the case of Hyperion mostly meant bugfixes and improvements to our state-of-the-art in-house machine-learning denoiser [Vogels et al. 2018]. From a computer graphics technology perspective, there is not really much new to tell about the work that went into Baymax!, but I think this in itself is actually an interesting thing to discuss. Production on Baymax! went very smoothly and required very little dedicated renderer development work, which I think is a testament to both how mature Hyperion has become since Big Hero 6 and how proficient our artists have become at using Hyperion. Of course, another factor to consider is that Baymax! has more or less exactly the same challenges that Big Hero 6 had, and all of those problems were solved already (and arguably Hyperion was custom built specifically to solve many of those problems), so of course having a second go at exactly the same kinds of production challenges should be easier than it was the first time, especially when using more advanced, evolved versions of the original solutions.

As an example, the Baymax character’s white translucent look comes from a ton of high-order scattering inside of his inflatable balloon shell, but the effect is different than what one gets from subsurface scattering. Instead of having some kind of solid medium inside, Baymax contains air inside, so the type of scattering occurring inside of Baymax is more akin to something like total internal reflection with no energy loss between surface events, as opposed to subsurface scattering which has extinction due to volumetric effects. The net result is that Baymax requires a ton of surface bounces in path tracing; often many more than our usual maximum path length. On Big Hero 6 the solution to this problem was to allow artists to specify materials and objects that would be permitted to raise the maximum path length for paths that interact with them [Driskill et al. 2015]. On Baymax!, all we had to do was to remember to set this setting for Baymax, and of course everything just worked because this problem had already been solved before.

Previously we’ve ported characters and assets from older shows forward to whatever the latest modern pipeline is many times, but Baymax! was a case where we had to port characters that originated from a previous Hyperion-based show forward. I’ve written before about how porting pre-Hyperion characters and assets to Hyperion is made a lot easier by a lot of our foundational shading technologies spanning between the pre-Hyperion and modern Hyperion pipeline versions, but porting from Big Hero 6 was made even easier by the fact that this was just porting between an older and newer version of the same renderer. Big Hero 6 was the first show to use our modern Disney BSDF [Burley et al. 2015], which expanded upon the older Disney BRDF [Burley et al. 2012] to add things such as refraction and subsurface scattering, so pretty much all solid surfaces ported over with essentially zero effort required. However, Big Hero 6 was also the last and only Hyperion-based show to use our older Tangled-era fur/hair shading model [Sadeghi et al. 2010], with all shows starting with Zootopia using our modern state-of-the-art (and now de-facto industry standard) fur/hair model [Chiang et al. 2016a], which meant that all characters with hair or fur had to have their look updated to use the modern fur/hair shading model. Similarly, for characters, shows starting with Frozen 2 moved off of normalized diffusion subsurface scattering [Christensen and Burley 2015] and onto path traced subsurface scattering [Chiang et al. 2016b], so this switch had to be made for Baymax! as well. On Frozen 2 we found that generally textures and settings meant for normalized diffusion translated to path traced subsurface scattering pretty well, although areas with thinner surfaces sometimes required some additional manual adjustment; the experience on Baymax! was similar. Additionally, additive features on top of the Disney BSDF, such as the softened shadow terminator handling we introduced on Frozen 2 [Chiang et al. 2019], just automatically made everything ported from Big Hero 6 look a bit nicer.

Aside from shading improvements, seeing how other technology and techniques developed in the years since Big Hero 6 fed back into improving the world of San Fransokyo on Baymax! was also very fun and cool. Here are three examples that I thought were really neat: First, for the swimming pool in the second episode of Baymax!, the photon mapping system originally developed for Moana [Burley et al. 2018] and expanded on Olaf’s Frozen Adventure proved useful for water effects once again, providing the caustics on the bottom of the pool. Second, in the fourth episode, a character runs a food truck that only serves fish soup, and to make shots of cooking the soup look really appealing and convincing, our effects artists borrowed from soup simulation and shading techniques originally developed for Raya and the Last Dragon. Third, the studio’s crowds and characters workflows and pipelines have improved by leaps and bounds since Big Hero 6, to the point where many of the new main characters in Baymax! are actually promoted versions of background crowd characters from Big Hero 6 [Hamed et al. 2015], chosen by the directors and upgraded by our artists to serve as new main characters that already fit into the show’s world.

At the end of the day though, I think one of the coolest technical things about Baymax! is simply that it looks every bit as good as the original Big Hero 6, if not even better in some places. Of course one would think that of course it would look as good as the original Big Hero 6 since it’s using improved versions of the same assets and an evolved version of the same pipeline and tooling and renderer, but the thing to note here is that Big Hero 6 was an enormous, high-risk, all-hands-on-deck heavy lift endeavour for the studio, whereas Baymax! is a television series made by a much smaller crew with much tighter resources. Why Baymax! is able to meet the same bar as Big Hero 6 is partially down to smart planning and decisions on the part of the directors and show supervisors, but another large part is due to how much the studio has improved and grown technically and artistically in the 8 years since Big Hero 6.

Baymax! is available for streaming on Disney+; I recommend projecting or casting Disney+ to the largest screen you can to best see all of the amazing work that went into making this television series look every bit as good as our feature films. Here is a selection of stills from Disney+, presented in no particular order:

Here is the credits frame for the Hyperion team, interspersed within the larger credits block for the entire production technology team at Disney Animation:

All images in this post are courtesy of and the property of Walt Disney Animation Studios.

References

Brent Burley. 2012. Physically Based Shading at Disney. In ACM SIGGRAPH 2012 Course Notes: Practical Physically-Based Shading in Film and Game Production.

Brent Burley. 2015. Extending the Disney BRDF to a BSDF with Integrated Subsurface Scattering. In ACM SIGGRAPH 2015 Course Notes: Physically Based Shading in Theory and Practice.

Brent Burley, David Adler, Matt Jen-Yuan Chiang, Hank Driskill, Ralf Habel, Patrick Kelly, Peter Kutz, Yining Karl Li, and Daniel Teece. 2018. The Design and Evolution of Disney’s Hyperion Renderer. ACM Transactions on Graphics 37, 3 (Jul. 2018), Article 33.

Matt Jen-Yuan Chiang, Benedikt Bitterli, Chuck Tappan, and Brent Burley. 2016. A Practical and Controllable Hair and Fur Model for Production Path Tracing. Computer Graphics Forum (Proc. of Eurographics) 35, 2 (May 2016), 275-283.

Matt Jen-Yuan Chiang, Peter Kutz, and Brent Burley. 2016. Practical and Controllable Subsurface Scattering for Production Path Tracing. In ACM SIGGRAPH 2016 Talks. Article 49.

Matt Jen-Yuan Chiang, Yining Karl Li, and Brent Burley. 2019. Taming the Shadow Terminator. In ACM SIGGRAPH 2019 Talks. Article 71.

Per H. Christensen and Brent Burley. 2015. Approximate Reflectance Profiles for Efficient Subsurface Scattering. In ACM SIGGRAPH 2015 Talks. Article 25.

Hank Driskill, Larry Wu, Adolph Lusinsky, and Sean D. Jenkins. 2015. Building San Fransokyo: Creating the World of Disney’s “Big Hero 6”. In ACM SIGGRAPH 2015 Production Sessions. 169.

Yasser Hamed, John Kahwaty, Andy Lin, Evan Goldberg, and Lawrence Chai. 2015. Crowd Character Complexity on Big Hero 6. In ACM SIGGRAPH 2015 Talks. Article 77.

Iman Sadeghi, Heather Pritchett, Henrik Wann Jensen, and Rasmus Tamstorf. 2010. An Artist Friendly Hair Shading System. ACM Transactions on Graphics (Proc. of SIGGRAPH) 29, 4 (Jul. 2010), Article 56.

Thijs Vogels, Fabrice Rousselle, Brian McWilliams, Gerhard Röthlin, Alex Harvill, David Adler, Mark Meyer, and Jan Novák. 2018. Denoising with Kernel Prediction and Asymmetric Loss Functions. ACM Transactions on Graphics (Proc. of SIGGRAPH) 37, 4 (Aug. 2018), Article 124.