Landmark in filmmaking and engineering

When Sir Peter Jackson’s first The Lord of the Rings film, The Fellowship of the Ring, hit the big screen in December 2001, audiences were dropped into a world that had never existed before. It instantly became a box-office hit, grossing almost US$900 million in cinemas worldwide.

Twenty-five years on, the Lord of the Rings trilogy still stands as a landmark in filmmaking and engineering. The tools, techniques and problem-solving audacity that came out of Wellington’s Wētā FX (then Weta Digital) didn’t just make three extraordinary films. They arguably redefined an entire industry. And, as those who were there will tell you, none of it was easy.

The scale of what Sir Peter and his team was attempting was unprecedented, so the team at Wētā FX had to invent most of what they needed as they went.

“I don’t think I could overstate what an impossibility it was in many ways,” says Wayne Stables, Visual Effects (VFX) Supervisor at Wētā FX, who has worked there since 1994.

“But there was just this blind faith. And a lot of that comes from Peter [Jackson] and his attitude of ‘how hard can it possibly be?’ There was never any doubt. Just never any doubt at all.”

“... the glory days of visual effects”

Ken McGaugh, a VFX Supervisor at Wētā FX, joined the company to work on The Two Towers, having watched what the Wellington team had achieved on Fellowship from the United States.

“From my perspective, those are the glory days of visual effects,” he says, adding,“You didn’t have to have a PhD or some kind of specialty to do cool, genuinely groundbreaking work. You could just be someone who was slightly technical, slightly creative. And it was fun. Hard, but fun.” Wayne agrees, describing his experience of The Lord of the Rings years as “a grand, ridiculous adventure”.

Kimball Thurston, now Chief Technology Officer at Wētā FX, describes himself as “an outsider looking in” during that time. He was working at hardware and software solution provider RFX Inc, making compositing software that the Wētā FX team occasionally used, watching the trilogy unfold with a mix of admiration and fascination.

“What the whole VFX industry was trying to do was to capture a human performance and then integrate and manipulate that in a way that looks like it’s something fantastical,” he says. “So, everyone was looking at Wētā.”

The early Wētā FX operation was small and scrappy, with no off-the-shelf solution for the problems they faced. If they needed a tool, they had to create it.

“The pipeline was written by the artists themselves,” says Ken. “There was no sanctioned route to doing most tasks.”

Wayne remembers it the same way: “If we didn’t have something, we would just write a tool to do it. And that gradually became an ecosystem that then got developed into much more sophisticated, formalised ways of doing things.”

Weta’s approach was, paradoxically, precisely what made the work so influential. Ideas like deep compositing, which lets artists manipulate layered 3D elements without constant re-rendering; the “digital intermediate” process for colour grading, VFX integration and mastering; and sophisticated crowd simulation systems are now standardised across the industry. However, they were all either invented or significantly advanced at Wētā FX during the Lord of the Rings years.

Beyond all count

Of all the technical achievements from the trilogy, the crowd simulation software Massive (Multiple Agent Simulation System in Virtual Environment) arguably received the most public attention. But Wayne says it may also be the most misunderstood tool created at Wētā.

“People always say ‘Oh, the crowds were rendered with Massive’ but in reality, what Massive generates is expansive motion. The art of rendering pictures from it used other processes.”

Wayne worked on multiple crowd sequences in the trilogy, including the battle at Helm’s Deep, which pitted 10,000 Uruk-hai against 2,000 defenders in an epic scene that regularly tops lists of the greatest ever cinematic battles.

“There was a huge amount of software that we put in place early on for the crowd scenes,” says Wayne: “Like assembling bits and pieces at render time to handle massive data sets. It’s everywhere in industry now, but none of that stuff existed then.”

Being at the cutting-edge often leads to competition, but it can also result in collaboration. Wayne continues: “The thing I love about this industry is that while we compete with different visual effects houses, there’s a mammoth amount of knowledge-sharing that just goes on, because we’re all trying to work out how the hell to do this stuff.”

It seems appropriate, then, that in May, Massive released a “fully featured” version of their software (and video tutorials) free for personal use.

Riddles in the dark

While Wayne looked after the advancing hordes, Ken faced another question: how to create the translucent, eerily convincing skin of a key character, Gollum.

“The skin shading was my primary focus when Joe [Letteri ONZM] brought me here,” he says.

“I was a self-taught programmer with an engineering degree but no official training in coding. I was still able to do all this, building off other things.”

When starting to work on Gollum, Ken came across a then-recently published research paper on subsurface scattering, which modelled how surfaces reflect light based on mathematical functions called Bidirectional Reflectance Distribution Function (BRDF).

“This was the first time I worked with different BRDFs – different lighting or shading models – because in my previous projects, we’d just used fixed numbers.”

While familiarising himself with it, Ken remembered work he’d done on underwater rendering which involved “tricking” a renderer called Renderman into producing data outside standard pipelines. “I realised that this trick would help us implement the paper. It was very much an inspired hack,” he says. “Nothing rigorous about it.”

Much later in the process, Ken realised that his model contained a mathematical error.

“There was basically a double exponent going on. But when I fixed it, the ‘incorrect’ version ended up being more artistically controllable and better looking than the ‘correct’ version. So, that’s what we used.” He laughs, adding: “I imagine if I was a properly trained scientist or software engineer, I wouldn’t have discovered that.”

Error or not, the results were so effective that in 2004, Ken, along with Christophe Hery and Joe Letteri, won an Academy Technical Achievement Award for their work.

An unexpected pipeline

A quarter of a century on from Fellowship, the demands and expectations placed on VFX studios continue to grow, powered by improvements in technology. Ken articulates a principle that rings true across the history of computing: “Increases in render efficiency or render speed don’t make your frames render faster. They mean you’re allowed to do more complex things.” Kimball notes with some amusement that this is now known in the industry as Blinn’s Law, after computer graphics pioneer Jim Blinn.

Despite the expansion, the self-sufficient, build-it-yourself culture Weta Digital was founded with remains a part of Wētā FX. Kimball describes the very first Wētā FX pipeline – for 1994 film Heavenly Creatures – as “… a film scanner, an SGI computer and a film recorder. That was it.” Three decades later, Wētā FX runs a hybrid graphics processing unit and central processing unit data centre with around 80,000 cores of compute. This year the company received an APAC data centre design award for its water-cooled render wall.

“Just like we innovate in pictures, we also innovate in how we efficiently use technology,” says Kimball.

He describes managing the data and pipeline integrity that sits behind all of this as “a constant battle” but one that three decades of tool-building has made significantly more tractable. He adds, with a grin, that he wouldn’t be surprised if some of Wayne’s original commits – save points for code – from The Lord of the Rings era are still sitting somewhere in the codebase.

AI is, of course, having an impact, but as Kimball explains, “We’ve been using forms of AI tools or machine learning tools to create artist-driven effects, really for decades, before we even knew to call it AI at all.” To him, each wave of technology that threatened to displace filmmakers (television, digital, streaming) only caused the industry to expand. He argues that the key to making sure that continues is to keep humans in the creative seat: “Don’t take solutions which do it all for you. Question these things. Experiment.”

All manner of folk

Wētā FX’s work attracts people from a wide array of disciplines. Ken studied electrical engineering but moved into VFX almost immediately. Wayne and Kimball came from computer science backgrounds. The team around them has included biologists, astrophysicists – Kimball notes they’ve recently hired two – and even a rocket scientist.

“Historically, our industry has been very diverse,” says Kimball. “And that is an incredible benefit.”

Ken describes an office he shared with other technical directors at visual effects company Industrial Light & Magic: “I had an engineering degree, one had a computer science degree, one had an art degree, one had an English degree, and one had no degree at all.”

For engineers or students thinking about this as a career path, the message from Wētā FX’s veterans is less about specific disciplines and more about disposition.

“The most important part is having a sense of adventure,” says Kimball. Wayne, who traces his love of visual effects back to watching the original Star Wars movie as a child, puts it simply: “If you’re a seven-year-old that gets really excited by a spaceship going overhead, it really doesn’t matter what your background is, you’ll love working in this industry.”

With the 25th anniversary of The Fellowship of the Ring this December, that engineering spirit might be the most enduring legacy of these films.

“We definitely were aware that what we were doing was pretty groundbreaking,” says Ken.

“And we just could not wait for people to see it.”

This article was first published in the September 2026 issue of EG magazine.