And the Academy Award goes to…
Perry Kivolowitz, professor and Chair of Computer Science at Carthage College, just won the Academy of Motion Picture Arts and Science Scientific and Technical Achievement Award for his computer software, SilhouetteFX.
The Academy’s Scientific and Technical Awards honor the men, women and companies whose discoveries and innovations have contributed in significant and lasting ways to motion pictures, according to the Academy Awards website. Honorees are celebrated two weeks prior to the Oscar ceremony. This year’s ceremony was held Feb. 9.
Kivolowitz says SilhouetteFX has been in continuous design and programming since 2004, developed by Kivolowitz and three partners unaffiliated with Carthage College. A product like this is often developed by dozens of people, according to Kivolowitz.
“It is pretty remarkable how small our team is, to produce a product that is so heavily entrenched in the industry,” Kivolowitz says.
“[Weta Digital] saw Silhouette demonstrate the very first version, pre-release at a trade show and said, ‘We’ve got to have this; where do we get it?’” Kivolowitz says. “They started using our software before it was introduced commercially.”
So what, exactly, is Silhouette? The software is used in three fundamental ways, according to Kivolowitz. First and foremost is rotoscoping.
“Rotoscoping itself, back then and within the modern era on computers, is the process of tracing the outlines which differentiate background from foreground,” Kivolowitz says. “Rotoscoping is used when you need to identity foreground from background in a more demanding environment.”
Using green screen or blue screen, sometimes even orange, is great when you’re in a studio or on a soundstage where lighting and background can be carefully controlled, Kivolowitz says, but it’s when a green or blue screen can’t be used that rotoscoping comes in handy.
An example of this, Kivolowitz says, is a helicopter lifting off from a parking lot in California that is supposed to be mountains in Afghanistan.
“It’s not reasonable to hang a giant blue curtain behind a real helicopter,” Kivolowitz says. “It can be a very time-consuming, labor-intensive process,” he says of rotoscoping the old-fashioned way. “Silhouette was given the Scientific and Technical Achievement award because of the tools that it offers to reduce the labor-intensive nature of rotoscoping.”
The second area in which Silhouette is commonly used is in Paint, a program similar to Adobe Photoshop.
“Silhouette was given the award for its ability to automatically apply repetitive paint strokes, adapting what an artist has done for one frame across many frames,” Kivolowitz says. “Potentially hundreds, even thousands of frames, taking into account that the thing being painted is moving, and sometimes it gets blocked by other things. It might get bigger, it might get smaller, it might rotate or it might move so quickly it gets blurred a bit. Silhouette is able to adjust for all of that automatically.”
Though the award did not include converting 2D to 3D motion, the third fundamental use of Kivolowitz’s technology, this feature is still important to the industry.
“The 3D nature of our perception is because we have two eyes, binocular vision. When a movie is filmed in normal ways with one camera, a second eye is missing,” Kivolowitz says. “So using a technique that is very similar to rotoscoping, Silhouette is able to synthesize the parallax of the missing eye, so that the parallax and depth information is then encoded for 3D projection so that both eyes see a slightly different perspective, giving us the 3D look.”
For viewers at home, the critical thing is what they don’t see. Kivolowitz explains it using an example from the 2016 movie The BFG.
“There is a scene in which a young girl stands in astonishment and has brightly colored fairies dancing in front of her face. The colors of the fairies wash over her skin and hair in a very real and natural way,” Kivolowitz says. “Doing that with a computer is pretty hard.”
Instead, the crew took light globes and attached them to broomsticks, moving the light around to project upon the actress, giving the correct physical lighting that the fairies had. Silhouette’s software was used to make sure the broomsticks and globes disappeared.
“The key is nobody saw any artifact left over by that. It’s not necessarily what you do see; it is what you don’t see,” Kivolowitz says.
Though Kivolowitz did not attend the Feb. 9 award ceremony, he says winning the award is the pinnacle of his career.
“In some ways, you feel relief because the process is very stressful and it is a relief when it comes to an end,” Kivolowitz says. “Of course, coming to an end may be a positive outcome or a negative outcome, but in my story, it’s a positive outcome.”
In previous years, this award has gone to developers of animation systems used by DreamWorks and Walt Disney Animation Studios, among other technological advancements (including something called The Animatronic Horse Puppet in 2017).
Kivolowitz says his experience watching a film is different now because of how long he has been involved in the film production and technology.
“I can’t help but watch a film critically,” Kivolowitz says. “Defects that go by in a blink of an eye, I see and I can’t help but notice. I have to very carefully put myself in a mindset where [I tell myself], ‘Okay, I’m not working, I’m just going to enjoy this film.’”
Surprisingly, Kivolowitz says, that is hard to do.