How Many Skin Colors Are There

This commodity originally appeared in Yale Engineering Mag.

Before long after arriving at Yale, Theodore Kim was invited to requite a invitee lecture on the history of calculator-generated imagery (CGI) for a Moving-picture show and Media Studies class.

"While I was assembling the materials, a pattern became clear to me," he said. That is, the degree of bias that exists in calculator graphics technology toward the features of white people. He met with the professor of the course, John MacKay, who confirmed that the design was real — and deep-seated in film history.

"He introduced me to the book 'White' by Richard Dyer, which described how similar biases pervaded film engineering in the analog era," said Kim, associate professor of computer science. "From there, it became articulate that there was a whole body of scholarship on this topic, merely its coverage of the digital historic period is still ongoing, especially with movie CGI."

In recent decades, the technology of computer graphics has fabricated remarkable progress. Still, when information technology comes to matters of race, and the ways to describe characters of different indigenous backgrounds, the field remains very much in the by. Kim, who co-leads the Yale Computer Graphics Group, is among those who are trying to change that. He's seen the issue from the perspectives of both the manufacture and academia. Earlier coming to Yale, he was a senior research scientist at Pixar, where his work can be seen in such movies as Cars 3, Coco, Incredibles 2, and Toy Story 4.

From the ground up, estimator graphics technology has been developed with the notion that the skin and hair of white people are the default when information technology comes to depicting humans. For instance, manufactures in computer graphics journals often include but computer-generated images of white people when discussing skin rendering, fifty-fifty when the topic is broadly claimed as "humans." And many of the lighting techniques used in computer graphics are based on guidelines for movie lighting developed earlier the 1940s — long before the modern calculator — and specifically designed for white skin.

It's a trouble that severely limits what computer graphics artists tin can exercise, and how broad of an audition they can achieve.

"We're supposed to exist the leaders in storytelling," said Kim. "In that location are lots of stories out in that location and we haven't told a agglomeration of them, so allow's get tell these stories."

Thanks in part to the efforts of Kim and others in the field, at that place's more awareness about the issue. Because racial bias is and then securely baked into the applied science, though, there's no quick ready. For instance, early in the evolution of models for human being features, computer graphics turned to the medical literature for guidance. Despite having the imprimatur of "hard scientific discipline," Kim notes, it turned out that much of the literature was made with the same biases, with Caucasian skin and hair being treated every bit the standard.

Gallery of user-generated character examples from the Arnold website. Commercial rendering programs like Arnold lack the full spectrum of human skin shades and hair types.

"We thought we were doing the right thing by going to the medical literature, but instead we inherited all the aforementioned things," Kim said. "Everybody needs to be more than careful almost this stuff and think a lot harder well-nigh what we're doing. Nosotros're trying to develop technology that nosotros claim is for all of humanity."

The commencement step is getting a substantive give-and-take going in the customs.

"At the very least, we're locating people who actually intendance most information technology," he said. "From in that location, it's a community-building exercise. There are people who care about it, merely we demand to form a community."

A big pace toward that goal happened after Kim published an article nigh the issue in Scientific American magazine in 2020. From that article:

Today's moviemaking technology has been congenital to tell white stories, because researchers working at the intersection of art and science take allowed white mankind and pilus to insidiously get the only form of humanity considered worthy of in-depth scientific inquiry. Going forward, we demand to ask whose stories this technology is furthering. What cases have been treated as "normal," and which are "special?" How many humans reside in those cases, and why?

That article got the attention of many others in the field with like concerns. Kim and an "all-star cast" of co-authors that includes beau Yale informatics professors Julie Dorsey and Holly Rushmeier submitted an extended abstract to SIGGRAPH 2021, a prestigious briefing for calculator graphics and interactive techniques hosted past the Association for Computing Machinery.

"You never know what's going to happen when bug are this controversial or hot-push," Kim said. "And what happened was we got seven reviews, which is usual. Five were extremely positive, i was neutral. And 1 was virulently negative, and in fact contained coded racist messages, and this person forced it to get rejected."

But Kim was invited to exist the opening speaker for the upshot'southward Multifariousness, Disinterestedness, and Inclusion Summit to give his talk, "Anti-Racist Graphics Research." He and Rushmeier as well led a town hall-style gathering, otherwise known as a Birds of a Feather, titled "Countering Racial Bias in Computer Graphics Requires Structural Change." The goal was to get others interested in joining them in submitting a broad range of extended abstracts for SIGGRAPH 2022. The rejection of the previous submission, Kim said, fabricated information technology clear that it was a "numbers game."

In his talk, Kim discussed a common lighting technique in reckoner graphics known every bit subsurface scattering, which creates a glowing effect. It adds realism to white pare, but is much less important in darker tones. While at that place are means to add together paint to the default white pare to make darker skin, details are lost in the process. The technique is even codified in elaborate mathematical equations, creating the sense that rigorous science is backside it.

"We carved out the slice of physics that's near important to white skin," he said. "This is not all pare."

The notion of bias built into technology can be specially distressing to people in the field who are used to thinking that "math is math."

"That's what attracted many of us to research to begin with," he said. "Nosotros become to look at these make clean, neutral problems all twenty-four hours don't get all tangled up in the ugly politics of the real world."

Raqi Syed, ane of Kim's co-authors, said she noticed the problem while working on a project in 2018, and "trying to make a grapheme look like me."

"I became aware that if I wanted to tell stories that reflect my experience and use the tools that I empathize from working in visual furnishings, and then that'south going to exist really challenging, because these tools aren't designed to practise that," she said.

A.M. Darke, another co-writer of the paper, encountered the results of anti-blackness bias in graphics technology while creating a virtual reality space called "In Passing," a 3D media project about how people navigate public spaces. When developing the avatars for Black characters, she establish a very express range of hairstyles that she could use. This prompted Darke to create the Open Source Afro Hair Library, which gives users a wider selection of hairstyles to choose from for their characters. When Darke tweeted about an award for the library, it went viral — a sign that this effect resonates well outside just the figurer graphics community.

"The response was really positive considering this was something that had already been understood tacitly in a non-specialist community," said Darke, an assistant professor at the University of California, Santa Cruz, in the department of Performance, Play and Pattern.

Spreading that awareness to the community of specialists is the next of import footstep.

"The mode we solve these issues is collectively, by opening upward a dialogue," Darke said. "The aim of what we did at SIGGRAPH was to encourage others in this community to write and research and get down this line of inquiry, and then that this knowledge and expertise tin exist made available, and so this community can exist amplified and heard."