AV

Engineering professors develop human-AI colorization tool for cartoonists

Body

AV professor Sungsoo Ray Hong has always been a huge fan of cartoons, comics, and animations. By combining his passion with his research area of human-computer interaction, he has created a new tool for cartoonists, ShadowMagic.

Hong, his colleague , and their PhD students published a paper, “” elucidating the need for the tool, which aims to boost product quality and designer productivity through study-informed human-AI interaction.

The team worked with professional cartoonists to find common workflow of colorization that is shared within the industry as a reference point for their research, which examines the evolution of designing technology as cartoonists transitioned from using PCs in the 1980s to artificial intelligence, said Hong, who is an assistant professor at the .

Creating a comic book is a team effort as there are many roles throughout the process including story drafting, sketching, colorizing, editing, and sending to a production company, said Hong.

“Professional cartoonists are sometimes producing content every other day,” said Hong, who is also director of George Mason’s . “You cannot commit to these kinds of deadlines doing everything by yourself. That's why the colorization industry is so helpful to professional cartoonists.”

Colorization is a sequential process of line drawing, flatting, base colorization, shadows and/or lighting, background and other special effects. The tool stems from a previous project, FlatMagic, another AI-driven tool designed to improve the creative process for digital comic professionals.

“This is a fairly standardized process, especially in South Korea and growing throughout the United States as well,” he explained. “The solutions to these network visualizations should be contextualized, but we have the limited resources. We can’t build everything for everybody. That’s my motivation here.”

Gingold also specializes in human-computer interaction and “has always been interested in sketching and how computers can help us express ourselves,” said the Department of Computer Science associate professor.

Yotam Gingold (left) and Ray Hong in Research Hall, presenting their colorization tool. Photo by Cristian Torres/Office of University Branding.

“We looked at artistic challenges, usually tedious things that aren’t so fun for the artists and break down those problems, so that they can focus more on the creative parts of it,” said Gingold, who is the director of George Mason’s .

There are millions of pixels in an image, and if an artist wants to move or change the color of a building for example, they point at one pixel among the millions. The artist wants the rest of the pixels to follow along naturally so the rest of the building should also change, he explained.

“The computers will do the math where we can write formulas to update all the other millions of pixels. Another example would be keeping lines parallel and corners perpendicular, and that’s where a lot of my research comes in,” said.

While a doctoral student at George Mason, Chuan Yan, PhD ’24, was recruited by Gingold for this project and conducted research on sketching and painting. Yan’s area of expertise encompasses computer graphics fundamentals and sketch-related computer vision techniques, which he had learned extensively from Gingold, he explained.

The ShadowMagic tool was built as an Adobe Photoshop plug-in, and since its release, the team “has garnered significant interest from artists inquiring about updates and expressing a strong desire for a tool that could free them from the labor-intensive flatting process,” said Yan, who is currently a postdoc researcher at Stanford University.

“This response is both exciting and humbling. It confirms that our research is addressing a real need in the industry yet also highlights the remaining challenges in making [the tool] ready for widespread adoption.”

Yan believes that as more sophisticated generative AI systems continue to emerge, AI-human collaboration will become increasingly vital.

“I am excited to witness and contribute to this transformation, helping to shape a future where AI acts as a powerful tool that amplifies human creativity rather than replacing it,” he said.