NEW YORK, NY.- Lance Weiler is preparing his students at Columbia University for the unknown. “What I’m going to show you might disturb you,” he warned the class in January, at the beginning of his graduate course on digital storytelling.
A filmmaker who made his reputation on the frontiers of entertainment technology by inventing a popular augmented reality game around his film, “Head Trauma,” Weiler parlayed his experiments into a job at the School of the Arts, where he shows how computers might become creative partners instead of professional dead ends. His classes have combined augmented reality with Edgar Allan Poe, virtual reality with Sherlock Holmes and machine learning with Mary Shelley’s Frankenstein.
Now, Weiler wants his students ready for an art world that is gradually embracing the latest digital tools. The widespread availability of artificial intelligence programs that serve as image and text generators is upending the rules of cultural production — and changing how students learn what it means to be an artist.
The 53-year-old professor grinned from behind his graying beard with the enthusiasm of a mad scientist meeting his test subjects. He told his class in a dramatic whisper: “I’m going to show how you can leverage these technologies in your artistic practices.”
“The class is about daring students to embrace the machines,” Weiler later explained in an interview, shedding his prophet mystique.
For months he has been rehearsing his students and their AI creations for a workshop this week at New York’s Lincoln Center and a performance at the Music Center in Los Angeles in the fall, where representatives from the art and entertainment industries will be in the audience, looking to hire young recruits. These immersive performances, co-productions of man and machine, employ AI programs like ChatGPT and Midjourney, which can produce scripts and artworks based on algorithms and replicate human creativity by devouring billions of datapoints from across the internet. Whether Weiler’s students find them tools for their own creativity is yet to be seen.
“I keep both programs open on my screen at all times,” said Behrang Garakani, 50, an aspiring artist who returned to college after a career in video game development. In Weiler’s class he uses AI to storyboard ideas that he didn’t have the technical skills to draw. “This is now part of my artist’s toolbox,” he said, comparing ChatGPT and Midjourney to the way Photoshop has become indispensable to photographers.
During the rehearsals, Weiler frequently reminded students of what the inventor Buckminster Fuller once said: “We are called to be architects of the future, not its victims.” Sometimes he needed to remind himself of that. He wasn’t some ivory-tower expert but a self-taught practitioner whose struggle to understand emerging technologies happened alongside his students. The risk of failure and criticism from his academic colleagues was high.
“Not every class lets students use ChatGPT, because many professors assume that it’s plagiarism,” said Haiyu Zhang, a student in Weiler’s class, referring to the ongoing debates over the truth of AI-generated “pictures.” “But he really emphasizes a hands-on perspective with these tools.”
Zhang, 22, an undergraduate studying information science and comparative literature, petitioned to enroll in Weiler’s graduate course. Like other students, she didn’t want to call herself a traditional artist but a coder, futurist and worldbuilder. “A lot of people are scared about this technology,” she reasoned. “But not me.”
AI: Tool or threat?
Weiler isn’t the only teacher experimenting with AI. Art schools nationwide, from the California Institute of the Arts to the Rhode Island School of Design, are offering young artists courses that prepare them to code with machine learning or employ programs like Midjourney, Stable Diffusion, and DALL-E 2 — image generators that convert words into images within seconds. Artists use them as the foundations of creative projects that might still involve traditional mediums like painting and illustration.
In February, Pratt Institute in New York hosted a seminar for its faculty to consider the impact of new technologies. “There is so much precedent for this type of experimentation,” Jane South, the school’s fine arts chair, said in an interview. She pointed to previous technological inventions that critics worried would kill the artist profession but only made it stronger. “Photography was supposed to be the end of art,” she said, “and then the Xerox machine came along and that was supposed to be the end of art, too.”
The artist today is expected to produce meaning, not just images, she said, and technologies can help develop new meaning about contemporary life.
“The more things are easily reproducible, the more valuable unique objects are to collectors,” South said.
Not everyone agrees. Some working artists — designers, illustrators, animators — have characterized artificial intelligence as an existential threat to their business models.
Last year, a Polish artist named Greg Rutkowski found his name being used to prompt fantastical images in Stable Diffusion. He claims that convincing forgeries of his own work, made by the AI, were confusing his buyers. In another case, an artist who goes by @ato1004fd on Twitch live-streamed a drawing session only to discover that a viewer had used the NovelAI image generator to complete the illustration; the impostor then accused the real artist on social media of being the copycat. And in January, three illustrators filed a class-action lawsuit against several AI companies, alleging that their image generators violated copyright and unfair competition laws when engineers designing algorithms generated images in the style of living artists.
“We’re taking our consent back,” Karla Ortiz, one of the illustrators, told The New York Times in February. “That data is my artwork, that’s my life. It feels like my identity.”
A spokesman for Stability AI, the company behind Stable Diffusion, one of the companies that was sued, said in a statement that anyone who “believes that this isn’t fair use does not understand the technology and misunderstands the law.”
Leaders in artificial intelligence research say that it’s impossible even for them to understand everything about the technology. “It has fundamentally changed over the past few years,” said Meg Mitchell, the chief ethics scientist at the AI firm Hugging Face.
Mitchell said that many companies are now declining to explain how their programs are getting smarter, citing fierce competition within the industry. When OpenAI released its latest model, GPT-4, for example, developers refused to share the details of their data set.
“I tend to be a buzzkill about the data,” Mitchell added, suggesting that within the industry, data has “been collected without the consent of artists, without giving them credit and without compensation.”
Weiler doesn’t sweat the legal challenges to artificial intelligence. He prefers to nurture the iconoclasts and rulebreakers, who, he said, “are challenging the status quo of how art is made and who gets to make it.”
That struggle, against the perceived gatekeepers of art, was once his struggle.
Weiler was raised in the suburbs of Philadelphia and joined the local film industry in the 1990s after high school. Those early years of being a journeyman and camera assistant on large commercial shoots ended in 1998, when he helped direct an indie thriller, “The Last Broadcast,” on a shoestring budget of $900. Celebrated as the first feature-length movie with an entirely digital distribution, it took in over $5 million in revenue.
Success encouraged Weiler to keep tinkering with emerging technologies, and he is willing to try anything once. He describes all the doodads in his digital workshop as “enchanted objects,” and cites the science-fiction writer Arthur C. Clarke, who wrote, “Any sufficiently advanced technology is indistinguishable from magic.”
In 2006, Weiler started incorporating elements of immersive theater and augmented-reality gaming into his film that had audiences interacting with characters through their mobile phones. It’s the kind of history that keeps students coming back for more. The digital storytelling lab will be the third course Garakani has taken with Weiler, who he says is “distilling something you can’t read in textbooks.”
But he takes a more cautious approach to artificial intelligence than his mentor. “I am still struggling and haven’t come up with a conclusion,” Garakani admitted. He hopes that companies will become more transparent. “From an academic perspective, it’s like — cite your sources.”
Other students have suggested that Weiler embraces the magic of artificial intelligence without fully grappling with its repercussions. For example, the digital storytelling lab might teach artists how to use image generators, but it doesn’t teach students how to build their own algorithms.
Weiler said in response that Columbia offers other courses that go into the computational side of programming artificial intelligence. “What we want to do is introduce students to emerging technology.”
Tryouts on the New Frontier
In late March, Weiler’s class gathered inside Lincoln Center to present their artworks to an audience of arts professionals.
The evening started with a provocation. Weiler asked everyone to close their eyes and imagine what values should be passed onto the future. Then the students broke into groups and came up to a podium to pitch their AI-generated artworks.
Zhang and her teammates adopted an ominous tone in “Dream Apocalypse,” an hourlong immersive experience that used AI to imagine a world where society has crumbled. They described it in their pitch as “‘Your greatest nightmare’ meets ‘Inception’ meets ‘Russian Doll’ meets Sisyphus.” Audiences would confront their fears, prompted by apocalyptic images from Midjourney, like one featuring a child gazing at the mushroom cloud of a nuclear blast.
Then Garakani’s team shared their moodboard — a collage of AI images — depicting a ruined aquatic civilization staffed by fishlike janitors. Participants would join in their goal to clean up the forgotten underwater world. Members of the group described the setting as “‘Guardians of the Galaxy’ meets ‘Atlantis’ meets ‘Fantasia.’”
But the concept art seemed to riff on “Star Wars” clichés like Baby Yoda and the Death Star, raising the question of just how original artists can be with tools that simply recycle pop culture memes.
These were just prototypes, and the students spent the next few weeks refining their text prompts until they could squeeze some originality out of the machine. “Refine, redo, merge, separate, contextualize,” Garakani repeated. “AI needs guidance to produce usable work.”
Based on feedback from the first audience at Lincoln Center, Garakani’s team switched topics and refocused on Carl Sagan’s Voyager Golden Record, two phonograph records with the sounds of Earth that NASA launched into space in 1977, for aliens to find. Midjourney produced one artwork for the team that looked like a fingerprint. Garakani figured it probably came from a student’s prompt with the word “haptic,” which might have inspired the AI program to generate an image related to the concept of touch: the cosmos in the shape of a fingerprint. It had students on Garakani’s team thinking about how participants in their performance might speak to aliens through a golden record of their own.
“It’s possible that we may have arrived at this point without AI, but the machine helped us spark this idea in seconds,” Garakani added.
More recently, Weiler unveiled his own AI work in progress, “Blockchain Fairy Tales,” which uses technology associated with cryptocurrencies and NFTs — non-fungible tokens — to depict crowdsourced fantasy worlds. The artwork will be shown later this year at the Music Center in LA alongside the immersive experiments of his students.
Zhang said that artificial intelligence allowed her to focus on the big concepts while the algorithm filled in the details. She predicted that wider adoption of these programs will put a greater premium on creativity.
“What makes artists special is their ability to imagine something new,” Zhang explained. “So while I think that AI tools help express our creativity, creativity will still be the driving force behind the future of art.”
From Weiler’s perspective, there wasn’t really another choice than for his class to embrace the machine. “What does it look like to slow down a cycle that is moving as fast as artificial intelligence?” he asked. “Well, nobody is slowing down. We’ve opened Pandora’s box. It’s already out of the box, man.”
This article originally appeared in
The New York Times.
