“Collaborating with the entire history of human expression”
Kirby Ferguson and Ale Matamala Ortiz on AI and filmmaking
A few weeks ago, we hosted an epic one-day festival to human expression in New York City called the Lux AI Summit, bringing together hundreds of founders, artists, engineers and visionaries who are redefining the future of media. Two of our speakers, Kirby Ferguson and Ale Matamala Ortiz joined us on the Riskgaming podcast to talk about the future of filmmaking in a generative AI world.
Kirby is a filmmaker, and he’s most well-known for his work around remix culture in a documentary series titled “Everything is a Remix.” Ale is the co-founder and Chief Design Officer of Runway, a Lux portfolio company that builds a generative AI studio for anyone producing films.
Kirby, Ale, Laurence, and I talk about the current economics of the film industry and the recession underway in Los Angeles, why filmmaking would have changed even without the rise of AI, how to think about remixing through software versus remixing through culture, the coming convergence of narrative forms like films and video games, and what the future of artists will look like with co-intelligences.
This conversation has been edited for length and clarity. For more of the discussion, please subscribe to our podcast.
Danny Crichton:
About a week ago, The Wall Street Journal had an article on the future of Los Angeles. There was a massive dive in the filmmaking industry. Obviously it’s in recession. Jobs are down 25%–30%. A lot of this has to do with the strike two years ago, but there’s also an AI component, which is that filmmaking is entirely changing in the next five to 10 years. The solutions we used in the past seem to not be what we’re going to project forward. So from both of your perspectives, what is filmmaking going to look like in the next couple of years?
Kirby Ferguson:
What a nice softball question to open with. Holy crap!
I’m skeptical about how much AI has to do with it, really. I’m not sure it has much of a role. It seems like it’s more a systemic economic thing. What do you think?
Ale Matamala Ortiz:
Yeah, I agree. When I talk to studios, to teams, one of the things that comes up often is that films have become really expensive to create. That’s one of the main reasons some of the teams are not making more films.
But I do think AI is going to transform the industry. On the positive side, we see that it is going to enable us to create new things that were harder to create before. We’re already seeing it. Somehow, some teams are getting the green light for projects that were not getting green-lit before, because they were either too expensive or too ambitious. So we are seeing it from a very positive side.
Kirby Ferguson:
I think there was just some oversupply going on for a while there, too. Do we need this much Star Wars or whatever?
Laurence Pevsner:
As I think about oversupply, one of the pros and cons of AI in terms of this field is that it’ll empower so many more people to become creators. We already have vibe coding, and we’re starting to get into vibe filmmaking. I’m curious what you think is going to happen as more people get access to these tools?
Ale Matamala Ortiz:
That’s been one of the key things since we started: democratization of who can actually tell the stories. I come from Chile, I dropped out from film school, but I wanted to be a filmmaker.
The idea of creating a film back in Chile was even more crazy than perhaps creating an AI lab here in the United States. Creating a film is hard, it’s complicated even in the traditional places, and we do believe that with these tools, with this technology, we’re going to start seeing more stories being told.
Generative AI is going to be one of the things that will unlock different creators around the world. We ran an AI film festival and we received 3,000 films from teams all over the world. People created films that ranged from five to seven minutes, and those were local stories that people wanted to tell but they otherwise couldn’t.
Laurence Pevsner:
Any favorites from the film festival?
Ale Matamala Ortiz:
The winner is a must-watch. It’s really, really good. It’s a story of how many images you can create within one image. So if you take the math of the pixels that create an image, you can create so many more possible images. It’s a very well-narrated film.
If you look at the history of any genre, any art, it’s poor people who are outside the system who look at these things in new ways and create new stuff.
Kirby Ferguson:
I think regular people will probably make the most exciting stuff with these tools. If you look at the history of any genre, any art, it’s poor people, it’s amateurs — like blues music, like jazz, whatever — it’s poor people who are outside the system who look at these things in new ways and create new stuff. So I think whole new genres are going to emerge out of this. New styles of filmmaking and new voices. That’s probably the most exciting part of it to me, rather than just professionals who are trying to get cheaper or make their stuff higher quality or whatever.
Danny Crichton:
I’m reminded of careers like Chris Nolan’s. His original film [Following], even before Memento, was a $100,000 budget film he did in school. And then Memento was also a very, very cheap film. He’s progressively grown into these massive Hollywood budgets. But to your point, democratization means that there will be many more Chris Nolans.
Kirby, you’ve been a filmmaker a very long time, but one of the themes that comes up frequently in your work is remixing, this idea of recombining previous material. Remix culture is something we see in certain fields, certain subcultures. But in others, it’s verboten, it’s considered offensive, you’re not an original artist. Talk about AI and remix. What do you see as the future of the culture there?
Kirby Ferguson:
It’s super fascinating, because AI is a remix, of course. It’s deep learning, it’s based on training on existing stuff and then creating new stuff out of the old stuff. I think a version of that is what we do for all sorts of creativity. So it mirrors us, in a way. There’s obviously a lot of stuff that is not in the AI mix — mysterious aspects of the human imagination are not there. But there’s this interesting parallel between the two, between of how it creates things and how we create things.
So it’s interesting just learning about how AI works and using it, because I do think it is a mirror of how we work. We don’t create out of nowhere, we learn music, we read books, we watch movies, and we start copying what other people are doing, and then we start putting our spin on it and combining it with other things we get from other places.
Laurence Pevsner:
As we think about AI in film, what do you think even defines it? We’ve had CGI forever, and that’s a form of using computers to advance film technology. Do you think there’s a distinction, or are we just looking at a curve of different kinds of advancing technology that we’re going to keep using as filmmakers adapt and use new tools?
We don’t create out of nowhere, we learn music, we read books, we watch movies, and we start copying what other people are doing, and then we start putting our spin on it and combining it with other things we get from other places.
Ale Matamala Ortiz:
At the AI film festival, we have a preface that, “some AI was being used to create this idea.” But I don’t think we’re going to be talking about AI films in the near future.
I do believe that as this technology progresses with new capabilities, such as real-time interactive outputs and multi-shot sequences, we can aim to have different kinds of stories — different ways of telling stories — that are perhaps going to blend what we know today as films and what we know as video games. I think then we’re going to start talking about new mediums rather than whether something is made with AI or not.
Danny Crichton:
As a filmmaker, when you think about authorial vision, we have this focus on how the director is in charge of the film, all the shots, all the framings. When you think about AI, does it empower the authorial vision as a collaborative agent for you?
Kirby Ferguson:
To me, AI is this social thing, where you’re collaborating with the human imagination itself.
I really like Ethan Mollick’s book Co-Intelligence on this subject. I like the idea of AI being a quirky, brilliant, weird collaborator. It’s brilliantly good at some things and incredibly inept and stupid at other things. I don’t think of it as collaborating with a machine, though, I’m collaborating with the entire history of human expression. Everything that was on the internet that it got trained on — I’m collaborating with that. So to me, it’s this social thing, where you’re collaborating with the human imagination itself.
Ale Matamala Ortiz:
Yeah. I see different layers. One is, as a creator, the way we design our tools and our software and our models is to make them collaborators, to empower artists to do the things they want to do and help them achieve their ideas. We’re not creating or designing systems that are one-pronged, one shot, give me a final output. But I do realize there are some use cases that might be very powerful on that end.
But even within the creative workflow, the collaborator can also expand to be something that can be more autonomous, one that gives you some results that perhaps you were not thinking of. Or, maybe it’s too complicated to do, or maybe you just need someone else who can run and do that thing. So on a feature film, a creator might have some ideas — this is the place, here is one scene — and they want to have fine control over that scene. But perhaps within that same feature film, there is a sequence of a car going through Manhattan that is one-minute long. Maybe we can use systems that are more autonomous, that can create multi-shot scenes, that can deliver a longer take with different scene cuts, that can also work for that creative.
On the other side, more on the consumer side of things, I think there is a future where we will be generating videos to consume on-demand for different use cases — for learning, for support, for entertainment in some cases. These are perhaps going to be a little bit more disposable, but they’re going to be run on their own by a machine.
Danny Crichton:
With films now, they are definitive — specific. They are the way the creators want them to be watched. With AI, you have this capability to say, not only can a film be in different formats, it can be in different languages, we can auto-translate, it can be dubbed, including with the right lip movements. And you could potentially change plot lines, it could be adaptive, the story itself could change, the film could change.
Does this worry you? Does it change the definition of filmmaking? Do films go from a singular vision to something where the film is more of a world, an environment — something you can explore?
Kirby Ferguson:
I think the scenario you outlined there, where AI can translate and convincingly convey it to cultures that don’t speak your language, that seems like a total win for me. Anything that can extend the reach of a filmmaker is definitely a huge win, so I love that aspect of it.
Ale Matamala Ortiz:
Why does a TV show have to end? Why do we have only one season finale? Why can’t we have the same season finale from a different character, from a different point of view?
To me, this is one of the most interesting opportunities, to be honest. We shouldn’t be designing this type of technology to continue doing everything that we have done in the same way as in the past. We should push for new ways of working and new ideas and new ways of telling a story.
Today we have films that end, but why do they have to end? Why can’t they keep going? Why can’t a viewer take action and continue the story, or maybe see the story from a different angle? Why does a TV show have to end? Why do we have only one season finale? Why can’t we have the same season finale from a different character, from a different point of view?
Those are the opportunities that we can start considering now. I’m not saying that we are there yet, but these are the opportunities that we should be looking at.
