Sora 2
I have been having a blast exploring Open AI’s latest incarnation of Sora. I actually used it to produce all the clips I edited together for my song The Button.
You can check out some of the things I’ve generated by checking out @wizardhead on Sora and if you’re registered you can create videos containing yours truly by using the cameo feature (just include “@wizardhead” in your prompt.)
Sora 2 is interesting because it has been designed to take a prompt and flesh out the concept of it before delegating to the various video models. For example, take a short prompt like “a 1980s tv commercial about cool pants for cats” and the result is what appears to be a fully scripted production of a tv commercial, complete with 3 different cats in cool pants, 1-800 number and an overlay that says “Pants for your Panther”. See it here
Granted, it is sloppy and there are all kinds of problems where characters and voices can be completely mismatched and timing can be wildly off, but overall it is much more complete and coherent than what you’d get by asking a video model to render some cats with pants. Under the hood it’s essentially just a preprocessor that takes a prompt and asks ChatGPT to generate a video production plan from it.
It has a couple of other useful features, to enable generating videos with consistent characters. There’s a “cameo” feature which lets you capture a video of yourself and it is able to let you (and anyone you give access to) create videos with a consistent version of you. And similarly there is a “characters” feature to let you upload short videos of other (non-human) characters to have them consistently participate in videos as well.
It’s a lot of fun, it can be super low effort, and it’s a great way to try out concepts.