Doors of Perception | AI Short Film 🎥

This is our submission for the 1 billion followers summit AI film competition!

For as long as we’ve been human, we’ve had to compress our experiences into language and symbols. A new technology changes that by letting people step back into their past exactly as it happened and even share dreams with others. It becomes the next step in the evolution of language, moving from words to pure experience, letting us finally communicate what we could never say.

CREATION PROCESS:

We started off by generating our images with imagen.

Then used nano banana pro to edit them to make them look more like our characters. This part was crucial so we could have good character consistency throughout.

We would then have to up res each image and manually splice on heads and faces and sometimes clothes in photoshop.

Since our story follows two characters as they get older nano banana was super helpful in helping us age each character accordingly. We would make character references for each character at each age and use those to help guide the model.

We would then run each image through VEO carefully prompting for camera angles, character performance, and scene dynamics ie: clouds moving, grass blowing in the wind.

We used a lot of first frame last frame to get seamless transitions from scene to scene in many cases.

We wrote the narration ourselves with some help from chat gpt and Claude to help spice it up a bit.

We spent a long time in eleven labs creating the perfect voices for the voice over and dialogue. We used chat gpt to help us describe the voices we were going for to make the perfect prompts for eleven labs.

For the lip sync we used a mixture of Act 2 from runway and Higgsfield. Recording a facial performance with Act 2 and using just our existing dialogue for Higgsfield.

We did all the video editing in Davinci Resolve. There is a lot of post production work that went into this.

We made all the music ourselves using ableton including all of the foley and sound design. We are singing on the song you hear at the end of the short.

We went through a lot of challenges to get this film looking the way we imagined it in our heads.


Originally we had planned on making every transition from scene to scene seamless with first frame last frame. That proved to be very difficult given the random nature of AI outputs. Each one you see is the result of a ton of trial and error and it was just too unrealistic to do it for every single scene.

Another hurdle that we faced was getting consistent characters from scene to scene at so much different ages. It’s obviously not perfect but we spent a crazy amount of time trying to make it look good enough for you to at least not be taken out of the story while watching. Again we generated hundreds of versions of these characters to get the correct look as they were aging. We spent along time in photoshop manually adding new faces and heads and clothing onto images that we generated.

To bring AI films to the next level it really takes a lot of touch up that people would usually skip in both pre production and post production phases. Ironically it’s the stuff that people usually think AI would handle for you.

I think the last thing that was really challenging for us was the voices. We spent a ton of time perfecting the realism of each voice. We really wanted them to sound human and conversational and not like an AI audiobook. We generated hundreds of voices while slowly adjusting the prompts to get them just right. We think the results really speak for themselves (pun intended). We’re super happy with how they turned out.

Hope you enjoy!