AI Animated This Movie: The Dream Life

Опубликовано: 28 Декабрь 2024
на канале: The Motivated Engineer

1,230

This is a film in which all scenes have been generated by AI model (stable diffusion). The script was written by Surya Penmetsa (with the guidance of another AI model GPT-3). I hope you guys enjoy the video.

More details on how the video was generated:
This is a short story based on the plot from William B. Irvine’s book “On Desire”, written by Surya Penmetsa (with the help of the AI language model GPT-3 [1]). All the images in the video have been generated by the AI model stable diffusion [2] using the prompt “a cartoon of [prompt] by Neal Adams”. All the prompt engineering to get the best suitable images are done by Surya Penmetsa. The 640x384p frames generated by stable diffusion have been upscaled to 3840x2304p using Vid2x model [3]. All of these images are imported into iMovie to make the final edits and render at 1080p.

Thanks for Vikram Voleti, Nishanth V. Anand and Raviteja Chunduru for the helpful discussions. Special thanks to all the amazing researchers and scientists who have developed these algorithms and made them easy to use.

References:
[1] https://paperswithcode.com/method/gpt-3
[2] https://github.com/CompVis/stable-dif...
[3] https://github.com/k4yt3x/video2x