I generated a photorealistic music video with AI (GauGAN2)
I fell in love with AI generated landscapes recently. Although this tool is only meant for images, I decided to see if making a cohesive video was possible. Choosing which parts of the landscape would represent different parts of the beat was a lot of fun! Maintaining continuity from one frame to another is challenging, but after much trial and error, I've found that minimizing the motion of the horizon maximizes cohesion between frames.
To support these projects and include your name in the next video, here is my Patreon page: https://www.patreon.com/doodlechaos
Discord: https://discord.com/invite/7FCrWAzDY7 (I spoil what the next video will be pretty often here)
Twitter: https://twitter.com/doodlechaos
Official tiktok: https://www.tiktok.com/@doodlechaos
Shorts Channel: https://www.youtube.com/channel/UCMqgJk1o2eWE7WeNtRIfnpg
Instagram Shorts: https://www.instagram.com/doodlechaos_shorts/
Email: contact@doodlechaos.com
Here is a link to the web version if you'd like to try it yourself!
http://gaugan.org/gaugan2/
Here is a quick explanation of my process for this project:
1. Animate segmentation maps in sync with music in After Effects.
2. Export this animation as a series of images.
3. Convert the image sequence into landscape photos via GauGan2.
I was able to automate the conversion process. My first attempt used AHK to move my mouse and click to upload the correct image, convert, download, and name. This was sketchy and actually crashed my chrome browser somehow (I think due to downloading so many images and they were all being cached?). Attempt two was to run it locally. Unfortunately, the NVIDIA Canvas downloadable software doesn't allow png segmentation maps as input (only .can files). My final solution works by sending images directly to the gaugan.org server via python. If you're interested in trying out something like this, I'll make the script I used available for download on patreon.
Resources:
https://www.nvidia.com/en-us/research/ai-demos/
https://blogs.nvidia.com/blog/2022/03/01/what-is-gaugan-ai-art-demo/
Music Credit:
Track: Lost Sky - Vision [NCS Release]
Music provided by NoCopyrightSounds.
Watch: https://youtu.be/7RU1l8GtGOc
Free Download / Stream: http://ncs.io/VisionID
Shoutout to Winkey for giving me the idea to make this Bob Ross themed.
-
51:39
Kimberly Guilfoyle
4 hours agoNO BORDERS BIDEN, Where We Go from Here, Live with Monica Crowley & Jaime Florez | Ep. 135
22.1K38 -
43:19
Game On!
2 hours agoJJ Redick land Laker's job, Patriots pay their young RB, and Dave Portnoy fed up with his employees
9.98K2 -
16:37
Scammer Payback
2 days agoHere's the Scammer's Location 👉 (28°34'05.1"N 77°24'00.9"E)
64.6K38 -
1:11:56
Battleground with Sean Parnell
7 hours agoDeep State Cover Up w/ Tony Shaffer
15.3K10 -
2:01:31
Revenge of the Cis
4 hours agoEpisode 1341 White Girl Summer
30.9K6 -
SwitzerlandPlayIT
6 hours agoLIVE - Retro THORsday - Blow The Ocarina! Blow It Good!
16.4K5 -
1:01:20
In The Litter Box w/ Jewels & Catturd
21 hours agoSCHIFFSHOW | In the Litter Box w/ Jewels & Catturd - Ep. 591 - 6/20/2024
55.2K40 -
20:44
Breaking Points
10 hours agoCNN Blames 'FAKE VIDEOS' For Biden Age Concern
39.8K25 -
2:24:05
Tucker Carlson
6 hours agoNeil Oliver: How Banks Took Over Empires, and the Truth About WWII, Brexit, & COVID
151K257 -
48:59
The Casey Adams Show
3 days agoDavid Sacks & Evan Owen - Glue: The AI Solution to Workplace Communication | The Casey Adams Show
53.7K6