2 years ago

I don't think Dream Studio has this yet!

2022·11·13 Deforum v0.5 Notebook Experiments
I've programmed a hybrid mode to include Video Input with 2D/3D.

- I'm doing this by extracting video frames like Video Input mode, but I'm mixing those into the prev_img in 2D/3D mode.
- I also just RANSAC'd Deforum. I'm using RANSAC method to analyze every two frames to get a transformation matrix which can then be applied to the prev_img in a similar way to how Deforum warps 2D perspective, but I'm doing it with a matrix extracted from video instead of with keyframe values converted to a matrix. And, 1792 frames only took 10 sec to process!
- The scale, translation x/y, rotation, and skew are transferred. Only every 5 frames were extracted @6fps. RIFE'd to 30fps.
- I'm also blending the original video frames in at 10% to each prev_img that gets fed to the Stable Diffusion generation. In this way, you can't see the original video at all. It only barely assists in the generation of each image.

“It essentially just helps guide the dream.” - reallybigname

My skills are in tuning things. I used to render a lot of 3D fractals and have been a digital artist/animator in various ways for many years. Once I can use a system enough, I see ways to make it better. I'm not the best programmer or math guy. In fact, I'm new to Python! But, I've been around... If anybody from Stability AI is reading this, you should hire me. I'll work for dirt cheap! Okay, actually... that's a lie. I know, that's a bad way to start a working relationship. It was a lie because really, I'll work for free - as you can see here. I shared exactly what I did. I'd just love to get paid something and have access to powerful servers! Ultimately, I'm an artist, and I'm compulsively driven to make more powerful tools to make cool stuff. I don't seem to be able to not create things. If I stop creating in one domain, I find myself creating in another. It'll come out one way or another. My only task is in focusing it on what I want.

#aiartist #aiart #aiartist #deforum #stablediffusion #stabilityai #ransac

🌲⭐🌃
🤜 Forest Star Walz
👉 #reallybigname
https://www.youtube.com/reallybigname
https://www.facebook.com/reallybigname
https://www.instagram.com/reallybigname
https://www.pinterest.com/reallybigname
https://www.twitter.com/reallybigname
https://www.snapchat.com/add/reallybigname
https://www.soundcloud.com/forest-star-walz