Deforum Experiment - Hybrid Video Input / 3D Mode by reallybigname - Stable Diffusion

🔊 Music From YouTube Audio Library
“URL Melt” by Unicorn Heads

⚙ Technical Details
I've been programming a hybridized mode in the Deforum v0.5 Python notebook that allows for Video Input during 2D or 3D renders. I don't know Python though, lol. Anyway, what I'm doing is mixing video frames in with the previous frame in different ways (blend, composite, etc). This one is using a composite. Composites require a mask. So, I make a mask using the difference of the prev_img and video frame, then use that mask in the composite to get sent back in as the prev_img. This is not to be confused with the mask feature in deforum. I do have another feature where you can use the mask I'm creating behind the scenes as the mask for the frame, but that isn't being used here.

I'm rendering on my home desktop with a reconditioned MSI 12GB 3060 ($325) running Windows 10, but running Ubuntu Linux under WSL, running Python using Anaconda, running Jupyter Notebooks, running deforum. This render was at 1024x576 resolution, which is exactly 16:9 (no cropping), and happens to be about the max size I can render anyway. Using euler_ancestral sampler (my favorite), set at 25 steps, at strength 0.8 though, so all but the first frame only did like 5 steps. Render took 2h 54m 11s. Post processing in Vegas Pro 19. I just used normal upscaling (no AI) to get to 1080p, but I used optical flow to up the 15fps to 60fps.

The original video I made some years ago was twice as long. It was just a live recording from GTA V (before they released the Rockstar Editor). I inserted classical music with the YouTube Audio Library after uploading from Xbox One. Original can be seen here:
https://www.youtube.com/watch?v=at2SxDorIOc

animation_prompts = {
0: "infinite diorama where everything is made of legos",
250: "infinite diorama where everything is made of finely detailed origami, folded paper, construction paper, folds, creases",
500: "infinite diorama where everything is made of sparkling jewels, jewelry, and precious metals",
750: "infinite diorama where everything is covered in urban graffiti",
1000: "infinite diorama where everything is steampunk",
1250: "infinite diorama where everything is in the style of a gingerbread house",
1500: "infinite diorama where everything is made of wooden toothpicks",
1750: "infinite diorama where everything is made of fruit",
2000: "in the style of r crumb, maximalist, hyperrealism"
}

#aiartist #aiartist #deforum #stablediffusion

🌲⭐🌃
🤜 Forest Star Walz
👉 #reallybigname
https://www.youtube.com/reallybigname
https://www.facebook.com/reallybigname
https://www.instagram.com/reallybigname
https://www.pinterest.com/reallybigname
https://www.twitter.com/reallybigname
https://www.snapchat.com/add/reallybigname
https://www.soundcloud.com/forest-star-walz