I expected it to be here six months ago, but its continued absence hasn’t changed my estimate from “any day now, and suddenly.” All of this is so weirdly democratized (and pornography-motivated) that we’re seeing the cool stuff before all the scary disinformation concerns.

And the underlying mechanisms are straight-up “the missile knows where it is, because it knows where it is not.” Stable Diffusion compares the noise estimate with and without a particular term, takes the difference, and then leaps outward along that vector.

Stable diffusion can do arbitrary sizes now, as long as you have the VRAM for it iirc

“Fisheye” generation seems obvious. Give the network a distorted view of an arbitrarily large image, where distant stuff scrunches inward toward a full-resolution point of focus. Predict only a small area - or even a single pixel. This would massively decrease the necessary network size, allowing faster training. (Or more likely, deeper networks). It’d also Hamburger Helper any size dataset by training on arbitrarily many spots within each image instead of swallowing the whole elephant.

Even without that, video only needs a few frames at a time. You want to predict a future frame from several past frames. You want to tween a frame in the middle of past and future frames. That’s… pretty much it. Time-lapse “past frames” by sampling one per second, and you can predict the next second instead of the next frame. Then the stuff between can be tweened.