I’ve been re-watching star trek voyager recently, and I’ve heard when filming, they didn’t clear the wide angle of filming equipment, so it’s not as simple as just going back to the original film. With the advancement of AI, is it only a matter of time until older programs like this are released with more updated formats?

And if yes, do you think AI could also upgrade to 4K. So theoretically you could change a SD 4:3 program and make it 4k 16:9.

I’d imagine it would be easier for the early episodes of Futurama for example due to it being a cartoon and therefore less detailed.

71 points
*

I think it would be possible. But adding previously unseen stuff would be changing/redirecting the movie/show.

Each scene is set up and framed deliberately by the director, should AI just change that? It’s a similar problem like with pan-and-scan, where content was removed to fit 4:3.

You wouldn’t want to add content to the left and right of the Mona Lisa, would you? And if so what? Continuing the landscape, which adds just more uninteresting parts? Now she is in a vast space, and you already changed the tone of the painting. Or would you add other people? This removes the focus from her, which is even worse. Well this is just a one frame example, there are even more problems with moving pictures.

It would be an interesting experiment, but imo it wouldn’t improve the quality of the medium, in contrary.

permalink
report
reply
12 points
*

I hate to brake it to you…

permalink
report
parent
reply
17 points

I hate to brake it to you…

You just proved his point lol

permalink
report
parent
reply
10 points

Yeah that sucks.

permalink
report
parent
reply
-4 points

I think both look great, better than the original because of the added content.

You still get the same detail of the original, nothing about it is changed, but with a more wide view.

permalink
report
parent
reply
10 points

But adding previously unseen stuff would be changing/redirecting the movie/show.

You could see this with The Wire 16:9 remake. They rescanned the original negatives that were shot in 16:9 but framed and cropped to 4:3. As a result the framing felt a bit off and the whole thing felt a bit awkward / amateurish.

permalink
report
parent
reply
3 points

There’s a what??!

permalink
report
parent
reply
0 points
*

Sometimes thats true, but not all things in a shot are very important. There may be buildings or plants or people whose placement in the shot is not important. They only exist in the shot to communicate that the film is happening in a real living world. 99% of directors don’t care about where a tree in the background is, unless the tree is the subject of the shot.

Ai improving a shot would be debatable, but it is definitely possible. 4:3 media on a 16:9 display is pretty annoying to most people seeing the black bars on the sides. Even if the AI only adds backgrounds or landscapes, simply removing the black bars would be an improvement enough for most viewers.

permalink
report
parent
reply
4 points

If the AI is only drawing in unimportant objects, I wonder what the value is?

At the risk of ruining the original framing, the potential gain is stuff you aren’t supposed to focus on?

Who is out there watching classic TV shows who isn’t adapted to the old framing?

permalink
report
parent
reply
-1 points

I think you’re looking at it from the wrong direction. Instead of adding new stuff in to get the width, you could get AI to stretch the image to fit 16:9 and then redraw everything there to no longer look stretched out. Slim the people and words back down. Things like bottles on a table would be slimmed down to look like normal bottles but have the horizontal table be drawn a bit longer to fill in the space etc.

If it were done this way there would be a minimal amount of things that the AI would have to artificially create that weren’t there in the original 4:3. It would just mostly be fixing things looking wider than they should look.

permalink
report
parent
reply
5 points

Stretching while preserving proportions is still stretching. You change the spacing and relative sizing between objects.

Framing is not only about the border of the frame.

permalink
report
parent
reply
-3 points

I mentioned how that would be taken care of with the bottles on tables description I made earlier. Also, the framing of shots would be changed very little.

permalink
report
parent
reply
2 points

What about person A putting an arm over person B’s shoulder? That’d have to be a pretty long arm.

permalink
report
parent
reply
0 points

If they were close enough in a 4:3 shot to do that, the stretching would be very minimal to go 16:9. Aside from that, ai could avoid changing spaces between physically interacting people and objects.

permalink
report
parent
reply
-1 points

The only thing that would seem wrong is that the actors stand closer than they have to. But other than that, I doubt many would notice.

permalink
report
parent
reply
33 points
*

Do you mean something like this? (warning: reddit link)

permalink
report
reply
13 points

Holy cow that is beyond impressive. Sure enough, sometimes it does hallucinate a bit, but it’s already quite wild. Can’t help but wonder where we’ll be in the next 5-10 years.

permalink
report
parent
reply
12 points

Eh, doing this on cherrypicked stationary scenes and then cherrypicking the results isn’t that impressive. I’ll be REALLY impressed when AI can extrapolate someone walking into frame.

permalink
report
parent
reply
6 points
*

The video seems a bit misleading in this context. It looks fine for what it is, but I don’t think they have accomplished what OP is describing. They’ve cherrypicked some still shots, used AI to add to the top and bottom of individual frames, and then gave the shot a slight zoom to create the illusion of motion.

I don’t think the person who made the content was trying to be disingenuous, just pointing out that we’re still a long ways from convincingly filling in missing data like this for videos where the AI has to understand things like camera moves and object permanence. Still cool, though.

permalink
report
parent
reply
4 points

Great points. I agree.

A proper working implementation for the general case is still far ahead and it would be much complex than this experiment. Not only it will need the usual frame-to-frame temporal coherence, but it will probably need to take into account info from potentially any frame in the whole video in order to be consistent with different camera angles of the same place.

permalink
report
parent
reply
1 point

It is the first iteration of this technology, things will only improve the more we use it.

That it can do still images is already infinitely more impressive than not being able to do it at all.

permalink
report
parent
reply

just fyi, your link is broken for me

i wonder if it’s a new url scheme, as i’ve never seen duplicates in a reddit url before, and if i switch it out for comments it works fine

permalink
report
parent
reply
4 points

Thanks! Fixed

i wonder if it’s a new url scheme, as i’ve never seen duplicates in a reddit url before

I think you’re right. It should work with the old frontend (which I have configured as the default when I’m logged in):

https://old.reddit.com/r/StableDiffusion/duplicates/14xojmf/using_ai_to_fill_the_scenes_vertically/

permalink
report
parent
reply

that’s weird. it’s actually a pretty useful feature, but it’s odd they’d add it to old reddit before new reddit, considering it’s basically deprecated. maybe it’s just an a/b rollout and i don’t have it yet

i have old.reddit as default as well, but i’m not logged in on my phone browser and it wouldn’t open in my app

permalink
report
parent
reply
27 points

You should be able to but remember that aspect ratios and framing are done intentionally so what is generated won’t be at all true to what should be in scene once the frame is there. You’d be watching derivative media. Upscaling should be perfectly doable but eventually details will be generated that will not have originally existed in scenes as well.

Probably would be fun eventually to try the conversion and see what differences you get.

permalink
report
reply
40 points

4:3 - Jumpscare, gremlin jumps in from off-camera.

16:9 AI upsized - Gremlin hangs out awkwardly to the left of the characters for half a minute, then jumps in.

permalink
report
parent
reply
10 points

I would watch the hell out of that movie.

permalink
report
parent
reply
1 point

Well for sure there’s some value in it, but let’s not pretend it wouldn’t completely change the intention and impact.

permalink
report
parent
reply
3 points

I was just thinking that. Or something like a comedy bit where the camera pans to a character who had just been out of frame.

Overall it seems like impressive technology to be able to reform old media, but I’d rather put it to use in tastefully sharpening image quality rather than reframing images.

permalink
report
parent
reply
2 points

Haha, yes. I spent 15 minutes trying to remember the term for the pan/zoom-to-reveal comedy effect before giving up and settling on a botched jumpscare.

permalink
report
parent
reply
7 points

Exactly, and to add to it, you can’t know the director’s vision or opinion on how the framing should be adjusted. AI can make images easily but it won’t understand subtext and context that was intended. No time soon at least.

permalink
report
parent
reply
4 points

This.

Surprise! If you want to go from 4:3 to wide screen, you still have exactly the same problem as when using pan&scan for going from wide to 4:3.

permalink
report
parent
reply
4 points

Very true, I remember a few years ago someone converting old cartoons to a consistent 60 frames a second.

If they’d asked an animator they’d have found out that animation purposely uses different rates of change to give a different feel to scenes. So the improvement actually ruined what they were trying to improve.

permalink
report
parent
reply
3 points

There’s a video from the YouTuber Noodle who does a good job explaining it

permalink
report
parent
reply
3 points
*

Yes, sometimes frame rates are intentional choices for artistic reasons and sometimes they are economic choices that animators work around.

Old Looney Tunes used a lot of smear frames in order to speed up production. They were 24 frames per second broadcast on doubles, which meant 12 drawn frames per second with each frame being shown twice. The smear frames gave the impression of faster movement. Enhancing the frame rate on those would almost certainly make them look weird.

If you want to see an artistic choice, the first Spiderverse movie is an easy example. It’s on doubles (or approximates being on doubles in CG) for most scenes to gives them a sort of almost stop motion look, and then goes into singles for action scenes to make them feel smoother.

permalink
report
parent
reply
2 points

Definitely. I remember hearing about The Wire being released in a 16:9 format, even though it was shot and broadcast in 4:3, and how that potentially messes up some of the shot framing.

They did I by cropping from top and bottom rather than AI infilling, but the issue is the same.

IIRC, David Simon wrote a really interesting piece about how they did it but did everything they could to try and stay true to Robert Colesberry’s carefully planned framing, as they were aware that had it been intended for 16:9 he’d have framed things differently.

Personally I wish they had kept it at 4:3 and only released it in a higher resolution. Glad I still have my old 4:3 DVDs.

permalink
report
parent
reply
22 points

Who is the person that enjoys old shows, but also can’t get past the old aspect ratio?

If the AI is just adding complimentary, unobtrusive parts to the shot, so as not to disrupt the original intent, I have to ask- is there really value being added? Why do this at all?

George Lucas thought CGI could make the original Star Wars movies better.

permalink
report
reply
2 points

A similar thought I’ve had is AI removal of laugh tracks (maybe introduce background based on non-laugh track scenes)

This would make old Scooby doos actually watchable for me, so I can judge modifying the original a little if it’d what someone prefers - you can always just not watch it

Thinking about it a bit more I 100% would be the type to use a 16:9 ratio, just cause I hate black bars

permalink
report
parent
reply
5 points
*

I’m not 100% against tweaking old media (so long as it’s an alternative to the original rather than a replacement), I just think the effort and outcome of widening shots is misguided. More for live action than cartoons. Like if an actor is entering a scene from a side are we going to trust the AI to perfectly add them and merge them without it looking obvious and weird?

For removing laugh tracks, the Scooby cartoons might be a good case for this. They seem paced properly without them. A lot of sitcoms pause for laugher so removing it makes the pacing weird. I’m sure the Big Bang Theory video with laugher removed proves how nightmarish scenes can become (you know in addition to being Big Bang Theory).

permalink
report
parent
reply
-7 points

I generally avoid old aspect ratio shows/movies, especially for animated stuff.

They just look so extremely dated.

permalink
report
parent
reply
7 points

That’s a mental thing. You’re missing out. Same with people who can’t enjoy the enormous amount of fantastic black and white movies out there.

permalink
report
parent
reply
21 points

Ahhh… the much-fabled “uncrop” operation!

permalink
report
reply
4 points
*

Almost as good as Enhance

permalink
report
parent
reply

No Stupid Questions

!nostupidquestions@lemmy.world

Create post

No such thing. Ask away!

!nostupidquestions is a community dedicated to being helpful and answering each others’ questions on various topics.

The rules for posting and commenting, besides the rules defined here for lemmy.world, are as follows:

Rules (interactive)


Rule 1- All posts must be legitimate questions. All post titles must include a question.

All posts must be legitimate questions, and all post titles must include a question. Questions that are joke or trolling questions, memes, song lyrics as title, etc. are not allowed here. See Rule 6 for all exceptions.



Rule 2- Your question subject cannot be illegal or NSFW material.

Your question subject cannot be illegal or NSFW material. You will be warned first, banned second.



Rule 3- Do not seek mental, medical and professional help here.

Do not seek mental, medical and professional help here. Breaking this rule will not get you or your post removed, but it will put you at risk, and possibly in danger.



Rule 4- No self promotion or upvote-farming of any kind.

That’s it.



Rule 5- No baiting or sealioning or promoting an agenda.

Questions which, instead of being of an innocuous nature, are specifically intended (based on reports and in the opinion of our crack moderation team) to bait users into ideological wars on charged political topics will be removed and the authors warned - or banned - depending on severity.



Rule 6- Regarding META posts and joke questions.

Provided it is about the community itself, you may post non-question posts using the [META] tag on your post title.

On fridays, you are allowed to post meme and troll questions, on the condition that it’s in text format only, and conforms with our other rules. These posts MUST include the [NSQ Friday] tag in their title.

If you post a serious question on friday and are looking only for legitimate answers, then please include the [Serious] tag on your post. Irrelevant replies will then be removed by moderators.



Rule 7- You can't intentionally annoy, mock, or harass other members.

If you intentionally annoy, mock, harass, or discriminate against any individual member, you will be removed.

Likewise, if you are a member, sympathiser or a resemblant of a movement that is known to largely hate, mock, discriminate against, and/or want to take lives of a group of people, and you were provably vocal about your hate, then you will be banned on sight.



Rule 8- All comments should try to stay relevant to their parent content.

Rule 9- Reposts from other platforms are not allowed.

Let everyone have their own content.



Rule 10- Majority of bots aren't allowed to participate here.

Credits

Our breathtaking icon was bestowed upon us by @Cevilia!

The greatest banner of all time: by @TheOneWithTheHair!

Community stats

  • 9.6K

    Monthly active users

  • 3.3K

    Posts

  • 129K

    Comments