The fact that Generative Adversarial Networks exists means it isn’t that hard.

By hard I mean hard like hard math problem, not hard like mopping the floor after a long shift is hard. When I say “not too hard” I mean “possible”.

And you’re right. It is possible.

What I described isn’t a GAN per se though. It’s based on a similar idea, but it’s not the same thing.