AWS is not a black box from the outside. The signal servers will have their own external IP addresses that you will connect with, your ISP could keep track of those connections. Furthermore, if you are worried that the government is using your ISP to spy, what makes you think that AWS wouldn’t be subject to that as well? Signal is absolutely a target in this respect too.

Of course you can do various things to potentially hide your connection to signal, for instance by using tor, but in some sense there’s no guarantee if you don’t trust anything external to you. I’m personally not too worried about the “this person uses signal” metadata, though.