Technically yes, but the birth certificate for both might be filled with the place of the landing simply.
At least for German law (and probably other ones) that’s what de facto would be required: You enter the exact town the child was born in, if known (but when moving 800 km/h over invisible town boundaries, who takes note of in which town you were at the exact moments the two were born?), or the place where the mother sets foot to ground otherwise.
In the US and Canada at least, there are laws that cover granting birthright citizenship to people born in their airspace of those countries. And since they share a border, it could happen, in theory. Would also depend on the citizenship status of the parents, I imagine…
In practice, I would hope you are right, this would not cause any issues. But if it happened, it would probably get news attention, and who knows what follows from that.
