Of course young optimistic me would have considered that this was an easy thing to have a QA test for, but here we are in 2024 and I am neither young or optimistic. Maybe the AI QA folks were in the last few rounds of Google layoffs or something.
This brings to mind a game instalment (probably COD) set in the 80s or something where you choose to be enby and reagan would respect your pronouns.
Riffing on this post title: https://awful.systems/post/1053906
Turns out that trying to correct for biased models by biasing your input does not make your results more reliable.