So if OpenAI complies with Robots.txt files then there’s no issue right?
Because then they’re identical. Open AI spent a bunch of money building a powerful system they feed those results to, as did Google.
Actually Google tries their hardest NOT to point you to content. They scrape the data from sites and display it directly in the search results so that you don’t need to visit any site except Google. Their new AI answers that they are pushing on users are just another step in that direction.
Literally every page Google shows you, where it also shows you those ads it makes money from, is Google’s content and it is derived from the data it gets scraping the web.
What the fuck are you even talking about? Making a list of website identifiers (names and URLs) so that people can go to them isn’t even slightly the same as making a derived work of the websites’ contents.