Isnt this copyright infringement? Does reddits policy allow them to do this?
#stockholderedging
$60mm a year seems really cheap, no? I know its shit data from the bot posters but still would think it would be like $100-150mm
It’s ludicrously cheap for the size and quality of the dataset. A set of 829 academic papers at University of Michigan is priced at $25,000—about 1/2400 of this sale. If you were to scale that dollar value to the size of the Reddit dataset, you’d expect it to contain about 2 million academic papers’ worth of data.
But Reddit has almost two decades of text written by 200 million chronically-online people. And sure, probably most Reddit users don’t write an academic paper amount of content every year; but the average is probably closer to that than not, especially when you consider that some of those subreddits like AskHistorians and AskScientists really are generating the equivalent of dozens of academic papers per day. Just based on the amount of text alone, Reddit should’ve sold us out for 50-100x what they got for just a single year of data, and 1000-2000x for the full twenty years (though, granted, they didn’t have that much data for that entire time, so let’s say half that).
Furthermore, those 829 papers in the U of M dataset are disconnected, unlinked text representing a tiny fraction of what U of M’s 50,000 students generate in even a single year. Reddit has data with links, images, conversational responses, prompt responses, Q&As, flash fiction, slash fiction, historical deep-dives, investigations, memes, inside jokes, a development of style and consensus over time, and a comprehensive understanding of what it means to interact online, generated by people around the world over the course of 18 years. It’s much better data for almost any LLM purpose that isn’t just writing academic papers from the perspective of students at a medium size 4-year undergrad institution in the Midwestern US. The quality of the dataset should’ve made the value even higher. It’s hard to say exactly how much higher, but let’s just be extremely conservative and say it should have doubled the total.
That means that, conservatively, the value of Reddit’s dataset—or, rather, our dataset, which Reddit freebooted from us—was about 1000x what they were paid, based on the proportional value of the U of M dataset.
They should’ve sold us out for billions.
Of course, we don’t know anything about what exclusivity deals or subset of data that they might have included with this deal. It might only be one year of data, and only 6 months of exclusivity. But assuming they sold the rights to the entire dataset, we got sold for pennies.
Honestly it’s probably the best search dataset in existence right now. You can make Google suck far less by appending “reddit” to most searches because you’ll get results from a group consisting of a higher ratio of actual humans instead of bots.
Yeah reddit is shit, but the rest of the internet is 10x worse at this point. Pretty much any writing that isn’t a labor of love on someone’s personal page or users interacting with each other in a semi organic way is rapidly becoming 100% GPT vomit as every company in existence lays off their writing staff
Whoever bought this got a fucking bargain.
Very glad I overwrote all of my comments with random words before deleting my account. They won’t be profiting off of me anymore.
Instead you’re posting to the Fediverse, which is even more open for use by third parties.
Its obscure enough that I don’t think it’s being sought out by AI companies. The nature of federated instances should make it a bit more challenging to pull a complete data set too
Yea, but it’ll be open forever, nobody can turn off an app overnight and profit from it.
Right. But my point is that they can profit from it. The issue lots of folks seem to be having is “how dare Reddit make money using something I did!”, and that issue is even worse for the Fediverse since lots of companies can be doing it.
I spent a chunk of this afternoon nuking my old reddit posts. Thousands and thousands of posts… thank goodness for shreddit.
I didn’t know about this. I just went into it.
edit: lol you have to pay for Shreddit. Nevermind, I don’t care about deleting my posts that much.
Most tools miss a ton bc of the limitations of the website and api. The best, pretty much only, way to get everything is to get an export of your data, then use that csv to delete all items one by one
