cross-posted from: https://lemmy.world/post/76533

One of the arguments made for Reddit’s API changes is that they are now the go to place for LLM training data (e.g. for ChatGPT).

https://www.reddit.com/r/reddit/comments/145bram/addressing_the_community_about_changes_to_our_api/jnk9izp/?context=3

I haven’t seen a whole lot of discussion around this and would like to hear people’s opinions. Are you concerned about your posts being used for LLM training? Do you not care? Do you prefer that your comments are available to train open source LLMs?

(I will post my personal opinion in a comment so it can be up/down voted separately)

11 points

I think the claim is nonsense. If that were their concern they would rather change the usage agreement and maybe take some of them to court.

What they actually did is everything in their power to drive mobile users to their mobile app. They want old fashioned user tracking data for advertising and selling on. Together with more in app ads.

permalink
report
reply
8 points

I totally agree that Reddit’s motivation is probably not related to LLMs and the link I posted is more of an excuse than anything. However, I am curious what people think about data scraping and LLMs in general.

permalink
report
parent
reply
11 points

Reddit provides a platform where regular users create the data. Moderators add value by ensuring the quality. Without any of these parties, there is no valuable data. Of course there is a cost in running the platform, but Reddit should avoid as much as possible charging users and especially moderators for using the platform.

Then there are search engines and 3rd party apps. They also add value. Search engines use the data, and in return they attract new contributors. 3rd party apps also attract regular users, and by providing a better experience make sure that the regular users stay active for longer. They should not be charged more than is required to keep the platform running and is reasonable with respect to their profits.

LLM trainers do not fit in this picture. They use large amounts of data, but do not provide anything in return that is valuable to the users, moderators or platform. Therefore, I absolutely support charging them more for accessing training data.

Users of the platform who provide value in return should not have to pay more than is reasonable and required than to keep the platform running. LLM trainers do not provide value in return, and I support charging them more. It is unreasonable to not differentiate between 3rd party app developers and LLM trainers.

permalink
report
reply
8 points

Reddit has every right to charge for their API, but the amount they wanted to charge was too high.

Other use cases aren’t relevant here either. They could have come to an agreement with Apollo etc that would have charged them reasonable rates while charging more to data scrapers. They could have done ads and dev share on the mobile apps. Most people wouldn’t have objected to that.

That part’s not a Reddit-specific problem though. I’ve seen a similar pattern play out at several companies I work for:

  • charge extra for a new premium feature
  • a new client with deep pockets comes along and wants part of that feature, but doesn’t want all of it, so doesn’t want to pay for it
  • sales really wants to catch this big fish
  • sales promises to build a new feature that does the same thing as the existing feature
  • the company loses more money than they would have by just giving the feature away for free, since now they’re also paying engineers to build the free version.
permalink
report
reply
9 points

I think another huge problem that you didn’t mention was the timeframe. Had they given the apps even 6 months from announcing the price they may have been able to pivot to subscriptions. The short timeframe (combined with the gaslighting from the CEO) makes it hard to want to try though.

permalink
report
parent
reply
8 points
*

I think Reddit does have a legitimate argument that the scales have tipped and Reddit eating the costs of “whales” abusing their APIs for for-profit use cases without Reddit being compensated at all is fair.

3P apps using the API at no cost while simultaneously monetizing Reddit’s content by showing their own ads does seem to be taking advantage.

That said, the way Reddit approached this was so scorched earth and bone headed.

For example. Reddit gets 10s of millions of dollars in free content moderation services from volunteers. The moderators of all their biggest subreddits rely on 3P moderation tools since Reddit’s are so poor.

So with the new API policy, they’re asking their unpaid moderators to PAY them for the privilege. It’s such a slap in the face.

Finally to address the original question, Reddit should absolutely block API consumers who are just training their glorified chat bots to regurgitate plagerized content.

permalink
report
reply
7 points

I hope cross posts are OK. But I am curious about Experienced Dev’s perspective on this as well since the question is rather technical.

Copying my opinion from the other thread in case you don’t want to look at my other thread:

My personal opinion is that high API usage fees hurt open source LLMs (e.g. GPT4All). I would rather not see this new technology monopolized by those who can pay API fees.

permalink
report
reply
6 points

I’d tend to agree. There are enough barriers to training large models without artificially increase them just because the largest players can afford it.

permalink
report
parent
reply

Experienced Devs

!experienced_devs@programming.dev

Create post

A community for discussion amongst professional software developers.

Posts should be relevant to those well into their careers.

For those looking to break into the industry, are hustling for their first job, or have just started their career and are looking for advice, check out:

Community stats

  • 20

    Monthly active users

  • 76

    Posts

  • 1.1K

    Comments