FaceDeer

I’ve been using ChatGPT since it came out and yet my brain isn’t nearly fried enough to fall for clickbait headlines this obvious.

I’m being genuine, perhaps explain your point differently. The thing you said was:

Because those things are not worth the damage it is doing.

The things you are saying are “not worth doing” would seem to be the birthday card and school assignment thing, that’s what the comment you’re responding to mentioned.

The “damage it is doing” is the part I responded to by pointing out that it’s not as bad as the clickbait articles that circulate about it say it is.

Which bit has been missed?

Writing birthday cards and cheating on school assignments? It can already do that quite well, that’s a goalpost that’s been passed already.

The term “AI” was first coined in 1956 at the Dartmouth workshop and covers a broad range of topics in computer science. Machine learning and language models most certainly do fall under that category.

Refusing to call LLMs “AI” looks to me like an instance of the AI effect in action, in which anything computers can do is no longer regarded as an example of “real” intelligence. It’s a goalpost shift.

Used to be that the Turing Test was a big deal. Or being able to beat a human chess grandmaster. Or a Go grandmaster, once chess was reliably being won by computers. Just the other day ChatGPT was able to generate a novel proof for an unsolved Erdos problem that mathematicians are now using as a basis for new discoveries. What’s the next goalpost?

In your personal opinion, and based on the articles you’ve seen describing the damage in dramatic terms that bait the clicks of people who are already predisposed to think negatively of AI.

And so the echo chamber resonates on.

It’s a common phenomenon. Look at how stunned so many people were when Trump won, for example. How could he have won? Everyone they knew voted against him! Everyone they talked to agreed that he was terrible! Except of course they were only talking to like-minded people, so they had no idea what “everyone” actually thought. Same thing applies to Trump supporters when he lost in 2020, they couldn’t believe it.

Places like the Fediverse are practically designed to become echo chambers. Look at the upvote/downvote mechanism, how’s the balance look in here for comments critical of AI versus comments that aren’t critical of it? I know karma is meaningless, even moreso here than on places like Reddit, but it’s still a psychological and social pressure declaring “you’re not one of us, your opinion is bad and you should feel bad.” Naturally the people who would post such unpopular opinions will tend to stop posting over time.

Heck, I’ve got my own personal stalker following me around lately posting about how much of a “troll” I am because I don’t toe the “AI bad” line. Disagreement with the local consensus is bad and wrong and I’m supposed to just shut up and go away I suppose. Doesn’t bother me, but people are social beings so I’m not surprised it bothers others.

“unlike many of the other PDFs” suggests it’s not unlike all of the other PDFs. I’m curious what other city documents are like it.

Negative examples are also useful when training AIs.

We must protect the children by <checks notes> removing protections from children!

On the one hand, up here in Canada this is going to likely have annoying ripple effects. We have a separate system (the ISED) with separate but similar certification criteria, and due to the similarity items are usually certified for both simultaneously. But I bet a lot of companies won’t bother doing just ISED certification on its own.

On the plus side, though; hey, international corporations, come set up shop in Canada instead. We don’t cut off Chinese electronics due to the random voices in a demented toddler-king’s brain.

Your obsession with me is quite silly. I recommend you just block me already, then you won’t have to see my “trolling.”

It’s always been that way, it’s just that until now the general public could say “well at least they pay me.”

So ironically this rise in anxiety is itself being driven by self-interest. People were fine with those people being in charge as long as they got a comfortable lifestyle out of it. A pattern seen throughout history.

Jeeves is public domain now.

I expect the trademark will be for sale soon.

Yes, English is a language with words. Why not use the properly?

Muphry’s Law strikes again.

This is the “uplifting news” community, so jumping on a slight lack of specificity in a headline looking for something to be angry about is kind of odd.

How about swapping in “Chinese researchers” for “China.” Happy now?

Alright, so instead of simply saying “include external data in your training run”, extend that to “and also filter the data to exclude erroneous stuff.” That’s a routine part of curating training data in real-world AI training as well, I was already writing a lot so I didn’t feel like adding more detail there would have enhanced it.

The basic point remains the same, that real world training accounts for the things that were necessary to force model collapse to happen in that old paper I linked. It’s a solved problem. We can see that it’s solved by the fact that AI models continue to get better, despite an increasing amount of AI-generated data being present in the world that training data is being drawn from. Indeed, most models these days use synthetic training data that is intentionally AI-generated.

A lot of people really want to believe that AI is going to just “go away” somehow, and this notion of model collapse is a convenient way to support that belief. So it’s very persistent and makes for great clickbait. But it’s just not so. If nothing else, the exact same training data that was used to create those earlier models is still around. AI models are never going to get worse than they are now because if they did get worse we’d just throw them out and go back to the earlier ones that worked better, perhaps re-training with the same data but better training techniques or model architectures.

Model collapse comes from using only training data generated by previous generations.

All that’s needed to avoid it is to add training data that isn’t directly from the previous “generation” of the LLM in question. The thing that causes model collapse is the loss of data from generation to generation, so you just need to keep the training data “fresh” with stuff that wasn’t directly generated by the earlier generation of your model.

You could do that with archived material you used for previous training runs. For more recent events you could do that with social media feeds. The Fediverse, for example, would probably be a perfectly fine source of new stuff. Sure, there’s some AI-generated stuff mixed in, but that’s not “poison.”

As I mentioned, the article that demonstrated model collapse did it using a very artificial set of circumstances. It’s not how real AI training is done.

The main mechanism leading to model collapse in that paper, as I understand it, are the loss of “rare” elements in the training data as each generation of model omits things that just don’t happen to be asked of it. Like, if the original training data has just one single line somewhere that says “birds are nice”, but the first generation of model never happens to be asked what it thinks of birds then this bit of information won’t be present in the second generation. Over time the training data becomes homogenized. It probably also picks up an increasing load of false or idiosyncratic bits of information that were hallucinated and get reinforced due to random happenstance, it’s been a long time since I read the article and details slip my mind.

I’m really not seeing how human filtering would mimic this process, so I think it’s safe. The filtering is being done with intent in that case, not due to random drift as is done with a purely automated generation like was done in the paper.

Semantic quibbling is one of the least interesting kinds of internet debate, so replace the word “understanding” with whatever word makes you happy. I continued with “and talking about” right afterwards so you can just delete the word entirely and the sentence still works fine. You could have just kept reading.

Since you didn’t read the rest of my comment, I should note that the rest of it after that sentence is about the other issue that OP raised and not even about model collapse at all.

Anyway. The article about model collapse that I see still crop up every once in a while is this one. It’s not that it has “methodological errors”, though, it’s just that it uses a very artificial training protocol to illustrate model collapse that doesn’t align with how LLMs are actually trained in real life. It’s like demonstrating the effects of inbreeding in animals by crossing brothers and sisters for twenty generations straight - you’ll almost certainly see some strong evidence, but it’s not a pattern of breeding that you are actually going to see in the wild.