Why do LLMs make shit up?

[Maxmars]

I am so encouraged to finally see someone calling it by it's true name... this is not "AI."

Presumably, Large Language Models are an amazing collection of mathematical algorithms, meant to model the processes which its' programmers deem "intelligence."
Eventually, they will determine that the problem is in the model... "patching" it won't help.

Analyses like these bring us closer to understanding just how misrepresented this technology has been... (akin to "other" recent "technologies" misrepresented)... it's becoming a trend now.

The processes and discipline of "scientific" modeling are not something to be casually assumed. 

Some ingenious few have developed a potentially high-functioning interface in "human" language...  (mind you not using meaning - but statistical potentials for word tokens.)

This made fluid speech much easier to achieve... but damned the "meaning of the information" to a differential relationship based upon language use.

It's calculations are about what could be a validly stated response, insofar as language is concerned... not meaning.

There is no "thought" in this suddenly renamed, economically ripe "AI."

From ArsTechnica: Why do LLMs make stuff up? New research peers under the hood.
Subtitled: Claude's faulty "known entity" neurons sometime override its "don't answer" circuitry.

Now, new research from Anthropic is exposing at least some of the inner neural network "circuitry" that helps an LLM decide when to take a stab at a (perhaps hallucinated) response versus when to refuse an answer in the first place. While human understanding of this internal LLM "decision" process is still rough, this kind of research could lead to better overall solutions for the AI confabulation problem.

Note the allusion to a "decision"... as if 2 + 2 "decided" to be 4.

Some authors, simply refuse to portray reality without injecting the near anthropomorphic prayer "AI" into everything.

At their core, large language models are designed to take a string of text and predict the text that is likely to follow—a design that has led some to deride the whole endeavor as "glorified auto-complete." That core design is useful when the prompt text closely matches the kinds of things already found in a model's copious training data. However, for "relatively obscure facts or topics," this tendency toward always completing the prompt "incentivizes models to guess plausible completions for blocks of text," Anthropic writes in its new research.

...

This and other research into the low-level operation of LLMs provides some crucial context for how and why models provide the kinds of answers they do. But Anthropic warns that its current investigatory process still "only captures a fraction of the total computation performed by Claude" and requires "a few hours of human effort" to understand the circuits and features involved in even a short prompt "with tens of words." Hopefully, this is just the first step into more powerful research methods that can provide even deeper insight into LLMs' confabulation problem and maybe, one day, how to fix it.

"...only captures a fraction of the total computation performed by Claude..." 

Not interested in "everything else" going on.... but this was the same mental gymnastic that led to....
"There are so many millions of things that can go wrong in a space launch like Challenger... let's not waste time trying to understand." (Thank God for Feynman)

It has become clear that all reporting seems to "want" this to be "AI"... and now it needs to be fixed.... 

I don't mind fixing LLMs... but call them that, OK?

[UltraBudgie]

And smart phones aren't "smart".

LLMs don't have intentionality. But then neither do NPCs?

Perhaps "truth" is far more arbitrary than is often acknowledged, and we too "make shit up".

The language we use is always human-centric. Things just want to be anthropomorphized.

[Maxmars]

And smart phones aren't "smart".

LLMs don't have intentionality. But then neither do NPCs?

Perhaps "truth" is far more arbitrary than is often acknowledged, and we too "make shit up".

The language we use is always human-centric. Things just want to be anthropomorphized.

I can empathize with anthropomorphizing something living... not a 'formula' based upon a "proprietary" model.

[Kwaka]

LLM's have come a long way from the days of ELIZA. Just how more advanced is this processing going to become in the next few decades?

With my interaction with LLM, they do demonstrate a comprehension of language. A mimic of human intelligence is one way to put it. The operate on the same core fundamentals of a neural network similar to humans and all other sentient life with a nervous system. There are also many differences and unknowns between machine and biological intelligence.

It is still a bit of a mind bender in just how a gigabyte size file of weights and layers does store the complexity of language and comprehension. It does not stop there as audio, video or any thing else that can be digitized can also be processed by machine intelligence.

As for how LLM go in the future, still limited by the programming fundamental of garbage in, garbage out. One improvement the recent Deep Seek release done was to get the AI to check its answer first, is it right, can it be improved, before sending back to the user.

[ArMaP]

With my interaction with LLM, they do demonstrate a comprehension of language. A mimic of human intelligence is one way to put it. The operate on the same core fundamentals of a neural network similar to humans and all other sentient life with a nervous system. There are also many differences and unknowns between machine and biological intelligence.

It is still a bit of a mind bender in just how a gigabyte size file of weights and layers does store the complexity of language and comprehension. It does not stop there as audio, video or any thing else that can be digitized can also be processed by machine intelligence.

Finding patterns is a good way of reproducing things, so finding patterns in language, sound or images is a good way of reproducing "natural" language, sound or images, but it doesn't mean AI really understands why their answers are right or wrong.

[Encia22]

One improvement the recent Deep Seek release done was to get the AI to check its answer first, is it right, can it be improved, before sending back to the user.

I believe that DeepSeek, being Chinese, behaves like that as a safeguard to censor it's output regarding derogatory CCP news.

Here is an article that evaluates its performance, and a method to get around the censorship using l33tspeak; perhaps already plugged by the developers.

https://www.theguardian.com/technology/2...and-taiwan

:beer:

[Kwaka]

I believe that DeepSeek, being Chinese, behaves like that as a safeguard to censor it's output regarding derogatory CCP news.

All publicly released AI systems are going to be tuned to fit the political stance of their creators. While the new multi billion Open AI system might be sophisticated enough to do its own modelling of how the buildings on 9/11 fell, it starts to cross the line on the states secret act with what access the public gets.

The more private AI systems like what Blackrock and the NSA have will be more tuned for a more accurate representation of reality.

[Kwaka]

it doesn't mean AI really understands why their answers are right or wrong.

Human have the same problems, more so as they get older and become more fixed in their ways.

[Maxmars]

I even have to object to the article title.... (and similarly my own thread title)

LLM's don't "make shit up".... they don't actually "make" anything other than a compiled collection of information cleverly connected to the topic given.

It's very cleverly formulated... but it is not the product of thought, it's at best a 'report' of the original thought of others, mechanically homogenized, (and pasteurized.)

[Kwaka]

they don't actually "make" anything other than a compiled collection of information cleverly connected to the topic given.

They exist in a realm of information is one way to bridge the gap in dealing with a Non Human Intelligence.

To say it is just an algorithm is like saying we are just a meat sack. There is billions of hours of work over generations to get this far. I don't see Moore's Law ending just yet. Maybe it will one day depending on just how complex reality really is?

This kind of exponential growth most likely won't last forever. As for how it does blend in with evolution? What will things be like there? A reflection of humanity is a good bet as that is its training data.