AI-powered virtual executive team — a single coherent executive persona backed by 8 specialist Claude agents (FastAPI + Next.js). - SenteLabsAI/OpenExecutive
If the data is organic and not synthetic, might the hallucination be avoided? Correct me if I’m wrong, but isn’t the basis of the hallucinations down to the compilation of synthetic data sets and the LLMs go off the rails because of that?
No, it’s a direct consequence of the structure of an LLM. You may have heard them called “fancy autocorrect” and that’s because in a very real way, that’s what an LLM does: based on the text in its conversation (the user’s, its own, the system prompt, search results, etc) it picks one of the most likely next words. Then starts over with that new word included.
“Most likely” in this case is based on the weights from training, but the important thing is that it’s inherently statistical in nature, there’s no enforcing of hard facts possible, and even limiting it to only the very most likely next word each time causes unnatural text and problems such as loops - while still not being 100% accurate.
Thank you for the clarification! I have a layperson’s understanding of LLMs as I just do not use any of the ai tools that are showing up in front of me.
I try to stay up on it, rather than just rejecting the technology outright.
Don’t use it, don’t plan to, but still want to understand it’s strengths and shortcomings.
So… not to go too far down the rabbit hole on this topic… my question is, how does the hallucination occur?
Is it a function of the language syntax within the data set or the contextual meanings of the words being lost as the LLM is compiling whatever answer it’s been asked to give?
Just from looking at some of the more humorous screengrabs of ai answers that get posted here… I can see that a lot of it appears to be poisoned data scraped from shit sources…
However, I have also seen a video of a guy trying to get the ai he was talking to, to tell him how many letter r’s appear in the word “strawberry” and it could not get it right… even as the guy was walking the ai through the spelling of the word and literally counting aloud each letter r as the machine spelled it out.
How does that breakdown happen in a machine model that was probably fed an Oxford dictionary of the English language before anything else?
So LLM’s are basically fancy auto correct. You type a phrase and it guesses what comes next, based on training and weights, etc.
Hallucinations happen when the guess is wrong. And then you iterate on the wrong guess. So the initial guess can be minor, it can guess “how can I drive a car” instead of “how can I drive a tractor”. Which on its own is a small deviation, but after looping on that wrong guess, the outcome is much different.
This is a high level view but I hope it makes sense.
That does make sense… the hallucinations arise from guesses… but if the query is “How can I drive a tractor?” and not a car, then that would be down to it’s not reading the inputs correctly… or the query itself wasn’t concise enough, no?
Hm.
So could the hallucinations be better prevented by framing the queries with more strict language and syntax rules?
I wonder what would happen to the economy if we had fully fact-based rational CEOs. Would it grow, shrink or stabilize?
Ai ceo’s would do great for a while. Then there’d be a single hallucination that would cause the company to fail and become insolvent.
So like human CEOs…
If the data is organic and not synthetic, might the hallucination be avoided? Correct me if I’m wrong, but isn’t the basis of the hallucinations down to the compilation of synthetic data sets and the LLMs go off the rails because of that?
No, it’s a direct consequence of the structure of an LLM. You may have heard them called “fancy autocorrect” and that’s because in a very real way, that’s what an LLM does: based on the text in its conversation (the user’s, its own, the system prompt, search results, etc) it picks one of the most likely next words. Then starts over with that new word included.
“Most likely” in this case is based on the weights from training, but the important thing is that it’s inherently statistical in nature, there’s no enforcing of hard facts possible, and even limiting it to only the very most likely next word each time causes unnatural text and problems such as loops - while still not being 100% accurate.
Thank you for the clarification! I have a layperson’s understanding of LLMs as I just do not use any of the ai tools that are showing up in front of me.
I try to stay up on it, rather than just rejecting the technology outright.
Don’t use it, don’t plan to, but still want to understand it’s strengths and shortcomings.
Yea the great weakness with LLM’s is that statistically a hallucination is inevitable. So its output always should be human verified.
So… not to go too far down the rabbit hole on this topic… my question is, how does the hallucination occur?
Is it a function of the language syntax within the data set or the contextual meanings of the words being lost as the LLM is compiling whatever answer it’s been asked to give?
Just from looking at some of the more humorous screengrabs of ai answers that get posted here… I can see that a lot of it appears to be poisoned data scraped from shit sources…
However, I have also seen a video of a guy trying to get the ai he was talking to, to tell him how many letter r’s appear in the word “strawberry” and it could not get it right… even as the guy was walking the ai through the spelling of the word and literally counting aloud each letter r as the machine spelled it out.
How does that breakdown happen in a machine model that was probably fed an Oxford dictionary of the English language before anything else?
So LLM’s are basically fancy auto correct. You type a phrase and it guesses what comes next, based on training and weights, etc.
Hallucinations happen when the guess is wrong. And then you iterate on the wrong guess. So the initial guess can be minor, it can guess “how can I drive a car” instead of “how can I drive a tractor”. Which on its own is a small deviation, but after looping on that wrong guess, the outcome is much different.
This is a high level view but I hope it makes sense.
That does make sense… the hallucinations arise from guesses… but if the query is “How can I drive a tractor?” and not a car, then that would be down to it’s not reading the inputs correctly… or the query itself wasn’t concise enough, no?
Hm.
So could the hallucinations be better prevented by framing the queries with more strict language and syntax rules?
That entirely depends on the strategy they choose to use to buy the government.
… they will definitely conclude that they need to do that, just as human CEOs have, but yeah, it depends on how they choose to pursue that goal.
Make an OpenCentralBankHead, then lemmy play with it a bit, and I can give you a better projection.