When Large Language Models (LLMs) like ChatGPT were first released, I experimented with bard and then gemini for a while; I noted some limitations but was impressed by its capabilities. But that use led me to refer to the Apostle’s creed as the Nicene creed for a significant amount of time. It was something which caused me embarrassment upon discovering. LLMs are especially bad because they have subtle lies. Outright lies are easy to detect. Subtle lies just slip under. After that incident, I stopped using LLMs.
Unfortunately LLMs migrated into my regular search, giving me really impressive, confident, irresistible and ‘useful’ summaries. It came first to Google Search and Bing. Then I migrated to Brave Search only occasionally checking Google for references. After a while, Brave Search also introduced Leo, spewing out beautiful summaries of what potentially does not exist, full with references. I felt angry and helpless. I tried to ignore the summaries but that did not seem to work. Then I tried turning it off. But I don’t know what happened. The LLM summaries just kept coming my way. It’s just too tempting, like a beautiful woman begging to flirt with you.
In the end, I started getting complacent, peeping into the AI summaries every so far often. One day, I searched for “when did the pillar of cloud leading the Jew disappear”. Behold there was a beautiful summary, almost relieving. But then, I decided to double check the results and I became angry. The answer to this question can be inferred from the fact that the pillar of cloud and fire was the primary campus for the Jews in the desert. When they entered the promised land, they would obviously not need the pillar of cloud. And what I needed was to find a Bible verse that made this reference to the end of this pillar of cloud and dust. And I got a beautiful summary (see image), complete with Bible quotations and references.
To summarise for you the result of my double checking; the basic premise of the summary was plausible (probably correct). But the Bible quotations did not support the summary. The referenced documents spoke about the subject matter but did not speak about the particular aspect. Some of the Bible verses could perhaps be used to justify the conclusion. But then, the logic would have to be inductive and not include such “phrases” like “specifically”. I have not found a Bible verse which specifically states when the pillar of cloud disappeared. But the AI was very confident and very specific about when it did. I was almost tempted to use the Bible quotations as a justification for that conclusion. Almost!
The reality is that we are all vulnerable to information that is nicely presented and projected with confidence. We perceive such information as coming from an expert and a fact that makes us even more gullible. At the same time, when we do a web search. We do so because we do not know. Worst of all when we meet a presentation that is referencing other sources, we are primed to believe what has been stated because we have been trained to know that when a person supports an assertion with a reference. That assertion must be right because they have taken their time to do their research. The only people who check references are skeptical reviewers intent on destroying an argument they do not believe. Most people rarely check references. Only people with an in depth knowledge in a specific field will make the attempt to check a reference, and that will only be done a small fraction of the time when they find that the assertions made seem wrong.
We search the web because we do not know. Most of the things we search for are not in our field of expertise. We are therefore rarely well placed to criticise an answer especially one that is well written and presented with confidence. For this reason, over reliance on AI will inevitably lead to the incorporation of half truth and lies into our knowledge base and dataset having believed a summary which is quoted.
I have gone shopping for alternatives to the traditional search engines (google, bing, and brave). It turns out there are many search engines out there, some of which use other search engines to process queries. It turns out there are over 24 search engines one can use. For the time being, I have settled on qwant.com and swisscow.com. If these also start providing automatic AI summaries, I will look for others. If the regular search engines can remove the experimental disclaimer on their summaries, I will come back to using them.
You can find out more about the different search engines. I know about AI, I want a plain search engine, which will not throw information that looks plausible but is unsupported by references. Even if AI summaries are not a result of hallucinations but are the genuine extraction of information from sources which my intuition suggests is not. The AI summary gives credence to those sources and may realize a conclusion that is not entirely true.
When the major search engines have the confidence to remove the experimental label and warning to search the results. I will go back to use them.
Appendix
screen shot of the search
Notes
Joshua: 1:13 “Remember the command that Moses the servant of the Lord gave you after he said, ‘The Lord your God will give you rest by giving you this land.’ “
Does not tell us that the pillar of cloud disappeared or that they entered the promised land.
Joshua 5: 10-12 “On the evening of the fourteenth day of the month, while camped at Gilgal on the plains of Jericho, the Israelites celebrated the Passover. 11 The day after the Passover, that very day, they ate some of the produce of the land: unleavened bread and roasted grain. 12 The manna stopped the day after[a] they ate this food from the land; there was no longer any manna for the Israelites, but that year they ate the produce of Canaan.”
We can use this to ascertain when the jews stopped receiving manna from heaven. It does not tell us when the pillar of the cloud disappeared.
You can continue the breakdown if you wish. But take the first reference Wikipedia is a generic reference to the pillar of cloud. It does not talk about the disappearance of the cloud using either deductive or inductive logic.
I do not have the appetite to continue an in depth analysis of this nonsense. The summary is not anything close to summary.
Alternative search engines.