“ChatGPT may sound confident, but when tested on complex scientific claims, it often guesses and even contradicts itself. Researchers found it struggles especially with spotting false information. Credit: Shutterstock”(ScitechDaily, ChatGPT Was Asked the Same Question 10 Times. The Answers Kept Changing)
“In the initial 2024 experiment, ChatGPT answered correctly 76.5% of the time. When the study was repeated in 2025, accuracy rose slightly to 80%. However, once the results were adjusted for random guessing, the performance looked far less reliable. The AI was only about 60% better than chance, which the researchers described as closer to a low D than strong performance.”(ScitechDaily, ChatGPT Was Asked the Same Question 10 Times. The Answers Kept Changing)
“The system had particular difficulty identifying false statements, correctly labeling them only 16.4% of the time. It also showed inconsistency. When given the exact same prompt 10 times, ChatGPT produced consistent results for only about 73% of the cases.”(ScitechDaily, ChatGPT Was Asked the Same Question 10 Times. The Answers Kept Changing)
Researchers talk about those cases like this. “We used 10 prompts with the same exact question. Everything was identical. It would answer true. Next, it says it’s false. It’s true, it’s false, false, true. There were several cases where there were five true, five false.” (ScitechDaily, ChatGPT Was Asked the Same Question 10 Times. The Answers Kept Changing)
The reason for that is the accumulation of information. When the researchers ask exactly the same questions hundreds or thousands of times. That causes data accumulation. Another thing is that. There is probably a pointer in the algorithm. That tells whether the answer satisfies the user. If the user does not stop the algorithm. While bombarding it with the same question.
The algorithm interprets. That situation. That there is something wrong in the answer. If the user drives the same algorithm. Again and again. There is a possibility. That some router. Or a switch is stuck. This can cause a situation. That the system gives different answers. When the AI gives an answer that remains. It's RAM memory. If the algorithm runs in a loop, the question repeats.
Time after time. That can also fill the memory of the central servers. So, before the next run. The user must remove the garbage or the data from the RAM. The system must have time to stop the algorithm and then clean its memory. If that is not done. The memory. It can be filled. And data. It can be polluted.
The user should finish the job first. That happens by telling the AI that it’s time to begin the new operation.
We all know that 1+1=2.But sometimes a stuck switch, or gate. Causes. That microchip fails in the simplest possible calculation. And tells researchers that 1+1=3. The failure in a processor’s internal structure. It can make it break mathematical rules.
The situation is similar. To cases. There are binary microchips that calculate 1+1=3. This happens sometimes. When the binary processors are tested by calculating 1+1 thousands or billions of times. Suddenly. A microchip. It can give an answer that can surprise us. The reason why the computer gives 1+1=3 is in Boolean algebra and switches. When the system runs 1+1 many times. That simple calculation jams the switch. And that causes the wrong answer.
The biggest problem with AI is that. The same AI should handle all types of things. There is no human on Earth. Who knows everything. The limit. Of the sector. Of AI use. It can make it more trustworthy. If the same AI is used for fun and for things like scientific work. That causes the data that it involves. Scientific data and other types of data. Like poems. The biggest problem is that. The AI will not make. A difference between the sources. The AI handles all homepages the same way. The thing that makes AI more trustworthy is that. Its data source. It can use. They are limited to trusted scientific homepages. In cases like entertainment purposes. The AI must not use data in a similar way.
Did you ask something? From the AI? Then you might notice that the answer changes all the time. The reason for that is in the AI, or LLM (Large Language Model). And especially in its structure. The AI is a cloud-based solution. The giant entirety. People who use AI might think that they are using their private software. The LLM or AI is a giant network of databases and servers. When somebody uses those systems. The system scales that interaction all over the network. So, every user in that kind of AI is participating.
In the AI-training mission. Even if other users cannot see directly what other people ask. That data grows and refines the dataset. That. The system involves. When we ask something. From the AI. It searches data. From its own registers. The AI searches. Is there the same question? And then the list of data sources that the LLM used. Questions. That people make. They might be from the same topics. But. They are a little. A bit differently written.
What is the heaviest stable element? It is not the same question. As the question: “What is the heaviest non-radioactive element”?
The AI uses statistical methods to generate answers. This means that the homepages that the AI uses must be involved. A certain number of references that match the question. The AI must know the trusted data sources it can use. If the AI is used for scientific work. Things like thesis banks, universities, and national institutions offer very qualified information. But then. We must be careful. When we use the AI. When we ask things like the heaviest known stable element. The AI might give an answer. That is wrong. It might tell.
That Oganesson is the heaviest stable element. Oganesson, element 118, is a very unstable element that decays in less than a microsecond. This means AI makes a mistake. And the reason for the mistake is in the way of thinking. The AI can “think” that the person who asks that question. Means the heaviest element that exists is confirmed. And the element is connected to the periodic table of elements. Normally, that question means the heaviest non-radioactive element. But as I just wrote. AI understands. This is the heaviest element. That has its place in the periodic table of elements. So, we should ask: what is the name of the heaviest non-radioactive element?
This means that the AI translates and treats them as different questions. The system searches data from its internal registers. But if the questions about the topics are a little bit different. The AI also searches data from the network. This means the AI accumulates information about the source lists in its memory. And when data is accumulated. The AI will use different data sources. The other thing is that. When the LLM makes a search and opens homepages. It changes the homepage's page rank. And that also causes a situation.
There, the AI’s answers are changing. The other thing is that the AI’s answers will be turned into the homepages. And that makes the AI recycle its output. This is the big problem for the net. The dead internet means that AI generates more and more material for it. That generates and accumulates. A data mass. The same data repeats again and again. This way of generating answers fills the servers. Another problem is that. For generating good answers. Those AIs require well-articulated questions. But another fact is that. A good-looking answer is probably not the best or right answer.
The problem is that. The wrong answers might look funny. People share them on social media. This causes a situation. Those incorrect answers affect the page ranking. They are visible in discussion forums. If those incorrect answers are often shared. They are seen in the statistics. And that can make the AI repeat them. The problem is that.
The AI searches data using statistical methods. Page ranking and the involvement of certain words in the list of words that are used in those pages. Make the AI select them. As the data source. Statistical methods. Don’t make a difference between right and wrong information. Or wrong information. The ability to limit. The use of data sources from trusted organizations. Makes AI more trusted.
The big problem is that. The AI causes. Those markings that people normally make become unclear. This breaks. The AI use. In medical work. If people do not make queries or make their markings as they should. That can break the AI. The AI’s purpose. It is to save the medical staff time. In the same way. It's made to make work easier. But the problem is that. Employers see the AI as a chance to fire workers. And if we think that way. The AI benefits only the owners of the companies.
https://scitechdaily.com/chatgpt-was-asked-the-same-question-10-times-the-answers-kept-changing/
