Skip to main content

Ahead of GPT-5 launch, another test shows that people cannot distinguish ChatGPT from a human in a conversation test — is it a watershed moment for AI?

Web Hosting & Remote IT Support

Popular AI tools such as GPT-4 generate fluent, human-like text and perform so well on various language tasks it’s becoming increasingly difficult to tell if the person you’re conversing with is human or a machine. 

This scenario mirrors Alan Turing's famous thought experiment, where he proposed a test to evaluate if a machine could exhibit human-like behavior to the extent that a human judge could no longer reliably distinguish between man and machine based solely on their responses.

The Department of Cognitive Science at UC San Diego decided to see how modern AI systems fared and evaluated ELIZA (a simple rules-based chatbot from the 1960’s included as a baseline in the experiment), GPT-3.5, and GPT-4 in a controlled Turing Test. Participants had a five-minute conversation with either a human or an AI and then had to decide whether their conversation partner was human.

No better than chance

GPT-4 and  GPT-3.5 were told how to respond to messages. Researchers instructed the model to “adopt the persona of a young person who was very concise, didn’t take the game very seriously, used slang, and made occasional spelling errors”. In addition, the prompt contained “general information about the setup of the game, and information about recent news stories that had occurred after the model’s training cutoff”. The two models were accessed through the OpenAI API with temperature set to 1 and to prevent the AI from responding too quickly, their messages were sent with a delay based on the number of characters in the message.

In the study, 500 participants were assigned to one of five groups. They engaged in a conversation with either a human or one of the three AI systems. The game interface resembled a typical messaging app. After five minutes, participants judged whether they believed their conversation partner was human or AI and provided reasons for their decisions.

The results were interesting. GPT-4 was identified as human 54% of the time, ahead of GPT-3.5 (50%), with both significantly outperforming ELIZA (22%) but lagging behind actual humans (67%). Participants were no better than chance at identifying GPT-4 as AI, indicating that current AI systems can deceive people into believing they are human.

Analysis of the results showed that interrogators often relied on linguistic style, socio-emotional factors, and knowledge-based questions to decide if they were talking to a human or a machine.

Details of the test and the full results can be seen in the paper published on the arXiv preprint server.

More from TechRadar Pro



via Hosting & Support
Labels:

Comments

Post a Comment

Web Hosting
Remote IT Support
Sites & Socials

Popular posts from this blog

AI tools are making social engineering attacks even more convincing, and I fear that this is only the beginning

Web Hosting & Remote IT Support Nick Park’s Wallace and Gromit were brought crashing into the 21st century in December 2024 with their latest adventure, Vengeance Most Fowl . The film challenges our growing dependence on smart technology in the form of a robotic garden gnome, built by Wallace to support his gardening business, which is then hacked by the Kubrick-esque Feathers McGraw for his own nefarious purposes. One of the more interesting but less commented on parts of the film shows Gromit cautiously entering his house and being greeted by what he thinks is Wallace’s reassuring voice, only to be confronted with Feathers and the robotic gnome. Technology’s ability to mimic linguistic patterns, to clone a person’s voice and understand and respond to questions has developed dramatically in the last few years. This has not gone unnoticed by the world’s criminals and scammers, with the result that social engineering attacks are not only on the rise but are more sophisticated a...
Post a Comment
Read more

Microsoft, Google, and Meta have borrowed EV tech for the next big thing in data centers: 1MW watercooled racks

Web Hosting & Remote IT Support Liquid cooling isn't optional anymore, it's the only way to survive AI's thermal onslaught The jump to 400VDC borrows heavily from electric vehicle supply chains and design logic Google’s TPU supercomputers now run at gigawatt scale with 99.999% uptime As demand for artificial intelligence workloads intensifies, the physical infrastructure of data centers is undergoing rapid and radical transformation. The likes of Google, Microsoft, and Meta are now drawing on technologies initially developed for electric vehicles (EVs), particularly 400VDC systems, to address the dual challenges of high-density power delivery and thermal management. The emerging vision is of data center racks capable of delivering up to 1 megawatt of power, paired with liquid cooling systems engineered to manage the resulting heat. Borrowing EV technology for data center evolution The shift to 400VDC power distribution marks a decisive break from legacy sy...
Post a Comment
Read more