[Updated] Are ‘AI written detectors’ accurate? A letter from 1963 marked as ‘AI written’ goes viral – (Full Video)

New updates are being added at the bottom of this story…….

Original story (published on May 19, 2023) follows:

The rise of artificial intelligence (AI) has undoubtedly brought about significant advancements in various fields, including natural language processing and text generation.

However, with the growing abuse of AI in academic settings, the need for tools to detect AI-generated work became apparent.

These tools were intended to maintain the integrity of educational institutions and ensure that students were producing their own original work.

Are ‘AI written detectors’ accurate?

Unfortunately, as the use of ‘AI written detectors’ increased, unforeseen consequences began to emerge, resulting in controversy and wrongful accusations.

With the advent of powerful language models like Chat GPT, students could effortlessly generate essays, reports, and even entire papers by leveraging the capabilities of AI.

This unethical behavior sparked concerns among educators and prompted the development of AI detection systems to identify instances of AI-generated content.

But recently a user fed GPTZero a letter from 1963, which then went viral across various social media platforms.

Surprisingly, the AI detection tool marked the letter as AI-written, causing a ripple effect of disbelief and concern.

The incident raised questions about the accuracy and reliability of the AI detection systems that had been implemented to safeguard integrity.

In the wake of the viral letter controversy, several individuals began speculating about the potential misjudgments made by AI detection systems (1,2,3,4,5).

This was reinforced by a recent test by The Washington Post on Turnitin which claims a 98 percent accuracy overall, got over half of essays at least partly wrong.

Also, in another test on Writer, The Bible scored only 11% human generated content while a copy-paste from ChatGPT scored 98% human generated content.

If an authentic human-written docvment like the 1963 letter and The Bible could be flagged as AI-generated, then it is plausible that countless students may have faced wrongful punishments due to erroneous AI detection.

This was exactly the case in Texas, where a professor made headlines by failing an entire class of seniors, claiming they had all used ‘Chat GTP’ to write their assignments.

This decision led to the blockage of their graduation, leaving these students devastated and their academic futures in jeopardy.

The professor’s explaining that he had used his own ChatGPT account to check the students’ assignments. If the AI claimed to have generated the content, the students were given a zero grade.

Such instances could be seen on Twitter where students claim that were failed wrongfully due to teacher not understanding how to use these tools or due to wrong results from them (1,2,3).

Funny enough, I Almost failed a history course in college because of one of those AI plagiarism detectors. Thing glitched out and accused my final paper about the Toledo War of being written by some Slavic author who wrote exclusively about Soviet control of Eastern Europe.
Source

In response to these alarming revelations, an educational website, ‘itwasntai.com,’ has been established to provide guidance to individuals who have been wrongly accused of using AI to write their papers.

This platform also aims to educate the teachers and empower those who may have fallen victim to flawed AI detection systems.

The existence of such a website emphasizes the urgent need for a comprehensive review of AI detection tools and the development of more accurate and reliable methods.

This controversy surrounding AI-written detectors has inevitably raised concerns about the blind trust placed in these systems.

The question arises: if we cannot fully trust the AI-generated content itself, why did we blindly trust the AI-written detectors?

Update 1 (May 25, 2023)

03:50 pm (IST): In another instance, a teacher ran final essays through the GPT zero and it flagged almost everything as AI-written.

Students tried to explain that those detectors are random number generators and flag false positives. He showed him how parts of official docvments and books we read were flagged as AI written.

But he told us they were flagged because ‘Chat GPT uses those as reference so of course they would be flagged.’

Update 2 (May 26, 2023)

02:50 pm (IST): There has been an official statement from GPTZero on their FAQ page stating that it should not be used as a tool to accuse students:

They also admit that there have been cases where AI is classified as human, and human is classified as AI. So, educators should use approaches that allow students to demonstrate their understanding in a controlled environment.

PiunikaWeb started as purely an investigative tech journalism website with main focus on ‘breaking’ or ‘exclusive’ news. In no time, our stories got picked up by the likes of Forbes, Foxnews, Gizmodo, TechCrunch, Engadget, The Verge, Macrumors, and many others. Want to know more about us? Head here.