How do AI (Originality) Detectors Work? Are They Reliable?

By 447 words2 min read

First Published on 11 April 2023.

Why Should You Read this Blog Post?

While AI has indeed become a powerful tool for optimizing workflows and accelerating career growth, the question arises: can we reliably detect AI-generated text? In this blog post, we will explore the importance of AI text detection, its implications, and the challenges it presents. Drawing insights from the provided articles, we’ll dive into the mechanisms behind AI detection and its accuracy, shedding light on this evolving field.

 

What is the Use of AI Text Detection?

Imagine AI whipping up content left and right, from news articles to marketing pitches, even academic papers and legal documents. This is an incoming reality. It’s not that AI-produced content is necessarily bad, but sometimes it’s handy to know if you’re dealing with a machine-crafted piece or a human masterpiece. The safety implications of publishing ChatGPT-generated thesis papers alone could be dangerous. Being able to spot AI text helps us assess the reliability of content up to a certain human standard. However, not all uses of AI require such measures: if you stumble upon a marketing copy written by AI, why not use the same tools to recreate the magic yourself?

How AI Writing Detection Works:

AI writing detection tools primarily rely on analyzing the context preceding the next word in a given text. Through extensive training on vast amounts of data, AI models can identify and analyze patterns within the contextual word set. For example, if the context is “I’m tired. I want to go to ___,” an AI language model might predict “bed” based on its training data, where it observed a high probability of the word “bed” following the phrases “tired” and “go to.” By calculating the likelihood of each word being the next predicted word, AI models generate text predictions based on context and patterns. Additionally, AI detectors such as OriginalityAI and GPTZero measure “burstiness” and “perplexity” values to conjure originality scores. Although not accurate, these detectors are great for all educators, institutions, and interested others to detect “AI plagiarism.” This brings us into our final segment.

The Accuracy of AI Detectors:

AI writing detection tools are not flawless; rather, they offer insights into the origin of the text. If suspicions arise regarding AI-generated content and a detector indicates a 100% likelihood of it being AI-written, it is likely to have been authored by an AI tool such as ChatGPT. However, it is crucial to approach these results with a degree of skepticism and cross-check such results across various detectors as they are likelihoods rather than definitiveness.

References:

A. “How AI Writing Detection Works” by Goldpenguin.org

B. “What is Responsible AI?” by PwC

Authored By: Harshaan Chugh

editor's pick

news via inbox

Nulla turp dis cursus. Integer liberos  euismod pretium faucibua