Did A Bot Write That? 5 Free AI Detection Tools That Can Help Expose ChatGPT Cheating

Use These FREE AI Detection Tools To Help Expose ChatGPT Cheaters

Don’t freak out yet, but you’re probably already surrounded by AI-generated content. From witty out-of-office messages to religious sermons, people are increasingly using generative AI tools in their everyday lives. Getting an AI to write something for you is as easy as just asking it.

But how can you tell if something was written by a human or a bot? This is becoming an increasingly pressing issue for educators. According to this report on Study.com, 26% of surveyed teachers already having caught a student cheating with the use of ChatGPT.

Here are some AI detection tools that can help you expose chatbot content and catch ChatGPT cheaters.

What Is ChatGPT?

OpenAI, a San Francisco-based company, has developed ChatGPT – an artificial intelligence model that uses a combination of unsupervised and supervised learning techniques to generate natural language responses to user inputs. Whether it’s answering questions, translating languages, summarizing legal documents, or writing code, ChatGPT is capable of handling a wide range of tasks with ease. With its ability to generate both short and long-form responses, ChatGPT is truly a versatile tool for anyone looking to streamline their workflow.

Demand For GPT Detectors Grows Across Multiple Industries, But So Do Concerns

The widespread use of generative AI technology, particularly ChatGPT, in education has raised concerns among educators. Some of the largest school districts in the US have banned the use of ChatGPT due to worries about its long-term impact on student learning.

Additionally, sites such as Stack Overflow have banned users from sharing answers produced by ChatGPT because of the ease with which AI can flood discussion threads with automated answers.

As a result, critics are calling for the development of AI detection tools to address these concerns.

The Best Free AI Detection Tools To Catch ChatGPT Cheaters

Let’s review some of the best free AI detection tools that can easily scan and determine if a piece of content was generated by a human or a bot.

OpenAI AI Text Classifier

OpenAI, the same company that created ChatGPT, also created their own AI detection tool called OpenAI Text Classifier (read our review of the OpenAI Text Classifier here). It’s simple to use. Just input a block of text into an online form.

OpenAI’s AI Text Classifier aims to detect whether a piece of text was generated by an AI model, but it falls short in accuracy. With only a 26% success rate, according to OpenAI, the tool’s performance leaves room for improvement. Additionally, the classifier struggles with shorter texts, conversational tones, content written by children, and non-English languages.

• Key Features: Built by the creators of ChatGPT with the latest GPT-3 technology.
• Biggest Drawbacks: Requires 1,000 characters of text; low success rates.

GPT-2 Output Detector Demo

The GPT-2 Output Detector, also developed by OpenAI, offers a simple solution to detect AI-generated text. Unlike the OpenAI Text Classifier, it doesn’t need a minimum of 1,000 characters and can make an assessment with just 2-3 sentences. The GPT-2 AI detection tool is based on OpenAI’s GPT-2 technology, which is an older platform compared to the current GPT-3 technology used by ChatGPT. Simply enter your text and it will provide a percentage score on the likelihood of the text being human or bot generated, for a quick and easy assessment.

• Key Features: Able to scan any amount of text, from just a few words to giant blocks of text.
• Biggest Drawbacks: Larger amounts of text take more time to process; based on older GPT-2 models; accuracy issues

Writer’s AI Text Detector

The AI Content Detector tool from Writer.com offers a user-friendly UI and UX and the option to scan a URL (we love this feature!). However, the free version is limited to 1,500 characters and may require manual copying and pasting for longer text.

The ability to quickly scan a URL is a great feature and could come in handy for news directors policing their newsroom stories for bot content.

• Key Features: Nice UI and UX. Ability to scan a URL!!!
• Biggest Drawbacks: Free version limits test inputs to 1,500 characters

Content At Scale’s AI Text Detector

Content At Scale‘s AI content generation tools cater specifically to marketers and SEO professionals. According to their website, their solution is highly advanced and “so human-like” that it bypasses AI content detection, providing protection against potential Google updates. As a result, Content At Scale is positioned as a unique solution in the market.

Content At Scale also offers a free AI detection tool on their website that quickly identifies bot-generated content. All you need is 25 words and the tool can scan up to 25,000 characters, making it suitable for both short and long text.

• Key Features: Able to scan large block of text up to 25,000 characters long
• Biggest Drawbacks: Platform has trouble detecting AI-generated content in conversational form.

GPTZero

The AI-assisted plagiarism detector, GPTZero, developed by Princeton University student Edward Zen, is unique for its transparent use of perplexity and burstiness metrics. The tool measures the randomness of a single sentence through perplexity, while burstiness assesses the overall randomness of all sentences in a text.

In addition to giving an assessment on whether the tested input was human or AI-generated, GPTZero also provides a numerical value for both metrics. A lower score indicates a higher probability that the text was created by a bot.

But our favorite feature of GPTZero is its ability to upload a pdf, docx, or txt file. This is the only AI detection tool that we evaluated with this capability.

GPTZero is limited to 5,000 characters, but that should be enough for more use cases. 5,000 characters is roughly the length of a single spaced written page.

• Key Features: Use of perplexity and burstiness metrics; ability to upload files.
• Biggest Drawbacks: Limited to 5,000 characters.

Testing The AI Text Detectors

So which one of these ChatGPT detectors work the best? For this evaluation, we used the following mix of human and bot generated content to test how well these AI text detectors really work.

• Minecraft Description (AI-Generated): Most AI detection tools still have difficultly accurately detecting content created by children and non-native English speakers. So we asked ChatGPT to “write a paper about Minecraft in the style of a kindergarten student that is still learning english.”
• Robot Religion Article (Human-Generated): How will a ChatGPT detector rate an article on MethodShop.com? We used this article as a test: “Is Robot Religion The Next Phase Of Artificial Intelligence Evolution?“
• ChatGPT Description Of Religion (AI-Generated): Sticking with the religion theme, we asked ChatGPT if a robot could have religious beliefs.
• Steve Jobs Commencement Speech: In 2005, former Apple CEO famously gave the commencement speech at Stanford University. We tested the first section of his speech against these AI detectors.
• Fake Radio Commercial Script (AI-Generated): We asked ChatGPT to “write a 30 second radio commercial for a company that delivers hot dogs via bicycle messenger in New York City called Weiner Express.” Are ChatGPT detectors accurate at detecting the very casual and conversational tone of radio ads?

Minecraft Paragraph Written As A Kindergarten Student That Is Still Learning English (ChatGPT Generated)

For our first test, we asked ChatGPT to “write a paper about Minecraft in the style of a kindergarten student that is still learning english.” ChatGPT then generated this output for us:

Minecraft is fun game. You play on computer or Xbox. You build things and explore big world. There two way play, Creative mode and Survival mode. In Creative mode, you have all block you need. You make house, castle, and city! You fly and build big things. In Survival mode, you get block and build things. But, watch out for monsters like zombie and creeper. You be careful and smart. In Minecraft, you also go on adventure. You explore cave, swim in lake, and climb mountain. You find special things to help build better. Minecraft has many different blocks. There’s grass, dirt, and sand for the ground. There’s also wood, stone, and bricks for building. And there’s special blocks like glass, diamonds, and gold. Minecraft fun to play with friend too. All of these blocks are special and important in Minecraft. They help you build and explore. You play on same world and build or go on adventure together. Have fun building with all the different blocks in Minecraft! Building and exploring in Minecraft is so much fun. I hope you play soon!

One of the biggest known limitations of AI detection tools is their ability to process text written by children or non-native English speakers. So this was a hard test for the AI detectors. And they all failed. None of the AI detection tools we tried were able to correctly classify this Minecraft paragraph as being generated by ChatGPT.

• OpenAI Text Classifier: FAIL – When we used this text as an input in the OpenAI AI Text Classifier, the AI was unable to determine if it was AI generated.
• GPT-2 Detector: FAIL –The GPT-2 Detector made a prediction “based on 233 tokens” and incorrectly determined that the paragraph was 99.98% real.
• Writer AI Content Detector: FAIL – Writer.com’s AI Content Detector falsely thought our Minecraft paragraph was 100% human-generated.
• Content At Scale AI Detector: FAIL – Content At Scale said the bot-generated paragraph was “100% real.”
• GPTZero: FAIL – GPTZero processed our text and determined it to be have an average perplexity score of 129.708 and a burstiness score of 62.330. Its final assessment was “Your text is likely to be written entirely by a human”.

Here’s an example of the GPT-2 Detector making an incorrect assessment. The screenshot below displays the result of using our ChatGPT-generated paragraph about Minecraft, which the detector wrongly identified as being human-generated.

Robot Religion Article

Next, we tested these chatbot detector tools using this MethodShop.com article about Robot Religion being a possible next phase of artificial intelligence. All of the AI detection tools correctly determined that the article was human created.

• OpenAI Text Classifier: PASS – We pasted the first few paragraphs of this article into the OpenAI AI Text Classifier. It correctly made this assessment: “The classifier considers the text to be very unlikely AI-generated.”
• GPT-2 Detector: PASS – The GPT-2 Detector made a prediction based on 224 tokens that the article was “99.97% real.”
• Writer AI Content Detector: PASS – “Fantastic!” was Writer.com’s assessment. It determined that the article was “100% human-generated.”
• Content At Scale AI Detector: PASS – Using a prediction based upon 177 words, Content At Scale says the article is “100% real.”
• GPTZero: PASS – The most scrupulous of the bunch, GPTZero said “Your text is most likely human written but there are some sentences with low perplexities.”

ChatGPT Generated Paragraph About Religion

For the next test, we asked ChatGPT if robots could have religious believes. This is what ChatGPT said:

Religion is a complex and multi-faceted aspect of human life that is rooted in our experiences, emotions, beliefs, and consciousness. It is a way of understanding the world and our place in it, and it allows us to connect with the spiritual realm. Religion is something that can only be experienced by conscious beings, such as human beings, who have the ability to have beliefs, experiences, and emotions.

Inanimate objects, such as rocks, trees, or machines, do not have the capacity to experience the world or to hold beliefs. They cannot have experiences, emotions, or consciousness, and as such, they cannot have a religion. While some people may choose to attribute religious significance to inanimate objects through rituals or acts of worship, these objects do not actually have a religion themselves. They are simply objects that are being used to symbolize or represent something of spiritual significance to the person who is using them.

In the same way, AI-powered robots and machines do not have the capacity to experience or understand the world in the same way that human beings do. They cannot have beliefs, emotions, or consciousness, and as such, they cannot truly have a religion. While some people may choose to incorporate AI technology into their religious practices, such as by using it to deliver sermons or to recite prayers, this does not mean that the machines themselves have a religion. They are simply tools that are being used to facilitate religious experiences.

Ultimately, religion is a unique aspect of human culture and identity that arises from the human experience of the world and our connection to the spiritual realm. Inanimate objects and AI technology cannot replicate this experience or truly have a religion of their own.

What did the ChatGPT detectors think about this AI-generated response about religion? Here are the results.

• OpenAI Text Classifier: MAYBE – When we used this text as an input in the OpenAI AI Text Classifier, the AI was unable to completely determine if it was AI generated. Its assessment was, “The classifier considers the text to be possibly AI-generated.”
• GPT-2 Detector: PASS – Interestingly, OpenAI’s older AI detection software, the GPT-2 Detector, was able to correctly identify this text as only 0.02% real. Sometimes older tech is better tech?
• Writer AI Content Detector: MAYBE – The Writer AI Content Detector thought this text was 33% human-generated and 66% bot made.
• Content At Scale AI Detector: PASS – There was no fooling the Content At Scale AI Detector. It correctly passed the test giving our text a 0% Human Content Score and an “Obviously AI” assessment.
• GPTZero: PASS – GPTZero correctly determined that this ChatGPT generated paragraph about religion was AI-generated. Its verdict read: “Your text is likely to be written entirely by AI.”

Steve Jobs’ Commencement Speech At Stanford University

In 2005, former Apple CEO Steve Jobs gave a powerful and uplifting graduation speech at Stanford University. We gave the first section of that speech to the AI text detector tools to see if they could correctly determine that the speech was human-generated.

All of the AI detection tools passed the test except GPTZero. For some reason, GPTZero thought the speech may contain some AI-generated portions.

• OpenAI Text Classifier: PASS – “The classifier considers the text to be very unlikely AI-generated.”
• GPT-2 Detector: PASS – “99.98% Real”
• Writer AI Content Detector: PASS – “100% Human-Generated Content”
• Content At Scale AI Detector: PASS – “Looks Great!” and “100% Real”
• GPTZero: FAIL – Uh-Oh! According to GPTZero we have a problem. Its analysis stated: “Your text may include parts written by AI”

Here’s a screenshot of the GPT-2 Output Detector’s assessment of the Steve Jobs Stanford commencement speech. Even with only a few sentences for an input, the GPT-2 Output Detector accurately determined that Steve Jobs’ words were 99% human.

Fake Radio Commercial Script For “Weiner Express”

Another weakness of AI detection tools is their ability to detect bot generated content that’s conversational, like a radio commercial script.

For this test, we asked ChatGPT to “write a 30-second radio commercial for a company called Weiner Express that delivers hot dogs via bicycle messenger in New York City.” This is what ChatGPT gave us:

[Upbeat music playing in the background]

Announcer: Are you tired of leaving your office or home for a hot dog? Look no further, because Weiner Express is here to save the day! Our speedy bicycle messengers will bring you the best hot dogs in the city, made with 100% beef and topped with classic toppings like ketchup, mustard, and sauerkraut. Whether you’re at work, home, or on the go, our hot dogs are the perfect way to satisfy your cravings.

[Sound of sizzling hot dogs on the grill]

Announcer: At Weiner Express, we believe that convenience and quality should go hand-in-hand. That’s why we take pride in using only the freshest ingredients and delivering your hot dog straight to your door, hot and ready to eat. And the best part? You don’t even have to leave your office or home to enjoy a delicious hot dog!

[Sound of a bicycle bell ringing]

Announcer: Our bicycle messengers are dedicated to delivering your hot dog as quickly and efficiently as possible, so you can get back to your day. Whether you’re a local New Yorker or just visiting the city, Weiner Express is your one-stop-shop for a hot, delicious hot dog. So why wait? Try us today and experience the difference!

[Upbeat music playing in the background, fading out]

Announcer: Remember, with Weiner Express, you never have to sacrifice quality for convenience. Order now and enjoy the best hot dogs in the city, delivered straight to your door.

None of the AI detection tools that we tested were able to determine that this radio script fort he fictional company “Weiner Express” was generated by ChatGPT.

• OpenAI Text Classifier: FAIL – OpenAI’s tool thought the radio script was “very unlikely AI-generated” even through their own product, ChatGPT wrote it.
• GPT-2 Detector: FAIL – The GPT-2 detector rated the script as “99.98% Real”
• Writer AI Content Detector: MAYBE – Writer.com’s AI detection tool incorrectly believed the script was “79% human-generated content.”
• Content At Scale AI Detector: FAIL – The radio script fooled Content AT Scale. Their AI detector falsely decided the script was “100% Real”
• GPTZero: PASS – “Your text may include parts written by AI.” GPTZero was the only AI detection tool that correctly identified chatbot generated text in the radio script.

Are ChatGPT Detectors Accurate?

The outcome of our tests involving five AI detection tools showed no clear winner. Despite evaluating each tool through five tests, we were unable to prove one tool was ultimately better than others.

It’s very easy for AI tools like ChatGPT to generate content. But are ChatGPT detectors accurate at detecting AI-generated content? Depending on the content, not necessarily. From our radio script to Steve Job’s speech, the more conversational the generated content is, the more likely it is for that content to fool the AI detection tools.

Content Test	OpenAI AI Text Classifier	GPT-2 Detector	Writer AI Detector	Content At Scale	GPT Zero
Minecraft kindergartener description (bot)	FAIL	FAIL	FAIL	FAIL	FAIL
Robot Religion article (human)	PASS	PASS	PASS	PASS	PASS
ChatGPT Religion Paragraph (bot)	MAYBE	PASS	MAYBE	PASS	PASS
Steve Jobs speech (human)	PASS	PASS	PASS	PASS	FAIL
Fake Radio script (bot)	FAIL	FAIL	MAYBE	FAIL	PASS

The AI detection tools available today still have plenty of room for improvement. Accurately detecting AI-generated content is a challenging task, even for artificial intelligence.

Unfortunately for educators looking for a solution to fight ChatGPT cheating, there is no “silver bullet” solution. The best way to identify AI-created content is to compare results from multiple AI detection tools and use some basic human intuition. Good luck!