The rise of generative AI, like ChatGPT, has transformed industries. Their capacity to create coherent, relevant text in real-time is revolutionary. But this power also reveals credibility issues in the employee training sector.
There’s growing concern that employees, trainees, or students may use generative AI to answer assessments. This compromises their learning and the validity of assessment results. Results may no longer be trusted and you may not be able to properly address skill gaps which could lead to further issues down the line.
How can you detect AI in assessments and avoid these potential pitfalls? Let’s dive into our suggested solutions below.
How to Detect AI in Assessments: Methods & Risks
Detecting AI-completed assessments demands a blend of technical and manual methods.
1. Technical Methods & Challenges
On the technical front, statistical analysis may be employed. This method compares a text’s patterns to known AI-generated outputs. Trained machine learning models may recognise these patterns and highlight potential AI-originated content. Another technical approach involves semantic analysis. This examines a text for abrupt topic shifts or irrelevant inclusions common in AI responses.
However, these technical methods come with inherent risks. Machine-trained algorithms may still lead to false positives. They may flag genuine human responses as AI-generated. At the same time, slow updates to these algorithms run the risk of them becoming obsolete. The rapid evolution of AI means that detection machines must continuously adapt, too.
2. Manual Methods & Challenges
Manually, the assessment may be scrutinised for telltale signs of AI generation. For example, there’s a lack of personal experiences or plenty of generic responses In this method, one can challenge employees with unique prompts or questions. These will require personal insight. AI lacks this personal human feature.
Yet, manual reviews aren’t foolproof. Human bias, fatigue, or unfamiliarity with AI quirks may also lead to oversights. It’s also possible to adapt prompts to provide more human-like responses, which can be increasingly difficult to recognise using manual methods.
3. Use for Initial Screening Only
In light of these challenges, you may view technical and manual detection methods as initial screening tools instead. They can act as the first line of defence. They may sift through submissions and flag obvious anomalies. A holistic assessment approach may offer a more robust solution to preserving the authenticity of assessments.
AI Detection Tools: Possibilities & Risks
Now, AI detection tools are widely available on the Internet for free or for a price. We’ve listed some of them below.
Free AI Detection Tools
GPTZero (Free)
GPTZero (view site) is an AI detection tool developed by Princeton student Edward Tian. It lauds itself as the gold standard of AI detection. It says it can detect texts generated by ChatGPT, GPT4, Bard, LLaMa, and other AI models.
The free GPTZero Classic has a 5,000-character limit per detection load and a 3-file limit per batch upload. GPTZero itself cautions that edge cases occur where AI is not detected correctly. It recommends that more holistic assessments be made to prevent and minimise possible AI use.
ZeroGPT
ZeroGPT (view site) is another free-to-access AI detector tool. It uses a proprietary DeepAnalyse Technology to identify text generated by ChatGPT, GPT 3, GPT 4, Bard, LLaMa, and other AI models.
It lauds itself as a 98% accurate AI detection tool. However, due to the hidden methodologies the machine model uses to detect AI, you may never be 100% sure of its accuracy.
Paid AI Detection Tools
GPTZero (Premium)
GPTZero (view site) also offers paid subscriptions for higher capacities in its machine. ZPTZero Educator costs US$9.99 per month, while GPTZero Pro costs US$19.99. It also has an API Subscription Plan for businesses at US$49.99. Again, even with these plans, the GPTZero machine may still make mistakes.
Originality.ai
Originality.ai (view site) is a business-exclusive service. It offers AI detection features as well as a plagiarism checker and paraphrase checker. It offers a single one-use scan for US$30. It also has a subscription for US$14.95 per month. Geared towards business organisations, Originality.ai can only detect so much. AI nowadays may be instructed to make their generated text more human-like.
How to Detect AI-Generated Text According to ChatGPT
Therein lies the ultimate challenge of AI detection. Even with intelligent machines, AI detection may not be 100% accurate. ChatGPT itself created an AI detection model but took it down due to low accuracy.
When we asked the AI itself if it could detect AI-generated text, ChatGPT responded that it could try but could never be completely sure. One reason given was highly variable AI outputs where some generated texts strongly resemble human-made texts (human mimicry). On the opposite end, humans-written content could also sometimes be flagged as ‘AI-generated’ simply because of how it was written.
The most important reason is that there is no ground truth. Unless there is a benchmark that definitively says that a text is AI-generated, the judgment of any text will prove tricky. Because of this, generative AI companies such as OpenAI have been thinking of watermarking generated text. This ensures that AI text may be judged as AI text. However, this feature has not yet been launched by any AI company, and it could negatively affect their customer base.
Detect AI Completed Assessments through Holistic Methods
Given the current challenges to AI detection today and the wait for watermarking promised by AI companies, AI detection may be done best through holistic methods of detection.
Use a mix of technological tools and manual processes to detect as many AI-completed assessments as possible. Compare and contrast past employee outputs to check outcomes from new learning opportunities now. Finally, employ preventive practices such as using assessment methods and other types of assessment that are difficult or impossible for AI to do. This may also stop employees from even thinking about using generative AI.
Other Tech Trend Topics
Interested in more tech trend topics? Stay up to date on our blog or check out some of our recent articles:
Deliver Your Skills Training Without Compromise
