Part 5 - How to Spot AI Mistakes Before Your Kids Don't

Part 5 of 12 in the Try It Yourself series — practical AI guidance for parents at blog.parentsguidetoai.ca

---

AI makes mistakes that look exactly like correct answers. That's what makes them worth paying attention to.

We covered this briefly in Post 2 — AI produces confident-sounding output regardless of whether it's accurate. In this post, we get practical. You're going to learn a simple routine for catching AI errors, so you can use it yourself and teach it to your children.

The good news: this doesn't require any special skills. It mostly requires slowing down for sixty seconds before you trust what you're reading.

---

The four questions the parent's guide gives students

Section 7 of the parent's guide lists four questions students should ask about AI-generated information. They're worth reading as a parent too, because they apply just as well to adults:

• Where did this information come from?
• Can I verify it?
• Does another source agree?
• Could this be fabricated?

These are good questions. The challenge is that they assume you know how to follow up on the answers. So let's make that concrete.

---

Where AI most often goes wrong

Not all AI errors are equally likely. Once you know the patterns, you'll catch mistakes faster.

Citations and academic sources are the highest-risk category. AI will generate plausible-looking citations, author names, journal titles, article titles, volume numbers, that simply do not exist. The citation looks real. The paper doesn't. This happens with regularity, even with the best AI tools, and it doesn't just happen in education - it's happened with scientific journals,¹ published reports from companies known for doing excellent consulting work, and many times in legal proceedings.²

If your child's homework references an academic paper or study, verify it exists before it gets submitted. You can also use a simple prompt to help you with this, something like "Review the attached document, and it's references, and verify that the references and citations actually exist, and prepare a step by step guide on how to validate them myself."

Statistics and specific numbers are another common problem. AI will state figures — percentages, populations, dollar amounts, growth rates — with the same confidence whether they're accurate or invented. Numbers feel authoritative. They're not always right.

Names, dates, and local information are also risky. Details about real people (including minor public figures), specific events, dates of legislation, or information specific to a region — like BC policy or Vancouver schools — are more likely to be wrong than general information.

Recent events are a particular vulnerability. AI systems have a knowledge cutoff — they were trained on data up to a certain date, and they don't reliably know what happened after that.³ The parent's guide itself notes this:

"AI tools, school policies, and support laws change quickly."

If you're asking AI about something from the last year or two, verify it independently.

---

A practical verification routine

Here's what to actually do when you want to check something AI told you.

Step 1: Identify the specific claim. Don't try to verify the whole response. Pick the most important fact — the one the rest of the response depends on. That's what to check.

Step 2: Use Perplexity for the same question. Perplexity is designed to show its sources inline. Ask it the same question and look at what sources it cites. If the sources are real and say what Perplexity says they say, that's a good sign. If Perplexity can't find sources, that's also useful information.

Step 3: Go to one primary source. Don't just check other AI tools — they may have the same wrong information. Go to an official site, a known news organization, or a government resource. For BC-specific information, the BC Ministry of Education and Vancouver School Board sites are reliable.

Step 4: For academic citations, search the title. If AI cited a paper, search for the exact title in Google Scholar or through a library database. If you can't find it, the paper probably doesn't exist.

This routine takes about ninety seconds for a straightforward claim. It takes longer for complex research. But it's the difference between using AI as a starting point and using it as the final word.

---

Teaching this to your kids

Here's the thing about verification: if you've done this yourself, you can explain it to your child in terms of a real example. That's much more effective than a general warning.

The parent's guide suggests asking: "Could you explain this work without using the AI?" A companion question that's equally useful: "Did you check whether this is actually true?"

Neither of these questions is an accusation. They're the same questions a good teacher or editor asks about any piece of work. AI just makes them more necessary.

---

The short version:

• AI mistakes look identical to correct information — there's no warning label.
• The riskiest areas: citations (often invented), statistics (often fabricated), local/recent information (often outdated or wrong).
• A simple routine: identify the key claim, check it in Perplexity for sourced results, then verify at a primary source.
• Doing this yourself gives you a real example to share with your child.
• The two questions to ask: "Could you explain this without the AI?" and "Did you check whether it's true?"

---

Now that you can spot mistakes, we're ready to look at what AI is genuinely good for. The next post is about using AI as your own personal research assistant — and how to do it without the same pitfalls we've been discussing.

Try it today: Take a piece of AI-assisted content — something your child brought home, or something you generated yourself this week — and run one claim through the verification routine. See if it holds up.

Next up: Post 6 — Using AI as Your Personal Research Assistant

---

Notes