🧪 Test Plans, and Where AI Gets It Wrong v0.1.0
Students write a test plan and use it to catch three planted mistakes of the kind AI really makes: off by one, a rule bent at the edge, and a confident wrong explanation. They end by asking an AI themselves and checking.
You will need
- The printable's test plan and three "AI wrote this" sheets
- Pencils
- A screen at the front for the last step, or nothing if you use the paper version
The lesson, step by step
Why check
Say that an AI writes code that looks right far more often than it is right, and it says it with the same confident voice either way. The way to trust a program is not how it sounds; it is that you ran it on cases you chose, including the odd ones.
Make a test plan
Show a test plan as a table with five columns: what I am testing, the input, what I expect, what I got, and pass or fail. Fill in two rows together for a rule such as "a free item at 10 points or more": 9 (no), 10 (yes, the edge), 11 (yes), 0 (no) and nothing at all.
Three AI-wrote-this sheets
Give out the sheet. It has three short programs, each with the claim an AI made about it. Students run each by hand, as in the last lesson, and fill in the table. They are not told which kinds of mistake are hiding.
Name the mistake
Gather the class. The first sheet is an off-by-one: the repeat runs 4 times when the job needs 5. The second bends a rule at the edge: more than 10, when the rule says 10 or more. The third is a confident explanation that is wrong: it says two hops where the steps give four. Students sort each into its kind.
Find the cheapest test
Ask: "Which one test would have caught all three?" Often it is the edge case, and often it is running the program on a number small enough to check by hand. Write the answer on the board.
Now ask the AI
Use the box below. This time the students are the testers and the AI is the one being tested.
Questions to ask while they work
- What are you testing, and what do you expect?
- What is the edge of this rule?
- Where did the AI sound sure and was wrong?
- Which one test would catch the most?
Now ask the AI
Pick a rule the class cares about, such as a score for a game or a number of days. The teacher types, at the front, on a screen everyone can see. Use an AI tool your school allows. No name, school or personal detail goes into the box, ever.
What to send:
Write steps in this plain style for the rule below. Use lines like set score to 0, repeat 3 times:, add 2 to score, if score is more than 5:, and say hop. Then tell me what it will say when the number is [9], [10] and [0]. [paste the rule]
What to check in what comes back:
- Run its steps by hand on the three numbers. Does it say what it claimed?
- Does it get the edge right (for example, exactly 10)?
- Did it sound more sure than it should have been?
- Which row of your test plan caught the mistake, if there was one?
Write it down: Write down what it got wrong, or, if it got everything right, which test you ran that proved it. Say what you would have missed if you had not run the test.
No AI today? Do it on paper: The planted-mistake swap
The same step, with people and paper, for when an AI is resting, not allowed at your school, or not what you want in the room.
- Each pair writes a rule on a card and steps in the plain style for it, with one mistake planted that is easy to miss: an edge, a count off by one or a missing case.
- Swap with another pair. Do not say what the mistake is.
- The new pair writes a five-row test plan from the rule alone, then runs the steps against it.
- Compare: which row caught the mistake? Was it an edge, a count, or a missing case? Write it down.
If it goes fast
- Write a program with a hidden mistake for another group to find with only a test plan.
- Count how many tests it takes to catch each of the three mistakes.
- Write the rule: "Never trust, always test" in your own words, and say where it should not be taken too far.
The printable
One sheet each. The teacher's key (shown on the lesson page, not on the sheet) is the output of each program, run as written; the mistakes are named in lesson step 4.
🧪 Test plan and three "AI wrote this" programs
Name: ______________________
Test it, don't trust it
1. AI said: "This adds up the numbers 1, 2, 3, 4 and 5." What does it really say?
set count to 0 set total to 0 repeat 4 times: add 1 to count add count to total say total
2. AI said: "This gives a free item at 10 points or more." The score is 10. What does it really say?
set score to 10 if score is more than 10: say free
3. AI said: "This says hop two times." What does it really say?
repeat 2 times:
repeat 2 times:
say hop4. Test plan for any rule you choose: what I test, the input, what I expect, what I got, pass or fail. Write five rows, with the edge in one.
Teacher key (not on the printable)
- Program 1 really says:
10 - Program 2 really says:
nothing at all - Program 3 really says:
hop, hop, hop, hop
◂ Say It So Exactly That a Machine Gets It · All ten lessons
Nothing on this page is sent to us. The printable has a blank name line and no child's details. No sign-in, no ads.