Make AI plant bugs in good code

How do you get faster at spotting mistakes in code you didn't write?

When AI writes the code, your job moves from typing to reviewing. Canva says it scores finding and fixing problems in AI-generated code. Most people have never practised that on purpose.

This drill is simple. Take code you know works, change a few small things so it's wrong, and find the changes. Because you know exactly what was planted, you get an honest score.

  • Mutation: A tiny change to working code, such as turning <= into <. Mutation testing uses the same idea to check whether tests catch bugs.
  • False alarm: A line you flagged that was actually fine. Too many waste time in a real round.
  • Seed: A number that makes random choices repeatable, so the same seed plants the same bugs.

Steps:

  1. Pick code you trust. Your own solutions, or code with passing tests.
  2. Plant three bugs. Use the script here, or the prompt for harder logic bugs.
  3. Set a timer and read. Don't run anything. Read for boundaries, empty input, and wrong conditions.
  4. Say each bug out loud. Say what breaks and which input proves it.
  5. Score it. Compare against the planted list, then run the tests to confirm.

Script or prompt?: The script makes small syntax-level changes, which build speed. An AI prompt can plant subtle logic bugs, such as a cache that never refreshes. Use both, but keep the answer key hidden until you've finished.

Prompt, Plant logic bugs:

Here is working code:
[code]
Make a copy with 3 realistic bugs of the kind AI assistants often write: a missed edge case, a wrong assumption about input, and one performance trap. Keep it plausible, with no comments that give it away. Show only the changed code. List the bugs only when I say 'answers'.

Code, Planting the bugs. Finds the places a small change is possible, then picks a few using a repeatable random order.

/** A seeded generator gives the same bugs for the same seed, so you can check your answers later. */
function seeded(seed: number): () => number {
  let s = seed >>> 0;
  return () => {
    s = (s + 0x6d2b79f5) >>> 0;
    let t = s;
    t = Math.imul(t ^ (t >>> 15), t | 1);
    t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
    return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
  };
}

export function plantBugs(code: string, count: number, seed: number): { code: string; planted: Planted[] } {
  const random = seeded(seed);
  const lines = code.split("\n");
  const options: { line: number; rule: number }[] = [];
  lines.forEach((text, line) => {
    if (text.trim().startsWith("//")) return;
    RULES.forEach(([re], rule) => {
      if (re.test(text)) options.push({ line, rule });
    });
  });
  const planted: Planted[] = [];
  while (planted.length < count && options.length > 0) {
    const [pick] = options.splice(Math.floor(random() * options.length), 1);
    if (planted.some((p) => p.line === pick.line + 1)) continue; // at most one bug per line
    const before = lines[pick.line];
    const [re, to] = RULES[pick.rule];
    lines[pick.line] = before.replace(re, to);
    planted.push({ line: pick.line + 1, before: before.trim(), after: lines[pick.line].trim() });
  }
  planted.sort((a, b) => a.line - b.line);
  return { code: lines.join("\n"), planted };
}
  • function seeded(seed: number): A tiny random generator. Math.random can't be repeated, and this can.
  • text.trim().startsWith("//"): Skips comments, because changing a comment plants no bug.
  • planted.some((p) => p.line === pick.line + 1): At most one bug per line, which makes scoring fair.
  • planted.sort: The answer key comes out in line order.

Code, Scoring your review. Three numbers tell you more than a pass or fail.

export function scoreReview(planted: Planted[], linesYouFlagged: number[]) {
  const found = planted.filter((p) => linesYouFlagged.includes(p.line)).length;
  const falseAlarms = linesYouFlagged.filter((l) => !planted.some((p) => p.line === l)).length;
  return { found, missed: planted.length - found, falseAlarms };
}
  • falseAlarms: Lines you flagged that were fine. In a live round, these burn time.

  • missed: The bugs that would have reached production.

  • Failure, Running it right away: Tests find the bug for you. Read first, then run.

  • Failure, Only planting easy bugs: Mix in the prompt version, or you'll only get good at spotting syntax-level slips.

Why use a seed?
The same seed plants the same bugs, so you can repeat a drill and measure progress, and check your answers later.
What should you look at first when reading unfamiliar code?
Boundaries, empty input, conditions, and anything that changes state. Most planted and real bugs live there.

Sources: