Module 08 · Lesson 2 of 4
Automation Without Nonsense
Automation is a servant of the method, and the method is the query set, the run, the log and the review. Automate the parts of that loop which are mechanical and mistake-prone. Keep your own eyes on everything that requires judgement. Most automation disasters come from swapping those two.
The honest split
| Automate it | Never automate it |
|---|---|
| Issuing the query set on schedule, in fixed conditions | Choosing the query set (a human decided what matters) |
| Capturing answers and cited domains into the log | Labelling context and sentiment (fixed vocabularies, human eyes) |
| Diffing this run against last run | Deciding what a change means |
| Alerting when thresholds from lesson 7.3 are crossed | The monthly interpretation sentences |
| Compiling the report numbers | The verdicts on experiments |
| Flagging pages with stale last-reviewed dates | Whether a page deserves update, merge or retire |
The left column buys back your time for the right column. That is the only justification for any automation: it converts hours of mechanical work into minutes, so the judgement work actually happens. Automation that removes the judgement instead is how an agency ends up confidently reporting nonsense for six months.
The three failure modes
- Silent drift: a surface changes, the bot keeps capturing the wrong element, and the log fills with plausible-looking junk. Defence: a monthly manual spot-check of a few rows against the real surface, forever.
- Conditions creep: automation quietly varies what manual runs held constant (session state, location, personalisation), so old and new rows are not comparable. Defence: when automation starts, start a new baseline; never mix regimes in one trend line.
- Confidence laundering: numbers produced by a machine feel more true, so weak signals get reported as findings. Defence: the interpretation rules stay human and the caveats stay in the report, whatever produced the counts.
The afternoon build
Your first automation should be the smallest useful one: the run diff. Take this week's log and last week's, and produce a short list of what changed: new cited domains, vanished domains, queries where the answer-changed column flipped. A spreadsheet formula version is fine; a script is fine; an agency licence is fine. The point is that Friday's ten minutes of diffing becomes one minute, and you never skip it again.
- Friday, run 14 logged. The diff fires: two new domains cited on the pricing cluster; your citation on one query vanished; answer-changed on four queries.
- You open the four answers yourself (that part stays human) and see a new comparison roundup is feeding the cluster.
- Action: one new opportunity card, one investigation note in the log. Time elapsed: twelve minutes, most of it reading.
The bot did the noticing; you did the thinking. That division of labour is the entire lesson.
- List every mechanical step in your weekly run. Circle what repeats identically each time; that is your automation candidate list.
- Build or configure the run diff (spreadsheet or script) this week and use it on your next two runs.
- Schedule the monthly manual spot-check of automated rows; put it next to the roll-up so it actually happens.
- Write one sentence in your methodology note: what this automation captures, under what conditions, started when. Future-you needs the regime boundary.
What is the only good reason to automate a task?
Why start a new baseline when automation begins?
What defends against silent drift?
Sources used in this lesson
Google Search Central: AI features and your website