scriptease.dev
Making Up Insights
I read that my coding assistant ships with a built-in command that studies how you work with it and reports back. Why not give it a shot? The first section — titled "At a Glance" — looked fine: actual feedback on how I prompt, what I do right. Then I got into the details and thought: that's not right. This is so specific, it's like it took five of my three thousand sessions and built the whole report out of them.
I had no idea how close that guess would land.
The report card
The command is called /insights. It samples your past sessions — the conversations you've had with the assistant — and writes you a usage report: what you worked on, what went well, where things ground, what to try next. A report card about our collaboration, written by the other party.
I liked the idea. I still do. And the page it produced was handsome: 3,210 sessions on record, 143 of them analyzed, 2,500 hours counted, and a claim to know where our work got stuck.
Then I read the suggestions.
"wtf is this"
One suggestion was to package my blog-writing workflow as a reusable tool. Sensible — except for the tool it proposed. Its first instruction, verbatim: "Find newest fishslop Maestro session."
Fish slop was last week's post. A one-off. Built, published, done. The report had watched me turn that one project into a blog post and templated the instance instead of the pattern — a "reusable" tool with a one-time fish baked into line one. It gets useful again the day I build a second underwater reef.
Another suggestion: try headless mode — running the AI on a schedule, no human present, like a cron job. The example it offered was a command to run my daily vault backup.
I run about half my sessions headless. Including — literally, specifically — the daily vault backup, every morning at seven. The report was recommending I try the mechanism that produced the logs it was reading.
I interrupted it mid-sentence with "wtf is this," which my dictation faithfully preserved.
Three out of three thousand
Next to the report, the tool leaves a folder of working files — one small analysis per session it looked at. I asked the assistant to inspect its own working files. Fifty of them. Forty-seven were not my sessions.
They belonged to my note-taker Claude-Mem. My setup includes a memory plugin, and for every real session it spawns a little observer session — a stenographer in the corner, transcribing what the main session does so future sessions can remember it. Those transcriptions are sessions too, technically. They live in the same folder as the real ones.
The report couldn't tell them apart. So every real working session got counted several times — once for the meeting, and once for each stenographer describing the meeting. One evening of blog writing became "ten sessions of blog publishing." The headline said 143 sessions analyzed; strip the stenographer copies from that pile and what remained was about five threads of actual work.
Five of three thousand. My annoyed guess on first read turned out to be the literal truth.
It had interviewed the stenographers instead of me.
Turning the insight inside out
I wanted to know if the process was careless or just blind, so I asked the assistant a slightly rude question: do you have access to the code that wrote this? The app ships as a single 256-megabyte file with the program text baked inside — but baked isn't hidden. It searched inside its own file and read the rules out to me.
The rule for "is this a real session" turned out to be two lines: at least two messages from the user, at least one minute long. That's the whole gate. The stenographer sends many messages over many minutes. It passes.
The best detail: there is one exclusion in the code. The tool filters out its own automatically generated sessions — it recognizes its own echo by a signature phrase in the first message. Everyone else's echoes count as work.
And the part that explained the suggestions: the analysis never looks at your setup. My rule files, my custom tools, my scheduled automations — the report sees none of it. It pattern-matches transcripts against generic advice, and it can't know that the advice is already installed. It was judging my workflow sight unseen and recommending me things I own.
Making up my own
The fix wasn't a complaint, it was an hour of writing Python. Same design, two changes.
First, who sent each message. Every message in the session logs carries a stamp nobody was using: whether a human typed it or software sent it. My rebuild keeps sessions with real typing, plus one special case — sessions I steer remotely from my phone, where I'm present but nothing is technically "typed." Stenographers and morning robots are counted, honestly, in their own column, and analyzed never.
Second, the rulebook goes in. Before writing suggestions, the report gets handed my actual config — the rules, the tools, the automations — with one instruction: suggest only what is not already covered, and if there are no genuine gaps, say so. An empty gap analysis is a valid result. Finding nothing was allowed.
The clean run over the same thirty days: 511 sessions — 208 with a human at the keyboard, 75 I steered remotely from my phone, and 228 robots doing their scheduled rounds. The new report's gap analysis came back mostly "covered, covered, covered."
Mostly. It found one real thing.
The AI who cried wolf
At work, code changes don't go straight into the shared codebase — they go through review first, via a system called Gerrit. Sending code out is called a push. A push to review is routine and safe: a colleague reads it before it lands. A direct push skips that checkpoint — that's the dangerous kind. And my rules say the AI never pushes without being explicitly told.
I even had a guard for it: a tripwire that pops a confirmation prompt whenever the AI is about to push. The clean report found the rule had been broken three times in a month anyway.
Not because the guard was off. Because it asked the same question every time. Three pushes to review — safe, routine — each raised the same prompt, and I approved each with the same reflex. The fourth push was direct. Same prompt, same reflex, approved.
A guard that cries wolf identically for sheep and wolves trains you to stop reading. The one time it mattered, I had long stopped.
Even without reading the text
So the guard got tiers. A push to review now passes in silence — no prompt at all. A direct push asks, prefixed ⚠️⚠️⚠️. A force push — the kind that rewrites history and is genuinely hard to undo — asks with 🚨🚨🚨.
Using emojis to make information scannable before you read it is something I use elsewhere too. In Postman, my production environment carries the same police lights: one glance at 🚨 and I know I'm in the system where I need to be careful. Every Postman user I've told this to has adopted it.
Steal this
If you want more insightful insights, take this: the rebuilt analyzer is a single Python script, insights-clean.py. It focuses on the sessions that matter — the ones a human actually drove — it hands over your actual config, and it's even allowed to come back with nothing. Together they would have caught every issue I ran into with the stock /insights.