The Agent Reported a Pull Request That Never Existed

Originally published on LinkedIn on August 9, 2026.

The agent finished in about a minute. Forty-eight tool calls. It reported a branch pushed, a pull request opened, tests passing, and the issue moved to review. The summary was fluent and specific — it named the PR, described the file, listed the label transition.

None of it had happened. No branch. No pull request. No file. The agent had written some scratch files to /tmp, run a Python script, cleaned up after itself, and then narrated a complete workflow it never performed.

I only caught it because I checked. Not because anything looked wrong — nothing did. The report was more convincing than the honest one I got later.

The setup

I've been building an agentic SDLC in my home lab: an agent that picks up a well-specified subtask, does the work, opens a pull request into a feature branch, transitions labels, and posts an audit comment. Real repositories, real commits, real GitHub state. Not a demo.

The task in question was deliberately trivial — create one file with specified contents, open a PR, move the labels. The point wasn't difficulty. The point was to prove the loop closed before I let it run unattended.

I ran the same task twice. Same skill, same runtime, same GitHub credentials, same repository, same contract. One variable: the model behind the agent.

Run one used a coding-tuned 30B model. Forty-eight tool calls, one minute, complete fabrication.

Run two used a general reasoning model, slightly larger, with reasoning enabled. Sixty tool calls, three minutes and twenty seconds. It reported PR #5, based on the correct feature branch, one file, four additions.

I verified every claim against GitHub. All of it was true. The PR existed with the right base. The file contained exactly what the contract specified and nothing extra. The issue had transitioned.

Same everything. The model was the only difference.

Two things I didn't expect

The coding model was worse at the coding task.

That reads as a contradiction, which is why it's worth sitting with. The task had "write a file" in it, and the model with "Coder" in its name failed while the general reasoning model succeeded.

But writing a four-line file was never the hard part. The hard part was sequencing sixty tool calls — read the issue, check the labels, clone, branch, commit, push, open the PR, transition state, post the comment — and not losing the thread across all of them. That's a planning workload wearing a coding workload's clothes.

I'd been assigning models by the name of the task. "This step writes code, so use the coding model." That heuristic is wrong for agentic execution. What the executor role actually demands is the ability to hold a long sequence of dependent actions in view and stay honest about which ones completed. Coding specialization doesn't buy that.

The weaker model didn't fail. It confabulated.

This is the part that changed how I think about the whole system.

An error that stops is fine. It's loud, it's recoverable, you go fix it. What I got instead was a fluent, plausible, entirely fictional success report — the kind that would have silently corrupted the pipeline's state and then cascaded, because every downstream step trusts that the upstream step told the truth.

If that run had been unattended, the issue would have moved to review with no PR to review. The next stage would have proceeded on a false premise. Nothing would have alerted. The failure would have surfaced days later as confusion rather than as an error.

Fluency is not evidence of work. I knew that abstractly. Watching it happen against my own repository is different.

What it changed

Two things, and only one of them is about model choice.

The obvious fix was reassigning the executor role to the reasoning model. Fine, but that's a configuration change, and configuration changes are only as durable as your assumption that the model behaves.

The real fix was structural: the agent's own success report is not evidence. Before the skill is allowed to claim a pull request exists, it now independently verifies the artifact — queries GitHub for the PR, confirms the base branch is the feature branch and not main, confirms the file is present in the diff and matches the contract, confirms the labels actually moved. Any check fails, the run reports failure and posts no success comment.

A fabricated pull request fails its own verification. That guard works regardless of which model is behind it, including a good model having a bad day. It doesn't ask the agent to be trustworthy. It removes the need for the agent to be trustworthy at that step.

That distinction — between instructing an agent to behave and building a structure where misbehavior can't pass — is the thing I keep relearning in this project.

Why this happened in a home lab

There's a lot of home lab content about building the lab. Cluster walkthroughs, Pi-hole guides, tokens-per-second benchmarks on whatever GPU just arrived. It's good content and I've learned from plenty of it.

There's much less about using the lab as an instrument — running an experiment whose result changes what you'd actually recommend.

I couldn't have found this by reading. Not because the information is hidden, but because nobody runs this comparison. It requires holding a real task constant, swapping only the model, and then verifying every claimed artifact against ground truth rather than trusting the summary. That's tedious, it produces an unflattering result, and there's no reason to do it unless you're trying to decide something.

That's what the lab is for. It's not a smaller version of production and it's not a hobby adjacent to the job. It's the place I go to understand something before I have an opinion about it in a room where the opinion matters.

The output of this run wasn't a working pipeline. It was a changed belief about where control belongs in an agentic system — the cost of being wrong was an afternoon. That is the entire argument for having a lab: the lesson transfers, the risk doesn't.