2026
Self-Building Software
GOTO Accelerate Chicago — what it takes to trust a machine with merge rights
Nov 2026
When Software Builds Itself: How the Engineering Gets Encoded
Beyond Conf
Upcoming
§01
The question
When an autonomous system acts, what authority is it acting under, who granted it, and how is it withdrawn? I treat autonomy as delegated authority rather than as a capability level: what an agent may do, what evidence it has to produce, what it is forbidden to do, and what the system does on a night when nobody is watching.
§02
Why it matters now
Agents now open pull requests, change infrastructure and merge. Any capable actor routes around a constraint with no enforcement behind it, politely, at speed, and with a plausible justification attached, and that is as true of people under deadline as it is of machines. Human-in-the-loop as a posture does not fix it. Written in advance, as a rule about which conditions need a person, it is governance. Written after the fact, it is a reflex.
§03
The landscape
The risk is widely named. The OWASP Top 10 for LLM Applications lists excessive agency, a model holding more permission or autonomy than its task needs, among its top risks. The NIST AI Risk Management Framework organises AI risk work into govern, map, measure and manage, and the EU AI Act requires human oversight of high-risk systems. On the engineering side, Anthropic’s “Building effective agents” argues for the simplest design that works, with guardrails and checkpoints around the agent. Most of this says what must be true. Much less of it says which mechanism makes it true, and what happens when that mechanism says no.
§04
What I’m testing
My current hypothesis is that one rule carries most of the weight: a system cannot enlarge its own authority. Inside the authority it was given, the machinery can propose and land changes, even amendments to its own standards. Creating authority stays outside it. The questions I am working: what is the smallest set of decisions that has to stay with a person? When does an operator stepping in help, and when does it quietly teach everyone the system cannot be trusted with that class of decision? And can “who did this, under what authority” be answered mechanically for every action, rather than reconstructed afterwards?
Related