Alvagante logo
AI Engineer / Blog

Building Abnormalia with Swamp, Part 4: Closing the Loop

Part 3 ended with a promise: personas next. I’m breaking that promise, and I’ll tell you why.

I sat down to write about the persona system and realized it’s a paragraph, not a post. A persona is a string input. The workflow passes it along, the extension maps it to a prompt directive, the LLM writes in that register. It’s content-pipeline trivia — specific to abnormalia, not interesting as swamp. The persona post would have been the weakest of the series, and series should end on their strongest note, not their most dutiful one.

The strongest note turned out to be something the pipeline did to itself.

There’s a new ixen at abnormalia.com/ixen/generate-ixens-workflow/. Its topic is the generate-ixens workflow. Its narrator is the generate-ixens workflow. It was generated by the generate-ixens workflow. The machine read its own source code and wrote a minisite about it, complete with hero image, concept illustrations, a cheatsheet of real swamp commands, and an industrial glitch ambient soundtrack titled — I did not choose this — “The Machine That Reads Its Own”.

This post is about what that run proves, and about the general swamp topics the first three parts left uncovered: what happens on the second run, and why an AI agent can operate this thing at all.

The self-documenting run as a genericity test

Parts 1 through 3 made a claim, mostly implicitly: the pipeline is topic-agnostic. The workflow doesn’t know about git repositories or Xen hypervisors or Puppet catalogs. It knows about slugs, topics, narrators, concepts. Everything specific arrives through the input YAML.

The cleanest way to test that claim is to hand the workflow a spec whose subject is the workflow. If anything were hardcoded — a topic assumption, a special case, a template that only fits certain kinds of subjects — this is the run where it would show. The input spec contains this:

- slug: generate-ixens-workflow
  topic: The swamp generate-ixens workflow that generates ixens on abnormalia.com
  narrator: The generate-ixens workflow itself (id 10051a2c-c09c-430d-9c38-e0a65a3e354d)

The narrator field contains the workflow’s own UUID. Same DAG, same models, same CEL expressions, zero special-casing. The run went through like any other: prepare versioned the previous output, images and music and cheatsheet ran in parallel, page composed the final HTML, register wrote the entry to _data/ixens.yml.

And here’s the callback to Part 3’s observation about who names the output. The register job pulls the title from data.latest("ixen-" + slug, "page").attributes.title — the title the model generated, not the one in my spec. So the workflow, narrating itself, named itself. The title in the site index is whatever the workflow decided to be called. I find this funnier than the recursion itself.

The persona system, since I owe you at least the paragraph: the spec sets ixenPersona: gonzo and the details field describes the desired voice as “a competent professional tool that has read too much Kafka and developed opinions about its own existence.” That’s it. Persona is an input field like any other, which — proving the point of this section — is exactly why it didn’t deserve its own post.

Day-2: run it again

The first three parts covered the first run. Nobody’s pipeline problem is the first run. The problem is the fifth run, when the images are fine, the music cost real Suno credits, the prose needs regenerating, and you’d rather not pay for everything again to change one thing.

The general swamp answer: partial re-runs are declared in the graph, not scripted around it.

Each ixen in the input spec carries optional regenerate flags:

regenerate:
  images: false
  cards: false
  infographic: false
  music: true

There is no if statement anywhere honoring these. They’re consumed by CEL expressions in the workflow definition. The restore-media job decides what to copy back from the versioned directory:

restoreImages: $

And the fan-out filters decide which ixens even enter a generation job. The music job’s filter is my favorite, because it encodes economics directly:

inputs.ixens.filter(ixen,
  inputs.musicProvider == "api" &&
  ((has(ixen.regenerate) && has(ixen.regenerate.music) && ixen.regenerate.music) ||
   data.latest("tracks-" + ixen.slug, "count").attributes.trackCount < inputs.musicSkipThreshold))

Generate music if explicitly asked, or if the ixen has fewer tracks than musicSkipThreshold (default: 5). Suno costs money per generation. The threshold is a workflow input, the track count comes from the data store, and the decision is an expression evaluated at run time. Cost control isn’t a wrapper script checking things before invoking the pipeline — it’s part of the DAG’s own logic, versioned in the same YAML as everything else.

The same pattern covers idempotency for images: the filter checks size(data.latest("ixen-" + ixen.slug, "media").attributes.missingImages) > 0, so a re-run generates only what’s absent. Run the workflow twice with nothing changed and the expensive jobs filter themselves down to empty lists.

When something does go wrong, debugging is querying, not log-grepping:

swamp workflow get generate-ixens --json     # the definition swamp actually executed
swamp data list ixen-generate-ixens-workflow # what this instance has produced, versioned
swamp data get ixen-generate-ixens-workflow page --json
swamp model get ixen-generate-ixens-workflow --json

Every job’s output landed in the store with a name and a version. The state of the system is inspectable as data, because the state of the system is data. Combined with prepare versioning the output directories (ixen/<slug>/1/, 2/, …), nothing is ever silently overwritten. Five generations of the git-repository ixen exist; I can diff what the model wrote in June against what it wrote in May.

Swamp as agent-legible infrastructure

Now the part I’ve been circling since Part 1’s “slightly absurd” section. Time to say it plainly: I didn’t type most of this. The extensions, the workflow YAML, the input specs, the runs themselves — the bulk of it was Claude Code driving the swamp CLI, with me directing, reviewing, and vetoing.

That’s only workable because swamp is unusually legible to an agent, and I think this is the most generally useful observation in the whole series.

Consider what an agent needs to operate a tool safely: discover what exists, understand contracts, act, verify. Swamp gives each of those a machine-readable answer.

  • swamp help returns a JSON schema of the entire CLI — commands, options, arguments. Not prose documentation the agent might misread. A schema.
  • swamp model type describe <type> --json is the contract for any model. Part 2 made the point that workflow authors don’t need the TypeScript source, only the described interface. That’s doubly true for an agent: it reads the contract, wires the CEL, and never hallucinates a method signature because the signature is right there.
  • Every read command takes --json. The agent verifies its own work by querying the store, the same way I debug.
  • swamp workflow validate catches wiring mistakes before anything runs or spends money.

The repository’s CLAUDE.md adds the behavioral layer: search community extensions before building anything, extend existing types rather than wrapping CLIs in shell, verify resource IDs before destructive operations, prefer fan-out methods over loops. Rules an agent can follow because the tooling makes following them cheaper than not.

The pattern generalizes past swamp. Typed contracts, introspectable state, machine-readable interfaces, declarative wiring instead of imperative glue — that’s just good infrastructure design, and we’ve known it for decades. What changed is who benefits. These properties used to pay off in maintainability, for the human who returns in six months having forgotten everything. Now they pay off immediately, for the agent that arrives with no memory at all. An agent facing a pile of bash scripts and tribal knowledge flails exactly where a human newcomer would. The disciplines we preached for future-us turn out to be the disciplines that make AI operation possible at all. I spent twenty years doing infrastructure as code; I appreciate when an argument comes home.

So the recursion in this series is one level deeper than “the workflow generated a page about the workflow.” An AI helped build the pipeline, an AI operates the pipeline, the pipeline generates AI content, and an AI writes these posts about it. The human in the loop — still me, still here — designs, decides, and takes the blame.

What held up, what bit

Part 1 asked whether any of this was a good idea and called it an experiment. Closing the series, the honest ledger.

Held up:

  • The DAG as the only orchestration. I never wrote sequencing logic. Ten jobs, declared dependencies, emergent parallelism. When the workflow grew from eight jobs to ten, I added nodes and edges, and swamp recomputed the rest.
  • The data store as coupling boundary. Jobs evolved independently the whole time. build-manifest changed internally more than once; page never noticed.
  • Vault references. Three API providers, several machines, zero credential files in the repo, zero “export this before running” documentation.
  • Date-based versioning. When something broke, I looked at what changed on that date. That’s the entire debugging story for extension regressions, and it worked every time.

Bit:

  • The bundler. Part 2’s Jimp afternoon. The lesson — test the bundle, not the source — got learned the expensive way, and a variant of it bit again later with image overlays. Bundled artifacts are a different execution environment from development source. I keep relearning this and each time I’m briefly surprised.
  • Per-model lock contention. Early on I fanned out by looping separate method calls against the same model. Parallel calls contend on the per-model lock and time out. The fix is the factory pattern — one method that handles all targets, acquiring the lock once. It’s now a rule in the repo’s CLAUDE.md, which is where lessons go to stop repeating.
  • The costs are real. Not ruinous, but real. Hence musicSkipThreshold, hence count-tracks, hence the missing-images filters. A generative pipeline without economics in its control flow is a pipeline you’ll hesitate to run, and a pipeline you hesitate to run stops being automation.

Was it a good idea? The site exists, the pipeline runs, adding an ixen is editing one YAML file, and the whole thing regenerates itself on demand — including the page that documents it. The early-2000s Alessandro who hand-built abnormalia in XHTML would recognize none of the tooling and all of the impulse.

The input spec for the self-documenting ixen ends with a thesis I can’t improve on, so I’ll steal it — from myself, or from the workflow, the distinction being the whole point of this series:

Generative AI pipelines are unglamorous infrastructure. YAML files, API keys, DAG resolvers, and quota limits. The poetry emerges from the plumbing, not despite it.

The workflow wrote that sentence. I just agreed with it.

Alvabot