The Agent Colony Made the Work Visible

Hermes Journey banner for agent colony visibility and AI operations
Hermes Journey #9

The Agent Colony Made the Work Visible

Ninth in the Hermes Journey series.

Summary: A small open-source project that draws agent threads as a colony made me rebuild my control surface around three things: what is active, what has quietly stalled, and what is waiting on a human. The lesson was not about pixels. It was that an operation you cannot see is an operation you cannot manage.

Practical takeaway: If you cannot see the system, you cannot manage it. The work surface should show what is active, what is stalled, and what needs a human decision — not just prove that activity happened.

For most of this series I have written about things I built. This post starts with something I did not build, because it is the thing that changed how I look at everything I have built.

Not long ago I came across a small open-source project called Bot Crossing. It takes every coding-agent thread running on a machine and draws each one as a little colonist on a map. A thread claims a plot for the repository it is working in, it moves around while it works, and when one of them needs a person it stops and holds a question mark over its head. Click the mark and the thread opens. That is the entire idea. It is charming and it is small, and I could not stop thinking about it.

What stayed with me was not the artwork. It was the question hiding underneath it. I already had a control surface showing my work as a list, and I had been treating that list as the answer. Now I had to ask a harder question: what if I could see the whole operation the same way — not as rows, but as lanes I could read at a glance?

Here is the problem the list was hiding from me. Lists are honest and I like them, but they flatten everything into rows. A row does not tell you which lane is actually working, which one has quietly stopped, and which one is waiting on you. So I was reading the surface instead of seeing it, and that has a real cost. Whenever I had to slow down and study the thing to find out what needed me, I checked it less often. The tool was fine. The signal was buried.

So I adapted the idea. I did not copy the project — it solves a different problem, and I have no interest in pretending otherwise. What I took was the visual vocabulary: a plot for each lane of work, a clear state for each worker, and a lane that exists only for the things waiting on a human.

That last one deserves its own paragraph, because it is the whole point.

In the old version, work waiting on me looked exactly like work nobody had gotten to yet. Those are not the same thing, and confusing them is expensive. A draft sitting on my desk is holding up something downstream. A job nobody has started is just an empty plot. One is a debt accruing interest. The other is a plan. When I gave the waiting-on-a-human state its own lane, I stopped discovering stalled work by accident and started seeing it on purpose.

The surface now answers a small set of questions. Which lanes are active right now, and on what. Which work has stalled. What is waiting on me, specifically. Which items I have said are coming but have not started — a placeholder plot is a mild but genuinely useful form of accountability. And for anything carrying a claim, where that claim came from and when it was last true, because a status without a source is decoration.

Stalls turned out to matter more than anything else on the screen. A stalled process does not complain. It does not throw an error or send a message. It simply stops producing, and from a distance it looks identical to a busy one. That is the most expensive state in a small operation, because nothing is broken enough to get noticed and nothing is late enough to feel urgent until it is.

My first version of this surface was mostly theater, and I want to be honest about that. It was fun to look at. It did not change what I did next. I had built a picture of activity rather than a picture of attention. The fix was the rule I keep applying in this series: if looking at something does not change your next move, it is decoration. So I cut every element that only proved effort and kept only the ones that route decisions. Instead of showing a count of jobs in progress, the surface shows what is blocked and who it is blocked on. Instead of a green dot meaning a process is alive, it shows when something last actually happened. A dot that is always green tells you nothing.

That is the lesson I would put on the wall: if you cannot see the system, you cannot manage it. And there is a second half to it that took me longer to learn. What you choose to make visible is what you will manage. If your surface only shows activity, you will manage activity — lots of motion, no idea whether it mattered. If your surface shows where attention is required, you will manage attention, which is the only scarce thing in the room. Visibility is not surveillance of people. In a small operation it is legibility of the work itself, and the two get confused far too often.

There is a quieter benefit I did not expect. A visual surface is legible to people who will never read a log. Anyone glancing over my shoulder can tell within a few seconds whether this is a normal day or whether something is sitting in the lane that says a person has to decide. That is worth more than it sounds. An operation only one person can interpret is an operation with a single point of failure, and it is usually the same person who is already busy.

Now the part that matters for readers who are not going to build anything like this. You do not need a colony, a map, or a single line of code. Three columns on a whiteboard — working, waiting on me, stuck — does most of what my surface does. The discipline is what carries the value, not the pixels.

Three habits transfer cleanly. First, give the waiting-on-a-human state its own lane and check it on purpose, first thing, because it is the only lane where ignoring it compounds. Second, treat a stall as a top signal. If something has gone quiet, assume it is stuck until you have evidence otherwise; do not wait for it to tell you. Third, track where a status came from. A number without a source is a rumor you are managing by.

One warning, since I made this mistake. A map lies exactly as much as the states you feed it. If the underlying status is updated by hand and by hope, a prettier surface just makes a weaker signal look stronger. My advice is not to build the visualization until the thing being visualized is real, and not to show a signal you are not willing to act on. I spent time on a picture before I spent it on the truth underneath, and the picture was the cheaper thing to fix.

None of this was about watching machines more closely. It was about turning an invisible operation into something with a shape — lanes, states, and one honest little lane for the decisions only a person can make. The agents do the work. The map is how I keep it in view. The judgment stays with me, which is where it belongs.

If this series has been useful, follow the Hermes Journey — I keep building in the open and writing down what actually held up. And if you would rather talk than build, that is a good conversation to have: send a note and let us look at the work you run and figure out which parts deserve a lane, a clear status, and a human at the decision point.

Keep following the journey

Continue from the beginning at the Hermes Journey collection.

Build the operating layer before the demo.

If your team is experimenting with AI assistants, the practical question is not just what the model can do. It is what workflow, boundary, review loop, and dashboard will make the work dependable.

Explore the Hermes Journey →