Parallel Agent Execution
How Colony enables multiple AI agents to work simultaneously with kernel-level isolation and zero container overhead
You can run as many agents as you want in parallel. Each one gets complete isolation at native performance. No containers, no virtualization overhead — just Linux kernel primitives doing what they do best.
Kernel-Level Isolation
Each agent runs in its own Linux network namespace. That’s the same isolation primitive Docker uses under the hood, but we skip the container layer entirely.
What Each Agent Gets
| Resource | Isolation Level |
|---|---|
| Network stack | Fully isolated (own IP, routing table, firewall) |
| Filesystem | Shared at native speed (no overlay, no volumes) |
| Process tree | Independent (can spawn services without conflicts) |
| Database | Dedicated SQLite instance per colony |
| VCS workspace | Separate Jujutsu workspace (conflict-free) |
Here’s the thing: agents need network isolation to run multiple web servers on the same port. They don’t need filesystem isolation. Sharing the filesystem at native speed is a feature, not a bug.
Zero Container Overhead
Colony starts in <50ms because we skip containers entirely:
# Traditional container startup
docker run → image pull → overlay mount → network setup
↓
2-5 seconds
# Colony namespace startup
ip netns add → veth pair → done
↓
`<50ms`
No Docker daemon, no image layers, no storage drivers. Just the Linux kernel doing what it does best.
OTP Actor Architecture
Mycelium (our orchestration layer) uses Erlang OTP actors to manage colonies. One actor per colony.
Supervisor
├─ Colony Actor 1 (agent-alpha)
│ ├─ Network namespace
│ ├─ Service owner process
│ └─ Log ring buffer
├─ Colony Actor 2 (agent-beta)
│ ├─ Network namespace
│ ├─ Service owner process
│ └─ Log ring buffer
└─ Colony Actor 3 (agent-gamma)
└─ ...
Each actor is fault-isolated. If one crashes, the supervisor restarts only that one. Other agents keep running.
Service Spawning
When an agent starts a web server or API, we spawn it via Erlang ports with a dedicated owner process:
- Agent requests service start via colony.toml
- Colony actor spawns a dedicated owner process
- Owner process opens Erlang port to OS command
- Owner monitors service PID and streams output
- On stop: kill(OS_PID) ensures clean shutdown
Why the owner process? The colony actor needs to stay responsive to API requests. If we did synchronous I/O inside the actor, it’d block and become unresponsive. The owner handles all the blocking work.
Practical Examples
Running 3-5 Agents Simultaneously
# Spawn three colonies with different tasks
mycelium spawn agent-frontend --repo user/app
mycelium spawn agent-backend --repo user/app
mycelium spawn agent-tests --repo user/app
# Each agent gets:
# - Own namespace (ns-agent-frontend, ns-agent-backend, ns-agent-tests)
# - Own Jujutsu workspace
# - Own subdomain (agent-frontend.colony.local, etc.)
All three can run npm start on port 3000 at the same time. No port conflicts, because they’re in separate network namespaces.
Real-World Performance
From our demo setup (seed/hello-colony/):
- Web service (port 4001): Express server with 4 routes
- API service (port 4002): JSON REST API
- Cold start: 42ms (namespace creation + veth setup)
- Memory overhead: ~2MB per namespace (kernel structures only)
- Filesystem: Native speed (no overlay mounts)
Colony’s isolation is surgical: isolate what conflicts (network), share everything else at native performance.
How Agents Don’t Interfere
Network Isolation
# Agent 1 runs web server on port 3000
curl http://agent-1-3000.colony.local/
# → Routes to namespace ns-agent-1
# Agent 2 also runs on port 3000 (no conflict!)
curl http://agent-2-3000.colony.local/
# → Routes to namespace ns-agent-2
Caddy handles the routing. It registers routes dynamically when services start.
VCS Isolation
Each colony gets its own Jujutsu workspace — a separate working copy sharing the underlying commit history:
repos/
├─ user-app/ # Shared .jj repository
└─ workspaces/
├─ agent-frontend/ # Independent working copy
├─ agent-backend/ # Independent working copy
└─ agent-tests/ # Independent working copy
Agents make changes independently. When you’re ready, review and merge using Jujutsu’s conflict-free workflow.
Comparison to Alternatives
| Approach | Cold Start | FS Performance | Memory/Agent | Port Conflicts |
|---|---|---|---|---|
| Docker | 2-5s | Degraded (overlay) | 50-200MB | Requires mapping |
| VMs | 10-30s | Degraded (virtio) | 512MB-2GB | Requires mapping |
| Colony | <50ms | Native | ~2MB | Impossible |
Technical Details
We use standard Linux tools for namespace management:
# Create namespace
ip netns add ns-colony-alpha
# Create veth pair (virtual ethernet)
ip link add veth-alpha type veth peer name veth-alpha-host
# Move one end into namespace
ip link set veth-alpha netns ns-colony-alpha
# Assign IPs and bring up interfaces
ip netns exec ns-colony-alpha ip addr add 10.200.1.2/24 dev veth-alpha
ip netns exec ns-colony-alpha ip link set veth-alpha up
All namespace operations are non-root — Colony runs entirely in userspace with proper capabilities.
Next Steps
- Live Preview — See agents work in real-time
- Colony Lifecycle — Understand spawn → stop states
- Why No Containers? — Deeper comparison with Docker