Skip to content
tokenworm ★ GitHub

Home / How it works

One core, a C ABI in the middle

tokenworm is a single Zig agent core with a C ABI as its load-bearing decision. The CLI and every language SDK are thin bindings over that boundary, so the agent loop, providers, tools, and sandbox are shared — not re-implemented per language.

The architecture

  CLI (src/cli/)   Python SDK   TypeScript SDK   Go SDK
        │              │             │            │
        └──────────────┴──────┬──────┴────────────┘
                              ▼
             ┌──────────────────────────────┐
             │  C ABI  (src/lib/ffi.zig)     │  init · run · cancel · session · free
             └──────────────┬───────────────┘
                            ▼
             ┌──────────────────────────────┐
             │  AgentRunner (agent_runner.zig)│ provider → tools → conversation → stats
             └──────────────┬───────────────┘
              ┌─────────────┼──────────────┐
              ▼             ▼              ▼
      ┌────────────┐ ┌────────────┐ ┌──────────────┐
      │ Providers  │ │  Tools     │ │  Sandbox     │
      │ zen/anthro │ │ read/write │ │ bwrap (linux)│
      │ openai/... │ │ bash/grep  │ │ seatbelt(mac)│
      └────────────┘ └────────────┘ └──────────────┘
                            ▼
                   .tworm binary session
             (history · metadata · O(1) offsets)

The flow, step by step

Whatever surface you drive it from, the same core runs the task. The C ABI is the boundary; everything below it is native Zig.

  1. 1

    A surface calls the C ABI

    The CLI, or one of the Python, TypeScript, and Go SDKs, calls into src/lib/ffi.zig — tokenworm_init, tokenworm_run, tokenworm_cancel, plus the session functions. Every surface is a thin binding over this same boundary.

  2. 2

    The AgentRunner runs the loop

    src/lib/agent_runner.zig orchestrates the canonical agent while-loop: call the provider, dispatch tools, update the conversation, account for stats. This is where a task actually runs.

  3. 3

    The provider is chosen at runtime

    Providers use vtable-based polymorphism over five LLMs — OpenCode Zen (default), Anthropic, OpenAI, Ollama, MiniMax — switchable with --provider on the CLI or switch_provider() in an SDK.

  4. 4

    Tools run, sandboxed

    The six built-in tools (read, write, edit, bash, grep, glob) are workspace-scoped. The bash tool runs through the sandbox layer: bubblewrap on Linux, sandbox-exec on macOS.

  5. 5

    Sessions persist as .tworm

    Conversation history, provider and model metadata, timestamps, token counts, and workspace snapshots are written to a .tworm binary file with an O(1) offset table — portable across every surface.

Read the design rationale

Why a systems language, why a C ABI, and what is in and out of scope.

About tokenworm Features Glossary