tangled
View as single-page

Spindles

Pipelines

Spindle workflows allow you to write CI/CD pipelines in a simple format. They’re located in the .tangled/workflows directory at the root of your repository, and are defined using YAML.

A workflow has a set of common fields that apply no matter which engine you pick:

  • Trigger: A required field that defines when a workflow should be triggered.
  • Engine: A required field that defines which engine a workflow should run on.
  • Clone options: An optional field that defines how the repository should be cloned.
  • Environment: An optional field that allows you to define environment variables.
  • Steps: An optional field that allows you to define what steps should run in the workflow.

On top of these, each engine has its own options for things like dependencies and images. See Engines for the per-engine fields.

Trigger

The first thing to add to a workflow is the trigger, which defines when a workflow runs. This is defined using a when field, which takes in a list of conditions. Each condition has the following fields:

  • event: This is a required field that defines when your workflow should run. It’s a list that can take one or more of the following values:
    • push: The workflow should run every time a commit is pushed to the repository.
    • pull_request: The workflow should run every time a pull request is made or updated.
    • manual: The workflow can be triggered manually.
  • branch: Defines which branches the workflow should run for. If used with the push event, commits to the branch(es) listed here will trigger the workflow. If used with the pull_request event, updates to pull requests targeting the branch(es) listed here will trigger the workflow. This field has no effect with the manual event. Supports glob patterns using * and ** (e.g., main, develop, release-*). Either branch or tag (or both) must be specified for push events.
  • tag: Defines which tags the workflow should run for. Only used with the push event - when tags matching the pattern(s) listed here are pushed, the workflow will trigger. This field has no effect with pull_request or manual events. Supports glob patterns using * and ** (e.g., v*, v1.*, release-**). Either branch or tag (or both) must be specified for push events.

For example, if you’d like to define a workflow that runs when commits are pushed to the main and develop branches, or when pull requests that target the main branch are updated, or manually, you can do so with:

when:
  - event: ["push", "manual"]
    branch: ["main", "develop"]
  - event: ["pull_request"]
    branch: ["main"]

You can also trigger workflows on tag pushes. For instance, to run a deployment workflow when tags matching v* are pushed:

when:
  - event: ["push"]
    tag: ["v*"]

You can even combine branch and tag patterns in a single constraint (the workflow triggers if either matches):

when:
  - event: ["push"]
    branch: ["main", "release-*"]
    tag: ["v*", "stable"]

Engine

Next is the engine on which the workflow should run, defined using the required engine field. The currently supported engines are:

  • nixery: This uses an instance of Nixery to run steps, which allows you to add dependencies from Nixpkgs (https://github.com/NixOS/nixpkgs). You can search for packages on https://search.nixos.org, and there’s a pretty good chance the package(s) you’re looking for will be there. See Nixery engine.
  • microvm: Runs the whole workflow inside its own microVM. Has configuration features for NixOS images that will let you enable services, do Docker-in-VM, etc. See microVM engine.

Example:

engine: "nixery"

Each engine also adds its own workflow fields (dependencies, images, services, and so on). These are documented under Engines.

Clone options

When a workflow starts, the first step is to clone the repository. You can customize this behavior using the optional clone field. It has the following fields:

  • skip: Setting this to true will skip cloning the repository. This can be useful if your workflow is doing something that doesn’t require anything from the repository itself. This is false by default.
  • depth: This sets the number of commits, or the “clone depth”, to fetch from the repository. For example, if you set this to 2, the last 2 commits will be fetched. By default, the depth is set to 1, meaning only the most recent commit will be fetched, which is the commit that triggered the workflow.
  • submodules: If you use Git submodules (https://git-scm.com/book/en/v2/Git-Tools-Submodules) in your repository, setting this field to true will recursively fetch all submodules. This is false by default.

The default settings are:

clone:
  skip: false
  depth: 1
  submodules: false

Environment

The environment field allows you define environment variables that will be available throughout the entire workflow. Do not put secrets here, these environment variables are visible to anyone viewing the repository. You can add secrets for pipelines in your repository’s settings.

Example:

environment:
  GOOS: "linux"
  GOARCH: "arm64"
  NODE_ENV: "production"
  MY_ENV_VAR: "MY_ENV_VALUE"

By default, the following environment variables are set:

  • CI - Always set to true to indicate a CI environment
  • TANGLED_PIPELINE_ID - The AT URI of the current pipeline
  • TANGLED_PIPELINE_KIND - One of push, pull_request or manual
  • TANGLED_REPO_KNOT - The repository’s knot hostname
  • TANGLED_REPO_DID - The DID of the repository owner
  • TANGLED_REPO_NAME - The name of the repository
  • TANGLED_REPO_DEFAULT_BRANCH - The default branch of the repository
  • TANGLED_REPO_URL - The full URL to the repository

These variables are only available when the pipeline is triggered by a push:

  • TANGLED_REF - The full git reference (e.g., refs/heads/main or refs/tags/v1.0.0)
  • TANGLED_REF_NAME - The short name of the reference (e.g., main or v1.0.0)
  • TANGLED_REF_TYPE - The type of reference, either branch or tag
  • TANGLED_SHA - The commit SHA that triggered the pipeline
  • TANGLED_COMMIT_SHA - Alias for TANGLED_SHA

These variables are only available when the pipeline is triggered by a pull request:

  • TANGLED_PR_SOURCE_BRANCH - The source branch of the pull request
  • TANGLED_PR_TARGET_BRANCH - The target branch of the pull request
  • TANGLED_PR_SOURCE_SHA - The commit SHA of the source branch

Steps

The steps field allows you to define what steps should run in the workflow. It’s a list of step objects, each with the following fields:

  • name: This field allows you to give your step a name. This name is visible in your workflow runs, and is used to describe what the step is doing.
  • command: This field allows you to define a command to run in that step. The step is run in a Bash shell, and the logs from the command will be visible in the pipelines page on the Tangled website. Any dependencies you added in your engine’s section (see Engines) will be available to use here.
  • environment: Similar to the global environment config, this optional field is a key-value map that allows you to set environment variables for the step. Do not put secrets here, these environment variables are visible to anyone viewing the repository. You can add secrets for pipelines in your repository’s settings.

Example:

steps:
  - name: "Build backend"
    command: "go build"
    environment:
      GOOS: "darwin"
      GOARCH: "arm64"
  - name: "Build frontend"
    command: "npm run build"
    environment:
      NODE_ENV: "production"

Engines

The common fields above apply to every workflow. Each engine then adds its own fields on top. Pick an engine with the engine field and use the matching section below.

Nixery engine

Dependencies

When you’re running a workflow you’ll usually need additional dependencies. The dependencies field lets you define which dependencies to get, and from where. It’s a key-value map, with the key being the registry to fetch dependencies from, and the value being the list of dependencies to fetch.

The registry URL syntax can be found on the nix manual.

Say you want to fetch Node.js and Go from nixpkgs, and a package called my_pkg you’ve made from your own registry at your repository at https://tangled.org/@example.com/my_pkg. You can define those dependencies like so:

dependencies:
  # nixpkgs
  nixpkgs:
    - nodejs
    - go
  # unstable
  nixpkgs/nixpkgs-unstable:
    - bun
  # custom registry
  git+https://tangled.org/@example.com/my_pkg:
    - my_pkg

Now these dependencies are available to use in your workflow!

Complete nixery workflow

# .tangled/workflows/build.yml

when:
  - event: ["push", "manual"]
    branch: ["main", "develop"]
  - event: ["pull_request"]
    branch: ["main"]

engine: "nixery"

# using the default values
clone:
  skip: false
  depth: 1
  submodules: false

dependencies:
  # nixpkgs
  nixpkgs:
    - nodejs
    - go
  # custom registry
  git+https://tangled.org/@example.com/my_pkg:
    - my_pkg

environment:
  GOOS: "linux"
  GOARCH: "arm64"
  NODE_ENV: "production"
  MY_ENV_VAR: "MY_ENV_VALUE"

steps:
  - name: "Build backend"
    command: "go build"
    environment:
      GOOS: "darwin"
      GOARCH: "arm64"
  - name: "Build frontend"
    command: "npm run build"
    environment:
      NODE_ENV: "production"

If you want another example of a workflow, you can look at the one Tangled uses to build the project.

microVM engine

Image

A workflow picks the image to boot with the top-level image field:

engine: microvm
image: nixos

There are two flavours of images:

  • NixOS images (e.g. nixos): the whole guest is built with Nix, so you can configure it from the workflow file itself. The dependencies, services, virtualisation, registry and caches fields below are all understood here, and the guest builds and activates that configuration before any of your steps run.
  • Non-NixOS images (e.g. alpine): there’s no NixOS to configure, so the workflow-level config fields above have no effect. You still get a full machine to run steps in.

The available image names depend on what the spindle operator has installed. nixos and alpine are examples. If image is omitted, the spindle’s configured default image is used.

Dependencies

On the microVM engine, dependencies is a flat list of packages that are made available to every step. This field only applies to NixOS images; for other images you can use the package manager included in a step.

The guest builds a nix develop-style devshell from your dependencies and uses it for each step, so you can, for example, add pkg-config and openssl and have the openssl-sys crate while compiling a Rust project just work.

A bare name like go is looked up in nixpkgs. You can also point at any flake with the flakeref#attr syntax, so github:nixos/nixpkgs#hello pulls hello straight out of that flake.

dependencies:
  - go
  - github:nixos/nixpkgs#hello

Registry

The registry field remaps flake references, the same way nix registry does. This lets you pin or alias the flakes used by dependencies.

For example, pin nixpkgs to nixos-unstable so that the bare go above resolves from unstable, and alias your own flake so you can use myflake#tool in dependencies:

registry:
  nixpkgs: github:nixos/nixpkgs/nixos-unstable
  myflake: github:me/x

Caches

The caches field is a map of Nix binary cache URL to its trusted public key. These are fed into the spindle’s read proxy, so the guest can substitute prebuilt paths from them instead of building everything from scratch.

caches:
  https://nix-community.cachix.org: "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="

Services and virtualisation

The services and virtualisation fields are passed straight through to NixOS. Anything you could write under services.* or virtualisation.* in a NixOS configuration, you can write here, and it’s brought up before any of your steps run.

As a convenience, true works as shorthand for .enable = true anywhere an enable option exists (e.g. virtualisation.docker: true).

services:
  postgresql:
    enable: true
    ensureDatabases: ["spindle-workflow"]
    ensureUsers:
      - name: spindle-workflow
        ensureDBOwnership: true

virtualisation:
  docker: true

Recipes

Lint, test and build a Node project
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: nixos

dependencies:
  - pnpm

steps:
  - name: "Install dependencies"
    command: pnpm install --frozen-lockfile
  - name: "Lint and test"
    command: |
      pnpm run lint
      pnpm test
  - name: "Build"
    command: pnpm run build
Check formatting
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: alpine # slimmer image for checking the formatting

steps:
  - name: "Install go"
    command: apk add go
  - name: "Check formatting"
    command: test -z $(gofmt -l .)
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: nixos

dependencies:
  - gcc
  - cargo
  - rustc
  - clippy
  - rustfmt
  - pkg-config # exports PKG_CONFIG_PATH for the libraries below
  - openssl # the C library + headers openssl-sys links against

steps:
  - name: "Check formatting"
    command: cargo fmt --check
  - name: "Clippy"
    command: cargo clippy --all-targets -- -D warnings
  - name: "Test"
    command: cargo test --all
  - name: "Release build"
    command: cargo build --release
Run migrations and integration tests against PostgreSQL
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: nixos

environment:
  DATABASE_URL: "postgresql:///spindle-workflow?host=/run/postgresql"

dependencies:
  - gcc
  - cargo
  - rustc
  - pkg-config
  - openssl
  - sqlx-cli

services:
  postgresql:
    enable: true
    # has to be same name as the user for peer auth to work automatically
    ensureDatabases: ["spindle-workflow"]
    ensureUsers:
      - name: spindle-workflow
        ensureDBOwnership: true

steps:
  - name: "Run migrations"
    command: sqlx migrate run
  - name: "Integration tests"
    command: cargo test --all
Build and push a Docker image on tag
when:
  - event: ["push"]
    tag: ["v*"]

engine: microvm
image: nixos

virtualisation:
  docker: true

steps:
  - name: "Build and push to ghcr.io"
    command: |
      set -euo pipefail

      echo "$REGISTRY_TOKEN" | docker login ghcr.io -u "$REGISTRY_USER" --password-stdin
      image="ghcr.io/$REGISTRY_USER/myapp:$TANGLED_REF_NAME"

      docker build -t "$image" -t "ghcr.io/$REGISTRY_USER/myapp:latest" .
      docker push "$image"
      docker push "ghcr.io/$REGISTRY_USER/myapp:latest"
Deploy to Cloudflare Workers on tag
# .tangled/workflows/deploy.yml
when:
  - event: ["push"]
    tag: ["v*"]

engine: microvm
image: nixos

dependencies:
  - pnpm

steps:
  - name: "Install dependencies"
    command: pnpm install --frozen-lockfile
  - name: "Deploy worker"
    # `wrangler` picks up `CLOUDFLARE_API_TOKEN` from the env.
    # set it under **Settings → Secrets**.
    command: pnpm exec wrangler deploy
Publish a release artifact
when:
  - event: ["push"]
    tag: ["v*"] # trigger on versions

engine: microvm
image: nixos

dependencies:
  - go

steps:
  - name: "Build release binary"
    command: |
      mkdir -p dist
      CGO_ENABLED=0 go build -trimpath -ldflags "-s -w" -o dist/myapp ./cmd/myapp

  - name: "Publish artifact record"
    command: |
      set -euo pipefail
      # change this if you're not on `tngl.sh`
      PDS="https://tngl.sh"
      # also update this to your handle or did
      ATP_IDENTIFIER="user.tngl.sh"
      ARTIFACT_PATH="dist/myapp"
      ARTIFACT_NAME="myapp"

      # set `ATP_APP_PASSWORD` under **Settings → Secrets**
      session=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.server.createSession" \
        -H "Content-Type: application/json" \
        -d "{\"identifier\":\"$ATP_IDENTIFIER\",\"password\":\"$ATP_APP_PASSWORD\"}")
      jwt=$(echo "$session" | jq -r .accessJwt)
      did=$(echo "$session" | jq -r .did)

      # upload the binary as a blob
      blob=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.uploadBlob" \
        -H "Authorization: Bearer $jwt" \
        -H "Content-Type: application/octet-stream" \
        --data-binary @"$ARTIFACT_PATH")

      # note that this requires an annotated tag (`git tag -a v1.0.0 -m ...`)
      tag_hash=$(git rev-parse "$TANGLED_REF_NAME^{tag}")
      tag_bytes=$(printf '%s' "$tag_hash" | xxd -r -p | base64 | tr -d '=')

      # the sh.tangled.repo.artifact record for your artifact
      record=$(jq -n \
        --arg did "$did" \
        --arg tag "$tag_bytes" \
        --arg name "$ARTIFACT_NAME" \
        --arg repo "$TANGLED_REPO_URL" \
        --arg created "$(date -Iseconds)" \
        --argjson blob "$(echo "$blob" | jq .blob)" '{
          repo: $did,
          collection: "sh.tangled.repo.artifact",
          validate: false,
          record: {
            "$type": "sh.tangled.repo.artifact",
            tag: {"$bytes": $tag},
            name: $name,
            repo: $repo,
            artifact: $blob,
            createdAt: $created
          }
        }')

      # create the record on the PDS
      curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.createRecord" \
        -H "Authorization: Bearer $jwt" \
        -H "Content-Type: application/json" \
        -d "$record"

Self-hosting guide

Prerequisites

  • Go
  • For the nixery engine: Docker (or Podman with Docker compatibility enabled).
  • For the microVM engine: a Linux host with KVM, plus the microVM host dependencies described in Running microVM workflows.

Configuration

Spindle is configured using environment variables. The following environment variables are available:

  • SPINDLE_SERVER_LISTEN_ADDR: The address the server listens on (default: "0.0.0.0:6555").
  • SPINDLE_SERVER_DB_PATH: The path to the SQLite database file (default: "spindle.db").
  • SPINDLE_SERVER_HOSTNAME: The hostname of the server (required).
  • SPINDLE_SERVER_JETSTREAM_ENDPOINT: The endpoint of the Jetstream server (default: "wss://jetstream1.us-west.bsky.network/subscribe").
  • SPINDLE_SERVER_DEV: A boolean indicating whether the server is running in development mode (default: false).
  • SPINDLE_SERVER_OWNER: The DID of the owner (required).
  • SPINDLE_SERVER_LOG_DIR: The directory to store workflow logs (default: "/var/log/spindle").
  • SPINDLE_SERVER_DOCKER_SOCKET: Path to Docker socket to expose to invoked Spindle containers (default: "").
  • SPINDLE_PIPELINES_NIXERY: The Nixery URL (default: "nixery.tangled.sh").
  • SPINDLE_PIPELINES_WORKFLOW_TIMEOUT: The default workflow timeout (default: "5m").

For the microVM engine, the following are also available (prefix SPINDLE_MICROVM_PIPELINES_):

  • SPINDLE_MICROVM_PIPELINES_IMAGE_DIR: Directory containing microVM images (required to use the engine). See Running microVM workflows.
  • SPINDLE_MICROVM_PIPELINES_DEFAULT_IMAGE: Image used when a workflow doesn’t set image (default: "nixos-x86_64").
  • SPINDLE_MICROVM_PIPELINES_OVERLAY_DIR: Where per-workflow temporary disks are created (default: the system temp dir).
  • SPINDLE_MICROVM_PIPELINES_ENABLE_KVM: Use KVM hardware acceleration (default: true). Without KVM, guests fall back to slow software emulation.
  • SPINDLE_MICROVM_PIPELINES_WORKFLOW_TIMEOUT: Default workflow timeout (default: "5m").

Optional resource limits (a value of 0 disables that limit). The limits cap usage across all running microVM workflows:

  • SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_MEMORY_MIB
  • SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_VCPUS
  • SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_DISK_MIB

Optional cgroup enforcement:

  • SPINDLE_MICROVM_PIPELINES_ENABLE_CGROUPS: Place each workflow’s QEMU and slirp4netns in a per-workflow cgroup= (default: false).
  • SPINDLE_MICROVM_PIPELINES_CGROUP_PARENT: Parent cgroup; self resolves the spindle service’s own cgroup (default: "self").
  • SPINDLE_MICROVM_PIPELINES_CGROUP_PIDS_MAX: Max processes per workflow cgroup (default: 4096).
  • SPINDLE_MICROVM_PIPELINES_CGROUP_SWAP_MAX_MIB: Max swap per workflow cgroup (default: 0, no swap).
  • SPINDLE_MICROVM_PIPELINES_CGROUP_SUPERVISOR_MEMORY_MIN_MIB: Memory protected for spindle itself so it isn’t OOM-killed before the workflows (default: 512).

To push paths built inside microVMs back to a shared Nix cache (and read from it), configure the cache (prefix SPINDLE_NIX_CACHE_):

  • SPINDLE_NIX_CACHE_READ_URLS: Comma-separated binary cache URLs the guest reads from.
  • SPINDLE_NIX_CACHE_TRUSTED_PUBLIC_KEYS: Comma-separated trusted public keys for those caches.
  • SPINDLE_NIX_CACHE_UPLOAD_URL: Cache URL that paths built in the guest are uploaded to.

Running spindle

  1. Set the environment variables. For example:

    export SPINDLE_SERVER_HOSTNAME="your-hostname"
export SPINDLE_SERVER_OWNER="your-did"

  2. Build the Spindle binary.

    cd core
go mod download
go build -o cmd/spindle/spindle cmd/spindle/main.go

  3. Create the log directory.

    sudo mkdir -p /var/log/spindle
sudo chown $USER:$USER -R /var/log/spindle

  4. Run the Spindle binary.

    ./cmd/spindle/spindle

Spindle will now start, connect to the Jetstream server, and begin processing pipelines.

Running microVM workflows

The microVM engine needs a few extra things on the host, and it needs images to boot.

Host dependencies

microVM workflows depend on a handful of host tools and devices. spindle checks for the ones an image needs right before it launches, so a missing dependency surfaces as a clear error. You’ll need:

  • qemu: the runner. The QEMU binary for the image’s arch must be present (e.g. qemu-system-x86_64).
  • mkfs.ext4 (from e2fsprogs): to format the per-workflow writable volumes.
  • slirp4netns, ip (from iproute2), mount and unshare (from util-linux): used to sandbox guest networking.
  • /dev/kvm: for hardware acceleration (unless you disable KVM with SPINDLE_MICROVM_PIPELINES_ENABLE_KVM=false).
  • /dev/vhost-vsock: the guest agent talks to spindle over vsock.

On NixOS, the spindle module puts qemu, e2fsprogs, slirp4netns, iproute2 and util-linux on the service’s PATH for you.

Building images

Images are built with Nix. The flake exposes packages for the two stock images (use the -tarball prefixed ones for a gzipped tarball you can copy to another host):

# a NixOS image
nix build .#spindle-nixos-image
# an Alpine image
nix build .#spindle-alpine-image

Installing images

Spindle looks for images in SPINDLE_MICROVM_PIPELINES_IMAGE_DIR. An image is resolved by the name a workflow puts in its image field, matched literally against what’s on disk:

  1. a directory <name>/ containing a spec.json (next to the kernel/initrd/store-disk), or
  2. a flat <name>.json self-contained spec.

Resolution depends only on the name and what’s on disk, never on the host doing the resolving, so the same workflow resolves to the same image on every spindle. If you keep multiple arches side by side, you can name them <name>-<arch> (e.g. nixos-x86_64, alpine-aarch64); the suffix is just part of the name. To make a name like nixos work if you are hosting multiple arches, you can use symlinks.

On NixOS, you’ll most likely want to use systemd.tmpfiles.rules to set these up declaratively.

Architecture

Spindle is a small CI runner service. Here’s a high-level overview of how it operates:

  • Listens for sh.tangled.spindle.member and sh.tangled.repo records on the Jetstream.
  • When a new repo record comes through (typically when you add a spindle to a repo from the settings), spindle then resolves the underlying knot and subscribes to repo events (see: sh.tangled.pipeline).
  • The spindle engine then handles execution of the pipeline, with results and logs beamed on the spindle event stream over WebSocket

The engines

Spindle has two execution backends, picked per-workflow with the engine field:

  • nixery: executes each step in a fresh Docker container (Podman works too, if Docker compatibility is enabled so that /run/docker.sock is created), with state persisted across steps within the /tangled/workspace directory. The base image for the container is constructed on the fly using Nixery, which is/rhandy for caching layers for frequently used packages.
  • microvm: runs the whole workflow inside its own microVM, supporting different images, with extra configuration for NixOS images (e.g. services in workflow file) See the engine README for the architecture in depth.

The pipeline manifest is specified here.

Secrets with openbao

This document covers setting up spindle to use OpenBao for secrets management via OpenBao Proxy instead of the default SQLite backend.

Overview

Spindle now uses OpenBao Proxy for secrets management. The proxy handles authentication automatically using AppRole credentials, while spindle connects to the local proxy instead of directly to the OpenBao server.

This approach provides better security, automatic token renewal, and simplified application code.

Installation

Install OpenBao from Nixpkgs:

nix shell nixpkgs#openbao   # for a local server

Setup

The setup process can is documented for both local development and production.

Local development

Start OpenBao in dev mode:

bao server -dev -dev-root-token-id="root" -dev-listen-address=127.0.0.1:8201

This starts OpenBao on http://localhost:8201 with a root token.

Set up environment for bao CLI:

export BAO_ADDR=http://localhost:8200
export BAO_TOKEN=root

Production

You would typically use a systemd service with a configuration file. Refer to @tangled.org/infra for how this can be achieved using Nix.

Then, initialize the bao server:

bao operator init -key-shares=1 -key-threshold=1

This will print out an unseal key and a root key. Save them somewhere (like a password manager). Then unseal the vault to begin setting it up:

bao operator unseal <unseal_key>

All steps below remain the same across both dev and production setups.

Configure openbao server

Create the spindle KV mount:

bao secrets enable -path=spindle -version=2 kv

Set up AppRole authentication and policy:

Create a policy file spindle-policy.hcl:

# Full access to spindle KV v2 data
path "spindle/data/*" {
  capabilities = ["create", "read", "update", "delete"]
}

# Access to metadata for listing and management
path "spindle/metadata/*" {
  capabilities = ["list", "read", "delete", "update"]
}

# Allow listing at root level
path "spindle/" {
  capabilities = ["list"]
}

# Required for connection testing and health checks
path "auth/token/lookup-self" {
  capabilities = ["read"]
}

Apply the policy and create an AppRole:

bao policy write spindle-policy spindle-policy.hcl
bao auth enable approle
bao write auth/approle/role/spindle \
    token_policies="spindle-policy" \
    token_ttl=1h \
    token_max_ttl=4h \
    bind_secret_id=true \
    secret_id_ttl=0 \
    secret_id_num_uses=0

Get the credentials:

# Get role ID (static)
ROLE_ID=$(bao read -field=role_id auth/approle/role/spindle/role-id)

# Generate secret ID
SECRET_ID=$(bao write -f -field=secret_id auth/approle/role/spindle/secret-id)

echo "Role ID: $ROLE_ID"
echo "Secret ID: $SECRET_ID"

Create proxy configuration

Create the credential files:

# Create directory for OpenBao files
mkdir -p /tmp/openbao

# Save credentials
echo "$ROLE_ID" > /tmp/openbao/role-id
echo "$SECRET_ID" > /tmp/openbao/secret-id
chmod 600 /tmp/openbao/role-id /tmp/openbao/secret-id

Create a proxy configuration file /tmp/openbao/proxy.hcl:

# OpenBao server connection
vault {
  address = "http://localhost:8200"
}

# Auto-Auth using AppRole
auto_auth {
  method "approle" {
    mount_path = "auth/approle"
    config = {
      role_id_file_path   = "/tmp/openbao/role-id"
      secret_id_file_path = "/tmp/openbao/secret-id"
    }
  }

  # Optional: write token to file for debugging
  sink "file" {
    config = {
      path = "/tmp/openbao/token"
      mode = 0640
    }
  }
}

# Proxy listener for spindle
listener "tcp" {
  address     = "127.0.0.1:8201"
  tls_disable = true
}

# Enable API proxy with auto-auth token
api_proxy {
  use_auto_auth_token = true
}

# Enable response caching
cache {
  use_auto_auth_token = true
}

# Logging
log_level = "info"

Start the proxy

Start OpenBao Proxy:

bao proxy -config=/tmp/openbao/proxy.hcl

The proxy will authenticate with OpenBao and start listening on 127.0.0.1:8201.

Configure spindle

Set these environment variables for spindle:

export SPINDLE_SERVER_SECRETS_PROVIDER=openbao
export SPINDLE_SERVER_SECRETS_OPENBAO_PROXY_ADDR=http://127.0.0.1:8201
export SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=spindle

On startup, spindle will now connect to the local proxy, which handles all authentication automatically.

Production setup for proxy

For production, you’ll want to run the proxy as a service:

Place your production configuration in /etc/openbao/proxy.hcl with proper TLS settings for the vault connection.

Verifying setup

Test the proxy directly:

# Check proxy health
curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/sys/health

# Test token lookup through proxy
curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/auth/token/lookup-self

Test OpenBao operations through the server:

# List all secrets
bao kv list spindle/

# Add a test secret via the spindle API, then check it exists
bao kv list spindle/repos/

# Get a specific secret
bao kv get spindle/repos/your_repo_path/SECRET_NAME

How it works

  • Spindle connects to OpenBao Proxy on localhost (typically port 8200 or 8201)
  • The proxy authenticates with OpenBao using AppRole credentials
  • All spindle requests go through the proxy, which injects authentication tokens
  • Secrets are stored at spindle/repos/{sanitized_repo_path}/{secret_key}
  • Repository paths like did:plc:alice/myrepo become did_plc_alice_myrepo
  • The proxy handles all token renewal automatically
  • Spindle no longer manages tokens or authentication directly

Troubleshooting

Connection refused: Check that the OpenBao Proxy is running and listening on the configured address.

403 errors: Verify the AppRole credentials are correct and the policy has the necessary permissions.

404 route errors: The spindle KV mount probably doesn’t exist—run the mount creation step again.

Proxy authentication failures: Check the proxy logs and verify the role-id and secret-id files are readable and contain valid credentials.

Secret not found after writing: This can indicate policy permission issues. Verify the policy includes both spindle/data/* and spindle/metadata/* paths with appropriate capabilities.

Check proxy logs:

# If running as systemd service
journalctl -u openbao-proxy -f

# If running directly, check the console output

Test AppRole authentication manually:

bao write auth/approle/login \
    role_id="$(cat /tmp/openbao/role-id)" \
    secret_id="$(cat /tmp/openbao/secret-id)"