CI

Run the tools and services from bothy.yaml in GitHub Actions and other CI systems.

The bothy.yaml that you use on your machine also works in CI. The CI job gets the same tool versions and the same services, from the same file. You do not write a second description of your stack in the workflow file.

Why use bothy in CI

  • One file describes the stack. If you add redis to bothy.yaml, CI gets redis in the next run. You do not edit a services: block in the workflow.
  • CI uses the same versions as your machine. bothy resolves each package from the nixpkgs revision that bothy pins, or the one that bothy.yaml names. The Nix settings of a machine do not change the result.
  • The services are plain processes. They run on macOS runners too, where GitHub Actions has no service containers.
  • Jobs on one self-hosted runner do not collide. Each checkout gets its own ports and its own data, as each worktree does on your machine.

How CI is different

On your machine, the shell hook starts the services when you cd into the worktree. A CI job has no interactive shell and no terminal, so you use three commands instead:

  1. Install the bothy binary in ~/.local/bin:

    curl -fsSL https://usebothy.dev/install.sh | bash

  2. Install Nix without a prompt. If Nix is already there, this does nothing:

    bothy install nix --yes

  3. Start the services, then run your command with the environment:

    bothy exec -- npm test

You do not need the shell hook in CI. Do not run bothy install without nix, because that also edits the rc file of the shell.

--yes is necessary. Without a terminal, bothy cannot ask before it installs Nix, so it stops with this error:

no terminal to confirm on. `bothy install nix --yes` installs without one.

The Nix installer uses sudo. GitHub-hosted runners have sudo without a password.

What bothy exec does

bothy exec -- npm test does these steps in order:

  1. It starts the bothy daemon, if it does not run.
  2. It resolves the tools and the service packages. On a new runner, Nix downloads them. bothy exec waits for this.
  3. It starts each service that has autostart set, in depends_on order. It runs each setup first, and waits until each service passes its health check.
  4. It runs npm test with the tools on PATH and the bothy variables set, such as DATABASE_URL.
  5. It exits with the exit code of npm test.

If a service fails to start, bothy exec prints the error and exits with a non-zero code. Your command does not run. If bothy cannot find your command, the exit code is 127.

Your command also gets the environment of the CI step, such as CI and GITHUB_SHA. The services do not. bothy clears the environment of each service. See What a service inherits.

When your command exits, bothy waits 60 seconds and then stops the services. The data directories stay until the runner goes away.

GitHub Actions

This workflow runs the tests of a Node project against postgres and redis. The bothy.yaml is the one from Getting started:

# .github/workflows/test.yml
name: test

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    env:
      # bothy downloads nixpkgs from GitHub. A token avoids the limit on anonymous requests.
      NIX_CONFIG: access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
    steps:
      - uses: actions/checkout@v5

      - name: install bothy
        run: |
          curl -fsSL https://usebothy.dev/install.sh | bash
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"

      - name: install nix
        run: bothy install nix --yes

      - name: test
        run: bothy exec -- npm test

The first run on a new runner takes longer, because Nix downloads every package. The job then spends most of its time in the test step.

Set NIX_CONFIG for the job, not for one step. bothy passes it to every nix command that it runs, and bothy does not edit nix.conf.

Run more than one command

Each run: step in GitHub Actions is a new shell. Each bothy exec is a new session in the worktree. Thus there are two ways to run more than one command.

To keep the commands together, run a shell inside one bothy exec:

- name: migrate and test
  run: |
    bothy exec -- sh -c '
      npm ci
      npm run migrate
      npm test
    '

To keep each command in its own step, put bothy exec in each step:

- run: bothy exec -- npm ci
- run: bothy exec -- npm run migrate
- run: bothy exec -- npm test

A second bothy exec in the same job gets the same ports and the same data. If the services still run, it uses them. If they stopped, it starts them again, and postgres keeps the tables from the earlier step. Each start runs setup again.

Use the tools in later steps

Sometimes a step needs only the tools, and no services. For example, a lint step needs node, but not postgres. Add the tool directories to $GITHUB_PATH one time:

- name: put the bothy tools on PATH
  run: |
    dirs="$(bothy exec -- sh -c 'printf "%s" "${BOTHY_TOOL_DIRS:-}"')"
    if [ -z "$dirs" ]; then
      echo "bothy exec resolved no tool directories" >&2
      exit 1
    fi
    printf '%s\n' "$dirs" | tr ':' '\n' >> "$GITHUB_PATH"

- run: node --version   # the node from bothy.yaml
- run: npm run lint

BOTHY_TOOL_DIRS holds the directories that bothy put on PATH. They are in the Nix store, so they stay valid after the session ends. bothy builds itself in CI this way.

Linux and macOS

bothy has releases for ubuntu-latest and macos-latest. One matrix runs the same bothy.yaml on both:

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    timeout-minutes: 20
    env:
      NIX_CONFIG: access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
    steps:
      - uses: actions/checkout@v5
      - run: |
          curl -fsSL https://usebothy.dev/install.sh | bash
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
      - run: bothy install nix --yes
      - run: bothy exec -- npm test

A GitHub service container runs on Linux runners only. A bothy service runs on both.

A Python project

This is a Django project with a Celery worker. The bothy.yaml declares python, uv, postgres and redis:

# bothy.yaml
tools:
  - python312
  - uv

services:
  db:
    type: postgres
    url_env: DATABASE_URL
  cache:
    type: redis
    url_env: CELERY_BROKER_URL

The test step is one line:

- name: test
  run: bothy exec -- uv run pytest

uv run installs the packages from uv.lock before it runs pytest. You do not need a separate setup step for python.

Services that do not start on entry

A service with autostart: false stays down in bothy exec too. Use --with to start it for one command:

services:
  db:
    type: postgres
    url_env: DATABASE_URL
  web:
    type: command
    command: "npm run start -- --port $PORT"
    setup: "npm ci && npm run build"
    port_env: PORT
    url: "http://127.0.0.1:{port}"
    url_env: APP_URL
    depends_on: [db]
    autostart: false
- name: unit tests
  run: bothy exec -- npm test

- name: end-to-end tests
  run: bothy exec --with web -- npx playwright test

The unit tests get postgres and no web server. The end-to-end tests get both, and Playwright finds the server at $APP_URL. Name --with one time for each service.

Secrets in CI

A CI checkout has no .env, because git ignores it. That is not an error. bothy starts the services with no secrets.

Your command gets the variables of the CI step. Thus a test that reads STRIPE_KEY gets it from the step:

- name: test
  env:
    STRIPE_KEY: ${{ secrets.STRIPE_KEY }}
  run: bothy exec -- npm test

A service does not get the variables of the step. If a service needs a secret, write a .env before bothy exec:

- name: write the secret source
  env:
    STRIPE_KEY: ${{ secrets.STRIPE_KEY }}
  run: printf 'STRIPE_KEY=%s\n' "$STRIPE_KEY" > .env

- name: test
  run: bothy exec -- npm test

If a service lists a secret in secrets: and the file does not set it, bothy prints a warning and starts the service. See Secrets.

Show the logs when a step fails

bothy writes the output of each service to a log file. The files stay after bothy exec exits. Add a step that runs only on failure:

- name: service logs
  if: failure()
  run: |
    bothy logs db -n 100
    bothy logs web -n 100

To keep the complete files, upload the log directory:

- name: upload service logs
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: bothy-logs
    path: ~/.local/state/bothy/w/*/.logs/

Check bothy.yaml in CI

bothy explain reads bothy.yaml and reports every error in it. It needs no Nix and no daemon, so it takes less than a second. Run it in a fast job, to find a broken file before a long job starts:

check-bothy-yaml:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v5
    - run: curl -fsSL https://usebothy.dev/install.sh | bash
    - run: ~/.local/bin/bothy explain

bothy explain --resolve also makes sure that every package resolves in nixpkgs. It needs Nix, so run bothy install nix --yes before it.

Other CI systems

bothy does not depend on GitHub Actions. The same three commands work in a shell script that any CI system can run:

#!/bin/sh
set -eu

curl -fsSL https://usebothy.dev/install.sh | bash
export PATH="$HOME/.local/bin:$PATH"

bothy install nix --yes
bothy exec -- npm test

The runner must have these things:

  • Linux on x86_64, or macOS on Apple silicon.
  • curl, and sudo that does not ask for a password.
  • Network access to the bothy releases, to the Nix installer, and to the Nix binary cache.

bothy itself runs in CI on the GitHub-hosted ubuntu-latest and macos-latest runners. Other runners are not tested. bothy is not tested inside a container job.

Cache between runs

bothy does not cache anything between CI runs. On a GitHub-hosted runner, every run is a new machine. Without a cache, every run downloads the same packages again. You add the cache with the tools of your CI system.

A run spends its time on three kinds of downloads:

what where it goes cache it with
The tools and service packages the Nix store, /nix nix-community/cache-nix-action
The nixpkgs source that bothy resolves against ~/.cache/nix the same action, as an extra path
The dependencies of your application for example ~/.npm or the uv cache actions/cache

Do not cache ~/.local/state/bothy. It holds the state of the daemon: ports, sessions and the data directories of the services. A copy from another runner does not match the new runner, and a CI run must start with empty databases.

Cache the Nix store

nix-community/cache-nix-action saves /nix at the end of the job and restores it at the start of the next one. It needs Nix on the runner first, so put it after bothy install nix --yes. The action supports Nix from the Determinate Systems installer, which is the installer that bothy runs.

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    # The cache action needs this to delete old caches. See `purge` below.
    permissions:
      contents: read
      actions: write
    env:
      NIX_CONFIG: access-tokens = github.com=${{ secrets.GITHUB_TOKEN }}
    steps:
      - uses: actions/checkout@v5

      - name: install bothy
        run: |
          curl -fsSL https://usebothy.dev/install.sh | bash
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"

      - name: install nix
        run: bothy install nix --yes

      - name: read the bothy version
        id: bothy
        run: echo "version=$(bothy --version | tr ' ' '-')" >> "$GITHUB_OUTPUT"

      - name: restore the nix store
        uses: nix-community/cache-nix-action@v7
        with:
          primary-key: bothy-${{ runner.os }}-${{ steps.bothy.outputs.version }}-${{ hashFiles('bothy.yaml') }}
          restore-prefixes-first-match: bothy-${{ runner.os }}-
          paths: ~/.cache/nix
          gc-max-store-size: 4G
          purge: true
          purge-prefixes: bothy-${{ runner.os }}-
          purge-created: 0
          purge-primary-key: never

      - name: test
        run: bothy exec -- npm test

The key holds three parts:

  • runner.os, because a Linux store does not work on macOS.
  • The bothy version, because each bothy release can pin a new nixpkgs, and a new nixpkgs gives new packages.
  • A hash of bothy.yaml, because the file decides which packages you need.

If the key is new, the action restores the most recent cache that starts with bothy-<os>-. Most of the packages are then already in the store, and Nix downloads only the packages that changed. At the end of the job, the action saves the store under the new key, and purge deletes the old caches of this branch.

gc-max-store-size runs nix store gc before the save, until the store is at most that size. bothy keeps a GC root for every package that a worktree uses, so the collection keeps the packages of this bothy.yaml. Set the size above the size of your store, or the cache stays incomplete.

Unfree packages

The public Nix binary cache, cache.nixos.org, does not hold packages with an unfree license. If tools names such a package, for example terraform, Nix builds it on the runner in every run. A cache of the Nix store saves the most time for such a project.

Cache the dependencies of your application

The setup-node and setup-python actions have a cache option, but with bothy you do not use those actions, because the tools come from bothy.yaml. Use actions/cache directly, and key it on your lockfile.

For npm:

- name: restore the npm cache
  uses: actions/cache@v6
  with:
    path: ~/.npm
    key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
    restore-keys: npm-${{ runner.os }}-

- name: test
  run: bothy exec -- sh -c 'npm ci && npm test'

For pnpm, ask pnpm where its store is, because the path is different on Linux and macOS:

- name: find the pnpm store
  id: pnpm
  run: echo "store=$(bothy exec -- pnpm store path)" >> "$GITHUB_OUTPUT"

- name: restore the pnpm store
  uses: actions/cache@v6
  with:
    path: ${{ steps.pnpm.outputs.store }}
    key: pnpm-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}
    restore-keys: pnpm-${{ runner.os }}-

For uv, do the same with bothy exec -- uv cache dir, and key the cache on uv.lock.

Put each restore step before the step that installs the dependencies. If setup in bothy.yaml installs them, put the restore step before the first bothy exec.

Other ways to cache

  • The Magic Nix Cache action from Determinate Systems stores Nix packages in the GitHub Actions cache, and works as a binary cache for Nix. Determinate Systems stopped it in February 2025 and brought it back in June 2025. Read its current status before you use it.
  • A binary cache service, such as Cachix or FlakeHub Cache, keeps packages outside GitHub. It helps most when you build packages on the runner, for example unfree ones. These services need an account and a token.

bothy is not tested with any of these cache tools. Measure a run with the cache and a run without it. A cache restore also takes time, and for a small store it can be slower than the download.

Self-hosted runners

A self-hosted runner keeps /nix and ~/.cache/nix between runs. You do not need a Nix cache there. bothy reuses the packages, and a new checkout with the same bothy.yaml starts with no download.