Writing a pipeline

A repository’s pipeline is one file, .squidci/pipelines.yaml, read at each commit. This page builds one up; the reference lists every setting.

# yaml-language-server: $schema=https://squidci.com/schema/pipelines.json

Start the file with that line and editors check it as you type.

Components and sections

The file is a map of components: parts of the repository built on their own. Each has its steps under one or more sections, and each section is one job.

web:
  base: ./web            # where its steps run; the repository's root when left out
  check:
    - pnpm: lint
  test:
    - pnpm: test
  build:
    - pnpm: build

The sections are check, test, build, publish, run and deploy. They take the same steps, and differ in when they run:

section	runs	on pull requests
check	at once	yes
test	at once	yes
build	at once	yes
publish	once its component’s check, test and build have passed	no
run	at once — for anything that fits nowhere else	no
deploy	per environment, once every check, test and build in the run passed	no

Component names are letters, digits, ., _ and -, in segments joined by /: web, services/api, runner/arm64.

Steps

A step is a command, or a map with one key naming a typed step.

api:
  build:
    - make build                       # a command, run by bash
    - pnpm: run build                  # pnpm, with Node and the lockfile set up
    - script: scripts/package.sh       # a file of the repository, run by its extension
    - docker:                          # build the Dockerfile in base, and push it
        image: 123456789012.dkr.ecr.eu-west-2.amazonaws.com/api
        tag: latest

• Commands run in order in one bash shell, started in the component’s base with set -eo pipefail: the job fails at the first command that fails, and a cd or export carries on to the steps after it. A multi-line command (|) is one step.
• pnpm: <args> runs pnpm <args> in a job SquidCI has set up for it: Node as .nvmrc or .node-version pins it, then pnpm install --frozen-lockfile, once, before the first pnpm step.
• script: <path> runs a file, relative to base: .ts, .mts, .js and the like with Node (TypeScript’s types stripped), .sh with bash, anything else by its shebang.
• docker: {image, tag, push} builds the Dockerfile in base as image:tag and pushes it, logging in first when the registry is ECR. push: false builds it to prove it builds.

Order: needs and after

Jobs run at once unless told otherwise. needs makes a component wait for every job of others — and brings what they made:

lib:
  build:
    - pnpm: build
  artifacts:
    - dist

app:
  needs: [lib]
  build:
    - pnpm: build       # .squidci/artifacts/lib/dist is here

after waits the same way, but brings nothing: an order, and only that. Use it when one component must go first but uses nothing of the other — an infra component deploying before the api on top of it.

Passing things between jobs

• Artifacts are paths a component’s build leaves, kept for the rest of the run: restored before its own publish, run and deploy at the same paths, and before each job of a component that needs it under .squidci/artifacts/<component>/<path>.
• Values are short strings. A step appends name=value lines to the file $SQUIDCI_OUTPUT names; a job that needs this one gets each as $SQUIDCI_<COMPONENT>_<NAME>, and all of them as JSON in the file $SQUIDCI_INPUTS names. Values are shown with the job, so never secrets.

image:
  publish:
    - docker: {image: public.ecr.aws/example/web, tag: latest}   # exports digest

notify:
  needs: [image]
  run:
    - echo "pushed $SQUIDCI_IMAGE_DIGEST"

Services

Containers that run beside each of a component’s jobs, on the job’s network, reached by name:

api:
  services:
    postgres:
      image: postgres:16
      env:
        POSTGRES_PASSWORD: squid
      healthcheck: pg_isready -U postgres     # the job starts once this passes
  test:
    - pnpm: test     # DATABASE_URL=postgres://postgres:squid@postgres:5432/postgres

The runner

runner says what a component’s jobs run in and on:

api:
  runner:
    image: ubuntu:24.04     # SquidCI's runner image when left out
    arch: arm64             # amd64 when left out
    vcpus: 2                # what of an instance each job reserves; 1 when left out
    spot: false             # keep off spot instances; they prefer spot otherwise

SquidCI’s runner image is Ubuntu 24.04 with git, Docker, the AWS CLI, Node 22, 24 and 26 through nvm, pnpm and the AWS CDK. Jobs run as root in it, mapped to an unprivileged user on the worker, and may run Docker.

runner.lambda: nodejs26.x runs each step as code in a Lambda function instead — quick, for small steps that call AWS or an API:

notify:
  runner:
    lambda: nodejs26.x
  run:
    - script: scripts/notify.ts

Caches

Each component gets a build cache, restored before its jobs and saved after them when it changed: the caches of npm, pnpm, Yarn, pip, uv, Poetry, Gradle, Maven and Go are kept there without any setting. cache: false turns it off for a component.

Unchanged components

A component whose files — its base, its paths, the .squidci folder and what it needs — have not changed since it last succeeded is not run again: its jobs end unchanged, which counts as passing. paths names more folders a component uses from elsewhere in the repository, so a change there runs it too. Deploys always run.

Environment variables

Every job has:

variable	value
CI	true
SQUIDCI_REPOSITORY	owner/name
SQUIDCI_COMMIT	the commit’s full SHA
SQUIDCI_BRANCH	the branch
SQUIDCI_REF	the ref, refs/heads/<branch>
SQUIDCI_ENVIRONMENT	for a deploy, its environment’s name
SQUIDCI_OUTPUT	the file to append name=value lines to
SQUIDCI_INPUTS	with needs, the file holding what they exported
AWS_PROFILE, AWS_REGION	with aws: see AWS access

Next

• Environments and deploys
• AWS access for jobs
• Every setting