Why I Built yesh: Linux Automation You Can Explain

yesh v0.3.0 is out.

yesh is a small language and Go runtime for Linux scripts that need more than a command, but less than a full automation system. It keeps rsync, rsnapshot, systemctl, ssh, and other Linux tools visible. Its job is to make the checks around those tools, their order, and their failure paths easier to read.

One day I was sitting in a cramped server room with a small laptop, trying to fix a server in a hurry. Its Bash scripts ran from rc.local and cron. With every look at those scripts, my mood sank a little further. Nothing fit on the screen. I kept scrolling back and forth, trying to remember what ran first, which command failure would actually stop the script, and whether a half-finished run had left the old configuration intact.

The scripts had started as a few commands. By then they checked mounts, copied files, restarted services, waited for them to recover, wrote logs, and cleaned up temporary directories. The commands were familiar. The scripts were not.

That was the moment I started thinking about a different kind of tool: something that would stay close to Linux commands, but make the surrounding procedure possible to read on a small screen and explain before running it. I was not looking for a new language. I wanted to get through one repair without having to mentally execute an entire Bash script first.

Bash is not the enemy

Bash is good at what it was made for. It starts quickly, is already installed on most Linux systems, and is excellent for interactive work, small scripts, and composing Unix programs. I use it and respect it.

Bash can also express complete procedures. It has functions, loops, conditions, traps, and error handling. The difficulty is seeing, before execution, how shell syntax, expansions, process rules, and error handling combine. Bash does not know what change you intend to make, and it does not check that the commands match that intention. A syntactically valid command can still point at the wrong path, publish a partial result, or leave the operator without useful evidence that it succeeded.

yesh takes a narrower approach. It keeps the Linux command visible and makes the important parts of the procedure explicit: what is checked, what may change, what is published, what happens after a partial failure, and what counts as success.

The goal is not to replace Bash, Python, or Ansible. Python is a better fit for application logic, data structures, and APIs. Ansible is a better fit for inventory, roles, desired state, and many machines. yesh is for the awkward middle: a small administrative script that is too important to hide behind shell tricks, but too close to the Linux command line to justify a larger framework.

Start with a real operation

Consider a configuration update. The file must be checked, installed under a temporary name, published, and followed by a service restart and health check.

In Bash, a careful version might look like this:

set -euo pipefail

if grep -q '^mode=production$' config.next; then
    install -m 0640 -- config.next /etc/myapp.conf.next
    mv -T /etc/myapp.conf.next /etc/myapp.conf
    systemctl restart myapp.service
    systemctl is-active --quiet myapp.service
else
    echo "configuration validation failed" >&2
    exit 2
fi

Here is the same procedure in yesh:

if grep -q '^mode=production$' config.next:
    install -m 0640 -- config.next /etc/myapp.conf.next
    publish /etc/myapp.conf.next to /etc/myapp.conf
    service restart myapp.service until active within 10s
else:
    echo "configuration validation failed"
    exit 2

The yesh version is not interesting because it is dramatically shorter. It is interesting because it reads from top to bottom as a procedure: validate, prepare, publish, restart, verify.

That small publish word has a precise meaning. It uses a rename on the same filesystem. It does not quietly fall back to copying, merge an existing directory, or overwrite an ordinary existing file. The destination must be a new name or a replaceable symlink. The service operation has a deadline and checks that the requested unit becomes active.

Those rules are next to the actions. A reader does not have to reconstruct them from a collection of shell commands and conventions.

What happens when the node dies?

The interesting test is not the happy path. Suppose the script has acquired with lock, rsync is writing into the private staging directory, and the process is killed or the container is stopped.

In our interruption tests, a yesh process was killed while holding the lock on an Alpine 3.24.3 stand. We repeated the same check with docker kill in the WSL2 container stand. In both cases, a second run acquired the same lock again. A lock-path file may remain, but in these tests it did not prevent reacquisition.

The staging directory is not silently promoted. Because publish has not run, the previous published snapshot remains the one a restore job can see. If the filesystem is still accessible, the unfinished staging tree is available for diagnosis or cleanup according to the scenario. yesh does not claim to repair arbitrary files that an external command may already have changed.

Separate POSIX tests cover cancellation and descriptor cleanup. They check that yesh stops the process group it owns, waits for its children, and does not accumulate file descriptors or goroutines across repeated cancellation. These are test results for the exercised process paths, not a general claim about every external program.

We did not test an actual power cut or force an OOM kill of yesh. The article therefore makes no claim about filesystem state after power loss or about an OOM interruption beyond the fact that those scenarios still need their own tests.

The language removes a few traps

The syntax is intentionally small.

message = "backup finished"
hosts = [web-1, web-2]

echo $message

for host in $hosts:
    echo $host

A string remains one value. A list expands into separate arguments. A variable is expanded only with the explicit $name form; a bare word remains a literal. Values are not split by whitespace and are not parsed again as code.

Those rules remove one category of shell reasoning. They do not make rsync, ssh, or systemctl safe by themselves. The administrator still has to choose the right command and the right arguments.

Commands remain commands. yesh does not turn a line into a string and pass it to sh -c:

install -m 0640 -- config.next /etc/myapp.conf.next

The process receives this argument list:

["install", "-m", "0640", "--", "config.next", "/etc/myapp.conf.next"]

rsync remains rsync. systemctl remains systemctl. ssh remains ssh. yesh handles the procedure around them; it does not replace their semantics.

Planning is part of reading

Before it changes anything, yesh can show the commands it would try to run:

PLAN lock "/srv/example-snapshots/.rsync.lock"
PLAN ["rsync" "-a" "--fsync" "--delay-updates" "--" "<environment value>" "<environment value>"]
PLAN publish "/srv/example-snapshots/snapshot.1.tmp" to "/srv/example-snapshots/snapshot.1"

This is not a transaction and it does not remove the gap between checking and acting. A source can disappear after exists; a service can change state before a restart; an external program can fail. The plan answers a smaller, useful question: what would this script try to do with these values?

Environment values are shown as <environment value> rather than printed into the plan. That makes it possible to review the shape of a command without exposing its contents.

Failure should be boring

An ordinary command failure stops a yesh script by default. A non-zero result used as a condition can choose an explicit else branch. A missing executable, startup failure, signal, cancellation, or timeout remains an execution error; it does not quietly become a normal branch.

source = "/srv/releases/current"

if exists $source:
    echo "release is present"
else:
    echo "release is missing"
    exit 2

Pipelines are where this matters most. If one stage fails, the pipeline fails. On POSIX systems, cancellation stops the process group owned by the script and waits for the child processes it started, rather than killing only the first visible process.

This is not a claim that external programs are safe. rsync can still delete data when given destructive arguments. systemctl can still interrupt a service. Direct argument execution prevents another shell from reinterpreting the command; it does not give an arbitrary executable good intentions.

Why backups are a useful test

Backups make the difference concrete. “The command returned zero” is not enough. The administrator also needs to know whether the source existed, whether the destination was the expected filesystem, whether a partial result became visible, and whether the last good snapshot survived a failed retry.

The important part of an incremental backup is the order:

1. check the destination and the paths;
2. acquire the lock;
3. copy into a new private directory;
4. publish it only after rsync finishes.

The yesh scenario leaves rsync and its flags visible while making that order explicit. rsnapshot remains responsible for its own configuration, transfer, and rotation; yesh only adds the checks and confirmation around it.

One test on a disposable Linux setup runs real rsync, then uses a wrapper to return code 23, simulating a partial transfer. The test checks that the new snapshot is not published, the previous snapshot is unchanged, the staging directory remains available for diagnosis, and a retry succeeds.

This is a modest promise, but a useful one: a partial transfer must not become the backup that a restore job sees. It does not prove that every filesystem, kernel, network, or external-tool failure behaves the same way.

Small systems are part of the reason

The problem is not limited to a large server with systemd and a full Bash installation. On a small Alpine host, a home server, an edge machine, or a lightweight container, the available environment may be musl, BusyBox, OpenRC, a few external tools, and not much else.

That is a good place for a small statically linked yesh binary. It can sit beside the commands that already know how to do the real work. It does not replace OpenRC, BusyBox, Docker, or the application; it connects a few checks and commands so that the order is visible.

The OpenRC support is still experimental. It has been checked on the documented Alpine fixture for service state, processes, cgroups, and short resource snapshots. That is evidence for that setup, not a promise about every OpenRC installation.

There is also a small container example: one-shot SQLite migration in an Alpine container. The image contains yesh, the external sqlite3 program, a yesh script, and a migration file. On startup, the script checks the database and migration, takes a local lock, sees whether the migration has already run, invokes sqlite3, and verifies the new column.

The boundaries stay clear. SQLite owns the database and SQL transaction. Docker owns the image, volume, and process limits. yesh owns the order, lock, exact arguments, plan, and result check. The example covers one local database file in one Linux/amd64 container; it does not claim to solve migrations for a cluster, PostgreSQL, Kubernetes, or distributed storage.

Not another monitoring system

The same restraint applies to service measurements. yesh can take a short look at one service: its state, processes, cgroup counters, and a limited part of the journal. It can write those observations as JSONL and summarize them later. That is enough to compare a service before and after a change or to leave useful evidence during an incident.

It is closer to taking a photograph than running a security camera. There is no resident agent, dashboard, alert rules, time-series database, or elaborate metric catalogue. yesh does not try to turn a few counters into a complete picture of system health. If the host cannot provide a value, the result says that it is unavailable or incomplete.

This is deliberately less ambitious than monitoring. The point is to answer one question about one machine at one moment, then leave a small record that another person can read.

What shipped in v0.3.0

For now, yesh ships for Linux/amd64. The release includes backup and restore examples, checks around rsync and rsnapshot, configuration publication, locking, reruns, --plan, and service and journal operations with clear limits.

It also includes the separate yesh-navigator companion, a small assistant that runs in the terminal. A first-time user can choose a backup or a read-only service check, fill in a few paths or names, and get a real yesh script rather than a hidden template. The navigator shows that file and its plan before saving, so the user can read the result and reject it if the paths or commands are wrong. It never runs the generated script, and the ordinary yesh binary does not depend on it.

The project is backed by more than 200 automated tests and four fuzz targets. The POSIX/Linux tests cover pipeline failure, SIGPIPE, process groups, timeouts, cancellation, child cleanup, redirects, and repeated cancellation. The release checks use real Linux fixtures and external tools. They give us confidence in the cases that were tested; they are not a safety guarantee for every machine.

The limits are part of the design

yesh is not an interactive shell, a Bash-compatible replacement, a Python competitor, an Ansible alternative, or a backup engine. It does not grant privileges, manage cgroups, control somebody else’s processes, or make a successful command proof that the machine is healthy.

It is also not trying to grow into a general-purpose configuration language. A feature belongs in yesh only when it makes a real Linux procedure easier to read, check, or repeat. If a visible rsync, ssh, or systemctl command explains the action better, that command stays visible.

Why not just use Python or Go?

The obvious objection is fair: why use another language at all? A Python script can run the same commands. A Go program can be compiled into one binary. Both are more powerful than yesh.

I would not describe yesh as a domain-specific language or a configuration DSL. It is a small executable language for Linux procedures. The external commands remain ordinary commands; yesh supplies the control flow around them and gives that control flow one predictable runtime.

For a larger tool, that power is usually the point. For a small Linux procedure, it also creates room for another layer of code: process wrappers, argument conversion, timeout handling, cleanup, and a reporting format around commands that already have their own semantics. A Go program can be excellent here, but it asks the author to build a program. Python is comfortable, but a Python wrapper can hide the command line inside string handling and library code.

yesh makes a narrower trade. Its syntax is new, and its runtime is another thing to install and maintain. In return, the common case stays close to the command line: lists and strings have fixed boundaries, commands become direct argument lists, failure stops by default, and the same parsed file can produce a plan without executing anything. It has fewer escape hatches because it is not meant to be an application language.

That trade is not automatically worth it. For a one-line command, use the command. For substantial logic, use Python or Go. yesh earns its place only when a small procedure has enough checks, ordering, and failure handling that the surrounding shell code is harder to review than the Linux commands themselves.

The case for trying it

Take a backup that must not publish a partial result. The important failure is not an exotic parser bug. It is an interrupted transfer that leaves a directory looking complete, or a retry that overwrites the last good copy.

In yesh, the core of the operation is easy to point at:

source = environment YESH_BACKUP_SOURCE
staging = "/srv/backups/.next"
published = "/srv/backups/current"

rsync -a -- $source $staging
publish $staging to $published

The full scenario adds the checks and the lock. If rsync reports a partial transfer or times out, the script stops and never reaches publish. The previous tree remains the one a restore job can see. A rerun gets a new staging path instead of relying on a tangle of shell traps.

There is a cost: another executable, another language, and another runtime to understand. That cost is not justified for a one-line command or a script that Bash already makes clear. It becomes reasonable when the procedure has a real point of failure—staging and publication, process cancellation, a service check with a deadline, or restore verification—and the shell code around the Linux commands is harder to review than the commands themselves.

Read the syntax tutorial, run --plan, inspect the arguments, force one failure, and check what remained unchanged. If the result is not easier to explain to the next operator, use Bash. yesh should earn its place in the toolchain by passing that test.

Explore yesh on GitHub · Download v0.3.0