Cairn
Priority-ordered failover clone, a host-aware credential helper, and redundant push across distinct-class git stores — so your canonical history survives any one store going away. The load-bearing idea is the independence class: two mirrors on one provider aren't redundancy.
The honest edge
It survives store loss, not corruption you push yourself. Push a bad commit and every mirror faithfully keeps your mistake.
Run it
./cairn.sh
smoke_test.sh (5/5)
Bash + git
The code — every file that ships
cairn.sh275 lineson GitHub →
#!/usr/bin/env bash
# cairn.sh — a self-healing, multi-store git fabric in one script.
#
# Keep the same repo on several INDEPENDENT git stores so the loss of any one
# store (a host outage, a deleted account, a revoked token, a dead disk) never
# costs you the repo. Three moves:
#
# clone Priority-ordered FAILOVER clone: try store 1, then 2, then 3;
# the first reachable+authed store wins. LOUD STOP only when ALL
# are dead. (This is the cold-boot path — how a fresh machine gets
# the repo when you don't know which store is up.)
#
# push Redundant fan-out push to every enabled store, in parallel, with
# a bounded per-store timeout. The write succeeds iff at least
# THRESHOLD distinct INDEPENDENCE CLASSES confirm it — never a raw
# store count. Two stores that share a class (two repos on the same
# provider) count ONCE, because losing that provider loses both.
# A store that fails is a "laggard": noted, never fatal, as long as
# the class threshold is met. A laggard is behind, not wrong.
#
# frontier Probe every store's main head and REPORT the MAX head among ones
# that are comparable (one is an ancestor of the other — a laggard
# just catches up). READ-ONLY: it records discovered heads under
# refs/cairn/heads/ and reports the frontier SHA — it moves nothing
# (no main, no HEAD, no laggard heal). To adopt, run yourself:
# git fetch && git merge --ff-only <frontier>. If two heads are
# INCOMPARABLE — a real fork,
# neither an ancestor of the other — that is a DIVERGENCE: it is
# recorded under refs/cairn/divergence/<utc>/ and surfaced LOUD, and
# is NEVER auto-merged. Divergence is a human decision.
#
# The whole point is expressed in the config, not the code: see `stores`
# (`stores.example` to start). Each store is tagged with a failure CLASS; the
# durability threshold is a number of DISTINCT CLASSES. Edit the config, never
# this script.
#
# Credentials never live here. Private stores authenticate through git's askpass
# (see `askpass.example.sh`): a helper that reads a token from a GITIGNORED file
# at call time and writes it ONLY to git's stdin — never to disk, a log, an
# argument, or a stored remote URL. This script contains no secret and needs to
# know none.
#
# git-only + POSIX-ish bash. No dependencies. MIT licensed.
#
# Usage:
# cairn.sh clone <dest> # failover clone into <dest>
# cairn.sh push [ref] # fan-out push (ref default: HEAD)
# cairn.sh frontier # REPORT max comparable head (read-only); loud on divergence
# cairn.sh doctor # validate the stores config
#
# Config resolution: $CAIRN_STORES (default: ./stores)
# Knobs (env): CAIRN_TIMEOUT (per-store seconds, default 10)
#
# Exit codes: 0 ok · 1 loud failure (below threshold / all stores dead) ·
# 2 frontier divergence · 3 config problem
set -euo pipefail
CAIRN_STORES="${CAIRN_STORES:-stores}"
CAIRN_TIMEOUT="${CAIRN_TIMEOUT:-10}"
CEILING="note: the Cairn survives the LOSS of a store (host, account, token, disk) \
— it does NOT survive corruption YOU push yourself; a bad commit fans out to every \
mirror. It heals AVAILABILITY, not correctness. Incomparable heads are surfaced as a \
divergence, never auto-merged."
die() { echo "cairn: $*" >&2; exit 3; }
# --- config parsing (git-only; no jq) ---------------------------------------
# A store line is 5 pipe-separated fields: name | class | priority | url | enabled
# The threshold is one line: threshold = <N> (default 2 if absent)
# Blank lines and lines beginning with '#' are ignored.
_threshold() {
awk -F= '
/^[[:space:]]*#/ {next}
tolower($1) ~ /threshold/ { v=$2; gsub(/[^0-9]/,"",v); if (v!="") { print v; exit } }
' "$1"
}
# Emit enabled stores as "name|class|priority|url", priority-ascending.
_enabled_stores() {
awk -F'|' '
/^[[:space:]]*#/ {next}
NF < 5 {next}
{
for (i=1;i<=5;i++){ gsub(/^[ \t]+|[ \t]+$/,"",$i) }
if ($5=="true") print $3"|"$1"|"$2"|"$4
}
' "$1" | sort -t'|' -k1,1n | awk -F'|' '{print $2"|"$3"|"$1"|"$4}'
}
_require_config() { [ -f "$CAIRN_STORES" ] || die "no stores config at '$CAIRN_STORES' (set CAIRN_STORES, or copy stores.example -> stores)"; }
# --- doctor: validate the config --------------------------------------------
cmd_doctor() {
_require_config
local stores thr classes ncanon problems=0
stores="$(_enabled_stores "$CAIRN_STORES")"
[ -n "$stores" ] || die "no enabled stores in '$CAIRN_STORES'"
thr="$(_threshold "$CAIRN_STORES")"; thr="${thr:-2}"
# distinct enabled classes
classes="$(printf '%s\n' "$stores" | awk -F'|' '{print $2}' | sort -u | wc -l)"
echo "cairn doctor: $(printf '%s\n' "$stores" | grep -c .) enabled store(s), $classes distinct class(es), threshold=$thr"
if [ "$classes" -lt "$thr" ]; then
echo " WARN: only $classes distinct class(es) enabled but threshold is $thr — a push cannot meet durability until you enable more independent classes." >&2
problems=1
fi
# duplicate priorities
if printf '%s\n' "$stores" | awk -F'|' '{print $3}' | sort | uniq -d | grep -q .; then
echo " WARN: duplicate priorities — failover order is ambiguous." >&2
problems=1
fi
[ "$problems" -eq 0 ] && echo " ok."
echo "$CEILING"
# GIFT-005: doctor is a validation gate, not a pretty-printer. A below-threshold
# or duplicate-priority config makes the durability contract impossible/ambiguous
# -> loud failure (exit 1, per the documented convention above), never exit 0.
[ "$problems" -eq 0 ] && return 0
return 1
}
# --- clone: priority-ordered failover ---------------------------------------
cmd_clone() {
local dest="${1:-}"
[ -n "$dest" ] || die "usage: cairn.sh clone <dest>"
_require_config
local n=0 name class prio url
while IFS='|' read -r name class prio url; do
[ -n "$name" ] || continue
n=$((n+1))
echo "cairn: trying $name ($class) ..." >&2
if timeout "$CAIRN_TIMEOUT" git clone --quiet "$url" "$dest" 2>/dev/null; then
echo "cairn: cloned from $name ($class)." >&2
echo "$CEILING" >&2
return 0
fi
echo "cairn: $name unreachable/auth-fail — failing over." >&2
done <<EOF
$(_enabled_stores "$CAIRN_STORES")
EOF
[ "$n" -gt 0 ] || die "no enabled stores in '$CAIRN_STORES'"
echo "cairn: LOUD STOP — every store is unreachable. Nothing to clone from." >&2
exit 1
}
# --- push: distinct-class fan-out -------------------------------------------
cmd_push() {
local ref="${1:-HEAD}"
_require_config
local thr work stores name class prio url
thr="$(_threshold "$CAIRN_STORES")"; thr="${thr:-2}"
stores="$(_enabled_stores "$CAIRN_STORES")"
[ -n "$stores" ] || die "no enabled stores in '$CAIRN_STORES'"
work="$(mktemp -d)"
# fan out, parallel, each store writes its own result file
while IFS='|' read -r name class prio url; do
[ -n "$name" ] || continue
(
if timeout "$CAIRN_TIMEOUT" git push --quiet "$url" "${ref}:refs/heads/main" >/dev/null 2>&1; then
echo "OK|$class"
else
echo "FAIL|$class"
fi
) >"$work/$name" &
done <<EOF
$stores
EOF
wait
# aggregate by DISTINCT class
local confirmed="" ok="" fail="" line st nc
while IFS='|' read -r name class prio url; do
[ -n "$name" ] || continue
line="$(cat "$work/$name")"; st="${line%%|*}"
if [ "$st" = "OK" ]; then
ok="$ok $name($class)"
case " $confirmed " in *" $class "*) : ;; *) confirmed="$confirmed $class" ;; esac
else
fail="$fail $name($class)"
fi
done <<EOF
$stores
EOF
nc="$(echo $confirmed | wc -w)"
echo "cairn push: confirmed classes=$nc (need >=$thr) |$ok | laggards:${fail:- none}"
echo "$CEILING"
rm -rf "$work"
if [ "$nc" -ge "$thr" ]; then
return 0
fi
echo "cairn push LOUD-FAIL: only $nc distinct class(es) [$confirmed ] confirmed, threshold is $thr — durability NOT met for this push." >&2
return 1
}
# --- frontier: REPORT max comparable head (read-only), loud on divergence ----
cmd_frontier() {
_require_config
git rev-parse --git-dir >/dev/null 2>&1 || die "frontier must run inside a git repo (it fetches store heads into refs/cairn/heads/)"
local stores name class prio url sha names="" a b
declare -A HEAD
stores="$(_enabled_stores "$CAIRN_STORES")"
[ -n "$stores" ] || die "no enabled stores in '$CAIRN_STORES'"
while IFS='|' read -r name class prio url; do
[ -n "$name" ] || continue
sha="$(timeout "$CAIRN_TIMEOUT" git ls-remote "$url" refs/heads/main 2>/dev/null | awk '{print $1}' | head -1)"
if [ -z "$sha" ]; then
echo "cairn: $name ($class) — no main head or unreachable (skip)." >&2
continue
fi
# fetch the objects so ancestry is decidable locally; a store that advertises
# a head it cannot actually serve is corrupt-at-that-head — fail past it.
if timeout "$CAIRN_TIMEOUT" git fetch --quiet "$url" refs/heads/main 2>/dev/null \
&& git cat-file -e "$sha" 2>/dev/null; then
git update-ref "refs/cairn/heads/$name" "$sha"
HEAD[$name]="$sha"
names="$names $name"
else
echo "cairn: $name ($class) advertises $sha but cannot serve it — failing past (fixity)." >&2
fi
done <<EOF
$stores
EOF
[ -n "$names" ] || die "frontier: no servable store heads found."
# find a store whose head is a descendant-or-equal of every other head
local best="" dominates
for a in $names; do
dominates=1
for b in $names; do
[ "$a" = "$b" ] && continue
if ! git merge-base --is-ancestor "${HEAD[$b]}" "${HEAD[$a]}" 2>/dev/null; then
dominates=0; break
fi
done
if [ "$dominates" -eq 1 ]; then best="$a"; break; fi
done
if [ -n "$best" ]; then
echo "cairn frontier: frontier=${HEAD[$best]} (from $best). READ-ONLY report — nothing was moved. To adopt: git fetch && git merge --ff-only ${HEAD[$best]}"
echo "$CEILING"
return 0
fi
# no dominator => incomparable heads => divergence. Record, never merge.
local ts; ts="$(date -u +%Y%m%dT%H%M%SZ)"
for a in $names; do
git update-ref "refs/cairn/divergence/$ts/$a" "${HEAD[$a]}"
done
echo "cairn frontier LOUD: incomparable store heads — this is a FORK, not a lag." >&2
echo "cairn frontier: divergence recorded at refs/cairn/divergence/$ts/* — NOT auto-merged; resolve by hand." >&2
echo "$CEILING" >&2
return 2
}
# --- dispatch ---------------------------------------------------------------
main() {
local cmd="${1:-}"; shift || true
case "$cmd" in
clone) cmd_clone "$@" ;;
push) cmd_push "$@" ;;
frontier) cmd_frontier "$@" ;;
reconverge) die "reconverge was renamed to 'frontier': it reports the max comparable head but never moved main or touched other stores (the old name overpromised). Use 'cairn frontier'." ;;
doctor) cmd_doctor "$@" ;;
""|-h|--help|help)
sed -n '2,40p' "$0" | sed 's/^# \{0,1\}//'
;;
*) die "unknown command '$cmd' (try: clone | push | frontier | doctor)" ;;
esac
}
main "$@"
askpass.example.sh62 lineson GitHub →
#!/bin/sh
# askpass.example.sh — a host-aware git credential helper. COPY to askpass.sh
# and edit the host arms + field names for your stores.
#
# WHY THIS EXISTS
# Private stores need a token. The unsafe habits are (a) baking the token into
# the remote URL (it lands in .git/config and every `git remote -v`), and
# (b) exporting it as an environment variable (it leaks into child processes
# and is gone across separate shells, so the NEXT push silently fails auth).
# This helper avoids both: git calls it at push/fetch time, it reads the token
# from a GITIGNORED file at CALL TIME, and writes it ONLY to git's stdin.
#
# SAFE TO COMMIT — once you've replaced the placeholders with your HOSTS and
# FIELD NAMES (which are not secret). The SECRET never appears here: it lives in
# a separate file you never commit. This example is verifiably credential-free.
#
# WIRE IT (before any push/fetch):
# cp askpass.example.sh askpass.sh # then edit the host arms below
# git config core.askpass "$PWD/askpass.sh"
# git config credential.username x-access-token # if your host wants a user
#
# THE CRED FILE
# Put your tokens in a file your .gitignore excludes, e.g. `cairn.cred`
# (add `cairn.cred` and `*.cred` to .gitignore). One token per line, keyed:
# HOST_A_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxx
# HOST_B_TOKEN=yyyyyyyyyyyyyyyyyyyyyyyy
# Point the helper at it: export CAIRN_CRED=/abs/path/to/cairn.cred
#
# Git passes the prompt string as $1, e.g. "Password for 'https://host-b.example':"
# and, for username prompts, a string beginning "Username". Route by host so ONE
# helper can serve MANY stores, each with its OWN scoped token (one leak reaches
# one store — the independence the class model buys). Each token is written only
# to git's stdin, never to disk, a log, or a URL.
CRED="${CAIRN_CRED:-./cairn.cred}"
# read a keyed value from the cred file, keeping only the value (no spaces/CR)
_field() {
grep -m1 -E "^[[:space:]]*$1[[:space:]]*=" "$CRED" 2>/dev/null \
| sed -E 's/^[^=]*=[[:space:]]*//' | tr -d '[:space:]'
}
case "$1" in
*host-b.example*)
# Example: a public host that wants username=account, password=token.
case "$1" in
Username*) printf '%s\n' "your-host-b-account" ;; # public, not a secret
*) printf '%s\n' "$(_field HOST_B_TOKEN)" ;;
esac
;;
*box.example*)
# Example: a self-hosted git server (git-over-HTTPS).
case "$1" in
Username*) printf '%s\n' "your-selfhost-account" ;; # public, not a secret
*) printf '%s\n' "$(_field SELFHOST_TOKEN)" ;;
esac
;;
*)
# Default / your primary host. Adjust the field name to your store.
printf '%s\n' "$(_field HOST_A_TOKEN)"
;;
esac
stores.example34 lineson GitHub →
# cairn stores — copy this to `stores` and edit. This file ships as an EXAMPLE;
# it contains NO real hosts, accounts, or credentials.
#
# One store per line, 5 pipe-separated fields:
#
# name | class | priority | url | enabled
#
# name a short handle for the store (used in output + ref namespacing)
# class the INDEPENDENCE CLASS. This is the load-bearing field. Two stores
# in the SAME class are NOT independent — if the thing that class
# names goes down (a provider, a machine, an account), both go with
# it. Give each genuinely-separate failure domain its own class.
# Examples of distinct classes: a big public host (cloud-A), a
# different public host (cloud-B), a box you run yourself
# (self-host), an offline bundle (offline).
# priority failover order for `clone` (lower first). Keep them unique.
# url a git remote URL (https://…/repo.git, ssh://…, or a local path).
# enabled true | false. Disabled stores are ignored entirely.
#
# The durability threshold is a number of DISTINCT CLASSES (not stores):
threshold = 2
# A push succeeds iff at least this many distinct classes confirm it. Set it to
# the number of independent failures you want to survive, plus one.
# name | class | priority | url | enabled
primary | cloud-A | 1 | https://host-a.example/you/repo.git | true
mirror-public | cloud-B | 2 | https://host-b.example/you/repo.git | true
mirror-selfhost | self-host | 3 | https://box.example/you/repo.git | true
# An offline class is real durability against a total network outage, but a live
# `file://` store inside an ephemeral machine is a phantom — it vanishes with the
# machine. Realize the offline class as an operator-held `git clone --all` bundle
# regenerated on a cadence, and keep this entry disabled:
# local-bundle | offline | 4 | file:///path/to/repo.bundle.git | false
smoke_test.sh157 lineson GitHub →
#!/usr/bin/env bash
# smoke_test.sh — hermetic proof that cairn.sh works. No network: every "store"
# is a local bare git repo standing in for a remote. Five scenarios, mirroring
# the durability claims:
#
# 1 failover clone skips a dead store and clones from the next live one
# 2 a distinct-class fan-out push succeeds when >= threshold classes confirm
# 3 two SAME-class stores count ONCE — below threshold => loud fail (exit 1)
# 4 frontier REPORTS the MAX head (read-only) when heads are comparable
# 5 frontier records INCOMPARABLE heads as divergence (exit 2), moves nothing
#
# Run: bash smoke_test.sh (expect: 5/5 passed)
set -uo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CAIRN="$HERE/cairn.sh"
PASS=0; FAIL=0
ROOT="$(mktemp -d)"; trap 'rm -rf "$ROOT"' EXIT
ok() { echo "ok $1"; PASS=$((PASS+1)); }
bad() { echo "FAIL $1: $2"; FAIL=$((FAIL+1)); }
git_q() { git -c init.defaultBranch=main -c user.name=t -c user.email=t@t "$@"; }
mk_source() { # $1 dest ; makes a repo on main with one commit
git_q init -q "$1"
( cd "$1"; echo "base" > file.txt; git_q add file.txt; git_q commit -q -m base )
}
mk_bare() { git_q init -q --bare "$1"; }
seed() { git_q -C "$1" push -q "$2" main:refs/heads/main; } # $1 src $2 bare
head_of() { git_q -C "$1" rev-parse HEAD; }
bare_main(){ git_q --git-dir="$1" rev-parse refs/heads/main 2>/dev/null; }
# --- 1: failover clone ------------------------------------------------------
t1() {
local d="$ROOT/t1"; mkdir -p "$d"
mk_source "$d/src"
mk_bare "$d/live.git"; seed "$d/src" "$d/live.git"
cat > "$d/stores" <<EOF
threshold = 2
dead | cloud-A | 1 | $d/does-not-exist.git | true
live | cloud-B | 2 | $d/live.git | true
EOF
if CAIRN_STORES="$d/stores" bash "$CAIRN" clone "$d/clone" >/dev/null 2>&1 \
&& [ -f "$d/clone/file.txt" ] \
&& [ "$(cat "$d/clone/file.txt")" = "base" ]; then
ok "failover clone skips dead store, clones from next"
else
bad "failover clone" "did not clone from the live store after the dead one"
fi
}
# --- 2: distinct-class push succeeds ----------------------------------------
t2() {
local d="$ROOT/t2"; mkdir -p "$d"
mk_source "$d/src"
mk_bare "$d/a.git"; mk_bare "$d/b.git"; mk_bare "$d/c.git"
cat > "$d/stores" <<EOF
threshold = 2
a | cloud-A | 1 | $d/a.git | true
b | cloud-B | 2 | $d/b.git | true
c | self-host | 3 | $d/c.git | true
EOF
local want; want="$(head_of "$d/src")"
if ( cd "$d/src"; CAIRN_STORES="$d/stores" bash "$CAIRN" push HEAD ) >/dev/null 2>&1 \
&& [ "$(bare_main "$d/a.git")" = "$want" ] \
&& [ "$(bare_main "$d/b.git")" = "$want" ] \
&& [ "$(bare_main "$d/c.git")" = "$want" ]; then
ok "distinct-class push confirmed 3 classes >= threshold 2"
else
bad "distinct-class push" "not all stores received main, or exit nonzero"
fi
}
# --- 3: two same-class stores count once => below threshold => loud fail -----
t3() {
local d="$ROOT/t3"; mkdir -p "$d"
mk_source "$d/src"
mk_bare "$d/a1.git"; mk_bare "$d/a2.git"
cat > "$d/stores" <<EOF
threshold = 2
a1 | cloud-A | 1 | $d/a1.git | true
a2 | cloud-A | 2 | $d/a2.git | true
EOF
# both pushes SUCCEED, but they share a class => 1 distinct class < 2 => exit 1
( cd "$d/src"; CAIRN_STORES="$d/stores" bash "$CAIRN" push HEAD ) >/dev/null 2>&1
if [ "$?" -eq 1 ]; then
ok "two same-class stores count once; below threshold is a loud fail"
else
bad "same-class threshold" "expected exit 1 (durability not met), got $?"
fi
}
# --- 4: frontier REPORTS the max comparable head (read-only) ----------------
t4() {
local d="$ROOT/t4"; mkdir -p "$d"
mk_source "$d/src"
mk_bare "$d/a.git"; mk_bare "$d/b.git"
seed "$d/src" "$d/a.git"; seed "$d/src" "$d/b.git" # both at base
# A moves ahead by one commit; B stays behind (a laggard)
( cd "$d/src"; echo more >> file.txt; git_q add file.txt; git_q commit -q -m ahead )
seed "$d/src" "$d/a.git"
local ahead; ahead="$(head_of "$d/src")"
git_q clone -q "$d/b.git" "$d/recon" # recon starts behind
cat > "$d/stores" <<EOF
threshold = 2
a | cloud-A | 1 | $d/a.git | true
b | cloud-B | 2 | $d/b.git | true
EOF
# frontier is DISCOVERY-ONLY: it must REPORT the ahead head, RECORD it under
# refs/cairn/heads/, and MOVE NOTHING. Assert by inspecting refs, never by
# accepting printed prose as proof of adoption (the false-green this fixes).
local before; before="$( cd "$d/recon"; git rev-parse HEAD )"
local out rc
out="$( cd "$d/recon"; CAIRN_STORES="$d/stores" bash "$CAIRN" frontier 2>&1 )"; rc=$?
local reported; reported="$( printf '%s' "$out" | sed -n 's/.*frontier=\([0-9a-f]\{7,\}\).*/\1/p' | head -1 )"
local after; after="$( cd "$d/recon"; git rev-parse HEAD )"
local recorded; recorded="$( cd "$d/recon"; git rev-parse refs/cairn/heads/a 2>/dev/null )"
if [ "$rc" -eq 0 ] && [ "$reported" = "$ahead" ] && [ "$after" = "$before" ] && [ "$recorded" = "$ahead" ]; then
ok "frontier reports the max comparable head, records it under refs/cairn/heads/, and does NOT move local HEAD (read-only)"
else
bad "frontier discovery" "rc=$rc reported=$reported ahead=$ahead before=$before after=$after recorded=$recorded"
fi
}
# --- 5: frontier records incomparable heads as divergence (moves nothing) ---
t5() {
local d="$ROOT/t5"; mkdir -p "$d"
mk_source "$d/base"
mk_bare "$d/a.git"; mk_bare "$d/b.git"
# two SIBLING commits on top of the same base => incomparable heads
git_q clone -q "$d/base" "$d/x"
( cd "$d/x"; echo x >> file.txt; git_q add file.txt; git_q commit -q -m fork-x )
seed "$d/x" "$d/a.git"
git_q clone -q "$d/base" "$d/y"
( cd "$d/y"; echo y >> file.txt; git_q add file.txt; git_q commit -q -m fork-y )
seed "$d/y" "$d/b.git"
git_q clone -q "$d/a.git" "$d/recon" # recon has X
cat > "$d/stores" <<EOF
threshold = 2
a | cloud-A | 1 | $d/a.git | true
b | cloud-B | 2 | $d/b.git | true
EOF
( cd "$d/recon"; CAIRN_STORES="$d/stores" bash "$CAIRN" frontier ) >/dev/null 2>&1
local rc=$?
local divs
divs="$(git_q -C "$d/recon" for-each-ref --format='%(refname)' 'refs/cairn/divergence/' | wc -l)"
if [ "$rc" -eq 2 ] && [ "$divs" -ge 2 ]; then
ok "frontier records incomparable heads as a divergence, never merges (exit 2)"
else
bad "frontier divergence" "expected exit 2 with recorded divergence refs; rc=$rc divs=$divs"
fi
}
t1; t2; t3; t4; t5
echo
echo "$((PASS))/$((PASS+FAIL)) passed"
[ "$FAIL" -eq 0 ]