Victor Augusteo

Inside Gas Town

Fri, 01 May 2026 00:00:00 GMT

An end-to-end, intuition-first explanation of Steve Yegge's multi-agent coding orchestrator, how it has evolved into Gas City, and what its design says about the next layer of developer tools. About a 30-minute read.

Prologue: what must be made durable

1. What a coding agent actually produces

The argument starts with a small reframing. When you run Claude Code on a task, it feels like the artifact is a diff. The agent reads, the agent thinks, the agent writes. You merge the diff and you're done.

But the diff is not really what the agent produces. The diff is what a session produces, using the agent. The agent itself produces something more like a trail: a sequence of decisions, lookups, mistakes, partial fixes, half-remembered constraints, scratch ideas about how to do the next part. The diff is a snapshot of where the trail happened to be when the context window ran out.

This trail is mostly invisible to you, mostly invisible to the next session, and entirely lost when the context window fills up. The next session starts from zero. Whatever the previous session learned, it learned for the duration of one context window, and then it took it to the grave.

Most of the friction in working with a single coding agent is downstream of this. The agent rediscovers facts you already taught it. It pursues approaches you already rejected. It writes the same broken function twice. It reaches the end of its context with the work two-thirds done and no good way to hand off. None of this is the model's fault, exactly. It's the medium.

The job of an orchestrator (which is what Gas Town is) is to fix the medium. You can't preserve the cognitive trail itself — the LLM's scratch reasoning is gone the moment the context fills up, and no orchestrator gets that back. But you can preserve the result of the trail: the work that's been done, the decisions that have been made explicit, the open questions a future session can pick up. Make the work survive the session, let multiple agents pass it between them without losing what's on it. Once you accept that as the goal, almost everything else in Gas Town's design follows.

2. The naive answer, and why it explodes

Suppose you accept that one Claude Code is not enough. You want twenty.

The naive way to do this is tmux new-window twenty times. Each window runs a fresh Claude Code in a separate worktree. Each one gets its own slice of work. You walk between them like a foreman, checking on each, prompting them when they get stuck, copying their output back to a master plan when they finish.

This works for about an hour. Then it doesn't.

Three things go wrong, all at the same time. First, the agents start stepping on each other. Two of them rebase against the same branch and pick different conflict resolutions, and now your repo has a private fork. Second, you forget what each window is doing. They all started with let's go, and after fifty turns of dialog the windows look identical. You cannot remember which one was redoing the auth migration and which one was fixing the flaky test. Third, half the agents have hit context exhaustion and are sitting silently waiting for input, which they are not going to receive because you are busy talking to one of the other agents who is currently telling you a long story about why the database schema is wrong.

Yegge's framing for the resulting environment is, by his own account, the project's namesake. Gas Town is "an industrialized coding factory manned by superintelligent robot chimps, and when they feel like it, they can wreck your shit in an instant." The chimps are competing for tokens and breaking each other's work. You are armed only with a copy of tmux send-keys.

<figure> <svg viewBox="0 0 760 360" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="Four parallel agents trying to merge into main, with conflicts and waiting agents."> <g> <g transform="translate(40,30)"> <rect x="0" y="0" width="150" height="80" fill="none" stroke="#1c1a15" stroke-width="1.5" rx="2"/> <text x="10" y="20" font-size="11" font-weight="600">polecat-1</text> <text x="10" y="40" font-family="JetBrains Mono" font-size="11" fill="#3a342a">$ git rebase main</text> <text x="10" y="55" font-family="JetBrains Mono" font-size="11" fill="#b8543e">! conflict: auth.go</text> <text x="10" y="70" font-family="JetBrains Mono" font-size="11" fill="#b8543e">push --force</text> </g> <g transform="translate(220,30)"> <rect x="0" y="0" width="150" height="80" fill="none" stroke="#1c1a15" stroke-width="1.5" rx="2"/> <text x="10" y="20" font-size="11" font-weight="600">polecat-2</text> <text x="10" y="40" font-family="JetBrains Mono" font-size="11" fill="#3a342a">$ git rebase main</text> <text x="10" y="55" font-family="JetBrains Mono" font-size="11" fill="#b8543e">! conflict: auth.go</text> <text x="10" y="70" font-family="JetBrains Mono" font-size="11" fill="#b8543e">push --force</text> </g> <g transform="translate(400,30)"> <rect x="0" y="0" width="150" height="80" fill="none" stroke="#1c1a15" stroke-width="1.5" rx="2"/> <text x="10" y="20" font-size="11" font-weight="600">polecat-3</text> <text x="10" y="40" font-family="JetBrains Mono" font-size="11" fill="#8a8377">waiting for input</text> <text x="10" y="55" font-family="JetBrains Mono" font-size="11" fill="#8a8377">since 11 min ago</text> <text x="10" y="70" font-family="JetBrains Mono" font-size="11" fill="#8a8377">. . .</text> </g> <g transform="translate(580,30)"> <rect x="0" y="0" width="150" height="80" fill="none" stroke="#1c1a15" stroke-width="1.5" rx="2"/> <text x="10" y="20" font-size="11" font-weight="600">polecat-4</text> <text x="10" y="40" font-family="JetBrains Mono" font-size="11" fill="#3a342a">explaining schema</text> <text x="10" y="55" font-family="JetBrains Mono" font-size="11" fill="#3a342a">to user, going on</text> <text x="10" y="70" font-family="JetBrains Mono" font-size="11" fill="#3a342a">third paragraph</text> </g> </g> <g stroke="#b8543e" stroke-width="1.4" fill="none"> <path d="M115,110 Q150,160 380,200" stroke-dasharray="4 3"/> <path d="M295,110 Q310,160 380,200" stroke-dasharray="4 3"/> <path d="M475,110 Q470,160 380,200" stroke-dasharray="4 3"/> <path d="M655,110 Q580,160 380,200" stroke-dasharray="4 3"/> </g> <text x="200" y="135" font-style="italic" font-size="11" fill="#b8543e">collisions</text> <text x="500" y="135" font-style="italic" font-size="11" fill="#b8543e">silent timeouts</text> <g transform="translate(255,205)"> <rect x="0" y="0" width="250" height="60" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.8" rx="2"/> </g> <g stroke="#b8543e" stroke-width="2" opacity="0.5"> <line x1="290" y1="220" x2="490" y2="255"/> <line x1="490" y1="220" x2="290" y2="255"/> </g> <g transform="translate(255,205)"> <text x="125" y="22" text-anchor="middle" font-size="13" font-weight="600">main</text> <text x="125" y="42" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#b8543e">two HEADs · history forked</text> </g> <text x="380" y="305" text-anchor="middle" font-style="italic" font-size="13" fill="#b8543e">"the kid's soccer team chasing the ball"</text> <text x="380" y="330" text-anchor="middle" font-size="11" fill="#3a342a">no merge queue. no health monitor. no shared notion of what's done.</text> </svg> <figcaption><strong>Fig 1.</strong> What twenty parallel coding agents look like with no orchestrator. The work is happening; it is just not converging. The merge queue does not exist as a concept yet, so every agent thinks it owns main.</figcaption> </figure>

The naive answer fails because it has no model of what the work is. The agents have no shared notion of what's done, what's queued, what's been merged, what's currently being attempted. There's no merge queue, no health monitor, no priority. There's just twenty independent processes shouting git push --force.

You need a substrate. That's where Beads comes in.

3. Beads, or what work looks like when AI reads it

Beads is Yegge's earlier project, launched October 2025. It's a git-backed issue tracker. Originally each issue was a JSON object on its own line in a JSONL file kept in your repo, with a SQLite cache layer hydrating from that file for fast queries. As of v1.0 in April 2026, the storage layer is embedded Dolt, a git-versioned SQL database. The principle hasn't changed: work-as-data, in version control, queryable by an LLM. Just the substrate did.

If you've used Linear or Jira, this looks like a step backwards. You're trading a slick web UI for a CLI and some structured text in a directory called .beads/.

But there are two things Beads gets right that Linear and Jira can't.

The first is that the issues are in the repo. The work tracker lives where your code lives, version-controlled by git. In the original SQLite+JSONL Beads, this meant your branch and your work tracker moved together: checkout an old commit, the issues rewound to match; fork a branch, the work plan forked with it. Beads v1.0's default Classic mode currently keeps all issues on Dolt's main branch regardless of your git branch position; restoring the original branch coupling is on the Dolt team's roadmap but not implemented yet. Either way, the issues are queryable structured data your agent can read directly, with full history committed alongside your code, no API call required.

The second is the data format. Whether the format is JSONL or a Dolt table, the agent gets structured records it can read, edit, append to, and version. There's no opaque database, no auth flow, no rate limit, no schema migration. The agent reads the work tracker the same way it reads your code: as queryable structured data, version-controlled by git.

# Original Beads (Oct 2025 - early 2026): JSONL in your repo
{"id":"bd-a7je4","title":"fix rate limiter race","status":"ready",...}
{"id":"bd-a7je5","title":"add backpressure metric","status":"blocked",...}

# Beads v1.0 (Apr 2026): same shape, embedded Dolt as the store
SELECT id, title, status FROM beads WHERE assignee = 'sully';

This is the foundation. Everything else in Gas Town is built on the assumption that work is a thing you can read, edit, append to, and version, the same way you do with code. Once you accept that as the substrate, a number of moves that look weird start to make sense.

4. The unit-of-work inversion

Here's the second move, and it's the central one.

A Claude Code session runs for a while and then ends. Either you /exit, or the context runs out, or the host process gets killed. The session has a beginning and an end. It is, in Kubernetes terms, a pod: ephemeral, replaceable, anonymous. You don't name your pods.

A pod managing a deployment, on the other hand, is persistent. It has a name, a role, an identity, a history. The deployment outlives any individual pod. Pods come and go; the deployment proceeds.

Gas Town does the same inversion at the agent layer. Sessions are cattle. Agents are pets. An agent in Gas Town is not a session. An agent is a Bead, a row in your work tracker, with a name (Sully, Cooper, Maxine), a role (polecat, crew, refinery), a hook (a queue of pending work), and a CV (a list of completed convoys, which is what Gas Town calls a batch of related beads worked in parallel and landed together). The Bead lives in git. The CV is committed. The hook is committed. The agent persists across crashes, compactions, restarts, and machine reboots, because the agent is data.

The session is the thing you spawn when the agent has work to do. The session reads the agent's identity from Beads, reads its hook to find the work, does the work, commits the result, and exits. Another session can come along five minutes later and pick up where the last one left off, because the work is still on the hook and the hook is in git.

<figure> <svg viewBox="0 0 760 320" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="A persistent agent shown as a horizontal pillar in git, with multiple ephemeral sessions attached at different times."> <g transform="translate(40,140)"> <rect x="0" y="0" width="680" height="50" fill="#fdfbf6" stroke="#1c1a15" stroke-width="2" rx="3"/> <rect x="0" y="0" width="180" height="50" fill="#f5efe2"/> <text x="20" y="22" font-size="13" font-weight="600">sully</text> <text x="20" y="40" font-size="11" fill="#3a342a">role: polecat · rig-A</text> <line x1="180" y1="0" x2="180" y2="50" stroke="#1c1a15" stroke-width="1.5"/> <text x="200" y="22" font-family="JetBrains Mono" font-size="11" fill="#3a342a">hook: [bd-a7je4, bd-a7je9, bd-a7jf2]</text> <text x="200" y="40" font-family="JetBrains Mono" font-size="11" fill="#8a8377">cv: [convoy-04, convoy-09, convoy-12, ...]</text> </g> <text x="40" y="135" font-size="10" fill="#8a8377" letter-spacing="0.05em" text-transform="uppercase">AGENT (persistent · in git)</text> <g> <line x1="60" y1="100" x2="720" y2="100" stroke="#8a8377" stroke-width="1" stroke-dasharray="2 4"/> <text x="60" y="90" font-size="10" fill="#8a8377">9 AM</text> <text x="180" y="90" font-size="10" fill="#8a8377">10 AM</text> <text x="320" y="90" font-size="10" fill="#8a8377">11 AM</text> <text x="450" y="90" font-size="10" fill="#8a8377">1 PM</text> <text x="580" y="90" font-size="10" fill="#8a8377">3 PM</text> <text x="690" y="90" font-size="10" fill="#8a8377">NOW</text> </g> <g> <g transform="translate(70,30)"> <rect x="0" y="0" width="80" height="44" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.2" rx="2"/> <text x="40" y="18" text-anchor="middle" font-size="11" fill="#3a342a">session 1</text> <text x="40" y="35" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">2h, ctx full</text> <line x1="40" y1="44" x2="40" y2="100" stroke="#b8543e" stroke-width="1" stroke-dasharray="2 3"/> </g> <g transform="translate(190,30)"> <rect x="0" y="0" width="80" height="44" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.2" rx="2"/> <text x="40" y="18" text-anchor="middle" font-size="11" fill="#3a342a">session 2</text> <text x="40" y="35" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">crashed</text> <line x1="40" y1="44" x2="40" y2="100" stroke="#b8543e" stroke-width="1" stroke-dasharray="2 3"/> </g> <g transform="translate(310,30)"> <rect x="0" y="0" width="80" height="44" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.2" rx="2"/> <text x="40" y="18" text-anchor="middle" font-size="11" fill="#3a342a">session 3</text> <text x="40" y="35" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">handed off</text> <line x1="40" y1="44" x2="40" y2="100" stroke="#b8543e" stroke-width="1" stroke-dasharray="2 3"/> </g> <g transform="translate(440,30)"> <rect x="0" y="0" width="80" height="44" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.2" rx="2"/> <text x="40" y="18" text-anchor="middle" font-size="11" fill="#3a342a">session 4</text> <text x="40" y="35" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">finished</text> <line x1="40" y1="44" x2="40" y2="100" stroke="#b8543e" stroke-width="1" stroke-dasharray="2 3"/> </g> <g transform="translate(570,30)"> <rect x="0" y="0" width="80" height="44" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.2" rx="2"/> <text x="40" y="18" text-anchor="middle" font-size="11" fill="#3a342a">session 5</text> <text x="40" y="35" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#b8543e">running</text> <line x1="40" y1="44" x2="40" y2="100" stroke="#b8543e" stroke-width="1.4"/> </g> </g> <text x="40" y="20" font-size="10" fill="#b8543e" letter-spacing="0.05em">SESSIONS (ephemeral)</text> <g transform="translate(40,230)"> <line x1="0" y1="20" x2="680" y2="20" stroke="#1c1a15" stroke-width="1"/> <g fill="#1c1a15"> <circle cx="50" cy="20" r="3"/> <circle cx="120" cy="20" r="3"/> <circle cx="190" cy="20" r="3"/> <circle cx="260" cy="20" r="3"/> <circle cx="330" cy="20" r="3"/> <circle cx="400" cy="20" r="3"/> <circle cx="470" cy="20" r="3"/> <circle cx="540" cy="20" r="3"/> <circle cx="610" cy="20" r="3"/> <circle cx="670" cy="20" r="3" fill="#b8543e"/> </g> <text x="0" y="50" font-size="10" fill="#8a8377">git history (commits, hook updates, CV entries)</text> </g> </svg> <figcaption><strong>Fig 2.</strong> The agent is the row in the database. The sessions are short-lived processes that borrow the row to do a piece of work and then disappear. The history of what the agent has done lives in git, not in any one session.</figcaption> </figure>

Once you make this inversion, the context-window problem becomes mundane. Sessions hit context limits all the time. Who cares. The agent is fine. Spawn another session. The work is still where it was. The agent's name is still on it.

This is the conceptual hinge of Gas Town. Almost every other design choice is in service of making this inversion structurally sound.

Act 1: The Ledger

5. MEOW, from the bottom up

Once you've accepted that work is data, the next question is: what kind of data? A flat list of issues is fine for one human, fine for one agent. It breaks the moment your work has shape. "Migrate the auth schema, then in parallel update five backend services, then in parallel update three frontends, then deploy" is not a list. It's a graph with sequencing, branching, and joins. You can hold it in your head. You cannot sling it onto an agent's hook as one ticket. You need composition: a way to say "this big thing is made of these smaller things, in this order, with these constraints, and a worker can pick up wherever the previous one left off."

The MEOW stack (Molecular Expression of Work) is Yegge's answer. It has four levels of composition (Beads, Molecules, Protomolecules, Formulas), with Epics living sideways as beads-with-children rather than as their own layer.

<figure> <svg viewBox="0 0 600 470" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="The MEOW stack: Formula, Protomolecule, Molecule, Bead."> <g transform="translate(120,30)"> <rect x="0" y="0" width="360" height="64" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="20" y="25" font-weight="600" font-size="14">Formula</text> <text x="20" y="48" font-size="11" fill="#3a342a">TOML source. composable, has macros, gates, loops.</text> <text x="350" y="25" text-anchor="end" font-family="JetBrains Mono" font-size="10" fill="#b8543e">.toml</text> </g> <g transform="translate(300,100)"> <line x1="0" y1="0" x2="0" y2="22" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="-4,18 4,18 0,26" fill="#1c1a15"/> <text x="14" y="18" font-style="italic" font-size="11" fill="#b8543e">cook</text> </g> <g transform="translate(105,128)"> <rect x="0" y="0" width="390" height="64" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="20" y="25" font-weight="600" font-size="14">Protomolecule</text> <text x="20" y="48" font-size="11" fill="#3a342a">template graph of beads. has placeholders, no values yet.</text> </g> <g transform="translate(300,198)"> <line x1="0" y1="0" x2="0" y2="22" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="-4,18 4,18 0,26" fill="#1c1a15"/> <text x="14" y="18" font-style="italic" font-size="11" fill="#b8543e">instantiate</text> </g> <g transform="translate(80,226)"> <rect x="0" y="0" width="440" height="64" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="20" y="25" font-weight="600" font-size="14">Molecule</text> <text x="20" y="48" font-size="11" fill="#3a342a">concrete bead-chain workflow. running instance.</text> <g transform="translate(290,28)"> <circle cx="0" cy="0" r="6" fill="#b8543e" stroke="#1c1a15"/> <line x1="6" y1="0" x2="20" y2="0" stroke="#1c1a15" stroke-width="1"/> <circle cx="26" cy="0" r="6" fill="#b8543e" stroke="#1c1a15"/> <line x1="32" y1="0" x2="46" y2="0" stroke="#1c1a15" stroke-width="1"/> <circle cx="52" cy="0" r="6" fill="#b8543e" stroke="#1c1a15"/> <line x1="58" y1="0" x2="72" y2="0" stroke="#1c1a15" stroke-width="1"/> <circle cx="78" cy="0" r="6" fill="#fdfbf6" stroke="#1c1a15"/> <line x1="84" y1="0" x2="98" y2="0" stroke="#1c1a15" stroke-width="1"/> <circle cx="104" cy="0" r="6" fill="#fdfbf6" stroke="#1c1a15"/> </g> </g> <g transform="translate(300,296)"> <line x1="0" y1="0" x2="0" y2="22" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="-4,18 4,18 0,26" fill="#1c1a15"/> <text x="14" y="18" font-style="italic" font-size="11" fill="#b8543e">walk one step at a time</text> </g> <g transform="translate(50,324)"> <rect x="0" y="0" width="500" height="80" fill="#fdfbf6" stroke="#1c1a15" stroke-width="2" rx="3"/> <text x="20" y="25" font-weight="600" font-size="14">Bead</text> <text x="20" y="46" font-size="11" fill="#3a342a">the atomic unit. one issue, version-controlled in git.</text> <text x="20" y="66" font-family="JetBrains Mono" font-size="10" fill="#8a8377">id · status · depends_on · acceptance criteria · assignee · ...</text> <text x="480" y="25" text-anchor="end" font-family="JetBrains Mono" font-size="10" fill="#b8543e">dolt</text> </g> <text x="300" y="426" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">EPICS = BEADS WITH CHILDREN · LIVE BESIDE, NOT BELOW</text> <text x="300" y="452" text-anchor="middle" font-style="italic" font-size="11" fill="#b8543e">all of this lives in your repo, queryable by SQL, version-controlled by git</text> </svg> <figcaption><strong>Fig 3.</strong> The MEOW stack. The atom is the bead. Going up the stack, you compose beads into runnable workflows, then templates, then a declarative source format. Going down the stack, you cook a formula into a protomolecule, instantiate it into a molecule, walk the molecule one bead at a time.</figcaption> </figure>

Beads are the atoms. A bead is a single issue. It can be claimed, worked on, closed, blocked. Most bugs and features start life as a bead.

Epics are beads that have children. The children can be epics themselves. The children are parallel by default; you add explicit dependencies if you need sequencing. Epics give you top-down planning: "ship the new auth flow" decomposes into "design the schema," "implement the backend," "implement the frontend," "write the tests," and so on. The leaves get done first; the root finishes last.

Molecules are workflows. They're bead chains with arbitrary shapes, including branches, joins, gates, and loops. Crucially, a molecule is executable. The agent walks the chain one bead at a time, claiming and closing each in turn. Because the chain is in git, the agent can crash partway through, and the next session can resume from wherever the last claim was.

Protomolecules are templates. A 20-step release process is a useful thing to have written down. A 20-step release process you can stamp out fresh on demand, with all the dependencies pre-wired, is more useful. A protomolecule is the template; a molecule is an instance, created by copying the template and substituting in the variables.

Formulas are the source form. They're TOML files describing a workflow declaratively, with macros and gates and loops. A formula gets cooked into a protomolecule, which gets instantiated into a molecule, which gets executed by an agent. Three layers of indirection, each earning its keep: formulas are reusable, protomolecules are concrete graphs, molecules are running instances.

There's also an escape hatch called wisps: ephemeral beads that live in the database but never get persisted. The orchestration agents (Refinery, Witness, Deacon) generate hundreds of wisps an hour for their internal patrol workflows. Persisting them all would bury your repo in orchestration noise, so wisps stay in memory and burn at the end of the run.

If you've worked on workflow systems, this looks familiar. You've seen something like it in Airflow (DAGs and tasks), in Temporal (workflows and activities), in Argo (workflow templates and workflows). The interesting thing about MEOW is not the structure. The interesting thing is that it lives in git, with the same affordances as your codebase. An agent can read your work graph the way it reads your imports.

6. GUPP, or how to fight LLM politeness

OK, so work is in git. Workflows are in git. Agents are in git. How does an agent know to start working?

Each agent has a hook. The hook is a special pinned bead, attached to the agent's identity. When you want to give an agent work, you gt sling it onto the agent's hook. The work sits there, waiting.

The principle, called GUPP (Gastown Universal Propulsion Principle), is short:

If there is work on your hook, you must run it.

That's the whole rule. Every agent is prompted with it on startup. The agent's first action, on every session, is to check its hook. If something's on the hook, the agent starts working. If not, the agent waits or sleeps.

This is what makes Gas Town go without you driving it. You sling work onto a hook and walk away. The next time a session spins up for that agent, it picks up the work and runs.

In theory.

In practice, Claude Code is too polite for its own good. The startup prompt explicitly tells the agent to check its hook and act. The agent acknowledges this. It says "I'll check my hook now." Then it sits there, waiting for you to type something. Anything. Until the user has produced some input, the agent doesn't believe it has permission to actually do work.

Yegge calls this "physics over politeness." The agent is physically allowed (and instructed) to do the work. It has the prompt. It has the tools. It has the hook. But its training pushes it toward waiting for human input, hard. So Gas Town has to nudge.

The nudge is mechanical: a tmux send-keys that fires a few seconds after the session starts, simulating user input that says, in effect, "do your job." This bypasses the politeness deadlock. The agent reads the nudge, checks its hook, and starts working.

There is something funny about this. A meaningful chunk of the orchestrator's complexity is dedicated to working around an alignment artifact. The agent has been so thoroughly trained to wait for the human that it cannot be unblocked by a system prompt alone. It needs to see input arrive. So the system fakes input arriving.

I'd guess this layer vanishes over the next two model generations. The model providers know about this failure mode. Once Claude Code (and Codex, and Gemini CLI) ship versions willing to act on hook work without a manual prompt, GUPP becomes pure declarative. Until then, GUPP is a contract between the orchestrator and the model that requires a small amount of theater to enforce.

7. Nondeterministic idempotence

GUPP gets the agent moving. The next question is what keeps the work durable when the session dies mid-step. That's the runtime model.

Temporal, the workflow engine that Yegge gestures at as a comparison, has a beautiful property called deterministic durable replay. You write a workflow as ordinary code. Temporal records every step it takes (every API call, every wait, every result). When the workflow crashes and resumes on a new machine, Temporal replays the recorded history to reconstruct local state, and then continues from where it left off. The replay is deterministic: given the same history, the workflow code produces exactly the same execution. This is what makes Temporal workflows feel like code that just doesn't crash.

Gas Town can't do this. The reason is simple: the steps are written by an LLM. There's no determinism. If you replay a molecule, the agent making decisions inside each step will produce different output, take different code paths, and possibly reach a different end state.

So Gas Town gets durability the other way. Yegge calls it Nondeterministic Idempotence (NDI). The shape is:

The workflow (the molecule) is persistent. It's a chain of beads in git.
Each step has well-specified acceptance criteria. The bead describes what "done" looks like, concretely enough that an agent reading the bead can tell.
Agents claim steps one at a time, do them, and close them. A closed step is closed. An open step is open. There's no in-between.
If a session crashes mid-step, the step stays open. The next session reads it, picks up where the last one left off (or starts the step over, if that's cheaper), and closes it.
The path is fully nondeterministic. The outcome is the workflow, completed.

It's a clean design. The bead acceptance criteria are the contract. The git history is the substrate. The agent is the reasoning engine that figures out, on each step, what "completing this bead" means in the current state of the world. If an agent makes a mistake, the next agent reads the bead, sees the work didn't match the criteria, and corrects it.

NDI is weaker than Temporal's guarantees. Temporal will execute exactly the workflow you wrote. NDI will execute some workflow that satisfies the acceptance criteria you wrote, and the path may meander. For the kinds of work coding agents do (where there are usually many valid solutions and the goal is the outcome, not the trajectory) this is plenty, as long as the acceptance criteria are tight enough to catch trajectory-sensitive failures like races or partial migrations. Where they aren't, the LLM can pass each step and still produce a globally wrong result, and you fall back to the script approach for those steps.

There's parallel academic work pointing in the same direction. The MAKER paper (Meyerson et al., Nov 2025) reports completing a single task spanning over a million dependent LLM steps with zero errors, by extreme decomposition into single-purpose microagents combined with multi-agent voting at each step. The benchmark is constructed; the principle is what travels: long LLM workflows complete reliably when each step is atomized, well-specified, and individually checkable. NDI is the less clinical version of the same insight, with Gas Town adding the working substrate and the role hierarchy on top.

The path is whatever the agent decides. The destination is whatever the bead says.

Checkpoint: what you know now.

Coding agents fail at scale because the work loses durability between sessions. Gas Town fixes this by inverting the unit-of-work: agents become persistent identities in git, sessions become disposable processes that borrow those identities for an hour at a time.

Work itself is structured into a four-level composition stack (Beads, Molecules, Protomolecules, Formulas), with Epics sitting sideways as beads-with-children. The atomic level is Beads, which started as JSONL and migrated to embedded Dolt at v1.0. Higher levels compose into runnable workflows. Wisps give you ephemeral workflows for orchestration noise.

The runtime model is Nondeterministic Idempotence: workflows live in git as bead chains, each step has clear acceptance criteria, sessions claim and close steps, and crashes are recovered by the next session re-reading the chain. The path varies; the outcome is invariant.

What we haven't covered yet: how agents in the swarm avoid eating each other.

Act 2: The Town

8. Seven roles, two tiers

A durable bead tells you what work exists. It doesn't tell you who notices when nobody is moving it. It doesn't pick the next polecat to spawn, decide whether two agents are about to step on the same file, or restart a Witness that quietly died at 2 AM. The ledger is necessary. It isn't sufficient. You need agents whose job is watching other agents, and rules for who reports to whom.

Gas Town's answer is seven roles, organized into two tiers. Twenty to thirty agents work concurrently across one or more git repos (called rigs); the coordination is what keeps them from eating each other.

<figure> <svg viewBox="0 0 760 460" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="Gas Town hierarchy: Overseer, then Mayor and Deacon as town-tier peers, with Dogs attached to Deacon, and per-rig Witnesses, Polecats, Refinery, Crew descending from Deacon."> <g transform="translate(330,15)"> <rect x="0" y="0" width="100" height="44" fill="#fdfbf6" stroke="#1c1a15" stroke-width="2" rx="22"/> <text x="50" y="20" text-anchor="middle" font-weight="600" font-size="13">Overseer</text> <text x="50" y="35" text-anchor="middle" font-size="11" fill="#3a342a">(you)</text> </g> <line x1="380" y1="59" x2="380" y2="72" stroke="#1c1a15" stroke-width="1.5"/> <line x1="200" y1="72" x2="560" y2="72" stroke="#1c1a15" stroke-width="1.5"/> <line x1="200" y1="72" x2="200" y2="90" stroke="#1c1a15" stroke-width="1.5"/> <line x1="560" y1="72" x2="560" y2="90" stroke="#1c1a15" stroke-width="1.5"/> <g transform="translate(130,90)"> <rect x="0" y="0" width="140" height="50" fill="#f5efe2" stroke="#1c1a15" stroke-width="2" rx="3"/> <text x="70" y="22" text-anchor="middle" font-weight="600" font-size="13">The Mayor</text> <text x="70" y="38" text-anchor="middle" font-size="11" fill="#3a342a">your front desk</text> </g> <g transform="translate(490,90)"> <rect x="0" y="0" width="140" height="50" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="70" y="22" text-anchor="middle" font-weight="600" font-size="13">The Deacon</text> <text x="70" y="38" text-anchor="middle" font-size="11" fill="#3a342a">patrols the town</text> </g> <line x1="630" y1="115" x2="657" y2="115" stroke="#1c1a15" stroke-width="1.5"/> <g transform="translate(657,98)"> <rect x="0" y="0" width="93" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="46" y="16" text-anchor="middle" font-weight="600" font-size="12">Dogs</text> <text x="46" y="28" text-anchor="middle" font-size="10" fill="#3a342a">helpers</text> </g> <line x1="40" y1="170" x2="720" y2="170" stroke="#d8d2c4" stroke-width="1" stroke-dasharray="3 4"/> <text x="40" y="163" font-size="10" fill="#b8543e" letter-spacing="0.05em">· TOWN TIER ·</text> <text x="720" y="163" text-anchor="end" font-size="10" fill="#8a8377" letter-spacing="0.05em">across all rigs</text> <text x="40" y="217" font-size="10" fill="#b8543e" letter-spacing="0.05em">· RIG TIER ·</text> <text x="720" y="217" text-anchor="end" font-size="10" fill="#8a8377" letter-spacing="0.05em">per repo</text> <line x1="560" y1="140" x2="560" y2="200" stroke="#1c1a15" stroke-width="1.5"/> <line x1="180" y1="200" x2="580" y2="200" stroke="#1c1a15" stroke-width="1.5"/> <line x1="180" y1="200" x2="180" y2="225" stroke="#1c1a15" stroke-width="1.5"/> <line x1="380" y1="200" x2="380" y2="225" stroke="#1c1a15" stroke-width="1.5"/> <line x1="580" y1="200" x2="580" y2="225" stroke="#1c1a15" stroke-width="1.5"/> <g transform="translate(135,225)"> <rect x="0" y="0" width="90" height="40" fill="#f5efe2" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="45" y="18" text-anchor="middle" font-weight="600" font-size="11">Witness</text> <text x="45" y="32" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">rig-A</text> </g> <g transform="translate(335,225)"> <rect x="0" y="0" width="90" height="40" fill="#f5efe2" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="45" y="18" text-anchor="middle" font-weight="600" font-size="11">Witness</text> <text x="45" y="32" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">rig-B</text> </g> <g transform="translate(535,225)"> <rect x="0" y="0" width="90" height="40" fill="#f5efe2" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="45" y="18" text-anchor="middle" font-weight="600" font-size="11">Witness</text> <text x="45" y="32" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">rig-C</text> </g> <line x1="180" y1="265" x2="180" y2="305" stroke="#1c1a15" stroke-width="1"/> <line x1="380" y1="265" x2="380" y2="305" stroke="#1c1a15" stroke-width="1"/> <line x1="580" y1="265" x2="580" y2="305" stroke="#1c1a15" stroke-width="1"/> <g transform="translate(110,305)"> <rect x="0" y="0" width="140" height="100" fill="none" stroke="#d8d2c4" stroke-width="1" stroke-dasharray="3 3" rx="3"/> <text x="70" y="-6" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">RIG-A WORKERS</text> <g transform="translate(8,12)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">polecat</text></g> <g transform="translate(72,12)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">polecat</text></g> <g transform="translate(8,40)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.4" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">refinery</text></g> <g transform="translate(72,40)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">crew</text></g> <g transform="translate(40,68)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">crew</text></g> </g> <g transform="translate(310,305)"> <rect x="0" y="0" width="140" height="100" fill="none" stroke="#d8d2c4" stroke-width="1" stroke-dasharray="3 3" rx="3"/> <text x="70" y="-6" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">RIG-B WORKERS</text> <g transform="translate(8,12)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">polecat</text></g> <g transform="translate(72,12)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">polecat</text></g> <g transform="translate(8,40)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">polecat</text></g> <g transform="translate(72,40)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.4" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">refinery</text></g> <g transform="translate(40,68)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">crew</text></g> </g> <g transform="translate(510,305)"> <rect x="0" y="0" width="140" height="100" fill="none" stroke="#d8d2c4" stroke-width="1" stroke-dasharray="3 3" rx="3"/> <text x="70" y="-6" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">RIG-C WORKERS</text> <g transform="translate(8,12)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">polecat</text></g> <g transform="translate(72,12)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.4" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">refinery</text></g> <g transform="translate(8,40)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">crew</text></g> <g transform="translate(72,40)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">crew</text></g> <g transform="translate(40,68)"><rect x="0" y="0" width="60" height="22" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1" rx="2"/><text x="30" y="15" text-anchor="middle" font-size="11" fill="#3a342a">crew</text></g> </g> <text x="380" y="440" text-anchor="middle" font-style="italic" font-size="11" fill="#b8543e">everything anchored in Beads · in git</text> </svg> <figcaption><strong>Fig 4.</strong> The Gas Town role hierarchy. The Mayor and Deacon (and the Deacon's Dogs) operate town-wide. Each rig has its own Witness, its own Refinery, its own Polecats, and its own Crew. Beads is the substrate everything writes to.</figcaption> </figure>

Three roles operate at the town level (across all rigs):

The Mayor is your front desk. It's the agent you talk to most. When you have an idea ("we need to fix the rate limiter"), you talk to the Mayor. The Mayor files beads, drafts plans, slings work to other agents, and gathers status when convoys land. It's the only agent with a global view of the town.

The Deacon is the watchdog. It runs a patrol loop, every couple of minutes, checking on the health of the town. It can dispatch maintenance work, restart stuck agents, escalate genuine problems to you. The Deacon is what makes Gas Town keep running while you're at lunch. (It is also, in early Gas Town versions, the role most likely to go feral. Yegge has talked openly about the Deacon's tendency to go on "serial killer sprees, viciously taking out random workers mid-job" — calling it the modern-day Butler in the Gas Town murder mysteries — and at one point recommended disabling it. Largely fixed by v1.0.)

The Dogs are the Deacon's hands, plus one structural exception. The patrol loop got too crowded with chores (clean up stale branches, run plugins, GC merged worktrees), so the Deacon hands those off to dedicated maintenance agents. The exception is Boot the Dog, who isn't a maintenance helper at all: Boot wakes up every five minutes and checks whether the Deacon is alive. Boot watches the Deacon, the other Dogs work for the Deacon. We'll come back to that distinction in section 10.

Four roles operate at the rig level (per repo):

Polecats are the workers. They're ephemeral, named, claimed for a task, and decommissioned when the task is done. A polecat picks up a bead, opens a worktree, does the work, files a merge request, and gets recycled. Polecats are how throughput happens.

The Refinery is the merge queue. When ten polecats finish their work simultaneously, you can't just merge ten branches into main. You'd get conflicts, broken builds, and a fork no one understands. The Refinery serializes merges, intelligently rebases, runs tests, and lands one MR at a time.

The Witness is the per-rig health monitor. Same idea as the Deacon, but scoped to one repo. The Witness keeps polecats unstuck, prods the Refinery if the queue stalls, and escalates to the Deacon if a problem is bigger than one rig.

The Crew are your collaborators. Crew agents are persistent, named (you pick the names), and work on whatever you tell them. They're the equivalent of the Claude Code session you used to keep open all day, except now there are several of them, each with its own identity and history. Crew is for the work where you want to be in the loop. Polecats are for the work where you don't.

If you squint, this looks like Kubernetes. The Mayor is the scheduler. The Deacon is the controller manager. The Witness is the kubelet. The Polecats are the pods. Beads is etcd. Yegge points this out at the end of his launch post and treats it as a discovery: these are the natural shapes that emerge when you need to herd cats at scale.

I think the Kubernetes resemblance is more than a coincidence. The shape of any system that coordinates unreliable workers toward a goal converges on something like this. There's a top-level brain. There's a thing that watches the watchers. There's a queue. There's a recycler. There's an audit log. There's a place to put the source of truth. The names change. The shape doesn't.

The interesting difference, and Yegge calls this out, is what each system optimizes for. Kubernetes asks "is it running?" It tries to keep N replicas alive forever. Gas Town asks "is it done?" It tries to land a convoy and then nuke the worker. K8s pods are anonymous; Gas Town polecats have names and CVs. K8s reconciles toward a continuous desired state; Gas Town proceeds toward a terminal goal. Same engine shape, different destination.

9. The Refinery, or why the merge queue is the actual product

If I had to point at one role and say "this is the thing that makes Gas Town more than a wrapper around tmux," it would be the Refinery.

Here's the problem the Refinery solves. You have ten polecats working in parallel, each on its own worktree, against a base branch that started identical. They each finish around the same time and submit their merge requests. Now you have ten changes that all want to land. They overlap in places. They were each written assuming the base branch wouldn't move. You merge the first one. The other nine are now stale.

A naive solution: rebase all nine and merge them in order. Half of them have textual conflicts. A few have semantic conflicts that the textual diff doesn't catch. One was working on the same file the second one already changed, and the rebase produces a perfectly valid-looking patch that breaks the test suite.

The Refinery is an agent specifically prompted for this. It uses what's called a Bors-style bisecting merge queue (named after Bors-NG, the merge bot the Rust language project used for years). The pattern is: instead of testing each MR one at a time, you batch them together, merge the batch onto main, and run the test suite once. If the batch passes, all the MRs land. If the batch fails, you split it in two and retry each half. The bad MR gets isolated in O(log N) batches instead of O(N) sequential tests, which is a meaningful difference when each test run takes ten minutes and you have thirty MRs in the queue.

<figure> <svg viewBox="0 0 760 320" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="Bors-style bisecting merge queue: batch fails, splits in two, half passes, half splits again, isolating the bad MR in log N batches."> <g transform="translate(60,30)"> <rect x="0" y="0" width="200" height="60" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="100" y="20" text-anchor="middle" font-weight="600" font-size="12">batch: MRs #1 #2 #3 #4 #5 #6 #7 #8</text> <text x="100" y="40" text-anchor="middle" font-size="11" fill="#3a342a">try to merge all</text> <text x="100" y="55" text-anchor="middle" font-style="italic" font-size="11" fill="#b8543e">test fails</text> </g>

<g transform="translate(40,125)"> <rect x="0" y="0" width="120" height="46" fill="#fdfbf6" stroke="#3a8c5c" stroke-width="1.5" rx="3"/> <text x="60" y="18" text-anchor="middle" font-weight="600" font-size="11">#1 #2 #3 #4</text> <text x="60" y="35" text-anchor="middle" font-size="11" fill="#3a8c5c">passes · landed</text> </g>

<g transform="translate(180,125)"> <rect x="0" y="0" width="120" height="46" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.5" rx="3"/> <text x="60" y="18" text-anchor="middle" font-weight="600" font-size="11">#5 #6 #7 #8</text> <text x="60" y="35" text-anchor="middle" font-size="11" fill="#b8543e">fails · split</text> </g>

<g transform="translate(150,205)"> <rect x="0" y="0" width="100" height="40" fill="#fdfbf6" stroke="#3a8c5c" stroke-width="1.5" rx="3"/> <text x="50" y="16" text-anchor="middle" font-weight="600" font-size="11">#5 #6</text> <text x="50" y="32" text-anchor="middle" font-size="11" fill="#3a8c5c">passes</text> </g>

<g transform="translate(260,205)"> <rect x="0" y="0" width="100" height="40" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.5" rx="3"/> <text x="50" y="16" text-anchor="middle" font-weight="600" font-size="11">#7 #8</text> <text x="50" y="32" text-anchor="middle" font-size="11" fill="#b8543e">fails · split</text> </g>

<g transform="translate(255,270)"> <rect x="0" y="0" width="60" height="36" fill="#fdfbf6" stroke="#3a8c5c" stroke-width="1.5" rx="3"/> <text x="30" y="14" text-anchor="middle" font-weight="600" font-size="11">#7</text> <text x="30" y="28" text-anchor="middle" font-size="11" fill="#3a8c5c">passes</text> </g>

<g transform="translate(320,270)"> <rect x="0" y="0" width="60" height="36" fill="#fdfbf6" stroke="#b8543e" stroke-width="1.5" rx="3"/> <text x="30" y="14" text-anchor="middle" font-weight="600" font-size="11">#8</text> <text x="30" y="28" text-anchor="middle" font-size="11" fill="#b8543e">isolated</text> </g>

<g transform="translate(450,90)"> <text x="0" y="0" font-weight="600" font-size="13">Bors-style bisection</text> <text x="0" y="22" font-size="11" fill="#3a342a">Bad MR found in log₂(N) batches.</text> <text x="0" y="40" font-size="11" fill="#3a342a">For 8 MRs with one bad, that's 4</text> <text x="0" y="58" font-size="11" fill="#3a342a">test runs instead of 8.</text> <text x="0" y="84" font-size="11" fill="#8a8377">Strategy invented for the Rust</text> <text x="0" y="102" font-size="11" fill="#8a8377">language repo. The Refinery</text> <text x="0" y="120" font-size="11" fill="#8a8377">is the agentic adaptation.</text> </g> </svg> <figcaption><strong>Fig 5.</strong> How the Refinery's bisecting merge queue works. Eight MRs in a batch with one bad apple. The full batch fails. Split in two; the good half lands; the bad half splits again; eventually the offending MR is isolated. With one bad MR in a batch of N, you spend roughly 1 + log₂N test runs instead of N.</figcaption> </figure>

Beyond the bisection, the Refinery has standing orders for what to do with the bad MR: ask the polecat to redo against new HEAD, replace it with another polecat's work, or escalate. It runs tests after each merge. It can restart its session and pick up where it left off, because the merge queue is in beads.

This is one role, but it's the role that makes the whole system tractable. Without the Refinery, parallel polecats are a pretty way to corrupt your repo. With it, you can let twenty agents work overnight and wake up to a clean main branch with twelve features merged and two convoys flagged for your review.

There's a lesson here that goes well beyond Gas Town. The hardest problems in multi-agent systems are not the agents. They're the points where parallel work tries to become serial. The merge queue. The shared resource. The handoff. Whoever owns those choke points owns the throughput. Most of the engineering effort in Gas Town went into the Refinery, the Witness, and the Deacon, not into making Polecats smarter. Polecats are just Claude Code. The thing Gas Town adds is the layer that turns a swarm of polecats into a working factory.

10. Self-watching systems

The other architectural choice worth pausing on is the patrol-and-heartbeat structure.

You'd think a multi-agent system would be event-driven. Polecat finishes a task, emits an event, Refinery wakes up and merges. Witness emits a "stuck" event, Deacon dispatches a Dog. This is the design you'd draw on a whiteboard.

Gas Town does some of this, but the dominant pattern is patrols: long-running loops that wake up periodically, check on the world, and act. The Deacon runs a patrol every couple of minutes. Each Witness runs a patrol per rig. The Refinery's main loop is a patrol that drains the merge queue.

Patrols have exponential backoff. If a Witness wakes up and finds nothing to do, it goes back to sleep with a longer interval. If something does happen (a mutating bd or gt command), a daemon wakes the relevant patrol agents up.

Why patrols and not pure events? Because LLM agents are flaky in ways event systems don't tolerate. An agent might miss an event. It might process the event and then go off and do something unrelated. It might get distracted by its own scratch thoughts and forget to come back. You don't want your merge queue depending on every relevant event reliably making it to a single subscriber.

A patrol is forgiving. It re-checks the state of the world from scratch every iteration. If the previous iteration missed something, the next one catches it. If an agent dies in the middle of a patrol, the next agent picks up where it left off. The system is resilient to its own components being unreliable, because the components keep checking each other.

Boot the Dog is the funniest, most-correct example of this principle. The Deacon patrols the town. But what if the Deacon falls asleep, or gets distracted, or quietly dies? Boot the Dog wakes up every five minutes and pokes the Deacon. Boot's whole job, on each wakeup, is to decide whether the Deacon needs a heartbeat, a nudge, a restart, or simply to be left alone, and then go back to sleep. It is the single dumbest agent in the whole system, and it has to be, because Yegge couldn't trust the Deacon to wake itself up reliably.

<figure> <svg viewBox="0 0 760 500" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="The watcher chain: deterministic daemon at top, then Boot the Dog, then Deacon, Dogs and Witnesses, then Polecats and Refinery."> <text x="380" y="22" text-anchor="middle" font-size="10" fill="#b8543e" letter-spacing="0.05em">↑ DUMBER · MORE RELIABLE</text> <g transform="translate(380,60)"> <g> <rect x="-100" y="0" width="200" height="50" fill="#1c1a15" stroke="#1c1a15" stroke-width="2" rx="3"/> <text x="0" y="18" text-anchor="middle" fill="#fdfbf6" font-weight="600" font-size="13">Daemon</text> <text x="0" y="32" text-anchor="middle" font-size="10" fill="#d8d2c4">deterministic Go process</text> <text x="0" y="44" text-anchor="middle" font-size="10" fill="#d8d2c4">heartbeat every 3 min</text> </g> <line x1="0" y1="50" x2="0" y2="78" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="-4,72 4,72 0,80" fill="#1c1a15"/> <text x="14" y="68" font-style="italic" font-size="11" fill="#b8543e">wakes every 5 min</text> <g transform="translate(0,82)"> <rect x="-100" y="0" width="200" height="50" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="0" y="22" text-anchor="middle" font-weight="600" font-size="13">Boot the Dog</text> <text x="0" y="38" text-anchor="middle" font-size="11" fill="#3a342a">simplest LLM role: wake the Deacon</text> </g> <line x1="0" y1="132" x2="0" y2="160" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="-4,154 4,154 0,162" fill="#1c1a15"/> <text x="14" y="150" font-style="italic" font-size="11" fill="#b8543e">decides if</text> <g transform="translate(0,164)"> <rect x="-100" y="0" width="200" height="50" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="0" y="22" text-anchor="middle" font-weight="600" font-size="13">The Deacon</text> <text x="0" y="38" text-anchor="middle" font-size="11" fill="#3a342a">town patrol · health monitor</text> </g> <line x1="0" y1="214" x2="0" y2="242" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="-4,236 4,236 0,244" fill="#1c1a15"/> <text x="14" y="232" font-style="italic" font-size="11" fill="#b8543e">dispatches / nudges</text> <g transform="translate(0,246)"> <line x1="0" y1="0" x2="0" y2="14" stroke="#1c1a15" stroke-width="1.5"/> <line x1="-160" y1="14" x2="160" y2="14" stroke="#1c1a15" stroke-width="1.5"/> <line x1="-160" y1="14" x2="-160" y2="28" stroke="#1c1a15" stroke-width="1.5"/> <line x1="160" y1="14" x2="160" y2="28" stroke="#1c1a15" stroke-width="1.5"/> <g transform="translate(-260,30)"> <rect x="0" y="0" width="200" height="44" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="100" y="20" text-anchor="middle" font-weight="600" font-size="12">Dogs</text> <text x="100" y="34" text-anchor="middle" font-size="11" fill="#3a342a">maintenance helpers</text> </g> <g transform="translate(60,30)"> <rect x="0" y="0" width="200" height="44" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="100" y="20" text-anchor="middle" font-weight="600" font-size="12">Witnesses</text> <text x="100" y="34" text-anchor="middle" font-size="11" fill="#3a342a">per-rig patrols</text> </g> <line x1="160" y1="74" x2="160" y2="100" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="156,94 164,94 160,102" fill="#1c1a15"/> </g> <g transform="translate(0,360)"> <rect x="60" y="0" width="200" height="44" fill="#f5efe2" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="160" y="20" text-anchor="middle" font-weight="600" font-size="12">Polecats / Refinery / Crew</text> <text x="160" y="34" text-anchor="middle" font-size="11" fill="#3a342a">the workers actually doing things</text> </g> </g> <text x="40" y="80" font-size="10" fill="#8a8377" letter-spacing="0.05em">deterministic root</text> <text x="40" y="98" font-size="10" fill="#8a8377" letter-spacing="0.05em">(the chain stops here)</text> <text x="720" y="180" text-anchor="end" font-style="italic" font-size="11" fill="#b8543e">each layer is dumber than what it watches</text> <text x="720" y="200" text-anchor="end" font-style="italic" font-size="11" fill="#b8543e">but more reliable about waking up on time</text> <text x="380" y="486" text-anchor="middle" font-size="10" fill="#b8543e" letter-spacing="0.05em">↓ SMARTER · LESS RELIABLE</text> </svg> <figcaption><strong>Fig 6.</strong> The watcher chain. The deterministic daemon at the top is the only thing in the system whose wake-up logic doesn't depend on an LLM staying coherent. Every layer below it is an LLM watching the layer below it. Each watcher is simpler than the thing it's watching, on purpose.</figcaption> </figure>

The general principle: in a system of unreliable agents, the watcher needs a watcher. Eventually, you reach a primitive (a daemon, a cron job, something deterministic) and the chain terminates. Everything beneath that primitive is an LLM checking whether the LLM beneath it is doing its job. This is a humbling architecture pattern. It says the way to build reliable systems out of unreliable agents is not to make the agents more reliable; it's to make the agents watch each other.

Checkpoint: what you know now.

Gas Town has seven roles in two tiers. The town tier (Mayor, Deacon, Dogs) coordinates across all repos. The rig tier (Witnesses, Polecats, Refinery, Crew) does the per-repo work.

The Refinery is the role that earns its keep on day one. It uses a Bors-style bisecting merge queue to land parallel MRs in log N test runs instead of N.

Reliability is not in any single agent. It's in the chain of patrols and watchers, with a deterministic daemon at the top of the chain heartbeat-pinging every three minutes.

Act 3: The City

11. From Gas Town to Gas City

Gas Town shipped with seven hardcoded roles. They are the right shape for coding agents. They aren't the right shape for moderating image submissions, running a nightly ops dashboard, or grading customer support tickets. The town worked; the town didn't generalize. The next problem isn't running one swarm. It's stamping out many differently shaped swarms.

Gas City is the answer.

Some chronology, because the timeline matters. Gas Town launched January 1, 2026: 17 days of vibe-coded Go, 75,000 lines, 2,000 commits. The community Discord and a handful of Fortune 100 trial deployments showed up almost immediately. By early March it had an 8-stage adoption framework, a Wasteland federation layer (a Dolt-backed trust network where thousands of Gas Towns can share a common Wanted Board and pick up work other Gas Towns post), and a measurable cost (one early adopter reported $100/hour in API spend during heavy use).

<g transform="translate(70,100)"> <circle cx="0" cy="0" r="6" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.8"/> <line x1="0" y1="-6" x2="0" y2="-22" stroke="#1c1a15" stroke-width="1"/> <text x="0" y="-32" text-anchor="middle" font-weight="600" font-size="12">Beads</text> <text x="0" y="-48" text-anchor="middle" font-size="11" fill="#3a342a">git-backed issue tracker</text> <text x="0" y="22" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">OCT 13, 2025</text> </g>

<g transform="translate(290,100)"> <circle cx="0" cy="0" r="7" fill="#b8543e" stroke="#1c1a15" stroke-width="1.8"/> <line x1="0" y1="6" x2="0" y2="22" stroke="#1c1a15" stroke-width="1"/> <text x="0" y="36" text-anchor="middle" font-weight="600" font-size="12">Gas Town</text> <text x="0" y="52" text-anchor="middle" font-size="11" fill="#3a342a">orchestrator (75k LOC)</text> <text x="0" y="-14" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">JAN 1, 2026</text> </g>

<g transform="translate(530,100)"> <circle cx="0" cy="0" r="6" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.8"/> <line x1="0" y1="-6" x2="0" y2="-22" stroke="#1c1a15" stroke-width="1"/> <text x="0" y="-32" text-anchor="middle" font-weight="600" font-size="12">Wasteland</text> <text x="0" y="-48" text-anchor="middle" font-size="11" fill="#3a342a">federation network</text> <text x="0" y="22" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">MAR 4, 2026</text> </g>

<g transform="translate(620,100)"> <circle cx="0" cy="0" r="6" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.8"/> <line x1="0" y1="6" x2="0" y2="22" stroke="#1c1a15" stroke-width="1"/> <text x="0" y="36" text-anchor="middle" font-weight="600" font-size="11">Beads + GT v1.0</text> <text x="0" y="52" text-anchor="middle" font-size="11" fill="#3a342a">Dolt migration done</text> <text x="0" y="-14" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">APR 3</text>

<circle cx="80" cy="0" r="7" fill="#b8543e" stroke="#1c1a15" stroke-width="1.8"/>
<line x1="80" y1="-7" x2="80" y2="-22" stroke="#1c1a15" stroke-width="1"/>
<text x="80" y="-32" text-anchor="middle" font-weight="600" font-size="12">Gas City</text>
<text x="80" y="22" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">APR 24, 2026</text>

<path d="M-10,68 Q35,76 90,68" fill="none" stroke="#b8543e" stroke-width="1" opacity="0.6"/>
<text x="40" y="86" text-anchor="middle" font-style="italic" font-size="10" fill="#b8543e">3 weeks apart</text>

</g>

<text x="380" y="180" text-anchor="middle" font-style="italic" font-size="11" fill="#b8543e">six months · one developer started this</text> </svg> <figcaption><strong>Fig 7.</strong> The trajectory from Beads to Gas City. Six months between the first git-backed issue tracker and a deployable SDK for building agent factories. The cluster at the right is real: v1.0 of two projects and the launch of a third all landed in the same three-week window of April 2026.</figcaption> </figure>

By April both Gas Town and Beads hit v1.0. Beads' storage layer migrated from JSONL+SQLite to embedded Dolt, a git-versioned SQL database; the "single binary, no daemon" experience came back, with version history and SQL queries built in. Yegge's framing for the Dolt migration is that it's been the unsung hero of the whole stack: Dolt gives the agents Git-style branch/merge semantics on structured data, which is exactly what a federated work system needs.

Then in late April, Yegge announced Gas City. Gas City is Gas Town disassembled and rebuilt as an SDK: roles, topology, and orchestration patterns become declarative packs that you can deploy and run. Yegge didn't write Gas City himself; it was built by Julian Knutsen and Chris Sells from the community, after Yegge sketched the vision. Gas Town is one factory; Gas City is the toolkit for stamping out new ones.

The strategic change is bigger than the technical one. The original Gas Town pitch was "Kubernetes for coding agents." The Gas City pitch is something more ambitious: agents-running-themselves, for any business process. Industry calls a facility that runs without humans watching a "dark factory." Yegge reframes Gas City as a "light factory" — or, in his phrasing, "a very well-lit dark one!" — because every action is observable and auditable in Dolt.

The example he gives is from his own MMORPG, Wyvern. Players who hit level 25 get to upload custom character images. Yegge has to review them for inappropriate content and approve or reject them. This is a small recurring task that has nothing to do with code. He describes a two-agent pack: one agent reviews each submission, another agent checks the first agent's work. Two agents instead of one because, as he puts it, any agent can go temporarily insane at any time. A second pair of eyes is cheap insurance.

<figure> <svg viewBox="0 0 760 240" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="A two-agent pack handling Wyvern's player image queue: submission, reviewer, approver, hall of fame."> <g transform="translate(20,90)"> <rect x="0" y="0" width="120" height="60" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="60" y="22" text-anchor="middle" font-weight="600" font-size="12">Submission</text> <text x="60" y="36" text-anchor="middle" font-size="11" fill="#3a342a">queue (web form)</text> <text x="60" y="50" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#8a8377">→ JSON</text> </g> <text x="80" y="80" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">INPUT</text> <line x1="142" y1="120" x2="178" y2="120" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="172,116 172,124 180,120" fill="#1c1a15"/> <g transform="translate(180,80)"> <rect x="0" y="0" width="160" height="80" fill="#f5efe2" stroke="#1c1a15" stroke-width="1.8" rx="3"/> <text x="80" y="22" text-anchor="middle" font-weight="600" font-size="13">Reviewer</text> <text x="80" y="40" text-anchor="middle" font-size="11" fill="#3a342a">reads submission</text> <text x="80" y="56" text-anchor="middle" font-size="11" fill="#3a342a">recommends:</text> <text x="80" y="72" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#b8543e">approve · reject · flag</text> </g> <line x1="340" y1="120" x2="420" y2="120" stroke="#1c1a15" stroke-width="1"/> <circle cx="380" cy="120" r="6" fill="#b8543e" stroke="#1c1a15" stroke-width="1"/> <text x="380" y="72" text-anchor="middle" font-size="10" fill="#b8543e" letter-spacing="0.05em">recommendation</text> <text x="380" y="178" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">(as a bead)</text> <g transform="translate(420,80)"> <rect x="0" y="0" width="160" height="80" fill="#f5efe2" stroke="#1c1a15" stroke-width="1.8" rx="3"/> <text x="80" y="22" text-anchor="middle" font-weight="600" font-size="13">Approver</text> <text x="80" y="40" text-anchor="middle" font-size="11" fill="#3a342a">checks the recommendation</text> <text x="80" y="56" text-anchor="middle" font-size="11" fill="#3a342a">takes action</text> <text x="80" y="72" text-anchor="middle" font-size="11" fill="#3a342a">logs decision</text> </g> <line x1="582" y1="120" x2="618" y2="120" stroke="#1c1a15" stroke-width="1.5"/> <polygon points="612,116 612,124 620,120" fill="#1c1a15"/> <g transform="translate(620,90)"> <rect x="0" y="0" width="120" height="60" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.5" rx="3"/> <text x="60" y="24" text-anchor="middle" font-weight="600" font-size="12">Hall of Fame</text> <text x="60" y="42" text-anchor="middle" font-size="11" fill="#3a342a">(or rejection)</text> </g> <text x="680" y="80" text-anchor="middle" font-size="10" fill="#8a8377" letter-spacing="0.05em">OUTPUT</text> <g transform="translate(80,200)"> <line x1="0" y1="0" x2="600" y2="0" stroke="#1c1a15" stroke-width="0.8" stroke-dasharray="4 3"/> <text x="300" y="20" text-anchor="middle" font-style="italic" font-size="11" fill="#b8543e">audit trail in Dolt: which submission · which agent · which decision · timestamp</text> </g> </svg> <figcaption><strong>Fig 8.</strong> A two-agent pack handling Wyvern's player image queue. The recommendation lives as a bead between the two agents. The full audit trail (which submission, which agent, which decision, which timestamp) is in Dolt. The pack itself is declarative TOML, not application code.</figcaption> </figure>

Packs make the orchestrator generalize. What you'd actually use one for is a separate question.

12. Replacing SaaS, badly enough

The bigger pitch in the Gas City post is about replacing SaaS. The argument goes like this:

SaaS started as a way for companies to share fixed costs across many customers.
It evolved into the superset of every customer's needs. Most customers use 20% of the features and subsidize the other 80%.
The 20% you actually use can be rebuilt in-house, badly, by an agent crew. And badly is good enough for a 200-person company, because you don't need Salesforce-grade compliance for a 200-person workflow.
Gas City is the substrate for building the in-house replacement: declarative packs, git-versioned audit trails, identity, memory, recovery from failure.

I was initially skeptical of this thesis. Replacing Atlassian with a Gas City pack is a much bigger lift than replacing your image-moderation queue. The evidence so far is two points, not a curve. Both are worth dwelling on.

The small end is from Yegge's v1.0 post. He describes a non-technical Comms major, four years out of school, who has been running Gas Town since "a few weeks after it came out" and has built an in-house replacement for a niche but pricey SaaS product her company was paying for. The substrate is now cheap enough that the rebuild does not require an engineer.

The large end is from a DoltHub blog post in late March. Tim Sehn, the CEO of DoltHub (and an old colleague of Yegge's from Amazon), used Gas Town to rewrite roughly 30,000 lines of Dolt's storage backend from Go into C, embedded inside SQLite. By 4pm that day, all 87,000 SQLite acceptance tests were passing. He had not written serious C in twenty years. The whole experiment cost him about $3,000 in API credits. An experienced systems engineer using Gas Town to do something he plausibly could not have done by hand at all.

The Comms major's case is small-SaaS displacement. Sehn's is an experienced engineer punching through a productivity ceiling Gas Town can move. Whether the wide territory between them, generic business processes run by small in-house packs that pay for themselves, is full or empty, no one knows yet. That's the Gas City bet, and the Gas City pivot says Yegge is making it. The interesting layer of the stack is no longer the coding orchestrator; it's the layer above it, where you stamp out small agent crews for whatever business process needs running. The coding case is just the first one anyone got working.

The skeptical case is worth registering. SaaS isn't only the 20% of features you actually use. It's the SOC 2 audit, the SSO integration, the data residency guarantees, the SLAs, the on-call team that takes incident calls at 2 AM, and the legal liability that travels with all of those. A pack that approves Wyvern character submissions doesn't need any of that. A pack that touches payroll, payments, customer PII, or anything regulated does. The question isn't whether you can build a working in-house pack — the Comms major proves you can. It's whether operating, securing, and being legally responsible for that pack stays below the SaaS bill once you add up everything the SaaS was actually doing. For small niche workflows, the math probably works. For replacing your CRM or your accounting system, probably not. Gas City's territory is the middle, and the shape of that middle is what's still empirical.

13. Locate yourself before installing

Sitting alongside the technical pieces of the project is a framework that doesn't appear in the codebase at all. Yegge sketched it at Gas Town's launch as eight stages of AI-assisted coding adoption; with Gas City, he extended it to eleven. It is the most opinionated piece of advice anyone associated with the project gives, and it is the part most likely to be ignored.

The point of the ladder is that Gas Town is not a beginner tool. Beads, hooks, the Refinery, the Deacon, NDI — every part of it is the answer to specific kinds of pain. If you haven't felt the pain, you can't tell which parts are load-bearing and which are baroque. You'll bypass the load-bearing parts, and your repo will catch fire.

<figure> <svg viewBox="0 0 760 600" xmlns="http://www.w3.org/2000/svg" width="100%" font-family="serif" role="img" aria-label="Yegge's 11-stage ladder of AI-assisted coding adoption, with three safety zones marked: stages 1–4 'don't even read about Gas Town' at the top, stages 5–6 'don't install yet' in the middle, and stages 7+ 'Gas Town safe to install' at the bottom. Stages 9, 10, 11 were added with Gas City in April 2026."> <text x="40" y="22" font-size="13" font-weight="600">The ladder of AI-assisted coding adoption</text> <text x="40" y="40" font-size="10" fill="#8a8377" letter-spacing="0.08em">YEGGE'S 11 STAGES · ORIGINALLY 8 AT GAS TOWN LAUNCH, EXTENDED WITH GAS CITY</text>

<g transform="translate(50,68)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">1</text> <text x="48" y="15" font-size="12" font-weight="600">Zero-AI</text> <text x="48" y="29" font-size="11" fill="#3a342a">human writes every line</text> </g> <g transform="translate(50,106)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">2</text> <text x="48" y="15" font-size="12" font-weight="600">IDE agent, permissions on</text> <text x="48" y="29" font-size="11" fill="#3a342a">Cursor, Copilot, every change reviewed</text> </g> <g transform="translate(50,144)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">3</text> <text x="48" y="15" font-size="12" font-weight="600">IDE agent, YOLO</text> <text x="48" y="29" font-size="11" fill="#3a342a">approval prompts off, lots of edits</text> </g> <g transform="translate(50,182)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">4</text> <text x="48" y="15" font-size="12" font-weight="600">IDE wide-agent</text> <text x="48" y="29" font-size="11" fill="#3a342a">agent with broader codebase reach</text> </g>

<g transform="translate(50,254)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">5</text> <text x="48" y="15" font-size="12" font-weight="600">CLI single-agent, YOLO</text> <text x="48" y="29" font-size="11" fill="#3a342a">Claude Code or similar, off the leash</text> </g> <g transform="translate(50,292)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">6</text> <text x="48" y="15" font-size="12" font-weight="600">CLI multi-agent (3–5)</text> <text x="48" y="29" font-size="11" fill="#3a342a">a small fleet, manually coordinated</text> </g>

<g transform="translate(50,360)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">7</text> <text x="48" y="15" font-size="12" font-weight="600">10+ agents, hand-managed</text> <text x="48" y="29" font-size="11" fill="#3a342a">tmux, baling wire, you-as-foreman</text> </g> <g transform="translate(50,398)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">8</text> <text x="48" y="15" font-size="12" font-weight="600">Building your own orchestrator</text> <text x="48" y="29" font-size="11" fill="#3a342a">where Gas Town itself was born</text> </g>

<g transform="translate(50,452)"> <line x1="0" y1="12" x2="490" y2="12" stroke="#b8543e" stroke-width="1" stroke-dasharray="3 3"/> <text x="245" y="6" text-anchor="middle" font-style="italic" font-size="10" fill="#b8543e">↓ added by Gas City, April 2026 · no longer about coding</text> </g>

<g transform="translate(50,472)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">9</text> <text x="48" y="15" font-size="12" font-weight="600">First deployed pack</text> <text x="48" y="29" font-size="11" fill="#3a342a">a real business process runs itself</text> </g> <g transform="translate(50,510)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">10</text> <text x="48" y="15" font-size="12" font-weight="600">Many deployed packs</text> <text x="48" y="29" font-size="11" fill="#3a342a">you become the bottleneck</text> </g> <g transform="translate(50,548)"> <rect x="0" y="0" width="490" height="34" fill="#fdfbf6" stroke="#1c1a15" stroke-width="1.2"/> <circle cx="22" cy="17" r="13" fill="#1c1a15"/> <text x="22" y="22" text-anchor="middle" font-size="12" font-weight="600" fill="#fdfbf6">11</text> <text x="48" y="15" font-size="12" font-weight="600">Factory builder</text> <text x="48" y="29" font-size="11" fill="#3a342a">a meta-orchestrator manages your packs</text> </g>

<g transform="translate(560,60)"> <rect x="0" y="0" width="180" height="164" fill="#ecd0c2" stroke="#1c1a15" stroke-width="0.6"/> <text x="90" y="22" text-anchor="middle" font-size="10" fill="#8a4a3a" letter-spacing="0.08em">STAGES 1–4</text> <text x="90" y="50" text-anchor="middle" font-size="13" font-weight="600">Don't even read</text> <text x="90" y="66" text-anchor="middle" font-size="13" font-weight="600">about Gas Town</text> <text x="90" y="100" text-anchor="middle" font-size="10" fill="#3a342a">framework solves problems</text> <text x="90" y="114" text-anchor="middle" font-size="10" fill="#3a342a">you haven't run into yet</text> </g> <g transform="translate(560,246)"> <rect x="0" y="0" width="180" height="84" fill="#f0d9b8" stroke="#1c1a15" stroke-width="0.6"/> <text x="90" y="22" text-anchor="middle" font-size="10" fill="#8a6a3a" letter-spacing="0.08em">STAGES 5–6</text> <text x="90" y="48" text-anchor="middle" font-size="13" font-weight="600">Don't install yet</text> <text x="90" y="68" text-anchor="middle" font-size="10" fill="#3a342a">you'll wreck your repo</text> </g> <g transform="translate(560,352)"> <rect x="0" y="0" width="180" height="234" fill="#dde6d3" stroke="#1c1a15" stroke-width="0.6"/> <text x="90" y="22" text-anchor="middle" font-size="10" fill="#5a6b48" letter-spacing="0.08em">STAGES 7+</text> <text x="90" y="50" text-anchor="middle" font-size="13" font-weight="600">Gas Town</text> <text x="90" y="66" text-anchor="middle" font-size="13" font-weight="600">becomes safe</text> <text x="14" y="100" font-size="10" fill="#3a342a">You've already felt the pain</text> <text x="14" y="114" font-size="10" fill="#3a342a">Gas Town's design solves.</text> <text x="14" y="128" font-size="10" fill="#3a342a">Each role earns its keep.</text> <line x1="14" y1="148" x2="166" y2="148" stroke="#b8543e" stroke-width="0.8" stroke-dasharray="3 3"/> <text x="90" y="170" text-anchor="middle" font-style="italic" font-size="10" fill="#b8543e">stages 9–11 are the</text> <text x="90" y="184" text-anchor="middle" font-style="italic" font-size="10" fill="#b8543e">Gas City extension</text> <text x="90" y="206" text-anchor="middle" font-size="10" fill="#3a342a">(no longer about code)</text> </g>

<g transform="translate(20,60)"> <line x1="0" y1="0" x2="0" y2="520" stroke="#8a8377" stroke-width="0.8"/> <polygon points="-3,516 3,516 0,520" fill="#8a8377"/> <text x="-12" y="260" text-anchor="middle" font-size="9" fill="#8a8377" letter-spacing="0.12em" transform="rotate(-90, -12, 260)">MORE AGENTS · MORE AUTONOMY</text> </g> </svg> <figcaption><strong>Fig 9.</strong> Yegge's 11-stage ladder of AI-assisted coding adoption. Stages 1–8 existed at Gas Town's launch; stages 9–11 were added with Gas City in April 2026, when the ceiling moved from "running your own orchestrator" to "running a fleet of deployable agent packs." The three safety zones are Yegge's, not mine: install Gas Town at Stage 7 or above, hold off at 5–6, and at 1–4 don't bother reading about it yet.</figcaption> </figure>

Yegge's three cut points are blunt. Stage 7 and up: install Gas Town. By the time you're hand-managing ten or more agents, you've already lived inside the problems Gas Town's roles are designed for. The Refinery makes obvious sense. The Witness makes obvious sense. You'll know which knobs to leave alone. Stage 5–6: don't install it. One agent, or a small handful, is fine; the orchestrator's overhead will eat you, and you'll fight machinery you don't yet need. He has begged the community, repeatedly, not to do this. Stage 1–4: don't even read about it. The framework is an answer to a question you haven't asked.

The shape of the extension is its own story. Stages 9, 10, 11 are no longer about coding. They're the trajectory of someone who got coding-orchestration working and is now stamping out small autonomous packs for the rest of the business: image moderation, compliance review, contract triage. Stage 7 used to be the top of the ladder; the framework had a ceiling at "run your own orchestrator." Gas City raised the ceiling, and what now sits above the old ceiling is no longer a developer-tools question. It's an operations question, and probably an org-design one.

For a serious user, the most useful thing about the ladder is that it tells you not to skip rungs. The Stage 9 win comes from having a working Stage 7–8 system to put it on top of. If you're at Stage 6 today, the next thing to chase is not Gas City. It's making your work durable enough that any Stage 7–8 orchestrator (Gas Town or somebody else's) can run on it.

Checkpoint: what you know now.

Gas City is the SDK abstraction over Gas Town. Roles, topology, and orchestration become declarative TOML packs. The same engine that runs a coding swarm can run image moderation, ops triage, anything you can describe as a graph of agents.

The strategic claim is SaaS replacement. A non-technical Comms major has displaced a niche SaaS at her company with a pack. Tim Sehn rewrote 30,000 lines of Dolt's storage backend in C in a day, for $3,000 in API credits. The territory between those two points is the bet, not yet proven.

Where you are determines what you can install. Yegge's 11-stage ladder is blunt: Stage 7+ for Gas Town, 5–6 for "not yet," 1–4 for "don't even read about it." The framework grew from 8 to 11 with Gas City because the top of the ladder is no longer about code.

Four ideas worth stealing

Make the work durable, not the agent

This is the central inversion. In a system where any individual component (sessions, agents, even hosts) is unreliable, the way to get reliability is to push state into a substrate that doesn't share the unreliability. Gas Town's substrate is git. Yours might be a database, or a queue, or a state machine in S3. The principle is: agents should be stateless and replaceable, work state should be persistent and idempotent. If you're building anything multi-agent, the first design question is "what does the work look like when no agent is currently holding it?" If you can't answer cleanly, your system will lose work.

Acceptance criteria beat scripts when the worker is an LLM

NDI is the runtime version of this idea. If you write your workflows as imperative scripts that an LLM has to execute step-by-step, you'll spend forever debugging the cases where the LLM did something slightly different. If you write your workflows as a sequence of well-specified outcomes ("this bead is done when X is true"), the LLM has room to figure out how to satisfy the criteria, and you can verify completion mechanically. The MAKER paper's million-step result points at the same conclusion: long LLM workflows complete reliably when each step is atomic, locally checkable, and isolated from the others. Gas Town and MAKER are the production and academic versions of the same principle.

Reliability is a chain of watchers

Boot the Dog watches the Deacon. The Deacon watches the Witnesses. The Witnesses watch the Polecats. At every layer, the watcher is dumber and more reliable than the thing it's watching. This is how Gas Town keeps a swarm of LLMs aimed at a goal. The pattern is older than Gas Town (it's how kubelet watches pods, it's how supervisord watches processes) but it's particularly sharp here because the things being watched are smart, the watchers don't have to be, and the bottom of the chain is a deterministic three-minute heartbeat. If you're building anything where agents need to keep going without human oversight, the design question is not "how do I make the agents more reliable?" It's "what's the simplest thing that can detect when an agent is stuck, and what does it do about it?"

Topology should be declarative, not hardcoded

Gas Town v1 baked seven roles into the binary. Gas City pulled them out into TOML packs you can ship and version. The reason this matters: once you've built one swarm that works, the second one shouldn't require rewriting the orchestrator. It should require writing a config. The crew that watches your image-moderation queue, the crew that runs your nightly compliance review, and the crew that triages customer tickets are the same engine with different topologies. If you're building anything multi-agent, the question after "what does the work look like" is "what does the topology look like, and can someone who isn't an engineer change it?" The orchestrators that win the next two years will be the ones where the answer is yes.

Epilogue

A year ago, the interesting engineering question in AI tooling was "how do I make a single coding agent work well." The answer (RAG, tool use, larger contexts, better prompts) was technically deep and mostly commodity by the end of 2025.

The interesting question this year is one layer up. "How do I make a fleet of coding agents work well." The honest answer is that we don't fully know yet. Gas Town is one attempt, and it's a good one, but it's a 2026-shaped attempt. The model layer underneath it will keep moving. The agents will get more compliant about acting on their hooks without a fake keystroke. The session size will grow. The orchestrators will consolidate around two or three patterns and the rest will fade.

What I think will not fade is the inversion at the heart of all of this. Work is persistent. Sessions are ephemeral. Agents are identities that connect work to sessions. Reliability comes from substrate, not from making the agents perfect. Acceptance criteria beat imperative scripts when the executor is a model. Watchers compose hierarchically. Topology is declarative. Whatever orchestrator we use a year from now, those facts will still be true.

If you're at Stage 5 or 6 (one or a few coding agents, hand-managed) the lesson is not "rush to install Gas Town." Yegge himself begs you not to, repeatedly, on the grounds that you'll wreck your repo. The lesson is to start writing your work down. Get it out of your head and your chat history and into a thing the next session can read. Once your work is durable, you can swap in any orchestrator you want.

We're going to start running this experiment ourselves at Boon, in a small way, this quarter. I'll write up what I learn.

References

Welcome to Gas Town. Steve Yegge, January 2026 (medium).
The Future of Coding Agents. Steve Yegge, January 2026 (medium).
Welcome to the Wasteland. Steve Yegge, March 2026 (medium).
Gas Town: from Clown Show to v1.0. Steve Yegge, April 2026 (medium).
Welcome to Gas City. Steve Yegge, April 2026 (medium).
A Week In Gas Town. Tim Sehn (DoltHub), March 2026 (dolthub blog). The Dolt-in-C case study cited above.
Solving a Million-Step LLM Task with Zero Errors (MAKER). Meyerson et al., Cognizant + UT Austin, November 2025 (arXiv:2511.09030).
Gas Town source. github.com/gastownhall/gastown.
Gas City source. github.com/gastownhall/gascity.
Beads source. github.com/gastownhall/beads (moved from steveyegge/beads at v1.0).
Dolt (git-versioned SQL database). github.com/dolthub/dolt.
Bors-NG (the bisecting merge queue Gas Town's Refinery is modeled after). github.com/bors-ng/bors-ng.

The Unified Multimodal Stack

Fri, 01 May 2026 00:00:00 GMT

import Figure from "@components/figure/Figure.svelte"; import VideoTokenExplosion from "@components/figures/omni-modal-stack/VideoTokenExplosion.svelte"; import EvsPatchPruning from "@components/figures/omni-modal-stack/EvsPatchPruning.svelte";

A sequel to The Unified Vision Stack. Picks up where C-RADIOv4 ended and asks what it takes to teach one eye to listen, watch, and speak. Worked example throughout: NVIDIA's Nemotron 3 Nano Omni, released April 2026. About a 45-minute read.

1. A single eye, three teachers

The previous post left us with C-RADIOv4. One vision backbone, three teachers, a single forward pass whose features could be read by three different downstream tasks at once.

<figure> <svg viewBox="0 0 680 260" xmlns="http://www.w3.org/2000/svg" width="100%" style="height: auto"> <g font-family="serif" font-size="14" font-weight="600"> <rect x="30" y="30" width="110" height="40" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5"/><text x="85" y="55" text-anchor="middle" fill="#2563EB">DINOv3</text> <rect x="30" y="110" width="110" height="40" fill="#D1FAE5" stroke="#059669" stroke-width="1.5"/><text x="85" y="135" text-anchor="middle" fill="#059669">SigLIP2</text> <rect x="30" y="190" width="110" height="40" fill="#EDE0FC" stroke="#9333EA" stroke-width="1.5"/><text x="85" y="215" text-anchor="middle" fill="#9333EA">SAM3</text> </g> <text x="85" y="18" text-anchor="middle" font-family="JetBrains Mono" font-size="10" letter-spacing="1.5" fill="#6B6258">TEACHERS</text> <g stroke="#8F8578"> <line x1="145" y1="50" x2="285" y2="120" marker-end="url(#a1)"/> <line x1="145" y1="130" x2="285" y2="130" marker-end="url(#a1)"/> <line x1="145" y1="210" x2="285" y2="140" marker-end="url(#a1)"/> </g> <defs><marker id="a1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="5" markerHeight="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="#8F8578"/></marker></defs> <rect x="290" y="100" width="120" height="60" fill="#1A1A1A" stroke="#B8651A" stroke-width="2"/> <text x="350" y="125" text-anchor="middle" font-family="serif" font-size="15" font-weight="700" fill="#F4EEE3">student</text> <text x="350" y="145" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#B8651A">C-RADIOv4</text> <line x1="412" y1="130" x2="432" y2="130" stroke="#8F8578" marker-end="url(#a1)"/> <rect x="435" y="115" width="100" height="30" fill="none" stroke="#B8651A" stroke-width="1.5"/> <text x="485" y="134" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#1A1A1A">unified feature</text> <g font-family="JetBrains Mono" font-size="10"> <line x1="540" y1="125" x2="565" y2="55" stroke="#8F8578" marker-end="url(#a1)"/> <rect x="565" y="40" width="100" height="30" fill="#DBEAFE" stroke="#2563EB"/><text x="615" y="59" text-anchor="middle" fill="#2563EB">DINOv3 head</text> <line x1="540" y1="130" x2="565" y2="130" stroke="#8F8578" marker-end="url(#a1)"/> <rect x="565" y="115" width="100" height="30" fill="#D1FAE5" stroke="#059669"/><text x="615" y="134" text-anchor="middle" fill="#059669">SigLIP head</text> <line x1="540" y1="135" x2="565" y2="205" stroke="#8F8578" marker-end="url(#a1)"/> <rect x="565" y="190" width="100" height="30" fill="#EDE0FC" stroke="#9333EA"/><text x="615" y="209" text-anchor="middle" fill="#9333EA">SAM3 head</text> </g> </svg> <figcaption><strong>Fig 1.</strong> Where C-RADIOv4 came from. Three frozen teachers feed one student backbone, and three small heads on the output read off the kind of feature each downstream task wants. The student is what we're about to bolt the rest of an omni-modal stack onto.</figcaption> </figure>

The prequel post walks through C-RADIOv4 in detail: DINOv3 brought dense patch geometry, SigLIP2 brought text alignment, SAM3 brought instance masks, and C-RADIOv4 inherited the three through AM-RADIO-style agglomerative distillation. At inference you can attach a small head to the shared backbone and read off whichever family of feature a downstream task wants.

For this post the relevant takeaway is narrower: the backbone is a strong "vision-only" model. Hand it an image and you can do dense prediction, retrieval, or segmentation through the appropriate head. Other vision tasks the family is reported to handle (depth, tracking, and beyond) are covered in the prequel; here we only need that the backbone can serve as a feature extractor for an LLM.

But "look at an image" is not the job most modern systems are being built for. The thing people want is an agent: something that takes a photo, listens to a question, watches a video, and answers in language. The system has to perceive across modalities and then talk about what it perceived. C-RADIOv4 only handles half of that loop, and only for one of the modalities.

What does the rest take?

There's a clue in C-RADIOv4 itself. One of its teachers was SigLIP2, so the SigLIP-aligned head on the backbone is trained to put image features into a space shared with text embeddings. SigLIP itself trains image and caption towers to score together (more on the loss in §3); inheriting that head means the C-RADIOv4 backbone can produce vectors that are aligned with caption-space text in the SigLIP sense. We already have, through that head, an image encoder that "speaks text" in that specific sense.

So why does Nemotron 3 Nano Omni, the open omni model NVIDIA shipped in April 2026, still bolt a separate text tokenizer onto the front of a 30-billion-parameter autoregressive language model and feed C-RADIOv4-H's features into it as if they were just another input? If the SigLIP head was meant to be the text bridge, what does the LLM do that the head can't?

The short answer is that SigLIP-style alignment scores; it doesn't generate. Once that's clear, the rest of the architecture falls into place. The post unfolds three architectural moves: a text-centric autoregressive decoder at the center, modality-specific encoders for everything that isn't text, and a stack of token-budget mechanisms to keep inference cost from collapsing under its own weight.

Act 1: the puzzle

Why a vision backbone that already aligns to text still cannot answer in language.

2. The SigLIP puzzle

The SigLIP head sitting on top of C-RADIOv4 produces caption-aligned features. An image goes in, a vector comes out, and that vector lives in the same embedding space SigLIP's text tower lives in. So you can take any sentence, embed it through the text tower, and ask: how close is its embedding to the image's? You get a similarity score. Higher score, the caption fits the image better. Lower score, it doesn't.

<text x="170" y="20" text-anchor="middle" font-family="JetBrains Mono" font-size="10" letter-spacing="1.5" fill="#6B6258">CONTRASTIVE (SIGLIP / CLIP)</text>

<rect x="30" y="40" width="80" height="80" fill="#EDE5D4" stroke="#8F8578"/> <rect x="30" y="40" width="80" height="32" fill="#93C5FD" opacity=".7"/> <rect x="30" y="72" width="80" height="22" fill="#D1AE7A" opacity=".8"/> <rect x="30" y="94" width="80" height="26" fill="#059669" opacity=".6"/> <circle cx="90" cy="55" r="8" fill="#FEF3C7"/> <text x="70" y="135" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">image</text>

<rect x="30" y="170" width="180" height="28" fill="#F9F5EB" stroke="#C9BEAA" rx="3"/> <text x="120" y="188" text-anchor="middle" font-family="serif" font-size="11" fill="#1A1A1A">"a dog on a beach"</text> <text x="120" y="213" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">caption</text>

<text x="378" y="110" text-anchor="end" font-family="JetBrains Mono" font-size="13" fill="#1A1A1A">= 0.81</text> <text x="295" y="244" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">one number per pair</text> <text x="295" y="261" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">"does this caption match</text> <text x="295" y="275" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">this image?"</text>

<text x="525" y="20" text-anchor="middle" font-family="JetBrains Mono" font-size="10" letter-spacing="1.5" fill="#6B6258">AUTOREGRESSIVE (LLM)</text>

<g font-family="JetBrains Mono" font-size="11"> <rect x="385" y="50" width="36" height="22" fill="#FBE9CE" stroke="#B8651A"/><text x="403" y="65" text-anchor="middle" fill="#1A1A1A">the</text> <rect x="423" y="50" width="38" height="22" fill="#FBE9CE" stroke="#B8651A"/><text x="442" y="65" text-anchor="middle" fill="#1A1A1A">dog</text> <rect x="463" y="50" width="28" height="22" fill="#FBE9CE" stroke="#B8651A"/><text x="477" y="65" text-anchor="middle" fill="#1A1A1A">is</text> <rect x="493" y="50" width="32" height="22" fill="#FBE9CE" stroke="#B8651A"/><text x="509" y="65" text-anchor="middle" fill="#1A1A1A">on</text> <rect x="527" y="50" width="36" height="22" fill="#FBE9CE" stroke="#B8651A"/><text x="545" y="65" text-anchor="middle" fill="#1A1A1A">the</text> <rect x="565" y="50" width="34" height="22" fill="none" stroke="#B91C1C" stroke-dasharray="3,2"/><text x="582" y="65" text-anchor="middle" fill="#B91C1C">?</text> </g> <text x="492" y="91" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">prefix</text>

<text x="492" y="194" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">P(NEXT TOKEN)</text> <g font-family="JetBrains Mono" font-size="9"> <rect x="395" y="200" width="14" height="44" fill="#2563EB"/><text x="402" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 402 252)">beach</text> <rect x="412" y="218" width="14" height="26" fill="#2563EB" opacity=".7"/><text x="419" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 419 252)">grass</text> <rect x="429" y="223" width="14" height="21" fill="#2563EB" opacity=".55"/><text x="436" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 436 252)">floor</text> <rect x="446" y="229" width="14" height="15" fill="#2563EB" opacity=".4"/><text x="453" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 453 252)">car</text> <rect x="463" y="232" width="14" height="12" fill="#2563EB" opacity=".3"/><text x="470" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 470 252)">deck</text> <rect x="480" y="236" width="14" height="8" fill="#2563EB" opacity=".22"/><text x="487" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 487 252)">sand</text> <rect x="497" y="238" width="14" height="6" fill="#2563EB" opacity=".18"/><text x="504" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 504 252)">moon</text> <rect x="514" y="240" width="14" height="4" fill="#2563EB" opacity=".15"/><text x="521" y="252" text-anchor="start" fill="#1A1A1A" transform="rotate(35 521 252)">...</text> </g> <text x="615" y="222" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">over</text> <text x="615" y="234" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">~100k</text> <text x="615" y="246" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">tokens</text>

<rect x="405" y="285" width="60" height="22" fill="#FBE9CE" stroke="#2563EB" stroke-width="1.5"/> <text x="435" y="300" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#2563EB">beach</text> <text x="510" y="300" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">sample, append, repeat</text> </svg> <figcaption><strong>Fig 2.</strong> Two different operations on the same example. Left: SigLIP-style scoring takes an image and a caption, projects each into a shared space, and outputs a single number. Right: an LLM takes a partial sentence and outputs a probability distribution over what the next token should be, then samples one and appends it. Scaling SigLIP doesn't get you the right side. Scaling an LLM doesn't, by itself, get you the left side.</figcaption> </figure>

What you can't do with that score is ask for a description of the image. The SigLIP text tower never learned to produce sentences. It only learned to score them.

This is easy to lose track of, because SigLIP and CLIP are routinely described as "vision-language models" in a way that suggests they understand language in some general sense. They don't, in the sense the rest of the post is going to need. Here's what they actually do, taken straight from the CLIP paper:

"The text sequence is bracketed with [SOS] and [EOS] tokens and the activations of the highest layer of the transformer at the [EOS] token are treated as the feature representation of the text which is layer normalized and then linearly projected into the multi-modal embedding space."

The text tower runs through the sentence once, takes one vector from one position in the final layer, and stops. Whatever happened in the per-token states gets thrown away. The output is a single embedding that says "this sentence as a whole means X" in some compressed form, and that is the entire output. The training loss never asked the model to expose a per-token next-token distribution, so it never learned to produce one.

SigLIP and SigLIP 2 keep the same shape, just with different losses (sigmoid binary classification per pair instead of softmax over the batch). SigLIP 2 even attaches a decoder during training to push the encoder representations harder, but the decoder gets thrown away before release:

"Finally, we note that the decoder only serves for representation learning here and is not part of the model release."

So the artifact you actually get is still a contrastive embedder. It scores. If you want to generate, you need a different organ.

The architectural consequence is the central design choice of every modern omni-modal stack. The text decoder sits in the middle. Vision goes in as features. Audio goes in as features. Video goes in as features. The decoder is the only thing in the system that can run a softmax over the next word, and everything else feeds it.

C-RADIOv4 is one of the best things you can put in front of a decoder. It is not, by itself, a decoder.

3. Contrastive vs autoregressive

The split between scoring and generating is structural, not a difference of degree. To see why, look at what each model is being trained to optimize.

CLIP's loss is the cleanest version. Take a batch of N image-caption pairs. Run each image through the image tower, each caption through the text tower. You now have N image vectors and N text vectors. Compute every pairwise dot product: that gives you an N×N matrix of similarities. The training loss says, for each row of this matrix, the diagonal entry should be the largest (because the image at row i pairs with the caption at column i, the matched pair). For each column, same thing. Apply softmax cross-entropy over the rows and over the columns, average them. That's the whole loss.

"Given a batch of N (image, text) pairs, CLIP is trained to predict which of the N × N possible (image, text) pairings across a batch actually occurred. ... We optimize a symmetric cross entropy loss over these similarity scores."

SigLIP swaps the softmax for a sigmoid binary classification on each cell of the matrix: each pair is either matched (label +1) or unmatched (label −1), and every cell contributes independently. Computationally cheaper to scale. Same architectural shape: dual towers, dot product, scalar score per pair.

What this loss does not teach the model is anything about next words. The matched-pair signal at any cell tells the towers "make these two embeddings closer." It says nothing about what should come after a given word in a sentence. Both towers, image and text, are trained to produce one embedding each, then compared. The text tower never has to predict the next token. There is no decoder loss. There is no autoregressive supervision anywhere in the training procedure.

A language model is doing the opposite thing. Take a sentence. For each prefix of the sentence (the first 1 word, the first 2 words, and so on), compute a probability distribution over what the next word should be. The loss is the negative log-likelihood of the actual next word, summed over all positions. This does teach the model about next words. It teaches almost only about next words. After enough data, you have a function that takes a partial sentence and returns a probability distribution over every possible continuation, which you can sample from to make new text.

These are two different functions, and the difference doesn't shrink with scale. The contrastive function takes (image, sentence) and returns a number. The autoregressive function takes (partial sentence) and returns a probability distribution over the next token. You can't use one to do the other's job, no matter how large you make it. Scaling SigLIP doesn't get you generation. Scaling an LLM doesn't, by itself, get you a calibrated image-caption similarity score; you have to fit a head on top to extract one.

This is why most production multimodal stacks ship both. SigLIP-class encoders for understanding what's in an image. An autoregressive LLM in the middle for producing language. (Some research designs go the other way, with an early-fusion model that tokenizes images straight into the LLM's vocabulary; Chameleon is the cleanest example of that branch. The pretraining cost story for that approach is much larger, and it isn't the path Nemotron 3 Nano Omni took.)

4. The text decoder is a different organ

The five things an autoregressive LLM contains, in the order an input flows through them: a tokenizer that splits text into sub-word pieces from a fixed vocabulary; a learned embedding table that maps token IDs to continuous vectors; a stack of attention (and sometimes state-space) blocks that produce contextualized hidden states; a learned LM head that projects each hidden state back to the vocabulary as logits; and an autoregressive sampling loop that picks one token from the final logits, appends it to the input, and runs the whole thing again.

Pieces 4 and 5 do not exist in SigLIP's text tower. SigLIP has the first two (tokenizer, embedding table) and something like the third (a small transformer over the tokens), but the output gets projected once into the shared image-text space and then the model stops. There is no head that produces a distribution over next tokens, and no sampling loop to consume one. The contrastive training objective specifically did not select for them.

So we need an LLM in the middle. The question becomes how to get the vision encoder's output into the LLM's input, and the parallel question for audio and video.

Act 2: three ways to wire it up

Once you know you need a generative LLM, the question is how you feed it.

5. The adapter zoo: Flamingo, BLIP-2, LLaVA

Suppose you have a strong text-only LLM. Pretraining cost millions of dollars and you don't want to redo it. You also have a strong vision encoder, frozen, that produces dense feature maps. The question is: how do you bolt them together so the LLM can answer questions about images without losing what it already knows about language?

Three approaches dominated the 2022-2024 era, and the differences between them turn out to be load-bearing for what came after.

<text x="113" y="20" text-anchor="middle" font-family="serif" font-size="14" font-weight="600" fill="#1A1A1A">Flamingo</text> <text x="340" y="20" text-anchor="middle" font-family="serif" font-size="14" font-weight="600" fill="#1A1A1A">BLIP-2</text> <text x="567" y="20" text-anchor="middle" font-family="serif" font-size="14" font-weight="600" fill="#1A1A1A">LLaVA</text>

<rect x="40" y="35" width="146" height="34" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5"/> <text x="113" y="51" text-anchor="middle" font-family="serif" font-size="11" fill="#2563EB">Vision encoder</text> <text x="113" y="64" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">(frozen)</text> <line x1="113" y1="72" x2="113" y2="88" stroke="#6B6258" marker-end="url(#a3)"/> <rect x="50" y="88" width="126" height="30" fill="#FBE9CE" stroke="#B8651A"/> <text x="113" y="103" text-anchor="middle" font-family="serif" font-size="11" fill="#B8651A">Perceiver Resampler</text> <text x="113" y="115" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#6B6258">→ 64 tokens</text> <line x1="113" y1="120" x2="113" y2="138" stroke="#6B6258" marker-end="url(#a3)"/> <rect x="40" y="138" width="146" height="148" fill="none" stroke="#1A1A1A" stroke-width="1.5"/> <text x="113" y="155" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">LLM (frozen)</text> <rect x="50" y="162" width="126" height="13" fill="#F4EEE3"/> <rect x="50" y="178" width="126" height="13" fill="#FEE7D5" stroke="#B8651A" stroke-width="0.8" stroke-dasharray="2,2"/> <rect x="50" y="194" width="126" height="13" fill="#F4EEE3"/> <rect x="50" y="210" width="126" height="13" fill="#FEE7D5" stroke="#B8651A" stroke-width="0.8" stroke-dasharray="2,2"/> <rect x="50" y="226" width="126" height="13" fill="#F4EEE3"/> <rect x="50" y="242" width="126" height="13" fill="#FEE7D5" stroke="#B8651A" stroke-width="0.8" stroke-dasharray="2,2"/> <rect x="50" y="258" width="126" height="13" fill="#F4EEE3"/> <text x="113" y="304" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#B8651A">tanh(α=0) at init</text> <text x="113" y="318" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">cross-attn inserted</text>

<rect x="267" y="35" width="146" height="34" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5"/> <text x="340" y="51" text-anchor="middle" font-family="serif" font-size="11" fill="#2563EB">Vision encoder</text> <text x="340" y="64" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">(frozen)</text> <line x1="340" y1="72" x2="340" y2="88" stroke="#6B6258" marker-end="url(#a3)"/> <rect x="267" y="88" width="146" height="62" fill="#FBE9CE" stroke="#B8651A" stroke-width="1.5"/> <text x="340" y="104" text-anchor="middle" font-family="serif" font-size="11" fill="#B8651A">Q-Former (188M)</text> <g font-family="JetBrains Mono" font-size="9"> <rect x="277" y="113" width="18" height="20" fill="#FEF3C7" stroke="#B8651A" stroke-width="0.6"/> <rect x="297" y="113" width="18" height="20" fill="#FEF3C7" stroke="#B8651A" stroke-width="0.6"/> <rect x="317" y="113" width="18" height="20" fill="#FEF3C7" stroke="#B8651A" stroke-width="0.6"/> <rect x="337" y="113" width="18" height="20" fill="#FEF3C7" stroke="#B8651A" stroke-width="0.6"/> <rect x="357" y="113" width="18" height="20" fill="#FEF3C7" stroke="#B8651A" stroke-width="0.6"/> <text x="390" y="127" fill="#6B6258">...</text> </g> <text x="340" y="146" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#6B6258">32 learned queries</text> <line x1="340" y1="152" x2="340" y2="170" stroke="#6B6258" marker-end="url(#a3)"/> <rect x="267" y="170" width="146" height="116" fill="none" stroke="#1A1A1A" stroke-width="1.5"/> <text x="340" y="187" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">LLM (frozen)</text> <g font-family="JetBrains Mono" font-size="9"> <rect x="277" y="195" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="296" y="195" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="315" y="195" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="334" y="195" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="353" y="195" width="18" height="14" fill="#F4EEE3" stroke="#1A1A1A" stroke-width="0.4"/> <rect x="372" y="195" width="18" height="14" fill="#F4EEE3" stroke="#1A1A1A" stroke-width="0.4"/> <rect x="391" y="195" width="18" height="14" fill="#F4EEE3" stroke="#1A1A1A" stroke-width="0.4"/> </g> <text x="340" y="227" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">queries → soft tokens</text> <text x="340" y="242" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">prepended to prompt</text> <text x="340" y="304" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#B8651A">2-stage training</text> <text x="340" y="318" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">connector module</text>

<rect x="494" y="35" width="146" height="34" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5"/> <text x="567" y="51" text-anchor="middle" font-family="serif" font-size="11" fill="#2563EB">CLIP encoder</text> <text x="567" y="64" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">(frozen)</text> <line x1="567" y1="72" x2="567" y2="88" stroke="#6B6258" marker-end="url(#a3)"/> <rect x="494" y="88" width="146" height="30" fill="#FBE9CE" stroke="#B8651A"/> <text x="567" y="103" text-anchor="middle" font-family="serif" font-size="11" fill="#B8651A">Linear / 2-layer MLP</text> <text x="567" y="115" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#6B6258">W · Z_v</text> <line x1="567" y1="120" x2="567" y2="138" stroke="#6B6258" marker-end="url(#a3)"/> <rect x="494" y="138" width="146" height="148" fill="none" stroke="#1A1A1A" stroke-width="1.5"/> <text x="567" y="156" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">LLM (frozen, ph 1)</text> <g font-family="JetBrains Mono" font-size="9"> <rect x="504" y="166" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="523" y="166" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="542" y="166" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="561" y="166" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="580" y="166" width="18" height="14" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="599" y="166" width="18" height="14" fill="#F4EEE3" stroke="#1A1A1A" stroke-width="0.4"/> </g> <text x="567" y="206" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">vision tokens prepended</text> <text x="567" y="220" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">to text prompt</text> <text x="567" y="304" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#B8651A">single matrix W</text> <text x="567" y="318" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">simplest connector</text> </svg> <figcaption><strong>Fig 3.</strong> The same problem solved three ways. Flamingo inserts new gated cross-attention layers (rust, dashed) into a frozen LLM and starts them at zero with tanh(α=0). BLIP-2 hangs a 188M-parameter Q-Former between two frozen models, with 32 learned query embeddings. LLaVA throws away the connector machinery and just multiplies CLIP's output through an MLP into the LLM's token space. All three keep the LLM frozen during pretraining; the differences are about how vision gets into the prompt.</figcaption> </figure>

Flamingo (DeepMind, 2022) inserts new layers into the frozen LLM. Specifically, between every pretrained block, Flamingo adds a GATED XATTN-DENSE block: a cross-attention layer where the query is the LLM's current hidden state and the key/value are visual features (after passing through a Perceiver Resampler that compresses the variable-size feature grid into 64 fixed slots). The cross-attention output is gated by a learned scalar α, multiplied through tanh(α) where α is initialized to zero. At step 0, every new block is a no-op:

"We freeze the pretrained LM blocks, and insert gated cross-attention dense blocks between the original layers, trained from scratch. To ensure that at initialization, the conditioned model yields the same results as the original language model, we use a tanh-gating mechanism."

The training process pushes α off zero only as the cross-attention starts contributing useful signal. The LLM weights themselves are never updated. Catastrophic forgetting is averted by construction.

BLIP-2 (Salesforce, 2023) takes a different shape. Instead of adding cross-attention into the LLM, it adds a separate small module (a 188-million-parameter Querying Transformer, or Q-Former) between the frozen vision encoder and the frozen LLM. Q-Former has a fixed set of learned "query" embeddings that attend to the vision encoder's outputs through cross-attention. The output is a short fixed-length sequence (32 vectors in the released checkpoints) that gets linearly projected and prepended to the LLM's prompt as if it were ordinary text tokens. Q-Former is trained in two stages: first in isolation against an image-text alignment objective, then together with the frozen LLM against a generation loss.

"BLIP-2 bridges the modality gap with a lightweight Querying Transformer, which is pre-trained in two stages. The first stage bootstraps vision-language representation learning from a frozen image encoder. The second stage bootstraps vision-to-language generative learning from a frozen language model."

LLaVA (Microsoft and Wisconsin, 2023) is the simplest possible version of the same idea. Skip the cross-attention. Skip the Q-Former. Take the dense feature map from a frozen CLIP encoder, multiply it by a single trainable matrix W into the LLM's word-embedding dimension, and prepend the resulting vectors to the prompt. That's it.

"We consider a simple linear layer to connect image features into the word embedding space. Specifically, we apply a trainable projection matrix W to convert Z_v into language embedding tokens H_v, which have the same dimensionality as the word embedding space in the language model."

LLaVA-1.5 upgraded the projector from a single linear layer to a two-layer MLP and reported gains, but the architectural shape stayed the same. One small module. No new layers in the LLM. The LLM weights are still frozen during stage-1 alignment training; they do get fine-tuned during stage-2 instruction tuning, but they never see the gradient of a pretraining loss applied to a multimodal sequence, only an instruction-tuning loss applied at the end.

The pattern across all three: the LLM was trained on text alone. Vision is a side-loaded computation, translated into something that looks like text tokens (or, in Flamingo's case, attended to as cross-attention values), and the LLM consumes that translation. The LLM never learns the joint distribution over multimodal sequences during its own pretraining. Whatever multimodal understanding shows up has to come through the projector and the late-stage fine-tune.

A note on Qwen2.5-VL, which sometimes gets lumped in with this group. The Qwen team uses the LLaVA-style MLP projector pattern, but they train the LLM jointly with a from-scratch dynamic-resolution ViT. So Qwen2.5-VL inherits the connector design from LLaVA but the training pattern from the next section's native-multimodal approach.

6. Native multimodal: one decoder, one sequence

The structural alternative is to drop the "frozen LLM" constraint. Instead of bolting a vision adapter onto a finished language model, you train one transformer decoder on a single token stream that contains both modalities from the start. The LLM doesn't have to be told "now you're seeing an image" through a side-channel; the multimodal sequences are part of its pretraining distribution.

<text x="170" y="20" text-anchor="middle" font-family="serif" font-size="13" font-weight="600" fill="#1A1A1A">Adapter (LLaVA, Flamingo)</text>

<text x="35" y="44" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">PRETRAINING SEQUENCE</text> <g font-family="JetBrains Mono" font-size="10"> <rect x="35" y="52" width="32" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="51" y="66" text-anchor="middle" fill="#1A1A1A">the</text> <rect x="69" y="52" width="32" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="85" y="66" text-anchor="middle" fill="#1A1A1A">dog</text> <rect x="103" y="52" width="32" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="119" y="66" text-anchor="middle" fill="#1A1A1A">is</text> <rect x="137" y="52" width="32" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="153" y="66" text-anchor="middle" fill="#1A1A1A">on</text> <rect x="171" y="52" width="32" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="187" y="66" text-anchor="middle" fill="#1A1A1A">the</text> <rect x="205" y="52" width="38" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="224" y="66" text-anchor="middle" fill="#1A1A1A">sand</text> <rect x="245" y="52" width="22" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="256" y="66" text-anchor="middle" fill="#1A1A1A">.</text> </g> <text x="151" y="86" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">text tokens only</text>

<text x="35" y="110" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">INFERENCE SEQUENCE</text> <g font-family="JetBrains Mono" font-size="10"> <rect x="35" y="118" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="46" y="132" text-anchor="middle" fill="#2563EB">v1</text> <rect x="59" y="118" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="70" y="132" text-anchor="middle" fill="#2563EB">v2</text> <rect x="83" y="118" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="94" y="132" text-anchor="middle" fill="#2563EB">v3</text> <rect x="107" y="118" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="118" y="132" text-anchor="middle" fill="#2563EB">v4</text> <rect x="133" y="118" width="38" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="152" y="132" text-anchor="middle" fill="#1A1A1A">what</text> <rect x="173" y="118" width="22" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="184" y="132" text-anchor="middle" fill="#1A1A1A">is</text> <rect x="197" y="118" width="38" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="216" y="132" text-anchor="middle" fill="#1A1A1A">this</text> <rect x="237" y="118" width="22" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="248" y="132" text-anchor="middle" fill="#1A1A1A">?</text> </g> <line x1="80" y1="155" x2="80" y2="140" stroke="#B91C1C" stroke-width="0.8" stroke-dasharray="2,2"/> <text x="80" y="166" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#B91C1C">vision (novel modality)</text>

<rect x="35" y="190" width="270" height="40" fill="#1A1A1A" stroke="#B8651A" stroke-width="2"/> <text x="170" y="210" text-anchor="middle" font-family="serif" font-size="13" font-weight="700" fill="#F4EEE3">LLM (text-pretrained)</text> <text x="170" y="224" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#FEF3C7">+ vision adapter on top</text>

<text x="170" y="252" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">vision tokens are out-of-distribution</text> <text x="170" y="266" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">relative to the LLM's pretraining,</text> <text x="170" y="280" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">arriving through a thin projector</text>

<text x="510" y="20" text-anchor="middle" font-family="serif" font-size="13" font-weight="600" fill="#1A1A1A">Unified-decoder training</text>

<text x="375" y="44" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">PRETRAINING SEQUENCE</text> <g font-family="JetBrains Mono" font-size="10"> <rect x="375" y="52" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="386" y="66" text-anchor="middle" fill="#2563EB">v1</text> <rect x="399" y="52" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="410" y="66" text-anchor="middle" fill="#2563EB">v2</text> <rect x="423" y="52" width="32" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="439" y="66" text-anchor="middle" fill="#1A1A1A">the</text> <rect x="457" y="52" width="22" height="20" fill="#D1FAE5" stroke="#15803D" stroke-width="0.8"/><text x="468" y="66" text-anchor="middle" fill="#15803D">a1</text> <rect x="481" y="52" width="22" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="492" y="66" text-anchor="middle" fill="#1A1A1A">is</text> <rect x="505" y="52" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="516" y="66" text-anchor="middle" fill="#2563EB">v3</text> <rect x="529" y="52" width="22" height="20" fill="#D1FAE5" stroke="#15803D" stroke-width="0.8"/><text x="540" y="66" text-anchor="middle" fill="#15803D">a2</text> <rect x="553" y="52" width="38" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="572" y="66" text-anchor="middle" fill="#1A1A1A">sand</text> <rect x="593" y="52" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="604" y="66" text-anchor="middle" fill="#2563EB">v4</text> <rect x="617" y="52" width="22" height="20" fill="#D1FAE5" stroke="#15803D" stroke-width="0.8"/><text x="628" y="66" text-anchor="middle" fill="#15803D">a3</text> </g> <text x="507" y="86" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">interleaved text + image + audio tokens</text>

<text x="375" y="110" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">INFERENCE SEQUENCE</text> <g font-family="JetBrains Mono" font-size="10"> <rect x="375" y="118" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="386" y="132" text-anchor="middle" fill="#2563EB">v1</text> <rect x="399" y="118" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="410" y="132" text-anchor="middle" fill="#2563EB">v2</text> <rect x="423" y="118" width="38" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="442" y="132" text-anchor="middle" fill="#1A1A1A">what</text> <rect x="463" y="118" width="22" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="474" y="132" text-anchor="middle" fill="#1A1A1A">is</text> <rect x="487" y="118" width="22" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/><text x="498" y="132" text-anchor="middle" fill="#1A1A1A">?</text> <rect x="511" y="118" width="22" height="20" fill="#DBEAFE" stroke="#2563EB" stroke-width="0.8"/><text x="522" y="132" text-anchor="middle" fill="#2563EB">v3</text> <rect x="535" y="118" width="22" height="20" fill="#D1FAE5" stroke="#15803D" stroke-width="0.8"/><text x="546" y="132" text-anchor="middle" fill="#15803D">a1</text> </g> <text x="466" y="166" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#15803D">same shape, in-distribution</text> <line x1="466" y1="156" x2="466" y2="140" stroke="#15803D" stroke-width="0.8" stroke-dasharray="2,2"/>

<rect x="375" y="190" width="270" height="40" fill="#1A1A1A" stroke="#B8651A" stroke-width="2"/> <text x="510" y="210" text-anchor="middle" font-family="serif" font-size="13" font-weight="700" fill="#F4EEE3">LLM (trained on</text> <text x="510" y="224" text-anchor="middle" font-family="serif" font-size="13" font-weight="700" fill="#F4EEE3">multimodal sequences)</text>

<text x="510" y="252" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">multimodal sequences flow through</text> <text x="510" y="266" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">the LLM's full capacity during training</text> <text x="510" y="280" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">(joint pretraining or staged SFT)</text> </svg> <figcaption><strong>Fig 4.</strong> Two views of what the LLM was exposed to multimodal data through. Adapter style (left) trains on text only; vision tokens show up at inference as a different shape from anything the LLM saw before, arriving through a thin projector. Unified-decoder style (right) trains the LLM on interleaved multimodal sequences, either as joint pretraining (Gemini) or as staged SFT on a text-pretrained backbone (Nemotron 3 Nano Omni). Either way, the LLM's full capacity sees multimodal data during training, not just at instruction-tune time.</figcaption> </figure>

Two flavors of this exist, and the difference between them matters for what kinds of modalities are easy to add later.

The first flavor is discrete-token unified vocabulary, exemplified by Meta's Chameleon. Quantize images into discrete codes from a fixed codebook (Chameleon uses 8192 entries). Extend the text tokenizer's vocabulary to include those codes. Now an image is just a longer sequence of "tokens" drawn from a vocabulary that happens to be larger than a pure-text one. Train a standard autoregressive decoder on a mix of text-only sequences and interleaved text-and-image sequences, where the image segments are these new tokens. The model can read an image and write text, or read text and write image tokens, with no architectural difference between the two.

"Our unified approach uses fully token-based representations for both image and textual modalities. By quantizing images into discrete tokens, analogous to words in text, we can apply the same transformer architecture to sequences of both image and text tokens, without the need for separate image/text encoders or domain-specific decoders."

The second flavor is continuous projected embeddings. Instead of quantizing, run each modality through a modality-specific encoder that produces a sequence of continuous feature vectors, then project those vectors (via a learned MLP, the projector) into the LLM's hidden dimension, and concatenate them into the LLM's input sequence. The LLM still consumes a sequence of vectors, the same way it always does. The only difference is that some of those vectors came from a vision encoder or an audio encoder rather than from the text embedding table. Loss is computed on text outputs, but the gradient flows back through the LLM, the projectors, and (during the training stages where they're unfrozen) the encoders themselves, so all of the per-token text predictions are conditioned on the modality-projected positions appearing in the sequence.

This is the family Gemini, Qwen2.5-Omni, and Nemotron 3 Nano Omni belong to. Gemini frames it as native-from-scratch:

"The Gemini models are natively multimodal, as they are trained jointly across text, image, audio, and video."

Nemotron 3 Nano Omni names the design pattern explicitly:

"Our model follows an encoder-projector-decoder design, combining the Nemotron 3 Nano 30B-A3B language model with modality-specific encoders for vision and audio, connected via MLP projectors. ... Visual, audio, and text tokens are concatenated and fed to the LLM."

A subtle point worth nailing down. The "tokens" the LLM sees in this setup are not raw pixels or raw waveform samples. They are the outputs of modality-specific encoders, projected by an MLP into the LLM's embedding dimension. So when we say the LLM consumes "vision tokens," we mean vectors in the LLM's hidden space that happen to have come from a ViT plus a projector, not 256-dimensional image patches. The projector matters. The encoder matters. The LLM sees the same kind of vector regardless of where it came from, and that's what makes the unified-decoder approach work as a drop-in extension of transformer architecture.

Qwen2.5-VL fits here as a vision-only special case of the continuous-projected pattern. It uses an MLP-based vision-language merger (LLaVA-style spatial grouping plus a 2-layer MLP) and a redesigned ViT with windowed attention so that vision-encoder cost scales linearly with patch count. Qwen2.5-VL is a vision-language model, not a four-modality omni model, but architecturally it sits closer to Nemotron 3 Nano Omni's continuous-feature design than to LLaVA's frozen-LLM adapter.

Qwen2.5-Omni adds the most subtle detail. Its speech-generation head, called Talker, doesn't read a transcript from the central decoder ("Thinker"). It reads Thinker's hidden states directly:

"Talker directly receives high-dimensional representations from Thinker and shares all of Thinker's historical context information. ... The high-dimensional representations provided by Thinker implicitly convey [content's tone and attitude before the entire text is fully generated]."

This matters for the next act. When the speech-out side of an agent loop reads hidden states instead of transcribed text, things like prosody and intent never go through a text bottleneck.

7. The tradeoff

Adapter style is the cheaper path. You inherit a pretrained text LLM, you inherit a pretrained vision encoder, and you train a small connector module against an instruction-tuning dataset. Total training compute is dominated by the connector training and the second-stage fine-tune; both are small compared to a from-scratch LLM pretraining run. You can swap LLMs without touching the vision side, and the work to add the modality is bounded.

What you give up is depth of joint understanding. The LLM was trained on text. It learned how text sequences behave, how words constrain other words, how to follow instructions. It never learned what an image looks like in the sense of having to predict or condition on one across the multimodal pretraining distribution. Whatever cross-modal inference shows up has to come through the projector. For tasks where the answer requires fluently weaving visual evidence into a text response, this can be enough. For tasks where the visual structure has to drive the reasoning (long video, multi-page documents, fine-grained audio cues), the projector ends up as a thin pipe.

Unified-decoder style is more expensive. You pretrain (or substantially continue-train) a multi-billion-parameter LLM on multimodal sequences, with curated text+image, text+audio, and text+video data in the training mix and the cluster engineering to run it. You can't easily swap the LLM. You're committed to one stack.

What you buy is depth. The LLM weights learn to expect multimodal input during pretraining. Reasoning that has to mix modalities (answer this question by reading the chart on page 4 of the PDF, or by listening to the speaker's tone in the third minute of the video) gets to use the LLM's full capacity, not just the capacity of a thin connector. And every additional modality you add (audio, video, robotics action sequences) plugs into the same input stream rather than adding a parallel adapter side-channel.

Where Nemotron 3 Nano Omni sits is at the unified end of this dial. It is not pretrained from scratch on multimodal data; the technical report describes a multi-stage post-training pipeline that takes the existing Nemotron 3 Nano 30B-A3B base LLM and runs it through seven SFT stages plus an RL phase, several of which unfreeze all parameters and train them jointly with the encoders and projectors over mixed-modality sequences. The HF model card's modality rollup gives a sense of scale:

"354,587,705 data points (~717.0B tokens) ... text+audio: 259,178,821 samples (~143,533.1M tokens); text+image: 70,143,901 samples (~180,347.1M tokens); text+video: 15,837,673 samples (~239,631.5M tokens); text+video+audio: 8,720,044 samples (~152,499.2M tokens); text: 707,187 samples (~958.4M tokens)."

The 30B-A3B backbone itself was pretrained as a text-first model; the omni capabilities arrive through staged mixed-modality post-training rather than through a from-scratch native-multimodal pretraining run (Gemini's framing). What's load-bearing is the shape of the mix: text+image, text+audio, and text+video sequences all show up with the LLM in the loss, across many billions of tokens.

Compare to an adapter-style model where a small instruction-tuning dataset sits on top of a frozen text-pretrained LLM. The two regimes train on different distributions. Adapter models work well when the multimodal signal the LLM needs to use is something a thin projector can encode and the LLM's text-only weights can interpret as token-shaped input. They tend to struggle when the input drifts further out of distribution from anything the LLM saw during pretraining (long video, dense documents, audio whose information lives in prosody more than wording), because the projector becomes a bottleneck on what the LLM can possibly attend to. Unified-decoder models carry that distribution into the LLM's training (whether as native pretraining or as staged SFT), so the bottleneck moves.

Act 3: the other senses

Why audio and video aren't more SigLIPs.

8. Audio is time-frequency, not pixel grid

The first thing to notice about audio is that the natural input format is not 1D and not pixel-shaped. A microphone produces a time series of pressure values (a waveform), but modern audio models almost always convert the waveform into a 2D representation, called a log-mel spectrogram, before any neural network touches it.

<text x="150" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">RAW WAVEFORM</text> <rect x="30" y="40" width="240" height="160" fill="#F9F5EB" stroke="#C9BEAA"/> <line x1="30" y1="120" x2="270" y2="120" stroke="#C9BEAA" stroke-width="0.5" stroke-dasharray="2,2"/>

<text x="150" y="240" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">16,000 samples / second</text>

<line x1="282" y1="120" x2="368" y2="120" stroke="#6B6258" stroke-width="1.5" marker-end="url(#a5)"/> <text x="325" y="105" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">25 ms window,</text> <text x="325" y="118" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">10 ms stride,</text> <text x="325" y="139" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">80 mel bins, log</text>

<text x="500" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">LOG-MEL SPECTROGRAM</text> <rect x="380" y="40" width="240" height="160" fill="#F4EEE3" stroke="#8F8578"/>

<text x="500" y="240" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">100 time frames × 80 mel bins</text> </svg> <figcaption><strong>Fig 5.</strong> A 1-second utterance and the log-mel tile every modern audio model turns it into. Left: the microphone signal, sampled 16,000 times per second. Right: what comes out after a sliding 25 ms window, an 80-bin mel filterbank, and a per-bin log. Rows are mel-spaced frequency channels (low at the bottom, high at the top); columns are 10 ms time frames. Time means the same thing on both axes; the vertical axis on the right has no spatial locality.</figcaption> </figure>

A spectrogram is what you get when you take a sliding window of, say, 25 milliseconds across the waveform, and for each window compute the energy of the signal at each of a fixed set of frequency bins. The "mel" part means the frequency bins are spaced on the perceptual mel scale rather than linearly, because human auditory perception is approximately log-frequency. The "log" means you take the log of the energy in each bin, because perception is approximately log-amplitude. You end up with a 2D matrix whose rows are frequency bins (typically 80 of them) and whose columns are time frames (typically one every 10 milliseconds).

Whisper uses exactly this representation, and it has become the de facto convention:

"All audio is re-sampled to 16,000 Hz, and an 80-channel log-magnitude Mel spectrogram representation is computed on 25-millisecond windows with a stride of 10 milliseconds."

So a 30-second audio clip becomes an 80×3000 matrix. A Conv1D stem then downsamples 2× along the time axis and projects the 80 mel bins into the model hidden dimension, producing a 1500-step sequence of model-dimension vectors that gets fed into a Transformer. The frequency axis is folded into channels at the stem rather than carried through as 80 separate rows; what survives downstream is a sequence-of-vectors view of the audio.

Here is why this matters for the omni-modal architecture. A vision encoder like C-RADIOv4 takes a 2D pixel grid and produces a sequence of patch tokens. The 2D grid has spatial locality: nearby patches depict the same object, lighting, texture. The encoder's inductive biases (patch tokenization, 2D rotary embeddings, sometimes window attention) are tuned for that.

A spectrogram is also a 2D grid, but the two axes mean different things. The horizontal axis is time, with strong sequential structure: what happens at second 10 is causally tied to what happened at second 9. The vertical axis is frequency channel, with no spatial locality at all: bin 5 (a low-frequency band, maybe 100 Hz) and bin 6 (slightly higher, maybe 150 Hz) describe related parts of the signal, but bin 5 and bin 78 describe completely different aspects of the same signal at the same time.

A vision ViT applied naively to a spectrogram would treat both axes the same and would miss the structure on both. You want temporal smoothing along the time axis (which is what attention plus convolutional downsampling buys) and you want every frequency bin to be visible to every step of the encoder, not split across patches. Audio encoders therefore use Conformer-style architectures, which interleave local convolutions with attention, instead of plain ViTs; they project the frequency axis into model channels at the stem rather than tokenizing it spatially, and they downsample aggressively along time.

C-RADIOv4 cannot just be reused for audio. The inductive biases are wrong. You need a different encoder built for this representation.

9. Parakeet: a token-emitting audio encoder

Nemotron 3 Nano Omni's audio encoder is Parakeet-TDT-0.6B-v2, an NVIDIA model that pairs a FastConformer encoder with a Token-and-Duration Transducer (TDT) decoder. 600 million parameters total. Its model card describes the shape:

"Architecture Type: FastConformer-TDT ... This model was developed based on FastConformer encoder architecture and TDT decoder. This model has 600 million model parameters."

In its original ASR setting (automatic speech recognition), Parakeet ingests a log-mel spectrogram and emits a transcript. The encoder is the part that produces a sequence of audio feature vectors, downsampled 8× along the time axis from the spectrogram input. The TDT decoder is what turns those features into characters. Together they're a full speech-to-text model.

In the omni-modal setting, the TDT decoder is thrown away. Nemotron uses only Parakeet's encoder, treating it as a feature extractor whose outputs pass through a 2-layer MLP projector and get concatenated into the LLM's input sequence:

"The audio side is powered by Parakeet-TDT-0.6B-v2, connected to the backbone through its own 2-layer MLP projector. Audio is sampled at 16 kHz, and the model is trained with inputs up to 1,200 seconds (20 minutes), while the LLM max context length supports 5+ hours."

<text x="50" y="32" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">INPUT</text> <rect x="20" y="110" width="60" height="60" fill="#F4EEE3" stroke="#8F8578"/> <rect x="20" y="155" width="60" height="15" fill="#B8651A" opacity="0.32"/> <rect x="22" y="120" width="14" height="32" fill="#B8651A" opacity="0.22"/> <rect x="40" y="125" width="12" height="28" fill="#B8651A" opacity="0.20"/> <rect x="56" y="118" width="14" height="38" fill="#B8651A" opacity="0.24"/> <rect x="22" y="113" width="3" height="18" fill="#B8651A" opacity="0.5"/> <rect x="56" y="115" width="3" height="14" fill="#B8651A" opacity="0.5"/> <text x="50" y="186" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">log-mel</text> <text x="50" y="200" text-anchor="middle" font-family="JetBrains Mono" font-size="9" fill="#6B6258">80×100</text>

<text x="155" y="86" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">SHARED ENCODER</text> <rect x="105" y="95" width="100" height="90" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5"/> <text x="155" y="118" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#2563EB">FastConformer</text> <text x="155" y="132" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#2563EB">encoder</text> <text x="155" y="151" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">600M params</text> <text x="155" y="164" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">8× temporal</text> <text x="155" y="176" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">subsample</text>

<g font-family="JetBrains Mono" font-size="9"> <rect x="230" y="125" width="14" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="246" y="125" width="14" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="262" y="125" width="14" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="278" y="125" width="14" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> <rect x="294" y="125" width="14" height="20" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.6"/> </g> <text x="270" y="160" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">~12.5 features/sec</text>

<text x="375" y="40" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258" opacity="0.55">DISCARDED IN OMNI-MODAL</text>

<g opacity="0.5"> <path d="M 270 122 Q 280 90 305 75 L 320 75" fill="none" stroke="#6B6258" stroke-width="1" stroke-dasharray="3,3" marker-end="url(#a6)"/> <rect x="325" y="55" width="100" height="44" fill="#EDE5D4" stroke="#8F8578" stroke-width="1.2" stroke-dasharray="4,3"/> <text x="375" y="75" text-anchor="middle" font-family="serif" font-size="11" fill="#6B6258">TDT decoder</text> <text x="375" y="91" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">(thrown away)</text> <line x1="430" y1="77" x2="448" y2="77" stroke="#6B6258" stroke-dasharray="3,3" marker-end="url(#a6)"/> <rect x="450" y="62" width="84" height="30" fill="#F9F5EB" stroke="#8F8578" stroke-dasharray="3,2"/> <text x="492" y="82" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#6B6258">"hello..."</text> <text x="544" y="80" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">(original ASR path)</text> </g>

<rect x="325" y="180" width="100" height="40" fill="#FBE9CE" stroke="#B8651A" stroke-width="1.5"/> <text x="375" y="198" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#B8651A">2-layer MLP</text> <text x="375" y="212" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">→ LLM hidden dim</text>

<rect x="450" y="180" width="200" height="40" fill="#1A1A1A" stroke="#B8651A" stroke-width="2"/> <text x="550" y="198" text-anchor="middle" font-family="serif" font-size="13" font-weight="700" fill="#F4EEE3">LLM input sequence</text> <text x="550" y="212" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#FEF3C7">audio + text + image tokens</text>

<text x="375" y="240" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">OMNI-MODAL HEAD</text> </svg> <figcaption><strong>Fig 6.</strong> What Nemotron actually keeps from Parakeet. The original Parakeet-TDT speech-recognition model is a FastConformer encoder feeding a Token-and-Duration Transducer that writes out a transcript. In omni-modal use, only the encoder runs; the TDT decoder is discarded, and the encoder's audio features (8× temporal subsampling of a 100 Hz log-mel spectrogram, so the encoder emits ~12.5 features per second of audio) pass through a 2-layer MLP projector into the LLM's input sequence instead.</figcaption> </figure>

This is the architectural counterpart of what happens on the vision side. C-RADIOv4-H is a vision encoder; its outputs flow through a projector into the LLM. Parakeet's encoder is an audio encoder; its outputs flow through a projector into the same LLM. Two MLPs, two encoders, one decoder.

The choice to use Parakeet's encoder rather than Whisper's encoder is itself worth a moment. Whisper is the more famous open speech model, and either encoder produces a log-mel-conditioned feature sequence that an LLM can in principle consume. The technical report and model card don't justify the choice in detail, so this post won't either; the architectural pattern (encoder plus projector into LLM) is what matters, and Parakeet versus Whisper sits below that as an implementation detail.

10. The cost of stitched chains

The alternative to using Parakeet's encoder this way is the older "stitched chain" pattern: run Whisper end-to-end (encoder plus decoder), produce a transcript, feed the transcript text to the LLM. This was the dominant approach for any system needing to handle audio before native multimodal LLMs landed, and it still appears in production stacks today.

It has three problems, in increasing order of how subtle they are.

First, repetition loops. Whisper's own paper documents the failure mode in §3.8:

"Whisper models are trained on 30-second audio chunks and cannot consume longer audio inputs at once... Transcribing long-form audio using Whisper relies on accurate prediction of the timestamp tokens to determine the amount to shift the model's 30-second audio context window by, and inaccurate transcription in one window may negatively impact transcription in the subsequent windows. We have developed a set of heuristics that help avoid failure cases of long-form transcription... we use beam search with 5 beams using the log probability as the score function, to reduce repetition looping which happens more frequently in greedy decoding."

The OpenAI authors are saying, plainly, that long-form Whisper occasionally falls into repetition loops in greedy decoding and needs hand-tuned heuristics to avoid window-to-window error compounding. When the LLM downstream consumes the transcript, those errors land in its context as if they were ground truth. There is no posterior signal to back out from.

Second, lost paralinguistic information. Speech carries content beyond the words: prosody, stress, intonation, speaker identity, disfluency, overlap. A transcript collapses all of it to a string. Tsiamas et al. ran a controlled benchmark of cascade vs end-to-end speech translation systems and found that cascades transmit prosody to a lesser extent than end-to-end systems, with the gap depending on what surface form the transcriber chooses:

"The prosody of a spoken utterance, including features like stress, intonation and rhythm, can significantly affect the underlying semantics, and as a consequence can also affect its textual translation."

A canonical example from their paper: "These are GERMAN teachers" (teachers from Germany) and "These are German TEACHERS" (teachers of German) are different statements that map to the same transcript if the transcriber doesn't capture the stress. The cascade has no way to recover what was lost; the end-to-end architecture can in principle preserve it.

Third, latency hop. Whisper has to finish a chunk before the LLM sees anything from that chunk. The architecture forces an extra serial step: any cascade with an autoregressive ASR head must commit to a transcript before the downstream model sees the input. There is no controlled benchmark for this in the agent-loop literature, so the claim stays qualitative; what's load-bearing is that the cascade introduces a latency hop the unified design doesn't, not any specific number.

Qwen2.5-Omni's Talker is the architectural answer in the other direction. Instead of generating speech via a text-to-speech head that reads a transcript from the LLM, Talker reads the LLM's hidden states directly:

"Talker directly receives high-dimensional representations from Thinker and shares all of Thinker's historical context information. ... The high-dimensional representations provided by Thinker implicitly convey [content's tone and attitude before the entire text is fully generated]."

The point is the same as on the input side. Any time speech information has to round-trip through a text bottleneck, you lose what couldn't be written down. Native multimodal models avoid the round-trip by sharing hidden states across the modality boundary.

That covers audio. Video has its own problem, and it's a bigger one.

11. Video is spatiotemporal redundancy

A 2D image with 16×16 patches at 384×384 resolution is around 576 tokens. A modest number; an LLM with a 32k context window has room for it. Now stretch the same model to video.

A video is a sequence of frames over time. The naive thing to do is treat each frame as an image, run the ViT, and concatenate the resulting tokens into one long sequence. The math, even at modest settings, is bad. The EVS paper opens with the cleanest version of this problem statement:

"a two-minute video at 24 FPS produces more than two million vision tokens, far beyond the effective context length of most language models"

Drag the duration slider above. Even at modest patch counts, the token total flies past a 262K-context LLM the moment the clip gets past about ten seconds. At 1024 patches per frame and 30 FPS, you're already over 262K at nine seconds. That's the cost ViViT-style naive frame-by-frame encoding pays, and it makes long-form video reasoning impossible without aggressive compression.

The good news, made vivid by VideoMAE in 2022, is that video is wildly redundant along the time axis. VideoMAE found that you can mask out 90 to 95 percent of the patches in a video and still reconstruct it, where the equivalent ratio for static images is around 75 percent.

"VideoMAE is in favor of extremely high masking ratios (e.g. 90% to 95%) compared with the ImageMAE."

The intuition is that two consecutive frames usually share most of their content. Static parts of a scene (the background, the unchanging foreground object, the camera angle) all contribute redundant patches at every frame. You don't need to encode them N times. You need to encode the parts that change, plus enough of the static structure to anchor where they change against.

ViViT (Google, 2021) was the first widely-used architecture to bake this into the encoder. Instead of tokenizing each frame independently, ViViT extracts non-overlapping spatio-temporal "tubes" that span both spatial extent and a small number of consecutive frames:

"Extract non-overlapping, spatio-temporal 'tubes' from the input volume, and to linearly project this to ℝ^d ... nt=⌊T/t⌋, nh=⌊H/h⌋ and nw=⌊W/w⌋, tokens are extracted from the temporal, height, and width dimensions respectively."

A single token now represents, say, a 2×16×16 chunk of video (two consecutive frames worth of one spatial patch). Tubelet tokenization halves the token count along the time axis at the input stage. Variants exist (TimeSformer's factorized space-time attention, MViT's pyramidal pooling) but the underlying observation is the same: time-axis redundancy is so high that you can collapse along it without losing the signal that matters.

TimeSformer makes the cost story explicit:

"In practice, joint space-time attention causes a GPU memory overflow once the spatial frame resolution reaches 448 pixels, or once the number of frames is increased to 32."

You can't stack a vanilla ViT and run full self-attention on a video; the (NF)² cost dominates the kernel. You either factorize attention (do spatial attention within frames, then temporal attention across frames) or you compress at the input (tubelets). Or both.

Tubelet encoding gets you a 2× reduction. Good but not enough to make a 2-minute video fit in a 262K-context LLM. You need another order of magnitude. That's what the next section is about.

12. Efficient Video Sampling

The trick that closes the gap is straightforward. If patches at the same spatial location in consecutive frames are nearly identical (because nothing in the scene changed there), drop the later one. Keep only patches where something changed. This is what the EVS paper proposes.

The intuition is easiest with raw pixels. Take a frame at time t and a frame at time t−1. For each spatial patch position p, compute the L1 distance between the two patches:

$$ D_{p,t} = |p_t - p_{t-1}|_1 $$

You now have a 2D map of "how much did this patch change between these two frames." Collect these distances across the whole clip, pick the q-th percentile, and use that as a threshold. Drop every patch whose distance is below the threshold:

"For every patch $p$ at time $0 < t \leq T$, compute $D_{p,t} = |p_t - p_{t-1}|1$, and denote ${D_t}$ as the differences of all patches between frames $t-1$ and $t$." "For each frame collect ${D_t}{t=1}^T$ and compute sequence-level cut-off threshold $d$ as $q$-th percentile, where $q$ is a user-selected pruning rate." "For all patches in the consecutive frames, keep those that satisfy $D_{p,t} \geq d$; this defines the binary mask $M_t$."

That's the algorithm. q is user-tunable: q=0 keeps everything; q=0.75 keeps the top 25% of changing patches; q=0.95 keeps almost nothing. The first frame is always kept whole, since there's no t−1 to diff against.

Drag the q slider. Notice what survives. The patches that get kept are the parts of the scene that are actually changing: the moving objects, the camera-relative motion, the cut transitions. The patches that get dropped are the static background already represented in earlier frames. The intuition is that most of what you drop is redundancy. The cost is real but small; the EVS authors report "up to 4×" speedups with "minimal accuracy loss," and on Video-MME at q=0.75 with uptraining the score drops by about 0.80 points (65.50 → 64.70 on 32-frame; see §12). Pruning is not lossless, just cheap relative to the compute it saves.

There's a wrinkle. Naively dropping tokens can disturb how downstream position embeddings interpret the resulting sequence, since they expect a contiguous progression of token indices. EVS's authors describe their algorithm as preserving positional identity:

"EVS preserves positional identity, requires no architectural changes or retraining."

The implementation detail varies. The EVS paper's framing is that surviving patches retain their original spatial-temporal indices, so position embeddings stay faithful. Nemotron 3 Nano Omni's public code applies a boolean retention mask to the post-projection token sequence ("inputs_embeds" in the HF model), which is a related but more pedestrian implementation: the kept feature vectors are concatenated and fed to the LLM in their original order, without re-indexing or re-positioning. Either way, no architectural change to the LLM is needed and the post-prune sequence stays a drop-in replacement for the original.

A separate EVS-paper detail is optional uptraining. The EVS authors sample q from a beta distribution during mini-batches so an uptrained model can tolerate a continuum of compression ratios:

"the pruning rate q is sampled from a beta distribution for every mini-batch. The model thus learns to be invariant to a continuum of compression ratios"

NVIDIA's Nemotron report does not claim this recipe for Nano Omni; it describes EVS as a runtime-only feature and evaluates q at inference (Table 12 fixes q=0.5, Table 13 sweeps q). So for Nemotron, treat q as an inference knob the report benchmarks, not as a compression level the omni checkpoint necessarily saw during training.

Second, the production deployment runs EVS in embedding space, not raw RGB. The simple RGB intuition above is exactly that, an intuition. The EVS paper itself runs and uptrains the algorithm on encoded feature similarity rather than raw pixels. In Nemotron 3 Nano Omni's specific implementation, the report and the public modeling code place EVS post-adapter. Pruning operates on the video token embeddings after Conv3D tubelet, pixel shuffle, and the MLP projector, immediately before the LLM. Raw RGB L1 over-weights pixel differences that are visually salient but semantically unimportant (lighting drift, sensor noise, mild camera jitter) and under-weights changes that are small in pixels but large in meaning (a face turning, a hand crossing into frame). Computing the threshold on encoded features fixes both directions because the encoder has already discounted nuisance variation. The algorithm structure is the same; only the input space changes.

Reported results, kept separate because they measure different things. The EVS paper reports a general "up to 4× LLM TTFT speedup with minimal accuracy loss," varying by model and q. Their Video-MME table at q=0.75 with uptraining shows an absolute 0.80-point drop on the 32-frame setting (65.50 → 64.70, equivalent to about 1.2% relative). The two numbers measure different things; don't bundle them into "4× at q=0.75."

EVS plus the tubelet encoding from §11 is what makes long video viable in a 262K-context LLM. Nemotron composes them in a specific order.

13. How Nemotron stacks the two

Nemotron 3 Nano Omni's video path runs the reductions in a specific order. First, frames are fused into tubelets at the input layer of the vision encoder (the ViViT idea). The vision encoder runs over the tubelets and emits patch features; those features go through the 4× pixel-shuffle reduction and the MLP projector to land in the LLM's hidden dimension. EVS then prunes the post-projection token sequence at inference time on encoded-feature similarity, immediately before the LLM:

"For video, Nemotron 3 Nano Omni uses a dedicated Conv3D tubelet embedding path. Instead of embedding each frame independently, every pair of consecutive frames is fused into a single 'tubelet' before the ViT, halving the number of vision tokens the language model has to attend to." "EVS is an important feature, enabled during inference time, that drops redundant video tokens after the vision encoder. ... The first frame of the video is kept entirely, then for each subsequent frame, EVS keeps the 'dynamic' tokens where the video is changing and drops the 'static' ones where nothing has changed from the previous frame."

<text x="340" y="20" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">VIDEO TOKEN BUDGET (30 S @ 24 FPS, 1024 PATCHES/FRAME)</text>

<rect x="20" y="55" width="55" height="40" fill="#F4EEE3" stroke="#8F8578"/> <rect x="25" y="60" width="55" height="40" fill="#F4EEE3" stroke="#8F8578"/> <rect x="30" y="65" width="55" height="40" fill="#EDE5D4" stroke="#8F8578"/> <rect x="30" y="65" width="55" height="14" fill="#93C5FD" opacity="0.6"/> <rect x="30" y="79" width="55" height="12" fill="#D1AE7A" opacity="0.7"/> <rect x="30" y="91" width="55" height="14" fill="#059669" opacity="0.5"/> <text x="55" y="125" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">raw frames</text> <text x="55" y="143" text-anchor="middle" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#1A1A1A">737,280</text> <text x="55" y="158" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">naive baseline</text>

<rect x="128" y="50" width="90" height="70" fill="#FBE9CE" stroke="#B8651A" stroke-width="1.5"/> <text x="173" y="73" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#B8651A">Conv3D</text> <text x="173" y="87" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#B8651A">tubelet</text> <text x="173" y="105" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">2 frames → 1 token</text> <text x="173" y="143" text-anchor="middle" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#1A1A1A">368,640</text> <text x="173" y="158" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">after time fusion</text>

<rect x="251" y="50" width="90" height="70" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5"/> <text x="296" y="73" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#2563EB">ViT</text> <text x="296" y="87" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#2563EB">C-RADIOv4-H</text> <text x="296" y="105" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">vision encoder</text> <text x="296" y="143" text-anchor="middle" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#1A1A1A">368,640</text> <text x="296" y="158" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">passes through</text>

<rect x="374" y="50" width="124" height="70" fill="#FBE9CE" stroke="#B8651A" stroke-width="1.5"/> <text x="436" y="73" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#B8651A">pixel-shuffle ×4</text> <text x="436" y="87" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#B8651A">+ EVS @ q=0.5</text> <text x="436" y="105" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">prune redundant 50%</text> <text x="436" y="143" text-anchor="middle" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#15803D">46,080</text> <text x="436" y="158" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#15803D">16× total reduction</text>

<rect x="531" y="50" width="129" height="70" fill="#1A1A1A" stroke="#B8651A" stroke-width="2"/> <text x="595" y="74" text-anchor="middle" font-family="serif" font-size="13" font-weight="700" fill="#F4EEE3">LLM</text> <text x="595" y="90" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#FEF3C7">Nemotron 30B-A3B</text> <text x="595" y="106" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#FEF3C7">262,144-token context</text>

<text x="595" y="143" text-anchor="middle" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#15803D">fits</text> <text x="595" y="158" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">18% of context budget</text>

<rect x="40" y="190" width="500" height="14" fill="#EDE5D4" stroke="#8F8578" stroke-width="0.5"/> <rect x="40" y="190" width="250" height="14" fill="#B8651A" opacity="0.45"/> <rect x="40" y="190" width="31" height="14" fill="#15803D" opacity="0.7"/> <line x1="178" y1="190" x2="178" y2="204" stroke="#1A1A1A" stroke-width="1" stroke-dasharray="2,2"/> <text x="178" y="216" text-anchor="middle" font-family="JetBrains Mono" font-size="8" fill="#6B6258">262K context</text>

<text x="555" y="200" font-family="JetBrains Mono" font-size="8" fill="#6B6258">naive 737K</text> <text x="555" y="211" font-family="JetBrains Mono" font-size="8" fill="#B8651A">+ tubelet 369K</text> <text x="555" y="222" font-family="JetBrains Mono" font-size="8" fill="#15803D">+ shuffle + EVS 46K</text> </svg> <figcaption><strong>Fig 9.</strong> The three compressions Nemotron stacks for video. Conv3D tubelet fusion at the input layer halves the token count along time (Nemotron 3 Nano Omni report, §2). The vision encoder runs over the fused tubelets and emits patch features. The Nemotron pipeline then applies a 4× pixel-shuffle reduction prior to projection ("Prior to projection, we apply a pixel shuffle operation with 4× downsampling to reduce the token count presented to the language model," §2). EVS is an optional inference-time pass that sits between the vision adapter and the LLM, dropping post-projection tokens that didn't change much from the previous frame; the report's published inference-efficiency setup runs EVS at q=0.5. Cascade total is ×0.5 × ×0.25 × ×0.5 = ×0.0625: a 30-second clip whose naive token count is 737,280 lands at 46,080 by the time it hits the LLM, well inside the 262,144-token context window. The cascade reaches ~23K if you crank EVS to q=0.75, but that isn't the Nemotron-deployed setting.</figcaption> </figure>

Three compressions compose multiplicatively in Nemotron's actual pipeline. Tubelet halves the token count along time. A 4× pixel-shuffle reduction sits between the vision encoder and the projector, dropping the per-frame post-encoder count by another factor of four. EVS, the optional inference-time pruner, sits between the vision adapter and the LLM and drops post-projection tokens that didn't change much from the previous frame; the report's published inference-efficiency setup runs EVS at q=0.5, so it removes about half of what's left. Together you're at roughly 1/16 of the naive frame-by-frame token total. For an illustrative 30-second clip at 24 FPS and 1024 patches per frame, the naive count is 30 × 24 × 1024 = 737,280 tokens. After tubelet fusion (×0.5), pixel-shuffle (×0.25), and EVS at q=0.5 (×0.5), you're at roughly 46,000 tokens. (Production deployments don't sample at 24 FPS in any case; the HF serving guidance recommends ~2 FPS / 256 frames as a default. The 24-FPS budget is a worst-case stress test for the compression chain, not the default ingestion path.) Crank EVS to q=0.75 (the EVS paper's headline setting) and the same 30-second clip lands at ~23K tokens; the report doesn't ship that as the default but the runtime knob is there.

For images (single frames, no temporal axis), the pipeline simplifies. Tubelet fusion isn't applicable; only the spatial ViT runs. The patch count varies with resolution:

"Each image can be represented using a variable number of 16 x 16 patches, with a minimum of 1,024 to a maximum of 13,312 visual patches per image."

A 13,312-patch image at 16-pixel patches is roughly 3.4 megapixels of input area (13,312 × 256 = 3,407,872 pixels), enough resolution for document understanding or fine-grained chart reading. The 1,024-patch lower bound (about 0.26 MP) is for thumbnail-sized images where dense detail isn't relevant.

Act 4: the inference math

The Nemotron-specific efficiency receipts, and what they actually show.

There's no controlled benchmark at the primary-source bar that proves "unified-decoder beats fragmented chains" in general. What we do have are three measured efficiency wins on specific models against specific baselines, plus the architectural reasons each one shows up. The post reports the numbers, names the baselines, and stops there.

14. The hybrid backbone

Nemotron 3 Nano Omni's language model is roughly 30 billion parameters total but only activates a few billion of them per input token. The omni model card rounds these as 31B / ~3B active; the underlying Nemotron 3 Nano backbone card the omni inherits from is more precise about the architecture, and that's where the layer counts and routing details below come from:

"The model employs a hybrid Mixture-of-Experts (MoE) architecture, consisting of 23 Mamba-2 and MoE layers, along with 6 Attention layers. Each MoE layer includes 128 experts plus 1 shared expert, with 6 experts activated per token. The model has 3.5B active parameters and 30B parameters in total."

Three things are happening here. The backbone is a hybrid of three layer types: Mamba-2 selective state-space layers, grouped-query attention layers, and Mixture-of-Experts feedforward layers. It uses sparse activation: of the 128 experts in each MoE layer, only the 6 the router picks (plus a shared expert that's always on) get computed for each token. And the layer counts are uneven: 23 Mamba-2 + 6 attention + 23 MoE = 52 total layers, with attention only ~11.5% of the stack.

<text x="340" y="20" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">52 LAYERS, INPUT TO OUTPUT (ORDERING ILLUSTRATIVE)</text>

<rect x="20" y="130" width="200" height="92" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5" rx="3"/> <text x="28" y="150" font-family="serif" font-size="12" font-weight="600" fill="#2563EB">Mamba-2 ×23</text> <text x="28" y="170" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">selective state-space,</text> <text x="28" y="183" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">O(N) cost in sequence</text> <text x="28" y="196" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">length, no attention</text>

<rect x="240" y="130" width="200" height="92" fill="#FBE9CE" stroke="#B8651A" stroke-width="1.5" rx="3"/> <text x="248" y="150" font-family="serif" font-size="12" font-weight="600" fill="#B8651A">GQA attention ×6</text> <text x="248" y="170" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">~11.5% of the stack;</text> <text x="248" y="183" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">kept for the kind of</text> <text x="248" y="196" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">exact-position lookup</text> <text x="248" y="207" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">attention is good at</text>

<rect x="460" y="130" width="200" height="92" fill="#D1FAE5" stroke="#15803D" stroke-width="1.5" rx="3"/> <text x="468" y="150" font-family="serif" font-size="12" font-weight="600" fill="#15803D">MoE feedforward ×23</text> <text x="468" y="170" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">128 routed experts,</text> <text x="468" y="183" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">top-6 active per token,</text> <text x="468" y="196" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">+ 1 always-on shared expert</text>

<line x1="20" y1="232" x2="660" y2="232" stroke="#C9BEAA"/> <text x="20" y="250" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#1A1A1A">30B parameters total</text> <text x="340" y="250" text-anchor="middle" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#6B6258">→</text> <text x="660" y="250" text-anchor="end" font-family="JetBrains Mono" font-size="11" font-weight="600" fill="#15803D">3.5B active per token</text> <text x="340" y="270" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">model card states the counts and the routing top-k, not the within-stack ordering of the attention layers</text> </svg> <figcaption><strong>Fig 10.</strong> The 52-layer hybrid backbone. Most of the stack alternates Mamba-2 (blue) selective state-space layers with MoE (green) feedforward layers; six grouped-query attention layers (rust) sit interspersed at roughly even spacing. Where each attention layer specifically falls is illustrative, not authoritative; the model card pins the counts and the routing but not the placement.</figcaption> </figure>

The model card states the counts and the routing top-k. It does not state the exact within-stack ordering of attention layers. NVIDIA's earlier Nemotron-H paper places attention layers "evenly dispersed throughout the model," but Nemotron-H is a different model and a literal copy of that arrangement isn't quoted for the omni model anywhere I could find. The figure draws an evenly dispersed layout because that's a reasonable default and matches the family pattern. The placement is illustrative, not authoritative.

A note worth pinning down: the routing is per token, not per modality. The same router sees text tokens, vision tokens, audio tokens (all of them, after projection, are vectors in the LLM's hidden space), and selects 6 experts for each one based on the token's content. Experts can and do specialize on patterns that show up disproportionately in one modality, but that specialization is learned, not architecturally enforced. There is no "vision gate" or "audio gate." The marketing material occasionally suggests there is. The published primary sources don't.

15. Why Mamba

The reason a hybrid uses Mamba layers at all is that long-context prefill is what kills attention.

Self-attention has prefill cost proportional to sequence length squared (every token attends to every other). For a 1K-token context, this doesn't matter; for a 256K-token context, it dominates time-to-first-token. Mamba is what you get when you keep the residual-stream architecture of a transformer but replace the attention layer with a selective state-space model, which processes the same context in cost proportional to sequence length. Linear, not quadratic. (Decode-step cost with a KV cache is a different story for both, scaling with the cached length per generated token; the prefill gap is what dominates 262K-context inference.)

"Letting the SSM parameters be functions of the input addresses their weakness with discrete modalities, allowing the model to selectively propagate or forget information." "Mamba enjoys fast inference (5× higher throughput than Transformers) and linear scaling in sequence length."

What "selective" means is that the state-update equations have parameters that depend on the current input token, instead of being shared across all positions. This is what lets Mamba pick what to remember and what to drop, with the input deciding rather than a fixed weight. The trade-off is that the recurrence is no longer the kind of linear time-invariant system you can compute with a single FFT-based convolution; you need a hardware-aware scan kernel to keep training fast.

Mamba-2 (Dao and Gu, 2024) shows that this scan and a restricted form of attention turn out to be the same matrix-multiplication structure under the hood, which the authors call structured state-space duality. Mamba-2's core layer is 2-8× faster than the original Mamba's:

"Our state space duality (SSD) framework allows us to design a new architecture (Mamba-2) whose core layer is a refinement of Mamba's selective SSM that is 2-8X faster, while continuing to be competitive with Transformers."

<text x="340" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">RELATIVE COMPUTE COST vs CONTEXT LENGTH (log–log)</text>

<text x="360" y="258" text-anchor="middle" font-family="serif" font-size="11" fill="#6B6258">context length (tokens)</text> <text x="80" y="42" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">cost (×1K-token baseline)</text>

<text x="555" y="170" font-family="serif" font-size="13" font-style="italic" fill="#2563EB">Mamba</text> <text x="555" y="184" font-family="JetBrains Mono" font-size="10" fill="#2563EB">O(N)</text>

<text x="555" y="100" font-family="serif" font-size="13" font-style="italic" fill="#B91C1C">Transformer</text> <text x="555" y="113" font-family="JetBrains Mono" font-size="10" fill="#B91C1C">O(N²)</text> </svg> <figcaption><strong>Fig 11.</strong> Prefill cost (full-context processing) scales linearly with sequence length for Mamba's selective state-space layer, and quadratically for self-attention. The two curves are indistinguishable at 1K tokens. At 16K, attention's full-context pass is already 16× more expensive than Mamba's; at 262K the gap is 262×. This is the ratio that makes long-context prefill affordable. Decode-step cost with a KV cache scales differently and is not what this figure measures.</figcaption> </figure>

At 262K, pure-attention prefill is unaffordable; this is what Mamba buys.

Why not pure Mamba? Because pure-Mamba models suffer on certain in-context retrieval tasks where attention's exact-position lookups matter. NVIDIA's Nemotron-H paper, which has a similar hybrid layout, says it directly:

"Roughly 8% of the total layers in the model are self-attention layers; these layers are evenly dispersed throughout the model."

Nemotron 3 Nano Omni is at ~11.5% attention, slightly higher than Nemotron-H's 8%, consistent with NVIDIA's framing about preserving stronger global interaction in the omni model. Either way, the principle is the same: keep enough attention layers to handle the retrieval-flavored tasks attention is good at, and replace the rest with Mamba to make long-context inference affordable.

The headline number from Nemotron-H, the closest primary source for the family's hybrid efficiency claims:

"Nemotron-H-56B-Base can generate 2.4× more output tokens per second per GPU compared to Qwen-2.5-72B and Llama-3.1-70B."

A separate measurement on long contexts: the Nemotron-H research page reports a 2.9× speedup for the smaller Nemotron-H-47B at 64K input, 1024 output tokens on H100, again versus Qwen-2.5-72B and Llama-3.1-70B. The same architectural choice carries forward into Nemotron 3 Nano Omni, where the language backbone is a smaller hybrid sized for omni-modal serving rather than monolingual chat.

16. MoE: total parameters versus active parameters

The other half of the "30B total / 3.5B active" math comes from the MoE layers. The idea is older than Mamba and goes back at least to Switch Transformer in 2021. Each token gets routed to a small subset of experts; the rest sit out:

"We simplify the MoE routing algorithm and design intuitive improved models with reduced communication and computational costs."

The high-level pitch is that you can grow the parameter count by adding experts without growing the per-token compute, because each token only sees a fixed number of them. Switch routes each token to exactly one expert (top-1). Mixtral 8x7B in 2024 generalized to top-2 across 8 experts, popularizing the "total params vs active params" naming convention:

"For every token, at each layer, a router network selects two experts to process the current state and combine their outputs." "Each token has access to 47B parameters, but only uses 13B active parameters during inference."

Nemotron 3 Nano scales this up to 128 experts per MoE layer with top-6 routing, plus a shared expert that runs for every token regardless of routing.

<rect x="290" y="20" width="100" height="22" fill="#FBE9CE" stroke="#B8651A" stroke-width="0.8"/> <text x="340" y="36" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#1A1A1A">input token</text>

<rect x="120" y="60" width="440" height="40" fill="#1A1A1A" stroke="#B8651A" stroke-width="1.5"/> <text x="340" y="80" text-anchor="middle" font-family="serif" font-size="13" font-weight="700" fill="#F4EEE3">Router (128-way softmax)</text> <text x="340" y="93" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#FEF3C7">scores expert i for this token</text>

<rect x="555" y="135" width="20" height="45" fill="#D1FAE5" stroke="#15803D" stroke-width="1.2"/> <text x="565" y="158" text-anchor="middle" font-family="JetBrains Mono" font-size="9" font-weight="600" fill="#15803D">+1</text> <text x="565" y="194" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#15803D">shared</text>

<text x="280" y="116" text-anchor="middle" font-family="JetBrains Mono" font-size="9" font-weight="600" fill="#B8651A">top-6 routed (highest scores)</text> <text x="565" y="123" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#15803D">always on</text>

<rect x="240" y="225" width="200" height="40" fill="#FBE9CE" stroke="#B8651A" stroke-width="1.5"/> <text x="340" y="245" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#B8651A">Σ weighted sum</text> <text x="340" y="258" text-anchor="middle" font-family="serif" font-size="9" font-style="italic" fill="#6B6258">6 routed × score + 1 shared</text>

<rect x="290" y="284" width="100" height="22" fill="#1A1A1A" stroke="#B8651A" stroke-width="1.5"/> <text x="340" y="300" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#F4EEE3">layer output</text> </svg> <figcaption><strong>Fig 12.</strong> What one MoE layer does for one token. The router runs a 128-way softmax and picks the six experts with the highest scores; the rest contribute nothing for this token. A separate "shared" expert runs for every token regardless of routing. The layer's output is a weighted sum of those six routed experts' outputs (weighted by router score) plus the shared expert. Aggregated over a batch, almost every expert sees some token, but each individual token only touches the seven the router and the shared slot select.</figcaption> </figure>

The arithmetic, in rough terms: 128 routed experts at some parameter count each, plus a shared expert. For each token, the router computes scores for all 128, picks the top 6, and the layer's output is a weighted combination of those 6 experts' outputs (weighted by router scores) plus the shared expert's contribution. The 122 unselected experts contribute nothing to this token's output. Their expert MLPs cost substantial FLOPs only when some token in the batch routes to them; the router still scores all 128 every step, but that's a cheap projection compared to the expert matmuls.

Aggregated over a batch of N tokens, the shared expert sees 100% of the tokens (it isn't routed) and each routed expert sees on average about 6N/128 ≈ 4.7% of the tokens under balanced routing. So 30B total parameters, only ~3.5B active per token, but during a forward pass on a batch of, say, 1024 tokens, the shared expert and most of the routed experts touch some token.

Three things follow from this. Capacity scales with parameter count; you can keep adding experts. Compute scales with the active count, not the total. Memory bandwidth dominates inference, because all 30B parameters live in GPU memory and have to be addressable even if any one token only uses 3.5B of them.

The third point is what NVFP4 attacks.

17. NVFP4: 4-bit weights, BF16 accuracy

The cheapest way to spend less memory bandwidth is to use fewer bits per weight. NVIDIA's Blackwell-era format for this is NVFP4, a 4-bit floating-point representation with a two-level scaling scheme. The format itself:

"NVFP4 employs a dual-tier approach: 1 shared FP8 scale per 16 value block (E4M3) and 1 FP32 per tensor second-level scaling factor."

Decoding the layout: each weight is stored as a 4-bit floating-point value with one sign bit, two exponent bits, and one mantissa bit (E2M1). Sixteen of those weights share an FP8 scale factor in E4M3 (the same format Hopper and Blackwell already had hardware support for). And the entire tensor shares a single FP32 global scale on top of that. The two-level structure lets the format keep both wide tensor-level dynamic range (the FP32 scale) and per-block precision tracking (the FP8 micro-scale), at a cost of about 5 effective bits per weight rather than a flat 4.

Memory savings versus the alternatives, on the Nemotron 3 Nano Omni reasoning checkpoint specifically:

"The NVFP4 variant uses a mixed-precision recipe inspired by Nemotron 3 Super: routed MoE experts are quantized to NVFP4 (FP4 E2M1 values with per-block FP8 E4M3 scales over groups of 16 elements and an additional per-tensor FP32 global scale), while the Mamba in_proj / out_proj, shared experts, and attention o_proj are quantized to FP8, yielding 4.98 effective bits per weight (20.9 GB). In both variants the vision and audio encoders and their MLP projectors are kept in BF16."

The numbers, plain: BF16 weights for the full model are 61.5 GB. FP8 is 32.8 GB. NVFP4 (with the mixed-precision recipe) is 20.9 GB. The reasoning-checkpoint mean across nine multimodal benchmarks: BF16 65.80, NVFP4 65.43. A 0.38-point drop on the mean for a 3× memory reduction.

<text x="340" y="20" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">NVFP4 STORAGE: ONE 16-WEIGHT BLOCK</text>

<rect x="120" y="32" width="440" height="22" fill="#FBE9CE" stroke="#B8651A" stroke-width="1.2"/> <text x="340" y="47" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#B8651A">FP32 global scale, 1 per tensor (32 bits)</text>

<rect x="120" y="62" width="440" height="22" fill="#D1FAE5" stroke="#15803D" stroke-width="1.2"/> <text x="340" y="77" text-anchor="middle" font-family="serif" font-size="11" font-weight="600" fill="#15803D">FP8 E4M3 micro-scale, 1 per 16-weight block (8 bits)</text>

<line x1="146" y1="148" x2="146" y2="160" stroke="#2563EB" stroke-width="0.8"/> <text x="146" y="172" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#2563EB">each = 4 bits (1 sign + 2 exp + 1 mantissa)</text>

<text x="340" y="195" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#1A1A1A">total: <tspan font-family="JetBrains Mono">16 × 4 + 8 + 32/N = ~5 effective bits per weight</tspan></text>

<text x="120" y="220" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">30B-MODEL WEIGHTS IN GPU MEMORY</text>

<rect x="120" y="280" width="400" height="18" fill="#EDE5D4" stroke="#8F8578" stroke-width="0.5"/> <rect x="120" y="280" width="136" height="18" fill="#15803D" opacity="0.65"/> <text x="525" y="293" font-family="JetBrains Mono" font-size="11" fill="#15803D">NVFP4 (mixed): 20.9 GB</text>

<text x="340" y="320" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">3× memory reduction over BF16 with a ~0.4-point mean benchmark drop</text> </svg> <figcaption><strong>Fig 13.</strong> The NVFP4 storage layout. Each weight in a 16-weight block is stored as a 4-bit E2M1 floating-point value (1 sign bit, 2 exponent bits, 1 mantissa bit). The block shares a single FP8 E4M3 micro-scale, and the entire tensor shares one FP32 global scale on top. Effective storage is around 5 bits per weight. Bottom: the same 30B model occupies 61.5 GB at BF16, 32.8 GB at FP8, and 20.9 GB under the NVFP4 mixed-precision recipe.</figcaption> </figure>

The mixed-precision recipe matters. Quantizing the routed experts to 4 bits is the bulk of the savings (most parameters in a 30B-A3B MoE live in those experts). Mamba projection layers, shared experts, and the attention output projection stay at FP8 because those layers are more sensitive. Vision and audio encoders stay at BF16 because they're small relative to the LLM and the accuracy cost isn't worth it.

The reason this works is that NVFP4 reuses E4M3 as the per-block scale factor. Blackwell already has hardware for E4M3, so the micro-scale comes nearly for free. The format is a careful reuse of formats Blackwell already accelerates, not an exotic ad-hoc design.

18. What's still missing

The post just walked through how to wire perception across four modalities into a single decoder. The reasoning loop on top of that perception is a different and unfinished problem.

What the model has at this point: a unified backbone that reads text, sees images, hears audio, and watches video, all in one forward pass; a hybrid Mamba+attention+MoE architecture that scales to 262K-token contexts; a quantization recipe that fits the whole thing in around 21 GB of weights. The model card lists tool-calling support and reports OSWorld-style computer-use scores, so the post is not making the claim that no agentic numbers exist; what it is claiming is that the broader question of long-horizon planning, multi-step decomposition, and benchmark performance on suites like BFCL or GAIA is not yet what the report or model cards center on. The launch announcement frames the model as a "multimodal perception-and-context sub-agent within larger agentic systems," not as a planner.

That framing is honest and worth taking seriously. Perception is not reasoning. A model that perceives well is a foundation for an agent loop; it is not the loop itself. The next problem in the stack is how the LLM in the middle generates plans, calls tools, and executes multi-step trajectories over the perceived state. None of that has the same shape as the perception story this post traced.

<text x="340" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#6B6258">EFFICIENCY EVIDENCE LEDGER (NEMOTRON-FAMILY MEASUREMENTS)</text>

<rect x="10" y="35" width="660" height="25" fill="#1A1A1A"/> <g font-family="serif" font-size="11" font-weight="700" fill="#F4EEE3"> <text x="20" y="52">Claim</text> <text x="170" y="52">Mechanism</text> <text x="370" y="52">Number (vs baseline)</text> <text x="570" y="52">Limitation</text> </g>

<rect x="10" y="63" width="660" height="70" fill="#F9F5EB" stroke="#C9BEAA" stroke-width="0.5"/> <rect x="10" y="63" width="4" height="70" fill="#2563EB"/> <g font-family="serif" font-size="11" fill="#1A1A1A"> <text x="22" y="80" font-weight="600" fill="#2563EB">Long-context</text> <text x="22" y="94" font-weight="600" fill="#2563EB">inference</text> <text x="22" y="114" font-style="italic" fill="#6B6258" font-size="10">throughput</text> <text x="170" y="80">Mamba-2 selective</text> <text x="170" y="94">SSM, O(N) cost in</text> <text x="170" y="108">sequence length, plus</text> <text x="170" y="122">6 attention for retrieval</text> <text x="370" y="80">2.9× tokens/sec at 64K</text> <text x="370" y="94">input, 1024 output, on H100</text> <text x="370" y="114" font-style="italic" font-size="10" fill="#6B6258">(vs Qwen-2.5-72B, Llama-3.1-70B)</text> <text x="570" y="80" font-size="10">Measured on</text> <text x="570" y="94" font-size="10">Nemotron-H-47B,</text> <text x="570" y="108" font-size="10">not on Nemotron 3</text> <text x="570" y="122" font-size="10">Nano Omni</text> </g>

<rect x="10" y="138" width="660" height="70" fill="#F9F5EB" stroke="#C9BEAA" stroke-width="0.5"/> <rect x="10" y="138" width="4" height="70" fill="#15803D"/> <g font-family="serif" font-size="11" fill="#1A1A1A"> <text x="22" y="155" font-weight="600" fill="#15803D">Active</text> <text x="22" y="169" font-weight="600" fill="#15803D">parameters</text> <text x="22" y="189" font-style="italic" fill="#6B6258" font-size="10">per token</text> <text x="170" y="155">MoE sparse routing:</text> <text x="170" y="169">top-6 of 128 experts</text> <text x="170" y="183">+ 1 always-on shared,</text> <text x="170" y="197">router decides per token</text> <text x="370" y="155">3.5B active /</text> <text x="370" y="169">30B total params</text> <text x="370" y="189" font-style="italic" font-size="10" fill="#6B6258">(per Nemotron 3 Nano Omni model card)</text> <text x="570" y="155" font-size="10">Wall-clock win</text> <text x="570" y="169" font-size="10">depends on serving</text> <text x="570" y="183" font-size="10">stack and batch</text> <text x="570" y="197" font-size="10">routing balance</text> </g>

<rect x="10" y="213" width="660" height="70" fill="#F9F5EB" stroke="#C9BEAA" stroke-width="0.5"/> <rect x="10" y="213" width="4" height="70" fill="#B8651A"/> <g font-family="serif" font-size="11" fill="#1A1A1A"> <text x="22" y="230" font-weight="600" fill="#B8651A">Memory</text> <text x="22" y="244" font-weight="600" fill="#B8651A">footprint</text> <text x="170" y="230">NVFP4 mixed precision:</text> <text x="170" y="244">routed experts FP4,</text> <text x="170" y="258">attention and Mamba projs FP8,</text> <text x="170" y="272">vision/audio encoders BF16</text> <text x="370" y="230">61.5 GB (BF16) →</text> <text x="370" y="244">20.9 GB (NVFP4) on</text> <text x="370" y="258">Nemotron 3 Nano Omni</text> <text x="370" y="272" font-style="italic" font-size="10" fill="#6B6258">(3× reduction)</text> <text x="570" y="230" font-size="10">−0.38-point mean</text> <text x="570" y="244" font-size="10">drop across 9</text> <text x="570" y="258" font-size="10">multimodal</text> <text x="570" y="272" font-size="10">benchmarks</text> </g>

<rect x="10" y="288" width="660" height="70" fill="#F9F5EB" stroke="#C9BEAA" stroke-width="0.5"/> <rect x="10" y="288" width="4" height="70" fill="#9333EA"/> <g font-family="serif" font-size="11" fill="#1A1A1A"> <text x="22" y="305" font-weight="600" fill="#9333EA">Prosody</text> <text x="22" y="319" font-weight="600" fill="#9333EA">preservation</text> <text x="170" y="305">Speech-out reads LLM</text> <text x="170" y="319">hidden states directly</text> <text x="170" y="333">(Qwen2.5-Omni Talker:</text> <text x="170" y="347">no transcript bottleneck)</text> <text x="370" y="305">Tsiamas et al. 2024:</text> <text x="370" y="319">cascades transmit prosody</text> <text x="370" y="333">to a "lesser extent"</text> <text x="570" y="305" font-size="10">Qualitative;</text> <text x="570" y="319" font-size="10">no Nemotron 3</text> <text x="570" y="333" font-size="10">Nano Omni</text> <text x="570" y="347" font-size="10">benchmark</text> </g> </svg> <figcaption><strong>Fig 14.</strong> What the post can claim and where each claim comes from. Two measured efficiency wins on Nemotron-family models against named baselines (long-context throughput from Nemotron-H, memory footprint from Nemotron 3 Nano Omni's NVFP4 recipe), one structural claim from the model card (3.5B active out of 30B total per token), and a fourth qualitative claim about prosody. The post does not promise a controlled benchmark of "unified-decoder beats stitched cascades" in the abstract; it reports these four rows and stops.</figcaption> </figure>

Three architectural moves take you from a vision backbone to a four-modality perception model. A text-centric autoregressive decoder. Modality-specific encoders feeding it through MLP projectors. A stack of token-budget mechanisms (tubelet fusion, EVS, hybrid Mamba+attention+MoE, NVFP4) to keep inference affordable. The last one is what makes the first two shippable.

None of them are agent reasoning. But you can't reason without seeing.

References

The Unified Vision Stack. the prequel post, Augusteo 2026.
Learning Transferable Visual Models From Natural Language Supervision (CLIP). Radford et al., OpenAI 2021.
Sigmoid Loss for Language Image Pre-Training (SigLIP). Zhai, Mustafa, Kolesnikov, Beyer (Google DeepMind) 2023.
SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features. Tschannen et al. (Google DeepMind) 2025.
AM-RADIO: Agglomerative Vision Foundation Model. Ranzinger et al. (NVIDIA) 2023, the framework that became C-RADIO.
RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models. Heinrich et al. (NVIDIA) 2024.
Flamingo: a Visual Language Model for Few-Shot Learning. Alayrac et al. (DeepMind) 2022.
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. Li et al. (Salesforce) 2023.
Visual Instruction Tuning (LLaVA). Liu, Li, Wu, Lee 2023.
Improved Baselines with Visual Instruction Tuning (LLaVA-1.5). Liu et al. 2023.
Qwen2.5-VL Technical Report. Bai et al. (Alibaba Qwen Team) 2025.
Qwen2.5-Omni Technical Report. Xu et al. (Alibaba Cloud Qwen Team) 2025.
Whisper: large-scale weak-supervision speech recognition. Radford et al. (OpenAI) 2022.
Speech is More Than Words. Tsiamas, Sperber, Finch, Garg 2024 (cascade vs end-to-end speech translation, prosody study).
Chameleon: Mixed-Modal Early-Fusion Foundation Models. Chameleon Team (FAIR at Meta) 2024.
Gemini: A Family of Highly Capable Multimodal Models. Gemini Team (Google) 2023.
NVIDIA Nemotron 3 Omni technical report. NVIDIA 2026.
Introducing NVIDIA Nemotron 3 Nano Omni: Multimodal Intelligence. NVIDIA 2026 (HF blog).
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 model card. NVIDIA 2026.
Nemotron-3-Nano-Omni-30B-A3B-Reasoning-NVFP4 model card. NVIDIA 2026.
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 base model card. NVIDIA 2026.
parakeet-tdt-0.6b-v2 model card. NVIDIA 2025.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces. Gu and Dao 2023.
Transformers are SSMs (Mamba-2). Dao and Gu 2024.
Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models. NVIDIA 2025.
Nemotron-H research page. NVIDIA ADLR 2025.
Jamba: A Hybrid Transformer-Mamba Language Model. AI21 Labs 2024.
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. Fedus, Zoph, Shazeer 2021.
Mixtral of Experts. Mistral AI 2024.
Introducing NVFP4 for Efficient and Accurate Low-Precision Inference. NVIDIA Developer Blog 2025.
FP8 Formats for Deep Learning. Micikevicius et al. (NVIDIA, Arm, Intel) 2022.
ViViT: A Video Vision Transformer. Arnab et al. (Google) 2021.
Is Space-Time Attention All You Need for Video Understanding? (TimeSformer). Bertasius, Wang, Torresani (Facebook AI) 2021.
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training. Tong, Song, Wang, Wang 2022.
VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking. Wang et al. 2023.
Efficient Video Sampling (EVS): Token Reduction for Long Videos in Multimodal LLMs. Bagrov et al. (NVIDIA + Deci) 2025.

Sequel to The Unified Vision Stack, written May 2026.

How PgBouncer Works

Thu, 30 Apr 2026 00:00:00 GMT

A from-the-ground-up explanation of PgBouncer: why Postgres needs a connection pooler, what each pooling mode actually does, what breaks under transaction pooling, and how to run the thing in production. About a 30-minute read. Versions reference PgBouncer 1.25.1, the current stable as of writing.

1. The thing PgBouncer fixes

If you have used Postgres for any length of time, you have probably run into some version of this story. Your app has been fine for months. Then you push a release that adds a few more workers, or traffic spikes during a launch, or your background job system starts running more parallel jobs than it used to. Suddenly the database starts refusing connections. Latency goes up. Memory usage on the database host climbs to ninety-something percent. The error logs are full of FATAL: sorry, too many clients already, and your max_connections setting, which seemed generous when you set it, is now the thing standing between you and a working application.

The textbook fix is "add a connection pooler." The textbook recommendation is PgBouncer. And then, if you are like most people, you set pool_mode = transaction, point your app at port 6432 instead of 5432, and walk away thinking the problem is solved.

It usually is. But the thing you just deployed is doing something quite specific, and if you do not understand what, the next bug you hit will be confusing in a way that direct Postgres bugs are not. Why doesn't SET search_path stick? Why did my long-running report time out at 5 seconds when I never set a 5-second timeout? Why does this Perl script work when I connect directly but fail through PgBouncer? PgBouncer presents itself as a transparent proxy, and in one mode it almost is, but in the mode you actually want to use, it isn't, and the gap between what looks transparent and what actually is transparent is where the bugs live.

This piece is about that gap. We'll start with why Postgres connections cost something in the first place, work through the three pooling modes and what each one buys and breaks, then look at how to actually run PgBouncer in production: how to size the pool, how to authenticate without storing every password twice, how to monitor it, and the long list of small things that go subtly wrong.

Act 1 — Why connections are expensive

The Postgres process model and why it does not love thousands of clients.

2. The fork that started it all

Postgres has been around since 1986. Its connection model dates from a time when the modern web did not exist and the typical workload was a handful of analysts running queries from terminals. The model is simple. There is one supervisor process called the postmaster. When a client connects, the postmaster calls fork() and produces a new child process called a backend. That backend is yours, exclusively, for the lifetime of the connection. When you disconnect, it exits. Every query you run, every transaction, every SET, every prepared statement, every temporary table, lives inside that one process.

This design has real virtues. Each connection is isolated, so a memory bug in one query cannot corrupt another client's session. Cancellation is a SIGINT to a specific PID. The OS scheduler does the heavy lifting of fairness between connections. It is a very Unix design and it ages well in many ways.

It does not age well in one specific way. A process is not free. Forking it is not free. Initializing the backend's catalog cache is not free. And keeping it around when it is doing nothing is not free either.

<figure> <svg viewBox="0 0 700 320" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Postgres process tree showing one postmaster forking a backend per client" width="100%" height="auto"> <rect fill="#f4ede0" x="0" y="0" width="700" height="320" rx="6"/> <rect x="270" y="30" width="160" height="46" rx="4" fill="#fff" stroke="#1a1815" stroke-width="1.4"/> <text font-family="serif" font-size="13" fill="#1a1815" x="350" y="50" text-anchor="middle">postmaster</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="66" text-anchor="middle">listens on 5432</text> <g stroke="#4a4742" fill="none" stroke-width="1"> <path d="M 350 76 L 110 130" stroke-dasharray="3 3"/> <path d="M 350 76 L 220 130" stroke-dasharray="3 3"/> <path d="M 350 76 L 350 130" stroke-dasharray="3 3"/> <path d="M 350 76 L 480 130" stroke-dasharray="3 3"/> <path d="M 350 76 L 600 130" stroke-dasharray="3 3"/> </g> <g> <rect x="40" y="130" width="140" height="58" rx="4" fill="#fff" stroke="#6a7d63" stroke-width="1.4"/> <text font-family="serif" font-size="13" fill="#1a1815" x="110" y="150" text-anchor="middle">backend</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="110" y="167" text-anchor="middle">pid 1812</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="110" y="180" text-anchor="middle">~5–15 MB</text>

<rect x="150" y="130" width="140" height="58" rx="4" fill="#fff" stroke="#6a7d63" stroke-width="1.4"/>
<text font-family="serif" font-size="13" fill="#1a1815" x="220" y="150" text-anchor="middle">backend</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="220" y="167" text-anchor="middle">pid 1813</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="220" y="180" text-anchor="middle">idle</text>

<rect x="280" y="130" width="140" height="58" rx="4" fill="#fff" stroke="#6a7d63" stroke-width="1.4"/>
<text font-family="serif" font-size="13" fill="#1a1815" x="350" y="150" text-anchor="middle">backend</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="167" text-anchor="middle">pid 1814</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="180" text-anchor="middle">running query</text>

<rect x="410" y="130" width="140" height="58" rx="4" fill="#fff" stroke="#6a7d63" stroke-width="1.4"/>
<text font-family="serif" font-size="13" fill="#1a1815" x="480" y="150" text-anchor="middle">backend</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="480" y="167" text-anchor="middle">pid 1815</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="480" y="180" text-anchor="middle">idle in tx</text>

<rect x="540" y="130" width="140" height="58" rx="4" fill="#fff" stroke="#6a7d63" stroke-width="1.4"/>
<text font-family="serif" font-size="13" fill="#1a1815" x="610" y="150" text-anchor="middle">backend</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="610" y="167" text-anchor="middle">pid 1816</text>
<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="610" y="180" text-anchor="middle">idle</text>

</g> <g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="110" y="240" text-anchor="middle">client A</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="220" y="240" text-anchor="middle">client B</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="240" text-anchor="middle">client C</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="480" y="240" text-anchor="middle">client D</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="610" y="240" text-anchor="middle">client E</text> <g stroke="#4a4742" fill="none" stroke-width="1"> <path d="M 110 220 L 110 195"/> <path d="M 220 220 L 220 195"/> <path d="M 350 220 L 350 195"/> <path d="M 480 220 L 480 195"/> <path d="M 610 220 L 610 195"/> </g> </g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="290" text-anchor="middle">one process per client. always. even when the client is doing nothing.</text> </svg> <figcaption><strong>Fig 1.</strong> The Postgres connection model. Every client gets a dedicated OS process. The processes are isolated, which is great for stability, and unshared, which is bad for scaling.</figcaption> </figure>

3. What an idle connection actually costs

Memory is the headline cost, and the number you usually hear is "10 MB per connection." That number is roughly right but slightly misleading. The truth is that ps and top overcount, because Linux uses copy-on-write for forked processes and does not split out shared memory cleanly in those tools. When you measure more carefully — Andres Freund's writeup uses a mix of /proc/[pid]/smaps_rollup (an approximation, not an exact figure) and direct PSS accounting — the incremental private memory of an idle Postgres backend is closer to 1.5 to 2 MB on a fresh connection, growing to 5 to 15 MB once the connection has done some work and accumulated catalog cache, prepared plans, and per-process state. The AWS database team's RDS-side measurements landed at 1.5 to 14.5 MB depending on workload, which lines up.

So it is not "10 MB per connection" exactly. It is more like "a few MB per connection, more if it does anything interesting, multiplied by however many connections you have, even if they are idle." On a host with 32 GB of RAM, where you also want most of that going to shared_buffers and the OS page cache, you cannot afford 2,000 idle connections. You probably cannot afford 1,000.

Memory is not even the whole story. Each backend process is also a thing the OS scheduler has to consider. Each backend has a slot in Postgres' shared procarray, which is scanned during snapshot acquisition: a process that does nothing all day still gets walked over every time another transaction takes a snapshot. There's a known cliff somewhere in the low thousands of connections where Postgres spends measurably more CPU just managing the procarray than running queries. The exact number depends on hardware and version, but as a rule of thumb, past a few hundred connections per CPU core, you are buying overhead and not throughput.

Then there is connection setup cost. Opening a fresh Postgres connection involves a TCP handshake, a TLS handshake (in any sane production setup), a fork on the server, a SCRAM exchange, sending startup parameters, loading the role's config, and setting the search path. On a healthy LAN this takes a few milliseconds. On a TLS connection across regions, it can be 30 to 100 ms. PHP-style "connect on every request" patterns make this a per-request tax. Hans-Jürgen Schönig benchmarked the difference between connection-per-query and pooled-connection at 98.7% throughput loss, going from a couple hundred thousand transactions per second down to 3,770. The fork is most of it.

So a Postgres connection costs you four things: memory, scheduler overhead, snapshot-acquisition overhead, and the multi-millisecond setup tax that fires every time a new one is opened. Any one of these gets bad at scale. All four together are why you cannot solve "scale to a thousand app workers" by just raising max_connections.

Raising max_connections is not scaling. It's just postponing.

4. What we actually want

Here's the observation that motivates everything that follows. Most application connections are doing nothing most of the time. A web request typically spends 1 ms talking to the database and 50 ms doing other work: rendering, calling APIs, formatting responses, waiting for the user. A worker process holds a connection for its lifetime even though it spends 95% of that lifetime idle.

If you could somehow let many app-side connections share a smaller pool of real Postgres connections, multiplexed so that whichever client actually needs the database right now gets a real connection, and everyone else gets a cheap proxy connection that costs essentially nothing while they're idle, you'd have your scaling. You'd also have a contention pattern that maps to your real workload (active query count) instead of your app's worker count.

This is what a connection pooler is. PgBouncer is the one most people use.

Act 2 — Three modes of multiplexing

Session, transaction, statement: the spectrum of how aggressively you're willing to lie.

5. Meet PgBouncer

PgBouncer is a single binary, written in C, single-threaded, event-driven, built on libevent. It speaks the Postgres wire protocol on both sides. To your application, it looks like a Postgres server. To the real Postgres, it looks like a bunch of clients. It sits in the middle and shuffles bytes back and forth, occasionally attaching and detaching client-to-server pairings according to whichever pooling mode you have configured.

The numbers are striking. The PgBouncer process is small: a base of a few MB plus roughly 2 KB of bookkeeping per client connection (more once you turn on max_prepared_statements tracking). A single node can hold many thousands of client connections; configurations of max_client_conn = 10000 are common in real deployments, with the practical ceiling set by file descriptors, TLS overhead, and the single-core CPU limit rather than memory. The per-query latency overhead is small (sub-millisecond in healthy LAN benchmarks), mostly because it does not parse SQL. It just looks at protocol messages, sees where one transaction ends and the next begins, and decides whether the current client/server pair should stay attached or get split apart.

That last sentence is the entire concept. The pooling mode is the rule for when PgBouncer can detach a client from a server connection and reuse that server connection for someone else. There are three modes.

<figure> <svg viewBox="0 0 700 360" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="PgBouncer sitting between many clients and few server connections" width="100%" height="auto"> <rect fill="#f4ede0" x="0" y="0" width="700" height="360" rx="6"/> <text font-family="serif" font-size="13" fill="#1a1815" x="40" y="30" text-anchor="start" font-weight="600">app workers</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="40" y="46" text-anchor="start">cheap proxy connections</text> <g> <g fill="#fff" stroke="#4a6f88" stroke-width="1"> <rect x="40" y="65" width="80" height="22" rx="3"/> <rect x="40" y="92" width="80" height="22" rx="3"/> <rect x="40" y="119" width="80" height="22" rx="3"/> <rect x="40" y="146" width="80" height="22" rx="3"/> <rect x="40" y="173" width="80" height="22" rx="3"/> <rect x="40" y="200" width="80" height="22" rx="3"/> <rect x="40" y="227" width="80" height="22" rx="3"/> <rect x="40" y="254" width="80" height="22" rx="3"/> <rect x="40" y="281" width="80" height="22" rx="3"/> <rect x="40" y="308" width="80" height="22" rx="3"/> </g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="80" text-anchor="middle">w01</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="107" text-anchor="middle">w02</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="134" text-anchor="middle">w03</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="161" text-anchor="middle">w04</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="188" text-anchor="middle">w05</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="215" text-anchor="middle">w06</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="242" text-anchor="middle">w07</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="269" text-anchor="middle">w08</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="296" text-anchor="middle">w09</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="80" y="323" text-anchor="middle">w10</text> </g>

<rect x="260" y="120" width="180" height="120" rx="6" fill="#fff" stroke="#1a1815" stroke-width="1.6"/> <text font-family="serif" font-size="13" fill="#1a1815" x="350" y="148" text-anchor="middle" font-weight="600">PgBouncer</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="170" text-anchor="middle">single process</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="186" text-anchor="middle">single thread</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="202" text-anchor="middle">~3 MB RAM</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="218" text-anchor="middle">listens on 6432</text>

<text font-family="serif" font-size="13" fill="#1a1815" x="660" y="30" text-anchor="end" font-weight="600">Postgres backends</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="660" y="46" text-anchor="end">expensive real connections</text> <g fill="#fff" stroke="#6a7d63" stroke-width="1.2"> <rect x="540" y="140" width="120" height="28" rx="3"/> <rect x="540" y="175" width="120" height="28" rx="3"/> <rect x="540" y="210" width="120" height="28" rx="3"/> </g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="600" y="158" text-anchor="middle">backend 1</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="600" y="193" text-anchor="middle">backend 2</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="600" y="228" text-anchor="middle">backend 3</text>

<g stroke="#6a7d63" fill="none" stroke-width="1"> <path d="M 440 154 L 540 154"/> <path d="M 440 189 L 540 189"/> <path d="M 440 224 L 540 224"/> </g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="280" text-anchor="middle">10 clients in</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="296" text-anchor="middle">3 server connections out</text> </svg> <figcaption><strong>Fig 2.</strong> PgBouncer's job in one picture. Many cheap proxy connections on the client side, a small number of real Postgres backends on the server side, and a multiplexer in the middle deciding who gets which backend, when. The ratio (10:3 here, often 100:20 in production) is the whole point.</figcaption> </figure>

6. Session pooling

Session pooling is the conservative default. A client connects to PgBouncer, PgBouncer hands them a real backend, and that backend belongs to that client until the client disconnects. From the application's point of view, this is identical to talking to Postgres directly.

What does this buy you? Not multiplexing. If you have 200 app workers, you have 200 server connections. The pooler is not really pooling anything during steady-state operation. What it does buy you is a few smaller wins. The connection setup cost is amortized: when a worker disconnects and another reconnects, the backend stays warm in PgBouncer's pool and gets reassigned without forking a new Postgres process. PgBouncer handles SCRAM authentication on the client side without holding open file descriptors during slow handshakes. And when Postgres restarts (for a parameter change, an HA failover, a scheduled maintenance), PgBouncer can reconnect on the back side while clients hold steady on the front side, which is genuinely useful for keeping your app from cascading.

Session mode is the safe choice. Every Postgres feature works exactly as you'd expect: prepared statements, temp tables, LISTEN/NOTIFY, session-level advisory locks, SET, search paths, the lot. There is essentially nothing you have to change in your application.

It is also, for most modern web workloads, the wrong choice. The whole reason we started this conversation is that 200 idle backends costs real money. Session pooling does not save you from that. It saves you only from the connect/disconnect tax, which is real but secondary.

7. Transaction pooling

Transaction pooling is the mode that makes PgBouncer worth running. The rule is simple: a server backend is bound to a client only for the duration of a transaction. As soon as the client commits or rolls back, the backend goes back to the pool, and the next client that wants to start a transaction gets it.

This is where the math changes. A web request typically takes 50 ms total but actually has the database connection in a transaction for maybe 5 ms of that. If you measure your real "active in transaction" time and divide by your total request time, you get a multiplier somewhere between 5x and 50x for typical workloads. That is your effective overcommit ratio. With transaction pooling, you can comfortably support 200 concurrent app workers on 20 server connections, because at any given instant only ~5% of them are actually mid-transaction.

<g> <rect x="40" y="20" width="14" height="10" fill="#6a7d63"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="60" y="29">server bound to client</text> <rect x="240" y="20" width="14" height="10" fill="#fff" stroke="#1a1815" stroke-width="1"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="260" y="29">server in pool, available</text> </g>

<text font-family="serif" font-size="13" fill="#1a1815" x="40" y="70" font-weight="600">session</text> <line x1="40" y1="80" x2="660" y2="80" stroke="#d9cfb9"/> <rect x="100" y="74" width="540" height="14" fill="#6a7d63"/> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="100" y="105">connect</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="640" y="105" text-anchor="end">disconnect</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="370" y="105" text-anchor="middle">held the whole time, even when idle</text>

<text font-family="serif" font-size="13" fill="#1a1815" x="40" y="160" font-weight="600">transaction</text> <line x1="40" y1="170" x2="660" y2="170" stroke="#d9cfb9"/> <rect x="100" y="164" width="60" height="14" fill="#6a7d63"/> <rect x="220" y="164" width="40" height="14" fill="#6a7d63"/> <rect x="320" y="164" width="80" height="14" fill="#6a7d63"/> <rect x="450" y="164" width="50" height="14" fill="#6a7d63"/> <rect x="560" y="164" width="70" height="14" fill="#6a7d63"/>

<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="130" y="195" text-anchor="middle">BEGIN…COMMIT</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="240" y="195" text-anchor="middle">tx</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="360" y="195" text-anchor="middle">tx</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="475" y="195" text-anchor="middle">tx</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="595" y="195" text-anchor="middle">tx</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="370" y="215" text-anchor="middle">in between, the backend is free for someone else</text>

<text font-family="serif" font-size="13" fill="#1a1815" x="40" y="260" font-weight="600">statement</text> <line x1="40" y1="270" x2="660" y2="270" stroke="#d9cfb9"/> <g fill="#6a7d63"> <rect x="100" y="264" width="14" height="14"/> <rect x="130" y="264" width="14" height="14"/> <rect x="170" y="264" width="14" height="14"/> <rect x="220" y="264" width="14" height="14"/> <rect x="280" y="264" width="14" height="14"/> <rect x="330" y="264" width="14" height="14"/> <rect x="370" y="264" width="14" height="14"/> <rect x="420" y="264" width="14" height="14"/> <rect x="470" y="264" width="14" height="14"/> <rect x="510" y="264" width="14" height="14"/> <rect x="560" y="264" width="14" height="14"/> <rect x="600" y="264" width="14" height="14"/> <rect x="640" y="264" width="14" height="14"/> </g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="370" y="298" text-anchor="middle">released after every single statement; multi-statement transactions forbidden</text> </svg> <figcaption><strong>Fig 3.</strong> The three modes, on the same timeline. Session holds a backend for the whole client lifetime. Transaction holds it only during <code>BEGIN…COMMIT</code> and releases it in the gaps. Statement releases after every query, and refuses to let you have multi-statement transactions at all. As you go down the spectrum, multiplexing improves and Postgres compatibility shrinks.</figcaption> </figure>

The catch is that transaction pooling lies to your application about how connections work. If you set a session variable in transaction A, and the next call you make happens in transaction B, transaction B might be on a different server backend, where the variable is unset. If you create a temp table in transaction A, it will not be visible in transaction B, because temp tables live in a backend's session and the next backend doesn't know about it. Anything that depends on Postgres' notion of a "session" is now broken, because what your app thinks of as a session is no longer mapped onto a single backend's session. We'll come back to this. It's the bulk of what you have to think about when you adopt transaction pooling.

8. Statement pooling

Statement pooling is transaction pooling's wilder cousin. The server is released after every single statement, and multi-statement transactions are forbidden outright: if your client sends BEGIN, PgBouncer returns an error.

This is genuinely too aggressive for almost any application. The use case is something like a key-value lookup workload run from a tool that does not need transactions, where the cost of even briefly holding a backend across statements is higher than the cost of losing transaction semantics. PL/Proxy historical setups used it. Most modern apps should not.

I'm going to mostly ignore statement pooling for the rest of this piece. Its existence is mostly important because it shows you the spectrum: session is "bind for the whole connection," transaction is "bind for one transaction," statement is "bind for one query." There's no rule that says PgBouncer has to stop there. Statement pooling shows you the limit case, where the server is held only while a single SQL command is in flight. Past that, you'd be parsing SQL and rewriting it to push specific queries to specific backends, which is what middleware like Pgpool-II does, with all the complexity that entails.

9. The compatibility table

Here is a condensed compatibility chart, drawn from PgBouncer's docs and a few extra rows that bite people in practice. It's worth memorizing if you're going to run transaction pooling.

Feature	session	transaction	statement
Multi-statement transactions	works	works	forbidden
`SET` / `RESET` across queries	works	broken	broken
`LISTEN` (receiving)	works	broken	broken
`NOTIFY` (sending)	works	works	broken
Session-level advisory locks	works	broken	broken
Transaction-level advisory locks	works	works	broken
Temporary tables	works	broken*	broken*
`WITH HOLD` cursors	works	broken	broken
Protocol prepared statements	works	works since 1.21	works since 1.21
`PREPARE` / `EXECUTE` SQL	works	broken	broken
`SET LOCAL` inside a transaction	works	works	broken
`pg_dump`	works	broken	broken
Logical replication connections	since 1.23	unsupported	unsupported

"Broken" here means "may silently misbehave." Temp tables created inside a single transaction with ON COMMIT DROP work fine, because they do not outlive the transaction. The asterisk on temp tables is for the common ON COMMIT PRESERVE ROWS case, where the table is meant to live for the session. That's the one that breaks.

Two of these have stories worth telling, because they're the ones that bite people most often. Prepared statements got a real fix in 2023. SET still doesn't, and the workarounds are awkward.

10. Prepared statements: the asterisk that took ten years to remove

For most of PgBouncer's history, you couldn't use prepared statements in transaction mode at all. The reason is straightforward. A prepared statement in Postgres is named, and it lives in a specific backend's session memory. If your driver does PREPARE q1 AS SELECT ... on backend A and then later does EXECUTE q1, but PgBouncer has by now moved you to backend B, backend B has never heard of q1 and you get an error.

The official advice for ten years was: turn off prepared statements at the driver level. JDBC users set prepareThreshold=0. PHP/PDO users set PDO::ATTR_EMULATE_PREPARES. Hibernate, ActiveRecord, and others have similar knobs. This works, but it costs you the parsing-and-planning savings that prepared statements were supposed to give you in the first place.

PgBouncer 1.21, released in October 2023, finally added support for protocol-level named prepared statements in transaction pooling mode. The trick is clever. PgBouncer intercepts the prepare command from the client, gives the statement an internal name like PGBOUNCER_1234, and tracks the mapping. When the same client (or any other client) later asks to EXECUTE that statement on whatever backend they happen to land on, PgBouncer checks whether the backend already has the statement under the internal name. If it does, the statement runs. If it doesn't, PgBouncer prepares it on the fly, transparently, before forwarding the execute. The internal LRU cache of prepared statements per backend is bounded by the max_prepared_statements setting.

<text font-family="serif" font-size="13" fill="#1a1815" x="80" y="30" text-anchor="middle" font-weight="600">client driver</text> <rect x="20" y="50" width="160" height="34" rx="3" fill="#fff" stroke="#4a6f88" stroke-width="1.2"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="100" y="72" text-anchor="middle">Parse "my_q": SELECT…</text> <rect x="20" y="100" width="160" height="34" rx="3" fill="#fff" stroke="#4a6f88" stroke-width="1.2"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="100" y="122" text-anchor="middle">Execute "my_q" (args)</text>

<rect x="220" y="40" width="260" height="160" rx="5" fill="#fff" stroke="#1a1815" stroke-width="1.4"/> <text font-family="serif" font-size="13" fill="#1a1815" x="350" y="62" text-anchor="middle" font-weight="600">PgBouncer</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="84" text-anchor="middle">global query cache</text> <rect x="240" y="96" width="220" height="32" rx="3" fill="#f1ebdc" stroke="#b07a4a" stroke-width="1"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="350" y="116" text-anchor="middle">my_q → PGBOUNCER_42</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="148" text-anchor="middle">prepares PGBOUNCER_42 on</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="164" text-anchor="middle">whichever backend the client</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="180" text-anchor="middle">lands on, transparently</text>

<rect x="540" y="65" width="140" height="50" rx="3" fill="#fff" stroke="#6a7d63" stroke-width="1.2"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="610" y="82" text-anchor="middle">backend 1</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="610" y="98" text-anchor="middle">PGBOUNCER_42 ✓</text>

<rect x="540" y="135" width="140" height="50" rx="3" fill="#fff" stroke="#6a7d63" stroke-width="1.2"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="610" y="152" text-anchor="middle">backend 2</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="610" y="168" text-anchor="middle">PGBOUNCER_42 ✓</text>

<rect x="540" y="205" width="140" height="50" rx="3" fill="#fff" stroke="#6a7d63" stroke-width="1.2"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="610" y="222" text-anchor="middle">backend 3</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="610" y="238" text-anchor="middle">PGBOUNCER_42 ✓</text>

<text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="290" text-anchor="middle">Same prepared SQL string is interned once. Each backend gets it</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="306" text-anchor="middle">prepared on first use, cached as PGBOUNCER_42, reused thereafter.</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="322" text-anchor="middle">The client thinks it's still talking about my_q. Everything stays consistent.</text> </svg> <figcaption><strong>Fig 4.</strong> How PgBouncer 1.21+ handles prepared statements under transaction pooling. The client's name is mapped to a PgBouncer-internal name. The same SQL string used by many clients gets a single internal identity. Each backend lazily prepares the statement on first use and caches it, bounded by <code>max_prepared_statements</code>.</figcaption> </figure>

As of PgBouncer 1.24.0 (January 2025), prepared statement support is enabled by default with max_prepared_statements = 200. If you're on a fresh deploy from 2025 onward, you have it for free. If you're on an older config, set the value yourself; reasonable values are 100 to 500 depending on how many distinct prepared statements your app actually uses. Higher means more memory per backend, both in PgBouncer's tracking and in Postgres' plan cache. The Tiger Data team recommends matching your max_prepared_statements to the count of distinct statements your ORM emits, so the cache fits exactly.

Two caveats remain. First, this only applies to protocol-level prepared statements, the ones your driver sends as Parse/Bind/Execute messages. SQL-level PREPARE foo AS ... followed by EXECUTE foo is still unsupported, because PgBouncer doesn't parse SQL and can't see those statements coming. If your app uses raw PREPARE, you have to stop. If your app uses an ORM or any modern driver, it's almost certainly using the protocol level, and you're fine.

Second, and this is irritating: client-side DEALLOCATE for individual prepared statements is still not transparently rewritten. DEALLOCATE ALL and DISCARD ALL are handled (added in PgBouncer 1.22, January 2024) and correctly clear PgBouncer's tracking. But a single DEALLOCATE my_q goes straight through to whichever backend the client happens to be on, which may not have the statement under that name, throwing an error. The protocol-level Close message for prepared statements has been part of Postgres' wire protocol for years; what's new is libpq 17 (Postgres 17, September 2024) exposing it via PQclosePrepared, so drivers can finally deallocate without sending raw SQL. Driver adoption is uneven. The pragmatic fix in older drivers is to disable client-side deallocation and trust PgBouncer's LRU to evict old statements. PHP/PDO needs PHP 8.4+ and libpq 17 to be compatible with PgBouncer's prepared-statement support; for older PHP, set PDO::ATTR_EMULATE_PREPARES = true. Most other modern drivers (psycopg3, JDBC, Go's pgx) work but each has their own knob for controlling deallocation behavior. Verify before you ship.

11. The other things that break, and what to do about them

Prepared statements got fixed. Most of the other items on the compatibility list didn't, and probably won't, because the underlying issue is fundamental to transaction pooling: any feature that depends on persistent session state across transactions is incompatible with a model where transactions get scattered across different backends. You can't paper over that without losing the multiplexing.

The list of practical things this means:

SET commands. SET search_path = 'tenant_42' at the start of a request will succeed, but as soon as the request commits and starts a new transaction, the next backend you land on doesn't have that search_path. The fix is SET LOCAL, which is scoped to the current transaction. If you need session-scoped settings (multi-tenant search paths are the most common case), you have to either set them on every transaction, or set them at the role level with ALTER ROLE ... SET search_path = ..., or use parameters PgBouncer specifically tracks across transactions. The default tracked set is small: client_encoding, application_name, DateStyle, TimeZone, standard_conforming_strings, and (since 1.20) IntervalStyle. You can extend this list with track_extra_parameters, but only for parameters that Postgres reports back to the client (the GUC must be marked GUC_REPORT). For search_path specifically, this works on Citus 12+ and on Postgres 18, both of which mark search_path as reportable. On older Postgres versions, you cannot track search_path through PgBouncer no matter what you set; see the CVE note in the auth section for why this matters.

statement_timeout, the cursed one. This is the bug that JP Camara wrote up and it's worth understanding because it bites people who think they understand the rules. If you set statement_timeout at the session level (SET statement_timeout = '5s'), and you set it on a backend that gets returned to the pool with that value still set, the next client to grab that backend inherits the 5-second timeout. They didn't ask for it. They might be a long-running background job. They will get killed at 5 seconds and never know why. This is solved by server_reset_query = DISCARD ALL in session pooling, which resets everything. But in transaction pooling mode, server_reset_query is not run by default, because the assumption is "you weren't doing anything session-y in the first place." If you set statement_timeout at the session level inside transaction pooling, you are leaking it to other clients. The fix is either to use SET LOCAL statement_timeout inside an explicit transaction, or to set the timeout at the role level with ALTER ROLE web_app SET statement_timeout = '5s' so it applies on connection establishment and never needs to change.

LISTEN / NOTIFY. LISTEN registers an interest on a specific backend. In transaction pooling, you'll only receive notifications when you happen to land on that exact backend, which is essentially never. You can keep a separate direct connection (port 5432) for the LISTEN side and use the pooler for everything else, which works but undermines the pooling savings. For real pub/sub, use a real pub/sub system.

Session-level advisory locks. Worse than the others. pg_advisory_lock(123) takes a lock that lives for the backend's session, not the client's transaction. In transaction pooling, your transaction ends at COMMIT, but the backend keeps running, holding your lock. Whoever lands on that backend next inherits the lock — meaning they can silently block on it, or release it themselves with an unrelated pg_advisory_unlock(123), or step on each other's locking logic in ways that look impossible from the application side. Use pg_advisory_xact_lock(123) if your locking scope fits inside a transaction, or use a real distributed lock if it doesn't.

Temporary tables that outlive a transaction. A temp table created with ON COMMIT DROP inside a single transaction works fine in transaction pooling. ON COMMIT PRESERVE ROWS (the default) creates a session-scoped temp table that won't be there next transaction. Most ORMs don't use temp tables, so this is mainly a problem for analytics tools and ETL.

pg_dump. Don't run pg_dump through PgBouncer. It expects session-level state and behaves badly. Use a direct connection for backups, period.

WITH HOLD cursors and LOAD. Both session-y. Both broken. Both rare enough that most apps don't notice.

The mental model that makes this all easier. Stop thinking of an app's connection to the database as a long-running thing. Start thinking of it as a request for a transaction. Each transaction is a brief encounter with some backend, and after it ends, you have no relationship to that backend. Anything you want to persist either has to be small (and PgBouncer-tracked, like application_name), or has to live somewhere that isn't a backend's memory: the database itself, your app's memory, Redis, whatever. As soon as you internalize this, the gotchas stop being surprises and start being obvious consequences of the pooling rules.

Checkpoint: what you know now.

Postgres uses one OS process per connection. Each connection costs a few MB of private memory plus scheduler and snapshot overhead, and the connect-time fork/TLS dance is multiple milliseconds.

PgBouncer is a single-threaded libevent-based proxy that multiplexes many client connections onto fewer real Postgres backends. It has three modes: session (one backend per client lifetime, mostly safe), transaction (backend bound only during BEGIN…COMMIT, the productive default), and statement (released after each query, too aggressive for normal apps).

Transaction mode breaks any feature that depends on session state spanning transactions. The most-bitten items are SET, LISTEN, session-scoped advisory locks, persistent temp tables, and statement-level PREPARE. Protocol-level prepared statements work as of 1.21 (2023), enabled by default since 1.24 (January 2025).

Act 3 — Running it for real

Sizing, auth, deployment topology, and the small operational decisions that determine whether you sleep through the night.

12. Sizing the pool

This is the question I get asked most often, and the one that most blog posts answer poorly. The honest answer is: do the math, and the math starts at the database, not the app.

Step one: figure out how many concurrent transactions Postgres can actually handle. There's no formula, but a useful starting point is "two to four times the number of CPU cores." A 16-core database server can probably comfortably run 32 to 64 concurrent active transactions. Past that, you're queueing on CPU or I/O, and adding more concurrent transactions makes everything slower without making throughput higher. This is your ceiling.

Step two: pick a default_pool_size below that ceiling, leaving headroom. If your ceiling is 50, set the pool size to 20 to 30. The pool size is per (database, user) pair: if you have one app user hitting one database, that's the whole pool. If you have multiple roles or multiple databases, those are separate pools that share the database's overall connection budget.

Step three: pick a max_client_conn that's much larger than the pool size. This is the cap on how many clients can connect to PgBouncer at all. Set it to your worst-case scenario: total worker count across all your app servers, plus headroom. 1,000 to 5,000 is normal. The whole reason you have PgBouncer is so that this number can be much larger than your pool size.

Step four: set max_db_connections as a safety cap. This is the absolute upper bound on real Postgres connections from this PgBouncer to this database, regardless of pool counts. Set it slightly below your Postgres max_connections, leaving room for direct admin sessions and replication.

There's a more analytical formulation that's worth knowing about, even if you don't use it. The pool size you actually need is roughly the peak rate of new transactions multiplied by the average transaction duration. If you're processing 500 transactions per second and each takes 20 ms, that's 500 × 0.02 = 10 active transactions on average. Round up generously, because the average hides spikes. Pool size of 20 is a reasonable starting point. This is Little's Law applied to transactions.

# A reasonable starting pgbouncer.ini for a single-tenant web app

[databases]
appdb = host=db.internal port=5432 dbname=appdb

[pgbouncer]
listen_addr     = 0.0.0.0
listen_port     = 6432
auth_type       = scram-sha-256
auth_file       = /etc/pgbouncer/userlist.txt

pool_mode             = transaction
max_client_conn       = 2000
default_pool_size     = 25
min_pool_size         = 5
reserve_pool_size     = 5
reserve_pool_timeout  = 3
max_db_connections    = 80
max_prepared_statements = 200

server_idle_timeout   = 600
server_lifetime       = 3600
query_wait_timeout    = 60

ignore_startup_parameters = extra_float_digits,options

Two settings deserve a footnote. reserve_pool_size and reserve_pool_timeout are PgBouncer's burst valve. If the regular pool is exhausted and clients have been waiting for more than reserve_pool_timeout seconds, PgBouncer can open up to reserve_pool_size additional backends to absorb the spike. Set this small. It's a release valve, not a primary sizing knob. If your reserve pool is firing constantly, your default_pool_size is too small.

And min_pool_size tells PgBouncer to keep this many connections warm even when nothing is using them. The classic use case is avoiding cold-start fork latency on the first request after a quiet period. Don't set it too high; idle warmed connections are still real backends consuming real memory.

13. Where to put PgBouncer

There are three reasonable places to deploy PgBouncer, and the choice has real consequences.

On the database host. One PgBouncer process running on the same machine as Postgres, listening on port 6432. This is the simplest setup and what most managed providers do (Azure Postgres, Crunchy Bridge, PlanetScale, Timescale Cloud). Latency between the pooler and the database is sub-millisecond because they're on a Unix socket or loopback. The downside is that if the database host falls over, the pooler falls over with it. Failover requires the standby to also have PgBouncer running.

On each app host (sidecar pattern). One PgBouncer per application server, listening on localhost. Each app's connection goes to its local PgBouncer over a Unix socket. The PgBouncers all then connect across the network to Postgres. Pros: no single point of failure on the pooler tier. Cons: you have N PgBouncers, each maintaining their own pool, so the total connection budget is harder to reason about. If each app host has a pool of 25 and you have 10 hosts, you've got 250 server connections, not 25.

On a dedicated pooler tier. A small number (2 to 4) of dedicated PgBouncer hosts behind a load balancer or VIP, fed by all app servers. The setup PgBouncer's docs explicitly support uses Linux's SO_REUSEPORT with multiple PgBouncer processes listening on the same port on the same host, which gives you parallelism without breaking the single-threaded model. This is the right pattern when your app servers fluctuate but your pooler tier should stay stable. It's also the most operationally complex.

For most small to medium apps, "on the database host" is the answer and worrying about the others is premature. For larger systems, the sidecar pattern is the most resilient if you can do the math on total connections. The dedicated tier is where the very high-throughput shops end up.

<figure> <svg viewBox="0 0 700 320" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Three deployment topologies: pooler on the database host, pooler as a sidecar, and a dedicated pooler tier" width="100%" height="auto"> <rect fill="#f4ede0" x="0" y="0" width="700" height="320" rx="6"/> <text font-family="serif" font-size="13" fill="#1a1815" x="120" y="30" text-anchor="middle" font-weight="600">on db host</text> <rect x="60" y="50" width="120" height="50" rx="3" fill="#fff" stroke="#4a6f88" stroke-width="1"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="120" y="80" text-anchor="middle">app servers</text> <rect x="60" y="155" width="120" height="100" rx="3" fill="#fff" stroke="#1a1815" stroke-width="1.4"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="120" y="180" text-anchor="middle">db host</text> <rect x="74" y="190" width="92" height="22" rx="3" fill="#f1ebdc" stroke="#b07a4a"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="120" y="206" text-anchor="middle">PgBouncer</text> <rect x="74" y="220" width="92" height="22" rx="3" fill="#fff" stroke="#6a7d63"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="120" y="236" text-anchor="middle">postgres</text> <line x1="120" y1="100" x2="120" y2="155" stroke="#4a4742" stroke-width="1"/> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="120" y="285" text-anchor="middle">simplest, single-host risk</text>

<text font-family="serif" font-size="13" fill="#1a1815" x="350" y="30" text-anchor="middle" font-weight="600">sidecar</text> <g> <rect x="280" y="50" width="60" height="60" rx="3" fill="#fff" stroke="#4a6f88" stroke-width="1"/> <rect x="288" y="80" width="44" height="22" rx="2" fill="#f1ebdc" stroke="#b07a4a" stroke-width="0.8"/> <text font-family="JetBrains Mono" fill="#1a1815" x="310" y="68" text-anchor="middle" font-size="10">app1</text> <text font-family="JetBrains Mono" fill="#1a1815" x="310" y="96" text-anchor="middle" font-size="10">PgB</text>

<rect x="350" y="50" width="60" height="60" rx="3" fill="#fff" stroke="#4a6f88" stroke-width="1"/>
<rect x="358" y="80" width="44" height="22" rx="2" fill="#f1ebdc" stroke="#b07a4a" stroke-width="0.8"/>
<text font-family="JetBrains Mono" fill="#1a1815" x="380" y="68" text-anchor="middle" font-size="10">app2</text>
<text font-family="JetBrains Mono" fill="#1a1815" x="380" y="96" text-anchor="middle" font-size="10">PgB</text>

</g> <rect x="290" y="170" width="120" height="50" rx="3" fill="#fff" stroke="#6a7d63" stroke-width="1"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="350" y="200" text-anchor="middle">postgres</text> <g stroke="#4a4742" fill="none" stroke-width="1"> <path d="M 310 110 L 320 170"/> <path d="M 380 110 L 380 170"/> </g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="285" text-anchor="middle">resilient, harder to reason</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="350" y="300" text-anchor="middle">about total connections</text>

<text font-family="serif" font-size="13" fill="#1a1815" x="580" y="30" text-anchor="middle" font-weight="600">dedicated tier</text> <g> <rect x="510" y="50" width="60" height="40" rx="3" fill="#fff" stroke="#4a6f88" stroke-width="1"/> <text font-family="JetBrains Mono" fill="#1a1815" x="540" y="76" text-anchor="middle" font-size="10">app1</text> <rect x="580" y="50" width="60" height="40" rx="3" fill="#fff" stroke="#4a6f88" stroke-width="1"/> <text font-family="JetBrains Mono" fill="#1a1815" x="610" y="76" text-anchor="middle" font-size="10">app2</text> </g> <g> <rect x="510" y="125" width="60" height="40" rx="3" fill="#f1ebdc" stroke="#b07a4a" stroke-width="1"/> <text font-family="JetBrains Mono" fill="#1a1815" x="540" y="150" text-anchor="middle" font-size="10">PgB</text> <rect x="580" y="125" width="60" height="40" rx="3" fill="#f1ebdc" stroke="#b07a4a" stroke-width="1"/> <text font-family="JetBrains Mono" fill="#1a1815" x="610" y="150" text-anchor="middle" font-size="10">PgB</text> </g> <rect x="510" y="200" width="130" height="40" rx="3" fill="#fff" stroke="#6a7d63" stroke-width="1"/> <text font-family="JetBrains Mono" font-size="12" fill="#1a1815" x="575" y="225" text-anchor="middle">postgres</text> <g stroke="#4a4742" fill="none" stroke-width="1"> <path d="M 540 90 L 540 125"/> <path d="M 610 90 L 610 125"/> <path d="M 540 165 L 575 200"/> <path d="M 610 165 L 575 200"/> </g> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="575" y="285" text-anchor="middle">most operationally</text> <text font-family="JetBrains Mono" font-size="12" fill="#4a4742" x="575" y="300" text-anchor="middle">flexible, most complex</text> </svg> <figcaption><strong>Fig 5.</strong> Three deployment topologies. The first one is what most managed Postgres providers do for you and is the right starting point. The third is what shops with high enough scale to justify a dedicated pooler tier do, often with <code>SO_REUSEPORT</code>-based parallelism on each pooler host.</figcaption> </figure>

14. Authentication, and the auth_query trick

By default, PgBouncer keeps its own user list in a file (userlist.txt), with one line per user containing the username and a password (plain text, MD5, or SCRAM secret). This is fine for tiny setups. For anything with more than a handful of users, or where users get added dynamically, you do not want to maintain a parallel password file in sync with Postgres' own auth.

The right pattern is auth_query. You give PgBouncer a single privileged user (call it pgbouncer), and you tell it to look up other users' password hashes by querying Postgres directly. The setup looks like this:

-- In Postgres, once per database:
CREATE ROLE pgbouncer LOGIN PASSWORD '...';

CREATE SCHEMA IF NOT EXISTS pgbouncer;
REVOKE ALL ON SCHEMA pgbouncer FROM PUBLIC;
GRANT USAGE ON SCHEMA pgbouncer TO pgbouncer;

CREATE OR REPLACE FUNCTION pgbouncer.get_auth(p_username TEXT)
RETURNS TABLE(username TEXT, password TEXT)
LANGUAGE sql
SECURITY DEFINER
SET search_path = pg_catalog, pg_temp
AS $$
    SELECT rolname::TEXT, rolpassword::TEXT
    FROM pg_catalog.pg_authid
    WHERE rolname = p_username
      AND rolcanlogin
      AND NOT rolsuper
      AND (rolvaliduntil IS NULL OR rolvaliduntil > CURRENT_TIMESTAMP);
$$;

REVOKE ALL ON FUNCTION pgbouncer.get_auth(TEXT) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION pgbouncer.get_auth(TEXT) TO pgbouncer;

# In pgbouncer.ini:
auth_type  = scram-sha-256
auth_user  = pgbouncer
auth_query = SELECT username, password FROM pgbouncer.get_auth($1)

# userlist.txt now only needs the pgbouncer user itself:
"pgbouncer" "SCRAM-SHA-256$4096:..."

Now any non-superuser role with login privilege can authenticate through PgBouncer without ever appearing in PgBouncer's own files. Adding a user is a single Postgres CREATE ROLE; rotating a password is a single ALTER ROLE.

Two things worth flagging, both of which have shown up as real CVEs in 2025. First, the SET search_path = pg_catalog, pg_temp line in the function definition is not optional. Without it, an attacker can hijack the function's name resolution by getting the right object to live earlier in the search path. CVE-2025-12819, fixed in PgBouncer 1.25.1 (December 2025), was a real-world example. When track_extra_parameters includes search_path and auth_user is configured, a malicious search_path in the client's StartupMessage propagates into the auth-time auth_query. PgBouncer's shipped default query contained an unqualified operator (<), which the attacker's schema could redefine, executing arbitrary SQL during authentication. The hardened example above pins the search path inside the function and schema-qualifies pg_authid, which neutralizes the attack. Always schema-qualify your catalog references in SECURITY DEFINER functions, and always pin the search path.

Second, notice the (rolvaliduntil IS NULL OR rolvaliduntil > CURRENT_TIMESTAMP) clause. This is also not optional. CVE-2025-2291, fixed in PgBouncer 1.24.1 (April 2025), was the embarrassing case where the shipped default auth_query didn't check VALID UNTIL, so an expired password would still authenticate through the pooler even though it wouldn't authenticate against Postgres directly. The default was fixed; if you wrote a custom auth_query before April 2025, double-check that you're filtering on rolvaliduntil. The example above does. Many older blog post examples don't.

Third, end-to-end SCRAM authentication is delicate. PgBouncer can pass SCRAM through to Postgres, but only when the SCRAM secrets in PgBouncer's userlist exactly match Postgres' (same salt, same iterations, not just the same password). Otherwise PgBouncer has to re-authenticate to Postgres on a separate channel, which introduces its own subtleties around credentials. For most setups, auth_query with a single privileged pgbouncer role is simpler and safer than trying to mirror SCRAM secrets exactly.

15. Watching it run

PgBouncer has a special database called pgbouncer that you can connect to (with an admin user) and run a small set of SHOW commands. They're how you understand what's actually happening.

The two most useful are SHOW POOLS and SHOW STATS. SHOW POOLS tells you, for each (database, user) pool, how many client connections are active or waiting, how many server connections are in each state (active, idle, used, tested), and what the pool's mode is. SHOW STATS gives you per-database counters: total transactions, total queries, total bytes, and the average wait time for clients to be assigned a server connection. That last number is the one that tells you whether your sizing is right.

Here's what I look at first when something is wrong:

cl_waiting in SHOW POOLS: how many clients are queued, waiting for a backend. If this is consistently nonzero, your pool is too small or your transactions are too slow.
avg_wait_time in SHOW STATS: average microseconds clients spent waiting for a server connection. Should be near zero. Tens of milliseconds means contention.
sv_active vs sv_idle: server connections currently doing work versus sitting in the pool. If sv_active is at the pool size and cl_waiting is climbing, you have your answer.
avg_xact_time and avg_query_time: how long the average transaction and query take. If transactions are way longer than queries, somebody is leaving transactions open.

You can scrape these into Prometheus with one of several PgBouncer exporters. The metric you most want a dashboard on is cl_waiting over time. The metric you most want an alert on is sustained cl_waiting > 0 for more than a minute or two, which means your pool sizing is wrong or your queries are slow.

16. Operational levers added in the last two years

A few features added since 2024 are worth knowing about because they fill specific operational gaps that used to require ugly workarounds.

transaction_timeout (1.25.0). Caps the duration of any transaction at the pooler level. Without this, a client that opens a transaction and forgets to commit holds a backend forever, and your only recourse is killing the backend manually or waiting for idle_in_transaction_session_timeout on the Postgres side. Set this to something reasonable (60 seconds is conservative for OLTP, longer for batch jobs that need their own pool) and runaway transactions clear themselves.

KILL_CLIENT (1.24.0). Lets you kill a specific client connection from the admin console without bouncing the whole pool. Combined with the unique IDs in SHOW CLIENTS output, you can surgically remove the one bad actor instead of the nuclear KILL pool. The combination of SHOW CLIENTS to find the offender and KILL_CLIENT to evict them is the modern equivalent of pg_terminate_backend() at the pooler tier.

max_user_client_connections and max_db_client_connections (1.24.0). These cap how many clients can connect to PgBouncer for a given user or database, separately from the cap on backend connections. The intuition: if a misbehaving service tries to open ten thousand connections to your pooler, the pooler accepts all of them and starves everyone else. These limits give you per-tenant fairness without splitting into separate PgBouncer instances.

query_wait_notify (1.25.0). When a client has been queued for a server connection for more than 5 seconds, PgBouncer sends a NOTICE message ("queued for X seconds"). This is great when debugging "why is my query slow" reports, because the latency might be entirely waiting on the pool, not the database. The default threshold is configurable.

None of these are exotic. They're the kind of features that an SRE spending time with PgBouncer eventually wants. If you're running an older 1.21 or 1.22 from a Linux distro package, an upgrade buys you these.

17. A real story: the Mattermost case

The Mattermost team published a postmortem in 2022 that's worth reading because it shows how pool-size choices interact with multi-database/multi-tenant deployments in non-obvious ways.

Mattermost runs thousands of customer workspaces grouped 10 to 15 to a logical database in a shared Postgres cluster, with each workspace getting its own PgBouncer pool. They put PgBouncer in front. Their initial config looked reasonable: pool_mode = transaction, min_pool_size = 20, default_pool_size = 20, reserve_pool_size = 5, max_client_conn = 10000, max_db_connections = 20.

Spot the problem? Each workspace gets its own pool of size 20. max_db_connections = 20 is the cap on real Postgres connections from all workspaces combined. With 10 to 15 workspaces per database, every workspace was effectively competing for those 20 backend slots, and any time more than one workspace had a burst of activity, they were starving each other. The visible symptom was high CPU on the RDS cluster from connection churn: PgBouncer was constantly opening and closing real connections trying to satisfy workspaces that were each demanding more than the global cap allowed.

The fix wasn't to raise the per-pool size proportionally. They dropped min_pool_size from 20 to 1 and default_pool_size from 20 to 5, so PgBouncer wasn't aggressively keeping warm connections that other pools needed. They raised reserve_pool_size to 20 and shortened reserve_pool_timeout from 5 seconds to 1, so bursty workspaces could grab reserve connections quickly. And critically, they raised max_db_connections from 20 to 100, so the global cap stopped being the bottleneck. The lesson is that in multi-tenant setups, pool sizes are not independent. They share a global cap, and configurations that look fine in isolation can deadlock or thrash when multiplied across tenants.

There was a secondary lesson too. They discovered that some of their migrations used pg_advisory_lock at the session level, which doesn't work in transaction pooling. They had to either route migrations to a direct connection or rewrite them to use transaction-scoped locks. This is the kind of thing that doesn't show up in load testing and only surfaces when you actually run the migration against the pooled connection, so it's worth auditing your migration tooling explicitly when you adopt PgBouncer.

18. When PgBouncer is the wrong tool

It's worth saying out loud: PgBouncer doesn't fix every problem.

If your app is genuinely doing heavy work on every connection, transaction pooling won't help much. Multiplexing only saves you if the connections are mostly idle. A worker process that's running queries 100% of the time will hold a backend the whole time, and PgBouncer adds latency without giving you ratio gains.

If you need session-level features that don't have transaction-level analogs, transaction pooling will fight you. LISTEN/NOTIFY-heavy designs, long-running cursors, session-locked migrations: stay on session pooling, or use direct connections for those code paths and pool the rest.

If you need read/write splitting, query routing, automatic failover, or replication management, PgBouncer doesn't do those. Pgpool-II does, at the cost of being much more complex (multi-process, parses SQL, has its own opinions about your queries). Recent Rust-based alternatives like PgCat are multi-threaded and add some routing features. Cloud-managed proxies like AWS RDS Proxy or Azure's built-in PgBouncer take the operational burden off you. Each of these is a real tradeoff. PgBouncer's appeal is that it does one thing and does it cleanly. The minute you want it to do more, you're better off picking a tool that's meant for the more.

And if the real problem is that your queries are slow, PgBouncer can't help you. A 2-second query holding a backend for 2 seconds is a backend that no other client can use, regardless of how many clients are waiting in PgBouncer's queue. Connection pooling is a multiplier on the time backends spend not running queries. If they're always running queries, the multiplier is 1.

19. Three ideas worth stealing

If you only take three things away from all this, I'd pick these.

Pool what's expensive, not what's logical. Postgres connections are expensive because they map 1:1 onto OS processes. App-side connections are cheap because they're just file descriptors. Resist the instinct to make your app's connection count match your real backend count. They aren't the same kind of thing. The same logic shows up everywhere: HTTP keepalive, TCP connection reuse, request-scoped sessions in web frameworks. Whenever you have a cheap thing the application thinks of as "the same as" an expensive thing, there's a multiplexer in your future.

Lying transparently is a strategy, until it isn't. Transaction pooling works because it lies about what a session is, and the lie holds for most apps because most apps don't actually use sessions for anything. The interesting question isn't "should I pool" but "which session features does my app accidentally rely on, and can I take them out?" If you can, pool aggressively. If you can't, the lying breaks down and you should know which lies you're tolerating. The real engineering work is the audit, not the config.

The expensive case is the one you'll meet in production. All the gotchas in this piece (statement_timeout leakage, search_path drift, advisory lock evaporation) only matter in the rare case. In the common case, transaction pooling is clean and obvious and your app works. The reason this stuff is worth knowing is because the rare cases are the ones that cause incidents, and incidents are the only times you'll be debugging connection pooling at all. The cost of learning this in a calm Tuesday afternoon is a lot lower than the cost of learning it during an outage.

Epilogue

The history of databases is full of these layers that don't quite fit. Postgres is great at being a transactional database. It is not great at handling a thousand short-lived connections, because that workload didn't exist when its connection model was designed. Putting PgBouncer in front of it is a fix for an impedance mismatch between two eras of computing: the one where a database was something a few analysts logged into directly, and the one where it's something serving billions of HTTP requests through dozens of stateless web workers.

What's interesting is that the fix is not a Postgres feature. It's a separate program, written by different people, that intercepts the wire protocol and shuffles bytes. It works because Postgres' protocol is open and stable, and because the place where the impedance mismatch lives (the cost of a backend process) is upstream of anything the app cares about. PgBouncer sits at the seam.

You can read the development of features like protocol-level prepared statements in 1.21 as a slow tightening of that seam. The original PgBouncer was content to break a lot of session features outright, on the assumption that the multiplexing was worth it. Successive versions have figured out, one by one, how to support more session-flavored things without giving up the multiplexing. The pattern repeats: find a feature that breaks, look for a way to track its state across pooled backends, ship the fix. Prepared statements were the big one. track_extra_parameters handles GUC propagation. LDAP support landed in 1.25 (November 2025). The seam stays a seam, but it gets less visible every release.

There's no big lesson at the end of this. PgBouncer is a small program that does a small thing well. Most of the value of writing about it is just demystifying it, so that the next time someone runs into one of its sharp edges, they know it's a sharp edge and not magic. Postgres' architecture is what it is. The pooler is a workaround. The workaround is solid, but it's a workaround, and knowing that is most of what you need to use it well.

The connection isn't the thing. The transaction is the thing. Everything else is bookkeeping.

References and further reading

PgBouncer 1.25.1 release and full changelog. pgbouncer.org/changelog.
PgBouncer features and pooling-mode compatibility table. pgbouncer.org/features.
PgBouncer configuration reference, including auth_query, max_prepared_statements, and the full pgbouncer.ini schema. pgbouncer.org/config.
Greg Sabino Mullane, "Prepared Statements in Transaction Mode for PgBouncer," Crunchy Data, 2023. The canonical writeup of how 1.21's prepared-statement support actually works.
JP Camara, "PgBouncer is useful, important, and fraught with peril," 2023. The best long-form writeup I've read on transaction-mode gotchas, with worked examples for statement_timeout leakage, advisory locks, and PHP/PDO quirks.
Andres Freund, "Measuring the Memory Overhead of a Postgres Connection," 2020. The careful per-connection memory measurement that "10 MB" comes from, with caveats.
Crunchy Data, "How to SCRAM in Postgres with pgBouncer." Walks through the SCRAM protocol mechanics that let PgBouncer authenticate to Postgres without ever seeing the plaintext password.
Mattermost engineering, "Optimizing Database Connection Loads With PgBouncer and Testwick," 2022. The multi-tenant pool-sizing case study used in this piece.
Cybertec, "PgBouncer Authentication Made Easy" (Laurenz Albe). Practical SCRAM and auth_query setup with a working SECURITY DEFINER pattern.
Heroku, "Best Practices for PgBouncer Configuration." Production-flavored sizing guidance and discussion of preboot interaction.
Microsoft Azure docs, "PgBouncer in Azure Database for PostgreSQL." Documents the managed-deployment pattern that ships PgBouncer 1.25.1 on the same VM as the database.

The Unified Vision Stack

Fri, 24 Apr 2026 00:00:00 GMT

An end-to-end, intuition-first explanation of how modern computer vision works, and why three papers published between late 2024 and early 2026 quietly changed the standard stack. For engineers new to the field. About a 45-minute read.

1. What a vision model actually does

Before anything else, we have to agree on what a computer vision model is supposed to produce. If you've used ML libraries you've probably called a function like model.predict(image) and gotten back a label. But the label is not really what the model makes. The label is what a tiny head stapled onto the model makes, using what the model actually produces: a pile of numbers.

That pile of numbers is called an embedding, or a feature, or a representation (the terms are used interchangeably). An image comes in. A vector comes out. The vector is a compact description of what's in the image, expressed in a language the model invented for itself during training.

Everything downstream of a vision model is a function from this vector to something useful. Classification is a small function that maps the vector to a list of probabilities. Semantic segmentation is a larger function that maps it to a per-pixel label. Image retrieval uses the vector directly. Find other vectors close to this one, and you've found similar images. Depth estimation, object detection, video tracking, visual question answering: all of them are small heads attached to a big backbone. The backbone is where the work lives.

So when we talk about vision foundation models, we're talking about one specific thing: the function that turns a raw image into a vector. The question that the entire field is really asking is: what makes one vector better than another?

A useful analogy for engineers: the vision backbone is like a compiler's intermediate representation. The IR is not what you care about directly, but every single optimization pass and every single backend target uses it. Get the IR right and every downstream pass gets easier. Get it wrong and you pay for it forever.

2. Two kinds of features, and why it matters

Here's where we need to make a distinction that will shape the rest of this piece. A vision model produces two different kinds of features, and they have very different jobs.

The first is a global feature. It's a single vector that describes the whole image. If the image is a golden retriever running on a beach, the global feature is something like "a vector that means beach-with-dog." You can compare it to other global features to find similar images. You can run a linear classifier on it to ask "is there a dog in this picture?" It is a summary.

The second is a set of dense features: one vector per region of the image. If you imagine the image cut into a grid of small patches, each patch gets its own vector. The dense features are a map. Patches of fur get one kind of vector, patches of sky get another, patches of sand get a third. The spatial arrangement is preserved: the vector in the top-right of the feature map corresponds to what was in the top-right of the image.

<figure> <svg viewBox="0 0 680 260" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <rect x="30" y="40" width="180" height="180" fill="#EDE5D4" stroke="#8F8578"/> <rect x="30" y="40" width="180" height="70" fill="#93C5FD" opacity=".7"/> <rect x="30" y="110" width="180" height="50" fill="#D1AE7A" opacity=".8"/> <rect x="30" y="160" width="180" height="60" fill="#059669" opacity=".6"/> <circle cx="155" cy="75" r="12" fill="#FEF3C7"/> <text x="120" y="240" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#6B6258">input image</text> <line x1="225" y1="130" x2="265" y2="130" stroke="#6B6258" stroke-width="1.5" marker-end="url(#a1)"/> <defs><marker id="a1" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="6" markerHeight="6" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="#6B6258"/></marker></defs> <text x="395" y="55" font-family="JetBrains Mono" font-size="10" fill="#6B6258" letter-spacing="1.5">GLOBAL FEATURE</text> <rect x="280" y="65" width="260" height="28" fill="none" stroke="#2563EB"/> <g font-family="JetBrains Mono" font-size="11" fill="#2563EB"><text x="290" y="84">0.7</text><text x="315" y="84">-0.2</text><text x="345" y="84">0.1</text><text x="368" y="84">0.5</text><text x="393" y="84">0.0</text><text x="418" y="84">-0.4</text><text x="450" y="84">0.8</text><text x="478" y="84">...</text><text x="505" y="84">0.3</text></g> <text x="395" y="115" text-anchor="middle" font-family="serif" font-size="12" fill="#6B6258" font-style="italic">one vector. says "beach with dog."</text> <text x="395" y="150" font-family="JetBrains Mono" font-size="10" fill="#6B6258" letter-spacing="1.5">DENSE FEATURES</text> <g transform="translate(280,160)"> <rect x="0" y="0" width="38" height="20" fill="#93C5FD"/><rect x="40" y="0" width="38" height="20" fill="#93C5FD"/><rect x="80" y="0" width="38" height="20" fill="#93C5FD"/><rect x="120" y="0" width="38" height="20" fill="#FEF3C7"/> <rect x="0" y="22" width="38" height="20" fill="#D1AE7A"/><rect x="40" y="22" width="38" height="20" fill="#D1AE7A"/><rect x="80" y="22" width="38" height="20" fill="#92400E"/><rect x="120" y="22" width="38" height="20" fill="#D1AE7A"/> <rect x="0" y="44" width="38" height="20" fill="#059669"/><rect x="40" y="44" width="38" height="20" fill="#059669"/><rect x="80" y="44" width="38" height="20" fill="#059669"/><rect x="120" y="44" width="38" height="20" fill="#059669"/> <g fill="none" stroke="#F4EEE3"><rect x="0" y="0" width="38" height="20"/><rect x="40" y="0" width="38" height="20"/><rect x="80" y="0" width="38" height="20"/><rect x="120" y="0" width="38" height="20"/><rect x="0" y="22" width="38" height="20"/><rect x="40" y="22" width="38" height="20"/><rect x="80" y="22" width="38" height="20"/><rect x="120" y="22" width="38" height="20"/><rect x="0" y="44" width="38" height="20"/><rect x="40" y="44" width="38" height="20"/><rect x="80" y="44" width="38" height="20"/><rect x="120" y="44" width="38" height="20"/></g> </g> <text x="395" y="245" text-anchor="middle" font-family="serif" font-size="12" fill="#6B6258" font-style="italic">a grid of vectors. says "sky here, dog there, sand below."</text> </svg> <figcaption><strong>Fig 1.</strong> The same image produces two kinds of output. A single global vector is enough to classify the whole image. A grid of dense vectors is required to tell one region from another.</figcaption> </figure>

Here's why the distinction matters enormously. For a long time, the dominant vision model in ML was OpenAI's CLIP, trained on hundreds of millions of image-caption pairs, designed to produce one clean global feature per image. CLIP is great at classification, retrieval, and zero-shot labeling. For dense tasks (segmentation, depth, anything pixel-precise) CLIP features are mush. They were never optimized for spatial detail, so they don't have it.

If you want a single vision backbone that can do everything (classify, segment, track, measure depth, correspond points between frames), you need both kinds of features to be good. Global features that summarize well. Dense features that preserve local geometry. Seems like an obvious thing to want. Turns out, until very recently, nobody knew how to train a single network to produce both at scale.

3. How a Vision Transformer sees

Modern vision is dominated by a single architecture family: the Vision Transformer, or ViT. It's worth spending a paragraph on how one works, because the rest of the story depends on it.

A ViT takes an image (say a 224×224 RGB picture) and chops it into a grid of non-overlapping patches, typically 16×16 pixels each. That's 14×14 = 196 patches. Each patch is flattened into a vector of pixel values and linearly projected to a fixed-width embedding, say 1024 dimensions. A special "classification token" (an extra learned vector) is prepended to the sequence. So the input to the transformer is a sequence of 197 vectors.

Then: standard transformer layers. Multi-head self-attention followed by feed-forward networks, repeated for dozens of layers. The attention mechanism lets each patch look at every other patch, so by the time we reach the top of the stack, every patch's vector has been shaped by information from every other patch.

# What a ViT forward pass looks like, roughly
def vit(image):
    patches = chop_into_16x16(image)           # 196 patches of 256 pixels
    tokens  = linear_project(patches)          # 196 vectors of dim 1024
    tokens  = prepend(CLS_TOKEN, tokens)       # now 197 vectors
    tokens  = add_positional_encoding(tokens)
    for layer in transformer_layers:
        tokens = layer(tokens)                 # each token attends to all others
    global_feature = tokens[0]                 # the CLS token
    dense_features = tokens[1:]                # the 196 patch tokens
    return global_feature, dense_features

At the end of the stack, the vector at position 0 (the CLS token) is your global feature. The vectors at positions 1 through 196 are your dense features, each one representing what's at that patch's location in the image. The attention mechanism has already mixed information between patches, but each final token is still positionally anchored to its input patch.

This is nice, clean, general. A single architecture that naturally produces both global and dense features. You can scale it by making the tokens wider, the layers deeper, or the patches smaller. The scaling behavior is smooth and well-understood from the NLP side.

And yet.

4. The paradox: scaling made it worse

If you take a ViT and train it on enough data for long enough, something strange happens. The global features keep getting better. Classification accuracy on ImageNet climbs monotonically, breaking past 85%, past 88%, past 90%. The model is clearly learning more and more about images.

The dense features, meanwhile, peak early and then get worse.

This isn't a subtle effect. Researchers have measured it carefully: on standard dense-feature benchmarks like ADE20k semantic segmentation and Pascal VOC, the quality of a large ViT's dense features rises during the first few hundred thousand training iterations, peaks, and then degrades steadily until the features become nearly unusable for pixel-level tasks. The phenomenon is called dense feature collapse, and it only really shows up at scale. Smaller models (around ViT-Large, roughly 300M params and below) don't suffer from it much, because they don't have the capacity to aggressively specialize toward any one objective and just produce mediocre-but-balanced representations. Past that scale, the model has the capacity to aggressively optimize the loudest signal in the loss function, which, for self-supervised training, is the global clustering objective. The fine-grained per-patch distinctions that dense tasks depend on get sacrificed to make the global summary cleaner.

<figure> <svg viewBox="0 0 680 320" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <line x1="60" y1="260" x2="620" y2="260" stroke="#8F8578"/> <line x1="60" y1="260" x2="60" y2="40" stroke="#8F8578"/> <g font-family="JetBrains Mono" font-size="10" fill="#6B6258" text-anchor="middle"> <text x="60" y="278">0</text><text x="172" y="278">200k</text><text x="284" y="278">400k</text><text x="396" y="278">600k</text><text x="508" y="278">800k</text><text x="620" y="278">1M</text><text x="340" y="300" font-size="11">training iterations</text> </g> <g stroke="#C9BEAA" stroke-width=".5" stroke-dasharray="2,3"><line x1="60" y1="100" x2="620" y2="100"/><line x1="60" y1="160" x2="620" y2="160"/><line x1="60" y1="220" x2="620" y2="220"/></g> <path d="M60,240 Q120,160 172,105 T284,75 T396,62 T508,58 T620,56" fill="none" stroke="#2563EB" stroke-width="2.5"/> <path d="M60,240 Q100,150 160,120 Q220,105 260,125 Q320,180 390,220 Q460,240 540,250 Q580,252 620,252" fill="none" stroke="#B91C1C" stroke-width="2.5"/> <g font-family="serif" font-size="12" font-style="italic"> <text x="425" y="28" fill="#2563EB">global accuracy (ImageNet)</text> <line x1="450" y1="34" x2="478" y2="54" stroke="#2563EB" stroke-width=".8"/> <text x="332" y="175" fill="#B91C1C">dense precision (ADE20k mIoU)</text> <line x1="352" y1="182" x2="372" y2="207" stroke="#B91C1C" stroke-width=".8"/> </g> <g font-family="serif" font-size="12" fill="#6B6258"> <text x="150" y="85" text-anchor="middle">rises and keeps rising</text> <text x="500" y="200" text-anchor="middle">rises, peaks, collapses</text> </g> </svg> <figcaption><strong>Fig 2.</strong> The two curves that define the problem. Global classification accuracy rises steadily through a million training iterations. Dense feature quality peaks around iteration 200,000 and then degrades for the rest of training. This is the failure mode every modern paper is ultimately trying to repair.</figcaption> </figure>

Why does this happen? The intuition, once you see it, is hard to unsee.

The self-supervised training objective is fundamentally a clustering objective. It asks: "given two views of the same image, make their global summaries land in the same place in embedding space." In other words, it rewards the model for saying that two patches (a patch of fur from view A and a patch of fur from view B) describe the same thing.

But "describe the same thing" is a slippery requirement. The easiest way to satisfy it is to make all patches of anything-dog-related look similar in feature space. Fur patches, eye patches, ear patches, paw patches: push them all toward a shared "dog" region of the feature space. That's great for classification, because now a simple average of the patch features is a clean signal for the dog concept. But it's disastrous for segmentation, because now the model has no way to tell a fur pixel from an eye pixel. They're in the same neighborhood of feature space.

The longer you train, the more the global objective wins. Patches get pulled toward semantic centers. Local distinctions dissolve. The model becomes a better classifier and a worse mapmaker, and there's no obvious way to stop the dissolution without also stopping the improvement.

The same loss that teaches the model to see teaches it to stop looking.

This is the paradox that every paper we're going to discuss is ultimately responding to. The story of modern vision foundation models is the story of a series of increasingly clever ways to preserve local detail while still letting global understanding improve. Three in particular (DINOv3, C-RADIOv4, and FeatSharp) form a coherent stack that now dominates the state of the art.

We'll take them one at a time.

Act 1 — The Lens

DINOv3, self-supervised learning at scale, and the trick that saved dense features.

5. Meet DINOv3

DINOv3 is Meta's self-supervised vision transformer, released in August 2025. It is, at the time of writing, the strongest general-purpose vision backbone in the world. It comes in sizes from 21 million parameters (deployable on a laptop) up to 6.7 billion parameters (the research flagship). It was trained on 1.7 billion images without a single human-provided label.

Three things make DINOv3 interesting. First, it took self-supervised learning to a scale nobody thought it could reach: 12× more data and roughly 6× more parameters than its predecessor DINOv2. Second, it produced dense features so good that a frozen DINOv3 backbone outperforms specialized networks on tasks like satellite canopy-height estimation and medical pathology, without any fine-tuning. Third, it solved dense feature collapse (the paradox we just described) with a clean technique called Gram anchoring.

Gram anchoring is the real contribution. Everything else (the scale, the data curation, the architecture tweaks) is table stakes for a 2025 foundation model. Gram anchoring is the idea that will still be influential in five years. Before we can appreciate it, though, we need to understand two pieces of machinery: self-supervised learning, and the Gram matrix itself.

6. Self-supervision, from first principles

If you're coming from supervised learning, the obvious question is: how do you train a model without labels?

The trick is to invent your own labels by exploiting the structure of the data. In images, a lot of structure is available for free. Two crops of the same image should "mean" the same thing. A masked-out patch of an image should be predictable from the patches around it. A rotated image is still the same image. These are all free sources of supervision. They require no human annotation, just a clever way of asking the model a question whose answer can be checked automatically.

The DINO family (DINO, DINOv2, and now DINOv3) uses a specific technique called self-distillation. Here's how it works.

You have two copies of the network: a student (which you actively train) and a teacher (which is updated as a slow-moving average of the student's weights, an exponential moving average, or EMA). Both networks are the same architecture. The teacher is always a slightly stale version of the student.

On each training step, you take a single image and produce two different "views" of it: two random crops, different augmentations, different scales. The student sees one view. The teacher sees the other. Both produce a feature vector. You compute a loss that says: "student, your feature should look like the teacher's feature." Then you backprop through the student, and the teacher updates by sliding toward the student's new weights.

<figure> <svg viewBox="0 0 680 240" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <rect x="20" y="90" width="80" height="80" fill="#EDE5D4" stroke="#8F8578"/> <rect x="20" y="90" width="80" height="30" fill="#93C5FD" opacity=".7"/> <rect x="20" y="130" width="80" height="20" fill="#D1AE7A" opacity=".8"/> <rect x="20" y="150" width="80" height="20" fill="#059669" opacity=".6"/> <text x="60" y="190" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#6B6258">source</text> <rect x="140" y="50" width="65" height="65" fill="#EDE5D4" stroke="#8F8578"/> <rect x="140" y="50" width="65" height="25" fill="#93C5FD" opacity=".7"/> <rect x="140" y="75" width="65" height="20" fill="#D1AE7A" opacity=".8"/> <rect x="140" y="95" width="65" height="20" fill="#059669" opacity=".6"/> <text x="172" y="132" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">view A</text> <rect x="140" y="150" width="65" height="65" fill="#EDE5D4" stroke="#8F8578"/> <rect x="140" y="150" width="65" height="20" fill="#93C5FD" opacity=".7"/> <rect x="140" y="170" width="65" height="25" fill="#D1AE7A" opacity=".8"/> <rect x="140" y="195" width="65" height="20" fill="#059669" opacity=".6"/> <text x="172" y="232" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">view B</text> <line x1="102" y1="115" x2="138" y2="82" stroke="#8F8578"/> <line x1="102" y1="145" x2="138" y2="182" stroke="#8F8578"/> <rect x="240" y="45" width="100" height="75" fill="none" stroke="#2563EB" stroke-width="1.5"/> <text x="290" y="75" text-anchor="middle" font-family="serif" font-size="14" fill="#2563EB" font-weight="600">student</text> <text x="290" y="95" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">being trained</text> <rect x="240" y="145" width="100" height="75" fill="none" stroke="#B8651A" stroke-width="1.5"/> <text x="290" y="175" text-anchor="middle" font-family="serif" font-size="14" fill="#B8651A" font-weight="600">teacher</text> <text x="290" y="195" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#6B6258">EMA of student</text> <line x1="205" y1="82" x2="238" y2="82" stroke="#6B6258" marker-end="url(#a2)"/> <line x1="205" y1="182" x2="238" y2="182" stroke="#6B6258" marker-end="url(#a2)"/> <defs><marker id="a2" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="5" markerHeight="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="#6B6258"/></marker></defs> <g font-family="JetBrains Mono" font-size="10"> <rect x="370" y="70" width="120" height="22" fill="none" stroke="#2563EB"/> <text x="430" y="86" text-anchor="middle" fill="#2563EB">[0.7, -0.1, 0.4, ...]</text> <rect x="370" y="170" width="120" height="22" fill="none" stroke="#B8651A"/> <text x="430" y="186" text-anchor="middle" fill="#B8651A">[0.8, -0.2, 0.3, ...]</text> </g> <line x1="342" y1="82" x2="368" y2="82" stroke="#6B6258" marker-end="url(#a2)"/> <line x1="342" y1="182" x2="368" y2="182" stroke="#6B6258" marker-end="url(#a2)"/> <line x1="430" y1="92" x2="430" y2="125" stroke="#6B6258" stroke-dasharray="3,3"/> <line x1="430" y1="168" x2="430" y2="135" stroke="#6B6258" stroke-dasharray="3,3"/> <rect x="520" y="115" width="120" height="38" fill="#1A1A1A" stroke="#B8651A"/> <text x="580" y="132" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#B8651A" letter-spacing="1">LOSS</text> <text x="580" y="147" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#F4EEE3">‖ s − t ‖²</text> <line x1="495" y1="130" x2="518" y2="130" stroke="#6B6258" marker-end="url(#a2)"/> <line x1="430" y1="130" x2="495" y2="130" stroke="#6B6258" stroke-dasharray="3,3"/> </svg> <figcaption><strong>Fig 3.</strong> Self-distillation: the student and teacher see different views of the same image. The student is trained to match the teacher's output. The teacher is a slow exponential moving average of the student. Neither network ever sees a label, but the pair pressures each other into producing stable, consistent features across augmentations.</figcaption> </figure>

This procedure has no obvious reason to work. There's no ground truth. The teacher is just another copy of the student, slightly behind in time. You could imagine the whole thing collapsing into a trivial solution: the student outputs a constant vector, the teacher outputs the same constant vector, everyone's happy and nothing has been learned.

Preventing that collapse is a big part of what the DINO papers are about. The tricks include: careful output-space normalization (via Sinkhorn-Knopp soft clustering), an asymmetric crop scheme (the teacher only sees two large "global" crops of the image; the student sees those global crops plus several smaller "local" crops, and has to match the teacher's global view from every one of its own views, including the tiny zoomed-in ones), and a lot of temperature-scaling gymnastics. What matters for us is that it does work, and when it works, it works spectacularly. The features DINO learns are so good that frozen DINO backbones (no task-specific fine-tuning) routinely beat specialized supervised networks on individual tasks.

On top of the self-distillation loss (which enforces global consistency), DINOv3 adds a second loss called iBOT, which is a masked patch prediction objective. Some percentage of the patches in the student's input are masked out (replaced with a special learnable "mask" token) and the student is trained to produce features for those masked patches that match what the teacher would have produced if it had seen them. This is the vision analog of BERT's masked-language-modeling objective, and it specifically targets local quality: the student has to learn something about each patch that lets it stand in for the real content.

Two losses. One global, one local. Trained together for a million iterations on 1.7 billion images. You might reasonably ask: if iBOT is explicitly a local-quality objective, shouldn't it prevent dense collapse? The short answer is that at multi-billion-parameter scale, the global DINO pressure (which is what the whole network is fundamentally oriented around) simply outweighs iBOT's local pressure. iBOT buys some protection early in training. It's not enough on its own. For the first couple hundred thousand iterations, everything is fine. Somewhere around iteration 200,000, something breaks.

An aside on data. The 1.7-billion-image dataset, called LVD-1689M, was not just scraped off the internet. It was filtered down from a starting pool of about 17 billion web images using an algorithm called hierarchical k-means clustering.

The intuition: the internet is skewed. If you train a model on whatever images happen to be popular on social media, it sees millions of selfies and vacation photos but almost no industrial equipment, medical imagery, or aerial photography. The resulting model learns the skew. Hierarchical k-means groups images into categories, then sub-categories, recursively. Training draws samples uniformly from the leaf clusters rather than from the raw distribution. Rare concepts get proportional attention. Common concepts don't drown them out.

Think of it as stratified sampling on steroids. You get to see every corner of visual reality, not just the bright-lit center.

7. A Gram matrix, from scratch

To understand DINOv3's fix for dense feature collapse, we need to introduce one more piece of math: the Gram matrix. It sounds technical but the idea is almost absurdly simple.

Start with a set of vectors. For our purposes, imagine the dense features coming out of a ViT: one vector per image patch. Say we have 196 patches, so 196 vectors, each maybe 1024 dimensions wide.

The Gram matrix of those vectors is a square table where entry (i, j) is the dot product between vector i and vector j. If you have 196 vectors, the Gram matrix is 196 by 196. It doesn't care about the raw values of the vectors. It only records how they relate to each other.

Let's do a concrete tiny example. Suppose we have only 4 patches, labeled A, B, C, and D. A and B are both patches of sky. C is a patch of fur. D is a patch of grass. The model produces a 3-dimensional feature vector for each patch:

<figure> <svg viewBox="0 0 680 310" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <g transform="translate(30,40)"> <rect x="0" y="0" width="70" height="70" fill="#93C5FD" stroke="#F4EEE3" stroke-width="2"/> <text x="35" y="42" text-anchor="middle" font-family="serif" font-size="28" font-weight="700" fill="#FFFFFF">A</text> <rect x="72" y="0" width="70" height="70" fill="#93C5FD" stroke="#F4EEE3" stroke-width="2"/> <text x="107" y="42" text-anchor="middle" font-family="serif" font-size="28" font-weight="700" fill="#FFFFFF">B</text> <rect x="0" y="72" width="70" height="70" fill="#92400E" stroke="#F4EEE3" stroke-width="2"/> <text x="35" y="115" text-anchor="middle" font-family="serif" font-size="28" font-weight="700" fill="#FFFFFF">C</text> <rect x="72" y="72" width="70" height="70" fill="#059669" stroke="#F4EEE3" stroke-width="2"/> <text x="107" y="115" text-anchor="middle" font-family="serif" font-size="28" font-weight="700" fill="#FFFFFF">D</text> <rect x="0" y="0" width="142" height="142" fill="none" stroke="#1A1A1A" stroke-width="1.5"/> <text x="71" y="170" text-anchor="middle" font-family="serif" font-size="12" font-style="italic" fill="#6B6258">A, B = sky</text> <text x="71" y="186" text-anchor="middle" font-family="serif" font-size="12" font-style="italic" fill="#6B6258">C = fur · D = grass</text> </g> <g transform="translate(230,40)" font-family="JetBrains Mono" font-size="12"> <text x="0" y="0" font-size="10" fill="#6B6258" letter-spacing="1">FEATURES</text> <text x="0" y="22" fill="#2563EB" font-weight="600">A =</text><text x="30" y="22" fill="#1A1A1A">[0.9, 0.1, 0.2]</text> <text x="0" y="42" fill="#2563EB" font-weight="600">B =</text><text x="30" y="42" fill="#1A1A1A">[0.95, 0.05, 0.15]</text> <text x="0" y="62" fill="#B8651A" font-weight="600">C =</text><text x="30" y="62" fill="#1A1A1A">[0.1, 0.85, 0.2]</text> <text x="0" y="82" fill="#059669" font-weight="600">D =</text><text x="30" y="82" fill="#1A1A1A">[0.3, 0.3, 0.8]</text> <text x="0" y="115" font-size="10" fill="#6B6258" letter-spacing="1">DOT PRODUCTS</text> <text x="0" y="137" fill="#1A1A1A">A·B = 0.89</text> <text x="0" y="155" fill="#1A1A1A">A·C = 0.21</text> <text x="0" y="173" fill="#1A1A1A">A·D = 0.46</text> <text x="0" y="191" fill="#1A1A1A">B·C = 0.17</text> <text x="0" y="209" fill="#1A1A1A">B·D = 0.42</text> <text x="0" y="227" fill="#1A1A1A">C·D = 0.44</text> </g> <g transform="translate(440,40)"> <text x="90" y="0" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#6B6258" letter-spacing="1">GRAM MATRIX</text> <text x="55" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="13" font-weight="600" fill="#1A1A1A">A</text> <text x="90" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="13" font-weight="600" fill="#1A1A1A">B</text> <text x="125" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="13" font-weight="600" fill="#1A1A1A">C</text> <text x="160" y="22" text-anchor="middle" font-family="JetBrains Mono" font-size="13" font-weight="600" fill="#1A1A1A">D</text> <g font-family="JetBrains Mono" font-size="11"> <text x="30" y="48" font-weight="600" fill="#1A1A1A">A</text> <rect x="40" y="32" width="30" height="22" fill="#2563EB"/><text x="55" y="48" text-anchor="middle" fill="#FFFFFF">1.0</text> <rect x="75" y="32" width="30" height="22" fill="#2563EB" opacity=".85"/><text x="90" y="48" text-anchor="middle" fill="#FFFFFF">0.89</text> <rect x="110" y="32" width="30" height="22" fill="#2563EB" opacity=".18"/><text x="125" y="48" text-anchor="middle" fill="#1A1A1A">0.21</text> <rect x="145" y="32" width="30" height="22" fill="#2563EB" opacity=".42"/><text x="160" y="48" text-anchor="middle" fill="#1A1A1A">0.46</text> <text x="30" y="72" font-weight="600" fill="#1A1A1A">B</text> <rect x="40" y="56" width="30" height="22" fill="#2563EB" opacity=".85"/><text x="55" y="72" text-anchor="middle" fill="#FFFFFF">0.89</text> <rect x="75" y="56" width="30" height="22" fill="#2563EB"/><text x="90" y="72" text-anchor="middle" fill="#FFFFFF">1.0</text> <rect x="110" y="56" width="30" height="22" fill="#2563EB" opacity=".15"/><text x="125" y="72" text-anchor="middle" fill="#1A1A1A">0.17</text> <rect x="145" y="56" width="30" height="22" fill="#2563EB" opacity=".38"/><text x="160" y="72" text-anchor="middle" fill="#1A1A1A">0.42</text> <text x="30" y="96" font-weight="600" fill="#1A1A1A">C</text> <rect x="40" y="80" width="30" height="22" fill="#2563EB" opacity=".18"/><text x="55" y="96" text-anchor="middle" fill="#1A1A1A">0.21</text> <rect x="75" y="80" width="30" height="22" fill="#2563EB" opacity=".15"/><text x="90" y="96" text-anchor="middle" fill="#1A1A1A">0.17</text> <rect x="110" y="80" width="30" height="22" fill="#2563EB"/><text x="125" y="96" text-anchor="middle" fill="#FFFFFF">1.0</text> <rect x="145" y="80" width="30" height="22" fill="#2563EB" opacity=".4"/><text x="160" y="96" text-anchor="middle" fill="#1A1A1A">0.44</text> <text x="30" y="120" font-weight="600" fill="#1A1A1A">D</text> <rect x="40" y="104" width="30" height="22" fill="#2563EB" opacity=".42"/><text x="55" y="120" text-anchor="middle" fill="#1A1A1A">0.46</text> <rect x="75" y="104" width="30" height="22" fill="#2563EB" opacity=".38"/><text x="90" y="120" text-anchor="middle" fill="#1A1A1A">0.42</text> <rect x="110" y="104" width="30" height="22" fill="#2563EB" opacity=".4"/><text x="125" y="120" text-anchor="middle" fill="#1A1A1A">0.44</text> <rect x="145" y="104" width="30" height="22" fill="#2563EB"/><text x="160" y="120" text-anchor="middle" fill="#FFFFFF">1.0</text> </g> <text x="0" y="162" font-family="serif" font-size="11" font-style="italic" fill="#6B6258"> <tspan x="0" dy="0">A·B is high (both sky).</tspan> <tspan x="0" dy="16">A·C is low (sky vs fur).</tspan> <tspan x="0" dy="16">Diagonal is always 1 (a vector's</tspan> <tspan x="0" dy="14">similarity to itself, assuming it's</tspan> <tspan x="0" dy="14">a unit vector).</tspan> </text> </g> </svg> <figcaption><strong>Fig 4.</strong> The Gram matrix of four patch features. Each cell (i, j) is the dot product of features i and j. It's a square, symmetric table that records relationships rather than values. Two patches that mean similar things will have a high dot product; two patches that mean different things will have a low one. The matrix is the model's picture of how every part of the image relates to every other part.</figcaption> </figure>

A few things worth noting about this object. The Gram matrix is square: if you have n vectors it's n-by-n. It's symmetric (dot products don't care about order), so entry (i, j) equals entry (j, i). The diagonal entries are each vector's similarity to itself, which, if the vectors are normalized to unit length, is always 1.

Here's the critical property. The Gram matrix doesn't care about the magnitudes or orientations of the underlying features. If you rotated all your feature vectors by the same angle in feature space, the Gram matrix would be unchanged. If you scaled them all by the same factor, it would change in a uniform, predictable way. What the Gram matrix captures is the relational structure: the geometry of the feature cloud, not its location.

For a vision model, the Gram matrix of the dense features tells you which patches the model thinks are similar and which it thinks are different. If the matrix is "clean" (high values for patches of the same content, low values for patches of different content), the model has a good local map. If the matrix is "noisy" (random-looking similarity everywhere), the model has lost its map.

Dense feature collapse, when you plot the Gram matrix of a late-training ViT, looks exactly like static on an old TV. Every patch looks moderately similar to every other patch. The fine distinctions are gone.

8. Gram anchoring: the blueprint trick

Now we have everything we need to understand Gram anchoring. The DINOv3 team observed the collapse, and they had a specific observation: the Gram matrix used to look fine, earlier in training. At iteration 100-200k or so, the dense features were sharp, the Gram matrix was clean, the model was doing the right thing. The collapse happened later, as the global objective started to dominate.

What if you could hold on to the early, healthy Gram matrix? What if, during late-stage training, you added a new loss term that said "student, your current Gram matrix should stay close to the Gram matrix of your own past self"?

This is Gram anchoring. The details:

Take a snapshot of the model at an earlier training checkpoint, chosen while the dense features are still sharp, before collapse has set in. Call this the Gram teacher. Hold it fixed for now. (It will be refreshed periodically, every 10k iterations or so, by copying in the current EMA teacher's weights. The early-snapshot intuition is what matters; the refresh schedule is a practical detail.)
During later training, on each batch, run the image through both the current student and the current Gram teacher. Compute both Gram matrices: $G_S$ and $G_T$.
Add a new term to the loss: the Frobenius norm of the difference between the two Gram matrices. Written out, it's $|G_S - G_T|^2_F$, which is just "sum up the squared differences between corresponding cells."
Train as normal. The student keeps its DINO and iBOT losses, but now also has to keep its Gram matrix from drifting away from the frozen snapshot.

It's worth pausing on why this works, because the logic is subtle. At first glance, anchoring a newer model to an older version of itself sounds like it would prevent learning. You're telling the model "be like your dumber self." Shouldn't that stop the model from improving?

It doesn't, and the reason is that Gram anchoring doesn't constrain the features themselves. It constrains only the relationships between features: the Gram matrix. The underlying feature vectors are free to evolve wildly. They can rotate, they can stretch, they can reshape themselves to capture new concepts. As long as the pairwise dot products stay consistent with the snapshot, the anchoring is satisfied.

Keep the blueprint. Upgrade the materials.

A good analogy: imagine a building. The Gram matrix is the blueprint: which walls are load-bearing, which rooms connect to which, how far the bathroom is from the kitchen. The individual features are the materials: the actual drywall, the carpet, the lighting fixtures. You can swap every piece of drywall for marble, change all the lighting, repaint every wall, and the blueprint is unchanged. The structure remains.

That's why the technique works. The early-training dense features had good structure. The Gram matrix was clean, distinct patches had low similarity, similar patches had high similarity. By preserving that structure during late training, you let the features themselves keep getting richer and more semantically loaded, while refusing to let them dissolve into an undifferentiated soup.

The result, measured on every dense benchmark that exists, is dramatic. Without Gram anchoring, ADE20k segmentation quality peaks early and then falls by tens of mIoU points over the remainder of training: the classic collapse. With Gram anchoring, it keeps climbing and settles at state-of-the-art levels. This is with the exact same backbone, the same data, the same number of iterations. The only difference is one extra loss term that keeps the spatial blueprint intact.

Gram anchoring is also the reason DINOv3 can be used frozen, with no fine-tuning, on a dizzying array of downstream tasks. Video tracking without ever seeing a video. Satellite canopy-height estimation that beats purpose-built Earth-observation networks. Medical pathology on atypical mitotic figures. The features are so spatially clean that they just transfer. You drop them into a new task, add a small linear head, and the result is often state of the art.

Checkpoint: what you know now.

Vision models produce two kinds of features: a global summary and a per-patch dense map. Both matter. Historically, scaling a self-supervised ViT improved one at the expense of the other.

The failure mode is called dense feature collapse: the pressure to group patches by semantic content ends up dissolving the per-patch distinctions that dense tasks need.

DINOv3 fixes it with Gram anchoring: during late-stage training, use an earlier, still-healthy checkpoint as a Gram teacher, and train the model to keep its current Gram matrix close to that teacher's. This preserves the structure of the patch features without freezing the features themselves.

DINOv3 is now the single strongest vision backbone in the world. But it has one big limitation: it was trained without text, so it has no idea what any object is called. That's the next problem.

Act 2 — The Prism

Why three specialist backbones can't coexist at inference, and how C-RADIOv4 fuses them into one.

9. Three teachers, one student

DINOv3 is the best dense-feature model ever made. It is also, in one specific way, useless: you can't talk to it. You can't ask "is there a dog in this picture?" and get a yes-or-no answer, because DINOv3 has never seen the word "dog." It has no alignment with language at all. It was trained on pure pixels, without captions or labels.

For anything involving natural language (zero-shot classification, text-to-image retrieval, visual question answering, multimodal assistants), you need a different kind of model. The standard is SigLIP2, Google's 2025 image-text model. Conceptually it sits in the lineage of OpenAI's CLIP (dual image and text encoders aligned in a shared space), but it's trained with a different loss: a pairwise sigmoid objective that treats each (image, caption) pair as an independent yes-or-no match. The result is the same kind of shared space. Image features and caption features live in the same embedding space, and you can do a dot product between them to measure how well a caption describes an image.

SigLIP2 is excellent at language alignment. It's also, in a different specific way, awkward for dense work. The standard high-resolution variant C-RADIOv4 distills from, SigLIP2-g-384, uses patch-16 tokenization at 384 pixels, which gives a 24×24 feature grid. Fine for classification, coarse for anything pixel-precise. SigLIP 2 does include a newer "NaFlex" variant that supports native aspect ratios and variable resolutions, but the fixed-resolution teacher is what's in play here. And SigLIP2's features don't preserve the sharp spatial geometry DINOv3 gives you. They're semantically rich but spatially mushy.

So: DINOv3 has dense features, no text. SigLIP2 has text, no dense features. Surely we also need high-resolution masking (you know, segment-anything), and that's a whole other specialist: SAM3, Meta's November 2025 segmentation model. (A point release, SAM 3.1, landed in March 2026 with tracking speed improvements but no architectural change to the backbone, which is what matters here.) SAM3 will happily take a text prompt and return pixel-precise masks for every instance in an image. But it's a segmenter, not a general-purpose encoder. It doesn't give you good global features for classification.

SAM3 and DINOv3 look redundant at first glance. They both produce dense features, so if DINOv3 is already excellent at dense work, why do we need SAM3 as a separate teacher? Because the two models come from completely different supervision regimes, and each regime teaches something the other one structurally can't. DINOv3 is self-supervised: it never sees a label, and certainly never sees a mask. Its loss is a clustering objective that pushes patches of similar content toward the same region of feature space. That's great for semantic grouping, but it's blind to where one instance of a class ends and another begins, because the loss has no ground truth for instance identity. Two cars parked next to each other sit in the same region of DINOv3's feature space, by design.

SAM3 was trained on pixel-exact mask supervision at scale, roughly 4 million unique concepts from Meta's SA-Co data engine (with instance masks, hard negatives, and both image and video annotations). Every training example carries explicit information about where a specific instance stops, which is information self-supervision cannot produce no matter how much data you scale to. SAM3's architecture is built around this: its DETR-based detector has a dedicated presence head that separates "is this concept in the image" from "where exactly is each copy of it." The backbone (Meta's Perception Encoder) is pre-trained separately and then coupled to the detector, with a later tracker stage that runs on top of a frozen backbone.

<figure> <svg viewBox="0 0 680 280" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <g transform="translate(205, 28)"> <circle cx="0" cy="0" r="5" fill="#0E7490"/> <text x="12" y="4" font-family="serif" font-size="13" fill="#1A1A1A">patches from car 1</text> <circle cx="160" cy="0" r="5" fill="#B8651A"/> <text x="172" y="4" font-family="serif" font-size="13" fill="#1A1A1A">patches from car 2</text> </g> <g transform="translate(30, 60)"> <rect x="0" y="0" width="290" height="200" fill="#F9F5EB" stroke="#C9BEAA" stroke-width="1" rx="4"/> <ellipse cx="145" cy="95" rx="92" ry="48" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5" opacity="0.9"/> <circle cx="100" cy="78" r="3.5" fill="#0E7490"/> <circle cx="118" cy="85" r="3.5" fill="#B8651A"/> <circle cx="130" cy="72" r="3.5" fill="#0E7490"/> <circle cx="135" cy="92" r="3.5" fill="#B8651A"/> <circle cx="148" cy="80" r="3.5" fill="#0E7490"/> <circle cx="162" cy="95" r="3.5" fill="#B8651A"/> <circle cx="172" cy="84" r="3.5" fill="#0E7490"/> <circle cx="184" cy="102" r="3.5" fill="#B8651A"/> <circle cx="108" cy="102" r="3.5" fill="#B8651A"/> <circle cx="125" cy="110" r="3.5" fill="#0E7490"/> <circle cx="155" cy="108" r="3.5" fill="#0E7490"/> <circle cx="178" cy="115" r="3.5" fill="#B8651A"/> <circle cx="122" cy="100" r="3.5" fill="#0E7490"/> <circle cx="168" cy="112" r="3.5" fill="#B8651A"/> <circle cx="145" cy="118" r="3.5" fill="#B8651A"/> <circle cx="140" cy="85" r="3.5" fill="#0E7490"/> <text x="145" y="165" text-anchor="middle" font-family="serif" font-size="16" font-weight="600" fill="#2563EB">DINOv3</text> <text x="145" y="184" text-anchor="middle" font-family="serif" font-size="12" fill="#6B6258" font-style="italic">one cluster. both cars blur together.</text> </g> <g transform="translate(360, 60)"> <rect x="0" y="0" width="290" height="200" fill="#F9F5EB" stroke="#C9BEAA" stroke-width="1" rx="4"/> <ellipse cx="85" cy="85" rx="52" ry="30" fill="#EDE0FC" stroke="#9333EA" stroke-width="1.5" opacity="0.9"/> <ellipse cx="215" cy="105" rx="52" ry="30" fill="#EDE0FC" stroke="#9333EA" stroke-width="1.5" opacity="0.9"/> <text x="85" y="48" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#9333EA">instance 1</text> <text x="215" y="145" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#9333EA">instance 2</text> <circle cx="65" cy="80" r="3.5" fill="#0E7490"/> <circle cx="78" cy="88" r="3.5" fill="#0E7490"/> <circle cx="85" cy="75" r="3.5" fill="#0E7490"/> <circle cx="95" cy="90" r="3.5" fill="#0E7490"/> <circle cx="102" cy="82" r="3.5" fill="#0E7490"/> <circle cx="72" cy="95" r="3.5" fill="#0E7490"/> <circle cx="92" cy="96" r="3.5" fill="#0E7490"/> <circle cx="108" cy="92" r="3.5" fill="#0E7490"/> <circle cx="195" cy="100" r="3.5" fill="#B8651A"/> <circle cx="208" cy="108" r="3.5" fill="#B8651A"/> <circle cx="215" cy="95" r="3.5" fill="#B8651A"/> <circle cx="225" cy="110" r="3.5" fill="#B8651A"/> <circle cx="232" cy="102" r="3.5" fill="#B8651A"/> <circle cx="200" cy="115" r="3.5" fill="#B8651A"/> <circle cx="222" cy="118" r="3.5" fill="#B8651A"/> <circle cx="240" cy="112" r="3.5" fill="#B8651A"/> <text x="145" y="165" text-anchor="middle" font-family="serif" font-size="16" font-weight="600" fill="#9333EA">SAM3</text> <text x="145" y="184" text-anchor="middle" font-family="serif" font-size="12" fill="#6B6258" font-style="italic">two clusters. instance identity preserved.</text> </g> </svg> <figcaption><strong>Fig 5.</strong> What each teacher's features encode for the same two-car scene. DINOv3's self-supervised clustering loss puts all "car" patches near each other, so the two cars' patches interleave in one region. SAM3's mask supervision pushes patches from different instance masks to different regions, so the two cars form separate clusters. DINOv3 carries "what is it." SAM3 carries "which one is it." C-RADIOv4 inherits both.</figcaption> </figure>

This matters for C-RADIOv4 in two ways. First, the student's dense features inherit information that's structurally absent from the DINOv3 signal: instance-level discriminability, and features shaped by a pixel-exact boundary loss. Second, because the student is distilled specifically to match the Perception Encoder's output format, C-RADIOv4 can function as a drop-in replacement for SAM3's backbone. Swap it in, keep SAM3's detector and tracker unchanged, and you get the whole promptable-segmentation and video-tracking stack at a fraction of the encoder cost. That kind of compatibility is at least part of the appeal of SAM3 as a teacher.

Three backbones. Three complementary strengths. And running all three in production is expensive enough that most teams won't do it. A ViT-H at 1024 pixels isn't cheap on its own; running two or three of them in parallel multiplies your inference cost for every request, and multi-encoder VLM stacks are the kind of thing you build reluctantly rather than by design. Getting the same capabilities out of one backbone is worth real money.

<figure> <svg viewBox="0 0 680 270" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <g> <circle cx="120" cy="120" r="80" fill="#DBEAFE" stroke="#2563EB" stroke-width="2.5"/> <text x="120" y="105" text-anchor="middle" font-family="serif" font-size="22" font-weight="700" fill="#1A1A1A">DINOv3</text> <text x="120" y="125" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#2563EB">self-supervised</text> <text x="120" y="220" text-anchor="middle" font-family="serif" font-size="12" fill="#1A1A1A">+ dense patch geometry</text> <text x="120" y="240" text-anchor="middle" font-family="serif" font-size="12" fill="#6B6258">− no text grounding</text> </g> <g> <circle cx="340" cy="120" r="80" fill="#D1FAE5" stroke="#059669" stroke-width="2.5"/> <text x="340" y="105" text-anchor="middle" font-family="serif" font-size="22" font-weight="700" fill="#1A1A1A">SigLIP2</text> <text x="340" y="125" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#059669">text-aligned</text> <text x="340" y="220" text-anchor="middle" font-family="serif" font-size="12" fill="#1A1A1A">+ zero-shot labels</text> <text x="340" y="240" text-anchor="middle" font-family="serif" font-size="12" fill="#6B6258">− locked at 384px</text> </g> <g> <circle cx="560" cy="120" r="80" fill="#EDE0FC" stroke="#9333EA" stroke-width="2.5"/> <text x="560" y="105" text-anchor="middle" font-family="serif" font-size="22" font-weight="700" fill="#1A1A1A">SAM3</text> <text x="560" y="125" text-anchor="middle" font-family="serif" font-size="11" font-style="italic" fill="#9333EA">fully supervised</text> <text x="560" y="220" text-anchor="middle" font-family="serif" font-size="12" fill="#1A1A1A">+ instance-aware masks</text> <text x="560" y="240" text-anchor="middle" font-family="serif" font-size="12" fill="#6B6258">− narrow semantics</text> </g> <g> <text x="230" y="110" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#B91C1C">LATENCY</text> <line x1="210" y1="120" x2="250" y2="120" stroke="#B91C1C" stroke-width="1.5" stroke-dasharray="4,3"/> <text x="450" y="110" text-anchor="middle" font-family="JetBrains Mono" font-size="9" letter-spacing="1.5" fill="#B91C1C">LATENCY</text> <line x1="430" y1="120" x2="470" y2="120" stroke="#B91C1C" stroke-width="1.5" stroke-dasharray="4,3"/> </g> </svg> <figcaption><strong>Fig 6.</strong> Three specialized backbones, each excellent at something, each deficient at something else. Running them together at inference time would give us a complete vision system, and also a multi-GPU deployment burden nobody can justify. The question: can we train a single student that gets all three of their strengths?</figcaption> </figure>

This is where C-RADIOv4 enters the story.

10. What distillation is, really

Before we can talk about C-RADIOv4, one more piece of background. The technique C-RADIOv4 relies on is called knowledge distillation, and it deserves a careful explanation because the whole story turns on it.

The simplest version of distillation was introduced by Geoffrey Hinton in 2015. You have a big, slow model (the teacher) that's been trained on some task. You want a small, fast model (the student) that does the same task almost as well. Training the small model from scratch on the original labels gives you mediocre results. But if you use the teacher's outputs as soft targets for the student, the student does dramatically better than it would if it were just mimicking the ground-truth labels.

The reason is that the teacher's outputs contain more information than the labels do. A label is a single correct answer. A teacher's output is a probability distribution over all possible answers, which encodes relationships: "this looks 80% like a cat, 15% like a lynx, 5% like a fox, nothing else." The student learns the teacher's full landscape of what's similar to what, not only the right answer.

Now: what if, instead of distilling from one teacher, we distill from several? What if we have a set of specialist teachers, each expert at a different aspect of vision, and we train a student whose job is to match all of their outputs simultaneously?

This is multi-teacher distillation, and it's the core idea behind the RADIO family of models (of which C-RADIOv4 is the latest). The student doesn't try to recreate any single teacher. It learns a unified embedding (a single feature vector) from which all the teachers' output features can be predicted. Small heads on top of the student are trained to reproduce DINOv3's output, SigLIP2's output, and SAM3's output, respectively. But the backbone itself is shared.

Crucially, this is an agglomerative process: the student ends up carrying information from all three teachers, because it has to serve all three of their prediction heads. At inference time, you run the student once (one forward pass) and you get features that can be used for any of the downstream tasks. If you need DINOv3-style dense geometry, read off the dense features. If you need SigLIP2-style text alignment, pass through the SigLIP2 head. If you need SAM3-style segmentation, pass through the SAM3 head. The expensive forward pass happens once. The lightweight heads run in microseconds.

<figure> <svg viewBox="0 0 680 260" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <g font-family="serif" font-size="14" font-weight="600"> <rect x="30" y="30" width="110" height="40" fill="#DBEAFE" stroke="#2563EB" stroke-width="1.5"/><text x="85" y="55" text-anchor="middle" fill="#2563EB">DINOv3</text> <rect x="30" y="110" width="110" height="40" fill="#D1FAE5" stroke="#059669" stroke-width="1.5"/><text x="85" y="135" text-anchor="middle" fill="#059669">SigLIP2</text> <rect x="30" y="190" width="110" height="40" fill="#EDE0FC" stroke="#9333EA" stroke-width="1.5"/><text x="85" y="215" text-anchor="middle" fill="#9333EA">SAM3</text> </g> <text x="85" y="18" text-anchor="middle" font-family="JetBrains Mono" font-size="10" letter-spacing="1.5" fill="#6B6258">TEACHERS (FROZEN)</text> <g stroke="#8F8578"> <line x1="145" y1="50" x2="285" y2="120" marker-end="url(#a3)"/> <line x1="145" y1="130" x2="285" y2="130" marker-end="url(#a3)"/> <line x1="145" y1="210" x2="285" y2="140" marker-end="url(#a3)"/> </g> <defs><marker id="a3" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="5" markerHeight="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="#8F8578"/></marker></defs> <rect x="290" y="100" width="120" height="60" fill="#1A1A1A" stroke="#B8651A" stroke-width="2"/> <text x="350" y="125" text-anchor="middle" font-family="serif" font-size="15" font-weight="700" fill="#F4EEE3">student</text> <text x="350" y="145" text-anchor="middle" font-family="serif" font-size="10" font-style="italic" fill="#B8651A">C-RADIOv4</text> <line x1="412" y1="130" x2="432" y2="130" stroke="#8F8578" marker-end="url(#a3)"/> <rect x="435" y="115" width="100" height="30" fill="none" stroke="#B8651A" stroke-width="1.5"/> <text x="485" y="134" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#1A1A1A">unified feature</text> <g font-family="JetBrains Mono" font-size="10"> <line x1="540" y1="125" x2="565" y2="55" stroke="#8F8578" marker-end="url(#a3)"/> <rect x="565" y="40" width="100" height="30" fill="#DBEAFE" stroke="#2563EB"/><text x="615" y="59" text-anchor="middle" fill="#2563EB">DINOv3 head</text> <line x1="540" y1="130" x2="565" y2="130" stroke="#8F8578" marker-end="url(#a3)"/> <rect x="565" y="115" width="100" height="30" fill="#D1FAE5" stroke="#059669"/><text x="615" y="134" text-anchor="middle" fill="#059669">SigLIP head</text> <line x1="540" y1="135" x2="565" y2="205" stroke="#8F8578" marker-end="url(#a3)"/> <rect x="565" y="190" width="100" height="30" fill="#EDE0FC" stroke="#9333EA"/><text x="615" y="209" text-anchor="middle" fill="#9333EA">SAM3 head</text> </g> </svg> <figcaption><strong>Fig 7.</strong> Multi-teacher distillation in action. Three frozen teachers produce outputs for an image. A shared student backbone learns a single unified feature, and three small heads, one per teacher, are trained to reproduce each teacher's output from that feature. At inference, the student runs once; the heads run as needed.</figcaption> </figure>

This all sounds elegant and easy. In practice, naive multi-teacher distillation with teachers that have very different activation statistics runs into a known failure mode: the student disproportionately learns the teacher with the loudest loss and underfits the quieter ones. The PHI-S paper documents this quantitatively. Text alignment capabilities are the typical casualty when DINO-style teachers are in the mix, because their feature distributions are much wider.

The reason, it turned out, was geometry.

11. Why three teachers clash

Let's go back to feature space. Imagine DINOv3's output features as a cloud of points in a high-dimensional space. Each image produces one point. The cloud as a whole occupies some region of the space. How big is that region? How spread out are the points?

This property (how widely a set of feature vectors spreads out in its space) has a technical name: angular dispersion. You can measure it empirically by taking a large batch of images, encoding each one, and computing the average angle between every pair of features. A cloud that's tightly clustered has low angular dispersion. A cloud that fans out widely has high angular dispersion.

Here are the measured values, from the C-RADIOv4 paper:

DINOv3-7B — dispersion $\approx 2.186$
SigLIP2-g — dispersion $\approx 0.694$

DINOv3's features spread over roughly three times as much angular volume as SigLIP2's. The geometric analogy: DINOv3 is a floodlight, casting a broad cone. SigLIP2 is a laser pointer, emitting a narrow beam.

<figure> <svg viewBox="0 0 680 280" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <circle cx="340" cy="80" r="5" fill="#1A1A1A"/> <text x="355" y="78" font-family="serif" font-size="11" font-style="italic" fill="#6B6258">shared feature origin</text> <path d="M340,80 L180,250 L500,250 Z" fill="#2563EB" fill-opacity=".25" stroke="#2563EB" stroke-width="2"/> <path d="M340,80 L295,250 L385,250 Z" fill="#059669" fill-opacity=".4" stroke="#059669" stroke-width="2"/> <rect x="130" y="253" width="150" height="22" fill="#2563EB"/> <text x="205" y="268" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#FFFFFF">DINOv3: 2.186</text> <rect x="400" y="253" width="150" height="22" fill="#059669"/> <text x="475" y="268" text-anchor="middle" font-family="JetBrains Mono" font-size="11" fill="#FFFFFF">SigLIP2: 0.694</text> </svg> <figcaption><strong>Fig 8.</strong> Two teacher geometries, drawn as cones originating from the shared feature-space origin. DINOv3's cone is wide; SigLIP2's is narrow. They live in different regions of the same space, but the DINOv3 cloud occupies roughly 3× the angular volume.</figcaption> </figure>

Why does this kill naive distillation? Because of how distillation losses work. The student is trained to match each teacher's output. The loss for each teacher is something like "squared distance between student's prediction and teacher's target," averaged across a batch.

If DINOv3's features are spread over a wide angular range, the typical distance from one DINOv3 feature to another is large. Large distances mean large loss values. So DINOv3's loss contribution to the total loss (summed across all teachers) is large. SigLIP2's features are bunched tightly together, so their distances are small, so their loss contribution is small.

When you backpropagate, gradient magnitudes scale with loss magnitudes. DINOv3's gradient dominates. SigLIP2's gradient gets drowned out. The student optimizes the thing that's loudest, which is DINOv3, and ignores the thing that's quiet, which is SigLIP2. End result: you've trained a DINOv3 lookalike with a vestigial, useless SigLIP2 head.

This is a geometric problem, and it has a geometric solution.

12. Three ingredients of the C-RADIO recipe

The C-RADIOv4 tech report builds on a set of tricks that have been accumulating in the RADIO line for a while. Two of them — PHI-S and shift equivariance — were introduced in earlier RADIO papers (2024) and refined in v4. The third, stochastic multi-resolution training, is the natural culmination of work on the "mode switching" problem that has been chipped away at across several releases. Taken together, they're the core ingredients that make agglomerative distillation work at all.

Ingredient 1: Balanced Summary Loss (PHI-S)

Divide each teacher's loss contribution by that teacher's angular dispersion. Before the losses get summed, DINOv3's loss gets scaled down by 2.186 and SigLIP2's loss gets scaled up (or equivalently, scaled down by 0.694). After this normalization, both teachers contribute the same total gradient magnitude. The floodlight and the laser pull with equal force.

The exact technique, which NVIDIA calls PHI-S (for "PHI-Standardization"), goes a bit further than dividing by a single dispersion number. Under the hood it applies a Hadamard rotation to each teacher's feature space and then standardizes the resulting dimensions uniformly, so that the distribution becomes isotropic: no direction in the feature space dominates any other. The practical effect is the one you'd expect — the student is forced to learn the shape of each teacher's feature space, not just its loudest directions — and the "divide by dispersion" framing above captures the spirit even if the mechanics are a little fancier. (Note: PHI-S was introduced earlier in the RADIO line, around v2.5 in 2024. C-RADIOv4 inherits it rather than introducing it.)

After PHI-S, multi-teacher distillation actually works. The student carries all three teachers' capabilities in a single embedding. But we're not done. There are still two subtle failure modes that show up when you try to distill from many teachers at once.

Ingredient 2: Shift Equivariance

Here's a problem nobody thinks about until they're deep into multi-teacher distillation: every teacher has bad habits. Little position-dependent quirks that aren't really about the image content but are artifacts of how the teacher was trained.

A famous example in the ViT literature: large self-supervised models tend to produce a handful of "artifact tokens": patches with anomalously large activations that don't correspond to anything in the image but instead get used as scratchpad memory for the network. DINOv3 actually addresses this with an architectural trick called register tokens: four extra learnable tokens prepended to the sequence, specifically designed to absorb the scratchpad role so that the real patch tokens can stay clean. That helps a lot. But it doesn't eliminate every position-locked quirk. Residual patterns survive in all the teachers. SigLIP2 has dead zones along the borders of its feature maps, because its training data was cropped and the border regions saw systematically less variation. SAM3 has grid-aligned artifacts that come from its windowed attention pattern. DINOv3 still has subtler traces tied to specific coordinates in its positional encoding.

None of these habits are properties of the input image. They're properties of the teacher's architecture and training, anchored to specific pixel coordinates. When you distill naively, the student learns these artifacts faithfully and bakes them into its own features forever.

The fix: shift equivariance. During training, take the student's input and translate it by a random small offset. Shift the image left by 7 pixels, say. The teacher sees the original un-shifted image. When you compare student and teacher outputs, you un-shift the student's feature map (moving it back to align with the teacher's) before computing the loss.

Now think about what these artifacts have in common. The border dead zones are at fixed pixel coordinates. The SAM3 grid artifacts are at fixed pixel coordinates. Any residual position-locked pattern lives at fixed pixel coordinates. In the un-shifted student's feature map, the "corresponding" patch is at a different pixel coordinate every batch, because the shift offset changes randomly. So the artifacts stop correlating with anything. The student can't learn to produce them, because there's no fixed position where they consistently appear. The student gives up memorizing positions and learns only the input-content-dependent part of each teacher's signal.

It's a minimal fix. One line of code during training, and an entire class of inherited garbage disappears from the student's features.

Ingredient 3: Stochastic Multi-Resolution

The third ingredient concerns a subtle issue you only notice when you try to use a model at many different resolutions. Standard ViT training happens at a single fixed resolution, usually 224px or 336px. When you deploy the model and someone sends it a 1024px image, you're asking it to generalize to a regime it never saw.

What ends up happening, in practice, is the model learns two modes. One mode works at the training resolution and assumes certain statistical properties of the input (fixed number of patches, fixed receptive field, fixed position indices). Another mode, activated when the image is much larger, has to awkwardly interpolate. The two modes conflict. The same features don't mean the same thing at 224px and 1024px. Inference becomes brittle. Downstream tasks that need a specific resolution work great; tasks that need a different one suddenly fail.

C-RADIOv4 solves this by randomly sampling the training resolution on every batch, from a discrete set that spans low resolutions (128, 192, 224, 256, 384, 432) and high resolutions (512, 768, 1024, 1152). The student learns a single representation function that works at every scale. No modes. No special cases. One checkpoint serves any resolution at inference.

This also has a lovely side effect: it lets C-RADIOv4 track the full scaling curve of DINOv3-7B at a fraction of the parameter count. A 631M-parameter C-RADIOv4-H student matches (and often exceeds) an 840M-parameter DINOv3-H+ on dense benchmarks, precisely because the student has been pressured by multiple teachers to be efficient. It can't afford to waste parameters on resolution-specific specializations.

Checkpoint: what you know now.

No single vision backbone does everything well. DINOv3 has dense self-supervised features but no text and no instance separation. SigLIP2 has text alignment but no dense spatial structure. SAM3 has mask-supervised features with instance separation but narrow global semantics.

Multi-teacher distillation can fuse them into one student, with three key ingredients accumulated across RADIO releases:

(a) PHI-S normalization so that the floodlight and the laser contribute equal gradient weight; (b) shift equivariance so the student doesn't inherit position-locked teacher artifacts; (c) stochastic multi-resolution so the student learns a single scale-invariant function instead of a per-resolution mode.

The result is C-RADIOv4: one backbone that runs all three teachers' capabilities in a single forward pass, with 412M or 631M parameters depending on the variant.

But we have one problem left. The trick only works if all three teachers can produce usable targets at the training resolution. DINOv3 can. SAM3 can. SigLIP2 cannot.

Act 3 — The Dial

FeatSharp, a 2D upsampler built on a 3D reconstruction trick.

13. Resolution, the last mile

SigLIP2 was trained at 384×384. Its feature map, at that input resolution, is 24×24, one vector per 16×16 pixel patch. If you ask SigLIP2 to process a 1152×1152 image, its internal machinery gets cranky. Position embeddings, which were learned at the training resolution, now have to extrapolate. The feature map becomes 72×72, but the patches at those new positions have position encodings the model has never seen. The features degrade, not catastrophically, but visibly. Boundaries become fuzzy. Patterns repeat where they shouldn't. Textures look like plaid.

You can see it with your eyes if you run a PCA on the feature map and visualize it as a color image. At 384px, SigLIP2's features look clean, crisp, object-aware. At 1152px via simple bilinear interpolation, they look like a defocused photograph.

The problem for C-RADIOv4: if the training resolution is random between 128px and 1152px, the student sometimes sees images at 1152px. At that resolution, the student needs to produce features that match SigLIP2's features. But SigLIP2 can't produce clean features at 1152px. The supervision signal is noise. The student learns nothing useful from SigLIP2 at high resolutions, which means its text-alignment capabilities stay weak.

We need a way to get clean SigLIP2 features at 1152px, without retraining SigLIP2. This is the job of FeatSharp.

14. The NeRF trick for features

To explain FeatSharp, I have to briefly talk about NeRF. NeRF (Neural Radiance Fields) is a 3D reconstruction technique from 2020. Its setup is this: you have a few dozen 2D photographs of a real-world scene, taken from different viewpoints. You want to reconstruct a 3D representation of the scene that you can render from any new viewpoint.

NeRF's core insight is a consistency principle. A real 3D point in the scene is observed in multiple 2D photos. When you look at where that point projects to in photo A, you see a specific color. When you look at where it projects to in photo B (from a different angle), you see that same color, or something close to it, depending on lighting. Features of real 3D structure are consistent across views.

Conversely: anything that varies across views is not real 3D structure. It's an artifact: sensor noise, specular reflection, glare, occlusion, something that isn't part of the stable underlying scene.

NeRF uses this to build a coherent 3D representation: it finds the 3D structure that, when rendered to the 2D photographs, is consistent with all of them. Parts that can't be made consistent are rejected as noise.

Now: the same principle applies, in a totally different setting, to feature upsampling.

Imagine you have a low-resolution encoder, say SigLIP2 at 384px. You want its feature map at 1152px. Naive bilinear interpolation will hallucinate details the encoder never committed to. But what if you exploit consistency across views of the feature map?

Here's the FeatSharp recipe:

Take the input image. Jitter it slightly: shift it by a few pixels, crop it differently, flip it horizontally, mess with its alignment. Generate many small variations ("views") of the same underlying scene.
Run the low-resolution encoder on every jittered view. Collect the feature maps. They're all low-resolution, but they're all slightly misaligned versions of roughly the same content.
Learn a high-resolution feature map such that, when you apply the corresponding jitter and downsample to low resolution, the result matches each view's low-res feature map.
Features that are consistent across views of the same content survive the upsampling process. Features that flicker, changing from one jitter to the next, get suppressed.

<figure> <svg viewBox="0 0 680 260" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <g font-family="JetBrains Mono" font-size="10" fill="#6B6258"><text x="80" y="22" text-anchor="middle" letter-spacing="1">JITTERED VIEWS</text></g> <g> <rect x="40" y="35" width="80" height="50" fill="#CFF0F6" stroke="#0E7490" stroke-width="1.5" transform="rotate(-3 80 60)"/> <rect x="40" y="95" width="80" height="50" fill="#CFF0F6" stroke="#0E7490" stroke-width="1.5" transform="rotate(2 80 120)"/> <rect x="40" y="155" width="80" height="50" fill="#CFF0F6" stroke="#0E7490" stroke-width="1.5" transform="rotate(-1 80 180)"/> </g> <g stroke="#8F8578"> <line x1="130" y1="60" x2="185" y2="95" marker-end="url(#a4)"/> <line x1="130" y1="120" x2="185" y2="120" marker-end="url(#a4)"/> <line x1="130" y1="180" x2="185" y2="145" marker-end="url(#a4)"/> </g> <defs><marker id="a4" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="5" markerHeight="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="#8F8578"/></marker></defs> <rect x="190" y="95" width="100" height="50" fill="none" stroke="#0E7490" stroke-width="1.5"/> <text x="240" y="120" text-anchor="middle" font-family="serif" font-size="13" font-weight="600" fill="#0E7490">SigLIP2</text> <text x="240" y="136" text-anchor="middle" font-family="serif" font-size="10" fill="#6B6258" font-style="italic">low-res encoder</text> <line x1="295" y1="120" x2="330" y2="120" stroke="#8F8578" marker-end="url(#a4)"/> <g font-family="JetBrains Mono" font-size="9" fill="#6B6258"><text x="395" y="22" text-anchor="middle" letter-spacing="1">LOW-RES FEATURE MAPS</text></g> <g> <rect x="335" y="35" width="60" height="50" fill="#CFF0F6" stroke="#0E7490"/> <rect x="400" y="35" width="60" height="50" fill="#CFF0F6" stroke="#0E7490"/> <rect x="335" y="95" width="60" height="50" fill="#CFF0F6" stroke="#0E7490"/> <rect x="400" y="95" width="60" height="50" fill="#CFF0F6" stroke="#0E7490"/> <rect x="335" y="155" width="60" height="50" fill="#CFF0F6" stroke="#0E7490"/> <rect x="400" y="155" width="60" height="50" fill="#CFF0F6" stroke="#0E7490"/> <text x="397" y="220" text-anchor="middle" font-family="serif" font-size="10" fill="#6B6258" font-style="italic">all jittered, misaligned</text> </g> <line x1="465" y1="120" x2="500" y2="120" stroke="#8F8578" marker-end="url(#a4)"/> <text x="505" y="110" font-family="serif" font-size="10" fill="#6B6258" font-style="italic">solve:</text> <text x="505" y="125" font-family="serif" font-size="10" fill="#6B6258" font-style="italic">the HR map that,</text> <text x="505" y="138" font-family="serif" font-size="10" fill="#6B6258" font-style="italic">downsampled, explains</text> <text x="505" y="151" font-family="serif" font-size="10" fill="#6B6258" font-style="italic">every view</text> <rect x="580" y="60" width="80" height="120" fill="#FBE9CE" stroke="#B8651A" stroke-width="2"/> <g stroke="#B8651A" stroke-width=".5" opacity=".4"> <line x1="580" y1="90" x2="660" y2="90"/><line x1="580" y1="120" x2="660" y2="120"/><line x1="580" y1="150" x2="660" y2="150"/> <line x1="600" y1="60" x2="600" y2="180"/><line x1="620" y1="60" x2="620" y2="180"/><line x1="640" y1="60" x2="640" y2="180"/> </g> <text x="620" y="200" text-anchor="middle" font-family="JetBrains Mono" font-size="10" fill="#B8651A" letter-spacing="1">HIGH-RES MAP</text> <text x="620" y="215" text-anchor="middle" font-family="serif" font-size="10" fill="#6B6258" font-style="italic">consistent detail</text> </svg> <figcaption><strong>Fig 9.</strong> FeatSharp's multi-view consistency: many jittered versions of the same input produce many low-res feature maps, misaligned in different ways. The algorithm solves for a single high-res feature map that, under each jitter and downsampling, reproduces every view. Details consistent across views are real; details that flicker are noise.</figcaption> </figure>

The result: a clean, high-resolution SigLIP2 feature map that is fully consistent with what SigLIP2 would have produced at low resolution, and contains only the details that survive the multi-view consistency test. No hallucinated boundaries. No plaid artifacts. A supervision signal C-RADIOv4 can actually learn from.

FeatSharp is a small module, about as expensive as one extra transformer block, and it plugs into the C-RADIOv4 training pipeline as a preprocessor for SigLIP2's targets. It's also useful independently: any time you have a pretrained low-resolution vision encoder and want to extract richer spatial information from it, FeatSharp can upgrade the feature resolution without retraining the encoder.

15. Interlude: borrowing from biology

Before we put the whole pipeline together, one more idea deserves its own section. It's used across all three papers we've discussed, but it's not really about computer vision. It's about how neural networks should be trained in the first place. And it's borrowed, more literally than is typical, from biology.

Here's the setup. When you train a neural network, a common trick for improving robustness is to inject noise somewhere in the training process. Add a bit of Gaussian noise to the inputs. Add dropout to the activations. Add weight decay to the gradients. All of these are additive perturbations, and they all work reasonably well.

Biological synapses work differently. Real synapses in real brains do something called multiplicative noise. The fluctuation in a synaptic signal is proportional to the signal's magnitude. When a synapse is firing weakly, its noise is small. When it's firing strongly, its noise is big. The ratio stays roughly constant, a 10% jitter around whatever the signal is. This follows a log-normal distribution, which has been measured experimentally in mouse brains, rat brains, human brains. It's a universal property of biological neural machinery.

Another way to see the difference: imagine the volume knob on a stereo. If the music is playing quietly and you bump the knob slightly, you barely notice. If the music is already at maximum volume and you bump the knob by the same amount, the change is enormous and chaotic. Biological noise is the second kind. Additive noise in standard ML training is the first kind. They produce qualitatively different optimization dynamics.

In 2024, a paper called DAMP (Data Augmentation via Multiplicative Perturbations) made a clever observation: an input corruption, to first order, looks a lot like a multiplicative perturbation of the weights in the downstream layer. So instead of augmenting the data, you can augment the weights: multiply each weight by a small random Gaussian during the forward pass, and train normally. DAMP demonstrated that this is enough to produce networks that are dramatically more robust to real-world input corruptions. No extra data, no extra forward passes. Just change the noise structure.

DAMP's justification is geometric, not biological. But the family of techniques it opens up turns out to line up closely with what biology does. In 2025, LMD (Log-Normal Multiplicative Dynamics) made the biological motivation explicit: it derives a Bayesian learning rule that assumes log-normal posterior distributions over weights, directly inspired by the log-normal distribution of biological synaptic weights, and uses multiplicative updates with both noise and regularization applied multiplicatively. LMD achieves stable training at very low numerical precision, which matters a lot for next-generation hardware.

C-RADIOv4's training recipe explicitly uses both DAMP and a related technique, MESA (Memory-Efficient Sharpness-Aware Training), a cousin of these methods that approximates Sharpness-Aware Minimization via a trajectory loss against an EMA teacher, driving the optimizer into "flat" regions of the loss landscape, where small perturbations don't change the loss much. Flat regions generalize better than sharp regions. So you get the same dataset, the same architecture, and a model that degrades more gracefully on unseen data. Closely related in spirit: ASAM (Adaptive SAM) has been shown to be mathematically equivalent to optimizing under adversarial multiplicative weight perturbations, which is what ties the sharpness-aware line back to the multiplicative-noise story in this section.

The bigger lesson: a lot of ML practice inherits assumptions from statistics textbooks (Gaussian noise, L2 penalties, isotropic distributions) that have nothing to do with how biological computation actually works. Occasionally, questioning those assumptions and reaching into neuroscience produces a free lunch. Multiplicative weight perturbation is one of the cleanest free lunches of the last few years.

There's a philosophical angle worth noting. We've spent two decades assuming that biological intelligence and artificial intelligence are "different kinds of systems," and that trying to directly translate neural biology into deep learning is a mug's game. Sometimes that's right. Backpropagation is not how brains learn, and anyone who says otherwise is selling something. But the log-normal synapse story is a counterexample. A specific, measured, boring property of real neurons (their noise is proportional to their signal) turns out to make artificial networks measurably better when imitated. Not every biological property will. Some of them will. The field is slowly learning which.

16. Putting it together: the pipeline

Let's zoom back out. We've now covered every piece. Here's the full training pipeline for a 2026 vision foundation model, from raw images to deployable checkpoint.

<figure> <svg viewBox="0 0 900 260" xmlns="http://www.w3.org/2000/svg" width="100%" height="auto"> <g font-family="serif" font-size="14" font-weight="600"> <rect x="30" y="40" width="200" height="192" fill="#F4EEE3" stroke="#2563EB" stroke-width="2"/> <rect x="30" y="40" width="200" height="52" fill="#2563EB"/> <text x="50" y="58" fill="#FFFFFF" font-size="11" letter-spacing="2" font-family="JetBrains Mono" font-weight="400">STAGE 1</text> <text x="130" y="80" text-anchor="middle" fill="#FFFFFF">Pretrain</text> <g font-family="serif" font-size="11.5" fill="#1A1A1A" font-weight="400"> <text x="45" y="112">· Curated 1.7B images</text> <text x="45" y="132">· DINO + iBOT losses</text> <text x="45" y="152">· 1M iterations</text> <text x="45" y="172">· ViT-7B, constant schedule</text> </g> <text x="45" y="207" font-family="JetBrains Mono" font-size="9" fill="#6B6258" letter-spacing="1.5">RESULT</text> <text x="45" y="222" font-family="serif" font-size="11" fill="#2563EB" font-style="italic">dense collapse at the end</text> </g> <g stroke="#8F8578" stroke-width="1.5"><line x1="233" y1="136" x2="252" y2="136" marker-end="url(#a5)"/></g> <defs><marker id="a5" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="5" markerHeight="5" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="#8F8578"/></marker></defs> <g> <rect x="255" y="40" width="200" height="192" fill="#F4EEE3" stroke="#15803D" stroke-width="2"/> <rect x="255" y="40" width="200" height="52" fill="#15803D"/> <text x="275" y="58" fill="#FFFFFF" font-size="11" letter-spacing="2" font-family="JetBrains Mono">STAGE 2</text> <text x="355" y="80" text-anchor="middle" fill="#FFFFFF" font-family="serif" font-weight="600" font-size="14">Gram Anchor</text> <g font-family="serif" font-size="11.5" fill="#1A1A1A"> <text x="270" y="112">· Early-ckpt Gram teacher</text> <text x="270" y="132">· Refresh every 10k iter</text> <text x="270" y="152">· Add Gram loss</text> <text x="270" y="172">· Refinement phase</text> </g> <text x="270" y="207" font-family="JetBrains Mono" font-size="9" fill="#6B6258" letter-spacing="1.5">RESULT</text> <text x="270" y="222" font-family="serif" font-size="11" fill="#15803D" font-style="italic">dense features restored</text> </g> <g stroke="#8F8578" stroke-width="1.5"><line x1="458" y1="136" x2="477" y2="136" marker-end="url(#a5)"/></g> <g> <rect x="480" y="40" width="200" height="192" fill="#F4EEE3" stroke="#0E7490" stroke-width="2"/> <rect x="480" y="40" width="200" height="52" fill="#0E7490"/> <text x="500" y="58" fill="#FFFFFF" font-size="11" letter-spacing="2" font-family="JetBrains Mono">STAGE 3</text> <text x="580" y="80" text-anchor="middle" fill="#FFFFFF" font-family="serif" font-weight="600" font-size="14">High-Res Adapt</text> <g font-family="serif" font-size="11.5" fill="#1A1A1A"> <text x="495" y="112">· Mixed crops 512/768</text> <text x="495" y="132">· Gram anchor still on</text> <text x="495" y="152">· 10k iterations</text> <text x="495" y="172">· Any-resolution support</text> </g> <text x="495" y="207" font-family="JetBrains Mono" font-size="9" fill="#6B6258" letter-spacing="1.5">RESULT</text> <text x="495" y="222" font-family="serif" font-size="11" fill="#0E7490" font-style="italic">stable up to 4k inference</text> </g> <g stroke="#8F8578" stroke-width="1.5"><line x1="683" y1="136" x2="702" y2="136" marker-end="url(#a5)"/></g> <g> <rect x="705" y="40" width="180" height="192" fill="#F4EEE3" stroke="#B8651A" stroke-width="2"/> <rect x="705" y="40" width="180" height="52" fill="#B8651A"/> <text x="725" y="58" fill="#FFFFFF" font-size="11" letter-spacing="2" font-family="JetBrains Mono">STAGE 4</text> <text x="795" y="80" text-anchor="middle" fill="#FFFFFF" font-family="serif" font-weight="600" font-size="14">Agglomerate</text> <g font-family="serif" font-size="11.5" fill="#1A1A1A"> <text x="720" y="112">· DINO + SigLIP + SAM</text> <text x="720" y="132">· FeatSharp upsamples</text> <text x="720" y="152">· PHI-S, shift eq, MESA</text> <text x="720" y="172">· Stoch. res 128–1152</text> </g> <text x="720" y="207" font-family="JetBrains Mono" font-size="9" fill="#6B6258" letter-spacing="1.5">RESULT</text> <text x="720" y="222" font-family="serif" font-size="11" fill="#B8651A" font-style="italic">unified 631M student</text> </g> </svg> <figcaption><strong>Fig 10.</strong> The four-stage pipeline. Each stage fixes a specific problem introduced by the previous one. Stage 1 trains a ViT-7B on curated data and exposes dense feature collapse. Stage 2 fixes collapse via Gram anchoring. Stage 3 extends the model to arbitrary resolutions. Stage 4 distills it, alongside SigLIP2 and SAM3, into a smaller, more efficient student that carries all three specialists' capabilities.</figcaption> </figure>

There's one more piece worth explaining: how the pipeline scales down for deployment. The 6.7-billion-parameter DINOv3-7B model is wonderful for research, terrible for production. It's slow, expensive, and hard to serve. Most real use cases want a model that fits on a single GPU, runs in under 100ms, and doesn't require a dedicated inference cluster.

The solution is multi-student distillation. Instead of training one small student to mimic the big teacher, you train an entire family of them simultaneously: a ViT-Small (21M params), a ViT-Base (86M), a ViT-Large (304M), a few ConvNeXt variants, all at once. The real trick: you only run the expensive teacher once per batch. Cache its outputs. Then broadcast those cached targets to every student. Each student consumes the same distillation signal but learns its own efficient architecture.

This is how a 6.7B self-supervised teacher ends up producing a 21M ConvNeXt you can put on a Jetson. It's also why the DINOv3 "model family" includes so many sizes: Meta trained them all in parallel, sharing the teacher's compute across every student.

Checkpoint: what you know now.

The full pipeline: pretrain a big ViT with self-supervision → repair dense features with Gram anchoring → adapt to arbitrary resolutions → distill multiple specialist teachers into a unified student → distill the student into many smaller students for deployment.

The output: a family of vision backbones ranging from 21M to 840M parameters (and a 6.7B research flagship on the DINOv3 side) that all share the same fundamental capabilities (classification, segmentation, depth estimation, text alignment, video tracking, cross-domain transfer) in a single forward pass. You pick the size that fits your hardware budget.

17. Three ideas worth stealing

We've covered a lot of ground: self-supervision, Gram matrices, multi-teacher distillation, angular dispersion, NeRF-inspired upsampling, log-normal synapses. If you only take three things away from all of it, I would pick these:

1. Regularize relationships, not values.

Gram anchoring is the cleanest example of a pattern that keeps showing up across machine learning: when some property of your model is degrading during training, the fix is often not to freeze the values of the relevant quantities but to freeze their relational structure. The Gram matrix preserves pairwise similarities while letting individual features evolve. The underlying idea, pin the geometry and free the content, is applicable far outside vision. Any time you have a representation that you want to stay structurally coherent while continuing to learn new things, ask what relational property captures the structure you care about, and regularize that.

2. Heterogeneous signals need geometric normalization.

The C-RADIOv4 three-teacher problem is, at its heart, a generic issue with combining evidence from sources with different natural scales. Naive weighted averaging fails silently when the sources have different variances or different angular ranges. You don't get a helpful error message; you get a model that ignored the quieter source. The fix is to normalize by each source's intrinsic scale before combining. PHI-S is one specific way to do this in feature space, but the general principle (measure each source's natural magnitude and divide it out before summing) applies broadly. Multi-loss optimization, multi-task learning, sensor fusion, ensemble methods: all of them have this failure mode, and the fix is always some variant of the same normalization.

3. Randomize what you don't want memorized.

Shift equivariance, stochastic multi-resolution, multiplicative weight perturbation: these are all, at some level, the same trick. If there's a structural property of your training setup that you don't want the model to encode as a shortcut, randomize it during training. Fixed pixel coordinates become random offsets. Fixed resolutions become sampled resolutions. Fixed noise levels become proportional fluctuations. The model learns the part that's invariant to the randomization, which, by construction, is the part you actually want it to learn. This is an old idea dressed up in many clothes, but it bears repeating: if your model is learning something you didn't intend, look for a way to make that something vary from batch to batch, and watch it disappear.

Epilogue

It's tempting to read the story of DINOv3 / C-RADIOv4 / FeatSharp as a procession of clever technical tricks, each patching over a specific failure mode. That's true, but it's not the whole truth. What's really happening, I think, is that the field is converging on a more mature understanding of what a vision model is.

The early ViT models were trained on one objective and evaluated on one benchmark. They were good at that benchmark and bad at everything else. The 2025–2026 generation is trained on multiple objectives, evaluated on dozens of benchmarks across domains the model was never specialized for, and expected to do all of it from a single frozen backbone. The architecture didn't really change. What changed is the training recipe and, more importantly, the understanding of what constraints the recipe needs to satisfy for the resulting features to be useful in the wild.

Gram anchoring is a constraint. Shift equivariance is a constraint. Multi-resolution sampling is a constraint. Multiplicative noise is a constraint. Each one narrows the space of functions the model can learn, in exchange for making the functions that remain more robust, more transferable, more general. The field is learning to write down the right constraints.

If there's a pattern here beyond computer vision, it's this: modern machine learning is increasingly about finding the right structure to preserve, not the right thing to learn from scratch. Gradient descent will learn almost anything, given enough data and compute. Making it learn the right thing (the thing that transfers, the thing that generalizes, the thing that survives contact with reality) is a matter of writing down what must remain invariant. Dense feature structure must remain invariant to continued training. Cross-teacher gradients must remain balanced. Feature maps must remain invariant to small input shifts. Performance must remain invariant to resolution. These are not subtle philosophical demands. They are engineering constraints, with concrete losses and concrete training tricks that enforce them.

The payoff is real. A single DINOv3 backbone beats specialized Earth-observation networks at satellite tree-height estimation. A single C-RADIOv4 backbone, at 631 million parameters, ties or beats its much larger peers (including DINOv3-H+, an 840M distilled sibling of its 7B teacher) on dense benchmarks. A single inference call provides classification, segmentation, depth, tracking, correspondence, text alignment, and open-vocabulary masking. The vision backbone is becoming, increasingly, just a backbone: a single spinal cord that a thousand downstream heads can read from.

Which is, when you think about it, pretty close to how biological vision works. A single visual cortex produces a representation that the rest of the brain interprets in a dozen different ways, for a dozen different purposes, using a dozen different downstream circuits. The representation is fixed; the interpretations are flexible. It took artificial vision a while to get there. It seems to be arriving.

We used to think the blurry mess was a failure of the machine. It turns out the blur was a failure of the objective.

References

DINOv3. Siméoni et al., Meta AI 2025 (arXiv:2508.10104).
C-RADIOv4 Tech Report. Ranzinger et al., NVIDIA 2026 (arXiv:2601.17237).
FeatSharp. Ranzinger et al., NVIDIA 2025 (arXiv:2502.16025).
PHI-S (distribution balancing). Ranzinger et al., NVIDIA 2024 (arXiv:2410.01680).
DAMP (multiplicative weight perturbations). Trinh et al. 2024 (arXiv:2406.16540).
LMD (log-normal multiplicative dynamics). Nishida et al., RIKEN AIP 2025 (arXiv:2506.17768).
SAM 3. Meta AI 2025 (arXiv:2511.16719).
SigLIP 2. Tschannen et al., Google 2025.

Originally prepared for the Boon engineering team, April 2026.

I Ship 20 PRs a Weekend with Claude Code. Here's the Workflow.

Tue, 03 Mar 2026 00:00:00 GMT

I watched Claude Code try to delete three pages from our app because I told it to "change the routing."

That's not what I wanted. I wanted to redirect one button to a different page. But Claude read "change the routing" and decided the old pages weren't needed anymore. If I hadn't read the plan before hitting execute, it would've nuked components we still needed elsewhere.

This is the most important thing I can tell you about working with AI coding tools: the planning phase is everything.

The Decision Tree

When a new task lands on your desk, the first question is simple: do you know exactly what to build?

Sometimes the spec is crystal clear. A Linear ticket with screenshots, reproduction steps, the works. In that case, skip the brainstorming and go straight to Feature Dev. Tell Claude Code what you need, and it'll generate a plan.

But most of the time? The spec is vague. A customer said something hand-wavy. Your PM wrote two sentences in a Slack thread. You kind of know what they want, but the details are fuzzy.

That's when you brainstorm first.

Brainstorm Mode Goes Deep

When I use Superpower Brainstorm, Claude doesn't just ask "what do you want to build?" It goes deep. Should this icon be here? What happens when a user clicks this button? Should we use this component or that one? Do you want a modal or a new page?

It's like a product discovery session, except it's just you and Claude, and it happens in five minutes instead of an hour-long meeting.

People complain about "AI slop" in code. I get it. But slop is just under-specified spec. If you tell Claude "build a login page" with no other context, yeah, you'll get generic slop. It doesn't know your component library. It doesn't know your auth patterns. It doesn't know you already have a FormInput component that handles validation.

But if you brainstorm first, if you tell it "reuse our existing FormInput, follow the pattern in the signup page, use our auth service not a new one," the output is clean. It matches your codebase. Because you told it exactly what "good" looks like in your project.

The engineers on my team who get the best output are the ones who spend the most time on the spec. The ones who get slop are the ones who skip the brainstorm and type a one-liner.

If your task involves any kind of UI, add Frontend Design to the brainstorm. It's surprisingly good at thinking through layouts and interactions before any code gets written.

Plan Mode: Shift+Tab Before Anything

Here's a tip that changed everything for me: always start in plan mode.

Press Shift+Tab until you're in plan mode. Do all your brainstorming, feature definition, and design exploration here. Claude will think through the problem and show you a plan.

Your job now is to read that plan.

I can't stress this enough. Read the plan. This is the actual job now.

Going back to my routing example: the plan said "remove pages X, Y, and Z since routing will change." I caught it because I read the plan. I told Claude "don't delete those pages, just change where the button points to." It updated the plan in seconds.

Another common one: Claude will want to create a brand new component when you already have one that does basically the same thing. I'll see that in the plan and say "reuse the existing DataTable component, don't create a new one. Follow the same pattern we use in the estimates view." Then it goes back into the codebase, finds the pattern, and adjusts the plan.

Same thing with tests. If the plan says "add unit tests" but it's only covering the happy path, tell it: "make sure the unit testing is comprehensive." It'll go deeper on edge cases, error handling, boundary conditions. The plan shapes how much effort Claude puts into each part of the execution. If you don't push back on thin test coverage during planning, you'll get thin test coverage in the code.

This back-and-forth takes maybe two or three rounds. Sometimes five for complex features. It's the most valuable time you'll spend.

Research During Planning

Here's something most people miss: Claude's training data is already outdated by the time you're using it.

Real example from my team: during a planning session, Claude proposed using tailscale serve for exposing a local service. I knew that was the old way. So I told it: "research the best practice for Tailscale in 2026." It did a web search, found that tailscale services is the newer approach (lets you run multiple services on the same node), and updated the plan.

Don't trust Claude's memory for anything that changes fast. Tell it to use Context7 for documentation lookups, or just say "research the current best practice for X." It combines web search with doc lookups and comes back with something current.

This is especially important for frameworks and libraries. React patterns from 2023 are not React patterns from 2026. Always verify.

Feed It Everything You've Got

The more context you give Claude during planning, the better your plan comes out.

We use the Linear MCP, so I just paste the ticket URL into Claude Code. It fetches everything (description, comments, acceptance criteria) automatically. I don't even need to describe the task most of the time.

Here's what that looks like in practice. I run /feature-dev with a Linear URL, and Claude fetches the ticket, spins up parallel exploration agents to understand the codebase, and builds a multi-phase plan:

If someone reports a bug with a video, I'll watch the video myself, take screenshots of the key moments (what's wrong, what it should look like), and drag them straight into the terminal. Claude handles images surprisingly well for UI problems. We even built an analyze-video skill that automatically extracts screenshots from videos for Claude. Happy to share if anyone wants it.[^1]

I also pull context from other repos. "Hey, look at the backend repo at ~/projects/boon-api and find how we handle PDF processing there." Claude reads the relevant files and incorporates that context into the plan. I keep my repos in sibling folders so they can reference each other easily.

If your product docs live in Notion or Google Docs, use those MCPs too. Pass in the PRD link and Claude pulls the context it needs.

Execute: Clear Context and Let It Run

Once my plan looks good, I execute it. And here's the counterintuitive part: I clear the context 90% of the time.

The plan itself contains everything Claude needs to execute. All that back-and-forth conversation about "should we use this or that?" is noise at execution time. Clear it. Start fresh with just the plan.

The 10% where I keep context: when there was a really specific conversation about an edge case or architectural decision that isn't captured in the plan. But that's rare.

On `--dangerously-skip-permissions`

Yeah, I use it. I've been using it since day one.

I know this scares people. You read horror stories online about AI agents deleting databases or nuking folders. But here's the thing: if you did the planning phase right, Claude already showed you exactly what it's going to do. You reviewed the plan. You know what files it'll touch.

The worst thing that ever happened to me: it merged directly to main instead of opening a PR. I reverted it in 30 seconds. That's it.

If you're uncomfortable with it, don't use it. But understand that every permission prompt interrupts your flow. That adds up fast, especially when you're running multiple instances (more on that in Part 2).

The PR Pipeline

OK so Claude finishes a task. It's run the linter, run the tests, everything passes. Now what?

Step 1: Superpower Code Review. Before pushing anything, I tell Claude to run the Superpower Code Review skill on its own work. Think of it as the "are you sure about this?" pass. It catches obvious mistakes, inconsistencies, things that don't match the codebase patterns.

Step 2: Push and open the PR.

Step 3: Automated PR review. At Boon, we have the Claude Code PR Review toolkit running alongside Cubic Reviewer on every PR. These catch stuff that the Superpower review missed. Different perspective, different checks.

Step 4: Resolve PR comments. This is where the pr-feedback plugin earns its keep.[^1] I go back to the same Claude Code instance (still has all the context from planning and execution) and say:

"Please look at the comments on the PR and then resolve them if needed and make sure to push the changes."

I do this on every single PR. Every one. Claude loads the worktree and PR feedback skills automatically, then fetches the PR comments:

Then it does something really smart. It makes a structured assessment table of every review comment, triaging each one: valid (will fix), already resolved, out of scope (follow-up PR), or pre-existing (not introduced by this PR). Then it implements only the real fixes.

It resolves every comment thread so the CI gates pass.

This works really well because Claude still has the full context from the planning and execution stages. It knows why it made each decision, so it can have an intelligent conversation with the reviewer about whether a comment is valid.

Step 5: Human review. By the time a human reviewer sees this PR, it's been through: your plan review, Superpower Code Review, automated PR toolkit review, Cubic review, and Claude's comment resolution pass. The human is looking at a clean, well-tested PR. Their job is to catch architectural concerns and business logic issues, not formatting bugs or missing null checks.

Tips

Working on multiple tickets? Use git worktrees so each Claude Code session has its own branch and working directory:

# I use atuin for cross-device alias sync, but a plain alias works fine
alias ccw='claude --dangerously-skip-permissions --worktree'

Type ccw and you get a new session with its own worktree. Three keystrokes. I'll cover the full parallel workflow in Part 2.

The Full Flow

Here's the complete single-instance workflow:

                    ┌─────────────┐
                    │  New Task   │
                    └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │ Spec clear? │
                    └──┬───────┬──┘
                   Yes │       │ No
                       │       │
              ┌────────▼┐  ┌──▼──────────────────────┐
              │ Feature │  │ Superpower Brainstorm    │
              │   Def   │  │ (+ Frontend Design if UI)│
              └────┬────┘  └────────────┬─────────────┘
                   │                    │
                   └────────┬───────────┘
                            │
                   ┌────────▼─────────┐
                   │    Plan Mode     │
                   │  (Shift + Tab)   │
                   └────────┬─────────┘
                            │
                   ┌────────▼─────────┐
                   │   Review Plan    │◄──────────┐
                   │  (READ IT!)      │           │
                   └────────┬─────────┘           │
                            │                     │
                   ┌────────▼─────────┐    ┌──────┴──────┐
                   │   Plan good?     │ No │ Iterate in  │
                   └───┬──────────┬───┘───►│  English    │
                   Yes │          └────────└─────────────┘
                       │
              ┌────────▼──────────┐
              │ Execute           │
              │ (clear context)   │
              └────────┬──────────┘
                       │
                  Claude codes...
                       │
              ┌────────▼──────────┐
              │ Superpower        │
              │ Code Review       │──► auto-fixes findings
              └────────┬──────────┘
                       │
              ┌────────▼──────────┐
              │ Manual Testing    │
              │ (you test it)     │
              └────────┬──────────┘
                       │
              ┌────────▼──────────┐
              │ Push & Open PR    │
              └────────┬──────────┘
                       │
              ┌────────▼──────────┐
              │ CI +              |
              | Automated Review  │
              │ (Cubic + PR       │
              │  Review Toolkit)  │
              └────────┬──────────┘
                       │
              ┌────────▼──────────┐
              │  /pr-feedback     │
              │  (resolve or      │
              │   recommend)      │──► shows you what to address
              └────────┬──────────┘
                       │
              ┌────────▼──────────┐
              │  Human Decision   │
              │  on comments      │──► "all of them" keeps going
              └────────┬──────────┘
                       │
              ┌────────▼──────────┐
              │  Human Review     │
              └────────┬──────────┘
                       │
              ┌────────▼──────────┐
              │     Merge ✓       │
              └───────────────────┘

Everything above is one Claude Code instance. One task, start to finish. Plan, execute, review, push, merge.

But I don't work one at a time. I run five of these in parallel. In one weekend, I shipped 20 PRs.

Part 2 is about how.

This is Part 1 of a 3-part series on my Claude Code workflow. Part 2: Parallel Execution drops tomorrow.

[^1]: Company Plugin Store. Claude Code lets you add custom marketplaces alongside the official one. We set up an internal repo and anyone on the team can install it with one command: /plugin marketplace add getboon/boon-plugins. Once added, it shows up right next to the official Anthropic marketplace: From there, engineers browse and install company-specific plugins. We've got plugins for Linear, Sentry, Slack, video analysis (analyze-video), and pr-feedback. Enable auto-update so everyone gets the latest plugins automatically. That's the beauty of a shared marketplace: when we add a new plugin or update an existing one, everyone gets it. No Slack messages saying "hey update your config." If you want to set up something similar for your team, the boon-plugins repo is a good reference.

My Claude Code Plugin Stack After Months of Trial and Error

Mon, 09 Feb 2026 00:00:00 GMT

I use Claude Code every day. Here's my full plugin stack, nothing hidden.

I only install from the official Anthropic plugin store. No community marketplaces. No random GitHub repos. It took months of trial and error to get here.

Why Official Only

Two reasons: simplicity and security.

I don't think you need 20 fancy plugins to ship working software. The official store has enough to cover my entire workflow. That's it. That's the simplicity argument.

The security argument is scarier.

Snyk published their ToxicSkills study in early 2026. They scanned 3,984 skills from community marketplaces. The findings: 13.4% contain critical security issues (malware, prompt injection, exposed secrets). Over a third of the ecosystem (36.82%) has at least one flaw. They found 1,467 malicious payloads total.

The barrier to publish a skill on community marketplaces? A markdown file and a GitHub account that's one week old. No code signing. No security review. No sandbox by default.

PromptArmor took it further and demonstrated how attackers can hijack Claude Code through injected marketplace plugins. The attack uses prompt injection to make Claude run curl commands that exfiltrate your codebase. Malicious hooks can bypass the human approval step entirely.

SentinelOne showed that marketplace skills can force Claude to install malicious dependencies without you knowing.

The official store is curated by Anthropic. Third-party plugins go through a review process before they're accepted. That's the trust baseline I'm comfortable with.

The Stack

Seven plugins and one MCP server. Here's what each one does and why it's in my stack.

Superpowers

This is the backbone. Created by Jesse Vincent, it's a skills framework that teaches Claude to actually think before it codes.

The skills I use most:

Brainstorming asks a ton of questions before you write any code. Depending on the task, it'll dig into requirements, edge cases, and alternatives you haven't considered.
Planning creates structured implementation plans that you review before execution. No surprises.
Code review catches issues before they ship. I pair this with the security-guidance plugin (more on that below).
Skill creation helps you write new skills for Claude Code. I've used it to build custom workflows for my blog writing process.

The whole philosophy is: stop and think before you code. It's the opposite of just letting Claude rip through your codebase unsupervised.

Context7

This one solves a real problem. LLMs hallucinate APIs. They suggest functions that don't exist, use deprecated patterns, or mix up versions. Context7 pulls up-to-date documentation straight from source repos and injects it into your prompts.

I use it constantly. Astro when building my blog, Bun, uv, Remix, Rails. Without it, Claude will confidently suggest APIs that were deprecated two versions ago. Context7 makes sure Claude has the right docs.

The llms.txt standard is getting better at this problem too. Over 844,000 websites have adopted it, and platforms like Mintlify now generate llms.txt automatically. But adoption isn't universal yet. Not every library has jumped on board. Context7 fills that gap.

Code Review

I use this for reviewing my own work before committing. It runs multiple review agents in parallel (checking for bugs, style, CLAUDE.md compliance) and gives confidence scores so you can focus on what actually matters.

PR Review Toolkit

I use this for reviewing other people's PRs on GitHub. Six specialized agents that each focus on a different aspect: code quality, test coverage, error handling, type design, comments, and code simplification.

The split is simple. Code Review for my code. PR Review Toolkit for everyone else's.

Feature Dev

Guided feature development with a 7-phase workflow: discovery, exploration, questions, architecture, implementation, review, summary. Sounds like a lot, but it forces Claude to understand the existing codebase before writing a single line.

I don't use this for every feature. Small changes don't need this much structure. But for anything non-trivial, it keeps things organized.

Frontend Design

This one surprised me. I expected it to generate generic-looking UI. Instead, it reads your existing design system (your CSS, tokens, component patterns) and designs new UI that actually fits. It explicitly avoids the "AI slop aesthetic" (overused Inter font, purple gradients, cookie-cutter components).

I use it even when I already have a bunch of existing UI. It respects what's there and adjusts accordingly.

Security Guidance

Runs as a pre-tool hook. When Claude tries to write code with security issues (command injection, XSS, unsafe input handling), it blocks the edit and explains what's wrong. You fix it on the next turn.

It pairs really well with superpowers' code reviewer. Together they catch both security vulnerabilities and code quality issues.

Claude in Chrome (MCP)

This is the one that feels like magic. Claude controls a real Chrome browser to test your frontend.

Here's how I use it: I change some backend logic in a Rails app that affects the frontend. Claude launches Chrome, navigates to the page, and verifies the UI looks right. I keep working on something else.

It's not about catching extra bugs. It's about automation. For backend developers who touch frontend, this is a huge time saver.

What I Tried and Dropped

I've tried pretty much everything in the official store. LSP plugins, Stripe, code-simplifier, and plenty of others. Most didn't make the cut. Here are the notable ones.

Serena. Opens a browser window every single time you start Claude Code. There are multiple GitHub issues with people complaining about this. If you're building a developer tool, don't break the developer's flow on startup. That's DX 101.

Playwright. Redundant once I had Claude in Chrome. Same job, one less dependency.

Ralph Loop. This is the autonomous loop plugin. You give Claude a prompt and it runs in a loop until it declares "DONE." No human in the loop. I've written before about why I don't trust fully autonomous workflows. I want to manage the plan. I want to review the diffs. Ralph is for people who are comfortable letting Claude run wild. I'm not there yet.

What's Next

This post is just the what. I'll write a follow-up on the how (my actual daily workflow with these plugins). But if you're getting started with Claude Code plugins, this is the stack I'd recommend.

Run /plugin in Claude Code. Go to the Discover tab. Start with superpowers and context7. Add the rest as you need them.

And stick to the official store. Your credentials will thank you.

The Claude Code Plugins I Use Every Day

Mon, 09 Feb 2026 00:00:00 GMT

I use Claude Code every day. Here's my full plugin stack, nothing hidden.

I only install from the official Anthropic plugin store. No community marketplaces. No random GitHub repos. It took months of trial and error to get here.

Why Official Only

Two reasons: simplicity and security.

I don't think you need 20 fancy plugins to ship working software. The official store has enough to cover my entire workflow. That's it. That's the simplicity argument.

The security argument is scarier.

The barrier to publish a skill on community marketplaces? A markdown file and a GitHub account that's one week old. No code signing. No security review. No sandbox by default.

SentinelOne showed that marketplace skills can force Claude to install malicious dependencies without you knowing.

The official store is curated by Anthropic. Third-party plugins go through a review process before they're accepted. That's the trust baseline I'm comfortable with.

The Stack

Seven plugins and one MCP server. Here's what each one does and why it's in my stack.

Superpowers

This is the backbone. Created by Jesse Vincent, it's a skills framework that teaches Claude to actually think before it codes.

The skills I use most:

Brainstorming asks a ton of questions before you write any code. Depending on the task, it'll dig into requirements, edge cases, and alternatives you haven't considered.
Planning creates structured implementation plans that you review before execution. No surprises.
Code review catches issues before they ship. I pair this with the security-guidance plugin (more on that below).
Skill creation helps you write new skills for Claude Code. I've used it to build custom workflows for my blog writing process.

The whole philosophy is: stop and think before you code. It's the opposite of just letting Claude rip through your codebase unsupervised.

Context7

Code Review

PR Review Toolkit

The split is simple. Code Review for my code. PR Review Toolkit for everyone else's.

Feature Dev

I don't use this for every feature. Small changes don't need this much structure. But for anything non-trivial, it keeps things organized.

Frontend Design

I use it even when I already have a bunch of existing UI. It respects what's there and adjusts accordingly.

Security Guidance

It pairs really well with superpowers' code reviewer. Together they catch both security vulnerabilities and code quality issues.

Claude in Chrome (MCP)

This is the one that feels like magic. Claude controls a real Chrome browser to test your frontend.

It's not about catching extra bugs. It's about automation. For backend developers who touch frontend, this is a huge time saver.

What I Tried and Dropped

I've tried pretty much everything in the official store. LSP plugins, Stripe, code-simplifier, and plenty of others. Most didn't make the cut. Here are the notable ones.

Playwright. Redundant once I had Claude in Chrome. Same job, one less dependency.

What's Next

This post is just the what. I'll write a follow-up on the how (my actual daily workflow with these plugins). But if you're getting started with Claude Code plugins, this is the stack I'd recommend.

Run /plugin in Claude Code. Go to the Discover tab. Start with superpowers and context7. Add the rest as you need them.

And stick to the official store. Your credentials will thank you.

Hand Tools, Power Tools, and the AI Coding Debate

Sun, 08 Feb 2026 00:00:00 GMT

My friend Paul kept his woodworking tools in his living room. Japanese hand saws, chisels, marking gauges.

My workshop is a two-car garage. Table saw, miter saw, and a growing pile of battery-powered everything.

We both worked at Apple. We bonded over woodworking and electric unicycles. But we built things in completely different ways.

The Dovetail Moment

One day Paul showed me his hand-cut castle joints and dovetail joints. He was smiling, proud. The joints were precise, beautiful, fitted together perfectly. You could tell someone spent hours on each one.

Then I showed him what I'd tried with my table saw. It wasn't even close.

I learned that day that table saws aren't really suitable for dovetail joints. I should've used a router. But that's the thing. My first instinct was always: how do I make this as efficient as possible? Paul's was: how do I make this beautiful?

Paul at Apple

When I joined Apple, I needed to learn internal systems fast. Paul was my go-to person.

Walking from the Apple Park parking lot to my building took 25 to 30 minutes. Paul was already riding an electric unicycle. He helped me pick one out and lent me his knee pads while I learned.

When we hit a bug we couldn't figure out together, I asked him: what do you do when you hit a roadblock like this?

"We just read the code for longer until something comes."

That was Paul. No shortcuts. No rushing. Just patience and attention.

He was a good person. He passed away in 2022 at 45. Way too young.

The Debate Paul Missed

Paul died before ChatGPT, before Claude, before any of the AI coding tools that now dominate my workflow. I'll never know which side he'd have taken.

Would he have been a hand-tool coder, reading the code for longer, savoring the process? Or would he have treated AI as just another tool in his collection? I genuinely don't know.

But the debate he missed is happening everywhere right now. And it sounds a lot like hand tools versus power tools.

Abhinav Omprakash recently wrote about trying Claude Code three times and deleting it each time. Same pattern: depression, lethargy, loss of purpose. Each time he deleted it, he rediscovered the joy of coding.

His argument is that writing code is thinking. When you outsource that to an AI, you skip the internalization. You become a reviewer of code you don't deeply understand.

I get it.

I Used to Be a Craftsman

In 2016, I gave a talk at Melbourne Mobile Meetup about Software Craftsmanship. I quoted Paul Graham: programmers as the last remaining craftsmen. I talked about masters and journeymen and deliberate practice. I believed it.

Around the same time, I was working on an iOS project with ReactiveCocoa and RxSwift. I learned to chain publishers and subscribers, replacing dozens of delegate functions with elegant reactive code. It was genuinely empowering. I still remember that feeling of everything clicking into place.

That's the hand-tool moment. When the craft itself is the reward.

But I've changed my mind.

Code Is a Liability

Every line you write is a line you have to maintain, debug, and eventually delete or rewrite. The less code, the better. What matters isn't how the code looks or how elegantly it works internally. What matters is whether it solves the user's problem and improves their life.

If it doesn't, the beauty of the code is irrelevant.

Recently, our customer support manager Stacy found a bug. When you edited a user's role in the admin panel and saved, it blanked out their password. A mistake in the form submission.

I spawned a debugging agent on Slack. It traced the routes, checked the controller, found the issue. I spawned another agent to fix it. It created a PR. CI passed. Merged.

I never opened VSCode. I never hunted through files trying to find which form handles that page. The old way would've taken me significantly longer. This took minutes.

25% of the latest Y Combinator batch has codebases that are 95% AI-generated. These aren't amateurs. These are highly technical founders who could write it all by hand but chose not to.

That number is only going up.

The craftsman coders who insist on hand-writing everything will have a harder time keeping up. Just like hand-tool woodworkers. They'll make beautiful things. They'll enjoy the process. But when speed matters, they'll be outpaced by AI-enabled engineers.

The Table Saw Lesson

Every woodworker measures twice and cuts once. That's the craft, not the tool.

The difference is what happens after the planning.

With hand tools, you feel the saw bite into the wood, adjust your angle, correct mid-cut. The material talks back to you. That's what Paul meant by "read the code for longer until something comes." The process itself is the reward.

With a table saw, the cut happens in a second. If you're on the wrong side of the line by 1/8th of an inch, the piece doesn't fit. You can't undo it. People lose fingers to table saws. The tool doesn't care about your intentions.

Both require thinking. Both require skill. But hand tools give you something power tools don't: the craft experience of the work itself.

AI coding is the same trade-off. I don't throw prompts and hope for the best. I plan the approach, review the architecture, then let the agent execute. The thinking is the same. But I've traded the joy of writing every line for the speed of not having to.

Two Kinds of Joy

Here's why I made that trade.

The nightstand I built years ago still sits next to my bed. The wardrobe shelves I made hold my wife's clothes in our bedroom and my kid's toys in their room.

I don't remember what it felt like to cut those pieces. The joy of building them is long gone. But I walk past them every day, still working, still useful. That joy is still here.

There's the joy of making something. And there's the joy of seeing it used. For me, the second one lasts longer.

That's what I optimize for now. Not the craft of the code, but the moment a customer's life gets a little better because of something I shipped.

Don't Judge, But Introspect

If you love hand-cutting dovetail joints, keep doing it. If writing code by hand brings you joy and flow, that's real and valid.

But be honest with yourself about why.

Are you writing code for the craft? For the beauty of it? For the feeling of mastery? That's fine. Paul would've understood that completely.

Or are you writing code to solve a problem for someone who doesn't care how you solved it? Because if so, reach for the power tools.

Paul's hand-cut joints were beautiful. My table-saw cuts were fast. We'd look at each other's work, nod, and go ride our electric unicycles around Cupertino.

There was room for both of us then. There still is now. Just know which one you are, and why.

Rest in peace, Paul. You are sorely missed.

Curiosity Is the Only Skill AI Can't Replace

Tue, 03 Feb 2026 00:00:00 GMT

Anthropic just published a study showing their own product makes developers worse. But buried in the data is the opposite finding: developers who asked "how does this work?" learned just as well as those who coded by hand.

The difference wasn't AI. It was curiosity.

The Study

52 professional engineers learned a new Python library called Trio. Half used Claude, half coded by hand. Then they took a quiz.

"On a quiz that covered concepts they'd used just a few minutes before, participants in the AI group scored 17% lower than those who coded by hand, or the equivalent of nearly two letter grades."

That's the headline number. But the averages hide the real story.

What Actually Mattered

The researchers identified six distinct patterns of AI usage. Three killed learning. Three preserved it.

"The low-scoring patterns generally involved a heavy reliance on AI, either through code generation or debugging... They showed less independent thinking and more cognitive offloading."

The low scorers used AI like a slot machine. Type in a problem, get code out, move on. When it didn't work, they'd paste the error back in and wait for the fix. No understanding required.

The high scorers did something different:

"The participants who showed stronger mastery used AI assistance not just to produce code but to build comprehension while doing so—whether by asking follow-up questions, requesting explanations, or posing conceptual questions."

One group asked "what should I write?" The other asked "how does this work?"

The Risk and the Opportunity

The study found the biggest gap was in debugging:

"The largest gap in scores between the two groups was on debugging questions, suggesting that the ability to understand when code is incorrect and why it fails may be a particular area of concern."

That's the risk. Here's the opportunity.

I'm at Boon, building a preconstruction platform with React on Rails. I'd used Rails for small projects, but never at production scale. For the last nine years I'd been doing iOS (CoreData, direct SQL on device). Production Rails is a different beast: Active Record, migrations, the Rails console. All new territory.

With AI, I learned it in weeks instead of months. By asking it to explain everything.

When I needed to explore the production database through the Rails console (terrifying for someone who'd spent a decade on mobile), AI generated the commands and told me which ones were safe. I wasn't just executing. I was building a mental model.

The key: I always asked it to verify. "Look this up online and double-check." It often corrected itself. When I wasn't sure, I asked my Rails-expert colleague. AI accelerated my learning, but I stayed curious.

Understanding Compounds

Every time you ask "why does this work this way?" you're building something AI can't give you directly.

Architecture knowledge is just accumulated understanding of tradeoffs. Debugging skill is pattern recognition from past failures. Senior engineers use AI as a junior dev to supervise (they already have the instincts to know when the output is wrong).

You can't shortcut this. But you can accelerate it (if you stay curious).

After AI implements something for me, I ask it to explain what it just did. I ask it to write an informative PR message. I go back and forth until I understand. AI will never get bored with your questions. That's its superpower (if you use it).

The Junior Developer Problem

"Junior developers may rely on AI to complete tasks as fast as possible at the cost of skill development—and notably the ability to debug issues when something goes wrong."

If AI stunts skill formation, and juniors are the pipeline to seniors, we have a problem. Some companies now run "AI-free learning sessions" for new hires.

But I don't think the solution is restricting AI. The solution is training people HOW to use it.

The habit that matters: stay curious and take ownership. AI is a tool. Ask your seniors how they use it. Keep up with how it's evolving (I've gone from Copilot to Claude Code to MCP servers to custom skills in two years). Try new things. See what works. Share what you learn.

Every company needs someone to experiment so they can share their learning with everyone else. Be that person.

What This Means

Anthropic published research showing their own product has downsides.

But the deeper lesson: AI is a learning accelerator or a learning replacement, depending on how you use it. The study participants who stayed curious scored just as well as those who coded by hand. The ones who delegated everything learned nothing.

Understanding compounds. The future belongs to the curious.

(via Anthropic Research)

How I Made npm Impossible in Claude Code

Fri, 30 Jan 2026 00:00:00 GMT

Last week Claude ignored my CLAUDE.md instructions and switched to npm in a Bun project. My advice was "stay engaged" and "check the diffs."

That's still good advice. But I found a better solution: make it impossible.

What Are Hooks?

Claude Code has a feature called hooks. They're shell commands that run automatically at specific points in Claude's workflow.

PreToolUse is the one we care about. It runs before Claude executes any tool. If your hook script exits with code 2, the action is blocked. Claude sees your error message and has to find another approach.

Think of it as a bouncer. Claude says "I want to run npm install." The hook checks the command, sees "npm," and says "No. Try again."

You configure hooks in .claude/settings.json in your project.

Hook 1: Block npm Commands

This one-liner intercepts every Bash command and blocks anything containing npm, yarn, npx, or pnpm:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          {
            "type": "command",
            "command": "jq -r '.tool_input.command // \"\"' | grep -qE '\\b(npm|yarn|npx|pnpm)\\b' && echo 'BLOCKED: This project uses Bun. Use bun instead of npm/yarn/npx/pnpm.' >&2 && exit 2 || exit 0"
          }
        ]
      }
    ]
  }
}

How it works:

jq extracts the command from Claude's input
grep checks for forbidden package managers (word boundaries prevent false matches like "pnpmrc")
If found: print error to stderr, exit 2 (block)
If not: exit 0 (allow)

When Claude tries to run npm install, it sees:

BLOCKED: This project uses Bun. Use bun instead of npm/yarn/npx/pnpm.

Then it retries with bun install. No human intervention needed.

Hook 2: Block npm in Dockerfiles

The first hook catches terminal commands. But my original incident was Claude editing a Dockerfile to use npm run start, a Write operation, not a Bash command.

So we need a second hook that intercepts file writes:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Write|Edit",
        "hooks": [
          {
            "type": "command",
            "command": "$CLAUDE_PROJECT_DIR/.claude/hooks/block-npm-in-files.sh"
          }
        ]
      }
    ]
  }
}

The script (.claude/hooks/block-npm-in-files.sh):

#!/bin/bash
set -e

input=$(cat)
file_path=$(echo "$input" | jq -r '.tool_input.file_path // ""')
content=$(echo "$input" | jq -r '.tool_input.content // .tool_input.new_string // ""')

# Only check files where package manager references matter
case "$file_path" in
  *Dockerfile*|*docker-compose*.yml|*.github/workflows/*.yml)
    if echo "$content" | grep -qE '\b(npm|yarn|npx|pnpm)\b'; then
      echo "BLOCKED: You're writing npm/yarn references to $file_path." >&2
      echo "This project uses Bun. Find a Bun-native solution." >&2
      exit 2
    fi
    ;;
esac

exit 0

Make it executable with chmod +x .claude/hooks/block-npm-in-files.sh.

Now if Claude tries to write CMD ["npm", "run", "start"] to a Dockerfile, it gets blocked. It has to find a Bun-native solution (like CMD ["bun", "run", "start"]).

The Difference

CLAUDE.md is guidance. Claude can ignore it under pressure, especially during long debugging sessions where the immediate problem overshadows standing rules.

Hooks are guardrails. They're deterministic. Every time Claude tries to use npm, it gets blocked. No exceptions. No "I know you said Bun but this would be faster."

The pattern works for any project convention:

Block rm -rf commands
Require tests to pass before commits
Prevent modifications to sensitive files
Enforce directory structure

Check the official hooks documentation for more examples.

Does It Work?

I tested these hooks in a fresh Claude Code session. Eight test cases:

Test	Result
`npm install`	Blocked
`yarn add lodash`	Blocked
`npx create-react-app`	Blocked
`bun install`	Works
Read `.pnpmrc` file	No false positive
Write `npm start` to Dockerfile	Blocked
Write `bun start` to Dockerfile	Works
Write `console.log('npm')` to `.js` file	Works (not a config file)

All eight passed. The hooks block what they should block and allow what they should allow.

You can find the test setup on GitHub if you want to try it yourself.

I'm still watching Claude work. But now I'm not watching for npm. That problem is solved.

Gemini Hallucinates Less When You Give It Python

Tue, 27 Jan 2026 00:00:00 GMT

When Google announced "agentic vision" for Gemini 3 Flash (the model can run Python code to analyze images), I had to test it. I ran 6 different tests. The results surprised me.

Summary: Code OFF vs Code ON

Test	Code OFF	Code ON	Verdict
Rebar counting	"60 rebar"	"0 rebar" (correct: image shows wood)	✅ Hallucination prevented
Pipe counting	"5 pipes"	"0 pipes" (looks like cables/supports)	✅ Hallucination prevented
Floor plan (trees)	10 trees (correct)	12 trees (wrong: zooms show 7+3=10)	❌ Overcounted
Floor plan (doors)	21 doors	18 doors + room-by-room list	✅ More detailed
Lighting fittings	17 new, 12 original	17 new, 12 original + per-room breakdown	✅ Same count, better structure
PPE detection	Lists items per worker	Lists items with specific colors/details	✅ More specific
Dashboard extraction	Extracts values	Extracts values + axis scales	✅ More complete

Key findings:

Hallucination prevention is the biggest win (60→0 on rebar, 5→0 on pipes)
Code execution makes the model more conservative (says "none" when unsure)
Not perfect: palm trees went from correct (10) to wrong (12), even with zoomed evidence showing 7+3=10
Every answer comes with visual evidence (38 zoomed crops across 6 tests)
Outputs are structured by region (Worker 1 vs 2, room-by-room, chart-by-chart)

What I Tested

I ran the same prompts with code execution OFF and ON. The difference isn't a parameter (it's a tool):

from google import genai
from google.genai import types

client = genai.Client(api_key=os.getenv("GOOGLE_API_KEY"))

# Code execution enabled
config = types.GenerateContentConfig(
    tools=[types.Tool(code_execution=types.ToolCodeExecution())]
)

response = client.models.generate_content(
    model="gemini-3-flash-preview",
    contents=[image, prompt],
    config=config,
)

The response includes executable_code and inline_data (generated images) when the model chooses to analyze visually.

Test 1: Rebar Counting (Hallucination Prevention)

This was the most dramatic result. I sent a construction site image with the prompt "count the rebar."

Code OFF:

{
  "text": "You can see 60 rebar (metal reinforcement bars) in this picture."
}

Code ON:

{
  "text": "Based on a careful examination of the image, including close-ups of the background and the worker's table, there are no rebar (metal reinforcement bars typically used for concrete) visible. The image depicts a construction worker working with large wooden beams and various tools (circular saw, drill, clamps, etc.) inside what appears to be a structural steel building. The visible metal elements are structural steel beams, bracing rods, and tools, rather than rebar.\n\nFinal count: 0 rebar.",
  "images_generated_count": 7
}

The image showed a woodworker with wooden beams. Not rebar. Without code execution, Gemini confidently hallucinated "60 rebar." With code execution, it zoomed into 7 regions, examined them, and correctly identified wood.

The code Gemini executed:

import PIL.Image
import PIL.ImageDraw

img = PIL.Image.open('input_file_0.jpeg')
width, height = img.size

# Looking for rebar - often found in concrete construction, but this looks like wood.
# Let's zoom into various parts to be sure.
regions = [
    [0, 0, 1000, 500],   # Top half (ceiling/background)
    [500, 0, 1000, 1000], # Bottom half (floor/bench)
    [0, 0, 500, 500],    # Top left
    [0, 500, 500, 1000],  # Top right
]

# Convert normalized to pixel coordinates
def norm_to_pixel(norm_coords, w, h):
    return [norm_coords[0] * h / 1000, norm_coords[1] * w / 1000,
            norm_coords[2] * h / 1000, norm_coords[3] * w / 1000]

for i, reg in enumerate(regions):
    p_reg = norm_to_pixel(reg, width, height)
    crop = img.crop((p_reg[1], p_reg[0], p_reg[3], p_reg[2]))
    crop.save(f'zoom_{i}.jpeg')

The model divides the image into regions, crops each one, and examines them separately. It's essentially giving itself a closer look at different parts of the image before committing to an answer.

In a second code block, it zooms into the background specifically:

# Zoom into the background more clearly
regions_bg = [
    [350, 600, 600, 900], # Right background
    [350, 300, 600, 600], # Center background
    [350, 0, 600, 300],   # Left background
]

for i, reg in enumerate(regions_bg):
    p_reg = norm_to_pixel(reg, width, height)
    crop = img.crop((p_reg[1], p_reg[0], p_reg[3], p_reg[2]))
    crop.save(f'bg_zoom_{i}.jpeg')

The model's zoomed regions:

Test 2: Pipe Counting (Conservative Mode)

Another case where code execution made the model more conservative.

Code OFF:

{
  "text": "This image contains five pipes."
}

Code ON:

{
  "text": "Based on a careful examination of the image, there are no clearly identifiable pipes present. The background contains several blurred structural elements, such as vertical rods and curved loops, which appear more like cables or solid metal supports than pipes. The welder is using a stick welding electrode, which is a solid rod, and is working on what looks like a flat or rectangular piece of metal.\n\nFinal Answer: 0",
  "images_generated_count": 3
}

The model questioned whether the objects were actually pipes. It decided they looked more like cables or supports.

The code Gemini executed:

import PIL.Image
import PIL.ImageDraw

img = PIL.Image.open('input_file_0.jpeg')
width, height = img.size

# Define potential pipe areas
objs = [
    {'box_2d': [310, 560, 680, 640], 'label': 'potential pipe 1'},
    {'box_2d': [0, 640, 420, 950], 'label': 'potential pipe background right'},
    {'box_2d': [480, 500, 780, 750], 'label': 'welding rod (pipe?)'},
]

# Create crops to see better
for i, obj in enumerate(objs):
    ymin, xmin, ymax, xmax = obj['box_2d']
    crop = img.crop((xmin * width / 1000, ymin * height / 1000,
                     xmax * width / 1000, ymax * height / 1000))
    crop.save(f'crop_{i}.png')

The model identifies potential pipe locations, crops them out, examines each one, then decides "these look more like cables." This could be seen as overly cautious, but for construction QC, conservative beats confident-but-wrong.

The model's zoomed regions:

Test 3: Floor Plan Counting (Trees & Doors)

A floor plan with pool, garage, and multiple rooms.

Code OFF:

{
  "text": "There are exactly 10 trees and 21 doors shown here."
}

Code ON:

{
  "text": "Based on the provided floor plan, here are the counts for trees and doors:\n\n### Trees: 12\nThere are 12 palm trees visible around the exterior of the property:\n- 8 palm trees are lined up along the top edge of the floor plan.\n- 4 palm trees are located in a row at the bottom right corner near the patio.\n- (Note: There are several other smaller green plants or bushes located on the terraces, in the gym, and in the bathroom, but these are generally represented as potted plants or small bushes rather than trees.)\n\n### Doors: 18\nThere are 18 doors (including door leaves and sliding panels) identified by their typical architectural symbols (arcs for swinging doors and parallel lines for sliding doors):\n1. 3 doors in the top-left exterior walkway area (one from the driveway, one to the terrace, and one from the laundry room).\n2. 1 door between the laundry room and the interior hallway.\n3. 1 door from the interior hallway into the gym.\n4. 1 door from the interior hallway to the terrace (located between the living room and the gym).\n5. 2 doors (a double door) from the living room to the terrace.\n6. 1 door from the main hall to the terrace.\n7. 1 door from the main hall leading to the outside on the right.\n8. 1 door from the main hall into the wine cellar.\n9. 1 door from the main hall into the guest bathroom.\n10. 1 door from the main hall into the guest bedroom.\n11. 1 door (sliding glass door) from the guest bedroom to the patio.\n12. 1 door from the gym to the garage.\n13. 1 door from the gym to the bathroom.\n14. 1 door from the gym to the terrace.\n15. 1 door from the bathroom to the terrace.\n16. 1 door from the driveway into the corridor leading to the terrace.\n\nTotal: 17 swinging door leaves (arcs) + 1 sliding door = 18 doors in total.",
  "images_generated_count": 8
}

Code execution gave a room-by-room breakdown, but got the tree count wrong. The zoomed images clearly show 7 trees on top and 3 on the bottom right (total: 10), yet the model reported 8+4=12. Code OFF was actually correct here. The model does distinguish between trees and potted plants, and between swinging doors (arcs) and sliding doors (parallel lines).

The code Gemini executed:

import PIL.Image
import PIL.ImageDraw

img = PIL.Image.open('input_file_0.jpeg')
width, height = img.size

# Detect trees and doors.
objs = [
    {'box_2d': [0, 140, 180, 520], 'label': 'top_trees'},
    {'box_2d': [740, 800, 995, 965], 'label': 'bottom_right_trees'},
    {'box_2d': [435, 212, 555, 330], 'label': 'left_doors'},
    {'box_2d': [230, 395, 385, 480], 'label': 'laundry_living_doors'},
    {'box_2d': [280, 750, 485, 955], 'label': 'hall_doors'},
    {'box_2d': [540, 240, 750, 455], 'label': 'gym_bath_doors'},
    {'box_2d': [380, 595, 450, 770], 'label': 'top_terrace_doors'},
    {'box_2d': [550, 765, 700, 900], 'label': 'guest_bedroom_doors'}
]

def save_crops(img, objs, prefix):
    for i, obj in enumerate(objs):
        ymin, xmin, ymax, xmax = obj['box_2d']
        left = xmin * width / 1000
        top = ymin * height / 1000
        right = xmax * width / 1000
        bottom = ymax * height / 1000
        crop = img.crop((left, top, right, bottom))
        crop.save(f'{prefix}_{i}.png')

save_crops(img, objs, 'crop')

The model creates labeled regions (top_trees, hall_doors, gym_bath_doors, etc.) and crops each one. This is how it can give a room-by-room breakdown in its response.

The model's zoomed regions:

Test 4: Lighting Fittings Count

A lighting plan (blue = new fittings, red = original).

Code OFF:

{
  "text": "Based on the lighting plan provided:\n\n- New / Extra Fittings (blue squares and rectangles): 17\n- Original Fittings (red circles): 12"
}

Code ON:

{
  "text": "Based on the lighting plan provided, here is the exact count for each category of fittings:\n\n### New / Extra Fittings (Blue squares and rectangles): 17\n- Entrance (top center): 2 squares (one on each side of the door).\n- Top-left room (dining): 2 squares (one in the top-left corner and one in the bottom-right area).\n- Top-right room (living): 5 fittings total (1 square in the top-left, 2 long horizontal rectangles near the sofa, 1 square below the lower rectangle, and 1 square near the TV).\n- Kitchen (bottom-left): 4 squares (1 on the top counter, 1 on the left counter, and 2 on the island).\n- Central hallway: 1 long vertical rectangle located between the kitchen and the bottom-right dining area.\n- Bottom-right room (dining): 3 squares (1 at the top entrance, 1 in the center of the table, and 1 on the right wall).\n\n### Original Fittings (Red circles): 12\n- Top-left room (dining): 3 circles (one near the top wall, one in the center of the round table, and one in the bottom-left corner).\n- Top-right room (living): 3 circles (one in the top-right corner, one large circle in the center, and one in the bottom-left area above the partition wall).\n- Kitchen (bottom-left): 4 circles (one in the top-left corner, one in the middle of the floor, one in the bottom-left corner, and one near the bottom of the island).\n- Bottom-right room (dining): 2 circles (one in the top-right corner and one in the bottom-right corner).\n\nSummary:\n- New / Extra Fittings: 17\n- Original Fittings: 12",
  "images_generated_count": 4
}

Same totals (17 new, 12 original). But Code OFF just says "17." Code ON says "Kitchen: 4 squares (1 on the top counter, 1 on the left counter, and 2 on the island)." For construction audits, that's the difference between a number and a checklist.

The code Gemini executed:

import PIL.Image
import PIL.ImageDraw

img = PIL.Image.open('input_file_0.jpeg')
width, height = img.size

# Detect objects manually for counting and zoom
# Red Circles (Original Fittings)
# Blue Squares/Rectangles (New / Extra Fittings)

objs = [
    # Red circles - with exact pixel coordinates
    {'box_2d': [210, 345, 265, 385], 'label': 'original_red'}, # TL room top
    {'box_2d': [435, 240, 485, 285], 'label': 'original_red'}, # TL room bottom left
    {'box_2d': [160, 730, 215, 770], 'label': 'original_red'}, # TR room top right
    # ... (11 red circles total)

    # Blue shapes - with exact pixel coordinates
    {'box_2d': [100, 350, 160, 395], 'label': 'new_blue_sq'}, # Entrance left
    {'box_2d': [100, 435, 160, 480], 'label': 'new_blue_sq'}, # Entrance right
    {'box_2d': [195, 550, 235, 705], 'label': 'new_blue_rect'}, # TR room upper rect
    # ... (17 blue shapes total)
]

# Create crops by quadrant to double check
img.crop((0, 0, width//2, height//2)).save('top_left.jpeg')
img.crop((width//2, 0, width, height//2)).save('top_right.jpeg')
img.crop((0, height//2, width//2, height)).save('bottom_left.jpeg')
img.crop((width//2, height//2, width, height)).save('bottom_right.jpeg')

The model creates a detailed inventory of every fitting with pixel coordinates, distinguishing between squares, rectangles, and circles. Then it crops the image into quadrants to verify its counts.

The model's zoomed regions:

Test 5: PPE Detection (Color Accuracy)

Two construction workers with various safety equipment.

Code OFF:

{
  "text": "Based on the image, the Personal Protective Equipment (PPE) visible on the workers includes:\n\nFor the worker kneeling in the background:\n- Hard Hat: An orange hard hat for head protection.\n- High-Visibility Vest: An orange safety vest with reflective strips to ensure visibility.\n- Work Gloves: Dark gloves to protect his hands while handling tools and materials.\n- Work Boots: Sturdy, dark-colored boots suitable for a construction environment.\n\nFor the worker standing in the foreground:\n- Face/Head Covering: A light blue cloth or balaclava covering the face and head, likely for protection against sun, dust, or wind.\n- High-Visibility Vest: An orange safety vest with reflective elements.\n- Work Gloves: Dark protective gloves on his hands.\n- Safety Boots: Tall, dark-colored waterproof or safety boots."
}

Code ON:

{
  "text": "Based on the image, the Personal Protective Equipment (PPE) items visible on the construction workers include:\n\n### Worker 1 (Kneeling):\n- Hard Hat: An orange safety helmet.\n- High-Visibility Vest: An orange safety vest featuring yellow/silver reflective strips.\n- Safety Boots: Dark, heavy-duty work boots.\n- Head/Neck Covering: A white cloth or scarf worn under the hard hat, likely for sun protection and sweat management.\n\n### Worker 2 (Standing):\n- High-Visibility Vest: An orange safety vest with yellow/silver reflective strips.\n- Safety Gloves: Blue and black protective gloves.\n- Safety Boots: Dark, tall work boots (possibly rubber or waterproof).\n- Face Mask/Neck Gaiter: A light blue fabric covering the face and neck.\n- Face Shield / Welding Mask: A dark protective shield or mask worn on the head, currently flipped up.\n\nBoth workers are also wearing full-length clothing (long sleeves and pants), which provides additional protection against abrasions and environmental factors on a construction site.",
  "images_generated_count": 2
}

Code ON added specific colors (blue and black gloves, yellow/silver reflective strips) and noticed details like the face shield flipped up on Worker 2's head.

The code Gemini executed:

import PIL.Image
import PIL.ImageDraw

img = PIL.Image.open('input_file_0.jpeg')
width, height = img.size

# List workers and their PPE items to zoom in on
workers_and_ppe = [
    {'box_2d': [113, 471, 424, 603], 'label': 'worker_1'},
    {'box_2d': [388, 452, 861, 561], 'label': 'worker_2'},
    {'box_2d': [114, 537, 185, 586], 'label': 'hard_hat_1'},
    {'box_2d': [179, 482, 332, 592], 'label': 'hi_vis_vest_1'},
    {'box_2d': [484, 477, 663, 558], 'label': 'hi_vis_vest_2'},
    {'box_2d': [547, 451, 603, 487], 'label': 'gloves_2'},
    {'box_2d': [416, 492, 513, 545], 'label': 'face_mask_2'},
    {'box_2d': [779, 487, 865, 545], 'label': 'boots_2'}
]

def save_crop(img, box, name):
    ymin, xmin, ymax, xmax = box
    left = xmin * width / 1000
    top = ymin * height / 1000
    right = xmax * width / 1000
    bottom = ymax * height / 1000
    crop = img.crop((left, top, right, bottom))
    crop.save(name)

# Crop workers for better visibility
save_crop(img, [100, 450, 450, 650], 'worker1_zoom.png')
save_crop(img, [350, 430, 880, 580], 'worker2_zoom.png')

The model identifies specific PPE items (hard_hat_1, gloves_2, face_mask_2) with bounding boxes, then crops each worker separately for closer inspection. This is why Code ON can report specific colors and details.

The model's zoomed regions:

Test 6: Dashboard Data Extraction (Axis Reading)

A web analytics dashboard with dense data.

Code OFF:

{
  "text": "Based on the dashboard visible on the laptop screen, here are the extracted labels and data values:\n\n### Top Header\n- Title: USERS: LAST 7 DAYS USING MEDIAN\n\n### Top-Left Chart: LOAD TIME VS BOUNCE RATE\n- Metrics/Legend: Page Load (LUX) (Blue histogram), Bounce Rate (Pink line)\n- Annotated Data Point: Median Page Load (LUX): 2.056s, Bounce Rate 7s 57.1%\n- Y-Axis (Left - Counts): 0, 15K, 30K, 45K, 50K, 75K\n- Y-Axis (Right - Percentage): 0%, 20%, 40%, 60%, 80%, 100%\n- X-Axis (Seconds): 0, 2.5, 5, 7.5, 10, 12.5, 15, 17.5\n\n### Top-Right Chart: START RENDER VS BOUNCE RATE\n- Median Start Render (LUX): 1.031s\n- Y-Axis (Left - Counts): 0, 8K, 16K, 24K, 32K, 40K\n\n### Bottom-Left Section: PAGE VIEWS VS ONLOAD\n- Page Load (LUX): 0.7s\n- Page Views (LUX): 2.7Mpvs\n- Bounce Rate (LUX): 40.6%\n\n### Bottom-Right Section: SESSIONS\n- Sessions (LUX): 479K\n- Session Length (LUX): 17min\n- PVs Per Session (LUX): 2pvs\n\n### Additional Interface Elements\n- Options: Every quadrant features an OPTIONS link.\n- Notification: Chat/help icon with red notification bubble showing 1."
}

Code ON:

{
  "text": "The dashboard titled USERS: LAST 7 DAYS USING MEDIAN displays the following key data points:\n\n### 1. LOAD TIME VS BOUNCE RATE\n- Median Page Load (LUX): 2.056s\n- Specific Callout: Bounce Rate at 7s is 57.1%.\n- Axes & Legend:\n  - X-axis (Time): 0 to 17.5 seconds (increments of 2.5s).\n  - Y-axis (Left - Count): 0, 15K, 30K, 45K, 60K, 75K.\n  - Y-axis (Right - Bounce Rate): 0% to 100% (increments of 20%).\n  - Legend: Page Load (LUX) is represented by blue bars; Bounce Rate is represented by a pink line.\n\n### 2. START RENDER VS BOUNCE RATE\n- Median Start Render (LUX): 1.031s\n- Axes & Legend:\n  - X-axis (Time): 0 to 5 seconds.\n  - Y-axis (Left - Count): 0, 8K, 16K, 24K, 32K, 40K.\n  - Y-axis (Right - Bounce Rate): 0% to 100% (increments of 20%).\n\n### 3. PAGE VIEWS VS ONLOAD\n- Page Load (LUX): 0.7s\n- Page Views (LUX): 2.7Mpvs\n- Bounce Rate (LUX): 40.6%\n- Chart Metrics:\n  - Left Y-axis (Load Time): 0.4s, 0.6s, 0.8s, 1s.\n  - Inner Right Y-axis (Page Views): 200K, 300K, 400K, 500K.\n  - Outer Right Y-axis (Bounce Rate): 40%, 60%, 80%, 100%.\n\n### 4. SESSIONS\n- Sessions (LUX): 479K\n- Session Length (LUX): 17min\n- PVs Per Session (LUX): 2pvs\n- Chart Metrics:\n  - Left Y-axis (PVs): 1.6 pvs, 2.4 pvs, 3.2 pvs, 4 pvs.\n  - Inner Right Y-axis (Sessions): 40K, 60K, 80K, 100K.\n  - Outer Right Y-axis (Session Length): 24 min, 32 min, 40 min.",
  "images_generated_count": 5
}

Both modes extracted detailed data. Code ON organized it more systematically and captured dual Y-axis scales.

The code Gemini executed:

import PIL.Image
import PIL.ImageDraw

img = PIL.Image.open('input_file_0.jpeg')
width, height = img.size

# List of regions to crop for better visibility of data points and labels
crops = [
    [150, 80, 260, 450],   # Dashboard title and top-left header
    [260, 100, 600, 480],  # Top-left chart (LOAD TIME VS BOUNCE RATE)
    [260, 500, 600, 880],  # Top-right chart (START RENDER VS BOUNCE RATE)
    [650, 100, 960, 480],  # Bottom-left section (PAGE VIEWS VS ONLOAD)
    [650, 500, 960, 880]   # Bottom-right section (SESSIONS)
]

def get_crop(img, box_norm):
    ymin, xmin, ymax, xmax = box_norm
    left = xmin * width / 1000
    top = ymin * height / 1000
    right = xmax * width / 1000
    bottom = ymax * height / 1000
    return img.crop((left, top, right, bottom))

for i, crop_box in enumerate(crops):
    crop_img = get_crop(img, crop_box)
    crop_img.save(f'crop_{i}.png')

The model divides the dashboard into logical sections (title, each of 4 chart areas) and crops each one. This allows it to read small text and axis labels more accurately.

The model's zoomed regions:

What This Means

From 6 tests, here's what I learned:

Hallucination prevention is the real win. Getting 60 vs 0 wrong (rebar) or 5 vs 0 (pipes) is a bigger problem than miscounting by 2. But code execution isn't magic: it got the palm tree count wrong (10→12) even while looking at zoomed images that clearly showed 10.
Code execution makes the model more conservative. In 2 of 6 tests, it gave lower counts or "none found" when it looked closely. For high-stakes inspection, conservative beats confident-but-wrong.
Per-region breakdown is auditable. When the model says "12 trees," it shows you which 8 are on the top edge and which 4 are in the bottom right corner.
Structured output is more useful. Worker 1 vs Worker 2, Room-by-room counts, Chart-by-chart extraction.
The code itself is informative. You can see exactly what regions the model examined and what it labeled them. The box_2d coordinates create an audit trail.

This is significant. The model now shows its work.

The code is on GitHub if you want to run your own tests.

Appendix: Complete API Response

Here's what the full API response looks like (base64 image data truncated for readability):

{
  "success": true,
  "text": "Based on a careful examination of the image, including close-ups of the background and the worker's table, there are no **rebar** (metal reinforcement bars typically used for concrete) visible. The image depicts a construction worker working with large wooden beams and various tools (circular saw, drill, clamps, etc.) inside what appears to be a structural steel building. The visible metal elements are structural steel beams, bracing rods, and tools, rather than rebar.\n\nFinal count: 0 rebar.",
  "code_executed": [
    "import PIL.Image\nimport PIL.ImageDraw\n\n# Load the image to get dimensions\nimg = PIL.Image.open('input_file_0.jpeg')\nwidth, height = img.size\n\n# Looking for rebar - often found in concrete construction, but this looks like wood.\n# Let's zoom into various parts to be sure.\nregions = [\n    [0, 0, 1000, 500],   # Top half (ceiling/background)\n    [500, 0, 1000, 1000], # Bottom half (floor/bench)\n    [0, 0, 500, 500],    # Top left\n    [0, 500, 500, 1000],  # Top right\n]\n\n# Convert normalized to pixel coordinates\ndef norm_to_pixel(norm_coords, w, h):\n    return [norm_coords[0] * h / 1000, norm_coords[1] * w / 1000, norm_coords[2] * h / 1000, norm_coords[3] * w / 1000]\n\nfor i, reg in enumerate(regions):\n    p_reg = norm_to_pixel(reg, width, height)\n    crop = img.crop((p_reg[1], p_reg[0], p_reg[3], p_reg[2]))\n    crop.save(f'zoom_{i}.jpeg')\n",
    "# Zoom into the background more clearly\nregions_bg = [\n    [350, 600, 600, 900], # Right background\n    [350, 300, 600, 600], # Center background\n    [350, 0, 600, 300],   # Left background\n]\n\nfor i, reg in enumerate(regions_bg):\n    p_reg = norm_to_pixel(reg, width, height)\n    crop = img.crop((p_reg[1], p_reg[0], p_reg[3], p_reg[2]))\n    crop.save(f'bg_zoom_{i}.jpeg')\n"
  ],
  "images_generated": [
    {
      "mime_type": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD... <truncated, 24532 chars>"
    },
    {
      "mime_type": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD... <truncated, 21176 chars>"
    },
    {
      "mime_type": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD... <truncated, 44788 chars>"
    },
    {
      "mime_type": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD... <truncated, 46680 chars>"
    },
    {
      "mime_type": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD... <truncated, 7760 chars>"
    },
    {
      "mime_type": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD... <truncated, 7332 chars>"
    },
    {
      "mime_type": "image/jpeg",
      "data": "/9j/4AAQSkZJRgABAQAAAQABAAD... <truncated, 8192 chars>"
    }
  ]
}

The code_executed array contains the Python scripts the model wrote and ran. The images_generated array contains the base64-encoded JPEGs of each cropped region (7 images totaling ~160KB).

The CLAUDE.md Said 'Never Use npm.' Claude Used npm Anyway.

Mon, 26 Jan 2026 00:00:00 GMT

I watched Claude Code switch to npm in my Dockerfile. The CLAUDE.md file said "Always use Bun instead of npm or yarn." Claude did it anyway.

This isn't a hit piece. I use Claude daily. It's production-ready and writes good code. But this incident is a perfect example of why you can't just let AI coding tools rip unsupervised.

What Happened

I'm working on a Bun-native project for fun. The Docker container wouldn't start. Claude diagnosed the issue: Bun doesn't fully support renderToPipeableStream, a Node.js-specific API.

Then it proposed a "solution":

⏺ There's a Bun runtime compatibility issue with React's server-side rendering.
   Bun doesn't fully support renderToPipeableStream.
   The solution is to use Bun for fast installs/builds but Node.js for
   the production runtime. Let me fix this.

⏺ Write(Dockerfile)
  ⎿  Added 6 lines, removed 5 lines
      23 +# Stage 3: Production image with Node.js
      24 +# (React Router SSR requires Node.js stream APIs not fully supported by Bun)
      25 +FROM node:20-alpine AS release
      ...
      31 +CMD ["npm", "run", "start"]

Claude swapped my Bun production image for Node.js. Changed bun run start to npm run start. The CLAUDE.md file explicitly said never do this. It did it anyway.

The Catch

I caught it immediately. Hit Escape before it could rebuild the image.

❯ NO! Never use npm, you need to remember this update Claude.md.
  Never, never, ever, ever use npm. If you encounter any issue,
  you need to fix it with Bun. Do a research into how to fix this with Bun.

This forced Claude to research instead of taking the shortcut.

The Correct Fix

After researching, Claude found the real solution: use renderToReadableStream instead of renderToPipeableStream. The first is a web standard. The second is Node-specific.

⏺ Write(app/entry.server.tsx)
  ⎿  Wrote 57 lines to app/entry.server.tsx
      import { renderToReadableStream } from "react-dom/server";

The Dockerfile stayed pure Bun. No Node fallback needed.

Claude's quick fix would have worked. The app would have run. But it was the wrong solution. The correct fix required research, and I had to push for it.

The Lucky Part

I don't always watch Claude work. Sometimes I kick off a task and check back later. This time I happened to be watching.

If I hadn't been? That npm Dockerfile ships to production.

The cost of that mistake: Bun installs are 20-40x faster than npm. It handles ~100k requests/second versus Node's ~25-30k. Teams report 35% lower serverless costs.

I chose Bun for a reason. Claude almost tossed those benefits to solve a problem faster.

Why This Happens

Classic context rot. The instruction "always use Bun" sits at the top of CLAUDE.md. But then there's a long troubleshooting session with errors and logs and attempts. The immediate problem (make the Docker container work) becomes more salient than the standing rule (use Bun, not npm).

Plus, npm is everywhere in Claude's training data. When it hits a wall, it defaults to what's familiar. Bun is newer. The muscle memory isn't there yet.

This is why fully autonomous workflows like Ralph Wiggum make me skeptical. It's a technique where Claude loops until it declares "DONE." No human in the loop. They work great until they don't. And when they don't, the wrong solution ships.

The Takeaway

Claude Code is ready for production work. I'm not arguing otherwise. But the human in the loop isn't optional yet.

When I caught this mistake, three things happened:

I interrupted before damage was done
Claude researched and found the correct solution
We updated CLAUDE.md with stronger language

That feedback loop matters. Over time, you're training your AI collaborator. The instructions get more specific. The guardrails get tighter. Claude learns your project's rules.

But it only works if you're watching.

If you're adopting AI coding tools, my advice: stay engaged. You don't have to watch every keystroke. But check the diffs before they ship. Read what it changed. When it takes a shortcut you don't like, stop it and make it find the right solution.

The AI is good. Really good. But "good enough to ship without review" isn't the same as "good enough to trust completely."

Estimation Isn't Impossible, Your Org Is Broken

Sat, 24 Jan 2026 00:00:00 GMT

Sean Goedecke's post on how he estimates work went viral on Hacker News. I disagree with almost every premise it rests on.

"Software engineering projects are not dominated by the known work, but by the unknown work, which always takes 90% of the time."

This isn't universally true. Every engineer knows what known work looks like. CRUD apps. Loading values from a database and displaying them in a table. Adding sorting and animations. Given a design, you can estimate these reliably.

Novel work is different. Building a computer vision model that no one in the world has built before? That's going to have huge error bars. Maybe you can't estimate it at all.

The conversation isn't about whether estimation is possible. It's about which category you're in and communicating that honestly.

"Estimates are political tools for non-engineers in the organization."

I don't think calling it "political" is helpful. It makes everything sound adversarial.

Product managers, sales teams, marketing teams need to know when they can promise a feature to users. They need to plan next quarter's work. They're relying on us engineers to set their expectations.

When I transitioned into engineering leadership, I started seeing the other side more clearly. As notjustanymike put it in the Hacker News discussion:

"After owning a product, I've developed a lot of sympathy for the people outside of engineering who have to put up with us... In a functioning org, there are lot of professionals depending on correct estimation to do their job."

Of course business and product people want everything as fast as possible. That's their job. They're trying to sell it and make money for the company. They're doing exactly what they should be doing.

They're also operating in a high-ambiguity environment. They have their own stakeholders, their own deadlines, their own promises to keep.

That's not politics. That's planning. Engineering is one of the most expensive investments a company makes. Estimation helps plan that investment.

"My job is to figure out the set of software approaches that match that estimate."

This is backwards. Requirements define work, not the other way around.

You understand the PRD. You get example inputs and outputs from the product team. You figure out whether this requires semantic search, highlighting, robust sourcing, or a mix of local and internet search. Then you estimate.

I always ask the product team: give me an example of input and output. By looking at that, I know what system I need to build.

"The estimate doesn't come from the engineers. It comes from management."

This makes no sense. Estimates come from engineers. They're just not for engineers.

But a lot of engineers don't feel empowered to push back. They don't feel they can say "there's no way we can deliver what you want in the time you want." They think they just have to accept the timeline and figure it out.

That's wrong. Our job is to communicate our limitations. We can't break the laws of physics. If you want it faster, something has to give. Quality, scope, or both.

When a stakeholder asks "Why will this take a month when that similar thing only took a week?" that's not politics. That's an opportunity for education.

You break it down: "For this to work by that date, we need a certain number of annotations, a certain number of GPUs, and we need to get lucky that the architecture we pick works on the first try. If it doesn't, we try another architecture. We might need to re-annotate the training data."

Now they understand. They can adjust their communication to customers accordingly.

One useful framing comes from moribvndvs in the same Hacker News thread. Offer "trim lines" like a car:

"I'll put together 3 'trim line' options:

Economy - bare functionality, fast delivery over reliability

Mid tier - good enough quality, reliable, no frills

Luxury - all the bells and whistles"

That's a conversation about trade-offs. You're working together to refine the requirements until they fit the timeline, or adjust the timeline to fit the requirements.

"Step 1: Extract the political context... What does management need? What kind of pressure are they under?"

We're on the same team. Imagine a basketball team where the point guard doesn't understand what the center is doing. That makes no sense.

Engineers need to learn what the product manager needs. What sales needs. What the CEO needs. And those stakeholders need to learn how to think about engineering work. You can't stay in silos.

I've worked at Apple and at a startup that had 7 engineers when I joined and now has 30. The estimation process is almost identical. A program manager (or at a startup, the CEO or PM lead) comes to the engineering lead and asks how long something will take. The lead works with the engineers to estimate. Priorities get tracked.

The only difference is startups don't have program managers. The fundamentals are the same.

The heuristics work well enough

Once you're having real conversations, the actual mechanics of estimation are straightforward.

We use the "multiply by two, add 20%" rule because we don't know what we don't know. It's a rough correction for optimism bias.

As you get more senior, you calibrate based on who's doing the work. "I can trust this engineer, she's really fast." "This engineer is thorough but slower." You factor that in.

None of this is political. It's just experience.

Conclusion

The article resonated with a lot of people, which tells me a lot of engineers work in organizations where estimation does feel adversarial. If that's your situation, I'm sorry.

But that's not estimation being impossible. That's communication being broken.

If we assume good will from everyone, this is a solvable problem. Business people need to understand that engineering is trying their best. Engineers need to understand that business people aren't the enemy.

Of course, if you have bad actors and assholes, that's another story entirely. But that's a people problem, not an estimation problem.

Estimation is hard. Not impossible. Talk to each other.

Turn On Your Camera

Sat, 06 Dec 2025 00:00:00 GMT

I've been working remotely for years now, and there's one thing I keep noticing: fewer and fewer people turn on their cameras during video calls. And I get it. Zoom fatigue is real. But here's the thing: the benefits of having your camera on far outweigh the costs.

This isn't about looking polished or having a nice background. It's about building trust, staying accountable, and actually connecting with your teammates in an environment where we're already operating at lower bandwidth than in-person work.

Let me break this down.

The Benefits for You

When you turn on your camera, people remember you. It's just human nature.

Think about a meeting with 10 people where only 5 have their cameras on. Your eyes naturally go to those 5 faces. You build a mental image of who they are. The other 5? They're technically there, but you don't feel their presence the same way.

If you're a junior employee trying to make an impression, this matters a lot. Leaders notice who shows up. Not just who attends the meeting, but who shows up.

Having your camera on also gives context. When I'm driving and can't fully participate, I still hold my phone up so people can see I'm in the car. Now they understand where I am. They set expectations accordingly. It builds another layer of trust.

This matters because remote work doesn't have the "water cooler chatter" of an office. We don't bump into each other in hallways. We don't grab coffee together. The camera is one of the few ways we can share a bit of who we are beyond just our voices.

The Benefits for Your Teammates

Communication over video is already limited compared to being in person. When you add your face to the call, you're giving your teammates more information to work with.

They can see you nod in agreement. They can catch you furrowing your brow in confusion. They can tell when you're engaged versus when you're zoning out. These cues matter. Non-verbal communication makes up a huge part of how humans understand each other.

It also makes meetings less awkward. There's nothing worse than presenting to a wall of black rectangles. Some of my coworkers call this "talking to the void." When people have their cameras off, the speaker has no feedback. No reactions. Nothing. It's honestly kind of depressing.

When you turn on your camera, you're showing respect for the people presenting. You're signaling that this meeting matters, that their time matters, that you're actually there with them.

The Downsides of Keeping Your Camera Off

We've all been in that meeting. Someone has their camera off. Their name gets called. Silence. Then after a few seconds: "Hey, can you repeat the question please?"

It's obvious they were doing something else. And look, I understand that multitasking is hard to avoid in modern work. But when your camera is off, it's way easier to drift. When your camera is on, your brain treats the meeting differently. You're more present. You're more focused.

Research backs this up too. People who keep cameras off are often perceived as less engaged, less trustworthy, and less suited for leadership roles. Fair or not, your teammates and your managers notice.

Here are some findings:

A Vyopta survey found that 92% of executives think workers who turn cameras off don't have a long-term future at the company.
A Korn Ferry survey found that 76% of professionals believe people who leave cameras off are looked upon negatively, and 60% say it's a "career minimizing move."
Research published in the European Journal of Social Psychology found that turning your camera off results in being perceived as less engaged, less moral, and less suitable for leadership.

You can either accept human nature or fight against it. But you're probably going to lose if you try to fight it.

But What About Zoom Fatigue?

I hear this one a lot. "Cameras are exhausting."

And yes, there's research showing that being on camera can increase fatigue, I'm not dismissing that.

But here's my take: everything in life is a cost-benefit analysis.

Sure, being on camera takes energy. But the trade-off is that your teammates trust you more. Your managers remember you. You build relationships. You stay focused. Those benefits are significant.

If you're worried about how you look, don't be. I've been in meetings in my pajamas plenty of times even at Apple. Let alone at startup. Nobody cares. People sometimes say it's cute, but it's genuinely not an issue. You're not in an interview anymore. You're not trying to impress people with your appearance. You're trying to build something together.

If you're worried about your living space, use a virtual background. Problem solved.

My Personal Rule

I turn my camera on about 95% of the time. The only times I don't are when I physically can't, like when I'm driving through a tricky area where Tesla FSD isn't handling things well and I need both hands on the wheel.

Even when I'm driving on easier roads, I'll hold my phone to show my face so people have context. They can see I'm in transit. They understand the situation.

I'd advocate the same for everyone working remotely. If you want to build relationships, if you want people to trust you, if you want to be remembered, just turn it on. It's fine. Really.

For Leaders

If you're a manager or team lead, this applies to you too. Maybe even more so.

You set the example. If you have your camera on, your team is more likely to follow. If you're always camera-off, don't be surprised when everyone else does the same.

Advocate for cameras-on culture in your team. Not as a rigid mandate, but as a norm. Explain why it matters. Make it clear that nobody's judging appearances or backgrounds. What matters is that you're present with each other.

Final Thoughts

Remote work is already lower bandwidth than being in person. We lose the hallway conversations, the lunch chats, the accidental run-ins that build relationships over time.

The camera is one of the few tools we have to close that gap. It lets people see your face, read your expressions, and feel like they're actually talking to a person rather than a name on a screen.

If you don't care about your career, if you're already looking for another job, sure, keep your camera off. But if you want better relationships with your teammates, if you want your leaders to remember you, if you want to actually be present in your work, just turn it on.

It's not about looking perfect. It's about showing up.

My rule: 95% cameras-on. The other 5% is when I literally can't. Try it for a month and see if your meetings feel different.

El Salvador with Kids: 5 Days of Adventure, Vomit, and Unexpected Familiarity

Sat, 29 Nov 2025 00:00:00 GMT

It started the way a lot of our family trips do: opening Google Flights during the kids' Thanksgiving break, looking for somewhere warm and cheap. Living in California, we wanted to go south, no snow, no cold. Mexico and most of South America were quoting us $3,500-4,500 for four people. El Salvador? $1,896 return.

That caught my attention. I remembered reading about Bukele's crackdown on MS-13, how the country had transformed from one of the most dangerous places in the Western Hemisphere to reportedly safer than Mexico. Did some research, confirmed it seemed legit, and booked it. Total planning time: maybe two weeks, mostly using Claude and ChatGPT to batch out activities by region rather than a rigid hour-by-hour itinerary.

What we got was five days that felt less like Central America and more like a time machine back to Indonesia.

Day 1: The Airport Adventure Begins

We landed around 6-7am after a red-eye from San Jose. The airport was surprisingly modern, we even took the obligatory photo with the country seal. And then things got interesting.

I'd booked with Sixt because they were half the price of everyone else. Couldn't find them anywhere in the airport car rental area. Using Google Translate, we talked to an officer who motioned for us to follow her. She grabbed a taxi driver, and suddenly we were walking through dirt paths outside the airport, following this guy to his car parked in some random lot, then driving to an unknown destination.

He was on WhatsApp the whole time, calling people, clearly trying to figure out where this place actually was. I pulled up Google Maps and showed him the address from my booking confirmation. He looked confused. We turned onto a rocky road that was clearly still under construction, and there it was: a tiny office in the middle of nowhere with maybe three cars parked outside.

Lesson learned: don't cheap out on car rentals in El Salvador. But also, it's an adventure. I had the same sketchy experience returning the car later. Just part of the charm.

The drive into San Salvador proper was my first "wait, is this Indonesia?" moment. The outskirts looked exactly like Indonesian villages, people riding on the open backs of pickup trucks, no safety equipment, just holding onto the frame. Coconut sellers everywhere. The chaos felt familiar.

Can’t figure out how to embed videos, so here’s a screenshot

We dropped luggage at the hotel (couldn't check in yet at 9am) and headed to MultiPlaza mall to get an eSIM. $15 for 12GB at the Claro shop. The signup was easy, but slow. Everyone here moves slowly. Very friendly, but slow. Just like Indonesia.

The mall had tons of kid stuff for Christmas, and we ended up spending way longer than planned because Chuck E. Cheese had a 30-minute unlimited play deal for $16. The trick is that there's a short cooldown between scans, so you just wait a few seconds and scan again. The kids racked up 471 tickets, then the live Chuck E. Cheese mascot came out, did a dance with them, and threw bonus tickets everywhere. Grace and George scrambled to collect another 65. Good times.

That evening we went to Picnic, this outdoor restaurant complex with moving animal statues, a massive fountain, and their famous rainbow slide. I'd give the slide 4 out of 5 stars, genuinely fun. They had a VR walking attraction that would've been 4.5 stars if the staff could explain it in English, but they only knew words like "fish" and "planets" and "you." The kids loved it anyway.

Dinner was a feast of seafood and steak for $58 total. Four people, stuffed.

Grace twisted her ankle near the fountain due to uneven ground. First casualty of the trip. We rode a carousel that inexplicably spoke Chinese (clearly imported directly from China, language settings untouched), walked through a Jurassic Park dinosaur exhibit, then hit up a Whole Foods-type supermarket. Fun fact: there's basically one brand of bottled water here called Cristal. Complete monopoly.

The city is beautiful at night. Light shows everywhere.

Day 2: City Tour and the Vomit Begins

Originally planned to hike a volcano, but everyone was exhausted from the red-eye. We pivoted to a city tour instead.

Started at the Mercado Nacional. Rows of stalls selling identical stuff: knick-knacks, knives, fridge magnets, Bukele figurines in various costumes. His face is everywhere here, along with the First Lady. Either people genuinely love him or it's mandatory. Based on the fridge magnets and mini figurines, I think they actually love him.

The historical district was chaos. So many roadblocks and one-way streets that it took forever to find parking. We eventually found a spot near a traditional market that was definitely not touristy. Loud, dirty, real.

The highlight was BINAES, the National Library. Seven stories, and the kids went nuts. The first floor is for small children, upper floors have LEGO, video games, even VR. Kids can earn gaming credits by reading books: 3 minutes of reading for 10 minutes of game time. Terrible exchange rate, but good incentive design. They had Harry Potter, Lord of the Rings, and Game of Thrones promotional displays. Interesting franchise choices for a library.

We visited the rainbow stained-glass church next to Plaza Libertad. Beautiful, packed with locals. Everyone stared at us. Worth noting: we saw almost no other Asian tourists the entire trip. One group at this church, one family at hotel breakfast, one mom with daughters at the airport on the way out. That's it. We stood out.

The Teatro Nacional was worth the tour. Private guide, very cheap. Painted ceiling like Italian theaters, the president has his own exclusive box, and there's a 7-meter deep pool under the stage to absorb sound. Candice played piano in the recital room.

We also hit the Palacio Nacional, the old government building with color-coded rooms. Pink for Supreme Court, blue for parliament. The courtyard had a massive tree, great for photos.

George needed to pee, so we went back to the library and ended up staying a while. They have beds and bean bags. The AC was glorious. Kids jumped around on bean bags while Candice and I caught our breath. On the walk back to the car, she bought a bag of rambutan from a street vendor for only $1. Cheap.

We tried Parque Cuscatlán at night because supposedly the lights are pretty. It was meh. No good food nearby either.

That's when George started vomiting. First one caught us off guard, went straight onto the car seat. We stopped at the hotel, cleaned what we could with tissues, then drove to a pharmacy for baking soda. The lady there gave us a bunch of plastic bags too. Nice lady.

Ate sushi at Bambú mall (ramen and California rolls), then back to the hotel where I covered the car seat in baking soda and cracked the windows. George vomited multiple times that night. No fever, just gastritis. Purely viral. Should be better tomorrow.

Day 3: Pyramids, Lakes, and More Vomit

Morning started with a car wash to clean the baking soda disaster. $20, about an hour. We walked to a pharmacy for hydrocortisone (Grace's ankle was swollen and bug-bitten), then waited at a mall while George lay down feeling miserable. Bought electrolytes.

Once the car was ready, we drove to San Andrés ruins. Really pretty pyramids sitting on lush green fields. There was a school group visiting, some students interviewed Candice for what seemed like a class project. Asked her which part was her favorite. "We don't know, we just arrived." They pointed us toward the pyramids.

It was brutally hot, but we found a tree in the middle of the field and rested in the shade. The coconut seller there was great, the kids brought their empty coconuts back and used a chopping hand gesture to ask her to split them open. She understood immediately. You'd be surprised how much you can communicate with just body language.

The drive to Lake Coatepeque was surreal. Armed soldiers and police stationed along the entire route, all holding M4 assault rifles. Not just at the lake, I saw heavily armed security everywhere in El Salvador. Even the supermarket security guards carry shotguns. Long ones.

At first it felt intimidating, but after watching documentaries about the MS-13 crackdown, I understood. These guys are defending against gangs who used to kill with impunity. Every interaction we had with police and soldiers was friendly. Smiles, waves. They could tell we were tourists.

Lunch was at Rancho Alegre by the lake. Live DJ singing and playing music. There was a diving board off the first floor, a sad little playground, a dirty swimming pool with three slides (we skipped it), and a trampoline. Grace and George played on the trampoline with a local kid. Kids don't care about language barriers, they just figured it out.

We ordered rabbit, mixed grill, chicken, fish. $40 total. Insane value.

The drive back took almost two hours due to traffic and construction. Construction everywhere in this country, buildings going up, streets being widened, Christmas decorations being installed. El Salvador is clearly in the middle of a renaissance. Waze took us through back streets that looked exactly like poor neighborhoods in Indonesia: small houses with zinc roofs, people sitting outside chatting with neighbors. Pre-internet vibes.

Candice drove for the first time this trip and handled it perfectly. The aggressive driving culture is similar to China, so she's no stranger to it.

Back at the hotel, we swam, hit the jacuzzi, then ordered room service: chicken soup, spinach linguine, salmon burgers. Good day despite George vomiting into a plastic bag during the drive. At least it didn't hit the seats this time.

Day 4: Beach Day and the Google Maps Disaster

Grace's turn to be sick. Vomited after hotel breakfast, multiple times. We started late.

Originally planned to hike to a waterfall, but after talking to another tourist (Zack from Miami) who said it took 3+ hours, we decided the kids weren't healthy enough. Pivoted to beach day instead.

Accidentally drove to La Libertad first instead of El Sunzal. Turned out to be a happy accident. La Libertad is more developed, proper restaurants, shops, even a run-down amusement park. We had lunch at a beachfront restaurant.

It was loud. Every restaurant blasting their own music, plus multiple mariachi bands competing to be heard. One band was mostly young kids and genuinely annoying. The others were more refined, slower songs, actually added to the atmosphere. We treat trips like this as novelty pumps. New experiences, even chaotic ones, are the point.

After lunch, we drove to El Sunzal and stopped for coconut along the way. George was struggling to scrape out the meat (too slow), so the lady just took over and fed him. He rolled with it.

El Sunzal had black sand beaches but was very undeveloped, reminded me of Pagar Timun in Ketapang. Bunch of food stalls, kids played in the sand, I had a fresca. Then back to the hotel.

Candice took the kids swimming while I returned the rental car. Forgot to fill up gas. Opened Google Maps to find the nearest station.

Big mistake.

Google Maps led me down increasingly sketchy roads, dirt, rocks, mud, trees everywhere, extremely narrow. Complete dead end. Locals were standing around staring at my car like "who is this idiot?" I had to reverse the entire way out. It was dark, no lights, completely alone. Every gang documentary I'd watched flashed through my mind. If something happened here, there'd be no recourse. I was genuinely relieved Candice and the kids were safe at the hotel pool.

Switched to Waze. The real road still didn't match the map perfectly, but I made it to the gas station. Lesson learned: always check which navigation app locals use. Waze works here. Google Maps will get you killed (okay, maybe not killed, but definitely lost in sketchy forests).

Returned the car successfully. Amazing. Took an Uber back to the hotel for $1.68.

Dinner at the hotel restaurant, good spaghetti and pupusas. Played pool with the kids until 10:30pm. It was Grace and George's first time ever playing pool, just messing around, learning the basics. Then packing, then sleep.

Day 5: The Fly Back (Barely)

4:30am wake-up call. We stayed at the Quality Hotel near the airport (yes, that's literally the name) and grabbed breakfast at 5am. Solid hotel breakfast, no complaints.

The airport was chaos.

The Avianca check-in line was absolutely packed. Looking at the queue, there was no mathematical way we'd make our flight. Luckily, they started calling out passengers flying to Miami or San Francisco to come to the front. We got through check-in with maybe 5 minutes before boarding. Cutting it way too close.

What surprised me was how strict security was compared to TSA at SFO. Back home, we don't even have to take off our shoes anymore. Here? Shoes off. They also caught my nail clipper buried in my backpack. TSA never flagged it on the way down. The security guy pulled it out, showed it to his supervisor, who eventually waved it through. But that little delay when you're already running late? Not fun.

Then there's a second security checkpoint right at the boarding gate, Singapore-style. Once you're through, you're trapped. No bathroom access, no water (they make you pour it out), no leaving the holding area. Plan accordingly.

The Avianca flight itself: cheap for a reason. This was the first flight I've ever been on where the seats don't recline at all. Not "limited recline." Zero recline. Completely vertical for 6 hours. No complimentary food but they gave us a bottle of water. Everything else costs extra. Pure budget airline experience.

Here’s a kid sleeping on the floor.

Thank god I brought my turtle travel pillow. Without it, sleeping would've been nearly impossible. The kids managed okay, probably because they can sleep anywhere. Candice and I just endured.

Landed at SFO, took a taxi home. Adventure complete.

Final Thoughts

On the Indonesia parallels: This was the biggest surprise. The pace of life, the driving chaos, the buses that stop anywhere to pick up passengers (they have "kernets" - guys hanging off the bus shouting destinations), and “bajaj”, the stratification between wealthy city centers and poor outskirts, the zinc-roof houses. It all felt familiar in a way I didn't expect from Central America.

Indonesian Kernet

![el-salvador-family-trip-2025-image 1.png](https://www.augusteo.com/assets/blog/el-salvador-family-trip-2025/el-salvador-family-trip-2025-image 1.png)

Indonesian Bajaj

On safety: Yes, there are armed soldiers everywhere. Yes, it felt surreal at first. But every interaction was friendly. The Bukele crackdown seems to have genuinely transformed the country. Construction everywhere suggests people finally feel safe enough to invest and build.

On food: Everything was cheap and good. Pupusa quality varies by chef, and the hotel restaurant ones were great. We avoided street food to prevent stomach issues (ironic, since both kids got gastritis anyway from who knows what). Horchata was fine, nothing special. Basically sweet rice drink. The mixed grills and seafood were the highlights.

On tourism infrastructure: This is still a raw place. Not ready for mass tourism. The rental car situation, the navigation app problems, the language barrier (almost no one speaks English), the lack of signage. It's all very DIY. Only come here if you want authentic adventure, not polished resort experiences.

On the people: Universally friendly and helpful. Even when communication was impossible, body language and Google Translate got us through. Don't let them type on your phone though. Some of the older folks type one letter at a time. As fast as Flash the DMV sloth from Zootopia.

On Bitcoin: Completely invisible. We never saw a single Bitcoin ATM or sign accepting crypto. Just credit cards and cash.

Would we go back? Yes, but not soon. When Grace and George are teenagers with stronger immune systems, we'll return for the volcano hike and waterfall we missed. The nature here is stunning and still pristine.

Three words for El Salvador: Raw, cheap, surprisingly familiar.

Trip stats: 5 days, 2 vomiting children, 1 Google Maps near-disaster, 1 nail clipper interrogation, 0 MS-13 encounters, 0 degrees of seat recline on Avianca, pupusas everywhere, and countless good food and drinks.

PS: All the juices are real and tasted so good.

Why Your Fear of Looking Stupid Is Your Biggest Career Risk

Sun, 06 Jul 2025 00:00:00 GMT

So I was scrolling through Hacker News yesterday and stumbled upon this discussion about Cate Hall's concept of the "Moat of Low Status". The idea hit me like a ton of bricks - not because it's revolutionary (people have been saying "embrace being a beginner" forever), but because of how urgent this mindset has become in 2025.

Here's the thing: the world is moving so fast now that whatever you're comfortable doing today will probably be obsolete in a few years. Maybe even months. The only sustainable advantage is getting really, really good at being bad at things.

The Status Paradox

But here's what really struck me from the discussion - the moat of low status isn't equally hard for everyone to cross. There's a cruel paradox at play here.

If you're very high status, you can afford to look stupid. People assume you're asking "dumb" questions strategically. You have enough social capital banked that a few withdrawals won't hurt. When a senior executive asks basic questions, people think "they're getting back to first principles" or "they're testing us."

If you're very low status, they also have nothing to lose. If you learned to always assume that they were the dumbest person in the room. This became a superpower - they could ask whatever they wanted because they had no status to protect.

But if you're in the middle? That's where the moat becomes a prison. You have just enough status to lose, but not enough to waste. You're trapped in what researchers call a social trap - where short-term social preservation leads to long-term stagnation.

This aligns perfectly with psychological research on social status. Middle-status individuals are the most anxious about their position, the most likely to conform, and the least likely to take risks that might threaten their precarious perch.

The Dumb Questions Superpower

One comment that resonated with me was about a senior engineer who deliberately asks the "dumb" questions in meetings because they can afford the social hit. They're using their high status as a shield to help juniors who might be thinking the same thing but are too afraid to speak up.

I've been doing this for years, though not always intentionally. In meetings, I'll ask questions constantly - sometimes about things I already understand, sometimes about things I genuinely don't know. And yeah, sometimes I look like an idiot because I wasn't paying attention and it was already explained.

But here's what I've learned: I'd rather look dumb for 30 seconds than be ignorant forever.

My kids hear this from me all the time. When they're struggling with something new, I remind them: "You're bad at this right now. Everyone is bad at things when they start. The only way to get better is to keep doing it."

The Beginner's Mind Is Not Optional

There's a concept in Zen Buddhism called shoshin, or "beginner's mind." As Shunryu Suzuki wrote in his classic book: "In the beginner's mind there are many possibilities, but in the expert's mind there are few".

This isn't just philosophical fluff. Research shows that intellectual humility - having a beginner's approach - is more of a state than a trait. It varies within us from situation to situation. Which means we need to actively cultivate it, constantly.

The danger is real. The more expert we become, the more we fall victim to what psychologists call the Einstellung effect - where a person becomes so accustomed to a certain way of doing things that they do not consider or acknowledge new ideas or approaches.

My Brazilian Jiu-Jitsu Reality Check

Last year, I decided to try Brazilian Jiu-Jitsu. I'd done various martial arts for 15+ years, but groundwork was completely new to me. My experience with Judo and Japanese Jiu-Jitsu was mostly standing techniques and throws.

Walking into that BJJ gym was humbling. I knew nothing. Everyone - and I mean everyone - could destroy me with ease.

But you know what? It was exhilarating.

I became that annoying white belt asking constant questions: "What happens if I do this? What if they grab here? How do I defend against that?" I was the human embodiment of the beginner's mind, and I learned fast.

Unfortunately, I had to stop. Turns out when you're my age, getting injured means days in bed with extreme back pain, muscle relaxants, and physiotherapy sessions. When you're young, you bounce back. When you're older... not so much. But I don't regret a single moment of getting my ass kicked on those mats.

The Voice AI Adventure

Here's a more recent example. My company wanted to build voice AI agents to handle phone calls. I knew absolutely nothing about voice AI.

So what did I do? I watched a bunch of videos, built some terrible prototypes, and then went to an AI engineering conference last month where I spent all my time in the voice track.

I learned about WebSocket vs WebRTC, how Voice Activity Detection (VAD) works, the difference between semantic VAD and basic activity detection, and the intricacies of pipeline architectures versus speech-to-speech models.

Was it embarrassing asking basic questions while surrounded by experts? Sure. But now we have a working prototype that we're showing to customers, and we're expecting massive uptake soon.

The satisfaction of going from zero to deployed product in a month? Priceless.

Breaking Out of the Middle Status Trap

So how do we escape the middle status trap? How do we give ourselves permission to be beginners again?

Research on workplace questions shows that 70% of employees face barriers to asking questions, with fear of ridicule being the number one obstacle. Yet 92% of managers believe curious people bring new ideas and see curiosity as driving innovation and high performance.

The disconnect is obvious - we're afraid of something that would actually make us more valuable.

Here's what works:

Frame your ignorance as curiosity - "I'm fascinated by this but want to make sure I understand..."
Use your research - "I've been reading about X and I'm curious how it applies here..."
Make it about the team - "Let me ask what might seem like a basic question to make sure we're all aligned..."
Own your beginner status - "I'm new to this area, so bear with me..."

As one HR expert noted, successful entrepreneurs ask "stupid" questions all the time. That's because no question is "stupid" if you don't know already the answer.

The Real Message

Look, we can dress this up with fancy terms like "growth mindset" or "continuous learning," but here's the brutal truth: the places where you know almost nothing are where you have the highest learning rate.

If you only do things you're good at, you'll improve - slowly. But when you do things you suck at? That's where the magic happens. That's where you compress years of growth into months.

The world is changing too fast for expertise to be a permanent state. As James Clear writes about shoshin, "There is a danger that comes with expertise. We tend to block the information that disagrees with what we learned previously and yield to the information that confirms our current approach".

The Bottom Line

The moat of low status isn't just about learning new skills. It's about recognizing that in a world changing at breakneck speed, your ability to be comfortably uncomfortable is your most valuable asset.

And if you're stuck in that middle status zone? Remember that both the very successful and the seemingly unsuccessful share one trait - they're not afraid to look stupid. Maybe it's time to pick a side.

Next time you're hesitating to try something because you might look stupid, remember: looking dumb is temporary, but ignorance is expensive.

So go ahead. Be the first person on the dance floor. Ask the obvious question in the meeting. Try that thing you've been putting off because you know you'll suck at it.

The world won't turn to ash. And you might just surprise yourself.

From MS-DOS to LLMs: Why This Is Computing's Next Great Transition

Wed, 18 Jun 2025 00:00:00 GMT

Watch the full talk on YouTube: Andrej Karpathy: Software Is Changing (Again)

I recently watched Andrej Karpathy's talk at Y Combinator on Youtube, and wow, it really crystallized a lot of thoughts I've been having about where software development is heading. As someone who learned to code on MS-DOS more than 20 years ago (memorizing all those commands because there was no GUI!), Karpathy's comparison of today's LLMs to 1960s-era operating systems really hit home.

We're Living Through a Fundamental Shift

Karpathy opened with a bold claim: software hasn't changed this fundamentally in 70 years. And then suddenly, it's changed twice in just a few years.

He breaks it down into three evolutionary stages:

Software 1.0: The Code We Write

This is traditional programming - the C++, Python, JavaScript that humans write directly. It's explicit instructions telling computers exactly what to do.

Software 2.0: Neural Network Weights

A few years ago, Karpathy observed that neural networks represented a new kind of software. Instead of writing code, we're tuning datasets and running optimizers to create neural network parameters. He pointed to Hugging Face as the "GitHub of Software 2.0" - a place where these models live and evolve.

Software 3.0: Programming in English

Here's where things get wild. With large language models, we're now programming computers in natural language. Your prompts are literally programs that configure these AI systems. As Karpathy tweeted (and it's now his pinned tweet): "The hottest new programming language is English."

The Tesla Autopilot Story: A Preview of What's Coming

Karpathy shared a fascinating story from his time at Tesla. When he worked on Autopilot, he watched as neural networks literally ate through the software stack. The C++ code was progressively deleted as more functionality moved from traditional code to neural nets.

I think we're seeing the exact same pattern now. Traditional software is being consumed by AI, layer by layer.

LLMs as the New Operating System

Here's where Karpathy's analysis gets really interesting - and where I strongly agree with him. LLMs aren't just utilities like electricity (though they share some characteristics). They're becoming more like operating systems.

Think about it:

Like utilities: We access them through metered APIs, pay per token, demand low latency and high uptime
Like chip fabs: Massive capital expenditure, deep tech trees, centralized expertise
Like operating systems: Complex software ecosystems with multiple providers

The comparison to operating systems is particularly apt. We have:

A few closed-source providers (OpenAI, Anthropic, Google) = Windows/macOS
Open-source alternatives (Llama ecosystem) = Linux
Apps that can run on multiple "OS" options (Cursor works with GPT, Claude, or Gemini)

The Commodity Future

What really resonates with me is how LLMs are becoming commoditized. Just like we don't care where our electricity comes from (as long as it works), we're rapidly approaching a point where the specific LLM won't matter much. They'll all be highly capable and cheap. The winners in this era won't be the foundation model providers - it'll be the application layer. Who will build the Microsoft Office or Adobe Photoshop of the LLM era?

The Psychology of LLMs: Autistic Savants with Superpowers

Karpathy's characterization of LLMs as "people spirits" with "jagged intelligence" is spot-on. They're like autistic savants who can memorize entire phone books but might insist that 9.11 > 9.9.

What They're Great At:

Encyclopedic knowledge and memory
Processing massive amounts of information
Pattern recognition across domains

Their Cognitive Deficits:

Hallucinations and confabulations
Jagged intelligence (superhuman in some areas, sub-human in others)
Anterograde amnesia (can't form new long-term memories)
Susceptibility to prompt injection and manipulation

However, I think this talk is incomplete without mentioning the recent breakthroughs in thinking models like OpenAI's o1 and o3. Many of the "jagged intelligence" problems Karpathy mentions are actually being solved through chain-of-thought reasoning. When we give LLMs more compute time to "think," they can work through problems step-by-step, catching their own errors. It's slower, yes, but the accuracy improvements are dramatic.

Building in the Age of Partial Autonomy

The most practical part of Karpathy's talk focused on how we should actually build with these fallible but powerful systems. His key insight: we need partial autonomy apps with:

1. Context Management

Apps shouldn't make users manually copy-paste between ChatGPT and their work. Look at Cursor - it automatically manages your codebase context.

2. Multiple Model Orchestration

Real apps need embedding models, chat models, diff models, etc. The complexity should be hidden from users.

3. Application-Specific GUIs

Text interfaces are terrible for humans! We need visual diffs, one-click accepts/rejects, and interfaces that utilize our "computer vision GPU" (our eyes and visual cortex).

4. The Autonomy Slider

This is crucial. Sometimes you want:

Low autonomy: Tab completion in Cursor
Medium autonomy: Command+K to modify a code block
High autonomy: Command+I to refactor your entire codebase

The key is keeping AI "on a leash" - getting work done without overwhelming humans with 10,000-line diffs we can't possibly review.

Vibe Coding: Everyone's a Programmer Now

Then there's "vibe coding" - Karpathy's term that unexpectedly went viral and now has its own Wikipedia page!

I love his example of kids using AI to build apps. It's such a wholesome vision of the future. This isn't about replacing programmers - it's about making programming accessible to everyone. It's a gateway drug to software development.

My Take on Vibe Coding

Having tried it myself (and watched the ecosystem explode in 2025), vibe coding is real but has limits. The actual coding part has become trivial - tools like Cursor, v0, Claude Artifacts, and Windsurf can generate working prototypes in minutes. But here's the catch: everything else is still hard.

Karpathy's Menu Gen app perfectly illustrates this. He built the core functionality in hours, but then spent a week on the "real" stuff:

Authentication
Payments
Domain setup
Deployment
All the DevOps clicking around

This is still the biggest barrier. We need to make the entire stack accessible to LLMs, not just the code generation part.

Building for Agents: The Next Frontier

The final piece of Karpathy's vision is perhaps the most forward-looking: we need to build for agents, not just humans.

This means:

llms.txt files (like robots.txt) to help LLMs understand your site
Documentation in markdown instead of human-oriented formats
Replacing "click here" instructions with curl commands that agents can execute
Tools like Gitingest that transform human interfaces into LLM-digestible formats

We're meeting LLMs halfway, making our digital infrastructure speak their language.

Looking Ahead: The Decade of Agents

Karpathy's self-driving car story serves as a crucial reality check. He had a perfect Waymo demo in 2013 and thought autonomous driving was imminent. Twelve years later, we're still working on it.

His prediction? This is the decade of agents - not the year of agents. We need to be patient, keep humans in the loop, and gradually slide that autonomy slider from left to right.

The Bottom Line

What an incredible time to enter the tech industry! We're not just iterating on existing paradigms - we're completely rewriting how software works. Three different programming paradigms (traditional code, neural networks, natural language) are competing and complementing each other.

The winners won't necessarily be those building the best LLMs. They'll be those who build the best experiences on top of these new operating systems. Just like the biggest tech companies today aren't OS vendors but application builders.

As someone who started with MS-DOS and watched GUIs revolutionize computing, I can't help but feel we're at a similar inflection point. We're in the MS-DOS era of AI - powerful but clunky, text-based, requiring memorization of commands. The GUI moment for AI is coming, and it's going to change everything.

The future isn't about AI replacing programmers. It's about AI augmenting human creativity, making programming accessible to kids who just want to build cool stuff, and gradually automating the tedious parts while we focus on what matters: solving real problems for real people.

Time to build! 🚀

Book Review - "An Absolutely Remarkable Thing" by Hank Green

Tue, 04 Mar 2025 00:00:00 GMT

Having been a fan of both John and Hank Green for years, I was excited to dive into Hank's debut novel. As a longtime follower of their work, I couldn't pass up the chance to see how Hank's storytelling would translate to fiction. Spoiler alert: he doesn't disappoint!

What I Loved About This Book

This book is captivating. Like, seriously captivating. I finished it in 2-3 days because I simply couldn't put it down. While reading, I experienced a whole range of emotions—from anger at the characters to wonder about the dream sequences and puzzles—but boredom was never one of them. I was constantly pulled forward, always wanting to know what would happen next.

"They tell you power corrupts. They never tell you how quickly."

This quote perfectly captures one of the book's central themes, and it stuck with me long after finishing the last page.

What's It About? (Major Spoilers Ahead!)

The story follows April May, who stumbles upon a giant robot statue in New York City one night. In my mind, I pictured it like a Gundam just standing there in the middle of Manhattan. April and her friend Andy decide to film it, and their video goes viral overnight.

Here's where it gets interesting: identical statues (called "Carls") have appeared simultaneously in major cities worldwide. These aren't just ordinary statues—they're made of unknown materials, impervious to damage, and they actually float. Physics doesn't apply to them the way it does to everything else on Earth.

As the world reacts to these mysterious appearances, people become divided on whether the Carls are a force for good or evil. Are they just observing us? Or are they planning to take over Earth?

The Dream Sequences: A Brilliant Plot Device

One of the most fascinating elements is how the Carls give everyone the same dream where they can lucid dream and work together to solve complex puzzles. These puzzles are deliberately designed to be cooperative—impossible to solve alone. They require diverse knowledge ranging from esoteric languages to obscure musical instruments and aviation engineering.

April theorizes that the Carls are testing humanity's ability to collaborate, which adds an intriguing layer to the story.

Characters That Feel Real

While April is our protagonist, I particularly appreciated Maya, her ex-girlfriend. Unlike other characters who often felt like sidekicks just helping April without question, Maya holds her own ground and tells it like it is. She's not afraid to challenge April's decisions and provides a necessary counterbalance to April's increasingly fame-driven choices.

Other key relationships in the book include:

Robin, April's assistant
Miranda, a scientist who reaches out for help with the puzzles
Andy, her friend who helps create the initial viral video

Fame, Social Media, and Our Modern World

The book delves deeply into how fame changes people, and it feels eerily relevant. We watch as April becomes addicted to her Twitter followers, talk show invitations, and public adoration. Her transformation from normal person to celebrity spokesperson for the Carls happens at breakneck speed, showing how quickly power can indeed corrupt.

There's also a powerful exploration of xenophobia and fear-mongering through characters like Peter Petrawicki, who writes a book called "Invaded" arguing that the Carls are dangerous. This aspect feels very reminiscent of real-world politics, where fear is peddled to gain power and popularity—a rather Trumpian tactic that the book critiques effectively.

The Climactic Ending

One of the most memorable moments comes near the end when April is trapped in a burning warehouse. The scene where she breaks through a window only to be crushed by a falling beam is heartbreaking—but then the twist comes when we learn she's somehow been transported into the Dream where she can speak directly with Carl. The revelation that Carl specifically chose her for first contact adds a whole new dimension to the story.

Final Thoughts

While I wouldn't call it a perfect 5-star book that I'll repeatedly return to, "An Absolutely Remarkable Thing" is definitely a solid 4-star read that I thoroughly enjoyed. The cliffhanger ending with April's cryptic "Knock Knock" text message has me eager to pick up the sequel, "A Beautifully Foolish Endeavor."

If you enjoy science fiction with a contemporary twist, explorations of social media fame, and mysteries that unfold in unexpected ways, this book is absolutely worth your time.

From Swift to Rails: My Foray Into Non-Trivial Ruby on Rails

Mon, 03 Mar 2025 00:00:00 GMT

I've been diving into Ruby on Rails after years of working with Swift for iOS and React. The mental shift has been quite interesting, and I wanted to share my observations while they're still fresh.

MVC vs. MVVM: Not What I Expected

Coming from the iOS world, I'm thoroughly accustomed to the MVVM (Model-View-ViewModel) pattern. iOS originally used MVC with UIKit, but Rails' implementation of MVC differs significantly from what I anticipated.

In iOS with CoreData, model files are typically small and lightweight, while the controller ends up doing most of the heavy lifting. This imbalance is precisely why iOS shifted toward MVVM – controllers were becoming unwieldy and overly complex. The ViewModel serves somewhat like a controller, but its role is more focused – it doesn't directly control the view; rather, it supplies the data for the view to respond to.

In Rails, I discovered the opposite problem. Controllers are designed to be thin, while models tend to become bloated! Developers often add increasing amounts of business logic into model files, transforming them from simple data definitions into repositories of helper methods and modifiers.

Here's what a "fat" Rails model typically looks like:

class Car < ApplicationRecord
  # Basic attributes: make, model, year

  # Validations
  validates :make, presence: true
  validates :year, numericality: { only_integer: true }

  # Business logic
  def legal_in_america?
    current_year = Date.today.year
    self.year >= (current_year - 10)
  end

  # More business logic
  def full_name
    "#{self.year} #{self.make} #{self.model}"
  end

  # Price calculation logic
  def depreciated_value(original_price)
    age = Date.today.year - self.year
    depreciation_factor = [0.1 * age, 0.9].min
    original_price * (1 - depreciation_factor)
  end

  # More methods...
  def needs_emission_test?
    self.year < Date.today.year - 2
  end

  # And more methods...
  def registration_fee
    base_fee = 100
    extra_fee = Date.today.year - self.year
    base_fee + (extra_fee * 5)
  end
end

In Swift MVVM, this logic would naturally reside in the ViewModel, not the model. Yet in Rails, all of it tends to accumulate in the model.

The Delegation Solution: Keeping Models Slim

As I explored further, I discovered additional components in the Rails ecosystem that address this problem quite elegantly. One particularly useful solution is using the SimpleDelegator pattern.

Ruby's standard library provides a SimpleDelegator class that allows you to wrap an object and delegate method calls to it, while also adding additional functionality. This is perfect for extracting logic from models without losing access to their attributes and methods.

Here's how it works:

# The slimmed-down model
class Car < ApplicationRecord
  # Just basic attributes and validations
  validates :make, presence: true
  validates :year, numericality: { only_integer: true }
end

# A decorator that adds legal-related functionality
class CarLegalDecorator < SimpleDelegator
  def legal_in_america?
    current_year = Date.today.year
    year >= (current_year - 10)
  end

  def needs_emission_test?
    year < Date.today.year - 2
  end
end

# A decorator for financial concerns
class CarFinanceDecorator < SimpleDelegator
  def depreciated_value(original_price)
    age = Date.today.year - year
    depreciation_factor = [0.1 * age, 0.9].min
    original_price * (1 - depreciation_factor)
  end

  def registration_fee
    base_fee = 100
    extra_fee = Date.today.year - year
    base_fee + (extra_fee * 5)
  end
end

# Usage in a controller
def show
  @car = Car.find(params[:id])
  @legal_car = CarLegalDecorator.new(@car)

  if @legal_car.legal_in_america?
    # Do something
  end
end

The beauty of SimpleDelegator is that it automatically forwards method calls to the wrapped object. Notice how in the legal_in_america? method, I can directly call year instead of car.year because the method call is delegated to the Car object.

This approach brings several advantages. Controllers can use only the specific decorator they need, without having to know about all the other business logic. For instance, only the legal controller needs to use the CarLegalDecorator.

Presentation Layer: JSON Resources

Another powerful component I discovered is the JSON Resources concept, which functions as a presenter layer. In Rails, libraries like Alba provide a clean way to define how to present a model as JSON.

Here's how a resource would look using the Alba gem:

# frozen_string_literal: true
class CarResource
  include Alba::Resource

  attributes :id, :make, :model, :year

  # Derived attributes
  attribute :name
  attribute :is_legal
  attribute :registration_status
  attribute :owner_info
  attribute :maintenance_history

  # Access decorated car for legal checks
  def decorated_car(car)
    @decorated_car ||= CarLegalDecorator.new(car)
  end

  # Access finance decorator for financial calculations
  def finance_decorator(car)
    @finance_decorator ||= CarFinanceDecorator.new(car)
  end

  def name(car)
    "#{car.year} #{car.make} #{car.model}"
  end

  def is_legal(car)
    decorated_car(car).legal_in_america?
  end

  def registration_status(car)
    car.year < Date.today.year - 5 ? "Needs renewal" : "Valid"
  end

  def owner_info(car)
    car.owner&.name || "No owner information"
  end

  def maintenance_history(car)
    car.maintenance_records&.map do |record|
      {
        date: record.service_date,
        description: record.description,
        cost: finance_decorator(car).adjust_for_inflation(record.cost)
      }
    end || []
  end
end

Then in the controller, instead of building complex JSON structures, you simply use the resource:

class CarsController < ApplicationController
  def index
    @cars = Car.includes(:owner, :maintenance_records).all

    respond_to do |format|
      format.html # Render the index.html.erb template
      format.json { render json: @cars.map { |car| CarResource.new(car).serializable_hash } }
    end
  end
end

The real power becomes evident when using this in an ERB template with data grids like AG Grid:

<%# In your index.html.erb template %>
<div id="cars-grid" class="ag-theme-alpine" style="height: 500px; width: 100%;"></div>

<script>
  document.addEventListener('DOMContentLoaded', function() {
    new agGrid.Grid(document.querySelector('#cars-grid'), {
      columnDefs: [
        { headerName: 'Car', field: 'name' },
        { headerName: 'Legal Status', field: 'is_legal',
          cellRenderer: params => params.value ? 'Legal' : 'Illegal' },
        { headerName: 'Registration', field: 'registration_status' },
        { headerName: 'Owner', field: 'owner_info' },
        // More columns...
      ],
      // The key part - using our resource to format the data
      rowData: <%= raw @cars.map { |car| CarResource.new(car).serializable_hash }.to_json %>,
      defaultColDef: {
        sortable: true,
        filter: true
      }
    });
  });
</script>

This approach dramatically simplifies both ERB/HTML views and controllers. The resource becomes the single source of truth for how your data is presented, while keeping presentation logic out of both models and controllers.

Here's an updated diagram illustrating how these components work together:

The BFF Pattern: Views as API Clients

Taking the Resource concept a step further, I've discovered another powerful pattern in Rails that blurs the traditional MVC boundaries: treating views as API clients. In this Backend-For-Frontend (BFF) pattern, the view makes AJAX calls back to the same controller but with a different format (JSON).

Here's how I implemented this with our car example:

# app/controllers/cars_controller.rb
# frozen_string_literal: true
class CarsController < ApplicationController
  before_action { @page_title = "Car Inventory" }
  before_action :set_car, only: %i[show update]

  def index
    respond_to do |format|
      format.html # Renders the index.html.erb template (lightweight)
      format.json do
        # Heavy database query only happens for JSON requests
        cars = Car
          .where(manufactured_year: 5.years.ago.year..)
          .order(created_at: :desc)
          .includes(:owner, :maintenance_records)

        # Use the resource for JSON serialization
        render json: CarResource.new(cars).serialize
      end
    end
  end

  def show
    respond_to do |format|
      format.html
      format.json { render json: CarResource.new(@car).serialize }
    end
  end

  private

  def set_car
    @car = Car.find(params[:id])
  end
end

The corresponding view then makes a fetch call to retrieve this JSON data:

<!-- app/views/cars/index.html.erb -->
<div class="mb-10 flex relative mx-auto items-center space-x-4">
  <!-- Filter controls for the grid -->
  <div class="w-1/2">
    <div class="inline-flex flex-nowrap whitespace-nowrap border border-blue-500 bg-white rounded-lg overflow-hidden">
      <button type="button" class="px-6 py-3 text-blue-500 font-medium focus:outline-none hover:bg-blue-100">
        All
      </button>
      <button type="button" class="px-6 py-3 text-blue-500 font-medium focus:outline-none hover:bg-blue-100">
        Legal
      </button>
      <button type="button" class="px-6 py-3 text-blue-500 font-medium focus:outline-none hover:bg-blue-100">
        Needs Renewal
      </button>
    </div>
  </div>
</div>

<div id="carsGrid" class="ag-theme-alpine" style="width: 100%; height: 600px;"></div>

<script type="module">
  import { createGrid } from "ag-grid-community";

  document.addEventListener("turbo:load", () => {
    initializeGrid();
  });

  function initializeGrid() {
    const gridContainer = document.getElementById("carsGrid");
    if (!gridContainer) return;

    const columnDefs = [
      {
        headerName: 'CAR',
        field: "name",
        minWidth: 180,
        cellRenderer: params => {
          const id = params.data?.id;
          const name = params.data?.name;
          return `<a href="/cars/${id}" data-turbo="false">${name}</a>`;
        }
      },
      {
        headerName: 'LEGAL STATUS',
        field: "is_legal",
        minWidth: 120,
        cellRenderer: params => params.value ? "Legal" : "Illegal"
      },
      {
        headerName: 'REGISTRATION',
        field: "registration_status",
        minWidth: 150
      },
      {
        headerName: 'OWNER',
        field: "owner_info",
        minWidth: 200
      }
    ];

    const gridOptions = {
      columnDefs: columnDefs,
      rowData: [],
      defaultColDef: {
        flex: 1,
        filter: true,
        sortable: true
      },
      onGridReady: function(params) {
        fetchDataAndSetRows(params.api);
      }
    };

    createGrid(gridContainer, gridOptions);
  }

  function fetchDataAndSetRows(gridApi) {
    fetch("/cars.json") // Calls the same controller with .json format
      .then(response => response.json())
      .then(data => {
        gridApi.setGridOption('rowData', data);
      })
      .catch(error => console.error("Error loading grid data:", error));
  }
</script>

What makes this pattern interesting is:

Separation of concerns: The initial HTML render is lightweight and quick, while the data-heavy operation happens asynchronously.
Resource reuse: The same Resource used for API endpoints can be reused for internal view components.
Progressive enhancement: The page is functional even before the JS-driven grid loads.
Performance optimization: Including complex data in the initial HTML render can slow down time-to-first-render, but this pattern avoids that.

This pattern combines the best of both worlds: Rails' server-side rendering for the initial page load and API-driven client-side rendering for data-heavy components. It feels very similar to how modern frontend frameworks operate, but without requiring a separate API backend.

The Missing Types: Swift vs. Ruby

The second major difference that struck me is how much I've come to rely on Swift's type system! Ruby's dynamic typing means there's no clear distinction between methods, functions, and variables as in Swift. Everything is simply defined with def, and you call methods on objects without any guarantee they'll respond appropriately.

The challenge here is that you can't depend on the compiler to verify correctness before deployment. Instead, you must write more comprehensive unit tests and adopt a defensive coding style.

In Swift, I could write something like:

struct Car {
    let make: String
    let model: String
    let year: Int

    func isLegalInAmerica() -> Bool {
        let currentYear = Calendar.current.component(.year, from: Date())
        return year >= (currentYear - 10)
    }
}

// Function with optional
func processRegistration(for car: Car?) {
    // Using guard let for early return
    guard let car = car else {
        print("No car provided")
        return
    }

    // Now we know car is not nil
    if car.isLegalInAmerica() {
        print("Car is legal: \(car.make) \(car.model)")
    } else {
        print("Car is too old to be registered")
    }
}

// Or with if-let
func displayCarInfo(car: Car?) {
    if let car = car {
        // Safe to use car here
        print("\(car.year) \(car.make) \(car.model)")
    } else {
        print("No car information available")
    }
}

Ruby offers no such safety net:

def process_registration(car)
  # No compile-time guarantee that car exists or has these methods
  if car && car.legal_in_america?
    puts "Car is legal: #{car.make} #{car.model}"
  else
    puts "Car is too old or invalid"
  end
end

# Using the safe navigation operator (&.)
def display_car_info(car)
  puts "#{car&.year} #{car&.make} #{car&.model}"
end

In Ruby, virtually anything can be nil. While you can use the safe accessor (&.) similar to Swift's optional chaining, there's no equivalent to the elegant guard let or if let constructs. You're forced to manually check for nil values, resulting in more defensive coding practices.

Solving the Type Safety Issue

To address these type safety concerns, Ruby developers rely on tools like:

RuboCop - A linter that catches common coding issues
Sorbet - A static type checker designed for Ruby

My current project employs RuboCop, but we're planning to implement Sorbet soon since we've encountered numerous "method not found" errors when objects unexpectedly return nil.

Conclusion: Different But Fascinating

Ruby on Rails is compelling because it provides established conventions that enable rapid development and productivity, even when working with unfamiliar codebases. If you know Rails well, you can join a new company and become productive quickly because they're likely following these same conventions.

The main drawback is the absence of static typing. After working extensively with Swift, TypeScript, and Go, I've grown to appreciate the safety and performance benefits of typed languages. While Ruby may be less performant, for most applications, raw speed isn't the primary concern.

What concerns me more is the potential for bugs arising from the lack of type safety. Fortunately, there appear to be effective mitigations through tools like Sorbet and architectural patterns using delegators and resources.

I'm excited to explore these solutions further in the coming weeks. Stay tuned for more updates on my journey from Swift to Rails!

Spicing Up Your Team Stand-ups with "Question of the Day"

Sun, 02 Mar 2025 00:00:00 GMT

Every day, countless teams worldwide participate in daily stand-ups or scrums. It's that short meeting where everyone shares what they did yesterday, what they plan to do today, and mentions any blockers they're facing.

But let's be honest - these meetings can quickly become routine. People show up, give their updates mechanically, and miss out on a perfect opportunity to actually connect with their teammates.

Enter "Question of the Day"

A few weeks ago, Sean from my team introduced something called "Question of the Day". The concept is simple: whoever moderates the stand-up asks one random, fun question that helps us learn more about each other.

Some examples of questions we've used:

What's your favorite caffeinated drink?
What's your favorite ice cream flavor?
What's your favorite candy?

But it doesn't always have to be about favorites! We also ask things like:

Share a fun fact about yourself
What would you do with an elephant if it were gifted to you and you couldn't sell it or give it away?

The answers can get pretty wild, and that's part of the fun!

How We Organize It

We rotate moderators weekly, which gives each person time to prepare a week's worth of questions. They're free to come up with questions on their own or generate them using an LLM if they want.

Every Friday, we use a spinning wheel to pick the next week's moderator. Once someone has been a moderator, they're removed from the wheel for that rotation. When everyone has had a turn, we reset the wheel and start over.

The Impact

We've only been doing this for several weeks, but I've already noticed we're starting to know each other better outside of work context. It's been great for team morale!

The "Question of the Day" brings a human element to our stand-ups, which is especially valuable in a remote setting where we don't get many chances to see each other in person. Since we have to set up explicit calls to talk, having this built into our existing meetings makes connecting more natural.

I highly recommend giving this practice a try with your own team! Find your own style and questions that work for your group - it's a small change that can make a big difference in how your team connects.

Why I Left Apple to join Boon

Fri, 28 Feb 2025 00:00:00 GMT

When I interview candidates at my new company Boon, I always give them time at the end to ask questions. Without fail, almost everyone asks me the same thing: Why did I leave Apple to join a small startup?

The Apple Years

At the beginning of these interviews, I typically introduce myself - sharing my experience in the industry, previous projects I've built, and mentioning that I spent the last 7 years at Apple where I worked on various projects.

Given how often I'm asked why I left, I've developed a mental script that I can recite almost automatically.

Why I Left Apple

The simple answer? Growth potential.

Apple has grown tremendously since I joined in 2017. When I started, their stock price was around $38 (post-split). Today, it's approximately $230 – that's about a 5x growth in just 7 years. While impressive, I don't have confidence that Apple can achieve another 5x growth in the next 7 years.

Here's my thinking:

Apple now builds really good devices that people don't need to replace as frequently anymore. Think about the leap from iPhone 4 to iPhone 5 – it was such a dramatic improvement that everyone wanted to upgrade immediately. The same thing happened when Apple Silicon replaced Intel chips in MacBooks.

But I'm still using my MacBook M1, and it works perfectly fine. I don't feel any need to upgrade yet. This is actually a testament to Apple's quality, but ironically, it's not great for continued stock growth.

While Apple is focusing on growing its services business, I'm not convinced that will be enough to 5x the company's value in the next 5-7 years.

The Big Tech Dilemma

This growth ceiling isn't unique to Apple. If I joined another Big Tech company, they'd likely face the same challenge - they're already so large that dramatic growth becomes increasingly difficult.

Why Not Other Startups?

During my job search, I actually interviewed with five or six different startups across various industries:

A startup doing voice AI in healthcare
Another in home defense and security
Several startups working on RAG (Retrieval-Augmented Generation) stacks for enterprise, helping companies search and write documents
A few others in different spaces

I deliberately chose not to join these companies for specific reasons:

Security companies: Security often feels like an afterthought for most consumers - we don't typically buy security products because most of the time we feel safe enough. More importantly, I didn't want to be responsible if software I built accidentally harmed someone innocent. That would be terrible, and it's a responsibility I preferred to avoid.

Healthcare companies: Having spent a couple of years at Apple Health, I had firsthand experience with the challenges in this industry. FDA approval processes and selling to insurance companies create incredibly long sales cycles. It's a difficult space with many regulatory hurdles.

RAG stack companies: Everyone is building these right now. There are so many competitors in this space that I couldn't see how a single company could successfully compete against much larger players like OpenAI and Google Gemini, who are also building similar solutions.

Why I Chose Boon

For startups, the biggest challenge is finding product-market fit. You want to enter an industry with:

A reasonable number of competitors (not zero, which would suggest no market)
Not too many competitors
A product with some competitive moat and velocity

What made Boon stand out to me was the team's domain expertise. The CEO and founder, Deepti, came from Samsara, a public company in the transportation and logistics industry. The team has been working in logistics for a while and really understands:

What to sell
Who to sell it to
How to sell it

Boon is building AI software to help logistics and transportation companies optimize their operations, and the team already has deep industry knowledge.

As an engineer, I'm confident in my ability to build things. What I can't necessarily do is sell. This makes for a perfect partnership: they know how to sell, and I know how to build.

My conversations with Deepti have been great, and I'm genuinely enjoying working with her. I believe Boon has the potential to become a major player in the industry within a few years. We just need to execute well.

And that's why I made the switch from Apple to Boon.

Book Review - "Wind and Truth" (Stormlight Archive Book 5)

Thu, 27 Feb 2025 00:00:00 GMT

I just finished listening to the audiobook of "Wind and Truth" yesterday, and I wanted to write up my thoughts while they're still fresh in my mind. This is a massive book—the audiobook is 65 hours long—and it took me almost two months to get through it since I don't listen to audiobooks that much anymore.

My Background with the Series

Before diving into my review, I should mention that I'm a huge fan of this series. I've read all the previous books (one through four) twice, and I even watched a comprehensive three-hour-long recap video on YouTube before starting this fifth book. Suffice to say, I'm a big fan of both the Stormlight Archive and Brandon Sanderson in general, so I had high expectations going in.

The Rating: ★★★½ (3.5/5)

I'm giving this book 3.5 out of 5 stars. It's almost great—almost a 4-star book—but it's riding mostly on the coattails of the previous books in the series. This book by itself has several problems that kept it from reaching the heights of books one and two. Instead, it feels closer to book four, where Sanderson started focusing more heavily on mental health challenges and diving deeper into those themes rather than the epic fantasy action that initially drew me to the series.

The Structure and Plot

The book is structured around a "Contest of Champions" that will take place in 10 days between the god Odium and humanity under Dalinar Kholin. The narrative follows a chronological structure, with each section covering one of these ten days. A lot happens during this short timeframe—that's the whole premise, after all. Something big is coming in ten days.

However, the way the author chose to set up and advance the plot feels suboptimal. I did enjoy learning about humanity's arrival on Roshar and their interactions with the native Singers, who had a more advanced civilization. The introduction of the Unmade and their attempt to seize god-like power was interesting but a bit convoluted. After all, the Unmade were supposedly created by Odium's power—they shouldn't be able to rebel against him and make their own decisions.

Character Arcs

Kaladin and Szeth

The storyline following Szeth and Kaladin as they travel to Shadesmar to figure out what's wrong with the Heralds was, honestly, quite painful to read. It could have been much faster-paced with more action sequences. Instead, there's a lot of Kaladin trying to provide therapy to Szeth and the Heralds. It just felt odd having this therapist-like character attempting to help demigod beings who have lived for thousands of years.

Adolin

The character development for Adolin was the best part of the book. His arc features plenty of action, fighting, and progress as he confronts good and evil. The surprise ending, where Maya breaks through to help Adolin, the Emperor, and the other survivors, was genuinely fun and exciting.

Shallan

Shallan's part was okay, but not great. It didn't leave much of an impression on me.

Dalinar

Dalinar's storyline was slightly more interesting, primarily because it revealed the past of how humanity came to Roshar and how they interacted with the Singers. I felt like this part had potential but wasn't fully realized.

Wit

I would have loved to read more about Wit, but he mostly serves as a character that ties the different books together. There are mentions of Scadrial (the Mistborn planet), suggesting there might be a battle there in the future—perhaps in Mistborn Era 3. This is the kind of cosmere connection I enjoy, but it wasn't explored deeply enough.

What Didn't Work

The book focuses too much on mental health issues, sexuality, and even contract law—the very things many of us read fantasy books to escape from, as we already deal with them enough in the real world. I was hoping for more scenes of flying knights battling villains (or villains who turn out to be good guys), but this book takes a different approach.

The Ghostbloods felt pretty weak in this installment. Their presence didn't add much to the story except to advance certain plot points.

Final Thoughts

"Wind and Truth" is a decent addition to the Stormlight Archive, but it doesn't reach the heights of the first two books. The shift toward deeper psychological exploration comes at the expense of the epic fantasy action and world-building that made me fall in love with the series. While I appreciate Sanderson's commitment to developing his characters' inner lives, I found myself missing the balance of the earlier books.

I'm still invested in the series and will definitely continue reading, but I hope future installments find a better balance between character development and the grand, epic fantasy storytelling that made the Stormlight Archive special in the first place.

My One Month Review of Tesla FSD: The Good, The Bad, and The Quirky

Mon, 24 Feb 2025 00:00:00 GMT

I've been using Tesla's Full Self-Driving (FSD) for about a month now, and I’ve taken every possible opportunity to test it out. Overall, I really like it—I'd say it does what it needs to do about 95% of the time. But, of course, the devil is in the details.

So let’s break this down into three parts: the good, the bad, and the things that make me go "huh?"

🚀 The Good: It’s (Mostly) Like Magic

The best way I can describe FSD right now is that it’s good enough to use daily without much hesitation. For instance, I can drive from my home in San Jose to my office in San Mateo—a commute that ranges from 45 minutes to 1.5 hours—without touching the steering wheel 95% of the time.

That’s the key part: "hands-off 95% of the time." It’s not perfect, but it's really, really close.

Using FSD is simple:

Get in the car.
Set your destination.
Hold down the FSD button.
Let it drive.

Tesla offers three driving modes:

Chill Mode – Supposed to be the smoothest and most relaxed.
Standard Mode – A balance between caution and efficiency.
Hurry Mode – More aggressive, takes tighter gaps, follows cars more closely.

I mostly use Hurry Mode, occasionally switching to Standard if it feels too reckless. I haven't really tried Chill Mode much, but maybe I should.

🤔 The Bad: When It Just Doesn’t Get It

While FSD is impressive, it still has some "what the heck was that?" moments. Here are the biggest issues I’ve run into:

1️⃣ Lane Selection Mistakes

One of the more common mistakes happens near my office. When turning right onto a street with multiple lanes, FSD sometimes picks the wrong one—choosing the rightmost lane, even when it’s a right-turn-only lane. Since I actually need to go straight, I have to take over at that moment. Not a huge deal, but annoying.

2️⃣ The Time It Almost Ran a Red Light

This happened once, but it was unsettling. I was waiting to turn left at a red arrow, and before the light changed, the car started creeping forward like it wanted to go. I had to hit the brakes.

It hasn’t happened again, but this is exactly why you still have to pay attention. You never really know when FSD might do something unexpected.

3️⃣ It Doesn’t See Objects Super Close to the Car

This is an issue when parking. My driveway has a curb and a slope leading to the street. The car doesn’t recognize the curb because it’s too close to the front cameras, so if I start FSD from a parked position, it will just drive straight over the curb.

Now, I always manually drive over the slope first, then enable FSD.

4️⃣ Yellow Light Aggression

Sometimes FSD is a little too eager to beat a yellow light instead of stopping. I suspect this is because I usually have it on Hurry Mode, but even when I switch to Standard, it still tries to push through more often than I’d like.

5️⃣ Randomly Driving in the Express Lane

Even when I don’t want to use the express lane (because, you know, tolls), FSD sometimes moves into it anyway. Switching to Standard Mode usually makes it return to a normal lane, but not always. When it refuses to leave the express lane, I have to take over and manually correct it.

🧐 The Quirks: Tesla’s Attempts to Keep You Engaged

Tesla really wants you to pay attention when using FSD, and I’ve tested the limits of how it enforces this.

1️⃣ Video Calls? Nope.

I once tried to take a work video call while driving, and Tesla immediately knew I was looking at my phone. It gave me two or three warnings, then disabled FSD completely until I parked the car and restarted it. Lesson learned.

2️⃣ Closing My Eyes

I wanted to see if the car could tell if I wasn’t paying attention, so I tried closing my eyes while my wife watched the road. Surprisingly, it didn’t immediately freak out. I didn’t time it, but I could close my eyes for a little while before it started demanding attention.

3️⃣ Sunglasses Trick

Wearing sunglasses seems to confuse the system a bit. Since the car can’t track my eyes, it doesn’t warn me about inattention as much. Instead, it compensates by periodically asking me to wiggle the steering wheel.

So theoretically—not saying you should do this—but if someone wanted to watch a video while using FSD, they could wear sunglasses and just shake the wheel when prompted. (Again, I am not recommending this. Just… you know… as an engineer, I was curious.)

🎯 Final Thoughts: Worth It?

Despite its flaws, I love Tesla’s FSD. It has exceeded my expectations, and I plan to keep paying for it.

My wife? Not convinced. She doesn’t fully trust it, and given the occasional weird behavior, I can’t blame her. But for me, FSD makes commuting effortless. I don’t worry about directions, I don’t stress over traffic—I just turn it on, play a podcast or audiobook, and relax.

It genuinely feels like living in the future. And I’m here for it.

My Journey from a Nissan Leaf to a Tesla Model Y: An Unexpected Switch

Mon, 10 Feb 2025 00:00:00 GMT

I've been driving my 2015 Nissan Leaf for years, but recently it's become clear that I need a replacement. The battery degradation has reached a point where I can't even reliably commute to work anymore. While it supposedly has 65 miles of range, this drops to around 55 miles on the freeway. My 30-mile commute to work eats up about 60% of the battery, and I've developed constant range anxiety about the car dying on me mid-journey.

The Search Process

I decided to look for a used electric car since I've already experienced the steep depreciation of buying new and didn't want to repeat that experience. My search began with opening multiple tabs across various websites like Edmunds, CarMax and Carvana, focusing on specific criteria:

Model year 2023 or newer, as I wanted something relatively recent
Traffic-aware cruise control - a feature my Nissan Leaf lacked that I really wanted
Sufficient cargo space to accommodate my family of four and our travel needs

Making an Objective Comparison

I created a comparison table of different EV options, looking at factors like price, cargo space, and range. Interestingly, this process led me to an unexpected conclusion. Despite my initial skepticism about Tesla due to concerns about build quality and what I perceived as poor value for money, the data told a different story. The used Tesla Model Y emerged as the clear winner, offering:

The highest range in its price bracket
Competitive pricing (alternatives were about $10,000 more expensive)
Sufficient cargo space for our needs
I found one with only 2,000 miles - practically new

The Remote Purchase Experience

I ventured into new territory by deciding to purchase the car from Southern California without seeing it in person. The process involved:

A remote inspection where the dealer sent video recordings of the exterior and interior
Multiple rounds of questions about the vehicle's condition
A straightforward purchase process: wire the money, wait about a week for it to clear
Transport arrangement that took a couple of days
Delivery right to my door via a car carrier at 8 AM as scheduled

The Result

I've been thoroughly enjoying the Tesla Model Y. The improved range and new technology have been great additions to my driving experience. While I haven't tried supercharging yet, I've been experimenting with the Full Self-Driving feature. It's impressive, though it does have some drawbacks that I'll save for another blog post.

How I Easily Turned My Voice Notes into Something Useful

Sat, 08 Feb 2025 00:00:00 GMT

Voice notes have become an essential part of my daily routine, helping me capture thoughts and ideas while staying active. Today, I want to share my experience using the Whisper application on Mac for transcribing these voice notes, and how it compares to Apple's built-in transcription.

My Voice Notes Setup

My preferred method is simple: I take my phone on walks and use the Voice Memos app to record my thoughts. While I initially tried using AirPods, the voice quality was consistently muddled, so I switched to recording directly through my phone's microphone. This setup has proven much more reliable.

Though you can record using a MacBook's microphone, I prefer the walking-and-talking approach. It combines exercise with productivity, and there's something therapeutic about voicing my thoughts while enjoying nature. The stream-of-consciousness style works well for me – no need for perfect organization at this stage.

The Transcription Process

Since I use Apple's Voice Memos app, the recordings automatically sync to my Mac. From there, the process involves a few steps:

Drag and drop the voice notes from the Mac app to a folder
Convert the M4A files to MP3 (Whisper only accepts MP3)
Run the transcription using Insanely Fast Whisper
Extract just the text from the resulting JSON file

To simplify this process, I created an alias that handles everything in one command:

alias ifwa='for f in *.m4a; do ffmpeg -i "$f" "${f%.m4a}.mp3"; done && for f in *.mp3; do insanely-fast-whisper --device-id mps --language en --file-name "$f" --transcript-path "${f%.mp3}.json"; done && for f in *.json; do jq -r ".text" "$f" > "${f%.json}.txt"; done && rm *.m4a *.mp3 *.json'

Breaking Down the Script

Let's break down what this alias does step by step:

Convert M4A to MP3:
```
for f in *.m4a; do ffmpeg -i "$f" "${f%.m4a}.mp3"; done
```
This loop takes all .m4a files in the folder and uses ffmpeg to convert them to .mp3. The ${f%.m4a}.mp3 syntax strips the .m4a extension and replaces it with .mp3.
Run Whisper Transcription:
```
for f in *.mp3; do insanely-fast-whisper --device-id mps --language en --file-name "$f" --transcript-path "${f%.mp3}.json"; done
```
This loop processes each .mp3 file through the insanely-fast-whisper tool. It specifies the device ID (mps for Apple Silicon), sets the language to English, and outputs the transcription in a .json file.
Extract the Transcribed Text:
```
for f in *.json; do jq -r ".text" "$f" > "${f%.json}.txt"; done
```
Here, jq extracts the transcribed text from each .json file and saves it into a .txt file. The -r flag ensures the text is outputted as raw strings without quotes.
Clean Up Temporary Files:
```
rm *.m4a *.mp3 *.json
```
Finally, it deletes all the original .m4a, intermediate .mp3, and .json files, leaving only the final .txt transcription files.

Beyond Just Blogging

This workflow isn't limited to blog writing. I use it for work, personal reflection, and mental clarity. There's something uniquely satisfying about talking through your thoughts as if you're having a conversation with someone.

Post-Processing

Once I have the transcription, I often feed it into an LLM like Claude or GPT to help structure the narrative. This helps transform the stream-of-consciousness recording into more organized, readable content – just like this blog post!

Here’s a sample prompt:

<Role>
Act as a professional ghostwriter & proofreader.

<Task>
- to rewrite my transcription in a much more clearly formatted and easy to digest format, but still in a narrative format instead of summary. 
- focus on cleaning up the rambling, but keep as much information as possible. 
- don't miss any details.
- keep the casual tone
- only write the output and nothing else
- be polite even if the transcription is angry or impolite

<Transcription>

I hope this workflow helps you find new ways to capture and process your thoughts. Sometimes the best ideas come when we're moving and speaking freely, rather than sitting at a desk.

Comparing Whisper and Apple Voice Notes Transcription

To understand the differences between Insanely Fast Whisper and Apple's built-in transcription, I ran the same voice note through both systems and compared the results.

The left is Apple VoiceNotes transcription and the right is insanely-fast-whisper.

Open image in new tab to see it in full size

Observations:

Accuracy: Whisper clearly outperforms Apple Voice Notes in transcription accuracy. It captures complex sentences and maintains coherence, whereas Apple's transcription introduces unusual words like "blockfast" and "Blockpus," and struggles with punctuation and sentence structure.
Consistency: Whisper handles repeated words and filler phrases more gracefully, while Apple's transcription often misinterprets or omits them, resulting in fragmented sentences.
Usability: While both methods sync well with my workflow, the cleaner and more accurate output from Whisper reduces the need for extensive post-editing, making it a more efficient tool overall.

Two Days, $95, and Endless Errors: My Painful Journey to Switch Carriers

Mon, 14 Oct 2024 00:00:00 GMT

So, here’s the story of my phone number porting adventure.

Last week, my phone number is still owned by Apple. To transfer it to my personal line, I had to go through a two-step process. First, I needed to port it from Apple’s business line to a personal line, and then from there, I could port it to whichever carrier I wanted. Sounds straightforward, right? Well, it wasn’t.

The Transfer to Personal Line

I opened a ticket with Apple Wireless support last week, and the wireless team got back to me within 2-3 days—not bad. Then, I got an email from Verizon saying my business line was in the process of being transferred to a personal line. It asked me to click a link to complete the process, so I did. The link took me to a form where I had to enter all sorts of personal information—address, marital status, etc. I filled everything in, hit confirm, and… error. It just kept failing with a vague error message.

Very helpful error.

Phone Call Fiasco

Frustrated, I called Verizon. The rep asked me the same questions I’d already answered online (don’t you love that?) and then told me the port was stuck. I couldn’t complete it over the phone either because there was a “pending port.” He opened a ticket internally and promised to call me back once it was sorted out. Oh, and I asked if I needed to wait a certain amount of time after the transfer to a personal line before porting to another carrier. Apparently, I’d need to wait 30 days.

Of course, I never heard back from him. So, the next day, I called again. This time, I got a different person, and we finally managed to do the transfer over the phone.

The Verizon App Disaster

Once my line was on a personal plan, I downloaded the Verizon app to check my plan. They didn’t ask me what plan I wanted, so I figured I’d better check it myself. I opened the app, and here’s where things got ridiculous. The app was clearly broken—screens were full of placeholder text. It looked like the developers never tested it. Just… wow. Amazing quality control, Verizon.

Don't you love those placeholders?

A couple of days later, I got a message saying I was almost out of data, even though I’d barely used the phone. So, I decided to say “screw it” and initiated the transfer to US Mobile, which is the carrier I wanted from the start.

The Billing Nightmare

Of course, I still owed Verizon $95 for two days of service. I had no problem paying it if it meant I’d never have to deal with them again. But—surprise, surprise—it wasn’t that easy. When I went to add a new payment method in the Verizon app, the button didn’t work. I clicked on the dropdown to select a credit or debit card, and nothing happened. It was just dead.

You would think you can add a new credit card here, right?

Selecting ‘Add a credit card’ does nothing

I had to go into account settings, manually add a payment method, then go back to the pay bill screen to finally settle up. Every single step with Verizon was a mess, whether it was the process itself or their terrible software.

Enter US Mobile: A Breath of Fresh Air

In contrast, porting my number to US Mobile was a breeze. Minimal clicks, all the information I needed was right there, and everything just worked. It was such a refreshing experience after dealing with Verizon’s chaos.

The Moral of the Story

This whole saga feels like a David vs. Goliath tale. Verizon, the Goliath, is this massive, bloated, and inefficient giant. Meanwhile, US Mobile, the David in this scenario, is nimble, smart, and focused. I think David’s going to win—at least, I know I’m rooting for him.

So yeah, if you’re thinking of switching carriers, take it from me: go with the little guy. You’ll save yourself a lot of headaches.

From Flappy Dove to TF Math: Our Incredible Cambrian Code Off Journey

Fri, 03 May 2024 00:00:00 GMT

Dear Grace and George,

Remember a couple of weeks ago when we went to the Cambrian Code Off together? It was so much fun! We got to be a family team, which was really cool because Dad could help you out.

When we got there, we found a seat and waited for everything to start. Some big kids told us that the theme was "education," but you know what? We didn't really care about that. Grace, you wanted to make a game like Flappy Bird, so that's what we did! We found a nice and easy example on the Scratch website, and you changed the code to make it your own. You called it "Flappy Dove," and it had a cute little dove flying and trying not to hit the pipes.

Then, we added a bunch of funny animals as obstacles. First, we put in a dinosaur, and we learned how to flip the pictures and make the dinosaur's mouth face the right way. We even made copies of the dinosaur and flipped them around. After that, you added a frog that shot its tongue out to catch a fly. The game had some silly problems, but it worked, and we had a blast making it!

<aside> 💡 Press ‘Spacebar’ to flap your wings.

</aside>

https://scratch.mit.edu/projects/1005024439/embed

While we were working on the game, Mom kept messaging me, saying that we should do something about education. I told her that I didn't mind if we didn't do education stuff because I just wanted you to have fun learning. But you know how Mom is – we can't ignore her! She had this idea that we could change the Flappy Dove game to be about a girl trying to get to school while avoiding obstacles. Grace, you said a girl can't flap, and when I suggested using the dove, you said, "The dove doesn't need to go to school. It's a bird!"

I asked an AI to give us some ideas for education games on Scratch, and one of them was about math. We searched the Scratch website and found a cool project that used Yoshi and Mario. There were math problems, and you had to choose the right answer by moving Yoshi with the arrow keys.

That's when you had your brilliant idea, Grace! You said we should make a true/false game, and I loved it! We quickly made a blackboard background with true and false buttons on each side. You added a cat and figured out how to make the cat move with the mouse all by yourself. I was so impressed!

I copied the number system from the Yoshi game and made it so that the game would show addition problems with random numbers. The player could move their mouse to click on the true or false button. You added the scoreboard and put it right in the middle. It was perfect!

You named it “TF Math”.

https://scratch.mit.edu/projects/1005041877/embed

We only had two hours to make our games, and we spent 90 minutes on the Flappy Dove game and looking for ideas. But the math game only took us 30 minutes! It was exciting and a little scary because we didn't have much time. Grace, you kept telling me how much time we had left, but I told you not to worry because I knew we would finish it. And we did!

Mom and George came to join us after his baseball game, and we showed her the game. Then, the judges came to look at our game. While we were waiting for the judging to end, we played some other fun games on Scratch. Grace, do you remember Stella? The girl sitting at our desk? She came over and played the Pikachu volleyball game with me!

When the judging was over, the organizers started giving out prizes. They showed us the prizes first, and there was a super cool coding robot. You said you really wanted it. Guess what? When they called out the winner, it was our ticket number! Both of you ran to the front to get the robot. George, you grabbed it and ran back to us so fast! The organizer had to stop you two to take a picture. It was amazing! You wanted the robot, and you got it. And George, you helped out with your quick moves.

Finally, they announced the winners for the individual, team, and family categories. They said who won second place, and it was Stella and her dad! We cheered for them. Then, they called out the first-place winner, and it was us! They said that our game looked like something people would really enjoy playing. Grace, you ran up to the front to get the trophy and the certificate. The trophy was shaped like a light bulb, and it said that you are a "Luminaries of Logic." You were so happy, and we celebrated with Stella and took some pictures together.

We came to the Cambrian Code Off, had an incredible time, and won both the raffle and the game. It was a day we'll never forget!

How Emotions are Made: The Secret Life of the Brain - Book Summary

Thu, 30 Mar 2023 00:00:00 GMT

"How Emotions are Made: The Secret Life of the Brain" is a book by psychologist and neuroscientist Lisa Feldman Barrett. In this book, Barrett presents a new theory of emotions, arguing that they are not hardwired into our brains, but rather, are constructed by our brains based on our experiences and the cultural context.

Chapter 1: Looking for Emotion's "Fingerprints"

Barrett questions the old-school view that emotions are hardwired into our brains and universal across all humans.
She highlights that research hasn't been able to find specific brain activity patterns (or "fingerprints") for each emotion.
For example, people from different cultures might not display or recognize emotions in the same way, which makes us question the universality of emotional expressions.
Instead, she suggests that emotions are constructed by our brains based on our past experiences and the stuff we've learned.
So, think of how we might learn to associate a smile with happiness based on our experiences, rather than being born with that knowledge.

Chapter 2: Emotions are Made, Not Born

Barrett explains that our brains create emotions based on our experiences and the cultural context we live in.
So, instead of having pre-programmed emotions in our brains, we learn to feel and express emotions based on our upbringing and what's considered "normal" in our culture.
Like how in some cultures, people might express grief by wailing loudly, while in others, they might remain silent and stoic.
This idea of constructed emotions helps explain why emotions are experienced and expressed differently across individuals and cultures.
Basically, our brains are like emotion-building machines that use our experiences and the stuff we've learned to create emotions in different situations.

Chapter 3: The Myth of the Emotional Brain

Barrett busts the myth that there are dedicated brain regions for specific emotions (like a "fear center" or a "happiness center").
She shows that research hasn't found consistent brain regions that are responsible for specific emotions.
Instead, our brains use networks of different areas to create emotions, kind of like a team effort.
So, it's like how you might use different ingredients to cook a dish, but no single ingredient is solely responsible for the final result.
In conclusion, emotions aren't neatly organized in our brains, but rather constructed through complex interactions between different brain areas.

Chapter 4: A Fresh Look at Human Nature

Barrett argues that this new view on emotions challenges the traditional idea that humans are governed by innate emotional instincts.
By understanding that emotions are constructed, we can better grasp how people experience and express emotions differently.
This new perspective also helps explain the diversity of emotional experiences across people and cultures.
For instance, imagine how someone who grew up in a culture that values emotional expression might react differently to a situation compared to someone from a culture that values emotional restraint.
The bottom line is that our emotional experiences are shaped by our brains, which use our past experiences and learned concepts to create emotions in different situations.

Chapter 5: Emotions and Health

Barrett explores how emotions are linked to our physical health, like how stress can affect our immune system and make us more prone to illnesses.
She explains that our brains construct emotions based on sensations from our bodies, which can impact our health and well-being.
For example, if we constantly feel stressed and anxious, our bodies might be more susceptible to inflammation and chronic diseases.
So, learning how to manage our emotions effectively can actually have a positive impact on our overall health.
It's like how practicing meditation or other relaxation techniques can help us reduce stress and improve our well-being.

Chapter 6: Mastering Your Emotions

Barrett emphasizes the importance of being able to accurately differentiate and label our emotions, which she calls "emotional granularity."
This skill can help us manage our emotions better, make smarter decisions, and improve our well-being.
For example, if we can recognize that we're feeling frustrated rather than just "upset," we can take more targeted steps to address the situation.
She suggests techniques like mindfulness, journaling, and learning new emotion words to improve our emotional granularity.
So, by understanding that emotions are constructed, we can become more in control of our emotional lives and better equipped to handle whatever life throws our way.

Chapter 7: Emotions and the Law

Barrett looks at how our understanding of emotions can impact the legal system, especially when it comes to eyewitness testimony and lie detection.
She discusses how emotions can influence our memories and perceptions, which could lead to biases and inaccuracies in eyewitness accounts.
For example, imagine being an eyewitness to a crime and feeling scared. Your fear might distort your memory, causing you to misremember certain details.
She also critiques methods like polygraph tests, saying that they're based on flawed assumptions about emotions and their connection to lying.
In the end, she highlights how understanding constructed emotions could help create better legal procedures and more accurate decision-making.

Chapter 8: Emotions in the Digital Age

Barrett examines how technology, like social media and digital communication, affects our emotional lives.
She talks about the limitations of digital communication when it comes to conveying emotions, which can lead to misunderstandings and misinterpretations.
Think of how a text message might be taken the wrong way if the recipient can't pick up on the sender's intended emotion.
She also looks at how emojis and other digital tools can help express emotions more accurately in online communication.
Overall, the chapter discusses the challenges and opportunities of navigating our emotions in an increasingly digital world.

Chapter 9: Building a Better Brain

Barrett wraps up the book by summarizing the main insights from the theory of constructed emotion and their implications for improving our well-being.
She highlights the importance of developing emotional granularity and effective emotional regulation for better mental and physical health.
By incorporating these ideas into education, therapy, and mental health interventions, we can potentially help people lead happier and healthier lives.
She also emphasizes the need to understand cultural differences in emotional experiences and expressions, as well as the impact of technology on our emotional lives.
In the end, the book encourages us to apply these insights to our own lives, empowering us to take control of our emotional experiences and build better brains.

I hope you found it as interesting as I did.

Remember, our brains construct our emotions, which means we have the power to shape and control our emotional lives!

When ChatGPT Goes Wrong - 3 Examples

Mon, 12 Dec 2022 00:00:00 GMT

We have seen tons of impressive examples of how ChatGPT wows and bedazzles with its seemingly superhuman knowledge and writing ability.

But let’s take a look at the other side and see how it can fail just as spectacularly.

Task 1: Comparing Bed Sizes

The AI does not understand numbers the way we do. It treats numbers like words, and as a result, makes silly mistakes that my 5-year-old could call out.

For example, when asked to compare bed sizes, the AI may say that 108 inches is narrower than 76 inches, which is clearly not true.

Task 2: Making a Topical Dad Joke

I planned to go to Tahoe last weekend but had to cancel because of the impending snow storm and avalanche risk. So I asked the AI to make a joke about it.

Unfortunately, the AI made a spurious connection between snowmen and avalanches, and then it hallucinated the double meaning of blizzard. The result was a joke that made no sense.

Task 3: Writing Python Code to Solve Titanic Dataset with Conformal Predictions

When asked to solve the famous Titanic problem with logistic regression, the AI was able to generate a pretty plausible code.

As a follow-up, I asked it to use conformal prediction instead of logistic regression, and it started to hallucinate.

ChatGPT suggested that we import scikit-cp, which stands for Computational Physics, not Conformal Prediction. It looks like it just made the assumption that the package could solve the problem and wrote the code without understanding what it was doing.

Summary

In conclusion, ChatGPT is a powerful tool that can generate impressive results, but it is not perfect. It has some limitations that can result in silly mistakes and nonsensical outputs. Some of these limitations include:

The AI does not understand numbers the way humans do, and can make mistakes when comparing sizes or quantities.
It can make spurious connections and hallucinations when asked to generate jokes or creative content.
It may import the wrong packages or use the wrong algorithms when asked to write code, if it does not understand the task properly.

Despite these limitations, ChatGPT is still an impressive tool that can be used for many useful applications. It just needs to be used with care and understanding of its limitations.

Zebra Floss Poem

Mon, 05 Dec 2022 00:00:00 GMT

Zebras with stripes so bold and bright Are known for their speed and their sight But there's something they often neglect Their dental care is often a suspect

They roam the savannah, wild and free But their teeth can get quite cavitied So they must take the time to floss To keep their mouth healthy, no matter the cost

With their tough, fibrous grasses to chew Their teeth can get quite yellow and blue But with some floss, they can keep them clean And avoid any dental scenes

So let's all take a lesson from the zebra herd And make sure to floss, it's not just a word Your teeth will thank you, I guarantee Flossing is important, as zebras can see!

—

Poem written with the help of ChatGPT

Zebra photo by yours truly

Moving From Markdown+Hugo to Notion+NextJS

Fri, 02 Dec 2022 00:00:00 GMT

After years of writing my blog in markdown and publishing it with Hugo, I have recently decided to switch to using Notion and NextJS. In this blog post, I want to share my experience and explain why I made this switch.

Pain Point

One of the biggest pain points with using markdown and Hugo was that I couldn't easily add images to my blog posts. I had to upload the images separately and then insert them into my markdown files using the appropriate syntax. This was time-consuming and painful, and it often resulted in broken links and missing images. With Notion, I can simply drag and drop an image into my blog post, and it will be automatically uploaded and inserted into the right place. This makes it much easier and more intuitive to add images to my blog posts.

Why Notion

One of the main reasons I decided to switch to Notion is because of its rich formatting options. Unlike markdown, which is a simple and plain text-based format, Notion allows me to add formatting, images, videos, and other rich media to my blog posts. This makes my writing more engaging and visually appealing, and allows me to express my ideas in a more creative and effective way.

Another advantage of using Notion is its ability to collaborate with other people. With markdown, I had to send my blog posts to other people as plain text files, which they would then have to edit and send back to me. This was a cumbersome and error-prone process. With Notion, I can share my blog posts with other people and work on them together in real-time. This makes it easier for me to get feedback and suggestions from other writers, and to improve the quality of my blog posts.

Why NextJS

In addition to the benefits of using Notion for writing, I also decided to switch to NextJS for publishing my blog.

One of the coolest features of NextJS is its support for Incremental Static Regeneration (ISR). This means that when I write a new blog post in Notion, NextJS will automatically detect the change and update the corresponding static page on my blog without me having to manually push a new commit to Git and rebuild the website. This allows me to write and publish blog posts quickly and easily, without having to worry about the technical details of building and deploying my blog.

Craigary on GitHub has a great template to get started: Nobelium

Migration Challenge

I have not finished migrating all of my old blog posts yet, so for now, they are still available on my subdomain https://hugo.augusteo.com

Moving from markdown and Hugo to Notion and NextJS was not without challenges.

The biggest challenge I am facing now is migrating my old blog posts from markdown to Notion.

This involves converting the markdown files to Notion pages, preserving the formatting and images, and organizing them into a logical structure.

Let’s see how it goes.

Walkable City by Jeff Speck - Book Club Discussion Points

Thu, 06 May 2021 00:00:00 GMT

My book club at work will discuss this book, and I’m sure it will be an exciting session.

Does this book convince you that a walkable city is something that every city should strive to be?
Do you currently drive a car? What would it take for you to ditch your car?
The author is against traffic study and believes that any increased supply will quickly be outpaced by increased demand. Do you agree with this?
Speck wrote very highly of the parking guru Donald Shoup and his 80% target occupancy rate. Why do you think it is?
Why did Speck recommend private parties to help pay for public transport? Do you agree with it?
Speck wrote, “nobody drove dangerously through this intersection, precisely because the intersection felt dangerous.” Would you support making your intersection feels more dangerous to reduce accidents potentially?
What was the most walkable city that you’ve ever visited? Does it map to Speck’s point about “Pedestrians need to feel safe and comfortable, but they also need to be entertained.”?
In the end, we need to pick our winners. Where can spending the least money make the most difference in your city?

How to Build Responsive Newsletter Signup Box for Hugo

Mon, 03 May 2021 00:00:00 GMT

This blog post focuses on Hugo, but you can use the same technique for other static site generators.

Starting last week, I moved my newsletter hosting from Moosend to Revue. Moosend came with its newsletter sign-up box with its builder. Revue does not. So I thought to create one by myself.

How hard can it be?

I started by just searching for a simple subscription box template on the internet and found this one. It has the structure that I wanted.

We will use that as the base and change it to suit our needs.

Make it responsive
Change the color palette
Implement the change on Hugo blog with partials

Making it responsive

To make it responsive, all I had to do was change:

form {
    width: 450px;
}

with:

form {
    width: 80%;
}

Choosing color

I used the Mermaid Lagoon color palette from Canva.

If you don’t like this one, you can just pick another from the list.

There are two places to change: the main embed and the submit button.

.revue-embed {
    background: #0c2d48;
}

#member_submit {
    background: #2e8bc0;
}

Hugo implementation

I’m using cocoa-eh as my theme, so please change your theme path accordingly.

First, we need to create a partial file to contain our form code.

site/themes/cocoa-eh/layouts/partials/newsletter_signup.html

<!-- Newsletter Signup -->
<style type="text/css" media="screen">
    {{ partial "css/newsletter.css" . | safeCSS }}
</style>
<link href='https://fonts.googleapis.com/css?family=Lato:400,700' rel='stylesheet' type='text/css'>

<div id="revue-embed" class="revue-embed">
    <form action="http://digest.augusteo.com/add_subscriber" method="post" id="revue-form" name="revue-form"
          target="_blank">
        <div class="revue-header">
            <p>Get your free weekly digest</p>
        </div>
        <div class="revue-description">
            <p>Once a week, I will send you a highly curated weekly digest on:</p>
            <br>
            <p>Science/Technology, Software/AI, Business/Finance, and Culture/Fun.</p>
            <br>
            <p>The easiest way to learn the most salient content every week.</p>
        </div>
        <div class="revue-input">
            <input type="email" class="button" name="member[email]" id="member_email" placeholder="Your email address...">
            <input type="submit" class="button" name="member[subscribe]" id="member_submit" value="Subscribe">
        </div>
    </form>
</div>

Next, we need to create css file for the styling.

site/themes/cocoa-eh/layouts/partials/css/newsletter.css

.revue-embed {
    background: #0c2d48;
    font-family: 'Lato', sans-serif;
    color: #FDFCFB;
    text-align: center;
    border-radius: 10px;
}

form {
    width: 80%;
    margin: 0.5em auto;
}

.revue-header {
    font-size: 26px;
    text-transform: uppercase;
    letter-spacing: 2px;
    padding-top: 0.5em;
    padding-bottom: 1em;
}

.revue-description {
    font-size: 15px;
    letter-spacing: 1px;
    line-height: 1.3em;
    margin: -2px 0 45px;
}

.revue-input {
    display: flex;
    align-items: center;
    padding-bottom: 2em;
}

.button {
    height: 44px;
    border: none;
}

#member_email {
    width: 65%;
    font-size: 14px;
    background: #FDFCFB;
    font-family: inherit;
    color: #737373;
    letter-spacing: 1px;
    text-indent: 5%;
    border-radius: 5px 0 0 5px;
}

#member_submit {
    width: 35%;
    height: 46px;
    font-size: 14px;
    background: #2e8bc0;
    font-family: inherit;
    font-weight: bold;
    color: inherit;
    letter-spacing: 1px;
    border-radius: 0 5px 5px 0;
    cursor: pointer;
    transition: background .3s ease-in-out;
}

#member_submit:hover {
    background: #d45d7d;
}

input:focus {
    outline: none;
    outline: 2px solid #E86C8D;
    box-shadow: 0 0 2px #E86C8D;
}

Once that’s done, all we have to do is to add the partials where we want the box to show up.

For example, I want the box to show up at the bottom of every blog post. So we need to add this partial:

{{ partial "newsletter_signup.html" . }}

To just below the blog post on:

site/themes/cocoa-eh/layouts/blog/single.html

That’s it. Just run the server and verify that it works!

Weekly Digest 15 - HQ Audio Makes You Smarter, NFT Gambling, FLoC, and Soft Robots

Sun, 18 Apr 2021 00:00:00 GMT

Must Read

Bad quality audio makes us think less of the speaker and what they’re saying. Ensure your audio is high quality in calls, online meetings, interviews, podcasts, or any other voice recording. When audio quality is high (vs low), people judge the content as better and more important. They also judge the speaker as more intelligent, competent, and likable.

I bought a Blue Yeti microphone and it had served me well throughout the pandemic.

Science & Technology

The study recruited three groups of students who took one of the two online courses. All participants took the courses between August 2018 and August 2020. The authors examined the participants’ well being both before and after taking one of the two courses. To do so, they used a psychometric self-report measure called the PERMA Profiler. In terms of their overall average score on the 15-item measure, the “science of well being” students went from a baseline average of about 6.6 to an average of 7.7 after completing the course. The “introduction to psychology” students, on the other hand, went from a baseline of 6.7 to a post-course average of about 7.3. Still an improvement, but a significantly smaller improvement.

By doing video surveillance deep in the ocean, researchers working in the North Pacific have discovered the densest accumulation of plastic waste ever recorded on an abyssal seafloor, finding that the majority of this waste is single-use packaging.

This self-extending soft robot has applications to archaeology, space exploration, and search and rescue — with a simple elegant design inspired by a plant.

Scientists in the US have developed a paint significantly “whiter than the whitest paint currently available”. Tests carried out by researchers at Purdue University on their “ultra-white” paint showed it reflected more than 98% of sunlight. That suggests, the scientists say, that it could help save energy and fight climate change.

Some experts say tooth-grinding is a behavior rather than a disorder, and the dentist’s chair isn’t the best place to address it.

Business & Technology

Goodreads enjoys a monopoly in the space of book social networks. Looking at data from Similarweb, Goodreads records 95mn visits per month, whereas its closest competitor, LibraryThing, is less than 1.5mn visits per month. The biggest moat is their superiority in organic search.

According to an internal memo published by BuzzFeed, Facebook plans to build “a version of Instagram that allows people under the age of 13 to safely use Instagram for the first time. Instagram, in particular, exploits young people’s fear of missing out and desire for peer approval to encourage children and teens to constantly check their devices and share photos with their followers. The platform’s relentless focus on appearance, self-presentation, and branding presents challenges to adolescents’ privacy and wellbeing.

Welcome to the non-fungible, memeified, cryptodenominated, degenerate future of finance. What is happening? It’s tempting to blame the money mutations of the past year on the coronavirus and to see the financial bets on Robinhood and the soaring prices of NFTs as the anguished outcries of the bored, locked down, and anhedonic. Roiling under this familiar surface is the reverberation of a much larger, fundamental shift: Something is changing in how we think about money. Maybe the question we should have been texting our financially literate friends wasn’t “What is an NFT?” but “What is money now?”

BitClout is pitching itself as a “crypto celebrity network.” The site offers consumers the opportunity to purchase tokens tied to the identities of 15,000 influential Twitter accounts, including the likes of Elon Musk and Katy Perry—albeit without their permission.

In the space of a week, BitClout has reportedly earned $160 million and attracted the interest of prominent investors and cultural figures. But already, crypto security firms are pointing to suspicious transactions patterns, and warning the site has all the appearances of a scam.

Software & AI

Google is leading the charge to replace third-party cookies with a new suite of technologies to target ads on the Web. Federated Learning of Cohorts (FLoC) is a new way that browsers could enable interest-based advertising on the web, in which the companies who today observe the browsing behavior of individuals instead observe the behavior of a cohort of similar people. The technology will avoid the privacy risks of third-party cookies, but it will create new ones in the process. It may also exacerbate many of the worst non-privacy problems with behavioral ads, including discrimination and predatory targeting.

Ward describes debt as the natural result of writing code about something we don’t have a proper understanding of. But how do we land to such disagreement? In the author’s experience, there are two offenders:

🎨 Wrong Design — what we built was wrong from the start!
🏃 Rapid Evolution — we built the right thing, but the landscape changed quickly and made it obsolete.

Learn the history and evolution of same-origin policy and CORS, understand CORS and the different types of cross-origin access in depth, and learn (some) best practices.

An estate agent has apologised after a 3D tour of a house for sale in Devon was published with a substantial amount of personal information visible. Financial paperwork in the study could be read by zooming in on the image. It included a shares dividend cheque, an insurance policy document and an invoice for a stairlift. Some family photos had also been left unblurred.

Culture & Fun

Easy 5 star book for me. The Silent Patient is a shocking psychological thriller of a woman’s act of violence against her husband—and of the therapist obsessed with uncovering her motive.

Starting with an old cell phone a friend gave him, 17-year-old Steven Ortiz of Glendora, CA, used Craigslist to trade up 14 times over two years and eventually ended up with a Porsche Boxster. Here’s how he did it.

Known simply as “her,” she would become the star of My Octopus Teacher, the Oscar-nominated Netflix documentary and surprise pandemic hit that told the story of Foster’s unlikely relationship with that eight-armed mollusk.

Video walk version of GeoGuessr. Watch a video walk in a city and guess where it is on a map!

The Australian government had to assert that COVID-19 vaccines do not – and cannot – connect you to the internet.

Weekly Digest 14 - Neuralink MindPong, Muon g-2, and Foot Fetish

Sun, 11 Apr 2021 00:00:00 GMT

Must Read

Neuralink is developing a fully-implanted, wireless, high-channel count brain-machine interface (BMI) with the goal of enabling people with paralysis to directly use their neural activity to operate computers and mobile devices with speed and ease. You can watch the video on how they enabled a macaque monkey, named Pager, to move a cursor on a computer screen with neural activity using a 1,024 electrode fully-implanted neural recording and data transmission device, termed the N1 Link.

Science & Tech

Jorge Cham, aka, PHD Comics, illustrates the excitement over the muon anomaly results in a set of cartoons he made for Physics.

Having a responsive, supportive partner minimizes the negative impacts of an individual’s depression or external stress on their romantic relationship, according to research by a University of Massachusetts Amherst social psychologist. The study found that being a responsive partner – one who focuses effort and energy to listen to their partner without reacting, tries to understand what’s being expressed and be supportive in a helpful way, and knows what their particular partner needs – is in general associated with better relationship quality, “which is what you would think,” Pietromonaco says.

Feasts your eyes on these amazing photographs of ants.

A review of Drink?: The New Science of Alcohol and Your Health by Professor David Nutt. Alcohol use is one of the top five causes of disease and disability in almost all countries in Europe. At least one in five cases of dementia is caused by alcohol. It is also the third leading risk factor for death and disability after smoking and obesity. The positive effect on cardiovascular health has never been definitely proven.

Some drugs classified as highly dangerous are less harmful than alcohol or tobacco. Why is alcohol legal but many other intoxicants not?

Coronavirus-related conspiracy theories (CT) have been found to be associated with fewer pandemic containment-focused behaviors. It is therefore important to evaluate associated cognitive factors. People who believe in COVID-19 conspiracy theories have the following cognitive biases: jumping-to-conclusions bias, bias against disconfirmatory evidence, and paranoid ideation, finds a new German study (n=1,684).

Business & Finance

Story of Missive, an innovative but somewhat hard-to-define email client. They reached US$1M in annual recurring revenue (ARR) last week.

Nearly 1.5m companies are incorporated in Delaware. How did this tiny state become a mecca for corporate activity? The answer is, of course, tax loopholes.

“In sharp contrast to regulators’ concerns that WSB investment advice is harming retail traders, our findings suggest that both WSB posters and users are skilled,” the study concluded.

PMC is what patent lawyers call a “non-practicing entity,” or NPE. The company has acquired over 100 patents that date back to applications from the 1980s, and it uses them to demand money from companies that do stuff—mostly internet companies.

Three factors weigh heavily in any unionization election: the outrageously vicious behavior of employers—some of it illegal, most fully legal—including harassing and intimidating workers, and telling bold lies (which, outside of countries with openly repressive governments, is unique to the United States); the strategies and tactics used in the campaign by the organizers; and the broader social-political context in which the union election is being held.

U.S. income inequality has varied inversely with union density over the past hundred years. The research finds consistent evidence that unions reduce inequality, explaining a significant share of the dramatic fall in inequality between the mid-1930s and late 1940s.

Software & AI

The Supreme Court surprised everyone with its API copyright ruling. By a 6-2 vote, the nation’s highest court held that Google’s copying of Oracle’s Java API was fair use. The ruling means Google won’t have to pay billions of dollars in damages to Oracle. It also has huge implications for the broader software industry. “It’s a minor miracle we got an opinion that reached the right result with pretty sound reasoning despite not one of the justices being able to explain how an API works,” Grimmelman said.

Git has a reputation for being confusing. Users stumble over terminology and phrasing that misguides their expectations. The author believes that Git becomes understandable if we peel back the curtain and look at how Git stores your repository data. After we investigate this model, we’ll explore how this new perspective helps us understand commands like git cherry-pick and git rebase.

Experts are raising doubts that artificial intelligence was used to create a video that police are calling a “deepfake” and is at the center of an ongoing legal battle playing out in Pennsylvania. Several experts have come forth and said that the vaping video did not appear to be artificially-generated.

The author breaks down the setup he uses to run a SaaS. From load balancing to cron job monitoring to payments and subscriptions. The business is https://panelbear.com/

Culture & Fun

“We’re receiving complaints about too much TV coverage of the death of HRH Prince Philip, Duke of Edinburgh,” reads a statement on the BBC Complaints page, which invites disgruntled viewers to submit an email address to register a complaint.

A surprisingly revealing interview with a foot fetishist. e.g.What makes a foot attractive to you?

I like the painted toes. I like an arch, the more pronounced the better. I’m kinda weird with the toes, I like a rounded big toe. If it’s more square it’s okay, but the rounded is better. I definitely like the soles. But I like the arches, that gets you turned on.

Weekly Digest 13 - NFTs, Facebook Hack, Microdosing, Inside istheshipstillstuck.com and others

Sun, 04 Apr 2021 00:00:00 GMT

Must Read

NFTs are cryptocurrency tokens that represent the ownership in a digital asset. Like all digital assets, NFTs can be sent across the world immediately and require little to no physical storage. But unlike other digital assets, they are unique, cannot be duplicated, and can only be transferred by their owners. That’s why they’re used to represent ownership of digital assets. It got popularized recently because someone paid $69 million for an NFT, and everyone else is scratching their heads wondering if these people have gone insane, or if it’s something they should pay attention to.

Science & Tech

Fascinating insight into how objects synchronize and how this affects everything from metronome, bridge building, to the celestial bodies like our moon.

Great visualization to see how much of an anomaly year 2020 was. Lots of excess death on a stacking line graph.

The personal data of over 500 million Facebook users has been posted online. The data includes phone numbers, full names, location, email address, and biographical information.

Female bots are perceived to have more positive human qualities, such as warmth, experience and emotion, than male bots, and this greater humanness leads consumers to prefer female artificial intelligence, a new study has found.

Of the 93% of children who play video games, up to 40% opened loot boxes
About 5% of gamers generate half the entire revenue from the boxes
Twelve out of 13 studies on the topic have established “unambiguous” connections to problem gambling behaviour
Young men are the most likely to use loot boxes - with young age and lower education correlating with increased uses

Microdosing involves taking a low dose of a psychedelic drug to enhance performance, or reduce stress and anxiety. The 24 participants experienced personality changes after a period of microdosing. They found a decrease in neuroticism and an increase in conscientiousness (people who are highly conscientious tend to be diligent, for example). Interestingly, a greater amount of experience with microdosing was associated with lower levels of neuroticism among the 76 participants.

The author of the article found that the book consistently overstates the problem of lack of sleep, sometimes egregiously so. It misrepresents basic sleep research and contradicts its own sources.

It’s pretty much impossible to stop companies from collecting data about you — even if you don’t use their services, they still have their ways. But you can at least limit how algorithms use it against you. There are also more private browsers to limit data collection and prevent sites from linking you to your past visits or data. Or you can just stop using those services entirely.

It seemed that patients were diagnosed with lung cancer at a surprisingly higher rate at 65 years old than, say, at 64 or 66. The researchers found a substantial rise nationwide in new cancer diagnoses at 65 — not only for lung cancer but also for breast, colon and prostate cancer. The four are the most common cancers in the United States.

In 2020, 47% of U.S. adults belonged to a church, synagogue or mosque
Down more than 20 points from turn of the century
Change primarily due to rise in Americans with no religious preference

Business & Finance

[Instagram Study] 102,700,573 Posts Show How Instagram Engagement Rate Decreases Back to Its Initial Values From 2019

Higher Instagram engagement is mainly brought by carousels, especially for small accounts. Shorter captions, up to 10 words, combined with carousels form the posts to which users prefer to interact with. Carousels are getting more likes than any other media type. People are more open to comment on video posts across all profile sizes.

Roth IRA is the best way to take advantage of current low income tax rates and set yourself up for years of tax-free income in retirement. However, people with high incomes are not allowed to contribute to a Roth IRA directly. For them the only way into a Roth is through the back door – with a strategy appropriately known as the backdoor Roth.

The COVID-19 pandemic accelerated digital adoption, especially in grocery shopping and healthcare, and this is expected to continue. Second, the pandemic and lockdowns reversed the long-standing trend of declining money and time spent at home, leading to “home nesting.” We expect this behavior to stick as some portion of high-income households will prefer to work from home to some degree after the pandemic and low-income households retain low-cost at-home alternatives such as digital entertainment.

Most people who are rich chose their parents wisely. Bill Gates might not have ever figured out 1960s-style computer science but he had the foresight to pick a father who is one of the richest, most prominent lawyers in the state of Washington. And before he and Paul Allen made the deal with IBM that gave them a monopoly on the PC operating system, Bill had the foresight to choose a mother who was personally acquainted with John Opel, CEO of IBM Corporation. None of this would have worked if Bill hadn’t been willing to take tremendous personal risks. Should Microsoft have failed, of course, Bill Gates would have had nothing to fall back on but a million dollar trust fund from his mother’s parents (bankers) and the resumption of his degree program at Harvard College.

This is an account of running istheshipstillstuck.com. The website received around 50 million views in a 5 day period. That puts it roughly on par with the New York Times. The author tried many fun ways to monetize the traffic but didn’t have much success.

A headline isn’t just an article title. It’s a tiny window of opportunity to connect with your audience. Top 5 insights:

Instructional headlines drove the most Facebook engagement.
Curiosity headline phrases – ie. those that hinted, teased or questioned something – tended to fare a lot better on Twitter.
The top headline phrase on Facebook had 590x more average shares than the top headline phrase on Twitter.
The ideal headline length is 11 words and 65 characters, according to the most shared headlines on both Facebook & Twitter.
The top Facebook headlines are no longer published by low-quality entertainment publications, but instead by authoritative news sites.

Software & AI

The basic functionality of fzf is very simple: it reads a set of lines from stdin, provides a user interface to select one or more lines, and writes the selected lines to stdout. This sounds very basic, but it is in fact very powerful.

Columbia’s Aho and Stanford’s Ullman Developed Tools and Fundamental Textbooks Used by Millions of Software Programmers around the World. Two of their most widely celebrated books include: The Design and Analysis of Computer Algorithms (1974) and Principles of Compiler Design (1977)

Culture & Fun

I have never seen this method of learning foreign language before. Very novel!

Marc Feren Claude Biart, an alleged member of the ‘Ndrangheta criminal organization based in southern Italy, reportedly hid his face in the cooking videos but failed to hide his tattoos, leading to his identification.

Weekly Digest 12

Sun, 28 Mar 2021 00:00:00 GMT

World News

It ‘Might Take Weeks’ To Free Ship Stuck In Suez Canal. Meanwhile, shipping traffic is quickly backing up at both ends of the canal, which normally sees about 50 cargo ships transit each day. Several dozen vessels, including other large container ships, oil and gas tankers and bulk carriers, are creating what is being described as one of the worst maritime traffic jams in years. Meanwhile, an Evergreen truck is blocking the highway in Taiwan too.

Old story from 2013.

“The Egyptian government endeavors to require as many people as possible to be involved in a single ship’s passage through the Suez Canal. Over the course of ten hours and a mere 200 kilometers, we had seven different pilots come on board the CC Rigoletto. They worked in pairs but it seemed that one pilot did all the work while the other was only present to argue, talk loudly over their walkie-talkie, take personal calls and generally create significant noise.”

Watching Anthony Fauci on Fox News makes people more willing to engage in pandemic reducing behaviors, study finds

In an initial survey of 415 individuals, the researchers found that those with warmer feelings towards scientists were more concerned about COVID-19 and engaged in greater preventative behaviors, such as social distancing. This was true even after controlling for cognitive ability, education, gender, income, and political ideology.

Science & Tech

China’s companies have been unsuccessful in their bid to match TSMC’s manufacturing prowess, but the US has also started to struggle: Intel is set to outsource some production of processors, its crown jewel, to the Taiwanese company. In Washington, the Pentagon has been quietly pressing for the US to invest more in advanced chipmaking so that its weapons are not dependent on foreign manufacturers.

This elegant study provides new evidence that people who are fluent in more than one language have some protection against dementia. Brain scans showed that lifelong bilinguals have stronger connections between certain brain areas compared to those who only speak one language – this appears to allow their brains to cope better with damage before they start to show outward signs of dementia.

Folding A4 paper to quarks and back again to all of the observable universe.

Despite being far more selective, women still match more frequently than men on Tinder

“Women match with 36% of those they like while men match with less than 2%”

Penises are shrinking and genitals becoming malformed because of pollution, an environmental scientist has warned in a new book detailing the challenges facing human reproduction. Her book, titled Count Down, examines “how our modern world is threatening sperm counts, altering male and female reproductive development, and imperilling the future of the human race”.

The researchers examined the correlation between trends for forest cover, plantations, population and disease around the globe using statistics from international institutions such as the World Health Organization, the World Bank, the Food and Agricultural Organization and the Gideon epidemic database. Over the period of study from 1990 to 2016, this covered 3,884 outbreaks of 116 zoonotic diseases that crossed the species barrier and 1,996 outbreaks of 69 vector-borne infectious diseases, mostly carried by mosquitoes, ticks or flies. The paper shows outbreaks increased over time, while plantations expanded rapidly and overall forest cover declined gradually. By itself, a correlation is not proof of causality because other factors may be involved, such as climate disruption.

When compared with our closest primate relatives, we have incredibly large brains. The researchers went on to identify a gene that is crucial to the process. Known as Zeb2, it switches on later in human tissue, allowing the cells to divide more before they mature. Tests showed that delaying the effects of Zeb2 made gorilla brain tissue grow larger, while turning it on sooner in human brain organoids made them grow more like the ape ones.

A study shows that harsh parenting practices in childhood have long-term repercussions for children’s brain development.

Repeatedly getting angry, hitting, shaking or yelling at children is linked with smaller brain structures in adolescence, according to a new study published in Development and Psychopathology.

Software & AI

Little-known company Flock has expanded from surveilling individual neighborhoods into a network of smart cameras that spans the United States. The cameras, which are sold to law enforcement, homeowners associations, and businesses, can automatically record when a “non-resident” vehicle drives into a community, and alert police to cars on a hotlist. Communities have created “virtual gates” around their neighborhoods, with cameras capturing each vehicle driving in and out of the area. Through a program called TALON, this little-known company is allowing police officers to track cars—and by extension, specific people—outside of their own jurisdictions.

I use Vercel to host my foundation website. “Vercel provides a good solid solution for Serverless Functions and makes the process of their creation seamless and hassle-free. Cloudflare Workers offer more functionality out-of-the-box (e.g. key-value data store, CRON) and look more mature and sophisticated.”

Todd Weaver pwns an IoT lamp.

77.2% of Google mobile search ends with zero click. Google is reaping the rewards but depriving websites of the valuable visits.

Business

The 2021 Early-Retirement Update

Six years update from someone who retired early at 38yo with a $1m asset.

How some of the top independent writers are making more than $78,400/month on Substack.

John O’Nolan of Ghost ($3.3M/year) talks creators, decentralization, no-code

The non-profit blogging platform Ghost is now doing over $3 million a year. That’s up from $1.5 million this time last year, and the company’s never received any funding. Moreover, that’s annual recurring revenues from the subscriptions!

Culture & Fun

No life, no hobbies, burnout, lost childhood — the price students pay for a prized IIT seat

Last year, 1.5 million students took the JEE to qualify for 13,000 seats in 23 IITs across the country – in other words, for each seat there were 115 aspirants. So intense is the pressure and so gruelling is the preparation required that students as young as 14 start the process, often missing out on the simple joys of adolescence. Most give up extra-curricular activities, relationships with friends and peers, and all forms of entertainment to achieve the goal. By the time they achieve their aim, if they do, many realise they have lost out on social skills, ability to communicate easily with others (an attribute now known as soft skills), and of course, some part of their youth.

An underrated family movie from the geniuses at Studio Ghibli. A love story between a girl who loves reading books, and a boy who has previously checked out all of the library books she chooses.

How UX designers describe children.

‘Salmon chaos’ in Taiwan as people change their names to get free sushi

The temporary deal from “Taiwan Sushiro” offered free all-you-can-eat sushi to anyone with “gui yu” — the Chinese characters for salmon — in their name.

Weekly Digest 11

Sun, 21 Mar 2021 00:00:00 GMT

Science & Technology

Study suggests that men and women actually prefer not to split household and childcare tasks equally

A new study suggests that the unequal division of household and childcare tasks within partnerships may reflect men’s and women’s actual preferences. For example, the study found that women enjoyed childcare tasks more than men did and also reported a greater desire for responsibility for these tasks compared to men. Men enjoyed tasks to do with outdoor labor and home maintenance more than women did. Women preferred cleaning, food prep, family scheduling, and home decorating. Moreover, this pattern mirrored the way the participants wanted these tasks to be divided — women tended to prefer men to take care of home maintenance tasks and men preferred women to take care of home decorating tasks.

This research is an expansion of an earlier study showing that a vitamin D deficiency (less than 20 ng/ml) may raise the risk of testing positive for COVID-19. In the current study, those results were further supported, finding that individuals with a vitamin D deficiency had a 7.2% chance of testing positive for the virus. A separate study recently found that over 80% of patients diagnosed with COVID-19 were vitamin D deficient.

Generalists vs specialists is not a new debate. There are areas where specialists have an advantage. Specialists do well in fields where there is one clear path to success. The world often hands us what are called “wicked problems”. These are settings where the rules of the game are unclear or incomplete. The patterns do not repeat and feedback is often delayed or misleading. The world is increasingly beginning to sound close to that description.

Milky Way, 12 years, 1250 hours of exposures and 125 x 22 degrees of sky

It took nearly twelve years to collect enough data for this high resolution gigapixel class mosaic image of the Milky Way. Total exposure time used is around 1250 hours between 2009 and 2021.

More than fifty published studies have documented meditation-induced mental health problems, including mania, dissociation, and psychosis. In 2012, leading meditation researchers in the United Kingdom published a set of guidelines for meditation instructors, noting “risks for participants,” including depression, traumatic flashbacks, and increased suicidal ideation.

Atlantic Ocean circulation at weakest in a millennium, say scientists

Decline in the system underpinning Gulf Stream could lead to more extreme weather in Europe and higher sea levels on the US east coast. The Atlantic Ocean circulation that underpins the Gulf Stream, the weather system that brings warm and mild weather to Europe, is at its weakest in more than a millennium, and climate breakdown is the probable cause, according to new data.

Software & AI

A gaping flaw in SMS lets hackers take over phone numbers in minutes by simply paying a company to reroute text messages. “Looking down at my phone, there was no sign it had been hacked. I still had reception; the phone said I was still connected to the T-Mobile network. Nothing was unusual there. But the hacker had swiftly, stealthily, and largely effortlessly redirected my text messages to themselves. And all for just $16.”

Imagine turning a 10-megapixel photo into a 40-megapixel photo. The term “Super Resolution” refers to the process of improving the quality of a photo by boosting its apparent resolution. Enlarging a photo often produces blurry details, but Super Resolution has an ace up its sleeve — an advanced machine learning model trained on millions of photos. Backed by this vast training set, Super Resolution can intelligently enlarge photos while maintaining clean edges and preserving important details.

The company’s AI algorithms gave it an insatiable habit for lies and hate speech. The Cambridge Analytica scandal would kick off Facebook’s largest publicity crisis ever. It compounded fears that the algorithms that determine what people see on the platform were amplifying fake news and hate speech, and that Russian hackers had weaponized them to try to sway the election in Trump’s favor. Everything the company does and chooses not to do flows from a single motivation: Zuckerberg’s relentless desire for growth.

Young female Japanese biker is really 50-year-old man with luscious hair using FaceApp

https://twitter.com/azusagakuyuki

Given the state of technological advancement these days, one can never really believe what they see online all the time. The man reportedly said that nobody wants to see an “uncle”, and so, he turned himself into a “beautiful woman” so that his photos would be popular.

What I learned in two years of moving government forms online

The author started at the City of Boston with a simple yet comically broad mandate: Get rid of paper forms and PDFs. Over the course of two years, they identified 425 forms and moved 122 of them online. Using a conservative estimate, moving those forms online saved Boston residents just under 10,000 hours. They also made government services much more accessible.

Amazon driver quits, saying the final straw was the company’s new AI-powered truck cameras that can sense when workers yawn or don’t use a seatbelt

Amazon was equipping all delivery vehicles with AI camera systems called Driveri, manufactured by a company called Netradyne. The cameras are always on and scan drivers’ body language, the speed of the vehicle, and even drowsiness. The system then uses “automated verbal alerts” to tell drivers if a violation has been detected.

Are Xiaomi browsers spyware? Yes, they are…

They send all visited websites to an analytics server. Once when the page starts loading, and another time when it finishes. And the Xiaomi blog post explains why this code exists: “The URL is collected to identify web pages which load slowly; this gives us insight into how to best improve overall browsing performance.” Are you convinced by this explanation?

Business & Finance

Bond markets offer ridiculously low yields.
The economics of investing in bonds (and most financial assets) has become stupid.
The world is a) substantially overweighted in bonds (and other financial assets, especially US bonds) at the same time that b) governments (especially the US) are producing enormous amounts more debt and bonds and other debt assets.

And many more reasons.

It’s 2021—it would be shocking if the New York Times wasn’t A/B testing headlines. The findings are very interesting.

How to do one-on-ones as a manager

If you are a line manager, your most important responsibility is to ensure the people you line manage are happy, engaged and are working as effectively as they can. One of the best ways to achieve this is to have quality one-on-ones with your line reports.

Culture & Fun

Reddit investors adopt 3,500 gorillas in six days

Around 3,500 gorillas have been adopted by Reddit’s WallStreetBets (WSB) community in six days. The Dian Fossey Gorilla Fund International, a charity for the protection of endangered mountain gorillas, has also received more than $350,000 (£252,000) in donations.

Zoom Escaper is a tool to help you escape Zoom meetings and other videoconferencing scenarios. It allows you to self-sabotage your audio stream, making your presence unbearable to others.

Player calculates the cost of obtaining his dream team in FUT 21 in response to EA saying you don’t need to spend money. 22,000 hours or £80,000 required

And related Reddit discussion with the usual jokes.

I liked the Netflix show and loved the book even more.

Eight year-old orphan Beth Harmon is quiet, sullen, and by all appearances unremarkable. That is until she plays her first game of chess. Her senses grow sharper, her thinking clearer, and for the first time in her life she feels herself fully in control. By the age of sixteen, she’s competing for the U.S. Open championship. But as she hones her skills on the professional circuit, the stakes get higher, her isolation grows more frightening, and the thought of escape becomes all the more tempting. Engaging and fast-paced, The Queen’s Gambit speeds to a conclusion as elegant and satisfying as a mate in four.

Once considered crazy by fellow villagers, Indonesian eco-warrior Sadiman has turned barren hills green after 24 years of effort. Some even thought he was a madman because he bartered saplings for the goats he reared. Sadiman also funds his work through a nursery of plants such as cloves and jackfruit that he can sell or barter.

Meet NASA’s Flight Director of Mars Perseverance, Who Came to U.S. with $300 and Cleaned Bathrooms

Diana Trujillo immigrated to the United States from Colombia when she was 17. “My parents got divorced when I turned 12. After that happened, my mom had nothing. No money. We didn’t even have food. We’d boil an egg and we’d cut it in half, and that was our lunch that day,” she said. To put herself through community college, she secured a housekeeping job, and was able to transfer to the University of Florida, where she majored in aerospace engineering.

ELI5: Why does an orchestra have many people playing the same instruments? How does it add to the overall performance?

Ten people place the horsehairs of their bows atop their strings and pull. They are all playing the same note. Their violins are the same size–or are they? They are almost the same size, but not quite; the margin of error in manufacturing is small, but greater than zero. The violins are made of different wood from different trees, each with its own voice. Each performer is pulling their bow at a slightly different speed. Their fingers are all placed expertly on the exact spot they need to be placed on their fingerboards, but as the violin is a fretless instrument, each finger is placed very slightly differently to all the rest, offering a barely discernible difference in pitch.

BEF Learning & Progress Update 2

Sat, 20 Mar 2021 00:00:00 GMT

Fund Distribution

We have encountered unexpected problems during fund distributions—the main reason being that we didn’t screen the kids before finding a sponsor for them.

Failure Analysis

The (faulty) process before:

School gave us a list of student
We reached out to potential donors
Once we matched a donor with a child, we will reach out to the parent to screen them

We found several issues:

A few families rejected our help by saying that they don’t need the help right now and that we should help other more needy kids instead.
One child was adamant that they wanted to drop out of school because they wanted to help their family make money. It took us a couple of weeks to convince the child and their family to accept our help and continue their study.
A few families are uncontactable. We tried contacting them via phone calls, WhatsApp (the default comms method in Indonesia) and sometimes contacting the kids directly without any success.

New & Improved Process

From now on, we will switch up the steps:

School gave us a list of student
We reach out to the families to screen them and make sure that they are receptive and require help
Only then will we reach out to potential donors to raise funds

Fundraising

We have entirely stopped fundraising while we close the loops on the first wave of 8 students.

Once we identify the next group of student, we will start the fundraising again.

Business Entity Registration

We have registered Borneo Education Foundation with the State of California.

I’m working to figure out the next steps now on the path to 501c3.

Website

The development is coming along. We aren’t in a hurry to get this done. The website’s primary purpose is for high scalability, and we aren’t ready for high volume yet.

I will write a more in-depth blog post about the website architecture later.

For now, I’m enjoying learning NextJs, Typescript, Serverless architecture, integration with Contentful, etc.

Lots of fun!

Borneo Education Foundation - Progress Update 1 - Mid February 2021

Sat, 13 Feb 2021 00:00:00 GMT

Since I published my previous blog post two weeks ago, we have made significant progress. I am grateful for all the help and interest from all of you!

KPI

Total children helped: 8

Updates

1. We had paid for the tuition fee for two children this semester

The children can now continue their education with one less worry. They are both covered until finishing high-school (at least).

2. Generous donors had pledged to sponsor six more children

The donations come from donors in Indonesia, Australia, and the USA. Thanks for all the pledges!

Some pledged to sponsor for a semester, some for a year, and some until the children finished with high-school.

3. Handful of donors are happy to donate and let us decide where to spend it

We are very grateful for your trust!

4. We have registered borneoeducationfoundation.org domain name

This name reflects our focus to help children in Borneo to get a good education.

5. We now have a web developer on board to help make the website

My high-school classmate, Wendy, had volunteered to develop the website.

A very early alpha version is in the works. Wendy wrote it in PHP with CodeIgniter.

Next Steps

1. Registering 501©(3) organization

My immigration lawyers have just confirmed that I can register a charitable organization despite my visa status. They just wanted to make sure that I won’t be compensated in any way. This won’t be a problem since all the money raised will go to the children.

If you know people who know how to do this best, please contact me!

2. Expanding partnership with other schools in Borneo

Right now, we are working with one school group only (Santu Petrus in Pontianak). We are planning to reach out to other schools to help more students.

3. Recruit more volunteers to help and raise more money!

Things are starting to heat up, and we will need more help. If you wanted to get involved, please contact me.

I'm starting a charity to help underprivileged children in Borneo, Indonesia to get quality education. Here's why and how you can help.

Sat, 30 Jan 2021 00:00:00 GMT

Why Borneo, Indonesia?

Because no one is doing it right now.

Searching for “West Borneo charity” or “Borneo education charity” on Google would show results for Orangutan & environment charities. Sure, it is good to help the animals, but we should help the human kids too.

There are many of other foundation that focuses on education in Indonesia. However, every single one that I’ve checked focuses on Java and Bali - the most populous and famous places. I haven’t been able to find a single one that helps kids in Borneo.

Some example: Emmanuel & Collibri both focus on Java.

Why education?

I was born and raised in West Borneo (Kalimantan Barat), Indonesia. I moved elsewhere after finishing high school to pursue higher education.

I am fortunate to be able to do the things I do now. It was all thanks to the education that I received along the way. I’m very thankful for the opportunity that I had received so far.

Many other kids are smarter than me but didn’t have the resources to continue studying due to various reasons. Economic being a significant barrier.

If we can take away the most significant barrier, they will surprise us with how high they can achieve.

What about free public schooling?

The government provides free primary education, but the quality is abysmal.

Based on a conversation with Yayasan Usaba in Ketapang, they said that the government-sponsored public school has:

36% of teachers who skip coming to class but still got paid.
Many principals only came twice to the school in a semester: at the beginning and end of the school semester.
50% of kids in second grade still couldn’t read or write.
20% of kids in sixth grade yet couldn’t read or write.

As a result, the kids will have to spend a long time playing catch up with their peers who got good quality education.

This data is limited to Ketapang and surrounding places, but the theme is clear. The public school’s quality is just awful.

What’s the finding so far?

One of my friends is now working as a high-school teacher at my alma mater. She helped to contact the principal and administrator and produced a list of at-risk students.

Some of the situations include:

Recently orphaned, living with 80yo grandfather.
Father passed away, mom working as a household maid.
Father works as a daily laborer.
Mom and dad divorced. Now the kid lives with 70yo grandmother.
Lots of stories about parents losing their income due to the COVID pandemic.

Those kids would lose their chance at getting a quality education without our help.

This school charges around IDR 660k (USD 46) per month per pupil.

I contacted my primary & middle school alma-mater, Yayasan Usaba.

They manage 13 schools ranging from primary to middle school, all located outside the city center of Ketapang. They serve 1,144 students and charge IDR 25-75k (USD 2-5) per month per pupil.

The typical students are children of local laborers who worked for palm oil plantations, miners, farmers, and fishers. Their livelihood had been decimated by deforestation, the COVID situation, and other climate-change related issues.

Other charitable organizations

I’ve known of three other organizations and in the process of establishing better communication and understanding.

Love Borneo - They raise funds and goes to remote villages to provide books to needy kids.
Yayasan Palung - They provide a college scholarship for students who want to give back to the Orang Utan conservation effort.
Aku Belajar - Recruits volunteers to teach poor kids in remote villages. They raise money by selling used clothing.

How can we help the kids?

I see two ways to help.

Get donors to sponsor individual kids to make sure they received adequate education until finishing college. The donor would get a regular update of the kid’s progress.
Raise a big pool of money and donate to local charitable foundations to help alleviate the situation.

Or both.

Getting started with sponsoring kids seems to be an excellent way to start. I have personally sponsored two kids for this semester and looking to support more.

My friends in Pontianak are currently helping to set up bank accounts, looking for potential at-risk kids, and surveying their situation to confirm that they needed help. All voluntarily.

Thanks, Carol and Budy!

How can YOU get involved?

There are two ways you can help.

Become a donor and send money to help the kids. 100% of your donation would help the kids directly.
Provide sweat capital. If you are passionate about helping kids, you can help us raise donations, create legal entities and accounts, spread the word, manage the logistics, etc.

This is my first try at starting a charity. I need all the help I can get!

Please get in touch, and I’ll be delighted to talk more about this.

Best Way to Prepare and Consume Soylent

Mon, 18 Jan 2021 00:00:00 GMT

I’ve been having Soylent lunch for approximately five years. I drank it every working day at lunch break. It is quick, nutritious, filling, and require very little preparation and clean up. Perfect for a lazy person like myself.

Think of these prep steps as the culmination of my five years of experience in consuming Soylent optimally.

Hardware needed: Blender with 24oz bottle. I use the Ninja brand and love it.

Summary

Fill the bottle with a little bit of water
Put two scoops of Soylent
Add whatever supplements you desire
Add chunky-cut fruits. I usually use Apples and Pears
Add soy milk or dairy milk if you are lactose tolerant.
Blend them
Drink
Clean up

Approximate time required: ~5 minutes.

Photos and details

Fill the bottle with a little bit of water. I usually run the tap for 1-2 seconds. This step is necessary so that the powders don’t stick to the container during blending.

Put two scoops of Soylent. I find that this is enough to sate me. Add more or less as necessary for you.

Add whatever supplements you desire. I added Creatine Monohydrate here. Feel free to add whatever supplement you take.

Add chunky-cut fruits. I usually use Apples and Pears. You should de-core the fruits, then quarter them and cut them a little smaller to help with the blending.

Add soy milk or dairy milk if you are lactose tolerant. Fill it up! Usually, I went way over the “MAX FILL” line, but it still works fine.

**Blend them for 30 seconds. **

After drinking, you need to rinse them off with water to prevent sticking. Then just put the dirty container into the dishwasher or wash manually.

Introducing My New Weekly Digest Newsletter

Sat, 02 Jan 2021 00:00:00 GMT

Do you want to get interesting links sent directly to your mailbox?

I spent many hours every day reading news, articles, and books. That’s on top of hours of listening to music, watching movies and TV shows, and playing games.

As a result, I’ve gathered a pile of contents that I loved that are too short for a blog post. So far, I’ve only shared those links with friends and family through chat or Facebook.

Now I wanted to open up these curated links to you.

You will receive an email every weekend with several links that I loved with a short description. The main goal is to provide you with a scan-able list so you can decide whether to open the link.

Since this is an entirely new thing for me, I appreciate your feedback on the kind of content that you like or don’t like to see.

To get started, please enter your email address below.

Victor's 2020 Book Awards

Sat, 19 Dec 2020 00:00:00 GMT

These are the best books I read this year. In no particular order.

“We will need an extended version” award

2BR02B by Kurt Vonnegut

I love this short book. The world-building is excellent, and the promise of immortality would make you think about the societal implications. I would love to read the full-length novel.

“Even better after second reading” award

Stormlight Archive series by Brandon Sanderson

I re-read the whole series a second time in preparation for the fourth book’s release. The story gripped me even more, and I noticed many more details that I missed the first time. Highly recommended.

“Oh, I didn’t know that!” award

Genghis Khan and the Making of the Modern World by Jack Weatherford

We all knew that the Mongols were just bloodthirsty barbarians who loved conquest and massacre. But do you know their real motivation and contributions to the world? You might be surprised!

“Remember your mortality” award

When Breath Becomes Air by Paul Kalanithi

High achieving neuro-surgeon was still a mortal. Paul’s story will give you a candid and personal look into his journey to the end of his life. Prepare tissue.

“Required yearly re-reading” award

How to Think Like a Roman Emperor: The Stoic Philosophy of Marcus Aurelius by Donald J Robertson

Robertson did a great job distilling the life philosophy of the late emperor. You can’t control what happens to you, but you can manage your reaction to it. It sounded simple yet tough to do consistently. This book will show you how.

“Action-adventure poetry” award

The Song of Achilles by Madeline Miller

You will fall in love with the characters and empathize with their pain and cheer them on their journeys. They can’t help but be tangled in the history-defining fateful encounters.

“Completely unexpected delight” award

The Seven Husbands of Evelyn Hugo by Taylor Jenkins Reid

This book isn’t the usual type of book that I would even consider reading. I’m so glad that I read it after recommended by a friend. I mean, who doesn’t like a story about a woman with seven husbands!

“How far can you take your lies” award

Bad Blood: Secrets and Lies in a Silicon Valley Startup by John Carreyrou

The saga of Theranos’s rise and fall has been popularized in the media lately. This book will give you the most profound insight with interviews and stories from many people who are personally involved with the company.

“Just fascinating” award

Creative Selection: Inside Apple’s Design Process During the Golden Age of Steve Jobs by Ken Kocienda

I’ve been working at Apple for three years now, but it seems like I knew very little about how things worked. It’s fascinating to read about some people with who I interacted in real life.

“Doesn’t have to be a book to move you” honorary award

The Fable of the Dragon-Tyrant by Nick Bostrom

Defeating Death is not just a pipe dream. This short story rings even more real this year with the Covid-19 pandemic. Humanity has spent way too much time ignoring our existential crises. We should put more serious effort into making real progress towards life extension.

Nightstand - Woodworking Project Step by Step

Thu, 10 Dec 2020 00:00:00 GMT

The final product sitting beside by bed.

First we start with the cutlist and Sketchup design.

Cut the boards to size with table saw and circular saw.

Now it’s time to glue the panel to the Redwood beam

Now we can use pocket screws to make the joint stronger. I admit that I probably went way overboard with the pocket screws. I ended up only using half of those holes.

Looking like a table now!

Now we can move on to assemble the drawers with glue and pocket screws.

Drawers looking good. Don’t worry about the holes since we will use a facade in front of these that will cover up the holes.

Time to put on the railing.

Once the railing is done, we can put on the drawers. I used Kreg’s drawer jig to make this easier.

Putting up the facade with glue and pin nails. You can’t have enough clamps.

Looking great!

Now it’s time to paint and stain them. I use Behr white paint on the surfaces and drawers. The Redwood beam and side panels are stained with Varathane Dark Walnut stain and finished with water based Polyurethane.

Time to install the drawer handles. I used some newspaper to correctly position the holes for drilling. You just need to put the drawer handle on a newspaper and poke the hole where the screws should be.

Clamping a wood to prevent blowout during drilling.

Handles installed.

Sitting proudly beside by bed! Mission accomplished!

The pocket holes are still quite visible as you can see. I’ve decided to not do anything with them since in most circumstances, we won’t ever get to see them much.

Think Life a Freak Book Club Discussion Points

Fri, 04 Sep 2020 00:00:00 GMT

how would you describe what it means to “think Like a Freak”? Do you think like a Freak? Would you encourage others to think like a Freak?
Why do Levitt and Dubner want you to “put away your moral compass”? How do you think your best friend would react if you encouraged her to put away her moral compass? How would you respond?
Describe the broad lessons that you learned from Kobi’s experience as a world famous hotdog-eating champion. How would you apply these lessons to a situation that you are currently facing
Problems are more likely to be resolved when causes, not symptoms, are addressed. Can you provide an example from your community that highlights a situation when symptoms are being targeted instead addressing root causes.
The authors ask, “Why is it so important to have fun?” Thinking like a Freak, what would your answer to this question be?
Do you believe you can persuade someone who doesn’t want to be persuaded? Think about someone who holds a different opinion than you about politics, the environment, taxes, or any other issue. What approach would you use to try to change his or her viewpoint?
Effectively using incentives means understanding what they are. Can you give some examples of financial, social, moral, legal or other kinds of incentives and describe how they might work in one setting and backfire in another.
This book introduces the concepts of declared preferences and revealed preferences. Explain the difference between these and offer examples of these two types of preferences that you have observed in your own experience.
Why do the authors advise a would-be persuader to steer clear of anecdotes? Do you agree with them? What is the problem with anecdotes?
The authors describe themselves as happy, serial quitters. What ventures or activities have you quit early and do you regret it?

Drafted and edited from various sources.

When Breath Becomes Air by Paul Kalanithi

Mon, 27 Jul 2020 00:00:00 GMT

5 out of 5 stars.

Death comes for all of us. For us, for our patients: it is our fate as living, breathing, metabolizing organisms.

This book made me cry several times. The author was able to tell his incredible story while stricken with cancer and working long hours as a neurosurgeon. His prose is terrific. The book explored interesting medical details and contrasting it to the big pictures of what makes life worth living.

It followed Paul’s story from his childhood to his death. Both his dad and his brother are doctors. He first studied English literature and transitioned to medical history before training as neurosurgeon and neuroscientist. His mom kindled his love for literature by pushing her kids to read a lot of books in the ‘college prep reading list.’

One day during his training, he visited a home for people with brain injury. There he encountered the point where studies met reality. Some patients are just left alone by themselves without any visitors. One of the professors even said that it was better for the patients to die. That event helped Paul to contemplate mortality from the start. He noted that “Neurosurgery attracted me as much for its intertwining of brain and consciousness as for its intertwining of life and death.”

You will read about medical details like how formaldehyde is a potent appetite stimulant. You’ll get hungry even while dissecting cadaver -or as they call it now, donors’ bodies. You’ll also read about how extremely overworked the doctors are with residents working upwards of one hundred hours per week.

He was a high achieving young professional with a bright future. He has excellent jobs lined up and research that would help many people. But, alas, fate has something else in store.

One day, his CT scan showed cancerous growth. His role instantly switched from doctor to patient. You’ll follow his harrowing story from the diagnosis to the first improvement and then to the eventual metastasis and death.

I liked how his oncologist never discussed prognosis and focused on finding his values and living in the present. Not ignoring his condition, but not narrowing his experience to just cancer. Under her care, he temporarily returned to neurosurgery before having to tap out due to a weakening body.

Time for me is now double-edged: every day brings me further from the low of my last relapse but closer to the next recurrence—and, eventually, death. Perhaps later than I think, but certainly sooner than I desire.

The epilogue by his wife closed out the book beautifully. You’ll learn how his family responds to his cancer and how their bond helps one another during a difficult time.

In summary, I highly recommend this book. It is one of the best memoirs I’ve read and one of the saddest.

You will confront your mortality, success, and values and hopefully live your life just a bit better.

If It Bleeds by Stephen King - Book Review

Sun, 19 Jul 2020 00:00:00 GMT

4 out of 5 stars overall. Recommended for anyone who loves a bit of mystery.

The book is broken into four short stories of varying lengths.

This is the first Stephen King book that I read. Definitely made me want to read his other works.

“Mr. Harrigan’s Phone”

5 out of 5 stars.

This is my favorite story from the book. Short and sweet with strong and memorable characters.

I especially love the focus on the iPhone and the wonder of being connected to the world in real-time.

“The Life of Chuck”

3 out of 5 stars.

I felt like I was promised a grand journey, but cut short without any explanation to the many mysteries it served.

Why was the world ending?

Why did Chuck’s face got plastered everywhere?

What triggered the mass suicides?

Chuck seemed like just a regular dude who stumbled upon a magical attic. Nothing from his backstory even hints at any answers to the questions above.

If Mr. King decided to write a full novel to expand this story, I’d read it in a heartbeat.

“If It Bleeds”

4 out of 5 stars.

This is a clinic in crafting flawed yet believable characters. I empathized with the decision that the character made. The antagonist is fascinating but felt a bit too weak and easy to defeat. It was supposed to be a (near) immortal who had lived for hundreds of years, so I expected it to be stronger. Seems like it was just explained away with immortal being underestimating mere humans.

Imagine that if you live more carefully, you can (probably) never die. You would definitely take extra steps to ensure that you survive and continue living.

This antagonist just went into the trap without much preparation on weapons nor exit strategy. A bit ridiculous if you asked me.

“Rat”

3 out of 5 stars.

The other stories felt like it happened in reality.

This story felt like it is a hallucination of a schizophrenic person. Maybe that is what the author intended.

The deal with the devil felt too easy too. Usually, you had to sacrifice something that you really love, either part of your body or your direct family. Somehow this devil is willing to take a friend’s life.

Way too easy.

How to set up Pi-Hole on AsusWRT Merlin from scratch for home-wide ad-blocking

Mon, 01 Jun 2020 00:00:00 GMT

There’s a world-wide pandemic raging, nation-wide riots over police brutality, lockdown, and curfew. What a perfect time to set up Pi-Hole.

I used NextDNS and loved it, as evidenced by my previous tutorial post.

However, I just moved house recently and found my old Raspi 3 B in some old box. I am definitely guilty of starting some random tech project even though I should probably unpack the millions of boxes in the garage.

This post will list the steps I took to set up the Pi-Hole.

1. Prepare your MicroSD card

Download the Raspi Imager for your OS.

I chose the Lite version since we won’t be needing any desktop environment. We are running headless! Just plug in your micro SD card and wait for it to finish.

2. Config your Raspi

Once the Raspi is on, you can log in with pi as username and raspberry as a password.

Then run sudo raspi-config to change the default password, set up hostname (i call it pi), enable ssh, and other localization options.

3. Run headless mode

Now that your Pi setup is done, feel free to unplug the keyboard and HDMI cables and let it run headless.

Open up your main machine and run ssh-copy-id pi@pi to enable password-less login next time.

Now ssh pi@pi should work flawlessly.

4. Install Pi-Hole

All you need to do is run this command:

curl -sSL[https://install.pi-hole.net](https://install.pi-hole.net/)| bash

I enabled the default 4 block list. It seems to work well.

Once it’s all set up, you should be able to login to the dashboard with the supplied password. Of course, we don’t want to use the short password, so head over to terminal and type pihole -a -p to change the password.

5. Configure DNS over HTTPS

You wouldn’t browse the web over HTTP, so why would you let people watch your DNS queries? No, no, no.

The team wrote a charming guide here, so you should follow that.

Just remember to use the ARM architecture.

I used the automated way to start the daemon. What could go wrong?

6. Setup AsusWRT Merlin with Pi-Hole

This is the last step. Congratz.

Some kind person has written an excellent step-by-step over on Reddit. Here are the screenshots of my setup for your reference.

LAN -> DHCP Server

LAN -> DNSFilter

WAN -> Internet Connection

And that’s it. Enjoy your ad-free home internet!

After some time, you should be able to see a bunch of information on the admin panel.

Criticism and Book Club Discussion Points for Talking to Strangers by Malcolm Gladwell

Sun, 23 Feb 2020 00:00:00 GMT

Criticisms of the book

James Mitchell who interrogated KSM:

“Gladwell implies that Enhanced Interrogation Techniques (EITs) were (1) used to pressure KSM into “confessing” to attacks; and (2), that efforts to question KSM about future attacks were marred because KSM was being subjected to EITs when he provided information that the CIA used to disrupt plots and capture or kill terrorists still at large. But, those two things are inaccurate. I know it makes for a better story, but that’s just not what happened.

“EITs were never used to pressure KSM to “confess” to anything…period…full stop. And, EITs were not—let me repeat— not being applied when KSM provided information that helped CIA prevent a second wave of 9⁄11 style catastrophic attacks on the West coast or aided in the capture terrorists still at large. I explain all this in my book (Enhanced Interrogation)”

“Gladwell makes much of KSM confessing in open court to a large variety of attacks and plots (including 9⁄11 and killing Daniel Pearl). Gladwell seems to be saying that KSM confessed to these things because, years after their brief use, the EITs compelled him to confess to things he didn’t actually do. To be clear, we did not discuss many of the things on KSM’s confessed list during his interrogations and debriefings. My guess is that if KSM confessed to crimes he didn’t commit, then it was to imbed his true crimes in a list of bogus plots in order to cast doubt on his confession later, should he need to or simply to mess with court proceedings. It is not necessary to evoke the boogie man of cognitive impairment to explain KSMs duplicitous behavior.”

source: review section of https://www.audible.com/pd/Talking-to-Strangers-Audiobook/1549150340

Andrew Ferguson of The Atlantic:

“Poets die young,” he writes, in a section on Sylvia Plath. “And of every occupational category, [poets] have far and away the highest suicide rates—as much as five times higher than the general population.”

“To get her “five times” figure, … studied the lives of “all major British and Irish poets born between 1705 and 1805.” She determined their “major” status by consulting old poetry anthologies. She decided there were 36—not 35, not 37, but 36—major poets, ranging from …. Of the 36 poets, two committed suicide. (It’s not clear that these two can even be classified as poets, however: One was a physician by trade, and the other died at 17, probably too young to qualify for an occupational category.) Jamison reckoned that two out of 36, proportionally, is five times the suicide rate for the general population.

Voilà! A statistic is born.

source: https://www.theatlantic.com/ideas/archive/2019/09/when-malcolm-gladwell-says-nothing-at-all/597697/

Steven Poole of the Guardian:

“In Gladwell’s world of large ideas, it may also be hard not to be a rapist when you’re drunk. He brings his forensic empathy to the case of Brock Turner, the Stanford college student who was caught sexually assaulting an unconscious woman on the ground outside a dorm building. Such a difficult case! Gladwell explains sorrowfully that consuming large amounts of alcohol causes mental “myopia”, where one is unable to consider the long-term consequences of one’s actions. It’s just too bad, he concludes, that these careless students both got so drunk at a party that the man could “tragically misunderstand” the woman’s intentions. You may object that plenty of men are able to get blattered without raping anyone, but that seems to be beyond Malcolm Gladwell.”

source: https://www.theguardian.com/books/2019/sep/11/talking-to-strangers-malcolm-gladwell-review

Book Club Discussion Points

Warm-up Question

What was your most memorable encounter with strangers?

Questions

There are many stories in the book. Which one impacted you the most?
The book is based on the ‘default to truth’ theory. Do you agree with it? Can you think of examples of when you ‘default to untruth’?
Gladwell wrote, “We fall out of truth-default mode only when the case against our initial assumption becomes definitive.” Do you have good examples from your life when the truth-default scales tipped over for you?
The bail judgment AI system has been shown to perform 25% better than human judges. Do you think we should reduce human judgment and increase the AI system’s role in our judicial system?
The “Holy Fool” is a truth-teller because he is an outcast & blurts out inconvenient truths or questions things the rest of us take for granted. Have you encountered a “Holy Fool”? What inconvenient truths did they blurt out?
After you read this book, what would you do differently when talking to the next stranger?
Has the book changed the way you see yourself?
The author wrote, “alcohol’s principal effect is to narrow our emotional and mental fields of vision. It makes the immediate aspects of the experience have a disproportionate influence on behavior and emotion.” Has this book changed your views about alcohol and its place in our culture? What do you think should be done to minimize the damage that alcohol will cause in the future?
How do you make sense of the statistic that 77 out of 114 soldiers falsely identified their interrogators in a photo lineup? If torture changes the mind so much, how can we reliably get critical information from the captured enemies?
515 people who tried to jump from the SF Golden Gate bridge had been unexpectedly restrained. Just 25 of them persisted in killing themselves some other way. Do you agree that suicide is coupled? Why is it tough for us to accept the idea that a behavior can be so tightly coupled to a place?
Firearm suicides make two-thirds of all gun deaths and half of the suicides in the US. The US firearm suicide rate is 10 times that of other high-income countries. Do you think that US suicide is coupled to the firearm, or is it the other way around, that firearm suicide is coupled to the USA? What can we do about this?
How has reading the book affected your views on the victimization of unarmed black people (Sandra Bland & Ferguson) to women & children being sexually assaulted at colleges (Brock Turner & Sandusky)? Do you agree with Gladwell that these are mere “communication” issues between strangers?
Gladwell is saying that the riots in Ferguson, Mo., are not about race, but about “a particular style of policing that had been practiced in the city for years.” Police officers approach civilians on the flimsiest of pretexts, looking for a needle in a haystack, resulting in obliteration of trust between police and community. What’s your take on this issue?
What problems does the author identify in our society that hasn’t been discussed?

33 Lessons from Talking To Strangers by Malcolm Gladwell

Thu, 20 Feb 2020 00:00:00 GMT

Throughout history, battles overwhelmingly involved neighbors directly across the border because people then can only interact with those who are physically close. However, today we are now thrown into contact all the time with people whose assumptions, perspectives, and backgrounds are different from our own.
Before WW2, Chamberlain misread Hitler’s intention. He was assuming that the information he got from his personal interaction with Hitler was uniquely revealing. In reality, the extra information actually helped to cloud his judgment. He was a master politician yet failed to read someone properly.
Why are the group of people you expect not to be fooled are actually the ones get fooled more?
Judges make bail decisions. This is the choice of whether to jail someone until trial or to let them go free on bail. They want to look at the defendant in the eyes to detect guilt or remorse to help them decide.
An AI system was trained on historical judges’ decisions based on the defendant’s age and criminal record. The AI results were far superior. The people whom the AI system released are 25% less likely to commit a crime while awaiting trial compared to the judges. The AI system couldn’t see the defendant’s facial expression and demeanor, unlike the judges. It also can’t ask additional questions. This turned out to be an advantage. How is it that meeting a stranger can make us worse at making sense of that person?
The ‘illusion of asymmetric insight’ dictates that we are convinced that we know others better than they know us. In one experiment, researchers asked the participant to fill in the blanks to complete words e.g., TOU_ _ → could either be TOUCH or TOUGH. Participants are convinced that their choices reveal nothing about them, but other’s choices reveal much. We think we can easily see into the hearts of others based on the flimsiest of clues. We jump at the chance to judge strangers. We would never do that to ourselves, of course.
Timothy Levine researched deception. He conducted an experiment where people are recorded on video and are given a chance to lie. It turns out that an average person could identify 54-56% of the liars. Police officers, judges, and CIA officers fared no better.
Enter “Truth-Default Theory” or TDT. We are much better than chance at correctly identifying the students who are telling the truth. But we’re much worse than chance at correctly identifying the students who are lying. We have a default to truth: our operating assumption is that the people we are dealing with are honest.
To snap out of truth-default mode requires what Levine calls a “trigger.” We fall out of truth-default mode only when the case against our initial assumption becomes definitive. In other words, we start by believing. And we stop believing only when our doubts and misgivings rise to the point where we can no longer explain them away. You believe someone not because you have no doubts about them. Belief is not the absence of doubt. You believe someone because you don’t have enough doubts about them. Just think about how many times you have criticized someone else, in hindsight, for their failure to spot a liar. You should have known. There were all kinds of red flags. You had doubts.
The “Holy Fool” is a truth-teller because he is an outcast. Those who are not part of existing social hierarchies are free to blurt out inconvenient truths or question things the rest of us take for granted. No one says a word except a small boy, who cries out, “Look at the king! He’s not wearing anything at all!” The little boy is a Holy Fool. We need Holy Fools in our society, from time to time. They perform a valuable role.
In real life, lies are rare. And those lies that are told are told by a tiny subset of people. That’s why it doesn’t matter so much that we are terrible at detecting lies in real life. Defaulting to truth makes logical sense. You don’t want to second guess every single coffee shop and groceries bill for the rest of your life. If you don’t begin in a state of trust, you can’t have meaningful social encounters. Default to truth becomes an issue when we are forced to choose between two alternatives, one of which is likely and the other of which is impossible to imagine. And in those rare instances where trust ends in betrayal, those victimized by default to truth deserve our sympathy, not our criticism.
Transparency is the idea that people’s behavior and demeanor—the way they represent themselves on the outside—provides an authentic and reliable window into the way they feel on the inside. It is the second of the crucial tools we use to make sense of strangers. Transparency is a myth. When a liar acts like an honest person or when an honest person acts like a liar, we’re flummoxed. Matched people conform with our expectations. Their intentions are consistent with their behavior. The mismatched are confusing and unpredictable. Are we sending perfectly harmless people to prison while they await trial simply because they don’t look right?
The most disturbing of Tim Levine’s findings was when he showed his lying videotapes to a group of seasoned law-enforcement agents—people with fifteen years or more of interrogation experience. On “matched” senders, the experienced interrogators were perfect. You or I would probably come in at 70 or 75 percent on that set of tapes. But everyone in Levine’s group of highly experienced experts got every matched sender right. On mismatched senders, however, their performance was abysmal: they got 20 percent right. And on the subcategory of sincere-acting liars, they came in at 14 percent—a score so low that it ought to give chills to anyone who ever gets hauled into an interrogation room with an FBI agent.
When you read through the details of the campus sexual-assault cases that have become so depressingly common, the remarkable fact is how many involve an almost identical scenario. A young woman and a young man meet at a party, then proceed to tragically misunderstand each other’s intentions—and they’re drunk.
Those who study alcohol now consider alcohol as an agent of myopia, and what they meant by myopia is that alcohol’s principal effect is to narrow our emotional and mental fields of vision. It makes the immediate aspects of experience have a disproportionate influence on behavior and emotion. Alcohol makes the thing in the foreground even more salient and the thing in the background less significant. When alcohol peels away those longer-term constraints on our behavior, it obliterates our true self.
It’s tough to tell, just by looking at someone, whether they’ve blacked out. It’s like trying to figure out if someone has a headache exclusively from the expression on their face. You can do anything in a blackout that you can do when you’re drunk.
We are failing to let women know that when they render themselves defenseless, terrible things can be done to them. Young women are getting a distorted message that their right to match men drink for drink is a feminist issue. The real feminist message should be that when you lose the ability to be responsible for yourself, you drastically increase the chances that you will attract the kinds of people who don’t have your best interest at heart. That’s not blaming the victim; that’s trying to prevent more victims.
In one study in Orange County, California, over 1,000 drivers were diverted to a parking lot late one night. They were asked to fill out a questionnaire about their evening, then interrogated by graduate students trained in intoxication detection. How did the driver talk? Walk? Was there alcohol on their breath? Were there bottles or beer cans in their car? After the interviewers made their diagnoses, the drivers were given a blood-alcohol test. Here’s how many drunk drivers were correctly identified by the interviewers: 20 percent.
Air Force’s SERE program—Survival, Evasion, Resistance, Escape - aims to teach key personnel what to do if they fall into enemy hands. The subjects would be cold, hungry, forced to stand—awake—inside a box for days. Then came the interrogation. One particularly effective technique developed at SERE was “walling.” You wrap a towel around someone’s neck to support their head, then bang them up against a specially constructed wall.
The point of the interrogation was to get the subject to talk—to crack open the subject’s memory and access whatever was inside. But what if the process of securing compliance proved so stressful to the interviewee that it affected what he or she could actually remember? In one study, participants couldn’t identify their interrogators. Out of the fifty-two students, twenty of them picked a doctor who weren’t even involved. In any court of law, the hapless physician would end up behind bars. Another more extensive study found that 77 out of 114 soldiers falsely identified their interrogators in a photo lineup—and this was 24 hours after interrogation! When these soldiers were asked how confident they were in their responses, there was no relationship between confidence and accuracy.
In his book Why Torture Doesn’t Work, neuroscientist Shane O’Mara writes that extended sleep deprivation “might induce some form of surface compliance“—but only at the cost of “long-term structural remodeling of the brain systems that support the very functions that the interrogator wishes to have access to.”
Poets die young. That is not just a cliché. The life expectancy of poets, as a group, trails playwrights, novelists, and nonfiction writers by a considerable margin. Something about writing poetry appears either to attract the wounded or to open new wounds.
In 1962, 5,588 people in England and Wales committed suicide. Of those, 2,469—44.2 percent—did so with carbon-monoxide poisoning. It was by then the leading cause of lethal self-harm in the United Kingdom. Nothing else—not overdosing on pills or jumping off a bridge—came close.
Displacement is the assumption that people would simply switch to another method. Displacement assumes that when people think of doing something as serious as committing suicide, they are tough to stop. Blocking one option isn’t going to make much of a difference. The alternative possibility is that suicide is a behavior coupled to a particular context.
Coupling is the idea that behaviors are linked to particular circumstances and conditions. If suicide is coupled, then it isn’t merely the act of depressed people. It’s the act of depressed people at a particular moment of extreme vulnerability and, in combination with a specific, readily available lethal means. Suicide goes way up when town gas first makes its way into British homes. And it comes plunging down as the changeover to natural gas begins in the late 1960s. In that ten-year window, as town gas was being slowly phased out, thousands of deaths were prevented.
Psychologist Richard Seiden. Seiden followed up on 515 people who had tried to jump from the bridge between 1937 and 1971 but had been unexpectedly restrained. Just 25 of those 515 persisted in killing themselves some other way. Overwhelmingly, the people who want to jump off the Golden Gate Bridge at a given moment want to jump off the Golden Gate Bridge only at that given moment. So when did the municipal authority that runs the bridge finally decide to install a suicide barrier? In 2018, more than eighty years after the bridge opened. Now, why is this? Is it because the people managing the bridge are callous and unfeeling? Not at all. It’s because it is really hard for us to accept the idea that a behavior can be so tightly coupled to a place. In one national survey, three quarters of Americans predicted that when a barrier is finally put up on the Golden Gate Bridge, most of those who wanted to take their life on the bridge would simply take their life some other way. But that’s absolutely wrong. Suicide is coupled.
The Law of Crime Concentration says, like suicide, crime is tied to particular places and contexts. 3.3 percent of the street segments in the city accounted for more than 50 percent of the police calls. The researcher looked at New York, Seattle, Cincinnati, Kansas City, Dallas, and Boston. And every place they looked, they saw the same thing: Crime in every city was concentrated in a tiny number of street segments.
AIDS is the same. Fifty census tracks out of fifty thousand had over half of the AIDS cases in the United States. AIDS didn’t look to him like a contagious disease roaming wildly and randomly across the land. It seemed to him like an interaction between certain kinds of people and certain particular places, an epidemic with its own internal logic.
The most prominent example of how our inability to understand suicide costs lives: roughly 40,000 Americans commit suicide every year, half of whom do so by shooting themselves. Handguns are the suicide method of choice in the United States—and the problem with that, of course, is that handguns are uniquely deadly. Handguns are America’s town gas. What would happen if the U.S. somehow eradicated its leading cause of suicide? It’s not hard to imagine. It would uncouple the suicidal from their chosen method. A very conservative estimate is that banning handguns would save 10,000 lives a year, just from thwarted suicides. That’s a lot of people.
Extra police patrol cars will make a difference—so long as officers took the initiative and stopped anyone they thought suspicious, got out of their cars as much as possible, and went out of their way to look for weapons. And once they’ve stopped a motorist, police officers are allowed, under the law, to search the car, so long as they have reason to believe the motorist might be armed or dangerous. And what was the principal implication of coupling? That law enforcement didn’t need to be bigger; it needed to be more focused. If criminals operated overwhelmingly in a few concentrated hot spots, those crucial parts of the city should be more heavily policed than anywhere else, and the kinds of crime-fighting strategies used by police in those areas ought to be very different from those used in the vast stretches of the city with virtually no crime at all.
There is something about the idea of coupling—of the notion that a stranger’s behavior is tightly connected to place and context—that eludes us. It leads us to misunderstand some of our greatest poets, to be indifferent to the suicidal, and to send police officers on senseless errands.
The anger in Ferguson wasn’t just about Brown’s death—or even mainly about Brown. It was, instead, about a particular style of policing that had been practiced in the city for years. The Ferguson Police Department was the gold standard of Kansas City policing. It was a place where the entire philosophy of law enforcement was to stop as many people as possible for as many reasons as possible. A police officer approaches a civilian on the flimsiest of pretexts, looking for a needle in a haystack—with the result that so many innocent people are caught up in the wave of suspicion that trust between police and community is obliterated. That’s what was being protested in the streets of Ferguson: years and years of police officers mistaking a basketball player for a pedophile. They instructed their police officers to disregard their natural inclination to default to the truth—and start imagining the worst: that young women coming from job interviews might be armed and dangerous, or young men cooling off after a pickup game might be pedophiles.
How many extra guns and drugs did the North Carolina Highway Patrol find with those 400,000 searches? Seventeen. Is it really worth alienating and stigmatizing 399,983 Mikes and Sandras to find 17 bad apples?

The Best Biography Books I Read in the Last Decade (2010-2019)

Wed, 29 Jan 2020 00:00:00 GMT

This article a follow up on the previous post about my last decade’s reading statistic.

I decided to break the book recommendation post into three sections because it has grown too long. The three parts are non-fiction, fiction, and biography (this post).

These are the books that I believe have significantly affected me and changed my life for the better.

Sorted in no particular order. However, If you only have time to read one book in that category, pick the top.

Business

Steve Jobs by Walter Isaacson walks you through Jobs’ life and gives you a glimpse of how he operates and what motivates him. The stories are based on actual interviews with him and people close to him. The story is filled with lessons about innovation, leadership, and a revelation of how terrible he treated the people who work with him.

Shoe Dog: A Memoir by the Creator of NIKE by Phil Knight is a riveting tale about the start and growth of Nike and Phil Knight. He started really small with only fifty dollars of loans and was able to take the world by storm. But it wasn’t always easy. He had to navigate politics across cultures and make lots of smart decisions. Great read overall.

Physical Feat

Endurance: Shackleton’s Incredible Voyage by Alfred Lansing shows you the peak of human perseverance in the face of almost-certain death. Shackleton’s crew was trapped in Antarctica and had to trek almost a thousand miles in freezing cold conditions. They had to focus one hundred percent on survival.

Open by Andre Agassi tells a gripping tale of how Agassi was raised. Since very young, he had been trained with the dragon, a cannon that shoots out tennis balls. He had a very complicated relationship with the game of tennis, and you can tell that he doesn’t really enjoy it all that much. This might be a cautionary tale of all the tiger parents out there. You can help your kids to be successful, but at what cost? Is it worth it?

Science & Technology

Surely You’re Joking, Mr. Feynman!: Adventures of a Curious Character by Richard P. Feynman is a must-read for everyone. He had a way of just having fun in everything he did. This book shows how curious and playful he was even while working on the nuclear bomb project and cracking safes. I certainly look up to him a lot and hope to emulate how he blitzed through life with such energy.

Just for Fun: The Story of an Accidental Revolutionary by Linus Torvalds is a refreshing approach to the Linux systems and how it came to be. For those who don’t know, Linux is an operating system that runs virtually all of the internet services that you use on a day to day basis. Linus was never in it for fame and glory. Instead, it was all for fun. Even now, he stills writes and improves on the codes of his creations.

An Astronaut’s Guide to Life on Earth by Chris Hadfield will give you a deep insight into astronauts and their lives. The parts that struck me most was the loneliness and isolation before launch. He couldn’t even be in the same room as his kids to avoid the risk of getting sick in space. You will learn a lot about his life and the process of becoming an astronaut.

Foreign Culture

Educated by Tara Westover gives us a window into the hard the life of survivalist and anti-government family would be. Her dad isolated the family, and the kids grew up without the normalcy that almost all of us enjoys. Despite the hardship, Tara managed to get high education at Harvard and Cambridge by her hard work and a fair amount of luck. It makes you wonder how many brilliant kids are there in the world, just waiting for a sliver of a chance.

Maus Series by Art Spiegelman is a Pulitzer prize-winning graphic novel depicting the life and suffering of the Jews in the hand of Nazi Germany. The stories are very personal, intimate, and told by a survivor of the genocide. The drawings are simple but meaningful, with mice representing the Jews and cats representing the Nazis. You will learn just how violent and evil we humans can be.

Born a Crime: Stories From a South African Childhood by Trevor Noah tells the story of growing up in apartheid South Africa and its hardship. Being born into a mixed-race brings about a complication of belonging. He is neither white nor black. You will read about the mischievousness of kids and just how they adapt to their environment despite ostracism and other evils. This is a funny and personal book that will make you laugh and cry.

Inspirational

Permanent Record by Edward Snowden reveals the extent of spying by the US government and how this one lone whistleblower put his life in jeopardy to inform the world. He couldn’t live with the disillusionment with how the government is treating its citizens. He chose to abandon a great job with a high salary, a loving partner, and stick to his life principles. The PRISM program has recorded and kept private information, photos, and webcam recording of millions of people for an indefinite time. Reading this will prompt you to take more control of your online privacy, but at times it will feel like Sisyphus.

Not Fade Away: A Memoir of Senses Lost and Found by Rebecca Alexander will show you the world of someone who is losing both their sense of sight and hearing due to genetic disease. On top of that, she even broke her legs. Despite all the hardship, she still managed to complete two master’s degrees, competed in endurance sports, and became a physiotherapist to help others to recover their mobility. Can you imagine what your life would be like if you are losing both your sight and hearing?

A Long Way Home by Saroo Brierley has been made into a Hollywood movie ‘Lion,’ and you’ve probably watched it. It is a tale about a lost Indian boy who was forced to survive on the streets. Later he was adopted by an Australian family. When he grew up, he used Google Maps and what recollection he had to find his original hometown. Miraculously, he managed to do just that.

Politics

A Higher Loyalty: Truth, Lies, and Leadership by James Comey is an honest and personal story by the former FBI director who was fired by Trump. In this book, you will learn how the court system and FBI works. He put a lot of care not to make any partisan decision and stick to his own principles. As a result, he ultimately lost his job. This book contains excellent lessons about loyalty, leadership, and integrity.

Free to Be Ruth Bader Ginsburg: The Story of Women and Law by Teri Kanefield is actually the first time I’ve ever heard of RBG since I didn’t grow up in the US. She brought about a lot of change in the court system regarding women empowerment. The book starts from her childhood and her family, then progresses to her early career and marriage, and ultimately to her supreme court appointment and the subsequent cases.

Becoming by Michelle Obama feels like a sincere telling of the former first lady’s life. She didn’t pull any punches. The book described both the good and the bad things alike. It starts from her childhood on the South Side of Chicago, her impressive careers, meeting Barrack, and ends with her endeavors as the first lady of the US. I especially love learning about the road to the presidency, and the tolls than campaigning will take.

The Best Fiction Books I Read in the Last Decade (2010-2019)

Fri, 24 Jan 2020 00:00:00 GMT

This article a follow up on the previous post about my last decade’s reading statistic.

I decided to break the book recommendation post into three sections because it has grown too long. The three parts are non-fiction, fiction (this post), and biographies.

These are the books that I believe have significantly affected me and changed my life for the better.

Sorted in no particular order. However, If you only have time to read one book in that category, pick the top.

Fan Fiction

Harry Potter and the Methods of Rationality by Eliezer Yudkowsky is the best fanfiction I’ve read in the last decade. What if Harry Potter is trained in the methods of rationality? What if instead of blindly following the teachers, the kids actually investigate the magic system themselves. He also fixed some of the more obvious plot holes like Hermoine joining Ravenclaw instead of Griffyndor like the original.

This is required reading for anyone who loves the Harry Potter series or wants to be more rational. You can download the ebook for free here on the official website.

High Fantasy

The Kingkiller Chronicle Series by Patrick Rothfuss is required reading for fantasy lovers. The story will grip you from start to finish, and you will have an intense love-hate relationship with the characters and appreciation for the silence of three parts.

Mistborn Series by Brandon Sanderson should be the entry point for people who wants to get into fantasy. The original trilogy is excellent, and the subsequent books build upon the world. It has a cohesive world-building and magic system, believable character development, and storytelling that will engage you throughout.

The Stormlight Archive Series by Brandon Sanderson is his latest epic fantasy that follows Kaladin in a world torn apart by the globe-circling storm. Unlike Mistborn, that’s mainly focused on the main character, this one will dazzle you with an array of seemingly unrelated characters that will work together to save the world.

The Lies of Locke Lamora by Scott Lynch is a unique book because it focuses on mind-games and lies. You follow the life of the con artist Locke and his exploits to lie and cheat his way through life.

A Song of Ice and Fire Series by George R.R. Martin is the source material to the world-famous Game of Thrones show by HBO. The books are better. Be prepared to whip up pen and paper so you can keep track of the hundreds of characters and their relationships.

The Witcher Saga by Andrzej Sapkowski is now an ongoing TV series by Netflix. It has also spawned 3+ best selling video games with Witcher 3 winning so many awards. Witchers earn a living by hunting monsters. They do not rely on muscle alone but rely more on their expansive detailed knowledge on the enemy’s strengths, and weaknesses and how to best defeats them. They bring rationality in a world beset by superstitions.

General Fantasy

The Ocean at the End of the Lane by Neil Gaiman is a fantastic fantasy that is set in a small town, not unlike the one you grew up on. The magic and supernatural events hit you from a blind spot. You wouldn’t expect them.

Watership Down by Richard Adams is now a Netflix series. It is an adult fantasy series about talking bunnies. One of the rabbits had a prophecy about impending doom, and a group set out to find a new burrow to avoid said doom. This is filled with intrigue, deception, betrayals, brutal murders, deposing authoritarian regimes, and calling in airstrikes. Definitely, do not read this story to your child.

Science Fiction

Remembrance of Earth’s Past Series by Liu Cixin is the best SciFi series I’ve read. It started with stories of scientists driven mad to suicide, and it ends with a universe-wide extinction event. You will learn about intergalactic politics and its allegory to our current nuclear deterrence, love across space-time, and mind-bending dimension-folding technologies. A must-read.

Chaos Walking Series by Patrick Ness tells the story of a spacefaring human colony that settled on a planet where some virus made it, so men’s thoughts are broadcasted to all women. This created extreme information imbalance, and the societies dealt with this in different ways. You will meet weird aliens and cross-species friendships.

The Origin Mystery Series by A.G. Riddle is a story of a weaponized global pandemic, Atlantis technologies, secret societies, and conspiracies.

Artemis by Andy Weir shows you how a colony in a moon might be like. You follow a little girl and her shenanigans while uncovering sinister plots that will affect the whole settlement.

Mythology

Norse Mythology by Neil Gaiman is a refreshing retelling of the old tales. You will learn about the gods, giants, and their relationships.

Mythos: The Greek Myths Retold by Stephen Fry and Circe by Madeline Miller are also modern retell just like the one above. I always love the intrigues of the Olympians.

Historical Fiction

Pachinko by Min Jin Lee is an epic story spanning generations and two countries. A Korean family is dealing with the Japanese invasion. You will learn the atrocities committed and how the civilians handle it. In case you still haven’t figured it out, the story is a sad one.

Shōgun by James Clavell is a classic tale about Westerners who crash-landed in Japan during the shogunate period. The newcomers will have to learn the local’s way of life and vice versa. This is a tale of battle, subversion, politics, and love.

Shantaram by Gregory David Roberts is about an Australian who flees to India after breaking out of prison. There he undergoes a journey of survival and self-discovery. You will meet local underbellies, police corruption, how the poor helps each other, and many more.

The Pillars of the Earth by Ken Follett will walk you through the struggle of building a cathedral in medieval times. You would think such a tale be boring, but that couldn’t be further from the truth. Follett wrote with such prose that you can’t help but feel like you are experiencing the journey firsthand.

All the Light We Cannot See by Anthony Doerr allows you to live in the eyes of a little girl and a man caught in World War II fights. A very moving story.

Literary Fiction

How to Get Filthy Rich in Rising Asia by Mohsin Hamid is the only book I’ve read that’s written in second person style. You reread that sentence in confusion, surely that’s a mistake. This prompted you to click that link and read the reviews for yourself and decided to get the book. The story impressed you as you follow the life of a man born in a poor household who rises through the ranks to become a rich man. But for what? What does he want in the end? You loved this book and told everyone you know to read this.

Where the Crawdads Sing by Delia Owens shows you the life of a girl who lived in a swamp. She ended up studying by herself through books and becomes a best-selling natural sciences author. Later she gets tangled in a murder case that pits herself against an entire town who hates and calls her the swamp girl. Did she do it?

The Overstory by Richard Powers starts with a collection of many seemingly unrelated stories that somehow ended up converging at the end. A group of eco-terrorist is trying to stop a logging company from cutting down ancient trees and accidentally killed one of their members. What are they going to do? You will learn about the hidden world of trees.

Graphic Novel

Habibi by Craig Thompson really impressed me. The Arabian story feels really authentic and alive. Do not give this book to your kids. This is for adults only. There are lots of sex, violence, and magic.

The Best Non-Fiction Books I Read in the Last Decade (2010-2019)

Sun, 19 Jan 2020 00:00:00 GMT

This article a follow up on the previous post about my last decade’s reading statistic.

I decided to break the book recommendation post into three sections because it has grown too long. The three parts are non-fiction (this post), fiction, and biographies.

These are the books that I believe have significantly affected me and changed my life for the better.

Sorted in no particular order. However, If you only have time to read one book in that category, pick the top.

Philosophy

How to Think Like a Roman Emperor: The Stoic Philosophy of Marcus Aurelius by Donald J. Robertson is the best step by step guide to Stoicism that I’ve ever read. If we all could learn to not over-react to adversity, there would be much fewer wars and suffering. This book is essentially a modernized form of Meditations by Marcus Aurelius. I plan to reread these two books again and again.

Free Will and The Moral Landscape: How Science Can Determine Human Values by Sam Harris heavily influenced how I view decision making and definitely improved my empathy. If free will is indeed an illusion, then how do we improve our lives and those we love? Without free will, those who made sub-par decisions don’t really have a choice. Next time when someone did something you think stupid, try to imagine that if you were born and raised in their exact environment and family, you would most certainly make the same decision that you considered stupid before.

Waking Up: A Guide to Spirituality Without Religion by Sam Harris and In Love with the World: A Monk’s Journey Through the Bardos of Living and Dying by Yongey Mingyur Rinpoche showed me the benefits & practicality of meditation. It is a wonder that the majority of the world’s population live their lives without ever thinking about how their mind works. Mindfulness meditation is surprisingly easy to learn and really helps you to understand what actually happens in your consciousness.

A Manual for Creating Atheists by Peter Boghossian gets an honorable mention here. The book taught me about the Socratic method and its application. You don’t have to go out and try to deconvert everyone you met. Just knowing the technique is absolutely beneficial, since you can apply the same principle to your beliefs to find out which ones should be dropped or kept.

Zen and the Art of Motorcycle Maintenance: An Inquiry Into Values by Robert M. Pirsig is a weird & polarizing book. It asks a lot of abstract questions like “What is quality?” and that makes people uncomfortable. However, if you do enjoy learning philosophy, this book would be a good step out of your comfort zone.

Sciences

A Brief History of Time by Stephen Hawking is a fun little book. I believe this is a must-read for everybody to learn about our home and the basics of how it works.

The Design of Everyday Things by Donald A. Norman showed me the hidden world of design. Steve Jobs would have probably recommended this book as well. You will learn about the designs of the objects you interact with every day and why it is so.

Communication & Negotiation

Fearless Salary Negotiation: A step-by-step guide to getting paid what you’re worth by Josh Doody taught me how to negotiate and gave me the confidence actually to do it in high stakes salary negotiation. This book would directly make you more money than the others.

Never Split the Difference: Negotiating As If Your Life Depended On It by Chris Voss is another great negotiation book that isn’t just about salary.

Non Violent Communication by Marshall B. Rosenberg will save & improve your relationship with others. We should all learn to communicate without causing further enmity.

The Game: Penetrating the Secret Society of Pickup Artists by Neil Strauss tells an amazing tale of a shy journalist’s journey in the world of people who specialized in pick-up, which really is just communication and negotiation. If you are the type of person who thinks that the pick-up community is toxic, just try to imagine this as an anthropology book where an Alien came to earth to learn about homo sapiens’ mating behavior. After all, pick-ups are just communications.

Anthropology & History

**Sapiens: A Brief History of Humankind,** Homo Deus: A History of Tomorrow & 21 Lessons for the 21st Century by Yuval Noah Harari is a must-read trilogy. He went into great detail of our species’ past, present and future. Sapiens talks about our species’ history, Homo Deus predicts our future, and the 21 Lessons combines them both to help us prepare.

Guns, Germs, and Steel: The Fates of Human Societies by Jared Diamond has a similar tone to Harrari’s but focuses more on why some nation advances quicker than others. This will probably help to dispel the notion that some race’s skin color made certain people superior to others. The biggest difference-maker is really luck that has been compounded over generations.

Scale: The Universal Laws of Growth, Innovation, Sustainability, and the Pace of Life in Organisms, Cities, Economies, and Companies by Geoffrey B. West will blow your mind. Everything is a system, and each system scales according to a set of power laws. Once you have learned this concept, a lot of things will make more sense. This is a must-read for everyone who works with any kind of system.

Enlightenment Now: The Case for Reason, Science, Humanism, and Progress by Steven Pinker tells the story that despite the bad news every day, our society is actually progressing positively at a rapid rate. Pinker presents his view with a plethora of charts and data.

Parenting

The Self-Driven Child: The Science and Sense of Giving Your Kids More Control Over Their Lives by William Stixrud and The Vanishing American Adult: Our Coming-of-Age Crisis—and How to Rebuild a Culture of Self-Reliance by Ben Sasse shows a better way to raise our kids in today’s age. We shouldn’t coddle our children and protect them from failure. Instead, we should let them fail early and often to assist learning. Despite lowering crime and kidnapping rates, parents nowadays are ever more afraid to let their children play by themselves. I certainly plan to give my kids control more of their own lives more.

How to Talk So Kids Will Listen & Listen So Kids Will Talk by Adele Faber will teach you how to communicate better not just to your kids but also to other adults.

Software Engineering

Algorithms to Live By: The Computer Science of Human Decisions by Brian Christian shows you how learning the basics of algorithms can help you make decisions in day to day life. I highly recommend this to both technical and non-tech people alike. Learn about the theory of ‘explore’ vs. ‘exploit’ and caching with many real-life examples.

Cracking the Coding Interview: 150 Programming Questions and Solutions by Gayle Laakmann McDowell helped me to get my current job in Silicon Valley. If you want to play the game, you got to learn the rules. This book is the best guide to get a tech job. The tech interview is often viewed as broken, but all the top companies still do it. It is so bad that even if you work with a complex system every day, you will always have to study for interviews. I do hope this trend changes.

RxSwift: Reactive Programming with Swift by Florent Pillet will teach you Functional Reactive Programming (FRP) for iOS. Once you grok FRP, you won’t want to use the old callback pattern anymore.

The Mythical Man-Month: Essays on Software Engineering by Frederick P. Brooks Jr. will show you the pain and suffering of scheduling and running software projects. This is a classic and required reading for project managers.

Leadership

Extreme Ownership: How U.S. Navy Seals Lead and Win by Jocko Willink is a brilliant book. Plenty of Navy Seals war stories juxtaposed with business scenarios. A mission or project can only be successful if someone takes extreme ownership. This is a highly recommended book for anyone who leads other people.

The Dichotomy of Leadership: Balancing the Challenges of Extreme Ownership to Lead and Win by Jocko Willink is a followup book that answers the questions of ‘how much is too extreme?’. You should only read this after reading the first one.

Radical Candor: Be a Kickass Boss Without Losing Your Humanity by Kim Malone Scott is a refreshing take on leadership without any bullshit. Just try to be honest and communicate more without being an asshole.

The Manager’s Path: A Guide for Tech Leaders Navigating Growth and Change by Camille Fournier is a book specifically for the tech industry. Highly recommended for anyone who wanted to transition or found themselves transitioning from individual contributor to leadership roles.

Medical & Health

Why We Sleep: Unlocking the Power of Sleep and Dreams by Matthew Walker will convince you to prioritize sleep. You will learn the downsides of lack of sleep, the benefit of sleeping more, and the techniques to get better sleep (a.k.a. sleep hygiene).

Being Mortal: Medicine and What Matters in the End by Atul Gawande discusses our mortality and the sorry state of end-of-life treatments. We will all die sooner or later. How should we handle all this and plan properly?

The Emperor of All Maladies: A Biography of Cancer by Siddhartha Mukherjee will teach you about the history of cancer and how badly our doctors ignored data about the screw-ups. We have come a long way, and we might have a lot longer way to go.

How to Change Your Mind: What the New Science of Psychedelics Teaches Us About Consciousness, Dying, Addiction, Depression, and Transcendence by Michael Pollan is a must-read if you are interested in how psychedelics medicines might be beneficial to us. The classification of psychedelics as an illegal drug has virtually stopped all research for decades. Only now have scientists started to look into it again. I wouldn’t be surprised if we would all be taking LSD or magic mushrooms as treatment or preventions for mental ailments in ten years.

The Epigenetics Revolution by Nessa Carey is a must-read if you are interested in genetics but do not know what epigenetics is. Fundamentally, genes can be turned on and off depending on an individual’s environments. A highly engaging book filled with real-life scenarios.

The Selfish Gene by Richard Dawkins opened my eyes to the shocking reality that we are merely a carrier of our genes. So much of our lives, desires, and what makes us happy can be explained as proxies to further the gene’s agenda. Next time you see an attractive person on the internet or the street and feel that arousing sensation, you are just doing what your genes want you to do.

The Nature Fix: Why Nature Makes Us Happier, Healthier, and More Creative by Florence Williams tells us to live close to nature to be happier. If you live in places without trees, do try to fill your surroundings with live plants. Hospitals with plants apparently heal people faster. If all the building managers read this book, we will all probably be happier and healthier.

Behavioral Economics

Thinking, Fast and Slow by Daniel Kahneman gave birth to the behavioral economics field. This is a must-read for everyone interested in how we make decisions and how we think. Why does 1+1 feel effortless while 36 x 7 feels effortful? It is because those are processed by the two different systems in your brain.

Predictably Irrational: The Hidden Forces That Shape Our Decisions by Dan Ariely cemented the concept that we are all predictably irrational. We probably know that we are sometimes irrational, but probably doesn’t yet understand that the irrationality can be predicted to a high degree of accuracy.

Scarcity: Why Having Too Little Means So Much by Sendhil Mullainathan shows that life is all about scarcity. You will look at debt, poverty, and public policy differently after reading this book.

Hooked: How to Build Habit-Forming Products by Nir Eyal revealed the strategy and techniques that companies use to hook people on their product. There are armies of people trying to get you hooked on their app or games right now, understanding the strategy would go a long way to defend against them.

Psychology & Self-Help

Range: Why Generalists Triumph in a Specialized World by David Epstein is probably my pick of the decade. This book provides a counterpoint to the plethora of other literature preaching the doctrine of specialization, such as Outliers by Malcolm Gladwell. The central thesis is that deliberate practice works really well for things that have unchanging rules e.g., playing the violin, but not optimal for inventing new things or finding connections between diverse domains of expertise. Read this. Love it or hate it, you won’t regret it. I’ve gifted this book to numerous people and always get a great review.

Peak: Secrets from the New Science of Expertise by K. Anders Ericsson and Grit by Angela Duckworth is very similar to Outliers mentioned above. They provide the view that specialization and perseverance is the key to the success of top performers. As I said above, this observation rings true for a lot of domains, but you might want to pair these books with Range to get a more complete understanding.

The Millionaire Fastlane: Crack the Code to Wealth and Live Rich for a Lifetime! by M.J. DeMarco provides a no-nonsense guide to getting rich. I like it because it doesn’t try to sell you new products or any pseudo-science. The fast lane really is just to find a way to increase your income significantly, then invest the money for early retirement.

Principles: Life and Work by Ray Dalio will show you the framework in which Bridgewater Associates and Ray Dalio himself make decisions, prioritize and solve problems. You might have a completely different life principle from him, but learning the framework will definitely be beneficial for your life.

Economics

Debt: The First 5,000 Years by David Graeber will teach you everything you need to know about the history of economics and our financial system. Mind you that this book is a tome, but it is worth it.

Antifragile: Things That Gain from Disorder by Nassim Nicholas Taleb revealed the nature of self-healing systems that benefits from shocks and damage but only up to a point. This is unlike robustness. Human muscle is antifragile because it gets stronger with micro-injuries from resistance training. The stock market is also antifragile, but government intervention can actually cause more damage. Read more to learn why.

Who Gets What — and Why: The New Economics of Matchmaking and Market Design by Alvin E. Roth goes deep into market and matchmaking. You will learn a lot of lessons about the market and economy in general.

If you have any book recommendations for me, please let me know!

I read 523 books last decade. Here are some fun stats

Sat, 04 Jan 2020 00:00:00 GMT

Photo by Janko Ferlic

I started tracking my book reading on Goodreads around 2009. Since then, I’ve read an average of 52 books per year.

One of my resolution in 2012 was to read more books, and I somehow was able to keep it up until now. I’ve no idea how.

Drilling more in-depth into the months, you can see that each month was different. The most were Oct 2012 and March 2018, in which I managed to read 17 books. Some months were either one, or no book read at all.

What does it all mean?

In my experience, you should just read when you can. There will be times when you are too busy or too into this game or that tv show, and that’s fine. Some breaks are good to recharge the fun of reading.

If you don’t like reading, that’s fine too. Don’t let anyone pressure you into doing something you don’t enjoy. However, you might just haven’t found your kind of books that will keep you coming back.

These charts are generated by Goodreads reading stats tool.

21 Lessons from ’Apollo: Race to the Moon’

Sun, 04 Aug 2019 00:00:00 GMT

by Charles Murray & Catherine Bly Cox

My rating: 5⁄5

This year is the 50th anniversary of the moon landing, so our book club at Apple voted to read this book. I love this book. It tells the story of the entirety of the Apollo program, from inception to completion. A lot can be learned from this fantastic tale.

Quoted texts are taken from the book. Emphasis mine.

1. Russians can be very cheeky

The United States … was working on a grapefruit-sized satellite weighing three and a half pounds. … launch this puny competitor to Sputnik in front of television and newspaper cameras from around the world. The Vanguard rocket being used for the launch rose four feet into the air, fell back, and crumpled onto the pad in a spectacular explosion. A few days later, the Soviet delegate to the United Nations inquired solicitously whether the United States was interested in receiving aid earmarked for underdeveloped countries.

2. Keep it simple

a.k.a. YAGNI principle

In the early day of NASA, they have a limited budget and haven’t built a sophisticated transport system for the spacecraft. They just lined up an old pickup truck with mattresses and got the job done. Too bad I couldn’t find the picture.

Pictures still in the files attest to the story, showing a battered pickup with a spacecraft on mattresses pulling up to the pad where an Atlas rocket stands waiting.

3. Building a construction marvel is easier than conveying its scale to the general public

To house the gigantic Saturn V rocket, NASA built a mega-warehouse that’s really massive in scale. The construction went fine, but they have trouble taking a photo to show the size.

Yet conveying this size in ways other than numbers turns out to be impossible. Ray Clark, who ran the Cape’s support facilities in the 1960s, recalled that as the V.A.B. was nearing completion, NASA wanted to show the public how big it really was. But nothing their photographers produced captured the immensity. NASA got in touch with Life magazine, whose editors replied loftily that NASA need not worry, Life’s photographers knew how to do that sort of thing. The Life photographers flew down and set up shooting positions from every conceivable angle. They shot hundreds of rolls of film and went back to their labs in New York. “A while later they came back to see us,” Clark remembered. “They said, ‘We don’t know how to do it either. There’s just no way to portray photographically, or any way we know of, what the building’s really like.’”

4. Planning for the impossible makes it easy to imagine building the extremely hard

If they had not been talking about the Nova, the huge Saturn would have seemed impossible. “The fact that we could postulate Nova with somewhat of a straight face automatically made the five-eighths-size Saturn credible.” It is one of the tricks engineers sometimes use on themselves, Maynard said, to maneuver themselves into taking on tasks that would otherwise be terrifying.

5. Rocket science is more uncomplicated than car

Liquid fuel is pumped into a combustion chamber in the presence of liquid oxygen and a flame. It burns. That’s all there is to it. There are no crankshafts to turn, no pistons to drive. The burning fuel produces energy in the form of gases that exit through the rocket’s nozzle. The force the gases produce against the top of the engine is called thrust. The thrust is transmitted through the rocket’s structure and, if it is greater than the weight of the rocket, the rocket lifts off. Put in its most basic terms, for any rocket to work there are two things that must be done extremely well: The propellants must be brought together, and then they must burn smoothly.

6. Exploding a bomb inside of the engine is a totally valid way to test the engine’s stability

The engineers were having a problem reproducing an engine instability problem. The problem arises seemingly in random. In software engineering, we call this Heisenbug . So how would they reproduce the issue consistently? To plant a bomb inside of it, of course.

… for testing purposes they must “be able to initiate this instability at our command,” as Castenholz put it. Nature, left to herself, wasn’t producing instabilities frequently enough for the team to learn how to control them. Therefore, they decided to explode a bomb in the combustion chamber while the test was in progress—if that didn’t drive the engine into instability, then nothing would, they reasoned. Then they could concentrate on making the engine damp out the instabilities they had induced.

After numerous bomb explosions and iteration, they got a system that would mend itself.

They fiddled and nudged, and the incidence of instability decreased still further until, by late 1964, they weren’t getting it anymore. The bombs would explode, the pressure in the combustion chamber would skyrocket—and then the engine would be running smoothly again, not just within the 400-millisecond goal the team had set for itself, but within 100 milliseconds.

7. The best way for higher up to know what’s going on is to communicate directly with the engineers

Joe Shea (Deputy Director of NASA) would get each engineering team to provide a weekly report. He would read and annotate it. This is the way they come out with to let the higher-ups stay in touch with the ground-level work.

Markley went away to produce the first edition of the loose-leaf notebook that he and his staff would prepare for Shea every week for the next 165 weeks. Each notebook ran to more than a hundred pages and was on Shea’s desk by close of business Thursday. Shea would get up at 4 A.M. Friday and start to annotate it, usually with technical comments and instructions, occasionally with remarks such as “You’ve gotta be kidding.” He would work on the notebook intermittently from Friday morning through the weekend and return it to Markley on Monday morning. By Monday afternoon, everyone in the ASPO network would have gotten his response from Shea. With the annotated notebook in his hands, Markley became the action officer. “I’d take this notebook, and then I’d fan it back out very quickly and say, ‘Hey, we’ve got these questions and I want the answers in the notebook by this Thursday.’ We’d get the answers back and then we’d take Joe’s marked-up notebook and start off in the front, anything he had marks on. We’d have the answers there. And then we’d have the brand-new section with the newest status too.” The scrawled notations to and fro were a way of keeping up “a running communication,” in Shea’s words, without having to type up long memos. “Even if I didn’t see [the project officers] directly, I’d at least been in touch with them, and they’d been back in touch with me.”

8. “If you understand it, you can make me understand it.”

And then Joe would begin to grill you. There was no point in trying to sidestep or fudge an answer. It made no difference if Shea wasn’t a specialist in your area. “If you understand it, you can make me understand it,” Shea kept saying, and you had better understand it absolutely cold or Shea would dissect you.

9. Keep it sweet and simple

a.k.a avoid over-engineering.

the people who wrote the specifications for the fuel gauges had reasoned that, because it was so important, they needed a highly accurate fuel gauge system. The way they would do it was to build a Geiger-counter arrangement that would infer the bulk of fuel left in the tank by the attenuation of radiation sources that would be transmitted through the fuel. It was a challenging bit of technology, no doubt about it—so challenging that the gauge they had developed so far was accurate only to about 10 percent and it kept failing quality tests for reliability. Shea let this process run for a couple of months and then stepped in. “This thing doesn’t make any sense to me,” he said. “We don’t need anything that complicated.” Shea had a Karmann Ghia that didn’t even need a fuel gauge, because it had a little reserve fuel tank that he could switch to. And that’s what they ended up doing for the R.C.S. fuel tanks. They had a big tank that they used during the mission, and a little tank, not to be touched unless the big tank ran out, but that could get the crew home safely. No more nuclear gas gauge.

10 .“The better is the enemy of the good”

This discussion comes up surprisingly often in tech. Should we rewrite our software in this newer, sexier framework/language? The answer is usually no because if you had time to ask this question, often you would already have a satisfactory working product.

“Because if we fail a qual test and if I have to send it back to redesign,” Shea explained, “then I have a less mature product when it comes back than I had already gotten up to that point.” The better is the enemy of the good.

11. Trust in reducible complexity

The system can be complicated, but you can tackle it by understanding each of the pieces.

“Hey, it isn’t that complicated. It is very understandable. The engines work this way, the guidance system works that way, the transistors work this way, so don’t get yourself in a state of mind thinking that it’s too complex. It really is very simple. It’s piece by piece.” It’s awfully big, that’s all, Shea kept saying. Piece by piece, it’s simple.

12. Even during the busy time, we need to have a calmer period to reflect

The most significant accident of the Apollo program happened during the test launch of Apollo 1. A catastrophe where all the crew were killed. The grief gave everyone time to pause and reflect.

“once the fire occurred, the flight schedule came to a screeching halt and everybody stopped and took stock.” People then had the time to go back and work in a less pressured fashion on “all of these things that everyone had in their back pocket that they should have worked on, and hadn’t had a chance to.”

13. “If you don’t know what to do, don’t do anything.”

“If you don’t know what to do, don’t do anything.” But it was simple as a koan is simple, for flight controllers were trained and conditioned to solve problems; the temptation was for a controller to think that he knew what to do when he really didn’t. True wisdom in flight control lay in being able to recognize one’s own ignorance. Much of the training of the flight controllers consisted of showing them how they could be fooled.

14. Hire smart people & trust them to make a right decision

Flight had to listen carefully to every word and nuance of the answers. “The most likely error we can make in the business is not listening,” said Kranz. “We’ve got very smart people [in the controllers]. We breed them to be very smart, we breed them to give opinions. We breed them to work in an arena of conflict. Sometimes they may disagree with you. Sometimes they may not be too smooth in words. Sometimes they may come in on an untimely basis and disrupt your train of thought.” But none of that can interfere with your listening and understanding precisely what it is that you are hearing.

Then another during the Apollo 13 accident.

For Glynn Lunney, standing alongside Kraft, the reaction of the men in the viewing room was inspiring. The option Kraft was recommending had the obvious disadvantage of leaving the crew at risk for twenty-four hours longer than the alternatives, but there was no quibbling. Lunney: “These men who we all grew up thinking were kings, absolute tops in our business, sat in this room and said to us, ‘Look, we’re here to support you guys, and we just want to hear what’s going on and we want you to know that if there’s anything at all that you need, you just let us know.’” Then and in the days to come, Lunney recalled, “they never second-guessed, never reversed anything, never even came close to that. No cover-your-ass. The whole set of interactions was one of the most supportive and reassuring you can imagine.” The gathering quickly ratified Kraft’s choice: The Apollo 13 crew would come home more slowly, but, they hoped, more surely.

15. Don’t mess with the rocket fuel tank

One day, a technician at White Sands working with a fully fueled test article was filling out a report and absentmindedly clicked the end of his ballpoint pen on a fuel tank. The pen exerted just enough extra pressure on the tank to open a pinprick leak. Eventually, or so the story goes, they found the pen embedded in a fence post, along with a nub of finger bone. The wire-thin stream of propellant had sliced through flesh and bone as neatly as a scalpel.

16. “The Eagle has landed”

When Neil Armstrong and Buzz Aldrin landed on the moon, they announced like so:

“Houston, Tranquility base here,” Neil Armstrong announced. “The Eagle has landed.” “Roger, Tranquility, we copy you on the ground. You’ve got a bunch of guys about to turn blue. We’re breathing again. Thanks a lot.”

17. Anything that could go wrong will go wrong, so always be prepared

Apollo 13 accident was so unlikely that they never even practised it in the simulator.

Such experiences led to a few procedures that could be pulled off the shelf during Apollo 13. Still, the flight control team had never developed procedures for an accident of the magnitude of Thirteen’s. They had imagined losing one of the two oxygen tanks, or a battery, or even the S.P.S. engine. But never had the simulations postulated that the C.S.M. would be completely dead. Surely, everything thought, anything that knocked out both oxygen tanks would also destroy the spacecraft. Apollo 13 brought a change in simulation policy. FIDO Dave Reed summarized the new ground rule: “They can throw anything at us they want and we won’t object.”

18. In unusual circumstances, forget the rulebook and improvise

When you encounter something that has never happened before, it might mean that you will have to do something you’ve never done before.

To conserve water and power, the engineers in the MER went systematically through the spacecraft, looking at every component to see whether it could be turned off completely. If not, they calculated the lowest voltage at which any given piece of equipment would operate and the minimal configuration in which a system would continue to perform the functions it absolutely had to perform. Forget the specs, forget the rulebooks, forget the operations manuals, Arabian announced. Go back to the physics of the design that no one knows better than you, the knowledge that’s not written down anywhere, and decide what’s the least we can really get away with.

19. Rely on checklists

NASA has checklists for everything . When the Apollo 13 accident happened, they needed the astronaut to solve some problem, and of course, they created a checklist for them too.

The checklist system is also widely used in medicine .

So as the White Team assembled Wednesday morning, they were a disheveled and weary crew, with a little more than two days left before splashdown. Under ordinary circumstances, preparing an entry checklist for a lunar mission was a three-month job.

The creation of the checklist was a controlled frenzy in which solutions were developed, rejected, and replaced in such rapid-fire sequence that it was difficult at any one moment to be sure how far they had to go.

20. Astronauts jobs are really, really hard

During the Apollo 13 accident, the astronaut had to survive on a meagre amount of water & sleep.

By Thursday, the temperature in Odyssey was thirty-eight degrees, about that of a refrigerator. Until then, the crew had used Odyssey as the bedroom of a two-room apartment; now they huddled together in the little LEM, which was marginally warmer. But Aquarius, too, was uncomfortable. The walls and windows of the LEM were moist with condensed water. The men were averaging three hours of fretful sleep per day. Also, concerned about having enough water to cool the electronics, all three astronauts had cut their fluid intake to near zero. They deliberately ignored the Surgeon’s instructions to drink—since they didn’t feel that thirsty anyway (one doesn’t, in space), they chose to save water for the equipment. The result was that, without knowing it, all three were becoming dehydrated. Haise had developed a kidney infection and was running a fever. Fatigue. Cold. Dehydration. As they neared time for entry, the normal performance levels of all three had deteriorated. Now they were going to have to assimilate a hastily written entry checklist, perform a crucial midcourse correction, and then bring the spacecraft home without any of the backup systems that ordinarily provided some margin for error.

21. If you want to maintain a good family relationship, don’t work too much

Work-life balance is imperative, especially when you have spouse and kids.

In many cases, families didn’t survive the stress—during the mid-1960s, the towns around K.S.C. had the highest divorce rate in the country.

The consensus is that this is no coincidence. Under the heat of Apollo, your marriage either melted or annealed.

How to use NextDNS with AsusWRT-Merlin [Tutorial]

Sat, 01 Jun 2019 00:00:00 GMT

Update 1/4/2020

Olivier mentioned that this is now directly integrated with Merlin.

Updated 7/27/2019

Recently I read about NextDNS from this Hackernews post. I used to run Pi-hole to manage DNS-level ad-blocking but its quite a pain to setup. There’s always some issue going on with the Raspberry Pi itself and needed to spend time to manage it.

Now with this service, I can have the DNS blocking like Pi-hole without the hassle!

Setting it up with AsusWRT-Merlin is easy, but NextDNS doesn’t have a setup guide for it. So I decided to write one.

1. Setup new account on NextDNS

Follow this link to get started: Create config

Then go to Setup panel and find the DNS-over-TLS & DNS Servers IP addresses.

It should look like:

2. Open DNS panel

You can get there by clicking WAN button on the left menu.

Previously I used CloudFlare’s DNS, and it worked quite well. But now we want to switch to NextDNS!

3. Enter the NextDNS details

First, you want to delete the existing DNS Server 1 & 2.

Then, just follow the config example above, then copy paste the DNS Servers to Addresses tab and DNS-over-TLS to TLS Hostname. The port is 853.

After that, you should reboot the router.

4. Done!

Now your whole network should be redirected to NextDNS filtering.

You can start enabling blocker lists like EasyList and StevenBlack from the List panel on NextDNS.

Edit 7⁄27: The below section is now outdated. They routed me to a server that’s very close to my area :)

Not all great… yet.

The service works wonderfully considering that this is so new and still in beta. Right now I’m being redirected to Miami VPS instead of the one in CA even though I’m in CA.

You can check that by going to the ping router page.

The founders are very responsive. I’ve chatted with them, and I’m delighted by their help.

That one time I visited Rotary club of Southbank

Fri, 15 Mar 2019 00:00:00 GMT

I am currently visiting Melbourne to activate my H1b visa.

Spent an evening with the Southbank Rotary club. Great people.

https://www.southbank.org.au/blog/volunteers-are-like-angels-on-earth

That one time I taught high school students web dev

Tue, 26 Feb 2019 00:00:00 GMT

Last month I taught students at Monta Vista High School (MVHS) on basics of web dev.

Full article here: https://elestoque.org/2019/01/22/briefs/mvhs-future-business-leaders-of-americas-kicks-off-technology-program/

Memorable quote:

“If you don’t know it, Google it,” Augusteo said. “Even engineers Google everything. Welcome to web development.”

Hackathon Resources

Sat, 19 Jan 2019 00:00:00 GMT

This article is a companion to a tech talk I’m giving to Monta Vista Highschool Entrehack event.

Click here for the Keynote presentation, and be sure to click ‘show presenter note’ on top left for my notes.

These links are a rough guide to help you get started with your hackathon project. Use them wisely.

The key is to focus on the vision and fleshing out your ideas during the hackathon and not to spend time dealing with app architecture and boilerplates.

Read this if your goal is to win. Otherwise, you can skip to the next section.

Ultimate 8 Step Guide to Winning Hackathons – garyyauchan – Medium

Tech links

Prototyping

Sometimes you could do really well with just prototyping the user experience without doing any coding. Invision is a really good service to try this. If you prefer a native Mac app, I highly recommend Flinto.

Web Builder

If you only need simple websites and prefer drag and drop, Wix.com & Webflow works really well.

Infrastructure

Firebase is the easiest to get started if you need to store data like user accounts, pictures, database, etc. For more complex infrastructure, use Amazon Web Services (AWS) or MS Azure.

Web Boilerplates

This project is a very popular web boilerplate that has a lot of services baked in. Easy to get started too. It also have a bunch of recommended libraries/links for you to use.

Some other boilerplates if you prefer React.js or Rails.

Cross Platform (Electron)

If you really need to build apps for all platform, you should use Electron.

iOS

iOS is pretty easy to get started, but this awesome page helps to find projects that you can reuse.

Explore

GitHub is the most popular source control website and you should use it to search for projects to clone or to learn from.

You should also checkout this mother of awesome lists to find out other awesome lists.

Machine Learning

Adding machine learning capabilities are actually really easy with these pre-trained models and services. Use CoreML Models if you are building for iOS. You can also try Amazon Rekognition or other ML services for web and mobile apps.

SMS

If you want to play around with SMS sending/receiving, you should try Twilio.

Hit me up if you have any questions.

My Favourite Books of 2018

Sun, 23 Dec 2018 00:00:00 GMT

This year I read eighty books which is my highest record. I might have to update my old how-to article.

We will start with fictions, then move on to non-fiction.

Fiction

The Wise Man’s Fear by Patrick Rothfuss is hands down one of the best fantasy tales I’ve ever read in my life. Rothfuss managed to outdo the magic of his first book The Name of the Wind and made it even grander. If you read just one book from my recommendation, please let it be this one.

The Dark Forest by Liu Cixin and its prequel The Three-Body Problem are fantastic sci-fi that ponders the question of what would humanity do on the face of an invasion of a superior alien civilization? This is probably one of the few Chinese works that become very popular in the west.

Oathbringer (The Stormlight Archive, #3) by Brandon Sanderson has left me wanting more of the classic Sanderson tale. You must read the previous books to understand the story. I love the magic system in this series, it just makes so much sense.

The Underground Railroad by Colson Whitehead showed me how terrible life under slavery could be and how people would fight systematic oppression. I didn’t know that the Underground Railroad isn’t a real railroad as it was depicted in the book.

The Ocean at the End of the Lane by Neil Gaiman this book went from zero to eleven really fast. What started as a slow-paced nostalgic story just changed gear into fantasy horror so suddenly, yet with a very believable transition.

Artemis by Andy Weir tells the story of a moon-base in the future. I love how Weir painted the details of the tech and how our future could be.

Pachinko by Min Jin Lee is the first book I’ve read about the Asian side of the second world war. Be warned that this book is a multi-generational epic and you might want to sketch the names of the characters to keep track of their relations.

Mythos: The Greek Myths Retold by Stephen Fry and Norse Mythology by Neil Gaiman are both excellent modern retelling of the classic mythology stories.

Habibi by Craig Thompson is an exceptional story from the middle east in the form of graphic novel.

Non-Fiction

Being Mortal: Medicine and What Matters in the End by Atul Gawande discusses the end-of-life treatment in modern medicine. What happens when there is no hope of recovery? What happens when the pain of chemotherapy get so high that it is better just to give up? At what point do we trade quality of life with the risk of treatments? I think we as a society aren’t paying enough attention to this topic and this book should be a must-read for everyone.

Extreme Ownership: How U.S. Navy SEALs Lead and Win by Jocko Willink and its sequel The Dichotomy of Leadership posits that a leader’s primary role is to take ownership of the mission. I enjoyed the format of the book wherein every chapter Willink and Babin tell their real combat story which was then followed by its learning and application in the business world. Highly recommended.

Surely You’re Joking, Mr. Feynman!: Adventures of a Curious Character by Richard Feynman is a fantastic biography of a genius who just wants to have fun. Feynman makes almost everything he does look easy and enjoyable.

An Astronaut’s Guide to Life on Earth by Chris Hadfield follows the commander’s story from his childhood to retirement. You will get a deep insight into the life of an astronaut and its challenges.

Free to Be Ruth Bader Ginsburg: The Story of Women and Law by Teri Kanefield tells the real-life story of the US supreme court judge that is still active as of now. You will come to appreciate just how much circumstances line up to pave her path to the highest judicial office in the nation.

Meditations by Marcus Aurelius is a classic that I would probably read once a year every year. The teaching of stoicism is really timeless and would apply to you where ever your current station in life might be.

Peak: Secrets from the New Science of Expertise by K. Anders Ericsson divulge the secrets of world-class performers. To give you a spoiler, it is deliberate practice. This book could help you understand how Feynman, Hadfield and Ginsburg did what they do.

A Primate’s Memoir: A Neuroscientist’s Unconventional Life Among the Baboons by Robert M. Sapolsky tells the story of the biologist who lived and gets to know the baboons intimately. He wrote the stories so well that you could imagine yourself sitting there in the savannah with them! I really enjoyed it, and it made me appreciate how dedicated some researcher might be.

Enlightenment Now: The Case for Reason, Science, Humanism, and Progress by Steven Pinker says that despite all the bad stuff happening in the news every day, we are progressing really well. There are less illness, poverty, war & crimes in the world compared to the past. This is the best time to be alive!

Drone photos from Cupertino and Apple Park Chapter Two

Wed, 19 Dec 2018 00:00:00 GMT

Second flight from Cupertino (Hamptons). This flight might be the last since there’s only so much you could shoot from here. Next Cupertino flight would probably be from Infinite Loop campus.

I enjoyed experimenting with the top-down drone style shoots and trying out different elevation and angles. The dynamism is a significant improvement over the traditional camera - which was limited by your feet.

However, I would love to have the dynamic range of SLR and better camera control on the drone. Even with five bracketed shot, you probably only get the same dynamic range as a single full-frame SLR RAW. I await the day that we can have a full-frame drone camera.

Hugo shortcode to display all images in a folder

Mon, 17 Dec 2018 00:00:00 GMT

On posts that intents to share photos like the drone post, I wanted to be able to include all the images from a single static folder into the post. Doing it manually is troublesome as I have to copy paste the filename one by one into markdown.

There’s gotta be an easy way to do that, but I’ve no experience working with Hugo, so I asked in the forum: Automating inserting markdown of all images in a subfolder?

As you can see, people suggested using the page resources which I don’t think works the way I wanted.

My curiosity got the better of me and I learned how to write shortcode to:

accept a parameter which contain the folder name in /static/img
loop through every file inside that folder
display the images with some padding

To use this shortcode, you need to create a file called album.html in your theme’s shortcode directory and paste these code:

<!-- put bunch of photos in /static/img/albumName -->
<!-- and insert shortcode "album albumName" in blogpost.md -->
{{ $albumUrl := print "/static/img/" ($.Get 0) }}
{{ range readDir $albumUrl }}
    {{ $imgURL := print "img/" ($.Get 0) "/" .Name | absURL }}
    <p><img src="{{ $imgURL }}"/></p>
{{ end }}

Now you can put all images that you wanted to post in a single folder, the just insert the shortcode in the post.

Drone photos from Cupertino and Apple Park

Thu, 13 Dec 2018 00:00:00 GMT

This flight was my first drone flight. My friend Alex was teaching me how to fly the drone and the camera options. It was fun but thrilling at the same time.

We left work around 4 pm and walked south to get outside of the SJO airport range as legally required by the FAA. Funnily, Apple Park was just on the edge of the airport range, so we didn’t have to walk far.

I took all the pictures with five bracketed shot, which were then processed together with AuroraHDR and Luminar. I am quite impressed that the drone was stable enough to take long exposure shots. Considering that they are floating in the air.

Vallejo Marina - Drone Photos

Tue, 11 Dec 2018 00:00:00 GMT

I went to Vallejo yesterday because I saw an interesting image on BayArea subreddit.

Google maps shows that the location only an hour drive from my house and there’s a big parking (municipal marina) and open space to launch drone from.

The weather was frigid and windy. This flight is my third flight with the drone, and I’m still experimenting with how to fly the drone effectively.

So far I love it because of the freedom to compose. Just maneuver your drone around until it looks good!

Why and How I Moved My Blog To Netlify

Sun, 09 Dec 2018 00:00:00 GMT

Why Netlify?

I have just recently moved my blog from Vultr to Netlify because I was lured by:

Free and easy to setup CI/CD. Just click one button and its all setup!
Free (unlimited?) CDN hosting! They actually called it ADN with a nice graph to compare.
Auto assets optimization like minifying asset files and lossless image compression.
Form handling and Lambda functions integration. I haven’t need to use these yet, but the fact that they are available is exciting. Oh, the possibilities!
If you use a static site generator, they offer NetlifyCMS that work with generators like Jekyll, Hugo, Hexo, or Gatsby.

Steps to create and move

Click on the ‘Deploy to Hugo’ button on this page. That will create a Netlify site and a repo on Github/Gitlab.
Pull the repo to your machine.
Move the site/static/admin folder from Netlify CMS to a temporary folder.
Replace the site folder content with your old Hugo folder.
Move back the admin folder from the temp to site/static.
Edit the config.yml to fit your old Hugo structure.
Commit and push.

Your site should be up now on url that looks like some-random-word12314.netlify.com.

Use your own domain name

(optional) Rename the Netlify url to something good e.g. augusteo.netlify.com
Transfer your domain NameServer to Netlify’s. Then let Netlify set the A and CNAME DNS record.
(optional) Change the MX and TXT record if you have mail server setup for your domain name.
Set SSL cert - they are using Let’s encrypt and will get you a wildcard cert.

Netlify’s amazing customer service

I was having a problem where my site was still using Netlify’s wildcard cert instead of my own. When I emailed their support, I was expecting a slow reply. However, they replied in 5 minutes and fixed my problem!

The quick response blew me away. This is better than some of the paid service! I highly recommend them based on this alone.

How can it be free? What’s the catch?

I asked them about it and how can they offer their services for free. Their answer:

Hi Victor,

The idea is that some users will use our paid services and we’ll have enough of those to cover the cost of people that don’t move to paid. We also count on people using our service to be evangelist and bring in others. Then there are devs that use our free service on free plans and get their companies to move to our our paid plans. In addition to these self-serve, we offer custom enterprise contracts and push support and features and performance for them that you can’t otherwise get on free or self-serve paid.

(emphasis mine).

This post is my effort to evangelist Netlify in return for their amazing free service.

You should give them a try.

Is coding the still the best thing to teach to kids?

Sat, 08 Dec 2018 00:00:00 GMT

I read this good article today:

I’m a Developer. I Won’t Teach My Kids to Code, and Neither Should You.

I agree halfway. I don’t think “teach kids to code” only means teaching the syntax of a particular language. Maybe it should mean teaching the ways of thinking that could be transferred to other activities.

I love his point about teaching curiosity:

You’re teaching them the world is full of interesting things to discover. You’re showing them how to be passionate and look for that ephemeral sense of quality in everything they do. The best part is that even if they don’t become coders—most shouldn’t and won’t—the same skills can be used in nearly any career, in every hobby, in every life.

With examples:

One day, my son was concerned that a chair of his was wobbly. We looked at it and he helped me isolate the problem: One of the screws was loose. I found one of our many leftover hex wrenches and showed him how to screw it back in. After that, he was curious what would happen if he screwed the other way, which he did until the screw came out. We ended up taking the chair all the way apart and putting it back together a couple of times, often mismatching pieces, before he was satisfied the job was finished. Try something. See how it works. Try again.

Norcal Renaissance Faire 2018

Tue, 02 Oct 2018 00:00:00 GMT

![norcal-renaissance-faire-2018-IMG_8991-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_8991-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_8992-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_8992-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9002-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9002-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9003-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9003-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9037-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9037-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9041-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9041-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9046-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9046-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9052-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9052-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9055-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9055-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9056-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9056-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9058-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9058-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9060-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9060-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9063-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9063-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9065-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9065-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9073-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9073-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9075-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9075-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9078-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9078-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9083-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9083-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9089-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9089-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9091-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9091-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9098-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9098-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9100-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9100-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9106-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9106-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9107-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9107-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9115-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9115-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9117-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9117-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9119-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9119-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9122-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9122-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9128-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9128-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9130-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9130-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9134-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9134-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9137-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9137-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9142-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9142-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9394-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9394-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9396-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9396-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9402-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9402-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9406-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9406-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9412-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9412-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9432-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9432-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9444-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9444-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9445-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9445-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9454-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9454-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9457-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9457-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9461-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9461-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9463-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9463-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9467-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9467-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9473-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9473-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9476-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9476-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9477-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9477-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9484-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9484-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9486-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9486-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9496-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9496-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9498-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9498-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9515-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9515-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9520-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9520-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9524-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9524-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9530-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9530-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9535-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9535-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9536-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9536-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9541-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9541-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9542-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9542-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9548-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9548-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9550-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9550-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9553-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9553-thumb 1.jpg)

![norcal-renaissance-faire-2018-IMG_9561-thumb 1.jpg](https://www.augusteo.com/assets/blog/norcal-renaissance-faire-2018/norcal-renaissance-faire-2018-IMG_9561-thumb 1.jpg)

License:

OpenAI Five - Study Notes

Sun, 08 Jul 2018 00:00:00 GMT

This blog post is my study notes on OpenAI Five. I am not involved in the research effort.

Why Dota?

objective -> destroy enemy’s throne. to do that you have to destroy the 4 layers of defensive towers while the other team is defending them/attacking your tower.
There are 115+ (and growing) unique heroes with unique spells and play styles. Each hero has access to 3 spells and one (usually long cooldown) ultimate spell.
Established game with millions of player worldwide. It was released the year 2003 as a map of Blizzard’s Warcraft 3 game.
IceFrog was then hired by Valve to make Dota 2. Valve is currently raising fund for ‘The International 2018’ prize pool
Super popular e-sports with largest-prize pool tournament in the world.
With top players earning millions of dollars
Very complex game
- OpenAI Five observes every fourth frame, yielding 20,000 moves. Chess usually ends before 40 moves, Go before 150 moves, with almost every move being strategic
- there is an average of ~1,000 valid actions each tick. The average number of actions in chess is 35; in Go, 250.
- Valve’s Bot API shows as many as 20,000 (mostly floating-point) numbers representing all information a human is allowed to access. A chess board is naturally represented as about 70 enumeration values; a Go board as about 400 enumeration values.
- I started playing this game more than 13 years ago when I was still in high school/college. The game has been actively developed for over a decade. The game also gets an update about once every two weeks, constantly changing the environment semantics.

Previous work:

1v1 bot - Dota 2
simpler version that only works for 1v1. With a single hero (Shadow Fiend).
result: beat all contestants, who themselves are top Dota players in the world.

Related research effort

What is OpenAI Five?

It’s a team of five neural networks that can work together to fight five human players in the game of Dota.
hardware stats:
- The 256 P100 optimizers are less than $400/hr. You can rent 128000 preemptible vcpus for another $1280/hr. Toss in some more support GPUs, and we’re at maybe $2500/hr all in.
- that is $57,600/day
- they have trained, 2+ months, so around $4 million so far just for hardware.
learns using a massively-scaled version of Proximal Policy Optimization PPO is a class of reinforcement learning algorithm
Each of OpenAI Five’s networks contain a single-layer, 1024-unit LSTM that sees the current game state (extracted from Valve’s Bot API ) and emits actions through several possible action heads. Each head has semantic meaning, for example, the number of ticks to delay this action, which action to select, the X or Y coordinate of this action in a grid around the unit, etc. Action heads are computed independently.
Current set of restrictions:
- 5 invulnerable couriers, no exploiting them by scouting or tanking
Even with OpenAI restrictions, there are hundreds of items, dozens of buildings, spells, and unit types, and a long tail of game mechanics to learn about — many of which yield powerful combinations. It’s not easy to explore this combinatorially-vast space efficiently.
OpenAI Five learns from self-play (starting from random weights), which provides a natural curriculum for exploring the environment. To avoid “strategy collapse”, the agent trains 80% of its games against itself and the other 20% against its past selves. In the first games, the heroes walk aimlessly around the map. After several hours of training, concepts such as laning , farming , or fighting over mid emerge. After several days, they consistently adopt basic human strategies: attempt to steal Bounty runes from their opponents, walk to their tier one towers to farm, and rotate heroes around the map to gain lane advantage. And with further training, they become proficient at high-level strategies like 5-hero push .
OpenAI Five does not contain an explicit communication channel between the heroes’ neural networks. Teamwork is controlled by a hyperparameter OpenAI dubbed “team spirit”. Team spirit ranges from 0 to 1, putting a weight on how much each of OpenAI Five’s heroes should care about its individual reward function versus the average of the team’s reward functions. OpenAI anneals its value from 0 to 1 overtraining.

Training Methods

implemented as a general-purpose RL training system called Rapid, which can be applied to any Gym environment. OpenAI used Rapid to solve other problems at OpenAI, including Competitive Self-Play
- gym: Gym
  - Gym is a toolkit for developing and comparing reinforcement learning algorithms. It supports teaching agents everything from walking to playing games like Pong or Pinball .
- Competitive Self-Play allows simulated AIs to discover physical skills like tackling, ducking, faking, kicking, catching, and diving for the ball, without explicitly designing an environment with these skills in mind. Self-play ensures that the environment is always the right difficulty for an AI to improve.
The training system is separated into rollout workers, which run a copy of the game and an agent gathering experience, and optimizer nodes, which perform synchronous gradient descent across a fleet of GPUs. The rollout workers sync their experience through Redis to the optimizers. Each experiment also contains workers evaluating the trained agent versus reference agents, as well as monitoring software such as TensorBoard , Sentry , and Grafana .
During synchronous gradient descent, each GPU computes a gradient on its part of the batch, and then the gradients are globally averaged. OpenAI originally used MPI’s allreduce for averaging, but now use OpenAI own NCCL2 wrappers that parallelize GPU computations and network data transfer.
- NCCL2 = The NVIDIA Collective Communications Library (NCCL) implements multi-GPU and multi-node collective communication primitives that are performance optimized for NVIDIA GPUs. NCCL provides routines such as all-gather, all-reduce, broadcast, reduce, reduce-scatter, that are optimized to achieve high bandwidth over PCIe and NVLink high-speed interconnect.
The latencies for synchronizing 58MB of data (size of OpenAI Five’s parameters) across different numbers of GPUs are shown on the right. The latency is low enough to be largely masked by GPU computation which runs in parallel with it. OpenAI implemented Kubernetes, Azure, and GCP backends for Rapid.

Reward Functions

Each hero’s reward is a linear combination of separate signals from the game. Each of these signals produces a score, and the agent’s reward is the increase in score from one tick to the next.
These signals and weights were designed by OpenAI local Dota experts at the start, and have only been tweaked a handful of times since.
- Individual Scores
- Building Scores
- Lane Assignments
  - agent receives a special reward to encourage exploration called “lane assignments.” During training, OpenAI assigns each hero a subset of the three lanes in the game. The model observes this assignment, and receives a negative reward (-0.02) if it leaves the designated lanes early in the game.
Zero Sum Each team’s mean reward is subtracted from the rewards of the enemy team to ensure that the sum of all ten rewards is zero, thereby preventing the two teams from finding positive-sum situations. It also ensures that whenever OpenAI assigns reward for a signal like “killing a tower,” the agent automatically receives a comparable signal for the reverse; “defending a tower that would have died.” hero_rewards[i] -= mean(enemy_rewards)
Team Spirit OpenAI wants them to take into account their teammates’ situations, rather than greedily optimizing for their own reward. For this reason, OpenAI average the reward across the team’s heroes using a hyperparameter τ called “team spirit”: hero_rewards[i] = τ * mean(hero_rewards) + (1 - τ) * hero_rewards[i] OpenAI anneals τ from 0.2 at the start of training to 0.97 at the end of OpenAI current experiment.
Time Scaling The majority of reward mass in OpenAI agent’s rollout experience comes from the later part of the game. However, the early game can be very important; if the agent plays badly at the start, it can be hard to recover. OpenAI wish OpenAI training scheme to place sufficient importance on the early part of the game. For this reason OpenAI scales up all rewards early in the game and scale down rewards late in the game, by multiplying all rewards by: hero_rewards[i] *= 0.6 ** (T/10 min)

Training progression

teams: 1. Best OpenAI employee team: 2.5k MMR (46th percentile) 2. Best audience players watching OpenAI employee match (including Blitz, who commentated the first OpenAI employee match): 4-6k MMR (90th-99th percentile), though they’d never played as a team. 3. Valve employee team: 2.5-4k MMR (46th-90th percentile). 4. Amateur team: 4.2k MMR (93rd percentile), trains as a team. 5. Semi-pro team: 5.5k MMR (99th percentile), trains as a team.

Versions: * The April 23rd version of OpenAI Five was the first to beat OpenAI scripted baseline. * The May 15th version of OpenAI Five was evenly matched versus team 1, winning one game and losing another. * The June 6th version of OpenAI Five decisively won all its games versus teams 1-3. OpenAI set up informal scrims with teams 4 & 5, expecting to lose soundly, but OpenAI Five won two of its first three games versus both.

Observations: * Repeatedly sacrificed its own safe lane (top lane for dire; bottom lane for radiant) in exchange for controlling the enemy’s safe lane, forcing the fight onto the side that is harder for their opponent to defend. This strategy emerged in the professional scene in the last few years, and is now considered to be the prevailing tactic. * Pushed the transitions from early- to mid-game faster than its opponents. It did this by: (1) setting up successful ganks (when players move around the map to ambush an enemy hero — see animation) when players overextended in their lane, and (2) by grouping up to take towers before the opponents could organize a counterplay. * Deviated from current playstyle in a few areas, such as giving support heroes (which usually do not take priority for resources) lots of early experience and gold. OpenAI Five’s prioritization allows for its damage to peak sooner and push its advantage harder, winning team fights and capitalizing on mistakes to ensure a fast win.

Difference versus humans

the bot instantly sees data like positions, healths, and item inventories that humans have to check manually. because just rendering pixels from the game would require thousands of GPUs. this is cost saving method.
OpenAI Five averages around 150-170 actions per minute (and has a theoretical maximum of 450 due to observing every 4th frame). Frame-perfect timing, while possible for skilled players, is trivial for OpenAI Five. OpenAI Five has an average reaction time of 80ms, which is faster than humans.
- different heroes have a different need for higher APM.
- the 5 heroes that OpenAI chose are considered ‘beginner’ heroes in that they require fairly low APM to play.
- all the 5 heroes also are ranged heroes who are considered easier to play.
- higher APM heroes example are Invoker - Dota 2 Wiki, Meepo - Dota 2 Wiki or Chen - Dota 2 Wiki due to having many skills (invoker has 14) and controlling many units at once (Chen’s creeps or meepo’s clones)

Surprising Findings

Binary rewards can give good performance. OpenAI 1v1 model had a shaped reward, including rewards for last hits, kills, and the like. OpenAI ran an experiment where OpenAI only rewarded the agent for winning or losing, and it trained an order of magnitude slower and somewhat plateaued in the middle, in contrast to the smooth learning curves OpenAI usually see. The experiment ran on 4,500 cores and 16 k80 GPUs, training to the level of semi-pros (70 TrueSkill ) rather than 90 TrueSkill of OpenAI best 1v1 bot).
- Binary rewards (win/loss score at the end of the rollout) scored a “good” 70.
- With sparse reward scored a better 90 and learned much faster.
Creep blocking can be learned from scratch. For 1v1, OpenAI learned creep blocking using traditional RL with a “creep block” reward.
bugs: large negative reward for reaching level 25. It turns out it’s possible to beat good humans while still hiding serious bugs!

next steps

The International 2018 - Dota 2 Wiki where OpenAI might battle the all-star or winning team like last year’s 1v1 bot test.
July 28th - OpenAI will try to battle a team of top players

discussions:

OpenAI Five | Hacker News https://www.reddit.com/r/programming/comments/8tse7u/openai_five_5v5_dota_2_bots/

Can 'Interview vs Audition' Approach Works for Software Engineers?

Sun, 13 May 2018 00:00:00 GMT

This article from Ryan Holiday has been making rounds lately:

Summary:

There are two types of football coaches, either they sit down and expects a barrage of questions from management or the one doing most of the asking. The first type is the typical interview while the second type conducts an audition. Ramit Sethi has an excellent video explaining the approach here: The Briefcase Technique - YouTube

Software Engineering Context

After all the brouhaha about the broken tech interview, the practice is still going strong. The problem stems from the lack of credibility of resumes. Just ask any seasoned interviewer at big tech companies, they will tell you impressive sounding resumes are dimes a dozen. Once those people got in and asked to solve a simple problem, a significant portion of them couldn’t produce the great code expected. That is the single thing a software engineer will do the most: write code. So it makes sense to see them write code.

The software interview process is not unlike actor audition. Actors still have to come in to act out parts of the script as if it is in front of a real camera and audience.

Now can someone audition as software engineer? the answer is partly.

Many companies have started the audition approach where engineers could submit their resume to one company (e.g. Hired or Triplebyte) and get matched with a lot of companies wanting to interview them. This process makes the first step of the job hunt - getting the interview - to be straightforward. They don’t have to submit resume to one thousand companies and hope one call back.

On the on-site interview, the traditional process still works best. The interviewee should expect the typical coding, architecture, logic and algorithm questions. The briefcase technique mentioned above would work here because company codebases and plans are typically closed, meaning the candidate won’t be able to research them beforehand to make suggestions and assertions. Also, companies wouldn’t like it when a candidate who knows nothing of their codebase says that they need to rewrite their whole stack with Javascript.

However, at every interview, there’s always that Q&A time in the end. The prepared candidate can shine here. Research the company and prepare a bunch of questions. Show interest in the niche.

I can tell you that this doesn’t work in software as well as another field because even though the candidate asks outstanding questions, they will still flunk the interview if they couldn’t solve the coding problems well. The Q&A is just icing on the cake.

What Is Bodpod and How Accurate Is It?

Fri, 04 May 2018 00:00:00 GMT

Yesterday, I used Bodpod for the first time. It was $10 per scan at my company, which was very cheap compared to the industry standard DEXA scan, albeit less accurate.

You have to wear very tight fitting clothes like a swimwear as to not screw with the measurement.

There is two thirty-second measurement, and the result comes out straight away. You will get bodyweight, lean mass and body fat percentage.

I was quite surprised to see that I had higher body fat percentage compared to what I expected but lost 6lbs since early this year. Does it mean I had even more body fat before? Is it even accurate?

This scientific paper compares BodPod with DEXA scan and find that BodPod tends to underestimate the BF% of high BF individual and underestimate low BF individual. This means that my real BF% might even be higher than what was measured at this time.

BodPod works with evaluating the air displacement in the tank. Breathing or moving might affect the measurement. The system will then calculate the relationship between pressure and volume using predetermined formula. Because of this, the result tends to be less accurate than DEXA which uses x-ray to get high accuracy measurement.

I’m leaning towards just skipping BodPod and using DEXA in the future for tracking progress.

Regardless of the accuracy, this measurement still shows that I’m still far off from my target leanness and will be adding more cardio and HIIT on top of my lifting routine.

5 Proven Ways That Your Brain Is Lying to You - Undoing Project Book Review & Summary

Mon, 23 Apr 2018 00:00:00 GMT

Review

An excellent biography of Behavioural Economics and its fathers. Before Daniel Kahneman and Amos Tversky, economics (and economist) has always assumed that human is mostly rational being. We used to believe that human will make the best decision based on available information at the time, and those decisions reproducible. Now we know that that is untrue. In the words of Dan Ariely, we are ‘predictably irrational’.

You would learn about the relationship of Danny and Amos, how their personality and work dynamic would result in one of the most prominent psychology and economic theory. Lewis interestingly wrote the journey, starting from their childhood to how the war in Israel affects their work together.

Lewis started the book with a story of his ‘Moneyball’ approach in NBA. I think this is the most significant weakness of the book because this chapter does not tie together with the rest. Once you started the next chapter, it felt like you are reading an entirely different book. It would have been better if he removed it.

I love the book and rate it a solid 4 out of 5 stars.

Summary

This book is a hard one to summarise because of its narrative and density. It reads like a joint biography but also contains a significant amount of distillable knowledge. I will try my best to summarise both.

Moneyball in the NBA

In NBA, scouts relied on their gut feeling when selecting new players for drafting. They were not aware of the many biases affecting that feeling, resulting in many bad decisions over the years.

Daryl Morey introduced the new way of analysing players using statistical data. He even avoided interviewing because he thinks the face to face meeting would affect the decision too much. This new approach worked well, but it is still not impervious to human bias. One of the most famous cases was Jeremy Lin, quoted from the book:

“He lit up our model,” said Morey. “Our model said take him with, like, the 15th pick in the draft.” The objective measurement of Jeremy Lin didn’t square with what the experts saw when they watched him play; a not terribly athletic Asian kid. Morey hadn’t completely trusted his model - and so had chickened out and not drafted Lin. A year after the Houston Rockets failed to draft Jeremy Lin, they began to measure the speed of a player’s first two steps: Jeremy Lin had the quickest first move of any player measured. He was explosive and was able to change direction far more quickly than most NBA players. “He’s incredibly athletic,” said Morey. “But the reality is that every person, including me, thought he was unathletic. And I can’t think of any reason for it other than he was Asian.”

The incident led to Morey improving his model even more and added more rules to his scouts. They can only make a comparison if the players are from a different racial background, so more comparing new black athlete with Michael Jordan or comparing Asian athlete with Yao Ming. Most of the time, the favourable comparison disappears.

Myth of Rational Actor, Biases and Heuristics

The belief that people mostly make the right decision given specific information after calculating the utility and value given the cost was prevalent in economics with the concept of Adam Smith’s invisible hand. It influences all sorts of government policy and decision making.

Danny and Amos did not originally set out just to disprove this, but the data steers them towards this conclusion. Along with their studies, they found that people use heuristics to make quick judgments instead of rational thinking. We will explore several examples of them:

1. Confirmation Bias

When a basketball scout likes a certain player, they will organise data to support their findings which is why the first impression is critical. The scouts usually made matters worse because they tend to favour players that resemble their younger selves, even though it is irrelevant to the analysis.

People do this all the time without realising. This bias gets exploited all the time in the political arena. Once a person is committed to a party or candidate, they will start to seek reasons to support their party and reasons to vilify the opponent. Over time, a great divide would polarise the nation further and further away. Two person with opposing viewpoint could look at an event and come to two widely different perception due to their bias.

2. Anchoring

People would anchor their future decision based on arbitrary priors. For example:

Two groups were asked to estimate the result of 8! (eight factorial). One of the group was presented 1x2x3x4x5x6x7x8 while the other got 8x7x6x5x4x3x2x1. The second group’s answer was reliably around five times larger than the firsts.

You could even anchor an answer to a completely random thing, like the last digits of a telephone number. People with telephone ending in 98 would give a much higher estimate compared to people with phone finishing with 06.

3. Present and Hindsight Bias

Present Bias says that people tend to undervalue the future with the present. People prefer cash today even though investing it will net more gains over the years.

The Present Bias is amplified by Hindsight Bias in which people tends to look at an unpredictable outcome and assume it was predictable all along. For example, suppose that the market just crashed last week. The people who bought a new car last month instead of investing the money would think to themselves that their hunch was right and they knew the crash was coming all along.

4. Availability Heuristics

Availability Heuristics is the bias that made many people avoid the news nowadays. People would make the judgement error of assuming that because it is easy to recall the instance of something happening, the more likely is that thing being true.

For example, the news covers death by terrorism en masse, thus increasing its ‘availability’. People who watch the headlines would think terrorism is on the rise and causing massive damages, even though the actual number of death is minimal compared to under-reported heart disease deaths.

Owen Shen made a fantastic graphical representation of Death: Reality vs Reported in which he compares the amount of reporting compared to the actual deaths. You should read his article in its entirety.

image credit: Owen Shen

These heuristics lead people to both ‘excessive fear and unjustified complacency’. Thus we have to be very careful with it.

5. Loss Aversion, Prospect Theory and Endowment Effect

These three heuristics are tightly intertwined. Endowment Effect is when people value things that they have much more than things that they do not, just because they own them which leads to Loss Aversion, which says that losses hurt more than gains feel good. We can estimate the pain of losses or joy of gains with this graph:

image credit: wikipedia

They conducted a study with the following questions:

First situation: You have $1,000, and you must pick one of the following choices: Choice A: You have a 50% chance of gaining $1,000, and a 50% chance of winning $0. Choice B: You have a 100% chance of winning $500.

Second situation: You have $2,000, and you must pick one of the following choices: Choice A: You have a 50% chance of losing $1,000, and 50% of losing $0. Choice B: You have a 100% chance of losing $500.

People would overwhelmingly pick B for the first situation and A for the second situation. This trend follows the asymmetrical value distribution of the Prospect Theory curve.

More Cognitive Biases

To learn more cognitive biases, please visit this Betterhuman article

Danny and Amos

Both were Israeli geniuses. Amos served as paratrooper while Danny escapes Nazi-occupied France to design a lot of the Israeli army psychology tests.

Amos was very confrontational while Danny avoids confrontation. Amos was an extrovert while Danny introvert. Amos was an outright optimist, while Danny was a pessimist.

Amos used to say, “when you are a pessimist, and the bad thing happens, you live it twice. Once when you worry about it, and the second time when it happens.” while Danny thinks that when you expect the worst, you are never disappointed.

They had very little in common besides having studied at the Hebrew University in Jerusalem together. However, they proved to be each other’s perfect match, until almost to the end.

Amos got the more prestigious job offers from Harvard and Stanford (which he took). Danny got a job at University of British Columbia, Vancouver, Canada which is of lower prestige compared to Amos. After their falling out, they begin to act like a divorced couple, trying to stay out of each other’s way and started to work with other people.

Daniel Kahneman won the Nobel prize in Economics for his works on human behaviours, which might have been shared with Amos Tversky had Amos not been taken away by cancer. Nobel prize does not get awarded posthumously. Kahneman wrote one of my favourite books of all time: “Thinking Fast and Slow” which explores a lot of the heuristics and biases that we talked about here but in much more detail.

Danny and Amos’s works arguably shaped a lot of current policies. Changing the default in retirement savings increases the saving rate accumulation in general population. People also start to pay attention to car accident rate when drivers were distracted on cellphones or drive too fast, resulting in law change to make road travel safer.

The Experience of Moving From Australia to Work in Silicon Valley

Wed, 11 Apr 2018 00:00:00 GMT

image credit

I moved here November last year from Australia. There are many questions from friends in Aussie, but the most frequent are:

“How’s life in Silicon Valley?”

Now that I have lived here for four months, I would try to paint the picture for you.

The Work

I was expecting a lot, but workplace has exceeded my expectation.

The calibre of the people here is incredible. My big boss is a professor at a world-class university. My manager was one of the original authors of a world-famous javascript web framework that I used to use back in Aussie. There are many PhDs in my team with papers which have been cited thousands of times. We have a weekly meeting discussing the latest scientific journals in Machine Learning and AI that mostly just went over my head.

It is glorious.

I have learned so much in just four months. We have people who wrote entirely new data structure just because there aren’t any which could solve our unique problem. Some people have 3-D printed custom enclosure, designed and printed PCBs, then coded the micro-controller and putting it all together, just for fun.

You hear about people like this on the internet. Being on the same team and working together with them is something else. The situation forces you to bring the best out of yourself and to step up to their level. It is challenging and fun.

The feeling that your everyday work helping to hundreds of millions of people navigate their modern life is very fulfilling. Because of the scale, the bugs that only happen to 0.1 percent of customers would mean millions of people having a bad experience. The highs are higher, but the pressure is much higher too.

I am fortunate to be working with a team that deal with many parts of the ecosystem from the desktop, the phone to the watch. I have reached out to many people whom with questions about their domain and everyone has been responding with pure kindness. One guy even travelled from another office to my office just to help me solve a problem because we could not do it online.

Now the office. It is not without its flaws, but it is the single most impressive office building that I have ever seen. Full of greenery and flowers everywhere. Everything is designed to work and look good together. You have to visit it to feel its grandiose. I might be able to give you a tour.

With that said, our team still needs more software engineers. If those above descriptions look great to you, please get in touch :)

The Valley

Let’s get the most significant negative out of the way: the price of housing. It is currently taking a large chunk of my after-tax income. Renting a small two bedroom apartment costs double the amount I paid for my large four bedroom house mortgage in Aussie. Silicon Valley is indeed one of the most expensive places to live in the world.

Aside from the housing price, life is pretty good here. You will get access to Amazon Prime, which doesn’t sound like much, but the two-day free shipping is light years ahead of Aussie. I got a gigabit internet just because I can, with the same price I paid for 100/40 in Aussie. The food is generally around the same price as Aussie, but with more substantial portions. Moreover, there are always unlimited drinks at any fast food restaurants.

Car price is low here compared to Aussie. People drive pretty badly. They rarely use the turn signal correctly. They ignore the speed limit and driving 15mph above the legal limit is a daily occurrence.

Driving on the wrong side of the road was disorienting at first, but you get used to it in a week or so. The unit conversion will take longer. Converting km to miles are approximately one and half times. Celsius to Fahrenheit does not make sense because they are not linear. Your best bet is to use certain milestones e.g. 70f == 21c, 60f == 16c, 50f == 10c and estimate from there.

My wife and baby seem to like this place a lot too. All the libraries here have baby classes where parents can take their babies in for story time, music class and art classes for free. My wife has been making friends with other moms this way.

One of the most problematic things is the concept of Credit Score. Since we just moved here, we do not have any credit history. Because of that, getting a credit card or car loan is tough. The only way I could get those now is because I could show them the offer letter from my company, which apparently helps a lot. It will be a couple of years before I could get good enough history to get a mortgage.

The future

Personally, I would love to stay here if things keep going well like this. There are still many things to try and places to explore.

The first four months here just makes me realise that the Aussie tech scene could improve substantially.

I think this is a perfect place to raise kids because they would grow up in a great environment, surrounded by kids from smart families.

Join me!

Wisdom of Insecurity - Book Review & Summary

Tue, 10 Apr 2018 00:00:00 GMT

Review

This book is full of pseudo-mysticism and pernicious nostalgia. It glorifies eastern philosophy at the expense of enlightenment ideology that brought the world all the technological advancement that we enjoy today.

The central thesis of the book that humanity is less happy now that we have strayed from religion and trapped by consumerism. However, is it true?

Steven Pinker’s Enlightenment book comes with 72 charts of human civilisation advancement. Chief among them is happiness and life satisfaction data:

source: https://ourworldindata.org/happiness-and-life-satisfaction

If the research is valid, it proves that we are not descending into atheism hell. Instead, we are getting happier.

Sure, the absolute number of people who are depressed and having a hard time finding meaning in life might be increasing, but we have to look at it in context. The world population has exploded significantly over the past several decades due to advance in medicine, so the absolute number of almost all demographic has increased with it.

The next big problem is how people deal with lack of faith in religion. The book postulates that modern people try to fill the void with stuff. Hedonic treadmill is a well-studied phenomenon, but it is not an exclusively modern one. The poor religious peasant in medieval Europe would desire an ever larger homestead and farmland. Once they have gotten that, they would be happy for a while before adapting to the new home, then repeat the process all over again. Their lack of ability to get rich is the great deterrent to hedonism.

The modern problem is that since we have more economic mobility and much wealthier compared to the peasant, people could afford larger homes and faster car. Because of this ability, people would spend a more substantial portion of their resources on the hedonic treadmill.

Instead of going back to the old eastern religion and philosophy, humanity could look forward to a different belief system like humanism. We could look at Star Trek and see what can we do to achieve interstellar travel. Let the past go. We have been using religion as a crutch for thousands of years without much to show for it.

In conclusion, this book is not malicious. It is just operating under false assumptions and doesn’t try to put things into modern context. We can still learn something from this book.

Rating: 3 out of 5

Summary

What is the meaning of life?

Religion offers strict doctrines that govern how people live their life. People feel more fulfilled when they follow the rules. Do good stuff according to the book, and one will live forever in heaven. The eternal afterlife is a very useful carrot and stick.

During the industrial revolution and rapid technological advancement, the world seems to move away from mysticism. Many people lost their sense of direction. When they lose faith in religion, they lose their compass in life. Why do we have to suffer in this life if there isn’t any afterlife to look forward?

The lost souls attempt to fill the hole with drugs, excessive partying or to become workaholics. The more they try to stimulate themselves, the more stimulation they need to feel alive. They have become desensitised. Thus the cycle of addiction continues.

Consumerism

We are not here in this world to buy and consume stuff. New car and new clothes are fun for a while, but they do not give life much meaning.

Without religion to guide them, people fell into hedonic treadmill where they always want to get the bigger house and the faster car. It is a treadmill for a reason. The thing they chase is designed never to get caught. We are playing a rigged game.

We train our children at an early age to follow the rules. Study hard to get good grades. Get good grades to go to college. Go to college to get a good job. Get a good job to make lots of money. Make lots of money to can buy stuff. Buy more stuff to be happy.

What happens when they are still unhappy at the end?

What if a child wanted to become a nurse to help others, but everyone around them steers them to go to medical school?

Pleasure & Pain

We want the sweet feelings without the bad. We want the joys of life but not the pain. These two are interconnected, and it all depends on perspective. We need to feel the lowest low to appreciate the highest high.

Just like Passenger sang on his song ‘Let her go.’

Well, you only need the light when it’s burning low Only miss the sun when it starts to snow Only know you love her when you let her go Only know you’ve been high when you’re feeling low Only hate the road when you’re missing home Only know you love her when you let her go And you let her go

Learn to view all events as temporary. Neither good nor bad. In Zen Buddhism, letting go of everything indeed is the path to enlightenment.

Let go of the past, embrace the present. Don’t worry about how the future is going to impact the present, just enjoy the now.

Sometimes thinking hard about a problem will bring you no closer to the solution. Instead, you should go for a walk in the park, or take a long hot shower. Ideas to solve your problem might emerge serendipitously. Inspiration comes when we aren’t thinking hard about it.

Grit - Book Review & Summary

Sat, 07 Apr 2018 00:00:00 GMT

Review

Grit is an excellent book with useful and actionable messages about what makes people successful. The primary hypotheses of the book centres around the concept of grittiness, which is the willingness to power through a challenging situation. The higher your grit score is, the more likely that you would succeed in your endeavour.

This idea is a modern rehash of the classic stoicism principles with a massive amount of anecdotes. If you are familiar with Seneca and Marcus Aurelius, this book has little to offer you aside from the modern context.

The redeeming quality of the book is the stuff other than just the concept of grit, where we would learn what kind of other attributes that make someone grittier than others.

Rating: 4 stars out of 5 stars

Summary

In general, people agree that effort is more important than raw talent. However, when given a choice to recruit either a talented or determined candidate, they would overwhelmingly choose the talented candidate, which seems to show that we have an innate preference towards the talented.

Effort

The “Effort” is a multiplier for both skill and talent, which means you would gain greater benefit when you increase your effort. To me, this doesn’t mean much if we agree that innate talent is a constant, and skill is a derivative of talent. The only variable in the equation is the effort, so it is the only one you can influence.

A super talented athlete who trains with low effort might get beaten by mildly talented one who works hard. However if the former trains with higher energy, she might be unbeatable by the latter.

The moral is that since talent is static, just don’t worry about it. If you put high enough effort, you would be good at something. The problem is that talent puts a ceiling for your accomplishment. If you are ok with not being world class, feel free to put effort into an activity that you aren’t talented at. For me, it is writing ;)

Sticking with it

Consistency is key. Doing small thing frequently is much better than doing a big thing once in a while.

A big goal of becoming a doctor is too high-level to visualise easily. Break down the goal into mid-level goals, e.g. pass the exam, and then to low-level goal, e.g. study chapter 12-15.

You will much more likely to stick with your plan if you love what you do, thus doing something you like is the key. On a poll, only 13 percent of US workers feel engaged with their work. With that attitude, it would be tough to improve consistently, which is good news because this means you have 87 percent chance to outperform your coworkers for the promotion if you love your job.

People have more than one thing they love. You don’t have to search for the ‘one true thing’ to love. Just something good enough would do. The quest for searching ‘one true thing’ follows a diminishing return curve.

One great predictor of ‘one true thing’ is that if you can think the calling to benefit other people. Jobs that is low pay, but serves the community tends to have much higher satisfaction, e.g. school teacher and zookeeper. These jobs give a greater sense of purpose in life and belief that they are making the world a better place.

If you are not a zookeeper, don’t despair. Even within the same job, different people would find a different meaning.

Practice

Now that you found the thing that you love, and you are having fun doing it, how do you improve?

There is a famous saying in software engineering field:

you could have ten years experience, or 1 year of experience repeated ten times.

The most significant difference is ‘Deliberate Practice’ where you go over the things you suck at and keep doing it until you improve. Malcolm Gladwell’s Outlier explored this concept with the 10,000 hours deliberate practice rule.

One thing you need to do to practice deliberately is to keep track of progress. If you are lifting weight, write down your reps and sets. Make charts to see your progression. If there is a particular lift that you aren’t progressing, change your routine so that you are doing extra volume on the muscle group.

Having a master of the craft to give you constant feedback is critical. By yourself, you won’t be able to notice slight mistakes that get repeated over an extended period. A master would see that and help you correct your course.

How to raise your kids to be gritty

Since talent is static, never tell your kids that they are brilliant. Smart kids who don’t know the value of hard work tend not to endure when things get hard.

Instead of saying, “You’re a natural, I love that,” you should tell your kids, “You’re a learner, I love that!”

Immersing yourself in a gritty culture helps you to be grittier. Lets say your personal running record is 5 miles, if you run with a group of friends who usually runs 8 miles, you would be working really hard to keep up and finish the 8 miles.

Set examples of a gritty culture for your kids. Make it a rule for the entire family to self-select one ‘hard thing’ to stick with for a determined period. Let the kids select it, and your job is to help them keep doing it until the time is up and to make sure they don’t quit.

If little Jane selected Basketball but wanted to quit after two weeks, kindly let her know that she can only stop after finishing the season. If little Sophia learns ballet, she has to stick with it until the scheduled talent show.

Of course, the parents need to set a good example and show grit, or it will all be for naught.

Using Goaccess to Replace Google Analytics feat. Caddy and Docker

Thu, 05 Apr 2018 00:00:00 GMT

Goaccess is a log analyser that can be used to visualise your website traffic.

Why use it?

It has three advantages compared to traditional web analytics like Google Analytics or Piwik.

Goaccess analyses the web server logs directly, so it doesn’t need to load javascript file. This will speed up the page load significantly.
All the data is stored in your server, not on Google’s server. You own it.
You can access it either using CLI or opening the generated html file.

Disadvantages

Harder to setup; you can’t just copy paste the javascript loading codes to <HEAD> section.
Very basic tracking mechanism. You can’t set customer sales funnel or drill down on granular data.

For my use case, where I just wanted to know the unique visitors number, referrer, etc. It is good enough compromise.

Dockerizing Goaccess with Caddy

The official Docker image contains good tutorial on how to use it.

The first step is to either build and run it, or just to use docker-compose. Any self-respecting Docker user will use the compose, so let’s take a look.

docker-compose.yml

version: '2'
services:
     caddy:
		   # other stuff
         volumes:
             - ./docker/caddy/logs:/etc/logs <- IMPORTANT
             - ./docker/goaccess/.html:/site/goaccess <- To view it on subdomain
     goaccess:
         image: allinurl/goaccess
         restart: unless-stopped
         volumes:
             - ./docker/goaccess/data:/srv/data
             - ./docker/goaccess/.html:/srv/report
             - ./docker/caddy/logs:/srv/logs

./docker/caddy/logs:/etc/logs This will share save the Caddy server logs to system disk folder, which in turn will be shared and read by Goaccess Docker image.

./docker/goaccess/.html:/site/goaccess This will share let Caddy server to serve the generated Goaccess html file on a subdomain

The Goaccess compose script is very standard, just following the default config on the Docker hub page.

Caddyfile

in Caddyfile, we just need to add this line to your desired host server to save the generated logs to disk:

log / /etc/logs/requests.log "{combined}"

This means we are logging all request (/) into /etc/logs/requests.log using the "{combined}" format. Combined is used here because it contains referrer and user agent.

You can read more information on the log syntax on the official docs.

If you wanted to access the generated html file on a subdomain, just add this to the Caddyfile:

 www.goaccess.example.com {
     basicauth / admin password
     root /site/goaccess
     gzip
 }

This will serve the generated index.html on the subdomain. Basic auth is important here if you don’t want just anyone to be able to see your analytics.

goaccess.conf

We need to specify the config file for Goaccess to tell it which file to load, like so:

 log-format COMBINED
 log-file /srv/logs/requests.log
 output /srv/report/index.html
 real-time-html true

COMBINED is used here because we tell Caddy server to generate Combined log format.

You are done, just run docker-compose up -d to spin up Caddy and Goaccess.

When you access www.goaccess.example.com you should see something like:

I hope this helps!

The Wonders of Caddyfile

Mon, 01 Jan 2018 00:00:00 GMT

It is incredibly easy to do a lot of things with Caddy server. We will explore reverse proxy, wildcard HTTPS cert and catch-all www redirection. Then we will see how to combine them together to do powerful stuff in one short Caddyfile.

Reverse Proxy

To reverse proxy a subdomain to docker container, all you need is:

 www.subdomain.augusteo.com {
     proxy / wordpress:8888 {
         transparent
     }
 }

With this, the subdomain will point to the wordpress container’s port 8888. If you aren’t using docker and just running things on localhost, you could replace that with localhost:3000 or whatever port you are running the app from.

Wildcard HTTPS

I’m hosting my blog on Vultr and uses its DNS, so I could use it for wildcard cert.

hugo.augusteo.com {
     root /site/augusteocom
     gzip
	   tls {
        dns vultr
     }
 }

This config will serve augusteo.com from the root path and initialise wildcard HTTP TLS using the Vultr DNS. To enable this, you need to either:

download caddy binary with Vultr plugin

Redirect all non-www to www

Its very easy to redirect one domain with the redir keyword, but its a bit tricky to do it for all subdomain at once:

 http://*.augusteo.com, https://*.augusteo.com {
     redir {
         if {host} not_starts_with www
         / https://www.{label1}.augusteo.com{uri}
     }
     gzip
     tls {
         dns vultr
     }
 }

We could use the not_starts_with function to check if non-www domain is being requested, then do 301 redirection to www version of it. We can also specify the tls here so all the subdomain get encrypted with the wildcard.

Combining them all

Now this is the current Caddyfile that I’m using on this server:

 # Main domain
 http://augusteo.com, https://augusteo.com {
     redir {
         if {host} not_starts_with www
         / https://hugo.augusteo.com{uri}
     }
     tls {
         dns vultr
     }
 }

 hugo.augusteo.com {
     root /site/augusteocom
     gzip
 }

 # Subdomains
 http://*.augusteo.com, https://*.augusteo.com {
     redir {
         if {host} not_starts_with www
         / https://www.{label1}.augusteo.com{uri}
     }
     gzip
     tls {
         dns vultr
     }
 }

 www.subdomain.augusteo.com {
     proxy / wordpress:8888 {
         transparent
     }
 }

This file would:

redirect all non-www to www page. We had to do it twice because the subdomain . doesn’t cover the main domain.
reverse proxy the subdomain to docker container
use wildcard HTTP TLS cert on all domains.

To add new subdomain, we would just need to add the reverse proxy or the root path of it, e.g.:

 www.subdomain2.augusteo.com {
     proxy / rubyonrails:3000 {
         transparent
     }
 }

 www.subdomain3.augusteo.com {
     root /site/jekyllstatic
 }

Running Caddy server extremely simple compared to running Nginx and a lot faster compared to Apache. Learn it and love it.

Summary of Fooled by Randomness

Mon, 01 Jan 2018 00:00:00 GMT

Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets by Nassim Nicholas Taleb

There are professions with low randomness (e.g. dentist) that can only be successful with skill; and professions with high randomness (e.g. day trader) that could see someone unskilled get successful purely based on luck and variance.
People often mistake luck and variance for skill, i.e. survivor-ship bias.
The absence of evidence is not evidence of absence. Just because we have only encountered white swans so far, doesn’t mean that all swans are white. It only takes one black swan to prove the hypotheses wrong.
The scientific method can only assess falsifiable hypotheses. Good hypotheses can be proven wrong quickly, like by a single black swan.
Theory of evolution postulates survival of the fittest, not the best. Sometimes things that survive the longest is the sub-optimal choice, e.g. QWERTY keyboard.
Nature and life are non-linear. We can build a sandcastle with a million grains of sand, yet adding another would bring the whole thing down.
The human mind is not evolved to handle sophisticated statistics and randomness. We depend on heuristics, e.g. availability heuristics where we value things that are easily visible more.
The heuristics cause us to attribute the wrong thing as a cause. e.g. our success is caused by our skills, but failure is bad luck
We are driven by emotion which helps us to make a quick decision, even though it is sub-optimal. Emotion influences our rational thinking rather than the other way around.
It is a lot easier to find ways to avoid triggering emotion rather than try to fight it; e.g. checking share portfolio once a month instead of daily to avoid impulse sale.
Hindsight bias is another big one that we need to be aware. Past events always seem predictable and easier to handle than they were at the time. We naturally find patterns where there were none.
Overfitting is when we try to find pattern, rule or formula to fit historical data too much that it becomes less able to predict future events.
We need to account for an unexpected event (black swan event). e.g. stock market crash and climate change.
Look for both chance and magnitude of an event. e.g. 99.9% chance of winning $1 with 0.1% chance of losing $10,000 looks good on paper, but the expected outcome is a loss of ~$9.
Too often we only looked at the chance and not enough of the magnitude. Rare events could be catastrophic.
Embrace beautiful randomness, e.g. poetry, art.
Stoicism philosophy is a great way to deal with randomness. We do not control randomness, but we manage our reaction to it. Expect it, don’t complain or blame. Face variance with dignity and wisdom.
Reading daily news takes a lot of effort for little reward. Therefore we should stop doing it. If the news is essential, it will travel through the grapevine to reach you somehow.
Same goes for checking portfolio every minute. We might be affected by small variance and let our emotion take over.
Learn the Monte Carlo simulation. I found this great website to visualize portfolio return. The website also have plenty of examples to play around.

Lake Tahoe Winter

Mon, 01 Jan 2018 00:00:00 GMT

![lake-tahoe-winter-2a7712aee623e81ddc03bbb183488669-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-2a7712aee623e81ddc03bbb183488669-thumb 1.jpg)

![lake-tahoe-winter-3aec308ca999f7a169dcfad468694256-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-3aec308ca999f7a169dcfad468694256-thumb 1.jpg)

![lake-tahoe-winter-411df3bdc4ccc5001654363881974b98-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-411df3bdc4ccc5001654363881974b98-thumb 1.jpg)

![lake-tahoe-winter-454a84bedb3b98841ceb05bca244f0e5-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-454a84bedb3b98841ceb05bca244f0e5-thumb 1.jpg)

![lake-tahoe-winter-4ea602f06bfc24c2adcd8e50d9dc794f-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-4ea602f06bfc24c2adcd8e50d9dc794f-thumb 1.jpg)

![lake-tahoe-winter-6478c494f8ee2d0090c8d04f24cb1f97-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-6478c494f8ee2d0090c8d04f24cb1f97-thumb 1.jpg)

![lake-tahoe-winter-8092907e6fed2bd9470f7fb20fdc3610-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-8092907e6fed2bd9470f7fb20fdc3610-thumb 1.jpg)

![lake-tahoe-winter-8a8ee1fca2d0b8fc9c1c144ac97b7697-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-8a8ee1fca2d0b8fc9c1c144ac97b7697-thumb 1.jpg)

![lake-tahoe-winter-a94d4a3160037a5ebafc4575711b4fa4-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-a94d4a3160037a5ebafc4575711b4fa4-thumb 1.jpg)

![lake-tahoe-winter-cd5b47e5b8ee42ae6cfbe089f83f021a-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-cd5b47e5b8ee42ae6cfbe089f83f021a-thumb 1.jpg)

![lake-tahoe-winter-e03a3318af598b49c2c0af0e9b37c869-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-e03a3318af598b49c2c0af0e9b37c869-thumb 1.jpg)

![lake-tahoe-winter-ec1592f7680f0932ab4ecd90ac399426-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-ec1592f7680f0932ab4ecd90ac399426-thumb 1.jpg)

![lake-tahoe-winter-fde7a6448c1aa892b5773843511b06c0-thumb 1.jpg](https://www.augusteo.com/assets/blog/lake-tahoe-winter/lake-tahoe-winter-fde7a6448c1aa892b5773843511b06c0-thumb 1.jpg)

License:

Running Gitlab CI With Docker

Sat, 18 Mar 2017 00:00:00 GMT

Recently I had the opportunity to play with Gitlab CI and had quite a bit of fun on the process. I wanted to share my experience in a tutorial format to share the fun with you :)

This tutorial assumes no prior knowledge of Docker, but you need to know what CI is and how SSH works.

Summary:

Setup .gitlab-ci.yml file to tell the CI what to do
Setup Gitlab CI with docker
Create your Docker image and put up on Docker Hub
Use the newly created Docker image to speed up CI cycle

Background

Our purpose is simple, and we want every single commit on dev branch to be rsync-ed to the server with an rsa ssh key. We are dealing with static files, so rsync is enough.

Now Gitlab offers a built in CI, but we still need to set up its runner, instructions and the docker images. Let’s get to it.

Setting up Gitlab CI

Config file

If you haven’t run any CI in Gitlab before, you need to set up the instructions first. To get started, just open up your project on Gitlab and click on Set up CI button.

It will bring up a new file screen with the name .gitlab-ci.yml pre-filled. Type this into it:

image: centos:latest

before_script:
  - yum install -y which
  - 'which ssh-agent || ( yum install -y openssh openssh-server openssh-clients openssl-libs )'
  - 'which rsync || ( yum install -y rsync )'
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh
  - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

staging-deployment:
  stage: deploy
  script:
    - rsync -hrvz files/* root@yourhost.com:/var/www/html/
  only:
    - dev

This file might not make sense now, but I promise by the end of this tutorial, you will understand it. Let’s go over them.

Line 1 tells the CI runner to fetch the Docker image centos:latest from Docker Hub . If you aren’t familiar with Docker, just think of it as a container of all the stuff you need to do a particular task. Please have a read the What is Docker page, but it might raise more question than it answers. We will explore it more later.

Any commands inside before_script will be executed before every CI cycle, so this is a good place to set up our SSH and dependencies.

Line 4–6 will install which, openssh and rsync to run our CI.

Line 7–10 sets up the SSH key that will be used by rsync. We should never put a private key into a repo, so we will be using Gitlab’s Secret Variable to store it. It can be referenced using the $SSH_PRIVATE_KEY line. We will set it up soon.

Line 12 sets up the name of the CI task. In this case we call it staging-deployment because we are deploying to staging environment. Then on the next line, we tell Gitlab that this is a deploy task.

Line 15 is the meat of the CI, which is the actual deployment. Rsync will use the ssh key that was set up previously and will start syncing the files from your repo to the yourhost.com server. The -hrvz modifier means:

/h/: human readable
/r/: recursive, so subfolders get synced too
/v/: verbose, so we can find out what went wrong from the logs
/z/: compressed while transferring, to speed up and use less bandwidth

Line 17 tells Gitlab only to execute this task when a commit is pushed to devbranch and not others. This will enable us to setup CI tasks for other branches as well. For example deploying to production server when a commit is pushed to release branch. Alright now that we understand this file, lets hit Commit Changes.

Setting up secret SSH variable

Once you have committed the CI config, click Settings and CI/CD Pipelinesand scroll down until you see the Secret Variables. Type in SSH_PRIVATE_KEY on the Key field and paste in your private key. You can easily copy the private key to clipboard using cat ~/.ssh/id_rsa | pbcopy. Then save them.

I am assuming you have already copied the public pair to the deployment server. If you haven’t, please follow this Gitlab tutorial .

Setting up CI runner

Now that our CI setup is done, we need something that will actually run the commands. The cheapest way would be to spin up a small linux VPS. You can provision one for $2.50/month from Vultr .

Head over to the Gitlab runner page for installation instructions for each of your operating systems. Make sure you select docker as the executor and enter centos:latest when asked for Docker image.

After this is done, refresh your CI/CD page on Gitlab and you should see the runner shown. For example:

If it doesn’t show up, try to read the runner documentation for help or ask around.

First CI task!

Now you have set up the runner and CI config, and you are ready! Since our config states that the runner will only be executed when a commit is pushed to dev branch, you should do that then.

$ cd test-gitlab-ci
$ mkdir files
$ cd files
$ vim index.html // and type in your html
$ cd ..
$ git checkout -b dev
$ git add .
$ git commit -m ‘initial index.html’
$ git push origin dev

Gitlab CI should detect the commit and initiate the task. Head over to Pipeline -> Jobs to see the job being executed:

Hopefully things run well, and you should see a job running. You can click on the job to see the details.

Turning on the Docker service

If the CI job fails and when you check the logs, you see something like:

ERROR: Preparation failed: Get http://unix.sock/v1.18/version: dial unix /var/run/docker.sock: connect: no such file or directory*

It means you have forgotten to turn on the docker service in the CI server. Just SSH into the runner machine and start the service. It should either be:

$ sudo service docker start

$ sudo systemctl *start* docker

Once the daemon is up, you can rerun the failed task again and it should pass.

What you have achieved so far

So up to this point, you have:

Set up Gitlab CI config file
Set up Gitlab CI runner
Run your first commit to dev branch
Fix an error!

Things seems to be going well, but there’s something that we can improve very easily.

If you see the running time of the CI, you should see that it runs for more than one minutes. That is because the vanilla CentOS box doesn’t come with rsync and openssh out of the box, so we need to install them. That process gets repeated every-time thus sucks a lot of time. We can easily fix this by creating your own Docker image of CentOS but with rsync and openssh preinstalled.

Creating your Docker image

First, lets install Docker. If you are on macOS, your can download the Docker app to handle all that for you. If you, look on the left sidebar for your operating system.

Then lets create a new folder and make a Dockerfile

$ mkdir my-docker
$ cd my-docker
$ vim Dockerfile

Type in this, but change the MAINTAINER to your own name.

FROM centos:latest
MAINTAINER Augusteo

RUN yum -y update; yum clean all
RUN yum -y install openssh openssh-server openssh-clients openssl-libs rsync which; yum clean all

To build the Docker image, just type in

$ docker build -t my-docker .

You should see it fetching the centos:latest and then installing all our dependencies. So next time you run your CI using this new image, it will skip the installation and goes straight to the deployment. In my case it cuts the deploy time from more than one minute to less than twenty seconds.

Pushing to Docker Hub

To use your new image, we need to push it to Docker hub. The process is pretty straightforward, just follow this:

If you want to skip all this work and wanted just to use a ready made image, you can use mine:

https://hub.docker.com/r/augusteo/centos-rsync-openssh/

Changing the Gitlab CI config

Now we need to change the Gitlab CI config file to make use of the new Docker image and remove the dependencies installation. Edit your .gitlab-ci.yml file to look like:

image: augusteo/centos-rsync-openssh

before_script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh
  - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config

staging-deployment:
  stage: deploy
  script:
    - rsync -hrvz files/* root@yourhost.com:/var/www/html/
  only:
    - dev

You can see that we have changed the image to the new CentOS image with rsync and openssh. If you have pushed your own image, feel free to change this to your own.

In before_script, we have removed the three dependencies installation lines since we have already done it on container level. Just commit and push this file, and you should see that your CI execution time reduced to less than twenty seconds.

Summary

You have learned:

Setup .gitlab-ci.yml file to tell the CI what to do
Setup Gitlab CI with docker
Create your Docker image and put up on Docker Hub
Use the newly created Docker image to speed up CI cycle

I hope you enjoyed this article and it helps you in your development life!

Fixing Slow Macos Finder Samba

Sat, 04 Mar 2017 00:00:00 GMT

As a bonus, this will also enable searching on Samba too.

Problem

I have a external HDD plugged into a Raspi which acts as a seed box of some kind. The files are served with Samba. The problem was:

Listing large amount of file in a directory takes ages.
Search doesn’t work at all, it just got stuck at “Searching”.
Transfer speed was very slow.

Solution

After much searching, the fix is quite simple actually.

First, SSH into the server with $ ssh serverIp

Then edit the Samba config by typing $ vim /etc/samba/smb.conf

Once it is open add these lines to your particularly slow Samba shares:

vfs objects = fruit
fruit:aapl = yes
fruit:encoding = native
fruit:locking = none
fruit:metadata = stream
fruit:resource = file

Lastly restart the server with $sudo service smbd restart

Result

Now the listing large directory should be pretty fast, search should work as normal and transfer speed should be around 10x faster.

References

Xcode Layout Constraint Issue

Mon, 20 Feb 2017 00:00:00 GMT

My typical solution for non-repeating view controllers nowadays is by using vertical UIStackView that is embedded within UIScrollView.

So the stack goes:

Main UIView -> UIScrollView -> UIStackView -> Views array

This stack is very easy to set up in storyboard or code and constrained with auto layout.

Naturally, the four sides of the scroll view will be constrained to the main UIView. The easiest way would be to use the tie-fighter icon. Notice that the top and bottom is glued to the layout guide instead of superview by default.

The problem arises when we embed this view controller inside a navigation controller, thus creating a navigation item bar. This action will then push the stack view (inside the scroll view) down 64 pixels, seemingly without reason.

If you constraint the scroll view to the top layout guide, the child stack view will be pushed down 64 pixels. That is 20 pixels for status bar and 44 pixels for navigation item. So you would need to change the constant to –64 pixels to stick the stack view to the top of the view. It is a fix but seems more of a hack than a real fix.

Why is this happening?

This is caused by constraining the scroll view to the top layout guide instead of the superview. But there is no obvious way to constraint the top of the scroll view to the top of main UIView.

The common effort of using ctrl+drag will only offer you to constraint the scroll view to the top layout guide. Using the tie-fighter icon will do the same as well.

After you set up the constraint to the top layout guide and tried to edit it, there’s no way to change it to the superview either.

So how the heck do we constraint the scroll view to the superview?

It is not obvious. After some searching, I found out that we can click the little arrow button (dropdown?) when setting constraint constant from the tie-fighter icon. From there you can select if you want to constraint the view to the superview or the top layout guide.

Voila, problem solved.

What should have happened

Given the hidden nature of the feature, maybe Xcode could have done better to inform us about this option.

When ctrl+dragging, a small tooltip should show up to a ask us if we want to constraint either to the top/bottom layout guide or the superview. This wouldn’t be a big problem because it should only be shown to a view that is directly under the main UIView. The other nested views should work normally.

When editing an existing constraint, there should be an option to change from top layout guide to superview and vice versa. Currently, the only way is to delete the constraint and recreate them.

Xcode could have done better.

How to Read 62 Books in a Year

Fri, 20 Jan 2017 00:00:00 GMT

From 2012 to 2016 I’ve read 310 books, which averages to 62 books per year. All the while working full time jobs, learning new languages, traveling, three weddings (to the same bride, of course), watched heaps of movies and TV shows, etc.

The point is that reading lots of books can be effortless. You don’t have to suddenly transform yourself into a reclusive bookworm. I’m going to share how I did it. It may or may not work, but give it a try and you might surprise yourself

“A reader lives a thousand lives before he dies, said Jojen. The man who never reads lives only one.”― George R.R. Martin, A Dance with Dragons

Track your reading

The first step is to set a goal of how many books you wanted to read this year. You don’t have to go straight to 52 (one a week). Start small. 12 is plenty and it will give you a full month to read one book.

I use two tools for this:

Goodreads reading challenge You should register a Goodreads account and start your challenge here:

2017 Reading Challenge

1178957 people participating - Motivate yourself to read more books: Enter the 2017 Reading Challenge!

Writing down books that you wanted to read I’m a fan of Bullet Journal and on New Year’s day, I wrote a list of books that I wanted to read for the year. Here’s the 2017 listing:

Here are some list of places where you can find inspiration:

Ok now that you have setup the challenges and list, you should start reading more.

Read whenever you can

Hours of our life are spent on commutes and chores that doesn’t require our full attention. You should read/listen to audiobooks during those:

Commute to and from work
Gardening, lawn mowing
Washing dishes, vacuuming and mopping the floor
Walking your dog

Many people would just browse Facebook or the news while commuting on the public transport. While that satisfies your boredom, it isn’t really beneficial for your mind. You should use that time to read books instead. For this purpose, getting a good ebook reader would help you greatly as they are lighter and easier to carry. I use the Amazon Kindle Voyage that’s more than 2 years old but still works well. If you prefer the printed paper, then by all means carry the paper book.

If you are driving, then use the time to play audiobook on your car’s stereo!

You would notice the rest of the listed activities, hands and eyes are undisposed. This brings me to the next point:

Audiobooks— Learn it, live it, love it

The major advantage of audiobook is that you free up your eyes and hands to do other thing while listening. This technique will be the main contributor of your upcoming prolific reading streak as it enables you to read while doing other stuff.

Invest in a pair of nice noice cancelling headphones if you can. I highly recommend the Bose QC35. It is wireless, so you don’t have to worry about cables. It might seem like a minor thing, but you won’t believe the difference it will make. If the price seems high to you, there are numerous deals going on throughout the year. I use OzBargain to track discounts. Bought it at 20% off RRP.

No matter what headphone that you get, get used to wearing it. Doing chores at home won’t be boring now!

Speed listening

Once you load up your phone with audiobooks, you should try listening at faster speed.

Start at 1x, then 1.5x, then 2x. I usually have 2x as base speed and increase according to the books. Some narrator has a calmer voice that you can listen at 3x while still understanding the words. Some books like the Witcher series are narrated with many voices, so I stayed on 2 to 2.5x for such books.

Many players offer up to 3x playback speed. I personally use:

Summary

To recap, there aren’t any secret to this. Those tips will help you, but the work is still in your hands.

Track your reading with Goodreads or journals
Read whenever your can, listen to audiobooks whenever u can’t

So how many books will you read this year?

Start Contributing to Open Source

Sun, 27 Nov 2016 00:00:00 GMT

Just recently my first PR to ReactiveCocoa has been merged and I’m still excited.

I have been using ReactiveCocoa/ReactiveSwift since the start of 2016 and enjoying it very much. The learning curve is not easy, but once you grok it, its incredible.

This brings us to contributing to open source. IMO the easiest way would be:

Use open source codes
Find functionality that you wanted but is lacking
Fork the repo and make a new branch with the changes along with the tests
Submit Pull Request with you new branch
Action the change requests and code styling that many repo enforces
Voila!

That’s what I did in the ReactiveCocoa PR. I wanted to bind attributedText of a UITextViewfrom a MutableProperty<NSAttributedString> but the framework doesn’t support that.

textView.reactive.attributedText <~ attributedStringProperty
  .producer
  .map {
    // whatever
  }

Without the binding, I have to use didSet { } on the property change, which isn’t that bad, but could be better.

So my colleague suggested to submit PR with the changes that I wanted, and I did!

Further resources

There are several guides for open source newbies:

First Timers Only

Github Guide

Another Github Guide

Good luck!

Reactive Way to Handle Dismissing Modal View Controller

Fri, 18 Nov 2016 00:00:00 GMT

We use modal presentation quite often in iOS. You must be familiar with:

self.present(vc, animated: true, completion: nil)

Now lets say we want something to happen with our current VC when the modal VC is dismissed, there are several ways to do that.

The Cocoa way would be to use presenting view controller, delegate pattern or notification center.

Lets go through each briefly.

Presenting view controller

This option is the worst because the child/presented VC needs to know quite a lot of detail about the parent/presenting VC. That should almost never be the case. The child should know as little about the parent as possible. Therefore lets avoid this:

class ModalVC: UIViewController {
  func someFunc() {
    dismiss(animated: true, completion: {
      if let parent = self.presentingViewController {
        parent.doSomething()
      }
    })
  }
}

The next pattern is a direct improvement over presentingViewController.

Delegate pattern

Create a protocol with a void method and get the parent VC to conform to it.

protocol ModalHandler {
  func modalDismissed()
}

class ParentVC: UIViewController, ModalHandler {
  func modalDismissed() {
    // do something
  }
}

Then set the ParentVC as delegate of ModalVC before presentation and call the method during dismissal.

class ModalVC: UIViewController {
  func someFunc() {
    dismiss(animated: true, completion: {
      delegate.modalDismissed()
    })
  }
}

This way is better that using presentingViewController because the child doesn’t know about the parent, but only the protocol/interface as a layer of abstraction. I recommend using this pattern in a non reactive projects.

Notification pattern

With this, we need to get ParentVC to observe the modal dismissed notification.

class ParentVC: UIViewController {
  override func viewDidLoad() {
    NotificationCenter.default.addObserver(self, selector: #selector(ParentVC.handleModalDismissed), name: NSNotification.Name(rawValue: "modalIsDimissed"), object: nil)
  }

  func handleModalDismissed() {
    // Do something
  }
}

class ModalVC: UIViewController {
  func someFunc() {
    dismiss(animated: true, completion: {
      NotificationCenter.default.post(name: NSNotification.Name(rawValue: "modalIsDimissed"), object: nil)
    })
  }
}

I don’t like notifications because both parent and modal VC needs to know the notification name to work.

Reactive pattern

When you use ReactiveCocoa/ReactiveSwift, you could easily handle modal dismissal with several lines of codes in the presenting VC. This is my preferred way because the child doesn’t even need to do any extra work. All the heavy lifting is done by ReactiveCocoa.

class ParentVC: UIViewController {
  override func viewDidLoad() {
    let modalVC = ModalVC()

    modalVC.reactive
      .trigger(for: #selector(onboardNav.viewDidDisappear(_:)))
      .observe { _ in self.handleModalDismissed() }

    present(modalVC, animated: false, completion: nil)
  }

  func handleModalDismissed() { }
}

In here, the ParentVC will observe modalVC before presenting. When modalVC calls viewDidDisappear(_:), ParentVC will call self.handleModalDismissed().

As you can see, it is very clean and quite easy to understand even if you aren’t as familiar with reactive concepts.

Easy Functional and Integration Testing with Mock Server | Tutorial and Example

Thu, 27 Oct 2016 00:00:00 GMT

You can download the presentation slides here.

Basic Testing

Increasingly, more and more mobile apps rely on some kind of api/backend server to work. Be it to save user data, fetch new data, or syncing live chat. Testing api integration usually require mocking/stubbing the network calls.

There are three basic layers of testing:

Unit test - making sure each function/unit of code works as expected
Functional test - making sure units interact with each other as expected
Integration test - making sure our app integrate with other app/api/services as expected

Mock server helps with all layers of tests, although it is particularly useful for functional and integration.

Why Use Mock Server

1. Reuse

Practically everyone deploys on multiple platform nowadays. Java on Android, JS on web, Swift on iOS and unless you do full hybrid like Xamarin or React Native, you will need to mock the web services for testing in each and every separate languages.

Mock server promotes sharing and code reuse, write once and use to test ios/android/web app. No need to write mock several times for different platform. You can just connect each app to the mock server and done.

2. Reducing Dependency of API

In the times when the API team is delayed/not in sync with the front end team’s sprint, mock server allows us to write our app and test agaisnt the mock server. This also applies when API server is down for whatever reason.

This reduces dependency on each other and reduces coupling, which is always a good thing.

3. Debugging

Sometimes it is hard to reproduce a bug, especially when the bug is triggered by a specific response from the server. You could either modify your code temporarily to mimic that incoming response, or you could just make the mock server return a specific bug-causing response.

4. Predictability and Consistency

When you use remote server to test, and it comes back with error or high latency, it is very hard to determine what’s wrong. How do you know the fault lies with the network connection or with the server?

Since mock server lives on your local machine, it is very easy see if it isn’t working properly.

What is Mock Server

Mock server is just a web server that will give us consistent response for particular request, that’s it. The only requirement is that the request and response format need to be consistent with the production server so that our test wouldn’t be able to tell the difference.

The main benefit is that the mock server is operated and maintained for the benefit of the development team, so you can make it as simple or as complex as you need it to.

It is just another tool in our toolbox. A very helpful one.

How to Use Mock Server

There are several existing solutions that’s quite popular. If you Google ‘mock server’, you will find:

If you love Objective-C, there is an embedded web server called GCDWebServer that will run on your Mac.

You should use whichever stack that your team is familiar with or the stack that the backend team uses. When you have some problem, you could always give them a shout!

For our tutorial, we will just write one from scratch using a combination of:

Basic Node & Express

Make sure that you have installed Node from the link above. For this tutorial, also please install Yarn.

Open up your terminal and start by making an empty directory and going into it.

$ mkdir mock
$ cd mock

Now we can initialize an empty app. When it asks question, you can either enter your app detail, or just keep pressing enter key to use the default value.

$  yarn init
yarn init v0.16.1
question name (mock2):
question version (1.0.0):
question description:
question entry point (index.js):
question git repository:
question author:
question license (MIT):
success Saved package.json
✨  Done in 59.45s.

Now when you list the files with ls, you should see only one package.json file in the directory. Let continue by installing Express and Morgan.

$ yarn add express morgan

After a while, it should download and install both packages. You can check by listing the files. Now we can start writing our basic Express app.

Let’s create a new file called app.js and open with with your favourite text editor. I will give example using Vim:

$ vim app.js

Inside the file, copy paste this block:

// 1
const express = require('express'),
app = express(),
morgan = require('morgan');

// 2
app.use(morgan('dev'));

// 3
app.get('/', function (req, res) {
  res.send('Hello World!');
});

// 4
app.listen(5678, function () {
   console.log('listening on port 5678!');
});

Here we tell Node that we require Express and Morgan, and that we assign app constant to the express module
We tell Express to use Morgan with the profile 'dev'
We tell Express to send 'Hello World!' when a GET request is sent to '/'
Lastly we tell Express to listen to port 5678 and prints a console log with that message

To test this, just run node app.js in the folder, and you should get listening on port 5678! message printed. To test this, you can open http://localhost:5678/ in your web browser to see ‘Hello World!’ printed.

Congrats, your have just built your first Node + Express app!

Now if you check out the terminal window where you run the app, you should see:

GET / 200 2.518 ms - 12

That message is from Morgan and it serves just a logging purposes to tell us when our server get hit by requests.

We will revisit the mock server later, now we will have a look at the sample app that we are going to test.

Favourite Food app

You can clone this repo to get a quick start.

This is a very simple app that will return the name of your favourite food when you put your name on. Right now the app only supports ‘foo’ and ‘bar’.

There are only two noteworthy file in the project: ViewController.swift and Favourite_FoodUITests.swift.

The ViewController manages the network request and UI interaction while the test file, well, manages the UI Test.

There are two schemes; One to hit production remote API, and the other to hit mock server.

Lets run the app using the Favourite Food scheme for now on the simulator. When it run, you can type foo as name and press calculate, it should return Pizza.

It should return Roast Pork Knucle when you enter bar as name.

Now if you rerun the app with FF-Mock scheme, it will crash when you press calculate because our mock server doesn’t know how to handle the request. Let’s fix that.

We will cover the multiple scheme for multiple api in another post.

Mock server for Favourite Food

lets open the app.js of our mock server again.

add this code block just above the app.listen line.

app.get('/3l2z2', function (req, res) {
  let favJson = {
    "foo": "Pizza",
    "bar": "Roast Pork Knuckle"
  };
  res.send(favJson);
});

So your app.js file should look like this now:

const express = require('express'),
app = express(),
morgan = require('morgan');

app.use(morgan('dev'));

app.get('/', function (req, res) {
  res.send('Hello World!');
});

app.get('/3l2z2', function (req, res) {
  let favJson = {
    "foo": "Pizza",
    "bar": "Roast Pork Knuckle"
  };
  res.send(favJson);
});

app.listen(5678, function () {
  console.log('listening on port 5678!');
});

All this block of code does, it to tell Express to return favJson object when /3l2z2 is hit by request. Simple.

Now lets kill the previous node instance and rerun it again. to do this, just go to the terminal window where you run node app.js before and press ctrl+c to cancel it. Then rerun node app.js.

To confirm that the new json is up, open http://localhost:5678/3l2z2 on your web browser to check. you should see:

{"foo":"Pizza","bar":"Roast Pork Knuckle"}

Awesome. Now if you run the iOS project with FF-Mock scheme, you should be getting Pizza and Roast Pork Knuckle successfully.

Intro to UI Test

Lets get back to Xcode and set the scheme to FavouriteFood. Then press cmd+u to run the tests. You will see that the simulator would launch and the app would run by itself.

To see what’s happening, lets open the Favourite_FoodUITests.swift file and take a look at one of the function:

func testFoo() {
  let textField = app.textFields["tf"]  // 1
  textField.tap()                       // 2
  textField.typeText("foo")             // 3
  app.buttons["Calculate"].tap()        // 4
  expectation(for: exists, evaluatedWith: app.staticTexts["Pizza"], handler: nil)
  waitForExpectations(timeout: 3, handler: nil) //5
}

Explanation:

Find the text field with accesibility label tf
Taps the field to bring up keyboard
Types foo
Taps the calculate button
Waits for 3 second for the app to show Pizza text

As you can see it is quite simple to write and run UI Test.

Now since our mock server is up and running, we can change the scheme to FF-Mock and press cmd+u to test it against the mock server. You should see that it is running correctly.

Simple Express Refactor

This express app works well because we only have one endpoint request. Imagine if we have 300 endpoints to take care of, we couldn’t just fit everything into app.js. To make things cleaner, we will refactor the app to different files.

First lets create a new directory instead of your mock directory, then make a file named fav.js

$ mkdir controllers
$ cd controllers
$ touch fav.js

Then open fav.js with your favourite editor and paste this block:

exports.index = (req, res) => {
  let favJson = {
    "foo": "Pizza",
    "bar": "Roast Pork Knuckle"
  };
  res.send(favJson);
};

All this does is exporting a function called index that returns favJson as response. Very similar to the one in app.js

Save this file. Then open up app.js and replace this block:

app.get('/3l2z2', function (req, res) {
  let favJson = {
    "foo": "Pizza",
    "bar": "Roast Pork Knuckle"
  };
  res.send(favJson);
});

With this:

const favController = require('./controllers/fav');

app.get('/3l2z2', favController.index);

This replacement simply tells Express to ask fav.js to perform index function when a GET request is sent to /3l2z2.

If you have more parts in the mock server, such as user management or adding new food, feel free to create new controllers and use the same require syntax.

Summary

We have learned:

Why, What, How of mock server
UITest in iOS
Node + Express app to act as mock server
Refactor Express

What was covered in the talk but didn’t make it to this post:

Using schemes in Xcode to manage multiple URLs
Using Fastlane to run tests against mock server on CI server

Those topics are better served by their own dedicated post.

Simplest Way to Sync Dotfiles and Config Using Git

Sun, 23 Oct 2016 00:00:00 GMT

Most devs have two or more computers that they use. In my case it is my home Macbook and one in the office. We want to have the same config, keybinding or aliases across those machines.

There are several ways to do it. I tried using symlinks to a folder that is synced through git, but some apps won’t load symlinked files. For example, if you try to symlink ~/.zshrc then zsh will load as if you don’t have an rc file.

After looking around for a while, I came upon a post in Hackernews about this issue. It led to this article by Atlassian. It is by far the best solution.

Git Bare

This technique is using git init --bare which I have never encountered before. You don’t really need to understand it to use it, but for reference sake, this is what --help says.

--bare
Create a bare repository. If GIT_DIR environment is not set, it is set to the current working directory.

Setup

That article is very succint and you could just follow the steps laid out there. In summary, use these commands to set it up.

git init --bare $HOME/.cfg
alias config='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'
config config --local status.showUntrackedFiles no
echo "alias config='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'" >> $HOME/.bashrc

Replace the last .bashrc to whatever shell that you are using e.g. in my case it is .zshrc.

Operation

All git operation are permitted, you just have to use config instead of git. For example:

config status
config add ~/.zshrc
config commit -m "Add vimrc"
config add ~/.xvimrc
config commit -m "Add xvimrc"

To sync those files with another computer, you need to setup a repo. I recommend using Gitlab as they have free private repos. After creating new repo in Gitlab, you can do:

config remote add origin git@gitlab.com:yourname/testrepo.git
config push -u origin master

From this point on, you can treat it like normal git repo. One interesting use case would be to create different branches for different settings that you wanted to use.

Setting up new machines

Now that you have the first machine set up and syncing nicely, you can start setting up the second machine by first adding this line to .zshrc or .bashrc.

alias config='/usr/bin/git --git-dir=$HOME/.cfg/ --work-tree=$HOME'

Then clone the repo from Gitlab to the folder:

git clone --bare git@gitlab.com:yourname/testrepo.git $HOME/.cfg

From here, you can pull, push, merge and checkout to your hearts content.

You really should read the source article since it explains the steps in detail except adding remote repo.

Thanks for reading!

iOS10 Crash - New Privacy Permission Settings and Descriptions

Tue, 04 Oct 2016 00:00:00 GMT

So we were getting crashes on development build when our app tries to ask user for camera permission on iOS10. It hasn’t happened on iOS9, and we didn’t know what went wrong. Setting up exception breakpoint on throw and catch doesn’t help either. Apple should really give us more clue on this. I was stuck for a while.

Until my team mate said that it might be the new privacy settings.

Lo and behold, he is right, turns out there are heaps of new privacy permission request description that we have to add now. Prior to iOS10, only location permission was required. The full list are:

NSBluetoothPeripheralUsageDescription
NSCalendarsUsageDescription
NSVoIPUsageDescription
NSCameraUsageDescription
NSContactsUsageDescription
NSHealthShareUsageDescription
NSHealthUpdateUsageDescription
NSHomeKitUsageDescription
NSLocationUsageDescription
NSLocationAlwaysUsageDescription
NSLocationWhenInUseUsageDescription
NSAppleMusicUsageDescription
NSMicrophoneUsageDescription
NSMotionUsageDescription
NSPhotoLibraryUsageDescription
NSRemindersUsageDescription
NSSpeechRecognitionUsageDescription
NSSiriUsageDescription
NSVideoSubscriberAccountUsageDescription

In the case of our app, adding NSCameraUsageDescription into info.plist fix the crashes and display a description message under the App would like to access your camera system alert.

Binary Search Algorith with Recursion and Generics - Swift Tutorial

Fri, 26 Aug 2016 00:00:00 GMT

Following through with my last post, we will explore how to implement the binary search algorithm in Swift. Of course it has to be generic too!

Problem and assumption

This algorithm assumes that we have an array of comparable objects. That is to say, the compiler should know how to compare those objects. If your object is not a primitive type, then you can follow the last post on how to make the struct/class/enum conform to Comparable protocol.

lets say we have an array of numbers:

let arr = [4,1,78,34,666,23,0,999,3245,54]

We can’t just apply binary search to this array because binary search assumes that the array is already sorted. So let’s do it.

let arr = [4,1,78,34,666,23,0,999,3245,54].sort()
// [0, 1, 4, 23, 34, 54, 78, 666, 999, 3245]

Please remember to sort your array before applying the search, because it wouldn’t return the correct result otherwise.

Algorithm

Binary search is very simple. We pass and array and a search target, then it will return the index of the target inside the array to us. We will return an optional nil in the case of target not found.

The process in short, is as follow:

Check if midpoint of the array is our target, if yes, just return the index
If not, check if the target is greater than the midpoint
If yes, search from midpoint to end of array recursively
If not, search from start of array to midpoint recursively

That’s it.

Swift code

I have used generics in this code because this assumes you have read the last post about it :)

func binarySearch<T: Comparable>(array: [T], target: T, range: Range<Int> = 0..<arr.count) -> Int? { // interface
  guard range.startIndex < range.endIndex else { return nil } // 1

  let midIndex = range.startIndex + (range.endIndex - range.startIndex) / 2 // 2
  guard array[midIndex] != target else { return midIndex } // 3

  let newRange = array[midIndex] > target ? range.startIndex..<midIndex
    : midIndex.advancedBy(1)..<range.endIndex // 4

  return binarySearch(array, target: target, range: newRange) // 5
}

Interface

The function will take a generic type T which conforms to Comparable protocol. The array and target is of type T.

We have a Range of type Int and not T because a range just need to act as our index, which usually is Int. We have also added a default value for this param so user of this func doesn’t need to specify the first range. The range is mostly for the recursion to work.

It returns an Optional Int. Nil for not found, and Int for index of target if found.

1. Range checking

This is the first recursion exit point. If the start of the range is not smaller than the end range, that means we couldn’t find the target. So just return nil

2. Index on the middle of the array

We specify the midIndex of the array. We need to add the startIndex + at the start to accomodate the recursion that starts if the target number is larger than midpoint.

3. Found the target!

The function will only proceed when the midpoint isn’t the target. If it is, the function will exit and returns the target index.

4. New range for recursion

When the midpoint is not the target, we have to find which half of the array to search. If the target is greater than midpoint, we will search the right side of the array and vice versa.

5. Recursion trigger

This is where we invoke recursion to re-search the midpoint of the same array, but with the new range that we calculated on // 4. The recursion will keep going until either // 1 hits, meaning the target isn’t in the array or when // 3 hits meaning we found the target and just returns the index.

Testing

So now that we have a working search function, we can give it a try:

binarySearch(arr, target: 23)  // 3
binarySearch(arr, target: 999) // 8
binarySearch(arr, target: 213) // nil
binarySearch(arr, target: 54)  // 5

It worked!

But how do we really check if it work? We can either manually go through the array and count the index of the target, or we can employ the Swift assertion, like so:

assert(binarySearch(arr, target: 23)  == arr.indexOf(23))
assert(binarySearch(arr, target: 213) == arr.indexOf(213))
assert(binarySearch(arr, target: 4)   == arr.indexOf(4))

Here we employed the compiler assertion to check if our searching algorithm returns the same index as the standard library searching function. If the compiler doesn’t complain, it means our work is done!

Conclusion

Binary search is a classic divide and conquer method for a more efficient searching. There are many ways to do it, but recursion is one of the funnest way!

Swift Quick Sort Algorithm With Recursion and Generics

Mon, 22 Aug 2016 00:00:00 GMT

There are many sorting algorithm and many way to implement it. Quicksort is one of the quickest.

Today we will learn how to implement quicksort in Swift with generic and protocol conformance.

Motivation

I was reading Learn You a Haskell book today and one of the example of recursion chapter was quicksort. It just tickled me to implement the same thing in Swift.

Great book by the way, you should consider this Joy of coding bundle for more books.

Sorting numbers

Lets say we have this array of jumbled Int that we wanted to sort

let arrNum = [6, 4, 7, 9, 22, 78, 11, 1, 0, 234]

Now lets define our function interface

func quickSort(array: [Int]) -> [Int] {
  return array
}

That codeblock will just return the unsorted array, which isn’t what we wanted. So lets get it working.

func quickSort(array: [Int]) -> [Int] {
  if array.isEmpty { return [] } // 1

  let first = array.first! // 2

  let smallerOrEqual = array.dropFirst().filter { $0 <= first } // 3
  let larger         = array.dropFirst().filter { $0 > first } // 4

  return quickSort(smallerOrEqual) + [first] + quickSort(larger) // 5
}

Every recursive function needs an escape scenario, else it would go to an infinite loop. In this case, we want the recursion to break when the array that is passed is empty.
We need to store the first element of the array to compare it with the smaller or larger number.
The first half of quicksort is all the values that are smaller or equal to our first number. We can filter the array after we drop the first value.
The second half is all the values that are larger than our first number.
The first and second half would then be recursed and added to the first value.

This graphic from Learn You a Haskell explains the concept perfectly:

Let’s test it!

quickSort(arrNum) // [0, 1, 4, 6, 7, 9, 11, 22, 78, 234]

Yay! It’s working!

Generics

As it is now, the function only works for Int type. What do we do if we want to sort a Double array?

We can either copy paste the whole function and just change the type to Double like so:

func quickSort(array: [Double]) -> [Double] {
  // implementation
}

But then we will have to copy paste it again if we need to compare String, Float or other types.

There must be a better way. And yes there is! We can use Generics

func quickSort<T: Comparable>(array: [T]) -> [T] {
  if array.isEmpty { return [] }

  let first = array.first!

  let smallerOrEqual = array.dropFirst().filter { $0 <= first }
  let larger         = array.dropFirst().filter { $0 > first }

  return quickSort(smallerOrEqual) + [first] + quickSort(larger)
}

As you can see, the implementation of the method doesn’t change. Only the interface and type is changed.

Our function will now be able to sort any types that conforms to the Comparable protocol. Swift already implemented this for most of the primitive types that we encounter in daily basis.

Let’s test it:

let arrStr = ["a", "x", "R", "hey", "c", "Jon Snow"]
let arrDouble = [1.4, 6.7, 9.2, 6.4, 1.4]

quickSort(arrNum)    // [0, 1, 4, 6, 7, 9, 11, 22, 78, 234]
quickSort(arrStr)    // ["Jon Snow", "R", "a", "c", "hey", "x"]
quickSort(arrDouble) // [1.4, 1.4, 6.4, 6.7, 9.2]

It’s working again! However it wouldn’t work if Swift isn’t able to infer the type of the array.

let emptyInt = []
genericQuickSort(emptyInt)
// error: 'NSArray' is not convertible to '[_]'; did you mean to use 'as!' to force downcast?

We can fix it by explicitly defining the type:

let emptyInt: [Int] = []
quickSort(emptyInt) // []

Now Swift compiler wouldn’t complain and will return an empty array as expected!

Comparable conformance

Our generic quickSort() will be able to sort anything that conforms to Comparable protocol. But it wouldn’t be able to sort a custom struct. For example:

struct Dog {
  let id: Int
  let name: String
}

let dog1 = Dog(id: 19123, name: "Bella")
let dog2 = Dog(id: 26234, name: "Ollie")
let dog3 = Dog(id: 31241, name: "Klue")

quickSort([dog2, dog1, dog3])
// error: cannot convert value of type '[Dog]' to expected argument type '[_]'

This happens because Swift doesn’t know how to sort a custom struct that doesn’t conform to Comparable. How do we fix it? By conforming it to Comparable of course!

struct Dog: Comparable { // 1
  // definition
}

func < (l: Dog, r: Dog) -> Bool { // 2
  return l.id < r.id
}

func == (l: Dog, r: Dog) -> Bool { // 3
  return l.id == r.id
}

Conforming to Comparable is as easy as 1-2-3

Conform the custom class or struct to Comparable
Implement the < operation by comparing a property of the struct that already is comparable
Do the same thing with == operation

Step 2 and 3 can be as simple as comparing ID to as complex as comparing all of the properties in the struct. It depends on your needs.

Now let’s test it again

quickSort([dog2, dog1, dog3])
// [Dog(id: 19123, name: "Bella"), Dog(id: 26234, name: "Ollie"), Dog(id: 31241, name: "Klue")]

It’s working!

Conclusion

It is very easy to implement the Swift version of recursive quicksort with generics. The type inference system helps us a lot.

Where to go from here?

Now that you can sort a randomized array, how about implementing a recursive binary sort with Swift?

I will post the answer on future blog post.

How to Host Static Websites Like Hugo on CentOS VPS with Nginx - Tutorial Part 1

Sat, 06 Aug 2016 00:00:00 GMT

Last time, we have learned how to get started with Hugo and run the server on our local machine. That alone is fun, but our friends couldn’t check it out. What’s the use of a blog if only you can read it? The blog need to be served on the internet through a domain name so people can read it easily.

In this tutorial, we will explore the delighful ways of hosting Hugo blog with CentOS, Nginx and side of Let’s Encrypt.

The steps that we will go through are:

Spinning up CentOS VPS instance
Pointing your domain name to the new VPS
Creating user account and setting up passwordless SSH login
Installing NginX and setting up site config
Writing deployment script with Rsync
Setting up HTTPS with free SSL cert from Let’s Encrypt

The contents will be spread out to several tutorial post to make it easier for you to digest.

VPS

We will be using Vultr to host our VPS. Why Vultr? I came highly recommended from my friend Curtis over at CentOS Blog. He is the devops guy at PwC and we used to work together, so I know to just trust his words on server stuff. Who better to ask from than the guy who play with server all day long?

Plus if you register to Vultr with this link, you will get $20 free credit that you can use to spin up server for free!

Creating account and spinning up VPS on Vultr

So lets over to Vultr and create a new account if you haven’t already. It is pretty straightforward and should be pretty standard account creation process.

After you are done, please log in. You should see this screen:

Let’s click the big blue plus button to spin up a new VPS instance. It should say Deploy New Server when you hover over it. Next screen will ask you series of questions regarding your new server:

On the top, select Compute instance

Location: Select the one close to you or your readers for better latency
Server Type: Select CentOS 7 x64
Server Size: Select the $5 option as it should be enough for our needs now. If you start getting millions of users, then we can easily scale up.
Additional Feature: You can check the IPv6 and leave the rest unchecked, they are all optional.
Startup Script: Leave it empty
SSH Keys: Add your RSA public key here or just leave it empty if you don’t know what it means
Server Hostname & Label: Enter your domain name on the first field or just leave it blank if you don’t have it now

All done, now let’s just press the Deploy Now button on the bottom.

You should be taken back to the dashboard with your list of server. The new server’s status will be shown as Installing for a couple minute. Just wait for it until it says Running.

That’s great! Now we have a server running with its very own IP address. It’s yours. Unique. The only one in the world.

But we don’t have anything on the server yet, so if you tried to open the IP address on browser or curl, you won’t get anything back

➜  ~ curl 45.32.190.53
curl: (7) Failed to connect to 45.32.190.53 port 80: Connection refused

Don’t worry, once we set up NginX on the server, everything will be fine.

Registering domain and pointing it to our server

I have just registered augusteo.xyz for this tutorial because it is the cheapest TLD I could find: only AUD 0.50. This tutorial will not cover the process of domain registration and I will assume you have your own domain name to use.

Let’s login to your domain registrar management page. The example shown is specific to CrazyDomain as it is where I register my .xyz domain name, but it will be easily applicable to any other registrar.

Find the option to change/edit your domain’s nameserver:

Change the first and second entry to:

ns1.vultr.com
ns2.vultr.com

And hit save.

Now lets get back to Vultr management page and select DNS on the top menu bar.

Let’s press the Add Domain button. In the Domain field, just enter your domain name. In my case I would enter augusteo.xyz. In the IP field, enter the IP address of your Vultr VPS instance. You can get it from the Instances list.

When you are happy, just press Add. You should see the finished DNS entry page:

We are all done with domain! Now if you access augusteo.xyz it should point to your new Vultr VPS server. The DNS propagation could take up to 24 until servers around the world knows where to point it to. You can check the propagation using What’s my DNS web app. While it is propagating, we can move on to setting up our server.

Summary

In this tutorial, we have learned how to spin up a new CentOS VPS and setting up DNS to point domain name to that VPS.

The next tutorial will go through:

Secure your CentOS server
Create your new user account
Adding RSA key for passwordless login
Installing NginX and setting up configs
Installing Let’s Encrypt Free SSL certs

Stay tuned!

How To Run Pokemon Go Bot On The Cloud With AWS EC2

Tue, 26 Jul 2016 00:00:00 GMT

Pokemon Go (PoGo) bots today works really well. It can run around spinning Pokestops and catch Pokemons for you automatically. You just need to tell them what to do.

The most popular PoGo bot right now is the Necro Bot. It only runs on windows so those of you who doesn’t have a Windows machine are kind of screwed.

Or are you?

There are three ways to run this bot:

Get a Windows machine and run it there 24⁄7 which might not be very helpful to your electricity bill
Run it on a Windows VM on your Mac or Linux machine
Run it on VPS such as AWS EC2 which runs all the time, accessible everywhere and very affordable (even free!)

I actually have a gaming PC with Windows, but it is too overpowered to just run this small app 24 hours a day. So let’s do it on Amazon’s server!

What do you need?

This guide is mostly for people who wants to run the bot on AWS EC2. It is totally fine to run it on your windows machine if you didn’t want to bother with AWS. Just go to the NecroBot wiki and follow the instruction there.

If you have never used AWS before. Great news, you can do all this for free. Amazon has the free trial for the first year for new customers. You could also just register a new email and get the free offer again.

So please head to AWS website to register or login.

Launching EC2

Once you inside the console, please click on Services -> EC2 on the menu at the top.

Let’s press the Launch Instance button to start. Scroll down until you see the one called:

Microsoft Windows Server 2012 R2 Base

And click on Select.

On the next screen, let’s go with the free tier one. Then please press Review and Launch

Then just press Launch when you are ready.

When prompted for key pair, select Create a new key pair and give it a memorable name, such as myKey. Download the key and press Launch Instance

That’s it. Your first cloud server is being initialized and will be ready in several minutes.

Click on Services button at the top and select EC2 again. This time you should see 1 Running Instances.

Click on that link and you will see one server listed on the table.

If the Status Checks still says Initializing then you just got to wait until it’s ready.

When it is ready, the text should change into 2/2 checks... with a green tick. It’s time to RDP into it!

Getting login details

Now please click on the only server you have and click the Connect button at the top.

Click on Download Remote Desktop File to get the .rdp file then click on Get Password.

In the next screen, click on Choose file and select the myKey.pem file that you downloaded earlier before launching the instance.

Once the key is populated, just click Decrypt Password. Now write down this password as you will need it to login to your instance.

Remote access into the instance

If you are on a mac, you will have to download the app Microsoft Remote Desktop from the Mac App Store.

On windows, you should have the Remote Desktop Connection app installed already.

Now please run the .rdp file that you downloaded on the previous step. It should run the RDP app and logs you into the server. Enter the password you get before and you are ready to proceed!

If this is your first time doing RDP, just treat the remote desktop like any other Windows machine. Pretty neat huh?

Downloading the bot

From the inside of your remote desktop, just click the start button and select Internet Explorer. You should see the warning about enhanced security configs. This means that IE will prompt you if you really wanted to access a webpage for your safety. So when you are prompted, just click Accept or Add to enable access.

Now let’s go to the NecroBot Installation Wiki and read the instructions. When you understand what to do, lets click on the Release.zip link.

On the next page, just click on the first Release.zip under Downloads.

If you get an error message saying Your current security settings do not allow this file to be downloaded. then just click the gear icon and go to Internet Options

From there go to Security tab and select Custom Level and scroll down until you see the disabled download. Just switch it to enable.

Now try to download the file again and you should be able to. Once it is done, just click on Open Folder

There you have it, the bot is extracted and ready to be configured. Just follow the steps outlined in the NecroBot wiki and you should be on your way to leveling up!

Multiple bots!

You can run multiple bots using the same windows instance. Just make a copy of the Release folder and change the login details. In no time you will be running hundreds of bots!

So that’s it. Good luck and let me know if you have any questions!

Super Quick Guide to Start Using Hugo

Sun, 24 Jul 2016 00:00:00 GMT

Wordpress is still the defacto CMS for blogging or just any simple website generally. I’ve created many Wordpress site both for clients and myself. With the plethora of plugins, it can do pretty much anything from e-commerce to paid-membership site.

So after all, why would you choose to change?

Before I knew about static site generators, the choice was between using CMS of some kind versus running straight up html/js/css. The former has a lot of advantages, but it is heavier and requires more maintenance. The latter is lighter, easier to run, but requires a lot of technical know how to code them.

Static site generator

The static site generator sits happily in the middle of those two. It is surprisingly easy to use and quite delightful, suprisingly.

There are many choices to explore and you can have a look at them at StaticGen. The choices really just boils down to preference as most of them works really well.

I chose Hugo mainly because it uses Go language instead of Ruby or JS like the others. Nothing wrong with those, it’s just that I wanted to try out new things.

There is a really good getting started guide by the folks at Hugo and I will just go through several of the most useful functions that it has.

Installation

It is very easy to install hugo if you have Brew on your mac. Just run this:

brew update && brew install hugo

After a while, you will be ready to go!

Initiating a new site

After installation, just cd to where you want your new project to be and run:

hugo new site {name of site}

It will produce a list of files to start.

We will be mainly playing with themes and config.

Installing theme

From the main folder, just do cd themes to move into themes directory and install any themes that you want. You can either choose one from the GoHugo theme list or you can clone my theme by running:

git clone https://github.com/augusteo/ghostwriter

I couldn’t find a theme that I really liked so I tinkered one out. This theme is combination of Ghostwriter and Cactus theme with several twist to make it cleaner and more minimalist.

after the theme is done, just edit the config.toml file to include the line

theme = "ghostwriter"

and you are done!

Writing a blogpost

This is my favourite part of static site gen. No longer do we need to go to the web browser or a specialized writer to start blogging. In fact I am writing with Vim now!

Markdown syntax that Hugo supports are a lot simpler compared to HTML tags and a lot faster to write.

Publishing

After writing your blogpost, depending on the theme, you might need to call undraft to make it visible on your Hugo site.

Once you make sure that they are undrafted, just run hugo server on the main directory and you should see the server running.

Now when you open localhost:1313 on your web browser, you should see your new blog displayed!

Deploying

I will write an article soon about spinning up a new CentOS VPS, installing Nginx and writing a script to automate deployment with Rsync and SSH. In the future we will also explore hosting the site on AWS S3 and CloudFront.

Summary

Hugo and the other static site generators does what it promises. By serving static files instead of generated pages and database fetches, you will save a lot of server cost in the long run. The more traffic you have, the more money you will save. Moreover, since static files doesn’t require processing at all, it will be served at a much faster speed compared to traditional CMS.

The best benefit in my opinion is that you can write markdown anywhere, edit and publish your articles without touching a web server.

By enabling CI or deployment script, getting your writing to the internet has never been easier!

How To Simulate GPS Movement in iOS

Sun, 10 Jul 2016 00:00:00 GMT

Be warned: doing this on Pokemon Go might get your account suspended for cheating.

Motivation

Since the release of Pokemon Go, the world has went insane catching those cute monsters. Myself included. Its all fun and games until you realize that if you live in the rural area, there’s not much that you can do.

I live 25km away from Melbourne CBD and there are two Pokestops within 5km of my house. Compare this to the CBD where there are hundreds of Pokestops and Gym everywhere. It’s ridiculous.

How about needing to walk 10km to hatch one Poke egg? I understand that it is good to have a walk around, but in the cold rainy winter day, I’d prefer to stay home.

There must be a better way to enjoy this game than driving 25km to city just to play. Right?

Solution

We are going to use the Xcode debugger to simulate GPS location and catch all the Pokemon!

Preparation

– very basic iOS development experience – Mac machine with Xcode installed (it’s free) – iPhone with Pokemon Go installed – Apple developer account to run app on your phone – text editor with regex find and replace function, such as Sublime Text

Steps

– draw the map that your trainer will walk through – process the map file into .gpx file that Xcode will accept with regex – create iPhone app that will be used to inject the .gpx file – catch ‘em all!

Now let’s get started

Drawing the map

Lets visit this website first: GPSies track creator

Then put 3 in Speed (mph) field to simulate average walking speed. If you wanted to test the limit, I’ve tried 6 mph and it still counts for hatching eggs.

After that, search for the big city that you wanted to catch your Pokemon in. In my case, I entered Melbourne

Zoom in until you can see the roads clearly and you can start clicking to draw waypoints. I’d recommend to draw around 5-10km worth of waypoints to reduce repetition and looping. The longer the better.

This is my track:

After we are done with drawing tracks, scroll down and find the Export file section and press download

Congrats, you have done drawing your map and exporting it!

Processing the gpx file

Now open the .gpx file that just been downloaded with Sublime Text. It should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<gpx xmlns="http://www.topografix.com/GPX/1/1" xmlns:gpsies="http://www.gpsies.com/GPX/1/0" creator="GPSies http://www.gpsies.com - GPSies Track" version="1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd http://www.gpsies.com/GPX/1/0 http://www.gpsies.com/gpsies.xsd">
  <metadata>
    <name>GPSies Track</name>
    <link href="http://www.gpsies.com/">
      <text>GPSies Track on GPSies.com</text>
    </link>
    <time>2016-07-10T05:47:48Z</time>
  </metadata>
  <trk>
    <name>GPSies Track on GPSies.com</name>
      <trkseg>
        <trkpt lat="-37.8117843" lon="144.9566173">
          <ele>62.000000</ele>
          <time>2010-01-01T00:00:00Z</time>
        </trkpt>
          ...
        <trkpt lat="-37.8118521" lon="144.9564456">
          <ele>59.000000</ele>
          <time>2010-01-01T03:27:06Z</time>
        </trkpt>
      </trkseg>
  </trk>
</gpx>

The problem is, we can’t just use the file like that. Xcode wouldn’t understand it. So lets get pumping!

First just replace all this part:

<gpx xmlns="http://www.topografix.com/GPX/1/1" xmlns:gpsies="http://www.gpsies.com/GPX/1/0" creator="GPSies http://www.gpsies.com - GPSies Track" version="1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd http://www.gpsies.com/GPX/1/0 http://www.gpsies.com/gpsies.xsd">
  <metadata>
    <name>GPSies Track</name>
    <link href="http://www.gpsies.com/">
      <text>GPSies Track on GPSies.com</text>
    </link>
    <time>2016-07-10T05:47:48Z</time>
  </metadata>
  <trk>
    <name>GPSies Track on GPSies.com</name>
    <trkseg>

With:

<gpx>

Then at the end of the file, just delete the lines:

    </trkseg>
  </trk>

You should have something like this now:

<?xml version="1.0" encoding="UTF-8"?>
<gpx>
  <trkpt lat="-37.8117843" lon="144.9566173">
    <ele>62.000000</ele>
    <time>2010-01-01T00:00:00Z</time>
  </trkpt>
  ...
  <trkpt lat="-37.8118521" lon="144.9564456">
    <ele>59.000000</ele>
    <time>2010-01-01T03:27:06Z</time>
  </trkpt>
</gpx>

That’s great! What follows is a bit tricky and not as straightforward. We will have to change the format of the data from this:

<trkpt lat="-37.8117843" lon="144.9566173">
  <ele>62.000000</ele>
  <time>2010-01-01T00:00:00Z</time>
</trkpt>

into this:

<wpt lat="-37.8117843" lon="144.9566173"><time>2010-01-01T00:00:00Z</time></wpt>

Doing them one by one is possible, but it takes a lot of time to say the least when you have hundreds of points. To make things simpler for us, let’s use Regular Expression or Regex. If you have never used Regex before, it is just a language rules that we can use to select, modify, extract or otherwise manipulate texts.

Now inside Sublime Text, open up the Find and Replace menu by pressing cmd+shift+f.

Click on the leftmost icon with a dot and asterisk symbol on it. There should be a popup saying Regular Expression when you hover over it.

Then in the Find field enter: \s+<trkpt\s(.+)\n.+\n\s+(.+)\n.+

And in the Replace field enter: \n<wpt $1$2</wpt>

You should have it looking like this:

Then just press Replace and OK when asked. Your waypoints should be transformed into Xcode readable file.

If you think what happened is totally ridiculous and looked like sorcery, you might be right. You can read the short explanation by going to Regex101, but I won’t go into details here. Regex is big topic by itself and there are many people smarter than me that has covered it in detail.

Your final gpx file should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<gpx>
<wpt lat="-37.8114622" lon="144.9647283"><time>2010-01-01T00:00:00Z</time></wpt>
...
<wpt lat="-37.8116487" lon="144.9645996"><time>2010-01-01T01:19:44Z</time></wpt>
</gpx>

Now you are ready to make your injector app!

Location injection app

The purpose of this app is just to inject the gpx file and run in the background while we play Pokemon Go. Thus it must be: – always be plugged in to the computer with Xcode – as light as possible to take up less memory – always run in the background

The first point is pretty easy. Second point is also easy since we will just have one empty controller with nothing on it.

Now the third point is tricky. We can’t just run a new empty app as iOS will kill it after several minutes of no activity to save battery and memory. We have to force it to keep running by using this trick:keep requesting location update in the background

With this two, I was able to keep the app running for several hours in the background while I play Pokemon Go.

Project Setup

Open Xcode and press cmd+shift+n to create a new project. Choose the Single View Application to start with and enter a name for your app. You can put anything here as it doesn’t matter. For example I named it LocTest

After the project is created, we have to do several things:

Backgrounding Capabilities

Go to the capabilities tab and turn on Background Modes. After that, turn on the Location checklist.

Info.plist

Open the Info.plist on the project navigator and add these two keys in the plist:

The first line enables the location permission prompt. Second line just help your app to not get closed during backgrounding.

Import files

There are two files that we need. One is the processes gpx file that we did previously.

Drag and drop the two files into your project navigator and select the Copy items if needed box.

Codes to prevent backgrounding

Open AppDelegate.swift and below import UIKit:

import CoreLocation

Make the AppDelegate class conforms to CLLocationManagerDelegate protocol

class AppDelegate: UIResponder, UIApplicationDelegate, CLLocationManagerDelegate {

Add the locationManager property below the window property:

var window: UIWindow?
let locationManager = CLLocationManager()

Add this function into the class:

func location() {
  locationManager.delegate = self
  locationManager.allowsBackgroundLocationUpdates = true
  locationManager.desiredAccuracy = kCLLocationAccuracyBest
  locationManager.requestAlwaysAuthorization()
  locationManager.startUpdatingLocation()
}

Finally just call the function inside didFinishLaunching:

func application(application: UIApplication, didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?) -> Bool {
  location()
  return true
}

And the app is done!

Just launch it and you should see a white screen with a popup asking for location permission. Just tap on Allow.

If it doesn’t ask you for permission, just go toSettings -> Privacy -> Location Services -> 'your app name' -> Always

There are better ways to debug this, but for the purpose of this tutorial, let’s just ignore the fact that we have a bug.

Injecting the location file

While the app is running, you should see the debugger console on the bottom half of Xcode. It looks like this:

If it’s not there, press cmd+shift+y to bring it up.

Click the arrow button and you should see your gpx file listed:

Click on it and your app is done!

Catch ‘em all

Now you can go to your iPhone and press home button to background the injector app. It should still be running in the background.

Open the Pokemon Go app and you should see your trainer walking around your pre-set path, passing through all the Pokestops and Pokemons!

I hope you enjoy the tutorial and let me know if you got any questions on the comment box below!

GPS error

Even though we have tried to make the app run in the background, sometimes iOS will still kill it after a long period of time. In my testing this is several hours. You will see an error in Xcode console that says:

Message from debugger: Terminated due to memory issue

Pokemon Go app will either: – show GPS error – reset your GPS to your real location

When that happens, just rerun the injector app and re-pick the gpx file. You will be good to go again!

I have not been able to find a more elegant solution that keeps iOS from killing the app. Let me know if you know a better way!

Summary

You have learned how to: – create a map waypoint – parse the gpx with regex – create app that will stay active in the background by – playing an empty mp3 file continuously – continously updates location in the background – cheat on Pokemon

Where to go from here? – create another map path on beaches/port to catch that Magikarp (for Gyarados of course) – use this technique to test (or cheat) all other GPS based apps – write a comment in the box below 🙂

Thanks for reading yo!

Software Craftsmanship

Sat, 09 Jul 2016 00:00:00 GMT

This post is a companion of my talk for Melbourne Mobile Meetup. You can get the slides here.

In the early years of software development, large numbers of software projects tends to be over budget and over time. The community then tries to bring in engineering practices to help make software projects predictable. Thus ‘Software Engineering’ was born.

The problem with engineering is that it:

Treats all developers equally as resources or automatons even though we know that developers are vastly different to each others
Assumes highly degree of control and predictability, but software projects is often unpredictable
Requires approved body of knowledge, certifications and college degree before you can call yourself engineers. In software, there are many highly achieving developers with only high school diploma and no more
Generally has end-goal set in advance. For example, a bridge will have to connect point a to point b. The two point will be unlikely to change. Contrast this with software project that changes on daily basis

Craftsmanship

Craftsmanship places the craftsman in the center of the process.

Craftsman are people who:

Dedicates their working life to getting better at their craft
Learns their skills through apprenticeship to a master craftsman
Aims for mastery of the craft
Teaches others through apprenticeship
Builds a reputation based on what they have delivered and not what exams they have passed

Passion is often a positive modifier for craftsman. I argue that passion is optional as you could succeed in a craft without passion although it definitely help.

But with the rise of industrialization there are fewer and fewer craftsmen. One of the biggest remaining groups is computer programmers.

~ Paul Graham

Software Craftsmanship

The concept isn’t new. The classics embraces it.

Pragmatic Programmer comes with ‘Journeyman to Master’ subtitle, which reflects the classic craftsmanship journey.

Software Craftsmanship: The New Imperative talked about the situation that our profession is facing right now. “As the demand for software has exploded, the software engineering establishment has attempted to adapt to the changing times with short training programs that teach the syntax of coding languages. But writing code is no longer the hard part of development; the hard part is figuring out what to write. This kind of know-how demands a skilled craftsman, not someone who knows only how to pass a certification course.” still true 15 years after the book is published.

Apprenticeship Pattern is similar to the design patterns and refactoring books. it goes through many patterns that we can follow on the path to mastery.

There is also a sequel to the Agile Manifesto, aptly titled Manifesto for Software Craftsmanship where you can sign and list your name on the page.

Stages of Craftsmanship

We all started as a Novice in a craft. As we pick up a few skills, we become Initiate. Once an initiate feels she wanted to pursue the craft, she should find a master to apprentice to.

Apprentice

In the medieval Europe, an apprentice who wants to learn new craft will have to look for a master to study from. Thankfully for us, with the advent of the internet and printing press, it’s easy to learn from the masters of the craft through books, videos and courses.

In modern times, I consider university student to be on apprenticeship stage. Just starting their craftsmanship journey, learning from the professors.

Journeyman

“The journeyman is focused on building an ever-larger portfolio of applications that demonstrates his progress in the craft; he moves between projects and masters, seeking to diversify and deepen his portfolio; he seeks to elevate his status within the community; and he strives to become ready to be a master.”

~ Apprenticeship Patterns, p. 31

That quote perfectly describes this stage.

Master

In the old days, a expert journeyman has to produce a ‘Masterpiece’ that has to be accepted by other masters before allowed to join the guild.

In software craftsmanship, this would be the people who has mastered their craft, produced masterpieces and shared their superior skills to advance their professions and the wider world. There is no certification for it. We can’t just copy someone else’s masterpiece either, because you have to create your own.

On scale of 1-10, on a good day, you might rate yourself as 8s or 9s. but once you see the works of the masters, you realize the scales goes up to 100s or 1000s.

Linus Torvalds is one of those people. his masterpiece is, of course, Linux and Git which changed the whole computing landscape forever.

My other favourite masters are Chris Lattner for creating LLVM, CLang, and Swift; and Richard Stallman for creating the GNU projects & the Free Software Foundation that started the open-source movement.

There are many other masters that has directly impacted our lives through their masterpieces, and all of us should aspire to do better than them.

Growing

Growing one’s skill is an integral part of any craftsman’s career. As we touched before, thanks to the internet, it is not very easy to learn from books, videos and courses from great masters.

Books

Despite the newer technologies, books are still the most time and cost effective way to learn new skills. There are several books that is a must for every aspiring software craftsman to read:

Refactoring

And books that I have read and personally recommend:

Swift

Practice

The old saying “Practice makes perfect” is only half true. Not all kinds of practice are created equal. Studies have shown that Deliberate Practice is the superior method to practice.

In other word, the practice has to be about improving specific skill, repeated, with feedback from expert and must be challenging. There is no point to keep practicing a skill that has already been mastered.

One way to practice software skills is to do Code Kata.

This way of practicing comes from the martial arts world. When a Karate practitioner is alone, he can’t practice Kumite (sparring). To practice, he can do a Kata, which is a set pattern of movements involving stepping, punching, kicking and other moves.

Code Kata is similar to that. You will solve a short programming exercise that can be done in multiple ways. Those katas are usually 15-60 minutes.

Remember to always get feedback for your practice to minimize errors and avoid forming bad habits.

Imagine that you have two groups of people. on the left you have team mango. on the right team avocado. they are about to attend a lecture.

Team mango was told that they will have to teach the material to others. Team avocado was told that they will be tested on this.

Which team do you think scores better on a test?

“When teachers prepare to teach, they tend to seek out key points and organize information into a coherent structure, our results suggest that students also turn to these types of effective learning strategies when they expect to teach.”

~ John F. Nestojko

Team mango performs significantly better than team avocado.

This is based on a study in St Louis Washington University

Even the intention to teach will improve your retention. So I want you to approach your learning from now on with different perspective.

We teach to learn more effectively.

Blogging

I know most of us codes everyday and can’t really be bothered to write after we get home, But blogging/writing is one of the most cost effective way to share knowledge. It benefit you as you need to deconstruct what you have learned and structure it.

Personally I have been blogging on and off for several years on different topics. Never consistently. I am planning to change that with the new revelation about the effectiveness of sharing and craftsmanship. There will be consistency.

It helps to keep me accountable.

Brown Bag Sessions

This is where you gather your work colleagues and do knowledge sharing over take-away lunch (usually in brown paper bags). An intimate atmosphere where you know most people and conducted in familiar place. After you’ve done a couple of those, try to find a larger forum to share. For example, Meetup group.

Meetup Groups

The easiest way to share with more people is to find a meetup group in your niche and register. You can then ask the organizer if they are still looking for people to speak. They usually are. When it is confirmed, just prepare yourself and practice, then deliver it confidently.

Summary

Software development is more craft than engineering because it relies more on the skill and experience of the craftsman than the process of engineering.

A good craftsman will always strive to grow themselves through continuous learning and sharing.

Beside helping others, sharing will also help you because it forces you to learn effectively and understand the material more.

Thanks for reading and I hope this essay benefit you.

Streamlining API with Swagger.io

Sat, 12 Mar 2016 00:00:00 GMT

This article is a companion of the talk I did for Melbourne Mobile about Swagger.

You can find the slides at SlideShare. I’m skipping majority of the contents covered in the slides, so please read through them first =)

What is Swagger?

It is a group of projects under the Swagger banner that works together to ease the pain of developing and documenting API. In this article’s scope, we are only going to play with Swagger-editor

Installing Swagger-editor locally

Just run these commands in your terminal of choice:

git clone https://github.com/swagger-api/swagger-editor.gitcd swagger-editornpm start

The ‘npm start’ will take quite a bit of time to start because it has to install all the dependencies and compiles the app. When it is done, it will launch a web browser with your locally installed Swagger-editor.

Swagger specs to write in the editor

When the editor launches, we can start writing our Swagger spec into it and see how it will be displayed on future Swagger-ui. If you are wondering about the syntax that Swagger can understand, please head over to the OpenAPI github page. That page can also be accessed from Swagger-editor; just click Help -> Swagger 2.0 Specs.

Play around with it for a while :)

Now put this code-block into the editor panel:

Our API specs for Hello World and sum two numbers

The code should be pretty self-explanatory. The only thing that isn’t obvious would be the property operationId, which means nothing in Swagger-UI now but will be used as method name in the code generated for server and client stub.

Now the UI (right hand side) should display the API in an easy-to-understand format. The ‘Try this operation’ will fail now, because we do not have a server running on port 4567 to handle the request yet.

Generating server and client stub

Now click on the ‘Generate Server’ and choose ‘Sinatra’. Wait for your web browser to download the zip file.

After that, please do the same for ‘Generate Client’ and ‘Swift’.

Fleshing out Sinatra implementation

When you unzip the generated server stub file, you should be getting this folder structure:

.├── Gemfile├── Gemfile.lock├── README.md├── api│ └── default_api.rb├── config.ru├── lib│ └── swaggering.rb├── my_app.rb└── swagger.yaml

Feel free check out the files to see what’s in there. The ‘README.md’ is especially helpful.

The bulk of the generated API will be inside ‘api/default_api.rb’. Please open that file using your favourite editor.

Now you shall see the two methods inside the file for the two API that we had specified in the editor before. Feel free to read through it.

Swagger-codegen will just generate the stubbed boilerplate for the api. It doesn’t know how to implement the functions to return correct result. If we just run the Sinatra server now, all we are getting would be:

{“message” => “yes, it worked”}

We don’t want that, do we?

Lets flesh out the implementation for the route /hello/{name}. Please replace this code block:

{“message” => “yes, it worked”}.to_json

to:

helloName = params[:name].to_s{“message” => “Hello #{helloName}”}.to_json

Sinatra will take the name parameter, parse it to string, and save it in variable helloName. It will then return the properly constructed Json message.

Let’s do the same for the route /sum/{first}/{second}. Replace the stubbed Json message with:

firstNum = params[:first].to_isecondNum = params[:second].to_isum = firstNum + secondNum{“message” => “The sum of #{firstNum} and #{secondNum} is #{sum}”}.to_json

You can probably guess what this code does by now :)

The default_api.rb file should look like this:

Yay, we are all done! See how easy it is to create a Sinatra server from scratch using the code generation?

Lets run the server using command:

$ rackup -p 4567 config.ru

This command will spin up a Rack server at port 4567 running our api.

After the server is running, you can go back to the Swagger-editor and play around with the ‘Try this operation’.

Consume api on Swift app

This step is a bit convoluted for non iOS dev as it requires dependency management using Cocoapod. So we speed through most of it. You can skip ahead and get the ready-to-use codes in github.

The short story for iOS dev who wants to follow:

unzip the generated swift client from Swagger-editor
create an empty iOS project in the same directory
disable ATS to enable http call (iOS 9 forces https)
close Xcode and cd into the project directory
do pod init to create an empty Podfile
enter pod ‘SwaggerClient’, :path => ‘../’ into the Podfile
run pod install

Once you are done with pod install, please open up the generated xcworkspace. It should show a structure like this image

project structure after pod install

The generated codes will be inside DefaultAPI.Swift file. You can see that it has two methods:

public class func greetUser(name name: String) -> RequestBuilder<Inline_response_200> { ... }public class func sumTwo(first first: Int, second: Int) -> RequestBuilder<Inline_response_200> { ... }

This is actually functions that we can use straight away in our iOS app. Lets get started!

Open up the ViewController.Swift and import SwaggerClient just below the import UIKit statement. The import should look like:

import UIKitimport SwaggerClient

Next we need to add a code to let SwaggerClient know our Sinatra server URL:

SwaggerClientAPI.basePath = “[http://localhost:4567](http://localhost:4567/)"

The viewDidLoad() method should now look like

Now that Swagger know where our server is, just call the generated method!

Press cmd+R to run the project. If everything is working like its supposed to, your Xcode console should look like:

Hello Melbourne

Great! We have connected the Swift client to the server by just writing three lines of codes! Lets add the next api call just below the greetUser().

The console should show:

Hello MelbourneThe sum of 2 and 4 is 6

All done!

Summary

During the course of this tutorial, we have learned how to:

install and run Swagger-editor locally
write Swagger spec using OpenAPI specs
generate Sinatra server and Swift client using Swagger-codegen
fleshing out the Sinatra implementation
consuming the api using generated Swift package

Thanks for reading, I hope this is useful for you. Please let me know if you have questions :)

The project is up on Github: https://github.com/augusteo/Swagger-Sinatra-Swift-Example