Claude Code, Anthropic’s agentic coding command-line interface (CLI) tool, has been a growing phenomenon.

What we will cover:

• A quick start for those who haven’t tried it
• Best practices and how to climb the ladder to being a Claude Code AI coding expert

One year ago, on February 24, 2025. Anthropic released Claude Code. Today, everyone can be a 10x developer, and it is causing enterprises to rethink dev toolchains, pipelines, and stacks, and markets to rethink valuations.

Claude Code is Anthropic’s agentic coding tool — a command-line interface (CLI) that lets you delegate coding tasks directly to Claude AI to automate development tasks. It understands your entire codebase and can work for extended periods across multiple files and tools to get tasks done.

Levels of Coding Assistance

Source: The Five Levels from Spicy Autocomplete to the Software Factory — Dan Shapiro

This guide should get you to level 2–3 and provide a roadmap to get to 4–5.

Why is Claude Code innovative, is it really ‘this time is different’?

• Excellent models: Sonnet and Opus, multi-model to ‘see’ output, tool-calling, long 200k context window (1m for Opus and Sonnet with extra charges). Curretnly rated significantly better than competitors for coding.
• A simple, effective harness with a ReAct control loop: Plan → code → test → iterate as necessary. ()
• Plan-driven long-term memory: It writes plans in .md files and uses them to stay on task. The ability to think and plan before acting, and then stay on plan, enables far longer and more complex work compared to vanilla single-turn LLM chat. It’s why ‘this time is different.’
• Rich internal tooling: A plethora of internal tools and subagents to understand, find, and edit code — plus MCP tool integrations. Good tools that look up exact function signatures or retrieve precise documentation outperform generic web searches or putting full code files in context, or large doc pages. (ChatGPT would often give answers from an outdated API due to training data cutoff; if you have a tool that fetches the exact doc or code you need, and patches a specific line of code, everything works better.) Along with the way it makes plans and notes to aid memory, this helps context management, Claude Code is carefully designed to have exactly what it needs in the context at all times.
• Integration: The CLI can do anything you can do on your computer from the command line, and control a browser. There is a growing community of skills, MCP servers and plugins, with strong third-party integration support for many SaaS platforms. (OpenAI’s Codex is good and adopting MCP and skills, but their strategy seems to be to push you toward their walled garden.)
• Multi-tasking subagents: Spawn isolated agents for parallel work.
• Continuous improvement: Anthropic can collect traces and up/down votes from Claude Code usage and integrate them into RLHF training, they have a flywheel going. More mature than OpenAI Codex and Gemini CLI which are closest comparisons. IDEs like Cursor, Windsurf, GitHub Copilot in VS Code are also developing agentic coding assistants. Combining Claude Code with your favorite IDE is most typical best practice.
• Autonomous operation: Can even run autonomously for many hours if you have the token quota and the right hooks (like Ralph).
• Real-world reliability: Claude Code can now solve complex problems and refactors, run for 30+ minutes on a complex plan without getting off track, understand brownfield codebases, produce minimal slop, get things 90%+ right, and fix most remaining issues iteratively.
• Usable code output has climbed from < 50% to > 90%
• Anthropic: How Claude Code Works
• YouTube: How Claude Code Works

With Claude Code, instead of being a developer with an assistant, you get promoted to the team lead of a team of agents planning and building stuff for you. You become more of a conductor leading an orchestra, and they do more of the work than was the case with GitHub Copilot, Cursor, Windsurf etc. And hopefully this makes you more productive. It’s getting closer to the autonomous engineer we were promised with e.g. Devin.

Claude Code’s power comes with a bit of a learning curve, but it’s well worth it; let’s get started.

Getting Started

Install

• Set up Claude Code
• Pricing: Pro, $100 and $200 Max plans. Start with Pro to learn, once you do real work you will run out of tokens from time to time, Pro gives you limited tokens in a 4-5 hour window. Use /status to see where you are with usage and when it resets. Once you start developing big plans that code for 30 minutes, you will run out very fast in Pro, need a $100 Max plan. The $200 plan is if you have multiple sessions and sub-agents running at once, or otherwise are running out in the $100 plan.
• Launch in a terminal window with command claude.
• This post is about the CLI. You can also use Claude Code within the desktop app and Web UI, but it is much more sandboxed out of the box. As a CLI running in your terminal without explicit permission restrictions and sandboxing, Claude Code can do anything you can do (so be careful!).

Essential Commands and Navigation

• Start claude in the root of your project, Claude.md and other files are typically project-specific.
• / commands — Type / to see all available commands.
• tab - Autocomplete commands and file paths
• Up / DownNavigate prompt history (works across sessions)
• ! run a shell command
• /status — Show current session status.
• Status line — Customizable; you can display context usage %, etc. using /statusline. Try /statusline what can I display?, /statusline set up a status line showing cwd, model, context percentage, with a progress bar . It’s a shell script, can access any environment variables, commands like git status, etc. Docs: Status Line
• [Shift+Tab] — Toggle between Plan mode (don’t code, just plan), and Default mode (prompt before code edits), and Auto-accept mode (accept all code edits).
• @ files — Reference specific files in prompts with @myfile.txt.
• Drag and drop images, or paste with Ctrl+V (not Cmd+V on Mac). Claude Code can read screenshots, images, diagrams. Paste a screenshot of a bug to fix. Or prompt it how to create an image output with Figma or Playwright, then check it, and iterate on it.
• /model — Switch between Sonnet and Opus.
• /ide — Claude can integrate with your IDE, see what you selected in IDE, read IDE listing errors etc., put proposed changes inline with diffs
• /vim — can use emacs (default) or vim keybindings in editor. /keybindings or open ~/.claude/keybindings.json to customize all keybindings.
• /config — Many configuration options.
• /copy — Copy last response to clipboard.
• Ctrl+RSearch prompt history
• /resume or claude --resume and claude --continue — Resume work in a previous conversation thread where you left off.
• [escape] — stop what it’s doing if you forget to tell it something or it’s going off the rails
• [escape] [escape] or /rewind — return to earlier point in conversation, get rid of digressive context
• Ctrl+D Ctrl+D Exit Claude Code
• Scan through all the commands with /. If you don’t know something, ask Claude — it has built-in tools to look up its own docs for you.
• Command line reference
• Startup options. If you find yourself always running commands at startup you can probably do them with startup options.

Claude.md - super important

• /init — Sets up CLAUDE.md file at project level, analyzes an existing project and saves what it thinks it needs to know to work in the project.
• Project-level - CLAUDE.md in repo root or ./.claude/CLAUDE.md - Shared team instructions committed to git. Rules, conventions, and context about how the repo works that every contributor (human or AI) should follow.
• User global - ~/.claude/CLAUDE.md - Your preferences in your home directory that apply across all projects. Style preferences, default behaviors, and personal rules that follow you everywhere.
• Project memory (local) - CLAUDE.local.md - Personal project-specific preferences, not committed to git, each dev has their own.
• Managed System directory like : /etc/claude-code/CLAUDE.md (or similar system directory on other OSes). IT administrators can set up preferences for all users.
• Add to CLAUDE.md: ... — puts it in the project instructions file
• /memory — helps you manage memory. Claude.md should generally be rules about the project that you curate carefully. Auto memory is facts and context about the project that Claude discovers and stores automatically, my understanding is, not persistent between sessions, more like working memory scratchpad.
• Keep CLAUDE.md short: definitely under 150 lines, ideally ~50. Longer files risk Claude forgetting items within them.
• claude.md starter kit — Point Claude at this repo and say: “Improve CLAUDE.md based on these criteria.”
• You can do multiple passes of “check my repo changes and update CLAUDE.md” — and update it regularly. It’s a living document describing how your repo works and how you like to develop.
• You can have it reference other files. For instance, say “When building skills, follow patterns and best practices in @CLAUDE_SKILLS_BEST_PRACTICES.md”
• Memory

Dev Loop: Spec-Driven Development

Recommended Plugins

• install via /plugin, I can’t recommend these plugins strongly enough

• context7 — Look up docs for any Python module (or other languages).

• Superpowers — Test-driven development workflow. Started as an open-source community project, now in the Anthropic plugin marketplace. Other similar plugins (these are not recommended, but give you a flavor of what people are doing):

  • BMAD/Spec Kit: Persona-based agents (Business Analyst, Architect, Developer, etc.), full PRD workflows — very heavyweight.

  • OpenSpec: Document-driven planning: Requirements → Design → Tasks → Changes (audit trail).

• Python-oriented:
  • pyright-lsp
  • code-review
• Front-end:
  • frontend-design
  • playwright — browser so Claude can visually check its work (or use the Chrome plugin; put your preference in CLAUDE.md, e.g., “Always check changes using Playwright”).
• Official plugins
• Plugin marketplaces

The Development Workflow

flowchart TD
    A[Brainstorm] --> B[Plan]
    B --> C{Plan Critic-Optimizer Loop}
    C --> D[Evaluate Plan]
    D --> E[Improve Plan]
    E --> C
    E --> G[Code]
    G --> H{Code Test-Evaluate-Optimize Loop}
    H --> I[Test]
    I --> J[Evaluate Test Results]
    J --> K[Improve Code]
    K --> H
    K --> L[Complete]

    style A fill:#e6f3ff,stroke:#4a90d9
    style B fill:#ffe6f0,stroke:#d94a90
    style C fill:#fff2e6,stroke:#d9a04a
    style D fill:#fff2e6,stroke:#d9a04a
    style E fill:#fff2e6,stroke:#d9a04a
    style G fill:#e6fdff,stroke:#4ac8d9
    style H fill:#e6ffe6,stroke:#4ad94a
    style I fill:#e6ffe6,stroke:#4ad94a
    style J fill:#e6ffe6,stroke:#4ad94a
    style K fill:#e6ffe6,stroke:#4ad94a
    style L fill:#f0e6ff,stroke:#9a4ad9

1. Start in Plan mode — Don’t write code yet. Go back and forth with the LLM to critique and improve the plan.
2. Shift+Tab to toggle between Plan mode (blocks code writing), Default mode (prompts for changes), and Auto-accept mode (makes all changes).
3. Pre-approve permissions — You’ll find you have to approve a lot of bash commands (reading the codebase, looking up docs). Use /permissions to pre-approve non-destructive commands like ls. You can also create an alias:

   alias claudeyolo='claude --dangerously-skip-permissions'
   

   But only use this in an environment where it can do no harm, like a throwaway VPC or container. Also consider adding hooks to block destructive commands like rm and git.

4. Brainstorm the plan — Say: “I would like to brainstorm and create a plan to build [high-level intention].” Then provide detailed info about what you want. The Superpowers plugin will ask questions to help write a plan.
5. Read and edit the plan — It’s your project, not Claude’s, especially at the planning stage. Ctrl-G opens the plan in the editor.
6. Cross-check with another AI — Use e.g. OpenAI Codex to review: “What details and edge cases might I have missed?” Iterate a few times. See: Rule of 5 Passes over Design, Code, etc.
7. Visual aids welcome — Give it a sketch, put ASCII art in the plan, or hook up the Figma plugin to mock up a UI.
8. Ask questions — e.g., “What are the tradeoffs between storing data in JSON vs. SQLite vs. Markdown files?”
9. When the plan looks good, tell it to write the code.
10. Error recovery patterns — If it goes off the rails, keeps producing broken code, or misunderstands the task (interrupt → clarify → /rewind → retry).
11. Important: Include acceptance tests in the plan — Give Claude a way to verify output: pytest, npm test, Playwright for UI, or rendering a PNG for graphical artifacts. Testable plans let it iterate, easily double the quality of the output. If you don’t have tests, you don’t have a full feedback loop, you are the ‘test’ part of the feedback loop.
12. Do several passes of code review — Example prompt: “Do a git diff and pretend you’re a senior dev doing a code review and you HATE this implementation. What would you criticize? What edge cases am I missing?” Go back and forth with OpenAI for diversity. Use the code-review plugin or Turing Skill.
13. Fix what you missed — You’ll probably notice gaps; ask Claude to fix them.
14. Don’t be afraid to start over — It’s cheap. When you realize the architecture is wrong, tell it to write a detailed plan describing everything it did, then build a whole new version in a parallel directory. As Claude Code creator Boris Cherny puts it: “Knowing everything you know, design a more elegant solution.”
15. A big unlock comes when you use an agentic loop to not just code, but to write detailed plans and specs, and have good tests so Claude Code can test that its code works against the spec. That’s when it starts coding for 30 minutes straight autonomously on a long implementation plan. The other (smaller) unlock is /ide integration, open Claude Code in the terminal pane in e.g. Cursor, you can see code as it writes it, can select something and ask a question about it.

Do what works for you — different people have different workflows. If people can build C compilers and operating systems with these tools, you can probably build most CRUD GUIs. The key is good specs and a way to verify.

Context Management Is The Key

Context management is the single most important skill for effective Claude Code usage.

• Too little context: Claude may not know APIs and will hallucinate, focus on the wrong things, or write overly defensive code.
• Too much context: Claude may forget or fail to retrieve important details.
• Bad context: Outdated or incorrect information leads to wrong answers.
• Context rot: Quality tends to fall when context reaches ~50% full.

Effective use of Claude Code is about giving it the right context at the right time, including using skills and Claude.md .

Essential Context Commands

• /clear — Do one task, then clear context. The 200K token context window (or 1m if enabled with /model) is large but not all equally usable. /clear also saves on tokens, and time.
• /context — Shows what Claude is currently tracking: system prompt, CLAUDE.md, MCP servers, skills, and messages.
• /compact — Summarizes context, but it’s somewhat of a black box. Prefer /clear and explicit, mindful context management over implicit /compact or auto-compact.
• /rewind (or Esc Esc) — If you had a digression to figure something out, rewind to an earlier point to “forget” useless context.
• /export — Save your context. Helpful if you want to review why Claude did what it did. Some people save at key checkpoints and even commit them to git.
• /resume — Scroll previous conversations and pick up where you left off.
• Escape — Anytime Claude is going off track, hit Escape to interrupt and redirect.

Extensions

Skills

• Procedural knowledge about how to do something with tools and subagents.
• Custom slash commands and skills have been merged, same thing now.
• Any boilerplate task — setting up a project, plan review, code review, security review — you can set up a skill so Claude does it the way you want.
• A skill is a .md file describing how to do something, with metadata (frontmatter), optionally with scripts and context file data. Importantly, skills always have access to a shell to run commands, that is their native ‘tool’.
• Anthropic: Complete Guide to Building Skills
• Anthropic Docs: Skills
• Very easy to build — there’s a built-in skill-builder skill. Just ask Claude to help.

MCP Servers

• An MCP server is a Web server with REST tools Claude Code and AI generally can call as necessary, plus some metadata about when to use them. You configure Claude to say it has access to these tools and here’s when to call them, just like skills. Anytime you ask Claude to do something, if it thinks the tool will help, it calls the tool and uses the output to respond.
• Use the /mcp command to manage servers.
• A couple of MCP servers is fine; 10+ might be too many. When MCP tool descriptions exceed 10% of your context, Claude automatically defers them and loads tool details on demand (“MCP Tool Search”). But it’s better to start lean.
• A little harder to build than skills, but there’s an MCP builder — ask Claude to help you create an MCP server as e.g. a Python script with FastMCP and tool decorators.
• Connectors: Third-party MCP integrations
• MCP Documentation

Tools vs. Skills: When to Use What

It sounds like MCP is similar to skills, but code and a server are mandatory, whereas skills just use prompts and the shell out of the box. There is some overlap. In general, prefer skills if they are available. Skills are currently more context-efficient and performant. They use progressive disclosure: by default, only a small metadata summary is loaded; the full instructions load only when needed. MCP servers consume more context; if there are too many, Claude puts them into a meta-tool search index (search for the tool first, then load its details).

If a third party wants to expose functionality that lives on their server, they might host an MCP server you can connect to. But if they have a CLI that exposes the same functionality, packaging it as a skill will generally work better.

In both cases, if prompts aren’t calling the tool/skill when you expect, you may need to improve the descriptive metadata or explicitly tell Claude to use it. If context gets large, Claude forgets things. If an MCP server has many tools, consider making a simpler version.

CLIs are free context for skills. Claude already knows how to shell out to gh, aws, docker, psql, stripe, etc. No tool definitions get loaded. No startup cost.

Skill-related tips:

• Skills use progressive disclosure and can scale to many — only frontmatter/metadata loads by default.
• If a skill isn’t triggering correctly, improve the frontmatter or prompt Claude explicitly to use it.
• Skills can include context data and accept arguments.
• Use MCP only for external integrations where a skill can’t work.
• Useful meta-skills: writing-skills, skill-creator (Anthropic built-in skill).

Plugins

• A plugin is a packaging mechanism to distribute skills, MCP servers, hooks, sub-agents, and their associated artifacts.
• Bundles skills (including definitions, scripts and data artifacts), hooks, subagents, MCP servers, into a single installable unit.

Hooks

Hooks let you run something before or after a chat turn or tool executes.

• Use cases: formatters, linters, testers, guardrails.
• 14+ hook trigger points (and growing): SessionStart, UserPromptSubmit, PreToolUse, PermissionRequest, PostToolUse, PostToolUseFailure, Notification, SubagentStart, SubagentStop, Stop, TeammateIdle, TaskCompleted, ConfigChange, PreCompact, SessionEnd.
• Run in YOLO mode (alias claudeyolo='claude --dangerously-skip-permissions') but add a hook for destructive commands (rm, git, etc.).
• Put guidance in CLAUDE.md too, but a hook guarantees code will run when something happens.
• Ralph Wiggum plugin: A hook that intercepts the “stop” event, lets you test when a prompt completes to ensure it met spec, and resumes if necessary. You create a test/promise; on exit, it runs the test. If the desired result hasn’t been achieved, it keeps going — enabling long-running prompts that perpetually find issues and fix them.
• Hook Ideas
  • PreToolUse — File Protection Guard to block edits on certain files
  • PostToolUse — Log tool use
  • SessionStart — Fetch some real-time stuff you always want initially in context

CI/CD Integration

• PR review — Analyzes diffs, finds bugs, flags security issues (often catches logic errors humans miss while humans nitpick variable names).
• Code implementation — Comment “@claude implement this” on an issue, and it creates a PR with working code.
• Bug fixes — “@claude fix this bug” generates a fix PR.
• PR summaries — Generates human-readable summaries of large PRs for easier review.
• Release notes — Trigger on tag push to summarize all PRs in a release.
• CI debugging — Reads workflow logs and diagnoses failures.
• Resources
  • Official GitHub Action
  • Anthropic docs: GitHub Actions
  • GitHub Marketplace listing

Security

• Claude Code CLI runs by default with all the permissions of the user and access to bash, which is too much for many enterprise environments. For instance my Docker was hung, I asked Claude Code to troubleshoot, it fixed it by deleting the troublesome container which was annoying because it had Postgres with some data (Langfuse). Claude Code CLI is agentic browsers on steroids and crack.
• To lock down the CLI
  • Claude Code has managed settings that override command line arguments, local, project, and user settings. These scopes are parsed in order of precedence:
    • Managed settings (highest priority)
    • Command line arguments
    • Local settings
    • Project settings
    • User settings (lowest priority)
  • /sandbox command will let you configure a sandbox to run bash commands in a more restricted container environment.
  • /permissions command lets you set permissions regarding what tools Claude Code can access.
  • Set up hooks, always intercept some calls you might not like like rm -rf /.
  • Log everything via hooks
  • Connect to LLM via proxy and log what it’s doing there. You could even have Claude Code point to a local model and never hit Anthropic using e.g. OpenRouter
  • Or consider running in a container with network allowlist to e.g. internal MCPs and select external websites. Can then sandbox at container level, only connect to internal-only MCP tools, some websites.
  • After writing some agents and skills, package into a web app that runs by spawning a Claude Code process with headless claude -p <prompt> (uses monthly Claude Code subsription, or using Claude Agent SDK which is essentially running Claude Code headless via the API (burns API tokens).
  • The desktop client and web UI are more locked down out of the box.

Advanced Topics

Worktrees

A lot of time is spent waiting for Claude — what do you do? Work on multiple things simultaneously without them clobbering each other.

• Git worktrees share the same .git subfolder.
• Can be annoying if they conflict on npm ports; you can pre-warm 5 worktrees with dependencies.
• worktrunk — A CLI for making worktrees easy.
• conductor.build — Mac GUI for managing multiple Claude Code sessions; takes away the complexity of working with worktrees and multiple sessions.
• Can also use multiple terminals / tmux

Tasks

Tasks are like job control in bash, but more.

• The simplest use is

apply the prompt below to each of the files in the "input" subdirectory. Run using concurrent tasks and save the results as {filename}.output in the output subdirectory. Prompt: read @{filename} and determine the top 5 most important actionable recommendations

• This should launch 5 background tasks. Use ctrl-b to make the current task a background task.

• Use /tasks to view a list of the running tasks, and select any of them to view their output.

Tasks are the replacement for deprecated todos. Unlike todos and the processes you can manage with job control in bash, tasks add persistent storage and explicit dependencies. See also this article.

As a more complex pattern, you can create a plan.md with a dependency graph expressed as markdown, then run a prompt like:

Read plan.md. Create tasks with TaskCreate for each incomplete item, wiring blockedBy dependencies from the file. Skip completed tasks — treat them as resolved preconditions only.

Start with all tasks that have no unresolved blockers and run them in parallel where possible, sequentially where blocked. Delegate each task to a subagent.

After each task completes, update plan.md to reflect the new status before starting the next wave. Continue until all tasks are complete.

Subagents

Multi-agent orchestration pattern. Each subagent runs in its own isolated context with its own tools, model, and permissions — like spinning up a fresh Claude session for a specific task. This keeps the main context clean. In contrast, skills are called in the current context.

Note: Subagents use the Task tool to run concurrently in their own execution context. Subagents cannot spawn other subagents. If you need nested delegation, chain subagents from the main conversation, or use skills.

Built-in agents

Claude Code ships with several built-in subagents it uses automatically:

Managing agents

• /agents — browse, create, and manage agents interactively. Select Create new agent, describe what it does, pick tools, model, and background color.
• Explicit invocation: “Use the code-reviewer subagent to look at my recent changes” — Claude delegates to a specific agent on request.
• Auto-delegation: Claude automatically picks the right subagent based on your request and each agent’s description field.

Creating custom subagents

A subagent is a .md file with YAML frontmatter + a system prompt. Two scopes:

• Project: .claude/agents/name.md — committed to git, team-shared
• User: ~/.claude/agents/name.md — personal, available in all projects

Example:

---
name: code-reviewer
description: Expert code review. Use proactively after writing or modifying code.
tools: Read, Grep, Glob, Bash
model: sonnet
---

You are a senior code reviewer. Run git diff, review modified files, and report:
- Critical issues (must fix)
- Warnings (should fix)
- Suggestions (consider improving)

Key frontmatter fields:

You can also define session-only agents at launch:

claude --agents '{"code-reviewer": {"description": "...", "prompt": "...", "tools": ["Read", "Grep"]}}'

Foreground vs. background

• Foreground (default): blocks the main conversation, permission prompts pass through to you.
• Background: runs concurrently while you keep working. Claude asks for tool permissions upfront, then auto-denies anything not pre-approved. Press Ctrl+B to background a running task, or ask: “run this in the background.” MCP tools are not available to background agents.

Persistent memory

Give a subagent memory that builds up over time across sessions:

memory: user      # ~/.claude/agent-memory/<name>/ — all projects
memory: project   # .claude/agent-memory/<name>/ — project-specific, shared via git
memory: local     # .claude/agent-memory-local/<name>/ — project-specific, gitignored

First 200 lines of the agent’s MEMORY.md load into every session. Prompt it: “Check your memory before starting, and update it when done.”

Worktree isolation

isolation: worktree

Runs the agent in a temporary git worktree — a clean copy of the repo. Great for risky experiments. The worktree is automatically deleted if no changes are made.

Resuming subagents

Each invocation starts fresh. To continue a previous run: “Continue that code review and now analyze the authorization module.” Claude resumes with full conversation history. Transcripts live at ~/.claude/projects/{project}/{sessionId}/subagents/.

Good use cases

• Code review: isolated context, read-only tools — “Have the code-reviewer check my recent changes”
• Deep research: send it to gather findings without polluting the main context
• Parallel work: multiple background agents on independent tasks
• Risky experiments: use isolation: worktree so it can’t break your working tree

• Persistent specialists: a debugger or data-scientist agent that builds domain knowledge over time via memory

• Docs: Creating Custom Subagents
• Awesome Claude Code Subagents

Agent Teams

Agent teams let you coordinate multiple Claude Code instances working together. One session acts as the team lead, coordinating work and assigning tasks. Teammates work independently, each in its own context window, and can communicate directly with each other — not just with the lead.

Status: Experimental. Enable by adding to settings.json:

{ "env": { "CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1" } }

####@ Subagents vs. Agent Teams

Use agent teams when workers need to talk to each other: share findings, challenge each other, and coordinate on their own. Use subagents when you workers that just report results back to the caller. Agent teams add complexity and burn more tokens.

Starting a Team

Tell Claude what you want in natural language — it creates the team, spawns teammates, and coordinates based on your prompt. This works well because the three roles are independent:

Create an agent team to design a CLI TODO tracker: one teammate on UX,
one on technical architecture, one playing devil's advocate.

Or be explicit about size and models:

Create a team with 4 teammates to refactor these modules in parallel.
Use Sonnet for each teammate.

Architecture

Team config and tasks are stored locally:

• ~/.claude/teams/{team-name}/config.json
• ~/.claude/tasks/{team-name}/

Display Modes

• In-process (default): all teammates run inside your terminal. Use Shift+Down to cycle through teammates and talk to them. Press Ctrl+T to toggle the task list.
• Split panes: each teammate gets its own pane. Requires tmux or iTerm2 with the it2 CLI.

{ "teammateMode": "in-process" }   // settings.json

claude --teammate-mode in-process  // override per session

The default "auto" uses split panes if already inside tmux, otherwise in-process.

Controlling the Team

Assign tasks: tell the lead to assign specific work to specific teammates, or let teammates self-claim from the shared task list after finishing.

Plan approval: require teammates to plan before implementing:

Spawn an architect teammate to refactor auth. Require plan approval before they make changes.

Teammates stay in read-only plan mode until the lead approves; the lead can reject with feedback.

Talk directly to teammates: in in-process mode, Shift+Down cycles to any teammate — type to send them a message or hit Escape to interrupt their turn.

Shut down teammates:

Ask the researcher teammate to shut down.

Clean up (always do this from the lead, not a teammate):

Clean up the team.

Quality Gates via Hooks

• TeammateIdle — runs when a teammate finishes; exit code 2 keeps the teammate working with feedback.
• TaskCompleted — runs when a task is being marked complete; exit code 2 blocks completion and sends feedback.

Best Use Cases

• Parallel code review: spawn a security reviewer, a performance reviewer, and a test-coverage reviewer simultaneously — each applies a different lens to the same PR.
• Competing hypotheses for debugging: each teammate investigates a different root cause theory and actively tries to disprove the others’.
• New features across layers: frontend teammate, backend teammate, test teammate each own their piece without stepping on each other.
• Research with multiple angles: explore a design space in parallel (UX, architecture, devil’s advocate) and synthesize findings.

Best Practices

• Give teammates enough context in the spawn prompt — they don’t inherit the lead’s conversation history.
• Size tasks right: too small and coordination overhead exceeds the benefit; too large and mates work too long without check-ins. Aim for 5–6 tasks per teammate.
• Avoid file conflicts — break work so each teammate owns different files.
• Start with research/review tasks to get comfortable before attempting parallel implementation.
• Monitor and steer — check in on teammates, redirect failing approaches early.
• Start with research and review — less coordination risk than parallel implementation.

Limitations

• /resume and /rewind don’t restore in-process teammates.
• Task status can lag — teammates sometimes fail to mark tasks complete.
• One team per session; teammates can’t spawn sub-teams.

• Split panes require tmux or iTerm2 — not supported in VS Code terminal, Windows Terminal, or Ghostty.

• Docs: Agent Teams

Ralph

The Ralph Wiggum technique, created by Geoffrey Huntley, is a method for running Claude Code in autonomous loops that iterate until a task is complete — named after the lovably persistent Simpsons character. You have a prompt completion hook that checks the previous task and continues if something is not totally right, and queues the next task. It’s now in the officially maintained Anthropic marketplace directory.

• Claude plugin
• How to Ralph Wiggum

Frontier stuff

• Steve Yegge’s Gas Town , multiple instances of agents that share memory and talk to each other
• Jeff Emanuel’s Agent Flywheel . Similar, can spread work across multiple Claude Code, OpenAI Codex, Gemini CLI instances so you are never rate-limited.

Other Best Practices / Recap

• Terminal apps like kitty, ghostty, wezterm, iterm2, have advanced features that let claude show images, links, split panes etc., and are recommended over default native terminals. (also running in your ide terminal pane)
• Keep asking questions — Claude Code can tell you how to use itself, how to write prompts, and how to build things. If you want to learn subagents, ask: “I do this task a lot; I’d like to make a custom subagent that can accomplish it.”
• Get the right plugins for your task:
  • context7 — Pull in all the docs.
  • Front-end: frontend-design, Playwright (so Claude can see its output).
  • Python: pyright-lsp for parse tree understanding, code-review for code review and simplification.
• Use the Superpowers plugin for brainstorming, or OpenSpec for document-driven plans.
• Always start in Plan mode.
• Be as specific as possible when telling Claude Code what do do, but try not to clutter the context with irrelevant details, or Claude may forget important details. It can help to specify skills and tools you want Claude to use, like: check the latest Claude docs with claude-code-guide.
• /clear context often, saves tokens and works faster and better without confusing Claude. Docs: Reducing token usage
• Git integration — Claude Code has native awareness of git diff, branch context, and how to leverage it for PR descriptions, commit messages, and changelog generation. There is a github plugin. Commit often.
• /doctor for info about config, current v. stable version. Some way wish to keep people on the latest stable version. Update version with claude update
• Opus for complex planning and complex tasks, can usually use Sonnet for coding (managing token cost); switch with /model.
• Multiple plan iterations for complex work — Tell Claude to write the plan to an .md file, edit it yourself, ask a different AI (like Codex) to review.
• The dev loop: Plan → Code (with tests) → Code review → Iterate.
• The “god prompt”: “You’re a cranky senior developer who hates me and my code. Tell me everything that’s wrong — all the edge cases I missed.”
• It’s cheap to throw away and start over — Say: “Write a new version that does the same thing but structures it like this.” The previous version functions as a spec. You may do this 2–3 times, because the first build reveals what you were missing, then you add the kitchen sink (Brooks’s Second-System Syndrome), then you realize much of the additions are unnecessary or simpler approaches can be generalized.
• Sandbox as thoroughly as possible — You might be tempted to develop with access to a production resource like a database or API — don’t. All bets are off. YOLO in a sandbox, then PR into a feature branch when something works.
• Voice input — Some people swear by Wispr Flow voice keyboard for talking to Claude Code.
• Give logs and screenshots of what happened to help troubleshoot.
• Headless/CI mode — Running Claude Code in CI pipelines (claude -p "prompt" --output-format json) for automated code review, test generation, or migration tasks. Can use --max-turns to control how many autonomous steps Claude takes before stopping for human review. Example: Pipe a log file to Claude: cat build-error.txt | claude -p 'concisely explain the root cause of this build error' > output.txt cat code.py | claude -p 'analyze this code for bugs' --output-format json > analysis.json find --help | claude -p 'read this and find all .py files modified in the last 7 days larger than 10kb' ffmpeg --help full | claude -p 'read this and give me the command to extract audio from input.mp4 as 128kbps mp3'
• SDK/library mode — Use Claude Code as a module from other scripts or a UI, building custom agentic workflows.

Concluding remarks

The Daily Standup Loop: A Week’s Work Every Day

With a workflow based on Ralph, you can run a continuous development cycle that produces a small team’s weekly output every day:

• Morning standup: Review the PRs and diffs your agents produced overnight. Approve, reject, leave comments — you’re the tech lead now, not the coder.

• Queue up work: Add tasks to the shared queue with clear specs and tests which AI helped you write.

• Agents work all day: Agent Teams claim tasks and code in isolated worktrees. In Ralph, loops keep them iterating until tests pass.

• Evening review: Review PRs, merge what’s good, queue overnight tasks. Agents keep working while you sleep.

• Repeat: 2–3 hours of reviews and specs per day, continuous output the rest of the time.

This is not science fiction, early adopters are doing this today. Coding resources are abundant now. Everyone is a 10x coder now. Knowing what to build, cognitive resources for specs and review are the bottleneck, and of course human change management in the org which was always most of the work.

The gap between “Claude as autocomplete” and “24/7 autonomous dev team” is mostly scaffolding: good specs, good tests, good task decomposition, and the discipline to review rather than write.

Go forth and vibe code!

Further reading:

• The documentation is excellent.
  • Common workflows
  • Best practices.
  • Failure patterns
  • How Claude Code Works
• Coding Agent Internals:
  • What Makes a Coding Agent? (Cline)
  • How Coding Agents Actually Work: Inside OpenCode
• Courses:
  • Claude Code in Action
  • Claude Code: A Highly Agentic Coding Assistant (older)
  • Sabrina.dev
• Best Practices, tips etc.
  • Claude Code Tips
  • Roadmap.sh Step-by-Step guide to becoming a Claude Code expert in 2026
  • Trail of Bits
  • Simon Willison: Agentic engineering patterns
• Extensions
  • Code simplifier
  • Serena

• Blog posts and recent discussion:

  • David Crawshaw: Usable output has climbed from < 50% to > 90%.

  • Steven Sinofsky: Software has to move up the value chain or be squeezed — and then presumably humans too.

  • Charity Majors: Humans shift to high ambiguity, high-stakes from low-ambiguity, low-stakes, high volume grunt work.

  • Andrej Karpathy: Moving from vibe coding to agentic engineering

  • Federico Pereiro: LLMs as the new high-level programming language, devs move from code monkey to prompt zookeeper.

  • Simon Willison: AI doesn’t reduce work, it intensifies it .

  • Margaret-Anne Storey: How Generative and Agentic AI Shift Concern from Technical Debt to Cognitive Debt.

  • Steve Yegge: Dracula effect, more productivity more exhaustion, need naps

You don’t need to be a rocket scientist. Investing is not a game where the guy with the 160 IQ beats the guy with 130 IQ. Success in investing doesn’t correlate with IQ once you’re above the level of 120. What you need is the temperament to control the urges that get other people into trouble in investing. If your IQ is 150, sell 30 points; it won’t hurt. - Warren Buffett

Today we will revisit portfolio theory, which I have posted about in the past, and try to present a coherent introduction to portfolio optimization and asset allocation, with data through 2025 and vibe-coded illustrations.

I would like to explain some things that were not explicitly made clear to me when I first encountered portfolio theory, and build a reasonably robust long term allocation from data and first principles.

The capital asset pricing model was popularized by Harry Markowitz and William Sharpe, although several people came up with versions of it independently. It lets you identify optimal risk/reward portfolios, if you make some (not 100% realistic) assumptions about how asset prices behave.

Ultimately we want to get to a useful version of the charts below.

This efficient frontier chart shows feasible combinations of risk and reward.

This transition map shows the composition of efficient portfolios at different levels of risk. Mouse over to see historical returns and asset composition:

We’ll proceed as follows:

1. Why volatility is a proxy for risk.
2. Why volatility is an imperfect proxy for risk.
3. Visualizing volatility math with pictures of triangles.
4. Visualizing optimal portfolios with efficient frontiers.
5. Picking an optimal portfolio.
6. Regularization and backoff - a midwit way.
7. Various scientific alternatives.
8. Conclusion: the data is sparse and ill-behaved, the math is not super realistic, but this gives us a reasonable starting point.

I hope you’ll have a better understanding of how to make reasonable decisions about allocations based on past market behavior and a little math.

Disclaimer: this is educational material, not financial advice.

1. Why volatility?

Suppose you are watching a peaceful ocean, and suddenly a shark eats a lone swimmer. That’s a bad outcome. The risk was probably there all along, but we didn’t perceive the risk because we weren’t aware of the possibility of a very bad outcome.

On the other hand, if you are watching Jaws, and you see the shark’s fin gliding toward unsuspecting swimmers, bad stuff could happen. Either the lifeguard will blow his whistle and they will narrowly escape being attacked, or someone will meet a grisly fate. We sense the danger. Suspense mounts, and our pulse quickens.

Risk means the future can follow many paths, including good and bad ones, but only one path will happen.

We can represent this as a probability distribution of possible outcomes. Less risky portfolios have narrow distributions clustered around the mean, with some better and some worse outcomes. Riskier portfolios have a wider confidence interval, with some extremely bad and some extremely good outcomes.

What this looks like in the market is this:

Notably, percentage return calculations are asymmetric in the sense that a 50% rise from 100 to 150, followed by a 50% drop to 75, leaves you down 25%. A 10% rise is the same number of dollars as a 10% drop, but it’s harder to catch up from the loss since your base is now smaller. The log diff of the total return index, i.e. \(log(1+r)\) where \(r\) is the percent return, is a more symmetric representation of change, and this chart of log diff returns is more representative of the fat tails (leptokurtosis) and left skew in returns, with most returns to the right of the mean and fat tails on the left.

Why is volatility used to measure risk? Volatility is the standard deviation of returns, and measures the size of typical daily (or longer, or shorter) swings. Volatility measures the width of the bell curve, or the range of possible outcomes. Historical volatility measures the size of past swings. Implied volatility measures what options price in about future swings. Lower volatility means outcomes are fairly predictable; higher volatility means a lot of different outcomes are possible.

Consider Ben Graham’s metaphor of manic-depressive Mr. Market.

One of your partners, named Mr. Market, is very obliging indeed. Every day he tells you what he thinks your interest is worth and furthermore offers either to buy you out or to sell you an additional interest on that basis. Sometimes his idea of value appears plausible and justified by business developments and prospects as you know them. Often, on the other hand, Mr. Market lets his enthusiasm or his fears run away with him, and the value he proposes seems to you a little short of silly.

On a typical day, bipolar Mr. Market swings a typical amount between euphoria and despair.

That volatility tells you how risky Mr. Market thinks the stock is, as a first-order approximation. In the short run, volatility just tells you market perception of risk. In the long run, it incorporates the good and bad things that actually did happen, and more closely approximates true risk.

Even then, a lot of things that could have happened didn’t happen. Maybe there was a too-great or not-great-enough probability of a disastrous war priced into the swings during the Cuban Missile Crisis. The concept of ‘risk’ is a Platonic ideal of things that are not knowable and don’t necessarily follow distributions you can model. Any attempt to put a number on it is inherently subjective. Still, volatility measures the range of returns that investors actually experienced, and what risk the market prices in, and these are good to know.

There are at least 3 different, related concepts here:

• The historical distribution of outcomes investors actually experienced.
• The true distribution of actual possible future outcomes for individual securities, which influences how you construct a portfolio. This is an abstract concept. Better models predict better. But that does not mean there is some perfect model, or that the future is acutally determined by some random process like picking fates out of a hat.
• The distribution investors currently believe, based on their knowledge of history, explcit models they build, implicit models revealed by their actions, which in turn impact short-term volatility.

We usually express volatility at an annual rate. We can do that because if you assume well-behaved log-normally distributed returns, then there is a simple relationship between daily volatility, monthly volatility, and annual volatility.¹

Let’s look at the distribution of stock returns over longer time frames.

We see tighter distributions as we look at longer spans. Many of the 20-year time spans overlap, so this probably overstates the convergence. But note that there has never been a 20-year negative real return in the S&P. If returns are uncorrelated and well behaved, we should expect returns over longer periods to converge toward the mean return according to a square root law. You are not guaranteed a great return, but if you stay invested for a long time the law of averages is on your side.

Let’s look at the distribution of real returns of various assets:

We can see that assets with high volatility tend to have higher mean returns:

The more risky an asset is, the higher the return investors demand to hold it. (Gold is an outlier; its lack of correlation with the other assets, and reputation for holding its real value over the long run in spite of inflation make people hold it, in spite of high volatility.)

More to the point, investors really care about nondiversifiable risk. Truly random, idiosyncratic volatility in a financial instrument is not a big deal if you can buy 20 securities with that type of risk which cancel each other out.

Why are some assets more volatile than others? Fixed income assets have a fixed payout, but longer duration can make them more sensitive to variations in interest rates. Some companies have higher variability in earnings, or may have no earnings and be a crapshoot, like a speculative drug or tech company. The earnings are farther in the future and duration comes into play. Also, if a company has a lot of debt and fixed expenses, i.e. high financial and operating leverage, then a swing in expectations about business conditions might impact future earnings and the stock price a lot more than an unlevered stock with very consistent earnings.

2. Issues with volatility

A model of log normal distributions is not totally realistic in practice:

• As seen above, we have asymmetric fat tails (the volatility smile). Why? Extreme events happen more frequently than predicted by normal distributions. Extreme negative events happen more frequently than equally extreme positive events. We have meltdowns but not meltups. We have seen nuclear power plants melt down into a pile of rubble and melted fuel. We have never seen a pile of rubble melt up into a working nuclear power plant. Entropy says you can demolish a building almost instantaneously with well-placed explosives, but you can’t construct a building the same way, however well you place the explosives.

• We have serial correlation. Markets can trend. Information can take time to be fully understood and reflected in stock prices. A decision a board or investment committee makes one day might result in many changes to companies or portfolios over long periods. Investor psychology is durably impacted by the recent past. Investors can also overreact and mean-revert. A high Shiller PE is associated with lower future returns. And people have loss aversion and a healthy fear of entropy and not being able to buy a GI Joe with a Kung Fu grip, and suffer from herding and crowd psychology. They will rush to sell when it’s down 40% and then rush to buy when it’s up 40%. Trending and mean-reversion can coexist at different time scales and for different types of perturbations.

• We do not have covariance stationarity. The S&P today, with >40% in Big Tech, has different risk/return characteristics compared to say 1980 energy-heavy stocks or 1960 manufacturing stocks. We have regime changes based on changes in economic and market structures like the gold standard, international trade, monetary and fiscal policy regimes. Observed volatility measures what Mr. Market thinks at a given time. Mr. Market evolves over time and what Mr. Market thinks today might not be an accurate reflection of the future. Past performance does not predict future results.

• Nonlinearity. Companies have complex dynamics that are not fully described by linear relationships over longer time periods and larger changes in the environment. Consider two identical companies, like gold mines, whose earnings depend on the price of gold and the cost of mining it. Suppose one decides to lever up and distribute cash. If the gold price goes down a lot, the unlevered company may no longer be profitable but it still owns the gold mine and the possibility of being profitable in the future. The second, levered company is bankrupt. The debtholders take over the mine to recover the debt, and the stock is worthless. On the other hand, if the stock goes up a lot, the second company organically delevers and the economics become similar to the first company. There is positive convexity, investors keep the upside and bondholders get a lot of downside in exchange for interest payments. At a high price the stocks may have a beta of 1 and a high correlation. At lower prices the levered one has a higher beta. Correlation is a first-order measure of how things move together, it doesn’t accurately reflect higher-order derivatives and the dynamics of large price moves.

• Investors may have complex distributional preferences that aren’t well described by volatility. For instance, they might care mostly about downside volatility in an asymmetric loss averse psychological model, maybe like Kahneman-Tversky prospect theory. Humans can be weird and boundedly rational. But weird utility functions make the math intractable, and lead to path dependencies and time inconsistencies. It’s hard to reason about unreasonable things.²

Humans don’t have a linear utility relationship with volatility. Markets don’t exhibit log normal behavior in realized returns. As Einstein said, if you want to describe reality correctly, you must leave elegance to the tailor. Some degree of mathematical perversity is a feature of real human beings and real financial markets.

Nevertheless, it is useful to start from tractable assumptions and simple models we can reason about, and see what they tell us. Even if they are imperfect in practice, they teach us useful ways to think. And from there maybe we can relax assumptions and build more realistic models.

Keeping these imperfections in mind, let’s proceed, with an awareness of monsters lurking below the surface.

3. Visualizing volatility math with triangles

The relationship between the volatility of individual stocks and the volatility of a portfolio of stocks can be expressed using the volatilities and correlation of returns, and visualized with vector addition.

Recall that the correlation is the change in expected return of one stock conditioned on the change in expected return of the other, each expressed in units of the number of standard deviations.

Suppose we have 2 stocks with actual returns \(r_1\) and \(r_2\), expected (mean) returns \(\mu_1\) and \(\mu_2\), standard deviations \(\sigma_1\) and \(\sigma_2\) and correlation \(\rho_{12}\). Then:

If the realized return \(r_1\) of stock 1 is 1 SD higher than its expected value \(\mu_1\), then the expected return of stock 2 \(r_2\) is \(\rho_{12}\) SDs higher than its expected value \(\mu_2\).

If we have a portfolio of these 2 stocks, the return of the portfolio is the weighted average of the two returns: \(r_p = w_1r_1 +w_2r_2\) where \(w_i\) is the weight of stock \(i\) between 0 and 1 and \(r_i\) is the return of stock \(i\).

The volatility of this portfolio is given by

This is a vector addition. Suppose we have a triangle of 3 vectors \(\vec{a}\), \(\vec{b}\), and \(\vec{c}=\vec{a}+\vec{b}\), and an angle \(\theta\) between \(\vec{a}\) and \(\vec{b}\). Denote the length of the three sides as \(a\), \(b\), \(c\) (properly \(\lVert \vec{a} \rVert ,\lVert \vec{b} \rVert ,\lVert \vec{c} \rVert\)).

Then the length of the 3rd side \(c\) is given by:

The equation for \(c\) has a form analogous to the equation for \(\sigma_{p}\), with correlation standing in for cosine.

If we define \(a=w_1\sigma_1\), \(b=w_2\sigma_2\), then we can draw a triangle with 2 sides of length \(a\) and \(b\). We set the angle \(\theta\) so \(\cos \theta\) = \(\rho_{12}\). Then the length \(c\) of the third side of the triangle represents the volatility of the portfolio of the 2 stocks.

The rest of this is strictly optional but if you define the portfolio weight vector:

and the correlation matrix of these 2 stocks:

and a diagonal matrix to scale it by the standard deviations:

then the covariance matrix is

And you can write the formula for the portfolio variance as:

And that formula generalizes to \(n\) assets with an \(n \times n\) covariance matrix \(\Sigma\) and a weight vector of \(n\) asset weights. A useful bit of notation, since large portfolios have a lot of terms.

This formula is an identity. It does not depend on returns being log-normally distributed. However, if returns are log-normally distributed, then so is the portfolio return. A 68–95–99.7 rule will hold. If you have deviations from normality, the formula will still hold, but the interpretation may be very different.

4. Visualizing optimal portfolios with efficient frontiers

Let’s build an efficient frontier which is a great way to visualize the risk and return of feasible portfolios.

We use data on major asset classes from Aswath Damodaran.

Descriptions of the data are in the 1st tab of the histretSP.xls Excel spreadsheet.

We can compute an efficient frontier, showing the best feasible combinations of risk (vol) and return, using the following process:

1. Estimate the covariance matrix of all the assets.
2. Find the maximum return portfolio, which is 100% of the highest mean return asset.
3. Find the minimum volatility portfolio via gradient descent (convex optimization).
4. Split the space between the max return vol and min vol into e.g. 200 equally spaced points.
5. For each vol \(\sigma_i\), find the portfolio weights that maximize the weighted average return, subject to a constraint of at most that vol, \(\sigma_p < \sigma_i\).
6. Plot the resulting portfolio returns vs. the portfolio vol.

Minimizing volatility (or maximizing return) is a convex optimization problem. ‘Convex’ means the return surface in weight space is like a big hill with a single peak, with no local maxima or saddle points. So if we just always move in the direction that gives better results, we are guaranteed to reach the best solution.

In math terms, we can solve a convex optimization problem by gradient descent. By convention we optimize in the direction of a minimum like the negative of the return. If the 2nd derivatives (Hessian in multiple dimensions) are nonnegative everywhere, then if we keep going downhill, then the first local minimum we find, where any weight change in any direction moves volatility up, is also the global minimum. Also, if you add constraints like “weights must sum to 1”, which form a boundary around the search space, the feasible space defined by constraints must be convex and not some weird shape, where gradient descent could lead you to a corner solution at a binding constraint, even if starting from somewhere else would lead you to a different, better solution.

We can use cvxpy which is a framework by Stephen P. Boyd that lets you use various optimization problem solvers.

We get this efficient frontier, the blue line, which shows the maximum mean return you can get for any portfolio risk budget.

Why the orange line? After computing the blue line, which is the portfolio’s mean return over the time span, I wanted to calculate the annualized percentage return, i.e. the geometric mean, for each optimal portfolio on the efficient frontier. I wanted to check the math that came out of the optimization. To compute the orange line, I took the asset weights for each portfolio on the blue line, and simulated its actual performance, rebalancing annually, and calculated its realized vol and realized annualized return. The math checked out, the blue line shows the optimal mean returns. But remember when we said percentage changes are asymmetric? When we simulate the portfolio, the geometric mean return is lower than the arithmetic mean return, because of volatility drag. The more volatile assets go up and down a lot, and if they go up 10% to 1.1 and down 10% to 0.99, the arithmetic mean return is 0, but the geometric return is down 1%.

Fortunately there is a trick we can use to improve our optimization. The geometric return including volatility drag can be estimated by average return - variance/2 . In our optimization, if we maximize this adjusted return of \(\mu -\sigma^2 / 2\), we get these efficient frontiers:

Maximizing the return less estimated volatility drag is very close to the actual realized efficient frontier using the estimated optimal portfolios.

Note: the efficient frontier stops before the ‘small cap’ marker, because it turns out that is not an efficient portfolio. Adding a bit of S&P to a 100% small cap portfolio reduces volatility, since S&P is less volatile, and also actually increases return, by reducing volatility drag.

Finally we can visualize individual portfolios via this transition map which shows how the portfolio evolves as you tolerate higher and higher risk (roll over for asset weights).

5. Picking the optimal portfolio on the frontier

There is a smart way to pick a good portfolio, which chooses the red star, and a midwit way, which chooses the green star.

First, the genius way:

• Let’s assume there is a risk-free rate on cash of 0%, which you can achieve with a volatility of 0%. Since we are using real returns, this means something that yields exactly the CPI every year. A guaranteed pre-tax return of CPI + some coupon is achievable with TIPS. The vol will only be 0 in real terms if you e.g. donate the coupons.
• If you add this risk-free asset to the mix, it is a marker at the origin.
• Then you can combine this risk-free asset with any portfolio on the efficient frontier and your return will be a weighted average of the risk-free rate of 0% and the chosen portfolio return.
• If you draw a line from (0, 0) so it is tangent to the outermost point of the efficient frontier, then the tangent portfolio is the portfolio with the maximum Sharpe ratio, which is defined as \((r_{portfolio} - r_{riskfree} ) / \sigma_{portfolio}\). It’s how much return you pick up over the risk-free return per unit of risk. You can combine the Sharpe-optimal portfolio with some amount of cash to achieve any risk-return profile on the dotted line.
• Then suppose you can borrow at some rate less than the return on your portfolio. For simplicity’s sake assume it’s the same 0% real rate. Then if you continue the line to the right of the Sharpe-optimal portfolio, that represents borrowing at 0% and buying more of the Sharpe-optimal portfolio. If you can borrow at e.g. 2% then the line will be a little flatter and be tangent to the efficient frontier starting a bit further right where the slopes line up.

Every point on this capital market line is a feasible portfolio with the same Sharpe ratio. In Sharpe world, you should always buy the risk asset portfolio with maximum Sharpe ratio. And then lever it up for more risk and return, like a hedge fund, or lever it down combining it with the risk-free asset if you prefer lower risk and return.

But there are some problems with this in our world:

1. We are trying to maximize real returns after inflation. On the one hand, there isn’t actually an asset that pays the CPI flat. On the other hand, TIPS should do better, but with some modest volatility. Ideally we would add TIPS to the asset universe. But they’ve only been around the last 30 years or less. We could try to model what TIPS returns would have been historically by estimating historical inflation expectations based on current inflation, the monetary/fiscal policy mix, other predictors like yield curves, gold prices, FX etc., and add some noise to match correlations. But that’s a rabbit hole. Assuming a zero return asset isn’t accurate, but it’s not crazy, it’s a little less return and volatility than is actually achievable.

2. Most retail investors can’t or don’t use leverage. You can invest on margin, but the margin rate is high. If you borrow, there is a path dependency where you get liquidated if your equity gets too low or negative. If the borrowing rate varies, that would have to be reflected in the optimization. Retail investors typically want to move to the right to increase their risk and return, and not simply lever up.

3. There is the complex problem of overfitting. We have taken average returns of assets and covariances over a long period. But we don’t really know how accurate these are as a forecast over our time horizon. And optimal portfolios are highly sensitive to return and covariance forecasts. In fact, optimization tends to maximize estimation errors. The better your optimal portfolio, the more likely it takes advantage of unlikely quirks in the forecasts.

In machine learning, we generally regularize to avoid overfitting. if we perform a regression, we typically add a penalty to parameters to shrink them toward 0, and the estimator toward the mean. We use cross-validation to determine how much to regularize, and balance overfitting and underfitting, by checking what regularization parameters generalize best out-of-sample.

This can be viewed as a Bayesian approach, where we start with a maximum entropy prior. For instance: a naive base case might be that all parameters are 0 with some distribution; then we update the prior in proportion to the strength of the evidence they are greater than 0. L2 regularization implies a Gaussian prior on parameter distribution, L1 implies a Laplacian prior.

Knuth has been quoted as saying, the root of all evil is premature optimization. Which is exactly the same as overfitting to an imperfect understanding of the problem. Bad government policy = overfitting to past problems. Bad heuristics and prejudices = overfitting to personal experience or what your grandparents or your lizard brain taught you. It takes a lot of wisdom to only learn from data in proportion to the evidence. You have to have a good model for how the evidence was created and what the underlying reality might look like.

Frequentists may object to a Bayesian approach because it seems non-mathy, where does this prior belief come from? On the other hand, even if you don’t use an implicit prior in linear regression, you can back a prior out via Bayes’s law, like a continuous distribution from -infinity to +infinity. So I would argue everyone is a Bayesian, but non-Bayesians don’t make priors explicit. The real question is whether a Bayesian prior improves out-of-sample performance, and it often does. If you use a prior distribution on IMDB movie ratings, you get a better rating sooner with fewer incoming votes. If you have some information about what the model should look like, you should use it, otherwise you are not getting the best model. Of course, you should only use the most defensible maximum-entropy prior, otherwise you might be introducing bias into your model.

If we don’t systematically regularize, our model will overfit and not generalize well out of sample.

As far as I know, there is no generally accepted way to do this for portfolio optimization. So, here is a midwit way to back off the optimized portfolio toward one that is more robust to estimation errors: just average all the optimal portfolios.

That gets us the green star, which looks like this:

Average over entire efficient frontier:

This is admittedly a very simplistic, possibly stupid methodology that loses Sharpe ratio, but not very much. The portfolios at the extremes are probably not ones you would realistically choose. You might want to average over the reasonable portfolios in the middle. But we could view it as a base case.

If we add a risk-free asset we get this.

Average over entire efficient frontier:

A risk-free asset unsurprisingly can help reduce your risk.

In general, inflation is a risk you want to protect against in a portfolio. It was a big deal in the 70s. The current policy mix may tend toward the inflationary: large fiscal deficits, political pressure on the Fed, tariffs, deglobalization, dollar depreciation, and labor market tightness exacerbated by immigration crackdowns.

Stocks offer some inflation protection over the long run. Buffett would probably argue against gold and TIPS generally, because they are low or zero-yielding assets, and a bet against the rationality and wisdom of policymakers. Also, if you are Buffett, your opportunity set is a bit different, if you can reliably find equities that will give you inflation protection in the form of moats and brand equity and pricing power over the very long haul, plus equity risk premium. Inflation protection is expensive if it means giving up equity risk premium. The index investor disclaims any such ability and is more at risk. In any event, these portfolios should be considered a starting point for thinking, not a substitute for it.

TIPS offer an inflation hedge and a safe real return, so they might dominate gold. In my opinion, there isn’t a strong theoretical argument gold should increase in value faster than inflation in the long run (gold bugs might disagree and think fiat currencies go to zero, but that’s my story and I’m sticking to it). I could see reasonable arguments why gold should maintain its real value: As the world gets richer, people may want the same value of jewelry in proportion to their wealth, the supply of gold is fairly fixed, with mining and industrial consumption resulting in little net change, and thus gold prices may keep pace with other assets. Also, there is monetary demand for gold; when people lose confidence in monetary authorities and fiat currency holding value, they demand gold because it is currency-like and supply is relatively fixed. So gold offers an inflation hedge.

The real estate series we have is the Case-Shiller index. Real estate is an important asset class, but this index is not investable. Some investors may have homes and already be exposed to real estate. Others would probably prefer liquid real estate ETFs, but we don’t have data on them until recently. If we take out real estate, the analysis looks like this:

Average over entire efficient frontier:

6. More scientific ways to pick an optimal portfolio

There is no generally accepted way to regularize this analysis, but there is a lot of work in this area, here are a few approaches:

• Ledoit-Wolf covariance matrix shrinkage - More robust covariances that work better out of sample.

• Just add a vol penalty to the optimization to force a lower risk, more diversified portfolio at each level of return. We are already adding a vol penalty to adjust for volatility drag. We can increase the penalty, and my theory is, this will force a more diversified portfolio and be more robust to estimation error. This begs the question of how big a penalty. We could do leave-one-out cross-validation or a similar Monte Carlo process to compute a Sharpe-optimal portfolio on many penalty parameters, and plot the average return, SD and Sharpe ratio for held-out data vs the penalty parameter, to find what works best out of sample. The machine learning engineer in me says go for it, the midwit finance/statistics guy finds it hard to justify rigorously. Maximizing long-run geometric return is Kelly-optimal and maps to log utility, doubling the penalty is something like moving to half-Kelly. Directionally it makes sense but there is no theoretically correct risk aversion parameter, we are left with what works best empirically.

• Near optimal portfolios One approach is, first find the highest Sharpe portfolio. Then we can say, find the lowest risk portfolio with no more than e.g. a 0.05 drop in Sharpe ratio. Since this portfolio is more diversified, i.e. most diversified within 0.05 of maximum Sharpe, it should be more robust out-of-sample.

• Subset resampled portfolios, Suppose we have 6 assets, do 6 optimizations, dropping one asset each time, then average all the portfolios. Similar to random forest, an ensemble of intentionally weakened models can perform better out of sample than a single overfitted model.

• Michaud resampling and MCOS. Do Monte Carlo simulations where we perturb the return forecasts and covariances randomly each time, and average all the resulting portfolios.

• Hierarchical Risk Parity and related methods like Nested Cluster Optimization. Create a tree of assets clustered by similarity. Then starting at the bottom at each non-leaf create a risk parity portfolio of assets under the node, and recursively climb the tree to get a global portfolio. If you create a portfolio of t-bonds and notes and bills and corporates (NCO) and MBS and munis, we will get some of each, even if one is dominated under MV optimization. Same if you do US stocks and various international markets with different market caps and geographies. Then as you combine clusters, all asset classes are represented, whereas a global optimization might omit some assets. Vanilla HRP actually ignores returns and substitutes cluster hierarchy for the covariance matrix, and creates a minimum risk portfolio assuming no covariance at each level. Which ignores a lot of information. Schur portfolios let you split the difference between vanilla HRP and optimization using the return and covariance information.

If you can come up with a good set of assumptions and a reasonable way to regularize covariance matrices and return forecasts and generate optimal portfolios that generalize well out of sample, you can gain fame and fortune.

7. Concluding thoughts

What have we learned?

• Markets are complex. All models are wrong (George Box), but some are useful. We didn’t even go into issues with the single-index beta model.
• Visualizing portfolio composition as vectors and correlations as cosines can help build intuition about volatility, correlation and diversification.
• If you have a risk-free asset, that changes the shape of the efficient frontier and gives you a line to the left of the Sharpe-optimal portfolio. If you can borrow and use leverage, that changes it again. If you assume some modest ability to pick stocks and short stocks, that improves the efficient frontier a lot, but not many people are actually able to do this consistently over the long run.
• Individual investors typically want to avoid leverage, shorts, and stock-picking. Even really smart and sophisticated investors get burned by leverage. For instance before the financial crisis, Citibank and Harvard both decided they should be taking more risk and leverage and got burned. Famously, the geniuses at Long Term Capital Management made highly levered bets on relatively sure things like buying illiquid off-the-run bonds and selling liquid on-the-run bonds, only to lose billions when markets went into crisis and markets rushed to risk-off via the most liquid instruments.
• Picking the right tradeoff on the frontier is important, hard, and requires a risk aversion assumption/choice.
• Because you don’t have accurate volatility and return estimates, you want to back off from optimal portfolios in the direction of more diversification and robustness to regime change. Regularization is important. The midwit average may be the simplest thing that might work.
• You can probably reduce and quantify the estimation error with some clever cross-validation.
• You may wish to take estimation error into account explicitly in picking a portfolio.
• Potentially you could adjust volatility estimates to additionally incorporate estimation uncertainty, and then pick an optimal portfolio using log utility/risk aversion. Kelly bets and log utility are two sides of the same coin.
• There are only 2 strategies in the world, mean reversion and trend following. Buy what’s cheap on the dip like Buffett, or buy the thing that goes up most and everyone else is buying because there is probably a reason. You can turn these 2 strategies into a 2x2 matrix, adding 2 corners: buy what looks cheap and also has momentum, like AQR, or ignore both and just buy the index.
• Returns to capital must come from GDP. If GDP goes up you want your fair payoff from the capital/labor share. Real interest rates and wages drive the capital/labor share to some extent. If the economy were to shrink, or the capital share were to shrink, all investors would likely suffer, nothing escapes their gravitational pull.
• A complementary view is, you want to take as many diversified risks as you can that you are reasonably well compensated for, interest rate risk, overall growth risk, inflation risk. In the long run equity risk has been well-compensated via the equity risk premium. But not always, fror instance from 1929 to 1932, late 60s to 1974, 2008. Historically you should buy the dip, but it might take many years to catch up.
• You get alpha when you understand the complex nonlinear dynamics of the world better than the market, and stay within your circle of competence.
• Being a good gambler and risk manager and staying within your circle of competence is more important than being a good analyst.
• A big part of investing is finding situations with asymmetry, optionality, and positive convexity. Then if things turn out better than expected you can make good money but if things turn out poorly your downside is limited.
• It’s easy to pontificate about investing and hard to invest.
• I’m a midwit so your mileage may vary.

Code is here

Related reading:

• CVXPY tutorial
• Boyd and Vandenberghe - Convex Optimization
• Cajas - Advanced Portfolio Optimization: A Cutting-edge Quantitative Approach
• David Swensen - Pioneering Portfolio Management: An Unconventional Approach to Institutional Investment
• Bernstein - The Intelligent Asset Allocator: How to Build Your Portfolio to Maximize Returns and Minimize Risk
• Kinlaw et al - Asset Allocation: From Theory to Practice and Beyond (Wiley Finance) 1st Edition
• Fabozzi and Markowitz, ed. - The Theory and Practice of Investment Management: Asset Allocation, Valuation, Portfolio Construction, and Strategies
• Muralidhar - Innovations in Pension Fund Management
• Ferri - All About Asset Allocation Paperback
• Faber - Global Asset Allocation: A Survey of the World’s Top Asset Allocation Strategies

The smartest thing anyone ever told me about their religion was “I don’t actually believe any of that stuff, I just like going to church.”

She was a Unitarian. I liked her. I like Unitarianism. It doesn’t make stuff worse.

There are mysteries. The human mind abhors a mystery, like nature abhors a vacuum. And we love a good narrative. An entire Hollywood and novel-publishing and theater industry attests to it. We have to fill in the blanks of the mystery with a narrative.

There is truth in Descartes. I directly sense that I exist. And that sensation is something inaccessible to science.

Consider Eddington’s ichtyologist:

Let us suppose that an ichthyologist is exploring the life of the ocean. He casts a net into the water and brings up a fishy assortment. Surveying his catch, he proceeds in the usual manner of a scientist to systematise what it reveals. He arrives at two generalisations :

(1) No sea-creature is less than two inches long. (2) All sea-creatures have gills.

These are both true of his catch, and he assumes tentatively that they will remain true however often he repeats it.

In applying this analogy, the catch stands for the body of knowledge which constitutes physical science, and the net for the sensory and intellectual equipment which we use in obtaining it. The casting of the net corresponds to observation; for knowledge which has not been or could not be obtained by observation is not admitted into physical science.

An onlooker may object that the first generalisation is wrong. “There are plenty of sea-creatures under two inches long, only your net is not adapted to catch them.”

The ichthyologist dismisses this objection contemptuously. “Anything uncatchable by my net is ipso facto outside the scope of ichthyological knowledge, and is not part ot the kingdom of fishes which has been defined as the theme of ichthyological knowledge. In short, what my net can’t catch isn’t fish.”

Or— to translate the analogy— If you are not simply guessing, you are claiming a knowledge ot the physical universe discovered in some other way than by the methods of physical science, and admittedly unvenfiablc by such methods. You are a metaphysician. Bah !”

A scientist can say consciousness is just an illusion. But it sure doesn’t feel like an illusion. Even the scientist doesn’t act as if it is an illusion, but tells themself some story that motivates them to do science. If it’s all an illusion creating more illusions, why bother? It’s hard out there for a reasoning creature. The first reality you experience defies reason.

Some metaphysics must exist? It’s just not all that clear whether it’s a benevolent God or Descartes’s evil demon deceiving us, or a fully indifferent one.

Whereof we cannot speak, we must pass over in silence - Ludwig Wittgenstein

We should probably just shut up, but we can’t. We are born crying. And live complaining, and die disappointed. We must be comforted. Demand for bullshit must be supplied.

Why does it always has to be such stupid bullshit though? Burning bushes and whatnot? Stuff that makes people hate each other?

David Hume said, on miracles:

No testimony is sufficient to establish a miracle unless it is of such a kind that its falsehood would be more miraculous than the fact that it tries to establish.

We can’t reasonably believe in miracles just because some dude or some ancient primitive book told us. But we are miracles. Our existence is a miracle. Nature is a miracle. Everything is a miracle. Reasoning can’t establish miracles, but we directly sense them.

Even if there is only one possible unified theory, it is just a set of rules and equations. What is it that breathes fire into the equations and makes a universe for them to describe? - Stephen Hawking

And everything is endogenous. You miraculously create your world with your thoughts.

When life itself seems lunatic, who knows where madness lies? Perhaps to be too practical is madness. To surrender dreams — this may be madness. Too much sanity may be madness — and maddest of all: to see life as it is, and not as it should be! - Miguel de Cervantes

All we are is stories in the end, so why not at least try to make it a good one? I’m not going to judge you if you make up a story to get you through the night. If you have to believe in something, if it makes you a better, happier person, more power to you.

But why does it always have to be a story that demands judging and othering and harming people? God gave us this strip of land and the right to kill people who think otherwise. But we are good people by definition! An avatar of Western Civilization! Certainly not, for instance, the world’s greatest example of main character syndrome.

Gandhi was asked what he thought about Western civilization, he said he thought it would be a very good idea.

We have just enough religion to make us hate, but not enough to make us love one another. - Jonathan Swift

I’m not religious but I’m not against religion. I’m against stupid religions. And religions are for the most part stupid, because people are for the most part stupid.

Inside every human there’s a John Lennon ‘Imagine all the people living life in peace’ and a Steely Dan ‘Everybody on the street has murder in their eyes’.

When you are dealing with effed-up people, say Hitler for example, you just have to fight and defeat them. There is no alternative, except a one-way trip to the showers. But when you run around thinking everyone is Hitler, and you have to do horrible stuff or it’s the end of ‘Western civilization’, you turn into the thing you swore was the ultimate evil.

A 12-dimension assessment of your company’s AI maturity and readiness, and a roadmap for developing an AI strategy

How to think about AI now

Gen AI is the ability to understand language

• Search and retrieval with deep semantic understanding
• Create a first draft of anything
• Especially coding
• And if you can code, you can do anything a computer can do: advanced agentic processes that compose sequences of retrieving or sensing, creating, and acting.

Everything is language (or at least everything a business can do or talk about)

A short digression into epistemology:

Eddington’s ichthyologist: If you go fishing with a net with 2-inch holes, you catch stuff more than 2 inches big.

Tired:

• Biology is a subset of chemistry, the application of chemistry to living things.
• Chemistry is a subset of physics, the application of physics to materials.
• Physics is the application of mathematics to the natural world.
• Mathematics is the abstraction of everything.

Math is our deepest abstraction, and also the formal foundation of computation. Checking proofs, which start from axioms and apply rules of logic, is something a Turing machine can do. It can also do search to find new valid proofs. Gödel’s incompleteness theorem is analogous to the halting problem of mathematics. The former says you can’t write a program that will tell if any program stops without running it; the latter says there are mathematical conjectures that must be either true or false but are unprovable.

Wired:

• Information theory is everything you can represent with symbols and language.
• Computer science is the study about computation and Turing machines and other simpler models of computation.
• Mathematics is the study of the set of Turing machines that represent proofs.

Probably the last 3 are all isomorphic. If you can’t prove it, you can’t know it.

The point is: everything knowable is expressed in language. If you do language, you can write any program and solve any solvable problem (eventually).

As for anything which is not language:

Wovon man nicht sprechen kann, darüber muß man schweigen. - Ludwig Wittgenstein (What we cannot speak about, we must attend in silence)

Pillars of an AI strategy

• People - if you have a good AI team, they can implement everything below.

• Data - AI works on data. You want all your processes to generate clean data that AI can use for retrieval, creation, automation. Data engineering should provide data quality, timeliness, reliable availability, process observability, security, compliance, governance, discoverability, interoperability, scalability, lifecycle management, change control, cost control.

• Tech foundations: ability to reliably scale AI processes with security, robustness, business continuity, disaster recovery.

• Process: industrialize automation, software dev etc.

A good metaphor is infinite interns, but possibly stoner interns and occasionally disobedient rogue interns.

Pitfalls to avoid

• IP and data protection

• Reputational / regulatory risk

• All security concerns amplified - a power multiplier for bad actors, and for cyber defenders.

It’s useful to think of AI as an insider threat. Those infinite stoner interns need to be limited in what they can do (least privilege) and subject to multiple layers of checking for correctness and guardrails so they don’t go rogue.

AI Maturity

AI maturity measures the degree to which organizations have mastered AI-related capabilities to achieve operational excellence. AI maturity is not defined by tech tools alone, but by how effectively the organization can improve outcomes by orchestrating people, technology, data, and process. Organizations that successfully navigate AI transformation gain competitive advantages, operate with greater efficiency, increase velocity and respond faster to market changes, delivering more value creation.

Here are levels of maturity:

• Level 1 (Crawl): Initial unstructured exploration and planning.
• Level 2 (Walk): Structured learning, training, planning, foundation-laying, and initial POCs and pilots.
• Level 3 (Run): Operational deployment at scale, with robust infrastructure, governance, and security, and business value measured against KPIs.
• Level 4 (Fly): An AI-native organization where AI is central and pervasive, deploying advanced use cases like autonomous agents, advanced fine-tuned models, with continuous improvement and adoption of emerging capabilities.

Finally, here is a link to the model itself. Answer questions to assess your own organization’s maturity. For each dimension, 50% completion of a level unlocks the next higher level. Of course, it’s a work in progress, and opinionated. I definitely threw in the kitchen sink, everything doesn’t apply to everyone. But it should inventory most things people should be thinking about doing. (You can also download the HTML and delete/edit questions that don’t apply)

Another Simon Willison post has motivated me to go down a rabbit hole.

AI coding assistants and vibecoding are a sea change in the world of software engineering. They make midwit code monkeys like me something like 20% faster at creating a given functionality (depends on task, others say different, parts one, two, three). But they also let me tackle things I never would have tackled, and the output is more robust in the long run, in terms of docs, tests, readability.

Suppose I want to do a traveling-salesman traversal of a bunch of news headlines in topic order. With Cursor or Claude Code, I can ask it to find all the Python modules that do traveling salesman and perform my task. And I can look at the code, run them, pick the one that is performant and makes sense. I can try different embeddings and choose the smallest one that works (or do dimensionality reduction).

I’ve known 10x devs who can just emit complex working C code fluently and it works on the first try, and I can’t do that. But using a coding assistant, I feel like one of them. It feels like I’ve been given superpowers. Or at least a fairly capable pair-coding partner.

And then with Claude Code I can say, add this field to the database schema, update the CRUD code to reflect it, write tests, and update the docstrings. And it will one-shot it.

I could even sketch a UI and say, make this using React and Bootstrap, and it will do a passable job. I can integrate it with GitHub, assign it PRs, and it will take a crack at them. I can spawn multiple sessions and have multiple agents working on multiple Git branches. I can give it my style hints on how I write code, write my own slash commands, lots of awesome stuff. AI-first dev is just different, although there is a messiness tax.

If you are a late adopter, update your toolchains! Take the mini courses on Claude Code and Cursor and start to use them effectively.

Now let’s talk about Claude Skills. Because they might also be a sea change, and let normies build agent workflows, without using a Python agent framework like LangGraph or OpenAI Agents SDK, or even a lightweight agent builder like Zapier or Copilot Studio.

“A skill is a Markdown file telling the model how to do something, optionally accompanied by extra documents and pre-written scripts that the model can run to help it accomplish the tasks described by the skill.” (direct quote from Simon Willison)

It’s an orchestration layer. It’s the world’s simplest, most effective agent framework.

Suppose you want the LLM to act like your travel agent, based on some MCP tools from airlines and booking.com. With MCP, you could tell the LLM, book me a ticket from New York to Paris. And the LLM will look up the right MCP tool, say OK! This lets me search for flights, book them etc.

Now with skills, I can write a markdown file that says: “always buy business class, no redeyes, window seat, because that’s how I roll. Prefer Delta because I am Diamond Medallion.”

On Claude Desktop you make a skill by creating a markdown file explaining that this is a skill for buying airline tickets, and here’s a workflow to do that.

When you start a chat it puts a token-efficient directory of available skills in the context.

Then when you say, book me a ticket, the first thing the LLM does is bring the full skill details into the context. It’s like the LLM now has a binder full of SOPs and when there is something relevant in the binder, before starting the task it takes out the checklist from the SOPs.

That may not sound like much, but this happens at runtime and you can also say, try booking.com, but if it times out retry 3 times. And then do Expedia and then Travelocity. So you can do a loop and control flow.

And then you could say, if I book a flight to Asia, give me an overnight in London so I can visit my gran, first book NY to London and then London to Singapore or wherever using the same airline ticket skill. So that’s recursion.

And then, one required tool in this context is computer use, running in a sandbox. You can give your skill bash scripts and Python scripts. So they can run complex behaviors, without writing full MCP tools.

You should be able to write agent patterns by writing small skills combining some scripts and plain English descriptions, and an orchestrator skill in plain English (or have the code assistant write a script which would be more predictable: in practice explaining an agentic flow to the LLM in English is not ideal).

In the Claude Desktop sandbox, the computer use tool can’t access enterprise files, just what you upload to the sandbox. Still, easy way to implement small tools like, extract something from a PDF.

In Claude Code CLI a skill script can access local resources, network and the Web (with permission), so you could write a tool that just runs locally as a script and accesses the network and enterprise files.

So skills are more than hints files, they are an ultra simple agent framework and orchestration layer. For example, have a few skills that perform different tasks on videos with ffmpeg; have another skill that runs an end to end production process using those skills based on a doc explaining the process like a directed acyclic graph.

Or even 3 skills like plan a to-do list, code, test, and then a high level skill that says, take an objective from user and loop through: 1) update to-do list, 2) code the 1st item on list, 3) test and record outcome, until the objective is achieved. You just built a ReAct agent and a primitive version of Claude Code.

OpenAI and Microsoft have ~9 agent frameworks between them. Anthropic has 0. And yet, Anthropic has the most successful agentic tool for the biggest enterprise use case which is coding. Because Claude Code is really a universal agent. With tools + skills + computer use you can tell it to do anything.

Claude Code works well because it has

• A great to-do list workflow so it doesn’t go off task.
• Great context engineering using skills, putting the right part of the code base in the context
• A great LLM well-tuned for this process.

Maybe there is more secret sauce, but that is what I can see.

Skills and MCP can bring Claude Code to a higher level. So you are cruising along using Claude Code or Claude Desktop to do a research project. You use it do download a bunch of PDF articles, normalize to text, apply prompts to extract parts relevant to your project and make a virtual index card knowledge base. You can do all this extraction with computer use, it will write code on the fly. It might be a bit of a trial and error process, so suppose you find some parts that work well and you want to keep doing it that way.

• You can write an MCP tool, which is a fairly heavy solution.
• You can write a skill with a script for the computer use tool which is a lot simpler. In this case the skill is like MCP-lite.
• The skill is easy to share as part of a git repo. (can be a per-project skill in the case of Claude Code).
• The progressive disclosure/gradual discovery of skill details on demand means you can use more of them without cluttering up the context or confusing the LLM.
• Skills operate at a higher level in the stack compared to tools and can call each other, so you can potentially implement more complex agent patterns. Although I would probably write Python with a framework like Agents SDK once agents get complex. But I could eventually see skills that abstract some of those patterns.

Skills add a lot of agent bang for minimal framework complexity bucks.

I can also see a world where Claude Code knows you are say a financial researcher, and constantly adds new skills, puts them in hierarchies to abstract tasks like, finding and understanding news and research, analytics, writing. And improves them as it watches you work. For now you have to write your own skills with Claude’s help though. (edit - this is changing with plugins and marketplaces)

The ‘agentic browsers’ are a bit disappointing to me, I can’t really tell it, read this folder full of PDFs and extract relevant parts using prompts into notes for this presentation I am doing. And I still have some security concerns about prompt injection. But I can do this research task with Claude Code. And over time all browsers will be agentic browsers with skills, probably.

Claude Code and skills punch above their weight. You can use Claude Code and skills and computer use as a universal agent to do anything on your computer implementing complex agent patterns.

If you haven’t already done it, download Claude Code, hook up some MCP servers and try some skills. I have barely scratched the surface and am excited to do more.

A couple of small skills for this research project here.

See also:

• Claude Code tutorial
• How I Use Every Claude Code Feature
• Exploring Claude Code: Features
• How an AI native actually works
• 50 ways non-technical people are using Claude Code in their work and life
• Awesome Claude Code

“He was very unsettled and he very deeply believed that vaccines hurt him and were hurting other people.”

RFK, Jr. promises to ‘clean up cesspool of corruption at CDC’.

“In such a world of conflict, a world of victims and executioners, it is the job of thinking people, not to be on the side of the executioners.”- Albert Camus

Medicine is high-bias, in the mathematical sense of a model that is relatively insensitive to changes in data. Like a linear model vs. a higher-order model that can follow complex multidimensional curves in the data. Medicine over-indexes on pedigrees like Harvard PhDs. It is slow to accept revolutionary ideas even in the face of evidence, and then gives Nobel laureates who originate them possibly undue deference. “Science progresses one funeral at a time”, as explored by Thomas Kuhn.

For life-and-death decisions, this is usually for the better.

Science needs falsifiability and verifiability, or it’s not science. Statistical evidence is a very dull, sample-inefficient knife. For instance, it’s generally accepted that cigarettes contribute to cancer, or that high cholesterol of certain kinds contributes to heart attacks. But it took a while. People have argued and maybe still argue that medicine is wrong, it’s more complex than that, that both cigarettes and cancer are downstream of e.g. stress and lifestyles and genetics that make people want to smoke.

So you can expose lab animals to cigarettes (or high cholesterol), or to the toxins in cigarette smoke, and see what happens. You can randomly target people with anti-smoking or anti-cholesterol interventions. Complex systems are hard, like maybe the cholesterol in arteries doesn’t come from dietary cholesterol but from high carb diets. But at some point, it becomes perverse to deny some degree of causality, as calculated by increasingly systematic methods, even if they only gradually approach the truth.

On the other hand, health influencers like Andrew Huberman, Dr. Mike, Dr. Phil, Dr. Oz (if it quacks like a quack) probably over-index on engagement.

There are probably ways to synthesize slow research and fast, extreme social media, to leverage all the crowdsourced data and discussions to make medicine better and more adaptable over time, and customized to different populations and individuals, and communicate and educate better. Doctors should be a bit concerned how often people report that ChatGPT does a better job of listening to patients and diagnosing.

People get mad when medicine is slow and hidebound. And they also get mad when it seems to change advice on e.g. masks or diets, and what was bad is now good. They can even get mad when it cures a pandemic. Not because it was too slow or too fast but because…well I’m not really sure what the problem is. They say the mRNA vaccine is ineffective and risks outweigh the benefits but it certainly seemed to end the pandemic, and evidence of widespread problems seems scant.

Also they are canceling research into messenger RNA therapies, not just the current crop of vaccines. Reprogramming the immune system is just too powerful, we don’t want to have cancer vaccines I guess. It’s not enough to say the climate science is wrong, we’re not allowed to even collect data now.

Cultishness might be a better description than engagement of what newer social-media movements over-index on, the health influencers, Trump and TESCREAL and Bitcoin.

Unfortunately it’s a recurring story of our era. There’s a lack of trust in institutions attributable to emergence of social media. Internet vibes are more real than what goes on in labs and models. And it might keep getting worse and collapse entirely. Or we might figure out how to use AI to make it better, more trustworthy.

There is truth to the notion that institutions have some catching up to do with technology. It’s not like we would have flying cars by now if government didn’t stand in the way, but government tends to slow things like new drugs and Ubers down. And we should have real-time payment clearing and real-time spending and jobs data.

But what’s happening is like living with a crazy person. With crazy people you can nod and say OK and let them think and do what they want. But the inmates are running the asylum, and now you can’t even get those therapies because they won’t be approved or offered and the research will happen elsewhere. It seems more about control than reality. You can’t tell me what to do, any thinking or data-gathering about systemic effects and behaviors is socialism or tyranny or something.

Then trolls like Q and Alex Jones find the pressure points, and the mountebanks like Dr. Oz, RFK Jr., Fox, Trump, Musk attack them relentlessly to gain wealth and power. And then Dr. Fauci gets death threats, and people shoot up hospitals and medical research offices.

Its a con man’s carnival, a grifter gravy train. The information space is just completely captured by memes divorced from reality, and by hustlers, scammers, trolls, and con artists. We really need decent reality-grounded folks to figure out how to recapture the marketplace of ideas, or we’re doomed to government of the shitposters, by the shitposters, for the shitposters. There are folks like MeidasTouch and the Krassensteins who sometimes get labeled grifters and clout chasers. But maybe every successful business is a bit of a cult or a racket, everything human that endures has an element of grift and graft and bullshit. They had to sell indulgences to build the Sistine Chapel.

US democracy is pretty low-bias and high-variance; It’s boundedly responsive to voters, and to money. It also has a lot of apathy, most people can’t be bothered by politics or understanding policy, and stuff generally just works without their participation. Until recently there was a reasonable amount of consensus around obvious stuff like measles and polio vaccines, and fluoride conspiracies were way outside the mainstream. But US democracy has pressure points, and you can make a buck selling crazy viral stuff. Folks like Putin and Murdoch and Musk figured out how to exploit the pressure points, and sell appealing self-serving nonsense to their benefit. The founding fathers talked about this a lot, they wanted indirect election of the president, lifetime judge appointments to introduce high bias (lower sensitivity to public opinion) and avoid high-variance populist demagogues and big changes in direction. Later governments did the same by delegating authority to commissioners and the Federal Reserve to conduct independent analyses and policies.

Of course, it was just a nut who shot at the CDC, just like it was just a nut who beat Paul Pelosi with a hammer, just like it was just a nut who gunned down legislators in Minnesota etc., etc.

But, you know Trump did January 6, he invoked 2nd amendment solutions, he wanted to call the military out against protestors, he repeatedly said stuff like, back in the day we took care of bad people. The Army is there to shoot bad people. Let’s pass laws to make it OK for righteous people to run over protestors or ‘stand their ground’ against bad people. When the looting starts, the shooting starts. He refused to call Gov. Walz because he’s a bad guy. His minions reshared stuff about the shooter being a Walz appointee, just like they shared stuff about Pelosi getting attacked as part of some kind of gay love triangle. Could write 10x as much on extreme right-wing violence and Trump stoking it with eliminationist rhetoric, and you know it’s true. Even Grok admits it.

There’s a shitposting strategy to constantly be doing and saying outrageous stuff to keep the narrative on him and away from sensitive topics, and to be so crazy and fast-moving that any reasoned, evidence-based response is responding to the stuff from 2 news cycles ago. It’s about control, not reality.

There’s an iterated prisoner’s dilemma game of building trust, and there’s a one-time prisoner’s dilemma game of defection, where you have to crash the cockpit door and kill the woke mind virus or else nothing else matters.

Of course, every election is a Flight 93 election now. So the Republicans want to redistrict Texas and do a new Census now. And some Democrats are like, OK in that case we’re going to redistrict NY and California to be 90% Democratic. On the one hand, OK you might have to fight fire with fire and play to win. But if one side wants to kill democracy, they are happy to see you abandon democratic principles. You go down that road it’s not much of a democracy.

There is a logic of peace which is to try to find common ground (prisoner’s dilemma cooperation), and there is a logic of war which is to destroy the enemy before they destroy you (defection). The side that wants to end democracy is guaranteed to win eventually. Defection breeds defection. They only have to win once, unless they are stopped in a relatively final way first. There’s a reason the extreme right hates Churchill.

In my lifetime the conservatives were the high-bias party, respecting tradition and hierarchy and wealth and pedigree, standing athwart history yelling ‘stop’. But now they are the party of revolution and chaos, they got hostile-taken-over by ends-justify-the-means Jacobin reactionaries, and it will evolve in exactly the same way. Poorly and violently.

It will make great TV, or TikTok. In the words of the greatest Doctor, we’re all just stories in the end so make it a good one. Not one that looks fire on TikTok (literally). Not one that future generations will say, wow, how could they believe this guy who can’t utter a coherent sentence, who is a pathological liar (Ibrought eleventy gajillion dollars into the US! I cut drug prices 1,500%! I prevented 32 wars this morning!), who is a congenital cheater and sex abuser, and likely child molester, and holds deep antipathy for huge swaths of Americans. I mean, given what we know about Epstein, how likely was the option presented, and how likely was it to have been taken.

In the words of Yeats:

Things fall apart; the centre cannot hold;
Mere anarchy is loosed upon the world,
The blood-dimmed tide is loosed, and everywhere   
The ceremony of innocence is drowned;
The best lack all conviction, while the worst   
Are full of passionate intensity.
Surely some revelation is at hand

This is a rant about cybersecurity and the information space around AI.

A couple of years ago, I was out sailing in NY harbor. A gust picked up my red L.L. Bean baseball cap and blew it into the water. I went home, and I was scrolling Facebook, and the first thing I saw was an ad for a red L.L. Bean baseball cap.

So what are the odds that Facebook really does listen to all your activities and show you ads according to them?

I’ve seen Mark Zuckerberg talk over the years and say things like, “They trust me. Dumb fucks.”

All too plausible that he would try things like that.

But I also know how machine learning works. And the algorithm knows I sometimes shop at L.L. Bean, and certainly that I looked at the red baseball cap if not actually bought it.

And it knows my location. It’s at least equally plausible that machine learning would learn that after a day doing outdoor activities, people sometimes buy baseball caps or outdoor wear, and that it would show me something that I had been interested in.

For sure there are some shenanigans that we don’t know about. But the known ways they monitor and manipulate us are as bad as the conspiracy theories. Who needs to spy on you when you let them know where you are, who with, and every online interaction?

Eventually I quit Facebook:

• I had a possibly ADD-related unhealthy relationship with it. I would spend too much time there…even though most of the time was not that positive. There was a lot of addictive clickbait-y junk and stupid stuff, and people getting in my business. And politics.
• I felt there was more upside to leaving in terms of overall mental well-being.
• Overall not trusting the company. Google and Apple know everything about me. I don’t need every company to know everything, especially ones who have repeatedly showed themselves to be manipulative and untrustworthy.

Now I want to share this tweet.

Okay, not gonna lie, this is stupid.

ChatGPT explicitly provides an opt out to training on your data. Language in contracts for products like ChatGPT Enterprise is also explicit. Here are personal GPT settings:

A lot of work goes into making a clean data corpus for pre-training. If you do a multi-million-dollar training run, and then realize the data set was contaminated with really bad stuff, there’s no way to fix it. You have to start over from scratch, or blacklist some questions because responses expose you to liability. It’s just a disaster and a PR nightmare.

LLMs struggle to acquire new knowledge through post-training and fine-tuning. Reinforcement Learning With Human Feedback, Direct Policy Optimization, Supervised Fine-Tuning are mostly about structuring better responses on the existing knowledge base.

It’s beyond unlikely that OpenAI is intentionally, systematically, taking your private conversations and training with them. I think if you have any picture of how the LLM sausage is made, the scenario does not seem remotely plausible. Of course it’s hallucinating that it read the board minutes.

Nevertheless, I’ve repeatedly had this conversation with, you know, smart IT people who are supposed to be familiar with the technology and ToS. And sometimes people have got a little bit testy with me for telling me their concerns are over the top. “I heard it from X! Trust me, bro.”

I’m not saying there might not be reasons to avoid ChatGPT. How should AI engineers and CTOs and CEOs think about the risks of putting private data, proprietary data into LLMs?

There are levels to this game.

Level 0: What, me worry? Let anybody put anything they want into any SaaS products, install any mobile apps, Chrome extensions that can read and change your Web pages.

This is clearly a very bad idea. Look at 23andMe. They had a severe breach, they went broke, they could have been bought by anybody who would then have had all your DNA information, and not been subject to the contracts that were previously signed.

And frankly this seems like a blind spot in our existing privacy and security landscape. Some rights you shouldn’t be able to sign away, or lose in bankruptcy court.

If you are Goldman Sachs, and Google knows everything you search for, Bloomberg has all your chats with your clients, they will know what you are up to. Other providers who know more than you might be comfortable with: NetJets, your ISP, your mobile provider, whoever runs all the networks of video cameras, personal assistants, butlers and system admins, and so on. Goldman basically started a company that is Signal with compliance because of discomfort with Bloomberg. And a huge hedge fund is basically, what if Google used every bit of online data exhaust for investing. It’s the Wild West out there. Software is eating the world so everything gets captured, and now AI can find everything. You have to do better than ‘nothing’.

Level 1: Understand terms of service and only use trusted SaaS endpoints, and exercise opt-outs, and implement internal policies and controls about what can be pasted into approved AI.

That seems like the bare minimum. Then, as long as you trust that ChatGPT complies properly with the-opt out and the terms of service, and they don’t suffer a breach, you should be okay.

Should you trust OpenAI? They are a startup. Things are a little bit chaotic. Nation-states spy on them. And they have had breaches where your conversations were shown to other users and where info was stolen.

Also they are currently under a court order to retain logs indefinitely and let plaintiffs search them. Presumably a separate pipeline that doesn’t impact e.g. other training and operational pipelines. But again, breaches happen.

So you need to exercise your own judgment about which SaaS services you can trust and what the upsides and the downsides are.

Level 2: Trusted public cloud. If you are a Fortune 500 company or a regulated investment company, or you handle medical or other data that’s subject to HIPAA and other regs, you need stronger representations about the privacy of the data. You might avoid public API endpoints. You might use a cloud provider like Microsoft’s Azure, which is certified ISO 27001 compliant and SOC-3 and HIPAA compliant and audited. And then you access hosted LLMs via that trusted public cloud.

If you use Microsoft Teams and hosted Microsoft Office 365, then you have chosen to trust Microsoft with your internal communications. There is no reason not to trust them for LLMs.

For most companies, major cloud providers are the best they can do. But if you are the CIA, or a really secretive investment firm like Renaissance, you are not even OK with public cloud.

Level 3: On-prem servers and trusted private clouds. You host your own email, and chat, and video-conferencing, and your own LLMs. No Teams, no Zoom, no Gmail. You implement things like biometric access, compartmentalization.

But if you build everything in-house, it’s expensive. You still have to trust someone: your employees, and the vendors who help you build it. And constantly monitor it and update it. And you have to trust that you’re able to do it more securely than Microsoft and AWS and Google, and it’s not their first rodeo. If you’re not extremely capable, you’re just gonna spend a lot more money and create a lot of hassle and end up less secure.

If government-sponsored hackers want to breach you, they still have a good shot, by compromising workers, having their spies work for you, intercepting equipment and tampering with it. With enough effort, they can probably get into anything. If Microsoft fails to stop them, again and again, what chance have you got?

As an IT manager or CTO or CEO, you have to pick a posture based on your own company’s sensitivity to a hack and loss of data and based on your resources, and for most companies, keeping everything in a single trusted cloud is the most appropriate solution.

So choose your own adventure. Pick your poison. How dead are you if you have a breach, and what can you afford to prevent it? Understand your threat models and what will kill you, like ransomware and spearphishers wiring firm or customer funds to a Manila casino. Make sure you are implementing best practices, spending appropriate resources combating appropriate realistic well-grounded threats, and not chasing your tail and making operations more complicated for no reason.

All this to say, there are valid reasons why you might not want to use OpenAI, but them spying on and publishing your private conversations is not one of them. (Unless you somehow leak them onto the Web and into the training corpus).

And you should have people who know what they are doing, not making decisions on the basis of tweets or panel discussions at your typical industry conference.

So what’s my point? Mostly I just wanted to get this off my chest, and the next time it comes up, point to it.

But also, there’s a lot of nonsense out there. There’s bad medical information out there, bad AI information, bad cybersecurity information. It’s a free country, anybody can be a viral influencer on social media.

If you’re a doctor or an IT guy, people already don’t listen, you give simple detailed instructions, they hear what they want to hear and do whatever they want to do. AI just makes it worse, they’ll find someone to tell them authoritatively whatever they want.

There are a lot of professional AI-haters, like Gary Marcus and Ed Zitron, for whom AI can do no right. Even the so-called experts, what they say does not always do a service to the community.

And of course, there’s an insane amount of hype about AI and people who sell it act like it’s better than it is, and will be AGI if you just give them seven trillion dollars, man. Some managers swallow it, and think that you can just say AI, do this complex task and it will spontaneously automate it flawlessly, or at least perform it as well as humans with no errors or hallucinations, and then you can cut staff.

AI figured out language. Which is a huge step forward. But it has a long way to go to be anything like AGI. It has to be able to learn on the fly. It has to integrate the left brain and right brain, Type 1 and Type 2 cognition. But just with a deep understanding of language, we can automate a lot of things, and it will disrupt a lot of professions and industries. And a lot of non-AI savvy companies are going to be disrupted by hungry, smart, AI-first companies.

No one is coming to save you. Our leaders don’t know what they’re doing. There is no Real Room, where real experts in charge with perfect information and real authority work. It’s just us. We are The Room. There’s no replacement for serious thinking grounded in reality, and it’s in short supply.

So prep for artificial intelligence, and also inconceivable amounts of human stupidity. Be careful out there.

(Obligatory disclosure, on some days, and some matters I am without doubt one of the Dumb People. Having made this clickbait title, there will without a doubt be a stupid typo in here.)

Any sufficiently advanced technology is indistinguishable from magic. — Arthur C. Clarke

What are AI agents? Simon Willison crowdsourced a lot of definitions that focus on:

1) Using AI to take action on the user’s behalf in the real world (i.e. what the agent does)
2) Using AI to control a loop or complex flow (i.e. how the agent does it).

An AI agent takes a sequence of actions based on an AI-determined control flow.

Agents use prompts as the CPU of a Turing machine that can manage state, memory, I/O, and control flow. The agent can access the Internet and tools to perform compute tasks, retrieve info, take actions via APIs, and use the outputs to determine next steps in a loop or complex control flow. Maybe even control a browser or computer.

In this post, we’ll try to develop a roadmap of agent concepts and patterns to learn, and resources to learn them.

Spinning Up: From Prompting to Single-Turn Agents

Before diving into agent patterns, let’s review 4 LLM foundational skills:

1. Prompting – The term ‘Prompt engineering’ contains an element of puffery, but we need to give clear, specific, instructions, so the LLM answers properly and in the right format. It’s an entry‑level skill that underpins every other pattern.
   • C-L-E-A-R Framework:
     • Contextualize - Specify a role or persona and intent: “You are a copy editor with years of experience polishing articles and blog posts for web publication. You will read the content carefully and suggest changes for clarity, style, and standard usage.”
     • Limits - Length; format e.g. ‘three bullet points’; tone or style e.g. ‘concisely’, or ‘like a tech journalist’, or ‘only use facts from this text’.
     • Elaborate and give Examples - Explain and provide as many details and specifics as possible. Use chain of thought and other advanced prompting methods.
     • Audience - Identify the audience the response is addressed to, such as ‘explain like I’m 5’.
     • Reflect or Review - Prompt ChatGPT to ask clarifying questions before answering, give itself space, such as “think step by step”, “make sure of x before answering”.
   • Alternatives: T-C-E-P-F-T (Task, Context, Example, Persona, Format, Tone). Or P-R-E-P-A-R-E-D (Propose, Role, Explain/Explicit, Presentation, Ask, Rate/Reflect, Emotions, Diversity). Use what resonates with you.
   • These days, you don’t need to spend hundreds of hours learning prompt engineering. Think about your intention, take a first crack using one of the above frameworks, and then ask your favorite LLM to improve it, iterating as necessary. OpenAI Cookbook: Enhance your prompts with meta prompting.
   • Side quest - proper evals, and automated prompt engineering with tools like DSPy. If you have good evals, you will eventually have good prompts and outputs through iteration. If you don’t have good evals, changes in underlying LLMs and assumptions will break your prompts and workflows.
   • As prompts become more dynamic and complex and we add tool use (discussed next), we start to think about it as ‘Context engineering’ to ensure the LLM has info it needs to provide a good answer. However we name it, good prompting and evals are a foundational skill. You need to know how to talk to the AI and measure the results.
     
   • See previous post: Practical ChatGPT Prompting: 15 Patterns to Improve Your Prompts
   • Also see GPT 4.1 prompting guide. The recent 4.1 OpenAI models are trained for agentic workflows, they do many things automatically that required complex prompting in previous models.
      
     

2. Tool Use – Expose a catalog of external APIs to the agent, like a Python function, search, SQL, REST, a browser, a full Python interpreter REPL, or a shell terminal, etc. Let the LLM decide whether to call a tool and which tool to call at each turn of the conversation. The LLM can respond with a request to call a tool using a provided calling signature; the user or client calls the tool and responds with the output; the LLM then uses tool output to determine the final answer or an additional tool call). See Microsoft: Tool Use Design Pattern

3. Retrieval-Augmented Generation (RAG): Give the agent documents and a tool (such as a vector database) to find relevant parts of the documents and respond using them via in-context learning, i.e. stuffing the prompt with data to ground the answers. “Please answer this question using the following text”. See Retrieval Augmented Generation (RAG) — A quick and comprehensive introduction.

4. Chain‑of‑Thought Prompting – Ask for step‑by‑step reasoning traces that make the model’s logic explicit and usually boost accuracy on math, logic and multi‑step tasks. “Before providing a final answer, please think step by step, walk through your complete research and thought process, and show your reasoning.” Kind of like ‘explaining your code to your rubber duck’, telling the model to show its work, and explain what it’s doing as it does it, forces it to think longer, and improves performance. Thought traces are also important for agents to remember what they did previously and why they did it. Paper: Wei et al.

With these 4 elements you can build highly capable single-turn OpenAI Assistants or Custom GPTs.

You can write a system prompt that describes a workflow to follow in response to user input, give it docs with detailed processes and reference information, and external tools to use. See for instance this Tuck AI Matrix custom GPT, which follows a proprietary methodology to do basic evaluations of M&A deals.

However, the OpenAI custom GPTs and Assistants (and their equivalents on other platforms) have a single prompt, so they aren’t true multi-turn agentic workflows. (They can call multiple tools though.) To level up to full-blown agentic workflows, we want more customizable multi-turn workflows that may use many different models, custom tools, sub-agents, and complex control flows.

16 Multi-turn Agent Patterns

1. ReAct (Reason + Act) Loops – Interleave “Thought → Action → Observation” so the agent both reasons and calls tools (search, code, DB) in the same dialog, allowing complex chains of thoughts and actions. This was the breakthrough behind AutoGPT, which you can run online here. Ask a question like ‘Find the best coffee grinder for espresso under $300’, and it will loop through a process of thinking, what are the tasks I need to do based on what I’ve done so far, what is the highest priority task, do it, observe the output, iterate until the goal is reached. Paper: Yao et al. ; Blog posts: Matt Webb; Simon Willison
    
   The ReAct pattern is fascinating and powerful, but the autonomy can make it unpredictable and hard to reason about, which is a general tradeoff when building agents. The more autonomy you give it, the more room for emergent behavior, but also the more risk it goes off the rails.

2. Prompt Chaining & Sequential Workflows – Break a complex task into ordered sub‑prompts with intermediate validation (“gate checks”) before moving to the next stage. LangChain: Build an Agent. Example: a news gathering workflow. Small prompts that perform composable tasks that can be tested and optimized independently are more predictable and easier to maintain than long ‘God’ prompts that try to do everything. See What We’ve Learned From A Year of Building with LLMs

3. Structured Output – Since errors tend to compound as you go down the agent’s trajectory, structured outputs and validation are critical. Ask the model to return JSON, then validate it, letting downstream code parse or act on the response safely. If output doesn’t validate, attempt to fix the output, or fix the input and retry. The GPT-4.1 models are exceptionally good at returning valid JSON, which you can also use Pydantic to specify and validate. Study the prompting guide thoroughly.

4. Human-in-the-loop – At the current maturity of AI development, fully autonomous agents are typically very challenging to achieve in complex, high-stakes environments, where they add the most value. It’s much more realistic to try to make agentic assistants and copilots that take humans through a structured process, than to aim for full autonomy. The AI can speed things up dramatically, but performance can also be hit-or-miss, so human supervision is critical. At key steps the human should evaluate and course-correct as necessary. Time travel allowing the human to go back to a previous step, adjust, and try again can also be useful.
    
   In fact, if you remember nothing else from this post:
    
   🤖🦾 Use AI for what AI is good at: parsing lots of information quickly and generating a first draft at a near-human level; fuzzy logic (easier to ask the LLM the human name for a URL than to parse the numerous ways websites provide titles)
    
   🛠️⚙️ - Use tools for what tools are good at: faithfully executing algorithmic workflows;
    
   👩‍⚕️💪 - Use humans in the loop for what humans are good at: critical thinking and creativity.
    
   

5. Reflection – After an initial answer, the agent critiques its own work and revises. For instance, a check for factuality may reduce hallucinations. Is this summary consistent with the text it summarizes? The agent can reflect and improve multiple times until satisfied, and/or perform multiple separate checks, like for factuality and also a Flesch-Kincaid readability benchmark. See DeepLearning.ai “Reflection” pattern. Paper: Shinn et al.

6. Evaluator‑Optimizer (Generator‑Critic) Loops – Divides the reflection pattern into substeps. One LLM prompt proposes an answer, another scores/criticizes it, and also provides directions for improvement; apply recommended improvements; and iterate until no further improvements are needed. Anthropic post “Building Effective AI Agents”

7. Task Routing / Mixture‑of‑Experts – A router runs a classification prompt based on the current state to determine the next action, such as a prompt or sub-agent workflow. Anthropic Agentic Systems - #2. Routing

8. Agentic RAG & Specialized Retrieval Teams – Multiple retrieval agents individually query their own knowledge pool; an aggregator agent synthesizes the evidence before final generation. IBM primer “What is Agentic RAG?”

9. Memory – In the basic ReAct workflow, the previous chain of the conversation serves as the record of everything learned so far. But you might want to structure the memory in the agent in a more efficient manner, and inject knowledge into the context in the manner the LLM can use it most effectively. See e.g. IBM: What is AI agent memory? ; Mem0

   • Short‑Term Memory – Keep just enough context (conversation buffer, sliding window, or summary) inside the model’s token limit for coherent multi‑turn chats. Context Windows: The Short‑Term Memory of LLMs

   • Long‑Term Memory – Persist facts or conversation extracts or summaries in a vector database, knowledge graph, SQL or NoSQL database, and retrieve them on demand so the agent “remembers” over the course of a long session working with lots of information, and across sessions. (A vector database has one job, fast nearest neighbors so you can look up text by semantic similarity.) Pinecone guide to conversational memory with LangChain

10. Plan‑and‑Execute (Hierarchical Planning) – First draft a high‑level plan, then execute each sub‑task in order. LangChain: Plan-and-Execute Agents; Paper: Wang et al.

11. Parallelization of Sub‑tasks – In contrast to sequential tasks, we can fan out independent LLM calls asynchronously (map‑reduce, parallel tools) and aggregate results for speed or consensus. We can perform similar tasks in different ways and pick the best one, or take all the outputs and synthesize a response from them. With ‘mini’ and ‘flash’ models (or local models), it can be cheap, fast and efficient to send hundreds of queries over a data set to process each one individually and simultaneously. LangChain: How to invoke runnables in parallel

12. Tree of Thoughts – Many agents run variations on search processes. The agent can explore multiple possible paths of thinking, decisions, or hypotheses before settling on an output or an action. Agents can go beyond single-step or linear reasoning by allowing branching and revisiting of different trajectories. See papers: Tree of Thoughts (ToT) and Graph of Thought (GoT)

13. Guardrails – Users may try to ‘jailbreak’ an app to use it in unexpected ways, and agentic processes can go off the rails and attempt costly or embarrassing or even dangerous actions. Guardrails. a form of reflection, can allow you to assert that the agent is on track at each step, and correct, or stop, or prompt the human if it strays.

14. Orchestrator‑Worker Architecture – A central orchestrator maintains state, assigns work to specialized worker agents, and merges their outputs—a pragmatic bridge to complex multi‑agent systems. The LangGraph state graph framework is one pattern. The state graph paradigm provides transparency, maps well to structured processes with time travel, and potentially in the future to low-code frameworks and cloud deployment.
     
    Alternatively, we can use the OpenAI Agents toolkit, make each sub-agent node a tool, and have a top-level reasoning prompt describing a workflow and telling the LLM to run the workflow using the available tools.
     
    You can also use low-code frameworks like Zapier, Make.com, n8n, and Automation Anywhere (and a ton of others), which could then call an AI prompt to make control flow branching decisions, run loops, call OpenAI Assistants or agents deployed in the cloud.

15. Multi‑Agent Collaboration – An orchestration pattern that uses distinct role‑based agents (e.g., Planner, Coder, Tester) that converse to solve problems that exceed a single model’s capacity, in a peer or hierarchical organizational structure. Multi-agent systems can be a bit like using Docker/Kubernetes microservices vs. monolithic architectures. They can provide a helpful decomposition, or make the system more complex and harder to reason about. For my money, you can get a lot of mileage with AI in a structured workflow running in LangGraph, n8n, Zapier, etc. In general I would recommend starting with simple chained pipelines, over complex multi-agent solutions. AutoGen is a leading multi-agent framework. Wired article.

16. Model Context Protocol and other communications protocols – When you create a tool, after implementing its functionality, you have to tell the LLM how to use it: the input schema, the output schema, when and why to use it. Model Context Protocol (MCP) is a standard for doing this.
     
    Another important evolving standard is Agent2Agent protocol (A2A). If one agent calls another agent, it may be a long-running process and a multi-turn chat conversation, unlike the REST call typically described by MCP. So there may be a need for a different standard to monitor long-running processes that come back to ask for more information, or that you want to send a sequence of interactive requests to. See the paper: A Survey of AI Agent Protocols

Where To Start

If you want to start building an agent to automate your pain points, here are some ways to start:

1. Connect Zapier to your SaaS services and LLM, like write a mail autoresponder that applies a prompt to extract stuff from an email and updates the CRM.
2. If you are enterprise as opposed to SMB, and putting credentials to all your SaaS in Zapier gives your the heebie-jeebies, use n8n on-prem instead. (Or standardize on an on-prem or cloud no-code framework, of which there are many.)
3. Write or grab some MCP wrappers (here is a list, or check out Jeff Emanuel’s ultimate MCP server with connections to tons of services), connect them in the Claude desktop client, and ask Claude to do stuff in your favorite services.
4. For the most advanced custom agent workflows, LangChain and LangGraph are great to get a POC up and running. If I want to figure out how to build an LLM workflow, typically I’ll look up how LangChain does it and try their variations, different LLMs and prompts, and take it from there. LangChain can be frustratingly complex and beta. Sometimes you realize all the magic is in the prompt and you can just write directly to e.g. OpenAI. They try to do all things for all people and have the most comprehensive ecosystem. But they can be architecture astronauts and try to do too much, make a lot of breaking changes, docs are not always great. OpenAI Agents SDK, CrewAI, AutoGen, LlamaIndex, SmolAgents are other alternatives.
5. Don’t be afraid to do things that need a lot of sequential prompts or to send a lot of prompts in parallel asynchronously. It’s hard to do things well with one prompt, our job is to make a reliable agent from unreliable LLM calls.

Final Thoughts

The era of agents is here. If you can explain a process in plain English, you can probably find or build MCP servers around the tools it uses, and build an agent to take a crack at it. If it’s a simple process, you might be able to fully automate it most of the time; if it’s a more complex process, you might be able to build a structured human-in-the-loop process around it that will level up productivity.

When you see an agent doing something simple, like my daily AI news bot, or something magical, like OpenAI’s Deep Research, it’s probably doing something that is a composition of patterns like these, and the right prompts applied to the right contexts. If you’ve read this far, you have an initial grasp of the basic concepts of agent engineering, and a few places to continue your learning journey!

Further Reading:

• Anthropic: Building effective agents
• Anthropic: How we built our multi-agent research system
• LangChain: Workflows and Agents
• Anthropic: Our framework for developing safe and trustworthy agents
• Chip Huyen on Agents
• Andrew Ng: The Batch
• DeepLearning.ai
• Mastering AI Agents: The 10 Best Free Courses, Tutorials & Learning Tools
• A Survey of Agentic AI, Multi-Agent Systems, and Multimodal Frameworks Architectures, Applications, and Future Directions

• AI Engineer summit 2025: Agent Engineering: Day 1 Day 2

• Agent Frameworks
  • LangGraph
  • OpenAI Agents SDK
  • Google ADK
  • LlamaIndex
  • AutoGen / AG2 (confusing fork under way)
  • CrewAI
  • SmolAgents are other alternatives.
• Druce’s related stuff
  • A slide presentation
  • I save articles here
  • Skynet and Chill daily AI reading (agent-generated with HITL)
  • AInewsbot agent to write an automated daily news letter. Barely agentic, it’s a long chain of prompts, the AI chooses stories and summarizes them and writes clever titles but not a complex AI-driven control flow.
• AI Engineering stuff (might break out into a separate post)
  • Matt Turck’s mega-landscape dataviz
  •  
• People to Follow
  • Simon Willison
  • Hamel Husein
  • Shawn ‘swyx’ Wang
  • Nir Diamant
  • Ethan Mollick
  • Andrew Ng
  • Harrison Chase
  • Jerry Liu
  • Logan Kilpatrick
  • Chip Huyen
  • Cameron Wolfe
  • Andrej Karpathy
  • Lilian Weng (whose blog template I appropriated a few years ago, many thanks!)
  • Heiko Hotz
  • Andriy Burkov
  • Latent Space Team
• Books
  • Jay Alammar - Hands-On Large Language Models: Language Understanding and Generation
  • Sebastian Raschka - Build a Large Language Model (From Scratch)
  • Chip Huyen - AI Engineering
  • Maxime Labonne - LLM Engineer’s handbook
  • Andriy Burkov - Machine Learning Engineering

Economics is the painful elaboration of the obvious.

Pundits sometimes say things like “AI is going to make our workers more productive, and they will reap the rewards with higher wages.”

It’s mostly worked out the way in the past. But the labor share of income has varied. How much labor benefits, and how much capital benefits, depends on how technology complements labor, versus substitutes for it. There is little support in economic theory for the notion that technological progress always raises everyone’s wages and standard of living. It’s empirically been significantly true in the past. But the notion that it’s an iron rule is just pop economics, a Panglossian belief based on motivated thinking.

AI is the most human-like technology ever invented, so it seems likely to be an effective substitute for human labor. It seems likely that we will get growth but also disruption, more income inequality, more concentration of wealth, and more people locked out of decent middle class and working class jobs. The worst case would be an ‘economic singularity’ of robots making more robots while masses are immiserated. We should think about how to detect the singularity and use policy to head it off.

Let’s break it down (painful as it may be).

In my first job out of college, I worked in the economics research department of a big investment bank. I wrote for the economics research reports, was occasionally quoted in the New York Times as ‘economist Druce Vertes’ if they spelled my name right, which they never did.

I also applied for economics PhD programs and was told to go back and get As in differential equations and linear algebra, and summarily rejected. All this to say, I’ve studied this stuff but I’m also kind of a midwit and it’s been a while.

AI might be as disruptive as previous industrial revolutions: steam, electricity, the internal combustion engine. Current AI impacts about 50% of jobs today. When we get really good robots mass-produced at scale in maybe 10 years or so, it will impact close to 100%.

Surprisingly, compared to previous revolutions, LLMs impact high-skill, high-education creative and analytical jobs like coders, writers, analysts, graphical artists more than low-skill, low-education jobs. And the AI community expects fast progress to continue: incremental improvements, bigger scale, maybe new breakthroughs in rapid training, knowledge representation, bridging the gap between Kahneman Type 1 perception and language and Type 2 algorithmic thinking.

AI can generate more human-like behavior than machines we’ve had before. It changes the relationship between capital and labor. Let’s look at the shape of the basic models economists use to think about how capital and labor interact, and then think about how AI changes those relationships and what it means.

The Cobb-Douglas Production Function

Economists use the Cobb-Douglas production function as a simple model of how capital and labor interact to produce GDP and national income.

If we assume complementarity between capital and labor, and also constant returns to scale so that doubling both capital and labor doubles output), then we can express the relationship with a production function of the form:

which is called the Cobb-Douglas production function (with constant returns to scale)

\(Y\) is real output (GDP). These are all flows per unit of time, like GDP per year.

\(L\) is the quantity of labor services—e.g., total hours worked, ideally human capital‑adjusted so that someone with more training delivers more labor services per hour worked.

\(K\) is the quantity of capital services that flow from factories, equipment, trucks, aircraft etc.

\(A\) scales the output. Higher \(A\) corresponds to better technology. You increase \(A\), then the same capital and labor produce more real output.

\(\alpha\) can be viewed as letting us express the shape of the technological relationship between capital and labor and tune the optimal ratio of capital and labor. Given a K+L budget, there is a maximum Y isoquant that can be achieved whose slope matches the slope of the budget line which is the relative price. The optimal ratio is a function of technology (ie shape of the isoquants) and the relative price of capital and labor.

We express everything in constant prices. We model the technological relationship between real factor quantities and output, not payments. But converting them to constant dollars expresses them in the same units. We must use constant prices to remove the influence of general or relative price‑level changes.

The curves are isoquants showing combinations of capital and labor producing the same output level \(Y\).

Isoquant slopes diminish as you move along them from left to right. This downward convexity, or convexity with respect to the origin, reflects the complementarity of capital and labor. Convexity expresses diminishing returns. Adding an additional unit of one input alone becomes increasingly less effective.

Over time,

• Firms invest and accumulate capital, moving up the Y-axis to better isoquants.
• The labor force grows, moving right along the X-axis to better isoquants.
• And technology improves, moving isoquants up and to the right for the same amount of capital and labor.

If we visualize this 2d topographical map in 3D, the Cobb-Douglas function describes a rolling hill.

Consider building houses:

• Without power tools (capital), using tons of workers with manual tools is labor-inefficient in the US, where capital is fairly abundant and cheap, with low real interest rates. But capital intensity is lower in developing countries with abundant low-cost labor and also scarcer capital.

• Some kind of giant automated 3D printing machine or gundam robot to make a house might be capital-inefficient.

• An optimal mix of capital and labor, like workers equipped with power tools, results in the most house for the money.

• There is a natural area of optimal total factor productivity, as you stray from it, total factor productivity diminishes.

Or consider truck drivers. You can have bigger or smaller trucks within reason, you could have teams of 2 drivers in 12-hour daily shifts, but you have one driver at a time. You couldn’t have one driver and a single giant truck driving all the day’s shipments between two major cities. There’s a natural, reasonable ratio between capital and labor.

There are critiques of the Cobb-Douglas function. At the level of an individual firm, there are myriad decisions that go into optimizing output. The unit cost of capital depends on capacity utilization. There are learning curves. Individuals are not lumps of labor. Neither are capital investments like a $100m ASML lithography machine. We may have breakpoints and variability in the isoquants over time.

But at the level of the macro economy, these assumptions are reasonable simplifications. Macro output is a function of how much capital, labor and tech you have, and diminishing returns to capital and labor individually make sense, and constant returns to scale make sense.

The CES Production Function

The degree of convexity is fixed in the Cobb-Douglas function. The elasticity of substitution between capital and labor is always 1. This means that the slopes of the isoquants are equal to the ratios of capital and labor.

To tune the degree of convexity, we can use a more general function like the Constant Elasticity of Substitution (CES) function.

Now we have \(\alpha\) to tune the optimal ratio of labor and capital, and also \(\rho\) to tune the degree of convexity or complementarity between labor and capital.

Explaining the function of \(\rho\):

• When \(\rho\) = 0, we get the Cobb-Douglas function.
• When \(\rho\) is between 0 and 1, the isoquants are less convex (flatter) than Cobb-Douglas.
• When \(\rho\) < 0, the isoquants are more convex than Cobb-Douglas.
• When \(\rho\) = 1, the isoquants are linear, and labor and capital are perfect substitutes.
• When \(\rho\) < 1, we have convexity and diminishing returns as we add one unit of labor or capital.
• When \(\rho \rightarrow -\infty\), the isoquants are right angles, the ratio of capital and labor is fixed, capital and labor are perfect complements, additional capital or labor has 0 marginal product. (But we run into numerical precision problems when we make the slider range bigger)
• We always have constant returns to scale, doubling labor and capital doubles production.

The Singularity

If you have capital that is a perfect substitute for labor (like a robot), then the form of the production function is:

Try it, set \(\rho\) to 1 in the CES function. Isoquants are straight lines.

Think of this as the edge case where we can build a humanoid robot that anywhere and everywhere could work exactly like a human.

If the isoquants are flat, then in practice you really only have one factor of production since labor and capital are perfect substitutes. Labor can only demand a wage equal to the cost of renting the functionally identical robot.

If you have the convex unit-elastic Cobb-Douglas production function, and perfect competition in labor markets, leading to workers being paid their marginal product, then the labor share of national income will equal the capital share. In the idealized classical economics world, firms will keep hiring until the next worker they add does not increase production enough to offset their wages. And the slope of the isoquants defines how much additional labor is worth relative to capital, it determines the wage rate. And unit elasticity of substitution between labor and capital under Cobb-Douglas means that as wages go up and the labor demanded goes down, the wage increase offsets lower employment exactly, so labor income remains the same.

Under Cobb-Douglas, wages will always rise as technology improves productivity and the labor share of national income will stay constant. This has very roughly been the case in the past, contra thinkers like Marx. But possibly only with the help of unions and policy choices to help counter the threat of Communism. The important thing to note is that the Cobb-Douglas function forces income share equality between labor and capital factor incomes.

If you move to the more general CES function, math alone doesn’t force it to happen, you need parameters like e.g. \(\alpha=0.5, \rho=0\) to achieve unit elasticity of substitution (or match \(\rho\) to a different \(\alpha \neq 0.5\)). More convexity and complementarity means a higher labor share, less convexity means a lower labor share.

There is no theoretical reason \(\rho\) must be equal to 0. And in fact the labor share has been declining since around 1970, although it’s still above 50%.

As you move \(\rho\) toward 1, the labor share decreases.

Technology that is labor-assisting and highly complementary to labor sharpens the curves of the isoquants. A power tool increases the productivity of the individual who wields it. Or an AI assistant may increase the productivity of a radiologist analyzing images.

A technology that is labor-replacing flattens the curves of the isoquants. Tap-to-pay completely eliminates the jobs of subway token clerks. It perfectly substitutes for them.

More than most technologies in the past, AI is more human-like and probably more labor-replacing across a wider range of occupations.

All that is really required for AI to favor capital over labor is for AI to flatten the isoquants, to increase the elasticity of substitution, to be less complementary to human labor than existing technologies.

But it what if it collapses the diminishing returns tradeoff between labor and capital and makes them near-perfect substitutes?

A little Economic History

We’ve passed through a few industrial revolutions: 

• The first industrial revolution of steam and rail and telegraphs starting in the late 1700s.

• The second industrial revolution of oil, cars, electric power, starting in the late 1800s.

• A third industrial revolution of analog electronics, TV and radio, and then digital electronics, computers, the web, mobile/cloud, starting in the 1950s.

• Now maybe we can consider AI the fourth.

When we transitioned from horses and buggies to automobiles, we no longer needed buggy drivers and whip makers and stable hands, but we needed (much more productive) truck drivers and auto mechanics.

When we went to factories on assembly lines with power tools, we needed electricians and workers who could handle them.

Every buggy driver could probably get retrained as a truck driver and certainly a lot of them could get trained as mechanics and even engineers.

That being said, labor adjustment takes a long time, there are winners and losers, and some people’s livelihoods don’t recover in their lifetimes. And that’s assuming the best case that there is an eventual adjustment. Potentially the new jobs might totally different and out of reach for retraining.

I’m here to tell you that the pop economics that technology always eventually leads to universal standard of living progress, always raises productivity and therefore wages in tandem, is bunk. Theory deoesn’t say that. History kind of said that, but it’s been a lot less true the last 50 years or so in the US, as the capital share has risen.

Consider the horse. The horse was a very versatile source of transportation and power. It ran on hay and water and could navigate bad roads and hostile off-road environments. Steam and then internal combustion engines came along, with railroads and better roads everwhere, and ultimately replaced horses by being better and cheaper. This time around, we might be the horse.

Even in ideal competitive markets, the optimistic view depends on a degree of complementarity between labor and capital that may no longer exist in a world full of AI. And in the imperfect real world, the balance of market and political power between labor and capital has a significant impact. If labor and capital were perfect complements, square isoquants, fixed ratio of labor and capital, then the factor in a stronger bargaining position, scarcer, better organized, more politically powerful, would extract the lion’s share of national income.

Most jobs don’t require all of humans’ creativity and knowledge. And your brain is just a slow, biased pattern-matcher trained on questionable data. Just like AI, but worse. In many but not all respects currently, but in 20 years, who knows?

A robot boot, stamping on a human face, forever?

There are tasks where a good AI today is better than the average human under some time budget. Take radiology. Maybe a 90th-percentile radiologist is better than AI if you give them 30 seconds, and maybe the average doctor is better if you give them 2 minutes. (Made-up numbers, although there is literature on this).

But AI improves. Next year, the 90th percentile becomes the 95th, and the average doctor needs 15 minutes to match the AI or can’t beat the AI no matter how long you give them.

Every year AI becomes better than a larger percentage of humans, at a larger percentage of tasks. AI scientists have learned the bitter lesson, which is that general methods and lots of data and compute are all it takes to beat humans at most tasks. GPU go brrr. Clever algorithms designed to mimic how humans solve problems generally just make things worse. Humans currently have an edge in adaptability, in fast learning without as much data and compute, in critical thinking. But we may not be special snowflakes who will always be a step or two ahead of the machines. In 20 years, who knows.

Now, radiology is a high-stakes, high skill use case. Typically in these use cases in 2025:

If a radiologist takes 10 minutes to review a case, they might continue to do that with an AI assistant as part of a structured workflow. First the radiologist reviews and highlights areas of concern, then the AI gives a probability of diagnosis on those areas and highlights other areas. AI can even bring up relevant literature and the doctor can ask for detailed comparisons with reference scans. AI writes the first draft of report based on doctor’s instructions, and the doctor edits.

In this way the human input remains similar but the output could be significantly higher quality with the help of AI. (Assuming the market and insurance companies don’t demand more automation and less human input.)

Something like medical care is a superior good, people consume more as their income goes up. It might be an income-proportional good, you might even demand the same amount in nominal terms if the price goes down. AI automates the work but there is a lot of demand for better work.

But in many occupations, we are not going to raise the quality bar. If you can have a long-distance truck convoy of 10 trucks supervised by 1 or 2 humans, or 10 robot hotel housekeepers supervised by one human, you get rid of the extra humans.

We can imagine a factory where the robots can maintain and repair each other. You still need humans to design factories and robots, write and debug and improve software. For now, anyway.

The level of skill that a human needs, in order to be able to do something better than AI, can increase beyond what the vast majority of humans can do, for the vast majority of the daily tasks needed to maintain civilization. You are only going to hire the human for a task and at a wage where they are cheaper and less trouble than a robot, which might not leave many jobs or much wages at all. There is always going to be some job, but it might be as an influencer (digital tout), or livestreaming yourself as a freak doing a gross-out act on ‘Dum Dummies’ from Black Mirror and it might not pay much.

Maybe there is a modest demand response, society gets richer, there is more demand for hotels and goods transport. But you don’t automatically 10X demand when you 10X productivity, and it’s hard for the housekeepers to retrain into jobs AI can’t do.

It is not at all a given that when technology destroys some jobs, it creates other, higher-paying jobs. You need to parameterize the equations to make that happen. I would argue that there is a good chance that AI changes the parameters dramatically in the other direction.

Potentially we can end up with robots building more robots, and an economic singularity where capital grows exponentially and wealth concentrates completely, while humans are immiserated.

If you combine AI with globalization, neoliberalism, loss of organized labor power etc., it upends the economic assumptions that underlie past wage growth.

Maybe we aren’t quite there yet. Maybe everybody will just get better at their jobs, and even midwits and less educated people can do software development and analytical jobs with AI. And we can shorten workweeks. But what would be the incentive to shorten workweeks in the absence of labor laws requiring it?

As a thought experiment, what is the difference between the state owning all the capital and being run by a politburo or junta, and a tiny number of individuals owning it, and writing all the rules of an economy and culture and marketplace of political ideas fully mediated by technology? To the extent capitalism works for most people, it requires competitive markets and not unlimited concentration of capital owned by a few. And maybe serfdom and extraction for the rest.

That is what the robot singularity looks like in economic terms. That is the economic challenge we will eventually face as a civilization, along with questions of universal income, robot taxes, which citizens reap the benefits of AI, what happens to people and countries outside that circle, what happens to economic growth, freedom, fairness, civil liberties, democratic governance, human values and human purpose.

We are ill-equipped to confront these questions. Even the places people think about them, the press, the universities, are under siege.

There is a risk of the fall of Rome, technofeudalism, barbarism. Maybe not next year, maybe not in 10 years, but it might be where we are headed.

At least, not knowing exactly how the tech and the equations evolve, we should probably be mindful of that possibility.

In the final analysis, everything is endogenous. AI changes human sytems a lot, and it seems impossible to say how. It would behoove us to be a little humble, and cautious, and not rely purely on historical parallels where maybe there aren’t any. And try to put in some guardrails to avoid the worst possible outcomes.

Further reading:

Rise of the Robots: Technology and the Threat of a Jobless Future, by Martin Ford

The Economic Singularity: Artificial intelligence and the death of capitalism, by Calum Chace

Displacement or Complementarity? The Labor Market Impact of Generative AI, by Wilbur Xinyuan Chen, Suraj Srinivasan, Saleh Zakerinia

The Simple Macroeconomics of AI, by David Autor

Automation and New Tasks: How Technology Displaces and Reinstates Labor, by Daron Acemoglu and Pascual Restrepo

Fed Governor Michael Barr: Artificial Intelligence and the Labor Market: A Scenario-Based Approach

Simon Willison has a great post on everything we learned about AI in 2024 (somewhat technical). Inspired by him, here is a roundup of the top events of 2024 in AI and where we are now.

1. Everybody caught up to OpenAI.

• The top tier is OpenAI, Google, and Anthropic. They are very close, for most purposes the difference is insignificant, for some use cases one is better than the other.

• With Gemini 2.0, Google seems to be the current leader by a modest amount on overall base model quality. Gemini is better enough, and cheaper enough that I would switch to it for some jobs.

  • Flash is very good, very fast, and very cheap

  • Long context

  • Great native multimodality (although OpenAI has caught up on vision/audio/video)

  • Deep research, thinking models, NotebookLM state-of-the-art text-to-speech.

• OpenAI is still ahead on product, things like structured outputs, o1, overall ecosystem. First-mover advantage is real, ChatGPT is synonymous with advanced AI in the public’s mind for now. Google now supports the OpenAI API semantics, which attests to OpenAI being the leader and Google the fast follower.

• Claude is very strong on technical discussions and coding. Definitely try it for deep conversations about code or research papers. They pioneered the Canvas, are doing interesting stuff on API discovery. (Punching below their weight in Chatbot Arena)

• Use Chatbot Arena and artificialanalysis.ai to compare models for different use cases.

2. The rise of small and open-source models nipping at the leaders’ heels.

• The thing that shocked me most in 2025 was Gemma2-9b-it-SIMPO. Gemma is Google’s excellent open source model. A group at Princeton fine-tuned it to score higher than early GPT-4, and shrank it to run adequately fast on a MacBook. Bananas.

• There are a slew of strong dark horses: Llama, Grok, Nova in the US; Qwen, Yi, DeepSeek from China, Mistral in Europe.

• The gap between the frontier models and the flash/mini or smaller open-source models shrank.

3. The scaling laws hit the wall.

• Gemini 2.0 is a significant improvement over 1.0 but not as big as GPT 3.5 to GPT-4

• OpenAI failed to deliver a foundation model that was a worthy successor to GPT-4 despite 2 gigantic training runs, according to reports. Anthropic also failed to update Opus, their largest model, leaving Sonnet as their flagship model.

• These top 3 bullet points reflect the commoditization of AI, no one has a monopoly, the price of pretty good AI has dropped by several orders of magnitude.

4. OpenAI pivoted to inference-time compute with o1 and o3.

• Faced with GPT-5 hitting the wall, OpenAI pivoted to models like o1 that think, run multiple LLM calls, use search algos to look for the best way to answer a question via a multi-step process.

• Probably the 2nd most shocking thing I saw in 2024 was this thread by math professor and Fields medalist Terence Tao, saying that o1 can do math at the level of a strong math major. I thought claims about it solving problems from the Putnam test (the leading US undergrad math competition) were probably overblown or training data contamination, but even if you fuzz them it can still solve a few of them at least. Bananas.

• o1 is very good but very expensive.

• o3 was announced, no one has seen it but OpenAI has hyped it for release in about a month, saying it is a superstar programmer and beat an AGI test. There is so much hype and so much pressure to beat Google that I am skeptical. I expect a notable improvement over o1. But with Google already releasing ‘thinking’ models, it’s not clear how big a lead OpenAI has. I don’t quite know what to make of the AGI claims, either it’s a bit overhyped or they found something important. To date LLMs are great with language but understanding is skin-deep, they don’t model complex abstractions deeply. There is so much pressure to be best and first that capabilities are overhyped and corners are cut. But I have been surprised before.

• Interesting consequences in the chip world. Possibly tired: 100,000-GPU Nvidia training clusters. Wired: Fast inference chips like d-Matrix, Groq, Cerebras, and a slew of others.

5. Everything is agents now.

• To me the definition of agent is when the LLM makes control flow decisions, whether with tool choice in a directed acyclic graph decision tree, or a full-blown autonomous loop. And every AI product does that now; when you use the chatbot it looks at your query and routes it to different models and tools. Devin-style full-service software developers are still a bit pie-in-the-sky but everything short of that, from low-code automation up to superintelligent assistants, is very much in play.

7. The Advanced Voice Mode with vision and other crazy sci-fi stuff.

• Point your phone at anything and have a conversation about it. Fast conversational multimodal FTW. There was a huge leap in video generation with Sora, Kling, Runway. Also associated tech like NotebookLM, Suno. We are not that far from a fully synthetic, realistic virtual or robotic avatar to interact with. (Apols for the NSFWish projection but I think OnlyFans creators might be the first victims of robots, before even truck drivers)

8. Self-driving cars achieved product-market fit, with one player left standing: Google’s Waymo.

• And possibly a European and Chinese competitor. Tesla is not really in the game yet, no robotaxis on the road yet even with safety drivers, maybe that will change and maybe it won’t. Waymo has a big lead.

• IMHO we will see this type of highly autonomous robot, static or on wheels, for 5 or maybe 10 years before we see a lot of humanoid robots. And ofc fully virtual robots as discussed above. Power, cost issues, normie workplaces can’t keep copiers and printers working.

• As an aside, the meme of AI killing Google is overblown, more likely Google integrates AI in search and laps the competition, cures cancer with AI and leads the AI revolution, even as search declines a bit in favor of chatbots. As long as the thing we interact with is a text box, Google should do OK. When AI becomes ubiquitous and phones and homes and smart glasses are environments we talk to that watch us and interact with us, that’s a bigger bridge to cross.

9. Safety took a back seat, and politics took center stage.

• US-China trade war, EU regs, California regs, major competition authority actions against all the Big Techs. With Musk in charge of tech policy in US, safe AI is so 2023, takes a back seat to accelerationism. But brace for the backlash, possibly the Butlerian Jihad (via Christopher Mims). Everything is mediated through big tech, and people view AI with suspicion as IP theft, surveillance state, manipulation and dark patterns. People look for scapegoats.

9. Dead internet and AI slop everywhere.

• Stack Overflow is in freefall. Most questions AI answers better, just because it reads the documentation carefully. Same for big social media like X and Facebook. Anything big enough gets overrun by slop and adverse selection. Back to artisanal social media like Discords, and painstakingly building your own small social and professional circle.

10. 2025 and beyond?

• Smart glasses, pervasive AI

• Social issues around IP theft, surveillance, deepfake attacks. We need a new social contract around AI and it’s hard.

• Hyper-personalization.

• Medical advances with AI: diagnostics, clinical decision support, AI drug discovery, surgical robots, wearables, personalized medicine, administrative productivity AI.

• Eventually a financial crash because markets are insane even as AI powers everything.

• I think AGI and super-intelligence are actually pretty close. Take what we got and add better knowledge representation and world models, the ability to learn on the fly instead of moonshot training runs, and you get something like AGI. On a scale of a few years or decades, with the scale of brainpower and compute currently deployed, surely that is imminent. On a lot of benchmarks and tasks like say detecting tuberculosis on a scan, the AI is much better than most professionals on a tight time budget like 15 seconds. And the best professionals will be much better than the AI on a higher time budget like 5 minutes. So there is some time crossover where the humans start to beat AI. And over time probably the AI will beat more humans at more jobs at any given time limit, and the crossover where humans outperform the AI will shift to higher time limits. AI assistants will infiltrate more and more knowledge work. Use AI for what it’s good at, processing lots of language, use tools for what they are good at, looking up knowledge and running predictable tasks, and use humans for judgment and critical thinking. And over time the human component will shrink.

And the social upheaval will be unmanageable unfortunately. The tech bros think they will solve all of society’s problems if you just give them a few trillion. I think everything is breaking and tech is one big reason why, it’s the cause of as many problems as it solves and these Elon Musk and Sam Bankman-Fried types are mountebanks, or buffoons, or blinded by greed, or in some cases just want to burn everything down. Mustafa Suleyman is correct that we need to spend a significant fraction of AI efforts on guardrails to avoid disasters, and preparing people and society for the impacts.

Every few years, the bandit, the revolutionary and the priest trade hats. And the tech folks have gone from revolutionaries to bandits. A guy I used to think highly of in tech, said that the one big thing in 2024 was Silicon Valley’s hostile takeover of the federal government, via an infiltration of Donald Trump’s MAGA movement, and Trump’s subsequent thrashing of Joe Biden’s “establishment” government in the November election. The thing is, if you are posting groyper memes, changing your name to Kekius Maximus, and pimping the AfD, who is infiltrating who? The word of the day is that Muslims, Mexicans, immigrants, whoever you don’t like, are not human, and democracy is for losers, and that does not end well. Never forget that Neville Chamberlain was a ‘realist’ conservative. These people are going to sell out democracy, sell out to Putin, and blow up the world, literally.

I had AI take a crack at the first version of this, Gemini 1.5 with Deep Research. Did not use AI at all after reading that output, blame me for any typos and bad writing.