Three Things, Tested Three Ways

The confusion comes from treating “the LLM feature” as one indivisible thing. It is three:

The first row is 90% of your code and needs no model at all. Get that boundary right and the rest is manageable.

Put an Interface in Front of the Model

Do not scatter client.Messages.New through your handlers. Define the narrowest interface your code actually needs and depend on that:

// Completer turns a prompt into text. It is deliberately narrow — narrower
// than the SDK — so tests can implement it in five lines.
type Completer interface {
    Complete(ctx context.Context, req Request) (Response, error)
}

type Request struct {
    System   string
    Messages []Message
    Tools    []Tool
}

type Response struct {
    Text       string
    ToolCalls  []ToolCall
    StopReason string
    Usage      Usage
}

This is just the dependency-inversion habit that makes any external service testable, and it earns its keep faster here than almost anywhere else. Two things fall out of it:

Your business logic never imports the SDK. It imports your Request and Response types. When the SDK’s union types change shape, one adapter file changes.

The fake is trivial. No HTTP, no fixtures, no mocking library:

type fakeCompleter struct {
    responses []Response
    err       error
    calls     []Request // recorded for assertions
}

func (f *fakeCompleter) Complete(_ context.Context, req Request) (Response, error) {
    f.calls = append(f.calls, req)
    if f.err != nil {
        return Response{}, f.err
    }
    if len(f.responses) == 0 {
        return Response{}, errors.New("fakeCompleter: no responses left")
    }
    resp := f.responses[0]
    f.responses = f.responses[1:]
    return resp, nil
}

Returning a queue rather than a single value is what lets you test multi-turn loops: the first call returns a tool request, the second returns the final answer.

Test the Loop, Not the Model

An agent loop has plenty of logic worth testing, none of which needs a model. Does it stop when it should? Does it return a result for every tool call? Does it hand a tool failure back rather than aborting?

func TestLoopReturnsResultForFailedTool(t *testing.T) {
    fake := &fakeCompleter{responses: []Response{
        {StopReason: "tool_use", ToolCalls: []ToolCall{
            {ID: "t1", Name: "lookup", Input: []byte(`{"id":"missing"}`)},
        }},
        {StopReason: "end_turn", Text: "I couldn't find that record."},
    }}

    agent := New(fake, map[string]ToolFunc{
        "lookup": func(context.Context, []byte) (string, error) {
            return "", errors.New("not found")
        },
    })

    got, err := agent.Run(context.Background(), "look up record missing")
    if err != nil {
        t.Fatalf("Run() error = %v, want nil", err)
    }
    if got != "I couldn't find that record." {
        t.Errorf("got %q", got)
    }

    // The failure must reach the model as a tool result, not abort the loop.
    second := fake.calls[1]
    result := findToolResult(t, second, "t1")
    if !result.IsError {
        t.Error("tool failure was not marked as an error result")
    }
}

That test catches a real bug — a loop that drops the failed call instead of reporting it — and runs in microseconds with no API key.

The same approach covers the rest of the loop’s contract — the invariants from the agent-loop article: that it stops at MaxIterations, that every tool_use block gets exactly one result, that results come back in one message. Table-driven tests fit this beautifully, since each case is just a different queue of canned responses.

Golden Files for Prompt Assembly

Prompt building is string manipulation, and it drifts. Someone adds a field, reorders a section, changes a heading — and unlike code, a prompt regression produces no compile error and no failing assertion, just slightly worse output that nobody attributes to the change.

Golden files make the diff visible in review:

var update = flag.Bool("update", false, "update golden files")

func TestBuildSystemPrompt(t *testing.T) {
    tests := []struct {
        name   string
        cfg    Config
        golden string
    }{
        {"default", Config{}, "system_default.txt"},
        {"with_tools", Config{Tools: allTools}, "system_with_tools.txt"},
        {"terse_mode", Config{Terse: true}, "system_terse.txt"},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got := BuildSystemPrompt(tt.cfg)
            path := filepath.Join("testdata", tt.golden)

            if *update {
                if err := os.WriteFile(path, []byte(got), 0o644); err != nil {
                    t.Fatal(err)
                }
                return
            }

            want, err := os.ReadFile(path)
            if err != nil {
                t.Fatal(err)
            }
            if diff := cmp.Diff(string(want), got); diff != "" {
                t.Errorf("prompt changed (-want +got):\n%s", diff)
            }
        })
    }
}

Run go test ./... -update to accept a change deliberately. The point is not that the golden file is correct — it is that changing it requires saying so out loud, in a diff a reviewer can read.

This also catches the caching bugs from the prompt caching article before they cost you anything. A golden test on the system prompt fails the moment somebody interpolates time.Now() into it, because the output differs on every run.

Which suggests a second, blunter test worth having:

func TestPromptAssemblyIsDeterministic(t *testing.T) {
    // Go randomises map iteration order, so a prompt built by ranging over a
    // map differs run to run — and silently never caches.
    first := BuildSystemPrompt(cfg)
    for i := 0; i < 20; i++ {
        if got := BuildSystemPrompt(cfg); got != first {
            t.Fatalf("prompt is not deterministic on run %d", i)
        }
    }
}

Twenty iterations is enough to make a randomised map order fail essentially every time.

Integration Tests, Behind a Build Tag

You do need a handful of tests that touch the real API — enough to catch an SDK upgrade that changed a union type, a model id that no longer resolves, or streaming that broke. But they cost money and need a key, so they must not run on every go test ./...:

//go:build integration

package llm_test

func TestStreamingReturnsCompleteMessage(t *testing.T) {
    if os.Getenv("ANTHROPIC_API_KEY") == "" {
        t.Skip("no API key; skipping integration test")
    }
    // ... real call, assert on shape rather than content
}

go test ./...                    # fast, free, no key
go test -tags=integration ./...  # the real thing, on demand

Assert on shape, never on wording. resp.Text being non-empty, StopReason being end_turn, Usage.OutputTokens being greater than zero, a streamed message accumulating to the same content as a non-streamed one. Those hold across model versions. “The answer contains the word Paris” does not, and a flaky test that fails once a month teaches your team to ignore failures.

There is a middle option that gets you a long way for free: point the SDK at a local httptest.Server via the ANTHROPIC_BASE_URL environment variable and serve canned JSON. That exercises the real SDK — its parsing, its retry behaviour, its streaming decoder — without a key or a bill. It is the closest analogue to what go-sqlmock does for the database layer: a real driver, a fake server.

It is also the only sane way to test the paths you cannot easily provoke on purpose:

// Rate limiting: does the caller back off, or hammer?
mux.HandleFunc("/v1/messages", func(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Retry-After", "1")
    w.WriteHeader(http.StatusTooManyRequests)
    fmt.Fprint(w, `{"type":"error","error":{"type":"rate_limit_error","message":"..."}}`)
})

Do the same for a 500, a connection dropped mid-stream, and a malformed body. Those are the failures that actually page you, and they are trivial to simulate and nearly impossible to trigger on demand against the real API.

The Refusal Case Nobody Tests

Worth its own paragraph because it is so easy to miss: a safety refusal comes back as HTTP 200 with StopReason set to refusal. Your error handling never fires. Code that goes from if err != nil straight to reading the first content block treats it as an empty answer.

func TestRefusalIsNotTreatedAsEmptyAnswer(t *testing.T) {
    fake := &fakeCompleter{responses: []Response{
        {StopReason: "refusal", Text: ""},
    }}

    _, err := New(fake, nil).Run(context.Background(), "...")
    if !errors.Is(err, ErrDeclined) {
        t.Errorf("got %v, want ErrDeclined", err)
    }
}

One line in the fake, and you have covered a branch most production code does not have at all. The sentinel-error pattern behind ErrDeclined is the one from error handling in Go.

Evals: For Quality, Not Correctness

Everything above tests whether your code is right. None of it tells you whether the answers are any good. That needs a different instrument, and the mistake is trying to force it into go test.

An eval is a fixed set of inputs, run against the real model, scored rather than asserted:

//go:build eval

func TestClassificationAccuracy(t *testing.T) {
    cases := loadEvalCases(t, "testdata/eval/classification.jsonl")

    var correct int
    for _, c := range cases {
        got, err := classify(context.Background(), realClient, c.Input)
        if err != nil {
            t.Fatal(err)
        }
        if got == c.Want {
            correct++
        } else {
            t.Logf("MISS: input=%q got=%q want=%q", c.Input, got, c.Want)
        }
    }

    accuracy := float64(correct) / float64(len(cases))
    t.Logf("accuracy: %.1f%% (%d/%d)", accuracy*100, correct, len(cases))

    // A floor, not an equality check. Below this, something regressed.
    if accuracy < 0.90 {
        t.Errorf("accuracy %.1f%% below the 90%% floor", accuracy*100)
    }
}

Two things make this useful rather than annoying. The threshold is a floor, not a target — you are detecting regression, not demanding perfection. And the misses get logged, because the list of what it got wrong is the actual output; the pass/fail is almost incidental.

Run evals when you change a prompt, a model, or an effort setting — not on every commit. They cost money and take minutes.

What Not to Do

Do not assert on model wording. strings.Contains(resp, "Paris") passes today and fails after a model update that phrases it differently. It is not testing your code.

Do not set temperature to zero and call it deterministic. Sampling parameters are not even accepted on current models, and identical output was never guaranteed regardless.

Do not mock the SDK’s types. Mocking anthropic.Message and its union blocks is a lot of work to test the adapter you wrote to avoid exactly that. Fake your own interface instead.

Do not let integration tests run by default. A test suite that needs an API key is a test suite that new contributors cannot run, and CI cost that grows with every push.

Do not skip testing the error paths because they are “just the SDK’s job”. Rate limits, refusals and mid-stream disconnects are the failures you will actually see in production, and they are the cheapest things in this article to cover.

Checklist

• A narrow interface between your logic and the SDK; business code never imports the SDK.
• A fake with a queue of canned responses for multi-turn loops.
• Unit tests for the loop’s contract: one result per tool call, errors handed back, iteration cap honoured.
• Golden files for prompt assembly, plus a determinism test.
• Integration tests behind //go:build integration, asserting on shape not wording.
• An httptest.Server for 429s, 500s and truncated streams.
• A test for the refusal path.
• Evals behind their own tag, scored against a floor, misses logged.

Conclusion

“You cannot test LLM code” conflates the model with the code around it. The code around it — the prompt builder, the parser, the loop, the tool handlers, the error branches — is ordinary Go, and it becomes easy to test the moment there is an interface between it and the SDK. Push non-determinism out to the edges, cover the edges with evals scored against a floor, and the rest of your suite stays fast, free and green.

Getting a Client

go get github.com/anthropics/anthropic-sdk-go

import (
    "github.com/anthropics/anthropic-sdk-go"
    "github.com/anthropics/anthropic-sdk-go/option"
)

// Reads ANTHROPIC_API_KEY from the environment.
client := anthropic.NewClient()

// Or pass it explicitly, if you load config yourself.
client := anthropic.NewClient(option.WithAPIKey(key))

Build the client once, at startup, and pass it around. It is safe for concurrent use and holds a connection pool — constructing one per request throws away keep-alive and gives you a fresh TLS handshake every time.

The First Call

resp, err := client.Messages.New(ctx, anthropic.MessageNewParams{
    Model:     "claude-opus-5",
    MaxTokens: 16000,
    Messages: []anthropic.MessageParam{
        anthropic.NewUserMessage(anthropic.NewTextBlock("Summarise this changelog in three bullets.")),
    },
})
if err != nil {
    return fmt.Errorf("summarise changelog: %w", err)
}

Two things about that snippet are worth slowing down on.

The model is a plain string. The SDK ships typed constants (anthropic.ModelClaudeOpus4_8 and friends), but anthropic.Model is an alias for string, so a model that has no constant yet is passed as its id. Either form compiles; check the SDK release notes before assuming a constant exists for the model you want.

MaxTokens is a ceiling, not a target. It is the point at which generation is cut off mid-sentence, and the model is not told about it. Setting it to 500 because you want a short answer does not produce a short answer — it produces a truncated one. Ask for brevity in the prompt and leave the ceiling generous. For non-streaming requests, something around 16000 keeps you clear of both truncation and the SDK’s HTTP timeout.

Content Is a List of Blocks, Not a String

This is where the first hour usually goes. A response is not resp.Text. It is resp.Content, a slice of union values that can hold text, thinking, tool calls and more:

for _, block := range resp.Content {
    switch variant := block.AsAny().(type) {
    case anthropic.TextBlock:
        fmt.Println(variant.Text)
    case anthropic.ThinkingBlock:
        // Reasoning, when you have asked for it to be shown.
        log.Debug("model reasoning", "text", variant.Thinking)
    }
}

block.AsAny() is the accessor that gets you a concrete type to switch on. Reaching for resp.Content[0].Text and hoping works right up until the day a thinking block or a tool call lands in position zero, at which point you silently return an empty string. Write the type switch once, in a helper, and use it everywhere:

// firstText returns the first text block in a response, or "" if there is none.
func firstText(msg *anthropic.Message) string {
    for _, block := range msg.Content {
        if t, ok := block.AsAny().(anthropic.TextBlock); ok {
            return t.Text
        }
    }
    return ""
}

Thinking, and Why You Probably Want It On

Current Claude models can reason before answering. The recommended mode is adaptive — you do not budget tokens for it, the model decides how much thinking a given request deserves:

adaptive := anthropic.ThinkingConfigAdaptiveParam{}

params := anthropic.MessageNewParams{
    Model:     "claude-opus-5",
    MaxTokens: 16000,
    Thinking:  anthropic.ThinkingConfigParamUnion{OfAdaptive: &adaptive},
    Messages:  messages,
}

There is no ThinkingConfigParamOfAdaptive helper — you construct the union literal and take the address of the variant, as above. That trips people up because almost every other option in the SDK does have a constructor function.

A word of warning if you are carrying settings over from older code: the fixed thinking budget (ThinkingConfigParamOfEnabled(N)) is gone on current models and returns a 400 rather than being ignored. If you want to spend less, the lever is effort, not a token budget — and effort lives inside output_config, not at the top level of the request.

The counter-intuitive part: on the newest models, turning thinking off is not reliably a cost saving. Lower effort with thinking on generally beats thinking off, and disabling it has failure modes of its own. Leave it on and turn effort down.

Streaming

Anything a user waits for should stream. It is not just perceived speed — a long non-streaming request is also the easiest way to hit an HTTP timeout.

stream := client.Messages.NewStreaming(ctx, anthropic.MessageNewParams{
    Model:     "claude-opus-5",
    MaxTokens: 64000,
    Messages:  messages,
})

for stream.Next() {
    event := stream.Current()
    switch ev := event.AsAny().(type) {
    case anthropic.ContentBlockDeltaEvent:
        switch delta := ev.Delta.AsAny().(type) {
        case anthropic.TextDelta:
            fmt.Print(delta.Text)
        }
    }
}
if err := stream.Err(); err != nil {
    return fmt.Errorf("stream response: %w", err)
}

Two nested type switches is not the prettiest Go you will write, but the shape is stable: outer switch on the event, inner switch on the delta.

Always check stream.Err(). stream.Next() returning false means “no more events” — it does not tell you whether that was a clean finish or a dropped connection. A loop that ignores Err() will happily serve a truncated answer as if it were complete.

If you want the whole message and the incremental deltas, accumulate as you go. There is no GetFinalMessage() on the Go stream:

stream := client.Messages.NewStreaming(ctx, params)

var message anthropic.Message
for stream.Next() {
    message.Accumulate(stream.Current())
    // ... also forward the delta to the user here
}
if err := stream.Err(); err != nil {
    return err
}
// message.Content is now the complete response.

Raising MaxTokens to 64000 in the streaming example is deliberate. Timeouts stop being the binding constraint once you stream, so you can give the model room.

Streaming to a Browser

Server-sent events are the path of least resistance, and Go’s http.ResponseController makes the flushing straightforward:

func (h *Handler) stream(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/event-stream")
    w.Header().Set("Cache-Control", "no-cache")
    w.Header().Set("X-Accel-Buffering", "no") // stop nginx buffering the stream

    rc := http.NewResponseController(w)

    stream := h.client.Messages.NewStreaming(r.Context(), params)
    for stream.Next() {
        ev, ok := stream.Current().AsAny().(anthropic.ContentBlockDeltaEvent)
        if !ok {
            continue
        }
        delta, ok := ev.Delta.AsAny().(anthropic.TextDelta)
        if !ok {
            continue
        }

        // SSE data frames must not contain raw newlines.
        payload, _ := json.Marshal(delta.Text)
        if _, err := fmt.Fprintf(w, "data: %s\n\n", payload); err != nil {
            return // client hung up
        }
        if err := rc.Flush(); err != nil {
            return
        }
    }

    if err := stream.Err(); err != nil {
        slog.Error("llm stream failed", "err", err)
        fmt.Fprint(w, "event: error\ndata: {}\n\n")
        _ = rc.Flush()
        return
    }
    fmt.Fprint(w, "event: done\ndata: {}\n\n")
    _ = rc.Flush()
}

Three details that cost me an afternoon each:

• X-Accel-Buffering: no. Without it nginx buffers your stream and delivers the whole thing at the end, which looks exactly like streaming being broken. (Nginx has strong opinions about proxied responses generally — I ran into a related set of them in making Nginx cache cookie aware.)
• JSON-encode the delta. A model can and will emit a newline mid-sentence, and a bare newline terminates an SSE frame.
• Pass r.Context(), not context.Background(). When the user closes the tab, the request context cancels, the SDK aborts the HTTP call, and you stop paying for tokens nobody will read.

Deadlines and Cancellation

Context is not decoration here. It is the only thing standing between a slow model call and a goroutine that lives forever:

ctx, cancel := context.WithTimeout(r.Context(), 90*time.Second)
defer cancel()

resp, err := client.Messages.New(ctx, params)

Set the deadline against how long the work should take, not a habit. A classification call has no business taking 90 seconds; a long agentic turn on a hard problem might legitimately run for several minutes. If you have not internalised how deadlines propagate through a call chain, understanding Golang context covers the machinery.

The corollary at shutdown: an in-flight model call is exactly the kind of long request that a naive SIGTERM handler will sever. Drain it properly — see graceful shutdown in Go web services.

Errors Worth Distinguishing

The SDK returns typed errors. Use errors.As to get at the status code rather than matching on message strings:

resp, err := client.Messages.New(ctx, params)
if err != nil {
    var apiErr *anthropic.Error
    if errors.As(err, &apiErr) {
        switch apiErr.StatusCode {
        case http.StatusTooManyRequests:
            return fmt.Errorf("rate limited: %w", ErrRetryable)
        case http.StatusBadRequest:
            // Your request is malformed. Retrying will not help.
            return fmt.Errorf("bad request to model: %w", err)
        default:
            return fmt.Errorf("model call failed (%d): %w", apiErr.StatusCode, err)
        }
    }
    // Not an API error: context cancellation, DNS, TLS, connection reset.
    return fmt.Errorf("model call failed: %w", err)
}

Wrapping with %w at each layer is what lets the HTTP boundary decide the status code without every layer needing to know about HTTP — the same pattern I laid out in error handling in Go.

The SDK already retries for you. By default it retries a couple of times on 408, 409, 429, 5xx and connection errors, with backoff. Two consequences people miss:

1. Do not add your own retry loop on top without lowering the SDK’s. Three of yours around two of its is nine attempts and a long tail of latency.
2. Wall-clock can reach timeout × (attempts + 1). Your context deadline is the real budget — set it deliberately, because the retry behaviour will happily use all of it.

A Refusal Is Not an Error

This one surprises people. If a safety classifier declines a request, you get HTTP 200 — a perfectly successful response whose StopReason says the model declined:

if resp.StopReason == anthropic.StopReasonRefusal {
    slog.Warn("model declined",
        "category", resp.StopDetails.Category,
        "explanation", resp.StopDetails.Explanation)
    return "", ErrDeclined
}

err is nil. resp.Content may hold nothing useful. Code that goes straight from if err != nil to reading Content[0] treats this as an empty answer and moves on. Check StopReason before you read content — and note the other values you care about: max_tokens means you were truncated, and tool_use means the model is waiting on you (which is a whole article of its own).

Watch the Usage Numbers

Every response carries a token accounting:

slog.Info("model call",
    "input_tokens", resp.Usage.InputTokens,
    "output_tokens", resp.Usage.OutputTokens,
    "cache_read", resp.Usage.CacheReadInputTokens,
    "cache_write", resp.Usage.CacheCreationInputTokens,
)

Log these from day one. They are the only ground truth about what a feature costs, and CacheReadInputTokens in particular is how you find out that your prompt caching silently stopped working three deploys ago — the failure mode there is not an error, just a bigger invoice. That is its own post, because it is the single biggest lever on what an LLM feature costs to run.

A structured logger pays for itself here: these are five numeric fields per call that you will want to aggregate later, which is exactly the case for structured logging with log/slog.

Checklist

• One client, built at startup, shared across handlers.
• Iterate resp.Content with a type switch; never index blindly into it.
• Adaptive thinking on; tune cost with effort, not by disabling it.
• Stream anything a human waits for, and always check stream.Err().
• r.Context() all the way down, with a deliberate deadline.
• errors.As into *anthropic.Error for status-code branching.
• Do not stack your retries on the SDK’s.
• Check StopReason before reading content — a refusal returns 200.
• Log the usage fields from the first commit.

Conclusion

The API surface is small — one endpoint, one loop, a handful of block types. What makes an LLM call different from any other HTTP call in your service is that it is slow, occasionally non-deterministic, priced per token, and able to succeed while declining to do what you asked. Go gives you good tools for exactly those problems, provided you use the context properly and read the response as the structured thing it is rather than the string you wish it were.

The Problem With the Obvious Version

// Do not ship this.
func fetchAll(urls []string) {
    var wg sync.WaitGroup
    for _, url := range urls {
        wg.Add(1)
        go func() {
            defer wg.Done()
            fetch(url)
        }()
    }
    wg.Wait()
}

Three separate problems:

1. No limit. len(urls) concurrent requests. The remote service rate-limits you, or your file descriptors run out, or both.
2. No errors. fetch returns one and it goes nowhere.
3. No cancellation. If the caller gives up, every goroutine keeps running to completion.

The concurrency itself is not the mistake — the missing back pressure is.

The Classic Channel Pool

The traditional shape is a jobs channel, a fixed number of workers reading from it, and a results channel:

type job struct {
    ID  int
    URL string
}

type result struct {
    JobID int
    Body  []byte
    Err   error
}

func workerPool(ctx context.Context, jobs []job, workers int) []result {
    jobCh := make(chan job)
    resCh := make(chan result, len(jobs))

    var wg sync.WaitGroup
    for i := 0; i < workers; i++ {
        wg.Add(1)
        go func(workerID int) {
            defer wg.Done()
            for j := range jobCh {
                body, err := fetch(ctx, j.URL)
                select {
                case resCh <- result{JobID: j.ID, Body: body, Err: err}:
                case <-ctx.Done():
                    return
                }
            }
        }(i)
    }

    // Feed the workers, stopping early if the caller cancels.
    go func() {
        defer close(jobCh)
        for _, j := range jobs {
            select {
            case jobCh <- j:
            case <-ctx.Done():
                return
            }
        }
    }()

    wg.Wait()
    close(resCh)

    out := make([]result, 0, len(jobs))
    for r := range resCh {
        out = append(out, r)
    }
    return out
}

This is worth understanding because you will read it in a lot of codebases, and because it shows the mechanics plainly. Note two things that are easy to get wrong:

• close(jobCh) is the workers’ exit signal. for j := range jobCh ends when the channel closes. Forget the close and wg.Wait() blocks forever.
• Every channel send is paired with <-ctx.Done(). Without that, a worker sending to a full resCh that nobody is reading leaks for the lifetime of the process.

It is also about forty lines to do something the standard extended library does in eight.

The errgroup Version

golang.org/x/sync/errgroup is a sync.WaitGroup that also collects the first error and cancels its siblings. SetLimit turns it into a bounded pool:

import "golang.org/x/sync/errgroup"

func fetchAll(ctx context.Context, urls []string) ([][]byte, error) {
    bodies := make([][]byte, len(urls))

    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(10) // at most 10 in flight

    for i, url := range urls {
        g.Go(func() error {
            body, err := fetch(ctx, url)
            if err != nil {
                return fmt.Errorf("fetch %s: %w", url, err)
            }
            // Each goroutine owns exactly one slot: no mutex needed.
            bodies[i] = body
            return nil
        })
    }

    if err := g.Wait(); err != nil {
        return nil, err
    }
    return bodies, nil
}

That is the whole pool. Behaviour worth knowing:

• g.Go blocks once the limit is reached, until a slot frees up. The for loop becomes its own back pressure — no jobs channel needed.
• errgroup.WithContext returns a derived context that is cancelled the moment any goroutine returns a non-nil error. Shadowing ctx with it, as above, is deliberate: every fetch gets the cancellable one.
• g.Wait() returns the first error, and waits for the rest regardless. Later errors are discarded — if you need all of them, collect them yourself (errors.Join is a good fit, see error handling in Go).
• Writing to bodies[i] is safe without a mutex because each goroutine writes one distinct element. Different elements of a slice are different memory; that is not a data race. Appending to a shared slice, or writing to a shared map, absolutely is — see concurrent map writing and reading in Go for what that failure looks like.

A Note on Loop Variables

The example above relies on Go 1.22’s per-iteration loop variables. Before 1.22, i and url were shared across iterations and every goroutine would see the final values — the single most common concurrency bug in Go. On older versions you must copy them:

for i, url := range urls {
    i, url := i, url // required before Go 1.22
    g.Go(func() error { /* ... */ })
}

Since Go 1.22 the copy is unnecessary. Leaving it in is harmless, and I still write it in code that must build on older toolchains. The loop semantics change was one of the more consequential recent additions to the language — I touched on the surrounding rules in mastering Golang for loops.

Streaming Results Instead of Preallocating

Indexing into a preallocated slice only works when you know the number of jobs up front. For a stream, send results down a channel and read them concurrently:

func processStream(ctx context.Context, in <-chan job, workers int) (<-chan result, func() error) {
    out := make(chan result)

    g, ctx := errgroup.WithContext(ctx)
    g.SetLimit(workers)

    go func() {
        // Closing `out` after every worker has finished lets the consumer
        // range over it and stop naturally.
        defer close(out)

        for j := range in {
            g.Go(func() error {
                body, err := fetch(ctx, j.URL)
                if err != nil {
                    return fmt.Errorf("job %d: %w", j.ID, err)
                }
                select {
                case out <- result{JobID: j.ID, Body: body}:
                    return nil
                case <-ctx.Done():
                    return ctx.Err()
                }
            })
        }
        _ = g.Wait()
    }()

    return out, g.Wait
}

The consumer ranges over out and then calls the returned function to get the error:

results, wait := processStream(ctx, jobs, 8)
for r := range results {
    save(r)
}
if err := wait(); err != nil {
    return fmt.Errorf("process stream: %w", err)
}

g.Wait() is safe to call more than once — subsequent calls return the same error immediately.

Picking the Limit

There is no universal number, but there is a reliable way to think about it.

CPU-bound work — parsing, hashing, image resizing, compression — saturates at roughly the number of cores. More goroutines just add scheduling overhead:

g.SetLimit(runtime.GOMAXPROCS(0))

I/O-bound work — HTTP calls, database queries, object storage — spends most of its time waiting, so the useful limit is much higher. But it is not “as high as possible”: it is whatever the slowest downstream dependency can absorb. If your database pool has 25 connections, a pool of 200 workers means 175 goroutines queueing on a mutex inside database/sql while your latency graph climbs.

// Match the constraint that actually binds.
g.SetLimit(db.Stats().MaxOpenConnections)

For outbound HTTP, remember that Go’s default transport keeps only 2 idle connections per host. Exceed that and you are opening a fresh TCP connection — plus a TLS handshake — for each extra request:

transport := http.DefaultTransport.(*http.Transport).Clone()
transport.MaxIdleConnsPerHost = 50
transport.MaxConnsPerHost = 50
client := &http.Client{Transport: transport, Timeout: 10 * time.Second}

Then set the pool limit to match. Tuning one without the other gets you nothing.

Whatever you pick, measure it. Run the job at 5, 10, 25 and 50 and look at total wall time and downstream latency — the fastest setting for your batch is often the one that makes everything else on the system slower. Load testing is the honest way to find out; I wrote about a lightweight setup in an easy way to load test your web apps.

Not Every Goroutine Belongs in a Pool

A pool is for a batch of similar work. Some situations want something else:

Waiting on several different things at once — no limit needed, just a group:

g, ctx := errgroup.WithContext(ctx)

var user *User
var orders []Order

g.Go(func() (err error) { user, err = loadUser(ctx, id); return })
g.Go(func() (err error) { orders, err = loadOrders(ctx, id); return })

if err := g.Wait(); err != nil {
    return nil, fmt.Errorf("load profile: %w", err)
}

Three sequential 100ms calls become one 100ms call. This is the highest-value use of errgroup in a typical request handler, and it needs no pool at all.

Work that must happen in order — a pool is the wrong shape entirely; you want a single consumer, like the simple queue implementation I wrote about earlier.

Fire-and-forget background work — resist it. A goroutine started in a request handler outlives the request, holds whatever it captured, and will be killed mid-flight when the process shuts down. If it matters, it belongs in a durable queue; if it does not, do it inline. The same reasoning applies at shutdown time, which I covered in graceful shutdown in Go web services.

Only trying if there is capacity — TryGo starts the goroutine only if a slot is free, and reports whether it did:

if !g.TryGo(func() error { return prefetch(ctx, url) }) {
    // Pool is busy; skip this optional work rather than blocking.
    metrics.PrefetchSkipped.Inc()
}

Pitfalls

Panic recovery is worth spelling out, because one bad input taking down the whole process is a common way for batch jobs to fail:

g.Go(func() (err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("panic processing %s: %v", url, r)
        }
    }()
    return process(ctx, url)
})

The named return value err is what makes this work — the deferred function assigns to it after the panic is recovered.

Conclusion

The pattern is small: pick a limit that matches your real bottleneck, use errgroup.WithContext so failures cancel their siblings, return errors instead of logging them, and give every blocking operation a context. Most of the time that is eight lines and no channel plumbing at all. Save the hand-rolled channel pool for the cases where you genuinely need to stream results or vary the shape of the work — and when you do write one, remember to close the jobs channel.

One place this pattern turns up more than you would expect: running the tool calls an LLM asks for, several at a time but not unboundedly — see tool use in Go.

The One Rule That Fixes Most of It

Handle an error once. Everywhere else, add context and pass it up.

“Handling” means doing something a caller cannot: returning a 404, retrying, falling back to a default, logging and moving on. If you are not doing one of those, you are just a link in the chain — and a link’s only job is to explain where it sits.

// Bad: the error is logged here and returned, so it will be logged again
// by every caller on the way up.
func (s *Store) User(ctx context.Context, id int64) (*User, error) {
    u, err := s.query(ctx, id)
    if err != nil {
        log.Printf("failed to load user: %v", err)
        return nil, err
    }
    return u, nil
}

// Good: add what this layer knows, and let the caller decide.
func (s *Store) User(ctx context.Context, id int64) (*User, error) {
    u, err := s.query(ctx, id)
    if err != nil {
        return nil, fmt.Errorf("load user %d: %w", id, err)
    }
    return u, nil
}

The second version produces messages that read like a stack trace written in English:

handle GET /users/42: load user 42: query users: sql: no rows in result set

Wrapping With %w

fmt.Errorf has a special verb, %w, that wraps the original error instead of flattening it to a string. The result behaves like a normal error, but the original stays reachable underneath.

_, err := os.Open("/etc/app/config.yaml")
wrapped := fmt.Errorf("read config: %w", err)

fmt.Println(wrapped)
// read config: open /etc/app/config.yaml: no such file or directory

fmt.Println(errors.Is(wrapped, os.ErrNotExist)) // true

That last line is the whole point. %v would have given you the same message, but errors.Is would have returned false because the chain was broken.

A few conventions that keep the output readable:

• Start the message with a lowercase verb phrase describing what this function was doing: "load user", "encode response", "dial redis".
• Do not end with punctuation, and do not include the word “error” or “failed” — the chain already reads as a failure.
• Put the wrapped error last, after a colon and a space.

Since Go 1.20 you can wrap more than one error in a single call, which is handy when you are cleaning up:

err := doWork()
if cerr := f.Close(); cerr != nil {
    err = fmt.Errorf("work failed: %w; close failed: %w", err, cerr)
}

Sentinel Errors: For Conditions Callers Branch On

A sentinel is a package-level error value that callers are meant to recognise.

package store

import "errors"

var (
    ErrNotFound  = errors.New("not found")
    ErrConflict  = errors.New("conflict")
    ErrForbidden = errors.New("forbidden")
)

Return them wrapped, so the caller gets both the identity and the context:

func (s *Store) User(ctx context.Context, id int64) (*User, error) {
    u, err := s.query(ctx, id)
    if errors.Is(err, sql.ErrNoRows) {
        return nil, fmt.Errorf("load user %d: %w", id, ErrNotFound)
    }
    if err != nil {
        return nil, fmt.Errorf("load user %d: %w", id, err)
    }
    return u, nil
}

And check them with errors.Is, which walks the whole chain:

u, err := store.User(ctx, id)
switch {
case errors.Is(err, store.ErrNotFound):
    http.Error(w, "user not found", http.StatusNotFound)
    return
case err != nil:
    log.Error("load user", "err", err)
    http.Error(w, "internal error", http.StatusInternalServerError)
    return
}

Never compare with == when the error might be wrapped, and never compare error strings. errors.Is(err, ErrNotFound) keeps working when someone adds another wrapping layer three months from now; err == ErrNotFound silently stops working.

Keep the list short. Sentinels are part of your package’s public API — every one you export is a promise. If callers cannot meaningfully branch on it, it should not be a sentinel.

Custom Error Types: When You Need to Carry Data

A sentinel says what went wrong. A custom type also says with what. Reach for one when the caller needs a field, not just an identity.

// ValidationError reports a single field that failed validation.
type ValidationError struct {
    Field  string
    Reason string
}

func (e *ValidationError) Error() string {
    return fmt.Sprintf("field %q is invalid: %s", e.Field, e.Reason)
}

errors.As finds it anywhere in the chain and assigns it to your variable:

var vErr *ValidationError
if errors.As(err, &vErr) {
    w.WriteHeader(http.StatusBadRequest)
    json.NewEncoder(w).Encode(map[string]string{
        "field":  vErr.Field,
        "reason": vErr.Reason,
    })
    return
}

Two details that trip people up:

1. Pass a pointer to the target. errors.As(err, &vErr) where vErr is already a *ValidationError. Passing vErr directly panics.
2. Be consistent about pointer vs. value receivers. If Error() is defined on *ValidationError, then only *ValidationError implements error. Return &ValidationError{...}, and match against *ValidationError.

If your type wraps another error, give it an Unwrap method so errors.Is can keep walking:

type QueryError struct {
    Query string
    Err   error
}

func (e *QueryError) Error() string { return e.Query + ": " + e.Err.Error() }
func (e *QueryError) Unwrap() error { return e.Err }

Now errors.Is(err, sql.ErrNoRows) still works even with a QueryError in the middle.

Collecting Several Errors With errors.Join

When you validate a whole struct, failing on the first problem makes for a frustrating API. errors.Join (Go 1.20) combines errors into one value that errors.Is and errors.As can still search.

func (r CreateUserRequest) Validate() error {
    var errs []error
    if r.Email == "" {
        errs = append(errs, &ValidationError{Field: "email", Reason: "required"})
    }
    if len(r.Password) < 12 {
        errs = append(errs, &ValidationError{Field: "password", Reason: "too short"})
    }
    // Join returns nil when every element is nil, so this is safe as-is.
    return errors.Join(errs...)
}

The joined error prints one message per line, and errors.As will find the first *ValidationError inside it.

Which Tool for Which Job

That last row matters more than it looks. A cache miss is not an error. An empty search result is not an error. Reserve errors for situations where the caller genuinely cannot continue as planned.

Errors at the HTTP Boundary

The boundary is where errors finally get handled, and it is worth doing that in exactly one place. Middleware is a natural home for it — the same pattern as the Go middleware example, just applied to errors instead of responses.

// handlerFunc is like http.HandlerFunc but may return an error.
type handlerFunc func(http.ResponseWriter, *http.Request) error

// wrap turns a handlerFunc into a plain http.Handler, translating
// errors into status codes in one place.
func wrap(h handlerFunc) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        err := h(w, r)
        if err == nil {
            return
        }

        var vErr *ValidationError
        switch {
        case errors.Is(err, store.ErrNotFound):
            http.Error(w, "not found", http.StatusNotFound)
        case errors.Is(err, store.ErrForbidden):
            http.Error(w, "forbidden", http.StatusForbidden)
        case errors.As(err, &vErr):
            http.Error(w, vErr.Error(), http.StatusBadRequest)
        case errors.Is(err, context.Canceled):
            // The client hung up. Nothing to report.
            return
        default:
            // Log the full chain, tell the client nothing.
            slog.Error("request failed",
                "method", r.Method, "path", r.URL.Path, "err", err)
            http.Error(w, "internal error", http.StatusInternalServerError)
        }
    })
}

Handlers become quiet and linear:

func getUser(w http.ResponseWriter, r *http.Request) error {
    id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
    if err != nil {
        return &ValidationError{Field: "id", Reason: "must be an integer"}
    }

    u, err := store.User(r.Context(), id)
    if err != nil {
        return fmt.Errorf("get user: %w", err)
    }

    return json.NewEncoder(w).Encode(u)
}

The context.Canceled case is easy to forget and shows up constantly in production: a user closes the tab, the request context is cancelled, and your dashboard fills with 500s that nothing went wrong for. If you have not met that machinery yet, understanding Golang context covers where those cancellations come from.

Common Pitfalls

One more, subtle enough to deserve its own note: a non-nil interface holding a nil pointer is not nil.

func find() *ValidationError { return nil }

func check() error {
    return find() // returns a non-nil error holding a nil *ValidationError
}

fmt.Println(check() == nil) // false — almost certainly not what you wanted

Declare the return type as error and return a literal nil, or check the concrete value before returning it.

Testing Error Paths

Assert on identity and type, never on the message text. Messages are for humans and will change.

func TestUserNotFound(t *testing.T) {
    _, err := store.User(context.Background(), 999)

    if !errors.Is(err, store.ErrNotFound) {
        t.Fatalf("got %v, want ErrNotFound", err)
    }
}

func TestValidation(t *testing.T) {
    err := CreateUserRequest{}.Validate()

    var vErr *ValidationError
    if !errors.As(err, &vErr) {
        t.Fatalf("got %v, want *ValidationError", err)
    }
    if vErr.Field != "email" {
        t.Errorf("got field %q, want email", vErr.Field)
    }
}

This pairs nicely with mocking the database layer, which I covered in how to test database interactions in Golang applications — you can force sql.ErrNoRows and check that your store translates it into ErrNotFound.

Conclusion

Go’s error handling is verbose, but it is also unusually honest: every failure is a value you can inspect, wrap and route. Wrap with %w on the way up, export a small set of sentinels for the conditions callers care about, use custom types when they need the details, and handle everything exactly once at the boundary. Do that and the if err != nil blocks stop feeling like noise and start reading like documentation.

One case that breaks the usual rules and deserves a look: an LLM call can return HTTP 200 while declining to answer, so err is nil and there is nothing in the content. Calling an LLM from Go covers that branch.