<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Streaming on WebDevStation</title>
    <link>https://webdevstation.com/tags/streaming/</link>
    <description>1 article tagged Streaming — tutorials, code examples and notes from building real systems, newest first.</description>
    <generator>Hugo</generator>
    <language>en</language>
    <managingEditor>Alex</managingEditor>
    <webMaster>Alex</webMaster>
    <copyright>© 2026 WebDevStation</copyright>
    <lastBuildDate>Thu, 27 Aug 2026 09:40:00 +0200</lastBuildDate>
    <atom:link href="https://webdevstation.com/tags/streaming/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Calling an LLM from Go: Streaming, Timeouts and the Parts That Bite</title>
      <link>https://webdevstation.com/posts/calling-claude-from-go/</link>
      <pubDate>Thu, 27 Aug 2026 09:40:00 +0200</pubDate>
      <author>Alex</author>
      <guid isPermaLink="true">https://webdevstation.com/posts/calling-claude-from-go/</guid>
      <description>A practical guide to wiring Claude into a Go service with the official SDK: content blocks, adaptive thinking, streaming, context deadlines, typed errors and the…</description>
      <content:encoded><![CDATA[<p>Most of the LLM tutorials I read are Python notebooks. That is fine for a prototype, but the moment the thing has to live inside a service — with timeouts, cancellation, retries and a bill attached — Go&rsquo;s constraints start to matter, and so do the details the notebooks skip. This is the write-up I wanted when I put my first Claude call into a Go HTTP handler.</p>
<h2 id="getting-a-client">Getting a Client</h2>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>go get github.com/anthropics/anthropic-sdk-go
</span></span></code></pre></div><div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#f92672">import</span> (
</span></span><span style="display:flex;"><span>    <span style="color:#e6db74">&#34;github.com/anthropics/anthropic-sdk-go&#34;</span>
</span></span><span style="display:flex;"><span>    <span style="color:#e6db74">&#34;github.com/anthropics/anthropic-sdk-go/option&#34;</span>
</span></span><span style="display:flex;"><span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// Reads ANTHROPIC_API_KEY from the environment.</span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">client</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">NewClient</span>()
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#75715e">// Or pass it explicitly, if you load config yourself.</span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">client</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">NewClient</span>(<span style="color:#a6e22e">option</span>.<span style="color:#a6e22e">WithAPIKey</span>(<span style="color:#a6e22e">key</span>))
</span></span></code></pre></div><p>Build the client <strong>once</strong>, at startup, and pass it around. It is safe for concurrent use and holds a connection pool — constructing one per request throws away keep-alive and gives you a fresh TLS handshake every time.</p>
<h2 id="the-first-call">The First Call</h2>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#a6e22e">resp</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">client</span>.<span style="color:#a6e22e">Messages</span>.<span style="color:#a6e22e">New</span>(<span style="color:#a6e22e">ctx</span>, <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">MessageNewParams</span>{
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">Model</span>:     <span style="color:#e6db74">&#34;claude-opus-5&#34;</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">MaxTokens</span>: <span style="color:#ae81ff">16000</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">Messages</span>: []<span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">MessageParam</span>{
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">NewUserMessage</span>(<span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">NewTextBlock</span>(<span style="color:#e6db74">&#34;Summarise this changelog in three bullets.&#34;</span>)),
</span></span><span style="display:flex;"><span>    },
</span></span><span style="display:flex;"><span>})
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;summarise changelog: %w&#34;</span>, <span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p>Two things about that snippet are worth slowing down on.</p>
<p><strong>The model is a plain string.</strong> The SDK ships typed constants (<code>anthropic.ModelClaudeOpus4_8</code> and friends), but <code>anthropic.Model</code> is an alias for <code>string</code>, so a model that has no constant yet is passed as its id. Either form compiles; check the SDK release notes before assuming a constant exists for the model you want.</p>
<p><strong><code>MaxTokens</code> is a ceiling, not a target.</strong> It is the point at which generation is cut off mid-sentence, and the model is not told about it. Setting it to <code>500</code> because you want a short answer does not produce a short answer — it produces a truncated one. Ask for brevity in the prompt and leave the ceiling generous. For non-streaming requests, something around <code>16000</code> keeps you clear of both truncation and the SDK&rsquo;s HTTP timeout.</p>
<h2 id="content-is-a-list-of-blocks-not-a-string">Content Is a List of Blocks, Not a String</h2>
<p>This is where the first hour usually goes. A response is not <code>resp.Text</code>. It is <code>resp.Content</code>, a slice of union values that can hold text, thinking, tool calls and more:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#66d9ef">for</span> <span style="color:#a6e22e">_</span>, <span style="color:#a6e22e">block</span> <span style="color:#f92672">:=</span> <span style="color:#66d9ef">range</span> <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">Content</span> {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">switch</span> <span style="color:#a6e22e">variant</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">block</span>.<span style="color:#a6e22e">AsAny</span>().(<span style="color:#66d9ef">type</span>) {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">case</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">TextBlock</span>:
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Println</span>(<span style="color:#a6e22e">variant</span>.<span style="color:#a6e22e">Text</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">case</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">ThinkingBlock</span>:
</span></span><span style="display:flex;"><span>        <span style="color:#75715e">// Reasoning, when you have asked for it to be shown.</span>
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">log</span>.<span style="color:#a6e22e">Debug</span>(<span style="color:#e6db74">&#34;model reasoning&#34;</span>, <span style="color:#e6db74">&#34;text&#34;</span>, <span style="color:#a6e22e">variant</span>.<span style="color:#a6e22e">Thinking</span>)
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p><code>block.AsAny()</code> is the accessor that gets you a concrete type to switch on. Reaching for <code>resp.Content[0].Text</code> and hoping works right up until the day a thinking block or a tool call lands in position zero, at which point you silently return an empty string. Write the type switch once, in a helper, and use it everywhere:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#75715e">// firstText returns the first text block in a response, or &#34;&#34; if there is none.</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">func</span> <span style="color:#a6e22e">firstText</span>(<span style="color:#a6e22e">msg</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">Message</span>) <span style="color:#66d9ef">string</span> {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">for</span> <span style="color:#a6e22e">_</span>, <span style="color:#a6e22e">block</span> <span style="color:#f92672">:=</span> <span style="color:#66d9ef">range</span> <span style="color:#a6e22e">msg</span>.<span style="color:#a6e22e">Content</span> {
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">t</span>, <span style="color:#a6e22e">ok</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">block</span>.<span style="color:#a6e22e">AsAny</span>().(<span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">TextBlock</span>); <span style="color:#a6e22e">ok</span> {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">t</span>.<span style="color:#a6e22e">Text</span>
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> <span style="color:#e6db74">&#34;&#34;</span>
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><h2 id="thinking-and-why-you-probably-want-it-on">Thinking, and Why You Probably Want It On</h2>
<p>Current Claude models can reason before answering. The recommended mode is <strong>adaptive</strong> — you do not budget tokens for it, the model decides how much thinking a given request deserves:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#a6e22e">adaptive</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">ThinkingConfigAdaptiveParam</span>{}
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">params</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">MessageNewParams</span>{
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">Model</span>:     <span style="color:#e6db74">&#34;claude-opus-5&#34;</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">MaxTokens</span>: <span style="color:#ae81ff">16000</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">Thinking</span>:  <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">ThinkingConfigParamUnion</span>{<span style="color:#a6e22e">OfAdaptive</span>: <span style="color:#f92672">&amp;</span><span style="color:#a6e22e">adaptive</span>},
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">Messages</span>:  <span style="color:#a6e22e">messages</span>,
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p>There is no <code>ThinkingConfigParamOfAdaptive</code> helper — you construct the union literal and take the address of the variant, as above. That trips people up because almost every other option in the SDK <em>does</em> have a constructor function.</p>
<p>A word of warning if you are carrying settings over from older code: the fixed thinking budget (<code>ThinkingConfigParamOfEnabled(N)</code>) is gone on current models and returns a 400 rather than being ignored. If you want to spend <em>less</em>, the lever is effort, not a token budget — and effort lives inside <code>output_config</code>, not at the top level of the request.</p>
<p>The counter-intuitive part: on the newest models, turning thinking <strong>off</strong> is not reliably a cost saving. Lower effort with thinking on generally beats thinking off, and disabling it has failure modes of its own. Leave it on and turn effort down.</p>
<h2 id="streaming">Streaming</h2>
<p>Anything a user waits for should stream. It is not just perceived speed — a long non-streaming request is also the easiest way to hit an HTTP timeout.</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#a6e22e">stream</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">client</span>.<span style="color:#a6e22e">Messages</span>.<span style="color:#a6e22e">NewStreaming</span>(<span style="color:#a6e22e">ctx</span>, <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">MessageNewParams</span>{
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">Model</span>:     <span style="color:#e6db74">&#34;claude-opus-5&#34;</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">MaxTokens</span>: <span style="color:#ae81ff">64000</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">Messages</span>:  <span style="color:#a6e22e">messages</span>,
</span></span><span style="display:flex;"><span>})
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">for</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Next</span>() {
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">event</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Current</span>()
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">switch</span> <span style="color:#a6e22e">ev</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">event</span>.<span style="color:#a6e22e">AsAny</span>().(<span style="color:#66d9ef">type</span>) {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">case</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">ContentBlockDeltaEvent</span>:
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">switch</span> <span style="color:#a6e22e">delta</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">ev</span>.<span style="color:#a6e22e">Delta</span>.<span style="color:#a6e22e">AsAny</span>().(<span style="color:#66d9ef">type</span>) {
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">case</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">TextDelta</span>:
</span></span><span style="display:flex;"><span>            <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Print</span>(<span style="color:#a6e22e">delta</span>.<span style="color:#a6e22e">Text</span>)
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Err</span>(); <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;stream response: %w&#34;</span>, <span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p>Two nested type switches is not the prettiest Go you will write, but the shape is stable: outer switch on the event, inner switch on the delta.</p>
<p><strong>Always check <code>stream.Err()</code>.</strong> <code>stream.Next()</code> returning <code>false</code> means &ldquo;no more events&rdquo; — it does not tell you whether that was a clean finish or a dropped connection. A loop that ignores <code>Err()</code> will happily serve a truncated answer as if it were complete.</p>
<p>If you want the whole message <em>and</em> the incremental deltas, accumulate as you go. There is no <code>GetFinalMessage()</code> on the Go stream:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#a6e22e">stream</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">client</span>.<span style="color:#a6e22e">Messages</span>.<span style="color:#a6e22e">NewStreaming</span>(<span style="color:#a6e22e">ctx</span>, <span style="color:#a6e22e">params</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">var</span> <span style="color:#a6e22e">message</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">Message</span>
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">for</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Next</span>() {
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">message</span>.<span style="color:#a6e22e">Accumulate</span>(<span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Current</span>())
</span></span><span style="display:flex;"><span>    <span style="color:#75715e">// ... also forward the delta to the user here</span>
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Err</span>(); <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">err</span>
</span></span><span style="display:flex;"><span>}
</span></span><span style="display:flex;"><span><span style="color:#75715e">// message.Content is now the complete response.</span>
</span></span></code></pre></div><p>Raising <code>MaxTokens</code> to <code>64000</code> in the streaming example is deliberate. Timeouts stop being the binding constraint once you stream, so you can give the model room.</p>
<h3 id="streaming-to-a-browser">Streaming to a Browser</h3>
<p>Server-sent events are the path of least resistance, and Go&rsquo;s <code>http.ResponseController</code> makes the flushing straightforward:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#66d9ef">func</span> (<span style="color:#a6e22e">h</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">Handler</span>) <span style="color:#a6e22e">stream</span>(<span style="color:#a6e22e">w</span> <span style="color:#a6e22e">http</span>.<span style="color:#a6e22e">ResponseWriter</span>, <span style="color:#a6e22e">r</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">http</span>.<span style="color:#a6e22e">Request</span>) {
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">w</span>.<span style="color:#a6e22e">Header</span>().<span style="color:#a6e22e">Set</span>(<span style="color:#e6db74">&#34;Content-Type&#34;</span>, <span style="color:#e6db74">&#34;text/event-stream&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">w</span>.<span style="color:#a6e22e">Header</span>().<span style="color:#a6e22e">Set</span>(<span style="color:#e6db74">&#34;Cache-Control&#34;</span>, <span style="color:#e6db74">&#34;no-cache&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">w</span>.<span style="color:#a6e22e">Header</span>().<span style="color:#a6e22e">Set</span>(<span style="color:#e6db74">&#34;X-Accel-Buffering&#34;</span>, <span style="color:#e6db74">&#34;no&#34;</span>) <span style="color:#75715e">// stop nginx buffering the stream</span>
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">rc</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">http</span>.<span style="color:#a6e22e">NewResponseController</span>(<span style="color:#a6e22e">w</span>)
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">stream</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">h</span>.<span style="color:#a6e22e">client</span>.<span style="color:#a6e22e">Messages</span>.<span style="color:#a6e22e">NewStreaming</span>(<span style="color:#a6e22e">r</span>.<span style="color:#a6e22e">Context</span>(), <span style="color:#a6e22e">params</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">for</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Next</span>() {
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">ev</span>, <span style="color:#a6e22e">ok</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Current</span>().<span style="color:#a6e22e">AsAny</span>().(<span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">ContentBlockDeltaEvent</span>)
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">ok</span> {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">continue</span>
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">delta</span>, <span style="color:#a6e22e">ok</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">ev</span>.<span style="color:#a6e22e">Delta</span>.<span style="color:#a6e22e">AsAny</span>().(<span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">TextDelta</span>)
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> !<span style="color:#a6e22e">ok</span> {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">continue</span>
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>        <span style="color:#75715e">// SSE data frames must not contain raw newlines.</span>
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">payload</span>, <span style="color:#a6e22e">_</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">json</span>.<span style="color:#a6e22e">Marshal</span>(<span style="color:#a6e22e">delta</span>.<span style="color:#a6e22e">Text</span>)
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">_</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Fprintf</span>(<span style="color:#a6e22e">w</span>, <span style="color:#e6db74">&#34;data: %s\n\n&#34;</span>, <span style="color:#a6e22e">payload</span>); <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">return</span> <span style="color:#75715e">// client hung up</span>
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">rc</span>.<span style="color:#a6e22e">Flush</span>(); <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">return</span>
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">stream</span>.<span style="color:#a6e22e">Err</span>(); <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">slog</span>.<span style="color:#a6e22e">Error</span>(<span style="color:#e6db74">&#34;llm stream failed&#34;</span>, <span style="color:#e6db74">&#34;err&#34;</span>, <span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Fprint</span>(<span style="color:#a6e22e">w</span>, <span style="color:#e6db74">&#34;event: error\ndata: {}\n\n&#34;</span>)
</span></span><span style="display:flex;"><span>        <span style="color:#a6e22e">_</span> = <span style="color:#a6e22e">rc</span>.<span style="color:#a6e22e">Flush</span>()
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">return</span>
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Fprint</span>(<span style="color:#a6e22e">w</span>, <span style="color:#e6db74">&#34;event: done\ndata: {}\n\n&#34;</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">_</span> = <span style="color:#a6e22e">rc</span>.<span style="color:#a6e22e">Flush</span>()
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p>Three details that cost me an afternoon each:</p>
<ul>
<li><strong><code>X-Accel-Buffering: no</code>.</strong> Without it nginx buffers your stream and delivers the whole thing at the end, which looks exactly like streaming being broken. (Nginx has strong opinions about proxied responses generally — I ran into a related set of them in <a href="/posts/how-to-make-nginx-cookie-aware/">making Nginx cache cookie aware</a>.)</li>
<li><strong>JSON-encode the delta.</strong> A model can and will emit a newline mid-sentence, and a bare newline terminates an SSE frame.</li>
<li><strong>Pass <code>r.Context()</code>, not <code>context.Background()</code>.</strong> When the user closes the tab, the request context cancels, the SDK aborts the HTTP call, and you stop paying for tokens nobody will read.</li>
</ul>
<h2 id="deadlines-and-cancellation">Deadlines and Cancellation</h2>
<p>Context is not decoration here. It is the only thing standing between a slow model call and a goroutine that lives forever:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#a6e22e">ctx</span>, <span style="color:#a6e22e">cancel</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">context</span>.<span style="color:#a6e22e">WithTimeout</span>(<span style="color:#a6e22e">r</span>.<span style="color:#a6e22e">Context</span>(), <span style="color:#ae81ff">90</span><span style="color:#f92672">*</span><span style="color:#a6e22e">time</span>.<span style="color:#a6e22e">Second</span>)
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">defer</span> <span style="color:#a6e22e">cancel</span>()
</span></span><span style="display:flex;"><span>
</span></span><span style="display:flex;"><span><span style="color:#a6e22e">resp</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">client</span>.<span style="color:#a6e22e">Messages</span>.<span style="color:#a6e22e">New</span>(<span style="color:#a6e22e">ctx</span>, <span style="color:#a6e22e">params</span>)
</span></span></code></pre></div><p>Set the deadline against how long the <em>work</em> should take, not a habit. A classification call has no business taking 90 seconds; a long agentic turn on a hard problem might legitimately run for several minutes. If you have not internalised how deadlines propagate through a call chain, <a href="/posts/understanding-golang-context/">understanding Golang context</a> covers the machinery.</p>
<p>The corollary at shutdown: an in-flight model call is exactly the kind of long request that a naive <code>SIGTERM</code> handler will sever. Drain it properly — see <a href="/posts/graceful-shutdown-in-go-web-services/">graceful shutdown in Go web services</a>.</p>
<h2 id="errors-worth-distinguishing">Errors Worth Distinguishing</h2>
<p>The SDK returns typed errors. Use <code>errors.As</code> to get at the status code rather than matching on message strings:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#a6e22e">resp</span>, <span style="color:#a6e22e">err</span> <span style="color:#f92672">:=</span> <span style="color:#a6e22e">client</span>.<span style="color:#a6e22e">Messages</span>.<span style="color:#a6e22e">New</span>(<span style="color:#a6e22e">ctx</span>, <span style="color:#a6e22e">params</span>)
</span></span><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#a6e22e">err</span> <span style="color:#f92672">!=</span> <span style="color:#66d9ef">nil</span> {
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">var</span> <span style="color:#a6e22e">apiErr</span> <span style="color:#f92672">*</span><span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">Error</span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">if</span> <span style="color:#a6e22e">errors</span>.<span style="color:#a6e22e">As</span>(<span style="color:#a6e22e">err</span>, <span style="color:#f92672">&amp;</span><span style="color:#a6e22e">apiErr</span>) {
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">switch</span> <span style="color:#a6e22e">apiErr</span>.<span style="color:#a6e22e">StatusCode</span> {
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">case</span> <span style="color:#a6e22e">http</span>.<span style="color:#a6e22e">StatusTooManyRequests</span>:
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;rate limited: %w&#34;</span>, <span style="color:#a6e22e">ErrRetryable</span>)
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">case</span> <span style="color:#a6e22e">http</span>.<span style="color:#a6e22e">StatusBadRequest</span>:
</span></span><span style="display:flex;"><span>            <span style="color:#75715e">// Your request is malformed. Retrying will not help.</span>
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;bad request to model: %w&#34;</span>, <span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>        <span style="color:#66d9ef">default</span>:
</span></span><span style="display:flex;"><span>            <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;model call failed (%d): %w&#34;</span>, <span style="color:#a6e22e">apiErr</span>.<span style="color:#a6e22e">StatusCode</span>, <span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>        }
</span></span><span style="display:flex;"><span>    }
</span></span><span style="display:flex;"><span>    <span style="color:#75715e">// Not an API error: context cancellation, DNS, TLS, connection reset.</span>
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> <span style="color:#a6e22e">fmt</span>.<span style="color:#a6e22e">Errorf</span>(<span style="color:#e6db74">&#34;model call failed: %w&#34;</span>, <span style="color:#a6e22e">err</span>)
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p>Wrapping with <code>%w</code> at each layer is what lets the HTTP boundary decide the status code without every layer needing to know about HTTP — the same pattern I laid out in <a href="/posts/error-handling-in-go/">error handling in Go</a>.</p>
<p><strong>The SDK already retries for you.</strong> By default it retries a couple of times on <code>408</code>, <code>409</code>, <code>429</code>, <code>5xx</code> and connection errors, with backoff. Two consequences people miss:</p>
<ol>
<li><strong>Do not add your own retry loop on top</strong> without lowering the SDK&rsquo;s. Three of yours around two of its is nine attempts and a long tail of latency.</li>
<li><strong>Wall-clock can reach <code>timeout × (attempts + 1)</code>.</strong> Your context deadline is the real budget — set it deliberately, because the retry behaviour will happily use all of it.</li>
</ol>
<h2 id="a-refusal-is-not-an-error">A Refusal Is Not an Error</h2>
<p>This one surprises people. If a safety classifier declines a request, you get <strong>HTTP 200</strong> — a perfectly successful response whose <code>StopReason</code> says the model declined:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#66d9ef">if</span> <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">StopReason</span> <span style="color:#f92672">==</span> <span style="color:#a6e22e">anthropic</span>.<span style="color:#a6e22e">StopReasonRefusal</span> {
</span></span><span style="display:flex;"><span>    <span style="color:#a6e22e">slog</span>.<span style="color:#a6e22e">Warn</span>(<span style="color:#e6db74">&#34;model declined&#34;</span>,
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;category&#34;</span>, <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">StopDetails</span>.<span style="color:#a6e22e">Category</span>,
</span></span><span style="display:flex;"><span>        <span style="color:#e6db74">&#34;explanation&#34;</span>, <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">StopDetails</span>.<span style="color:#a6e22e">Explanation</span>)
</span></span><span style="display:flex;"><span>    <span style="color:#66d9ef">return</span> <span style="color:#e6db74">&#34;&#34;</span>, <span style="color:#a6e22e">ErrDeclined</span>
</span></span><span style="display:flex;"><span>}
</span></span></code></pre></div><p><code>err</code> is nil. <code>resp.Content</code> may hold nothing useful. Code that goes straight from <code>if err != nil</code> to reading <code>Content[0]</code> treats this as an empty answer and moves on. Check <code>StopReason</code> before you read content — and note the other values you care about: <code>max_tokens</code> means you were truncated, and <code>tool_use</code> means the model is waiting on you (which is <a href="/posts/tool-use-in-go-agent-loop/">a whole article of its own</a>).</p>
<h2 id="watch-the-usage-numbers">Watch the Usage Numbers</h2>
<p>Every response carries a token accounting:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-go" data-lang="go"><span style="display:flex;"><span><span style="color:#a6e22e">slog</span>.<span style="color:#a6e22e">Info</span>(<span style="color:#e6db74">&#34;model call&#34;</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#e6db74">&#34;input_tokens&#34;</span>, <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">Usage</span>.<span style="color:#a6e22e">InputTokens</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#e6db74">&#34;output_tokens&#34;</span>, <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">Usage</span>.<span style="color:#a6e22e">OutputTokens</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#e6db74">&#34;cache_read&#34;</span>, <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">Usage</span>.<span style="color:#a6e22e">CacheReadInputTokens</span>,
</span></span><span style="display:flex;"><span>    <span style="color:#e6db74">&#34;cache_write&#34;</span>, <span style="color:#a6e22e">resp</span>.<span style="color:#a6e22e">Usage</span>.<span style="color:#a6e22e">CacheCreationInputTokens</span>,
</span></span><span style="display:flex;"><span>)
</span></span></code></pre></div><p>Log these from day one. They are the only ground truth about what a feature costs, and <code>CacheReadInputTokens</code> in particular is how you find out that your prompt caching silently stopped working three deploys ago — the failure mode there is not an error, just a bigger invoice. That is <a href="/posts/prompt-caching-llm-cost/">its own post</a>, because it is the single biggest lever on what an LLM feature costs to run.</p>
<p>A structured logger pays for itself here: these are five numeric fields per call that you will want to aggregate later, which is exactly the case for <a href="/posts/structured-logging-in-go-with-slog/">structured logging with log/slog</a>.</p>
<h2 id="checklist">Checklist</h2>
<ul>
<li>One client, built at startup, shared across handlers.</li>
<li>Iterate <code>resp.Content</code> with a type switch; never index blindly into it.</li>
<li>Adaptive thinking on; tune cost with effort, not by disabling it.</li>
<li>Stream anything a human waits for, and always check <code>stream.Err()</code>.</li>
<li><code>r.Context()</code> all the way down, with a deliberate deadline.</li>
<li><code>errors.As</code> into <code>*anthropic.Error</code> for status-code branching.</li>
<li>Do not stack your retries on the SDK&rsquo;s.</li>
<li>Check <code>StopReason</code> before reading content — a refusal returns 200.</li>
<li>Log the usage fields from the first commit.</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>The API surface is small — one endpoint, one loop, a handful of block types. What makes an LLM call different from any other HTTP call in your service is that it is slow, occasionally non-deterministic, priced per token, and able to succeed while declining to do what you asked. Go gives you good tools for exactly those problems, provided you use the context properly and read the response as the structured thing it is rather than the string you wish it were.</p>]]></content:encoded>
      <category>AI Engineering</category>
      <category>Go Programming</category>
      <category>Backend Development</category>
    </item>
  </channel>
</rss>
