We wanted a small, boring feature: let an operator read a running instance's recent error logs when something is wrong — and crucially, when *MCP itself* is the thing that's wrong. So the path can't depend on MCP. The design was deliberately humble: an opt-in in-memory ring buffer that tees slog output, plus a plain GET /_logs (Admin, bearer) — the same shape as the existing /_stats and /_audit endpoints.
app.CaptureLogs() // keep the last 500 WARN+ records in memory
// GET /_logs → {"capacity":500,"count":2,"dropped":0,"entries":[...]}CaptureLogs wraps the current default handler so log lines still go to stderr *and* into the ring — a tee, not a replacement. Wrap slog.Default().Handler(), call slog.SetDefault, done. It worked. Tests passed. Then the full suite hung for ten minutes and died with a goroutine dump.
The trap
The stack told the story: a re-entrant lock inside log.Logger.output, reached *twice* in the same goroutine. The cause is a piece of slog behaviour that's easy to forget:
> slog.SetDefault also repoints the standard log package through the new handler.
And the other half: slog's built-in, zero-config handler (*slog.defaultHandler) doesn't write to stderr directly — it writes through the log package.
Put those together and wrapping the built-in handler builds a loop:
log.Output → (handlerWriter) → our tee → inner defaultHandler → log.Output → …The first log.Output takes the log mutex; the nested one tries to take it again; the goroutine deadlocks. The worst part: this only triggers for apps that *didn't* configure their own handler — i.e. the most common, zero-config case. It would freeze a default app on its very first warning.
The fix
We don't wrap the built-in handler. When we detect it, we forward to a fresh text handler on os.Stderr instead — which writes directly, no log bridge, no loop. Apps that configure their own handler (the recommended path) are wrapped unchanged, so their formatting is preserved.
Detecting it is the one slightly uncomfortable part: slog exposes no public type for its default handler, so we match the concrete type name. To keep that honest, a test asserts the detection and fails loudly with the new type name if a future Go release ever renames it — turning a silent re-deadlock into a one-line CI failure.
The lesson
slog and log are two doors into the same room. The moment you call slog.SetDefault, anything that writes via log can come back around through your handler. If your handler also writes via log, you've built a cycle. Tee to a concrete writer, not to a handler that loops back — and call CaptureLogs *after* any slog.SetDefault of your own.
The feature stayed boring. Getting there wasn't.