Skip to content

stdio server makes two worker-thread hops for every message it writes #3888

Open

Description

@patrick-immometrica

Description

stdout_writer in src/mcp/server/stdio.py does

await stdout.write(json + "\n")
await stdout.flush()

stdout is an anyio.AsyncFile, so each of those awaits is its own anyio.to_thread.run_sync round trip. The write is no use to the client until the flush, and the writer handles one message at a time, so every response and notification pays for two hops where one would do. It shows up when a tool sends progress or log notifications: each one queues behind two thread round trips.

Hop count, standalone (counts only the writer's hops):

import io

import anyio
import anyio.to_thread
from mcp_types import JSONRPCNotification

from mcp.server.stdio import stdio_server
from mcp.shared.message import SessionMessage

hops = 0
_run_sync = anyio.to_thread.run_sync


async def counting_run_sync(func, *args, **kwargs):
    global hops
    hops += 1
    return await _run_sync(func, *args, **kwargs)


async def main() -> None:
    global hops
    stdin = anyio.AsyncFile(io.StringIO())  # empty: the reader hits EOF straight away
    stdout = anyio.AsyncFile(io.StringIO())
    async with stdio_server(stdin=stdin, stdout=stdout) as (read_stream, write_stream):
        async with read_stream:
            async for _ in read_stream:
                pass
        anyio.to_thread.run_sync = counting_run_sync  # count only the writer from here on
        n = 10
        for i in range(n):
            note = JSONRPCNotification(
                jsonrpc="2.0", method="notifications/progress", params={"progressToken": 1, "progress": i}
            )
            await write_stream.send(SessionMessage(note))
        await write_stream.aclose()
    print(f"{n} messages written: {hops} worker-thread hops")


anyio.run(main)

On main (77001c9) this prints 10 messages written: 20 worker-thread hops. I'd expect 10.

Timing over a real stdio subprocess (Client spawning the server via StdioServerParameters), interleaved before/after runs, median wall time:

workload before after change
tool sending 5 progress notifications per call, 300 calls 578 ms 402 ms -30%
tool sending 20 progress notifications per call, 300 calls (legacy protocol) 1326 ms 851 ms -36%
50 concurrent async tool calls, 1000 total 544 ms 422 ms -22%
plain sequential call_tool, 800 calls 498 ms 443 ms -11%

The machine was busy during these runs (load average around 10), so treat the absolute numbers loosely. Plain one-request/one-response traffic gains little; it matters once a request produces several messages.

Doing the write and the flush in one anyio.to_thread.run_sync call on stdout.wrapped fixes it. The output bytes are identical and it's about 10 lines. One thing to note: it would break a duck-typed stdout that has async write/flush but no .wrapped, though the parameter is typed anyio.AsyncFile[str]. Open PR #2680 rewrites this writer for a different reason (shutdown after EOF) and would remove the double hop as a side effect. I have a branch with the small fix and a test if you want it.

Environment: macOS 27.2 on Apple M5, Python 3.12.16, mcp at 77001c9, anyio 4.14.2.

References

I used an AI assistant to help find and measure this; I ran the repro above myself.

Activity

  1. added
    enhancementRequest for a new feature that's not currently supported
    on Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementRequest for a new feature that's not currently supported

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions