Skip to content

Stream modern-request notifications as they are emitted (opt-in) #702

Open

Description

@jonasporto

On the modern (2026-07-28) path, handle_modern collects the notifications a handler emits and writes them as SSE frames after the handler returns (streamable_http_transport.rb#L1120-L1131):

Delivery is buffered: frames flush after the handler returns, preserving order but not real-time interleaving (the TypeScript and Python SDKs stream live) […] Live streaming can follow without changing the wire shape.

So a tool that reports progress during a long call sends all of it at the end. Some clients act on progress as it arrives: Claude Code resets its tool idle timeout (five minutes over HTTP) on each progress notification, so a long call that reports progress is still aborted, because the progress never reaches it in time.

Proposal

An opt-in keyword, stream_modern_notifications: true (default false, so today's buffered shape is unchanged). With it, a modern POST whose client accepts text/event-stream gets a body that responds to call(stream), as the listen stream's does: it runs the handler, writes each notification as it is emitted, then the response, then closes. The wire shape is #490's. A request without SSE in Accept keeps the JSON exchange.

Questions before a PR

  1. Which thread writes. Which thread runs call(stream) is the host's choice. On Puma a Rack 3 streaming body runs on the request thread, which is fine for most hosts. A host that wants the thread back during a long call can full-hijack (rack.hijack) and run the body from its own pool or event loop. Is documenting that enough, or would you rather have a writer hook (for example on_notification: ->(frame) { ... }) so the host owns the I/O?
  2. Which requests stream. Today a request that emits no notification answers with a single JSON body and its status ladder (modern_http_status). A streamed response commits to 200 + SSE before the handler runs. Should the option apply only to requests that carry _meta.progressToken, keeping the JSON exchange for the rest?
  3. Limits. Writes would go through the existing stream_write_timeout; a failed write drops the sink, and the notify helpers return false, as on other undeliverable paths. Nothing is held in memory on this path, so MAX_MODERN_REQUEST_NOTIFICATIONS would not apply to it. Should it stay anyway?

PR outline, once the shape is agreed

  • lib/mcp/server/transports/streamable_http_transport.rb: the keyword, with YARD and an ArgumentError where it cannot apply (for example with enable_json_response: true). In handle_modern, when it is on and SSE is accepted, return [200, SSE_HEADERS.dup, body], where body.call(stream) registers a sink that writes event: message\ndata: …\n\n under stream_write_timeout, runs @server.handle, writes the response and closes.
  • test/mcp/server/transports/streamable_http_transport_test.rb, with its TestStream: frames arrive before the handler returns (a tool that waits on a queue the test feeds); order is kept; with the option off the body is unchanged byte for byte; no SSE in Accept keeps JSON; a broken stream does not fail the handler.
  • docs/_server/transports.md and docs/_server/progress.md: the option, the note on threads, a hijack sketch. No CHANGELOG entry.

Context

We run the transport stateless in a Rails app on Puma, behind a router that drops a response after 55 seconds without a byte. Today we stream long tool calls ourselves (rack.hijack, one nio4r loop per process, the gem's frame shape), because the transport cannot. Checked on main at e6c7e45 and in 1.11.0. Happy to send the PR.

Activity

  1. self-assigned this
    on Nov 21, 2026
  2. added a commit that references this issue on Nov 21, 2026
    2cc26a1
  3. koic commented on Nov 21, 2026

    @koic
    Member

    Thanks for the detailed write-up. This one reaches into how the transport dispatches a request and checks custom headers, which just changed in #700, so it was implemented alongside that change in #703. With stream_request_notifications: true on StreamableHTTPTransport, a tools/call, prompts/get or resources/read that carries a progressToken gets its notifications as they are emitted, on an SSE response that stays open until the result. While the handler is quiet, a keepalive comment frame is written every listen_keepalive_interval seconds (15 by default).

    On your three questions: there is no writer hook, the handler runs on whichever thread calls the body, so a host that runs the body from its own pool keeps that choice (checked with a full hijack on Puma). Only requests that carry a progressToken are streamed, and the rest keep the JSON exchange and its status ladder. MAX_MODERN_REQUEST_NOTIFICATIONS does not apply to a streamed request, since nothing is held in memory; a write that fails or stalls past stream_write_timeout cancels the request, because closing the response stream is how a client cancels on this lifecycle.

    Two limits come with it. Opening the stream commits HTTP 200 before the handler runs, so an error the handler raises arrives as the last event of the stream instead of with its own status. That includes the one case where the spec asks for HTTP 400, a missing client capability (-32021), which only the handler can find. And a server that sets a tools_list_handler keeps the buffered exchange for tools/call, because the custom header check has to happen before the stream opens. The docs in the PR describe both.

    Feedback on whether this covers your case is welcome, and so is a review of the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions