Skip to content

match() round-trip guarantee vs. the documented lenient decode: which one wins? #3887

Open

Description

@SyedTashfin

Initial Checks

Release line

2.x (current stable) — reproduced on main at 77001c96f9, src/mcp/shared/uri_template.py

Description

match() can return a value set that is not a pre-image of the URI it just matched: expand(match(uri)) != uri. That breaks the contract the module docstring states and that test_match_inverts_expand_for_every_parseable_template asserts ("Exact equality with the original values is not required … as long as it re-expands identically").

Two shapes, same root cause family (captured values are percent-decoded, but re-expanding the decoded value does not always reproduce the original text).

A. Reserved expansion passing an existing percent-escape through. Two lines, no literals involved:

t = UriTemplate.parse("{+v}")
uri = t.expand({"v": "%33"})     # '%33'
t.match(uri)                     # {'v': '3'}      <- decoded
t.expand(t.match(uri))           # '3'             <- not the URI, so not a pre-image

{+v} leaves an existing pct-triplet untouched on expansion, so decoding it on match cannot be inverted by re-expansion.

B. An encoded separator inside a bounded variable, followed by a literal and a multi-segment variable. This is the shape that shows up in practice (a token or query value containing =, then a literal, then {+…}):

t = UriTemplate.parse("{a}x{+b}")
uri = t.expand({"a": "x=1", "b": ""})   # 'x%3D1x'
t.match(uri)                            # {'a': '', 'b': '=1x'}
t.expand(t.match(uri))                  # 'x=1x'  <- differs

Expected: {'a': 'x=1', 'b': ''}, which re-expands to x%3D1x.

Trace: in _scan_prefix, the ifemp path is not taken here, so the bounded capture does uri.find(nxt.text, pos, …) — find("x") hits the value's own first character at pos, so a is captured as "", the literal x consumes it, and the greedy {+b} takes %3D1 and unquotes it to =1. + re-emits = as =, not %3D, so the parse diverges from the URI.

Also affected, same mechanism: {a}end{+b} with a="end=end" (end%3Dendend → {'a': '', 'b': '=endend'}) and {?a}end{+b}.

Not affected (so this needs a multi-segment variable after the literal): {a}end{b} (correct), {a}end{/b} (correct), {a}end{?b} (correct), {a}end (correct, no greedy variable), and a plain non-encoded value like {"a": "xy"} (correct).

This is distinct from #3852: there the returned parse is a genuine pre-image (both readings expand identically, as discussed in that thread). Here the returned dict does not expand back to the URI at all.

Why the suite does not catch it

tests/shared/test_uri_template.py draws property values from a "literal-disjoint alphabet":

_VALUE_CHARS = string.ascii_lowercase + string.digits

so no generated value ever contains % or =. Extending that one line to ... + "=%" makes test_match_inverts_expand_for_every_parseable_template fail on the first accepted template, e.g.:

'{+v0}{&v1}~{;v2}{?v3}': match('%33&v1=%25f~;v2=erau?v3=l')
  -> {'v0': '3', 'v2': 'erau', 'v1': '%f', 'v3': 'l'}
  which re-expands to '3&v1=%25f~;v2=erau?v3=l' != input

Measured locally at 77001c9: 307 passed unchanged; with "=%" added to _VALUE_CHARS, that property test fails immediately.

Possible direction

  • Case B is the cheap one: after the lazy capture, check expand(candidate) == uri and, when it fails, retry at the next occurrence of the following literal before giving up. That leaves the documented lazy ordering intact for the cases that already work.
  • Case A looks like a contract decision: either stop decoding pct-escapes that the operator would re-emit differently, or extend the docstring so reserved-operator values containing pct-triplets are explicitly named as outside the round-trip guarantee (it currently names only the operator's separator).
  • Independently of the fix, _VALUE_CHARS including "=%" would keep both covered, since the property test already encodes the invariant that is being violated.

Python & MCP Python SDK

  • mcp: 2.3.0, and main at 77001c96f9
  • Python: 3.12.12

(I investigated and drafted this with AI assistance; every claim above was verified by running the code on 77001c9 rather than taken from the model. Happy to carry the fix if an outside PR is welcome for this one.)

Activity

  1. SyedTashfin commented on Nov 21, 2026

    @SyedTashfin
    Author

    Correction, after a closer read of this file's own tests. The round-trip failure I reported is real and reproducible, but as a conflict between two contracts this repository asserts, not as an outright defect — the lenient decoding has deliberate, commented tests behind it:

    • test_match_reserved_expansion_handles_slash: {+path} matching a%2Fb returns {"path": "a/b"}
    • test_match_encoded_delimiters_roundtrip: file{.ext} matching file.a%2Eb returns {"ext": "a.b"}
    • test_match_double_encoding_decoded_once: ..%252Fetc returns "..%2Fetc"

    Every one of those decodes to a value that re-expands to a different string, so the invariant below cannot hold alongside them. I had not read them when I filed, and I withdraw the framing that the matcher is wrong; I also withdraw the patch I was preparing, since "decode only when invertible" necessarily breaks those three tests. What I think is still worth a maintainer decision is which contract is authoritative, because a third test asserts the opposite:

    test_match_inverts_expand_for_every_parseable_template asserts t.expand(t.match(uri)) == uri for every parseable template. It passes only because property values are drawn from _VALUE_CHARS = string.ascii_lowercase + string.digits, so no generated value ever contains % (and a pct-escape is the only way to produce a non-invertible decode). Adding "=%" to that alphabet makes it fail on the first accepted template:

    '{+v0}{&v1}~{;v2}{?v3}': match('%33&v1=%25f~;v2=erau?v3=l')
      -> {'v0': '3', 'v2': 'erau', 'v1': '%f', 'v3': 'l'}
      which re-expands to '3&v1=%25f~;v2=erau?v3=l' != the input
    

    Both measurements are on 77001c9, Python 3.12.12, with the suite at 307 passed unchanged.

    Two more concrete cases, in case the distinction between "lenient decode" and "wrong split" matters to whoever decides:

    # a: pure lenient decode (the case the tests above bless)
    t = UriTemplate.parse("{+v}")
    t.match(t.expand({"v": "%33"}))          # {'v': '3'}, re-expands to '3'
    
    # b: the lazy literal match lands on the value's own first character, so a
    #    non-empty variable is reported empty and the encoded remainder moves
    t = UriTemplate.parse("{a}x{+b}")
    t.match(t.expand({"a": "x=1", "b": ""}))  # {'a': '', 'b': '=1x'} from 'x%3D1x'
                                              # expected {'a': 'x=1', 'b': ''}

    Case (a) is fully explained by the documented leniency. Case (b) is the one I am not sure about: a is reported empty for a URI whose value was x=1, and the literal x is consumed off the front of that value. {a}end{b}, {a}end{/b} and {a}end{?b} all get this right, so it needs a multi-segment variable after the literal.

    So my question is narrower than the original report:

    1. Is expand(match(uri)) == uri a guarantee at all, or is it only guaranteed for URIs with no pct-escapes in bounded spans? The docstring currently names only the operator's separator as the exception, which does not cover any of the examples above.
    2. If the lenient decode is authoritative, should test_match_inverts_expand_for_every_parseable_template and the _VALUE_CHARS comment say so explicitly? Right now the property test reads as a general inverse guarantee, and the alphabet that keeps it passing is the only thing preventing it from contradicting the three leniency tests.
    3. For case (b) specifically: is the empty capture intended? If not, taking the next occurrence of the following literal when the first one yields an empty capture would recover {'a': 'x=1', 'b': ''} without touching the decode policy.

    Happy to turn any of those into a patch, and equally happy to close this if the answer is "leniency wins, the property test just needs its alphabet documented".

  2. changed the title [-]UriTemplate.match() can return a non-pre-image: expand(match(uri)) != uri[/-] [+]match() round-trip guarantee vs. the documented lenient decode: which one wins?[/+] on Nov 21, 2026
  3. added
    bugSomething isn't working
    v2Affects the v2 line (2.x on main)
    on Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingv2Affects the v2 line (2.x on main)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions