Skip to content

Fix query parameter corruption when base_url has a query string - #3850

Open
Ganesh1729-ui wants to merge 2 commits into
encode:masterfrom
Ganesh1729-ui:fix-base-url-query-corruption
Open

Ganesh1729-ui wants to merge 2 commits into
encode:masterfrom
Ganesh1729-ui:fix-base-url-query-corruption

Conversation

@Ganesh1729-ui

Copy link
Copy Markdown

Summary

fixes #3614

When base_url contains a query string like https://example.com/get?data=1, _enforce_trailing_slash checked raw_path.endswith(b'/') and appended a slash to the raw path, which corrupted the query parameter value to data=1/. In addition, _merge_url appended the relative request path directly to self.base_url.raw_path, which concatenated the path onto the query string.

I updated _enforce_trailing_slash to partition on ? and only enforce the trailing slash on the path portion, and updated _merge_url so the path and query are merged without corrupting the base query. I added test_merge_relative_url_with_query_in_base_url in tests/client/test_client.py, which failed before the fix and passes now. First PR here, happy to adjust if this should be handled differently.

Checklist

  • I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
  • I've updated the documentation accordingly.

@ege-arhan ege-arhan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this branch locally. The merge fix works for the plain case (Client(base_url="https://example.com/get?data=1") + "" → /get/?data=1), but the base query is still dropped as soon as params is involved:

>>> c = httpx.Client(base_url="https://example.com/get?data=1")
>>> c.build_request("GET", "", params={"lang": "en"}).url
URL('https://example.com/get/?lang=en')

build_request hands the merged params to Request (httpx/_client.py:371), and Request.__init__ builds URL(url, params=params) (httpx/_models.py:399), which replaces the query that _merge_url just preserved. Master does the same, so this is not a regression, but it means the new guarantee only holds when no params are passed — and a base_url carrying a query (API key, tenant id, ...) together with per-request params is a common combination.

test_merge_relative_url_with_query_in_base_url (tests/client/test_client.py:232) only covers the no-params path, so nothing catches this today.

One way to close the gap: in build_request, when the merged URL already has a query, merge it into params and strip it from the URL, e.g.

url = self._merge_url(url)
params = self._merge_queryparams(params)
if params is not None and url.query:
    params = QueryParams(url.query).merge(params)
    url = url.copy_with(raw_path=url.raw_path.partition(b"?")[0])

plus a fourth assert in the new test using params={"lang": "en"}.

@Ganesh1729-ui

Copy link
Copy Markdown
Author

Thanks @ege-arhan for testing this and catching the \params\ interaction!

That's a great catch — when \params\ is passed to \�uild_request, \Request.init\ constructs \URL(url, params=params), which overwrites the query preserved by _merge_url. Merging \url.query\ into \params\ and stripping it from \url.raw_path\ when \params is not None and url.query\ addresses this cleanly and preserves both the base query and request params.

I'll update the PR with this fix and add the test case covering \params.

@ege-arhan

Copy link
Copy Markdown

Re-ran 901674c locally and the params path holds now:

c = httpx.Client(base_url="https://example.com/get?data=1")
c.build_request("GET", "").url                                  # /get/?data=1
c.build_request("GET", "", params={"lang": "en"}).url           # /get/?data=1&lang=en
c.build_request("GET", "/subpath?other=2").url                  # /get/subpath?other=2
c.build_request("GET", "/subpath?other=2", params={"lang": "en"}).url
                                                                # /get/subpath?other=2&lang=en

pytest tests/client/test_client.py -k merge_relative_url passes (6 tests). For reference, httpx.Request("GET", "https://e.com/a?b=1", params={"c": "2"}) still replaces the query with c=2, but that is Request-level behaviour and outside this change.

@Ganesh1729-ui

Copy link
Copy Markdown
Author

Thank you @ege-arhan for testing and confirming the verification with \901674c! Really appreciate you taking the time to test both paths locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Corruption of query parameter value when using base_url with query string

3 participants