From 38a78cbf252b5b422b0a5a71096c7a574cee9b72 Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 15:25:30 +0800 Subject: [PATCH 1/9] feat: install CPython from uv-build by default on Linux and macOS On Linux and macOS, a CPython install now takes the prebuilt uv-build archive when the uv-build index lists the version, and builds from source with pyenv otherwise. Windows, GraalPy, PyPy and Conda keep their current paths. - VFOX_PYTHON_USE_UV_BUILD is three-state: unset selects the default above, 0/false/no/off builds every CPython from source, and 1/true/yes/on uses uv-build only, as before. - Available lists the pyenv names together with the CPython releases only the index lists, newest first; pyenv -dev snapshots sort last. - A partial name such as python@3.12 resolves in the index first, so it installs the newest 3.12 that has an archive. Only a line the index does not list falls back to the combined list. latest is unchanged. - PreInstall sets the note "uv-build" on an archive. vfox passes the note to PostInstall in sdkInfo, so PostInstall makes no second index request. A source build refuses to start over a Python already in the install path. - With VFOX_PYTHON_MIRROR set and VFOX_PYTHON_UV_BUILD_MIRROR unset, CPython stays on the python.org mirror and builds from source, because uv-build downloads come from GitHub. - An index that cannot be read fails the install and names VFOX_PYTHON_USE_UV_BUILD=0; it never starts a silent source build. - The source-build E2E subtests set VFOX_PYTHON_USE_UV_BUILD=0, and the container test installs the exact uv-build version. - A pyenv index failure reports the HTTP status instead of a Lua concatenation error. --- README.md | 36 ++- README_CN.md | 25 +- docs/changes.md | 15 ++ docs/development.md | 30 ++- docs/e2e.md | 8 +- docs/mirror.md | 8 + docs/platforms.md | 60 +++-- docs/uv.md | 46 +++- docs/versions.md | 16 +- docs/zh-CN/changes.md | 9 + docs/zh-CN/development.md | 19 +- docs/zh-CN/e2e.md | 12 +- docs/zh-CN/mirror.md | 5 + docs/zh-CN/platforms.md | 46 ++-- docs/zh-CN/uv.md | 33 ++- docs/zh-CN/versions.md | 11 +- hooks/available.lua | 8 +- hooks/post_install.lua | 2 +- hooks/pre_install.lua | 29 ++- lib/util.lua | 278 ++++++++++++++++++++-- metadata.lua | 4 +- tests/e2e/linux/container_test.go | 12 +- tests/e2e/linux/e2e_test.go | 6 +- tests/partial_version_test.lua | 3 +- tests/uv_build_default_test.lua | 369 ++++++++++++++++++++++++++++++ 25 files changed, 936 insertions(+), 154 deletions(-) create mode 100644 tests/uv_build_default_test.lua diff --git a/README.md b/README.md index 14c6c6e..ce82b23 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,9 @@ Python plugin for [vfox](https://vfox.dev/). -It installs CPython from python.org, compiles CPython from source on Linux and -macOS, or downloads a prebuilt uv-build archive. GraalPy comes from Oracle's +It installs CPython from python.org on Windows. On Linux and macOS it downloads a +prebuilt uv-build archive when one exists, and compiles CPython from source +otherwise. GraalPy comes from Oracle's release archives on every supported platform. It also adopts the Pythons already on the machine — system packages, uv, pyenv, Conda and others — and reads a project's `.python-version`, so pyenv and uv projects open on the @@ -52,14 +53,15 @@ The available versions depend on the platform: | Platform | Default source | Versions | | --- | --- | --- | -| Linux | pyenv source build for CPython; Oracle archive for GraalPy | every release pyenv knows, including GraalPy, PyPy and Conda | -| macOS | pyenv source build for CPython; Oracle archive for GraalPy | every release pyenv knows, including GraalPy, PyPy and Conda | +| Linux | uv-build archive for CPython when the index has the version, otherwise a pyenv source build; Oracle archive for GraalPy | every release pyenv knows, including GraalPy, PyPy and Conda, plus CPython releases only uv-build lists | +| macOS | uv-build archive for CPython when the index has the version, otherwise a pyenv source build; Oracle archive for GraalPy | every release pyenv knows, including GraalPy, PyPy and Conda, plus CPython releases only uv-build lists | | Windows | python.org installer for CPython; Oracle zip for GraalPy | stable CPython installers for the architecture, plus GraalPy amd64 releases | On Windows, source-only security releases and pre-release or free-threaded CPython builds are left out because python.org's stable installer names do not -cover them. `VFOX_PYTHON_USE_UV_BUILD=1` switches to prebuilt uv-build archives -on Linux, macOS and Windows; uv-build contains CPython only. See +cover them. `VFOX_PYTHON_USE_UV_BUILD=0` builds every CPython from source on +Linux and macOS, and `VFOX_PYTHON_USE_UV_BUILD=1` uses prebuilt uv-build archives +only, on Linux, macOS and Windows; uv-build contains CPython only. See [docs/uv.md](docs/uv.md). `py.exe` does not read `PATH`. Windows installs are registered automatically @@ -120,16 +122,26 @@ echo graalpy-24.2.0 > .python-version ## uv -uv-build is the optional prebuilt source. `python@3.12` installs the newest 3.12 -release in its index: +On Linux and macOS, a CPython install takes the prebuilt uv-build archive when the +index lists the version, so it needs no compiler or build dependencies. The index +lists only some releases, normally the newest patch of each line, so an older +patch builds from source with pyenv, as does a version the index does not list. +GraalPy comes from Oracle's archive; PyPy and Conda build from source with pyenv. +A partial name such as `python@3.12` picks the newest 3.12 that the index lists, and +installs its archive. Only a line the index does not list falls back to the newest +match in `vfox search python`, which is built from source. To build a newer patch +that the index does not list yet, name its exact version: ```bash -export VFOX_PYTHON_USE_UV_BUILD=1 -vfox install python@3.12 +vfox install python@3.12 # newest 3.12 in the uv-build index, as an archive +vfox install python@3.12.16 # exactly 3.12.16 ``` -uv-build needs no compiler or build dependencies. On Linux the libc is -detected automatically; set `VFOX_PYTHON_UV_LIBC=gnu` or +Set `VFOX_PYTHON_USE_UV_BUILD=0` to always build from source, or +`VFOX_PYTHON_USE_UV_BUILD=1` to use uv-build only, which fails for a version the +index does not list. If `VFOX_PYTHON_MIRROR` is set, CPython keeps its source +build on that mirror, unless `VFOX_PYTHON_UV_BUILD_MIRROR` is set too. On Linux the +libc is detected automatically; set `VFOX_PYTHON_UV_LIBC=gnu` or `VFOX_PYTHON_UV_LIBC=musl` to override it. uv-managed Pythons from `uv python install` are picked up by `vfox link --detect python` like any other installation. See [docs/uv.md](docs/uv.md) for the archive and for how to make diff --git a/README_CN.md b/README_CN.md index 7c57427..2ee2f45 100644 --- a/README_CN.md +++ b/README_CN.md @@ -4,8 +4,8 @@ [vfox](https://vfox.dev/) 的 Python 插件。 -它可以从 python.org 安装 CPython,在 Linux 和 macOS 上从源码编译 CPython,也 -可以下载 uv-build 的预编译包。GraalPy 在所有支持的平台上都来自 Oracle 发布的 +它在 Windows 上从 python.org 安装 CPython。在 Linux 和 macOS 上,有对应的 uv-build +预编译包时下载它,否则从源码编译 CPython。GraalPy 在所有支持的平台上都来自 Oracle 发布的 压缩包。它还能纳入本机已有的 Python——系统包管理器、uv、pyenv、Conda 等等—— 并读取项目里的 `.python-version`,让为 pyenv 和 uv 写的项目按文件里写的版本 打开。 @@ -48,14 +48,15 @@ vfox install python@graalpy-24.2.0 | 平台 | 默认来源 | 版本范围 | | --- | --- | --- | -| Linux | CPython 用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | pyenv 收录的全部版本,包括 GraalPy、PyPy 和 Conda | -| macOS | CPython 用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | pyenv 收录的全部版本,包括 GraalPy、PyPy 和 Conda | +| Linux | CPython 在 uv-build 索引列出该版本时用 uv-build 预编译包,否则用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | pyenv 收录的全部版本,包括 GraalPy、PyPy 和 Conda,以及只有 uv-build 列出的 CPython 版本 | +| macOS | CPython 在 uv-build 索引列出该版本时用 uv-build 预编译包,否则用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | pyenv 收录的全部版本,包括 GraalPy、PyPy 和 Conda,以及只有 uv-build 列出的 CPython 版本 | | Windows | CPython 用 python.org 安装包;GraalPy 用 Oracle 压缩包 | 当前架构的稳定 CPython 安装包,以及 GraalPy 的 amd64 版本 | Windows 上,python.org 没有为仅提供源码的安全版本发布安装包,稳定的安装包名也 覆盖不了预发布版和自由线程的 CPython 构建,所以这些版本不在列表里。设置 -`VFOX_PYTHON_USE_UV_BUILD=1` 后,Linux、macOS 和 Windows 都改为下载 uv-build -的预编译包;uv-build 只包含 CPython。详见 +`VFOX_PYTHON_USE_UV_BUILD=0` 后,Linux 和 macOS 上的每个 CPython 都从源码编译;设置 +`VFOX_PYTHON_USE_UV_BUILD=1` 后,只使用 uv-build 的预编译包,适用于 Linux、macOS 和 +Windows;uv-build 只包含 CPython。详见 [docs/zh-CN/uv.md](docs/zh-CN/uv.md)。 `py.exe` 不读取 `PATH`。Windows 上的安装会自动注册到 vfox 自己的 PEP 514 @@ -114,15 +115,17 @@ echo graalpy-24.2.0 > .python-version ## uv -uv-build 是可选的预编译来源。`python@3.12` 会安装它的索引里最新的 3.12 版本: +在 Linux 和 macOS 上,CPython 安装在 uv-build 索引列出该版本时使用它的预编译包,因此不需要编译器或构建依赖。 +索引只列出部分版本,通常是每个版本线的最新补丁版,所以较旧的补丁版会用 pyenv 从源码编译,索引没有列出的版本也是如此。GraalPy 来自 Oracle 的压缩包;PyPy 和 Conda 用 pyenv 从源码编译。部分版本号(如 `python@3.12`)会先在 uv-build 索引里选出最新的 3.12 版本,并安装它的预编译包。只有索引没有列出的版本系列,才会改为选中 `vfox search python` 列出的、最新的匹配版本,并从源码编译。要编译索引里还没有的较新补丁版,请写出它的精确版本号: ```bash -export VFOX_PYTHON_USE_UV_BUILD=1 -vfox install python@3.12 +vfox install python@3.12 # uv-build 索引里最新的 3.12,安装预编译包 +vfox install python@3.12.16 # 精确的 3.12.16 ``` -uv-build 不需要编译器或构建依赖。Linux 上会自动识别 libc;要覆盖识别结果, -设置 `VFOX_PYTHON_UV_LIBC=gnu` 或 `VFOX_PYTHON_UV_LIBC=musl`。 +设置 `VFOX_PYTHON_USE_UV_BUILD=0` 可以始终从源码编译;设置 `VFOX_PYTHON_USE_UV_BUILD=1` 则只使用 uv-build,索引里没有列出的版本会失败。若设置了 `VFOX_PYTHON_MIRROR`,CPython 的源码构建会继续经由该镜像,除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`。 +Linux 上会自动识别 libc;要覆盖识别结果,设置 `VFOX_PYTHON_UV_LIBC=gnu` 或 +`VFOX_PYTHON_UV_LIBC=musl`。 `uv python install` 装的 Python 和别的安装一样,会被 `vfox link --detect python` 找到。预编译包的细节,以及怎样让 uv 使用 vfox 选中的解释器而不是再下载一份,见 diff --git a/docs/changes.md b/docs/changes.md index 6557086..843f2dc 100644 --- a/docs/changes.md +++ b/docs/changes.md @@ -4,6 +4,21 @@ ## Unreleased +- On Linux and macOS, `vfox install python@` now installs a CPython + release from its prebuilt uv-build archive when the uv-build index lists it, + and builds from source only when the index does not. Set + `VFOX_PYTHON_USE_UV_BUILD=0` to keep building every CPython from source, or + `VFOX_PYTHON_USE_UV_BUILD=1` to use uv-build only. GraalPy, PyPy and Conda + installs are unchanged. `vfox search python` lists the pyenv names together + with the CPython releases only uv-build lists. Installs made before this + change are not touched. A `VFOX_PYTHON_MIRROR` setting keeps CPython on the + source path, unless `VFOX_PYTHON_UV_BUILD_MIRROR` is set too. +- In the default mode on Linux and macOS, a partial name such as `python@3.12` + resolves in the uv-build index first, so it installs the newest 3.12 that has an + archive. An exact name still selects any release, including a newer pyenv patch + that the index does not list. +- When the pyenv version index cannot be read, the error names the HTTP status + instead of failing with a Lua concatenation error. - A partial version such as `python@3.12` now selects a linked install named with `-opt`, such as `3.12.8-opt`, and `python@3.13t` selects a linked free-threaded install, such as `3.13.1t-uv`. The source suffix is no longer diff --git a/docs/development.md b/docs/development.md index df17bd9..8200221 100644 --- a/docs/development.md +++ b/docs/development.md @@ -56,8 +56,10 @@ for the older hooks; the newer modules (`python_version`, `detect_python`, It returns `version`, `url`, `headers`, `sha256` and `note`; vfox downloads the URL, verifies `sha256` when present, unpacks the archive and names the install -after `version`. A partial version such as `3.12` is resolved first, to the newest -stable release that `Available({})` names for this mode; an exact version is not +after `version`. A partial version such as `3.12` is resolved first. In the default +mode on Linux and macOS it resolves to the newest match in the uv-build index when +the index has one; otherwise, and in every other mode, it resolves to the newest +stable release that `Available({})` names for this mode. An exact version is not looked up. - `version = "latest"` is resolved to `Available({})[1].version`. The list is @@ -76,6 +78,14 @@ looked up. archive must be HTTPS; an unusable checksum fails the install instead of skipping verification. A GraalPy version is rejected because the index contains CPython only. +- The default mode (no `VFOX_PYTHON_USE_UV_BUILD`, Linux and macOS, and no + `VFOX_PYTHON_MIRROR` unless `VFOX_PYTHON_UV_BUILD_MIRROR` is set) checks the + index for a CPython version only; a non-CPython name is never looked up there. + When the index lists the version, the result is the same as uv-build mode. When + it does not, PreInstall prints a line and returns no archive, so vfox runs the + pyenv source build. A failed index request is an error that names + `VFOX_PYTHON_USE_UV_BUILD=0`. When no libc can be detected, the source build is + used. - `note = "uv-build"`, `"GraalPy"` or `"GraalPy Community"` is shown by vfox. ### PostInstall @@ -92,10 +102,14 @@ directory (`path`) and name (`version`). under its `graalpy*-` directory and runs the bundled `ensurepip`. On Unix it repairs shebangs; on Windows it runs `bin/python.exe` and creates pip shims if the launchers are missing. -- In uv-build mode, the archive has already been unpacked by vfox; the hook - finds the payload under `path/python-`, repairs shebangs on Unix and - ensures pip on Windows. `resolvePythonInstallPath` handles the MSI, uv-build - and GraalPy layouts. +- In uv-build mode, and in the default mode when PreInstall returned an archive, + vfox has already unpacked the archive. PostInstall takes the archive path when + the note that PreInstall set is `uv-build`; vfox passes the note on in `sdkInfo`, + so PostInstall does not ask the index again. A source build stops if a Python + already sits in the install path: vfox clears the directory of a failed or + interrupted install, so that Python came from an archive, and building over it + would replace it silently. The hook repairs shebangs on Unix and ensures pip on + Windows. `resolvePythonInstallPath` handles the MSI, uv-build and GraalPy layouts. - Paths outside `ctx.rootPath` are rejected for uv-build, so a malformed index cannot make the hook write elsewhere. - Unless `VFOX_PYTHON_REGISTER_PY_LAUNCHER` is `0`, `false`, `no` or `off`, the @@ -240,7 +254,7 @@ It checks native argument encoding, MSI target quoting, the `%` junction, exit-code propagation and every failure path. The E2E suite lives in `tests/e2e` and is written in Go. The Linux package -installs CPython from source, through the mirror, through uv-build and GraalPy, +installs CPython from source (`VFOX_PYTHON_USE_UV_BUILD=0`), through the mirror, through uv-build and GraalPy, and also runs `vfox link --detect` and a project `.python-version` on the runner. The Windows package installs CPython and uv-build, installs GraalPy with PEP 514 registration, and checks `.python-version`. Both packages also @@ -326,7 +340,7 @@ updating that tool does not release this plugin. | Variable | Used by | Meaning | | --- | --- | --- | | `VFOX_PYTHON_MIRROR` | PreInstall, PostInstall | python.org mirror; a trailing `/` is added | -| `VFOX_PYTHON_USE_UV_BUILD` | Available, PreInstall, PostInstall | use uv-build archives | +| `VFOX_PYTHON_USE_UV_BUILD` | Available, PreInstall, PostInstall | unset: uv-build for a CPython version the index lists, on Linux and macOS; `0`, `false`, `no` or `off`: source build; `1`, `true`, `yes` or `on`: uv-build only | | `VFOX_PYTHON_UV_LIBC` | Available, PreInstall | `gnu` or `musl` override | | `VFOX_PYTHON_UV_BUILD_MIRROR` | PreInstall | HTTPS mirror for uv-build archives | | `VFOX_PYTHON_REGISTER_PY_LAUNCHER` | PostInstall, PreUninstall | register Windows installs under `HKCU\Software\Python\VersionFox`; on by default, `0`, `false`, `no` or `off` disables | diff --git a/docs/e2e.md b/docs/e2e.md index 25c267e..3def196 100644 --- a/docs/e2e.md +++ b/docs/e2e.md @@ -13,7 +13,7 @@ in Docker containers. `.github/workflows/e2e.yml` only installs Go and runs | --- | --- | | `common/` | builds the plugin zip, installs vfox 1.1.0, runs host commands, reads the uv-build index, asserts | | `linux/e2e_test.go` | `TestNativeE2E`: source build, mirror, uv-build, GraalPy, `vfox link --detect`, `.python-version` | -| `linux/container_test.go` | `TestContainerE2E`: a bare Ubuntu 24.04 container for uv-build, and one for each GraalPy release | +| `linux/container_test.go` | `TestContainerE2E`: a bare Ubuntu 24.04 container for the default install, and one for each GraalPy release | | `linux/Dockerfile` | Ubuntu 24.04 with `ca-certificates`, `curl`, `tar`, `xz-utils` and `zip`; no compiler and no `-dev` packages | | `windows/e2e_test.go` | `TestNativeE2E`: python.org EXE installer, mirror, uv-build, GraalPy with launcher registration, `.python-version` | | `windows/container_test.go` | `TestContainerE2E`: a Server Core container, driven by the docker CLI, with PowerShell 7, the Visual C++ runtime and vfox, GraalPy | @@ -26,8 +26,8 @@ into the container and drive every step from there. Linux, on the runner: -- `source build`: CPython 3.12.2 built from source. -- `mirror`: CPython 3.12.1 from the huaweicloud mirror. +- `source build`: CPython 3.12.2 built from source, with `VFOX_PYTHON_USE_UV_BUILD=0`. +- `mirror`: CPython 3.12.1 built from source through the huaweicloud mirror, with `VFOX_PYTHON_USE_UV_BUILD=0`. - `uv-build`: `python@3.11` installs the newest CPython 3.11 uv-build archive, `vfox use` selects it, and the check reads the full version and pip. - `graalpy`: GraalPy 24.2.0, asserted by implementation name. - `link`: `vfox link --detect python` adopts an installed Python, and `vfox list` shows it as linked. @@ -35,7 +35,7 @@ Linux, on the runner: Linux, in containers: -- `uv-build without build packages`: the uv-build CPython imports `ssl`, `sqlite3`, `bz2`, `lzma`, `ctypes`, `readline`, `curses`, `zlib` and `hashlib`, and pip runs. This is the check behind the answer in [platforms.md](platforms.md#linux-build-dependencies): uv-build needs no apt packages. +- `default install without build packages`: `vfox install` of the exact CPython version that the uv-build index lists, with the default settings, takes the uv-build archive. The installed CPython imports `ssl`, `sqlite3`, `bz2`, `lzma`, `ctypes`, `readline`, `curses`, `zlib` and `hashlib`, and pip runs. This is the check behind the answer in [platforms.md](platforms.md#linux-build-dependencies): a uv-build install needs no apt packages. - `graalpy-24.2.0` and `graalpy-community-24.2.0`: GraalPy installs and runs with pip, and a `.python-version` selects the uv-build CPython over the global GraalPy. Windows, on the runner: diff --git a/docs/mirror.md b/docs/mirror.md index a0b8fae..85996f3 100644 --- a/docs/mirror.md +++ b/docs/mirror.md @@ -64,6 +64,14 @@ unpacking. A mirror that serves different bytes fails the install with `Checksum error`, which is the intended behaviour: change the index or the mirror, do not bypass the check. +On Linux and macOS, the default mode takes this mirror for every CPython version +the index lists, when `VFOX_PYTHON_MIRROR` is not set. A `VFOX_PYTHON_MIRROR` +setting keeps CPython on the python.org mirror and builds it from source, unless +`VFOX_PYTHON_UV_BUILD_MIRROR` is set too. A source build still clones pyenv from +GitHub, and neither mirror variable changes that clone. Set +`VFOX_PYTHON_USE_UV_BUILD=0` to build from source, or `VFOX_PYTHON_USE_UV_BUILD=1` +to use uv-build only. + ## GraalPy Neither mirror variable applies to GraalPy. Every platform downloads Oracle's diff --git a/docs/platforms.md b/docs/platforms.md index 38631c6..63a33f5 100644 --- a/docs/platforms.md +++ b/docs/platforms.md @@ -4,14 +4,17 @@ ## What is installed -| Platform | Default | `VFOX_PYTHON_USE_UV_BUILD=1` | -| --- | --- | --- | -| Linux x86_64 | source build with pyenv for CPython; Oracle archive for GraalPy | prebuilt uv-build archive | -| Linux arm64 | source build with pyenv for CPython; Oracle archive for GraalPy | prebuilt uv-build archive | -| Linux armv7 | source build with pyenv for CPython; no GraalPy build | prebuilt uv-build archive when available | -| Linux musl / Alpine | source build with pyenv | prebuilt musl archive | -| macOS Intel and Apple silicon | source build with pyenv for CPython; Oracle archive for GraalPy | prebuilt uv-build archive | -| Windows x86_64, arm64, 386 | python.org EXE or MSI for CPython; GraalPy amd64 zip | prebuilt uv-build archive | +| Platform | Default | `VFOX_PYTHON_USE_UV_BUILD=0` | `VFOX_PYTHON_USE_UV_BUILD=1` | +| --- | --- | --- | --- | +| Linux x86_64 | prebuilt uv-build archive for CPython when the index lists the version, otherwise a source build with pyenv; Oracle archive for GraalPy | source build with pyenv for CPython; Oracle archive for GraalPy | prebuilt uv-build archive | +| Linux arm64 | same as Linux x86_64 | same as Linux x86_64 | prebuilt uv-build archive | +| Linux armv7 | same as Linux x86_64, with no GraalPy build | source build with pyenv; no GraalPy build | prebuilt uv-build archive when available | +| Linux musl / Alpine | prebuilt musl archive when the index lists the version, otherwise a source build with pyenv | source build with pyenv | prebuilt musl archive | +| macOS Intel and Apple silicon | same as Linux x86_64 | source build with pyenv for CPython; Oracle archive for GraalPy | prebuilt uv-build archive | +| Windows x86_64, arm64, 386 | python.org EXE or MSI for CPython; GraalPy amd64 zip | python.org EXE or MSI for CPython; GraalPy amd64 zip | prebuilt uv-build archive | + +In the default mode, a `VFOX_PYTHON_MIRROR` setting keeps CPython on the source +path, unless `VFOX_PYTHON_UV_BUILD_MIRROR` is also set. The CPython source build takes the pyenv tag and version list, clones pyenv at that tag, and runs its `python-build`. It needs a C toolchain and the headers @@ -36,7 +39,9 @@ amd64. filename; use uv-build for a pre-release on Windows. - **uv-build:** CPython archives that the index has for the platform, architecture and libc, including `freethreaded` builds. GraalPy is not - available in this mode. + available in this mode. In the default mode on Linux and macOS, `vfox search + python` lists the pyenv names together with these CPython releases; a version + the index lacks is built from source. A free-threaded release is selected with the same `t` suffix on every source: @@ -47,26 +52,28 @@ VFOX_PYTHON_USE_UV_BUILD=1 vfox install python@3.13t A partial name resolves in each mode: `python@3.13t` is the newest free-threaded 3.13 release that the mode lists, and a plain `python@3.13` never picks a -free-threaded build. +free-threaded build. In the default mode on Linux and macOS, the uv-build index is +searched first; see [uv.md](uv.md). Windows free-threaded releases come through uv-build; the stable installer list does not include the `t` builds. ## Linux build dependencies -Only a **source build** needs the packages in this section, and a source build -is the default for CPython on Linux. Two other paths need none of them: +Only a **source build** needs the packages in this section. A CPython install +from uv-build needs none of them, and uv-build is the default for a version the +index lists: | What you install | Needs these packages? | | --- | --- | -| CPython, source build (default) | yes | -| CPython, with `VFOX_PYTHON_USE_UV_BUILD=1` | no | +| CPython, source build (a version the index lacks, or `VFOX_PYTHON_USE_UV_BUILD=0`) | yes | +| CPython, uv-build (the default for a listed version, or `VFOX_PYTHON_USE_UV_BUILD=1`) | no | | GraalPy | no | uv-build downloads a prebuilt CPython archive and GraalPy is an Oracle archive, -so neither runs a compiler. The Linux container suite checks the uv-build case: +so neither runs a compiler. The Linux container suite checks the default case: it starts an Ubuntu 24.04 image with no compiler and no `-dev` packages, -installs a uv-build CPython, and imports `ssl`, `sqlite3`, `bz2`, `lzma`, +installs a CPython from the uv-build index, and imports `ssl`, `sqlite3`, `bz2`, `lzma`, `ctypes`, `readline`, `curses`, `zlib` and `hashlib`. On Ubuntu, the source build needs these packages: @@ -84,15 +91,17 @@ builds no `zlib` module, and `make install` then fails in ensurepip: pip is installed from a wheel, which is a zip archive and needs zlib to be read (`No module named 'zlib'`). -To avoid that step on Ubuntu, use uv-build. It needs no `sudo` and no `apt`: +To avoid that step on Ubuntu, install a version the uv-build index lists. It +needs no `sudo` and no `apt`, and it is the default on Linux: ```bash -export VFOX_PYTHON_USE_UV_BUILD=1 -vfox install python@3.12 +vfox install python@3.12.16 +vfox use -g python@3.12.16 ``` -`python@3.12` installs the newest 3.12 release in the uv-build index, and -`vfox use -g python@3.12` selects it. +A partial name such as `python@3.12` picks the newest 3.12 that the uv-build index +lists, and installs its archive. A newer pyenv patch that the index does not list is +built from source, and only its exact name selects it. Fedora and RHEL, for a source build: @@ -216,7 +225,9 @@ https://vault.vfox.dev/python/uv-build?os=&arch=-amd64.exe`, `python-.exe` on 386, or `python--arm64.exe`, and the matching @@ -171,8 +178,9 @@ macOS it is a `.tar.gz`; on Windows it is a `graalpy*-windows-amd64.zip`. In uv-build mode, `display_version` wins when the index provides it; otherwise a `freethreaded` build gets the `t` suffix, so `3.13.1` with `variant = "freethreaded"` is `3.13.1t`. Only CPython builds with an HTTPS archive URL are -listed. A GraalPy version is rejected with a message that asks you to unset -`VFOX_PYTHON_USE_UV_BUILD`. See [uv.md](uv.md). +listed. With `VFOX_PYTHON_USE_UV_BUILD=1`, a GraalPy version is rejected with a +message that asks you to unset `VFOX_PYTHON_USE_UV_BUILD` or set it to `0`. See +[uv.md](uv.md). A partial name such as `3.12` or `3.13t` resolves to the newest matching release in the index, and the install is recorded under the full name, such as `3.12.16`. diff --git a/docs/zh-CN/changes.md b/docs/zh-CN/changes.md index 7259d3f..8c7b2d7 100644 --- a/docs/zh-CN/changes.md +++ b/docs/zh-CN/changes.md @@ -4,6 +4,15 @@ ## 未发布 +- 在 Linux 和 macOS 上,`vfox install python@<版本>` 现在会在 uv-build 索引列出某个 CPython 版本时, + 从它的 uv-build 预编译包安装;只有索引没有列出时才从源码构建。设置 `VFOX_PYTHON_USE_UV_BUILD=0` + 可以让每个 CPython 都继续从源码构建,设置 `VFOX_PYTHON_USE_UV_BUILD=1` 则只使用 uv-build。GraalPy、 + PyPy 和 Conda 的安装保持不变。`vfox search python` 会把 pyenv 的名字和只有 uv-build 列出的 CPython 版本一起列出。 + 此变更之前已完成的安装不会被改动。若设置了 `VFOX_PYTHON_MIRROR`,CPython 保持走源码构建的路径, + 除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`。 +- 在 Linux 和 macOS 的默认模式下,部分版本号(如 `python@3.12`)先在 uv-build 索引里解析, + 所以会安装索引列出的最新 3.12 预编译包。精确版本号仍可选中任何版本,包括索引里没有的较新 pyenv 补丁版。 +- 无法读取 pyenv 版本索引时,报错会写出 HTTP 状态码,而不是因为 Lua 字符串拼接错误而失败。 - 部分版本号(如 `python@3.12`)现在会选中名字带 `-opt` 的链接安装,如 `3.12.8-opt`; `python@3.13t` 会选中自由线程的链接安装,如 `3.13.1t-uv`。来源后缀不再被当作版本的一部分。 - `.python-version` 中的部分版本号(如 `3.13`)与 `vfox use` 遵循同一规则:不再选中自由线程 diff --git a/docs/zh-CN/development.md b/docs/zh-CN/development.md index 5467015..ee8c95f 100644 --- a/docs/zh-CN/development.md +++ b/docs/zh-CN/development.md @@ -54,8 +54,8 @@ vfox 加载 `metadata.lua`,命令需要时调用相应的钩子,并向钩子 ### PreInstall 返回 `version`、`url`、`headers`、`sha256` 和 `note`;vfox 下载 URL,有 -`sha256` 时校验,解压后按 `version` 命名。部分版本号(如 `3.12`)会先解析成 `Available({})` 里该模式最新的 -稳定版;精确版本号不会查列表。 +`sha256` 时校验,解压后按 `version` 命名。部分版本号(如 `3.12`)会先解析。在 Linux 和 macOS 的默认模式下,若 uv-build 索引里有匹配版本, +就解析成它;否则(以及其他所有模式下)解析成 `Available({})` 里该模式最新的稳定版。精确版本号不会查列表。 - `version = "latest"` 会解析成 `Available({})[1].version`。CPython 在列表最 前面、从新到旧排列,所以可能是预发布版;GraalPy 要用完整版本名。 @@ -68,6 +68,10 @@ vfox 加载 `metadata.lua`,命令需要时调用相应的钩子,并向钩子 使用前校验对应的 `.sha256`。 - uv-build 模式返回索引里的包地址和校验和。地址必须是 HTTPS;校验和不可用时 安装失败,而不是跳过校验。GraalPy 会被拒绝,因为索引里只有 CPython。 +- 默认模式(未设置 `VFOX_PYTHON_USE_UV_BUILD`,在 Linux 和 macOS 上,且未设置 `VFOX_PYTHON_MIRROR`,除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`)只在索引里查 CPython 版本, + 非 CPython 的名字从不在那里查找。索引列出该版本时,结果与 uv-build 模式相同。索引没有列出时, + PreInstall 会打印一行,并且不返回包,于是 vfox 运行 pyenv 源码构建。索引请求失败会报错, + 错误信息里写明 `VFOX_PYTHON_USE_UV_BUILD=0`。检测不到 libc 时,使用源码构建。 - `note = "uv-build"`、`"GraalPy"` 或 `"GraalPy Community"` 会显示在 vfox 里。 ### PostInstall @@ -83,8 +87,11 @@ vfox 加载 `metadata.lua`,命令需要时调用相应的钩子,并向钩子 - GraalPy 压缩包已经由 vfox 解压;钩子在 `graalpy*-<平台>` 目录下找 payload, 运行包里自带的 `ensurepip`。Unix 上修 shebang;Windows 上用 `bin/python.exe` 运行,缺少启动器时创建 pip shim。 -- uv-build 模式下,包已经由 vfox 解压;钩子在 `path/python-<版本>` 下找 - payload,Unix 上修 shebang,Windows 上确保 pip 可用。 +- uv-build 模式下,以及默认模式里 PreInstall 返回了归档的情况下,包已经由 vfox 解压。 + PostInstall 在 PreInstall 设置的 `note` 为 `uv-build` 时走归档路径;vfox 会把 `note` 放进 + `sdkInfo` 传给 PostInstall,所以 PostInstall 不会再问一次索引。如果安装目录里已经有 Python, + 源码构建会停止:vfox 会清理失败或中断的安装目录,因此那个 Python 来自归档,直接覆盖会悄悄替换掉它。 + 钩子负责在 Unix 上修 shebang,在 Windows 上确保 pip 可用。 `resolvePythonInstallPath` 处理 MSI、uv-build 和 GraalPy 三种布局。 - uv-build 会拒绝 `ctx.rootPath` 之外的路径,避免错误的索引让钩子写到别处。 - 除非 `VFOX_PYTHON_REGISTER_PY_LAUNCHER` 设为 `0`、`false`、`no` 或 `off`,钩子都会按 @@ -200,7 +207,7 @@ local versions = plugin:Available({}) `%` junction、退出码传递和每条失败路径。 E2E 套件在 `tests/e2e` 里,使用 Go 编写。Linux 包在 runner 上安装 CPython -源码构建、镜像构建、uv-build 和 GraalPy,还会运行 `vfox link --detect` 和项目 +源码构建(`VFOX_PYTHON_USE_UV_BUILD=0`)、镜像构建、uv-build 和 GraalPy,还会运行 `vfox link --detect` 和项目 的 `.python-version`。Windows 包安装 CPython 和 uv-build,安装带 PEP 514 注册的 GraalPy,并检查 `.python-version`。两个包也在 Docker 容器里跑 GraalPy 检查:Linux 包通过 testcontainers-go,Windows 包直接调用 `docker` 命令行,因为 testcontainers-go 会为每个容器 @@ -276,7 +283,7 @@ vfox 的全局变量,只关闭 `duplicate-set-field`(测试会有意替换 ` | 变量 | 使用者 | 含义 | | --- | --- | --- | | `VFOX_PYTHON_MIRROR` | PreInstall、PostInstall | python.org 镜像;结尾没有 `/` 会自动补上 | -| `VFOX_PYTHON_USE_UV_BUILD` | Available、PreInstall、PostInstall | 使用 uv-build 预编译包 | +| `VFOX_PYTHON_USE_UV_BUILD` | Available、PreInstall、PostInstall | 未设置:在 Linux 和 macOS 上,索引列出的 CPython 版本用 uv-build;`0`、`false`、`no` 或 `off`:源码构建;`1`、`true`、`yes` 或 `on`:只用 uv-build | | `VFOX_PYTHON_UV_LIBC` | Available、PreInstall | 覆盖 `gnu` / `musl` 识别 | | `VFOX_PYTHON_UV_BUILD_MIRROR` | PreInstall | uv-build 包的 HTTPS 镜像 | | `VFOX_PYTHON_REGISTER_PY_LAUNCHER` | PostInstall、PreUninstall | 把 Windows 安装注册到 `HKCU\Software\Python\VersionFox`;默认开启,`0`、`false`、`no`、`off` 关闭 | diff --git a/docs/zh-CN/e2e.md b/docs/zh-CN/e2e.md index 700d729..878767b 100644 --- a/docs/zh-CN/e2e.md +++ b/docs/zh-CN/e2e.md @@ -12,7 +12,7 @@ | --- | --- | | `common/` | 构建插件 zip、安装 vfox 1.1.0、运行主机命令、读取 uv-build 索引、断言 | | `linux/e2e_test.go` | `TestNativeE2E`:源码构建、镜像、uv-build、GraalPy、`vfox link --detect`、`.python-version` | -| `linux/container_test.go` | `TestContainerE2E`:一个没有构建包的 Ubuntu 24.04 容器跑 uv-build,每个 GraalPy 版本各一个容器 | +| `linux/container_test.go` | `TestContainerE2E`:一个没有构建包的 Ubuntu 24.04 容器跑默认安装,每个 GraalPy 版本各一个容器 | | `linux/Dockerfile` | Ubuntu 24.04,只装 `ca-certificates`、`curl`、`tar`、`xz-utils` 和 `zip`;没有编译器,也没有 `-dev` 包 | | `windows/e2e_test.go` | `TestNativeE2E`:python.org EXE 安装包、镜像、uv-build、带启动器注册的 GraalPy、`.python-version` | | `windows/container_test.go` | `TestContainerE2E`:由 docker 命令行驱动的 Server Core 容器,装有 PowerShell 7、Visual C++ 运行时和 vfox,跑 GraalPy | @@ -24,8 +24,8 @@ Dockerfile 只负责准备环境。Go 测试把插件复制进容器,然后在 Linux,在 runner 上: -- `source build`:从源码构建 CPython 3.12.2。 -- `mirror`:从华为云镜像安装 CPython 3.12.1。 +- `source build`:从源码构建 CPython 3.12.2,设置 `VFOX_PYTHON_USE_UV_BUILD=0`。 +- `mirror`:通过华为云镜像从源码构建 CPython 3.12.1,设置 `VFOX_PYTHON_USE_UV_BUILD=0`。 - `uv-build`:`python@3.11` 安装最新的 CPython 3.11 uv-build 包,`vfox use` 选中它,检查完整版本号和 pip。 - `graalpy`:GraalPy 24.2.0,按实现名断言。 - `link`:`vfox link --detect python` 纳入已安装的 Python,`vfox list` 显示为已链接。 @@ -34,9 +34,9 @@ Linux,在 runner 上: Linux,在容器里: -- `uv-build without build packages`:uv-build 的 CPython 能导入 `ssl`、`sqlite3`、`bz2`、 - `lzma`、`ctypes`、`readline`、`curses`、`zlib` 和 `hashlib`,pip 也能运行。这是 - [平台](platforms.md#linux-构建依赖) 里“uv-build 不需要 apt 包”这一说法的依据。 +- `default install without build packages`:用默认设置对 uv-build 索引列出的某个 CPython 精确版本执行 `vfox install`,会使用 uv-build 包。 + 安装好的 CPython 能导入 `ssl`、`sqlite3`、`bz2`、`lzma`、`ctypes`、`readline`、`curses`、`zlib` 和 `hashlib`,pip 也能运行。这是 + [平台](platforms.md#linux-构建依赖) 里“uv-build 安装不需要 apt 包”这一说法的依据。 - `graalpy-24.2.0` 和 `graalpy-community-24.2.0`:GraalPy 能安装并运行、pip 可用,并且 `.python-version` 能让 uv-build 的 CPython 优先于全局的 GraalPy。 diff --git a/docs/zh-CN/mirror.md b/docs/zh-CN/mirror.md index 8b002cf..c05c72c 100644 --- a/docs/zh-CN/mirror.md +++ b/docs/zh-CN/mirror.md @@ -59,6 +59,11 @@ export VFOX_PYTHON_UV_BUILD_MIRROR=https://registry.npmmirror.com/-/binary/pytho 解压前,插件会用 uv-build 索引里的 SHA-256 校验每个包。镜像提供的字节不一致时 安装会以 `Checksum error` 失败,这是有意的:应该改索引或镜像,不要绕过校验。 +在 Linux 和 macOS 上,未设置 `VFOX_PYTHON_MIRROR` 时,默认模式会对索引列出的每个 CPython 版本使用这个镜像。 +若设置了 `VFOX_PYTHON_MIRROR`,CPython 会保持经 python.org 镜像从源码构建,除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`。 +源码构建仍然从 GitHub 克隆 pyenv,两个镜像变量都不会改变这次克隆。设置 `VFOX_PYTHON_USE_UV_BUILD=0` 可以从源码构建; +设置 `VFOX_PYTHON_USE_UV_BUILD=1` 则只使用 uv-build。 + ## GraalPy 两个镜像变量都不适用于 GraalPy。所有平台都直接从 GitHub 下载 Oracle 的发布 diff --git a/docs/zh-CN/platforms.md b/docs/zh-CN/platforms.md index 185c5c9..084dba0 100644 --- a/docs/zh-CN/platforms.md +++ b/docs/zh-CN/platforms.md @@ -4,14 +4,16 @@ ## 安装的是什么 -| 平台 | 默认 | `VFOX_PYTHON_USE_UV_BUILD=1` | -| --- | --- | --- | -| Linux x86_64 | CPython 用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | uv-build 预编译包 | -| Linux arm64 | CPython 用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | uv-build 预编译包 | -| Linux armv7 | CPython 用 pyenv 从源码编译;没有 GraalPy 构建 | 有包时用 uv-build 预编译包 | -| Linux musl / Alpine | 用 pyenv 从源码编译 | uv-build 的 musl 预编译包 | -| macOS Intel 和 Apple silicon | CPython 用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | uv-build 预编译包 | -| Windows x86_64、arm64、386 | CPython 用 python.org 的 EXE 或 MSI;GraalPy 用 amd64 压缩包 | uv-build 预编译包 | +| 平台 | 默认 | `VFOX_PYTHON_USE_UV_BUILD=0` | `VFOX_PYTHON_USE_UV_BUILD=1` | +| --- | --- | --- | --- | +| Linux x86_64 | CPython 在 uv-build 索引列出该版本时用 uv-build 预编译包,否则用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | CPython 用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | uv-build 预编译包 | +| Linux arm64 | 同 Linux x86_64 | 同 Linux x86_64 | uv-build 预编译包 | +| Linux armv7 | 同 Linux x86_64,没有 GraalPy 构建 | 用 pyenv 从源码编译;没有 GraalPy 构建 | 有包时用 uv-build 预编译包 | +| Linux musl / Alpine | 索引列出该版本时用 uv-build 的 musl 预编译包,否则用 pyenv 从源码编译 | 用 pyenv 从源码编译 | uv-build 的 musl 预编译包 | +| macOS Intel 和 Apple silicon | 同 Linux x86_64 | CPython 用 pyenv 从源码编译;GraalPy 用 Oracle 压缩包 | uv-build 预编译包 | +| Windows x86_64、arm64、386 | CPython 用 python.org 的 EXE 或 MSI;GraalPy 用 amd64 压缩包 | CPython 用 python.org 的 EXE 或 MSI;GraalPy 用 amd64 压缩包 | uv-build 预编译包 | + +在默认模式下,若设置了 `VFOX_PYTHON_MIRROR`,CPython 会保持走源码构建的路径,除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`。 CPython 的源码构建取 pyenv 的 tag 和版本列表,把 pyenv 克隆到那个 tag,再运行 它的 `python-build`。它需要 C 工具链和 Python 链接的头文件。Windows 则下载 python.org @@ -30,7 +32,8 @@ macOS 有 amd64 和 arm64 资源,Windows 只有 amd64。 安全版本不在列表里,2.5 以前的版本也不列出;预发布安装包的文件名匹配不上稳定 的精确文件名,Windows 上需要预发布版时请用 uv-build。 - **uv-build:** 索引里当前平台、架构和 libc 有的 CPython 包,包括 - `freethreaded` 构建。GraalPy 不在此模式中。 + `freethreaded` 构建。GraalPy 不在此模式中。在 Linux 和 macOS 的默认模式下, + `vfox search python` 会把 pyenv 的名字和这些 CPython 版本一起列出;索引没有的版本则从源码编译。 自由线程版本在每种来源上都用同样的 `t` 后缀选中: @@ -40,25 +43,26 @@ VFOX_PYTHON_USE_UV_BUILD=1 vfox install python@3.13t ``` 部分版本号在每种模式下都会解析:`python@3.13t` 选出该模式列出的、最新的 3.13 自由线程版本; -只写 `python@3.13` 则不会选中自由线程构建。 +只写 `python@3.13` 则不会选中自由线程构建。在 Linux 和 macOS 的默认模式下,先搜索 uv-build 索引, +详见 [uv.md](uv.md)。 Windows 的自由线程版本通过 uv-build 提供;稳定安装包列表里没有 `t` 构建。 ## Linux 构建依赖 -只有 **源码构建** 需要本节列出的包,而源码构建是 Linux 上 CPython 的默认方式。另外两条 -路径都不需要这些包: +只有 **源码构建** 需要本节列出的包。从 uv-build 安装的 CPython 不需要这些包, +索引列出的版本默认就用 uv-build: | 安装的内容 | 需要这些包吗? | | --- | --- | -| CPython,源码构建(默认) | 需要 | -| CPython,设置 `VFOX_PYTHON_USE_UV_BUILD=1` | 不需要 | +| CPython,源码构建(索引没有的版本,或设置 `VFOX_PYTHON_USE_UV_BUILD=0`) | 需要 | +| CPython,uv-build(索引列出的版本默认使用,或设置 `VFOX_PYTHON_USE_UV_BUILD=1`) | 不需要 | | GraalPy | 不需要 | uv-build 下载的是预编译的 CPython 包,GraalPy 是 Oracle 的压缩包,两者都不运行编译器。 -Linux 容器套件会检查 uv-build 这条路径:它启动一个没有编译器、也没有 `-dev` 包的 -Ubuntu 24.04 镜像,安装 uv-build 的 CPython,然后导入 `ssl`、`sqlite3`、`bz2`、`lzma`、 -`ctypes`、`readline`、`curses`、`zlib` 和 `hashlib`。 +Linux 容器套件会检查默认的情况:它启动一个没有编译器、也没有 `-dev` 包的 +Ubuntu 24.04 镜像,从 uv-build 索引安装一个 CPython,然后导入 `ssl`、`sqlite3`、`bz2`、 +`lzma`、`ctypes`、`readline`、`curses`、`zlib` 和 `hashlib`。 在 Ubuntu 上做源码构建,需要安装这些包: @@ -74,14 +78,14 @@ sudo apt install --assume-yes make build-essential libssl-dev zlib1g-dev \ 随后会在 ensurepip 步骤失败:pip 是从 wheel 安装的,而 wheel 是 zip 压缩包,读取它需要 zlib (报错为 `No module named 'zlib'`)。 -想在 Ubuntu 上省掉这一步,就用 uv-build。它不需要 `sudo`,也不需要 `apt`: +想在 Ubuntu 上省掉这一步,就安装 uv-build 索引列出的版本。它不需要 `sudo`,也不需要 `apt`,在 Linux 上也是默认方式: ```bash -export VFOX_PYTHON_USE_UV_BUILD=1 -vfox install python@3.12 +vfox install python@3.12.16 +vfox use -g python@3.12.16 ``` -`python@3.12` 会安装 uv-build 索引里最新的 3.12 版本,`vfox use -g python@3.12` 选中它。 +部分版本号(如 `python@3.12`)会选中 uv-build 索引列出的最新 3.12 版本,并安装它的预编译包。pyenv 列出的、索引里没有的较新补丁版会从源码编译,只有写出它的精确版本号才会选中它。 Fedora 和 RHEL 的源码构建: diff --git a/docs/zh-CN/uv.md b/docs/zh-CN/uv.md index 0f245f6..5462dca 100644 --- a/docs/zh-CN/uv.md +++ b/docs/zh-CN/uv.md @@ -11,16 +11,29 @@ ## uv-build -安装前设置 `VFOX_PYTHON_USE_UV_BUILD=1`。部分版本号会解析成 uv-build 索引里最新的对应版本, -所以 `python@3.12` 装的是最新的 3.12,`python@3.13t` 装的是最新的 3.13 自由线程版本: +在 Linux 和 macOS 上,CPython 默认使用 uv-build:索引列出该版本时,插件安装它的预编译包; +否则用 pyenv 从源码编译。部分版本号先在索引里解析,所以 `python@3.12` 装的是索引列出的最新 3.12, +`python@3.13t` 装的是索引列出的最新 3.13 自由线程版本。只有索引没有列出的版本系列,才会改用 +`vfox search python` 列出的列表,并从源码编译: ```bash -export VFOX_PYTHON_USE_UV_BUILD=1 vfox install python@3.12 ``` -这条路径不需要源码构建所需的任何 `apt` 包,也不需要 `sudo`。在 Ubuntu 上,跳过 -[平台](platforms.md#linux-构建依赖) 里的 apt 步骤,设置这个变量后直接安装即可。 +这条路径不需要源码构建所需的任何 `apt` 包,也不需要 `sudo`。源码构建仍然需要它们, +见 [平台](platforms.md#linux-构建依赖)。 + +索引只列出部分版本,通常是每个版本线的最新补丁版。pyenv 列出的补丁版比索引新时,`python@3.12` 仍然安装索引列出的最新补丁版;较新的补丁版只有写出精确版本号时才会从源码编译。 + +`VFOX_PYTHON_USE_UV_BUILD` 决定使用哪种模式: + +- 未设置:索引列出的 CPython 版本用 uv-build,其他 CPython 版本用源码构建;GraalPy、PyPy 和 Conda 保持各自的路径。当设置了 `VFOX_PYTHON_MIRROR` + 且未设置 `VFOX_PYTHON_UV_BUILD_MIRROR` 时,CPython 是经 python.org 镜像的源码构建, + 因为 uv-build 的下载会来自 GitHub。 +- `0`、`false`、`no` 或 `off`:在 Linux 和 macOS 上源码构建。 +- `1`、`true`、`yes` 或 `on`:只用 uv-build。索引里没有列出的版本会报错。 + +如果无法读取索引,安装会失败,并提示 `VFOX_PYTHON_USE_UV_BUILD=0`,因此不会在没有说明的情况下开始源码构建。如果索引在该平台上没有这个版本的包(比如索引不覆盖某个架构),安装会改为源码构建,并打印一行说明。在 Linux 上,如果无法识别 libc,安装会从源码构建;要使用 uv-build,请设置 `VFOX_PYTHON_UV_LIBC`。 插件向 `https://vault.vfox.dev/python/uv-build` 询问当前平台、架构和 libc 的 构建,然后把包地址和 SHA-256 交给 vfox。vfox 下载、校验、解压,再调用 @@ -30,10 +43,10 @@ vfox install python@3.12 - Windows 上,如果包里带了 `ensurepip` 却没有 pip 命令,会安装 pip,并生成 `pip.cmd`、`pip3.cmd` 两个 shim。 -列表里只有 CPython 的包。GraalPy 和其他实现不在 uv-build 里;在这个模式下 -执行 `vfox install python@graalpy-24.2.0` 会失败,并提示取消 -`VFOX_PYTHON_USE_UV_BUILD`。GraalPy 的安装路径见 [GraalPy](graalpy.md)。插件按 -这个顺序优先: +列表里只有 CPython 的包。GraalPy 和其他实现不在 uv-build 里。设置 +`VFOX_PYTHON_USE_UV_BUILD=1` 时,执行 `vfox install python@graalpy-24.2.0` 会失败, +并提示取消该变量或把它设为 `0`。不设置该变量时,GraalPy 从 Oracle 的压缩包安装, +见 [GraalPy](graalpy.md)。插件按这个顺序优先: 1. `install_only_stripped` 2. `install_only` @@ -50,7 +63,7 @@ vfox install python@3.12 | 变量 | 作用 | | --- | --- | -| `VFOX_PYTHON_USE_UV_BUILD` | `1`、`true`、`yes` 或 `on` 时改用 uv-build | +| `VFOX_PYTHON_USE_UV_BUILD` | 未设置:索引列出该 CPython 版本时用 uv-build,否则源码构建。`0`、`false`、`no` 或 `off`:源码构建。`1`、`true`、`yes` 或 `on`:只用 uv-build | | `VFOX_PYTHON_UV_LIBC` | `gnu` 或 `musl`,覆盖 Linux 的识别结果 | | `VFOX_PYTHON_UV_BUILD_MIRROR` | 让预编译包走镜像下载 | diff --git a/docs/zh-CN/versions.md b/docs/zh-CN/versions.md index 07b2438..53eb967 100644 --- a/docs/zh-CN/versions.md +++ b/docs/zh-CN/versions.md @@ -26,7 +26,7 @@ vfox install python@3.12.2 # 正好这个版本 vfox use python@3.12 # 已安装的 3.12.x 里最新的 ``` -部分版本号两边都会解析。`PreInstall` 选出版本列表里最新的稳定版;`PreUse` 选出已安装里 +部分版本号两边都会解析。`PreInstall` 选出版本列表里最新的稳定版;在 Linux 和 macOS 的默认模式下,它先选出 uv-build 索引里最新的匹配版本,索引里没有时才用版本列表;`PreUse` 选出已安装里 最新的匹配项,同一版本时裸名字优先于带 `-system` 等来源后缀的名字。普通请求从不选自由线程 (`t`)或预发布版,所以 `python@3.13` 不会选到 `python@3.13t`。 @@ -38,10 +38,13 @@ vfox use python@3.12 # 已安装的 3.12.x 里最新的 | 平台 | 来源 | 列出什么 | | --- | --- | --- | -| Linux、macOS | 插件里的 pyenv 索引 | pyenv 收录的全部版本,包括 CPython、GraalPy、PyPy 和 Conda | +| Linux、macOS | 插件里的 pyenv 索引;默认模式下还包括 uv-build 索引 | pyenv 收录的全部版本,包括 CPython、GraalPy、PyPy 和 Conda;默认模式下还包括只有 uv-build 列出的 CPython 版本 | | Windows | python.org 下载页和 Oracle 的 GraalPy 发布 | 当前架构的稳定 CPython 安装包,以及 GraalPy 的 Windows amd64 版本 | | 任意平台,`VFOX_PYTHON_USE_UV_BUILD=1` | uv-build 索引 | 当前平台、架构和 libc 的 CPython 包 | +在 Linux 和 macOS 的默认模式下,两个列表都列出的 CPython 版本只列一次。最新的版本排在最前面, +正式版排在它的预发布版和自由线程构建之前。每个安装用哪种来源,由 PreInstall 决定,而不是由列表决定:见 [uv.md](uv.md)。 + Windows 上,插件只读一次下载页,然后按当前架构只留下安装包名完全匹配的条目: `python-<版本>-amd64.exe`,386 上是 `python-<版本>.exe`,arm64 上是 `python-<版本>-arm64.exe`,以及对应的 MSI 名。只有源码的安全版本、嵌入式 zip、 @@ -146,7 +149,7 @@ SHA-256 后解压。Linux 和 macOS 上是 `.tar.gz`,Windows 上是 uv-build 模式下,索引提供 `display_version` 时用它;否则 `freethreaded` 构建会加 `t`,也就是 `3.13.1` 配 `variant = "freethreaded"` 时叫 `3.13.1t`。 -列表里只保留有 HTTPS 包地址的 CPython 构建。GraalPy 版本会被拒绝,并提示取消 -`VFOX_PYTHON_USE_UV_BUILD`。详见 [uv.md](uv.md)。 +列表里只保留有 HTTPS 包地址的 CPython 构建。设置 `VFOX_PYTHON_USE_UV_BUILD=1` 时, +GraalPy 版本会被拒绝,并提示取消 `VFOX_PYTHON_USE_UV_BUILD` 或把它设为 `0`。详见 [uv.md](uv.md)。 部分版本号(如 `3.12`、`3.13t`)会解析为索引里最新的匹配版本,安装记录的是完整版本名(如 `3.12.16`)。 diff --git a/hooks/available.lua b/hooks/available.lua index 1cada93..9fc2f07 100644 --- a/hooks/available.lua +++ b/hooks/available.lua @@ -31,7 +31,11 @@ function PLUGIN:Available(ctx) print("Warning: Could not list GraalPy releases: " .. tostring(graalPy)) end return result - else - return parseVersionFromPyenv() end + + if uvBuildAutoEnabled() then + return parseVersionFromAuto() + end + + return parseVersionFromPyenv() end diff --git a/hooks/post_install.lua b/hooks/post_install.lua index 7cd77a9..7ff895d 100644 --- a/hooks/post_install.lua +++ b/hooks/post_install.lua @@ -19,7 +19,7 @@ function PLUGIN:PostInstall(ctx) local version = ctx.sdkInfo["python"].version if isGraalPyVersion(version) and parseGraalPyVersion(version) then result = graalPyInstall(ctx) - elseif useUvBuild() then + elseif useUvBuild() or uvBuildNoteArchive(ctx) then result = uvBuildInstall(ctx) elseif OS_TYPE == "windows" then result = windowsInstall(ctx) diff --git a/hooks/pre_install.lua b/hooks/pre_install.lua index 307ff2b..b5504a5 100644 --- a/hooks/pre_install.lua +++ b/hooks/pre_install.lua @@ -15,14 +15,24 @@ require("util") local python_version = require("python_version") ---- Resolves a partial request such as "3.12" to the newest stable release that ---- the version list for this mode names. An exact request is returned without ---- reading the list. A partial request that matches nothing is returned as it ---- is, so the install reports the request the user typed. +--- Resolves a partial request such as "3.12" to a release. In the default mode on +--- Linux and macOS the uv-build index is searched first, so the newest release it +--- lists on that line is installed as an archive. Only a line the index does not +--- list falls back to the list Available returns, and is then built from source. +--- An exact request is returned without reading any list. A partial request that +--- matches nothing is returned as it is, so the install reports the request the +--- user typed. local function resolvePartialVersion(plugin, requested) if not python_version.is_partial(requested) then return requested end + if uvBuildAutoEnabled() then + local indexNames = uvBuildAutoVersionSet() + local match = indexNames and python_version.find_release(indexNames, requested) + if match then + return match + end + end local names = {} for _, item in ipairs(plugin:Available({})) do names[item.version] = true @@ -41,7 +51,7 @@ function PLUGIN:PreInstall(ctx) if useUvBuild() then if isGraalPyVersion(version) then error( - "uv-build provides CPython only; unset VFOX_PYTHON_USE_UV_BUILD to install " + "uv-build provides CPython only; unset VFOX_PYTHON_USE_UV_BUILD (or set it to 0) to install " .. version .. " from pyenv or Oracle's GraalPy releases." ) @@ -60,6 +70,15 @@ function PLUGIN:PreInstall(ctx) -- pyenv's python-build definition. end + -- Linux and macOS by default: a CPython release that the uv-build index lists + -- is installed from its archive, and any other release is built from source. + if uvBuildAutoEnabled() and string.match(version, "^%d") then + local package = uvBuildAutoPreInstall(version) + if package ~= nil then + return package + end + end + if OS_TYPE == "windows" and not checkIsReleaseVersion(version) then error("The current version is not released") return diff --git a/lib/util.lua b/lib/util.lua index 1c5a975..ab45be8 100644 --- a/lib/util.lua +++ b/lib/util.lua @@ -16,6 +16,7 @@ local http = require("http") local html = require("html") local json = require("json") local windowsCommand = require("windows_command") +local python_version = require("python_version") -- get mirror local PYTHON_URL = "https://www.python.org/ftp/python/" @@ -413,13 +414,27 @@ end local pyenvBranch = "" -function useUvBuild() +-- VFOX_PYTHON_USE_UV_BUILD: 1, true, yes or on asks for uv-build only; 0, false, +-- no or off asks for the source build (or the installer on Windows). Unset, or a +-- value that is neither, leaves the default in place. nil means unset. +local function uvBuildSetting() local value = os.getenv(UV_BUILD_ENV) if value == nil then + return nil + end + value = string.lower((value:gsub("^%s+", ""):gsub("%s+$", ""))) + if value == "1" or value == "true" or value == "yes" or value == "on" then + return true + end + if value == "0" or value == "false" or value == "no" or value == "off" then return false end - value = string.lower(value) - return value == "1" or value == "true" or value == "yes" or value == "on" + return nil +end + +--- Whether VFOX_PYTHON_USE_UV_BUILD asks for uv-build only. +function useUvBuild() + return uvBuildSetting() == true end local function containsTraversalSegment(value) @@ -549,7 +564,9 @@ local function runtimeArch() return archType end -local function runtimeLibc(osType) +--- The libc an index request names: the VFOX_PYTHON_UV_LIBC override, "none" off +--- Linux, the detected libc, or nil on Linux when detection fails. +local function detectLibc(osType) local configuredLibc = os.getenv("VFOX_PYTHON_UV_LIBC") if configuredLibc and configuredLibc ~= "" then return configuredLibc @@ -595,10 +612,38 @@ local function runtimeLibc(osType) end end + return nil +end + +local function runtimeLibc(osType) + local libc = detectLibc(osType) + if libc ~= nil then + return libc + end print("Warning: Could not detect libc, using gnu as default. Set VFOX_PYTHON_UV_LIBC to override.") return "gnu" end +local function isSetEnv(name) + local value = os.getenv(name) + return value ~= nil and value ~= "" +end + +--- Whether CPython installs try uv-build first when the index lists the version: +--- with no VFOX_PYTHON_USE_UV_BUILD setting, on Linux and macOS. Windows keeps the +--- python.org installers unless uv-build is asked for. A python.org mirror keeps +--- CPython on the source path, because uv-build downloads come from GitHub unless +--- VFOX_PYTHON_UV_BUILD_MIRROR is set too. +function uvBuildAutoEnabled() + if uvBuildSetting() ~= nil or runtimeOs() == "windows" then + return false + end + if isSetEnv("VFOX_PYTHON_MIRROR") and not isSetEnv(UV_BUILD_MIRROR_ENV) then + return false + end + return true +end + local function getUvBuildPlatform() local osType = runtimeOs() return osType, runtimeArch(), runtimeLibc(osType) @@ -738,11 +783,12 @@ local function getUvBuilds(osType, archType, libc) return jsonObj.items or jsonObj.versions or {} end -local function findUvBuild(version) - local osType, archType, libc = getUvBuildPlatform() +--- The build the listing has for version, the one with the best asset priority, +--- or nil when the listing has none. +local function selectUvBuild(builds, version) local selectedBuild = nil local selectedPriority = nil - for _, build in ipairs(getUvBuilds(osType, archType, libc)) do + for _, build in ipairs(builds) do if isSupportedUvBuild(build) and uvBuildVersion(build) == version then local priority = uvBuildAssetPriority(build) if selectedBuild == nil or priority < selectedPriority then @@ -751,6 +797,12 @@ local function findUvBuild(version) end end end + return selectedBuild +end + +local function findUvBuild(version) + local osType, archType, libc = getUvBuildPlatform() + local selectedBuild = selectUvBuild(getUvBuilds(osType, archType, libc), version) if selectedBuild ~= nil then return selectedBuild end @@ -766,6 +818,60 @@ local function findUvBuild(version) ) end +--- The error for an index that cannot be read in the default mode. It names the +--- opt-out, so the user is not left with a silent source build. +local function uvBuildAutoIndexError(detail) + return "Could not read the uv-build index: " + .. tostring(detail) + .. ". Set VFOX_PYTHON_USE_UV_BUILD=0 to build from source with pyenv instead." +end + +--- The build for a CPython version on this machine in the default mode. It returns +--- nil and the reason when the index has none, so PreInstall can build from +--- source. A failed request raises instead: the message names the opt-out, so the +--- user is not left with a silent source build. +local function findAutoUvBuild(version) + local osType, archType = runtimeOs(), runtimeArch() + local libc = detectLibc(osType) + if libc == nil then + return nil, "the C library could not be detected; set VFOX_PYTHON_UV_LIBC to use uv-build" + end + local ok, builds = pcall(getUvBuilds, osType, archType, libc) + if not ok then + error(uvBuildAutoIndexError(builds)) + end + local selectedBuild = selectUvBuild(builds, version) + if selectedBuild == nil then + return nil, "uv-build has no build for " .. osType .. "/" .. archType .. "/" .. libc + end + return selectedBuild +end + +--- The CPython versions the uv-build index lists for this platform, as a set of +--- names. A partial request in the default mode resolves against this set first. +--- It is nil when the libc is unknown, because the index is then not asked (see +--- findAutoUvBuild). A request that fails raises, so the install stops before it +--- starts a source build. +function uvBuildAutoVersionSet() + local osType, archType = runtimeOs(), runtimeArch() + local libc = detectLibc(osType) + if libc == nil then + return nil + end + local ok, builds = pcall(getUvBuilds, osType, archType, libc) + if not ok then + error(uvBuildAutoIndexError(builds)) + end + local names = {} + for _, build in ipairs(builds) do + local version = isSupportedUvBuild(build) and uvBuildVersion(build) or nil + if version ~= nil then + names[version] = true + end + end + return names +end + local function pathExists(path) local file = io.open(path, "r") if file then @@ -982,10 +1088,10 @@ function isLinkedPythonInstall(versionPath, version) return string.find(output, "yes", 1, true) ~= nil end -function uvBuildPreInstall(version) +--- The PreInstall result for a selected build. vfox downloads the URL, checks the +--- sha256 and unpacks the archive. +local function uvBuildPackage(build, version) local ok, value = pcall(function() - local build = findUvBuild(version) - -- Return url/sha256 so vfox core performs download, checksum verification, and archive extraction. return { version = version, url = uvBuildDownloadUrl(build), @@ -998,17 +1104,42 @@ function uvBuildPreInstall(version) local err = value error("uv-build PreInstall failed: " .. tostring(err)) end - local uvBuildPackage = value - if uvBuildPackage == nil then + local pkg = value + if pkg == nil then error("uv-build PreInstall did not provide install metadata") end - if isNilOrEmpty(uvBuildPackage.url) then + if isNilOrEmpty(pkg.url) then error("uv-build PreInstall failed: url is required") end - if isNilOrEmpty(uvBuildPackage.sha256) then + if isNilOrEmpty(pkg.sha256) then error("uv-build PreInstall failed: sha256 is required") end - return uvBuildPackage + return pkg +end + +function uvBuildPreInstall(version) + local ok, build = pcall(findUvBuild, version) + if not ok then + error("uv-build PreInstall failed: " .. tostring(build)) + end + return uvBuildPackage(build, version) +end + +--- PreInstall for a CPython version in the default mode: the archive when the +--- index lists it, or nil when PreInstall should return the source build instead. +function uvBuildAutoPreInstall(version) + local build, reason = findAutoUvBuild(version) + if build == nil then + print( + "Python " + .. version + .. " has no uv-build archive here (" + .. reason + .. "). Building it from source with pyenv." + ) + return nil + end + return uvBuildPackage(build, version) end local graalPyReleaseCaches = {} @@ -1194,10 +1325,29 @@ function parseGraalPyWindowsVersions() return result end +--- A source build starts from an empty install path. vfox removes the directory of +--- a failed install and clears an interrupted one before it runs again, so a Python +--- already in the path was unpacked from an archive. Building over it would replace +--- it without a word, so the build stops and says how to recover. +local function uvBuildRejectUnpackedArchive(path, version) + if pathExists(path .. "/bin/python") then + error( + "Python " + .. version + .. " is already in " + .. path + .. ", so the source build is not started. Run `vfox uninstall python@" + .. version + .. "` and install it again." + ) + end +end + function linuxCompile(ctx) local sdkInfo = ctx.sdkInfo["python"] local path = sdkInfo.path local version = sdkInfo.version + uvBuildRejectUnpackedArchive(path, version) local pyenv_url = "https://github.com/pyenv/pyenv.git" local dest_pyenv_path = ctx.rootPath .. "/pyenv" local branch = "" @@ -1236,6 +1386,14 @@ function linuxCompile(ctx) end end +--- Whether PreInstall returned a uv-build archive for this install. PreInstall sets +--- the note of that package to "uv-build", and vfox passes the note on to +--- PostInstall in sdkInfo, so PostInstall needs no second request to the index. +function uvBuildNoteArchive(ctx) + local sdkInfo = ctx.sdkInfo["python"] + return sdkInfo ~= nil and sdkInfo.note == "uv-build" +end + function uvBuildInstall(ctx) local sdkInfo = ctx.sdkInfo["python"] local path = sdkInfo.path @@ -1431,7 +1589,8 @@ function parseVersionFromPyenv() headers = REQUEST_HEADERS, }) if err ~= nil or resp.status_code ~= 200 then - error("paring release info failed." .. err) + local statusCode = resp and resp.status_code or "none" + error("parsing release info failed. Status: " .. statusCode .. ", Error: " .. (err or "none")) end local result = {} local jsonObj = json.decode(resp.body) @@ -1497,6 +1656,93 @@ function parseVersionFromUvBuild() return result end +--- A name with a dash, such as 3.16-dev, is a development snapshot from pyenv. It +--- sorts after every release, so `latest` never picks one. +local function isSnapshot(name) + return string.find(name, "-", 1, true) ~= nil +end + +--- Ranks names with the same numbers: a final build outranks its free-threaded +--- build, which outranks a pre-release, and a GIL pre-release outranks its +--- free-threaded pre-release. +local function releaseRank(name) + local free = string.sub(name, -1) == "t" + local base = free and string.sub(name, 1, -2) or name + local pre = string.find(base, "%d%a") ~= nil + if pre then + return free and 0 or 1 + end + return free and 2 or 3 +end + +--- Newest first: snapshots last, then the release numbers, then releaseRank. Ties +--- sort by name, so the order does not depend on the order of the input. +local function isNewerCandidate(a, b) + local snapshotA, snapshotB = isSnapshot(a), isSnapshot(b) + if snapshotA ~= snapshotB then + return snapshotB + end + local order = python_version.compare(string.match(a, "^[%d.]+"), string.match(b, "^[%d.]+")) + if order ~= 0 then + return order > 0 + end + local rankA, rankB = releaseRank(a), releaseRank(b) + if rankA ~= rankB then + return rankA > rankB + end + return a > b +end + +--- The list for the default mode on Linux and macOS: every release the pyenv list +--- names, plus the CPython releases only the uv-build index lists. The source of +--- each install is chosen in PreInstall, so the list is the same whichever source +--- an install ends up using. +function parseVersionFromAuto() + local okPyenv, pyenvList = pcall(parseVersionFromPyenv) + local okUv, uvList = pcall(parseVersionFromUvBuild) + if not okPyenv and not okUv then + error("Could not list Python versions. pyenv: " .. tostring(pyenvList) .. "; uv-build: " .. tostring(uvList)) + end + if not okPyenv then + print("Warning: Could not list the pyenv versions: " .. tostring(pyenvList)) + pyenvList = {} + end + if not okUv then + print("Warning: Could not list the uv-build versions: " .. tostring(uvList)) + uvList = {} + end + + local numeric = {} + local seen = {} + local function addNumeric(version) + if not seen[version] then + seen[version] = true + table.insert(numeric, version) + end + end + local named = {} + for _, item in ipairs(uvList) do + addNumeric(item.version) + end + for _, item in ipairs(pyenvList) do + if string.match(item.version, "^%d") then + addNumeric(item.version) + else + table.insert(named, item) + end + end + table.sort(numeric, isNewerCandidate) + + local result = {} + for _, version in ipairs(numeric) do + table.insert(result, { version = version, note = "" }) + end + for _, item in ipairs(named) do + table.insert(result, item) + end + return result +end + -- Fix shebang lines in Python scripts that point to temporary directories function fixShebangLines(installPath) return fixShebangForVersion(installPath, nil) diff --git a/metadata.lua b/metadata.lua index a32b939..dad4a79 100644 --- a/metadata.lua +++ b/metadata.lua @@ -47,7 +47,9 @@ PLUGIN.notes = { "Documentation: https://github.com/version-fox/vfox-python/blob/main/README.md", "Mirror: set VFOX_PYTHON_MIRROR to a python.org mirror, for example", "`export VFOX_PYTHON_MIRROR=https://mirrors.huaweicloud.com/python/`", - "uv-build: set VFOX_PYTHON_USE_UV_BUILD=1 to use prebuilt Python archives.", + "CPython on Linux and macOS: uv-build when the index has the version, else a source build.", + "A VFOX_PYTHON_MIRROR keeps the source build unless VFOX_PYTHON_UV_BUILD_MIRROR is set.", + "Set VFOX_PYTHON_USE_UV_BUILD=0 to always build from source, or =1 to use uv-build only.", "Set VFOX_PYTHON_UV_LIBC=gnu|musl to override Linux libc detection.", "Set VFOX_PYTHON_UV_BUILD_MIRROR to mirror uv-build archive downloads.", "GraalPy: install python@graalpy-24.2.0 or python@graalpy-community-24.2.0.", diff --git a/tests/e2e/linux/container_test.go b/tests/e2e/linux/container_test.go index 8053dca..16683c8 100644 --- a/tests/e2e/linux/container_test.go +++ b/tests/e2e/linux/container_test.go @@ -33,8 +33,8 @@ import ( // TestContainerE2E builds tests/e2e/linux/Dockerfile once and runs each // scenario in its own Ubuntu container. The image holds only the tools the -// Dockerfile installs and no build packages, so the uv-build subtest proves -// that uv-build needs no apt install. The assertions stay in Go. +// Dockerfile installs and no build packages, so the default-install subtest +// proves that a CPython install needs no apt install. The assertions stay in Go. func TestContainerE2E(t *testing.T) { if testing.Short() { t.Skip("skip container E2E in short mode") @@ -52,17 +52,17 @@ func TestContainerE2E(t *testing.T) { } uvVersion := common.NewestUvBuildVersion(t) - t.Run("uv-build without build packages", func(t *testing.T) { + t.Run("default install without build packages", func(t *testing.T) { ctr := startContainer(t, ctx, image, plugin) output := execOK(t, ctx, ctr, ` set -euo pipefail eval "$(vfox activate bash)" -VFOX_PYTHON_USE_UV_BUILD=1 vfox install "python@$UV_MINOR" -vfox use -g "python@$UV_MINOR" +vfox install "python@$UV_VERSION" +vfox use -g "python@$UV_VERSION" eval "$(vfox env -s bash)" python -c 'import ssl, sqlite3, bz2, lzma, ctypes, readline, curses, zlib, hashlib, sys; print("stdlib ok", sys.version.split()[0])' python -m pip --version -`, "UV_MINOR="+common.MinorVersion(uvVersion)) +`, "UV_VERSION="+uvVersion) common.RequireContains(t, output, "stdlib ok "+uvVersion, "uv-build stdlib without build packages") common.RequireContains(t, output, "pip", "uv-build pip") }) diff --git a/tests/e2e/linux/e2e_test.go b/tests/e2e/linux/e2e_test.go index 764d7ea..ad07327 100644 --- a/tests/e2e/linux/e2e_test.go +++ b/tests/e2e/linux/e2e_test.go @@ -55,8 +55,9 @@ func TestNativeE2E(t *testing.T) { // .python-version subtest asks for it, so it runs after uv-build. var uvVersion string + // The default is uv-build for CPython, so the source-build subtests opt out. t.Run("source build", func(t *testing.T) { - output := run(t, nil, ` + output := run(t, map[string]string{"VFOX_PYTHON_USE_UV_BUILD": "0"}, ` set -euo pipefail eval "$(vfox activate bash)" vfox install python@3.12.2 @@ -69,7 +70,8 @@ python -V t.Run("mirror", func(t *testing.T) { output := run(t, map[string]string{ - "VFOX_PYTHON_MIRROR": "https://mirrors.huaweicloud.com/python/", + "VFOX_PYTHON_MIRROR": "https://mirrors.huaweicloud.com/python/", + "VFOX_PYTHON_USE_UV_BUILD": "0", }, ` set -euo pipefail eval "$(vfox activate bash)" diff --git a/tests/partial_version_test.lua b/tests/partial_version_test.lua index ecf061f..3da1388 100644 --- a/tests/partial_version_test.lua +++ b/tests/partial_version_test.lua @@ -25,7 +25,8 @@ local reads = 0 local plugin = test.load({ os = "linux", arch = "amd64", - env = {}, + -- The source-build path: the default would also read the uv-build index. + env = { VFOX_PYTHON_USE_UV_BUILD = "0" }, http = function() reads = reads + 1 local quoted = {} diff --git a/tests/uv_build_default_test.lua b/tests/uv_build_default_test.lua new file mode 100644 index 0000000..fc02823 --- /dev/null +++ b/tests/uv_build_default_test.lua @@ -0,0 +1,369 @@ +-- Copyright 2026 Han Li and contributors +-- +-- Licensed under the Apache License, Version 2.0 (the "License"); +-- you may not use this file except in compliance with the License. +-- You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, software +-- distributed under the License is distributed on an "AS IS" BASIS, +-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +-- See the License for the specific language governing permissions and +-- limitations under the License. + +-- With VFOX_PYTHON_USE_UV_BUILD unset, Linux and macOS install a CPython release +-- from its uv-build archive when the index lists it, and build any other release +-- from source. A partial request resolves in the index first. Available keeps every +-- pyenv name and adds the uv-only CPython releases. VFOX_PYTHON_USE_UV_BUILD=0 keeps +-- the source build, and =1 stays strict. +local test = require("vfox.test") +local json = require("json") + +local realOpen, realPopen = io.open, io.popen + +local github = "https://github.com/astral-sh/python-build-standalone/releases/download/20260101/" +local UV_INDEX = "https://vault.vfox.dev/python/uv-build?" +local PYENV_INDEX = "https://vault.vfox.dev/python/pyenv" + +local function uvBuild(version, variant, checksum) + local filename = "cpython-" + .. version + .. "+20260101-x86_64-unknown-linux-gnu-" + .. (variant == "freethreaded" and "freethreaded-" or "") + .. "install_only_stripped.tar.gz" + return { + implementation = "cpython", + version = version, + variant = variant, + filename = filename, + url = github .. filename, + asset = { sha256 = string.rep(checksum, 64) }, + } +end + +local state = { + requests = {}, + uvStatus = 200, + pyenvStatus = 200, + uvItems = { + uvBuild("3.12.16", "default", "1"), + uvBuild("3.13.18", "default", "2"), + uvBuild("3.13.18", "freethreaded", "3"), + uvBuild("3.16.0", "default", "4"), + uvBuild("3.16.0a2", "default", "5"), + }, +} + +-- 3.13.19 is a pyenv-only patch that the index does not list. A partial request +-- for 3.13 takes the index's 3.13.18; only the exact name selects 3.13.19. +-- 3.16-dev is a pyenv development snapshot: it must not be the newest entry. +local pyenvVersions = + { "2.7.18", "3.12.16", "3.13.18", "3.13.18t", "3.13.19", "3.16-dev", "pypy3.10-7.3.15", "graalpy-24.2.0" } + +local function http(request) + table.insert(state.requests, request.url) + if request.url == PYENV_INDEX then + if state.pyenvStatus ~= 200 then + return { status_code = state.pyenvStatus } + end + return { status_code = 200, body = json.encode({ tagName = "v2.6.0", versions = pyenvVersions }) } + end + if string.find(request.url, UV_INDEX, 1, true) == 1 then + if state.uvStatus ~= 200 then + return { status_code = state.uvStatus } + end + return { status_code = 200, body = json.encode({ items = state.uvItems }) } + end + error("unexpected request: " .. request.url) +end + +-- test.load may run once per file, so the environment is switched here. The +-- plugin reads os.getenv at call time, so each case sets the variables it needs. +local env = { VFOX_PYTHON_UV_LIBC = "gnu" } + +local plugin = test.load({ + os = "linux", + arch = "amd64", + env = { VFOX_PYTHON_UV_LIBC = "gnu" }, + http = http, +}) + +os.getenv = function(name) + return env[name] +end + +local function uvRequests() + local count = 0 + for _, url in ipairs(state.requests) do + if string.find(url, UV_INDEX, 1, true) == 1 then + count = count + 1 + end + end + return count +end + +local function reset() + state.requests = {} +end + +-- A source install returns no archive. The test harness reports the missing URL +-- as an empty string, and vfox treats an empty URL the same way. +local function isSource(package) + return package.url == nil or package.url == "" +end + +local function namesOf(list) + local names = {} + for i, item in ipairs(list) do + names[i] = item.version + end + return names +end + +-- Available: the CPython releases of both lists, newest first and once each, +-- then the names only pyenv lists. A final release outranks its pre-release. +local names = namesOf(plugin:Available({})) +local expected = { "3.16.0", "3.16.0a2", "3.13.19", "3.13.18", "3.13.18t", "3.12.16", "2.7.18", "3.16-dev" } +for i, version in ipairs(expected) do + assert(names[i] == version, "Available[" .. i .. "] is " .. tostring(names[i]) .. ", want " .. version) +end +assert(#names == 10, table.concat(names, " ")) +assert( + (names[9] == "pypy3.10-7.3.15" and names[10] == "graalpy-24.2.0") + or (names[9] == "graalpy-24.2.0" and names[10] == "pypy3.10-7.3.15"), + table.concat(names, " ") +) + +-- An exact CPython release in the index installs its archive. +reset() +local package = plugin:PreInstall({ version = "3.12.16" }) +assert(package.note == "uv-build", tostring(package.note)) +assert( + package.url == github .. "cpython-3.12.16+20260101-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + package.url +) +assert(package.sha256 == string.rep("1", 64), tostring(package.sha256)) +assert(uvRequests() == 1, "one index request for an exact CPython version") + +package = plugin:PreInstall({ version = "3.13.18t" }) +assert(package.sha256 == string.rep("3", 64), "free-threaded request must take its own build") + +-- A partial request resolves in the index first: the newest release the index lists +-- on that line. The pyenv-only 3.13.19 is not picked for "3.13". The request reads +-- the index twice: once to resolve the name, and once for the archive. +reset() +package = plugin:PreInstall({ version = "3.13" }) +assert(package.version == "3.13.18" and package.sha256 == string.rep("2", 64), tostring(package.version)) +assert(package.note == "uv-build", tostring(package.note)) +assert(uvRequests() == 2, "a partial request reads the index to resolve, then for the archive") + +-- A free-threaded partial request resolves to the free-threaded build. +package = plugin:PreInstall({ version = "3.13t" }) +assert(package.version == "3.13.18t" and package.sha256 == string.rep("3", 64), tostring(package.version)) + +-- The exact pyenv-only patch has no archive, so it is built from source. +package = plugin:PreInstall({ version = "3.13.19" }) +assert(package.version == "3.13.19" and isSource(package), tostring(package.version)) + +-- A line the index does not list resolves in the pyenv list, and is built from source. +package = plugin:PreInstall({ version = "2" }) +assert(package.version == "2.7.18" and isSource(package), tostring(package.version)) + +-- latest keeps its meaning: the first entry of the list. +package = plugin:PreInstall({ version = "latest" }) +assert(package.version == "3.16.0" and package.note == "uv-build", tostring(package.version)) + +-- A release only pyenv lists is built from source: no archive is returned. +reset() +package = plugin:PreInstall({ version = "2.7.18" }) +assert( + package.version == "2.7.18" and isSource(package), + "version=" .. tostring(package.version) .. " url=" .. tostring(package.url) +) +assert(uvRequests() == 1, "the index is checked before the source build") + +-- A non-CPython name never reads the uv-build index. +reset() +package = plugin:PreInstall({ version = "pypy3.10-7.3.15" }) +assert(package.version == "pypy3.10-7.3.15" and isSource(package), tostring(package.url)) +assert(uvRequests() == 0, "a non-CPython name must not read the uv-build index") + +-- An index that cannot be read is an error that names the opt-out; it is not a +-- silent source build. The list still shows the pyenv names. +state.uvStatus = 503 +local ok, err = pcall(function() + plugin:PreInstall({ version = "3.12.16" }) +end) +assert(not ok and tostring(err):find("Could not read the uv-build index", 1, true), tostring(err)) +assert(tostring(err):find("VFOX_PYTHON_USE_UV_BUILD=0", 1, true), tostring(err)) + +-- A partial request fails the same way: it must not fall back to a source build. +ok, err = pcall(function() + plugin:PreInstall({ version = "3.13" }) +end) +assert(not ok and tostring(err):find("Could not read the uv-build index", 1, true), tostring(err)) +assert(#plugin:Available({}) == 8, "the pyenv names are listed when the index fails") +state.uvStatus = 200 + +-- A pyenv failure still lists the uv-build CPython releases. +state.pyenvStatus = 500 +names = namesOf(plugin:Available({})) +assert(#names == 5 and names[1] == "3.16.0", table.concat(names, " ")) +state.pyenvStatus = 200 + +-- macOS asks for its own platform and never sends a libc. The plugin reads the +-- platform at call time, so it is switched for this case only. +local runtimeOs, runtimeArch, hostOs = RUNTIME.osType, RUNTIME.archType, OS_TYPE +RUNTIME.osType, RUNTIME.archType, OS_TYPE = "darwin", "arm64", "darwin" +env = {} +reset() +package = plugin:PreInstall({ version = "3.12.16" }) +env = { VFOX_PYTHON_UV_LIBC = "gnu" } +RUNTIME.osType, RUNTIME.archType, OS_TYPE = runtimeOs, runtimeArch, hostOs +assert(package.note == "uv-build", tostring(package.note)) +local macRequest = state.requests[#state.requests] +assert(string.find(macRequest, "os=darwin&arch=aarch64", 1, true), macRequest) +assert(not string.find(macRequest, "libc=", 1, true), macRequest) + +-- With no detectable libc, the source build is used rather than a guessed archive. +env = {} +io.open = function(path, mode) + if string.find(path, "musl", 1, true) then + return nil + end + return realOpen(path, mode) +end +io.popen = function() + return nil +end +reset() +package = plugin:PreInstall({ version = "3.12.16" }) +io.open, io.popen = realOpen, realPopen +assert(isSource(package), "without a detectable libc the source build is used") +assert(uvRequests() == 0, "no index request without a libc") +env = { VFOX_PYTHON_UV_LIBC = "gnu" } + +-- A python.org mirror keeps CPython on the source path. The uv-build mirror, +-- when it is set too, moves the uv-build downloads to that mirror. +env = { VFOX_PYTHON_MIRROR = "https://mirror.example/python/", VFOX_PYTHON_UV_LIBC = "gnu" } +reset() +names = namesOf(plugin:Available({})) +assert(#names == 8, table.concat(names, " ")) +package = plugin:PreInstall({ version = "3.12.16" }) +assert(isSource(package), "a python.org mirror keeps the source build") +assert(uvRequests() == 0, "no uv-build index request with a python.org mirror") +env = { + VFOX_PYTHON_MIRROR = "https://mirror.example/python/", + VFOX_PYTHON_UV_BUILD_MIRROR = "https://uv.example/", + VFOX_PYTHON_UV_LIBC = "gnu", +} +package = plugin:PreInstall({ version = "3.12.16" }) +assert( + package.url + == "https://uv.example/20260101/cpython-3.12.16+20260101-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + package.url +) +env = { VFOX_PYTHON_UV_LIBC = "gnu" } + +-- VFOX_PYTHON_USE_UV_BUILD=0: the source build and the pyenv list only. +env = { VFOX_PYTHON_USE_UV_BUILD = "0", VFOX_PYTHON_UV_LIBC = "gnu" } +reset() +names = namesOf(plugin:Available({})) +assert(#names == 8, table.concat(names, " ")) +for _, version in ipairs(names) do + assert(version ~= "3.16.0" and version ~= "3.16.0a2", "uv-only release listed in source mode") +end +package = plugin:PreInstall({ version = "3.12.16" }) +assert(isSource(package), "=0 must build from source") +assert(uvRequests() == 0, "=0 must not read the uv-build index") + +-- VFOX_PYTHON_USE_UV_BUILD=1: only the index, with the strict errors. +env = { VFOX_PYTHON_USE_UV_BUILD = "1", VFOX_PYTHON_UV_LIBC = "gnu" } +names = namesOf(plugin:Available({})) +assert(#names == 5, table.concat(names, " ")) +package = plugin:PreInstall({ version = "3.12.16" }) +assert(package.note == "uv-build", tostring(package.note)) +ok, err = pcall(function() + plugin:PreInstall({ version = "2.7.18" }) +end) +assert(not ok and tostring(err):find("No uv-build prebuilt Python found for version 2.7.18", 1, true), tostring(err)) +ok, err = pcall(function() + plugin:PreInstall({ version = "graalpy-24.2.0" }) +end) +assert(not ok and tostring(err):find("uv-build provides CPython only", 1, true), tostring(err)) +assert(tostring(err):find("(or set it to 0)", 1, true), tostring(err)) + +-- PostInstall takes the archive path when PreInstall set the uv-build note, which +-- vfox passes on in sdkInfo. It does not ask the index again. uvBuildInstall and +-- linuxCompile are replaced so only the choice is checked. The real linuxCompile +-- is kept for the check that refuses to build over an unpacked archive. +local realLinuxCompile = linuxCompile +local calls = {} +uvBuildInstall = function() + calls[#calls + 1] = "uv" +end +linuxCompile = function() + calls[#calls + 1] = "source" +end + +-- A path holds a Python only when a test puts one there, as vfox does for an archive. +local present = {} +io.open = function(path, mode) + if present[path] then + return { + close = function() end, + } + end + return realOpen(path, mode) +end + +local root = "/home/test/.vfox" +local function installPath(version) + return root .. "/cache/python/v-" .. version .. "/python-" .. version +end + +local function postInstall(version, note) + calls = {} + plugin:PostInstall({ + rootPath = root, + sdkInfo = { + python = { path = installPath(version), version = version, note = note }, + }, + }) + return table.concat(calls, ",") +end + +env = { VFOX_PYTHON_UV_LIBC = "gnu" } +reset() +assert(postInstall("3.12.16", "uv-build") == "uv", "the uv-build note takes the archive path") +assert(postInstall("2.7.18", "") == "source", "an install without the note is built from source") +assert(postInstall("2.7.18", nil) == "source", "an install with no note at all is built from source") +assert(postInstall("pypy3.10-7.3.15", nil) == "source", "a non-CPython name is built from source") +assert(uvRequests() == 0, "PostInstall does not ask the index") + +env = { VFOX_PYTHON_USE_UV_BUILD = "1", VFOX_PYTHON_UV_LIBC = "gnu" } +assert(postInstall("2.7.18", nil) == "uv", "=1 always takes the archive path") + +env = { VFOX_PYTHON_USE_UV_BUILD = "0", VFOX_PYTHON_UV_LIBC = "gnu" } +assert(postInstall("3.12.16", nil) == "source", "=0 builds from source") + +env = { VFOX_PYTHON_MIRROR = "https://mirror.example/python/", VFOX_PYTHON_UV_LIBC = "gnu" } +assert(postInstall("3.12.16", nil) == "source", "a python.org mirror keeps the source build") +env = { VFOX_PYTHON_UV_LIBC = "gnu" } + +-- A Python already in the install path was unpacked from an archive. The source +-- build refuses to start rather than build over it, and the message says how to +-- recover. +present = { [installPath("3.12.16") .. "/bin/python"] = true } +local guardOk, guardErr = pcall(realLinuxCompile, { + rootPath = root, + sdkInfo = { python = { path = installPath("3.12.16"), version = "3.12.16" } }, +}) +assert(not guardOk and tostring(guardErr):find("is already in", 1, true), tostring(guardErr)) +assert(tostring(guardErr):find("vfox uninstall python@3.12.16", 1, true), tostring(guardErr)) +present = {} + +io.open, io.popen = realOpen, realPopen + +print("PASS: uv-build by default for CPython on Linux and macOS, with source fallbacks and opt-outs") From 4c46fb287ae0fa8ddda6657d77f01560b9cc2d97 Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 15:30:44 +0800 Subject: [PATCH 2/9] test: keep latest on the newest release in the default mode latest stays the newest entry of the combined list. It is not resolved in the uv-build index, because that would silently install an older release, or a pre-release the index lists in place of a final one. The newest release comes from its archive when the index lists it, and is built from source, with a line saying so, when it does not. Add a test for the case where pyenv lists a newer release than the index, and document the behaviour in English and Chinese. --- docs/uv.md | 4 ++++ docs/versions.md | 5 ++++- docs/zh-CN/uv.md | 2 ++ docs/zh-CN/versions.md | 4 +++- tests/uv_build_default_test.lua | 10 +++++++++- 5 files changed, 22 insertions(+), 3 deletions(-) diff --git a/docs/uv.md b/docs/uv.md index c7dd23b..684401b 100644 --- a/docs/uv.md +++ b/docs/uv.md @@ -33,6 +33,10 @@ pyenv lists a newer patch than the index, a partial name such as `python@3.12` s installs the newest patch the index lists. The newer patch builds from source only when its exact version is named. +`latest` is not resolved in the index. It stays the newest release in the list, so +when pyenv lists a newer release than the index, `latest` builds that release from +source. + `VFOX_PYTHON_USE_UV_BUILD` selects the mode: - Unset: uv-build for a CPython release the index lists, and a source build for diff --git a/docs/versions.md b/docs/versions.md index 897d96f..3f31555 100644 --- a/docs/versions.md +++ b/docs/versions.md @@ -38,7 +38,10 @@ pre-release name, so `python@3.13` is not `python@3.13t`. `latest` is resolved by `PreInstall` to the first version `Available` returns. The list is newest CPython first, so this is the newest CPython release the platform lists, which can be a pre-release when one exists. GraalPy is -selected by its full name. +selected by its full name. In the default mode on Linux and macOS, `latest` is not +resolved in the uv-build index, so it stays the newest release in the list: it +installs from its archive when the index lists that release, and is built from +source, with a line saying so, when pyenv lists a newer one. ## Where the list comes from diff --git a/docs/zh-CN/uv.md b/docs/zh-CN/uv.md index 5462dca..15c7a78 100644 --- a/docs/zh-CN/uv.md +++ b/docs/zh-CN/uv.md @@ -25,6 +25,8 @@ vfox install python@3.12 索引只列出部分版本,通常是每个版本线的最新补丁版。pyenv 列出的补丁版比索引新时,`python@3.12` 仍然安装索引列出的最新补丁版;较新的补丁版只有写出精确版本号时才会从源码编译。 +`latest` 不在索引里解析:它仍是列表里最新的版本。pyenv 列出的版本比索引新时,它会从源码编译。 + `VFOX_PYTHON_USE_UV_BUILD` 决定使用哪种模式: - 未设置:索引列出的 CPython 版本用 uv-build,其他 CPython 版本用源码构建;GraalPy、PyPy 和 Conda 保持各自的路径。当设置了 `VFOX_PYTHON_MIRROR` diff --git a/docs/zh-CN/versions.md b/docs/zh-CN/versions.md index 53eb967..f75faf4 100644 --- a/docs/zh-CN/versions.md +++ b/docs/zh-CN/versions.md @@ -32,7 +32,9 @@ vfox use python@3.12 # 已安装的 3.12.x 里最新的 `latest` 由 `PreInstall` 解析成 `Available` 返回的第一个版本。列表按从新到旧 排列,所以它是这个平台列出的最新 CPython 版本;有预发布版时,也可能是一个 -预发布版。GraalPy 需要写出完整版本名。 +预发布版。GraalPy 需要写出完整版本名。在 Linux 和 macOS 的默认模式下,`latest` 不在 uv-build 索引里解析, +它仍是列表里最新的版本:索引列出该版本时安装它的预编译包;pyenv 列出的版本比索引新时,则从源码编译, +并打印一行说明。 ## 版本列表从哪里来 diff --git a/tests/uv_build_default_test.lua b/tests/uv_build_default_test.lua index fc02823..b354a42 100644 --- a/tests/uv_build_default_test.lua +++ b/tests/uv_build_default_test.lua @@ -170,10 +170,18 @@ assert(package.version == "3.13.19" and isSource(package), tostring(package.vers package = plugin:PreInstall({ version = "2" }) assert(package.version == "2.7.18" and isSource(package), tostring(package.version)) --- latest keeps its meaning: the first entry of the list. +-- latest is the newest release of the list. It is never an older release that the +-- index lists instead, and it takes the archive when the index lists that release. package = plugin:PreInstall({ version = "latest" }) assert(package.version == "3.16.0" and package.note == "uv-build", tostring(package.version)) +-- When pyenv lists a newer release than the index, latest is that release, and it is +-- built from source. The extra entry is removed again straight after the check. +table.insert(pyenvVersions, "3.16.1") +package = plugin:PreInstall({ version = "latest" }) +table.remove(pyenvVersions) +assert(package.version == "3.16.1" and isSource(package), tostring(package.version)) + -- A release only pyenv lists is built from source: no archive is returned. reset() package = plugin:PreInstall({ version = "2.7.18" }) From 25c5d35e7120a155c55b773cc91b982e87102ecd Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 15:43:27 +0800 Subject: [PATCH 3/9] fix: keep the version lists newest first and skip free-threaded builds for latest With VFOX_PYTHON_USE_UV_BUILD=1 the list kept the order the index returned, and latest took its first entry, so latest depended on that order. The pyenv list sorted names with compare_versions, which reads 3.13.18t as 3.13.0. The uv-build list and the pyenv list now use one comparator, python_version.newer_first: release numbers first, then a final build before its free-threaded build before its pre-release, with snapshots last. latest is now the first entry that is not a free-threaded build. A free-threaded build is installed only when its name asks for it, as the README describes, so latest never selects one. The =1 test now expects the newest-first list, and new tests cover the comparator, the pyenv order and latest. --- docs/changes.md | 6 +++ docs/development.md | 7 +-- docs/uv.md | 6 +-- docs/versions.md | 9 ++-- docs/zh-CN/changes.md | 3 ++ docs/zh-CN/development.md | 4 +- docs/zh-CN/uv.md | 2 +- docs/zh-CN/versions.md | 6 +-- hooks/pre_install.lua | 14 ++++- lib/python_version.lua | 55 ++++++++++++++++++++ lib/util.lua | 92 +++++++++++---------------------- tests/partial_version_test.lua | 29 +++++++++++ tests/uv_build_default_test.lua | 16 ++++++ tests/uv_build_test.lua | 8 +-- 14 files changed, 176 insertions(+), 81 deletions(-) diff --git a/docs/changes.md b/docs/changes.md index 5255b22..e36b432 100644 --- a/docs/changes.md +++ b/docs/changes.md @@ -4,6 +4,12 @@ ## Unreleased +- The uv-build index and pyenv version lists are sorted newest first with one + comparator. With `VFOX_PYTHON_USE_UV_BUILD=1`, the list used to keep the order + the index returned, so `latest` depended on that order. The pyenv list compared + a name with a suffix, such as `3.13.18t`, as if it were `3.13.0`. `latest` now + picks the newest standard release and skips free-threaded builds, which are + installed only when named. - On Linux and macOS, `vfox install python@` now installs a CPython release from its prebuilt uv-build archive when the uv-build index lists it, and builds from source only when the index does not. Set diff --git a/docs/development.md b/docs/development.md index d3eae93..7d2e447 100644 --- a/docs/development.md +++ b/docs/development.md @@ -62,9 +62,10 @@ the index has one; otherwise, and in every other mode, it resolves to the newest stable release that `Available({})` names for this mode. An exact version is not looked up. -- `version = "latest"` is resolved to `Available({})[1].version`. The list is - newest CPython first, so this can be a pre-release; GraalPy is selected by - its full name. +- `version = "latest"` is resolved to the first entry of `Available({})` that is + not a free-threaded build. The list is newest CPython first, so this can be a + pre-release; a free-threaded build is installed only when its name asks for it, + and GraalPy is selected by its full name. - On Windows, `Available` lists only the exact installer names on the download page. PreInstall still sends a HEAD request for the EXE and then for the MSI, and stops before any download when neither answers 200, with `Python diff --git a/docs/uv.md b/docs/uv.md index 684401b..ff45458 100644 --- a/docs/uv.md +++ b/docs/uv.md @@ -33,9 +33,9 @@ pyenv lists a newer patch than the index, a partial name such as `python@3.12` s installs the newest patch the index lists. The newer patch builds from source only when its exact version is named. -`latest` is not resolved in the index. It stays the newest release in the list, so -when pyenv lists a newer release than the index, `latest` builds that release from -source. +`latest` is not resolved in the index. It stays the newest standard release in the +list, so when pyenv lists a newer release than the index, `latest` builds that +release from source. `VFOX_PYTHON_USE_UV_BUILD` selects the mode: diff --git a/docs/versions.md b/docs/versions.md index 87be337..b2605ea 100644 --- a/docs/versions.md +++ b/docs/versions.md @@ -35,10 +35,11 @@ name that matches, and a bare name beats the same release with a source suffix such as `-system`. A plain request never picks a free-threaded (`t`) or pre-release name, so `python@3.13` is not `python@3.13t`. -`latest` is resolved by `PreInstall` to the first version `Available` returns. -The list is newest CPython first, so this is the newest CPython release the -platform lists, which can be a pre-release when one exists. GraalPy is -selected by its full name. In the default mode on Linux and macOS, `latest` is not +`latest` is resolved by `PreInstall` to the first version `Available` returns that +is not a free-threaded build. The list is newest CPython first, so this is the newest +standard CPython release the platform lists, which can be a pre-release when one +exists. A free-threaded build such as `3.13.18t` is installed only when its name asks +for it. GraalPy is selected by its full name. In the default mode on Linux and macOS, `latest` is not resolved in the uv-build index, so it stays the newest release in the list: it installs from its archive when the index lists that release, and is built from source, with a line saying so, when pyenv lists a newer one. diff --git a/docs/zh-CN/changes.md b/docs/zh-CN/changes.md index e63531f..0396c45 100644 --- a/docs/zh-CN/changes.md +++ b/docs/zh-CN/changes.md @@ -4,6 +4,9 @@ ## 未发布 +- uv-build 索引和 pyenv 版本列表现在用同一个比较函数从新到旧排序。过去 `VFOX_PYTHON_USE_UV_BUILD=1` + 的列表保留索引返回的顺序,`latest` 依赖这个顺序;pyenv 列表则把带后缀的名字(如 `3.13.18t`)当成 + `3.13.0` 来比较。`latest` 现在选最新的常规版本,并跳过自由线程版;自由线程版只有写出名字时才会安装。 - 在 Linux 和 macOS 上,`vfox install python@<版本>` 现在会在 uv-build 索引列出某个 CPython 版本时, 从它的 uv-build 预编译包安装;只有索引没有列出时才从源码构建。设置 `VFOX_PYTHON_USE_UV_BUILD=0` 可以让每个 CPython 都继续从源码构建,设置 `VFOX_PYTHON_USE_UV_BUILD=1` 则只使用 uv-build。GraalPy、 diff --git a/docs/zh-CN/development.md b/docs/zh-CN/development.md index 091f8aa..b72593d 100644 --- a/docs/zh-CN/development.md +++ b/docs/zh-CN/development.md @@ -57,8 +57,8 @@ vfox 加载 `metadata.lua`,命令需要时调用相应的钩子,并向钩子 `sha256` 时校验,解压后按 `version` 命名。部分版本号(如 `3.12`)会先解析。在 Linux 和 macOS 的默认模式下,若 uv-build 索引里有匹配版本, 就解析成它;否则(以及其他所有模式下)解析成 `Available({})` 里该模式最新的稳定版。精确版本号不会查列表。 -- `version = "latest"` 会解析成 `Available({})[1].version`。CPython 在列表最 - 前面、从新到旧排列,所以可能是预发布版;GraalPy 要用完整版本名。 +- `version = "latest"` 会解析成 `Available({})` 里第一个不是自由线程版的条目。CPython 在列表最 + 前面、从新到旧排列,所以可能是预发布版;自由线程版只有写出名字时才会安装;GraalPy 要用完整版本名。 - Windows 上,`Available` 只列出下载页里当前架构的精确安装包文件名。PreInstall 仍会先对 EXE、 再对 MSI 发 HEAD 请求;两者都没有返回 200 时,在任何下载开始前停止,并提示 `Python <版本> has no Windows installer on python.org`。 diff --git a/docs/zh-CN/uv.md b/docs/zh-CN/uv.md index 15c7a78..cba9441 100644 --- a/docs/zh-CN/uv.md +++ b/docs/zh-CN/uv.md @@ -25,7 +25,7 @@ vfox install python@3.12 索引只列出部分版本,通常是每个版本线的最新补丁版。pyenv 列出的补丁版比索引新时,`python@3.12` 仍然安装索引列出的最新补丁版;较新的补丁版只有写出精确版本号时才会从源码编译。 -`latest` 不在索引里解析:它仍是列表里最新的版本。pyenv 列出的版本比索引新时,它会从源码编译。 +`latest` 不在索引里解析:它仍是列表里最新的常规版本(跳过自由线程版)。pyenv 列出的版本比索引新时,它会从源码编译。 `VFOX_PYTHON_USE_UV_BUILD` 决定使用哪种模式: diff --git a/docs/zh-CN/versions.md b/docs/zh-CN/versions.md index 532bc9b..6b4139a 100644 --- a/docs/zh-CN/versions.md +++ b/docs/zh-CN/versions.md @@ -30,9 +30,9 @@ vfox use python@3.12 # 已安装的 3.12.x 里最新的 最新的匹配项,同一版本时裸名字优先于带 `-system` 等来源后缀的名字。普通请求从不选自由线程 (`t`)或预发布版,所以 `python@3.13` 不会选到 `python@3.13t`。 -`latest` 由 `PreInstall` 解析成 `Available` 返回的第一个版本。列表按从新到旧 -排列,所以它是这个平台列出的最新 CPython 版本;有预发布版时,也可能是一个 -预发布版。GraalPy 需要写出完整版本名。在 Linux 和 macOS 的默认模式下,`latest` 不在 uv-build 索引里解析, +`latest` 由 `PreInstall` 解析成 `Available` 返回的、第一个不是自由线程版的版本。列表按从新到旧 +排列,所以它是这个平台列出的最新常规 CPython 版本;有预发布版时,也可能是一个 +预发布版。自由线程版(如 `3.13.18t`)只有写出名字时才会安装。GraalPy 需要写出完整版本名。在 Linux 和 macOS 的默认模式下,`latest` 不在 uv-build 索引里解析, 它仍是列表里最新的版本:索引列出该版本时安装它的预编译包;pyenv 列出的版本比索引新时,则从源码编译, 并打印一行说明。 diff --git a/hooks/pre_install.lua b/hooks/pre_install.lua index b5504a5..ef293f6 100644 --- a/hooks/pre_install.lua +++ b/hooks/pre_install.lua @@ -40,11 +40,23 @@ local function resolvePartialVersion(plugin, requested) return python_version.find_release(names, requested) or requested end +--- The version `latest` installs: the first entry of the list that is not a +--- free-threaded build. The list is newest first, and a free-threaded build is +--- installed only when a request names it. +local function latestVersion(list) + for _, item in ipairs(list) do + if not python_version.is_free_threaded(item.version) then + return item.version + end + end + error("No standard CPython release is listed, so latest has nothing to install. Name a version instead.") +end + function PLUGIN:PreInstall(ctx) local version = ctx.version if version == "latest" then - version = self:Available({})[1].version + version = latestVersion(self:Available({})) end version = resolvePartialVersion(self, version) diff --git a/lib/python_version.lua b/lib/python_version.lua index 4d3b60f..341c3dd 100644 --- a/lib/python_version.lua +++ b/lib/python_version.lua @@ -292,4 +292,59 @@ function python_version.find_release(names, request) return best end +--- Whether a name is a development snapshot from pyenv, such as 3.16-dev. A snapshot +--- has a dash; a release name never does. +local function isSnapshot(name) + return string.find(name, "-", 1, true) ~= nil +end + +--- The release numbers of a name, without its suffix: 3.16.0a2 gives 3.16.0 and +--- 3.13.18t gives 3.13.18, so a pre-release or a free-threaded build sorts with its +--- release. +local function numberPart(name) + local release = releaseOf(name) + return release:match("^[%d.]+") or release +end + +--- Ranks names that have the same release numbers: a final build outranks its +--- free-threaded build, which outranks a pre-release, which outranks a +--- free-threaded pre-release. +local function sameNumberRank(name) + local _, free, pre = releaseOf(name) + if pre then + return free and 0 or 1 + end + return free and 2 or 3 +end + +--- Whether a name is a free-threaded build, such as 3.13.18t. A free-threaded build is +--- installed only when a request names it, so `latest` skips it. +function python_version.is_free_threaded(name) + if type(name) ~= "string" then + return false + end + local _, free = releaseOf(name) + return free +end + +--- Whether `a` is listed before `b` in a newest-first list of CPython names. The +--- release numbers decide, then sameNumberRank. A snapshot such as 3.16-dev comes +--- after every release. Names that still tie sort by name, so the order does not +--- depend on the order of the input. +function python_version.newer_first(a, b) + local snapshotA, snapshotB = isSnapshot(a), isSnapshot(b) + if snapshotA ~= snapshotB then + return snapshotB + end + local order = python_version.compare(numberPart(a), numberPart(b)) + if order ~= 0 then + return order > 0 + end + local rankA, rankB = sameNumberRank(a), sameNumberRank(b) + if rankA ~= rankB then + return rankA > rankB + end + return a > b +end + return python_version diff --git a/lib/util.lua b/lib/util.lua index 8d60d29..9b80ee2 100644 --- a/lib/util.lua +++ b/lib/util.lua @@ -1642,6 +1642,27 @@ local function pyenvVersionNote(version) return "" end +--- Sorts CPython items ({ version = ... }) newest first, by python_version.newer_first. +--- An item whose version does not start with a digit, such as a GraalPy or PyPy name, +--- keeps its place after the CPython releases, in its original order. +local function sortNewestFirst(items) + local releases, others = {}, {} + for _, item in ipairs(items) do + if string.match(item.version, "^%d") then + table.insert(releases, item) + else + table.insert(others, item) + end + end + table.sort(releases, function(a, b) + return python_version.newer_first(a.version, b.version) + end) + for _, item in ipairs(others) do + table.insert(releases, item) + end + return releases +end + function parseVersionFromPyenv() fixHeaders() local resp, err = http.get({ @@ -1662,32 +1683,22 @@ function parseVersionFromPyenv() local versions = jsonObj.versions - local numericVersions = {} + local releases = {} local namedVersions = {} for _, version in ipairs(versions) do if string.match(version, "^%d") then - table.insert(numericVersions, version) + table.insert(releases, { version = version, note = "" }) else table.insert(namedVersions, version) end end - table.sort(numericVersions, function(a, b) - return compare_versions(a, b) > 0 - end) - table.sort(namedVersions, function(a, b) return compare_versions(a, b) > 0 end) - for _, version in ipairs(numericVersions) do - table.insert(result, { - version = version, - note = "", - }) - end - + result = sortNewestFirst(releases) for _, version in ipairs(namedVersions) do table.insert(result, { version = version, @@ -1713,44 +1724,7 @@ function parseVersionFromUvBuild() end end end - return result -end - ---- A name with a dash, such as 3.16-dev, is a development snapshot from pyenv. It ---- sorts after every release, so `latest` never picks one. -local function isSnapshot(name) - return string.find(name, "-", 1, true) ~= nil -end - ---- Ranks names with the same numbers: a final build outranks its free-threaded ---- build, which outranks a pre-release, and a GIL pre-release outranks its ---- free-threaded pre-release. -local function releaseRank(name) - local free = string.sub(name, -1) == "t" - local base = free and string.sub(name, 1, -2) or name - local pre = string.find(base, "%d%a") ~= nil - if pre then - return free and 0 or 1 - end - return free and 2 or 3 -end - ---- Newest first: snapshots last, then the release numbers, then releaseRank. Ties ---- sort by name, so the order does not depend on the order of the input. -local function isNewerCandidate(a, b) - local snapshotA, snapshotB = isSnapshot(a), isSnapshot(b) - if snapshotA ~= snapshotB then - return snapshotB - end - local order = python_version.compare(string.match(a, "^[%d.]+"), string.match(b, "^[%d.]+")) - if order ~= 0 then - return order > 0 - end - local rankA, rankB = releaseRank(a), releaseRank(b) - if rankA ~= rankB then - return rankA > rankB - end - return a > b + return sortNewestFirst(result) end --- The list for the default mode on Linux and macOS: every release the pyenv list @@ -1772,31 +1746,27 @@ function parseVersionFromAuto() uvList = {} end - local numeric = {} + local releases = {} local seen = {} - local function addNumeric(version) + local function addRelease(version) if not seen[version] then seen[version] = true - table.insert(numeric, version) + table.insert(releases, { version = version, note = "" }) end end local named = {} for _, item in ipairs(uvList) do - addNumeric(item.version) + addRelease(item.version) end for _, item in ipairs(pyenvList) do if string.match(item.version, "^%d") then - addNumeric(item.version) + addRelease(item.version) else table.insert(named, item) end end - table.sort(numeric, isNewerCandidate) - local result = {} - for _, version in ipairs(numeric) do - table.insert(result, { version = version, note = "" }) - end + local result = sortNewestFirst(releases) for _, item in ipairs(named) do table.insert(result, item) end diff --git a/tests/partial_version_test.lua b/tests/partial_version_test.lua index 3da1388..b46f467 100644 --- a/tests/partial_version_test.lua +++ b/tests/partial_version_test.lua @@ -185,4 +185,33 @@ assert(use("3.13t").version == "3.13.18t", use("3.13t").version) -- Only the free-threaded 3.13.18t is installed, and a plain request never picks it. assert(use("3.13").version == "3.13", use("3.13").version) +-- newer_first orders a newest-first list: the release numbers decide, a final build +-- comes before its free-threaded build and before its pre-release, and a pyenv +-- snapshot comes after every release. +local function newestFirst(list) + local copy = {} + for i, name in ipairs(list) do + copy[i] = name + end + table.sort(copy, python_version.newer_first) + return table.concat(copy, " ") +end +assert( + newestFirst({ "3.13.5", "3.13.18t", "3.12.16", "3.16-dev", "3.16.0a2", "3.16.0", "3.13.18" }) + == "3.16.0 3.16.0a2 3.13.18 3.13.18t 3.13.5 3.12.16 3.16-dev", + newestFirst({ "3.13.5", "3.13.18t", "3.12.16", "3.16-dev", "3.16.0a2", "3.16.0", "3.13.18" }) +) +-- A free-threaded build sorts by its release, not by the digits of its suffix. +assert(python_version.newer_first("3.13.18t", "3.13.5"), "3.13.18t is newer than 3.13.5") +assert(not python_version.newer_first("3.13.5", "3.13.18t"), "3.13.5 is not newer than 3.13.18t") +-- A final build outranks the pre-release of the same numbers. +assert(python_version.newer_first("3.16.0", "3.16.0a2"), "3.16.0 is newer than its pre-release") +-- latest skips free-threaded builds, and a plain request never picks one either. +assert(python_version.is_free_threaded("3.13.18t"), "3.13.18t is free-threaded") +assert(python_version.is_free_threaded("3.16.0a2t"), "3.16.0a2t is free-threaded") +assert(not python_version.is_free_threaded("3.13.18"), "3.13.18 is not free-threaded") +assert(not python_version.is_free_threaded("3.16.0a2"), "a pre-release is not free-threaded") +assert(not python_version.is_free_threaded("graalpy-24.2.0"), "GraalPy is not free-threaded") +assert(not python_version.is_free_threaded("3.16-dev"), "a snapshot is not free-threaded") + print("PASS: partial versions resolve in PreInstall and PreUse") diff --git a/tests/uv_build_default_test.lua b/tests/uv_build_default_test.lua index b354a42..4de3406 100644 --- a/tests/uv_build_default_test.lua +++ b/tests/uv_build_default_test.lua @@ -182,6 +182,13 @@ package = plugin:PreInstall({ version = "latest" }) table.remove(pyenvVersions) assert(package.version == "3.16.1" and isSource(package), tostring(package.version)) +-- A free-threaded build newer than every standard release is never latest: latest is +-- the newest standard release, here the index's 3.16.0 with its archive. +table.insert(pyenvVersions, "3.16.1t") +package = plugin:PreInstall({ version = "latest" }) +table.remove(pyenvVersions) +assert(package.version == "3.16.0" and package.note == "uv-build", tostring(package.version)) + -- A release only pyenv lists is built from source: no archive is returned. reset() package = plugin:PreInstall({ version = "2.7.18" }) @@ -282,6 +289,15 @@ assert(#names == 8, table.concat(names, " ")) for _, version in ipairs(names) do assert(version ~= "3.16.0" and version ~= "3.16.0a2", "uv-only release listed in source mode") end +-- The pyenv list is newest first too. A 3.13.5 that pyenv lists falls below 3.13.18t, +-- since 3.13.18t sorts by its release; the snapshot 3.16-dev comes after the releases. +table.insert(pyenvVersions, "3.13.5") +names = namesOf(plugin:Available({})) +table.remove(pyenvVersions) +assert( + table.concat(names, " ", 1, 7) == "3.13.19 3.13.18 3.13.18t 3.13.5 3.12.16 2.7.18 3.16-dev", + table.concat(names, " ") +) package = plugin:PreInstall({ version = "3.12.16" }) assert(isSource(package), "=0 must build from source") assert(uvRequests() == 0, "=0 must not read the uv-build index") diff --git a/tests/uv_build_test.lua b/tests/uv_build_test.lua index 59a5a2a..a9435be 100644 --- a/tests/uv_build_test.lua +++ b/tests/uv_build_test.lua @@ -104,9 +104,10 @@ local plugin = test.load({ local available = plugin:Available({}) assert(#available == 3, tostring(#available)) -assert(available[1].version == "3.12.8", available[1].version) -assert(available[2].version == "3.13.1t", available[2].version) --- The invalid checksum is rejected by PreInstall, not hidden from the list. +-- The list is newest first whatever order the index returns: 3.13.1 free-threaded is +-- newer than 3.12.8. The invalid checksum is rejected by PreInstall, not hidden here. +assert(available[1].version == "3.13.1t", available[1].version) +assert(available[2].version == "3.12.8", available[2].version) assert(available[3].version == "3.11.11", available[3].version) local package = plugin:PreInstall({ version = "3.12.8" }) @@ -119,6 +120,7 @@ assert( ) assert(package.sha256 == string.rep("b", 64), tostring(package.sha256)) +-- latest skips the free-threaded 3.13.1t: it is newer, but is installed only when named. local latest = plugin:PreInstall({ version = "latest" }) assert(latest.version == "3.12.8", latest.version) From 2ab0c571438f47db9eb96dba42e48b20334649e9 Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 16:00:11 +0800 Subject: [PATCH 4/9] fix: retry transient failures of the index requests The uv-build and pyenv index requests were sent once, so a brief network error, HTTP 429 or 5xx failed a CPython install in the default mode. A transient failure is now sent again: up to three attempts in all, with pauses of one and then two seconds, and each retry is announced. A failure that persists still fails the install and names VFOX_PYTHON_USE_UV_BUILD=0. The default mode does not fall back to a source build for it, and other statuses such as 404 are not retried. The pause is replaced in the tests that fail an index, because os.execute is disabled in plugin tests. tests/index_retry_test.lua covers the retries, the pause commands on Linux and Windows, and the pyenv index. The default-mode tests count the requests that a failing index receives. --- docs/changes.md | 4 + docs/development.md | 11 ++- docs/uv.md | 8 +- docs/zh-CN/changes.md | 1 + docs/zh-CN/development.md | 3 +- docs/zh-CN/uv.md | 2 +- lib/util.lua | 99 +++++++++++++++++-- tests/index_retry_test.lua | 162 ++++++++++++++++++++++++++++++++ tests/uv_build_default_test.lua | 8 ++ tests/uv_build_test.lua | 5 + 10 files changed, 289 insertions(+), 14 deletions(-) create mode 100644 tests/index_retry_test.lua diff --git a/docs/changes.md b/docs/changes.md index e36b432..68cd05e 100644 --- a/docs/changes.md +++ b/docs/changes.md @@ -4,6 +4,10 @@ ## Unreleased +- Index requests to the uv-build and pyenv indexes are sent again on a network error, + HTTP 429 or a 5xx: up to three attempts in all, with pauses of one and then two seconds. + A failure that persists still fails the install. The default mode does not fall back to + a source build for it. - The uv-build index and pyenv version lists are sorted newest first with one comparator. With `VFOX_PYTHON_USE_UV_BUILD=1`, the list used to keep the order the index returned, so `latest` depended on that order. The pyenv list compared diff --git a/docs/development.md b/docs/development.md index 7d2e447..86fff20 100644 --- a/docs/development.md +++ b/docs/development.md @@ -84,9 +84,14 @@ looked up. index for a CPython version only; a non-CPython name is never looked up there. When the index lists the version, the result is the same as uv-build mode. When it does not, PreInstall prints a line and returns no archive, so vfox runs the - pyenv source build. A failed index request is an error that names - `VFOX_PYTHON_USE_UV_BUILD=0`. When no libc can be detected, the source build is - used. + pyenv source build. A failed index request is retried (see below); an index that + still fails is an error that names `VFOX_PYTHON_USE_UV_BUILD=0`. When no libc can be + detected, the source build is used. +- Index requests (`getIndexWithRetry` in `lib/util.lua`) send a GET again on a transport + error, HTTP 429 or a 5xx: up to three attempts in all, with pauses of one and then two + seconds. Other statuses are returned as they are. The uv-build index and the pyenv index + both use it. A test that makes an index fail replaces `pauseBeforeIndexRetry`, because + `os.execute` is disabled in plugin tests. - `note = "uv-build"`, `"GraalPy"` or `"GraalPy Community"` is shown by vfox. ### PostInstall diff --git a/docs/uv.md b/docs/uv.md index ff45458..82c76b0 100644 --- a/docs/uv.md +++ b/docs/uv.md @@ -48,9 +48,11 @@ release from source. - `1`, `true`, `yes` or `on`: uv-build only. A version the index does not list is an error. -If the index cannot be read, the install fails and names -`VFOX_PYTHON_USE_UV_BUILD=0`, so it never starts a source build without saying -so. If the index has no archive for this version on this platform, such as an +An index request that fails with a network error, HTTP 429 or a 5xx status is +sent again, up to three attempts in all, with a pause of one second and then two +seconds. Other statuses are not retried. If the index still cannot be read after +that, the install fails and names `VFOX_PYTHON_USE_UV_BUILD=0`, so it never starts a +source build without saying so. If the index has no archive for this version on this platform, such as an architecture it does not cover, the install builds from source and prints a line saying so. On Linux, if the libc cannot be detected, the install builds from source; set `VFOX_PYTHON_UV_LIBC` to use uv-build. diff --git a/docs/zh-CN/changes.md b/docs/zh-CN/changes.md index 0396c45..6934c2e 100644 --- a/docs/zh-CN/changes.md +++ b/docs/zh-CN/changes.md @@ -4,6 +4,7 @@ ## 未发布 +- uv-build 和 pyenv 索引的请求遇到网络错误、HTTP 429 或 5xx 时会重发:总共最多三次,间隔 1 秒和 2 秒。持续失败仍会使安装失败;默认模式不会因此改为源码构建。 - uv-build 索引和 pyenv 版本列表现在用同一个比较函数从新到旧排序。过去 `VFOX_PYTHON_USE_UV_BUILD=1` 的列表保留索引返回的顺序,`latest` 依赖这个顺序;pyenv 列表则把带后缀的名字(如 `3.13.18t`)当成 `3.13.0` 来比较。`latest` 现在选最新的常规版本,并跳过自由线程版;自由线程版只有写出名字时才会安装。 diff --git a/docs/zh-CN/development.md b/docs/zh-CN/development.md index b72593d..98919e1 100644 --- a/docs/zh-CN/development.md +++ b/docs/zh-CN/development.md @@ -70,8 +70,9 @@ vfox 加载 `metadata.lua`,命令需要时调用相应的钩子,并向钩子 安装失败,而不是跳过校验。GraalPy 会被拒绝,因为索引里只有 CPython。 - 默认模式(未设置 `VFOX_PYTHON_USE_UV_BUILD`,在 Linux 和 macOS 上,且未设置 `VFOX_PYTHON_MIRROR`,除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`)只在索引里查 CPython 版本, 非 CPython 的名字从不在那里查找。索引列出该版本时,结果与 uv-build 模式相同。索引没有列出时, - PreInstall 会打印一行,并且不返回包,于是 vfox 运行 pyenv 源码构建。索引请求失败会报错, + PreInstall 会打印一行,并且不返回包,于是 vfox 运行 pyenv 源码构建。索引请求失败会先重试(见下),仍失败则报错, 错误信息里写明 `VFOX_PYTHON_USE_UV_BUILD=0`。检测不到 libc 时,使用源码构建。 +- 索引请求(`lib/util.lua` 里的 `getIndexWithRetry`)遇到网络错误、HTTP 429 或 5xx 时会重发:总共最多三次,两次之间分别等待 1 秒和 2 秒。其他状态码原样返回。uv-build 索引和 pyenv 索引都用它。让索引失败的测试要替换 `pauseBeforeIndexRetry`,因为插件测试里 `os.execute` 被禁用。 - `note = "uv-build"`、`"GraalPy"` 或 `"GraalPy Community"` 会显示在 vfox 里。 ### PostInstall diff --git a/docs/zh-CN/uv.md b/docs/zh-CN/uv.md index cba9441..b9b6137 100644 --- a/docs/zh-CN/uv.md +++ b/docs/zh-CN/uv.md @@ -35,7 +35,7 @@ vfox install python@3.12 - `0`、`false`、`no` 或 `off`:在 Linux 和 macOS 上源码构建。 - `1`、`true`、`yes` 或 `on`:只用 uv-build。索引里没有列出的版本会报错。 -如果无法读取索引,安装会失败,并提示 `VFOX_PYTHON_USE_UV_BUILD=0`,因此不会在没有说明的情况下开始源码构建。如果索引在该平台上没有这个版本的包(比如索引不覆盖某个架构),安装会改为源码构建,并打印一行说明。在 Linux 上,如果无法识别 libc,安装会从源码构建;要使用 uv-build,请设置 `VFOX_PYTHON_UV_LIBC`。 +索引请求遇到网络错误、HTTP 429 或 5xx 状态码时会重发,总共最多三次,两次之间分别等待 1 秒和 2 秒;其他状态码不重试。重试后仍无法读取索引,安装会失败,并提示 `VFOX_PYTHON_USE_UV_BUILD=0`,因此不会在没有说明的情况下开始源码构建。如果索引在该平台上没有这个版本的包(比如索引不覆盖某个架构),安装会改为源码构建,并打印一行说明。在 Linux 上,如果无法识别 libc,安装会从源码构建;要使用 uv-build,请设置 `VFOX_PYTHON_UV_LIBC`。 插件向 `https://vault.vfox.dev/python/uv-build` 询问当前平台、架构和 libc 的 构建,然后把包地址和 SHA-256 交给 vfox。vfox 下载、校验、解压,再调用 diff --git a/lib/util.lua b/lib/util.lua index 9b80ee2..a0a0ee3 100644 --- a/lib/util.lua +++ b/lib/util.lua @@ -828,15 +828,95 @@ local function uvBuildDownloadUrl(build) return trimTrailingSlash(mirror) .. "/" .. release .. "/" .. filename end +--- How many times an index request is sent before a failure stands. Both indexes are +--- read on the way to a CPython install, so one brief failure of a server should not +--- fail the install by itself. +local INDEX_ATTEMPTS = 3 + +--- Whether a response to an index request is worth another attempt: a transport error, +--- HTTP 429, or a 5xx. Any other status is an answer, such as a 404, and is kept. +local function isTransientIndexFailure(resp, err) + if err ~= nil or resp == nil then + return true + end + local status = resp.status_code + return status == 429 or (type(status) == "number" and status >= 500) +end + +--- Describes a failed index request for a warning, such as "HTTP 503". +local function describeIndexFailure(resp, err) + if err ~= nil then + return tostring(err) + end + if resp == nil then + return "no response" + end + return "HTTP " .. tostring(resp.status_code) +end + +--- The part of an error message that says how often the request was sent: nothing for +--- one attempt, otherwise " after 3 attempts". +local function attemptsNote(attempts) + if attempts == 1 then + return "" + end + return " after " .. tostring(attempts) .. " attempts" +end + +--- Waits before the next attempt at an index request: one second before the second +--- attempt and two before the third. The wait runs in the shell, so if it fails the +--- next attempt simply comes sooner. Tests replace this function to skip the wait. +function pauseBeforeIndexRetry(attempt) + local seconds = tostring(attempt) + if runtimeOs() == "windows" then + os.execute(powerShellCommand("Start-Sleep -Seconds " .. seconds)) + else + os.execute("sleep " .. seconds) + end +end + +--- Sends a GET request to an index. A transient failure is sent again, up to +--- INDEX_ATTEMPTS attempts in all, and each retry is announced. It returns the last +--- response, its error and the number of attempts, so a failure that persists is +--- reported by the caller as before. +local function getIndexWithRetry(name, request) + local attempt = 1 + local resp, err = http.get(request) + while isTransientIndexFailure(resp, err) and attempt < INDEX_ATTEMPTS do + print( + "Warning: the " + .. name + .. " request failed (" + .. describeIndexFailure(resp, err) + .. "). Trying again, attempt " + .. tostring(attempt + 1) + .. " of " + .. tostring(INDEX_ATTEMPTS) + .. "." + ) + pauseBeforeIndexRetry(attempt) + attempt = attempt + 1 + resp, err = http.get(request) + end + return resp, err, attempt +end + local function getUvBuilds(osType, archType, libc) fixHeaders() - local resp, err = http.get({ + local resp, err, attempts = getIndexWithRetry("uv-build index", { url = buildUvBuildUrl(osType, archType, libc), headers = REQUEST_HEADERS, }) - if err ~= nil or resp.status_code ~= 200 then + if err ~= nil or resp == nil or resp.status_code ~= 200 then local statusCode = resp and resp.status_code or "none" - error("parsing uv-build release info failed. Status: " .. statusCode .. ", Error: " .. (err or "none")) + error( + "parsing uv-build release info failed" + .. attemptsNote(attempts) + .. ". Status: " + .. statusCode + .. ", Error: " + .. (err or "none") + ) end local jsonObj = json.decode(resp.body) @@ -1665,13 +1745,20 @@ end function parseVersionFromPyenv() fixHeaders() - local resp, err = http.get({ + local resp, err, attempts = getIndexWithRetry("pyenv index", { url = version_vault_url, headers = REQUEST_HEADERS, }) - if err ~= nil or resp.status_code ~= 200 then + if err ~= nil or resp == nil or resp.status_code ~= 200 then local statusCode = resp and resp.status_code or "none" - error("parsing release info failed. Status: " .. statusCode .. ", Error: " .. (err or "none")) + error( + "parsing release info failed" + .. attemptsNote(attempts) + .. ". Status: " + .. statusCode + .. ", Error: " + .. (err or "none") + ) end local result = {} local jsonObj = json.decode(resp.body) diff --git a/tests/index_retry_test.lua b/tests/index_retry_test.lua new file mode 100644 index 0000000..1114781 --- /dev/null +++ b/tests/index_retry_test.lua @@ -0,0 +1,162 @@ +-- Copyright 2026 Han Li and contributors +-- +-- Licensed under the Apache License, Version 2.0 (the "License"); +-- you may not use this file except in compliance with the License. +-- You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, software +-- distributed under the License is distributed on an "AS IS" BASIS, +-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +-- See the License for the specific language governing permissions and +-- limitations under the License. + +-- An index request that fails transiently is sent again: up to three attempts in all, +-- with a pause of one second and then two before the retries. A failure that persists +-- is still an error, and a client error such as 404 is not retried. os.execute is +-- disabled in plugin tests, so the pause is replaced and the test records it instead. +local test = require("vfox.test") +local json = require("json") + +local UV_INDEX = "https://vault.vfox.dev/python/uv-build?" +local PYENV_INDEX = "https://vault.vfox.dev/python/pyenv" +local SHA = string.rep("a", 64) + +-- The uv-build index for linux/x86_64/gnu, with one CPython release. +local UV_BODY = json.encode({ + items = { + { + implementation = "cpython", + version = "3.12.16", + variant = "default", + filename = "cpython-3.12.16+20260101-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + url = "https://github.com/astral-sh/python-build-standalone/releases/download/20260101/" + .. "cpython-3.12.16+20260101-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + asset = { sha256 = SHA }, + }, + }, +}) + +local PYENV_BODY = json.encode({ tagName = "v2.6.0", versions = { "3.12.16" } }) + +-- Each request takes the next step, and the last step repeats. A step is +-- { error = "..." } for a transport error, or { status = N, body = "..." }. +local state = { steps = {}, urls = {} } +local pauses = {} +local env = {} + +local plugin = test.load({ + os = "linux", + arch = "amd64", + env = {}, + http = function(request) + table.insert(state.urls, request.url) + local step = state.steps[math.min(#state.urls, #state.steps)] + if step.error ~= nil then + return nil, step.error + end + return { status_code = step.status, body = step.body } + end, +}) + +-- The real pause, kept before the test replaces it below. +local realPause = pauseBeforeIndexRetry + +-- The plugin reads its variables at call time, so each case sets the ones it needs. +os.getenv = function(name) + return env[name] +end + +pauseBeforeIndexRetry = function(attempt) + table.insert(pauses, attempt) +end + +--- Runs fn against the given steps. It returns whether fn succeeded, its result or +--- error, the number of requests sent, and the pauses taken, as "1,2". +local function run(steps, fn) + state.urls = {} + state.steps = steps + pauses = {} + local ok, result = pcall(fn) + -- table.concat hands back a lone number unchanged in GopherLua, so it is made a string. + return ok, result, #state.urls, tostring(table.concat(pauses, ",")) +end + +local function available() + return plugin:Available({}) +end + +local function fails(ok, err, expected) + assert(not ok and tostring(err):find(expected, 1, true), tostring(err)) +end + +local OK = { status = 200, body = UV_BODY } +local TRANSPORT = { error = "connection reset by peer" } +local UV_ENV = { VFOX_PYTHON_USE_UV_BUILD = "1", VFOX_PYTHON_UV_LIBC = "gnu" } +local PYENV_ENV = { VFOX_PYTHON_USE_UV_BUILD = "0", VFOX_PYTHON_UV_LIBC = "gnu" } + +-- A 503, then a success: one pause, two requests. +env = UV_ENV +local ok, result, sent, paused = run({ { status = 503 }, OK }, available) +assert(ok, tostring(result)) +assert(result[1].version == "3.12.16", tostring(result[1] and result[1].version)) +assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) + +-- A transport error, then a success. +ok, result, sent, paused = run({ TRANSPORT, OK }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) + +-- A 429, then a 500, then a success: two pauses, of one and then two seconds. +ok, result, sent, paused = run({ { status = 429 }, { status = 500 }, OK }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 3 and paused == "1,2", "sent " .. sent .. ", paused " .. paused) + +-- A failure that persists: three attempts, then an error that says so. +ok, result, sent, paused = run({ { status = 503 } }, available) +fails(ok, result, "Status: 503") +fails(ok, result, "after 3 attempts") +assert(sent == 3 and paused == "1,2", "sent " .. sent .. ", paused " .. paused) + +-- A transport error that persists is retried the same way. +ok, result, sent, paused = run({ TRANSPORT }, available) +fails(ok, result, "connection reset by peer") +assert(sent == 3 and paused == "1,2", "sent " .. sent .. ", paused " .. paused) + +-- A 404 is an answer, not a server failure: no retry, no pause, and no attempt count. +ok, result, sent, paused = run({ { status = 404 } }, available) +fails(ok, result, "Status: 404") +assert(not tostring(result):find(" attempts", 1, true), tostring(result)) +assert(sent == 1 and paused == "", "sent " .. sent .. ", paused " .. paused) + +-- An install retries the same way: PreInstall reads the index for an exact version. +ok, result, sent, paused = run({ { status = 503 }, OK }, function() + return plugin:PreInstall({ version = "3.12.16" }) +end) +assert(ok and result.note == "uv-build" and result.sha256 == SHA, tostring(result)) +assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) + +-- The pyenv list is retried as well, in the source-build mode. +env = PYENV_ENV +ok, result, sent, paused = run({ { status = 500 }, { status = 200, body = PYENV_BODY } }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) +assert(state.urls[1] == PYENV_INDEX, state.urls[1]) + +-- The real wait is a shell command: `sleep` on Linux and macOS, PowerShell on Windows. +-- The shell is stubbed here to capture the command instead of running it. +local commands = {} +os.execute = function(command) + table.insert(commands, command) + return 0 +end +realPause(1) +RUNTIME.osType = "windows" +realPause(2) +RUNTIME.osType = "linux" +assert(commands[1] == "sleep 1", commands[1]) +assert(string.find(commands[2], "powershell", 1, true), commands[2]) +assert(string.find(commands[2], "Start-Sleep -Seconds 2", 1, true), commands[2]) + +print("PASS: index requests are retried on transient failures and not on client errors") diff --git a/tests/uv_build_default_test.lua b/tests/uv_build_default_test.lua index 4de3406..1d0f1f6 100644 --- a/tests/uv_build_default_test.lua +++ b/tests/uv_build_default_test.lua @@ -93,6 +93,10 @@ os.getenv = function(name) return env[name] end +-- A transient failure is sent again after a pause. The pause is replaced here, since +-- os.execute is disabled in plugin tests; index_retry_test.lua checks the retries. +pauseBeforeIndexRetry = function() end + local function uvRequests() local count = 0 for _, url in ipairs(state.requests) do @@ -207,17 +211,21 @@ assert(uvRequests() == 0, "a non-CPython name must not read the uv-build index") -- An index that cannot be read is an error that names the opt-out; it is not a -- silent source build. The list still shows the pyenv names. state.uvStatus = 503 +reset() local ok, err = pcall(function() plugin:PreInstall({ version = "3.12.16" }) end) assert(not ok and tostring(err):find("Could not read the uv-build index", 1, true), tostring(err)) assert(tostring(err):find("VFOX_PYTHON_USE_UV_BUILD=0", 1, true), tostring(err)) +assert(uvRequests() == 3, "a failing index is sent three times before the install gives up") -- A partial request fails the same way: it must not fall back to a source build. +reset() ok, err = pcall(function() plugin:PreInstall({ version = "3.13" }) end) assert(not ok and tostring(err):find("Could not read the uv-build index", 1, true), tostring(err)) +assert(uvRequests() == 3, "a partial request also retries the index before it fails") assert(#plugin:Available({}) == 8, "the pyenv names are listed when the index fails") state.uvStatus = 200 diff --git a/tests/uv_build_test.lua b/tests/uv_build_test.lua index a9435be..922fadb 100644 --- a/tests/uv_build_test.lua +++ b/tests/uv_build_test.lua @@ -102,6 +102,10 @@ local plugin = test.load({ end, }) +-- A transient failure is sent again after a pause. os.execute is disabled in plugin +-- tests, so the pause is replaced here; index_retry_test.lua checks the retries. +pauseBeforeIndexRetry = function() end + local available = plugin:Available({}) assert(#available == 3, tostring(#available)) -- The list is newest first whatever order the index returns: 3.13.1 free-threaded is @@ -186,5 +190,6 @@ local ok, err = pcall(function() plugin:Available({}) end) assert(not ok and tostring(err):find("503", 1, true), tostring(err)) +assert(tostring(err):find("after 3 attempts", 1, true), tostring(err)) print("PASS: uv-build listing, archive priority, checksum and mirror") From bba17a946b727a4a97c1ee06f2a3779859e9e154 Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 16:00:11 +0800 Subject: [PATCH 5/9] docs: say that latest does not pick a newer free-threaded build latest selects the newest standard release even when a free-threaded build is newer, and a free-threaded build is installed only when its name asks for it. Both modes are tested (uv_build_test.lua and uv_build_default_test.lua); the English and Chinese version pages now say so as well. --- docs/versions.md | 3 ++- docs/zh-CN/versions.md | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/versions.md b/docs/versions.md index b2605ea..1fe6f38 100644 --- a/docs/versions.md +++ b/docs/versions.md @@ -39,7 +39,8 @@ pre-release name, so `python@3.13` is not `python@3.13t`. is not a free-threaded build. The list is newest CPython first, so this is the newest standard CPython release the platform lists, which can be a pre-release when one exists. A free-threaded build such as `3.13.18t` is installed only when its name asks -for it. GraalPy is selected by its full name. In the default mode on Linux and macOS, `latest` is not +for it, even when it is newer than every standard release. GraalPy is selected by its +full name. In the default mode on Linux and macOS, `latest` is not resolved in the uv-build index, so it stays the newest release in the list: it installs from its archive when the index lists that release, and is built from source, with a line saying so, when pyenv lists a newer one. diff --git a/docs/zh-CN/versions.md b/docs/zh-CN/versions.md index 6b4139a..5e402b5 100644 --- a/docs/zh-CN/versions.md +++ b/docs/zh-CN/versions.md @@ -32,7 +32,7 @@ vfox use python@3.12 # 已安装的 3.12.x 里最新的 `latest` 由 `PreInstall` 解析成 `Available` 返回的、第一个不是自由线程版的版本。列表按从新到旧 排列,所以它是这个平台列出的最新常规 CPython 版本;有预发布版时,也可能是一个 -预发布版。自由线程版(如 `3.13.18t`)只有写出名字时才会安装。GraalPy 需要写出完整版本名。在 Linux 和 macOS 的默认模式下,`latest` 不在 uv-build 索引里解析, +预发布版。自由线程版(如 `3.13.18t`)即使比所有常规版本都新,也只有写出名字时才会安装。GraalPy 需要写出完整版本名。在 Linux 和 macOS 的默认模式下,`latest` 不在 uv-build 索引里解析, 它仍是列表里最新的版本:索引列出该版本时安装它的预编译包;pyenv 列出的版本比索引新时,则从源码编译, 并打印一行说明。 From ad2a040d7ee6473d68eb3d564d57eb14b71f0671 Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 17:22:23 +0800 Subject: [PATCH 6/9] fix: honour Retry-After and retry index bodies that are not the index An index request reads the Retry-After header, in seconds, and waits for it when it is at most ten seconds. Without a usable header it waits one second, then two. A longer Retry-After fails at once and names the wait. HTTP-date values are not read. A 200 response that is not the index is sent again too. Before, an HTML page raised a Lua error, and a JSON object without the list was read as an empty list. The body check is inside the retry loop, so it applies to both indexes. The reads and the waits live in readIndex in lib/util.lua. The tests cover the Retry-After forms, the captive-portal body, the pause commands on Linux and Windows, and the uv-build and pyenv indexes. --- lib/util.lua | 155 ++++++++++++++++++++----------------- tests/index_retry_test.lua | 71 +++++++++++++---- 2 files changed, 137 insertions(+), 89 deletions(-) diff --git a/lib/util.lua b/lib/util.lua index a0a0ee3..5960a8f 100644 --- a/lib/util.lua +++ b/lib/util.lua @@ -828,30 +828,38 @@ local function uvBuildDownloadUrl(build) return trimTrailingSlash(mirror) .. "/" .. release .. "/" .. filename end ---- How many times an index request is sent before a failure stands. Both indexes are ---- read on the way to a CPython install, so one brief failure of a server should not ---- fail the install by itself. +--- An index request is sent up to INDEX_ATTEMPTS times in all. Before each further +--- attempt the plugin waits as long as the server's Retry-After asks, up to +--- INDEX_MAX_WAIT_SECONDS. Without a usable Retry-After it waits INDEX_BACKOFF_SECONDS. +--- A longer Retry-After is not waited for: the install fails and says how long it asked. local INDEX_ATTEMPTS = 3 - ---- Whether a response to an index request is worth another attempt: a transport error, ---- HTTP 429, or a 5xx. Any other status is an answer, such as a 404, and is kept. -local function isTransientIndexFailure(resp, err) - if err ~= nil or resp == nil then - return true +local INDEX_MAX_WAIT_SECONDS = 10 +local INDEX_BACKOFF_SECONDS = { 1, 2 } + +--- The number of seconds a Retry-After header asks for, or nil when there is no such +--- header or it is an HTTP date, which is not read. The header name is matched without +--- regard to case, because the response keeps the spelling the server used. +local function retryAfterSeconds(resp) + if type(resp) ~= "table" or type(resp.headers) ~= "table" then + return nil end - local status = resp.status_code - return status == 429 or (type(status) == "number" and status >= 500) + for name, value in pairs(resp.headers) do + if type(name) == "string" and string.lower(name) == "retry-after" and type(value) == "string" then + local seconds = string.match(value, "^%s*(%d+)%s*$") + return seconds and tonumber(seconds) or nil + end + end + return nil end ---- Describes a failed index request for a warning, such as "HTTP 503". -local function describeIndexFailure(resp, err) - if err ~= nil then - return tostring(err) - end - if resp == nil then - return "no response" - end - return "HTTP " .. tostring(resp.status_code) +--- Whether a decoded uv-build index has the list the plugin reads. +local function isUvBuildIndex(value) + return type(value.items) == "table" or type(value.versions) == "table" +end + +--- Whether a decoded pyenv index has the list the plugin reads. +local function isPyenvIndex(value) + return type(value.versions) == "table" end --- The part of an error message that says how often the request was sent: nothing for @@ -863,64 +871,76 @@ local function attemptsNote(attempts) return " after " .. tostring(attempts) .. " attempts" end ---- Waits before the next attempt at an index request: one second before the second ---- attempt and two before the third. The wait runs in the shell, so if it fails the ---- next attempt simply comes sooner. Tests replace this function to skip the wait. -function pauseBeforeIndexRetry(attempt) - local seconds = tostring(attempt) +--- Waits `seconds` before the next attempt at an index request. The wait is a shell +--- command: `sleep` on Linux and macOS, PowerShell on Windows. Tests replace it. +function pauseBeforeIndexRetry(seconds) if runtimeOs() == "windows" then - os.execute(powerShellCommand("Start-Sleep -Seconds " .. seconds)) + os.execute(powerShellCommand("Start-Sleep -Seconds " .. tostring(seconds))) else - os.execute("sleep " .. seconds) - end -end - ---- Sends a GET request to an index. A transient failure is sent again, up to ---- INDEX_ATTEMPTS attempts in all, and each retry is announced. It returns the last ---- response, its error and the number of attempts, so a failure that persists is ---- reported by the caller as before. -local function getIndexWithRetry(name, request) - local attempt = 1 - local resp, err = http.get(request) - while isTransientIndexFailure(resp, err) and attempt < INDEX_ATTEMPTS do + os.execute("sleep " .. tostring(seconds)) + end +end + +--- Reads an index and decodes it. A transient failure is sent again: a transport error, +--- HTTP 429 or a 5xx, and a 200 whose body is not JSON or lacks the list the caller +--- reads, since a proxy or a captive portal can answer that way. Returns the decoded +--- index, or nil with the reason for the last failure and the number of attempts. +local function readIndex(name, request, isIndex) + local attempts = 0 + while true do + attempts = attempts + 1 + local resp, err = http.get(request) + local transient, reason + if err ~= nil or resp == nil then + transient = true + reason = "Status: none, Error: " .. tostring(err or "none") + elseif resp.status_code == 200 then + local ok, value = pcall(json.decode, resp.body or "") + if ok and type(value) == "table" and isIndex(value) then + return value, nil, attempts + end + transient = true + reason = "Status: 200, Error: the body is not the expected index" + else + local status = resp.status_code + transient = status == 429 or (type(status) == "number" and status >= 500) + reason = "Status: " .. tostring(status) .. ", Error: none" + end + if not transient or attempts >= INDEX_ATTEMPTS then + return nil, reason, attempts + end + local asked = retryAfterSeconds(resp) + local wait = asked or INDEX_BACKOFF_SECONDS[attempts] + if wait > INDEX_MAX_WAIT_SECONDS then + return nil, + reason .. ", and the index asked to wait " .. tostring(asked) .. " seconds, more than the plugin waits", + attempts + end print( "Warning: the " .. name .. " request failed (" - .. describeIndexFailure(resp, err) + .. reason .. "). Trying again, attempt " - .. tostring(attempt + 1) + .. tostring(attempts + 1) .. " of " .. tostring(INDEX_ATTEMPTS) .. "." ) - pauseBeforeIndexRetry(attempt) - attempt = attempt + 1 - resp, err = http.get(request) + pauseBeforeIndexRetry(wait) end - return resp, err, attempt end local function getUvBuilds(osType, archType, libc) fixHeaders() - local resp, err, attempts = getIndexWithRetry("uv-build index", { + local index, reason, attempts = readIndex("uv-build index", { url = buildUvBuildUrl(osType, archType, libc), headers = REQUEST_HEADERS, - }) - if err ~= nil or resp == nil or resp.status_code ~= 200 then - local statusCode = resp and resp.status_code or "none" - error( - "parsing uv-build release info failed" - .. attemptsNote(attempts) - .. ". Status: " - .. statusCode - .. ", Error: " - .. (err or "none") - ) + }, isUvBuildIndex) + if index == nil then + error("parsing uv-build release info failed" .. attemptsNote(attempts) .. ". " .. reason) end - - local jsonObj = json.decode(resp.body) - return jsonObj.items or jsonObj.versions or {} + return index.items or index.versions or {} end --- The build the listing has for version, the one with the best asset priority, @@ -1745,23 +1765,14 @@ end function parseVersionFromPyenv() fixHeaders() - local resp, err, attempts = getIndexWithRetry("pyenv index", { + local jsonObj, reason, attempts = readIndex("pyenv index", { url = version_vault_url, headers = REQUEST_HEADERS, - }) - if err ~= nil or resp == nil or resp.status_code ~= 200 then - local statusCode = resp and resp.status_code or "none" - error( - "parsing release info failed" - .. attemptsNote(attempts) - .. ". Status: " - .. statusCode - .. ", Error: " - .. (err or "none") - ) + }, isPyenvIndex) + if jsonObj == nil then + error("parsing release info failed" .. attemptsNote(attempts) .. ". " .. reason) end local result = {} - local jsonObj = json.decode(resp.body) local tagName = jsonObj.tagName if tagName then diff --git a/tests/index_retry_test.lua b/tests/index_retry_test.lua index 1114781..f75cff6 100644 --- a/tests/index_retry_test.lua +++ b/tests/index_retry_test.lua @@ -12,10 +12,12 @@ -- See the License for the specific language governing permissions and -- limitations under the License. --- An index request that fails transiently is sent again: up to three attempts in all, --- with a pause of one second and then two before the retries. A failure that persists --- is still an error, and a client error such as 404 is not retried. os.execute is --- disabled in plugin tests, so the pause is replaced and the test records it instead. +-- An index request that fails transiently is sent again, up to three attempts in all. +-- The wait is the Retry-After the server sent when it is usable (ten seconds at most), +-- and otherwise one second and then two. A 200 whose body is not the expected index is +-- sent again too, since a proxy or a captive portal can answer that way. Client errors +-- such as 404 are not retried. os.execute is disabled in plugin tests, so the pause is +-- replaced here and the test records the seconds instead. local test = require("vfox.test") local json = require("json") @@ -40,8 +42,8 @@ local UV_BODY = json.encode({ local PYENV_BODY = json.encode({ tagName = "v2.6.0", versions = { "3.12.16" } }) --- Each request takes the next step, and the last step repeats. A step is --- { error = "..." } for a transport error, or { status = N, body = "..." }. +-- Each request takes the next step, and the last step repeats. A step is { error = ... } +-- for a transport error, or { status = N, body = ..., headers = ... } for a response. local state = { steps = {}, urls = {} } local pauses = {} local env = {} @@ -56,30 +58,28 @@ local plugin = test.load({ if step.error ~= nil then return nil, step.error end - return { status_code = step.status, body = step.body } + return { status_code = step.status, body = step.body, headers = step.headers } end, }) --- The real pause, kept before the test replaces it below. -local realPause = pauseBeforeIndexRetry - -- The plugin reads its variables at call time, so each case sets the ones it needs. os.getenv = function(name) return env[name] end -pauseBeforeIndexRetry = function(attempt) - table.insert(pauses, attempt) +-- The real pause, kept for the check of its commands below, and replaced for the cases. +local realPause = pauseBeforeIndexRetry +pauseBeforeIndexRetry = function(seconds) + table.insert(pauses, tostring(seconds)) end ---- Runs fn against the given steps. It returns whether fn succeeded, its result or +--- Runs fn against the given steps. It returns whether fn succeeded, its result or its --- error, the number of requests sent, and the pauses taken, as "1,2". local function run(steps, fn) state.urls = {} state.steps = steps pauses = {} local ok, result = pcall(fn) - -- table.concat hands back a lone number unchanged in GopherLua, so it is made a string. return ok, result, #state.urls, tostring(table.concat(pauses, ",")) end @@ -96,7 +96,7 @@ local TRANSPORT = { error = "connection reset by peer" } local UV_ENV = { VFOX_PYTHON_USE_UV_BUILD = "1", VFOX_PYTHON_UV_LIBC = "gnu" } local PYENV_ENV = { VFOX_PYTHON_USE_UV_BUILD = "0", VFOX_PYTHON_UV_LIBC = "gnu" } --- A 503, then a success: one pause, two requests. +-- A 503, then a success: one pause of one second, two requests. env = UV_ENV local ok, result, sent, paused = run({ { status = 503 }, OK }, available) assert(ok, tostring(result)) @@ -137,9 +137,46 @@ end) assert(ok and result.note == "uv-build" and result.sha256 == SHA, tostring(result)) assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) --- The pyenv list is retried as well, in the source-build mode. +-- A Retry-After in seconds replaces the fixed wait, whatever the case of its name. +ok, result, sent, paused = run({ { status = 429, headers = { ["Retry-After"] = "3" } }, OK }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 2 and paused == "3", "sent " .. sent .. ", paused " .. paused) + +ok, result, sent, paused = run({ { status = 503, headers = { ["retry-after"] = "4" } }, OK }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 2 and paused == "4", "sent " .. sent .. ", paused " .. paused) + +-- A Retry-After longer than ten seconds is not waited for: one request, and the failure +-- names the wait the server asked for. +ok, result, sent, paused = run({ { status = 503, headers = { ["Retry-After"] = "60" } }, OK }, available) +fails(ok, result, "asked to wait 60 seconds") +fails(ok, result, "Status: 503") +assert(sent == 1 and paused == "", "sent " .. sent .. ", paused " .. paused) + +-- An HTTP date is not read, so the fixed wait applies. +ok, result, sent, paused = + run({ { status = 503, headers = { ["Retry-After"] = "Wed, 21 Oct 2026 07:28:00 GMT" } }, OK }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) + +-- A 200 with a body that is not JSON is sent again, as is one whose JSON lacks the list. +ok, result, sent, paused = run({ { status = 200, body = "captive portal" }, OK }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) + +ok, result, sent, paused = run({ { status = 200, body = '{"message":"rate limited"}' }, OK }, available) +assert(ok and result[1].version == "3.12.16", tostring(result)) +assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) + +-- A body that never becomes an index fails after three attempts, and says what it was. +ok, result, sent, paused = run({ { status = 200, body = "not json" } }, available) +fails(ok, result, "the body is not the expected index") +fails(ok, result, "after 3 attempts") +assert(sent == 3 and paused == "1,2", "sent " .. sent .. ", paused " .. paused) + +-- The pyenv index is retried in the same way, in the source-build mode. env = PYENV_ENV -ok, result, sent, paused = run({ { status = 500 }, { status = 200, body = PYENV_BODY } }, available) +ok, result, sent, paused = run({ { status = 200, body = "" }, { status = 200, body = PYENV_BODY } }, available) assert(ok and result[1].version == "3.12.16", tostring(result)) assert(sent == 2 and paused == "1", "sent " .. sent .. ", paused " .. paused) assert(state.urls[1] == PYENV_INDEX, state.urls[1]) From 701cc71ef903ae57d6448456a56b8e5eae317fc3 Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 17:22:23 +0800 Subject: [PATCH 7/9] ci: check the Windows index retry wait and the docs in both languages test.yml runs a search through a closed proxy on Windows, with VFOX_PYTHON_USE_UV_BUILD=1. It must fail after three attempts, show two retries, and take at least three seconds, which only happens when the PowerShell pause runs through os.execute. The plugin tests never sleep, so this is the only check of the real Windows wait. lint.yml gets a docs job that runs .github/scripts/check_docs_parity.py. It compares each Chinese page with its English page: both exist, both name the same VFOX_* variables, both have the same number of fenced code blocks, and every relative link resolves. Sentences are not compared; translations differ on purpose. --- .github/scripts/check_docs_parity.py | 105 +++++++++++++++++++++++++++ .github/workflows/lint.yml | 10 +++ .github/workflows/test.yml | 31 ++++++++ 3 files changed, 146 insertions(+) create mode 100644 .github/scripts/check_docs_parity.py diff --git a/.github/scripts/check_docs_parity.py b/.github/scripts/check_docs_parity.py new file mode 100644 index 0000000..d7f268b --- /dev/null +++ b/.github/scripts/check_docs_parity.py @@ -0,0 +1,105 @@ +# Copyright 2026 Han Li and contributors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Checks that the Chinese documents keep the structure of the English ones. + +The sentences are translated, so they are not compared. The checks are the ones a +reader trips over when the two languages drift apart: + +- every English page has a Chinese page, and every Chinese page has an English one; +- both pages name the same VFOX_* environment variables; +- both pages have the same number of fenced code blocks; +- every relative link resolves from the folder of the page that holds it. + +Run it from the repository root: + + python3 -I .github/scripts/check_docs_parity.py . +""" + +import pathlib +import re +import sys + +FENCE = re.compile(r"^[ \t]*```[^\n]*\n(.*?)^[ \t]*```[ \t]*$", re.S | re.M) +ENV_NAME = re.compile(r"\bVFOX_[A-Z0-9_]+\b") +LINK = re.compile(r"\]\(([^)\s]+)") +SCHEME = re.compile(r"^[A-Za-z][A-Za-z0-9+.-]*:") + + +def read(path): + return path.read_text(encoding="utf-8") + + +def page_pairs(root): + pairs = [(root / "README.md", root / "README_CN.md")] + for english in sorted((root / "docs").glob("*.md")): + pairs.append((english, root / "docs" / "zh-CN" / english.name)) + return pairs + + +def relative_links(text): + # Code blocks can hold anything that looks like a link, so they are skipped. + for target in LINK.findall(FENCE.sub("", text)): + if SCHEME.match(target) or target.startswith("#"): + continue + yield target.split("#", 1)[0] + + +def check_pair(root, english, chinese, problems): + name = english.relative_to(root) + if not chinese.is_file(): + problems.append(f"{name}: no Chinese page at {chinese.relative_to(root)}") + return 0 + + en_text = read(english) + zh_text = read(chinese) + en_env = set(ENV_NAME.findall(en_text)) + zh_env = set(ENV_NAME.findall(zh_text)) + for variable in sorted(en_env - zh_env): + problems.append(f"{name}: {variable} appears in the English page only") + for variable in sorted(zh_env - en_env): + problems.append(f"{name}: {variable} appears in the Chinese page only") + + en_blocks = len(FENCE.findall(en_text)) + zh_blocks = len(FENCE.findall(zh_text)) + if en_blocks != zh_blocks: + problems.append(f"{name}: {en_blocks} code blocks in English, {zh_blocks} in Chinese") + + for page in (english, chinese): + for target in relative_links(read(page)): + if not (page.parent / target).exists(): + problems.append(f"{page.relative_to(root)}: the link to {target} does not exist") + return 1 + + +def main(argv): + root = pathlib.Path(argv[1] if len(argv) > 1 else ".").resolve() + problems = [] + checked = 0 + for english, chinese in page_pairs(root): + checked += check_pair(root, english, chinese, problems) + for chinese in sorted((root / "docs" / "zh-CN").glob("*.md")): + if not (root / "docs" / chinese.name).is_file(): + problems.append(f"{chinese.relative_to(root)}: no English page for it") + + for problem in problems: + print(problem) + if problems: + return 1 + print(f"Docs parity: {checked} pages checked, the English and Chinese pages agree.") + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv)) diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 5d50a7a..fb5a345 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -45,3 +45,13 @@ jobs: run: mise x aqua:LuaLS/lua-language-server@3.19.1 -- lua-language-server --check . --checklevel=Warning - name: Check workflow files run: mise x aqua:rhysd/actionlint@1.7.7 -- actionlint + + docs: + name: Docs in English and Chinese + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + with: + persist-credentials: false + - name: Check that the Chinese pages match the English ones + run: python3 -I .github/scripts/check_docs_parity.py . diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 1256458..6d65d1e 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -76,6 +76,37 @@ jobs: # its own and disables os.execute and io.popen until a test stubs them. - name: Run the offline hook tests run: vfox plugin test . + # On Windows, VFOX_PYTHON_USE_UV_BUILD=1 reads the uv-build index. A closed proxy + # fails every attempt at once, so the search must wait between attempts (through + # PowerShell), announce each retry, and then fail after three attempts. + - name: Check that index retries wait on Windows + if: runner.os == 'Windows' + shell: pwsh + env: + VFOX_HOME: ${{ runner.temp }}\vfox-retry-home + VFOX_PYTHON_USE_UV_BUILD: '1' + HTTPS_PROXY: 'http://127.0.0.1:1' + run: | + $ErrorActionPreference = 'Continue' + [System.IO.Directory]::CreateDirectory($env:VFOX_HOME) | Out-Null + $archive = Join-Path $env:RUNNER_TEMP 'python-plugin-retry.zip' + Compress-Archive -Path metadata.lua,hooks,lib -DestinationPath $archive + vfox add --source $archive + if ($LASTEXITCODE -ne 0) { throw "vfox add failed: $LASTEXITCODE" } + $watch = [System.Diagnostics.Stopwatch]::StartNew() + $output = vfox search python 2>&1 + $exitCode = $LASTEXITCODE + $watch.Stop() + $text = [regex]::Replace(($output | Out-String), "\x1b\[[0-9;]*m", '') + Write-Output $text + if ($exitCode -eq 0) { throw 'vfox search python succeeded through a closed proxy' } + if ($text -notmatch 'Status: none') { throw 'the index failure was not a transport error' } + if ($text -notmatch 'after 3 attempts') { throw 'the index failure does not report three attempts' } + $retries = ([regex]::Matches($text, 'Trying again')).Count + if ($retries -lt 2) { throw "expected at least two retries, saw $retries" } + $seconds = $watch.Elapsed.TotalSeconds + if ($seconds -lt 3) { throw "the retries did not wait: the search took $seconds seconds" } + Write-Output "Index retries waited for $([math]::Round($seconds, 1)) seconds in all" layouts: name: Windows installer command layouts From bcf6253c0baa503e065f1e2dbc210086518eda88 Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 17:22:23 +0800 Subject: [PATCH 8/9] docs: describe the retry limits and review the Chinese pages The index retries, Retry-After and the archive download limit are described in English and Chinese. The stale getIndexWithRetry references are gone. The Chinese pages changed in this branch were read against their English pages. The mirror paragraph now names VFOX_PYTHON_UV_BUILD_MIRROR instead of "this mirror", and a few sentences were reworded for clarity. --- docs/changes.md | 7 ++++--- docs/development.md | 29 ++++++++++++++++++++++------- docs/mirror.md | 5 +++-- docs/uv.md | 34 ++++++++++++++++++++++------------ docs/zh-CN/changes.md | 4 ++-- docs/zh-CN/development.md | 9 ++++++--- docs/zh-CN/mirror.md | 2 +- docs/zh-CN/platforms.md | 2 +- docs/zh-CN/uv.md | 6 +++++- 9 files changed, 66 insertions(+), 32 deletions(-) diff --git a/docs/changes.md b/docs/changes.md index 68cd05e..0fad1fa 100644 --- a/docs/changes.md +++ b/docs/changes.md @@ -5,9 +5,10 @@ ## Unreleased - Index requests to the uv-build and pyenv indexes are sent again on a network error, - HTTP 429 or a 5xx: up to three attempts in all, with pauses of one and then two seconds. - A failure that persists still fails the install. The default mode does not fall back to - a source build for it. + HTTP 429, a 5xx, or a 200 response that is not the index: up to three attempts in + all. The wait follows the server's `Retry-After` in seconds when it is at most ten, + and otherwise one second and then two. A failure that persists still fails the + install; the default mode does not fall back to a source build for it. - The uv-build index and pyenv version lists are sorted newest first with one comparator. With `VFOX_PYTHON_USE_UV_BUILD=1`, the list used to keep the order the index returned, so `latest` depended on that order. The pyenv list compared diff --git a/docs/development.md b/docs/development.md index 86fff20..6b4a4ad 100644 --- a/docs/development.md +++ b/docs/development.md @@ -25,6 +25,7 @@ tests/windows_install_layouts.lua the Windows command paths, with stubbed proce tests/e2e/ Go end-to-end suites for Linux and Windows; see e2e.md tests/fixtures/ .python-version files used by the legacy-file test .github/workflows/ test, lint, license, e2e and publish +.github/scripts/ the docs parity check docs/, docs/zh-CN/ documentation in English and Chinese Injection.lua type hints for what vfox injects, never loaded stylua.toml, .luarc.json Lua formatter and language-server settings @@ -87,11 +88,17 @@ looked up. pyenv source build. A failed index request is retried (see below); an index that still fails is an error that names `VFOX_PYTHON_USE_UV_BUILD=0`. When no libc can be detected, the source build is used. -- Index requests (`getIndexWithRetry` in `lib/util.lua`) send a GET again on a transport - error, HTTP 429 or a 5xx: up to three attempts in all, with pauses of one and then two - seconds. Other statuses are returned as they are. The uv-build index and the pyenv index - both use it. A test that makes an index fail replaces `pauseBeforeIndexRetry`, because - `os.execute` is disabled in plugin tests. +- Index requests go through `readIndex` in `lib/util.lua`. A request is sent again on + a transport error, HTTP 429, a 5xx, or a 200 whose body is not JSON with the list the + caller reads, since a proxy or a captive portal can answer that way. There are three + attempts in all. Before a retry the plugin waits for the `Retry-After` seconds when + the header is present and at most ten; otherwise for one second, then two. A longer + `Retry-After` fails at once, and the error names the wait. An HTTP-date `Retry-After` + is not read. Other statuses are returned to the caller as they are. The uv-build + index and the pyenv index both use it. `pauseBeforeIndexRetry` runs `sleep`, or + PowerShell on Windows, through `os.execute`, which plugin tests disable. The tests + that make an index fail replace it, and `tests/index_retry_test.lua` checks both + commands. - `note = "uv-build"`, `"GraalPy"` or `"GraalPy Community"` is shown by vfox. ### PostInstall @@ -294,6 +301,8 @@ VFOX_SKIP_CONTAINER_E2E=1 go test -v ./linux | Hook tests | `vfox plugin test .` on Linux and Windows | `test.yml` | | Windows command paths | GopherLua | `test.yml` | | Native and container E2E | Go tests on Linux and Windows | `e2e.yml` | +| Index retry waits | `vfox search` through a closed proxy, on Windows | `test.yml` | +| Docs in English and Chinese | `.github/scripts/check_docs_parity.py` | `lint.yml` | `stylua.toml` uses four spaces, a 120-column limit and Unix line endings; `.gitattributes` keeps checkouts LF so the check agrees with Windows. LuaLS @@ -310,8 +319,14 @@ is missing. - `test.yml` runs the offline tests on `ubuntu-latest` and `windows-latest` with vfox 1.1.0 (checksum-pinned), the GopherLua layouts on Linux, and a real install of CPython 3.14.5 through the EXE installer on Windows, with paths full - of spaces and shell characters, driven by the released vfox 1.0.12. -- `lint.yml` runs StyLua, LuaLS and actionlint. + of spaces and shell characters, driven by the released vfox 1.0.12. On Windows it + also searches through a closed proxy with `VFOX_PYTHON_USE_UV_BUILD=1`, and checks + that the index retries wait and then fail after three attempts. +- `lint.yml` runs StyLua, LuaLS and actionlint. Its `docs` job runs + `.github/scripts/check_docs_parity.py`. That check compares the structure of the + Chinese pages with the English ones, not their sentences: page pairs, `VFOX_*` + names, code block counts and relative links. Translation quality still needs a + reader. - `license.yml` runs the header check. - `e2e.yml` builds and installs real Pythons on Linux and Windows, including a mirror, uv-build and GraalPy, plus the Windows PEP 514 check. diff --git a/docs/mirror.md b/docs/mirror.md index 85996f3..f2b9695 100644 --- a/docs/mirror.md +++ b/docs/mirror.md @@ -64,8 +64,9 @@ unpacking. A mirror that serves different bytes fails the install with `Checksum error`, which is the intended behaviour: change the index or the mirror, do not bypass the check. -On Linux and macOS, the default mode takes this mirror for every CPython version -the index lists, when `VFOX_PYTHON_MIRROR` is not set. A `VFOX_PYTHON_MIRROR` +On Linux and macOS, when `VFOX_PYTHON_MIRROR` is not set, the default mode downloads +every CPython archive the index lists through `VFOX_PYTHON_UV_BUILD_MIRROR`, or from +GitHub when that is not set. A `VFOX_PYTHON_MIRROR` setting keeps CPython on the python.org mirror and builds it from source, unless `VFOX_PYTHON_UV_BUILD_MIRROR` is set too. A source build still clones pyenv from GitHub, and neither mirror variable changes that clone. Set diff --git a/docs/uv.md b/docs/uv.md index 82c76b0..f77ca79 100644 --- a/docs/uv.md +++ b/docs/uv.md @@ -48,14 +48,20 @@ release from source. - `1`, `true`, `yes` or `on`: uv-build only. A version the index does not list is an error. -An index request that fails with a network error, HTTP 429 or a 5xx status is -sent again, up to three attempts in all, with a pause of one second and then two -seconds. Other statuses are not retried. If the index still cannot be read after -that, the install fails and names `VFOX_PYTHON_USE_UV_BUILD=0`, so it never starts a -source build without saying so. If the index has no archive for this version on this platform, such as an +An index request is sent again when it fails with a network error, HTTP 429 or a +5xx status, or when a 200 response does not hold the index, such as a captive +portal's page. It is sent up to three times in all. Before a retry, the plugin waits +for the seconds in the server's `Retry-After` header when the header is present and +is at most ten; otherwise it waits one second, then two. A longer `Retry-After` is +not waited for: the request fails at once, and the error says how long the server +asked to wait. An HTTP-date `Retry-After` is not read. Other statuses, such as 404, +are not retried. If the index still cannot be read, the install fails and names +`VFOX_PYTHON_USE_UV_BUILD=0`, so it never starts a source build without saying so. + +If the index has no archive for this version on this platform, such as an architecture it does not cover, the install builds from source and prints a line -saying so. On Linux, if the libc cannot be detected, the install builds from -source; set `VFOX_PYTHON_UV_LIBC` to use uv-build. +saying so. On Linux, if the libc cannot be detected, the install builds from source; +set `VFOX_PYTHON_UV_LIBC` to use uv-build. The plugin asks `https://vault.vfox.dev/python/uv-build` for the builds of the running platform, architecture and libc, then hands vfox the archive URL and its @@ -67,12 +73,16 @@ SHA-256. vfox downloads and verifies the archive, unpacks it, and calls - On Windows, if the archive has `ensurepip` but no pip command, pip is installed and `pip.cmd`/`pip3.cmd` shims are created. +The archive download belongs to vfox, not to this plugin. vfox 1.1.0 has no setting +for retrying a download, and the plugin cannot retry it, so run the install again if +a download fails. The retries described above cover the index requests only. + Only CPython archives are listed. GraalPy and other implementations are not -part of uv-build. With `VFOX_PYTHON_USE_UV_BUILD=1`, `vfox install -python@graalpy-24.2.0` fails with a message asking you to unset the variable or -set it to `0`. Without the variable, GraalPy installs from Oracle's archive, as -described in [GraalPy](graalpy.md). The plugin prefers, -in order: +part of uv-build. With `VFOX_PYTHON_USE_UV_BUILD=1`, +`vfox install python@graalpy-24.2.0` fails with a message asking you to unset +the variable or set it to `0`. Without the variable, GraalPy installs from +Oracle's archive, as described in [GraalPy](graalpy.md). The plugin prefers, in +order: 1. `install_only_stripped` 2. `install_only` diff --git a/docs/zh-CN/changes.md b/docs/zh-CN/changes.md index 6934c2e..034c406 100644 --- a/docs/zh-CN/changes.md +++ b/docs/zh-CN/changes.md @@ -4,13 +4,13 @@ ## 未发布 -- uv-build 和 pyenv 索引的请求遇到网络错误、HTTP 429 或 5xx 时会重发:总共最多三次,间隔 1 秒和 2 秒。持续失败仍会使安装失败;默认模式不会因此改为源码构建。 +- uv-build 和 pyenv 索引的请求遇到网络错误、HTTP 429、5xx,或 200 响应不是索引时会重发:总共最多三次。等待时间跟随服务器的 `Retry-After`(秒数,不超过 10 秒),否则依次为 1 秒和 2 秒。持续失败仍会使安装失败;默认模式不会因此改为源码构建。 - uv-build 索引和 pyenv 版本列表现在用同一个比较函数从新到旧排序。过去 `VFOX_PYTHON_USE_UV_BUILD=1` 的列表保留索引返回的顺序,`latest` 依赖这个顺序;pyenv 列表则把带后缀的名字(如 `3.13.18t`)当成 `3.13.0` 来比较。`latest` 现在选最新的常规版本,并跳过自由线程版;自由线程版只有写出名字时才会安装。 - 在 Linux 和 macOS 上,`vfox install python@<版本>` 现在会在 uv-build 索引列出某个 CPython 版本时, 从它的 uv-build 预编译包安装;只有索引没有列出时才从源码构建。设置 `VFOX_PYTHON_USE_UV_BUILD=0` - 可以让每个 CPython 都继续从源码构建,设置 `VFOX_PYTHON_USE_UV_BUILD=1` 则只使用 uv-build。GraalPy、 + 可以让每个 CPython 都从源码构建,设置 `VFOX_PYTHON_USE_UV_BUILD=1` 则只使用 uv-build。GraalPy、 PyPy 和 Conda 的安装保持不变。`vfox search python` 会把 pyenv 的名字和只有 uv-build 列出的 CPython 版本一起列出。 此变更之前已完成的安装不会被改动。若设置了 `VFOX_PYTHON_MIRROR`,CPython 保持走源码构建的路径, 除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`。 diff --git a/docs/zh-CN/development.md b/docs/zh-CN/development.md index 98919e1..b55e514 100644 --- a/docs/zh-CN/development.md +++ b/docs/zh-CN/development.md @@ -25,6 +25,7 @@ tests/windows_install_layouts.lua Windows 命令路径,进程调用用 stub tests/e2e/ Go 端到端套件,覆盖 Linux 和 Windows,见 e2e.md tests/fixtures/ .python-version 测试用的文件 .github/workflows/ test、lint、license、e2e 和 publish +.github/scripts/ 文档中英文结构检查 docs/, docs/zh-CN/ 中英文文档 Injection.lua vfox 注入对象的类型提示,不会被加载 stylua.toml, .luarc.json Lua 格式化和语言服务器设置 @@ -72,7 +73,7 @@ vfox 加载 `metadata.lua`,命令需要时调用相应的钩子,并向钩子 非 CPython 的名字从不在那里查找。索引列出该版本时,结果与 uv-build 模式相同。索引没有列出时, PreInstall 会打印一行,并且不返回包,于是 vfox 运行 pyenv 源码构建。索引请求失败会先重试(见下),仍失败则报错, 错误信息里写明 `VFOX_PYTHON_USE_UV_BUILD=0`。检测不到 libc 时,使用源码构建。 -- 索引请求(`lib/util.lua` 里的 `getIndexWithRetry`)遇到网络错误、HTTP 429 或 5xx 时会重发:总共最多三次,两次之间分别等待 1 秒和 2 秒。其他状态码原样返回。uv-build 索引和 pyenv 索引都用它。让索引失败的测试要替换 `pauseBeforeIndexRetry`,因为插件测试里 `os.execute` 被禁用。 +- 索引请求经过 `lib/util.lua` 里的 `readIndex`。遇到网络错误、HTTP 429、5xx,或 200 响应的正文不是带列表的 JSON(代理和强制门户都可能这样返回)时会重发,总共最多三次。重试前,如果响应带有不超过 10 秒的 `Retry-After`,就等待它指定的秒数;否则依次等待 1 秒和 2 秒。超过 10 秒的 `Retry-After` 立即失败,错误信息里写明等待时间。HTTP 日期格式的 `Retry-After` 不会被读取。其他状态码原样返回给调用方。uv-build 索引和 pyenv 索引都用它。`pauseBeforeIndexRetry` 通过 `os.execute` 运行 `sleep`(Windows 上是 PowerShell);插件测试禁用了 `os.execute`,所以让索引失败的测试会替换它,`tests/index_retry_test.lua` 检查这两条命令。 - `note = "uv-build"`、`"GraalPy"` 或 `"GraalPy Community"` 会显示在 vfox 里。 ### PostInstall @@ -238,6 +239,8 @@ VFOX_SKIP_CONTAINER_E2E=1 go test -v ./linux | 钩子测试 | 在 Linux 和 Windows 上 `vfox plugin test .` | `test.yml` | | Windows 命令路径 | GopherLua | `test.yml` | | 原生和容器 E2E | Linux 和 Windows 上的 Go 测试 | `e2e.yml` | +| 索引重试等待 | Windows 上经关闭的代理执行 `vfox search` | `test.yml` | +| 中英文文档 | `.github/scripts/check_docs_parity.py` | `lint.yml` | `stylua.toml` 用四空格、120 列、Unix 换行;`.gitattributes` 让 checkout 保持 LF,这样 Windows 上的检查和 Linux 一致。LuaLS 读 `.luarc.json`,里面声明了 @@ -251,8 +254,8 @@ vfox 的全局变量,只关闭 `duplicate-set-field`(测试会有意替换 ` - `test.yml` 在 `ubuntu-latest` 和 `windows-latest` 上用 vfox 1.1.0(校验和 固定)跑离线测试,在 Linux 上跑 GopherLua 布局测试,并用带空格和 Shell 字符的路径, - 由发布版 vfox 1.0.12 在 Windows 上通过 EXE 安装包真实安装一次 CPython 3.14.5。 -- `lint.yml` 跑 StyLua、LuaLS 和 actionlint。 + 由发布版 vfox 1.0.12 在 Windows 上通过 EXE 安装包真实安装一次 CPython 3.14.5。在 Windows 上,它还用关闭的代理执行 `VFOX_PYTHON_USE_UV_BUILD=1` 的搜索,检查索引重试确实等待,并在三次尝试后失败。 +- `lint.yml` 跑 StyLua、LuaLS 和 actionlint。它的 `docs` 作业运行 `.github/scripts/check_docs_parity.py`,比较中文页与英文页的结构而不比较句子:页面是否成对、`VFOX_*` 变量名、代码块数量和相对链接。译文是否通顺仍需人工阅读。 - `license.yml` 跑许可证头检查。 - `e2e.yml` 在 Linux 和 Windows 上真实构建、安装 Python,包括镜像、uv-build 和 GraalPy,并检查 Windows 上的 PEP 514 注册。 diff --git a/docs/zh-CN/mirror.md b/docs/zh-CN/mirror.md index c05c72c..fe9d229 100644 --- a/docs/zh-CN/mirror.md +++ b/docs/zh-CN/mirror.md @@ -59,7 +59,7 @@ export VFOX_PYTHON_UV_BUILD_MIRROR=https://registry.npmmirror.com/-/binary/pytho 解压前,插件会用 uv-build 索引里的 SHA-256 校验每个包。镜像提供的字节不一致时 安装会以 `Checksum error` 失败,这是有意的:应该改索引或镜像,不要绕过校验。 -在 Linux 和 macOS 上,未设置 `VFOX_PYTHON_MIRROR` 时,默认模式会对索引列出的每个 CPython 版本使用这个镜像。 +在 Linux 和 macOS 上,未设置 `VFOX_PYTHON_MIRROR` 时,默认模式会通过 `VFOX_PYTHON_UV_BUILD_MIRROR` 下载索引列出的每个 CPython 版本的包;未设置它时从 GitHub 下载。 若设置了 `VFOX_PYTHON_MIRROR`,CPython 会保持经 python.org 镜像从源码构建,除非同时设置了 `VFOX_PYTHON_UV_BUILD_MIRROR`。 源码构建仍然从 GitHub 克隆 pyenv,两个镜像变量都不会改变这次克隆。设置 `VFOX_PYTHON_USE_UV_BUILD=0` 可以从源码构建; 设置 `VFOX_PYTHON_USE_UV_BUILD=1` 则只使用 uv-build。 diff --git a/docs/zh-CN/platforms.md b/docs/zh-CN/platforms.md index 2c55325..f3f1a9f 100644 --- a/docs/zh-CN/platforms.md +++ b/docs/zh-CN/platforms.md @@ -78,7 +78,7 @@ sudo apt install --assume-yes make build-essential libssl-dev zlib1g-dev \ 随后会在 ensurepip 步骤失败:pip 是从 wheel 安装的,而 wheel 是 zip 压缩包,读取它需要 zlib (报错为 `No module named 'zlib'`)。 -想在 Ubuntu 上省掉这一步,就安装 uv-build 索引列出的版本。它不需要 `sudo`,也不需要 `apt`,在 Linux 上也是默认方式: +想在 Ubuntu 上省掉这一步,就安装 uv-build 索引列出的版本。它不需要 `sudo` 或 `apt`,而且在 Linux 上是默认方式: ```bash vfox install python@3.12.16 diff --git a/docs/zh-CN/uv.md b/docs/zh-CN/uv.md index b9b6137..7645ce4 100644 --- a/docs/zh-CN/uv.md +++ b/docs/zh-CN/uv.md @@ -35,7 +35,9 @@ vfox install python@3.12 - `0`、`false`、`no` 或 `off`:在 Linux 和 macOS 上源码构建。 - `1`、`true`、`yes` 或 `on`:只用 uv-build。索引里没有列出的版本会报错。 -索引请求遇到网络错误、HTTP 429 或 5xx 状态码时会重发,总共最多三次,两次之间分别等待 1 秒和 2 秒;其他状态码不重试。重试后仍无法读取索引,安装会失败,并提示 `VFOX_PYTHON_USE_UV_BUILD=0`,因此不会在没有说明的情况下开始源码构建。如果索引在该平台上没有这个版本的包(比如索引不覆盖某个架构),安装会改为源码构建,并打印一行说明。在 Linux 上,如果无法识别 libc,安装会从源码构建;要使用 uv-build,请设置 `VFOX_PYTHON_UV_LIBC`。 +索引请求遇到网络错误、HTTP 429 或 5xx 状态码,或者 200 响应里不是索引(比如强制门户的页面)时会重发,总共最多三次。重试前,如果服务器返回了 `Retry-After` 头且不超过 10 秒,就等待它指定的秒数;否则依次等待 1 秒和 2 秒。超过 10 秒的 `Retry-After` 不等待:请求立即失败,错误信息里写明服务器要求的等待时间。HTTP 日期格式的 `Retry-After` 不会被读取。其他状态码(如 404)不重试。重试后仍无法读取索引,安装会失败,并提示 `VFOX_PYTHON_USE_UV_BUILD=0`,因此不会在没有说明的情况下开始源码构建。 + +如果索引在该平台上没有这个版本的包(比如索引不覆盖某个架构),安装会改为源码构建,并打印一行说明。在 Linux 上,如果无法识别 libc,安装会从源码构建;要使用 uv-build,请设置 `VFOX_PYTHON_UV_LIBC`。 插件向 `https://vault.vfox.dev/python/uv-build` 询问当前平台、架构和 libc 的 构建,然后把包地址和 SHA-256 交给 vfox。vfox 下载、校验、解压,再调用 @@ -45,6 +47,8 @@ vfox install python@3.12 - Windows 上,如果包里带了 `ensurepip` 却没有 pip 命令,会安装 pip,并生成 `pip.cmd`、`pip3.cmd` 两个 shim。 +归档由 vfox 下载,不由插件下载。vfox 1.1.0 没有下载重试的设置,插件也无法重试下载,所以下载失败时请重新运行安装。上面描述的重试只作用于索引请求。 + 列表里只有 CPython 的包。GraalPy 和其他实现不在 uv-build 里。设置 `VFOX_PYTHON_USE_UV_BUILD=1` 时,执行 `vfox install python@graalpy-24.2.0` 会失败, 并提示取消该变量或把它设为 `0`。不设置该变量时,GraalPy 从 Oracle 的压缩包安装, From 084671674fba5fc57eb4688e93151468c7af032d Mon Sep 17 00:00:00 2001 From: Qiheng He Date: Sat, 14 Nov 2026 17:34:36 +0800 Subject: [PATCH 9/9] ci: end the Windows retry check with its own result The search in the Windows retry check is meant to fail. Its exit code stayed in $LASTEXITCODE, which the runner reads, so the step failed even though every check held: run 43580252144 waited 5.8 seconds, reported three attempts and two retries. The step now exits 0 once its checks pass. --- .github/workflows/test.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 6d65d1e..a122c28 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -107,6 +107,9 @@ jobs: $seconds = $watch.Elapsed.TotalSeconds if ($seconds -lt 3) { throw "the retries did not wait: the search took $seconds seconds" } Write-Output "Index retries waited for $([math]::Round($seconds, 1)) seconds in all" + # The search above is meant to fail. Its exit code stays in $LASTEXITCODE, which + # the runner reads, so the step states its own result once every check holds. + exit 0 layouts: name: Windows installer command layouts