Skip to content

gh-162308: Check native extension modules with the target interpreter - #162311

Closed
superajun-wsj wants to merge 3 commits into
python:mainfrom
superajun-wsj:fix/gh-162308-native-module-check
Closed

superajun-wsj wants to merge 3 commits into
python:mainfrom
superajun-wsj:fix/gh-162308-native-module-check

Conversation

@superajun-wsj

@superajun-wsj superajun-wsj commented Nov 20, 2026 •

Copy link
Copy Markdown

Fixes #162308.

Native builds configured with --with-build-python currently run extension
checks in the external build interpreter. If the target builds _bisect as
a builtin while the build interpreter has it as a shared extension, the
checker reports _bisect is not a built-in module even though the target
module imports successfully.

Select a dedicated module-check command from configure's cross_compiling
state. Native checks run the newly built target Python; cross checks retain
the external build Python. Pass the mode explicitly to prevent an inherited
_PYTHON_HOST_PLATFORM from skipping native imports. Use this command for
both missing-stdlib information generation and the final checks, since both
paths call ModuleChecker.check().

This PR is based on #161530 at
c7bf53b4d659f6838c3f11044034e139cf01f939 and includes its prerequisite commit
with the original author and coauthors preserved. #161530 removes the empty
platform marker that previously caused these native builds to skip imports.
The additional fix is commit 51fa2643fffc3fc8ab50ba74e92f464c4fb56327.
Current main (b09a7e92fb0ab958a52ad94e376402cdd3da09b2) is merged so the
PR applies cleanly alongside its RUNSHARED_FOR_BUILD changes. The selected
native checker uses RUNSHARED to load the target interpreter's libraries.

Validation:

  • Built fresh native CPython 3.16 host and target interpreters, with shared
    _bisect in the host and builtin _bisect in the target. The original
    recipe reproduces the import error in both calls; the fixed recipe reports
    zero import failures, including with an inherited platform marker.
  • A real x86_64-to-AArch64 configure and make all build completed using the
    external interpreter. All enabled target modules passed the file checks.
  • test_sysconfig, test_build_details, and test_tools.test_makefile passed:
    68 tests run, 8 skipped.
  • Autoconf 2.72 regeneration, workflow YAML and shell syntax, patch whitespace,
    and clean application of both commits were verified.

The local native build lacks optional _decimal and _zstd dependencies;
the cross sysroot lacks 17 optional module dependencies. These are reported
as missing; all enabled modules have zero import/check failures. The added
GitHub Actions regression job has not yet run on GitHub.

Prepared with assistance from TRAE CLI. The change was checked against the
actual native and cross build paths described above.

itamaro and others added 3 commits November 6, 2026 21:03
configure sets _PYTHON_HOST_PLATFORM only for cross builds, but passed it
to the build Python for every --with-build-python build. In a native
build it was empty, so sysconfig.get_platform() returned "" and
build-details.json ended up with "platform": "".

Co-authored-by: Zanie Blue <contact@zanie.dev>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Select MODULE_CHECK_COMMAND from configure's cross_compiling state.
Native builds use the newly built interpreter even when --with-build-python
selects an external build Python. Cross builds keep using the build Python.
Pass the check mode explicitly so an inherited _PYTHON_HOST_PLATFORM cannot
skip native import checks.

Use the selected command both when generating missing stdlib information
and when reporting extension checks, since both paths check the modules.
Exercise differing builtin module configurations in the existing Linux
build-Python CI job, and add a Build NEWS entry. Regenerate configure with
Autoconf 2.72.

Based on PR python#161530 at c7bf53b.

Validation:
- Native and x86_64-to-AArch64 configure and make all builds passed.
- The original recipe reports _bisect import errors twice; the fixed
  native recipe and inherited-platform check report zero import failures.
- test_sysconfig, test_build_details, and test_tools.test_makefile passed
  (61 tests run, 8 skipped).
- Workflow YAML, shell syntax, and git diff --check passed.

Co-authored-by: TRAE CLI <traecli@bytedance.com>
Merge b09a7e9 while preserving the
prerequisite PR python#161530 and pythongh-162308 fix commits. Resolve the checksharedmods
recipe conflict by using RUNSHARED around MODULE_CHECK_COMMAND: native
checks run the target interpreter and need its runtime library path.
Other upstream RUNSHARED_FOR_BUILD uses remain part of the merged main.
Regenerate configure with Autoconf 2.72.

Validation on the updated source:
- Fresh native host and target builds passed with differing _bisect
  configurations. The original recipe reproduced both import errors;
  the fixed recipe and inherited-platform check had zero import failures.
- A real x86_64-to-AArch64 configure and make all build passed, with all
  enabled target modules passing the file checks.
- test_sysconfig, test_build_details, and test_tools.test_makefile passed:
  68 tests run, 8 skipped.
- Updated workflow YAML, shell syntax, and diff whitespace checks passed.

Co-authored-by: TRAE CLI <traecli@bytedance.com>
@python-cla-bot

Copy link
Copy Markdown

The following commit authors need to sign the Contributor License Agreement:

CLA not signed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

checksharedmods uses the wrong Python interpreter in native builds with --with-build-python

2 participants