Skip to content

Use Argument Clinic Vectorcall for types where it provides substantial benefit #162350

Open

Description

@cmaloney

Feature or enhancement

Argument Clinic can now generate vectorcall for class __init__ and __new__ (gh-87613). This is a meta tracking issue for moving objects to use Argument Clinic generated Vectorcall for these cases:

  1. Replacing a handwritten vectorcall with the same behavior and equal or better performance.
    • Minor changes in exception text for bad arguments are okay
  2. Replacing PyArg_Parse* with Argument Clinic + Vectorcall
    • At least a 10% performance boost in pyperf microbenchmark.
    • Minor changes in exception text for bad arguments are okay
  3. Adding Vectorcall to objects which currently use Argument Clinic but do not support it
    • For these one of the following should apply:
      • The objects are common in performance critical code. This should generally be shown with an improvement on a pyperformance benchmark. If it's critical for a real application but there are no benchmarks, we should add them.
      • Speedup is substantial: over 15% in microbenchmarks. If the operation is a small part of a larger common operation (ex. json.Encoder in json.dumps), that operation must improve by at least 5% on a typical input. This is because vectorcall adds significant code for the performance it enables.

For all the cases it's important to validate

  • Behavior is the same across a wide range of inputs. Changes must be documented and noted. Substantial chagnes mean a different issue from this one is required.
  • There are no regressions for specific argument patterns.
  • tp_call keeps strong references in the argument tuple. In vectorcall the arguments are borrowed references. Make sure to hold a strong reference (Py_INCREF) before calling arbitrary Python code.

If new features are neededed those should be worked on in other issues. Ideally don't want to both convert a type and change existing codegen in one PR. Issues with AC Vectorcall code generation should be reported in gh-87613.

Good Candidates

  • itertools
  • operator: itemgetter, attrgetter
  • types: MethodType, CellType, FrameLocalsProxy, MappingProxyType, FunctionType
  • memoryview
  • property
  • complex
  • sentinel
  • string.templatelib: Template, Interpolation
  • typing: TypeAliasType, ParamSpecArgs, ParamSpecKwargs
  • pickle.PickleBuffer
  • contextvars.ContextVar
  • array.array
  • csv
  • datetime: timezone, timedelta
  • zoneinfo
  • queue.SimpleQueue
  • io.IncrementalNewlineDecoder
  • ctypes.CField
  • map
  • json - gh-162350: Use AC vectorcall for _json.make_encoder to speed up json.dumps #162376

Blocked: Require known feature work

  • gh-87613: Supprt **kwargs in AC VC and use for xml.etree.ElementTree.Element #161597: Support **kwargs in AC vectorcall
    • collections.defaultdict, operator.methodcaller, functools.partial, object
    • collections.OrderedDict (also needs the self cast fix)
    • Exception hierarchy
    • asyncio.Task, asyncio.Future
    • xml.etree.ElementTree
  • Other known improvements needed
    • set, frozenset
    • super
    • sqlite3.Cursor, sqlite3.Row
    • datetime.date, datetime.time, datetime.datetime

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions