VHDL to Python interface: python_pkg on a native bridge for NVC, GHDL and Questa

[ru551n]

Summary

This PR brings the VHDL-to-Python interface of the python_pkg branch (Lars Asplund, VUnit/vunit@22af9848) to master, with a new native implementation underneath and a few additions to its API.

The API of the python_pkg branch is the interface. python_pkg.vhd, python_context.vhd, the VHPI package and its C sources, run_script_path, the check_result_t getters and the test cases of tb_python_pkg.vhd are taken over; the testbench's 8 error demonstrations, which failed by design, are now negative tests checking the reported message through the mocked logger, so the whole testbench passes on NVC, GHDL and Questa.

Enable with vu.add_vhdl_builtins() followed by vu.add_python(); in VHDL, context vunit_lib.python_context;. add_python() builds whatever the simulator needs under the output path, so a run script needs nothing else, and the interpreter starts on first use, so python_setup/python_cleanup are optional.

What is underneath

• NVC and GHDL: the two VHPIDIRECT applications of the branch and their compile_* run-script steps are replaced by the VUnit Python bridge (vunit/python_bridge, C in native/): compiled on first use on Linux/macOS and cached by a fingerprint of its inputs; prebuilt MSVC DLLs per CPython version on Windows (built in CI, shipped in releases, never committed). Loading is automatic (--load for NVC, the search path for GHDL, per backend).
• Questa/ModelSim: the same bridge through an FLI front end (native/fli.c), built and cached by add_python() like the others (the system C compiler on Linux, the MinGW compiler bundled with Questa on Windows), so the API behaves identically on NVC, GHDL and Questa. Verified on Questa Altera Starter FPGA Edition 2025.3 (Linux). The branch's FLI application is therefore not included.
• Riviera-PRO/Active-HDL: the branch's VHPI application, now built by add_python() as well and starting the interpreter on first use. Untested (no licence available); its run-script helper compile_vhpi_application is not needed and not included.
• The interpreter is started with PyConfig as the Python running VUnit (virtual environments and editable installs resolve), the GIL is taken on every entry, only scalars and fixed-size chunks cross the FFI (no length limit, same ABI on all simulators), and Python errors are reported through python_logger (vunit_lib:python), so they can be tested with mock/check_only_log.

Added to the API, and why

• Sessions: a trailing, defaulted session parameter on every operation; a session is a VUnit object (a record around an integer_vector_ptr_t, created with new_session(name) or, from an existing identity, new_session(id), with get_id/name accessors) whose name is an identity under vunit_lib:python; every session logs on the logger of its identity, python_logger being the parent. Two Python models that both define model or config can live in separate namespaces of the same interpreter.
• More arg/kwarg value types: std_ulogic (→ bool), arg_unsigned/arg_signed and their kwarg forms (any width, exact Python integers), real_vector, integer_vector_ptr_t, integer_array_t. The typed names for unsigned/signed exist because an arg overload would make a string literal argument such as arg("Hello") ambiguous. A value that cannot be converted (metavalues) is reported on python_logger and becomes an argument that raises in Python, so a call never proceeds with a missing argument. They came out of migrating a real testbench that hands 32-bit CSR counters to a Python checker.
• Argument groups: kwarg("a", 1) & kwarg("b", 2) becomes one argument (**dict(...)), arg(1) & arg(2) becomes *(1, 2,), and mixed groups follow Python's rules; this lifts the 10-argument limit of call and lets arguments be built in loops (null_arg is the identity), without touching arg_t or to_call_str.
• integer_array_t as NumPy arrays, in both directions, with shape and word size preserved (get(a, x, y) is a[y, x]): bulk data such as images and weight tensors are not viable as source text.
• More result types: eval_boolean/call_boolean, eval_std_ulogic, width-checked std_ulogic_vector/signed/unsigned results, call_string, call_real_vector, call_integer_vector_ptr, eval_integer_array/call_integer_array.
• exec_file: executes a Python file (a relative name is relative to the testbench directory, tb_path) into the session namespace, so a verification component can load its own model from a generic without the testbench knowing that Python is involved.
• The string-argument forms of to_call_str and call are removed; arguments are always built with arg/kwarg.
• real values are not range-checked against single precision on the bridge (VHDL real is a double on these simulators); any finite value is exchanged exactly and a non-finite one is reported as a failure.

The embedded Python example gained cases for each of these, in the style of the existing ones, including a minimal verification component whose behaviour is a Python function.

NumPy as a dependency

numpy is added to the package dependencies. integer_array_t values are exchanged as NumPy arrays and a user of the feature should not have to discover that from an error message; the runtime itself only imports NumPy lazily, so nothing changes for users who never call add_python().

Tests and CI

• vunit/vhdl/python/run.py: the branch's tb_python_pkg.vhd plus tb_python_pkg_bridge.vhd (75 cases for the additions), on NVC, GHDL (mcode, llvm-jit, llvm, gcc) and Questa.
• python_bridge.yml: Linux with CPython 3.10/3.14 × NVC 1.22.1 and every GHDL backend on the latest release; canaries (allowed to fail) on NVC built from master and the GHDL nightly builds; macOS with NVC and GHDL; the MSVC DLL build (reusable python_bridge_dlls.yml) and DLL-fed Windows tests on NVC and GHDL with CPython 3.10/3.12/3.14; a packaging check. push.yml only gains the DLL download in the release job.
• Unit tests for the build/cache logic, the generated packages and the simulator hooks; documentation in docs/python_bridge/user_guide.rst.

Review

The first review round is addressed in one commit per comment (24035be … 91ff756, sessions completed in 6f34c38), each answered on its thread.

Disclosure

This work was done with substantial use of an AI assistant (Claude Code, Anthropic): the design, the C, Python and VHDL code, the tests, the workflows and this description were produced with it, under my direction and review, and verified by running the test suites on the simulators listed above. Please review it with that in mind.

[LarsAsplund]

I'm not done reviewing but this is a start. As a general comment it looks very good with nice additions. I do see that this will be a feature with a lot of development going forward. For that reason, I think most of it should be external to the VUnit core, for example in repo vunit/python_bridge. That way it can have its own life-cycle with more frequent major releases. I have started an embryo of a package system for VUnit that should be used. See how json4vhdl became a VUnit package that can be downloaded from pypi and imported as Python package (https://pypi.org/project/vunit-json-for-vhdl)

[LarsAsplund]

It feel like this should be re-written to cover the larger picture. It feels like Claude (?) focused on the additions such as std_logic support.

[ru551n]

Fixed in 24035be: rewritten as an announcement of the feature itself.

[LarsAsplund]

VUnit provides join(a, b) to concatenate path segments in a way that works regardless if the path separator is present in a or not

[ru551n]

Fixed in b9430da: join(tb_path(runner_cfg), "...") everywhere in the example.

[LarsAsplund]

Other "typed" VUnit functions/procedures use _std_ulogic(_vector)

[ru551n]

Fixed in d7a4009: eval_std_ulogic, eval_std_ulogic_vector, call_std_ulogic, call_std_ulogic_vector and arg/kwarg(std_ulogic), with std_ulogic/std_ulogic_vector parameter and result types.

[LarsAsplund]

I think we should be explicit rather than implicit regarding the path. That follows the pattern we've used before. That way we don't have to decide what the best root is. tb_path is probably equally good as default.

[ru551n]

Fixed in 4ce146e: a relative file name is resolved from tb_path (in VHDL, from runner_state), an absolute path is used as given, and the run-script-relative resolution is gone.

[LarsAsplund]

join

[ru551n]

Fixed in b9430da.

[LarsAsplund]

call with string arguments should be remove. This was my initial approach

[ru551n]

Fixed in 4d5c399: both string forms are removed; the example uses arg(...).

[LarsAsplund]

Should this be an error instead? If the user calls arg and it becomes a null_arg, it will completely disappear from the Python call

[ru551n]

Agreed and fixed in d33a067: after the failure is logged, the argument becomes __vunit__.error("<message>"), so the call raises in Python with the same message instead of proceeding with a missing argument (only observable when the logger is mocked, but that is exactly when it matters).

[LarsAsplund]

Can positional arguments be grouped in the same way?

[ru551n]

Yes. Done in e4ffa82: arg(1) & arg(2) becomes one argument *(1, 2,), a mixed group arg(1) & kwarg("a", 2) becomes *(1,), **dict(a=2), and appending a positional argument after keyword arguments is an error, as in Python.

[LarsAsplund]

Is grouping will null_arg a real valid use case?

[ru551n]

Yes, it is the identity element of &, and three things rely on it: (1) building a group in a loop, args := null_arg; for i in regs'range loop args := args & kwarg(names(i).all, regs(i)); end loop;, which otherwise needs a special first iteration; (2) optional arguments, since VHDL has no conditional expression the idiom is a helper that returns null_arg when the argument does not apply, and the caller just writes base & optional_kwarg(...); (3) an empty group, call("f", args) with args = null_arg, is a valid call with no arguments, consistent with the unused slots of call. It also costs nothing: after the change in d33a067 a failed conversion no longer degrades to null_arg, so it is only ever produced deliberately. Documented in the comment and the user guide in 9e0cdfd.

[LarsAsplund]

session_t should probably be an object that we can build upon, i.e. a record type with a p_data field of type integer_vector_ptr_t. It is created with new_session function. I would like to see that its name is an identity (id_t) and that log calls are made to a logger with that identity. Going forward, I think we may see message passing into actors within the session namespaces. If so, actors in that namespace will be children of the session identiy. As an initial step, it would be sufficient to place the session name in such an record type but leave everything else as is.

[ru551n]

Done in 91ff756: python_session_t is a record with p_data : integer_vector_ptr_t, created with new_session(name); the name is an identity under vunit_lib:python (the identity of python_logger), name(session) returns it, and the operations of a session log on the logger of that identity, a child of python_logger (the default session logs on python_logger itself). Sessions are compared by identity, so new_session("golden") twice is the same session.

[ru551n]

Now exactly as you describe, in 6f34c38: python_session_t is a record with p_data : integer_vector_ptr_t; new_session(name) creates its identity under vunit_lib:python and, following new_actor, new_session(id : id_t) takes an existing identity so that a session can sit anywhere in the identity tree (and actors can be its children later); get_id(session) and name(session) are the accessors; and every session, the default one included, logs on get_logger(get_id(session)) — the earlier exception where the default session logged on python_logger is gone; python_logger is the logger of the parent identity. The Python namespace is keyed by the full identity name, so sessions with the same leaf name under different parents are distinct.

[ru551n]

I'm not done reviewing but this is a start. As a general comment it looks very good with nice additions. I do see that this will be a feature with a lot of development going forward. For that reason, I think most of it should be external to the VUnit core, for example in repo vunit/python_bridge. That way it can have its own life-cycle with more frequent major releases. I have started an embryo of a package system for VUnit that should be used. See how json4vhdl became a VUnit package that can be downloaded from pypi and imported as Python package (https://pypi.org/project/vunit-json-for-vhdl)

Regarding the packaging, I have created #1221 as a means to support passing simulator flags (hooks into sim_if) as well as running setup hooks (compiling the C python bridge).

I have also created vunit-python-bridge which will be a package similar to vunit-json-for-vhdl, We will merge this later to a repo under the VUnit banner when this feature is completed.

For the sake of context I propose that we keep reviewing here, then sync to the vunit-python-bridge repo when the review is done. When we are satisified we can create a repo under the VUnit group and merge my personal repo to there.

[ru551n]

Regarding docs, this is quite complex feature, and a user manual would be nice (in the scope as we already have now). The question is though where it should live. Either it can live here, with an added maintanability issue (syncing with the package repo), or it can live in it's own package repo and get linked to from VUnit main repo.

Maybe add a list of available packages and links to their respective documentation?

As per discussion, as much as possible should reside in the package itself.

[ru551n]

Also I feel that the examples should be in the VUnit core. They should fail with an exact error message if the package is not installed.

As per discussion, move examples to package.