Configuration File API Reference

This document describes the low-level API for PyOxidizer configuration files. For a higher-level overview of how configuration files work, see Configuration Files.

Types

The following custom data types are defined in the Starlark environment:

FileContent
Represents the content of a file on the filesystem.
FileManifest
Represents a mapping of filenames to file content.
PythonBytecodeModule
Represents a .pyc file containing Python bytecode for a given module.
PythonDistribution

Represents an implementation of Python.

Used for embedding into binaries and running Python code.

PythonEmbeddedResources
Represents resources made available to a Python interpreter.
PythonExecutable
Represents an executable file containing a Python interpreter.
PythonExtensionModule
Represents a compiled Python extension module.
PythonInterpreterConfig
Represents the configuration of a Python interpreter.
PythonPackageDistributionResource
Represents a file containing Python package distribution metadata.
PythonPackageResource
Represents a non-module resource data file.
PythonSourceModule
Represents a .py file containing Python source code.

Constants

PyOxidizer provides global constants as defined by the following sections.

BUILD_TARGET_TRIPLE

The string Rust target triple that we’re currently building for. Will be a value like x86_64-unknown-linux-gnu or x86_64-pc-windows-msvc. Run rustup target list to see a list of targets.

CONFIG_PATH

The string path to the configuration file currently being evaluated.

CONTEXT

Holds build context. This is an internal variable and accessing it will not provide any value.

CWD

The current working directory. Also the directory containing the active configuration file.

Functions for Manipulating Global State

set_build_path(path)

Configure the directory where build artifacts will be written.

Build artifacts include Rust build state, files generated by PyOxidizer, staging areas for built binaries, etc.

If a relative path is passed, it is interpreted as relative to the directory containing the configuration file.

The default value is $CWD/build.

Important

This needs to be called before functionality that utilizes the build path, otherwise the default value will be used.

Functions for Managing Targets

register_target(name, fn, depends=[], default=False, default_build_script=False)

Registers a named target that can be resolved by the configuration file.

A target consists of a string name, callable function, and an optional list of targets it depends on.

The callable may return one of the types defined by this Starlark dialect to facilitate additional behavior, such as how to build and run it.

depends is an optional list of target strings this target depends on. If specified, each dependency will be evaluated in order and its returned value (possibly cached from prior evaluation) will be passed as a positional argument to this target’s callable.

default indicates whether this should be the default target to evaluate. The last registered target setting this to True will be the default. If no target sets this to True, the first registered target is the default.

default_build_script indicates whether this should be the default target to evaluate when run from the context of a Rust build script (e.g. from pyoxidizer run-build-script. It has the same semantics as default.

Note

It would be easier for target functions to call resolve_target() within their implementation. However, Starlark doesn’t allow recursive function calls. So invocation of target callables must be handled specially to avoid this recursion.

resolve_target(target)

Triggers resolution of a requested build target.

This function resolves a target registered with register_target() by calling the target’s registered function or returning the previously resolved value from calling it.

This function should be used in cases where 1 target depends on the resolved value of another target. For example, a target to create a FileManifest may wish to add a PythonExecutable that was resolved from another target.

resolve_targets()

Triggers resolution of requested build targets.

This is usually the last meaningful line in a config file. It triggers the building of targets which have been requested to resolve by whatever is invoking the config file.

Python Distributions

Python distributions are entities that define an implementation of Python that can be used to create a binary embedding Python and that can be used to execute Python code.

Python distributions are defined by the PythonDistribution type. This type can be constructed from parameters or via default_python_distribution(flavor="standalone", build_target=None).

PythonDistribution(sha256, local_path=None, url=None, flavor="standalone")

Defines a Python distribution that can be embedded into a binary.

A Python distribution is a zstandard-compressed tar archive containing a specially produced build of Python. These distributions are typically produced by the python-build-standalone project. Pre-built distributions are available at https://github.com/indygreg/python-build-standalone/releases.

A distribution is defined by a location, and a hash.

One of local_path or url MUST be defined.

sha256 (string)
The SHA-256 of the distribution archive file.
local_path (string)
Local filesystem path to the distribution archive.
url (string)
URL from which a distribution archive can be obtained using an HTTP GET request.
flavor (string)
The distribution flavor. Must be standalone.

Examples:

linux = PythonDistribution(
    sha256="11a53f5755773f91111a04f6070a6bc00518a0e8e64d90f58584abf02ca79081",
    local_path="/var/python-distributions/cpython-linux64.tar.zst"
)

macos = PythonDistribution(
     sha256="b46a861c05cb74b5b668d2ce44dcb65a449b9fef98ba5d9ec6ff6937829d5eec",
     url="https://github.com/indygreg/python-build-standalone/releases/download/20190505/cpython-3.7.3-macos-20190506T0054.tar.zst"
)

default_python_distribution(flavor="standalone", build_target=None)

Resolves the default PythonDistribution for the given distribution flavor and build target, which default to a standalone distribution and the active build target as defined by BUILD_TARGET, respectively.

flavor is a string denoting the distribution flavor. Values can be one of the following:

standalone
A distribution produced by the python-build-standalone project. The distribution may be statically or dynamically linked, depending on the build_target and availability.
standalone_static
This is like standalone but the distribution must have a statically linked libpython.
standalone_dynamic
This is like standalone but the distribution must have a dynamically linked libpython.

The pyoxidizer binary has a set of known distributions built-in which are automatically available and used by this function. Typically you don’t need to build your own distribution or change the distribution manually.

PythonDistribution Methods

PythonDistribution.source_modules()

Returns a list of PythonSourceModule representing Python source modules present in this distribution.

PythonDistribution.package_resources(include_test=False)

Returns a list of PythonPackageResource representing resource files present in this distribution.

The include_test boolean argument controls whether resources associated with test packages are included.

PythonDistribution.extension_modules(filter='all', preferred_variants=None)

Returns a list of PythonExtensionModule representing extension modules in this distribution.

The filter argument denotes how to filter the extension modules. The following values are recognized:

all
Every named extension module will be included.
minimal
Return only extension modules that are required to initialize a Python interpreter. This is a very small set and various functionality from the Python standard library will not work with this value.
no-libraries

Return only extension modules that don’t require any additional libraries.

Most common Python extension modules are included. Extension modules like _ssl (links against OpenSSL) and zlib are not included.

no-gpl

Return only extension modules that do not link against GPL licensed libraries.

Not all Python distributions may annotate license info for all extensions or the libraries they link against. If license info is missing, the extension is not included because it could be GPL licensed. Similarly, the mechanism for determining whether a license is GPL is based on an explicit list of non-GPL licenses. This ensures new GPL licenses don’t slip through.

The preferred_variants argument denotes a string to string mapping of extension module name to its preferred variant name. If multiple variants of an extension module meet the filter requirements, the preferred variant from this mapping will be used. Otherwise the first variant will be used.

Important

Libraries that extension modules link against have various software licenses, including GPL version 3. Adding these extension modules will also include the library. This typically exposes your program to additional licensing requirements, including making your application subject to that license and therefore open source. See Licensing Considerations for more.

PythonDistribution.pip_install(args, extra_envs={})

This method runs pip install <args> with the specified distribution.

args
List of strings defining raw process arguments to pass to pip install.
extra_envs
Optional dict of string key-value pairs constituting extra environment variables to set in the invoked pip process.

Returns a list of objects representing Python resources installed as part of the operation. The types of these objects can be PythonSourceModule, PythonBytecodeModule, PythonPackageResource, etc.

The returned resources are typically added to a FileManifest or PythonExecutable to make them available to a packaged application.

PythonDistribution.read_package_root(path, packages)

This method discovers resources from a directory on the filesystem.

The specified directory will be scanned for resource files. However, only specific named packages will be found. e.g. if the directory contains sub-directories foo/ and bar, you must explicitly state that you want the foo and/or bar package to be included so files from these directories will be read.

This rule is frequently used to pull in packages from local source directories (e.g. directories containing a setup.py file). This rule doesn’t involve any packaging tools and is a purely driven by filesystem walking. It is primitive, yet effective.

This rule has the following arguments:

path (string)
The filesystem path to the directory to scan.
packages (list of string)

List of package names to include.

Filesystem walking will find files in a directory <path>/<value>/ or in a file <path>/<value>.py.

Returns a list of objects representing Python resources found in the virtualenv. The types of these objects can be PythonSourceModule, PythonBytecodeModule, PythonPackageResource, etc.

The returned resources are typically added to a FileManifest or PythonExecutable to make them available to a packaged application.

PythonDistribution.read_virtualenv(path)

This method attempts to read Python resources from an already built virtualenv.

Important

PyOxidizer only supports finding modules and resources populated via traditional means (e.g. pip install or python setup.py install). If .pth or similar mechanisms are used for installing modules, files may not be discovered properly.

It accepts the following arguments:

path (string)

The filesystem path to the root of the virtualenv.

Python modules are typically in a lib/pythonX.Y/site-packages directory (on UNIX) or Lib/site-packages directory (on Windows) under this path.

Returns a list of objects representing Python resources found in the virtualenv. The types of these objects can be PythonSourceModule, PythonBytecodeModule, PythonPackageResource, etc.

The returned resources are typically added to a FileManifest or PythonExecutable to make them available to a packaged application.

PythonDistribution.setup_py_install(...)

This method runs python setup.py install against a package at the specified path.

It accepts the following arguments:

package_path
String filesystem path to directory containing a setup.py to invoke.
extra_envs={}
Optional dict of string key-value pairs constituting extra environment variables to set in the invoked python process.
extra_global_arguments=[]
Optional list of strings of extra command line arguments to pass to python setup.py. These will be added before the install argument.

Returns a list of objects representing Python resources installed as part of the operation. The types of these objects can be PythonSourceModule, PythonBytecodeModule, PythonPackageResource, etc.

The returned resources are typically added to a FileManifest or PythonExecutable to make them available to a packaged application.

PythonDistribution.to_python_executable(...)

This method constructs a PythonExecutable instance. It essentially says build an executable embedding Python from this distribution.

The accepted arguments are:

name (str)
The name of the application being built. This will be used to construct the default filename of the executable.
resources_policy (str)

The policy to apply when adding resources to the produced instance.

See Python Resources Policy for documentation on allowed values. The default value is in-memory-only.

config (PythonEmbeddedConfig)

The default configuration of the embedded Python interpreter.

Default is what PythonInterpreterConfig() returns.

extension_module_filter (str)

The filter to apply to determine which extension modules to add.

See PythonDistribution.extension_modules(filter='all', preferred_variants=None) for what values are accepted and their behavior.

Default is all.

preferred_extension_module_variants (dict of string to string)

Preferred extension module variants to use. See See PythonDistribution.extension_modules(filter='all', preferred_variants=None) for behavior.

Default is None, which will use the first variant.

include_sources (bool)

Boolean to control whether sources of Python modules are added in addition to bytecode.

Default is True.

include_resources (bool)

Boolean to control whether non-module resource data from the distribution is added.

Default is False.

include_test (bool)

Boolean to control whether test-specific objects are included.

Default is False.

Python Resources

At run-time, Python interpreters need to consult resources like Python module source and bytecode as well as resource/data files. We refer to all of these as Python Resources.

Configuration files represent Python Resources via the types PythonSourceModule, PythonBytecodeModule, PythonPackageResource, PythonPackageDistributionResource, and PythonExtensionModule.

These are described in detail in the following sections.

PythonSourceModule

This type represents Python source modules, agnostic of location.

Each instance has the following attributes:

name (string)
Fully qualified name of the module. e.g. foo.bar.
is_package (bool)
Whether this module is also a Python package (or sub-package).

Instances cannot be manually constructed.

PythonBytecodeModule

This type represents a Python module defined through bytecode.

Each instance has the following attributes:

name (string)
Fully qualified name of the module. e.g. foo.bar
optimize_level (int)
Optimization level of compiled bytecode. Must be the value 0, 1, or 2.
is_package (bool)
Whether the module is also a Python package (or sub-package).

PythonPackageResource

This type represents a resource _file_ in a Python package. It is effectively a named blob associated with a Python package. It is typically accessed using the importlib.resources API.

Each instance has the following attributes:

package (string)
Python package this resource is associated with.
name (string)
Name of this resource.

PythonPackageDistributionResource

This type represents a named resource to make available as Python package distribution metadata. These files are typically accessed using the importlib.metadata API.

Each instance represents a logical file in a <package>-<version>.dist-info or <package>-<version>.egg-info directory. There are specifically named files that contain certain data. For example, a *.dist-info/METADATA file describes high-level metadata about a Python package.

Each instance has the following attributes:

package (string)
Python package this resource is associated with.
name (string)
Name of this resource.

PythonExtensionModule

This type represents a compiled Python extension module.

Each instance has the following attributes:

name (string)
Unique name of the module being provided.

Python Resources Policy

There are various ways to add resources (typically Python resources) to a binary. For example, you can import modules from memory or the filesystem. Often, configuration files may wish to be explicit about what behavior is and is not allowed. A Python Resources Policy is used to apply said behavior.

A Python Resources Policy is defined by a str. The following values are recognized.

in-memory-only
Resources are to be loaded from in-memory only. If a resource cannot be loaded from memory (e.g. dynamically linked Python extension modules in some configurations), an error will (likely) occur.
filesystem-relative-only:<prefix>
Values starting with filesystem-relative-only: specify that resources are to be loaded from the filesystem from paths relative to the produced binary. Files will be installed at the path prefix denoted by the value after the :. e.g. filesystem-relative-only:lib will install resources in a lib/ directory.
prefer-in-memory-fallback-filesystem-relative:<prefix>
Values starting with prefer-in-memory-fallback-filesystem-relative represent a hybrid between in-memory-only and filesystem-relative-only:<prefix>. Essentially, if in-memory resource loading is supported, it is used. Otherwise we fall back to loading from the filesystem from paths relative to the produced binary.

Python Interpreter Configuration

A Python interpreter has settings to control how it runs. Configuration files represent these settings through the PythonInterpreterConfig(...) type.

PythonInterpreterConfig(...)

This type configures the default behavior of the embedded Python interpreter.

Embedded Python interpreters are configured and instantiated using a pyembed::PythonConfig data structure. The pyembed crate defines a default instance of this data structure with parameters defined by the settings in this type.

Note

If you are writing custom Rust code and constructing a custom pyembed::PythonConfig instance and don’t use the default instance, this config type is not relevant to you and can be omitted from your config file.

The following arguments can be defined to control the default PythonConfig behavior:

bytes_warning (int)

Controls the value of Py_BytesWarningFlag.

Default is 0.

filesystem_importer (bool)

Controls whether to enable Python’s filesystem based importer. Enabling this importer allows Python modules to be imported from the filesystem.

Default is False (since PyOxidizer prefers embedding Python modules in binaries).

ignore_environment (bool)

Controls the value of Py_IgnoreEnvironmentFlag.

This is likely wanted for embedded applications that don’t behave like python executables.

Default is True.

inspect (bool)

Controls the value of Py_InspectFlag.

Default is False.

interactive (bool)

Controls the value of Py_InteractiveFlag.

Default is False.

isolated (bool)
Controls the value of Py_IsolatedFlag.
legacy_windows_fs_encoding (bool)

Controls the value of Py_LegacyWindowsFSEncodingFlag.

Only affects Windows.

Default is False.

legacy_windows_stdio (bool)

Controls the value of Py_LegacyWindowsStdioFlag.

Only affects Windows.

Default is False.

optimize_level (bool)

Controls the value of Py_OptimizeFlag.

Default is 0, which is the Python default. Only the values 0, 1, and 2 are accepted.

This setting is only relevant if dont_write_bytecode is false and Python modules are being imported from the filesystem.

parser_debug (bool)

Controls the value of Py_DebugFlag.

Default is False.

quiet (bool)
Controls the value of Py_QuietFlag.
raw_allocator (string)

Which memory allocator to use for the PYMEM_DOMAIN_RAW allocator.

This controls the lowest level memory allocator used by Python. All Python memory allocations use memory allocated by this allocator (higher-level allocators call into this pool to allocate large blocks then allocate memory out of those blocks instead of using the raw memory allocator).

Values can be jemalloc, rust, or system.

jemalloc will have Python use the jemalloc allocator directly.

rust will use Rust’s global allocator (whatever that may be).

system will use the default allocator functions exposed to the binary (malloc(), free(), etc).

The jemalloc allocator requires the jemalloc-sys crate to be available. A run-time error will occur if jemalloc is configured but this allocator isn’t available.

Important: the rust crate is not recommended because it introduces performance overhead.

Default is jemalloc on non-Windows targets and system on Windows. (The jemalloc-sys crate doesn’t work on Windows MSVC targets.)

run_eval (string)

Will cause the interpreter to evaluate a Python code string defined by this value after the interpreter initializes.

An example value would be import mymodule; mymodule.main().

run_file (string)

Will cause the interpreter to evaluate a file at the specified filename.

The filename is resolved at run-time using whatever mechanisms the Python interpreter applies. i.e. this is little different from running python <path>.

run_module (string)

The Python interpreter will load a Python module with this value’s name as the __main__ module and then execute that module.

This mode is similar to python -m <module> but isn’t exactly the same. python -m <module> has additional functionality, such as looking for the existence of a <module>.__main__ module. PyOxidizer does not do this. The value of this argument will be the exact module name that is imported and run as __main__.

run_noop (bool)
Instructs the Python interpreter to do nothing after initialization.
run_repl (bool)
The Python interpreter will launch an interactive Python REPL connected to stdio. This is similar to the default behavior of running a python executable without any arguments.
site_import (bool)

Controls the inverse value of Py_NoSiteFlag.

The site module is typically not needed for standalone Python applications.

Default is False.

stdio_encoding (string)

Defines the encoding and error handling mode for Python’s standard I/O streams (sys.stdout, etc). Values are of the form encoding:error e.g. utf-8:ignore or latin1-strict.

If defined, the Py_SetStandardStreamEncoding() function is called during Python interpreter initialization. If not, the Python defaults are used.

sys_frozen (bool)

Controls whether to set the sys.frozen attribute to True. If false, sys.frozen is not set.

Default is False.

sys_meipass (bool)

Controls whether to set the sys._MEIPASS attribute to the path of the executable.

Setting this and sys_frozen to true will emulate the behavior of PyInstaller and could possibly help self-contained applications that are aware of PyInstaller also work with PyOxidizer.

Default is False.

sys_paths (array of strings)

Defines filesystem paths to be added to sys.path.

Setting this value will imply filesystem_importer = true.

The special token $ORIGIN in values will be expanded to the absolute path of the directory of the executable at run-time. For example, if the executable is /opt/my-application/pyapp, $ORIGIN will expand to /opt/my-application and the value $ORIGIN/lib will expand to /opt/my-application/lib.

If defined in multiple sections, new values completely overwrite old values (values are not merged).

Default is an empty array ([]).

terminfo_resolution (string)

How the terminal information database (terminfo) should be configured.

See Terminfo Database for more about terminal databases.

The value dynamic (the default) looks at the currently running operating system and attempts to do something reasonable. For example, on Debian based distributions, it will look for the terminfo database in /etc/terminfo, /lib/terminfo, and /usr/share/terminfo, which is how Debian configures ncurses to behave normally. Similar behavior exists for other recognized operating systems. If the operating system is unknown, PyOxidizer falls back to looking for the terminfo database in well-known directories that often contain the database (like /usr/share/terminfo).

The value none indicates that no configuration of the terminfo database path should be performed. This is useful for applications that don’t interact with terminals. Using none can prevent some filesystem I/O at application startup.

The value static indicates that a static path should be used for the path to the terminfo database. That path should be provided by the terminfo_dirs configuration option.

terminfo is not used on Windows and this setting is ignored on that platform.

terminfo_dirs

Path to the terminfo database. See the above documentation for terminfo_resolution for more on the terminfo database.

This value consists of a : delimited list of filesystem paths that ncurses should be configured to use. This value will be used to populate the TERMINFO_DIRS environment variable at application run time.

unbuffered_stdio (bool)

Controls the value of Py_UnbufferedStdioFlag.

Setting this makes the standard I/O streams unbuffered.

Default is False.

use_hash_seed (bool)

Controls the value of Py_HashRandomizationFlag.

Default is False.

user_site_directory (bool)

Controls the inverse value of Py_NoUserSiteDirectory.

Default is False.

write_bytecode (bool)

Controls the inverse value of Py_DontWriteBytecodeFlag.

This is only relevant if the interpreter is configured to import modules from the filesystem.

Default is False.

write_modules_directory_env (string)

Environment variable that defines a directory where modules-<UUID> files containing a \n delimited list of loaded Python modules (from sys.modules) will be written upon interpreter shutdown.

If this setting is not defined or if the environment variable specified by its value is not present at run-time, no special behavior will occur. Otherwise, the environment variable’s value is interpreted as a directory, that directory and any of its parents will be created, and a modules-<UUID> file will be written to the directory.

This setting is useful for determining which Python modules are loaded when running Python code.

Python Binaries

Binaries containing an embedded Python interpreter can be defined by configuration files. They are defined via the PythonExecutable type. In addition, the PythonEmbeddedResources type represents the collection of resources made available to an embedded Python interpreter.

PythonEmbeddedResources

The PythonEmbeddedResources type represents resources made available to a Python interpreter. The resources tracked by this type are consumed by the pyembed crate at build and run time. The tracked resources include:

  • Python module source and bytecode
  • Python package resources
  • Shared library dependencies

While the type’s name has embedded in it, resources referred to by this type may or may not actually be embedded in a Python binary or loaded directly from the binary. Rather, the term embedded comes from the fact that the data structure describing the resources is typically embedded in the binary or made available to an embedded Python interpreter.

Instances of this type are constructed by transforming a type representing a Python binary. e.g. PythonExecutable.to_embedded_resources().

If this type is returned by a target function, its build action will write out files that represent the various resources encapsulated by this type. There is no run action associated with this type.

PythonExecutable

The PythonExecutable type represents an executable file containing the Python interpreter, Python resources to make available to the interpreter, and a default run-time configuration for that interpreter.

Instances are constructed from PythonDistribution instances using PythonDistribution.to_python_executable(...).

PythonExecutable.add_in_memory_module_source(module)

This method registers a Python source module with a PythonExecutable instance. The module will be imported from memory at run-time. The argument must be a PythonSourceModule instance.

If called multiple times for the same module, the last write wins.

PythonExecutable.add_filesystem_relative_module_source(prefix, module)

This method registers a Python source module with a PythonExecutable instance. The module will be imported from the filesystem from a file relative to the built executable. The directory prefix for the generated file is defined by prefix.

If called multiple times for the same module, the last write wins.

PythonExecutable.add_module_source(module)

This method registers a Python source module with a PythonExecutable instance.

This method is a glorified proxy to PythonExecutable.add_in_memory_module_source(module) or PythonExecutable.add_filesystem_relative_module_source(prefix, module) depending on the Python Resources Policy in effect.

PythonExecutable.add_in_memory_module_bytecode(module, optimize_level=0)

This method registers Python module bytecode with a PythonExecutable instance. The module will be imported from memory at run-time.

The first argument must be a PythonSourceModule instance. The 2nd argument the value 0, 1, or 2.

PythonExecutable.add_filesystem_relative_module_bytecode(prefix, module, optimize_level=0)

This method registers Python module bytecode with a PythonExecutable instance. The module will be imported from the filesystem from a file relative to the built executable. The directory prefix for the generated file is defined by prefix.

The module argument must be a PythonSourceModule instance. The optimize_level argument must be the value 0, 1, or 2.

If called multiple times for the same module, the last write wins.

PythonExecutable.add_module_bytecode(module, optimize_level=0)

This method registers a Python module bytecode with a PythonExecutable instance.

This method is a glorified proxy to PythonExecutable.add_in_memory_module_bytecode(module, optimize_level=0) or PythonExecutable.add_filesystem_relative_module_bytecode(prefix, module, optimize_level=0) depending on the Python Resources Policy in effect. See these other methods for documentation of behavior.

PythonExecutable.add_in_memory_package_resource(resource)

This method adds a PythonPackageResource instance to the PythonExecutable instance, making that resource available via in-memory access.

If multiple resources sharing the same (package, name) pair are added, the last added one is used.

PythonExecutable.add_filesystem_relative_package_resource(prefix, resource)

This method adds a PythonPackageResource instance to the PythonExecutable instance. The resource will be materialized on the filesystem next to the produced executable at a path derived from the resource’s attributes. The directory prefix for the generated file is defined by prefix.

If multiple resources sharing the same (prefix, package, name) tuple are added, the last added one is used.

PythonExecutable.add_package_resource(resource)

This method adds a PythonPackageResource instance to the PythonExecutable instance.

This method is a glorified proxy to PythonExecutable.add_in_memory_package_resource(resource) or PythonExecutable.add_filesystem_relative_package_resource(prefix, resource) depending on the Python Resources Policy in effect. See these other methods for documentation of behavior.

PythonExecutable.add_in_memory_package_distribution_resource(resource)

This method adds a PythonPackageDistributionResource instance to the PythonExecutable instance, making that resource available via in-memory access.

If multiple resources sharing the same (package, name) pair are added, the last added one is used.

PythonExecutable.add_filesystem_relative_package_distribution_resource(prefix, resource)

This method adds a PythonPackageDistributionResource instance to the PythonExecutable instance. The resource will be materialized on the filesystem next to the produced executable at a path derived from the resource’s attributes. The directory prefix for the generated file is defined by prefix.

If multiple resources sharing the same (prefix, package, name) tuple are added, the last added one is used.

PythonExecutable.add_package_distribution_resource(resource)

This method adds a PythonPackageDistributionResource instance to the PythonExecutable instance.

This method is a glorified proxy to PythonExecutable.add_in_memory_package_distribution_resource(resource) or PythonExecutable.add_filesystem_relative_package_distribution_resource(prefix, resource) depending on the Python Resources Policy in effect. See these other methods for documentation of behavior.

PythonExecutable.add_in_memory_extension_module(module)

This method registers a PythonExtensionModule instance with a PythonExecutable instance. The extension module will be loaded from memory via one of the following mechanisms:

  • Linking the extension module’s symbols directly into the produced binary.
  • Embedded the extension module’s shared library into the produced binary and loading it from memory.

If multiple extension modules with the same name are added, the last added one is used.

PythonExecutable.add_filesystem_relative_extension_module(prefix, module)

This method registers a PythonExtensionModule instance with a PythonExecutable instance. The extension module will be loaded from the filesystem from a path relative to the produced executable.

If multiple extension modules with the same name are added, the last added one is used.

PythonExecutable.add_extension_module(module)

This method registers a PythonExtensionModule instance with a PythonExecutable instance. The extension module will be made available to the executable using whatever means are possible, constrained by the Python Resources Policy set by this PythonExecutable.

The extension module could be packaged in the following manner depending on the capabilities of the underlying Python distribution and resources policy:

  • If the object files for the extension module are available, the extension module may be statically linked into the produced binary.
  • If loading extension modules from in-memory import is supported, the extension module will have its dynamic library embedded in the binary.
  • The extension module will be materialized as a file next to the produced binary and will be loaded from the filesystem. (This is how Python extension modules typically work.)

If multiple extension modules with the same name are added, the last added one is used.

PythonExecutable.add_in_memory_python_resource(...)

This method registers a Python resource of various types for in-memory loading. It accepts a resource argument which can be a PythonSourceModule, PythonBytecodeModule, PythonPackageResource, or PythonExtensionModule and registers that resource with this instance. This method is a glorified proxy to the appropriate add_in_memory_* method.

The following arguments are accepted:

resource
The resource to add to the embedded Python environment.
add_source_module (bool)

When the resource is a PythonSourceModule, this flag determines whether to add the source for that resource.

Default is True.

add_bytecode_module (bool)

When the resource is a PythonSourceModule, this flag determines whether to add the bytecode for that module source.

Default is True.

optimize_level (int)
Bytecode optimization level when compiling bytecode.

PythonExecutable.add_filesystem_relative_python_resource(prefix, ...)

This method registers a Python resource of various types for filesystem loading in a path relative to the produced executable. The arguments for this method are the same as PythonExecutable.add_in_memory_python_resource(...) except the first argument is the str path prefix to install files into.

PythonExecutable.add_python_resource(...)

This method registers a Python resource of various types with the instance. The location the resource will be loaded from will be determined by the Python Resources Policy in effect for the PythonExecutable.

The arguments are the same as for PythonExecutable.add_in_memory_python_resource(...).

PythonExecutable.add_in_memory_python_resources(...)

This method registers an iterable of Python resources of various types. This method is identical to PythonExecutable.add_in_memory_python_resource() except the first argument is an iterable of resources. All other arguments are identical.

PythonExecutable.add_filesystem_relative_python_resources(prefix, ...)

This method registers an iterable of Python resources of various types. This method is identical to PythonExecutable.add_filesystem_relative_python_resource(prefix, ...) except the first argument is a path prefix to install files to and the second argument is an iterable of resources. All other arguments are identical.

PythonExecutable.add_python_resources(...)

This method registers an iterable of Python resources of various types. This method is identical to PythonExecutable.add_python_resource(...) except the argument is an iterable of resources. All other arguments are identical.

PythonExecutable.filter_from_files(files=[], glob_patterns=[])

This method filters all embedded resources (source modules, bytecode modules, and resource names) currently present on the instance through a set of resource names resolved from files.

This method accepts the following arguments:

files (array of string)
List of filesystem paths to files containing resource names. The file must be valid UTF-8 and consist of a \n delimited list of resource names. Empty lines and lines beginning with # are ignored.
glob_files (array of string)

List of glob matching patterns of filter files to read. * denotes all files in a directory. ** denotes recursive directories. This uses the Rust glob crate under the hood and the documentation for that crate contains more pattern matching info.

The files read by this argument must be the same format as documented by the files argument.

All defined files are first read and the resource names encountered are unioned into a set. This set is then used to filter entities currently registered with the instance.

PythonExecutable.to_embedded_resources()

Obtains a PythonEmbeddedResources instance representing resources to be made available to the Python interpreter.

See the PythonEmbeddedResources type documentation for more.

Interacting With the Filesystem

FileManifest()

The FileManifest type represents a set of files and their content.

FileManifest instances are used to represent things like the final filesystem layout of an installed application.

Conceptually, a FileManifest is a dict mapping relative paths to file content.

FileManifest.add_manifest(manifest)

This method overlays another FileManifest on this one. If the other manifest provides a path already in this manifest, its content will be replaced by what is in the other manifest.

FileManifest.add_python_resource(prefix, value)

This method adds a Python resource to a FileManifest instance in a specified directory prefix. A Python resource here can be a PythonSourceModule, PythonBytecodeModule, PythonPackageResource, PythonPackageDistributionResource, or PythonExtensionModule.

This method can be used to place the Python resources derived from another type or action in the filesystem next to an application binary.

FileManifest.add_python_resources(prefix, values)

This method adds an iterable of Python resources to a FileManifest instance in a specified directory prefix. This is effectively a wrapper for for value in values: self.add_python_resource(prefix, value).

For example, to place the Python distribution’s standard library Python source modules in a directory named lib:

m = FileManifest()
dist = default_python_distribution()
m.add_python_resources(dist.source_modules())

FileManifest.install(path, replace=True)

This method writes the content of the FileManifest to a directory specified by path. The path is evaluated relative to the path specified by BUILD_PATH.

If replace is True (the default), the destination directory will be deleted and the final state of the destination directory should exactly match the state of the FileManifest.

FileContent

This type represents the content of a single file.

glob(include, exclude=None, strip_prefix=None)

The glob() function resolves file patterns to a FileManifest.

include is a list of str containing file patterns that will be matched using the glob Rust crate. If patterns begin with / or look like a filesystem absolute path, they are absolute. Otherwise they are evaluated relative to the directory of the current config file.

exclude is an optional list of str and is used to exclude files from the result. All patterns in include are evaluated before exclude.

strip_prefix is an optional str to strip from the beginning of matched files. strip_prefix is stripped after include and exclude are processed.

Returns a FileManifest.