PythonInterpreterConfig
¶
-
class
starlark_pyoxidizer.
PythonInterpreterConfig
¶ This type configures the default behavior of the embedded Python interpreter.
Embedded Python interpreters are configured and instantiated using a Rust
pyembed::OxidizedPythonInterpreterConfig
data structure. Thepyembed
crate defines a default instance of this data structure with parameters defined by the settings in this type.Note
If you are writing custom Rust code and constructing a custom
pyembed::OxidizedPythonInterpreterConfig
instance and don’t use the default instance, this config type is not relevant to you and can be omitted from your config file.Danger
Some of the settings exposed by Python’s initialization APIs are extremely low level and brittle. Various combinations can cause the process to crash/exit ungracefully. Be very cautious when setting these low-level settings.
Instances are constructed by calling
PythonDistribution.make_python_interpreter_config()
.Instance state is managed via attributes.
There are a ton of attributes and most attributes are not relevant to most applications. The bulk of the attributes exist to give full control over Python interpreter initialization.
The following attributes control features provided by the
pyembed
Rust crate, which manages the embedded Python interpreter in generated executables. These attributes provide features and level of control over embedded Python interpreters beyond what is possible with Python’s initialization C API.allocator_backend
allocator_raw
allocator_mem
allocator_obj
allocator_pymalloc_arena
allocator_debug
oxidized_importer
filesystem_importer
argvb
multiprocessing_auto_dispatch
multiprocessing_start_method
sys_frozen
sys_meipass
terminfo_resolution
write_modules_directory_env
The following attributes correspond to fields of the PyPreConfig C struct used to initialize the Python interpreter.
config_profile
allocator
configure_locale
coerce_c_locale
coerce_c_locale_warn
development_mode
isolated
legacy_windows_fs_encoding
parse_argv
use_environment
utf8_mode
The following attributes correspond to fields of the PyConfig C struct used to initialize the Python interpreter.
base_exec_prefix
base_executable
base_prefix
buffered_stdio
bytes_warning
check_hash_pycs_mode
configure_c_stdio
dump_refs
exec_prefix
executable
fault_handler
filesystem_encoding
hash_seed
home
import_time
inspect
install_signal_handlers
interactive
legacy_windows_stdio
malloc_stats
module_search_paths
optimization_level
parser_debug
pathconfig_warnings
prefix
program_name
pycache_prefix
python_path_env
quiet
run_command
run_filename
run_module
show_ref_count
site_import
skip_first_source_line
stdio_encoding
stdio_errors
tracemalloc
user_site_directory
verbose
warn_options
write_bytecode
x_options
-
allocator_backend
¶ (
string
)Configures a custom memory allocator to be used by Python.
Accepted values are:
default
Let Python choose how to configure the allocator.
This will likely use the
malloc()
,free()
, etc functions linked to the binary.jemalloc
Use the jemalloc allocator.
(Not available on Windows.)
mimalloc
- Use the mimalloc allocator (https://github.com/microsoft/mimalloc).
rust
- Use Rust’s global allocator (whatever that may be).
snmalloc
- Use the snmalloc allocator (https://github.com/microsoft/snmalloc).
The
jemalloc
,mimalloc
, andsnmalloc
allocators require the presence of additional Rust crates. A run-time error will occur if these allocators are configured but the binary was built without these crates. (This should not occur when usingpyoxidizer
to build the binary.)When a custom allocator is configured, the autogenerated Rust crate used to build the binary will configure the Rust global allocator (
#[global_allocator] attribute
) to use the specified allocator.Important
The
rust
allocator is not recommended because it introduces performance overhead. But it may help with debugging in some situations.Note
Both
mimalloc
andsnmalloc
require thecmake
build tool to compile code as part of their build process. If this tool is not available in the build environment, you will encounter a build error with a message similar tofailed to execute command: The system cannot find the file specified. (os error 2) is `cmake` not installed?
.The workaround is to install cmake or use a different allocator.
Note
snmalloc
only supports targeting to macOS 10.14 or newer. You will likely see build errors when building a binary targeting macOS 10.13 or older.Default is
jemalloc
on non-Windows targets anddefault
on Windows. (Thejemalloc-sys
crate doesn’t work on Windows MSVC targets.)
-
allocator_raw
¶ (
bool
)Controls whether to install a custom allocator (defined by
allocator_backend
) into Python’s raw allocator domain (PYMEM_DOMAIN_RAW
in Python C API speak).Setting this to
True
will replace the system allocator (e.g.malloc()
,free()
) for this domain.A value of
True
only has an effect ifallocator_backend
is some value other thandefault
.Defaults to
True
.
-
allocator_mem
¶ (
bool
)Controls whether to install a custom allocator (defined by
allocator_backend
) into Python’s mem allocator domain (PYMEM_DOMAIN_MEM
in Python C API speak).Setting this to
True
will replacepymalloc
as the allocator for this domain.A value of
True
only has an effect ifallocator_backend
is some value other thandefault
.Defaults to
False
.
-
allocator_obj
¶ (
bool
)Controls whether to install a custom allocator (defined by
allocator_backend
) into Python’s obj allocator domain (PYMEM_DOMAIN_OBJ
in Python C API speak).Setting this to
True
will replacepymalloc
as the allocator for this domain.A value of
True
only has an effect ifallocator_backend
is some value other thandefault
.Defaults to
False
.
-
allocator_pymalloc_arena
¶ (
bool
)Controls whether to install a custom allocator (defined by
allocator_backend
) into Python’spymalloc
to be used as its arena allocator.The
pymalloc
allocator is used by Python by default and will use the system’s allocator functions (malloc()
,VirtualAlloc()
, etc) by default.Setting this to
True
will have no effect ifpymalloc
is not being used (theallocator_mem
andallocator_obj
settings areTrue
and have replacedpymalloc
as the allocator backend for these domains).A value of
True
only has an effect ifallocator_backend
is some value other thandefault
.Defaults to
False
.
-
allocator_debug
¶ (
bool
)Whether to enable debug hooks for Python’s memory allocators.
Enabling debug hooks enables debugging of memory-related issues in the Python interpreter. This setting effectively controls whether to call PyMem_SetupDebugHooks() during interpreter initialization. See the linked documentation for more.
Defaults to
False
.
-
oxidized_importer
¶ (
bool
)Whether to install the
oxidized_importer
meta path importer (oxidized_importer Python Extension) onsys.meta_path
andsys.path_hooks
during interpreter initialization. If installed, we will always occupy the first element in these lists.Defaults to
True
.
-
filesystem_importer
¶ (
bool
)Whether to install the standard library path-based importer for loading Python modules from the filesystem.
If disabled,
sys.meta_path
andsys.path_hooks
will not have entries provided by the standard library’s path-based importer.Due to quirks in how the Python interpreter is initialized, the standard library’s path-based importer will be registered on
sys.meta_path
andsys.path_hooks
for a brief moment when the interpreter is initialized. Ifsys.path
contains valid entries that would be serviced by this importer andoxidized_importer
isn’t able to service imports, it is possible for the path-based importer to be used to import some Python modules needed to initialize the Python interpreter. In many cases, this behavior is harmless. In all cases, the path-based importer is disabled after Python interpreter initialization, so future imports won’t be serviced by the path-based importer if it is disabled by this flag.The filesystem importer is enabled automatically if
PythonInterpreterConfig.module_search_paths
is non-empty.
-
argvb
¶ (
bool
)Whether to expose a
sys.argvb
attribute containingbytes
versions of process arguments.On platforms where the process receives
char *
arguments, Python normalizes these values tounicode
and makes them available viasys.argv
. On platforms where the process receiveswchar_t *
arguments, Python may interpret the bytes as a certain encoding. This encoding normalization can be lossy.Enabling this feature will give Python applications access to the raw
bytes
values of arguments that are actually used. The single or double width bytes nature of the data is preserved.Unlike
sys.argv
which may chomp off leading arguments depending on the Python execution mode,sys.argvb
has all the arguments used to initialize the process. The first argument is always the executable.
-
multiprocessing_auto_dispatch
¶ (
bool
)Controls whether the main execution routine of the binary will detect when the process is supposed to act as a
multiprocessing
worker process and will dispatch tomultiprocessing
automatically, instead of any other configured code.Values of
True
have the same effect as callingmultiprocessing.freeze_support()
in your application code’s__main__
and replace the need to do so.Default value is
True
.See Automatic Detection and Dispatch of multiprocessing Processes for more.
-
multiprocessing_start_method
¶ (
str
)Controls how to call
multiprocessing.set_start_method()
upon the import of themultiprocessing
module.Accepted values are:
none
Do not call
multiprocessing.set_start_method()
automatically.This mode is what Python programs do by default.
auto
- Call
multiprocessing.set_start_method()
with the appropriate value for the environment. This likely maps tospawn
on Windows andfork
on non-Windows. fork
- Call with the value
fork
. forkserver
- Call with the value
forkserver
. spawn
- Call with the value
spawn
.
The default value is
auto
.When set to a value that is not
none
, whenoxidized_importer.OxidizedFinder
services an import of themultiprocessing
module, it will automatically callmultiprocessing.set_start_method()
to configure how worker processes are created.If the
multiprocessing
module is not imported byoxidized_importer.OxidizedFinder
, this setting has no effect.
-
sys_frozen
¶ (
bool
)Controls whether to set the
sys.frozen
attribute toTrue
. Iffalse
,sys.frozen
is not set.Default is
True
.
-
sys_meipass
¶ (
bool
)Controls whether to set the
sys._MEIPASS
attribute to the path of the executable.Setting this and
sys_frozen
toTrue
will emulate the behavior of PyInstaller and could possibly help self-contained applications that are aware of PyInstaller also work with PyOxidizer.Default is
False
.
-
terminfo_resolution
¶ (
string
)Defines how the terminal information database (
terminfo
) should be configured.See Terminfo Database for more about terminal databases.
Accepted values are:
dynamic
Looks at the currently running operating system and attempts to do something reasonable.
For example, on Debian based distributions, it will look for the
terminfo
database in/etc/terminfo
,/lib/terminfo
, and/usr/share/terminfo
, which is how Debian configuresncurses
to behave normally. Similar behavior exists for other recognized operating systems.If the operating system is unknown, PyOxidizer falls back to looking for the
terminfo
database in well-known directories that often contain the database (like/usr/share/terminfo
).none
- The value
none
indicates that no configuration of theterminfo
database path should be performed. This is useful for applications that don’t interact with terminals. Usingnone
can prevent some filesystem I/O at application startup. static:<path>
Indicates that a static path should be used for the path to the
terminfo
database.This values consists of a
:
delimited list of filesystem paths thatncurses
should be configured to use. This value will be used to populate theTERMINFO_DIRS
environment variable at application run time.
terminfo
is not used on Windows and this setting is ignored on that platform.
-
write_modules_directory_env
¶ (
string
orNone
)Environment variable that defines a directory where
modules-<UUID>
files containing a\n
delimited list of loaded Python modules (fromsys.modules
) will be written upon interpreter shutdown.If this setting is not defined or if the environment variable specified by its value is not present at run-time, no special behavior will occur. Otherwise, the environment variable’s value is interpreted as a directory, that directory and any of its parents will be created, and a
modules-<UUID>
file will be written to the directory.This setting is useful for determining which Python modules are loaded when running Python code.
-
config_profile
¶ (
string
)This attribute controls which set of default values to use for attributes that aren’t explicitly defined. It effectively controls which C API to use to initialize the
PyPreConfig
instance.Accepted values are:
-
allocator
¶ (
string
orNone
)Controls the value of PyPreConfig.allocator.
Accepted values are:
None
- Use the default.
not-set
PYMEM_ALLOCATOR_NOT_SET
default
PYMEM_ALLOCATOR_DEFAULT
debug
PYMEM_ALLOCATOR_DEBUG
malloc
PYMEM_ALLOCATOR_MALLOC
malloc-debug
PYMEM_ALLOCATOR_MALLOC_DEBUG
py-malloc
PYMEM_ALLOCATOR_PYMALLOC
py-malloc-debug
PYMEM_ALLOCATOR_PYMALLOC_DEBUG
-
configure_locale
¶ (
bool
orNone
)Controls the value of PyPreConfig.configure_locale.
-
coerce_c_locale
¶ (
string
orNone
)Controls the value of PyPreConfig.coerce_c_locale.
Accepted values are:
LC_CTYPE
- Read
LC_CTYPE
C
- Coerce the
C
locale.
-
coerce_c_locale_warn
¶ (
bool
orNone
)Controls the value of PyPreConfig.coerce_c_locale_warn.
-
development_mode
¶ (
bool
orNone
)Controls the value of PyPreConfig.development_mode.
-
isolated
¶ (
bool
orNone
)Controls the value of PyPreConfig.isolated.
-
legacy_windows_fs_encoding
¶ (
bool
orNone
)Controls the value of PyPreConfig.legacy_windows_fs_encoding.
-
parse_argv
¶ (
bool
orNone
)Controls the value of PyPreConfig.parse_argv.
-
use_environment
¶ (
bool
orNone
)Controls the value of PyPreConfig.use_environment.
-
utf8_mode
¶ (
bool
orNone
)Controls the value of PyPreConfig.utf8_mode.
-
base_exec_prefix
¶ (
string
orNone
)Controls the value of PyConfig.base_exec_prefix.
-
base_executable
¶ (
string
orNone
)Controls the value of PyConfig.base_exectuable.
-
base_prefix
¶ (
string
orNone
)Controls the value of PyConfig.base_prefix.
-
buffered_stdio
¶ (
bool
orNone
)Controls the value of PyConfig.buffered_stdio.
-
bytes_warning
¶ (
string
orNone
)Controls the value of PyConfig.bytes_warning.
Accepted values are:
None
none
warn
raise
-
check_hash_pycs_mode
¶ (
string
orNone
)Controls the value of PyConfig.check_hash_pycs_mode.
Accepted values are:
None
always
never
default
-
configure_c_stdio
¶ (
bool
orNone
)Controls the value of PyConfig.configure_c_stdio.
-
dump_refs
¶ (
bool
orNone
)Controls the value of PyConfig.dump_refs.
-
exec_prefix
¶ (
string
orNone
)Controls the value of PyConfig.exec_prefix.
-
executable
¶ (
string
orNone
)Controls the value of PyConfig.executable.
-
fault_handler
¶ (
bool
orNone
)Controls the value of PyConfig.fault_handler.
-
filesystem_encoding
¶ (
string
orNone
)Controls the value of PyConfig.filesystem_encoding.
-
filesystem_errors
¶ (
string
orNone
)Controls the value of PyConfig.filesystem_errors.
-
hash_seed
¶ (
int
orNone
)Controls the value of PyConfig.hash_seed.
PyConfig.use_hash_seed
will automatically be set if this attribute is defined.
-
home
¶ (
string
orNone
)Controls the value of PyConfig.home.
-
import_time
¶ Controls the value of PyConfig.import_time.
-
inspect
¶ (
bool
orNone
)Controls the value of PyConfig.inspect.
-
install_signal_handlers
¶ (
bool
orNone
)Controls the value of PyConfig.install_signal_handlers.
-
interactive
¶ (
bool
orNone
)Controls the value of PyConfig.interactive.
-
legacy_windows_stdio
¶ (
bool
orNone
)Controls the value of PyConfig.legacy_windows_stdio.
-
malloc_stats
¶ (
bool
orNone
)Controls the value of PyConfig.malloc_stats.
-
module_search_paths
¶ (
list[string]
orNone
)Controls the value of PyConfig.module_search_paths.
This value effectively controls the initial value of
sys.path
.The special string
$ORIGIN
in values will be expanded to the absolute path of the directory of the executable at run-time. For example, if the executable is/opt/my-application/pyapp
,$ORIGIN
will expand to/opt/my-application
and the value$ORIGIN/lib
will expand to/opt/my-application/lib
.Setting this to a non-empty value also has the side-effect of setting
filesystem_importer = True
-
optimization_level
¶ (
int
orNone
)Controls the value of PyConfig.optimization_level.
Allowed values are:
None
0
1
2
This setting is only relevant if
write_bytecode
isTrue
and Python modules are being imported from the filesystem using Python’s standard filesystem importer.
-
parser_debug
¶ (
bool
orNone
)Controls the value of PyConfig.parser_debug.
-
pathconfig_warnings
¶ (
bool
orNone
)Controls the value of PyConfig.pathconfig_warnings.
-
prefix
¶ (
string
orNone
)Controls the value of PyConfig.prefix.
-
program_name
¶ (
string
orNone
)Controls the value of PyConfig.program_name.
-
pycache_prefix
¶ (
string
orNone
)Controls the value of PyConfig.pycache_prefix.
-
python_path_env
¶ (
string
orNone
)Controls the value of PyConfig.pythonpath_env.
-
quiet
¶ (
bool
orNone
)Controls the value of PyConfig.quiet.
-
run_command
¶ (
string
orNone
)Controls the value of PyConfig.run_command.
-
run_filename
¶ (
string
orNone
)Controls the value of PyConfig.run_filename.
-
run_module
¶ (
string
orNone
)Controls the value of PyConfig.run_module.
-
show_ref_count
¶ (
bool
orNone
)Controls the value of PyConfig.show_ref_count.
-
site_import
¶ (
bool
orNone
)Controls the value of PyConfig.site_import.
The
site
module is typically not needed for standalone/isolated Python applications.
-
skip_first_source_line
¶ (
bool
orNone
)Controls the value of PyConfig.skip_first_source_line.
-
stdio_encoding
¶ (
string
orNone
)Controls the value of PyConfig.stdio_encoding.
-
stdio_errors
¶ (
string
orNone
)Controls the value of PyConfig.stdio_errors.
-
tracemalloc
¶ (
bool
orNone
)Controls the value of PyConfig.tracemalloc.
-
user_site_directory
¶ (
bool
orNone
)Controls the value of PyConfig.user_site_directory.
-
verbose
¶ (
bool
orNone
)Controls the value of PyConfig.verbose.
-
warn_options
¶ (
list[string]
orNone
)Controls the value of PyConfig.warn_options.
-
write_bytecode
¶ (
bool
orNone
)Controls the value of PyConfig.write_bytecode.
This only influences the behavior of Python standard path-based importer (controlled via
filesystem_importer
).
-
x_options
¶ (
list[string]
orNone
)Controls the value of PyConfig.xoptions.
Starlark Caveats¶
The PythonInterpreterConfig
Starlark type is backed by a Rust data
structure. And when attributes are retrieved, a copy of the underlying
Rust struct field is returned.
This means that if you attempt to mutate a Starlark value (as opposed to assigning an attribute), the mutation won’t be reflected on the underlying Rust data structure.
For example:
config = dist.make_python_interpreter_config()
# assigns vec!["foo", "bar"].
config.module_search_paths = ["foo", "bar"]
# Creates a copy of the underlying list and appends to that copy.
# The stored value of `module_search_paths` is still `["foo", "bar"]`.
config.module_search_paths.append("baz")
To append to a list, do something like the following:
value = config.module_search_paths
value.append("baz")
config.module_search_paths = value