Rust Projects¶
PyOxidizer uses Rust projects to build binaries embedding Python.
If you just have a standalone configuration file (such as when running
pyoxidizer init-config-file
), a temporary Rust project will be
created as part of building binaries and the existence of Rust should
be largely invisible (except for the output from building the Rust project).
If you use pyoxidizer init-rust-project
to initialize a
PyOxidizer
application, the Rust project exists side-by-side with
the PyOxidizer
configuration file and can be modified like
any other Rust project.
Either way, the PyOxidizer
configuration file works alongside Rust
to build binaries.
Layout¶
Generated Rust projects all have a similar layout:
$ find pyapp -type f | grep -v .git
Cargo.toml
src/main.rs
pyembed/Cargo.toml
pyembed/build.rs
pyembed/src/config.rs
pyembed/src/data.rs
pyembed/src/importer.rs
pyembed/src/lib.rs
pyembed/src/pyalloc.rs
pyembed/src/pyinterp.rs
pyembed/src/pystr.rs
Main Application Project¶
The Cargo.toml
file is the configuration file for the Rust project.
Read more in
the official Cargo documentation.
The magic lines in this file to enable PyOxidizer are the following:
[dependencies]
pyembed = { path = "pyembed" }
These lines declare a dependency on the pyembed
package in the directory
pyembed
. Cargo.toml
is overall pretty straightforward.
Next let’s look at pyapp/src/main.rs
. If you aren’t familiar with Rust
projects, the src/main.rs
file is the default location for the source
file implementing an executable. If we open that file, we see a
fn main() {
line, which declares the main function for our executable.
The file is relatively straightforward. We import some symbols from the
pyembed
crate. We then construct a config object, use that to construct
a Python interpreter, then we run the interpreter and pass its exit code
to exit()
. Succinctly, we instantiate and run an embedded Python
interpreter. That’s our executable.
The pyembed
Package¶
The bulk of the files in our new project are in the pyembed
directory.
This directory defines a Rust project whose job it is to build and manage
an embedded Python interpreter. This project behaves like any other Rust
library project: there’s a Cargo.toml
, a src/lib.rs
defining the
main library define, and a pile of other .rs
files implementing the
library functionality. The only functionality you will likely be concerned
about are the PythonConfig
and MainPythonInterpreter
structs. These
types define how the embedded Python interpreter is configured and executed.
If you want to learn more about this crate and how it works, run cargo doc
and read pyembed Crate.
There are a few special properties about the pyembed
package worth
calling out.
First, the package is a copy of files from the PyOxidizer project. Typically,
one could reference a crate published on a package repository like
https://crates.io/ and we wouldn’t need to have local files. However,
pyembed
is currently relying on modifications to some other published
crates (we plan to upstream all changes eventually). This means we can’t
publish pyembed
on crates.io
. So we need to vendor a copy next to
your project. Sorry about the (temporary) inconvenience!
Speaking of modification to the published crates, the pyembed
’s
Cargo.toml
enumerates those crates. If pyoxidizer
was run from
an installed executable, these modified crates will be obtained from
PyOxidizer’s canonical Git repository. If pyoxidizer
was run out of
the PyOxidizer source repository, these modified crates will be obtained
from the local filesystem path to that repository. You may want to
consider making copies of these crates and/or vendoring them next to your
project if you aren’t comfortable fetching dependencies from the local
filesystem or a Git repository.
Build Artifacts for pyembed
¶
The pyembed
crate needs to reference special artifacts as part of its
build process in order to compile a Python interpreter into a binary.
These special artifacts are generated by the pyembed
crate’s build.rs
build script. This file defines a program that runs as part of building the
crate. The main goal of the build.rs
script is to read the auto-generated
artifact defining metadata needed by Rust’s build system and to print it.
In order to do so, it may need to invoke PyOxidizer
to generate this
metadata file.
The build artifacts required by pyembed
are generated by resolving
a configuration file target returning a PythonEmbeddedData
instance. In the auto-generated pyoxidizer.bzl
configuration file, the
embedded
target facilitates this purpose.
There are multiple ways for the build.rs
script to invoke PyOxidizer
.
The default option is to call pyoxidizer run-build-script
. This command
is a special variation of pyoxidizer build
that knows it is running in
the context of a Rust build script and it will take appropriate actions.
For example, artifacts required by pyembed
will be written to OUT_DIR
,
In addition, the content of the generated cargo_metadata.txt
file is
printed so the pyembed
crate is properly configured to embed Python.
Under the hood, pyoxidizer run-build-script
calls a function inside
the pyoxidizer
crate. Should the build script wish to avoid the dependency
on a pyoxidizer
executable and call the equivalent code as a library
(by compiling PyOxidizer
as a build dependency), it can do so. The
function it should call is
pyoxidizerlib::project_building::run_from_build()
. An example of this
is included in the auto-generated build.rs
script when running
pyoxidizer init-rust-project
.
A final option for the build script is to not invoke PyOxidizer
directly
and instead rely on artifacts built out of band. In this case, all you need
to do is read the cargo_metadata.txt
file generated by PyOxidizer
and print its contents.