If Python developers have one consistent gripe about their beloved language, it tends to be this: Why is it so hard to take a Python program and deploy it as a standalone artifact, the way C, C++, Rust, Go, and even Java can be deployed? Are we stuck with requiring everyone to install the Python runtime first before they can use a Python program? And why are all the workarounds for this problem so clunky?

One of the features that makes Python so appealing — its dynamism — is also the reason Python apps are so difficult to bundle and deploy. Not impossible, but challenging. Bundled Python apps end up being big packages, never less than a dozen megabytes or more. Plus, the tools for creating those bundles aren’t the friendliest or most convenient.

So what is it about Python’s dynamism that’s reponsible for this?

The pleasures and perils of Python’s dynamism

When we talk about Python as a “dynamic” language, that means more than the fact that Python apps are executed with an interpreter. It also means that many decisions about the behaviors of Python apps are made at run time, and not ahead of time.

Many of Python’s conveniences come from this design choice. Variables don’t have to be declared in advance, and they are automatically garbage-collected when no longer in use. Imports can be declared ahead of time, but they can also be generated at run time — and theoretically they can import code from anywhere. Python code can even be generated and interpreted at run time.

These flexible behaviors all come at a cost: it’s hard to predict what a Python program may do at run time. One of the factors that make it so difficult is that the code in a Python program can theoretically be changed by other code. A library can be imported, have methods overridden, even have its bytecode altered. It’s hard to optimize Python for high performance because so many optimizations rely on knowing what the code will do ahead of time (although the new JIT and other changes are helping with that).

Two big consequences arise from all this:

1. The most reliable way to run a Python program is through an instance of the Python runtime, so that all of Python’s dynamic behaviors can be recreated. Any solution that turns a Python app into some kind of redistributable package must include the runtime in some form. And any solution that would include “just enough” of the Python runtime to run that program would break promises about Python’s dynamism.
2. It’s difficult to package a Python app for standalone use because it’s difficult to predict which Python capabilities the app will need at runtime. Not impossible, but difficult enough that it’s far from trivial. It also means that any third-party libraries the app requires must be bundled with the app in their entirety.

Third-party libraries: All or nothing

Python apps require very clear declarations of which libraries they need to run, via pyproject.toml or requirements.txt. What’s more, Python’s dynamism means you can’t make any assumptions about which parts of those libraries are actually used.

In the world of C++ or Rust, you can compile statically-linked binaries that omit any code that isn’t called from within your program. Python libraries can’t work this way; any part of the library could be called by any code at any time. Therefore the entire library — including all of its own dependencies, like binaries — must be included.

Thus any attempt to package a Python app as a standalone executable must include all of its dependencies. The result can be quite a large package — large enough to be off-putting to those who don’t want to deliver, say, a 300MB artifact to their users. But Python’s dynamism requires including everything.

In theory you could trace the call path of a Python program and “tree-shake” it — remove everything that never gets called. But that would work only for that particular run of the program. Guaranteeing this would work for any run of the program, including those where Python’s dynamism gets exploited, is all but impossible.

The only workable solution: A total package

All of these issues mean we have only a few ways to deploy a Python program reliably:

• Install it into an existing Python interpreter. This is the most common scenario, but it requires setting up a copy of the interpreter. At best, this means an entirely separate step, one fraught with complexity if Python versions already exist on the system. This is also the scenario people want to avoid in the first place, because they want to make their app as easy to redistibute as possible.
• Bundle the interpreter with the program and its dependencies. This is the approach taken by projects like PyInstaller and Nuitka. The downsides are that the deliverables tend to be quite large, and creating them requires learning the quirks of these projects. But they do work.
• Use a system like Docker to bundle the program. Docker containers introduce their own world of trade-offs. On the one hand, you get absolutely everything you need to run the program, including any system-level dependencies. On the other hand, the resulting container can be positively hefty. And, of course, using Docker means adopting an additional software ecosystem.

Some of the newer solutions to the problem try to solve one particular pain point or another, as a way to make the whole issue less unpalatable. For instance, PyApp uses Rust to build a self-extracting binary that installs the needed Python distribution, your app, and all its dependencies. It has two big drawbacks: you need the Rust compiler to build it for your project, and your project must be an installable package that uses the pyproject.toml standard. The first of these requirements is likely to be the larger hurdle; most Python projects need a pyproject.toml of some kind at this point.

Another solution is one I wrote myself: pydeploy. It also requires the project in question be installable via pip install. Otherwise, pydeploy needs nothing more than Python’s standard library to generate a self-contained deliverable with the Python runtime included. Its big drawback right now is that it only works for Microsoft Windows, but in theory it could work on any operating system.

Maybe someday

All the recent major changes being proposed for Python, such as the new native JIT and full concurrency or multithreading, are meant to enhance Python’s behavior as a dynamic language. Any proposals designed to change that dynamism essentially would mean creating a new language with different expectations about its behavior.

While there have been attempts to launch variants of Python that address one limitation or another (e.g., Mojo), the original Python language, for all its limits, remains a massive center of gravity. As the language continues to develop, there’s always the chance we’ll someday see a “blessed”, Python-native solution to the problem of distributing standalone Python apps. In the meantime, the solutions we have may be less than elegant, but at least we have solutions.