When a client asked me why their CI pipeline suddenly started downloading a zip file that executed a shell script, I traced it back to a from_pretrained call on a Hugging Face model. The call itself looks harmless—just a Python function that returns a PreTrainedModel instance. Under the hood, however, it pulls a repository, extracts files, and runs any setup.py or custom torchscript conversion code that lives in the repo. If you haven't inspected the repo contents, you are essentially giving a third-party package full control over your build environment.

The hidden steps behind from_pretrained

Hugging Face's transformers library follows a fairly simple algorithm:

from transformers import AutoModel
model = AutoModel.from_pretrained("user/model-name")
  • model is a string that the library treats as a Git-LFS path.
  • It contacts https://huggingface.co/api/models/... to fetch metadata.
  • The metadata includes a list of files to download (usually pytorch_model.bin, config.json, and optionally tokenizer.json).
  • Files are cached under ~/.cache/huggingface/transformers.
  • If a setup.py or requirements.txt is present, the library may invoke pip install -r requirements.txt during the first load.
  • Finally, the binary weights are loaded into PyTorch or TensorFlow.

The crucial line is the optional pip install. It is not advertised in the docs, but the code checks for a requirements.txt in the repo and runs it in a subprocess. That means any malicious package can be pulled in silently.

Why this matters for production pipelines

Most teams treat model loading as a read-only operation. In practice, a CI job that runs from_pretrained can end up executing arbitrary code on the build server. A compromised repo could:

  • Install a package that exfiltrates environment variables.
  • Add a cron job that runs later.
  • Replace system binaries with trojaned versions.

I saw this happen when a data-science colleague used a community model that referenced a private GitHub repo in its requirements.txt. The CI job installed a package that wrote the CI token to a public gist. The breach was caught only after the token was revoked.

Securing the loading process

There is no single switch to disable the requirements.txt execution, but you can mitigate the risk with a combination of policies and code changes.

1. Pin trusted model sources

Maintain a whitelist of model identifiers that you have reviewed. Store the list in a JSON file and validate the identifier before calling from_pretrained.

import json, sys
from transformers import AutoModel

with open('trusted_models.json') as f:
    trusted = json.load(f)

model_id = sys.argv[1]
if model_id not in trusted:
    raise ValueError('Untrusted model source')

model = AutoModel.from_pretrained(model_id)
  • trusted_models.json contains only the models you have inspected.
  • The script aborts early if the identifier is unknown.

2. Disable automatic dependency installation

Set the environment variable TRANSFORMERS_NO_DEPENDENCIES=1 before launching your Python process. The library will then skip any pip install step and raise an exception if a requirements.txt is found.

export TRANSFORMERS_NO_DEPENDENCIES=1
python load_model.py user/model-name
  • This forces you to install dependencies manually, where you can audit them.

3. Use a sandboxed environment

Run the loading code inside a Docker container that has no network access except to huggingface.co. Use --network=none and mount a read-only cache directory. If the model tries to fetch additional resources, the container will fail fast.

docker run --rm \
  --network=none \
  -v $HOME/.cache/huggingface:/cache:ro \
  -v $(pwd):/app \
  -w /app python:3.11-slim bash -c "\
    pip install transformers && \
    TRANSFORMERS_NO_DEPENDENCIES=1 python load_model.py user/model-name\"
  • The container cannot reach external URLs, so any hidden download attempt is blocked.

4. Verify repository contents before use

Hugging Face provides a repo_info endpoint that returns a file manifest. Pull the manifest, scan for suspicious filenames (setup.py, requirements.txt, *.sh), and reject the model if any appear.

import requests

def safe_repo(model_id):
    url = f"https://huggingface.co/api/models/{model_id}"  # simplified
    data = requests.get(url).json()
    for file in data.get('siblings', []):
        if file['rfilename'] in {'setup.py', 'requirements.txt'}:
            return False
    return True
  • safe_repo returns False for repos that contain executable setup files.

Practical warning: the cache can lie

Even after you whitelist a model, the local cache may already contain a compromised version from an earlier run. Always clear the cache when you change the whitelist or when you enable TRANSFORMERS_NO_DEPENDENCIES.

rm -rf ~/.cache/huggingface/transformers/*

If you forget this step, the next load will use the stale, potentially malicious files.

When to accept the risk

In research environments where you need to experiment with dozens of community models, the overhead of strict vetting can be prohibitive. In those cases, I recommend:

  • Running the experiments on an isolated VM that has no access to production secrets.
  • Using a separate service account with limited permissions for the CI job.
  • Rotating API keys frequently so that any leak has a short window.

FAQ

How can I tell if a model has a hidden requirements.txt?

Check the model page on huggingface.co; the file list is displayed under the Files and versions tab. You can also call the api/models/{model_id} endpoint and look for a requirements.txt entry.

Does setting TRANSFORMERS_NO_DEPENDENCIES=1 break any official models?

Some official models ship with optional dependencies (e.g., sentencepiece for tokenizers). The flag will raise an error if those are missing, so you need to install them manually beforehand.

Is there a way to sign model repositories so I can verify integrity?

Hugging Face supports model signatures via GPG, but the feature is not widely adopted yet. Until it becomes mainstream, rely on the whitelist and manifest checks.

Wrapping up

Treat model loading as a code-execution step, not just a data fetch. By pinning trusted identifiers, disabling automatic installs, sandboxing the process, and verifying repo contents, you can keep your pipelines safe without sacrificing the flexibility that Hugging Face offers. If you ever doubt a model, pull it into an isolated container first—better to waste a minute than to compromise a whole system.