Django Dependency Management Workflow¶

This guide explains how Python dependencies are managed in the Django app using pip-tools, how to pin and sync them, and how this integrates with Docker and the Makefile.

It’s written for new developers joining the project — follow it exactly and you’ll always have a clean, reproducible environment.

Project Layout (relevant parts)¶

django/
├── Dockerfile
├── piptools_helper.py
├── src/
│   └── requirements/
│       ├── base.in
│       ├── base.txt
│       ├── dev.in
│       ├── dev.txt
│       ├── prod.in
│       └── prod.txt
└── ...

*.in → human-edited “input” lists (un-pinned)
*.txt → machine-generated “lock” files (fully pinned)
piptools_helper.py → helper script (runs pip & pip-tools inside the container)

Tools¶

We use:

pip-tools → to pin and sync dependencies.
Makefile targets → to standardize commands and wrap docker calls.
Docker multi-stage builds → to bake pinned requirements into images.

Quick Start for a New Dev¶

1. Build the dev image¶

make build

This builds the Docker image, installing dependencies from the pinned lock file requirements/dev.txt.

2. Start your environment¶

make up

Your Django container now runs with exactly the dependencies defined in the lock files.

3. Run tests or open a shell¶

make dj-test
make dj-shell

The Dependency Workflow (Full Cycle)¶

Step 1 — Compile / Pin Versions¶

Run this whenever you modify a .in file (or periodically to refresh versions):

make deps-compile-all

This calls:

pip-compile base.in → base.txt
pip-compile dev.in  → dev.txt
pip-compile prod.in → prod.txt

Each .txt will contain:

exact versions
SHA256 hashes
deterministic resolver output

Step 2 — Rebuild the image¶

make build

Because the Dockerfile installs from requirements/dev.txt, this step bakes the new pinned dependencies into the image.

Step 3 — Start (or restart) containers¶

make restart
# or
make up

Now your containers use the freshly pinned environment — no manual pip commands needed.

Step 4 — Verify¶

make deps-check

Checks for any dependency conflicts.

Updating or Adding Packages¶

Add a new dev-only dependency¶

make deps-add FILE=dev.in PKG="django-debug-toolbar"
make deps-compile-dev
make build
make restart

Remove a dependency¶

make deps-remove FILE=dev.in PKG="django-debug-toolbar"
make deps-compile-dev
make build
make restart

Upgrade everything to latest versions¶

make deps-update-all
make build
make restart

Upgrade specific packages¶

make deps-upgrade PKG="Django djangorestframework"
make build
make restart

Optional: Live Sync (without rebuild)¶

If you just want to update the running container quickly (not recommended for CI):

make deps-sync-dev-live

This runs pip-sync requirements/dev.txt inside the live container, bringing it in line with the lock file immediately.

⚠️ This is temporary — if you rebuild the image later, it’ll revert to whatever is in the lock files.

What Each Command Does¶

Command	Description	Scope
`make deps-compile-all`	Regenerates all `.txt` lock files from `.in`	Host
`make deps-sync-dev`	Tests syncing in a throw-away container (does not change your running one)	Ephemeral
`make deps-sync-dev-live`	Runs pip-sync inside your current Django container	Live
`make build`	Rebuilds the Docker image and installs dependencies from `requirements/dev.txt`	Image
`make up` / `make restart`	Runs containers from the latest image	Runtime
`make deps-check`	Runs `pip check` to detect conflicts	Inside container
`make deps-outdated`	Lists outdated packages in the container	Inside container

How pip-tools Works (for Context)¶

pip-compile reads .in files → resolves versions → writes pinned .txt
pip-sync reads .txt → installs exactly those versions (and removes extras)

Using both ensures every environment is identical across dev, CI, and prod.

Example Developer Flow¶

Clone repo & build environment

make build
make up

Add a new package

make deps-add FILE=dev.in PKG="django-environ"
make deps-compile-dev
make build
make restart

Confirm everything works

make dj-test

Commit the lock files

git add django/src/requirements/*.txt
git commit -m "Pin dependencies after adding django-environ"

Production Builds¶

Prod images install from requirements/prod.txt.

To build the prod image:

make prod-build

This stage uses the same pinned mechanism to guarantee deterministic deployments.

Troubleshooting¶

Problem	Cause / Fix
`pg_config not found` when compiling	Add `libpq-dev` to dev image (already handled in Dockerfile)
“unsafe packages” (`pip`, `setuptools`) warning	Harmless — shown because we use `--allow-unsafe` for full reproducibility
`pip-sync` removes unexpected packages	Expected: it makes the env exactly match the lock file
Dependencies changed but rebuild didn’t pick them up	Run `make build` again — Docker caches layers until the `requirements/` folder changes

Summary Cheat-Sheet¶

Action	Command
Compile pins	`make deps-compile-all`
Sync exact versions (test)	`make deps-sync-dev`
Sync inside live container	`make deps-sync-dev-live`
Rebuild image with pins	`make build`
Restart containers	`make restart`
Upgrade all deps	`make deps-update-all`
Upgrade specific deps	`make deps-upgrade PKG="Django"`
Add package	`make deps-add FILE=dev.in PKG="newlib"`
Remove package	`make deps-remove FILE=dev.in PKG="oldlib"`
Check env health	`make deps-check`
See outdated deps	`make deps-outdated`

Key Takeaways¶

The .in files are editable; the .txt files are machine-generated locks.
Always commit the .txt lock files after compilation.
A normal flow is: make deps-compile-all → make build → make restart.
Rebuilding the image automatically installs the new pinned dependencies — no manual pip-sync needed.
pip-sync is for local or CI verification, not a regular step after every build.

< Backend Development - Environment File Setup

Next: Backend Development - GitHub CLI Install Setup >