How do you handle dependency versioning and reproducible builds?
Quick Answer
Declare loose, compatible version ranges in your project's dependency list (requests>=2.28,<3.0) to allow reasonable updates, but pin exact resolved versions (including transitive dependencies) in a lock file (poetry.lock, Pipfile.lock, or a pip-compile-generated requirements.txt) for actual deployments. This guarantees every environment (dev, CI, production) installs the identical set of package versions, not just versions that happen to satisfy the range at install time.
Detailed Answer
The problem: unpinned dependencies drift over time
# requirements.txt -- loose, no pinning
requests
django
pip install -r requirements.txt # today: gets requests 2.31, django 4.2
# ... three months later, on a fresh machine ...
pip install -r requirements.txt # gets requests 2.32, django 5.0 -- different versions!
Without pinning, "the same" requirements.txt can resolve to entirely
different package versions depending on when it's installed. A
transitive dependency's new release could introduce a breaking change or
a subtle behavior difference, and the bug only shows up on a fresh
install (a new developer's machine, a rebuilt CI image, a redeployed
production server), not in the environment where it was originally
tested.
The fix: separate "what I depend on" from "what I actually installed"
# pyproject.toml -- loose ranges, expressing compatibility intent
[project]
dependencies = [
"requests>=2.28,<3.0",
"django>=4.2,<5.0",
]
# poetry.lock / Pipfile.lock (generated, committed to version control)
# exact, fully-resolved versions of EVERY dependency, including transitive ones:
requests==2.31.0
urllib3==2.0.7 <- a transitive dependency of requests, also pinned
django==4.2.7
sqlparse==0.4.4 <- a transitive dependency of django, also pinned
The pyproject.toml/Pipfile declares acceptable ranges (compatibility
intent — "any 2.x of requests is fine"). The lock file records the exact
versions that were actually resolved and tested, including every
transitive dependency, down to a fully reproducible tree. Everyone
running poetry install/pipenv install from the same lock file gets
byte-for-byte identical dependency versions.
Achieving the same with plain pip: pip-tools
# requirements.in -- loose, human-maintained
requests>=2.28,<3.0
django>=4.2,<5.0
pip-compile requirements.in # generates requirements.txt with EVERY package pinned
pip install -r requirements.txt # exact, reproducible install
pip-compile (from pip-tools) fills the same role as poetry.lock for
projects using plain pip. A fully pinned, reproducible
requirements.txt generated from a loose, human-edited input file.
Why loose ranges still matter, not just exact pins everywhere
dependencies = ["requests==2.31.0"] # too strict for a LIBRARY's own dependency declaration
If a library (as opposed to a deployable application) pins exact versions for its own dependencies, it forces every consumer of that library into the exact same versions too, creating conflicts when two libraries in the same project pin incompatible exact versions of a shared dependency. Libraries should declare loose, compatible ranges; applications (the actual deployable unit) are what should carry a fully pinned lock file for their own reproducible deployment.
Security: keeping pinned dependencies from going stale
poetry update requests # deliberately bump one pinned dependency, re-lock
pip-audit # scan installed/locked dependencies for known CVEs
Pinning solves reproducibility but introduces a new responsibility:
dependencies need periodic, deliberate updates (not just "never touch
it again") to pull in security patches. Tools like pip-audit,
Dependabot, or poetry show --outdated help surface when a pinned
version has a known vulnerability.
Declare loose, compatible version ranges for
what a project depends on, but pin exact, fully-resolved versions
(including transitive dependencies) in a committed lock file
(poetry.lock, Pipfile.lock, or pip-compile's output) for actual
deployment reproducibility. Revisit those pins periodically for
security updates rather than freezing them permanently.