What's the role of `pyproject.toml`, and how has Python packaging evolved?

The old way: setup.py as executable code

# setup.py (legacy)
from setuptools import setup

setup(
    name="myproject",
    version="1.0.0",
    install_requires=["requests>=2.0"],
)

Because setup.py is a Python script, building or installing a package required actually executing arbitrary code just to read its metadata. This is a real reproducibility and security concern: a malicious or broken setup.py could do anything at install time, and different environments could produce different results running the "same" setup.py.

The modern way: declarative pyproject.toml

# pyproject.toml
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "myproject"
version = "1.0.0"
dependencies = ["requests>=2.0"]
requires-python = ">=3.9"

[project.optional-dependencies]
dev = ["pytest", "mypy", "ruff"]

[project.scripts]
mycli = "myproject.cli:main"

This is plain, static TOML data — no code execution needed to read project metadata, dependencies, or entry points. [build-system] (PEP 518) declares what's needed to build the project before even importing setuptools. [project] (PEP 621) standardizes metadata that used to be scattered across setup.py/setup.cfg/Pipfile in tool-specific formats.

Why this matters: one format, many tools

[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.0"

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.ruff]
line-length = 100

Beyond the standardized [project] table, tools can add their own [tool.*] sections in the same file. poetry, pytest, ruff, black, mypy all support configuration directly in pyproject.toml, consolidating what used to be setup.cfg, pytest.ini, .flake8, and various other tool-specific config files into one place.

The evolution, in short

1. distutils/setup.py (original, Python 2 era) — code-based, minimal metadata standardization.
2. setuptools + setup.cfg — moved some metadata to a declarative INI-style file, but setup.py was often still required as a shim.
3. pyproject.toml (current standard, PEP 518/621) — fully declarative project metadata and build-system requirements. setup.py is no longer required at all for most modern projects (though setuptools can still use one for complex custom build logic).

Building and publishing today

python -m build          # builds a wheel (.whl) and sdist (.tar.gz) from pyproject.toml
python -m twine upload dist/*   # publishes to PyPI

The build package is the modern, backend-agnostic way to build a distributable package purely from pyproject.toml, regardless of which build backend (setuptools, hatchling, poetry-core) the project uses.

pyproject.toml replaced the historical mix of executable setup.py and scattered config files with one declarative, standardized file for build requirements, project metadata, and dependencies. It's read by pip, build, and virtually every modern Python tool, eliminating the need to execute arbitrary code just to discover a package's metadata.