What's the role of `pyproject.toml`, and how has Python packaging evolved?

The old way: setup.py as executable code

# setup.py (legacy)
from setuptools import setup

setup(
    name="myproject",
    version="1.0.0",
    install_requires=["requests>=2.0"],
)

Because setup.py is a Python script, building or installing a package required actually executing arbitrary code just to read its metadata — a real reproducibility and security concern (a malicious or broken setup.py could do anything at install time, and different environments could produce different results running the "same" setup.py).

The modern way: declarative pyproject.toml

# pyproject.toml
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "myproject"
version = "1.0.0"
dependencies = ["requests>=2.0"]
requires-python = ">=3.9"

[project.optional-dependencies]
dev = ["pytest", "mypy", "ruff"]

[project.scripts]
mycli = "myproject.cli:main"

This is plain, static TOML data — no code execution needed to read project metadata, dependencies, or entry points. [build-system] (PEP 518) declares what's needed to build the project before even importing setuptools; [project] (PEP 621) standardizes metadata that used to be scattered across setup.py/setup.cfg/Pipfile in tool-specific formats.

Why this matters: one format, many tools

[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.0"

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.ruff]
line-length = 100

Beyond the standardized [project] table, tools can add their own [tool.*] sections in the same file — poetry, pytest, ruff, black, mypy all support configuration directly in pyproject.toml, consolidating what used to be setup.cfg, pytest.ini, .flake8, and various other tool-specific config files into one place.

The evolution, in short

1. distutils/setup.py (original, Python 2 era) — code-based, minimal metadata standardization.
2. setuptools + setup.cfg — moved some metadata to a declarative INI-style file, but setup.py was often still required as a shim.
3. pyproject.toml (current standard, PEP 518/621) — fully declarative project metadata and build-system requirements; setup.py is no longer required at all for most modern projects (though setuptools can still use one for complex custom build logic).

Building and publishing today

python -m build          # builds a wheel (.whl) and sdist (.tar.gz) from pyproject.toml
python -m twine upload dist/*   # publishes to PyPI

The build package is the modern, backend-agnostic way to build a distributable package purely from pyproject.toml, regardless of which build backend (setuptools, hatchling, poetry-core) the project uses.

Interview-ready summary: pyproject.toml replaced the historical mix of executable setup.py and scattered config files with one declarative, standardized file for build requirements, project metadata, and dependencies — read by pip, build, and virtually every modern Python tool, eliminating the need to execute arbitrary code just to discover a package's metadata.