A Python script starts as forty lines that solve one problem and ends up scheduled on a server, called by two other teams, and impossible to debug when it fails at 3am. The techniques below are the ones that make that transition survivable: proper argument handling, subprocess calls that cannot be hijacked, logging that goes somewhere useful, and concurrency chosen to match the kind of work. None of them require a framework, and most are standard library. They are worth applying the second time you open the file, not the tenth.
Every script that survives a month acquires flags. argparse is in the standard library and handles subcommands through add_subparsers(), which is enough for a tool with three or four verbs. Typer and Click are worth the dependency once you want shell completion, coloured help, or arguments derived from type hints — Typer reads your function signature and builds the parser from it, so count: int = 10 becomes a validated --count.
For configuration that comes from a file or the environment rather than flags, parse it into a dataclass or a Pydantic model at startup and fail loudly there. A script that dies on line 400 because a config key was a string instead of an int has wasted whatever it did in between. And use pathlib.Path throughout: p.with_suffix(".csv") and p.parent / "out" beat nested os.path.join calls, and Path objects are accepted everywhere strings are.
The rule is one list, no shell:
import shutil, subprocess, sys
if shutil.which("ffmpeg") is None:
sys.exit("ffmpeg not found on PATH")
r = subprocess.run(
["ffmpeg", "-i", str(src), "-c:v", "libx264", str(dst)],
check=True, capture_output=True, text=True, timeout=600,
)shell=True with any interpolated value is a command injection waiting for a filename containing a semicolon. check=True turns a non-zero exit into an exception instead of silently continuing. timeout stops a hung child holding the script forever. Check for the binary with shutil.which up front so the failure message names the missing tool.
print writes to stdout, which is where your actual output belongs. Send diagnostics to stderr through logging, wire -v to DEBUG, and pick the handler to match the destination: Rich's RichHandler for a human terminal, a JSON formatter when the output lands in a log aggregator. Then exit with a meaningful code — sys.exit(0) for success, non-zero for failure, and a distinct code for "nothing to do" if a caller needs to tell those apart. Guard the entry point with if __name__ == "__main__": so the module can also be imported by tests.
Waiting on network or disk is IO-bound, and threads release the GIL while they wait, so ThreadPoolExecutor(max_workers=16) over a list of URLs is close to free speed. Hashing, image resizing and parsing are CPU-bound, and threads buy nothing there — use ProcessPoolExecutor, remembering that arguments and results are pickled across the process boundary, so passing a 200MB DataFrame costs more than the work.
Past a few hundred concurrent requests, threads get expensive and asyncio with httpx.AsyncClient is the better shape — one client reused across all requests so connections are pooled, and an asyncio.Semaphore to cap how many are in flight so you do not get rate-limited. Do not mix: a blocking call inside a coroutine stalls the entire loop.
Iterating a file object yields lines lazily, and a generator function that filters and yields keeps peak memory flat regardless of input size. The same idea applies to database cursors and paginated APIs: yield rows, do not build a list. Where the same expensive pure function is called repeatedly with the same arguments, @functools.lru_cache(maxsize=None) is one line, though it holds every result forever, so cap the size on anything unbounded.
PEP 723 lets a single script declare what it needs in a comment block, and uv run script.py creates a throwaway environment, installs them and runs it — no virtualenv to activate, nothing to explain to whoever inherits the file.
# /// script
# requires-python = ">=3.11"
# dependencies = ["httpx", "rich"]
# ///Use tempfile.TemporaryDirectory() as a context manager rather than writing to /tmp/scratch and hoping. For work that must not be interrupted mid-write, install a signal.SIGTERM handler that sets a flag the main loop checks, and register final cleanup with atexit. When the script is slow, do not guess: python -m cProfile -s cumtime script.py ranks by cumulative time, and py-spy top --pid 1234 samples a process that is already running in production without restarting or instrumenting it.