Six months of using Python for scheduled jobs, scraping and glue code produces a clear split: a few tools that removed work permanently, and several that added maintenance nobody accounted for.
uv handles virtual environments, dependency resolution and installs, and it is fast enough that the difference is behavioural — you stop avoiding a clean rebuild. uv venv && uv pip install -r requirements.txt on a moderate dependency set completes in a couple of seconds rather than most of a minute, and uv pip compile requirements.in -o requirements.txt gives you a pinned lockfile with hashes. The one caveat: it is a resolver and installer, not a build backend, so packages with unusual build steps still need attention.
Requests remains the most predictable HTTP client in any language. The habit worth forming is never calling it bare — always a Session with a mounted retry adapter, always an explicit timeout, because the default is no timeout and a hung connection will silently stall a cron job forever. httpx is the upgrade when you need async or HTTP/2; the API is close enough that porting is mechanical. Six months in, the single biggest source of stuck jobs was a missing timeout, not a library bug.
For anything that needs a browser, Playwright's auto-waiting removes the entire category of flaky sleep-based tests. page.get_by_role("button", name="Submit").click() waits for the element to be actionable rather than for a fixed duration. playwright install chromium pulls the browser, page.wait_for_load_state("networkidle") handles most single-page apps, and headless mode works in a container. The costs are honest: several hundred megabytes of browser per image, and roughly 200-400 MB of memory per browser context, so parallelism is bounded by RAM. Use it only when the target genuinely requires JavaScript — an HTTP client plus selectolax is an order of magnitude cheaper when the HTML is server-rendered.
Type hints become the argument parser. A function with def sync(source: Path, dry_run: bool = False) gets validation, --help, and shell completion for free. The practical effect over six months was that scripts became reusable: a script with a real --dry-run flag and validated paths gets run by other people, and a script with positional string arguments does not.
APScheduler works and has a genuine trap — the default in-memory job store loses everything on restart, so a job that should run daily silently stops after a deploy. Use a persistent job store, or accept the simpler answer: system cron, or a systemd timer, with the script writing a heartbeat somewhere you monitor. Over six months, the two worst incidents were both a scheduled job that had stopped running and nobody noticed for days. A dead man's switch — an alert that fires when the heartbeat is absent — would have caught both.
For anything under a few thousand rows, the csv module and a dict are faster to write, faster to run and much easier to debug. pandas earns its place on real analysis and on files large enough that manual iteration is painful, but importing it into a small automation script adds seconds of import time and a large dependency. Polars is worth trying for larger workloads — the lazy API and explicit schemas catch mistakes pandas silently coerces.
logging configured once with timestamps, replacing every print. Jobs you cannot see are jobs you cannot fix.tenacity for retries with exponential backoff, applied only to operations that are genuinely idempotent.python -m package.module to avoid import path surprises.Free tools, guides, and resources across the SPUNK13 network.
Visit spunk.bet400+ Free Tools