pip install uv
# or: brew install uv

uv venv
source .venv/bin/activate
uv pip install playwright

The activation step matters. After it, python and pip resolve to the binaries inside .venv/, not the ones on your system.

If you cannot install uv, the stdlib has everything you need. Both the venv module and pip ship with Python itself since 3.3 and 3.4 respectively – no extra tool required.

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install playwright

The only difference is speed: uv resolves and installs dependencies in parallel and caches aggressively; pip is sequential and cold-caches often. Pick one and stick with it for the rest of the session.

F-strings are the modern way to build strings. A leading f tells Python to evaluate expressions inside {} braces and insert the result.

name = "Ada"
age = 36
print(f"{name} is {age} years old.")
print(f"Next birthday: {age + 1}.")
print(f"{name=}, {age=}")

The last form – adding = inside the braces – prints both the expression and its value. name='Ada', age=36. It exists for one reason only: throwaway debug prints that are still readable three weeks later.

Most operations look the same on lists, tuples, dicts, and sets. in tests membership, len() gives size, iteration yields elements (for dicts: keys). The differences are in mutation and shape.

urls = ["https://a.com", "https://b.com"]
urls.append("https://c.com")
"https://a.com" in urls        # True

seen = {"https://a.com"}
seen.add("https://b.com")

page = {"url": "https://a.com", "status": 200}
page["title"] = "A"   # dicts grow by assignment

Python uses indentation instead of braces. Four spaces per level is the convention, enforced by every editor and lint tool. No end, no }, no semicolons.

status = 404

if status == 200:
    print("ok")
elif 300 <= status < 400:
    print("redirect")
else:
    print("problem")

for url in urls:
    if "localhost" in url:
        continue
    print(url)

The continue and break keywords do what you expect. elif is the Python spelling of else if – no separate keyword.

def greet(
    name: str,
    loud: bool = False,
) -> str:
    end = "!" if loud else "."
    return f"Hello, {name}{end}"

greet("Ada")
# Hello, Ada.
greet("Ada", loud=True)
# Hello, Ada!
greet(name="Ada")
# same call, keyword form

Type hints are documentation that compiles. A linter like ruff or a type checker like mypy reads them and flags mismatches before the code runs. Your future self will thank you.

Comprehensions build a list, dict, or set from an existing iterable in a single expression. They read like “{element} for each item in source, optionally filtered”.

urls = ["https://a.com/", "https://b.com", "mailto:x@y"]

https_only = [u for u in urls if u.startswith("https://")]
lengths = {u: len(u) for u in https_only}
domains = {u.split("/")[2] for u in https_only}

Prefer a comprehension over a for-loop with .append(). The comprehension form is more compact, slightly faster, and signals intent: “I am building a collection”, not “I am performing side effects”.

Exceptions are Python's error channel. When something goes wrong, a function raises an exception; a caller further up the stack catches it with try/except and decides what to do.

try:
    value = int(user_input)
except ValueError as exc:
    print(f"Not a number: {exc}")
    value = 0

The as exc clause binds the exception object to a name so you can inspect it. Drop it when you only care that something failed, not what failed: except ValueError:.

pathlib replaces string path manipulation with path objects. The / operator joins segments; methods like .read_text(), .mkdir(parents=True), and .glob() do what their names suggest.

from pathlib import Path

here = Path(__file__).parent
report = here / "out" / "report.txt"
report.parent.mkdir(parents=True, exist_ok=True)
report.write_text("hello\n")

for md in here.glob("**/*.md"):
    print(md.relative_to(here))

Cross-platform correctness comes for free. Path normalizes slashes and drive letters so the same code runs on Linux, macOS, and Windows without os.path.join gymnastics.

Parsing URLs with a regex is almost always a mistake. urllib.parse already knows about schemes, userinfo, punycode hosts, port defaults, and path normalization.

from urllib.parse import urlparse

u = urlparse("https://example.com/a?x=1")
u.scheme   # "https"
u.netloc   # "example.com"
u.path     # "/a"

from urllib.parse import urljoin

urljoin("https://ex.com/a/", "b/c")  # .../a/b/c
urljoin("https://ex.com/a/", "/d")   # .../d

The scanner uses both: urljoin to turn relative links into absolute URLs, and urlparse to check that a link stays on the same host before we visit it.

Reach for re when pattern matching is the right tool, not before. If you only need “starts with” or “contains”, str.startswith, str.endswith, and in are faster to write and faster to read.

import re

pattern = re.compile(r"^https?://")
pattern.match("https://example.com")   # Match object
pattern.match("mailto:x@y")            # None

Compile once, match many. re.compile caches the compiled pattern; calling .match() on the compiled object skips the compile step on every call.

argparse turns a list of argument descriptions into a full CLI. Help text, type coercion, default values, error messages – all generated from add_argument calls.

import argparse

p = argparse.ArgumentParser(description="Scan a page for link health.")
p.add_argument("url", help="URL to start from")
p.add_argument("--max", type=int, default=20, help="max links")
args = p.parse_args()

print(args.url, args.max)

python scanner.py --help already works. Three lines of setup, and the user gets a standards-conforming CLI with Unix-style flags and a readable usage block.

async def defines a coroutine; await suspends it until the awaited operation completes. asyncio.gather starts several coroutines in parallel and waits for all of them.

import asyncio

async def fetch(name: str, delay: float) -> str:
    await asyncio.sleep(delay)  # a network call, pretend
    return f"done: {name}"

async def main() -> None:
    results = await asyncio.gather(
        fetch("a", 1.0),
        fetch("b", 1.0),
        fetch("c", 1.0),
    )
    print(results)

asyncio.run(main())

Three one-second sleeps, total runtime: about one second. That is the whole point of async I/O.

After pip install playwright you still need the browser itself. Playwright ships a small CLI to download a pinned Chromium build into its cache.

playwright install chromium

Pinned means reproducible. The next developer on the project runs the same command and gets the same Chromium version, not whatever ships with today's operating system.

The pattern is always the same: open a context, launch a browser, navigate, query, close. async with guarantees cleanup even if the page raises.

import asyncio
from playwright.async_api import async_playwright

async def main() -> None:
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        await page.goto("https://example.com")
        print(await page.title())
        await browser.close()

asyncio.run(main())

await on every browser call. Each one is a round-trip over a socket to the browser process, so every call is an I/O wait – exactly what async is for.

page.evaluate runs JavaScript inside the page and returns the result as plain Python data. Anything JSON-serializable crosses the boundary.

hrefs = await page.evaluate("""
  () => Array.from(document.querySelectorAll('a[href]'))
             .map(a => a.href)
""")

hrefs comes back as a Python list of strings, already absolute. The browser resolves relative <a href> against the current page URL before handing them over.

python scanner.py https://example.com --max 10

The output is deliberately grep-friendly. One line per page, flags first, URL last. Pipe it into grep -v '^ok' to see only the pages that have a problem.

ok                                       https://example.com/
no-description                           https://example.com/about
status=404 no-title                      https://example.com/oops