Data types and control flow

Learning objectives

Model records with the right Python types, write explicit control flow, and reject invalid data early.

Core model

Python data pipelines commonly move between dict, list, tuple, set, and typed scalar values. Choose types by semantics: tuples for fixed records, sets for membership, and dictionaries for named fields.

order = {"order_id": 1001, "quantity": 2, "unit_price": 39.90, "status": "paid"}
allowed = {"paid", "refunded", "cancelled"}

if order["status"] not in allowed:
    raise ValueError(f"invalid status: {order['status']}")
gross_amount = order["quantity"] * order["unit_price"]

Prefer guard clauses over deeply nested branches. Never silently coerce a value when invalid input should stop the pipeline.

Lab

Download the orders dataset. Write validate_order(record) that rejects non-positive quantities for paid orders, missing IDs, and unknown statuses.

Checkpoint

Why is a set a better fit than a list for allowed? What should happen to a cancelled order with quantity zero?

Reference solution

def validate_order(row):
    required = {"order_id", "quantity", "status"}
    if missing := required - row.keys():
        raise ValueError(f"missing fields: {sorted(missing)}")
    if row["status"] not in {"paid", "refunded", "cancelled"}:
        raise ValueError("unknown status")
    if row["status"] == "paid" and int(row["quantity"]) <= 0:
        raise ValueError("paid quantity must be positive")
    return row

Why this matters

Reliable data products depend on engineers making data types and control flow explicit, testable, and repeatable. The techniques in this lesson reduce silent corruption and make pipeline behavior reviewable by another engineer.

Mental model

Treat the pipeline as a sequence of contract boundaries: input is observed, validated, transformed, and emitted with evidence. Each boundary should make assumptions and failure behavior visible.

Independent challenge

Create a fresh synthetic fixture with a valid row, a boundary row, and two distinct failures. Implement the lesson's contract without copying the reference solution, then produce a deterministic test or report proving the behavior.

Common mistakes

• Allowing implicit coercion to hide invalid source values.
• Mutating raw input and losing the evidence needed for diagnosis or replay.
• Testing only the happy path or asserting implementation details instead of outcomes.
• Producing output without run identity, rejection reasons, or deterministic retry behavior.

Key takeaways

• Make data contracts and failure paths explicit at every pipeline boundary.
• Preserve raw evidence and use small synthetic fixtures for repeatable tests.
• Prefer deterministic, idempotent transformations whose outcomes can be audited.

Next step

Continue with Functions, modules, and environments, carrying forward the tested contract and fixtures from this lesson.