Quality Begins with a Consumer Decision

Why this matters

At 08:10 on Monday, Northstar Commerce's customer-support lead sees 312 orders labelled “delivery problem.” The number looks alarming, but it does not yet tell the team what to do. Some orders were placed only an hour ago. Some have a carrier scan that arrived after the report was generated. Some were never paid and should not have entered fulfillment at all.

If an analyst begins by asking whether the carrier_scan_at column is complete, the answer may be technically correct and operationally useless. The support lead is not trying to perfect a column. She is deciding which customers need proactive contact before a delayed delivery becomes a complaint.

That decision changes the quality question. A missing carrier scan is material for a paid order that has waited 48 hours. The same absence is expected for an unpaid order or an order created five minutes ago. One field condition can therefore be harmful, harmless, or premature depending on the record and its use.

This lesson establishes the course's governing idea: quality begins with a declared consumer decision. Dimensions, metrics, rules, dashboards, and tools become useful after that decision is understood. Starting with a tool usually creates large collections of checks that nobody can prioritize when they fail.

Learning objectives

By the end of the lesson, you will be able to:

• distinguish a data observation from harm to a consumer decision;
• describe a consumer, decision, data boundary, and failure mode precisely;
• build a decision chain from business use to measurable evidence;
• reject quality statements that lack context or an accountable response;
• implement a small decision-specific control without pretending it measures all aspects of quality; and
• document what the control cannot establish.

Mental model

Quality is a relationship, not a label

A dataset is not simply “high quality” or “low quality.” It is fit—or unfit—for a particular use under stated conditions.

Consider one daily orders table used by three teams:

The table has one physical form, but three quality contracts. A defect that blocks finance may have little effect on marketing. A delay tolerated in a weekly campaign analysis may be unacceptable in a support queue refreshed every 15 minutes.

This does not mean quality is arbitrary. Context makes a claim testable. Once a decision and boundary are explicit, the team can collect evidence and judge whether the dataset meets the declared need.

The decision chain

Use the following sequence before proposing a quality rule:

consumer
   ↓
decision or action
   ↓
data boundary and grain
   ↓
failure mode
   ↓
decision consequence
   ↓
observable evidence
   ↓
acceptance condition
   ↓
owner and response

Each link prevents a common mistake.

1. Consumer: Name a role or system, not “the business.” Different consumers can have conflicting needs.
2. Decision or action: State what changes because of the data. “Uses the dashboard” describes an interface, not a decision.
3. Data boundary and grain: Identify the dataset version, time window, and what one record represents. A uniqueness rule is meaningless until the expected grain is known.
4. Failure mode: Describe how the evidence could mislead the decision. Avoid using a dimension name as the whole explanation.
5. Decision consequence: State the harm: missed contact, incorrect payment, compliance exposure, wasted work, or delayed response.
6. Observable evidence: Define what can actually be measured at the decision boundary.
7. Acceptance condition: Specify the rule or threshold and the evaluation window. A threshold is a policy choice, not a universal constant.
8. Owner and response: Identify who evaluates, contains, communicates, corrects, or accepts the risk.

The chain does not always produce a row-level rejection. It may produce a warning, a delayed publication, a reconciliation task, a consumer notice, or an approved exception.

Observation, defect, and incident are different states

Suppose 4% of Monday's orders have no carrier scan.

• The observation is measurable: carrier_scan_at is absent for 4% of records in the evaluated boundary.
• A defect exists when a record violates its declared contract. An unpaid order may not require a scan, so absence alone is not enough.
• An incident exists when a defect or collection of defects materially threatens or harms a consumer outcome and requires coordinated response.

Collapsing these states creates alert fatigue. Treating every observation as an incident trains operators to ignore alarms. Treating every observation as harmless conceals risk. The decision chain supplies the missing context.

Data quality does not prove truth by itself

A record can satisfy every implemented rule and still be wrong. A plausible amount may refer to the wrong order. A valid country code may not be the customer's actual destination. Automated controls establish conformance to observable conditions; they do not create omniscience.

For that reason, a credible control declares both its evidence and its limit. The control in this lesson identifies orders that satisfy a support-priority policy. It does not prove that the carrier failed, predict a delivery date, or establish that every underlying timestamp is accurate.

Worked example

Northstar's support policy says:

Prioritize a paid order when at least 48 complete hours have elapsed since creation and no carrier scan has been observed by the evaluation time.

Before writing code, translate the policy through the decision chain.

Now implement only the declared decision. The code uses aware UTC timestamps and keeps the rule separate from orchestration.

from datetime import datetime, timezone
from typing import TypedDict


class Order(TypedDict):
    order_id: str
    created_at: datetime
    paid: bool
    carrier_scan_at: datetime | None


def needs_proactive_contact(order: Order, *, evaluated_at: datetime) -> bool:
    """Return whether an order meets the declared support-contact policy."""
    if evaluated_at.tzinfo is None or evaluated_at.utcoffset() is None:
        raise ValueError("evaluated_at must include a timezone")
    if order["created_at"].tzinfo is None or order["created_at"].utcoffset() is None:
        raise ValueError("created_at must include a timezone")
    age_hours = (evaluated_at - order["created_at"]).total_seconds() / 3600
    return order["paid"] and age_hours >= 48 and order["carrier_scan_at"] is None


evaluated_at = datetime(2026, 7, 20, 8, tzinfo=timezone.utc)
orders: list[Order] = [
    {
        "order_id": "O-101",
        "created_at": datetime(2026, 7, 20, 7, tzinfo=timezone.utc),
        "paid": True,
        "carrier_scan_at": None,
    },
    {
        "order_id": "O-102",
        "created_at": datetime(2026, 7, 17, 8, tzinfo=timezone.utc),
        "paid": True,
        "carrier_scan_at": datetime(2026, 7, 18, 10, tzinfo=timezone.utc),
    },
    {
        "order_id": "O-103",
        "created_at": datetime(2026, 7, 18, 8, tzinfo=timezone.utc),
        "paid": True,
        "carrier_scan_at": None,
    },
    {
        "order_id": "O-104",
        "created_at": datetime(2026, 7, 17, 7, tzinfo=timezone.utc),
        "paid": True,
        "carrier_scan_at": None,
    },
    {
        "order_id": "O-105",
        "created_at": datetime(2026, 7, 17, 7, tzinfo=timezone.utc),
        "paid": False,
        "carrier_scan_at": None,
    },
]

priority_ids = [
    order["order_id"]
    for order in orders
    if needs_proactive_contact(order, evaluated_at=evaluated_at)
]

assert priority_ids == ["O-103", "O-104"]

Trace the boundary records:

• O-101 has no scan, but it is only one hour old.
• O-102 is old and paid, but a scan exists.
• O-103 is exactly 48 hours old. The >= 48 policy includes it.
• O-104 satisfies all three conditions.
• O-105 is old and lacks a scan, but it is unpaid.

The assertion is not decoration. It records the expected result derived from the policy. A reviewer can dispute the policy or the implementation separately.

What this control does not establish

It does not establish that:

• created_at reflects the true order time;
• paid agrees with the payment processor;
• a carrier scan was not lost before ingestion;
• the customer has not already contacted support;
• the 48-hour threshold is economically optimal; or
• orders omitted from the queue are generally “high quality.”

Those questions require other evidence and owners. Naming the limits keeps one useful rule from becoming an unjustified quality score.

Guided practice

Northstar's finance analyst uses the same orders dataset to decide which captured payments enter a daily revenue reconciliation. Complete the decision chain before proposing checks.

Use this brief:

• The reconciliation covers captures observed from 00:00:00Z through, but not including, the next 00:00:00Z.
• One output record represents one payment capture, not one order.
• A capture needs a stable capture ID, order ID, exact amount, currency, capture time, and reversal state.
• Finance compares the output total and record set with the payment processor's control report.
• An unexplained difference stops ledger publication for that accounting day.

Fill the table.

Then write three candidate controls. At least one must compare two sources rather than inspect a single field. For each control, label it:

• preventive when it blocks an invalid state from being accepted at a controlled boundary;
• detective when it discovers a condition after or while records are produced; or
• reconciliatory when it compares independently produced evidence.

Do not choose thresholds yet. First state the exact failure each control can detect and one important failure it cannot detect.

Independent challenge

Choose one of these synthetic consumers:

1. a warehouse team releasing pick lists;
2. a fraud-review service selecting payments for manual review; or
3. a merchandising analyst measuring stock-out duration.

Create a one-page decision brief containing:

• a named consumer role or system;
• one consequential decision;
• dataset boundary, time window, and record grain;
• two distinct failure modes;
• the consequence of each failure;
• observable evidence for each failure;
• a proposed acceptance condition;
• an accountable owner and response; and
• one explicit limitation for every proposed control.

Add three synthetic records: one ordinary case, one exact boundary, and one deceptive case that appears valid under a weak field-only rule. Explain the expected decision for each record before writing any code.

Your work is incomplete if it only lists dimensions such as completeness or validity. The reviewer must be able to determine what action changes when a control fails.

Knowledge check

1. A table has no null order_id values. What can you conclude? - A. Every order identifier is accurate. - B. Every expected order is present. - C. Every observed row has some value in order_id; further identity and completeness evidence is still required. - D. The table is fit for every consumer.

2. Why is “the dashboard must be accurate” not yet a usable quality requirement? - A. Dashboards cannot be tested. - B. It does not identify the decision, boundary, observable condition, acceptance rule, or response. - C. Accuracy is never a data-quality concern. - D. A dashboard requires only freshness checks.

3. A campaign report arrives six hours later than usual but before its documented weekly decision deadline. Which statement is best supported? - A. Any delay is automatically a quality incident. - B. The report is necessarily accurate. - C. The delay is an observation; its materiality depends on the declared service objective and consumer decision. - D. Freshness never matters for weekly reports.

4. In the worked example, why is O-103 included? - A. Its age is exactly on an inclusive 48-hour boundary and the other conditions hold. - B. Every order without a scan is included. - C. It is the third record. - D. Its identifier sorts before O-104.

5. What is the strongest reason to record what a control cannot establish? - A. It makes the lesson longer. - B. It prevents local conformance evidence from being misrepresented as proof of broader truth or fitness. - C. It eliminates the need for owners. - D. It allows every failed check to be ignored.

Common mistakes

Beginning with a universal list of rules

“Check every column for nulls” creates activity but not priority. Some nulls are expected, some fields are irrelevant to the decision, and some catastrophic errors contain no nulls at all. Begin with failure consequences and work backward to evidence.

Naming “the business” as consumer

An abstract organization cannot clarify a rule or respond to a breach. Name the role, service, report, or workflow whose action depends on the data.

Confusing a dashboard view with a decision

“The manager views weekly revenue” does not say what the manager decides. Budget allocation, cash forecasting, and sales compensation can require different boundaries and tolerances even when they use the same chart.

Treating dimensions as independent scores

A completeness percentage can conceal missing high-value records. A validity percentage can pass while values belong to the wrong entity. Dimensions help classify questions; they do not remove the need to understand grain, distribution, and consequence.

Setting zero-defect targets without a response design

Some identifiers genuinely require zero duplicates at a trusted publication boundary. Other controls need warnings, sampling, quarantine, or a tolerated budget. “Zero defects” is credible only when the condition, measurement, cost, and failure response support it.

Hiding uncertainty

If a control detects conformance rather than truth, say so. If a baseline is not established, do not invent a threshold. If an owner has not accepted the response, the control is not operationally complete.

Key takeaways

• Data quality expresses fitness for a declared use, not a universal property of a table.
• A consumer decision gives quality work a boundary, priority, and consequence.
• The decision chain connects consumer, action, grain, failure, harm, evidence, acceptance, owner, and response.
• An observation is not automatically a defect, and a defect is not automatically an incident.
• Automated rules prove only the conditions they observe; credible controls state their limits.
• A rule without an accountable failure response is an unfinished control.

Next step

The next lesson, Define the Dataset Boundary and Grain, makes two parts of the decision chain precise. You will learn why row counts, uniqueness, completeness, and reconciliation cannot be interpreted until the evaluated population and record meaning are explicit.

Before continuing, save your independent decision brief. It becomes the consumer-risk input for the rest of Quality Foundations.