Question
Bring a transformation, join, partitioning, or execution-plan question. Include the shape of your data and the behaviour you observed so the community can reason about the trade-offs with you.
1 answer
✓ Accepted answer
Begin with the execution plan rather than adding more resources. Check whether the workload introduces wide shuffles, skewed keys, unnecessary repartitioning, or repeated recomputation. Compare input size, partition count, and task duration. Cache only reused expensive results, select columns early, and validate each improvement with the same representative workload.