DO02 · DataOps & MLOps

Spark Partitioning & Cost

Choose partition columns, fix data skew and compare job time and cost.

📶 L400⏱ 90 min📘 Guided

🎯 You will learn to

Choose partition keys
Fix skew
Estimate cost

📘 Guided lab

This mission is a guided walkthrough: follow the steps on your own computer or practice account. The in-browser simulator for it is planned for a later release.

✅ Steps & checkpoints

Partition by date
Find the skewed key
Salt it
Compare runtime

🗺️ Partitioning and skew

Good partition columns let queries skip files; salting a hot key spreads the work evenly.

The 2D diagram shows the exact flow, ports and trust boundaries. The 3D scene is a concept view of the same parts; packets travel along the same arrows.

🤔 Think about it

Why can partitioning by a high-cardinality column hurt?

Your answer (kept only on this device)

🛡️ Safety & cost

Synthetic data only. No personal data is used.

🧹 Cleanup

Nothing to clean up.

📚 Reference documents

Official sources, checked on 5 Nov 2026. Cloud services change often; check the official page before relying on a feature. All references