Choose record serialization, table semantics and physical layout from the workload. The modelling chapter explains row meaning and relationships; platform design explains catalogs, compute and analytical serving.
In this chapter
- Parquet, Avro, ORC and row-oriented file formatsChoose serialization, columnar scans, schema handling and compression.
- Delta Lake, Iceberg and Hudi: selection and operationsCommit semantics, compatibility, update costs and retention.
- Lake partitioning, clustering and Parquet file sizingEstimate bytes per partition and test file layout against reads and updates.
- Choosing storage from data shape, access pattern and scaleMatch transactions, telemetry, documents, analytics and search to a store.