Databricks Interview Questions

Q1. What is a Lakehouse and how is it different from a data warehouse and a data lake?

A data warehouse gives you ACID transactions and fast SQL but is expensive and awkward for raw or semi-structured data. A data lake gives you cheap, scalable object storage but no transactions, no schema guarantees and no reliable updates. A Lakehouse puts warehouse-grade reliability — ACID, schema enforcement, performance — directly on top of cheap lake storage, using Delta Lake as that reliability layer. The practical win is one copy of the data serving both BI and ML, instead of maintaining a lake and a separate warehouse and copying between them.

Q2. How does Delta Lake provide ACID transactions on object storage that isn't transactional?

Through the transaction log — the _delta_log folder next to the Parquet files. Every write creates a new, ordered JSON commit that lists which files were added and removed. Readers reconstruct the current table state from the log, so they only ever see fully committed versions; a failed write simply never gets a commit. That ordered log is also what enables time travel, because any previous version is just an earlier point in the log.

Q3. Walk me through the medallion architecture and why you'd use it.

Bronze holds raw ingested data, append-only, so I keep full history and lineage exactly as it arrived. Silver is cleaned and conformed — de-duplicated, type-cast, joined, with quality rules applied. Gold is business-level aggregates and star schemas that feed dashboards and ML features. Each layer is a Delta table and each transformation is idempotent, so the biggest benefit is reprocessing: if business logic changes, I can rebuild Silver and Gold from Bronze without going back to the source systems.

Q4. Your Databricks job cost doubled this month with the same data volume. Where do you look?

First, cluster type — is scheduled work running on an always-on all-purpose cluster instead of a job cluster that terminates after the run. Second, cluster sizing and auto-terminate — an idle cluster left running overnight is pure waste. Third, the small-files problem: thousands of tiny files inflate read time and shuffle, so I'd check whether OPTIMIZE is running. Fourth, a skewed join or an accidental full recompute from a missing checkpoint. I'd confirm in the Spark UI which stage dominates before changing anything.

Q5. When do you partition a Delta table versus using ZORDER?

Partitioning physically splits data into folders and works best on low-cardinality columns you filter on constantly, like date or region — over-partitioning on something high-cardinality creates the small-files problem. ZORDER co-locates related data within files on one or more high-cardinality columns, so reads skip files via data-skipping statistics without exploding the folder count. In practice I partition by date and ZORDER by a frequently filtered id, then run OPTIMIZE to apply it.

Q6. What does Unity Catalog give you that workspace-level access control doesn't?

Unity Catalog is a single governance layer that spans workspaces, with a three-level namespace — catalog.schema.table — instead of governance being trapped per workspace. It adds centralised fine-grained access control, including column- and row-level security, plus automatic data lineage and audit logging. So instead of each team's workspace being its own island of permissions, you get one consistent, auditable model across the whole org.

Yes — Databricks runs on Spark, so Spark fundamentals (shuffles, narrow vs wide transformations, data skew, joins) come up alongside Delta Lake. Pair this guide with the PySpark interview guide.

No. Data analysts use Databricks SQL and dashboards, ML engineers use it for feature engineering and MLflow, and BI developers pull Gold-layer tables into Power BI or Tableau. The Lakehouse basics are worth knowing for any data role.

SQL warehouses are optimised for BI/SQL analytics workloads; all-purpose clusters are for interactive notebook development; job clusters are for scheduled production runs and are the cheapest for that. Knowing when to use each is a strong signal.