What if a laptop could hold a working lakehouse without a sprawling stack of services? DuckLake brings that idea into reach with DuckDB, Python, and Parquet files, creating a practical path from raw data to snapshots, schema changes, and time travel.
DuckLake is an open lakehouse table format from the DuckDB team. DuckLake 1.0 shipped in April 2026 with a stable specification and guaranteed backward compatibility, and this walkthrough builds a working lakehouse with Python and the duckdb package.
Why DuckLake Keeps the Architecture Simple
A DuckLake has two parts. The data lives in Parquet files in a folder or object store, while everything else sits in a catalog database: the table list, schemas, snapshots, and file statistics.
That split gives DuckLake a clear job for each layer. Parquet stores the data, and the catalog tracks the information needed to work with it. In DuckLake, finding the right files is one SQL query.
DuckLake can use any SQL database as its catalog, as long as it supports ACID transactions and primary key constraints. The documentation suggests DuckDB for a single client, SQLite for several local clients, and PostgreSQL for a multi-user lakehouse. For this laptop setup, the choice is direct: “We use DuckDB, the simplest option.”
Formats like Apache Iceberg and Delta Lake track much of this information in metadata files. DuckLake puts the metadata in SQL, creating a structure that fits naturally with DuckDB sessions and SQL queries.
There are limits to keep in mind. Like other lakehouse formats, DuckLake does not support indexes, primary keys, foreign keys, or UNIQUE and CHECK constraints. The format focuses on table data, catalog metadata, snapshots, and files rather than those database features.
Set Up the Laptop Project
DuckLake 1.0 is supported by DuckDB v1.5.2 or newer. You need a recent Python 3 installation and an internet connection the first time, because DuckDB downloads the ducklake extension on first use.
Install the Python package at the minimum version DuckLake 1.0 needs with pip install “duckdb>=1.5.2”. Then run python -c “import duckdb; print(duckdb.__version__)”. The second command prints the installed version, which should be 1.5.2 or higher.
Create an empty project folder and place six scripts inside it:
- step1_make_data.py
- common.py
- step2_load.py
- step3_change.py
- step4_time_travel.py
- step5_maintenance.py
Run the scripts in order from that folder. The sequence moves from sample data to loading, changes, time travel, and maintenance, so each step builds on the lake created before it.
Start with Repeatable Parquet Data
A lakehouse needs data to hold, so the first script uses DuckDB itself to write two months of fake orders as plain Parquet files. Each month contains 5,000 rows, and the source describes the output as two months of fake orders, 5,000 rows each, written as plain Parquet files.
range(5000) produces 5,000 numbers, with each number becoming a row. The order ID counts up from a starting value, the date cycles through 28 days, and the region and status cycle through fixed lists, making the data repeatable.
The amount is cast to DECIMAL(10, 2), a type suited to money. The last line reads both files back and prints the row count and date range, which should be 10,000 rows from January 1 to February 28.
Attach the Lake and Prepare for Change
Every later script starts the same way, so the shared setup goes in common.py. Its connect() function opens a DuckDB session with the DuckLake attached and selected.
The setup creates the folder lake/data with os.makedirs(“lake/data”, exist_ok=True), then runs INSTALL ducklake and LOAD ducklake. If lake/metadata.ducklake already exists, the code attaches ducklake:lake/metadata.ducklake as lake. Otherwise, it attaches the same catalog with DATA_PATH ‘lake/data/’.
The path in ATTACH ‘ducklake:…’ is the catalog database, while DATA_PATH identifies where the Parquet files go. The catalog folder must already exist, which is why the code creates it first. The data path is stored in the catalog, so later sessions skip DATA_PATH.
Then the code runs USE lake. That command makes plain table names point at the lake instead of DuckDB’s temporary in-memory database, avoiding the mistake the documentation warns about. Finally, connect() returns the connection.
The helper also includes mark() and markers_so_far(). The marker function saves the newest snapshot ID under a name in the small JSON file at lake/markers.json. It gets the ID with SELECT max(snapshot_id) FROM lake.snapshots(), loads existing markers, stores the new name, writes the result with json.dump(markers, f), and returns the snapshot ID.
markers_so_far() returns json.load(f) when the marker file exists and returns {} when it does not. This matters because snapshot numbers depend on exactly what ran before. Naming snapshots lets the time travel step avoid hard-coding IDs.
From there, the remaining scripts can load the Parquet data, change rows and schema, travel back through snapshots, and run maintenance. The result is a compact DuckLake lakehouse that shows how SQL catalog metadata and Parquet data work together on a laptop.
Based on
