MiniLake

A morden mini lakehouse based on Spark and Iceberg running in the docker.

Usage

Build and run

docker compose up --build

Attach the spark container

docker exec -it spark-iceberg /opt/spark/bin/spark-sql

Create table

CREATE TABLE student (id INT, name STRING, age INT) USING ICEBERG LOCATION 's3://minilake/student';

Insert data

INSERT INTO student VALUES (1, 'jay', 15), (2, 'dove', 15);

Execute query

SELECT * FROM student;

TODO

1. A standalone catalog server.
2. Ingesting real-time data from Kafka.
3. CDC.