Warp Solutions: DuckDB/Kwack <-> WarpStream

Save
1 year ago
8
Technology Education

Series: Warp Solutions
Subject: Use DuckDB extension Kwack to read directly from WarpStream

Shawn uses the new DuckDB extension, Kwack, to connect to two Apache Kafka topics in WarpStream, read them, and join them. After that, he illustrates how to export the topics to a Parquet file or DuckDB database.

WarpStream - www.warpstream.com
DuckDB - duckdb.org
Kwack - github.com/rayokota/kwack (requires Java 11 or later)

ShadowTraffic was used for the creation of the sample data, available at shadowtraffic.io
Harlequin was used as a terminal-based SQL IDE, available at https://harlequin.sh/

#apachekafka #warpstream #streambased #duckdb #datastreaming #dataengineering

Loading comments...