DuckDB Github stars over time

DuckDB — What’s the Hype About?

This was a blog post that I already planned to write during the spring when I saw that the hype around DuckDB started taking new heights. Since then the discussion around DuckDB has only intensified in the developer and data engineering community. I currently see two trends within the data community with high engagement levels: DuckDB and if Rust will take over data engineering. But what’s the hype around DuckDB really about? Let’s scratch the surface a little bit.

“Most workloads aren’t massive. Instead of requiring a scale-out database in the sky, most analyses are faster with an optimized database on your computer that can leverage the cloud when needed.”

Google Trends search data for “DuckDB”

The growing momentum

“I’ve heard a lot of good things about DuckDB lately and have found podcasts with both Jordan Tigani (founder of MotherDuck and BigQuery celebrity) and Hannes Mühleisen (creator founder of DuckDB Labs) really good. Hence I had to give it a try. My first program was to create a DuckDB table by reading directly from a BigQuery table using the BigQuery Storage Read API since it supports arrow tables (and no compute). Turned out to be really easy, sharing as a gist. Can’t wait to experiment some more with DuckDB and with bigger data volumes, it sure has huge potential.”

“Opinion: One of the most exciting new technologies for Data Engineering/ Data Science is DuckDB. DuckDB is insanely fast and with Apache Arrow, the duo is capable of delivering astonishing results. Another important point behind DuckDB is it’s simple. It doesn’t claim any groundbreaking stuff but sticks to the core of simple and faster data access.”

Image source Madrona

“Users want easy and fast answers to their questions — they don’t want to wait for the cloud… The fact is that a modern laptop is faster than a modern data warehouse. Cloud data vendors are focused on the performance of 100TB queries, which is not only irrelevant for the vast majority of users, but also distracts from vendors’ ability to deliver a great user experience.”

But what’s this hype all about? Let’s scratch the surface a little bit.

Database workload types (image source Kojo Osei)
Database deployment types (image source Kojo Osei)
Image source Kojo Osei

Final thoughts

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Oliver Molander

Co-founder at Validio and early-stage tech investor at J12 Ventures. Preaching about the realities & possibilities of Data & ML.