282 points · baotiao · 1 day ago
github.comruhith
infogulch
1. integrate an off the shelf OLAP engine
forward OLAP queries to it
deal with continued issues keeping the two datasets in sync
2. rebase OLTP and OLAP engines to use a unified storage layer
storage layer supports both page-aligned row-oriented files and column-oriented files and remote files
still have data and semantic inconsistencies due to running two engines
3. merge the engines
policy to automatically archive old records to a compressed column-oriented file format
option to move archived record files to remote object storage, fetch on demand
queries seamlessly integrate data from freshly updated records and archived records
only noticeable difference is queries for very old records seem to take a few seconds longer to get the results backlinuxhansl
polskibus
BTW, Would be great to hear apavlo’s opinion on this.
jimmyl02
The most interesting part of this is the improvements to transaction handling that it seems they've made in https://github.com/alibaba/AliSQL/blob/master/wiki/duckdb/du... (its also a good high level breakdown of MySQL internals too). Ensuring that the sync between the primary tables and the analytical ones are fast and most importantly, transactional, is awesome to see.
dzonga
at the moment I use PG + Tiger Data - couldn't find a mysql equivalent
so this as one.
anentropic
Keyframe
enamya
redwood
manishsharan
And I get the benefit of resiliency and DR for free.
If you are a developing for My SQL and you are using Java/kotlin/closure/scala consider this as well.
cies
Let's all hope Ali will pick it up :)
I'm fully invested on Postgres though.
aussieguy1234