#336 — January 8, 2021
Database Weekly
Using Checksums to Verify Syncing 100M Database Records — A Shopify engineer thinks through the challenge of rapidly checking that two datastores are in sync using SQL.
Simon Hørup Eskildsen
Presto Poised for a Breakout Year as Data Explosion Continues — Presto is a federated SQL query engine developed at Facebook as a follow-on to Apache Hive and its popularity is growing. Facebook uses it for running interactive queries against several internal data stores, including their 300PB data warehouse.
Alex Woodie
Try DataStax Astra DBaaS | Get Started with 5 GB Free — Start building cloud-native apps fast with Astra, the open-source, multi-cloud stack for modern data apps. With Astra you can build faster with REST, GraphQL, CQL, and JSON/Document APIs. Try it today.
DataStax Astra sponsor
Why CockroachDB and PostgreSQL Are Compatible — Postgres and its wire protocol have continued to increase their status as building blocks of other databases and CockroachDB deliberately implemented Postgres compatibility as a way ‘to be accessible to an ecosystem of drivers, existing code, and developer knowledge’. We’ll see more of this in 2021, I’m sure.
Raphael 'kena' Poss
TimescaleDB 2.0 Now GA: A Petabyte-Scale Relational Database for Time-Series — We announced TimescaleDB 2.0 back in October but it’s now officially GA. What does 2.0 get us? Distributed hypertables, user-defined actions, and a swathe of enterprise features added into the community edition.
Ajay Kulkarni and Mike Freedman (Timescale)
PostgreSQL Is DB-Engines' 'DBMS of the Year 2020' — DB-Engines is a popular DBMS knowledge base and each year they have a ‘DBMS of the Year’ based on popularity growth. MySQL had the honor in 2019.
Paul Andlinger and Matthias Gelbmann
Ingesting MongoDB Atlas Data using Amazon EventBridge — EventBridge is an AWS service that provides an event bus for tying together various SaaS apps and other AWS services.
James Beswick (AWS)
Using Amazon DocumentDB's Read Autoscaling — Amazon Document DB (with MongoDB compatibility) is AWS’s MongoDB-(mostly)-compatible document database service and as storage and compute are separated, scaling is really flexible, as demonstrated here.
Randy DeFauw (AWS)
Average Pull Request Merge Time with AskGit SQL — AskGit is an open source command line tool for running SQL queries on git repositories.
Patrick DeVivo
New Hard Disk Write Head Analytical Technology To Increase HDD Capacity — Some rather mind boggling research has been taking place that involves imaging the magnetization dynamics of HDD write heads in order to eventually increase hard drive capacity.
Tohoku University
▶ Database Jiu Jitsu: How ScyllaDB Open Sourced a DynamoDB-Compatible API — Corey Quinn (who you might know from Last Week in AWS and his good humored Twitter snark) joined ScyllaDB’s CEO to discuss vendor lock-in, the economics of open source, and what ScyllaDB is up to.
ScyllaDB
▶ Breakdown Data Silos: Why S3 is the Best Place To Build Your Data Lake — From AWS’s recent AWS Storage Day. It’s basically a 20-minute pitch but ties together a variety of ideas and may be useful if you’re using or considering S3 for anything beyond mere file storage.
Matt Sidley (AWS)
🔨 Code and Tools
Another Redis Desktop Manager 1.4.0 — An open source desktop client for working with Redis that promises it ‘won’t crash’ if dealing with large numbers of keys. Supports TLS, dark mode, and more – seems like the author has put in some serious effort here.
qii404
simple-graph: A Simple Graph Database on Top of SQLite — Thanks to its recursive CTE support, you can rig up a graph database on SQLite anyway, but if you’re a Pythonista you might find this abstraction interesting.
Denis Papathanasiou
CYBERTEC PostgreSQL Enterprise Edition: Fully Encrypted & High-Performing — Check out the highly secure PostgreSQL distribution with advanced features like user-friendly monitoring and 24/7 support.
CYBERTEC sponsor
rqlite 5.8: A Distributed Relational Database Built on SQLite — I sometimes wonder if Postgres and SQLite are in a secret fight to have the most number of other databases built on top of them :-) 5.8.0 of this popular distributed database adds support for TLS connections.
rqlite
Database Lab Engine 2.1: Instant Clones of Large Postgres Databases for Dev Environments — Provision independent non-production environments with multi-terabyte Postgres databases in a few seconds without extra costs.
Postgres.ai
💼 Job
DevOps Engineer at X-Team (Remote) — Join the most energizing community for developers and work on projects for Riot Games, FOX, Sony, Coinbase, and more.
X-Team