I Am Building a Database

A few months ago I was pondering on what sort of 'lifelong' project I wanted to do, and I thought 'well why not something that I would never get a chance to work on' which lead me to a simple answer: a database.

But databases are not simple pieces of software (as we know them) even though you could have a file and search inside it, there's a lot of pieces in play. I don't have much time either to work on it, but when I do, I figured this would really push my own engineering growth because of it being an anomoly.

I warmed up with implementing a subset of the redis specification, just enough to wonder if this is what I wanted to do. Once I had enough, I started writing my b+tree implementation "ok here's my structure and oh wait I need to persist this to disk oh it's so over" yeah having a data structure based around disk changes how you think a lot compared to in-memory.

So far I have only worked on the b+tree implementation and it's been a lot of fun, I didn't read anything (pseudocode, actual code, behaviour etc) and rawdogged it by using this cool site: BPlusTree App

By looking at the behaviour, I would figure out the edge cases:

So going blind has been really fun, and that will apply to other features as well. I don't have any concrete plans of what I want this to be, I think it will emerge overtime. However, I'm thinking of it being embedded/abstraction around a single file kinda like sqlite. Since it'll take time to slowly build parts, I figure there's no point trying to live up to a specific design right now (sumtypes tonight? sumtypes tonight king??).

I've got a few books lined up to read during times away from my laptop, similarly I think I'll dive into other projects (sqlite, postgres, cassandra etc) and see what's up.

I'm writing this in Java (always the latest JDK + preview features) + compiling with GraalVM vs e.g. Rust, mainly because there's a lot of apis I don't touch often and I think it'll be good to keep myself sharp with them. It'll be open on github as well once it's in an mvp state.

The overall structure of the code is based on one idea: fun. I will write the code however I want to because yeah I wanna do recursion miss me with your bullshit.

Here's my blog on tail call recursion in Java

I'll do blog updates when I think I've made enough interesting progress to share. Might be slow however because there's a lot of reading around databases I need to consume.

The working name is 'ButterflyDB' because butterfly is my favourite guard in jiujitsu.