awarm.spacenewsletter | fast | slow

Diverse Distributed Databases

Following my gut from last week, I experimented a little this week with Cloudflare workers, making a tiny app: https://workers-test.awarm.workers.dev/hello-world

Feel free to play around with it, it's not much more than a little toy. It's a simple note-taking app with very basic hyperlinks. But, it did what it was supposed to do, which was take me from 0 to 1 with Cloudflare workers.

One of the metrics that matter most for libraries, infrastructure providers, APIS, and any other creative tool is time to fun. It seems like the Cloudflare Workers team is intentionally optimizing for this, and they kind of nail it.

There was a wee bit of wrangling with wrangler, the CLI tool for setting up new projects, but once that was set up, it was a tiny command with a few seconds wait to get my code deployed and live.

This on it's own isn't that fun. You can do the same with many serverless and serverful tools alike. What really get's that dopamine firing for me, is making use of Workers KV.

This may be a view I've internalized as a software engineer, but dealing with storing data is fundamentally a pain. Part of the reason I leaned so hard into Ethereum and decentralized app architectures is because it removed the question of how/where/why to set up a database.

Cloudflare Workers does the same thing. Directly in my script I have access to a persistant, fast, simple datastore. I don't have to provision anything, set any configuration strings, or worry about where the data is going to live. It just happens. Even though it's quite limited, this just feels incredibly powerful.

Whats so special about distributed databases?

Part of what makes Workers KV is it's distributed. That means that even though the code you write could execute in any one of Cloudflare's many datacenters it can still access the same data every time.

From my perspective, these kinds of databases have been one of the defining trends of the last decade. Both "traditional" databases, as global tech companies try and move their data ever closer to their users, and more experimental systems, that try to do away entirely with centers of operation.

While these distributed systems solve hard technical problems for practical (and philosophical) reasons, they also end up with something that's just plain fun. It's incredibly satisfying to publish a website with beaker browser and be able to share it with anyone with just a link. And even more so to extend it to a full-fledged app that's reading and writing data local to whomever you just sent it to.

This is exciting from the lens of a single developer ("Look at all the things I don't have to worry about"), but becomes even more so when you look at distributed communities around these databases.

We haven't yet seen "end user" applications at scale that take advantage of these systems, but they're tailor made for a new generation of social tools. While the growth of Web 2.0 companies drove the creation of massive data silos, the promise of Web 3.0 is in truly networked applications. I'm pretty convinced that new applications basically have to do this if they want to compete with the dominant ones today.

An interesting piece of news in this vein is Roam's API announcement. If Conor and Co can pull it off Roam could be the distributed database at the heart of a new wave of networked thinking tools. And, it would come with a native interface for humans to read and write it.

A quick aside on hyperlink. Almost all our first courses launched this last week, and it's impossible to escape how much more we need to build.

There are three concerns currently on my mind:

  1. Increasing learner agency within a course
  2. Enabling learners to take things from a course to use in their own practice
  3. Connecting courses to other courses

Importantly, I want to enable systems across all those concerns. I want to build a tool for spaced repetition that can be an element of a course, that a learner can add to their personal deck, and that allows other courses to refrence your recall rate for particular items.

I'm not sure how exactly yet, but I feel like the database systems I've been talking about will be useful for this, though perhaps not immediately. For now their just two neurons that are firing together.