diff --git a/Database-Design.md b/Database-Design.md index c25b03d..ac7a37e 100644 --- a/Database-Design.md +++ b/Database-Design.md @@ -6,17 +6,33 @@ Why Sled? * Mostly because I felt like it. +## High-Level Architecture Overview + +A simple overview of how the database code is organized. + +### Database Code + +A struct `Database` is the entrypoint to all database functionality. The struct has fields, each their own sub-type, which expose the specific functionality. For example, the `Database` instance has a `Variables` field and a `Migrations` field. The `Variables` type relates to user variables, and the `Migrations` field manages the database migration version in the database. + +### Data Migrations + +The dicebot has very simple (and frankly dangerous) support for migrating data. The database keeps track of its current migration version, and when the application starts up, it checks to see if it needs to upgrade the format of data. The current migration version is stored in the config (currently hardcoded into the binary). + +Migrations are simply functions that receive the `Database` instance, and can execute whatever they need to do to alter the data in the database. Migration functions are idempotent and make their changes in one or more atomic steps. This is necessary because the database version is updated in a transaction separate from any transactions the migration starts. + +The dicebot currently has no support for automatically migrating Sled database file versions. + ## Guiding Principles Sled essentially operates as a bunch of `Map<[u8], [u8]>`s (that is, a map of byte slice keys and byte slice values), which means both keys and values can be pretty much any arbitrary type. To keep things simple, we follow these guidelines: * Separate `Tree`s for different types of data. A `Tree` in Sled is an isolated keyspace. -* Strongly typed access to keys, converted into `Vec` keys by private code. -* Keys are UTF8 strings, sometimes separated by `0xfe` and `0xff` delimiter bytes. `0xfe` and `0xff` are not valid UTF8, making them useful delimiters. +* Strongly typed access to data, converted into `[u8]` keys by code private to the DB layer. +* Keys are UTF8 strings, sometimes separated by `0xfe` and `0xff` delimiter bytes. `0xfe` and `0xff` are not valid UTF8, which makes them useful delimiters. ### Trees, Keys, and Delimiters -Sled supports opening a number of `Tree` instances, each which acts as an isolated set of key-value pairs. We use trees to categorize data much like a relational SQL table. +Sled supports opening multiple `Tree` instances, each which acts as an isolated set of key-value pairs. We use trees to categorize data much like a relational SQL table. * In general, a single tree should store one kind of data. * Ideally, the tree does not have more that one format used for keys. @@ -37,15 +53,19 @@ The value stored in a tree can be anything, including serialized structs. Follow Where possible, higher-level Sled APIs should be used. Batches, `compare_and_swap` and `fetch_and_update` are both very useful. In cases where this is not possible, transactions should be used to preserve atomicity. -# High-Level Architecture Overview +### Migrations -How things are designed. +Designing database migrations is difficult when the database doesn't really have anything resembling a query language, and the data stored is completely arbitrary! That said, some basic rules make them less painful: -# Design: User Variables +* Isolate all functionality for the migration inside its function, or inside a module only the migration function can access. +* Avoid use of any APIs from the dicebot database layer. These APIs change to match the latest requirements of the application. The migration should be able to craft and decode keys in a way completely decoupled from the API. +* Migrations must be idempotent. If the migration crashes halfway through execution, or the DB version update does not commit, the migration must be able to run again and produce the same result. + +## Design: User Variables This documents the database design of user-defined variables. -## Database Schema +### Database Schema User variables are implemented with two trees: * `room_user_variables`: Key format ` 0xfe 0xff `. Value type is `i32`.