projectmoon
/
tenebrous-dicebot
1
0
Fork 1
Code Issues 12 Pull Requests 1 Projects Releases 10 Wiki Activity

Updated Database Design (markdown)

ProjectMoon 2020-11-03 21:24:18 +00:00
parent efa8df30b1
commit 088b42465c
1 changed files with 27 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
Database-Design.md

@ -6,17 +6,33 @@ Why Sled?
* Mostly because I felt like it.
## High-Level Architecture Overview
A simple overview of how the database code is organized.
### Database Code
A struct `Database` is the entrypoint to all database functionality. The struct has fields, each their own sub-type, which expose the specific functionality. For example, the `Database` instance has a `Variables` field and a `Migrations` field. The `Variables` type relates to user variables, and the `Migrations` field manages the database migration version in the database.
### Data Migrations
The dicebot has very simple (and frankly dangerous) support for migrating data. The database keeps track of its current migration version, and when the application starts up, it checks to see if it needs to upgrade the format of data. The current migration version is stored in the config (currently hardcoded into the binary).
Migrations are simply functions that receive the `Database` instance, and can execute whatever they need to do to alter the data in the database. Migration functions are idempotent and make their changes in one or more atomic steps. This is necessary because the database version is updated in a transaction separate from any transactions the migration starts.
The dicebot currently has no support for automatically migrating Sled database file versions.
## Guiding Principles
Sled essentially operates as a bunch of `Map<[u8], [u8]>`s (that is, a map of byte slice keys and byte slice values), which means both keys and values can be pretty much any arbitrary type. To keep things simple, we follow these guidelines:
* Separate `Tree`s for different types of data. A `Tree` in Sled is an isolated keyspace.
* Strongly typed access to keys, converted into `Vec<u8>` keys by private code.
* Keys are UTF8 strings, sometimes separated by `0xfe` and `0xff` delimiter bytes. `0xfe` and `0xff` are not valid UTF8, making them useful delimiters.
* Strongly typed access to data, converted into `[u8]` keys by code private to the DB layer.
* Keys are UTF8 strings, sometimes separated by `0xfe` and `0xff` delimiter bytes. `0xfe` and `0xff` are not valid UTF8, which makes them useful delimiters.
### Trees, Keys, and Delimiters
Sled supports opening a number of `Tree` instances, each which acts as an isolated set of key-value pairs. We use trees to categorize data much like a relational SQL table.
Sled supports opening multiple `Tree` instances, each which acts as an isolated set of key-value pairs. We use trees to categorize data much like a relational SQL table.
* In general, a single tree should store one kind of data.
* Ideally, the tree does not have more that one format used for keys.
@ -37,15 +53,19 @@ The value stored in a tree can be anything, including serialized structs. Follow
Where possible, higher-level Sled APIs should be used. Batches, `compare_and_swap` and `fetch_and_update` are both very useful. In cases where this is not possible, transactions should be used to preserve atomicity.
# High-Level Architecture Overview
### Migrations
How things are designed.
Designing database migrations is difficult when the database doesn't really have anything resembling a query language, and the data stored is completely arbitrary! That said, some basic rules make them less painful:
# Design: User Variables
* Isolate all functionality for the migration inside its function, or inside a module only the migration function can access.
* Avoid use of any APIs from the dicebot database layer. These APIs change to match the latest requirements of the application. The migration should be able to craft and decode keys in a way completely decoupled from the API.
* Migrations must be idempotent. If the migration crashes halfway through execution, or the DB version update does not commit, the migration must be able to run again and produce the same result.
## Design: User Variables
This documents the database design of user-defined variables.
## Database Schema
### Database Schema
User variables are implemented with two trees:
* `room_user_variables`: Key format `<username> 0xfe <room_id> 0xff <variable_name>`. Value type is `i32`.