Schemas
This section provides guidance about how to model your data inside Ditto's collection and document schemaless database.
Creating relationships
Ditto does not support nesting documents or collections. If you need to create relationships
between documents, use a foreign key relationship by referencing the _id of
the document.
⚠️ Bad pattern: Big documents
You can embed a map in another map to create a nested map comprised of multiple, hierarchical levels. For example, a people collection that contains documents with the following schema:
This would translate to a document with a nested map cars where each car can be arbitrarily large.
{ "name": "Susan", "age": 43, "cars": { "abc123": { "make": "Hyundai", "color": "red", "mileage": 13000 }, "def456": { "make": "Jeep", "color": "blue", "mileage": 34000 }}This approach results in slow replication performance as follows:
When internet connection is limited or when using only Bluetooth to sync across connected peers, the process of replicating documents that contain large amounts of data is slow. For instance, when using only Bluetooth LE to replicate a document of a typical size, the rate of replication is 20 KB per second maximum. Given this, a document of 250 KB or larger in size may require 10 seconds or more to replicate for the first time between peer devices.
A slow replication rate results in a loading spinner being displayed to your end users until the replication process fully completes. This is due to the callback being unable to render the returned data; the end-to-end replication process requires that documents be broken down into smaller parts before being synced across the mesh, and then, until the client receives all of the smaller parts and then reconstructed, the document is not returned.
Instead of using a single document to encode all of your large dataset, use a series of smaller documents.
Note that if a document exceeds 250 KB in size, a stdout warning prints, and any documents larger than 5 MB will not sync to other peers.
👍 Good pattern: Flat models
Ditto syncs and queries documents based on a combination of the collection name and the document _id.
Collections are a way to create an index to a set of related documents. You can
create foreign relationships between documents by referencing the _id. For
example, you may have a people collection and a cars collection.
In the above example, each car is owned by a person. We represent this by using
the _id.ownerId field to relate to the person's _id.
When Susan gets a car, you can create a new document in the cars collection
with the foreign key relationship to the people collection.
In the people collection:
{ "_id": "abc123", "name": "Susan", "age": 31}In the cars collection.
{ "_id": { "id": "def456", "ownerId": "abc123" }, "make": "Hyundai", "color": "red",}info
Remember, _id is immutable, so only use this pattern when you are
dealing with static identifiers within your foreign key relationship.
Using a Write Transaction
When inserting a new car and a person at the same time, we want to use a write transaction. This allows a device to perform atomic transactions across collections within a single database call. This means that both documents will be synchronized at the same time to other devices.
In our above example, this ensures that a peer will always see the person document for Susan if they have received the Hyundai document.
ditto.store.write { transaction in let cars = transaction.scoped(toCollectionNamed: "cars") let people = transaction.scoped(toCollectionNamed: "people") let docId = "abc123" do { try people.upsert(["_id": docId, "name": "Susan"] as [String: Any?]) try cars.upsert(["make": "Ford", "color": "red", "owner": docId] as [String: Any?]) try cars.upsert(["make": "Toyota", "color": "black", "owner": docId] as [String: Any?]) } catch (let err) { print(err.localizedDescription) } people.findByID(docId).evict()}For more information about write transactions, see the section on Writing -> Batching.
Role-based permissions to documents
There are situations where you want to show a document in a view only for some devices. For example, if a device is being used by a First Class passenger, you will want to show special features just for those authorized users.
In these cases, we recommend that the model have a boolean property that determines if the received data should be shown in the UI.
// Document content
let note = ["text": text, "firstClassOnly": false]
if !note.firstClassOnly { // Show it on UI}To offer authentication for role-based permission, we recommend using the OnlineWithAuthentication identity.
Versioning
Ditto's replication protocol is backwards-compatible and reliable. This means that eventually you will have the "couch device problem" (i.e., a device that fell behind a couch). In other words, a device in your mesh may be offline for a significant amount of time before connecting back with other devices.
If the shape of your documents are significantly different on that device, there could be documents that do not conform with your new application code. Synchronizing with this "couch device" could cause other devices to crash unexpectedly in production if precautions aren't taken in your application schema.
When do I version my documents?
Changing your schema is inevitable. To ensure reliability over time, you should create your own schema versioning pattern for each Ditto document.
Same-version compatibility
Some applications do not need backward- or forward- compatibility, which can
simplify their business logic significantly. If that sounds like your
application, we recommend that you use a pattern where you change the name of
the collection for each schema version of your application. This enforces
further that field types never change. For example, you can use
myCollection_v{number} as a convention to specify the collection schema
version your app will be listening to. When a schema change is necessary, bump
the number. Collections are very cheap to create in Ditto, so this will scale
even for applications that run for many years.
You could also only synchronize documents that come from schema versions that are the same as your current schema version.
const query = 'name == $args.name && age <= $args.age && _schemaVersion == 1'collection.find(query, () => { age: 32, name: 'Max',})Forward-compatibility
In a typical centralized database like PostgreSQL, developers often focus on backward-compatbility, where newer versions of the application can open old documents. In a distributed system, you do not have central control of all modifications to data. In an offline peer-to-peer mesh, it is difficult and sometimes impossible to control all versions of your application that are active in production environments. Because of these constraints, you need to not only think of backward-compatibility, but also forward-compatibility.
An application is forward-compatible when existing code is able to read new data. We can see forward-compatibility in web development.
To achieve forward-compatibility of your database, you should never change the type of an existing field. In other words, developers should only ever add new fields, and never remove or modify old fields. You can ensure this by creating a controller that encapsulates Ditto and is used across your application(s) to validate the field values and their associated types before upserting those values into the database.
Backward-compatibility
Older data could be very important, or it could not be. It's your choice to decide what to do with these old documents: you could accept (as-is), reject (ignore), or migrate them to the new schema.
For example, here's a breaking version change where we add a new field and change the type of an old field:
App version 2
private struct V1Car { let _id: String let make: String let model: String let year: String var version: Int init(doc: DittoDocument) { self._id = doc["_id"].stringValue self.make = doc["make"].stringValue self.model = doc["model"].stringValue self.year = doc["year"].stringValue self.version = doc["version"].intValue }}
private struct V2Car { let _id: String let make: String let model: String let year: Int let hometown: String var version: Int init(_id: String, make: String, model: String, year: Int, hometown: String, version: Int) { self._id = _id self.make = make self.model = model self.year = year self.hometown = hometown self.version = version } init(doc: DittoDocument) { self._id = doc["_id"].stringValue self.make = doc["make"].stringValue self.model = doc["model"].stringValue self.year = doc["year"].intValue self.hometown = doc["hometown"].stringValue self.version = doc["version"].intValue }}
func decode(car: DittoDocument) -> V2Car { switch(car.version) { case 1 { let oldCar = V1Car(car) let migratedCar = V2Car(_id: oldCar._id, make: oldCar.make, model: oldCar.model, year: Int(oldCar.year), hometown: "N/A", version: 2) return migratedCar } case 2 { return V2Car(car) } }}App version 1
You also may want to ignore documents that come from incompatible applications.
private struct V1Car { let _id: String let make: String let model: String let year: String var version: Int init(doc: DittoDocument) { self._id = doc["_id"].stringValue self.make = doc["make"].stringValue self.model = doc["model"].stringValue self.year = doc["year"].stringValue self.version = doc["version"].intValue }}
func decode(car: DittoDocument) -> V1Car { switch(car.version) { case 1 : let oldCar = V1Car(car) return oldCar default: // create default car item, or ignore document altogether return }}Supporting the latest version
When a new application version is detected, you can stop synchronizing. You can detect that a new application version is available by querying for a
_schemaVersion that is greater than the current version. If a new version is
detected, stop sync and tell the user they need to upgrade their app to the
latest version.
const query = '_schemaVersion > 1'collection.find(query).observeLocal(() => { // Notify user to update to latest application version. ditto.stopSync()})This is a common pattern that many applications use. For example, Apple Notes warns users that they are on an older version and will experience degraded features until they upgrade.