Ind.ie is now Small Technology Foundation.
Commit f6caf448 authored by Aral Balkan's avatar Aral Balkan

Update index copy, add CRDT spike info

parent c3ee4c90
---
title: "ActivityPub"
weight: 10
weight: 30
---
Explore ActivityPub implementation and interoperability (e.g., with Mastodon) for the [Indie Site](/site) project.
......
---
title: "CRDT: Conflict-free/Commutative Replicated Data Types"
weight: 10
---
Indie Site is expected to:
1. Work offline (in web parlance/marketing speak: “offline first”/“progressive web app”)
2. Be able to sync changes between the server and clients
3. Eventually be able to support peer-to-peer communication between clients
All of these require a robust sync solution for sharing state between the various nodes (the server is merely seen as another node in this approach). And sync is hard.
## OT vs CRDT
We can rule out OT from the outset as there is a body of evidence that [it is overly-complicated to implement](https://en.wikipedia.org/wiki/Operational_transformation#Critique_of_OT):
“Joseph Gentle who is a former Google Wave engineer and an author of the Share.JS library wrote, "Unfortunately, implementing OT sucks. There's a million algorithms with different tradeoffs, mostly trapped in academic papers. The algorithms are really hard and time consuming to implement correctly. [] Wave took 2 years to write and if we rewrote it today, it would take almost as long to write a second time.”
Our spikes will focus on evaluating various CRDT algorithms and, when possible, existing JavaScript implementations of the underlying algorithms.
## CRDTs
CRDT stands for [conflict-free/commutative replicated datatype](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type).
### WOOT (and WOOTO, WOOTH, etc.)
[Aral] My own research in this area dates back six years when I was looking at the WOOT algorithm ([Objective-C implementation](https://github.com/aral/trovedata), talk at (Warning: YouTube link) [Realtime Conf 2013](https://www.youtube.com/watch?v=NSTZ4mIv_wk))
In [Evaluating CRDTs for Real-time Document Editing](https://hal.inria.fr/inria-00629503/document) (PDF) by Mehdi Ahmed-Nacer, et. al, even the original researchers rule out WOOT as a real-world algorithm due to performance/scalability issues. The optimised WOOTO and WOOTH algorithms do not fair much either.
### Logoot
The algorithm that comes up on top in the above-mentioned paper is [Logoot](https://hal.archives-ouvertes.fr/inria-00432368/document) (PDF).
(Also, there is a variant of Logoot called Logoot-undo that implements undo/redo anywhere.)
TODO:
* [ ] Spike logoot
* [ ] Share findings
JavaScript implementations in the wild:
* [bnoguchi/logoot](https://github.com/bnoguchi/logoot)
* [usecanvas/logoot-js](https://github.com/usecanvas/logoot-js) (has a great readme, by the way)
### Automerge
Spike: https://source.ind.ie/indienet/spikes/crdt/automerge
Automerge is a JavaScript library for building distributed systems using a conflict-free JSON data structure. It is based on the paper [A Conflict-Free Replicated JSON Datatype](https://arxiv.org/abs/1608.03960) by Martin Kleppmann and Alastair R. Beresford.
I summarised my initial (very basic) performance/storage-related findings at:
https://github.com/automerge/automerge/issues/89
In its current state, I’m not sure we can use this. To have a conflict-free JSON structure would be amazing but it currently comes with a lot of overhead. There’s nothing we want to achieve that we cannot using a linear structure such as the ones used by more traditional CRDTs like Logoot. My next spike will be to explore this option.
## Resources
* [CRDT Playground](https://github.com/archagon/crdt-playground)
\ No newline at end of file
......@@ -13,6 +13,8 @@ But what is a Federated Personal Web Site? Good question. It’s:
* __Web Site:__ apart from the above two qualifications, an FPWS is a regular web site. It is build using web technologies and experienced through a standard web browsers. Indie site functionality can also be experienced via native apps in the future that use its REST And WebSocket APIs.
__May 2018, update:__ While Indie site requires a server component for findability and availability (two areas in which strictly p2p apps struggle), it will eschew the standard client/server model in favour of a peer-to-peer approach using an “offline first” approach to its web client and a replicated datastore. We are currently researching various conflict-free/commutative replicated data types/algorithms. Work on this is currently taking place on the [Spikes](/other/spikes/) section.
## Sections
{{% children %}}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment