...
 
Commits (1)
......@@ -3,10 +3,20 @@ title = "Design"
weight = "10"
+++
The Indienet Engine is a loosely-coupled ActivityPub server implementation for single-person web apps in JavaScript (Node.js) that handles the following features via REST and WebSocket APIs and [message persistence](/engine/technology-stack/database/) (see [technology stack](../technology-stack)):
Indie Personal Activity Database (Indie PAD Base) is a personal server-to-server ActivityPub implementation for single-person web apps in JavaScript (Node.js) that handles the following:
* Server-to-server ActivityPub requests
* Persistence
* Creation of meaningful message feeds via map/reduce
* Expose high-level realtime change feeds / interface.
## TODO
Move to functionality to other component – Indie Pad (site):
via REST and WebSocket APIs and [message persistence](/engine/technology-stack/database/) (see [technology stack](../technology-stack)):
* [ ] Authentication
* [ ] Server-to-server interactions
* [ ] Client-to-server interactions
{{<mermaid align="left">}}
......
......@@ -3,7 +3,13 @@ title = "Technology Stack"
weight = "20"
+++
Indienet Engine is an implementation of the ActivityPub [protocol](protocol/) written in the JavaScript (ES6) [language](language/). It runs on a Node.js [server](server/) and is built using the FeathersJS [framework](framework/) and a RethinkDB [database](database/). It exposes websocket and REST [interfaces](interfaces/) and implements publickey and JWT [authentication](authentication/).
Indienet Engine is an implementation of the ActivityPub [protocol](protocol/) written in the JavaScript (ES6) [language](language/). It runs on a Node.js [server](server/) and is built as Express middleware. It uses the in-process LevelDB database and exposes a realtime API for meaningful changefeeds.
**TODO: Move to Indie Pad:**
…using the FeathersJS [framework](framework/) and a RethinkDB [database](database/). It exposes websocket and REST [interfaces](interfaces/) and implements publickey and JWT [authentication](authentication/).
***
The overriding design goal of Indienet Engine is to be a basic foundational component for creating federated apps and sites for individual use. As such, we want it to be as accessible and loosely-coupled as possible. Our technology choices attempt to reflect these goals.
......
......@@ -3,72 +3,11 @@ title = "Database"
weight = "70"
+++
**Note:** this decision is under review. Options:
## LevelDB
Indie PAD uses LevelDB with level-sublevel.
**Note:** What can we learn from our experience with [Heartbeat Node](https://source.ind.ie/project/heartbeat-node) - e.g., regarding the use of LevelDB for this purpose? (e.g., see the [implementation of the StreamWeaver class](https://source.ind.ie/project/heartbeat-node/blob/master/StreamWeaver.coffee)).
* [Awesome Level](https://github.com/Level/awesome)
* [github.com/Level/levelup/wiki/Modules](https://github.com/Level/levelup/wiki/Modules)
## Secure Scuttlebutt
[Secure Scuttlebutt](https://github.com/ssbc/secure-scuttlebutt) is built on LevelDB, specifically for p2p messaging. Would the higher-level abstraction aid or limit us?
Related:
* [Kappa architecture](http://milinda.pathirage.org/kappa-architecture.com/)
* [Scuttlebot](https://github.com/ssbc/scuttlebot)
## Other realtime/distributed data stores
* [Gun](https://github.com/amark/gun): “A realtime, decentralized, offline-first, graph database engine.” (Also see: [Gun Auth](https://github.com/amark/gun/wiki/auth))
* [HyperDB](https://github.com/mafintosh/hyperdb): “Distributed scalable database” (as used in [DAT project](https://datproject.org).)
## Other data persistence solutions
* [Hoodie](hood.ie)
## RethinkDB
RethinkDB is a realtime database that has a changefeeds feature that could simplify our production footprint/deployment requirements, server-side code dramatically and enable us to keep an event-driven workflow throughout.
Although RethinkDB supports clustering natively, scalability of the database should not be an issue for personal web sites/apps (as there’s a single owner).
### Possible issues
* Changefeed connections might fail: [ReQL proposal: restarting feeds](https://github.com/rethinkdb/rethinkdb/issues/3471)
### Related projects
* [Penseur](https://github.com/hueniverse/penseur): Lightweight RethinkDB wrapper. Changes API includes reconnect functionality:
```
await db[table].changes(criteria, [options])
Subscribe to changes matching the given criteria for the table.
criteria - db criteria functions chained together
options - optional object with the following properties
handler - handler function to execute when changes occur.
reconnect - boolean, reconnect if the connection to the feed is interrupted
initial - boolean, include the initial results in the change feed
```
* [node-rethinkdb-job-queue](https://github.com/grantcarthew/node-rethinkdb-job-queue): A persistent job or task queue backed by RethinkDB. (Also see, regarding changefeed connection failures: [Add rethinkdb-changefeed-reconnect](https://github.com/grantcarthew/node-rethinkdb-job-queue/issues/77))
### Useful links
* [Troubleshooting common RethinkDB problems](https://rethinkdb.com/docs/troubleshooting/)
## Other options
A more traditional database / message queue combination.
* [rsmq](https://github.com/smrchy/rsmq): Redis Simple Message Queue (Node)
* (RabbitMQ, etc.…)
* (For database, PostgreSQL, etc., or Mongo, etc.)
* …?
......@@ -5,6 +5,8 @@ weight = "60"
## Back-end/API
Every component aims to use the least amount of dependencies possible. The goal is for the core system to be a single process to simplify development and deployment. When in doubt, choose “human scale” over “enterprise”. Avoid premature optimisation/abstraction.
### FeathersJS
“A REST and realtime API layer for modern applications.”
......
......@@ -9,4 +9,4 @@ JavaScript is the language of the web.
Knowledge of JavaScript is a core skill that any modern developer should (and probably does) have in their toolset. We have chosen JavaScript as the language of our implementation of Engine.
However, since the goal of the Indienet Engine is to be as accessible as possible, we actively support ports to other languages.
However, since the goal of the Indienet Engine is to be as accessible as possible, we actively encourage ports to other languages.