manual, devs, set internals toc

This commit is contained in:
Mik-TF 2024-05-14 12:09:00 -04:00
parent 678ca4b842
commit bc773126e0
3 changed files with 34 additions and 18 deletions

View File

@ -4,16 +4,33 @@ We present in this section of the developers book a partial list of system compo
<h2> Table of Contents </h2> <h2> Table of Contents </h2>
- [Reliable Message Bus (RMB)](rmb_toc.md) - [Reliable Message Bus - RMB](rmb_toc.md)
- [Introduction to RMB](rmb_intro.md) - [Introduction to RMB](rmb_intro.md)
- [RMB Specs](rmb_specs.md) - [RMB Specs](rmb_specs.md)
- [RMB Peer](peer.md) - [RMB Peer](peer.md)
- [RMB Relay](relay.md) - [RMB Relay](relay.md)
- [Zero-OS](zos_readme.md)
- [ZOS](zos_readme.md)
- [Manual](manual.md) - [Manual](manual.md)
- [Workload Types](workload_types.md) - [Workload Types](workload_types.md)
- [Internal Modules](internals.md) - [Internal Modules](internals.md)
- [Identity](identity_readme.md)
- [Node ID Generation](identity.md)
- [Node Upgrade](upgrade.md)
- [Node](node_readme.md)
- [Storage](storage_readme.md)
- [Network](network_readme.md)
- [Introduction](introduction.md)
- [Definitions](definitions.md)
- [Mesh](mesh.md)
- [Setup](setup_farm_network.md)
- [Flist](flist_readme.md)
- [Container](container_readme.md)
- [VM](vmd_readme.md)
- [Provision](provision_readme.md)
- [Capacity](capacity.md) - [Capacity](capacity.md)
- [Performance Monitor Package](performance.md) - [Performance Monitor Package](performance.md)
- [Public IPs Validation Task](publicips.md)
- [CPUBenchmark](cpubench.md)
- [IPerf](iperf.md)
- [Health Check](healthcheck.md)
- [API](api.md) - [API](api.md)

View File

@ -14,7 +14,7 @@
- [Building](#building) - [Building](#building)
- [Running tests](#running-tests) - [Running tests](#running-tests)
*** ---
## What is RMB ## What is RMB
@ -27,7 +27,7 @@ Out of the box RMB provides the following:
- Support for 3rd party hosted relays. Anyone can host a relay and people can use it safely since there is no way messages can be inspected while using e2e. That's similar to `home` servers by `matrix` - Support for 3rd party hosted relays. Anyone can host a relay and people can use it safely since there is no way messages can be inspected while using e2e. That's similar to `home` servers by `matrix`
![layout](img/layout.png) ![layout](img/layout.png)
***
## Why ## Why
RMB is developed by ThreefoldTech to create a global network of nodes that are available to host capacity. Each node will act like a single bot where you can ask to host your capacity. This enforced a unique set of requirements: RMB is developed by ThreefoldTech to create a global network of nodes that are available to host capacity. Each node will act like a single bot where you can ask to host your capacity. This enforced a unique set of requirements:
@ -45,17 +45,17 @@ Starting from this we came up with a more detailed requirements:
- Then each message then can be signed by the `bot` keys, hence make it easy to verify the identity of the sender of a message. This is done both ways. - Then each message then can be signed by the `bot` keys, hence make it easy to verify the identity of the sender of a message. This is done both ways.
- To support federation (using 3rd party relays) we needed to add e2e encryption to make sure messages that are surfing the public internet can't be sniffed - To support federation (using 3rd party relays) we needed to add e2e encryption to make sure messages that are surfing the public internet can't be sniffed
- e2e encryption is done by deriving an encryption key from the same identity seed, and share the public key on `tfchain` hence it's available to everyone to use - e2e encryption is done by deriving an encryption key from the same identity seed, and share the public key on `tfchain` hence it's available to everyone to use
***
## Specifications ## Specifications
For details about protocol itself please check the [specs](rmb_specs.md). For details about protocol itself please check the [specs](rmb_specs.md).
***
## How to Use RMB ## How to Use RMB
There are many ways to use `rmb` because it was built for `bots` and software to communicate. Hence, there is no mobile app for it for example, but instead a set of libraries where you can use to connect to the network, make chitchats with other bots then exit. There are many ways to use `rmb` because it was built for `bots` and software to communicate. Hence, there is no mobile app for it for example, but instead a set of libraries where you can use to connect to the network, make chitchats with other bots then exit.
Or you can keep the connection forever to answer other bots requests if you are providing a service. Or you can keep the connection forever to answer other bots requests if you are providing a service.
***
## Libraries ## Libraries
If there is a library in your preferred language, then you are in luck! Simply follow the library documentations to implement a service bot, or to make requests to other bots. If there is a library in your preferred language, then you are in luck! Simply follow the library documentations to implement a service bot, or to make requests to other bots.
@ -64,14 +64,14 @@ If there is a library in your preferred language, then you are in luck! Simply f
- Golang [rmb-sdk-go](https://github.com/threefoldtech/rmb-sdk-go) - Golang [rmb-sdk-go](https://github.com/threefoldtech/rmb-sdk-go)
- Typescript [rmb-sdk-ts](https://github.com/threefoldtech/rmb-sdk-ts) - Typescript [rmb-sdk-ts](https://github.com/threefoldtech/rmb-sdk-ts)
***
### No Known Libraries ### No Known Libraries
If there are no library in your preferred language, here's what you can do: If there are no library in your preferred language, here's what you can do:
- Implement a library in your preferred language - Implement a library in your preferred language
- If it's too much to do all the signing, verification, e2e in your language then use `rmb-peer` - If it's too much to do all the signing, verification, e2e in your language then use `rmb-peer`
***
## What is rmb-peer ## What is rmb-peer
think of `rmb-peer` as a gateway that stands between you and the `relay`. `rmb-peer` uses your mnemonics (your identity secret key) to assume your identity and it connects to the relay on your behalf, it maintains the connection forever and takes care of think of `rmb-peer` as a gateway that stands between you and the `relay`. `rmb-peer` uses your mnemonics (your identity secret key) to assume your identity and it connects to the relay on your behalf, it maintains the connection forever and takes care of
@ -85,11 +85,11 @@ Then it provide a simple (plain-text) api over `redis`. means to send messages (
> More details can be found [here](rmb_specs.md) > More details can be found [here](rmb_specs.md)
***
## Download ## Download
Please check the latest [releases](https://github.com/threefoldtech/rmb-rs/releases) normally you only need the `rmb-peer` binary, unless you want to host your own relay. Please check the latest [releases](https://github.com/threefoldtech/rmb-rs/releases) normally you only need the `rmb-peer` binary, unless you want to host your own relay.
***
## Building ## Building
```bash ```bash
@ -97,7 +97,7 @@ git clone git@github.com:threefoldtech/rmb-rs.git
cd rmb-rs cd rmb-rs
cargo build --release --target=x86_64-unknown-linux-musl cargo build --release --target=x86_64-unknown-linux-musl
``` ```
***
## Running tests ## Running tests
While inside the repository While inside the repository

View File

@ -15,7 +15,7 @@
- [End2End Encryption](#end2end-encryption) - [End2End Encryption](#end2end-encryption)
- [Rate Limiting](#rate-limiting) - [Rate Limiting](#rate-limiting)
*** ---
# Introduction # Introduction
@ -51,7 +51,7 @@ On the relay, the relay checks federation information set on the envelope and th
When the relay receive a message that is destined to a `local` connected client, it queue it for delivery. The relay can maintain a queue of messages per twin to a limit. If the twin does not come back online to consume queued messages, the relay will start to drop messages for that specific twin client. When the relay receive a message that is destined to a `local` connected client, it queue it for delivery. The relay can maintain a queue of messages per twin to a limit. If the twin does not come back online to consume queued messages, the relay will start to drop messages for that specific twin client.
Once a twin come online and connect to its peer, the peer will receive all queued messages. the messages are pushed over the web-socket as they are received. the client then can decide how to handle them (a message can be a request or a response). A message type can be inspected as defined by the schema. Once a twin come online and connect to its peer, the peer will receive all queued messages. the messages are pushed over the web-socket as they are received. the client then can decide how to handle them (a message can be a request or a response). A message type can be inspected as defined by the schema.
***
# Overview of the Operation of RMB Relay # Overview of the Operation of RMB Relay
![relay](img/relay.png) ![relay](img/relay.png)
@ -201,7 +201,6 @@ A response message is defined as follows this is what is sent as a response by a
Your bot (server) need to make sure to set `destination` to the same value as the incoming request `source` Your bot (server) need to make sure to set `destination` to the same value as the incoming request `source`
The
> this response is what is pushed to `msgbus.system.reply` > this response is what is pushed to `msgbus.system.reply`
```rust ```rust
@ -223,7 +222,7 @@ pub struct JsonOutgoingResponse {
pub error: Option<JsonError>, pub error: Option<JsonError>,
} }
``` ```
***
# End2End Encryption # End2End Encryption
Relay is totally opaque to the messages. Our implementation of the relay does not poke into messages except for the routing attributes (source, and destinations addresses, and federation information). But since the relay is designed to be hosted by other 3rd parties (hence federation) you should Relay is totally opaque to the messages. Our implementation of the relay does not poke into messages except for the routing attributes (source, and destinations addresses, and federation information). But since the relay is designed to be hosted by other 3rd parties (hence federation) you should
@ -246,7 +245,7 @@ As you already understand e2e is completely up to the peers to implement, and ev
- derive the same shared key - derive the same shared key
- `shared = ecdh(B.sk, A.pk)` - `shared = ecdh(B.sk, A.pk)`
- `plain-data = aes-gcm.decrypt(shared-key, nonce, encrypted)` - `plain-data = aes-gcm.decrypt(shared-key, nonce, encrypted)`
***
# Rate Limiting # Rate Limiting
To avoid abuse of the server, and prevent DoS attacks on the relay, a rate limiter is used to limit the number of clients' requests.\ To avoid abuse of the server, and prevent DoS attacks on the relay, a rate limiter is used to limit the number of clients' requests.\