tfgrid/info_tfgrid
33
3
Fork 0
Code Issues Pull Requests 2 Packages Projects Releases Activity

updated smaller collections for manual

Browse Source
This commit is contained in:
mik-tf
2024-04-29 17:49:02 +00:00
parent 4806cb355b
commit b1e98c2051
1872 changed files with 50236 additions and 3303 deletions
Download Patch File Download Diff File Expand all files Collapse all files

127
collections/developers/grid_deployment/deploy_dashboard.md Normal file
View File

@@ -0,0 +1,127 @@
<h1>Deploy the Dashboard</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Create an SSH Tunnel](#create-an-ssh-tunnel)
- [Editor SSH Remote Connection](#editor-ssh-remote-connection)
- [Set the VM](#set-the-vm)
- [Build the Dashboard](#build-the-dashboard)
- [Dashboard Public Access](#dashboard-public-access)
- [Questions and Feedback](#questions-and-feedback)
***
## Introduction
We show how to deploy the Dashboard (devnet) on a full VM. To do so, we set an SSH tunnel and use the VSCodium Remote Explorer function. We will then be able to use a source-code editor to explore the code and see changes on a local browser.
We also show how to provide a public access to the Dashboard by setting a gateway domain to your full VM deployment. Note that this method is not production-ready and should only be used to test the Dashboard.
## Prerequisites
- TFChain account with TFT
- [Deploy full VM with WireGuard connection](../../system_administrators/getstarted/ssh_guide/ssh_wireguard.md)
- [Make sure you can connect via SSH on the terminal](../../system_administrators/getstarted/ssh_guide/ssh_openssh.md)
In this guide, we use WireGuard, but you can use other connection methods, such as [Mycelium](../../system_administrators/mycelium/mycelium_toc.md).
## Create an SSH Tunnel
- Open a terminal and create an SSH tunnel
```
ssh -4 -L 5173:127.0.0.1:5173 root@10.20.4.2
```
Simply leave this window open and follow the next steps.
If you use an IPv6 address, e.g. with Mycelium, set `-6` in the line above instead of `-4`.
## Editor SSH Remote Connection
You can connect via SSH through the source-code editor to a VM on the grid. In this example, WireGuard is set.
- Add the SSH Remote extension to [VSCodium](https://vscodium.com/)
- Add a new SSH remote connection
- Set the following (adjust with your own username and host)
```
Host 10.20.4.2
HostName 10.20.4.2
User root
```
- Click on `Connect to host`
## Set the VM
We set the VM to be able to build the Dashboard.
```
apt update && apt install build-essential python3 -y
wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
nvm install 18
npm install -g yarn
```
## Build the Dashboard
We now build the Dashboard.
Clone the repository, then install, build and run the Dashboard. Note that here it is called `playground`:
```
git clone https://github.com/threefoldtech/tfgrid-sdk-ts
cd tfgrid-sdk-ts/
yarn install
make build
make run project=playground
```
You can then access the dev net Dashboard on your local browser.
To stop running the Dashboard, simply enter ̀`Ctrl-C` on the terminal window.
## Dashboard Public Access
> Note: This method is not production-ready. Use only for testing purposes.
Once you've tested the Dashboard with the SSH tunnel, you can explore how to access it from the public Internet. For this, we will create a gateway domain and bind the host to `0.0.0.0`.
On the Full VM page, [add a domain](../../dashboard/solutions/add_domain.md) to access your deployment from the public Internet.
- Under `Actions`, click on `Manage Domains`
- Go to `Add New Domain`
- Choose a gateway domain under `Select domain`
- Set the port 5173
- Click on `Add`
To run the Dashboard from the added domain, use this instead of the previous `make run` line:
```
cd packages/playground
yarn dev --host 0.0.0.0
```
You can then access the Dashboard from the domain you just created.
## Questions and Feedback
If you have any questions or feedback, please let us know by either writing a post on the [ThreeFold Forum](https://forum.threefold.io/), or by chatting with us on the [TF Grid Tester Community](https://t.me/threefoldtesting) Telegram channel.

12
collections/developers/grid_deployment/grid_deployment.md Normal file
View File

@@ -0,0 +1,12 @@
# Grid Deployment
The TFGrid whole source code is open-source and instances of the grid can be deployed by anyone thanks to the distribution of daily grid snapshots of the complete ThreeFold Grid stacks.
This section also covers the steps to deploy the Dashboard locally. This can be useful when testing the grid or contributing to the open-source project.
## Table of Contents
- [TFGrid Stacks](./tfgrid_stacks.md)
- [Full VM Grid Deployment](./grid_deployment_full_vm.md)
- [Grid Snapshots](./snapshots.md)
- [Deploy the Dashboard](./deploy_dashboard.md)

175
collections/developers/grid_deployment/grid_deployment_full_vm.md Normal file
View File

@@ -0,0 +1,175 @@
<h1> Grid Deployment on a Full VM </h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Deploy All 3 Network Instances](#deploy-all-3-network-instances)
- [DNS Settings](#dns-settings)
- [DNS Verification](#dns-verification)
- [Prepare the VM](#prepare-the-vm)
- [Set the Firewall](#set-the-firewall)
- [Launch the Script](#launch-the-script)
- [Access the Grid Services](#access-the-grid-services)
- [Manual Commands](#manual-commands)
- [Update the Deployment](#update-the-deployment)
***
## Introduction
We present the steps to deploy an instance of the TFGrid on a full VM.
For this guide, we will be deploying a mainnet instance. While the steps are similar for testnet and devnet, you will have to adjust your deployment depending on which network you use. Details are provided when needed.
We also provide information to deploy the 3 different network instances.
## Prerequisites
For this guide, you will need to deploy a full VM on the ThreeFold Grid with at least the following minimum specs:
- IPv4
- IPv6
- 32GB of RAM
- 1000 GB of SSD
- 8 vcores
After deploying the full VM, take note of the IPv4 and IPv6 addresses to properly set the DNS records and then SSH into the VM.
It is recommended to deploy on a machine with modern hardware and NVME storage disk.
## Deploy All 3 Network Instances
To deploy the 3 network instances, mainnet, testnet and mainnet, you need to follow the same process for each network on a separate machine or at least on a different VM.
This means that you can either deploy each network instance on 3 different machines, or you can also deploy 3 different VMs on the same machine, e.g. a dedicated node. Then, each VM will run a different network instance. In this case, you will certainly need a machine with NVME storage disk and modern hardware.
## DNS Settings
You need to set an A record for the IPv4 address and an AAAA record for the IPv6 address with a wildcard subdomain.
The following table explicitly shows how to set the A and AAAA records for your domain for all 3 networks. Note that both `testnet` and `devnet` have a subdomain. The last two lines are for mainnet since no subdomain is needed in this case.
| Type | Host | Value |
| ---- | ---- | -------------- |
| A | \*.dev | <devnet_ipv4_address> |
| AAAA | \*.dev | <devnet_ipv6_address> |
| A | \*.test | <testnet_ipv4_address> |
| AAAA | \*.test | <testnet_ipv6_address> |
| A | \* | <mainnet_ipv4_address> |
| AAAA | \* | <mainnet_ipv6_address> |
As stated above, each network instance must be on its own VM or machine to work properly. Make sure to adjust the DNS records accordingly.
### DNS Verification
You can use tools such as [DNSChecker](https://dnschecker.org/) or [dig](https://linux.die.net/man/1/dig) on a terminal to check if the DNS propagadation is complete.
## Prepare the VM
We show the steps to prepare the VM to run the network instance.
If you are deploying on testnet or devnet, simply replace `mainnet` by the proper network in the following lines.
- Download the ThreeFold Tech `grid_deployment` repository
```
git clone https://github.com/threefoldtech/grid_deployment
cd grid_deployment/docker-compose/mainnet
```
- Generate a TFChain node key with `subkey`
- Note: If you deploy the 3 network instances, you can use the same node key for all 3 networks. But it is recommended to use 3 different keys to facilitate management.
```
echo .subkey_mainnet >> .gitignore
../subkey generate-node-key > .nodekey_mainnet
cat .nodekey_mainnet
```
- Create and the set environment variables file
```
cp .secrets.env-example .secrets.env
```
- Adjust the environment file
```
nano .secrets.env
```
- To adjust the `.secrets.env` file, take into account the following:
- **DOMAIN**="example.com"
- Write your own domain
- **TFCHAIN_NODE_KEY**="abc123"
- Write the output of the command `cat .nodekey_mainnet`
- **ACTIVATION_SERVICE_MNEMONIC**="word1 word2 ... word24"
- Write the seed phrase of an account on mainnet with at least 10 TFT in the wallet
- **GRID_PROXY_MNEMONIC**="word1 word2 ... word24"
- Write the seed phrase of an account on mainnet with at least 10 TFT in the wallet and a registered twin ID\*
> \*Note: If you've created an account using the ThreeFold Dashboard on a given network, the twin ID is automatically registered for this network.
## Set the Firewall
You can use UFW to set the firewall:
```
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 30333/tcp
ufw allow 22/tcp
ufw enable
ufw status
```
## Launch the Script
Once you've prepared the VM, you can simply run the script to install the grid stack and deploy it online.
```
sh install_grid_bknd.sh
```
This will take some time since you are downloading the whole mainnet grid snapshots.
## Access the Grid Services
Once you've deployed the grid stack online, you can access the different grid services by usual the usual subdomains:
```
dashboard.example.com
metrics.example.com
tfchain.example.com
graphql.example.com
relay.example.com
gridproxy.example.com
activation.example.com
stats.example.com
```
In the case of testnet and devnet, links will also have the given subdomain, such as `dashboard.test.example.com` for a `testnet` instance.
## Manual Commands
Once you've run the install script, you can deploy manually the grid stack with the following command:
```
docker compose --env-file .secrets.env --env-file .env up -d
```
You can also check if the environment variables are properly set:
```
docker compose --env-file .secrets.env --env-file .env config
```
If you want to see the output during deployment, remove `-d` in the command above as follows:
```
docker compose --env-file .secrets.env --env-file .env up
```
This can be helpful to troubleshoot errors.
## Update the Deployment
Go into the folder of the proper network, e.g. mainnet, and run the following commands:
```
git pull -r
docker compose --env-file .secrets.env --env-file .env up -d
```

259
collections/developers/grid_deployment/snapshots.md Normal file
View File

@@ -0,0 +1,259 @@
<h1>Snapshots for Grid Backend Services</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Services](#services)
- [ThreeFold Public Snapshots](#threefold-public-snapshots)
- [Requirements](#requirements)
- [Files for Each Net](#files-for-each-net)
- [Deploy All 3 Network Instances](#deploy-all-3-network-instances)
- [Deploy a Snapshot Backend](#deploy-a-snapshot-backend)
- [Deploy the Services with Scripts](#deploy-the-services-with-scripts)
- [Create the Snapshots](#create-the-snapshots)
- [Start All the Services](#start-all-the-services)
- [Stop All the Services](#stop-all-the-services)
- [Expose the Snapshots with Rsync](#expose-the-snapshots-with-rsync)
- [Create the Service Configuration File](#create-the-service-configuration-file)
- [Start the Service](#start-the-service)
***
## Introduction
To facilitate deploying grid backend services, we provide snapshots to significantly reduce sync time. This can be setup anywhere from scratch. Once all services are synced, one can use the scripts to create snapshots automatically.
To learn how to deploy your own grid stack, read [this section](./grid_deployment_full_vm.md).
## Services
There are 3 grid backend services that collect enough data to justify creating snapshots:
- ThreeFold blockchain - TFChain
- Graphql - Indexer
- Graphql - Processor
## ThreeFold Public Snapshots
ThreeFold hosts all available snapshots at: [https://bknd.snapshot.grid.tf/](https://bknd.snapshot.grid.tf/). Those snapshots can be downloaded with rsync:
- Mainnet:
```
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshots/tfchain-mainnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshots/indexer-mainnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshots/processor-mainnet-latest.tar.gz .
```
- Testnet:
```
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotstest/tfchain-testnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotstest/indexer-testnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotstest/processor-testnet-latest.tar.gz .
```
- Devnet:
```
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotsdev/tfchain-devnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotsdev/indexer-devnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotsdev/processor-devnet-latest.tar.gz .
```
## Requirements
To run your own snapshot backend, you need the following:
- Configuration
- A working docker environment
- 'node key' for the TFchain public RPC node, generated with `subkey generate-node-key`
Hardware
- min of 8 modern CPU cores
- min of 32GB RAM
- min of 1TB SSD storage (high preference for NVMe based storage), preferably more (as the chain keeps growing in size)
- min of 2TB HDD storage (to store and share the snapshots)
Dev, QA and Testnet can do with a Sata SSD setup. Mainnet requires NVMe based SSDs due to the data size.
**Note**: If a deployment does not have enough disk input/output operations per second (iops) available, you might see the processor container restarting regulary and grid_proxy errors regarding processor database timeouts.
### Files for Each Net
Each folder contains the required deployment files for its net. Make sure to work in the folder that has the name of the network you want to create snapshots for.
What does each file do:
- `.env` - contains environment files maintaned by Threefold Tech
- `.gitignore` - has a list of files to ignore once the repo has been cloned. This has the purpose to not have uncommited changes to files when working in this repo
- `.secrets.env-examples` - is where you have to add all your unique environment variables
- `create_snapshot.sh` - script to create a snapshot (used by cron)
- `docker-compose.yml` - has all the required docker-compose configuration to deploy a working Grid stack
- `open_logs_tmux.sh` - opens all the docker logs in tmux sessions
- `typesBundle.json` - contains data for the Graphql indexer and is not to be touched
- `startall.sh` - starts all the (already deployed) containers
- `stopall.sh` - stops all the (already deployed) containers
### Deploy All 3 Network Instances
To deploy the 3 network instances, mainnet, testnet and mainnet, you need to follow the same process for each network on a separate machine or at least on a different VM.
This means that you can either deploy each network instance on 3 different machines, or you can also deploy 3 different VMs on the same machine, e.g. a dedicated node. Then, each VM will run a different network instance. In this case, you will certainly need a machine with NVME storage disk and modern hardware.
## Deploy a Snapshot Backend
Here's how to deploy a snapshot backend of a given network.
- Go to the corresponding network folder (e.g. `mainnet`).
```sh
cd mainnet
cp .secrets.env-example .secrets.env
```
- Open `.secrets.env` and add your generated subkey node-key.
- Check that all environment variables are correct.
```
docker compose --env-file .secrets.env --env-file .env config
```
- Deploy the snapshot backend. Depending on the disk iops available, it can take up until a week to sync from block 0.
```sh
docker compose --env-file .secrets.env --env-file .env up -d
```
## Deploy the Services with Scripts
You can deploy the 3 individual services using known methods such as [Docker](../../system_administrators/computer_it_basics/docker_basics.md). To facilitate the process, scripts are provided that run the necessary docker commands.
The first script creates the snapshots, while the second and third scripts serve to start and stop all services.
You can use the start script to start all services and then set a cron job to execute periodically the snapshot creation script. This will ensure that you always have the latest version available on your server.
### Create the Snapshots
You can set a cron job to execute a script running rsync to create the snapshots and generate logs at a given interval.
- First download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/create_snapshot.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/create_snapshot.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/create_snapshot.sh
```
- Set the permissions of the script
```
chmod +x create_snapshot.sh
```
- Make sure to a adjust the snapshot creation script for your specific deployment
- Set a cron job
```
crontab -e
```
- Here is an example of a cron job where we execute the script every day at 1 AM and send the logs to `/var/log/snapshots/snapshots-cron.log`.
```sh
0 1 * * * sh /root/code/grid_deployment/grid-snapshots/mainnet/create_snapshot.sh > /var/log/snapshots/snapshots-cron.log 2>&1
```
### Start All the Services
You can start all services by running the provided scripts.
- Download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/startall.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/startall.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/startall.sh
```
- Set the permissions of the script
```
chmod +x startall.sh
```
- Run the script to start all services via docker engine.
```
./startall.sh
```
### Stop All the Services
You can stop all services by running the provided scripts.
- Download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/stopall.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/stopall.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/stopall.sh
```
- Set the permissions of the script
```
chmod +x stopall.sh
```
- Run the script to stop all services via docker engine.
```
./stopall.sh
```
## Expose the Snapshots with Rsync
We use rsync with a systemd service to expose the snapshots to the community.
### Create the Service Configuration File
To setup a public rsync server, create and edit the following file:
`/etc/rsyncd.conf`
```sh
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsync.log
port = 34873
max connections = 20
exclude = lost+found/
transfer logging = yes
use chroot = yes
reverse lookup = no
[gridsnapshots]
path = /storage/rsync-public/mainnet
comment = THREEFOLD GRID MAINNET SNAPSHOTS
read only = true
timeout = 300
list = false
[gridsnapshotstest]
path = /storage/rsync-public/testnet
comment = THREEFOLD GRID TESTNET SNAPSHOTS
read only = true
timeout = 300
list = false
[gridsnapshotsdev]
path = /storage/rsync-public/devnet
comment = THREEFOLD GRID DEVNET SNAPSHOTS
read only = true
timeout = 300
list = false
```
### Start the Service
Start and enable via systemd:
```sh
systemctl start rsync
systemctl enable rsync
systemctl status rsync
```

206
collections/developers/grid_deployment/snapshots_archive.md Normal file
View File

@@ -0,0 +1,206 @@
<h1>Snapshots for Grid Backend Services</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Services](#services)
- [ThreeFold Public Snapshots](#threefold-public-snapshots)
- [Deploy the Services with Scripts](#deploy-the-services-with-scripts)
- [Start All the Services](#start-all-the-services)
- [Stop All the Services](#stop-all-the-services)
- [Create the Snapshots](#create-the-snapshots)
- [Expose the Snapshots with Rsync](#expose-the-snapshots-with-rsync)
- [Create the Service Configuration File](#create-the-service-configuration-file)
- [Start the Service](#start-the-service)
***
## Introduction
To facilitate deploying grid backend services, we provide snapshots to significantly reduce sync time. This can be setup anywhere from scratch. Once all services are synced, one can use the scripts to create snapshots automatically.
## Prerequisites
There are a few prerequisites to properly run the ThreeFold services.
- [Docker engine](../computer_it_basics/docker_basics.md#install-docker-desktop-and-docker-engine)
- [Rsync](../computer_it_basics/file_transfer.md#rsync)
## Services
There are 3 grid backend services that collect enough data to justify creating snapshots:
- ThreeFold blockchain - TFChain
- Graphql - Indexer
- Graphql - Processor
## ThreeFold Public Snapshots
ThreeFold hosts all available snapshots at: [https://bknd.snapshot.grid.tf/](https://bknd.snapshot.grid.tf/). Those snapshots can be downloaded with rsync:
- Mainnet:
```
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshots/tfchain-mainnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshots/indexer-mainnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshots/processor-mainnet-latest.tar.gz .
```
- Testnet:
```
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotstest/tfchain-testnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotstest/indexer-testnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotstest/processor-testnet-latest.tar.gz .
```
- Devnet:
```
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotsdev/tfchain-devnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotsdev/indexer-devnet-latest.tar.gz .
rsync -Lv --progress --partial rsync://bknd.snapshot.grid.tf:34873/gridsnapshotsdev/processor-devnet-latest.tar.gz .
```
Let's now see how to use those snapshots to run the services via scripts.
## Deploy the Services with Scripts
You can deploy the 3 individual services using known methods such as [Docker](https://manual.grid.tf/computer_it_basics/docker_basics.html). To facilitate the process, scripts are provided that run the necessary docker commands.
The first script creates the snapshots, while the second and third scripts serve to start and stop all services.
You can use the start script to start all services and then set a cron job to execute periodically the snapshot creation script. This will ensure that you always have the latest version available on your server.
### Start All the Services
You can start all services by running the provided scripts.
- Download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/startall.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/startall.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/startall.sh
```
- Set the permissions of the script
```
chmod +x startall.sh
```
- Run the script to start all services via docker engine.
```
./startall.sh
```
### Stop All the Services
You can stop all services by running the provided scripts.
- Download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/stopall.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/stopall.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/stopall.sh
```
- Set the permissions of the script
```
chmod +x stopall.sh
```
- Run the script to stop all services via docker engine.
```
./stopall.sh
```
### Create the Snapshots
You can set a cron job to execute a script running rsync to create the snapshots and generate logs at a given interval.
- First download the script.
- Main net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/mainnet/create_snapshot.sh
```
- Test net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/testnet/create_snapshot.sh
```
- Dev net
```
wget https://github.com/threefoldtech/grid_deployment/blob/development/grid-snapshots/devnet/create_snapshot.sh
```
- Set the permissions of the script
```
chmod +x create_snapshot.sh
```
- Make sure to a adjust the snapshot creation script for your specific deployment
- Set a cron job
```
crontab -e
```
- Here is an example of a cron job where we execute the script every day at 1 AM and send the logs to `/var/log/snapshots/snapshots-cron.log`.
```sh
0 1 * * * sh /opt/snapshots/create-snapshot.sh > /var/log/snapshots/snapshots-cron.log 2>&1
```
## Expose the Snapshots with Rsync
We use rsync with a systemd service to expose the snapshots to the community.
### Create the Service Configuration File
To setup a public rsync server, create and edit the following file:
`/etc/rsyncd.conf`
```sh
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsync.log
port = 34873
max connections = 20
exclude = lost+found/
transfer logging = yes
use chroot = yes
reverse lookup = no
[gridsnapshots]
path = /storage/rsync-public/mainnet
comment = THREEFOLD GRID MAINNET SNAPSHOTS
read only = true
timeout = 300
list = false
[gridsnapshotstest]
path = /storage/rsync-public/testnet
comment = THREEFOLD GRID TESTNET SNAPSHOTS
read only = true
timeout = 300
list = false
[gridsnapshotsdev]
path = /storage/rsync-public/devnet
comment = THREEFOLD GRID DEVNET SNAPSHOTS
read only = true
timeout = 300
list = false
```
### Start the Service
Start and enable via systemd:
```sh
systemctl start rsync
systemctl enable rsync
systemctl status rsync
```
If you're interested about hosting your own instance of the grid to strenghten the ThreeFold ecosystem, make sure to read the next section, [Guardians of the Grid](./tfgrid_guardians.md).

32
collections/developers/grid_deployment/tfgrid_stacks.md Normal file
View File

@@ -0,0 +1,32 @@
<h1> TFGrid Stacks </h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Advantages](#advantages)
- [Run Your Own Stack](#run-your-own-stack)
***
## Introduction
ThreeFold is an open-source project and anyone can run the full stack of the TFGrid in a totally decentralized manner. In practice, this means that anyone can grab a docker compose file shared by ThreeFold of the TFGrid stacks and run an instance of the grid services on their own domain.
This means that you could host your own instance of the ThreeFold Dashboard at `dashboard.yourdomain.com` that would serve your own instance of the complete TFGrid stack! Users could then access the ThreeFold Dashboard via your own domain.
The process is actually very straightforward and we even provide a script to streamline the process.
## Advantages
Setting such instances of the TFGrid ensures resiliency and decentralization of the ThreeFold ecosystem.
As a very concrete example, image that one instance of the Dashboard goes offline, `dashboard.grid.tf`, then users could still access the Dashboard from another instance. The more users of the TFGrid deploy their own instance, the more resilient the grid becomes.
The overall ThreeFold ecosystem becomes more resilient to failures of individual nodes.
## Run Your Own Stack
To set your own instance of the TFGrid, you can download a snapshot of the grid and deploy the TFGrid services with Docker. We even provide scripts to quicken the whole process!
Read more about snapshots in the [next section](./grid_deployment_full_vm.md).