Merge pull request 'updated sync with info_grid' (#140) from development_update_new into development

Reviewed-on: #140

books/manual/SUMMARY.md

@@ -10,11 +10,11 @@
       - [Pricing Calculator](dashboard/deploy/pricing_calculator.md)
       - [Node Finder](dashboard/deploy/node_finder.md)
       - [Virtual Machines](dashboard/solutions/vm_intro.md)
-        - [Micro and Full VM Differences ](dashboard/solutions/vm_differences.md)
         - [Full Virtual Machine](dashboard/solutions/fullvm.md)
         - [Micro Virtual Machine](dashboard/solutions/microvm.md)
         - [Nixos MicroVM](dashboard/solutions/nixos_micro.md)
-        - [Add a Domain](dashboard/solutions/add_domain.md)
+        - [Micro and Full VM Differences ](dashboard/solutions/vm_differences.md)
+        - [Add a Domain to a VM](dashboard/solutions/add_domain.md)
       - [Orchestrators](dashboard/deploy/orchestrators.md)
         - [Kubernetes](dashboard/solutions/k8s.md)
         - [Caprover](dashboard/solutions/caprover.md)
@@ -161,6 +161,7 @@
       - [TFGrid Stacks](developers/grid_deployment/tfgrid_stacks.md)
       - [Full VM Grid Deployment](developers/grid_deployment/grid_deployment_full_vm.md)
       - [Grid Snapshots](developers/grid_deployment/snapshots.md)
+      - [Z-OS Boot Generator](developers/grid_deployment/zos_boot_generator/zos_boot_generator.md)
       - [Deploy the Dashboard](developers/grid_deployment/deploy_dashboard.md)
       - [Application Case Study: Nextcloud](developers/grid_deployment/app_case_study_nextcloud.md)
   - [Farmers](farmers/farmers.md)
@@ -174,6 +175,7 @@
     - [Farming Requirements](farmers/farming_requirements.md)
     - [Farming Optimization](farmers/farming_optimization/farming_optimization.md)
       - [GPU Farming](farmers/3node_building/gpu_farming.md)
+      - [HDD-Only Nodes](farmers/farming_optimization/hdd_only_nodes.md)
       - [Set Additional Fees](farmers/farming_optimization/set_additional_fees.md)
       - [Minting Receipts](farmers/3node_building/minting_receipts.md)
       - [Minting Periods](farmers/farming_optimization/minting_periods.md)
@@ -255,6 +257,9 @@
       - [CLI and Scripts Basics](system_administrators/computer_it_basics/cli_scripts_basics.md)
       - [Docker Basics](system_administrators/computer_it_basics/docker_basics.md)
       - [Git and GitHub Basics](system_administrators/computer_it_basics/git_github_basics.md)
+      - [Gitea](system_administrators/computer_it_basics/gitea/gitea_toc.md)
+        - [Gitea Basics](system_administrators/computer_it_basics/gitea/gitea_basics.md)
+        - [Gitea API](system_administrators/computer_it_basics/gitea/gitea_api.md)
       - [Firewall Basics](system_administrators/computer_it_basics/firewall_basics/firewall_basics.md)
         - [UFW Basics](system_administrators/computer_it_basics/firewall_basics/ufw_basics.md)
         - [Firewalld Basics](system_administrators/computer_it_basics/firewall_basics/firewalld_basics.md)
@@ -281,6 +286,7 @@
         - [nopCommerce](system_administrators/advanced/ecommerce/nopcommerce.md)
       - [HTTPS with Caddy](system_administrators/advanced/https_caddy.md)
       - [Node Status Bot](system_administrators/advanced/node_status_bot.md)
+      - [Minetest](system_administrators/advanced/minetest.md)
   - [ThreeFold Token](threefold_token/threefold_token.md)
     - [TFT Bridges](threefold_token/tft_bridges/tft_bridges.md)
       - [TFChain-Stellar Bridge](threefold_token/tft_bridges/tfchain_stellar_bridge.md)

collections/collaboration/collaboration_tools/circle_tool.md

@@ -19,7 +19,7 @@
 
 ## Introduction
 
-The [__ThreeFold Circle Tool__](https://circles.threefold.me ) is our own self-hosted (desktop only) project management tool based on [Taiga](https://www.taiga.io/), an open-source project management tool for cross-functional agile. It offers a lot of different project management kits and features such as the scrum board, kanban board, issues management, and many more. 
+The [__ThreeFold Circle Tool__](https://circles.threefold.me ) is our own self-hosted (desktop only) project management tool based on [Taiga](https://taiga.io/), an open-source project management tool for cross-functional agile. It offers a lot of different project management kits and features such as the scrum board, kanban board, issues management, and many more. 
 
 Our teams at ThreeFold use the Circle Tool to self-manage our tasks, thus it is deemed necessary for the new onboarded team members to learn how to use the tool. Unfortunately we only provide the desktop version of the tool at this moment since we normally manage our projects on the computer.
 

collections/dashboard/solutions/add_domain.md

@@ -13,7 +13,7 @@
 
 ## Introduction
 
-We cover the overall process to add a domain to a virtual machine running on the ThreeFold Grid. 
+We cover the overall process to add a domain to a virtual machine running on the ThreeFold Grid. This process works for both micro and full virtual machines. 
 
 ## Preparation
 

collections/dashboard/solutions/nextcloud.md

@@ -16,13 +16,16 @@
   - [TURN](#turn)
   - [Use Talk](#use-talk)
 - [Backups and Updates](#backups-and-updates)
-  - [Create a Backup](#create-a-backup)
+  - [BorgBackup](#borgbackup)
-  - [Automatic Backups and Updates](#automatic-backups-and-updates)
+    - [Create a BorgBackup](#create-a-borgbackup)
+    - [Automatic Borg Backups and Updates](#automatic-borg-backups-and-updates)
+  - [Secondary VM Backup](#secondary-vm-backup)
+    - [SSH Connection](#ssh-connection)
+    - [Secondary VM Backup Script](#secondary-vm-backup-script)
 - [Troubleshooting](#troubleshooting)
   - [Retrieve the Nextcloud AIO Password](#retrieve-the-nextcloud-aio-password)
   - [Access the Nextcloud Interface Page](#access-the-nextcloud-interface-page)
   - [Check the DNS Propagation](#check-the-dns-propagation)
-- [Questions and Feedback](#questions-and-feedback)
 
 ***
 
@@ -62,12 +65,15 @@ If you're not sure and just want the easiest, most affordable option, skip the p
     * **Standard**: {cpu: 2, memory: 8gb, diskSize: 500gb }
     * **Recommended**: {cpu: 4, memory: 16gb, diskSize: 1000gb }
     * Or choose a **Custom** plan
-* If want to reserve a public IPv4 address, click on Network then select **Public IPv4**
+- Choose the network
-* If you want a [dedicated node](node_finder.md#dedicated-nodes) and/or a certified node, select the corresponding option
+   - `Public IPv4` flag gives the virtual machine a Public IPv4
-* Choose the location of the node
+   - `Mycelium` flag gives the virtual machine a Mycelium address
-   * `Country`
+- `Dedicated` flag to retrieve only dedeicated nodes 
-   * `Farm Name`
+- `Certified` flag to retrieve only certified nodes 
-* Select a node 
+- Choose the location of the node
+   - `Region`
+   - `Country`
+   - `Farm Name`
 * If you want to use a custom domain, click on **Custom domain** under **Domain Name** and write your domain name
   * Example: `nextcloudwebsite.com`
 * The **Select gateway** box will be visible whenever a gateway is required. If so, click it and choose a gateway
@@ -156,7 +162,11 @@ Note that the host of the video meeting might need to turn the VPN off before cr
 
 # Backups and Updates
 
-## Create a Backup
+In this section, we cover how to make a BorgBackup on the Nextcloud VM and we also cover how to make a backup of the Nextcloud BorgBackup to a secondary VM for additional redundancy.
+
+## BorgBackup
+
+### Create a BorgBackup
 
 In the section **Backup and restore**, you can set a [BorgBackup](https://www.borgbackup.org/) of your Nextcloud instance.
 
@@ -166,7 +176,7 @@ In the section **Backup and restore**, you can set a [BorgBackup](https://www.bo
   * This will stop all containers, run the backup container and create the backup. 
 * Once the backup is complete, you can click on **Start containers** to restart the Nextcloud instance.
 
-## Automatic Backups and Updates
+### Automatic Borg Backups and Updates
 
 After the first manual backup of your Nextcloud instance is complete, you can set automatic backups and updates.
 
@@ -174,7 +184,88 @@ After the first manual backup of your Nextcloud instance is complete, you can se
 * In the section **Daily backup and automatic updates**, choose a time for your daily backup and click **Submit backup time**.
   * To set automatic updates, make sure that the option **Automatically update all containers, the mastercontainer and on** is selected.
 
+## Secondary VM Backup
 
+To allow for another layer of redundancy, you can set a secondary VM on the grid and make a daily backup from the BorgBackup of your Nextcloud instance to the secondary VM. The following shows how to do this. It is based on the [File Transfer section](system_administrators@@file_transfer) of the manual.
+
+For the following, we take into account that the BorgBackup is located at `/mnt/backup` on the VM running Nextcloud. 
+
+You will need to deploy a full VM on the TFGrid and SSH into this secondary VM.
+
+### SSH Connection
+
+We want to set an SSH connection between the Nextcloud VM and the secondary VM.
+
+* Create SSH key pair on the secondary VM
+```
+ssh-keygen
+```
+* Install openssh-client on the secondary VM
+```
+apt install openssh-client
+```
+* Install openssh-server on the Nextcloud VM
+```
+apt install openssh-server
+```
+* Copy the public key of the secondary VM
+```
+cat ~/.ssh/id_rsa.pub
+```
+* Create the SSH directory on the Nextcloud VM
+```
+mkdir ~/.ssh
+```
+* Add the secondary VM public key in the file **authorized_keys** on the Nextcloud VM
+```
+nano ~/.ssh/authorized_keys
+```
+* Check the openssh-server status
+``` 
+service ssh status
+```
+* As a test, you can check if you can SSH into the Nextcloud VM from the secondary VM
+```
+ssh root@<Nextcloud_VM_IP_Address>
+```
+
+Once the SSH connection is set up, we need to prepare a backup script. This is covered in the next subsection.
+
+### Secondary VM Backup Script
+
+We now cover how to set up a backup script on the secondary VM.
+
+On the secondary VM, do the following:
+
+* Set a directory for your Nextcloud backup
+```
+mkdir -p /root/nextcloud_backup
+```
+* Create the script file
+```
+nano /root/rsync_nextcloud_backup.sh
+```
+* Write the following script. Here the log is saved in the same directory.
+```
+#!/bin/bash
+
+sudo rsync -avz --progress --delete --log-file=/root/nextcloud_backup/rsync_nextcloud_storage.log /root/nextcloud_backup/ root@<Nextcloud_VM_IP_Address>:/mnt/backup
+```
+* Give permission to execute the script
+```
+sudo chmod +x /root/rsync_nextcloud_backup.sh
+```
+* Set a cron job to run the script periodically
+* Open the cron file
+```
+sudo crontab -e
+```
+* At the bottom of the cron file, add the following to run the script everyday. For this example, we set the time at 18:00PM
+```
+0 18 * * * /root/rsync_nextcloud_backup.sh
+```
+
+Once this is set up, this secondary VM will make a daily backup of the Nextcloud BorgBackup. It is a good idea to set this secondary backup to run the script a few hours after the daily BorgBackup.
 
 # Troubleshooting
 
@@ -200,8 +291,3 @@ To access the Nextcloud interface page, follow those stepse
 
 You can check if the DNS records are propagated globally with DNS propagation check services such as [DNS Checker](https://dnschecker.org/). You can use this tool to verify that your domain is properly pointing to the IPv4 address of the VM you deployed on.
 
-
-
-# Questions and Feedback
-
-If you have any questions, you can ask the ThreeFold community for help on the [ThreeFold Forum](http://forum.threefold.io/) or on the [ThreeFold Grid Tester Community](https://t.me/threefoldtesting) on Telegram.

collections/dashboard/solutions/nodepilot.md

@@ -24,7 +24,10 @@ This is a simple instance of upstream [Node Pilot](https://nodepilot.tech).
 
 - Fill in the instance name: it's used to reference the node-pilot in the future.
 
-- Minimum CPU allowed is 8 cores and minimum memory allowed is 8192.
+- Minimum specs:
+  - 1 vcore
+  - 256 MB of memory
+  - 15 GB of storage
 
 - `Dedicated` flag to retrieve only dedeicated nodes 
 - `Certified` flag to retrieve only certified nodes 

collections/dashboard/solutions/taiga.md

@@ -11,7 +11,7 @@
 
 ## Introduction
 
-[Taiga](https://www.taiga.io/) is the project management tool for multi-functional agile teams. It has a rich feature set and at the same time it is very simple to start with through its intuitive user interface.
+[Taiga](https://taiga.io/) is the project management tool for multi-functional agile teams. It has a rich feature set and at the same time it is very simple to start with through its intuitive user interface.
 
 ## Prerequisites
 
@@ -38,7 +38,7 @@
    - `Region`
    - `Country`
    - `Farm Name`
-- Choose the node to deploy the Tiaga instance on
+- Choose the node to deploy the Taiga instance on
 > Or you can select a specific node with manual selection.
 - `Custom Domain` flag lets the user to use a custom domain
 - Choose a gateway node to deploy your Funkwhale instance on.

collections/dashboard/solutions/vm_intro.md

@@ -4,8 +4,8 @@ On the TFGrid, you can deploy both micro and full virtual machines.
 
 <h2> Table of Contents </h2>
 
+- [Full Virtual Machine](fullVm.md)
+- [Micro Virtual Machine](vm.md)
+- [Nixos Micro VM](nixos_micro.md)
 - [Micro and Full VM Differences ](vm_differences.md)
-- [Full Virtual Machine](fullvm.md)
+- [Add a Domain to a VM](add_domain.md)
-- [Micro Virtual Machine](microvm.md)
-- [Nixos MicroVM](nixos_micro.md)
-- [Add a Domain](add_domain.md)

collections/developers/developers.md

@@ -7,86 +7,11 @@ For complementary information on the technology developed by ThreeFold, refer to
 <h2> Table of Contents </h2>
 
 - [Javascript Client](grid3_javascript_readme.md)
-  - [Installation](grid3_javascript_installation.md)
-  - [Loading Client](grid3_javascript_loadclient.md)
-  - [Deploy a VM](grid3_javascript_vm.md)
-  - [Capacity Planning](grid3_javascript_capacity_planning.md)
-  - [Deploy Multiple VMs](grid3_javascript_vms.md)
-  - [Deploy CapRover](grid3_javascript_caprover.md)
-  - [Gateways](grid3_javascript_vm_gateways.md)
-  - [Deploy a Kubernetes Cluster](grid3_javascript_kubernetes.md)
-  - [Deploy a ZDB](grid3_javascript_zdb.md)
-  - [Deploy ZDBs for QSFS](grid3_javascript_qsfs_zdbs.md)
-  - [QSFS](grid3_javascript_qsfs.md)
-  - [Key Value Store](grid3_javascript_kvstore.md)
-  - [VM with Wireguard and Gateway](grid3_wireguard_gateway.md)
-  - [GPU Support](grid3_javascript_gpu_support.md)
 - [Go Client](grid3_go_readme.md)
-  - [Installation](grid3_go_installation.md)
-  - [Loading Client](grid3_go_load_client.md)
-  - [Deploy a VM](grid3_go_vm.md)
-  - [Deploy Multiple VMs](grid3_go_vms.md)
-  - [Deploy Gateways](grid3_go_gateways.md)
-  - [Deploy Kubernetes](grid3_go_kubernetes.md)
-  - [Deploy a QSFS](grid3_go_qsfs.md)
-  - [GPU Support](grid3_go_gpu.md)
 - [TFCMD](tfcmd.md)
-  - [Getting Started](tfcmd_basics.md)
-  - [Deploy a VM](tfcmd_vm.md)
-  - [Deploy Kubernetes](tfcmd_kubernetes.md)
-  - [Deploy ZDB](tfcmd_zdbs.md)
-  - [Gateway FQDN](tfcmd_gateway_fqdn.md)
-  - [Gateway Name](tfcmd_gateway_name.md)
-  - [Contracts](tfcmd_contracts.md)
 - [TFROBOT](tfrobot.md)
-  - [Installation](tfrobot_installation.md)
-  - [Configuration File](tfrobot_config.md)
-  - [Deployment](tfrobot_deploy.md)
-  - [Commands and Flags](tfrobot_commands_flags.md)
-  - [Supported Configurations](tfrobot_configurations.md)
 - [ThreeFold Chain](tfchain.md)
-  - [Introduction](introduction.md)
-  - [Farming Policies](farming_policies.md)
-  - [External Service Contract](tfchain_external_service_contract.md)
-  - [Solution Provider](tfchain_solution_provider.md)
 - [Grid Proxy](proxy_readme.md)
-  - [Introducing Grid Proxy](proxy.md)
-  - [Setup](setup.md)
-  - [DB Testing](db_testing.md)
-  - [Commands](commands.md)
-  - [Contributions](contributions.md)
-  - [Explorer](explorer.md)
-  - [Database](database.md)
-  - [Production](production.md)
-  - [Release](release.md)
 - [Flist](flist.md)
-  - [ThreeFold Hub Intro](zos_hub.md)
-  - [Generate an API Token](api_token.md)
-  - [Convert Docker Image Into Flist](convert_docker_image.md)
-  - [Supported Flists](grid3_supported_flists.md)
-  - [Flist Case Studies](flist_case_studies.md)
-    - [Case Study: Debian 12](flist_debian_case_study.md)
-    - [Case Study: Nextcloud AIO](flist_nextcloud_case_study.md)
 - [Internals](internals.md)
-  - [Reliable Message Bus (RMB)](rmb_toc.md)
+- [Grid Deployment](grid_deployment.md)
-    - [Introduction to RMB](rmb_intro.md)
-    - [RMB Specs](rmb_specs.md)
-    - [RMB Peer](peer.md)
-    - [RMB Relay](relay.md)
-  - [ZOS](index.md)
-    - [Manual](manual.md)
-    - [Workload Types](workload_types.md)
-    - [Internal Modules](zos_internals.md)
-    - [Capacity](capacity.md)
-    - [Performance Monitor Package](performance.md)
-      - [Public IPs Validation Task](publicips.md)
-      - [CPUBenchmark](cpubench.md)
-      - [IPerf](iperf.md)
-      - [Health Check](healthcheck.md)
-    - [API](api.md)
-- [Grid Deployment](grid_deployment.md)
-  - [TFGrid Stacks](tfgrid_stacks.md)
-  - [Full VM Grid Deployment](grid_deployment_full_vm.md)
-  - [Grid Snapshots](snapshots.md)
-  - [Deploy the Dashboard](deploy_dashboard.md)
-  - [Application Case Study: Nextcloud](app_case_study_nextcloud.md)

collections/developers/grid_deployment/zos_boot_generator/zos_boot_generator.md

@@ -0,0 +1,200 @@
+
+<h1>Zero-OS Boot Generator</h1>
+
+<h2>Table of Contents</h2>
+
+- [Introduction](#introduction)
+- [Set the Environment](#set-the-environment)
+  - [With Docker Ubuntu 22.04](#with-docker-ubuntu-2204)
+  - [With Ubuntu 22.04 Micro VM on TFGrid](#with-ubuntu-2204-micro-vm-on-tfgrid)
+- [Prepare the Machine](#prepare-the-machine)
+- [Set a DNS A Record](#set-a-dns-a-record)
+- [Set a Firewall](#set-a-firewall)
+- [Set HTTPS with Caddy](#set-https-with-caddy)
+  - [Manually](#manually)
+  - [With Zinit](#with-zinit)
+- [Run the Development Server](#run-the-development-server)
+- [Visit the Boot Generator](#visit-the-boot-generator)
+
+---
+
+## Introduction
+
+We cover how to deploy the development server of the Zero-OS Boot Generator Assistant.
+
+Visit the [0-bootstrap repo](https://github.com/threefoldtech/0-bootstrap) for more information.
+
+## Set the Environment
+
+There are many ways to set your environment. Here we show with Docker and a micro VM.
+
+Using Docker should only be used as a test before deploying on a micro VM with IPv4 and IPv6.
+
+### With Docker Ubuntu 22.04
+
+- Deploy Ubuntu 22.04 (Jammy) with Docker
+
+```
+sudo docker pull ubuntu:jammy
+sudo docker run -it ubuntu:jammy /bin/bash
+```
+
+### With Ubuntu 22.04 Micro VM on TFGrid
+
+- Deploy an Ubuntu 22.04 micro VM on the [Dashboard](https://dashboard.grid.tf/)
+  - Set IPv4 and IPv6 as `Network`
+- SSH into the VM
+  - It is recommended to use VSCodium Explorer to facilitate the file management and editing
+
+## Prepare the Machine
+
+Set the machine to deploy the server
+
+- Set the machine in i386
+```
+dpkg --add-architecture i386
+```
+- Update the packages
+```
+apt update
+```
+- Install python3-flask
+```
+echo "2" | apt install -y python3-flask
+```
+- Install the prerequisites
+```
+apt install -y mtools syslinux isolinux libc6-dev-i386 libc6-dbg:i386 git wget genisoimage liblzma-dev build-essential sqlite3 nano
+```
+
+## Set a DNS A Record
+
+Set a DNS A Record pointing to the server hosting the micro VM.
+
+* Go to your domain name registrar
+  * In the section **Advanced DNS**, add a **DNS A Record** to your domain and link it to the IP address of the VM you deployed on:
+    * Type: A Record
+    * Host: @
+    * Value: <IPv4_Address>
+    * TTL: Automatic
+  * It might take up to 30 minutes to set the DNS properly.
+  * To check if the A record has been registered, you can use a common DNS checker:
+    * ```
+      https://dnschecker.org/#A/example.com
+      ```
+
+## Set a Firewall
+
+We set a firewall.
+
+- Install ufw
+```
+apt install -y ufw
+```
+- Set the ports
+```
+ufw allow 80
+ufw allow 443
+ufw allow 22
+```
+- Enable and see the status
+```
+ufw enable
+ufw status
+```
+
+## Set HTTPS with Caddy
+
+We set HTTPS with Caddy. First, we test manually, then we set a zinit service.
+
+### Manually
+
+- Install Caddy
+```
+apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
+curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
+curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' > /etc/apt/sources.list.d/caddy-stable.list
+apt update
+apt install caddy
+```
+
+- Start Caddy
+```
+caddy reverse-proxy -r --from example.com --to :5555
+```
+
+You can also set Caddy to run with zinit, as shown below.
+
+### With Zinit
+
+We manage Caddy with zinit.
+
+- Open the file for editing
+```bash
+nano /etc/zinit/caddy.yaml
+```
+- Insert the following line with your own domain and save the file
+```
+exec: caddy reverse-proxy -r --from example.com --to :80
+```
+- Add the new Caddy file to zinit
+```bash
+zinit monitor caddy
+```
+
+Zinit will start up Caddy immediately, restart it if it ever crashes, and start it up automatically after any reboots. Assuming you tested the Caddy invocation above and used the same form here, that should be all there is to it. 
+
+Here are some other Zinit commands that could be helpful to troubleshoot issues:
+
+- See status of all services (same as "zinit list")
+```
+zinit
+```
+- Get logs for a service
+```
+zinit log caddy
+```
+- Restart a service (to test configuration changes, for example)
+```
+zinit stop caddy
+zinit start caddy
+```
+
+Now that we set the domain and HTTPS, let's deploy the development server with Python.
+
+## Run the Development Server
+
+We show how to deploy the Boot Generator with Python.
+
+
+- Clone the repository  
+```
+git clone https://github.com/threefoldtech/0-bootstrap
+cd 0-bootstrap
+```
+- Copy the sample file to config.py and add the proper info (e.g. set domain)
+```
+cp config.py.sample config.py
+```
+- In config.py, set the proper info, e.g. replace `http://default.tld` with your own domain, e.g. `https://example.com`
+```
+sed -i 's/http:\/\/default\.tld/https:\/\/example\.com/g' config.py
+```
+- Create the database
+```
+cat db/schema.sql | sqlite3 db/bootstrap.sqlite3
+```
+- Run the template script
+```
+bash setup/template.sh
+```
+- Run the development server with Python
+```
+python3 bootstrap.py
+```
+
+## Visit the Boot Generator
+
+You can now access the boot generator on `https://example.com`.
+
+![](./img/zos_boot_generator_main.png)

collections/farmers/3node_building/2_bootstrap_image.md

@@ -24,6 +24,8 @@
 
 We will now learn how to create a Zero-OS bootstrap image in order to boot a DIY 3Node.
 
+If you are building an HDD-only node, read [this section](hdd_only_nodes.md) too.
+
 ## Download the Zero-OS Bootstrap Image
 
 Let's download the Zero-OS bootstrap image.

collections/farmers/3node_building/3_set_hardware.md

@@ -31,6 +31,8 @@ The essential information are available in the section [3Node Requirements Summa
 
 You need a theoretical minimum of 500 GB of SSD and 2 GB of RAM on a mini pc, desktop or server. In short, for peak optimization, aim for 100 GB of SSD and 8GB of RAM per thread. (Thread is equivalent to virtual core or logical core.)
 
+If you are building an HDD-only node, read [this section](hdd_only_nodes.md) too.
+
 Also, TFDAO might implement a farming parameter based on [passmark](https://www.cpubenchmark.net/cpu_list.php). From the ongoing discussion on the Forum, you should aim at a CPU mark of 1000 and above per core.
 
 > 3Node optimal farming hardware ratio -> 100 GB of SSD + 8 GB of RAM per Virtual Core
@@ -58,6 +60,7 @@ Any computer with the following specifications can be used as a DIY 3Node.
 
 *A passmark of 1000 per core is recommend and will be a minimum requirement in the future. This is not yet an official requirement. A 3Node with less than 1000 passmark per core of CPU would not be penalized if it is registered before the DAO settles the [Passmark Question](https://forum.threefold.io/t/cpu-benchmarking-for-reward-calculations/2479).
 
+> Note: It is also possible to build an HDD-only node. Read more about this [here](hdd_only_nodes.md).
 
 
 ## Bandwidth Requirements

collections/farmers/farming_optimization/farming_optimization.md

@@ -6,6 +6,7 @@ The section [Build a 3Node](3node_building.md) covered the notions necessary to
 
 - [GPU Farming](gpu_farming.md)
 - [Set Additional Fees](set_additional_fees.md)
+- [HDD-Only Nodes](hdd_only_nodes.ms)
 - [Minting Receipts](minting_receipts.md)
 - [Minting Periods](minting_periods.md)
 - [Room Parameters](farm_room_parameters.md)

collections/farmers/farming_optimization/hdd_only_nodes.md

@@ -0,0 +1,37 @@
+<h1> HDD-Only Nodes</h1>
+
+<h2> Table of Contents </h2>
+
+- [Introduction](#introduction)
+- [Requirements](#requirements)
+- [BIOS Parameters](#bios-parameters)
+- [Bootstrap Image for HDD-Only Nodes](#bootstrap-image-for-hdd-only-nodes)
+
+---
+
+## Introduction
+
+It is possible to create an HDD-only node. This means that this node doesn't have SSD disks, but only HDD disks.
+
+The steps are almost all the same, except for the creation of the Zero-OS Bootstrap image, which needs an extra kernel argument.
+
+The requirements and BIOS parameters differences are also covered.
+
+## Requirements
+
+An HDD-only node needs at least 100GB of HDD disk.
+
+## BIOS Parameters
+
+In the BIOS, make sure to set `AHCI` for SATA disks.
+
+## Bootstrap Image for HDD-Only Nodes
+
+To create an HDD-only node, the bootstrap image needs an extra kernel argument:
+
+- Navigate to the [Zero-OS Bootstrap site](https://bootstrap.grid.tf/).
+- Select `Expert mode` from the menu at the top.
+- Fill in all the needed inputs (Farm ID, network, kernel and format).
+  - You can keep the default kernel
+- In the `Extra kernel arguments` input field, type `missing-ssd`.
+- Generate and download the image.

collections/system_administrators/advanced/minetest.md

@@ -0,0 +1,142 @@
+<h1>Minetest on the TFGrid</h1>
+
+<h2>Table of Contents</h2>
+
+- [Introduction](#introduction)
+- [Prerequisites](#prerequisites)
+- [Deploy a Full VM](#deploy-a-full-vm)
+- [DNS Settings](#dns-settings)
+  - [DNS Verification](#dns-verification)
+- [Run a Minetest Server on the TFGrid](#run-a-minetest-server-on-the-tfgrid)
+- [Run Minetest Locally](#run-minetest-locally)
+- [Access the Server from Minetest](#access-the-server-from-minetest)
+- [Public Server Notes](#public-server-notes)
+- [References](#references)
+
+---
+
+## Introduction
+
+We show how to deploy a Minetest server on the TFGrid, install Minetest on a local machine and access the Minetest server from the local machine.
+
+![](./img/minetest_1.png)
+
+## Prerequisites
+
+- [A TFChain account](dashboard@@wallet_connector)
+- TFT in your TFChain account
+  - [Buy TFT](threefold_token@@buy_sell_tft)
+  - [Send TFT to TFChain](threefold_token@@tft_bridges)
+
+## Deploy a Full VM
+
+We start by deploying a full VM on the ThreeFold Dashboard.
+
+* On the [Threefold Dashboard](https://dashboard.grid.tf/#/), go to the [full virtual machine deployment page](https://dashboard.grid.tf/#/deploy/virtual-machines/full-virtual-machine/)
+* Deploy a full VM (Ubuntu 22.04) with an IPv4 address and at least the minimum specs for a full VM
+  * Network
+    * Set IPv4 Address (required)
+    * Set Wireguard (optional)
+  * Minimum vcores: 1vcore
+  * Minimum MB of RAM: 512MB
+  * Minimum storage: 15GB
+* After deployment, note the VM IPv4 address
+* SSH into the VM
+    ```
+    ssh root@vm_ipv4_address
+    ```
+
+
+## DNS Settings
+
+You need to set an A record for the IPv4 address.
+
+The following table explicitly shows how to set the A record for your domain. Note that we set `minetest` as a subdomain.
+
+| Type | Host | Value          |
+| ---- | ---- | -------------- |
+| A    | minetest   | <ipv4_address> |
+
+
+You will then be able to access your server on Minetest with the URL `minetest.example.com:30033`. More on the port below.
+
+### DNS Verification
+
+You can use tools such as [DNSChecker](https://dnschecker.org/) or [dig](https://linux.die.net/man/1/dig) on a terminal to check if the DNS propagadation is complete.
+
+
+- Check that the subdomain resolves to the IPv4 address of the full VM
+    ```
+    minetest.example.com
+    ```
+
+## Run a Minetest Server on the TFGrid
+
+
+- Install the Minetest server
+```
+apt update
+apt install minetest-server -y
+```
+- Set the Minetest server
+```
+nano /etc/minetest/minetest.conf
+```
+- Update the following lines
+```
+server_name = Server Name
+
+server_address = http://minetest.example.com
+
+server_announce = true
+
+port = 30033
+```
+- Run the Minetest server
+```
+minetestserver --worldname world
+```
+
+## Run Minetest Locally
+
+- Update the local machine
+```
+apt update
+```
+- Install Minetest
+```
+apt install flatpak -y
+apt install gnome-software-plugin-flatpak -y
+flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
+```
+- Reboot the machine 
+- Install Minetest
+```
+flatpak install flathub net.minetest.Minetest
+```
+- Run Minetest
+```
+flatpak run net.minetest.Minetest
+```
+
+## Access the Server from Minetest
+
+- On Minetest, go to the server you previously set
+
+![](./img/minetest_2.png)
+
+- You can then access the Minetest world!
+
+![](./img/minetest_3.png)
+
+## Public Server Notes
+
+Once you set the Minetest server publicly, you can see it displayed on https://servers.minetest.net/.
+
+- Example of publicly available server
+
+![](./img/minetest_4.png)
+
+## References
+
+For further information on Minetest, read the [Minetest documentation](https://wiki.minetest.net/Main_Page).

collections/system_administrators/computer_it_basics/computer_it_basics.md

@@ -9,8 +9,7 @@ In this section, tailored specifically for system administrators, we'll delve in
 - [CLI and Scripts Basics](cli_scripts_basics.md)
 - [Docker Basics](docker_basics.md)
 - [Git and GitHub Basics](git_github_basics.md)
+- [Gitea](gitea_toc.md)
 - [Firewall Basics](firewall_basics.md)
-  - [UFW Basics](ufw_basics.md)
-  - [Firewalld Basics](firewalld_basics.md)
 - [File Transfer](file_transfer.md)
 - [Screenshots](screenshots.md)

collections/system_administrators/computer_it_basics/gitea/gitea_api.md

@@ -0,0 +1,98 @@
+<h1>Gitea API</h1>
+
+<h2>Table of Contents</h2>
+
+- [Introduction](#introduction)
+- [Swagger](#swagger)
+- [CLI](#cli)
+- [Swagger and CLI](#swagger-and-cli)
+- [API Authorization](#api-authorization)
+- [References](#references)
+
+---
+
+## Introduction
+
+We show how to use the Gitea API with both Swagger and CLI.
+
+## Swagger
+
+The Gitea API is available with the slugs `/api/swagger#` after your Gitea domain URL.
+
+- For ThreeFold, we use the following Gitea API:
+```
+https://git.ourworld.tf/api/swagger#/
+```
+
+You can access different levels of the Gitea instance:
+
+![](./img/api_overview.png)
+
+You can use the Swagger for different queries. Parameters are expained for each example so it is a fairly intuitive use.
+
+For example, to get the raw content of a file, you can use the following:
+
+- Go to the section `Get a file from a repository`.
+
+- In this example, we query the tfgrid organization and its repository `info_tfgrid` for the branch `main`. We write the path of the file.
+
+![](./img/api_example_file.png)
+
+Once you tried a query on the API, you can take the curl command example to query the same information from the CLI. We check this in the next section.
+
+## CLI
+
+To use the API with a command line interface, you can use curl.
+
+For example, we take the raw content of a file:
+
+```
+curl -X 'GET' \
+  'https://git.ourworld.tf/api/v1/repos/tfgrid/info_tfgrid/raw/heroscript%2Fduniayetu%2Fbook_collections.md?ref=main' \
+  -H 'accept: application/json'
+```
+
+We can save this in a file by adding `> filename.txt` after the line above:
+
+```
+curl -X 'GET' \
+  'https://git.ourworld.tf/api/v1/repos/tfgrid/info_tfgrid/raw/heroscript%2Fduniayetu%2Fbook_collections.md?ref=main' \
+  -H 'accept: application/json' > filename.txt
+```
+
+## Swagger and CLI
+
+A combination of the Swagger and the CLI can be very effective.
+
+First, find the proper `curl` command by using the Gitea Swagger, then use the CLI to pass the `curl` commands.
+
+## API Authorization
+
+If you want to query information that is access-protected, such as a private repository, you first need to create a Gitea access token and then log into the Gitea API with the token.
+
+Once this is done, every Swagger request you ask will be accompanied with the access token.
+
+- Create an access token directly on Gitea
+  - Go to the `Applications` section of the `User Settings` [on Gitea](https://git.ourworld.tf/user/settings/applications)
+  - Under `Manage Access Tokens`, choose a `Token Name`
+  - Select either ̀`Public only` or `All (public, private and limited)`. For private access, choose the latter
+  - You must select at least one permission to create a token. Click on `Select permissions` and choose the permissions for your token
+  - Click on `Generate token`
+  - Copy the generate token and keep it somewhere safe
+
+![](./img/gitea_token.png)
+
+- Log in the Gitea API with your access token
+  - On the top of the Gitea API page, click on `Authorize`
+
+<p align="center">
+  <img src="./img/gitea_authorize_token.png" />
+</p>
+
+  - In `AuthorizationHeaderToken (apiKey)` write `token` followed by a space and then paste your token, e.g.:
+    - `token 1234567890`
+- Once you're logged in with your token, you can make queries with the API
+
+## References
+
+You can learn more by reading the [Gitea API documentation](https://docs.gitea.com/development/api-usage).

collections/system_administrators/computer_it_basics/gitea/gitea_basics.md

@@ -0,0 +1,194 @@
+<h1>Gitea Basics</h1>
+
+<h2>Table of Contents</h2>
+
+- [Introduction](#introduction)
+- [Prerequisites](#prerequisites)
+- [Register](#register)
+- [Sign In](#sign-in)
+- [Set an SSH Key Pair](#set-an-ssh-key-pair)
+- [Start the SSH Agent](#start-the-ssh-agent)
+- [Clone a Repository](#clone-a-repository)
+- [Create and Push a New Branch](#create-and-push-a-new-branch)
+  - [On the Browser and Git](#on-the-browser-and-git)
+  - [Only With Git](#only-with-git)
+- [Create a Pull Request](#create-a-pull-request)
+  - [Create a Draft Pull Request](#create-a-draft-pull-request)
+- [Useful Commands](#useful-commands)
+- [Gitea References](#gitea-references)
+
+---
+
+## Introduction
+
+We present a basic guide for Gitea, a forge software package for hosting software development version control using Git as well as other collaborative features like bug tracking, code review, continuous integration, kanban boards, tickets, and wikis.
+
+ThreeFold hosts its own git server on [Gitea OurWorld](https://git.ourworld.tf). We will be using OurWorld's Gitea instance for this guide.
+
+## Prerequisites
+
+You should have git installed on your computer to work easily with Gitea. It is not necessary to use Gitea but recommended to work with files.
+
+- Install [git](../git_github_basics.md#install-git)
+  - Optional, to update repositories from your local computer
+- Install [VSCodium](../git_github_basics.md#vs-codium)
+  - Optional, to edit your files
+- Install [OpenSSH](../../getstarted/ssh_guide/ssh_openssh.md)
+  - Optional, to use SSH
+
+## Register
+
+- Go to the main gitea URL, e.g. [https://git.ourworld.tf](https://git.ourworld.tf)
+- On the top right click on `Register`
+- Set your credentials
+  - Enter a `Username`
+  - Enter your `Email Address`
+  - Enter and confirm a `Password`
+- Click on `Register Account`
+
+![](./img/gitea_register.png)
+
+## Sign In
+
+- Go to the main Gitea URL, e.g. [https://git.ourworld.tf](https://git.ourworld.tf)
+- On the top right click on `Sign In`
+- Set your credentials
+  - Enter your `Username` or your `Email Address`
+  - Enter your `Password`
+- Click on `Sign In`
+
+![](./img/gitea_sign_in.png)
+
+## Set an SSH Key Pair
+
+- Generate an SSH key pair. You can leave default option `~/.ssh/id_rsa.pub`
+```
+ssh-keygen
+```
+- Note the public key
+```
+cat ~/.ssh/id_rsa.pub
+```
+- [Add the SSH public key on git.ourworld.tf](https://git.ourworld.tf/user/settings/keys)
+  - Click on `Add Key` on the top right
+  - Paste the public key in `Content`
+  - The `Key Name` will be added automatically
+  - Click on `Add key` under `Content`
+
+![](./img/gitea_ssh_key.png)
+
+- Optional: on git.ourworld.tf, verify the key. Make sure to put the proper token.
+```
+echo -n 'token' | ssh-keygen -Y sign -n gitea -f ~/.ssh/id_rsa
+```
+
+## Start the SSH Agent
+
+- Start the ssh agent
+```
+eval $(ssh-agent)
+```
+- Add private key to the agent
+```
+ssh-add ~/.ssh/id_rsa.pub
+```
+
+## Clone a Repository
+
+- Clone a repository and change directory
+```
+git clone <repository_url>
+cd <repository_url>
+```
+
+## Create and Push a New Branch
+
+### On the Browser and Git
+
+- On the repo of the Gitea instance, click on the ̀`Branch` icon (e.g. `development`)
+- Write a branch name with the `development_branch_name` convention
+- Click `Create branch`
+
+![](./img/gitea_new_branch.png)
+
+```
+git checkout development
+git fetch
+git pull
+```
+- Make changes in the files
+- When changes are done, make a new branch, add the new files, commit and push to the remote server
+```
+git add .
+git commit -m "Commit message for new branch"
+git push
+```
+
+### Only With Git
+
+- Start by working on the latest development release
+```
+git checkout development
+git fetch
+git pull
+```
+- Make changes in the files
+- When changes are done, make a new branch, add the new files, commit and push to the remote server
+```
+git checkout -b development_new_branch
+git add .
+git commit -m "Commit message for new branch"
+git push --set-upstream origin development_new_branch
+```
+
+## Create a Pull Request
+
+- Go to the repo of the Gitea instance
+- Click on `Pull Request` then `New Pull Request`
+
+![](./img/gitea_new_pr.png)
+
+- Choose the branch you want to merge into development for `pull from` then click `New Pull Request`
+  - At ThreeFold, we protect the master/main and development branches and always make Pull Request from `development_branch_name` to `development`
+
+![](./img/gitea_pr_create.png)
+
+- On the next page, choose a `Title` and a `Description`, then click `Create Pull Request`
+  - Usually, we write at least any related issue and a brief description of the work done
+
+![](./img/gitea_pr_title_description.png)
+
+- At any time you can close your own Pull Request
+
+![](./img/gitea_close_pr.png)
+
+### Create a Draft Pull Request
+
+To create a draft pull request, you simply need to add `WIP:` before the title of your pull request. This will set the pull request in Draft mode.
+
+- Example of a draft pull request:
+```
+WIP: New Pull Request in Draft Mode
+```
+
+## Useful Commands
+
+- See current branch
+```
+git branch
+```
+- See all branches locally
+```
+git branch -r
+```
+- See the git status
+```
+git status
+```
+
+## Gitea References
+
+There are great Gitea references available online.
+
+- [Gitea Docs](https://docs.gitea.com/)
+- [Gitea Cheat Sheet](https://docs.gitea.com/administration/config-cheat-sheet)

collections/system_administrators/computer_it_basics/gitea/gitea_toc.md

@@ -0,0 +1,6 @@
+# Gitea
+
+## Table of Contents
+
+- [Gitea Basics](./gitea_basics.md)
+- [Gitea API](./gitea_api.md)

collections/system_administrators/system_administrators.md

@@ -7,89 +7,11 @@ For complementary information on ThreeFold grid and its cloud component, refer t
 <h2> Table of Contents </h2>
 
 - [Getting Started](tfgrid3_getstarted.md)
-  - [SSH Remote Connection](ssh_guide.md)
-      - [SSH with OpenSSH](ssh_openssh.md)
-      - [SSH with PuTTY](ssh_putty.md)
-      - [SSH with WSL](ssh_wsl.md)
-      - [WireGuard Access](ssh_wireguard.md)
-  - [Remote Desktop and GUI](remote_desktop_gui.md)
-      - [Cockpit: a Web-based Interface for Servers](cockpit_guide.md)
-      - [XRDP: an Open-Source Remote Desktop Protocol](xrdp_guide.md)
-      - [Apache Guacamole: a Clientless Remote Desktop Gateway](guacamole_guide.md)
 - [Planetary Network](planetarynetwork.md)
 - [TFGrid Services](tf_grid_services_readme.md)
 - [GPU](gpu_toc.md)
-  - [GPU Support](gpu.md)
 - [Terraform](terraform_toc.md)
-  - [Overview](terraform_readme.md)
-  - [Installing Terraform](terraform_install.md)
-  - [Terraform Basics](terraform_basics.md)
-  - [Full VM Deployment](terraform_full_vm.md)
-  - [GPU Support](terraform_gpu_support.md)
-  - [Resources](terraform_resources_readme.md)
-    - [Using Scheduler](terraform_scheduler.md)
-    - [Virtual Machine](terraform_vm.md)
-    - [Web Gateway](terraform_vm_gateway.md)
-    - [Kubernetes Cluster](terraform_k8s.md)
-    - [ZDB](terraform_zdb.md)
-    - [Zlogs](terraform_zlogs.md)
-    - [Quantum Safe Filesystem](terraform_qsfs.md)
-      - [QSFS on Micro VM](terraform_qsfs_on_microvm.md)
-      - [QSFS on Full VM](terraform_qsfs_on_full_vm.md)
-    - [CapRover](terraform_caprover.md)
-  - [Advanced](terraform_advanced_readme.md)
-    - [Terraform Provider](terraform_provider.md)
-    - [Terraform Provisioners](terraform_provisioners.md)
-    - [Mounts](terraform_mounts.md)
-    - [Capacity Planning](terraform_capacity_planning.md)
-    - [Updates](terraform_updates.md)
-    - [SSH Connection with Wireguard](terraform_wireguard_ssh.md)
-    - [Set a Wireguard VPN](terraform_wireguard_vpn.md)
-    - [Synced MariaDB Databases](terraform_mariadb_synced_databases.md)
-    - [Nomad](terraform_nomad.md)
-    - [Nextcloud Deployments](terraform_nextcloud_toc.md)
-      - [Nextcloud All-in-One Deployment](terraform_nextcloud_aio.md)
-      - [Nextcloud Single Deployment](terraform_nextcloud_single.md)
-      - [Nextcloud Redundant Deployment](terraform_nextcloud_redundant.md)
-      - [Nextcloud 2-Node VPN Deployment](terraform_nextcloud_vpn.md)
 - [Pulumi](pulumi_readme.md)
-  - [Introduction to Pulumi](pulumi_intro.md)
-  - [Installing Pulumi](pulumi_install.md)
-  - [Deployment Examples](pulumi_examples.md)
-  - [Deployment Details](pulumi_deployment_details.md)
 - [Mycelium](mycelium_toc.md)
-    - [Overview](overview.md)
-    - [Installation](installation.md)
-    - [Additional Information](information.md)
-    - [Message](message.md)
-    - [Packet](packet.md)
-    - [Data Packet](data_packet.md)
-    - [API YAML](api_yaml.md)
 - [Computer and IT Basics](computer_it_basics.md)
-  - [CLI and Scripts Basics](cli_scripts_basics.md)
+- [Advanced](advanced.md)
-  - [Docker Basics](docker_basics.md)
-  - [Git and GitHub Basics](git_github_basics.md)
-  - [Firewall Basics](firewall_basics.md)
-      - [UFW Basics](ufw_basics.md)
-      - [Firewalld Basics](firewalld_basics.md)
-  - [File Transfer](file_transfer.md)
-  - [Screenshots](screenshots.md)
-- [Advanced](advanced.md)
-  - [Token Transfer Keygenerator](token_transfer_keygenerator.md)
-  - [Cancel Contracts](cancel_contracts.md)
-  - [Contract Bills Reports](contract_bill_report.md)
-  - [Listing Free Public IPs](list_public_ips.md)
-  - [Cloud Console](cloud_console.md)
-  - [Redis](grid3_redis.md)
-  - [IPFS](ipfs_toc.md)
-      - [IPFS on a Full VM](ipfs_fullvm.md)
-      - [IPFS on a Micro VM](ipfs_microvm.md)
-  - [MinIO Operator with Helm3](minio_helm3.md) 
-  - [Hummingbot](hummingbot.md)
-  - [AI & ML Workloads](ai_ml_workloads.md)
-  - [Ecommerce](ecommerce.md)
-    - [WooCommerce](woocommerce.md)
-    - [nopCommerce](nopcommerce.md)
-  - [Hummingbot](hummingbot.md)
-  - [HTTPS with Caddy](https_caddy.md)
-  - [Node Status Bot](node_status_bot.md)

collections/system_administrators/terraform/terraform_readme.md

@@ -13,6 +13,7 @@ In this section, we'll embark on a journey to explore the powerful capabilities
 - [Get Started](#get-started)
 - [Features](#features)
 - [What is Not Supported](#what-is-not-supported)
+- [OpenTofu: Alternative to Terraform](#opentofu-alternative-to-terraform)
 
 ***
 
@@ -43,3 +44,7 @@ Threefold loves Open Source! In v3.0 we are integrating one of the most popular
 ## What is Not Supported
 
 - we don't support updates/upgrades, if you want a change you need to destroy a deployment & re-create your deployment this in case you want to change the current running instances properties or change the node, but adding a vm to an existing deployment this shouldn't affect other running vm and same if we need to decommission a vm from a deployment this also shouldn't affect the others
+
+## OpenTofu: Alternative to Terraform
+
+[OpenTofu](https://opentofu.org/) is a fully open-source Terraform fork that is backward compatible with all prior versions of Terraform up to version 1.6. This alternative can be used instead of Terraform for the following sections. You might need to make changes depending on the version you are working with. Check the [OpenTofu Docs](https://opentofu.org/docs/) for more information.