manual transfer done for documentation, still hero issues for parsing
@@ -0,0 +1,88 @@
|
||||
<h1> 1. Create a Farm </h1>
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Create a TFChain Account](#create-a-tfchain-account)
|
||||
- [Create a Farm](#create-a-farm)
|
||||
- [Create a ThreeFold Connect Wallet](#create-a-threefold-connect-wallet)
|
||||
- [Add a Stellar Address for Payout](#add-a-stellar-address-for-payout)
|
||||
- [Farming Rewards Distribution](#farming-rewards-distribution)
|
||||
- [More Information](#more-information)
|
||||
|
||||
***
|
||||
|
||||
## Introduction
|
||||
|
||||
We cover the basic steps to create a farm with the ThreeFold Dashboard. We also create a TFConnect app wallet to receive the farming rewards.
|
||||
|
||||
## Create a TFChain Account
|
||||
|
||||
We create a TFChain account using the ThreeFold Dashboard.
|
||||
|
||||
Go to the [ThreeFold Dashboard](https://dashboard.grid.tf/), click on **Create Account**, choose a password and click **Connect**.
|
||||
|
||||

|
||||
|
||||
Once your profile gets activated, you should find your Twin ID and Address generated under your Mnemonics for verification. Also, your Account Balance will be available at the top right corner under your profile name.
|
||||
|
||||

|
||||
|
||||
## Create a Farm
|
||||
|
||||
We create a farm using the dashboard.
|
||||
|
||||
In the left-side menu, select **Farms** -> **Your Farms**.
|
||||
|
||||

|
||||
|
||||
Click on **Create Farm**, choose a farm name and then click **Create**.
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
## Create a ThreeFold Connect Wallet
|
||||
|
||||
Your farming rewards should be sent to a Stellar wallet with a TFT trustline enabled. The simplest way to proceed is to create a TF Connect app wallet as the TFT trustline is enabled by default on this wallet. For more information on TF Connect, read [this section](../../threefold_token/storing_tft/tf_connect_app.md).
|
||||
|
||||
Let's create a TFConnect Wallet and take note of the wallet address. First, download the app.
|
||||
|
||||
This app is available for [Android](https://play.google.com/store/apps/details?id=org.jimber.threebotlogin&hl=en&gl=US) and [iOS](https://apps.apple.com/us/app/threefold-connect/id1459845885).
|
||||
|
||||
- Note that for Android phones, you need at minimum Android Nougat, the 8.0 software version.
|
||||
- Note that for iOS phones, you need at minimum iOS 14.5. It will be soon available to iOS 13.
|
||||
|
||||
Open the app, click **SIGN UP**, choose a ThreeFold Connect Id, write your email address, take note of the seed phrase and choose a pin. Once this is done, you will have to verify your email address. Check your email inbox.
|
||||
|
||||
In the app menu, click on **Wallet** and then click on **Create Initial Wallet**.
|
||||
|
||||
To find your wallet address, click on the **circled i** icon at the bottom of the screen.
|
||||
|
||||

|
||||
|
||||
Click on the button next to your Stellar address to copy the address.
|
||||
|
||||

|
||||
|
||||
You will need the TF Connect wallet address for the next section.
|
||||
|
||||
> Note: Make sure to keep your TF Connect Id and seed phrase in a secure place offline. You will need these two components to recover your account if you lose access.
|
||||
|
||||
## Add a Stellar Address for Payout
|
||||
|
||||
In the **Your Farms** section of the dashboard, click on **Add/Edit Stellar Payout Address**.
|
||||
|
||||

|
||||
|
||||
Paste your Stellar wallet address and click **Submit**.
|
||||
|
||||

|
||||
|
||||
### Farming Rewards Distribution
|
||||
|
||||
Farming rewards will be sent to your farming wallet around the 8th of each month. This can vary depending on the situation. The minting is done automatically by code and verified by humans as a double check.
|
||||
|
||||
## More Information
|
||||
|
||||
For more information, such as setting IP addresses, you can consult the [Dashboard Farms section](../../dashboard/farms/farms.md).
|
@@ -0,0 +1,177 @@
|
||||
<h1> 2. Create a Zero-OS Bootstrap Image </h1>
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Download the Zero-OS Bootstrap Image](#download-the-zero-os-bootstrap-image)
|
||||
- [Burn the Zero-OS Bootstrap Image](#burn-the-zero-os-bootstrap-image)
|
||||
- [CD/DVD BIOS](#cddvd-bios)
|
||||
- [USB Key BIOS+UEFI](#usb-key-biosuefi)
|
||||
- [BalenaEtcher (MAC, Linux, Windows)](#balenaetcher-mac-linux-windows)
|
||||
- [CLI (Linux)](#cli-linux)
|
||||
- [Rufus (Windows)](#rufus-windows)
|
||||
- [Additional Information (Optional)](#additional-information-optional)
|
||||
- [Expert Mode](#expert-mode)
|
||||
- [Use a Specific Kernel](#use-a-specific-kernel)
|
||||
- [Disable GPU](#disable-gpu)
|
||||
- [Bootstrap Image URL](#bootstrap-image-url)
|
||||
- [Zeros-OS Bootstrapping](#zeros-os-bootstrapping)
|
||||
- [Zeros-OS Expert Bootstrap](#zeros-os-expert-bootstrap)
|
||||
|
||||
***
|
||||
|
||||
## Introduction
|
||||
|
||||
We will now learn how to create a Zero-OS bootstrap image in order to boot a DIY 3Node.
|
||||
|
||||
## Download the Zero-OS Bootstrap Image
|
||||
|
||||
Let's download the Zero-OS bootstrap image.
|
||||
|
||||
In the Farms section of the Dashboard, click on **Bootstrap Node Image**
|
||||
|
||||

|
||||
|
||||
or use the direct link [https://v3.bootstrap.grid.tf](https://v3.bootstrap.grid.tf):
|
||||
|
||||
```
|
||||
https://v3.bootstrap.grid.tf
|
||||
```
|
||||
|
||||

|
||||
|
||||
This is the Zero-OS v3 Bootstrapping page.
|
||||
|
||||

|
||||
|
||||
Write your farm ID and choose production mode.
|
||||
|
||||

|
||||
|
||||
If your system is new, you might be able to run the bootstrap in UEFI mode.
|
||||
|
||||

|
||||
|
||||
For older systems, run the bootstrap in BIOS mode. For BIOS CD/DVD, choose **ISO**. For BIOS USB, choose **USB**
|
||||
|
||||
Download the bootstrap image. Next, we will burn the bootstrap image.
|
||||
|
||||
|
||||
|
||||
## Burn the Zero-OS Bootstrap Image
|
||||
|
||||
We show how to burn the Zero-OS bootstrap image. A quick and modern way is to burn the bootstrap image on a USB key.
|
||||
|
||||
### CD/DVD BIOS
|
||||
|
||||
For the BIOS **ISO** image, download the file and burn it on a DVD.
|
||||
|
||||
### USB Key BIOS+UEFI
|
||||
|
||||
There are many ways to burn the bootstrap image on a USB key. The easiest way that works for all operating systems is to use BalenaEtcher. We also provide other methods.
|
||||
|
||||
#### BalenaEtcher (MAC, Linux, Windows)
|
||||
|
||||
For **MAC**, **Linux** and **Windows**, you can use [BalenaEtcher](https://www.balena.io/etcher/) to load/flash the image on a USB stick. This program also formats the USB in the process. This will work for the option **EFI IMG** for UEFI boot, and with the option **USB** for BIOS boot. Simply follow the steps presented to you and make sure you select the bootstrap image file you downloaded previously.
|
||||
|
||||
> Note: There are alternatives to BalenaEtcher (e.g. [usbimager](https://gitlab.com/bztsrc/usbimager/)).
|
||||
|
||||
**General Steps with BalenaEtcher:**
|
||||
|
||||
1. Download BalenaEtcher
|
||||
2. Open BalenaEtcher
|
||||
3. Select **Flash from file**
|
||||
4. Find and select the bootstrap image (with your correct farm ID)
|
||||
5. Select **Target** (your USB key)
|
||||
6. Select **Flash**
|
||||
|
||||
That's it. Now you have a bootstrap image on Zero-OS as a bootable removable media device.
|
||||
|
||||
|
||||
#### CLI (Linux)
|
||||
|
||||
For the BIOS **USB** and the UEFI **EFI IMG** images, you can do the following on Linux:
|
||||
|
||||
sudo dd status=progress if=FILELOCATION.ISO(or .IMG) of=/dev/sd*
|
||||
|
||||
Here the * is to indicate that you must adjust according to your disk. To see your disks, write lsblk in the command window. Make sure you select the proper disk!
|
||||
|
||||
*If you USB key is not new, make sure that you format it before burning the Zero-OS image.
|
||||
|
||||
#### Rufus (Windows)
|
||||
|
||||
For Windows, if you are using the "dd" able image, instead of writing command line, you can use the free USB flashing program called [Rufus](https://sourceforge.net/projects/rufus.mirror/) and it will automatically do this without needing to use the command line. Rufus also formats the boot media in the process.
|
||||
|
||||
## Additional Information (Optional)
|
||||
|
||||
We cover some additional information. Note that the following information is not needed for a basic farm setup.
|
||||
|
||||
### Expert Mode
|
||||
|
||||
You can use the [expert mode](https://v3.bootstrap.grid.tf/expert) to generate specific Zero-OS bootstrap images.
|
||||
|
||||
Along the basic options of the normal bootstrap mode, the expert mode allows farmers to add extra kernel arguments and decide which kernel to use from a vast list of Zero-OS kernels.
|
||||
|
||||
#### Use a Specific Kernel
|
||||
|
||||
You can use the expert mode to choose a specific kernel. Simply set the information you normally use and then select the proper kernel you need in the **Kernel** drop-down list.
|
||||
|
||||

|
||||
|
||||
#### Disable GPU
|
||||
|
||||
You can use the expert mode to disable GPU on your 3Node.
|
||||
|
||||

|
||||
|
||||
In the expert mode of the Zero-OS Bootstrap generator, fill in the following information:
|
||||
|
||||
- Farmer ID
|
||||
- Your current farm ID
|
||||
- Network
|
||||
- The network of your farm
|
||||
- Extra kernel arguments
|
||||
- ```
|
||||
disable-gpu
|
||||
```
|
||||
- Kernel
|
||||
- Leave the default kernel
|
||||
- Format
|
||||
- Choose a bootstrap image format
|
||||
- Click on **Generate**
|
||||
- Click on **Download**
|
||||
|
||||
### Bootstrap Image URL
|
||||
|
||||
In both normal and expert mode, you can use the generated URL to quickly download a Zero-OS bootstrap image based on your farm specific setup.
|
||||
|
||||
Using URLs can be a very quick and efficient way to create new bootstrap images once your familiar with the Zero-OS bootstrap URL template and some potential varations.
|
||||
|
||||
```
|
||||
https://<grid_version>.bootstrap.grid.tf/<image_format>/<network>/<farm_ID>/<arg1>/<arg2>/.../<kernel>
|
||||
```
|
||||
|
||||
Note that the arguments and the kernel are optional.
|
||||
|
||||
The following content will provide some examples.
|
||||
|
||||
#### Zeros-OS Bootstrapping
|
||||
|
||||
On the [main page](https://v3.bootstrap.grid.tf/), once you've written your farm ID and selected a network, you can copy the generated URL of any given image format.
|
||||
|
||||
For example, the following URL is a download link to an **EFI IMG** of the Zero-OS bootstrap image of farm 1 on the main TFGrid v3 network:
|
||||
|
||||
```
|
||||
https://v3.bootstrap.grid.tf/uefimg/prod/1
|
||||
```
|
||||
|
||||
#### Zeros-OS Expert Bootstrap
|
||||
|
||||
You can use the generated sublink at the **Generate step** of the expert mode to get a quick URL to download your bootstrap image.
|
||||
|
||||
- After setting the parameters and arguments, click on **Generate**
|
||||
- Add the **Target** content to the following URL `https://v3.bootstrap.grid.tf`
|
||||
- For example, the following URL sets an **ipxe** script of the Zero-OS bootstrap of farm 1 on the main TFGrid v3 network, with the **disable-gpu** function enabled as an extra kernel argument and a specific kernel:
|
||||
- ```
|
||||
https://v3.bootstrap.grid.tf/ipxe/test/1/disable-gpu/zero-os-development-zos-v3-generic-b8706d390d.efi
|
||||
```
|
@@ -0,0 +1,188 @@
|
||||
<h1> 3. Set the Hardware </h1>
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Hardware Requirements](#hardware-requirements)
|
||||
- [3Node Requirements Summary](#3node-requirements-summary)
|
||||
- [Bandwidth Requirements](#bandwidth-requirements)
|
||||
- [Link to Share Farming Setup](#link-to-share-farming-setup)
|
||||
- [Powering the 3Node](#powering-the-3node)
|
||||
- [Surge Protector](#surge-protector)
|
||||
- [Power Distribution Unit (PDU)](#power-distribution-unit-pdu)
|
||||
- [Uninterrupted Power Supply (UPS)](#uninterrupted-power-supply-ups)
|
||||
- [Generator](#generator)
|
||||
- [Connecting the 3Node to the Internet](#connecting-the-3node-to-the-internet)
|
||||
- [Z-OS and Switches](#z-os-and-switches)
|
||||
- [Using Onboard Storage (3Node Servers)](#using-onboard-storage-3node-servers)
|
||||
- [Upgrading a DIY 3Node](#upgrading-a-diy-3node)
|
||||
|
||||
***
|
||||
|
||||
|
||||
## Introduction
|
||||
|
||||
In this section of the ThreeFold Farmers book, we cover the essential farming requirements when it comes to ThreeFold 3Node hardware.
|
||||
|
||||
The essential information are available in the section [3Node Requirements Summary](#3node-requirements-summary).
|
||||
|
||||
## Hardware Requirements
|
||||
|
||||
|
||||
You need a theoretical minimum of 500 GB of SSD and 2 GB of RAM on a mini pc, desktop or server. In short, for peak optimization, aim for 100 GB of SSD and 8GB of RAM per thread. (Thread is equivalent to virtual core or logical core.)
|
||||
|
||||
Also, TFDAO might implement a farming parameter based on [passmark](https://www.cpubenchmark.net/cpu_list.php). From the ongoing discussion on the Forum, you should aim at a CPU mark of 1000 and above per core.
|
||||
|
||||
> 3Node optimal farming hardware ratio -> 100 GB of SSD + 8 GB of RAM per Virtual Core
|
||||
|
||||
Note that you can run Zero-OS on a Virtual Machine (VM), but you won't farm any TFT from this process. To farm TFT, Zero-OS needs to be on bare metal.
|
||||
|
||||
Also, note that ThreeFold runs its own OS, which is Zero-OS. You thus need to start with completely wiped disks. You cannot farm TFT with Windows, Linux or MAC OS installed on your disks. If you need to use such OS temporarily, boot it in Try mode with a removable media (USB key).
|
||||
|
||||
Note: Once you have the necessary hardware, you need to [create a farm](./1_create_farm.md), [create a Zero-OS bootstrap image](./2_bootstrap_image.md), [wipe your disks](./4_wipe_all_disks.md) and [set the BIOS/UEFI](./5_set_bios_uefi.md) . Then you can [boot your 3Node](./6_boot_3node.md). If you are planning in building a farm in data center, [read this section](../advanced_networking/advanced_networking_toc.md).
|
||||
|
||||
|
||||
|
||||
### 3Node Requirements Summary
|
||||
|
||||
|
||||
|
||||
Any computer with the following specifications can be used as a DIY 3Node.
|
||||
|
||||
- Any 64-bit hardware with an Intel or AMD processor chip.
|
||||
- Servers, desktops and mini computers type hardware are compatible.
|
||||
- A minimum of 500 GB of SSD and a bare minimum of 2 GB of RAM is required.
|
||||
- A ratio of 100GB of SSD and 8GB of RAM per thread is recommended.
|
||||
- A wired ethernet connection is highly recommended to maximize reliability and the ability to farm TFT.
|
||||
- A [passmark](https://www.passmark.com/) of 1000 per core is recommended and will probably be a minimum requirement in the future.
|
||||
|
||||
*A passmark of 1000 per core is recommend and will be a minimum requirement in the future. This is not yet an official requirement. A 3Node with less than 1000 passmark per core of CPU would not be penalized if it is registered before the DAO settles the [Passmark Question](https://forum.threefold.io/t/cpu-benchmarking-for-reward-calculations/2479).
|
||||
|
||||
|
||||
|
||||
## Bandwidth Requirements
|
||||
|
||||
<!---
|
||||
This section should be checked and validated with the TF Team. We can change the constant (here it's 10) if needed. Or use another equation if this one is deemed suboptimal. This equation is an attempt at a synthesis of the discussions we had on the TF Forum.
|
||||
-->
|
||||
|
||||
A 3Node connects to the ThreeFold Grid and transfers information, whether it is in the form of compute, storage or network units (CU, SU, NU respectively). The more resources your 3Nodes offer to the Grid, the more bandwidth will be needed to transfer the additional information. In this section, we cover general guidelines to make sure you have enough bandwidth on the ThreeFold Grid when utilization will be happening.
|
||||
|
||||
Note that the TFDAO will need to discuss and settle on clearer guidelines in the near future. For now, we propose those general guidelines. Being aware of these numbers as you build and scale your ThreeFold farm will set you in the proper direction.
|
||||
|
||||
> **The strict minimum for one Titan is 1 mbps of bandwidth**.
|
||||
|
||||
If you want to expand your ThreeFold farm, you should check the following to make sure your bandwidth will be sufficient when there will be Grid utilization.
|
||||
|
||||
**Bandwidth per 3Node Equation**
|
||||
|
||||
> min Bandwidth per 3Node (mbps) = 10 * max((Total SSD TB / 1 TB),(Total Threads / 8 Threads),(Total GB / 64 GB)) + 10 * (Total HDD TB / 2)
|
||||
|
||||
This equation means that for each TB of HDD you need 5 mbps of bandwidth, and for each TB of SSD, 8 Threads and 64GB of RAM (whichever is higher), you need 10 mbps of bandwidth.
|
||||
|
||||
This means a proper bandwidth for a Titan would be 10 mbps. As stated, 1 mbps is the strict minimum for one Titan.
|
||||
|
||||
|
||||
|
||||
## Link to Share Farming Setup
|
||||
|
||||
|
||||
If you want ideas and suggestions when it comes to building DIY 3Nodes, a good place to start is by checking what other farmers have built. [This post on the Forum](https://forum.threefold.io/t/lets-share-our-farming-setup/286) is a great start. The following section also contains great DIY 3Node ideas.
|
||||
|
||||
## Powering the 3Node
|
||||
|
||||
### Surge Protector
|
||||
|
||||
A surge protector is highly recommended for your farm and your 3Nodes. This ensures your 3Nodes will not overcharge if a power surge happens. Whole-house surge protectors are also an option.
|
||||
|
||||
### Power Distribution Unit (PDU)
|
||||
|
||||
A PDU (power distribution unit) is useful in big server settings in order to manage your wattage and keep track of your power consumption.
|
||||
|
||||
|
||||
### Uninterrupted Power Supply (UPS)
|
||||
|
||||
|
||||
A UPS (uninterrupted power supply) is great for a 3Node if your power goes on and off frequently for short periods of time. This ensures your 3Node does not need to constantly reboot. If your electricity provider is very reliable, a UPS might not be needed, as the small downtime resulting from rare power outages with not exceed the DIY downtime limit*. (95% uptime, 5% downtime = 36 hours per month.) Of course, for greater Grid utilization experience, considering adding a UPS to your ThreeFold farm can be highly beneficial.
|
||||
|
||||
Note: Make sure to have AC Recovery Power set properly so your 3Node goes back online if power shutdowns momentarily. UPS are generally used in data center to make sure people have enough time to do a "graceful" shutdown of the units when power goes off. In the case of 3Nodes, they do not need graceful shutdowns as Zero-OS cannot lose data while functioning. The only way to power down a 3Node is simply to turn it off directly on the machine.
|
||||
|
||||
|
||||
### Generator
|
||||
|
||||
|
||||
A generator will be needed for very large installation with or without an unsteady main power supply.
|
||||
|
||||
|
||||
|
||||
## Connecting the 3Node to the Internet
|
||||
|
||||
As a general consideration, to connect a 3Node to the Internet, you must use an Ethernet cable and set DHCP as a network management protocol. Note that WiFi is not supported with ThreeFold farming.
|
||||
|
||||
The general route from the 3Node to the Internet is the following:
|
||||
|
||||
> 3Node -> Switch (optional) -> Router -> Modem
|
||||
|
||||
Note that most home routers come with a built-in switch to provide multiple Ethernet ports. Using a stand-alone switch is optional, but can come quite handy when farmers have many 3Nodes.
|
||||
|
||||
|
||||
|
||||
### Z-OS and Switches
|
||||
|
||||
Switches can be managed or unmanaged. Managed switches come with managed features made available to the user (typically more of such features on premium models).
|
||||
|
||||
Z-OS can work with both types of switches. As long as there's a router reachable on the other end offering DHCP and a route to the public internet, it's not important what's in between. Generally speaking, switches are more like cables, just part of the pipes that connect devices in a network.
|
||||
|
||||
We present a general overview of the two types of switches.
|
||||
|
||||
**Unmanaged Switches**
|
||||
|
||||
Unmanaged are the most common type and if someone just says "switch" this is probably what they mean. These switches just forward traffic along to its destination in a plug and play manner with no configuration. When a switch is connected to a router, you can think of the additional free ports on the switch as essentially just extra ports on the router. It's a way to expand the available ports and sometimes also avoid running multiple long cables. My nodes are far from my router, so I run a single long ethernet cable to a switch next to the nodes and then use multiple shorter cables to connect from the switch to the nodes.
|
||||
|
||||
**Managed Switches**
|
||||
|
||||
Managed switches have more capabilities than unmanaged switches and they are not very common in home settings (at least not as standalone units). Some of our farmers do use managed switches. These switches offer much more control and also require configuration. They can enable advanced features like virtual LANs to segment the network.
|
||||
|
||||
|
||||
|
||||
## Using Onboard Storage (3Node Servers)
|
||||
|
||||
If your 3Node is based on a server, you can either use PCIe slots and PCIe-NVME adapter to install SSD NVME disk, or you can use the onboard storage.
|
||||
|
||||
Usually, servers use RAID technology for onboard storage. RAID is a technology that has brought resilience and security to the IT industry. But it has some limitations that ThreeFold did not want to get stuck with. ThreeFold developed a different and more efficient way to [store data reliably](https://library.threefold.me/info/threefold#/cloud/threefold__cloud_products?id=storage-quantum-safe-filesystem). This Quantum Safe Storage overcomes some of the shortfalls of RAID and is able to work over multiple nodes geographically spread on the TF Grid. This means that there is no RAID controller in between data storage and the TF Grid.
|
||||
|
||||
For your 3Nodes, you want to bypass RAID in order for Zero-OS to have bare metals on the system.
|
||||
|
||||
To use onboard storage on a server without RAID, you can
|
||||
|
||||
1. [Re-flash](https://fohdeesha.com/docs/perc.html) the RAID card
|
||||
2. Turn on HBA/non-RAID mode
|
||||
3. Install a different card.
|
||||
|
||||
For HP servers, you simply turn on the HBA mode (Host Bus Adapter).
|
||||
|
||||
For Dell servers, you can either cross, or [re-flash](https://fohdeesha.com/docs/perc.html), the RAID controller with an “IT-mode-Firmware” (see this [video](https://www.youtube.com/watch?v=h5nb09VksYw)) or get a DELL H310-controller (which has the non-RAID option). Otherwise, you can install a NVME SSD with a PCIe adaptor, and turn off the RAID controller.
|
||||
|
||||
|
||||
|
||||
Once the disks are wiped, you can shutdown your 3Node and remove the Linux Bootstrap Image (USB key). Usually, there will be a message telling you when to do so.
|
||||
|
||||
|
||||
|
||||
## Upgrading a DIY 3Node
|
||||
|
||||
|
||||
|
||||
As we've seen in the [List of Common DIY 3Nodes](#list-of-common-diy-3nodes), it is sometimes necessary, and often useful, to upgrade your hardware.
|
||||
|
||||
**Type of upgrades possible**
|
||||
|
||||
- Add TBs of SSD/HDD
|
||||
- Add RAM
|
||||
- Change CPU
|
||||
- Change BIOS battery
|
||||
- Change fans
|
||||
|
||||
For some DIY 3Node, no upgrades are required and this constitutes a good start if you want to explore DIY building without going into too much additional steps.
|
||||
|
||||
For in-depth videos on how to upgrade mini-pc and rack servers, watch these great [DIY videos](https://www.youtube.com/user/floridanelson).
|
@@ -0,0 +1,14 @@
|
||||
<h1> Building a DIY 3Node </h1>
|
||||
|
||||
This section of the ThreeFold Farmers book presents the necessary and basic steps to build a DIY 3Node.
|
||||
|
||||
For advanced farming information, such as GPU farming and room parameters, refer to the section [Farming Optimization](../farming_optimization/farming_optimization.md).
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [1. Create a Farm](./1_create_farm.md)
|
||||
- [2. Create a Zero-OS Bootstrap Image](./2_bootstrap_image.md)
|
||||
- [3. Set the Hardware](./3_set_hardware.md)
|
||||
- [4. Wipe All the Disks](./4_wipe_all_disks.md)
|
||||
- [5. Set the BIOS/UEFI](./5_set_bios_uefi.md)
|
||||
- [6. Boot the 3Node](./6_boot_3node.md)
|
@@ -0,0 +1,106 @@
|
||||
<h1> 4. Wipe All the Disks </h1>
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Main Steps](#main-steps)
|
||||
- [1. Create a Linux Bootstrap Image](#1-create-a-linux-bootstrap-image)
|
||||
- [2. Boot Linux in *Try Mode*](#2-boot-linux-in-try-mode)
|
||||
- [3. Use wipefs to Wipe All the Disks](#3-use-wipefs-to-wipe-all-the-disks)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
|
||||
***
|
||||
|
||||
## Introduction
|
||||
|
||||
In this section of the ThreeFold Farmers book, we explain how to wipe all the disks of your 3Node.
|
||||
|
||||
|
||||
|
||||
## Main Steps
|
||||
|
||||
It only takes a few steps to wipe all the disks of a 3Node.
|
||||
|
||||
1. Create a Linux Bootstrap Image
|
||||
2. Boot Linux in *Try Mode*
|
||||
3. Wipe All the Disks
|
||||
|
||||
ThreeFold runs its own OS, which is Zero-OS. You thus need to start with completely wiped disks. Note that ALL disks must be wiped. Otherwise, Zero-OS won't boot.
|
||||
|
||||
An easy method is to simply download a Linux distribution and wipe the disk with the proper command line in the Terminal.
|
||||
|
||||
We will show how to do this with Ubuntu 20.04. LTS. This distribution is easy to use and it is thus a good introduction for Linux, in case you haven't yet explored this great operating system.
|
||||
|
||||
|
||||
|
||||
## 1. Create a Linux Bootstrap Image
|
||||
|
||||
Download the Ubuntu 20.04 ISO file [here](https://releases.ubuntu.com/20.04/) and burn the ISO image on a USB key. Make sure you have enough space on your USB key. You can also use other Linux Distro such as [GRML](https://grml.org/download/), if you want a lighter ISO image.
|
||||
|
||||
The process here is the same as in section [Burning the Bootstrap Image](./2_bootstrap_image.md#burn-the-zero-os-bootstrap-image), but with the Linux ISO instead of the Zero-OS ISO. [BalenaEtcher](https://www.balena.io/etcher/) is recommended as it formats your USB in the process, and it is available for MAC, Windows and Linux.
|
||||
|
||||
|
||||
|
||||
## 2. Boot Linux in *Try Mode*
|
||||
|
||||
When you boot the Linux ISO image, make sure to choose *Try Mode*. Otherwise, it will install Linux on your computer. You do not want this.
|
||||
|
||||
|
||||
|
||||
## 3. Use wipefs to Wipe All the Disks
|
||||
|
||||
When you use wipefs, you are removing all the data on your disk. Make sure you have no important data on your disks, or make sure you have copies of your disks before doing this operation, if needed.
|
||||
|
||||
Once Linux is booted, go into the terminal and write the following command lines.
|
||||
|
||||
First, you can check the available disks by writing in a terminal or in a shell:
|
||||
|
||||
```
|
||||
lsblk
|
||||
```
|
||||
|
||||
To see what disks are connected, write this command:
|
||||
|
||||
```
|
||||
fdisk -l
|
||||
```
|
||||
|
||||
If you want to wipe one specific disk, here we use *sda* as an example, write this command:
|
||||
|
||||
```
|
||||
sudo wipefs -a /dev/sda
|
||||
```
|
||||
|
||||
And replace the "a" in sda by the letter of your disk, as shown when you did *lsblk*. The term *sudo* gives you the correct permission to do this.
|
||||
|
||||
To wipe all the disks in your 3Node, write the command:
|
||||
|
||||
```
|
||||
sudo for i in /dev/sd*; do wipefs -a $i; done
|
||||
```
|
||||
|
||||
If you have any `fdisk` entries that look like `/dev/nvme`, you'll need to adjust the command line.
|
||||
|
||||
For a nvme disk, here we use *nvme0* as an example, write:
|
||||
|
||||
```
|
||||
sudo wipefs -a /dev/nvme0
|
||||
```
|
||||
|
||||
And replace the "0" in nvme0 by the number corresponding to your disk, as shown when you did *lsblk*.
|
||||
|
||||
To wipe all the nvme disks, write this command line:
|
||||
|
||||
```
|
||||
sudo for i in /dev/nvme*; do wipefs -a $i; done
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you're having issues wiping the disks, you might need to use **--force** or **-f** with wipefs (e.g. **sudo wipefs -af /dev/sda**).
|
||||
|
||||
If you're having trouble getting your disks recognized by Zero-OS, some farmers have had success enabling AHCI mode for SATA in their BIOS.
|
||||
|
||||
If you are using a server with onboard storage, you might need to [re-flash the RAID card](../../faq/faq.md#is-there-a-way-to-bypass-raid-in-order-for-zero-os-to-have-bare-metals-on-the-system-no-raid-controller-in-between-storage-and-the-grid).
|
||||
|
||||
|
@@ -0,0 +1,172 @@
|
||||
<h1> 5. Set the BIOS/UEFI </h1>
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Z-OS and DHCP](#z-os-and-dhcp)
|
||||
- [Regular Computer and 3Node Network Differences](#regular-computer-and-3node-network-differences)
|
||||
- [Static IP Addresses](#static-ip-addresses)
|
||||
- [The Essential Features of BIOS/UEFI for a 3Node](#the-essential-features-of-biosuefi-for-a-3node)
|
||||
- [Setting the Remote Management of a Server with a Static IP Address (Optional)](#setting-the-remote-management-of-a-server-with-a-static-ip-address-optional)
|
||||
- [Update the BIOS/UEFI firmware (Optional)](#update-the-biosuefi-firmware-optional)
|
||||
- [Check the BIOS/UEFI version on Windows](#check-the-biosuefi-version-on-windows)
|
||||
- [Check the BIOS/UEFI version on Linux](#check-the-biosuefi-version-on-linux)
|
||||
- [Update the BIOS firmware](#update-the-bios-firmware)
|
||||
- [Additional Information](#additional-information)
|
||||
- [BIOS/UEFI and Zero-OS Bootstrap Image Combinations](#biosuefi-and-zero-os-bootstrap-image-combinations)
|
||||
- [Troubleshoot](#troubleshoot)
|
||||
|
||||
|
||||
***
|
||||
|
||||
## Introduction
|
||||
|
||||
In this section of the ThreeFold Farmers book, we explain how to properly set the BIOS/UEFI of your 3Node.
|
||||
|
||||
Note that the BIOS mode is usually needed for older hardware while the UEFI mode is usually needed for newer hardware, when it comes to booting properly Zero-OS on your DIY 3Node.
|
||||
|
||||
If it doubt, start with UEFI and if it doesn't work as expected, try with BIOS.
|
||||
|
||||
Before diving into the BIOS/UEFI settings, we will present some general considerations on Z-OS and DHCP.
|
||||
|
||||
## Z-OS and DHCP
|
||||
|
||||
The operating system running on the 3Nodes is called Zero-OS (Z-OS). When it comes to setting the proper network for your 3Node farm, you must use DHCP since Z-OS is going to request an IP from the DHCP server if there's one present, and it won't get network connectivity if there's no DHCP.
|
||||
|
||||
The Z-OS philosophy is to minimize configuration wherever possible, so there's nowhere to supply a static config when setting your 3Node network. Instead, the farmer is expected to provide DHCP.
|
||||
|
||||
While it is possible to set fixed IP addresses with the DHCP for the 3Nodes, it is recommended to avoid this and just set the DHCP normally without fixed IP addresses.
|
||||
|
||||
By setting DHCP in BIOS/UEFI, an IP address is automatically assigned by your router to your 3Node every time you boot it.
|
||||
|
||||
### Regular Computer and 3Node Network Differences
|
||||
|
||||
For a regular computer (not a 3Node), if you want to use a static IP in a network with DHCP, you'd first turn off DHCP and then set the static IP to an IP address outside the DHCP range. That being said, with Z-OS, there's no option to turn off DHCP and there's nowhere to set a static IP, besides public config and remote management. In brief, the farmer must provide DHCP, either on a private or a public range, for the 3Node to boot.
|
||||
|
||||
### Static IP Addresses
|
||||
|
||||
In the ThreeFold ecosystem, there are only two situations where you would work with static IP addresses: to set a public config to a 3Node or a farm, and to remotely manage your 3Nodes.
|
||||
|
||||
**Static IP and Public Config**
|
||||
|
||||
You can [set a static IP for the public config of a 3Node or a farm](./1_create_farm.md#optional-add-public-ip-addresses). In thise case, the 3Node takes information from TF Chain and uses it to set a static configuration on a NIC (or on a virtual NIC in the case of single NIC systems).
|
||||
|
||||
**Static IP and Remote Management**
|
||||
|
||||
You can [set a static IP address to remotely manage a 3Node](#setting-the-remote-management-of-a-server-static-ip-address).
|
||||
|
||||
|
||||
|
||||
## The Essential Features of BIOS/UEFI for a 3Node
|
||||
|
||||
There are certain things that you should make sure are set properly on your 3Node.
|
||||
|
||||
As a general advice, you can Load Defaults (Settings) on your BIOS, then make sure the options below are set properly.
|
||||
|
||||
* Choose the correct combination of BIOS/UEFI and bootstrap image on [https://bootstrap.grid.tf/](https://bootstrap.grid.tf/)
|
||||
* Newer system will use UEFI
|
||||
* Older system will use BIOS
|
||||
* Hint: If your 3Node boot stops at *Initializing Network Devices*, try the other method (BIOS or UEFI)
|
||||
* Set Multi-Processor and Hyperthreading at Enabled
|
||||
* Sometimes, it will be written Virtual Cores, or Logical Cores.
|
||||
* Set Virtualization at Enabled
|
||||
* On Intel, it is denoted as CPU virtualization and on ASUS, it is denoted as SVM.
|
||||
* Make sure virtualization is enabled and look for the precise terms in your specific BIOS/UEFI.
|
||||
* Set AC Recovery at Last Power State
|
||||
* This will make sure your 3Node restarts after losing power momentarily.
|
||||
* Select the proper Boot Sequence for the 3Node to boot Zero-OS from your bootstrap image
|
||||
* e.g., if you have a USB key as a bootstrap image, select it in Boot Sequence
|
||||
* Set Server Lookup Method (or the equivalent) at DNS. Only use Static IP if you know what you are doing.
|
||||
* Your router will assign a dynamic IP address to your 3Node when it connects to Internet.
|
||||
* Set Client Address Method (or the equivalent) at DHCP. Only use Static IP if you know what you are doing.
|
||||
* Your router will assign a dynamic IP address to your 3Node when it connects to Internet.
|
||||
* Secure Boot should be left at disabled
|
||||
* Enable it if you know what you are doing. Otherwise, it can be set at disabled.
|
||||
|
||||
|
||||
|
||||
|
||||
## Setting the Remote Management of a Server with a Static IP Address (Optional)
|
||||
|
||||
|
||||
Note from the list above that by enabling the DHCP and DNS in BIOS, dynamic IP addresses will be assigned to 3Nodes. This way, you do not need any specific port configuration when booting a 3Node.
|
||||
|
||||
As long as the 3Node is connected to the Internet via an ethernet cable (WiFi is not supported), Zero-OS will be able to boot. By setting DHCP in BIOS, an IP address is automatically assigned to your 3Node every time you boot it. This section concerns 3Node servers with remote management functions and interfaces.
|
||||
|
||||
You can set up a node through static routing at the router without DHCP by assigning the MAC address of the NIC to a IP address within your private subnet. This will give a static IP address to your 3Node.
|
||||
|
||||
With a static IP address, you can then configure remote management on servers. For Dell, [iDRAC](https://www.dell.com/support/kbdoc/en-us/000134243/how-to-setup-and-manage-your-idrac-or-cmc-for-dell-poweredge-servers-and-blades) is used, and for HP, [ILO](https://support.hpe.com/hpesc/public/docDisplay?docId=a00045463en_us&docLocale=en_US) is used.
|
||||
|
||||
|
||||
|
||||
## Update the BIOS/UEFI firmware (Optional)
|
||||
|
||||
|
||||
Updating the BIOS firmware is not always necessary, but to do so can help prevent future errors and troubleshootings. Making sure the Date and Time are set correctly can also help the booting process.
|
||||
|
||||
Note: updating the BIOS/UEFI firmware is optional, but recommended.
|
||||
|
||||
|
||||
### Check the BIOS/UEFI version on Windows
|
||||
|
||||
Hit *Start*, type in *cmd* in the search box and click on *Command Prompt*. Write the line
|
||||
|
||||
> wmic bios get smbiosbiosversion
|
||||
|
||||
This will give you the BIOS or UEFI firmware of your PC.
|
||||
|
||||
### Check the BIOS/UEFI version on Linux
|
||||
|
||||
Simply type the following command
|
||||
|
||||
> sudo dmidecode | less
|
||||
|
||||
or this line:
|
||||
|
||||
> sudo dmidecode -s bios-version
|
||||
|
||||
### Update the BIOS firmware
|
||||
|
||||
1. On the manufacturer's website, download the latest BIOS/UEFI firmware
|
||||
2. Put the file on a USB flash drive (+unzip if necessary)
|
||||
3. Restart your hardware and enter the BIOS/UEFI settings
|
||||
4. Navigate the menus to update the BIOS/UEFI
|
||||
|
||||
## Additional Information
|
||||
|
||||
### BIOS/UEFI and Zero-OS Bootstrap Image Combinations
|
||||
|
||||
To properly boot the Zero-OS image, you can either use an image made for a BIOS system or a UEFI system, this depends on your system.
|
||||
|
||||
BIOS is older technology. It means *Basic Input/Output System*.
|
||||
|
||||
UEFI is newer technology. It means *Unified Extensible Firmware Interface*. BIOS/UEFI is, in a way, the link between the hardware and the software of your computer.
|
||||
|
||||
In general, setting a 3Node is similar whether it is with a BIOS or UEFI system. The important is to choose the correct combination of boot media and boot mode (BIOS/UEFI).
|
||||
|
||||
The bootstrap images are available [here](https://bootstrap.grid.tf/).
|
||||
|
||||
The choices are:
|
||||
|
||||
1. EFI IMG - UEFI
|
||||
2. EFI FILE - UEFI
|
||||
3. iPXE - Boot from network
|
||||
4. ISO - BIOS
|
||||
5. USB - BIOS
|
||||
6. LKRN - Boot from network
|
||||
|
||||
Choices 1 and 2 are for UEFI (newer models).
|
||||
Choices 4 and 5 are for BIOS (newer models).
|
||||
Choices 3 and 6 are mainly for network boot.
|
||||
|
||||
Refer to [this previous section](./2_bootstrap_image.md) for more information on creating a Zero-OS bootstrap image.
|
||||
|
||||
For information on how to boot Zero-OS with iPXE, read [this section](./6_boot_3node.md#advanced-booting-methods-optional).
|
||||
|
||||
### Troubleshoot
|
||||
|
||||
You might have to try UEFI first and if it doesn't work, try BIOS. Usually when this is the case (UEFI doesn't work with your current computer), the following message will be shown:
|
||||
|
||||
> Initializing Network Devices...
|
||||
|
||||
And then... nothing. This means that you are still in the BIOS of the hardware and boot is not even started yet. When this happens, try the BIOS mode of your computer.
|
@@ -0,0 +1,169 @@
|
||||
<h1> 6. Boot the 3Node </h1>
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [1. Booting the 3Node with Zero-OS](#1-booting-the-3node-with-zero-os)
|
||||
- [2. Check the 3Node Status Online](#2-check-the-3node-status-online)
|
||||
- [3. Receive the Farming Rewards](#3-receive-the-farming-rewards)
|
||||
- [Advanced Booting Methods (Optional)](#advanced-booting-methods-optional)
|
||||
- [PXE Booting with OPNsense](#pxe-booting-with-opnsense)
|
||||
- [PXE Booting with pfSense](#pxe-booting-with-pfsense)
|
||||
- [Booting Issues](#booting-issues)
|
||||
- [Multiple nodes can run with the same node ID](#multiple-nodes-can-run-with-the-same-node-id)
|
||||
|
||||
***
|
||||
|
||||
|
||||
## Introduction
|
||||
|
||||
We explain how to boot the 3Node with the Zero-OS bootstrap image with a USB key. We also include optional advanced booting methods using OPNSense and pfSense.
|
||||
|
||||
One of the great features of Zero-OS is that it can be completely run within the cache of your 3Node. Indeed, the booting device that contains your farm ID will connect to the ThreeFold Grid and download everything needed to run smoothly. There are many benefits in terms of security and protection of data that comes with this.
|
||||
|
||||
## 1. Booting the 3Node with Zero-OS
|
||||
|
||||
To boot Zero-OS, insert your Zero-OS bootstrap image USB key, power on your computer and choose the right booting sequence and parameters ([BIOS or UEFI](./5_set_bios_uefi.md)) in your BIOS/UEFI settings. Then, restart the 3Node. Zero-OS should boot automatically.
|
||||
|
||||
Note that you need an ethernet cable connected to your router or switch. You cannot farm on the ThreeFold Grid with Wifi.
|
||||
|
||||
The first time you boot a 3Node, it will be written: “This node is not registered (farmer : NameOfFarm). This is normal. The Grid will create a node ID and you will be able to see it on screen. This can take a couple of minutes.
|
||||
|
||||
If time passes (an hour and more) and the node does not get registered, in many cases, [wiping the disks](./4_wipe_all_disks.md) all over again and trying another reboot usually resolves this issue.
|
||||
|
||||
Once you have your node ID, you can also go on the ThreeFold Dashboard to see your 3Node and verify that your 3Node is online.
|
||||
|
||||
## 2. Check the 3Node Status Online
|
||||
|
||||
You can use the ThreeFold [Node Finder](../../dashboard/deploy/node_finder.md) to verify that your 3Node is online.
|
||||
|
||||
* [ThreeFold Main Net Dashboard](https://dashboard.grid.tf/)
|
||||
* [ThreeFold Test Net Dashboard](https://dashboard.test.grid.tf/)
|
||||
* [ThreeFold Dev Net Dashboard](https://dashboard.dev.grid.tf/)
|
||||
* [ThreeFold QA Net Dashboard](https://dashboard.qa.grid.tf/)
|
||||
|
||||
|
||||
## 3. Receive the Farming Rewards
|
||||
|
||||
The farming reward will be sent once per month at the address you gave when you set up your farm. You can review this process [here](./1_create_farm.md#add-a-stellar-address-for-payout).
|
||||
|
||||
That's it. You've now completed the necessary steps to build a DIY 3Node and to connect it to the Grid.
|
||||
|
||||
## Advanced Booting Methods (Optional)
|
||||
|
||||
### PXE Booting with OPNsense
|
||||
|
||||
> This documentation comes from the [amazing Network Booting Guide](https://forum.ThreeFold.io/t/network-booting-tutorial/2688) by @Fnelson on the ThreeFold Forum.
|
||||
|
||||
Network booting ditches your standard boot USB with a local server. This TFTP server delivers your boot files to your 3 nodes. This can be useful in bigger home farms, but is all but mandatory in a data center setup.
|
||||
|
||||
Network boot setup is quite easy and is centered about configuring a TFTP server. There are essentially 2 options for this, a small dedicated server such as a raspberry pi, or piggybacking on your pfsense or opnsense router. I would recommend the latter as it eliminates another piece of equipment and is probably more reliable.
|
||||
|
||||
**Setting Up Your Router to Allow Network Booting**
|
||||
|
||||
These steps are for OPNsense, PFsense may differ. These set are required regardless of where you have your TFTP server.
|
||||
|
||||
> Services>DHCPv4>LAN>Network Booting
|
||||
|
||||
Check “Enable Network Booting”
|
||||
|
||||
Enter the IP address of your TFTP server under “Set next-server IP”. This may be the router’s IP or whatever device you are booting from.
|
||||
|
||||
Enter “pxelinux.0” under Set default bios filename.
|
||||
|
||||
Ignore the TFTP Server settings.
|
||||
|
||||
|
||||
**TFTP server setup on a debian machine such as Ubuntu or Raspberry Pi**
|
||||
|
||||
> apt-get update
|
||||
>
|
||||
> apt-get install tftpd-hpa
|
||||
>
|
||||
> cd /srv/tftp/
|
||||
>
|
||||
> wget http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/netboot.tar.gz
|
||||
>
|
||||
> wget http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/pxelinux.0
|
||||
>
|
||||
> wget https://bootstrap.grid.tf/krn/prod/<FARMID> --no-check-certificate
|
||||
>
|
||||
> mv <FARMID> ipxe-prod.lkrn
|
||||
>
|
||||
> tar -xvzf netboot.tar.gz
|
||||
>
|
||||
> rm version.info netboot.tar.gz
|
||||
>
|
||||
> rm pxelinux.cfg/default
|
||||
>
|
||||
> chmod 777 /srv/tftp/pxelinux.cfg (optional if next step fails)
|
||||
>
|
||||
> echo 'default ipxe-prod.lkrn' >> pxelinux.cfg/default
|
||||
|
||||
|
||||
**TFTP Server on a OPNsense router**
|
||||
|
||||
> Note: When using PFsense instead of OPNsense, steps are probably similar, but the directory or other small things may differ.
|
||||
|
||||
The first step is to download the TFTP server plugin. Go to system>firmware>Status and check for updates, follow prompts to install. Then click the Plugins tab and search for tftp, install os-tftp. Once that is installed go to Services>TFTP (you may need to refresh page). Check the Enable box and input your router ip (normally 192.168.1.1). Click save.
|
||||
|
||||
Turn on ssh for your router. In OPNsense it is System>Settings>Administration. Then check the Enable, root login, and password login. Hop over to Putty and connect to your router, normally 192.168.1.1. Login as root and input your password. Hit 8 to enter the shell.
|
||||
|
||||
In OPNsense the tftp directory is /usr/local/tftp
|
||||
|
||||
> cd /usr/local
|
||||
>
|
||||
> mkdir tftp
|
||||
>
|
||||
> cd ./tftp
|
||||
>
|
||||
> fetch http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/netboot.tar.gz
|
||||
>
|
||||
> fetch http://ftp.nl.debian.org/debian/dists/buster/main/installer-amd64/current/images/netboot/pxelinux.0
|
||||
>
|
||||
> fetch https://bootstrap.grid.tf/krn/prod/<FARMID>
|
||||
>
|
||||
> mv <FARMID> ipxe-prod.lkrn
|
||||
>
|
||||
> tar -xvzf netboot.tar.gz
|
||||
>
|
||||
> rm version.info netboot.tar.gz
|
||||
>
|
||||
> rm pxelinux.cfg/default
|
||||
>
|
||||
> echo 'default ipxe-prod.lkrn' >> pxelinux.cfg/default
|
||||
|
||||
You can get out of shell by entering exit or just closing the window.
|
||||
|
||||
**3Node Setup**
|
||||
|
||||
Set the server to BIOS boot and put PXE or network boot as the first choice. At least on Dell machines, make sure you have the network cable in plug 1 or it won’t boot.
|
||||
|
||||
|
||||
|
||||
### PXE Booting with pfSense
|
||||
|
||||
> This documentation comes from the [amazing Network Booting Guide](https://forum.threefold.io/t/network-booting-tutorial/2688/7) by @TheCaptain on the ThreeFold Forum.
|
||||
|
||||
These are the steps required to enable PXE booting on pfSense. This guide assumes you’ll be using the router as your PXE server; pfSense allows boot file uploads directly from its web GUI.
|
||||
|
||||
* Log into your pfSense instance
|
||||
* Go to System>Package Manager
|
||||
* Search and add ‘tftpd’ package under ‘Available Packages’ tab
|
||||
* Go to Services>TFTP Server
|
||||
* Under ‘Settings’ tab check enable and enter the router IP in TFTP Server Bind IP field
|
||||
* Switch to ‘Files’ tab under Services>TFTP Server and upload your ‘ipxe-prod.efi’ file acquired from https://v3.bootstrap.grid.tf/ (second option labeled ‘EFI Kernel’)
|
||||
* Go to Services>DHCP Server
|
||||
* Under ‘Other Options’ section click Display Advance next to ‘TFTP’ and enter router IP
|
||||
* Click Display Advance next to ‘Network Booting’
|
||||
* Check enable, enter router IP in Next Server field
|
||||
* Enter ipxe-prod.efi in Default BIOS file name field
|
||||
|
||||
That's it! You’ll want to ensure your clients are configured with boot priority set as IPv4 in first spot. You might need to disable secure boot and enable legacy boot within BIOS.
|
||||
|
||||
## Booting Issues
|
||||
|
||||
### Multiple nodes can run with the same node ID
|
||||
|
||||
This is a [known issue](https://github.com/threefoldtech/info_grid/issues/122) and will be resolved once the TPM effort gets finalized.
|
||||
|
@@ -0,0 +1,72 @@
|
||||
<h1>GPU Farming</h1>
|
||||
|
||||
Welcome to the *GPU Farming* section of the ThreeFold Manual!
|
||||
|
||||
In this guide, we delve into the realm of GPU farming, shedding light on the significance of Graphics Processing Units (GPUs) and how they can be seamlessly integrated into the ThreeFold ecosystem.
|
||||
|
||||
<h2>Table of Contents</h2>
|
||||
|
||||
- [Understanding GPUs](#understanding-gpus)
|
||||
- [Get Started](#get-started)
|
||||
- [Install the GPU](#install-the-gpu)
|
||||
- [GPU Node and the Farmerbot](#gpu-node-and-the-farmerbot)
|
||||
- [Set a Price for the GPU Node](#set-a-price-for-the-gpu-node)
|
||||
- [Check the GPU Node on the Node Finder](#check-the-gpu-node-on-the-node-finder)
|
||||
- [Reserving the GPU Node](#reserving-the-gpu-node)
|
||||
- [Questions and Feedback](#questions-and-feedback)
|
||||
|
||||
***
|
||||
|
||||
## Understanding GPUs
|
||||
|
||||
A Graphics Processing Unit, or GPU, is a specialized electronic circuit designed to accelerate the rendering of images and videos. Originally developed for graphics-intensive tasks in gaming and multimedia applications, GPUs have evolved into powerful parallel processors with the ability to handle complex computations, such as 3D rendering, AI and machine learning.
|
||||
|
||||
In the context of ThreeFold, GPU farming involves harnessing the computational power of Graphics Processing Units to contribute to the decentralized grid. This empowers users to participate in the network's mission of creating a more equitable and efficient internet infrastructure.
|
||||
|
||||
## Get Started
|
||||
|
||||
In this guide, we focus on the integration of GPUs with a 3Node, the fundamental building block of the ThreeFold Grid. The process involves adding a GPU to enhance the capabilities of your node, providing increased processing power and versatility for a wide range of tasks. Note that any Nvidia or AMD graphics card should work as long as it's supported by the system.
|
||||
|
||||
## Install the GPU
|
||||
|
||||
We cover the basic steps to install the GPU on your 3Node.
|
||||
|
||||
* Find a proper GPU model for your specific 3Node hardware
|
||||
* Install the GPU on the server
|
||||
* Note: You might need to move or remove some pieces of your server to make room for the GPU
|
||||
* (Optional) Boot the 3Node with a Linux distro (e.g. Ubuntu) and use the terminal to check if the GPU is recognized by the system
|
||||
* ```
|
||||
sudo lshw -C Display
|
||||
```
|
||||
* Output example with an AMD Radeon (on the line `product: ...`)
|
||||

|
||||
* Boot the 3Node with the ZOS bootstrap image
|
||||
|
||||
## GPU Node and the Farmerbot
|
||||
|
||||
If you are using the Farmerbot, it might be a good idea to first boot the GPU node without the Farmerbot (i.e. to remove the node in the config file and restart the Farmerbot). Once you've confirmed that the GPU is properly detected by TFChain, you can then put back the GPU node in the config file and restart the Farmerbot. While this is not necessary, it can be an effective way to test the GPU node separately.
|
||||
|
||||
## Set a Price for the GPU Node
|
||||
|
||||
You can [set additional fees](../farming_optimization/set_additional_fees.md) for your GPU dedicated node on the [TF Dashboard](https://dashboard.grid.tf/).
|
||||
|
||||
When a user reserves your 3Node as a dedicated node, you will receive TFT payments once every 24 hours. These TFT payments will be sent to the TFChain account of your farm's twin.
|
||||
|
||||
## Check the GPU Node on the Node Finder
|
||||
|
||||
You can use the [Node Finder](../../dashboard/deploy/node_finder.md) on the [TF Dashboard](https://dashboard.grid.tf/) to verify that the node is displayed as having a GPU.
|
||||
|
||||
* On the Dashboard, go to the Node Finder
|
||||
* Under **Node ID**, write the node ID of the GPU node
|
||||
* Once the results are displayed, you should see **1** under **GPU**
|
||||
* If you are using the Status bot, you might need to change the node status under **Select Nodes Status** (e.g. **Down**, **Standby**) to see the node's information
|
||||
|
||||
> Note: It can take some time for the GPU parameter to be displayed.
|
||||
|
||||
## Reserving the GPU Node
|
||||
|
||||
Now, users can reserve the node in the **Dedicated Nodes** section of the Dashboard and then deploy workloads using the GPU. For more information, read [this documentation](../../dashboard/deploy/dedicated_machines.md).
|
||||
|
||||
## Questions and Feedback
|
||||
|
||||
If you have any questions or feedback, we invite you to discuss with the ThreeFold community on the [ThreeFold Forum](https://forum.threefold.io/) or on the [ThreeFold Farmer chat](https://t.me/threefoldfarmers) on Telegram.
|
@@ -0,0 +1 @@
|
||||
farming_30.png
|
After Width: | Height: | Size: 10 KiB |
After Width: | Height: | Size: 138 KiB |
After Width: | Height: | Size: 40 KiB |
After Width: | Height: | Size: 5.3 KiB |
After Width: | Height: | Size: 25 KiB |
After Width: | Height: | Size: 10 KiB |
After Width: | Height: | Size: 24 KiB |
After Width: | Height: | Size: 14 KiB |
After Width: | Height: | Size: 8.9 KiB |
After Width: | Height: | Size: 8.7 KiB |
After Width: | Height: | Size: 57 KiB |
After Width: | Height: | Size: 53 KiB |
After Width: | Height: | Size: 51 KiB |
After Width: | Height: | Size: 40 KiB |
After Width: | Height: | Size: 28 KiB |
After Width: | Height: | Size: 10 KiB |
After Width: | Height: | Size: 7.3 KiB |
After Width: | Height: | Size: 3.6 KiB |
After Width: | Height: | Size: 1.4 MiB |
After Width: | Height: | Size: 238 KiB |
After Width: | Height: | Size: 33 KiB |
After Width: | Height: | Size: 95 KiB |
After Width: | Height: | Size: 81 KiB |
After Width: | Height: | Size: 124 KiB |
After Width: | Height: | Size: 119 KiB |
After Width: | Height: | Size: 22 KiB |
After Width: | Height: | Size: 10 KiB |
After Width: | Height: | Size: 45 KiB |
After Width: | Height: | Size: 35 KiB |
After Width: | Height: | Size: 27 KiB |
After Width: | Height: | Size: 13 KiB |
After Width: | Height: | Size: 35 KiB |
After Width: | Height: | Size: 79 KiB |
After Width: | Height: | Size: 21 KiB |
After Width: | Height: | Size: 57 KiB |
After Width: | Height: | Size: 25 KiB |
After Width: | Height: | Size: 30 KiB |
After Width: | Height: | Size: 46 KiB |
After Width: | Height: | Size: 57 KiB |
After Width: | Height: | Size: 120 KiB |
After Width: | Height: | Size: 118 KiB |
After Width: | Height: | Size: 36 KiB |
After Width: | Height: | Size: 30 KiB |
After Width: | Height: | Size: 54 KiB |
After Width: | Height: | Size: 23 KiB |
After Width: | Height: | Size: 34 KiB |
@@ -0,0 +1,98 @@
|
||||
<h1>Minting Receipts</h1>
|
||||
|
||||
<h2>Table of Contents</h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Access the Reports](#access-the-reports)
|
||||
- [Available Information](#available-information)
|
||||
|
||||
***
|
||||
|
||||
## Introduction
|
||||
|
||||
Once you have the receipt hash of your node minting, you can get the [minting report](../../dashboard/tfchain/tf_minting_reports.md) of your node.
|
||||
|
||||
## Access the Reports
|
||||
|
||||
- On the Dashboard, go to **TFChain** -> **TF Minting Reports**
|
||||
- Enter your receipt hash
|
||||
- Consult your minting report
|
||||
|
||||
## Available Information
|
||||
|
||||
The ThreeFold Alpha minting tool will present the following information for each minting receipt hash:
|
||||
|
||||
- Node Info
|
||||
- Node ID
|
||||
- Farm Name and ID
|
||||
- Measured Uptime
|
||||
- Node Resources
|
||||
- CU
|
||||
- SU
|
||||
- NU
|
||||
- CRU
|
||||
- MRU
|
||||
- SRU
|
||||
- HRU
|
||||
- TFT Farmed
|
||||
- Payout Address
|
||||
|
||||
<!-- NOTE: This is removed from the new dashboard, but might be brought back.
|
||||
|
||||
## Introduction
|
||||
|
||||
You can easily consult minting receipts of all your 3Nodes on the ThreeFold Dashboard to get essential minting information of your 3Nodes and your ThreeFold farm. With your minting receipt hash, you can then query the ThreeFold Alpha minting tool for further minting information.
|
||||
|
||||
## Download Minting Receipts of Your Farm
|
||||
|
||||
You can download minting receipts of your whole farm directly on the ThreeFold Dashboard.
|
||||
|
||||
- On the [ThreeFold Dashboard](https://dashboard.grid.tf/), go to **TFChain** -> **TF Minting Reports**
|
||||
- In the section **Your Farms**, on the left of your **Farm ID**, click on the down arrow button
|
||||
- Click on **Download Minting Receipts**
|
||||
|
||||
## Download Minting Receipts of a 3Node
|
||||
|
||||
You can download minting receipts of a single 3Node directly on the ThreeFold Dashboard.
|
||||
|
||||
- On the [ThreeFold Dashboard](https://dashboard.grid.tf/), go to **Portal** -> **Farms**
|
||||
- In the section **Your Farm Nodes**, on the left of your **Node ID**, click on the down arrow button
|
||||
- Click on **Node Statistics**
|
||||
- Click on **Download Node Receipt**
|
||||
|
||||
## Minting Receipts Information
|
||||
|
||||
The minting receipts contain the following information:
|
||||
|
||||
- Minting: <minting_receipt_hash>
|
||||
- start: <start of minting period>
|
||||
- end: <end of minting period>
|
||||
- TFT: <TFT minted by the 3Node>
|
||||
- Cloud Units: <3Node Resources>
|
||||
|
||||
## Alpha Minting Tool
|
||||
|
||||
You can query additional minting information by using the [Dashboard Alpha Minting tool](https://dashboard.grid.tf/other/minting).
|
||||
|
||||
- Download the minting receipts of your farm or of a single 3Node
|
||||
- Copy a minting receipt hash
|
||||
- Open the ThreeFold Alpha Minting tool by clicking on **Minting** on the left-side [ThreeFold Dashboard](https://dashboard.grid.tf/) menu
|
||||
- Paste the minting receipt hash
|
||||
|
||||
The ThreeFold Alpha minting tool will present the following information for each minting receipt hash:
|
||||
|
||||
- Node ID
|
||||
- Farm Name
|
||||
- Measured Uptime
|
||||
- Node Resources
|
||||
- CU
|
||||
- SU
|
||||
- NU
|
||||
- CRU
|
||||
- MRU
|
||||
- SRU
|
||||
- HRU
|
||||
- TFT Farmed
|
||||
- Payout Address
|
||||
|
||||
-->
|
@@ -0,0 +1,13 @@
|
||||
<h1> Advanced Networking </h1>
|
||||
|
||||
Welcome to the *Advanced Networking* section of the ThreeFold Manual.
|
||||
|
||||
In this section, we provide advanced networking tips for farms with public IPs and in data centers (DC). We also cover the differences between IPv4 and IPv6 networking.
|
||||
|
||||
<h2>Table of Contents</h2>
|
||||
|
||||
- [Networking Overview](./networking_overview.md)
|
||||
- [Network Considerations](./network_considerations.md)
|
||||
- [Network Setup](./network_setup.md)
|
||||
|
||||
> Note: This documentation does not constitute a complete set of knowledge on setting farms with public IP addresses in a data center. Please make sure to do your own research and communicate with your data center and your Internet service provider for any additional information.
|
@@ -0,0 +1,120 @@
|
||||
<h1>Network Considerations</h1>
|
||||
|
||||
<h2>Table of Contents</h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Running ZOS (v2) at Home](#running-zos-v2-at-home)
|
||||
- [Running ZOS (v2) in a Multi-Node Farm in a DC](#running-zos-v2-in-a-multi-node-farm-in-a-dc)
|
||||
- [Necessities](#necessities)
|
||||
- [IPv6](#ipv6)
|
||||
- [Routing/Firewalling](#routingfirewalling)
|
||||
- [Multi-NIC Nodes](#multi-nic-nodes)
|
||||
- [Farmers and the TFGrid](#farmers-and-the-tfgrid)
|
||||
|
||||
***
|
||||
|
||||
## Introduction
|
||||
|
||||
Running ZOS on a node is just a matter of booting it with a USB stick, or with a dhcp/bootp/tftp server with the right configuration so that the node can start the OS.
|
||||
Once it starts booting, the OS detects the NICs, and starts the network configuration. A Node can only continue it's boot process till the end when it effectively has received an IP address and a route to the Internet. Without that, the Node will retry indefinitely to obtain Internet access and not finish it's startup.
|
||||
|
||||
So a Node needs to be connected to a __wired__ network, providing a dhcp server and a default gateway to the Internet, be it NATed or plainly on the public network, where any route to the Internet, be it IPv4 or IPv6 or both is sufficient.
|
||||
|
||||
For a node to have that ability to host user networks, we **strongly** advise to have a working IPv6 setup, as that is the primary IP stack we're using for the User Network's Mesh to function.
|
||||
|
||||
## Running ZOS (v2) at Home
|
||||
|
||||
Running a ZOS Node at home is plain simple. Connect it to your router, plug it in the network, insert the preconfigured USB stick containing the bootloader and the `farmer_id`, power it on.
|
||||
|
||||
## Running ZOS (v2) in a Multi-Node Farm in a DC
|
||||
|
||||
Multi-Node Farms, where a farmer wants to host the nodes in a data centre, have basically the same simplicity, but the nodes can boot from a boot server that provides for DHCP, and also delivers the iPXE image to load, without the need for a USB stick in every Node.
|
||||
|
||||
A boot server is not really necessary, but it helps! That server has a list of the MAC addresses of the nodes, and delivers the bootloader over PXE. The farmer is responsible to set-up the network, and configure the boot server.
|
||||
|
||||
### Necessities
|
||||
|
||||
The Farmer needs to:
|
||||
|
||||
- Obtain an IPv6 prefix allocation from the provider. A `/64` will do, that is publicly reachable, but a `/48` is advisable if the farmer wants to provide IPv6 transit for User Networks
|
||||
- If IPv6 is not an option, obtain an IPv4 subnet from the provider. At least one IPv4 address per node is needed, where all IP addresses are publicly reachable.
|
||||
- Have the Nodes connected on that public network with a switch so that all Nodes are publicly reachable.
|
||||
- In case of multiple NICS, also make sure his farm is properly registered in BCDB, so that the Node's public IP Addresses are registered.
|
||||
- Properly list the MAC addresses of the Nodes, and configure the DHCP server to provide for an IP address, and in case of multiple NICs also provide for private IP addresses over DHCP per Node.
|
||||
- Make sure that after first boot, the Nodes are reachable.
|
||||
|
||||
### IPv6
|
||||
|
||||
IPv6, although already a real protocol since '98, has seen reluctant adoption over the time it exists. That mostly because ISPs and Carriers were reluctant to deploy it, and not seeing the need since the advent of NAT and private IP space, giving the false impression of security.
|
||||
But this month (10/2019), RIPE sent a mail to all it's LIRs that the last consecutive /22 in IPv4 has been allocated. Needless to say, but that makes the transition to IPv6 in 2019 of utmost importance and necessity.
|
||||
Hence, ZOS starts with IPv6, and IPv4 is merely an afterthought ;-)
|
||||
So in a nutshell: we greatly encourage Farmers to have IPv6 on the Node's network.
|
||||
|
||||
### Routing/Firewalling
|
||||
|
||||
Basically, the Nodes are self-protecting, in the sense that they provide no means at all to be accessed through listening processes at all. No service is active on the node itself, and User Networks function solely on an overlay.
|
||||
That also means that there is no need for a Farm admin to protect the Nodes from exterior access, albeit some DDoS protection might be a good idea.
|
||||
In the first phase we will still allow the Host OS (ZOS) to reply on ICMP ping requests, but that 'feature' might as well be blocked in the future, as once a Node is able to register itself, there is no real need to ever want to try to reach it.
|
||||
|
||||
### Multi-NIC Nodes
|
||||
|
||||
Nodes that Farmers deploy are typically multi-NIC Nodes, where one (typically a 1GBit NIC) can be used for getting a proper DHCP server running from where the Nodes can boot, and one other NIC (1Gbit or even 10GBit), that then is used for transfers of User Data, so that there is a clean separation, and possible injections bogus data is not possible.
|
||||
|
||||
That means that there would be two networks, either by different physical switches, or by port-based VLANs in the switch (if there is only one).
|
||||
|
||||
- Management NICs
|
||||
The Management NIC will be used by ZOS to boot, and register itself to the GRID. Also, all communications from the Node to the Grid happens from there.
|
||||
- Public NICs
|
||||
|
||||
### Farmers and the TFGrid
|
||||
|
||||
A Node, being part of the Grid, has no concept of 'Farmer'. The only relationship for a Node with a Farmer is the fact that that is registered 'somewhere (TM)', and that a such workloads on a Node will be remunerated with Tokens. For the rest, a Node is a wholly stand-alone thing that participates in the Grid.
|
||||
|
||||
```text
|
||||
172.16.1.0/24
|
||||
2a02:1807:1100:10::/64
|
||||
+--------------------------------------+
|
||||
| +--------------+ | +-----------------------+
|
||||
| |Node ZOS | +-------+ | |
|
||||
| | +-------------+1GBit +--------------------+ 1GBit switch |
|
||||
| | | br-zos +-------+ | |
|
||||
| | | | | |
|
||||
| | | | | |
|
||||
| | | | +------------------+----+
|
||||
| +--------------+ | | +-----------+
|
||||
| | OOB Network | | |
|
||||
| | +----------+ ROUTER |
|
||||
| | | |
|
||||
| | | |
|
||||
| | | |
|
||||
| +------------+ | +----------+ |
|
||||
| | Public | | | | |
|
||||
| | container | | | +-----+-----+
|
||||
| | | | | |
|
||||
| | | | | |
|
||||
| +---+--------+ | +-------------------+--------+ |
|
||||
| | | | 10GBit Switch | |
|
||||
| br-pub| +-------+ | | |
|
||||
| +-----+10GBit +-------------------+ | +---------->
|
||||
| +-------+ | | Internet
|
||||
| | | |
|
||||
| | +----------------------------+
|
||||
+--------------------------------------+
|
||||
185.69.167.128/26 Public network
|
||||
2a02:1807:1100:0::/64
|
||||
|
||||
```
|
||||
|
||||
Where the underlay part of the wireguard interfaces get instantiated in the Public container (namespace), and once created these wireguard interfaces get sent into the User Network (Network Resource), where a user can then configure the interface a he sees fit.
|
||||
|
||||
The router of the farmer fulfills 2 roles:
|
||||
|
||||
- NAT everything in the OOB network to the outside, so that nodes can start and register themselves, as well get tasks to execute from the BCDB.
|
||||
- Route the assigned IPv4 subnet and IPv6 public prefix on the public segment, to which the public container is connected.
|
||||
|
||||
As such, in case that the farmer wants to provide IPv4 public access for grid proxies, the node will need at least one (1) IPv4 address. It's free to the farmer to assign IPv4 addresses to only a part of the Nodes.
|
||||
On the other hand, it is quite important to have a proper IPv6 setup, because things will work out better.
|
||||
|
||||
It's the Farmer's task to set up the Router and the switches.
|
||||
|
||||
In a simpler setup (small number of nodes for instance), the farmer could setup a single switch and make 2 port-based VLANs to separate OOB and Public, or even wit single-nic nodes, just put them directly on the public segment, but then he will have to provide a DHCP server on the Public network.
|
@@ -0,0 +1,86 @@
|
||||
<h1>Network Setup</h1>
|
||||
|
||||
<h2>Table of Contents</h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Network Setup for Farmers](#network-setup-for-farmers)
|
||||
- [Step 1. Testing for IPv6 Availability in Your Location](#step-1-testing-for-ipv6-availability-in-your-location)
|
||||
- [Step 2. Choosing the Setup to Connect Your Nodes](#step-2-choosing-the-setup-to-connect-your-nodes)
|
||||
- [2.1 Home Setup](#21-home-setup)
|
||||
- [2.2 Data Center/Advanced Setup](#22-data-centeradvanced-setup)
|
||||
- [General Notes](#general-notes)
|
||||
|
||||
***
|
||||
|
||||
# Introduction
|
||||
|
||||
0-OS nodes participating in the Threefold grid, need connectivity of course. They need to be able to communicate over
|
||||
the Internet with each-other in order to do various things:
|
||||
|
||||
- download its OS modules
|
||||
- perform OS module upgrades
|
||||
- register itself to the grid, and send regular updates about it's status
|
||||
- query the grid for tasks to execute
|
||||
- build and run the Overlay Network
|
||||
- download flists and the effective files to cache
|
||||
|
||||
The nodes themselves can have connectivity in a few different ways:
|
||||
|
||||
- Only have RFC1918 private addresses, connected to the Internet through NAT, NO IPv6
|
||||
Mostly, these are single-NIC (Network card) machines that can host some workloads through the Overlay Network, but
|
||||
cant't expose services directly. These are HIDDEN nodes, and are mostly booted with an USB stick from
|
||||
bootstrap.grid.tf .
|
||||
- Dual-stacked: having RFC1918 private IPv4 and public IPv6 , where the IPv6 addresses are received from a home router,
|
||||
but firewalled for outgoing traffic only. These nodes are effectively also HIDDEN
|
||||
- Nodes with 2 NICs, one that has effectively a NIC connected to a segment that has real public
|
||||
addresses (IPv4 and/or IPv6) and one NIC that is used for booting and local
|
||||
management. (OOB) (like in the drawing for farmer setup)
|
||||
|
||||
For Farmers, we need to have Nodes to be reachable over IPv6, so that the nodes can:
|
||||
|
||||
- expose services to be proxied into containers/vms
|
||||
- act as aggregating nodes for Overlay Networks for HIDDEN Nodes
|
||||
|
||||
Some Nodes in Farms should also have a publicly reachable IPv4, to make sure that clients that only have IPv4 can
|
||||
effectively reach exposed services.
|
||||
|
||||
But we need to stress the importance of IPv6 availability when you're running a multi-node farm in a datacentre: as the
|
||||
grid is boldly claiming to be a new Internet, we should make sure we adhere to the new protocols that are future-proof.
|
||||
Hence: IPv6 is the base, and IPv4 is just there to accomodate the transition.
|
||||
|
||||
Nowadays, RIPE can't even hand out consecutive /22 IPv4 blocks any more for new LIRs, so you'll be bound to market to
|
||||
get IPv4, mostly at rates of 10-15 Euro per IP. Things tend to get costly that way.
|
||||
|
||||
So anyway, IPv6 is not an afterthought in 0-OS, we're starting with it.
|
||||
|
||||
# Network Setup for Farmers
|
||||
|
||||
This is a quick manual to what is needed for connecting a node with zero-OS V2.0
|
||||
|
||||
## Step 1. Testing for IPv6 Availability in Your Location
|
||||
As descibed above the network in which the node is instaleld has to be IPv6 enabled. This is not an afterthought as we are building a new internet it has to ba based on the new and forward looking IP addressing scheme. This is something you have to investigate, negotiate with you connectivity provider. Many (but not all home connectivity products and certainly most datacenters can provide you with IPv6. There are many sources of infromation on how to test and check whether your connection is IPv6 enabled, [here is a starting point](http://www.ipv6enabled.org/ipv6_enabled/ipv6_enable.php)
|
||||
|
||||
## Step 2. Choosing the Setup to Connect Your Nodes
|
||||
|
||||
Once you have established that you have IPv6 enabled on the network you are about to deploy, you have to make sure that there is an IPv6 DHCP facility available. Zero-OS does not work with static IPv6 addresses (at this point in time). So you have choose and create one of the following setups:
|
||||
|
||||
### 2.1 Home Setup
|
||||
|
||||
Use your (home) ISP router Ipv6 DHCP capabilities to provide (private) IPv6 addresses. The principle will work the same as for IPv4 home connections, everything happens enabled by Network Adress Translation (just like anything else that uses internet connectivity). This should be relatively straightforward if you have established that your conenction has IPv6 enabled.
|
||||
|
||||
### 2.2 Data Center/Advanced Setup
|
||||
|
||||
In this situation there are many options on how to setup you node. This requires you as the expert to make a few decisions on how to connect what what the best setup is that you can support for the operaitonal time of your farm. The same basics principles apply:
|
||||
- You have to have a block of (public) IPv6 routed to you router, or you have to have your router setup to provide Network Address Translation (NAT)
|
||||
- You have to have a DHCP server in your network that manages and controls IPV6 ip adress leases. Depending on your specific setup you have this DHCP server manage a public IPv6y range which makes all nodes directly connected to the public internet or you have this DHCP server manage a private block og IPv6 addresses which makes all you nodes connect to the internet through NAT.
|
||||
|
||||
As a farmer you are in charge of selecting and creating the appropriate network setup for your farm.
|
||||
|
||||
# General Notes
|
||||
|
||||
The above setup will allows your node(s) to appear in explorer on the TFGrid and will allow you to earn farming tokens. At stated in the introduction ThreeFold is creating next generation internet capacity and therefore has IPv6 as it's base building block. Connecting to the current (dominant) IPv4 network happens for IT workloads through so called webgateways. As the word sais these are gateways that provide connectivity between the currenct leading IPv4 adressing scheme and IPv6.
|
||||
|
||||
We have started a forum where people share their experiences and configurations. This will be work in progress and forever growing.
|
||||
|
||||
**IMPORTANT**: You as a farmer do not need access to IPV4 to be able to rent capacity for IT workloads that need to be visible on IPV4, this is something that can happen elsewhere on the TFGrid.
|
||||
|
@@ -0,0 +1,94 @@
|
||||
<h1> Networking Overview </h1>
|
||||
|
||||
<h2> Table of Contents </h2>
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Possible Configurations](#possible-configurations)
|
||||
- [Overall Requirements](#overall-requirements)
|
||||
- [Notes and Warnings](#notes-and-warnings)
|
||||
- [Management Interfaces](#management-interfaces)
|
||||
- [Data Center Cable Management](#data-center-cable-management)
|
||||
- [Static IP Uplink](#static-ip-uplink)
|
||||
- [Testing the Setup](#testing-the-setup)
|
||||
- [Questions and Feedback](#questions-and-feedback)
|
||||
|
||||
***
|
||||
|
||||
## Introduction
|
||||
|
||||
In this section, we provide advanced networking tips for farms with public IPs and in data centers (DC). The information available in this section is a combination of documentation from ThreeFold and tips and advice from community members who experienced first-hand the creation of ThreeFold farms that make use of public IPs block in data centers, personal data centers and home farms. A special thank you to those who contributed to improving the TFGrid and its knowledge base documentation.
|
||||
|
||||
## Possible Configurations
|
||||
|
||||
For farmers who have public IPs, extra considerations are needed in setting up the network of the farm. We will go through the main considerations in this section.
|
||||
|
||||
First, we must acknowledge that by the open-source and design of ThreeFold farming, a farm can range from a simple [single 3Node](../3node_building/3node_building.md) setup, to a multi-rack farm hosted in a typical data center, and everything in-between, from the farmer experiencing with public IP blocks, to the entrepreneur who builds their own data center at home.
|
||||
|
||||
There are thus many types of farms and each will have varying configurations. The simplest way to set up a farm has been extensively discussed in the first steps of creating a farm. But what are the other more complex configurations possible? Let's go through some of those:
|
||||
|
||||
- Network link
|
||||
- DC provides a network link into the farmer's rack
|
||||
- Router and switch
|
||||
- The farmer provider their own router and switch
|
||||
- DC provides a router and/or switch in the rack
|
||||
- Gateway IP and public IP
|
||||
- Gateway IP provided is in the same range as the public IPs
|
||||
- Gateway IP is in a different range than the public IPs
|
||||
- Segmenting
|
||||
- Farmer segments the OOB ("Zos"/private) interfaces and the public interfaces into
|
||||
- separate VLANs, OR;
|
||||
- uses separate switches altogether
|
||||
- No segmenting is actually necessary, farmer connects all interfaces to one switch
|
||||
|
||||
## Overall Requirements
|
||||
|
||||
There are overall requirements for any 3Node farm using IP address blocks in a data centere or at home:
|
||||
|
||||
- There must be at least one interface that provide DHCP to each node
|
||||
- Public IPs must be routable from at least one interface
|
||||
|
||||
Note that redundancy can help in avoiding single point of failure [(SPOF)](https://en.wikipedia.org/wiki/Single_point_of_failure).
|
||||
|
||||
## Notes and Warnings
|
||||
|
||||
### Management Interfaces
|
||||
|
||||
You should make sure to never expose management interfaces to the public internet.
|
||||
|
||||
|
||||
### Data Center Cable Management
|
||||
|
||||
It's important to have a good cable management, especially if you are in a data center. Proper cable management will improve the cooling streams of your farm. There shouldn't be any cable in front of the fans. This way, your servers will last longer. If you want to patch a rack, you have to have all lenght of patch cables from 30cm to 3m. Also, try to keep the cables as short as possible. Arrange the cables in bundles of eight and lead them to the sides of the rack as much as possible for optimal airflow.
|
||||
|
||||
<!--
|
||||
|
||||
## Configuring Node Interfaces
|
||||
Add info on this:
|
||||
|
||||
There's a program floating around that allows farmers to configure the interfaces on their nodes. This is done over RMB, and I also developed a script for this purpose. Both might be deprecated with the 3.9 release and new RMB. There should be a documented way of doing this that's relatively accessible.
|
||||
|
||||
-->
|
||||
|
||||
<!--
|
||||
|
||||
### NIC
|
||||
|
||||
QUESTION: Does it matter which NICs are used? -->
|
||||
|
||||
### Static IP Uplink
|
||||
|
||||
If your DC uplink is established by simple static IP (which is the case in most DCs), there is a simple setup possible. Note that if you have to use PPPoE or pptp/L2TP (like a consumer internet connection at most homes), this would not work.
|
||||
|
||||
If your WAN is established by static IP, you can simply attach the WAN uplink provided by the DC to one of the switches (and not to the WAN-side of your own router). Then, the WAN-side of the router needs to be attached to the switch too. By doing so, your nodes will be able to connect directly to the DC gateway, in the same way that the router is connecting its WAN-side to the gateway, without the public IP traffic being routed/bridged through the router (bypassing).
|
||||
|
||||
With a network configured like this, it is absolutely not important on which ports you connect which NIC of your nodes. You can just randomly plug them anywhere. But there is one restriction: the DC uplink must use a static IP. Dynamic IP would also not work because you would then have two DHCP servers in the same physical network (the one from the DC and your own router).
|
||||
|
||||
## Testing the Setup
|
||||
|
||||
Manual and automatic validation of the network of a farm are possible. More information on automatic validation will be added in the future.
|
||||
|
||||
You can test the network of your farm manually by deploying a workload on your 3Nodes with either a gateway or a public IP reserved.
|
||||
|
||||
## Questions and Feedback
|
||||
|
||||
If you have any questions, you can ask the ThreeFold community for help on the [ThreeFold Forum](http://forum.threefold.io/) or on the [ThreeFold Farmer Chat](https://t.me/threefoldfarmers) on Telegram.
|
@@ -0,0 +1,406 @@
|
||||
<h1>Building a DIY 3Node: Desktop Computer</h1>
|
||||
|
||||
In the following 3Node DIY guide, you will learn how to turn a Dell Optiplex 7020 into a 3Node farming on the ThreeFold Grid.
|
||||
|
||||
Note that the process is similar for other desktop computers.
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
|
||||
|
||||
<h2>Table of Contents</h2>
|
||||
|
||||
|
||||
|
||||
- [Prerequisite](#prerequisite)
|
||||
- [DIY 3Node Computer Requirements](#diy-3node-computer-requirements)
|
||||
- [DIY 3Node Material List](#diy-3node-material-list)
|
||||
- [1. Create a Farm](#1-create-a-farm)
|
||||
- [Using Dashboard](#using-dashboard)
|
||||
- [Using TF Connect App](#using-tf-connect-app)
|
||||
- [2. Create a Zero-OS Bootstrap Image](#2-create-a-zero-os-bootstrap-image)
|
||||
- [Download the Zero-OS Boostrap Image](#download-the-zero-os-boostrap-image)
|
||||
- [Burn the Zero-OS Bootstrap Image](#burn-the-zero-os-bootstrap-image)
|
||||
- [3. Set the Hardware](#3-set-the-hardware)
|
||||
- [4. Wipe All the Disks](#4-wipe-all-the-disks)
|
||||
- [1. Create a Linux Boostrap Image](#1-create-a-linux-boostrap-image)
|
||||
- [2. Boot Linux in Try Mode](#2-boot-linux-in-try-mode)
|
||||
- [3. Use wipefs to Wipe All Disks](#3-use-wipefs-to-wipe-all-disks)
|
||||
- [5. Set the BIOS/UEFI](#5-set-the-biosuefi)
|
||||
- [The Essential Features of BIOS/UEFI for a 3Node](#the-essential-features-of-biosuefi-for-a-3node)
|
||||
- [Set the BIOS/UEFI on a Dell Optiplex 7020](#set-the-biosuefi-on-a-dell-optiplex-7020)
|
||||
- [6. Boot the 3Node](#6-boot-the-3node)
|
||||
- [Check the Node Status](#check-the-node-status)
|
||||
- [Farming Rewards Distribution](#farming-rewards-distribution)
|
||||
- [Additional Information](#additional-information)
|
||||
|
||||
***
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
|
||||
|
||||
# Prerequisite
|
||||
|
||||
## DIY 3Node Computer Requirements
|
||||
|
||||
|
||||
|
||||
Any computer with the following specifications can be used as a DIY 3Node.
|
||||
|
||||
- Any 64-bit hardware with an Intel or AMD processor chip.
|
||||
- Servers, desktops and mini computers type hardware are compatible.
|
||||
- A minimum of 500 GB of SSD and a bare minimum of 2 GB of RAM is required.
|
||||
- A ratio of 100GB of SSD and 8GB of RAM per thread is recommended.
|
||||
- A wired ethernet connection is highly recommended to maximize reliability and the ability to farm TFT.
|
||||
- A [passmark](https://www.passmark.com/) of 1000 per core is recommended and will be a minimum requirement in the future.
|
||||
|
||||
In this guide, we are using a Dell Optiplex 7020. It constitutes a perfect affordable entry DIY 3Node as it can be bought refurbished with the proper ratio of 100GB of SSD and 8GB of RAM per thread, and this, without any need of upgrades or updates.
|
||||
|
||||
|
||||
|
||||
## DIY 3Node Material List
|
||||
|
||||
|
||||
|
||||
* Any computer respecting the DIY 3Node Computer Requirements stated above
|
||||
* Ethernet cable
|
||||
* Router + Modem
|
||||
* Surge Protector
|
||||
* 2x USB key 4 Go
|
||||
* Android/iOS Phone
|
||||
* Computer monitor and cable, keyboard and mouse
|
||||
* MAC/Linux/Windows Computer
|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
# 1. Create a Farm
|
||||
|
||||
You can create a farm with either the ThreeFold Dashboard or the ThreeFold Connect app.
|
||||
|
||||
## Using Dashboard
|
||||
|
||||
The Dashboard section contains all the information required to [create a farm](../../../dashboard/farms/your_farms.md).
|
||||
|
||||
## Using TF Connect App
|
||||
|
||||
You can [create a ThreeFold farm](../../../threefold_token/storing_tft/tf_connect_app.md) with the ThreeFold Connect App.
|
||||
|
||||
|
||||
# 2. Create a Zero-OS Bootstrap Image
|
||||
|
||||
## Download the Zero-OS Boostrap Image
|
||||
|
||||
We will now learn how to create a Zero-OS Bootstrap Image in order to boot a DIY 3Node.
|
||||
|
||||
Go on the [ThreeFold Zero-OS Bootstrap Link](https://v3.bootstrap.grid.tf) as shown above.
|
||||
|
||||

|
||||
|
||||
This is the Zero-OS v3 Bootstrapping page.
|
||||
|
||||

|
||||
|
||||
Write your farm ID and choose production mode.
|
||||
|
||||

|
||||
|
||||
Download the bootstrap image. Next, we will burn the bootstrap image.
|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
## Burn the Zero-OS Bootstrap Image
|
||||
|
||||
|
||||
|
||||
For **MAC**, **Linux** and **Windows**, you can use [BalenaEtcher](https://www.balena.io/etcher/) to load/flash the image on a USB stick. This program also formats the USB in the process. This will work for the option **EFI IMG** for UEFI boot, and with the option **USB** for BIOS boot. Simply follow the steps presented to you and make sure you select the correct bootstrap image file you downloaded previously.
|
||||
|
||||
General Steps:
|
||||
|
||||
1. Download BalenaEtcher at [https://balena.io/etcher](https://balena.io/etcher)
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
2. Open BalenaEtcher
|
||||
|
||||

|
||||
|
||||
3. Select **Flash from file**
|
||||
|
||||

|
||||
|
||||
1. Find and select the bootstrap image in your computer
|
||||
|
||||
2. Select **Target** (your USB key)
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
6. Select **Flash**
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
|
||||
That's it. Now you have a bootstrap image on Zero-OS as a bootable removable media device.
|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
# 3. Set the Hardware
|
||||
|
||||
Setting the hardware of this DIY 3node is very easy as there are no updates or upgrades needed. Simply unbox the computer and plug everything.
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
Plug the computer cable in the surge protector
|
||||
|
||||

|
||||
|
||||
Connect the computer cable, the ethernet cable, the mouse and keyboard cable and the monitor cable.
|
||||
|
||||

|
||||
|
||||
Plug the ethernet cable in the router (or the switch)
|
||||
|
||||

|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
# 4. Wipe All the Disks
|
||||
|
||||
In this section, we will learn how to create a Linux bootstrap image, boot it in Try mode and then wipe all the disks in your 3Node. To create a Linux boostrap image, follow the same process as when we burnt the Zero-OS Boostrap Image.
|
||||
|
||||
|
||||
|
||||
## 1. Create a Linux Boostrap Image
|
||||
|
||||
|
||||
|
||||
1. Download the Ubuntu 20.04 ISO file [here](https://releases.ubuntu.com/20.04/)
|
||||
2. Burn the ISO image on a USB key with Balena Etcher
|
||||
|
||||
|
||||
|
||||
## 2. Boot Linux in Try Mode
|
||||
|
||||
|
||||
|
||||
1. Insert your Linux boostrap image USB key in your computer and boot it
|
||||
2. During boot, press F12 to enter into Settings
|
||||
3. Select your booting device, here it is: *UEFI: USB DISK 2.0*
|
||||
|
||||

|
||||
|
||||
4. Select Try or install Ubuntu
|
||||
|
||||

|
||||
|
||||
5. Select Try Ubuntu
|
||||
|
||||

|
||||
|
||||
|
||||
|
||||
## 3. Use wipefs to Wipe All Disks
|
||||
|
||||
|
||||
|
||||
Once Ubuntu is booted, you will land on the main page.
|
||||
|
||||

|
||||
|
||||
At the bottom left of the screen, click on Applications.
|
||||
|
||||

|
||||
|
||||
In Applications, select Terminal.
|
||||
|
||||

|
||||
|
||||
If you don't see it, write terminal in the search box.
|
||||
|
||||

|
||||
|
||||
You will land in the Ubuntu Terminal.
|
||||
|
||||

|
||||
|
||||
Write the command line *lsblk* as shown below. You will then see the disks in your computer. You want to wipe the main disk, but not the USB key we are using, named *sdb* here. We can see here that the SSD disk, *sda*, has 3 partitions: *sda1*, *sda2*, *sda3*. Note that when wiping the disks, we want no partition.
|
||||
|
||||
In this case, the disk we want to wipe is *sda*.
|
||||
|
||||

|
||||
|
||||
Write the command line *sudo wipefs -a /dev/sda*. This will wipe the disk *sda*.
|
||||
|
||||

|
||||
|
||||
If you write the command line *lsblk* once more, you should see that your SSD disk has no more partition. The disk has been properly wiped.
|
||||
|
||||

|
||||
|
||||
Power off the computer by selecting *Power Off* after having clicked on the button at the top right of the screen as shown below.
|
||||
|
||||

|
||||
|
||||
That's it! The disks are all wiped. All that is left now is to set the BIOS/UEFI settings and then boot the 3Node!
|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
# 5. Set the BIOS/UEFI
|
||||
|
||||
Before booting the main operating system, in our case Zero-OS, a computer will boot in either BIOS or UEFI mode. Older systems use BIOS and newer systems uses UEFI. Both BIOS and UEFI are low-lewel softwares needed to interact between the hardware and the main OS of the computer. Note that BIOS is also called Legacy BIOS.
|
||||
|
||||
## The Essential Features of BIOS/UEFI for a 3Node
|
||||
|
||||
|
||||
|
||||
There are certain things that you should make sure are set properly on your 3Node.
|
||||
|
||||
As a general advice, you can Load Defaults (Settings) on your BIOS, then make sure the options below are set properly.
|
||||
|
||||
* Choose the correct combination of BIOS/UEFI and bootstrap image on [https://bootstrap.grid.tf/](https://bootstrap.grid.tf/)
|
||||
* Newer system will use UEFI --> the Dell Optiplex 7020 uses UEFI
|
||||
* Bootstrap image: *EFI IMG* and *EFI FILE*
|
||||
* Older system will use Legacy BIOS
|
||||
* Bootstrap image: *ISO* and *USB*
|
||||
* Set *Multi-Processor* and *Hyperthreading* at Enabled
|
||||
* Sometimes, it will be written *Virtual Cores*, or *Logical Cores*.
|
||||
* Set *Virtualization* at Enabled
|
||||
* On Intel, it is denoted as *CPU virtualization* and on ASUS, it is denoted as *SVM*.
|
||||
* Make sure virtualization is enabled and look for the precise terms in your specific BIOS/UEFI.
|
||||
* Enable *Network Stack* (sometimes called *Network Boot*)
|
||||
* Set *AC Recovery* at *Last Power State*
|
||||
* This will make sure your 3Node restarts after losing power momentarily.
|
||||
* Select the proper *Boot Sequence* for the 3Node to boot Zero-OS from your bootstrap image
|
||||
* e.g., if you have a USB key as a bootstrap image, select it in *Boot Sequence*
|
||||
* Set *Server Lookup Method* (or the equivalent) at *DNS*.
|
||||
* Only use Static IP if you know what you are doing.
|
||||
* Your router will automatically assign a dynamic IP address to your 3Node when it connects to Internet.
|
||||
* Set *Client Address Method* (or the equivalent) at *DHCP*. Only use Static IP if you know what you are doing.
|
||||
* Your router will automatically assign a dynamic IP address to your 3Node when it connects to Internet.
|
||||
* *Secure Boot* should be left at disabled
|
||||
* Enable it if you know what you are doing. Otherwise, it can be set at disabled.
|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
## Set the BIOS/UEFI on a Dell Optiplex 7020
|
||||
|
||||
|
||||
|
||||
1. Insert your Zero-OS boostrap image USB key in your computer and boot it.
|
||||
2. During boot, press F12 to enter into *Settings* then choose *BIOS Setup*.
|
||||
|
||||

|
||||
|
||||
3. In BIOS Setup, click on Load Default and confirm by clicking on *OK*
|
||||
|
||||

|
||||
|
||||
4. Leave the BIOS Setup (Exit) and re-enter. This will set the default settings.
|
||||
|
||||
5. Go through each page and make sure you are following the guidelines set in the section Essential Features as shown in the following pictures.
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
|
||||
6. Once you are done, click on *Exit* and then click *Yes* to save your changes. The 3node will now boot Zero-OS.
|
||||
|
||||

|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
# 6. Boot the 3Node
|
||||
|
||||
If your BIOS/UEFI settings are set properly and you have the Zero-OS bootstrap image USB key plugged in, your 3node should automatically boot Zero-OS every time that it is turned on.
|
||||
|
||||
1. Power on the 3Node with the Zero-OS boostrap image USB key
|
||||
2. Let the 3Node load Zero-OS
|
||||
* The first time it boots, the 3node will register to the TF Grid
|
||||
3. Verify the 3Node's status on ThreeFold Explorer
|
||||
|
||||
The first time you boot a 3Node, it will be written: “This node is not registered (farmer : NameOfFarm). This is normal. The Grid will create a node ID and you will be able to see it on screen. This can take a couple of minutes.
|
||||
|
||||
This is the final screen you should see when your 3Node is connected to the ThreeFold Grid. Note that it is normal if it is written *no public config* next to *PUB* as we did not set any public IP address.
|
||||
|
||||
Naturally, your node ID as well as your farm ID and name will be shown.
|
||||
|
||||

|
||||
|
||||
Once you have your node ID, you can also go on the ThreeFold Dashboard to see your 3Node and verify that your 3Node is online.
|
||||
|
||||
|
||||
|
||||
<div style="break-after:page"></div>
|
||||
|
||||
## Check the Node Status
|
||||
|
||||
You can use the [Node Finder](../../../dashboard/deploy/node_finder.md) on the [TF Dashboard](https://dashboard.grid.tf/) to verify that the node is online.
|
||||
|
||||
Enter your node ID and click **Apply**.
|
||||
|
||||
## Farming Rewards Distribution
|
||||
|
||||
|
||||
|
||||
The farming reward will be sent once per month directly in your ThreeFold Connect App wallet. Farming rewards are usually distributed around the 5th of each month.
|
||||
|
||||
|
||||
|
||||
# Additional Information
|
||||
|
||||
Congratulations, you now have built your first ThreeFold 3Node server!
|
||||
|
||||
If you have any questions, you can ask the ThreeFold community for help on the [ThreeFold Forum](https://forum.threefold.io/) or on the [ThreeFold Telegram Farmer Group](https://t.me/threefoldfarmers).
|
@@ -0,0 +1,19 @@
|
||||
3node_diy_desktop_111.png
|
||||
3node_diy_desktop_114.png
|
||||
3node_diy_desktop_115.png
|
||||
3node_diy_desktop_116.png
|
||||
3node_diy_desktop_117.png
|
||||
3node_diy_desktop_118.png
|
||||
3node_diy_desktop_119.png
|
||||
3node_diy_desktop_120.png
|
||||
3node_diy_desktop_121.png
|
||||
3node_diy_desktop_122.png
|
||||
3node_diy_desktop_123.png
|
||||
3node_diy_desktop_124.png
|
||||
3node_diy_desktop_125.png
|
||||
3node_diy_desktop_126.png
|
||||
3node_diy_desktop_127.png
|
||||
3node_diy_desktop_128.png
|
||||
3node_diy_desktop_129.png
|
||||
3node_diy_desktop_130.png
|
||||
farming_30.png
|
After Width: | Height: | Size: 247 KiB |
After Width: | Height: | Size: 23 KiB |
After Width: | Height: | Size: 27 KiB |
After Width: | Height: | Size: 34 KiB |
After Width: | Height: | Size: 10 KiB |
After Width: | Height: | Size: 229 KiB |
After Width: | Height: | Size: 68 KiB |
After Width: | Height: | Size: 1.3 MiB |
After Width: | Height: | Size: 285 KiB |
After Width: | Height: | Size: 726 KiB |
After Width: | Height: | Size: 490 KiB |
After Width: | Height: | Size: 778 KiB |
After Width: | Height: | Size: 798 KiB |
After Width: | Height: | Size: 691 KiB |
After Width: | Height: | Size: 599 KiB |
After Width: | Height: | Size: 788 KiB |
After Width: | Height: | Size: 758 KiB |
After Width: | Height: | Size: 713 KiB |
After Width: | Height: | Size: 796 KiB |
After Width: | Height: | Size: 753 KiB |
After Width: | Height: | Size: 819 KiB |
After Width: | Height: | Size: 670 KiB |
After Width: | Height: | Size: 728 KiB |
After Width: | Height: | Size: 682 KiB |
After Width: | Height: | Size: 709 KiB |
After Width: | Height: | Size: 67 KiB |
After Width: | Height: | Size: 901 KiB |
After Width: | Height: | Size: 229 KiB |
After Width: | Height: | Size: 382 KiB |
After Width: | Height: | Size: 399 KiB |
After Width: | Height: | Size: 352 KiB |
After Width: | Height: | Size: 408 KiB |
After Width: | Height: | Size: 42 KiB |
After Width: | Height: | Size: 56 KiB |
After Width: | Height: | Size: 146 KiB |
After Width: | Height: | Size: 55 KiB |
After Width: | Height: | Size: 340 KiB |
After Width: | Height: | Size: 25 KiB |