tfgrid/info_tfgrid
33
3
Fork 0
Code Issues Pull Requests 2 Packages Projects Releases Activity

new book manual

Browse Source
This commit is contained in:
mik-tf
2024-04-15 17:42:25 +00:00
parent b46a6df7e0
commit a567404ef3
1772 changed files with 450 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
collections/manual/documentation/farmers/advanced_networking/advanced_networking_toc.md Normal file
View File

@@ -0,0 +1,13 @@
<h1> Advanced Networking </h1>
Welcome to the *Advanced Networking* section of the ThreeFold Manual.
In this section, we provide advanced networking tips for farms with public IPs and in data centers (DC). We also cover the differences between IPv4 and IPv6 networking.
<h2>Table of Contents</h2>
- [Networking Overview](./networking_overview.md)
- [Network Considerations](./network_considerations.md)
- [Network Setup](./network_setup.md)
> Note: This documentation does not constitute a complete set of knowledge on setting farms with public IP addresses in a data center. Please make sure to do your own research and communicate with your data center and your Internet service provider for any additional information.

120
collections/manual/documentation/farmers/advanced_networking/network_considerations.md Normal file
View File

@@ -0,0 +1,120 @@
<h1>Network Considerations</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Running ZOS (v2) at Home](#running-zos-v2-at-home)
- [Running ZOS (v2) in a Multi-Node Farm in a DC](#running-zos-v2-in-a-multi-node-farm-in-a-dc)
- [Necessities](#necessities)
- [IPv6](#ipv6)
- [Routing/Firewalling](#routingfirewalling)
- [Multi-NIC Nodes](#multi-nic-nodes)
- [Farmers and the TFGrid](#farmers-and-the-tfgrid)
***
## Introduction
Running ZOS on a node is just a matter of booting it with a USB stick, or with a dhcp/bootp/tftp server with the right configuration so that the node can start the OS.
Once it starts booting, the OS detects the NICs, and starts the network configuration. A Node can only continue it's boot process till the end when it effectively has received an IP address and a route to the Internet. Without that, the Node will retry indefinitely to obtain Internet access and not finish it's startup.
So a Node needs to be connected to a __wired__ network, providing a dhcp server and a default gateway to the Internet, be it NATed or plainly on the public network, where any route to the Internet, be it IPv4 or IPv6 or both is sufficient.
For a node to have that ability to host user networks, we **strongly** advise to have a working IPv6 setup, as that is the primary IP stack we're using for the User Network's Mesh to function.
## Running ZOS (v2) at Home
Running a ZOS Node at home is plain simple. Connect it to your router, plug it in the network, insert the preconfigured USB stick containing the bootloader and the `farmer_id`, power it on.
## Running ZOS (v2) in a Multi-Node Farm in a DC
Multi-Node Farms, where a farmer wants to host the nodes in a data centre, have basically the same simplicity, but the nodes can boot from a boot server that provides for DHCP, and also delivers the iPXE image to load, without the need for a USB stick in every Node.
A boot server is not really necessary, but it helps! That server has a list of the MAC addresses of the nodes, and delivers the bootloader over PXE. The farmer is responsible to set-up the network, and configure the boot server.
### Necessities
The Farmer needs to:
- Obtain an IPv6 prefix allocation from the provider. A `/64` will do, that is publicly reachable, but a `/48` is advisable if the farmer wants to provide IPv6 transit for User Networks
- If IPv6 is not an option, obtain an IPv4 subnet from the provider. At least one IPv4 address per node is needed, where all IP addresses are publicly reachable.
- Have the Nodes connected on that public network with a switch so that all Nodes are publicly reachable.
- In case of multiple NICS, also make sure his farm is properly registered in BCDB, so that the Node's public IP Addresses are registered.
- Properly list the MAC addresses of the Nodes, and configure the DHCP server to provide for an IP address, and in case of multiple NICs also provide for private IP addresses over DHCP per Node.
- Make sure that after first boot, the Nodes are reachable.
### IPv6
IPv6, although already a real protocol since '98, has seen reluctant adoption over the time it exists. That mostly because ISPs and Carriers were reluctant to deploy it, and not seeing the need since the advent of NAT and private IP space, giving the false impression of security.
But this month (10/2019), RIPE sent a mail to all it's LIRs that the last consecutive /22 in IPv4 has been allocated. Needless to say, but that makes the transition to IPv6 in 2019 of utmost importance and necessity.
Hence, ZOS starts with IPv6, and IPv4 is merely an afterthought ;-)
So in a nutshell: we greatly encourage Farmers to have IPv6 on the Node's network.
### Routing/Firewalling
Basically, the Nodes are self-protecting, in the sense that they provide no means at all to be accessed through listening processes at all. No service is active on the node itself, and User Networks function solely on an overlay.
That also means that there is no need for a Farm admin to protect the Nodes from exterior access, albeit some DDoS protection might be a good idea.
In the first phase we will still allow the Host OS (ZOS) to reply on ICMP ping requests, but that 'feature' might as well be blocked in the future, as once a Node is able to register itself, there is no real need to ever want to try to reach it.
### Multi-NIC Nodes
Nodes that Farmers deploy are typically multi-NIC Nodes, where one (typically a 1GBit NIC) can be used for getting a proper DHCP server running from where the Nodes can boot, and one other NIC (1Gbit or even 10GBit), that then is used for transfers of User Data, so that there is a clean separation, and possible injections bogus data is not possible.
That means that there would be two networks, either by different physical switches, or by port-based VLANs in the switch (if there is only one).
- Management NICs
The Management NIC will be used by ZOS to boot, and register itself to the GRID. Also, all communications from the Node to the Grid happens from there.
- Public NICs
### Farmers and the TFGrid
A Node, being part of the Grid, has no concept of 'Farmer'. The only relationship for a Node with a Farmer is the fact that that is registered 'somewhere (TM)', and that a such workloads on a Node will be remunerated with Tokens. For the rest, a Node is a wholly stand-alone thing that participates in the Grid.
```text
172.16.1.0/24
2a02:1807:1100:10::/64
+--------------------------------------+
| +--------------+ | +-----------------------+
| |Node ZOS | +-------+ | |
| | +-------------+1GBit +--------------------+ 1GBit switch |
| | | br-zos +-------+ | |
| | | | | |
| | | | | |
| | | | +------------------+----+
| +--------------+ | | +-----------+
| | OOB Network | | |
| | +----------+ ROUTER |
| | | |
| | | |
| | | |
| +------------+ | +----------+ |
| | Public | | | | |
| | container | | | +-----+-----+
| | | | | |
| | | | | |
| +---+--------+ | +-------------------+--------+ |
| | | | 10GBit Switch | |
| br-pub| +-------+ | | |
| +-----+10GBit +-------------------+ | +---------->
| +-------+ | | Internet
| | | |
| | +----------------------------+
+--------------------------------------+
185.69.167.128/26 Public network
2a02:1807:1100:0::/64
```
Where the underlay part of the wireguard interfaces get instantiated in the Public container (namespace), and once created these wireguard interfaces get sent into the User Network (Network Resource), where a user can then configure the interface a he sees fit.
The router of the farmer fulfills 2 roles:
- NAT everything in the OOB network to the outside, so that nodes can start and register themselves, as well get tasks to execute from the BCDB.
- Route the assigned IPv4 subnet and IPv6 public prefix on the public segment, to which the public container is connected.
As such, in case that the farmer wants to provide IPv4 public access for grid proxies, the node will need at least one (1) IPv4 address. It's free to the farmer to assign IPv4 addresses to only a part of the Nodes.
On the other hand, it is quite important to have a proper IPv6 setup, because things will work out better.
It's the Farmer's task to set up the Router and the switches.
In a simpler setup (small number of nodes for instance), the farmer could setup a single switch and make 2 port-based VLANs to separate OOB and Public, or even wit single-nic nodes, just put them directly on the public segment, but then he will have to provide a DHCP server on the Public network.

86
collections/manual/documentation/farmers/advanced_networking/network_setup.md Normal file
View File

@@ -0,0 +1,86 @@
<h1>Network Setup</h1>
<h2>Table of Contents</h2>
- [Introduction](#introduction)
- [Network Setup for Farmers](#network-setup-for-farmers)
- [Step 1. Testing for IPv6 Availability in Your Location](#step-1-testing-for-ipv6-availability-in-your-location)
- [Step 2. Choosing the Setup to Connect Your Nodes](#step-2-choosing-the-setup-to-connect-your-nodes)
- [2.1 Home Setup](#21-home-setup)
- [2.2 Data Center/Advanced Setup](#22-data-centeradvanced-setup)
- [General Notes](#general-notes)
***
# Introduction
0-OS nodes participating in the Threefold grid, need connectivity of course. They need to be able to communicate over
the Internet with each-other in order to do various things:
- download its OS modules
- perform OS module upgrades
- register itself to the grid, and send regular updates about it's status
- query the grid for tasks to execute
- build and run the Overlay Network
- download flists and the effective files to cache
The nodes themselves can have connectivity in a few different ways:
- Only have RFC1918 private addresses, connected to the Internet through NAT, NO IPv6
Mostly, these are single-NIC (Network card) machines that can host some workloads through the Overlay Network, but
cant't expose services directly. These are HIDDEN nodes, and are mostly booted with an USB stick from
bootstrap.grid.tf .
- Dual-stacked: having RFC1918 private IPv4 and public IPv6 , where the IPv6 addresses are received from a home router,
but firewalled for outgoing traffic only. These nodes are effectively also HIDDEN
- Nodes with 2 NICs, one that has effectively a NIC connected to a segment that has real public
addresses (IPv4 and/or IPv6) and one NIC that is used for booting and local
management. (OOB) (like in the drawing for farmer setup)
For Farmers, we need to have Nodes to be reachable over IPv6, so that the nodes can:
- expose services to be proxied into containers/vms
- act as aggregating nodes for Overlay Networks for HIDDEN Nodes
Some Nodes in Farms should also have a publicly reachable IPv4, to make sure that clients that only have IPv4 can
effectively reach exposed services.
But we need to stress the importance of IPv6 availability when you're running a multi-node farm in a datacentre: as the
grid is boldly claiming to be a new Internet, we should make sure we adhere to the new protocols that are future-proof.
Hence: IPv6 is the base, and IPv4 is just there to accomodate the transition.
Nowadays, RIPE can't even hand out consecutive /22 IPv4 blocks any more for new LIRs, so you'll be bound to market to
get IPv4, mostly at rates of 10-15 Euro per IP. Things tend to get costly that way.
So anyway, IPv6 is not an afterthought in 0-OS, we're starting with it.
# Network Setup for Farmers
This is a quick manual to what is needed for connecting a node with zero-OS V2.0
## Step 1. Testing for IPv6 Availability in Your Location
As descibed above the network in which the node is instaleld has to be IPv6 enabled. This is not an afterthought as we are building a new internet it has to ba based on the new and forward looking IP addressing scheme. This is something you have to investigate, negotiate with you connectivity provider. Many (but not all home connectivity products and certainly most datacenters can provide you with IPv6. There are many sources of infromation on how to test and check whether your connection is IPv6 enabled, [here is a starting point](http://www.ipv6enabled.org/ipv6_enabled/ipv6_enable.php)
## Step 2. Choosing the Setup to Connect Your Nodes
Once you have established that you have IPv6 enabled on the network you are about to deploy, you have to make sure that there is an IPv6 DHCP facility available. Zero-OS does not work with static IPv6 addresses (at this point in time). So you have choose and create one of the following setups:
### 2.1 Home Setup
Use your (home) ISP router Ipv6 DHCP capabilities to provide (private) IPv6 addresses. The principle will work the same as for IPv4 home connections, everything happens enabled by Network Adress Translation (just like anything else that uses internet connectivity). This should be relatively straightforward if you have established that your conenction has IPv6 enabled.
### 2.2 Data Center/Advanced Setup
In this situation there are many options on how to setup you node. This requires you as the expert to make a few decisions on how to connect what what the best setup is that you can support for the operaitonal time of your farm. The same basics principles apply:
- You have to have a block of (public) IPv6 routed to you router, or you have to have your router setup to provide Network Address Translation (NAT)
- You have to have a DHCP server in your network that manages and controls IPV6 ip adress leases. Depending on your specific setup you have this DHCP server manage a public IPv6y range which makes all nodes directly connected to the public internet or you have this DHCP server manage a private block og IPv6 addresses which makes all you nodes connect to the internet through NAT.
As a farmer you are in charge of selecting and creating the appropriate network setup for your farm.
# General Notes
The above setup will allows your node(s) to appear in explorer on the TFGrid and will allow you to earn farming tokens. At stated in the introduction ThreeFold is creating next generation internet capacity and therefore has IPv6 as it's base building block. Connecting to the current (dominant) IPv4 network happens for IT workloads through so called webgateways. As the word sais these are gateways that provide connectivity between the currenct leading IPv4 adressing scheme and IPv6.
We have started a forum where people share their experiences and configurations. This will be work in progress and forever growing.
**IMPORTANT**: You as a farmer do not need access to IPV4 to be able to rent capacity for IT workloads that need to be visible on IPV4, this is something that can happen elsewhere on the TFGrid.

94
collections/manual/documentation/farmers/advanced_networking/networking_overview.md Normal file
View File

@@ -0,0 +1,94 @@
<h1> Networking Overview </h1>
<h2> Table of Contents </h2>
- [Introduction](#introduction)
- [Possible Configurations](#possible-configurations)
- [Overall Requirements](#overall-requirements)
- [Notes and Warnings](#notes-and-warnings)
- [Management Interfaces](#management-interfaces)
- [Data Center Cable Management](#data-center-cable-management)
- [Static IP Uplink](#static-ip-uplink)
- [Testing the Setup](#testing-the-setup)
- [Questions and Feedback](#questions-and-feedback)
***
## Introduction
In this section, we provide advanced networking tips for farms with public IPs and in data centers (DC). The information available in this section is a combination of documentation from ThreeFold and tips and advice from community members who experienced first-hand the creation of ThreeFold farms that make use of public IPs block in data centers, personal data centers and home farms. A special thank you to those who contributed to improving the TFGrid and its knowledge base documentation.
## Possible Configurations
For farmers who have public IPs, extra considerations are needed in setting up the network of the farm. We will go through the main considerations in this section.
First, we must acknowledge that by the open-source and design of ThreeFold farming, a farm can range from a simple [single 3Node](../3node_building/3node_building.md) setup, to a multi-rack farm hosted in a typical data center, and everything in-between, from the farmer experiencing with public IP blocks, to the entrepreneur who builds their own data center at home.
There are thus many types of farms and each will have varying configurations. The simplest way to set up a farm has been extensively discussed in the first steps of creating a farm. But what are the other more complex configurations possible? Let's go through some of those:
- Network link
- DC provides a network link into the farmer's rack
- Router and switch
- The farmer provider their own router and switch
- DC provides a router and/or switch in the rack
- Gateway IP and public IP
- Gateway IP provided is in the same range as the public IPs
- Gateway IP is in a different range than the public IPs
- Segmenting
- Farmer segments the OOB ("Zos"/private) interfaces and the public interfaces into
- separate VLANs, OR;
- uses separate switches altogether
- No segmenting is actually necessary, farmer connects all interfaces to one switch
## Overall Requirements
There are overall requirements for any 3Node farm using IP address blocks in a data centere or at home:
- There must be at least one interface that provide DHCP to each node
- Public IPs must be routable from at least one interface
Note that redundancy can help in avoiding single point of failure [(SPOF)](https://en.wikipedia.org/wiki/Single_point_of_failure).
## Notes and Warnings
### Management Interfaces
You should make sure to never expose management interfaces to the public internet.
### Data Center Cable Management
It's important to have a good cable management, especially if you are in a data center. Proper cable management will improve the cooling streams of your farm. There shouldn't be any cable in front of the fans. This way, your servers will last longer. If you want to patch a rack, you have to have all lenght of patch cables from 30cm to 3m. Also, try to keep the cables as short as possible. Arrange the cables in bundles of eight and lead them to the sides of the rack as much as possible for optimal airflow.
<!--
## Configuring Node Interfaces
Add info on this:
There's a program floating around that allows farmers to configure the interfaces on their nodes. This is done over RMB, and I also developed a script for this purpose. Both might be deprecated with the 3.9 release and new RMB. There should be a documented way of doing this that's relatively accessible.
-->
<!--
### NIC
QUESTION: Does it matter which NICs are used? -->
### Static IP Uplink
If your DC uplink is established by simple static IP (which is the case in most DCs), there is a simple setup possible. Note that if you have to use PPPoE or pptp/L2TP (like a consumer internet connection at most homes), this would not work.
If your WAN is established by static IP, you can simply attach the WAN uplink provided by the DC to one of the switches (and not to the WAN-side of your own router). Then, the WAN-side of the router needs to be attached to the switch too. By doing so, your nodes will be able to connect directly to the DC gateway, in the same way that the router is connecting its WAN-side to the gateway, without the public IP traffic being routed/bridged through the router (bypassing).
With a network configured like this, it is absolutely not important on which ports you connect which NIC of your nodes. You can just randomly plug them anywhere. But there is one restriction: the DC uplink must use a static IP. Dynamic IP would also not work because you would then have two DHCP servers in the same physical network (the one from the DC and your own router).
## Testing the Setup
Manual and automatic validation of the network of a farm are possible. More information on automatic validation will be added in the future.
You can test the network of your farm manually by deploying a workload on your 3Nodes with either a gateway or a public IP reserved.
## Questions and Feedback
If you have any questions, you can ask the ThreeFold community for help on the [ThreeFold Forum](http://forum.threefold.io/) or on the [ThreeFold Farmer Chat](https://t.me/threefoldfarmers) on Telegram.