...

collections/technology/primitives/compute/beyond_containers.md

@@ -0,0 +1,17 @@
+## Beyond Containers
+
+![](img/container_native.jpg)
+
+
+Default features:
+
+- compatible with Docker
+- compatible with any Linux workload
+
+We have following unique advantages:
+
+- no need to work with images, we work with our unique zos_fs
+- every container runs in a dedicated virtual machine providing more security
+- the containers talk to each other over a private network: zos_net
+- the containers can use web_gw to allow users on the internet connect to the applications as running in their secure containers 
+- can use core-x to manage the workload

collections/technology/primitives/compute/compute.md

@@ -0,0 +1,8 @@
+
+## TFGrid Compute Layer
+
+![](img/tfgrid_compute_.jpg)
+
+We are more than just Container or VM technology, see [our Beyond Container Document](../../primitives/compute/beyond_containers.md).
+
+For more information see [ZeroOS](../../zos/zos_toc.md)

collections/technology/primitives/compute/corex.md

@@ -0,0 +1,13 @@
+
+# CoreX
+
+![](img/corex.jpg)
+
+This tool allows you to manage your ZMachine over web remotely.
+
+ZMachine process manager
+
+- Provide a web interface and a REST API to control your processes
+- Allow to watch the logs of your processes
+- Or use it as a web terminal (access over https to your terminal)!
+

collections/technology/primitives/compute/zkube.md

@@ -0,0 +1,30 @@
+# ZKube
+
+TFGrid is compatible with Kubernetes Technology.
+
+![](img/kubernetes_0_.jpg)
+
+Each eVDC as shown above is a full blown Kubernetes deployment.
+
+### Unique for our Kubernetes implementation
+
+- The Kubernetes networks are on top of our [ZNet](znet) technology which means all traffic between containers and kubernetes hosts is end2end encrypted independent of where your Kubernetes nodes are deployed.
+- You can mount a QSFS underneath a Kubernetes Node (VM), which means that you can deploy containers on top of QSFS to host unlimited amounts of storage in a super safe way.
+- You Kubernetes environment is for sure 100% decentralized, you define where you want to deploy your Kubernetes nodes and only you have access to the deployed workloads on the TFGrid.
+
+### Features
+
+*   integration with znet (efficient, secure encrypted network between the zmachines)
+*   can be easily deployed at the edge
+*   single-tenant!
+
+<!-- 
+### ZMachine Benefits
+
+*   [ZOS Protect](zos_protect): no hacking surface to the Zero-Nodes, integrate silicon route of trust
+*   [ZNet](znet) and [Planetary Net](planetary_network): a true global single backplane network connecting us all -->
+
+### Architecture
+
+![](img/zkube_architecture_.jpg)
+

collections/technology/primitives/compute/zmachine.md

@@ -0,0 +1,22 @@
+# ZMachine
+
+
+
+### Features
+
+*   import from docker (market std for containers)
+*   can be easily deployed at the edge (edge cloud)
+*   single-tenant, fully decentralized!
+*   can deploy unlimited amounts of storage using our qsfs
+*   minimal hacking surface to the Zero-Nodes, integrate silicon route of trust
+*   ZOS Filesystem: dedupe, zero-install, hacker-proof
+*   Webgateway: intelligent connection between web (internet) and container services
+*   integration with ZNet (efficient, secure encrypted network between the zmachines)
+*   Planetary Net: a true global single backplane network connecting us all
+
+### Architecture
+
+![](img/zmachine_zos_.jpg)
+
+A ZMachine is running as a virtual machine on top of Zero-OS.
+

collections/technology/primitives/network/network.md

@@ -0,0 +1,13 @@
+# Network Primitives
+
+- [Planetary network](planetary_network.md): 
+    - is a planetary scalable network, we have clients for windows, osx, android and iphone.
+- [ZOS Net](znet.md):
+    - is a fast end2end encrypted network technology, keep your traffic between your z_machines 100% private.
+- [ZOS NIC](znic.md): 
+    - connection to a public ipaddress
+- [WEB GW](webgw3.md): 
+    - web gateway, a secure way to allow internet traffic reach your secure Z-Machine.
+
+
+

collections/technology/primitives/network/planetary_network.md

@@ -0,0 +1,47 @@
+
+# Planetary Network
+
+![](img/planet_net_.jpg)
+
+The planetary network is an overlay network which lives on top of the existing internet or other peer2peer networks created. In this network, everyone is connected to everyone. End-to-end encryption between users of an app and the app running behind the network wall.
+
+Each user end network point is strongly authenticated and uniquely identified, independent of the network carrier used. There is no need for a centralized firewall or VPN solutions, as there is a circle based networking security in place.
+
+Benefits :
+- It finds shortest possible paths between peers
+- There's full security through end-to-end encrypted messaging
+- It allows for peer2peer links like meshed wireless
+- It can survive broken internet links and re-route when needed
+- It resolves the shortage of IPV4 addresses
+
+
+Whereas current computer networks depend heavily on very centralized design and configuration, this networking concept breaks this mould by making use of a global spanning tree to form a scalable IPv6 encrypted mesh network. This is a peer2peer implementation of a networking protocol.
+
+The following table illustrates high-level differences between traditional networks like the internet, and the planetary threefold network:
+
+| Characteristic                                                  | Traditional | Planetary Network |
+| --------------------------------------------------------------- | ----------- | ----------------- |
+| End-to-end encryption for all traffic across the network        | No          | Yes               |
+| Decentralized routing information shared using a DHT            | No          | Yes               |
+| Cryptographically-bound IPv6 addresses                          | No          | Yes               |
+| Node is aware of its relative location to other nodes           | No          | Yes               |
+| IPv6 address remains with the device even if moved              | No          | Yes               |
+| Topology extends gracefully across different mediums, i.e. mesh | No          | Yes               |
+
+## What are the problems solved here?
+
+The internet as we know it today doesn’t conform to a well-defined topology. This has largely happened over time - as the internet has grown, more and more networks have been “bolted together”. The lack of defined topology gives us some unavoidable problems:
+
+- The routing tables that hold a “map” of the internet are huge and inefficient
+- There isn’t really any way for a computer to know where it is located on the internet relative to anything else
+- It’s difficult to examine where a packet will go on its journey from source to destination without actually sending it
+- It’s very difficult to install reliable networks into locations that change often or are non-static, i.e. wireless mesh networks
+
+These problems have been partially mitigated (but not really solved) through centralization - rather than your computers at home holding a copy of the global routing table, your ISP does it for you. Your computers and network devices are configured just to “send it upstream” and to let your ISP decide where it goes from there, but this does leave you entirely at the mercy of your ISP who can redirect your traffic anywhere they like and to inspect, manipulate or intercept it.
+
+In addition, wireless meshing requires you to know a lot about the network around you, which would not typically be the case when you have outsourced this knowledge to your ISP. Many existing wireless mesh routing schemes are not scalable or efficient, and do not bridge well with existing networks.
+
+![](img/planetary_net.jpg)
+
+The planetary network is a continuation and implementation of the [Planetary Network](https://Planetary Network-network.github.io/about.html) network initiative. This technology is in beta but has been proven to work already quite well.
+

collections/technology/primitives/network/webgw.md

@@ -0,0 +1,40 @@
+
+
+# WebGW
+
+The Web Gateway is a mechanism to connect the private networks to the open Internet, in such a way that there is no direct connection between internet and the secure workloads running in the ZMachines.
+
+![](img/webgateway.jpg)
+
+
+- Separation between where compute workloads are and where services are exposed
+- Redundant
+    - Each app can be exposed on multiple webgateways at once
+- Support for many interfaces...
+- Helps resolve shortage of IPv4 addresses
+
+### Implementation
+
+Some 3nodes supports gateway functionality (configured by the farmers). A 3node with gateway configuration can then accept gateway workloads and then forward traffic to ZMachines that only have Planetary Network (planetary network) or Ipv6 addresses.
+
+The gateway workloads consists of a name (prefix) that need to be reserved on the block chain first. Then the list of backend IPs. There are other flags that can be set to control automatic TLS (please check terraform documentations for the exact details of a reservation).
+
+Once the 3node receives this workloads, the network configure proxy for this name and the Planetary Network IPs.
+
+### Security
+
+ZMachines have to have a Planetary Network IP or any other IPv6 (also IPv4 are accepted), it means that any person who is connected to the Planetary Network, can also reach the ZMachine without the need for a proxy.
+
+So it's up to the ZMachine owner/maintainer to make sure it is secured and only have the required ports open.
+
+### Redundant Network Connection
+
+![](img/redundant_net.jpg)
+
+
+### Unlimited Scale
+
+![](img/webgw_scaling.jpg)
+
+
+The network architecture is a pure scale-out network system, it can scale to unlimited size, there is simply no bottleneck. Network "supply" is created by network farmers, and network "demand" is done by TF Grid users. Supply and demand scale independently, for supply there can be unlimited network, farmers providing the web gateways on their own 3nodes, and unlimited compute farmers providing 3nodes for compute and storage. The demand side is driven by developers creating software that runs on the grid, system integrators creating solutions for enterprises. This demand side is exponentially growing for data processing and storage use cases.

collections/technology/primitives/network/znet.md

@@ -0,0 +1,33 @@
+
+
+# ZNET
+
+Decentralized networking platform allowing any compute and storage workload to be connected together on a private (overlay) network and exposed to the existing internet network. The Peer2Peer network platform allows any workload to be connected over secure encrypted networks which will look for the shortest path between the nodes.
+
+![](img/zos_network_overlay.jpg)
+
+
+### Secure mesh overlay network (peer2peer)
+
+Z_NET is the foundation of any architecture running on the TF Grid. It can be seen as a virtual private datacenter and the network allows all of the *N* containers to connect to all of the *(N-1)* other containers. Any network connection is a secure network connection between your containers, it creates peer 2 peer network between containers. 
+
+![](img/overlay_net1.jpg)
+
+No connection is made with the internet. The ZNet is a single tenant network and by default not connected to the public internet. Everything stays private. For connecting to the public internet, a Web Gateway is included in the product to allows for public access if and when required.
+
+### Redundancy
+
+As integrated with [WebGW](webgw):
+
+![](img/znet_redundancy.jpg)
+
+- Any app can get (securely) connected to the internet by any chosen IP address made available by ThreeFold network farmers through [WebGW](webgw)
+- An app can be connected to multiple web gateways at once, the DNS round robin principle will provide load balancing and redundancy
+- An easy clustering mechanism where web gateways and nodes can be lost and the public service will still be up and running
+- Easy maintenance. When containers are moved or re-created, the same end user connection can be reused as that connection is terminated on the Web Gateway. The moved or newly created Web Gateway will recreate the socket to the Web Gateway and receive inbound traffic.
+
+### Interfaces in Zero-OS
+
+![](img/znet_znic1.jpg)
+
+

collections/technology/primitives/network/znic.md

@@ -0,0 +1,11 @@
+# ZNIC
+
+ZNIC is the network interface which is connected to ZMachine.
+
+Can be implemented as interface to
+
+- planetary_network
+- public ip address on a Zero-OS
+
+![](img/znet_znic.jpg)
+

collections/technology/primitives/primitives.md

@@ -0,0 +1,49 @@
+![](img/layer0_.jpg)
+
+# TFGrid Low Level Functions = Primitives
+
+The following are the low level constructs which can be deployed on the TFGrid.
+
+The following functionalities allow you to create any solutions on top of the grid.
+Any application which can run on linux can run on the TFGrid.
+
+### Compute (uses CU)
+
+- [ZKube](compute/zkube.md)
+    - kubernetes deployment
+- [ZMachine](compute/zmachine.md)
+    - the container or virtual machine running inside ZOS
+- [CoreX](compute/corex.md) 
+    - process manager (optional), can be used to get remote access to your zmachine
+
+A 3Node is a Zero-OS enabled computer which is hosted by any of the ThreeFold Farmers.
+
+### There are 4 storage mechanisms which can be used to store your data:
+
+- [ZOS FS](storage/zos_fs.md) 
+    - is our dedupe unique filesystem, replaces docker images 
+- [ZOS Mount](storage/zmount.md)     
+    - is a mounted disk location on SSD, this can be used as faster storage location 
+- [Quamtum Safe Filesystem](../qsss/qss_filesystem.md)
+    - this is a super unique storage system, data can never be lost or corrupted. Please be reminded that this storage layer is only meant to be used for secondary storage applications
+- [ZOS Disk](storage/zdisk.md)
+    - a virtual disk technology, only for TFTech OEM partners
+
+### There are 4 ways how networks can be connected to a Z-Machine.
+
+- [Planetary network](network/planetary_network.md): 
+    - is a planetary scalable network, we have clients for windows, osx, android and iphone
+- [ZOS Net](network/znet.md):
+    - is a fast end2end encrypted network technology, keep your traffic between your z_machines 100% private
+- [ZOS NIC](network/znic.md): 
+    - connection to a public ipaddress
+- [WEB GW](network/webgw.md): 
+    - web gateway, a secure way to allow internet traffic reach your secure Z-Machine.
+
+
+
+
+
+
+
+

collections/technology/primitives/storage/qsfs.md

@@ -0,0 +1,29 @@
+# Quantum Safe Filesystem
+
+![](img/zos_zstor.jpg)
+
+presents itself as a filesystem to the ZMachine.
+
+### Benefits
+
+- Safe
+- Hacker Proof
+- Ultra Reliable
+- Low Overhead
+- Ultra Scalable
+- Self Healing = recovers service automatically in the event of outage with no human 
+
+
+### Can be used as
+
+- backup and archive system
+- Blockchain Storage Backend (OEM ONLY)
+
+### Implementation
+
+see how its implemented in:
+
+- [Quantum Safe Storage](../../qsss/qsss_home.md)
+- [Quantum Safe Filesystem](../../qsss/qss_filesystem.md)
+- [Quantum Safe Algo](../../qsss/qss_algorithm.md)
+

collections/technology/primitives/storage/storage.md

@@ -0,0 +1,9 @@
+# Storage Primitives
+
+- [ZOS Filesystem](zos_fs.md) : deduped immutable filesystem
+- [ZOS Mount](zmount.md) : a part of a SSD (fast disk), mounted underneath your zmachine
+- [Quantum Safe Filesystem](qsfs.md) : unbreakable storage system (secondary storage only)
+- [Zero-DB](zdb.md) : the lowest level storage primitive, is a key value stor, used underneath other storage mechanisms typically
+- [Zero-Disk](zdisk.md) : OEM only, virtual disk format
+
+Uses [Storage Units = SU](../../../grid/concepts/cloudunits.md).

collections/technology/primitives/storage/zdb.md

@@ -0,0 +1,8 @@
+# ZOS-DB (ZDB)
+
+![](img/zdb_arch.jpg)
+
+0-db is a fast and efficient key-value store redis-protocol compatible, which makes data persistent inside an always append datafile, with namespaces support.
+
+> ZDB is being used as backend storage for [Quantum Safe Filesystem](qsfs.md).
+

collections/technology/primitives/storage/zdisk.md

@@ -0,0 +1,10 @@
+# ZOS_Disk
+
+Virtual disk creates the possibility to create and use virtual disks which can be attached to containers (and virtual machines).  
+
+The technology is designed to be redundant without having to do anything.
+
+## Roadmap
+
+- The virtual disk technology is  available for OEM's only, contact ThreeFold Tech.
+

collections/technology/primitives/storage/zmount.md

@@ -0,0 +1,10 @@
+# ZOS_Mount
+
+A SSD storage location on which can be written inside a VMachine and VKube.
+
+The SSD storage location is mounted on a chosen path inside your Z-Machine.
+
+![](img/zmount.jpg)
+
+> ZMounts are not incrypted, if you need security, use a quantum safe filesystem.
+

collections/technology/primitives/storage/zos_fs.md

@@ -0,0 +1,38 @@
+
+# ZOS FileSystem (ZOS-FS)
+
+![](img/zosfs.png)  
+
+
+A deduped filesystem which is more efficient compared to images as used in other Virtual Machine technology.
+
+
+## Uses FLIST Inside
+
+In Zero-OS, `flist` is the format used to store zmachine images. This format is made to provide a complete mountable remote filesystem, but downloading only the files contents that you actually needs.
+
+In practice, Flist itself is a small database which contains metadata about files and directories, file payload are stored on a tfgrid hub. You only need to download payload when you need it, this dramatically reduce zmachine boot time, bandwidth and disk overhead.
+
+### Why this ZFlist Concept
+
+Have you ever been in the following situation: you need two small files but they are embedded in a large archive.  How to get to those two files in an efficient way?  What a disappointment when you see this archive is 4 GB large and you only need four files of 2 MB inside. You'll need to download the full archive, store it somewhere to extract only what you need. Time, effort and bandwidth wasted.
+
+You want to start a Docker container and the base image you want to use is 2 GB. What do you need to do before being able to use your container? Waiting to get the 2 GB downloaded. This problem exists everywhere but in Europe and the US where the bandwidth speeds are such that this doesn't present a real problem anymore, hence none of the leading (current) tech companies are looking for solutions.
+
+We believe that there should be a smarter way of dealing with this than simply throwing larger bandwidth at the problem: what if you could only download the files you actually want and not the full blob (archive, image, whatever...).
+
+ZFList is splitting metadata and data. Metadata is referential information about everything you need to know about content of the archive, but without the payload. Payload is the content of the referred files. The ZFList is exactly that: it consists of metadata with references that point to where to get the payload itself. So if you don't need it you won't get it.
+
+As soon as you have the flist mounted, you can see the full directory tree, and walk around it. The files are only downloaded and presented at moment that you try to access them. In other words, every time you want to read a file, or modify it, Zero FS will download it, so that the data is available too. You only download on-the-fly what you need which reduces dramatically the bandwidth requirement.
+
+
+## Benefits
+
+- Efficient usage of bandwidth makes this service perform with and without (much) bandwidth.
+
+## Flist Tool
+
+![](img/flist.png)  
+
+> to see our tool for flists see: https://hub.grid.tf/
+