manual, update new pr

2024-05-14 16:21:27 -04:00 · 2024-05-14 16:21:27 -04:00 · 71bd49d905
commit 71bd49d905
parent 6b20cce94f
7 changed files with 237 additions and 2 deletions
--- a/books/manual/SUMMARY.md
+++ b/books/manual/SUMMARY.md
@ -242,6 +242,7 @@
        - [UFW Basics](system_administrators/computer_it_basics/firewall_basics/ufw_basics.md)
        - [Firewalld Basics](system_administrators/computer_it_basics/firewall_basics/firewalld_basics.md)
      - [File Transfer](system_administrators/computer_it_basics/file_transfer.md)
+      - [Screenshots](system_administrators/computer_it_basics/screenshots.md)
    - [Advanced](system_administrators/advanced/advanced.md)
      - [Token Transfer Keygenerator](system_administrators/advanced/token_transfer_keygenerator.md)
      - [Cancel Contracts](system_administrators/advanced/cancel_contracts.md)
@ -251,6 +252,7 @@
      - [IPFS](system_administrators/advanced/ipfs/ipfs_toc.md)
        - [IPFS on a Full VM](system_administrators/advanced/ipfs/ipfs_fullvm.md)
        - [IPFS on a Micro VM](system_administrators/advanced/ipfs/ipfs_microvm.md) 
+      - [AI & ML Workloads](system_administrators/advanced/ai_ml_workloads.md)
  - [ThreeFold Token](threefold_token/threefold_token.md)
    - [TFT Bridges](threefold_token/tft_bridges/tft_bridges.md)
      - [TFChain-Stellar Bridge](threefold_token/tft_bridges/tfchain_stellar_bridge.md)
--- a/collections/system_administrators/advanced/advanced.md
+++ b/collections/system_administrators/advanced/advanced.md
@ -12,3 +12,4 @@ In this section, we delve into sophisticated topics and powerful functionalities
 - [IPFS](ipfs_toc.md)
  - [IPFS on a Full VM](ipfs_fullvm.md)
  - [IPFS on a Micro VM](ipfs_microvm.md)
+- [AI & ML Workloads](ai_ml_workloads.md)
--- a/collections/system_administrators/advanced/ai_ml_workloads.md
+++ b/collections/system_administrators/advanced/ai_ml_workloads.md
@ -0,0 +1,125 @@
+<h1> AI & ML Workloads </h1>
+
+<h2> Table of Contents </h2>
+
+- [Introduction](#introduction)
+- [Prerequisites](#prerequisites)
+- [Prepare the System](#prepare-the-system)
+- [Install the GPU Driver](#install-the-gpu-driver)
+- [Set a Python Virtual Environment](#set-a-python-virtual-environment)
+- [Install PyTorch and Test Cuda](#install-pytorch-and-test-cuda)
+- [Set and Access Jupyter Notebook](#set-and-access-jupyter-notebook)
+- [Run AI/ML Workloads](#run-aiml-workloads)
+
+***
+
+## Introduction
+
+We present a basic method to deploy artificial intelligence (AI) and machine learning (ML) on the TFGrid. For this, we make use of dedicated nodes and GPU support.
+
+In the first part, we show the steps to install the Nvidia driver of a GPU card on a full VM Ubuntu 22.04 running on the TFGrid.
+
+In the second part, we show how to use PyTorch to run AI/ML tasks.
+
+## Prerequisites
+
+You need to reserve a [dedicated GPU node](../../dashboard/deploy/node_finder.md#dedicated-nodes) on the ThreeFold Grid.
+
+## Prepare the System
+
+- Update the system
+    ```
+    dpkg --add-architecture i386
+    apt-get update
+    apt-get dist-upgrade
+    reboot
+    ```
+- Check the GPU info
+    ```
+    lspci | grep VGA
+    lshw -c video
+    ```
+
+## Install the GPU Driver
+
+- Download the latest Nvidia driver
+  - Check which driver is recommended
+      ```
+      apt install ubuntu-drivers-common
+      ubuntu-drivers devices
+      ```
+  - Install the recommended driver (e.g. with 535)
+      ```
+      apt install nvidia-driver-535
+      ```
+  - Reboot and reconnect to the VM
+- Check the GPU status
+    ```
+    nvidia-smi
+    ```
+
+Now that the GPU node is set, let's work on setting PyTorch to run AI/ML workloads.
+
+## Set a Python Virtual Environment
+
+Before installing Python package with pip, you should create a virtual environment.
+
+- Install the prerequisites
+  ```
+  apt update
+  apt install python3-pip python3-dev
+  pip3 install --upgrade pip
+  pip3 install virtualenv
+  ```
+- Create a virtual environment
+  ```
+  mkdir ~/python_project
+  cd ~/python_project
+  virtualenv python_project_env
+  source python_project_env/bin/activate
+  ```
+
+## Install PyTorch and Test Cuda
+
+Once you've created and activated a virtual environment for Pyhton, you can install different Python packages.
+
+- Install PyTorch and upgrade Numpy
+    ```
+    pip3 install torch
+    pip3 install numpy --upgrade
+    ```
+
+Before going further, you can check if Cuda is properly installed on your machine.
+
+- Check that Cuda is available on Python with PyTorch by using the following lines:
+    ```
+    import torch
+    torch.cuda.is_available()
+    torch.cuda.device_count() # the output should be 1
+    torch.cuda.current_device() # the output should be 0
+    torch.cuda.device(0)
+    torch.cuda.get_device_name(0)
+    ```
+
+## Set and Access Jupyter Notebook
+
+You can run Jupyter Notebook on the remote VM and access it on your local browser.
+
+- Install Jupyter Notebook 
+    ```
+    pip3 install notebook
+    ```
+- Run Jupyter Notebook in no-browser mode and take note of the URL and the token
+  ```
+  jupyter notebook --no-browser --port=8080 --ip=0.0.0.0
+  ```
+- On your local machine, copy and paste on a browser the given URL but make sure to change `127.0.0.1` with the WireGuard IP (here it is `10.20.4.2`) and to set the correct token.
+  ```
+  http://10.20.4.2:8080/tree?token=<insert_token>
+  ```
+
+## Run AI/ML Workloads
+
+After following the steps above, you should now be able to run Python codes that will make use of your GPU node to compute AI and ML workloads.
+
+Feel free to explore different ways to use this feature. For example, the [HuggingFace course](https://huggingface.co/learn/nlp-course/chapter1/1) on natural language processing is a good introduction to machine learning.
--- a/collections/system_administrators/computer_it_basics/computer_it_basics.md
+++ b/collections/system_administrators/computer_it_basics/computer_it_basics.md
@ -12,4 +12,5 @@ In this section, tailored specifically for system administrators, we'll delve in
 - [Firewall Basics](firewall_basics.md)
  - [UFW Basics](ufw_basics.md)
  - [Firewalld Basics](firewalld_basics.md)
- [File Transfer](file_transfer.md)
+- [File Transfer](file_transfer.md)
+- [Screenshots](screenshots.md)
--- a/collections/system_administrators/computer_it_basics/screenshots.md
+++ b/collections/system_administrators/computer_it_basics/screenshots.md
@ -0,0 +1,75 @@
+<h1> Screenshots </h1>
+
+<h2>Table of Contents</h2>
+
+- [Introduction](#introduction)
+- [Linux](#linux)
+- [MAC](#mac)
+- [Windows](#windows)
+
+***
+
+## Introduction
+
+In this section, we show how to easily take screenshots on Linux, MAC and Windows.
+
+## Linux
+
+- Copy to the clipboard a full screenshot 
+```
+PrintScreen
+```
+- Copy to the clipboard a screenshot of an active window 
+```
+Alt + PrintScreen
+```
+- Copy to the clipboard a screenshot of an active app 
+```
+Control + Alt + PrintScreen
+```
+- Copy to the clipboard a screenshot of a selected area
+```
+Shift + PrintScreen
+```
+
+## MAC
+
+- Save to the desktop a full screenshot 
+```
+Shift + Command (⌘) + 3
+```
+- Save to the desktop a screenshot of an active window
+```
+Shift + Command (⌘) + 4 + Spacebar
+```
+- Copy to the clipboard a screenshot of an active window
+```
+Shift + Control + Command (⌘) + 3
+```
+- Save to the desktop a screenshot of a selected area
+```
+Shift + Command (⌘) + 4
+```
+- Copy to the clipboard a screenshot of a selected area
+```
+Shift + Control + Command (⌘) + 4
+```
+
+## Windows
+
+- Copy to the clipboard a full screenshot 
+```
+PrintScreen
+```
+- Save to the pictures directory a full screenshot  
+```
+Windows key + PrintScreen
+```
+- Copy to the clipboard a screenshot of an active window 
+```
+Alt + PrintScreen
+```
+- Copy to the clipboard a selected area of the screen
+```
+Windows key + Shift + S
+```
--- a/collections/system_administrators/mycelium/information.md
+++ b/collections/system_administrators/mycelium/information.md
@ -14,6 +14,7 @@
 - [API](#api)
 - [Message System](#message-system)
 - [Inspecting Node Keys](#inspecting-node-keys)
+- [Permanently Enable Mycelium](#permanently-enable-mycelium)

 ***

@ -150,4 +151,32 @@ Where the output could be something like this:
 ```sh
 Public key: a47c1d6f2a15b2c670d3a88fbe0aeb301ced12f7bcb4c8e3aa877b20f8559c02
 Address: 27f:b2c5:a944:4dad:9cb1:da4:8bf7:7e65
+```
+
+## Permanently Enable Mycelium
+
+It is possible to permenently enable Mycelium on your client.
+
+For Linux, we use systemd to manage the mycelium daemon in `/storage/`. Here's an example:
+
+```
+[Unit]
+Description=End-2-end encrypted IPv6 overlay network
+Wants=network.target
+After=network.target
+Documentation=https://github.com/threefoldtech/mycelium
+[Service]
+ProtectHome=true
+ProtectSystem=true
+SyslogIdentifier=mycelium
+CapabilityBoundingSet=CAP_NET_ADMIN
+StateDirectory=mycelium
+StateDirectoryMode=0700
+ExecStartPre=+-/sbin/modprobe tun
+ExecStart=/storage/mycelium --no-tun --disable-peer-discovery -k %S/mycelium/key.bin --peers tcp://[2a01:4f8:221:1e0b::2]:9651 tcp://[2a01:4f8:212:fa6::2]:9651 tcp://[2a02:1802:5e:0:8478:51ff:fee2:3331]:9651 tcp://[2a02:1802:5e:0:8c9e:7dff:fec9:f0d2]:9651 tcp://[2a01:4f9:5a:1042::2]:9651
+Restart=always
+RestartSec=5
+TimeoutStopSec=5
+[Install]
+WantedBy=multi-user.target
 ```
--- a/collections/system_administrators/system_administrators.md
+++ b/collections/system_administrators/system_administrators.md
@ -72,6 +72,7 @@ For complementary information on ThreeFold grid and its cloud component, refer t
    - [UFW Basics](ufw_basics.md)
    - [Firewalld Basics](firewalld_basics.md)
  - [File Transfer](file_transfer.md)
+  - [Screenshots](screenshots.md)
 - [Advanced](advanced.md)
  - [Token Transfer Keygenerator](token_transfer_keygenerator.md)
  - [Cancel Contracts](cancel_contracts.md)
@ -80,4 +81,5 @@ For complementary information on ThreeFold grid and its cloud component, refer t
  - [Redis](grid3_redis.md)
  - [IPFS](ipfs_toc.md)
    - [IPFS on a Full VM](ipfs_fullvm.md)
-    - [IPFS on a Micro VM](ipfs_microvm.md) 
+    - [IPFS on a Micro VM](ipfs_microvm.md) 
+  - [AI & ML Workloads](ai_ml_workloads.md)