...

_archive/technology/qsss/specs_todo/policy.md

@@ -0,0 +1,3 @@
+# zstor filesystem (zstor) Policy
+
+Describe how it works...

_archive/technology/qsss/specs_todo/qss_specs.md

@@ -0,0 +1,68 @@
+![specs](img/specs_header.jpg)
+
+# System requirements
+
+System that is easy to provision storage capacity on the TF grid
+- user can create X storage nodes on a random or specific locations
+- user can list their storage nodes
+- check node status/info in some shape or form in a monitoring solution
+- external authentication/payment system using threefold connect app
+- user can delete their storage nodes
+- user can provision mode storage nodes
+- user can increase total size of storage solutions
+- user can install the quantum safe filesystem on any linux based system, physical or virtual
+
+# Non-functional requirements
+
+- How many expected concurrent users: not application - each user will have it's own local binary and software install.
+- How many users on the system: 10000-100000
+- Data store: fuse filesystem plus local and grid based ZDB's
+- How critical is the system? it needs to be alive all the time.
+- What do we know about the external payment system?
+Threefold Connect, use QR code for payments and validate on the blockchain
+- Life cycle of the storage nodes? How does the user keep their nodes alive? The local binary / application has a wallet from which it can pay for the existing and new storage devices.  This wallet needs to be kept topped up.
+- When the user is asked to sign the deployment of 20 storage nodes:
+   - will the user sign each single reservation? or should the system itself sign it for the user and show the QR code only for payments?
+- Payments should be done to the a specific user wallet and with a background service with extend the user pools or /extend in the bot conversation? to be resolved
+- Configuration and all metadata should be stored as a hash / private key.  With this information you are able to regain access to your stored data from everywhere.
+
+
+# Components mapping / SALs
+
+- Entities: User, storage Node
+- ReservationBuilder: builds reservation for the user to sign (note the QR code data size limit is 3KB)
+- we need to define how many nodes can we deploy at a time, shouldn't exceed 3KB for the QR Code, if it exceeds the limit should we split the reservations?
+   
+- UserInfo: user info are loaded from threefold login system
+- Blockchain Node (role, configurations) 
+- Interface to Threefold connect (authentication+payment)  /identify + generate payments
+- User notifications / topup
+- Monitoring: monitoring + redeployment of the solutions again if they go down, when redeploying who owns the reservation to delete -can be fixed with delete signers field- and redeploy, but to deploy we need the user identity or should we inform the user in telegram and ask him to /redeploy
+- Logging 
+
+# Tech stack
+
+- [JS-SDK[](https://github.com/threefoldtech/js-sdk) (?)
+- [0-db](https://github.com/threefoldtech/0-db-s)
+- [0-db-fs](https://github.com/threefoldtech/0-db-fs)
+- [0-stor_v2](https://github.com/threefoldtech/0-stor_v2)
+- [quantum_storage](https://github.com/threefoldtech/quantum-storage)
+
+
+
+# Blockers
+
+
+Idea from blockchain jukekebox brainstorm:
+
+## payments
+- QR code contains threebot://signandpay/#https://tf.grid/api/a6254a4a-bdf4-11eb-8529-0242ac130003 (can also be uni link)
+- App gets URL
+- URL gives data
+- { DataToSign : {RESERVATIONDETAILS}, Payment: {PAYMENTDETAILS}, CallbackUrl: {CALLBACKURL} }
+- App signs reservation, makes payment, calls callbackurl { SingedData : {SINGEDRESERVATION}, Payment: {FINISHED_PAYMENTDETAILS}}
+
+Full flow:
+- User logs in using normal login flow
+- User scans QR
+- User confirms reservation and payment in the app

_archive/technology/qsss/specs_todo/quantum_safe_filesystem.md

@@ -0,0 +1,3 @@
+# Specs zstor filesystem
+
+- [Quantum Safe File System](quantum_safe_filesystem_2_6)

_archive/technology/qsss/specs_todo/quantum_safe_filesystem_2_6.md

@@ -0,0 +1,15 @@
+# zstor filesystem 2.6
+
+## requirements
+
+- redundancy/uptime
+  - data can never be lost if older than 20 min (avg will be 7.5 min, because we use 15 min push)
+  - if a datacenter or node goes down and we are in storage policy the storage stays available
+- reliability
+  - data cannot have hidden data corruption, when bitrot the FS will automatically recover
+- self healing
+  - when data policy is lower than required level then should re-silver (means make sure policy is intact again)
+
+## NEW
+
+- 100% redundancy

_archive/technology/qsss/specs_todo/quantum_safe_filesystem_architecture.md

@@ -0,0 +1,37 @@
+
+## zstor Architecture
+
+```mermaid
+graph TD
+    subgraph TFGridLoc2
+        ZDB5
+        ZDB6
+        ZDB7
+        ZDB8
+        ETCD3
+    end
+    subgraph TFGridLoc1
+        ZDB1
+        ZDB2
+        ZDB3
+        ZDB4
+        ETCD1
+        ETCD2
+        KubernetesController --> ETCD1
+        KubernetesController --> ETCD2
+        KubernetesController --> ETCD3
+    end
+
+
+    subgraph eVDC
+        PlanetaryFS --> ETCD1 & ETCD2 & ETCD3
+        PlanetaryFS --> MetadataStor
+        PlanetaryFS --> ReadWriteCache
+        MetadataStor --> LocalZDB
+        ReadWriteCache --> LocalZDB
+        LocalZDB & PlanetaryFS --> ZeroStor
+        ZeroStor --> ZDB1 & ZDB2 & ZDB3 & ZDB4 & ZDB5 & ZDB6 & ZDB7 & ZDB8
+    end    
+ 
+
+```

_archive/technology/qsss/specs_todo/quantum_safe_filesystem_sequence_graph.md

@@ -0,0 +1,40 @@
+## zstor Sequence Diagram
+
+```mermaid
+sequenceDiagram
+  participant user as user
+  participant fs as 0-fs
+  participant lzdb as local 0-db
+  participant zstor as 0-stor
+  participant etcd as ETCD
+  participant zdbs as backend 0-dbs
+  participant mon as Monitor
+
+  alt Writing data
+    user->>fs: write data to files
+    fs->>lzdb: write data blocks
+    opt Datafile is full
+      lzdb->>zstor: encode and chunk data file
+      zstor->>zdbs: write encoded datafile chunks to the different backends
+      zstor->>etcd: write metadata about encoded file to metadata storage
+    end
+  else Reading data
+    user->>fs: read data from file
+    fs->>lzdb: read data blocks 
+    opt Datafile is missing
+      lzdb->>zstor: request retrieval of data file
+      zstor->>etcd: load file encoding and storage metadata
+      zstor->>zdbs: read encoded datafile chunks from multiple backends and rebuilds original datafile
+      zstor->>lzdb: replaces the missing datafile
+    end
+  end
+
+  loop Monitor action
+    mon->>lzdb: delete local data files which are full and have been encoded, AND have not been accessed for some time
+    mon->>zdbs: monitors health of used namespaces
+    opt Namespace is lost or corrupted
+      mon->>zstor: checks storage configuration
+      mon->>zdbs: rebuilds missing shard on new namespace from storage config
+    end
+  end
+```