info_tfgrid/collections/tech/qsss/qss_filesystem.md

62 lines
3.2 KiB
Markdown
Raw Normal View History

2024-02-27 11:11:12 +00:00
# Quantum Safe Filesystem
2024-04-23 12:20:22 +00:00
![](qss_fs_arch.png)
2024-04-04 12:02:26 +00:00
2024-02-27 11:11:12 +00:00
A redundant filesystem, can store PB's (millions of gigabytes) of information.
Unique features:
2024-04-04 12:02:26 +00:00
- Unlimited scalability (many petabytes)
2024-02-27 11:11:12 +00:00
- Quantum Safe:
2024-04-23 12:20:22 +00:00
- No farmer knows what the data is
2024-04-04 12:02:26 +00:00
- Even a quantum computer cannot decrypt the data
2024-02-27 11:11:12 +00:00
- Data can't be lost
- Protection for datarot, data will autorepair
2024-04-04 12:02:26 +00:00
- Data is kept forever (data does not get deleted)
2024-02-27 11:11:12 +00:00
- Data is dispersed over multiple sites
2024-04-04 12:02:26 +00:00
- Even if the sites go down the data will not be lost
2024-02-27 11:11:12 +00:00
- Up to 10x more efficient than storing on classic storage cloud systems
2024-04-23 12:20:22 +00:00
- Can be mounted as filesystem on any OS or any deployment system (OSX, Linux, Windows, Docker, Kubernetes etc.)
2024-02-27 11:11:12 +00:00
- Compatible with ± all data workloads (not high performance data driven workloads like a database)
- Self-healing: when a node or disk is lost, the storage system can get back to the original redundancy level
2024-04-04 12:02:26 +00:00
- Helps with compliance for regulations like GDPR (as the hosting facility has no view on what is stored: information is encrypted and incomplete)
- Hybrid: can be installed onsite, public and private
2024-02-27 11:11:12 +00:00
- Read-write caching on encoding node (the front end)
2024-04-04 12:02:26 +00:00
## Mount Any Files In Your Storage Infrastructure
2024-02-27 11:11:12 +00:00
2024-04-04 12:02:26 +00:00
The QSFS is a mechanism to mount any file system (in any format) on the grid, in a quantum secure way.
2024-02-27 11:11:12 +00:00
2024-04-23 12:20:22 +00:00
This storage layer relies on 3 primitives:
2024-02-27 11:11:12 +00:00
- [0-db](https://github.com/threefoldtech/0-db) is the storage engine.
2024-04-04 12:02:26 +00:00
It is an always append database, which stores objects in an immutable format. It allows history to be kept out-of-the-box, good performance on disk, low overhead, easy data structure and easy backup (linear copy and immutable files).
2024-02-27 11:11:12 +00:00
- [0-stor-v2](https://github.com/threefoldtech/0-stor_v2) is used to disperse the data into chunks by performing 'forward-looking error-correcting code' (FLECC) on it and send the fragments to safe locations.
It takes files in any format as input, encrypts the file with AES based on a user-defined key, then FLECC-encodes the file and spreads out the result
2024-04-04 12:02:26 +00:00
to multiple 0-DBs. The number of generated chunks is configurable to make it more or less robust against data loss through unavailable fragments. Even if some 0-DBs are unreachable, you can still retrieve the original data, and missing 0-DBs can even be rebuilt to have full consistency. It is an essential element of the operational backup.
2024-02-27 11:11:12 +00:00
- [0-db-fs](https://github.com/threefoldtech/0-db-fs) is the filesystem driver which uses 0-DB as a primary storage engine. It manages the storage of directories and metadata in a dedicated namespace and file payloads in another dedicated namespace.
Together they form a storage layer that is quantum secure: even the most powerful computer can't hack the system because no single node contains all of the information needed to reconstruct the data.
2024-04-23 12:20:22 +00:00
2024-02-27 11:11:12 +00:00
This concept scales forever, and you can bring any file system on top of it:
- S3 storage
- any backup system
- an ftp-server
- IPFS and Hypercore distributed file sharing protocols
## Architecture
2024-04-23 12:20:22 +00:00
By using our filesystem inside a Virtual Machine or Kubernetes, the cloud user can deploy any storage application on top e.g. Minio for S3 storage, OwnCloud as online fileserver.
2024-02-27 11:11:12 +00:00
![](img/qsstorage_architecture.jpg)
Any storage workload can be deployed on top of the zstor.