tfgrid/circle_engineering
22
0
Fork 0
Code Issues 61 Pull Requests Packages Projects 1 Releases Wiki Activity

rfs & zos improvements #59

New Issue
Closed
opened 2024-05-17 07:10:56 +00:00 by despiegk · 3 comments
despiegk commented 2024-05-17 07:10:56 +00:00
Owner
Copy Link
  • support compression of flist (choose a strong one)
  • rfs: add cmd to change source of data (so flist points to e,g, data on S3 but was built on FS)
  • rfs: copy one location e.g. dir to other location e.g.s3
  • rfs: multithreads (or coroutines) when building flist or when copying, so we get 10x speed
  • rfs: support super simple http on top of the DIR (so can be like https://[ipv6]/myflista/a/bdjfhadfhjgiadhfgjb)
  • rfs: make DIR 2 levels, now too many files in 1 dir, this makes it slow if too many flists, e.g. first byte is subdir
    • make sure rfs suppors old and new model (2 dirs, 1 dir, so it can mount both based on this option as specified in flist)
  • rfs: merge 2 flists
  • make sure ZOS can use mycelium addresses as backend for S3, HTTP, Flist location...
- support compression of flist (choose a strong one) - rfs: add cmd to change source of data (so flist points to e,g, data on S3 but was built on FS) - rfs: copy one location e.g. dir to other location e.g.s3 - rfs: multithreads (or coroutines) when building flist or when copying, so we get 10x speed - rfs: support super simple http on top of the DIR (so can be like https://[ipv6]/myflista/a/bdjfhadfhjgiadhfgjb) - rfs: make DIR 2 levels, now too many files in 1 dir, this makes it slow if too many flists, e.g. first byte is subdir - make sure rfs suppors old and new model (2 dirs, 1 dir, so it can mount both based on this option as specified in flist) - rfs: merge 2 flists - make sure ZOS can use mycelium addresses as backend for S3, HTTP, Flist location...
despiegk added this to the (deleted) project 2024-05-17 08:05:45 +00:00
despiegk added the
Story
 label 2024-05-17 08:05:51 +00:00
despiegk commented 2024-05-17 08:06:44 +00:00
Author
Owner
Copy Link

maybe not all should be in 3.14, but at least the ZOS integration with Mycelium so we can use the new format from people who host their flist and content on mycelium enabled location

maybe not all should be in 3.14, but at least the ZOS integration with Mycelium so we can use the new format from people who host their flist and content on mycelium enabled location
despiegk commented 2024-05-22 04:49:45 +00:00
Author
Owner
Copy Link

the file handling is being done in V so we can probably delay that, but what we need urgently is the ZOS support for mycelium

the file handling is being done in V so we can probably delay that, but what we need urgently is the ZOS support for mycelium
despiegk modified the project from (deleted) to tfgrid_3_14 2024-05-22 07:56:25 +00:00
despiegk commented 2024-06-02 05:10:44 +00:00
Author
Owner
Copy Link

done as part of hero now, appart form ZOS parts which is other story

done as part of hero now, appart form ZOS parts which is other story
despiegk removed this from the tfgrid_3_14 project 2024-06-04 12:21:48 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
  
Issue

   
Question

   
Roadmap

   
Story

   
Task

   
Urgent
No Label
Issue
Question
Roadmap
Story
Task
Urgent
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: tfgrid/circle_engineering#59
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?