Commit Graph

567 Commits

Author SHA1 Message Date
3dbcf00e9f vfs_basics working 2025-03-09 20:14:08 +01:00
708147435e vfs working 2025-03-09 20:01:00 +01:00
26289bb00f Merge branch 'development_actions007' into development_ourdb_new
# Conflicts:
#	lib/data/dedupestor/dedupestor.v
#	lib/data/dedupestor/dedupestor_test.v
2025-03-09 19:07:12 +01:00
1489b9f44f ... 2025-03-09 18:34:58 +01:00
e4045ef179 Merge branch 'development_grid_deploy' into development_actions007 2025-03-09 17:40:20 +01:00
13f482fa12 Merge branch 'development_bizmodel' into development_grid_deploy 2025-03-09 17:38:23 +01:00
3e2013576f ... 2025-03-09 17:37:37 +01:00
e5aa8bca09 Merge branch 'development_bizmodel' of github.com:freeflowuniverse/herolib into development_bizmodel 2025-03-09 17:25:22 +01:00
4dd3908ff7 ... 2025-03-09 17:24:44 +01:00
09f388e2ff ... 2025-03-09 15:59:57 +01:00
baecb9bbe4 Merge branch 'development_grid_deploy' into development_generator_docusaurus
* development_grid_deploy:
  Restore all needed for basic deployments, add vm example
  Update griddriver to use prebuilt binary
  Return Deployer and update references
  Update module paths
  Update grid proxy module path
  commetbft
  cleanup client for grid
2025-03-08 10:56:16 +01:00
2c5986295e Merge branch 'development_bizmodel' into development_generator_docusaurus
* development_bizmodel: (93 commits)
  s
  Revert "test: add cmdline parser tests"
  test: add cmdline parser tests
  markdown code
  ...
  revert
  ...
  ..deployments
  ...
  bump version to 1.0.21
  ...
  bump version to 1.0.20
  ...
  fix tests and example
  bump version to 1.0.19
  bump version to 1.0.18
  bump version to 1.0.17
  ...
  ...
  bump version to 1.0.16
  ...

# Conflicts:
#	lib/web/docusaurus/config.v
2025-03-08 10:55:58 +01:00
126f23dfa2 s 2025-03-08 08:56:14 +01:00
Scott Yeager
407f3f85bc Restore all needed for basic deployments, add vm example 2025-03-07 18:09:28 -08:00
timurgordon
94da4fff5e fix delete data decoding 2025-03-08 01:55:10 +01:00
Scott Yeager
0fa54f1354 Update griddriver to use prebuilt binary 2025-03-07 16:54:44 -08:00
timurgordon
80677f7177 lock support fixes 2025-03-08 01:53:39 +01:00
timurgordon
01cac0f741 fixes to chunked file data processing 2025-03-08 01:52:53 +01:00
Scott Yeager
30546a34f9 Return Deployer and update references 2025-03-07 16:39:52 -08:00
Scott Yeager
0ccf317564 Update module paths 2025-03-07 16:36:01 -08:00
Scott Yeager
0c49e83a68 Update grid proxy module path 2025-03-07 16:00:15 -08:00
51db8257f5 Revert "test: add cmdline parser tests"
This reverts commit ecc2977581.
2025-03-07 21:56:33 +01:00
ecc2977581 test: add cmdline parser tests 2025-03-07 21:54:04 +01:00
ffafef0c88 markdown code 2025-03-07 21:03:55 +01:00
ca3bac1d76 ... 2025-03-07 20:02:37 +01:00
43f7bc7943 revert 2025-03-07 18:36:45 +01:00
b3555aa54e ... 2025-03-07 10:09:39 +01:00
timurgordon
afad769066 Merge branch 'development_ourdb_new' of https://github.com/freeflowuniverse/herolib into development_ourdb_new 2025-03-06 15:50:03 +01:00
timurgordon
2da37ee4a5 fix propfind request handling and parsing, and file sending 2025-03-06 15:50:00 +01:00
timurgordon
90c81a1234 add path to fsentry metadata 2025-03-06 15:48:37 +01:00
Mahmoud Emad
34ea12ca23 Merge branch 'development_ourdb_new' of https://github.com/freeflowuniverse/herolib into development_ourdb_new 2025-03-06 13:33:07 +02:00
Mahmoud Emad
d2c1be5396 feat: Add basic key-value store example
- Added a client and server for a simple key-value store.
- Improved documentation with client and server usage examples.
- Created client and server implementations using the V language.
2025-03-06 13:32:57 +02:00
timurgordon
fe934bba36 webdav fixes 2025-03-06 01:24:04 +01:00
timurgordon
b01e40da40 fix vfsimplemetation, add path metadata 2025-03-06 01:23:17 +01:00
Mahmoud Emad
ae7e7ecb84 feat: Add OurDB key-value store server
- Adds a new lightweight key-value store server implemented in V.
- Includes basic CRUD operations (`set`, `get`, `delete`).
- Provides configurable host and operation restrictions for security.
- Offers middleware for logging and request validation.
- Supports incremental mode for automatic ID generation.
- Includes comprehensive documentation and example usage.
- Adds unit tests to ensure functionality and stability.
2025-03-05 23:02:35 +02:00
Mahmoud Emad
fdf540cbd0 docs: Add documentation for Mycelium Streamer example
- Added a new documentation file explaining the Mycelium Streamer
  example, covering setup, prerequisites, and execution.
- This provides users with clear instructions on how to use the
  example project.
2025-03-04 21:34:02 +02:00
timurgordon
7cb26e5569 fix webdav lock handling and xlm parsing 2025-03-04 00:50:40 +01:00
timurgordon
913b0cb790 fix vfs_db and add metadata_db 2025-03-04 00:49:57 +01:00
Mahmoud Emad
485b47d145 feat: Improve MyceliumStreamer worker communication
- Add worker registration to MyceliumStreamer:  Allows for explicit
  addition of workers, improving management and control.
- Simplify worker message handling: Streamlines message processing
  for increased efficiency and readability.  Removes unnecessary
  logging and simplifies message routing.
- Remove redundant message handling: Eliminates duplicate code
  paths for cleaner and more maintainable code.
- Improve worker data retrieval: Facilitates direct data retrieval
  from workers, enhancing efficiency and reliability.
2025-03-04 00:40:05 +02:00
fb9c9b8070 commetbft 2025-03-03 17:32:28 -05:00
Mahmoud Emad
5b69f935a5 feat: Improve MyceliumStreamer to read from workers
- Allow reading data directly from specific workers by specifying
  their public key in `MyceliumStreamer.read()`. This improves
  data retrieval flexibility and allows for distributed data access.
- Add master data reading to ensure data consistency and allow
  comparison between master and worker data.  This helps debug and
  verify data replication.
- Implement JSON encoding/decoding for database transfer between
  master and worker nodes. This enables efficient and structured
  data exchange.
2025-03-03 22:44:53 +02:00
Mahmoud Emad
9dbc36d634 feat: Enhance Deduplicated Mycelium example with multiple workers
- Updated the master node to support multiple workers, allowing for
  increased scalability and redundancy.
- Modified the worker node to simplify initialization and connection
  to the master.
- Added logging statements for better monitoring and debugging.
2025-03-03 17:34:41 +02:00
Mahmoud Emad
42b0c4d48f refactor: Improve MyceliumStreamer's message handling
- Removed unnecessary test data from `deduped_mycelium_master.vsh`.
- Simplified `MyceliumStreamer.listen()` to efficiently handle
  incoming messages, removing redundant code and improving readability.
- Enhanced error handling in `MyceliumStreamer.listen()` for more robust
  operation.
2025-03-03 16:58:13 +02:00
Mahmoud Emad
5343d9bff6 feat: Add unique ID to MyceliumStreamer
- Added a unique ID to the MyceliumStreamer struct for better
  identification of individual streamer instances.
2025-03-03 14:21:25 +02:00
Mahmoud Emad
eb47c8490d feat: Enhance Mycelium streamer with worker support
- Added worker ID to master and worker configurations for improved
  identification and management.
- Implemented worker registration and data synchronization mechanisms
  to enable distributed data access.
- Added a read function to retrieve data from specific workers,
  enhancing data access flexibility.
- Improved logging for better monitoring and debugging of the system.
2025-03-03 14:18:55 +02:00
Mahmoud Emad
1de9be2a8a feat: Improve Mycelium Streamer functionality
- Added continuous data writing and verification to the master node
  to ensure data persistence and integrity.
- Simplified worker update handling in the `listen` function for
  better efficiency and error handling.  The previous implementation
  had unnecessary complexity and potential for hangs.
2025-03-03 13:48:55 +02:00
Mahmoud Emad
d852ecc5b1 feat: Improve Mycelium client and streamer
- Changed Mycelium worker port to avoid conflict with master.
- Added debug print statements to Mycelium client for better troubleshooting.
- Removed unnecessary `SyncData` struct, simplifying data handling.
- Updated data encoding/decoding to directly use base64 for efficiency.
- Clarified message topic names for better understanding.
2025-03-03 12:50:59 +02:00
Mahmoud Emad
368edcd93a feat: Implement distributed database using Mycelium
- Refactor database streamer to support multiple workers.
- Add master node to manage and distribute data updates.
- Implement worker nodes to receive and apply updates.
- Remove unnecessary slave node.
- Improve error handling and logging.
- Use base64 encoding for JSON compatibility in data transfer.
2025-03-03 12:19:03 +02:00
Mahmoud Emad
71906fd891 feat: Improve Mycelium slave and streamer communication
- Renamed the topic for database synchronization messages from
  'db_sync' to 'sync_db' for clarity.
- Updated the Mycelium slave to decode base64 payload before
  processing and to log received messages and their source.
- Added logging to the Mycelium streamer to track sent messages.
- Added a new feature to retrieve and log the last index from the
  worker after syncing updates.  This improves monitoring and
  debugging capabilities.
2025-03-02 22:01:44 +02:00
Mahmoud Emad
dbd187a017 feat: Streamline mycelium example and add server port config
- Remove unnecessary code from the example to improve clarity.
- Add a configurable server port to the `MyceliumStreamer`.
2025-03-02 21:47:12 +02:00