Commit Graph

29 Commits

Author SHA1 Message Date
Mahmoud Emad
eb47c8490d feat: Enhance Mycelium streamer with worker support
- Added worker ID to master and worker configurations for improved
  identification and management.
- Implemented worker registration and data synchronization mechanisms
  to enable distributed data access.
- Added a read function to retrieve data from specific workers,
  enhancing data access flexibility.
- Improved logging for better monitoring and debugging of the system.
2025-03-03 14:18:55 +02:00
Mahmoud Emad
1de9be2a8a feat: Improve Mycelium Streamer functionality
- Added continuous data writing and verification to the master node
  to ensure data persistence and integrity.
- Simplified worker update handling in the `listen` function for
  better efficiency and error handling.  The previous implementation
  had unnecessary complexity and potential for hangs.
2025-03-03 13:48:55 +02:00
Mahmoud Emad
d852ecc5b1 feat: Improve Mycelium client and streamer
- Changed Mycelium worker port to avoid conflict with master.
- Added debug print statements to Mycelium client for better troubleshooting.
- Removed unnecessary `SyncData` struct, simplifying data handling.
- Updated data encoding/decoding to directly use base64 for efficiency.
- Clarified message topic names for better understanding.
2025-03-03 12:50:59 +02:00
Mahmoud Emad
368edcd93a feat: Implement distributed database using Mycelium
- Refactor database streamer to support multiple workers.
- Add master node to manage and distribute data updates.
- Implement worker nodes to receive and apply updates.
- Remove unnecessary slave node.
- Improve error handling and logging.
- Use base64 encoding for JSON compatibility in data transfer.
2025-03-03 12:19:03 +02:00
Mahmoud Emad
71906fd891 feat: Improve Mycelium slave and streamer communication
- Renamed the topic for database synchronization messages from
  'db_sync' to 'sync_db' for clarity.
- Updated the Mycelium slave to decode base64 payload before
  processing and to log received messages and their source.
- Added logging to the Mycelium streamer to track sent messages.
- Added a new feature to retrieve and log the last index from the
  worker after syncing updates.  This improves monitoring and
  debugging capabilities.
2025-03-02 22:01:44 +02:00
Mahmoud Emad
dbd187a017 feat: Streamline mycelium example and add server port config
- Remove unnecessary code from the example to improve clarity.
- Add a configurable server port to the `MyceliumStreamer`.
2025-03-02 21:47:12 +02:00
Mahmoud Emad
52a3546325 feat: Improve Mycelium-based data synchronization
- Refactor data synchronization logic to use Mycelium messages for
  efficient updates between master and worker nodes.  This removes
  the previous inefficient polling method and simplifies the code.
- Update the slave node to receive and apply updates from the
  master, improving synchronization efficiency and robustness.
- Change the default slave port to 9000.
- Rename `db` variable to `worker` for clarity.
2025-03-02 21:40:37 +02:00
Mahmoud Emad
a23eb0ff0b feat: Implement MyceliumStreamer for distributed data synchronization
- Introduces `MyceliumStreamer` for synchronizing data across a
  Mycelium network, enabling distributed data access.
- Allows adding multiple worker nodes to the streamer for data
  replication and redundancy.
- Provides `write` and `read` methods for seamless data
  management across nodes.
2025-03-02 20:25:29 +02:00
Mahmoud Emad
b40c366335 feat: Update slave communication in mycelium example
- Rename `get_last_id` to `get_last_index` for clarity.
- Correctly send sync message to the slave instead of master.
2025-03-02 15:17:00 +02:00
Mahmoud Emad
d2ad18e8ec feat: Implement two-way communication between master and slave nodes
- Added message reply functionality to both master and slave
  nodes to enable a two-way communication flow for database
  synchronization. This improves the robustness and reliability
  of the database synchronization process.
- Enhanced the database synchronization process by allowing the
  slave node to send the last inserted record ID to the master
  node.  This provides better tracking of data changes.
2025-03-02 15:14:51 +02:00
Mahmoud Emad
9b737c9280 feat: Improve Mycelium example with bidirectional communication
- Remove unnecessary public key printing in master node.
- Use variable for slave public key in master node.
- Add message receiving functionality to master node.
- Remove redundant sending logic from slave node.
2025-03-02 14:57:33 +02:00
Mahmoud Emad
a0b53126ca feat: Add example scripts for Mycelium inter-node communication
- Added `deduped_mycelium_master.vsh` to demonstrate a master node
  sending data.
- Added `deduped_mycelium_slave.vsh` to demonstrate a slave node
  receiving data.  These scripts showcase basic inter-node
  communication using the Mycelium library.
2025-02-27 14:46:57 +02:00
e34d804dda ... 2025-02-07 11:59:52 +03:00
0a757e8634 ... 2025-02-07 07:03:10 +03:00
a66ef2e8b3 ... 2025-02-06 21:09:20 +03:00
f4f5eb06a4 location fixed for postgresql client 2025-02-06 07:15:32 +03:00
c93fe755fd location postgresql 2025-02-06 06:57:23 +03:00
6bdc4a5b8f location is importing in psql 2025-02-06 06:48:47 +03:00
5ad2062e5c format 2025-02-06 06:26:44 +03:00
cfa9f877b3 search 2025-02-02 12:30:25 +03:00
e4f883d35a location 2025-02-02 12:17:33 +03:00
0ad1d27327 location 2025-02-02 11:48:19 +03:00
3bf2473c3a location 2025-02-02 08:10:32 +03:00
74ab68d05f ... 2025-01-31 15:39:44 +03:00
27cb6cb0c6 ... 2025-01-31 09:58:53 +03:00
5670efc4cb ... 2025-01-31 08:29:17 +03:00
f0fdf146a8 formatting 2024-12-30 17:36:22 +02:00
523a58cfec ... 2024-12-30 08:39:45 +01:00
7894f7d420 ... 2024-12-30 08:01:17 +01:00