Commit Graph

240 Commits

Author SHA1 Message Date
9fcdcc3aff ... 2025-03-31 09:32:58 +02:00
598b312140 ... 2025-03-24 06:44:39 +01:00
0df10f5cb3 Merge branch 'development_grid_deploy' into development
* development_grid_deploy:
  Update shebang
  Add simple filter example
2025-03-24 05:30:21 +01:00
2c748a9fc8 Merge branch 'development_actions007' into development
* development_actions007: (49 commits)
  ...
  bump version to 1.0.22
  add baobab mcp
  feat: Improve path normalization in `namefix`
  feat: Improve Qdrant client library
  test: Skip Jina client for now
  feat: Remove redundant Jina client code
  feat: Remove optional age field from Person struct
  feat: Improve DedupeStore and update tests
  test: Improve test coverage for fenced code block and list item parsers
  test: Improve test coverage for paragraph parsing
  test: Improve test coverage for markdown block parser
  test: Improve list parsing test cases
  feat: Improve Markdown parser list and table detection
  fix: Fix CI
  feat: Improve RadixTree debugging output
  refactor: Simplify ContactsDB methods
  feat: Add calendar VFS implementation
  feat: Add Contacts VFS module
  feat: Add contacts database and VFS implementation
  ...

# Conflicts:
#	.gitignore
#	lib/clients/qdrant/qdrant_client.v
#	lib/core/texttools/namefix.v
2025-03-24 05:30:15 +01:00
a2e1b4fb27 Merge branch 'development_action007_mahmoud' into development
* development_action007_mahmoud:
  feat: Add upsert points functionality to Qdrant client
  feat: Remove unnecessary delete_collection call in example
  feat: Add Qdrant client's retrieve_points functionality
  feat: Improve Qdrant client example
  ...
  feat: Add Jina server health check
  feat: Add multi-vector API support
  feat: Add classifier deletion functionality
  qdrant
  feat: Add classifier listing functionality
  feat: Enhance Jina client with improved classification API
  feat: Add train functionality to Jina client
  feat: Add Jina client training and classification features
  feat: Add reranking functionality to Jina client
  feat: Enhance Jina client with additional embedding parameters
  feat: Add create_embeddings function to Jina client
  fix: Ensure the code compiles and add a test example
  ...
  jina specs
2025-03-24 05:29:51 +01:00
9b0da9f245 Merge branch 'development_ourdb_new' into development
* development_ourdb_new: (115 commits)
  webdav completion wip
  Remove path from fsentry metadata, make vfs and webdav work again with fixes
  feat: Implement database synchronization using binary encoding
  Add documentation and tests for model_property.v
  feat: Add diagrams and README for OurDB syncer
  circle core objects work again
  ...
  ...
  radix tree has now prefix
  names
  models
  ...
  ....
  ...
  ...
  vfs_basics working
  vfs working
  ...
  ...
  ...
  ...

# Conflicts:
#	.gitignore
#	lib/code/generator/installer_client/ask.v
#	lib/code/generator/installer_client/factory.v
#	lib/code/generator/installer_client/generate.v
#	lib/code/generator/installer_client/model.v
#	lib/code/generator/installer_client/readme.md
#	lib/code/generator/installer_client/scanner.v
#	lib/code/generator/installer_client/templates/atemplate.yaml
#	lib/code/generator/installer_client/templates/heroscript_client
#	lib/code/generator/installer_client/templates/heroscript_installer
#	lib/code/generator/installer_client/templates/objname_actions.vtemplate
#	lib/code/generator/installer_client/templates/objname_factory_.vtemplate
#	lib/code/generator/installer_client/templates/objname_model.vtemplate
#	lib/code/generator/installer_client/templates/readme.md
#	lib/code/generator/installer_client_OLD/ask.v
#	lib/code/generator/installer_client_OLD/do.v
#	lib/code/generator/installer_client_OLD/generate.v
#	lib/code/generator/installer_client_OLD/model.v
#	lib/code/generator/installer_client_OLD/readme.md
#	lib/code/generator/installer_client_OLD/scanner.v
#	lib/code/generator/installer_client_OLD/templates/atemplate.yaml
#	lib/code/generator/installer_client_OLD/templates/heroscript_client
#	lib/code/generator/installer_client_OLD/templates/heroscript_installer
#	lib/code/generator/installer_client_OLD/templates/objname_actions.vtemplate
#	lib/code/generator/installer_client_OLD/templates/objname_factory_.vtemplate
#	lib/code/generator/installer_client_OLD/templates/objname_model.vtemplate
#	lib/code/generator/installer_client_OLD/templates/readme.md
#	lib/core/generator/installer_client_OLD/ask.v
#	lib/core/generator/installer_client_OLD/factory.v
#	lib/core/generator/installer_client_OLD/generate.v
#	lib/core/generator/installer_client_OLD/model.v
#	lib/core/generator/installer_client_OLD/readme.md
#	lib/core/generator/installer_client_OLD/scanner.v
#	lib/core/generator/installer_client_OLD/templates/atemplate.yaml
#	lib/core/generator/installer_client_OLD/templates/heroscript_client
#	lib/core/generator/installer_client_OLD/templates/heroscript_installer
#	lib/core/generator/installer_client_OLD/templates/objname_actions.vtemplate
#	lib/core/generator/installer_client_OLD/templates/objname_factory_.vtemplate
#	lib/core/generator/installer_client_OLD/templates/objname_model.vtemplate
#	lib/core/generator/installer_client_OLD/templates/readme.md
#	lib/core/texttools/namefix.v
2025-03-24 05:29:46 +01:00
b27f0a5017 ... 2025-03-17 11:13:39 +01:00
Timur Gordon
c47002430e implement mcc baobab generation example 2025-03-17 01:25:26 +01:00
Timur Gordon
ea0637423e Merge branch 'development' into development_actions007 2025-03-17 00:39:38 +01:00
Timur Gordon
133e7c9809 Merge branch 'development' into 6-openrpc-code-generator 2025-03-16 19:15:04 +01:00
Mahmoud Emad
228abe36a3 feat: Add upsert points functionality to Qdrant client
- Added `upsert_points` method to the Qdrant client to allow
  inserting and updating points in a collection. This enhances
  the client's ability to manage data efficiently.
- Improved error handling in Qdrant client methods to provide
  more informative error messages. This improves the user
  experience by providing better feedback on failed operations.
2025-03-16 14:58:23 +02:00
Mahmoud Emad
c3fe788a5b feat: Remove unnecessary delete_collection call in example
- Removed the `delete_collection` call from the Qdrant example
  to avoid unnecessary collection deletion. This simplifies the
  example and prevents potential issues if the collection doesn't
  exist.
- Updated `RetrievePointsParams` struct to use optional parameters
  for `shard_key`, `with_payload`, and `with_vectors`. This
  improves flexibility and reduces the required parameters.  The
  change simplifies the request structure.
2025-03-16 11:56:55 +02:00
Mahmoud Emad
025e8fba69 feat: Add Qdrant client's retrieve_points functionality
- Added a new `retrieve_points` function to the Qdrant client
  to retrieve points by their IDs. This allows for efficient
  fetching of specific points from a collection.
- Renamed `is_exists` to `is_collection_exists` for clarity
  and consistency.
- Added  `RetrievePointsRequest`, `RetrievePointsParams`, and
  `RetrievePointsResponse` structs for better structured data.
2025-03-16 11:40:52 +02:00
Scott Yeager
59a0519b4e Update shebang 2025-03-14 19:19:14 -07:00
Scott Yeager
dfcaeec85f Add simple filter example 2025-03-14 19:19:02 -07:00
Timur Gordon
ef922d162e implement groq example using openai client 2025-03-14 23:07:13 +01:00
Timur Gordon
475e812ba3 add mpc inspector example 2025-03-14 02:37:48 +01:00
Mahmoud Emad
e374520654 feat: Improve Qdrant client example
- Simplify Qdrant client example script, removing unnecessary
  boilerplate and improving readability.
- Add functions for creating, getting, deleting and listing
  collections.
- Add function to check collection existence.
- Improve error handling and logging.
2025-03-13 16:06:23 +02:00
Timur Gordon
47c95f76e9 Merge branch 'development' into development_actions007 2025-03-13 13:33:46 +01:00
Timur Gordon
c09e424890 refactor and reorganize code modules 2025-03-13 12:39:38 +01:00
9c09af83a2 Merge branch 'development_action007_mahmoud' into development_actions007 2025-03-13 07:15:40 +01:00
e69d67d238 Merge branch 'development_grid_deploy' into development_actions007 2025-03-13 07:15:34 +01:00
01c6ea66ac Merge branch 'development_ourdb_new' into development_actions007 2025-03-13 07:15:27 +01:00
f9ea731a6e Merge branch 'development_actions007' into development_action007_mahmoud
# Conflicts:
#	lib/clients/jina/jina_client.v
#	lib/clients/jina/jina_factory_.v
#	lib/clients/jina/jina_model.v
#	lib/clients/jina/rank_api.v
2025-03-13 07:14:09 +01:00
Scott Yeager
413b805823 Update caddy gateway example 2025-03-12 17:08:24 -07:00
5ba11aab46 ... 2025-03-12 16:36:17 +01:00
Mahmoud Emad
4abb46b4bf feat: Add Jina server health check
- Added a health check to the Jina client to verify server availability.
- Improved error handling and messaging for failed health checks.
- Enhanced client robustness by providing feedback on server status.
2025-03-12 16:38:38 +02:00
Mahmoud Emad
a7976c45f9 feat: Add multi-vector API support
- Added a new `create_multi_vector` function to the Jina client
  to support creating multi-vector embeddings.
- Added a new `multi_vector_api.v` file containing the
  implementation for the multi-vector API.
- Updated the `jina.vsh` example to demonstrate the usage of the
  new multi-vector API.
2025-03-12 16:16:37 +02:00
Mahmoud Emad
5194fabe62 feat: Add classifier deletion functionality
- Added `delete_classifier` function to delete a classifier by ID.
- Added corresponding unit tests for the new function.
- Updated the client example to demonstrate classifier deletion.
- Renamed `jina_client_test.v` to `api_test.v` for better organization.
- Renamed `model_embed.v` to `embeddings_api.v` for better organization.
- Refactored the embedding API to use enums for task and truncate types,
  and added error handling for invalid inputs.
2025-03-12 14:29:40 +02:00
timurgordon
ff430c2e4d Remove path from fsentry metadata, make vfs and webdav work again with fixes 2025-03-12 02:16:40 +01:00
1581628ea3 Merge branch 'development_action007_mahmoud' into development_actions007 2025-03-11 20:41:52 +01:00
237f9bd742 qdrant 2025-03-11 20:41:33 +01:00
Mahmoud Emad
cf27e7880e feat: Add classifier listing functionality
- Added a new function to list available classifiers.
- Extended the Jina client with `list_classifiers()` method.
- Added unit tests to verify the new functionality.
2025-03-11 21:38:06 +02:00
Mahmoud Emad
ad300c068f feat: Enhance Jina client with improved classification API
- Update `jina.vsh` example to showcase the new classification API
  with support for both text and image inputs. This improves
  the flexibility and usability of the client.
- Introduce new structs `TextDoc`, `ImageDoc`, `ClassificationInput`,
  `ClassificationOutput`, `ClassificationResult`, and `LabelScore`
  to represent data structures for classification requests and
  responses. This enhances code clarity and maintainability.
- Implement the `classify` function in `jina_client.v` to handle
  classification requests with support for text and image inputs,
  model selection, and label specification. This adds a crucial
  feature to the Jina client.
- Add comprehensive unit tests in `jina_client_test.v` to cover
  the new `classify` function's functionality. This ensures the
  correctness and robustness of the implemented feature.
- Remove redundant code related to old classification API and data
  structures from `model_embed.v`, `model_rank.v`, and
  `jina_client.v`. This streamlines the codebase and removes
  obsolete elements.
2025-03-11 21:11:04 +02:00
Mahmoud Emad
9ecc2444aa feat: Add Jina client training and classification features
- Added `train` function to the Jina client for training
  classifiers.
- Added `ClassificationTrain` struct to define training
  parameters.
- Added `TrainingExample` struct to represent training data.
- Added `ClassificationTrainOutput` struct for the training
  response.
- Added a new `classification_api.v` module for classifier
  training functionalities.
- Added a new `classify` function to the Jina client for
  classification tasks (currently commented out).
2025-03-11 20:17:35 +02:00
Mahmoud Emad
0e1836c5d0 feat: Add reranking functionality to Jina client
- Added a new `rerank` function to the Jina client for reranking documents.
- Added a new `RerankParams` struct to define parameters for reranking.
- Added unit tests for the new `rerank` function.
- Updated the example script to demonstrate reranking.
- Improved error handling and added more comprehensive logging.
2025-03-11 19:27:01 +02:00
Mahmoud Emad
7965883744 feat: Enhance Jina client with additional embedding parameters
- Add `type_`, `truncate`, and `late_chunking` parameters to the
  `create_embeddings` function for finer control over embedding
  generation. This allows users to specify embedding type,
  truncation method, and whether to apply late chunking.
- Rename model parameter to `model` for clarity and consistency.
- Improve model enum naming for better readability and API consistency.
- Add unit tests for the `create_embeddings` function to ensure
  correct functionality and handle potential errors.
2025-03-11 17:45:55 +02:00
Mahmoud Emad
b006bb1e41 feat: Add create_embeddings function to Jina client
- Added a `create_embeddings` function to the Jina client to
  generate embeddings for given input texts.
- Improved the `create_embeddings` function input parameters
  for better flexibility and error handling.
- Updated `TextEmbeddingInput` struct to handle optional
  parameters for embedding type, truncation type, and late
  chunking.  This improves the flexibility of the embedding
  generation process.
2025-03-11 17:18:47 +02:00
Mahmoud Emad
27c9018c48 fix: Ensure the code compiles and add a test example
- Fixed compilation issues and ensured the code builds successfully
- Created an example to test the client functionality
- Started implementing additional endpoints
2025-03-11 16:49:39 +02:00
9448ae85cf ... 2025-03-11 14:07:16 +01:00
b1453e3580 jina specs 2025-03-11 11:23:58 +01:00
a96903da0e ... 2025-03-09 21:17:32 +01:00
3dbcf00e9f vfs_basics working 2025-03-09 20:14:08 +01:00
26289bb00f Merge branch 'development_actions007' into development_ourdb_new
# Conflicts:
#	lib/data/dedupestor/dedupestor.v
#	lib/data/dedupestor/dedupestor_test.v
2025-03-09 19:07:12 +01:00
13f482fa12 Merge branch 'development_bizmodel' into development_grid_deploy 2025-03-09 17:38:23 +01:00
e5aa8bca09 Merge branch 'development_bizmodel' of github.com:freeflowuniverse/herolib into development_bizmodel 2025-03-09 17:25:22 +01:00
4dd3908ff7 ... 2025-03-09 17:24:44 +01:00
Scott Yeager
407f3f85bc Restore all needed for basic deployments, add vm example 2025-03-07 18:09:28 -08:00
Scott Yeager
0ccf317564 Update module paths 2025-03-07 16:36:01 -08:00
Scott Yeager
0c49e83a68 Update grid proxy module path 2025-03-07 16:00:15 -08:00