EPIC - Update 3rd Party Dependencies

[markpollack]

Need to update to the latest versions of third party dependencies for vector stores and AI models.

Vector Stores

• Azure Cosmos DB
• Azure Search
• Cassandra
• Chroma
• Coherence
• Couchbase
• Elasticsearch
• Gemfire
• HanaDB
• MariaDB
• Milvus
• MongoDB Atlas
• Neo4j
• OpenSearch
• Oracle
• PostgreSQL (pgvector)
• Pinecone
• Qdrant
• Redis
• Typesense
• Weaviate

AI Models

• Amazon Bedrock
• Azure
• Oracle
• Transformers
• Vertex Embedding
• Vertex Gemini
• Watson

[markpollack]

1. Azure Cosmos DB
   - Current: 5.17.1
   - Latest: 5.22.0
2. Azure Search Documents
   - Current: 11.6.1
   - Latest: 11.7.6
3. Coherence
   - Current: 24.09
   - Latest: 24.10
4. Couchbase Client
   - Current: 3.7.8
   - Latest: 3.7.9
5. OpenSearch Java
   - Current: 2.10.1
   - Latest: 2.23.0
6. MariaDB JDBC
   - Current: 3.5.1
   - Latest: 3.5.3
7. Typesense
   - Current: 0.5.0
   - Latest: 1.3.0
8. Milvus
   - Current: 2.5.4
   - Latest: 2.5.7
9. Weaviate
   - Current: 5.0.1
   - Latest: 5.2.0
10. Redis Jedis
    - Current: 5.1.0
    - Latest: 5.2.0
11. Neo4j neo4j-cypher-dsl-bom
    - Current: 2024.0.2
    - Latest: 2024.5.1

The SDKs that appear to be current include:

• Pinecone
• SAP HANA
• pgvector (PostgreSQL)
• qdrant
• Elasticsearch (managed by Boot)
• MongoDB Atlas (managed by Boot)
• Oracle (managed by Boot)
• Cassandra (managed by Boot)

[markpollack]

AI Model SDK Version Report

Current vs Latest Versions

Key Findings

1. All SDKs need updates: Every third-party AI model SDK is behind the latest version.
2. AWS Bedrock: Major version update from 2.29.29 to 2.31.23. This is the most significant version leap among all SDKs.
3. DJL Ecosystem: All DJL components (API, Model-Zoo, HuggingFace Tokenizers, PyTorch Engine) need to be updated from 0.30.0 to 0.32.0. Since they're typically
   updated together, this should be a coordinated update.
4. Azure OpenAI: Still in beta stage (1.0.0-beta.14 to 1.0.0-beta.16), indicating ongoing development.

Potential Impact and Considerations

1. Breaking Changes:
   - AWS Bedrock and Azure OpenAI updates may include API changes as they're still evolving
   - DJL's upgrade from 0.30.0 to 0.32.0 might introduce compatibility issues
2. New Capabilities:
   - ONNX Runtime 1.21.0 likely includes performance improvements and new operator support
   - Vertex AI updates may add support for newer models like Gemini 1.5
3. Authentication and API Structure:
   - No indication of authentication method changes in the version updates
   - AWS SDK updates often introduce new service features
4. Implementation Strategy:
   - Consider updating the DJL ecosystem components together
   - Test AWS Bedrock and Azure OpenAI updates thoroughly due to their beta/evolving status
   - Create integration tests for each updated SDK to verify functionality

SDKs Using HTTP Clients (No Direct SDK Dependency)

The following AI models don't use dedicated SDKs but rely on Spring's HTTP clients:

• Anthropic
• OpenAI
• Mistral AI
• Ollama
• Stability AI
• HuggingFace (uses Swagger-generated client from OpenAPI spec)

These implementations require manual checking of API compatibility with the services.

Recommended Next Steps

1. Update Azure OpenAI SDK to 1.0.0-beta.16 [DONE]
2. Update AWS Bedrock SDK to 2.31.23
3. Update IBM Watson SDK Core to 9.23.1
4. Coordinate update of all DJL components to 0.32.0
5. Update ONNX Runtime to 1.21.0
6. Check for deprecated methods or API changes in release notes before updating
7. Run integration tests after updates to verify functionality