Feature Request: Support for `pgvector` in YugabyteDB

[fire]

Description

pgvector is a PostgreSQL extension that provides efficient storage and operations for vector data. It's particularly useful for machine learning applications, where high-dimensional vectors are common.

Benefits

1. Efficient Storage: pgvector stores vectors as binary data, which is more space-efficient than storing them as arrays.
2. Fast Operations: pgvector supports various vector operations like dot product, cosine similarity, etc., which can be executed much faster than equivalent operations on arrays.
3. Machine Learning Applications: With the rise of AI and ML, support for pgvector would make YugabyteDB an attractive option for developers in these fields.

Proposed Changes

• Add support for pgvector data type
• Implement functions for common vector operations.
• Ensure compatibility with existing PostgreSQL pgvector functions.

-- Example from the pgvector github
CREATE EXTENSION vector;
-- Create a vector column with 3 dimensions
CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(3));
-- Insert vectors
INSERT INTO items (embedding) VALUES ('[1,2,3]'), ('[4,5,6]');
-- Get the nearest neighbors by L2 distance
SELECT * FROM items ORDER BY embedding <-> '[3,1,2]' LIMIT 5;

The above SQL commands should work seamlessly in YugabyteDB after implementing this feature.

Conclusion

Supporting pgvector would significantly enhance YugabyteDB's capabilities, especially for AI/ML applications. This feature would make YugabyteDB a more versatile and powerful database solution.

This feature request was assisted by AI/ML technologies.

Warning: Please confirm that this issue does not contain any sensitive information

• I confirm this issue does not contain any sensitive information.

[fire]

Duplicate of #16166