Managed vector database example

[arun-gupta]

A lot of customers use managed Vector database service. The current examples demonstrate a Vector database as a bundled microservice. To address the SaaS customers, there is a need to have an example that uses a managed Vector database.

[endomorphosis]

There is already an example of managed vector database, for example postgres, its just that there is not a managed pinecone database.

https://github.com/opea-project/GenAIComps/tree/main/comps/vectorstores/langchain/pgvector

[arun-gupta]

I'm looking for an example to use a managed service, such as Pinecone or Zilliz or something similar.

[pallavijaini0525]

@arun-gupta - have added implementations for the pinecone db.

dataprep - https://github.com/opea-project/GenAIComps/tree/main/comps/dataprep/pinecone
retriever - https://github.com/opea-project/GenAIComps/tree/main/comps/retrievers/langchain/pinecone
vector store - https://github.com/opea-project/GenAIComps/tree/main/comps/vectorstores/langchain/pinecone

[arun-gupta]

@pallavijaini0525 this is awesome. How do I use this with ChatQnA example?

[pallavijaini0525]

To use the pinecone DB,

1. instead of redis DB docker up and running, create the Pinecone API key and the index creation -https://github.com/opea-project/GenAIComps/tree/main/comps/vectorstores/langchain/pinecone -> No need to run a docker container for Pinecone as Pinecone DB is in cloud.
2. for Retriever service and Dataprep services in ChatQnA use the pinecone Dockerfiles instead of redis to build the images.
3. In the docker compose make the necessary changes to use the pinecone images.

[arun-gupta]

This is helpful.

Can you contribute a Docker Compose file that allows developers to run this entire configuration?

[pallavijaini0525]

@arun-gupta - Created the PR - #540 and opea-project/GenAIComps#420 to support Pinecone DB for the ChatQnA-

Verified the full pipeline