w.r.t. LLM applications using the RAG (retriever-augmented-generation) architecture, people have started taken it for granted that it will be powered by a vector database. e.g., see this:
The most important piece of the preprocessing pipeline, from a systems standpoint, is the vector database.
Why can't lucene index (full-text search) be used for the retriever? Is there any objective study that has been done comparing quality of results using full-text search vs. using a vector database?
As I was writing this, even lucene seems to have jumped on the vector bandwagon. see this