Introduction
Vector databases are a new paradigm optimized for handling machine learning models and vectorized data used in modern applications like recommendations, search, and AI. In this article, we will understand what vector databases are, their technical architecture, when to use them, and walk through a case study highlighting their benefits.
What are Vector Databases?
Vector databases are a category of NoSQL database optimized for storing and querying vector data types like embeddings or feature vectors used in machine learning models. Examples of vector data include:
- Word, document, product embeddings used in search and recommendations
- Image feature vectors extracted by CNNs
- Audio embeddings extracted from speech data
- Node embeddings in knowledge graphs
Traditional databases like PostgreSQL or MySQL are not designed for efficiently handling such large, high-dimensional vector data. Vector databases fill this gap with capabilities like:
- Optimized storage for dense numerical vector data
- Support for common vector operations like KNN, similarity search
- Integration with data science notebooks and frameworks
- Auto-indexing of vectors to accelerate queries
- APIs for embedding management and lifecycle
This makes vector databases ideal for productionalizing ML models that rely heavily on embeddings and vector data.
Technical Architecture
Vector databases synthesize capabilities from several technologies:
- Document store – Stores vectors as documents for optimized data encoding
- Search engine – Indexes vectors allowing low latency similarity searches
- Column store – Column storage maximizes retrieval performance
- Distributed compute – Clustering provides scalability and availability
Additionally, they incorporate algorithmic techniques like hierarchical navigable small world graphs, locality-sensitive hashing, and more tailored specifically for efficient vector indexing and neighbor lookups.
Many vector databases provide client libraries in Python, Java, Go etc. to query easily from data science notebooks and applications. Auto-generated API clients are also common.
How does Vector Databases works?
Most of us are familiar with the workings of traditional databases—they store data in the form of strings, numbers, and other scalar types within rows and columns. However, when it comes to vector databases, the operating principles and optimization techniques are distinctly different.
In conventional databases, we typically search for rows where the data matches our query precisely. In contrast, vector databases employ a similarity metric to locate a vector that closely resembles our query.
Vector databases leverage an assortment of algorithms, all contributing to the task of Approximate Nearest Neighbor (ANN) search. These algorithms enhance search efficiency through techniques like hashing, quantization, or graph-based searches.
These algorithms are orchestrated into a pipeline that delivers swift and precise retrieval of neighboring vectors in response to a query. Since vector databases offer approximate results, there is a fundamental trade-off between accuracy and speed. The higher the accuracy desired, the slower the query may become. Nevertheless, a well-designed system can achieve lightning-fast searches while maintaining nearly impeccable accuracy.
Vector Database Operations Explained:
- Indexing: Vector databases employ algorithms like PQ, LSH, or HNSW (details below) to index vectors. This critical step transforms vectors into a data structure optimized for swift retrieval.
- Querying: The vector database then compares the indexed query vector with the dataset’s indexed vectors to locate the closest matches, employing a similarity metric inherent to the chosen indexing method.
- Post Processing: Occasionally, the vector database retrieves the ultimate nearest neighbors from the dataset and undertakes post-processing to deliver the final results. This phase may involve re-ranking the nearest neighbors using an alternate similarity measure
When to Use Vector Databases
Some good use cases include:
- Recommendation systems – Powering product recommendations using item or user embeddings
- Information retrieval – Finding similar documents or passages using text embeddings
- Fraud detection – Clustering user sessions and behavioral vectors to detect outliers
- Conversational AI – Storing chatbot conversation flows as vectors
- Inventory optimization – Organizing product inventory positions using demand forecast vectors
- Content moderation – Flagging abusive text by comparing against unsafe text vectors
Essentially any application leveraging vectorized representations of data for improved semantics can benefit from a vector database.
Case Study: Powering Recommendations at Scale
Let’s walk through how OnlineShop, an e-commerce site, leveraged a vector database to improve its product recommendation engine.
The Problem
OnlineShop was using a PostgreSQL database to store product metadata like title, description, images etc. To generate recommendations, product embeddings were generated by analyzing user views and co-viewing patterns using neural networks.
These 500-dimensional product embedding vectors were initially stored in PostgreSQL as floats across 500 columns. Generating recommendations required computationally intensiveKNN similarity lookups across these columns.
As the product catalog and user activity grew, recommendation latency increased exponentially. The database could not handle the explosion in vector data volume and dimensionality.
The Solution
OnlineShop migrated its product embedding storage to Weaviate, an open-source vector db. Weaviate imported the existing embedding data seamlessly. The special data schema optimized for vectors provided 20x compression rate.
Now similarity searches like finding top 10 products nearest to product X’s embedding vector ran blazingly fast. Multi-threaded batching allowed generating recommendations for all products in parallel.
Weaviate scaled seamlessly with OnlineShop’s growth. Recommendation latency reduced from minutes to milliseconds while traffic doubled. OnlineShop also saved compute resources worth $100K/month previously spent on ML infrastructure.
Key Takeaways
By adopting a vector db, OnlineShop unlocked excellent product recommendations at scale. The project demonstrated key vector database capabilities:
- Drastic reduction in storage needs via vector-optimized encoding
- Massive acceleration in similarity search performance
- Easy integration with existing ML pipelines
- Scalability to handle increasing users and catalog without degradation
- Significant cost savings from optimized infrastructure
The vector data model gave OnlineShop a future-proof foundation for its recommendation engine.
Conclusion
Vector dbs fill a crucial gap in the data engineering stack – efficiently storing and querying vectorized data used pervasively in modern machine learning. Their unique architecture combines the capabilities of multiple technologies like search engines, column stores, distributed systems etc. tailored for high-dimensional vector workloads. If your applications leverage embeddings or feature vectors, evaluating vector databases could provide big gains in performance, scale and TCO. Multiple open source and cloud offerings make it easy to try them out today.
Frequently Asked Questions
- What are the typical use cases for a vector database?
Typical use cases include powering recommendations using embeddings, finding similar documents or passages in search, detecting fraud based on user behavior vectors, inventory optimization using demand forecast vectors, and more.
- How do vector databases encode the vector data?
They encode vectors in an optimized way leveraging column storage, document storage, and search engine indexing concepts to allow fast retrieval and dimensionality reduction.
- What operations do vector dbssupport?
Typical operations are searches to find nearest vector neighbors, clustering vectors, aggregations based on vectors, retrieving vectors for prediction, and lifecycle management.
- How do vector databases scale?
They leverage distributed architectures to scale across nodes and provide high availability. Auto-sharding vectors across nodes is common.
- What programming languages do they support?
Most support client libraries for Python, Java, Go. Some also offer REST APIs, gRPC interfaces and auto-generated clients.
- Can I use a vector database with scikit-learn?
Yes, you can query and retrieve vectors from the database into NumPy arrays or Pandas dataframes for use with libraries like scikit-learn.
- Does migrating to a vector database require remodeling?
In most cases, migrating existing vectors is straightforward. Special modeling constructs like custom schemas may be provided.
- How are vector db different from document databases?
They incorporate document storage principles but also add search engine, columnar storage, and vector-specific optimizations.
- When should I use a vector vs traditional database?
If your workload involves vectorized data like embeddings, feature vectors, use a vector db. Otherwise traditional RDBMS/NoSQL databases are likely better.
- Are there open source options available?
Yes, Weaviate and VectDB are two leading open source vector databases. Cloud hosted versions are also available.