Vector Databases Compared — pgvector vs Qdrant vs Weaviate vs Pinecone

Why Every AI Stack Now Has a Vector Layer

Two years ago, adding a vector database to a production system was an exotic engineering decision. Today it is a default assumption. The rapid adoption of LLM-powered features — semantic search, RAG pipelines, recommendation engines, duplicate detection — created a new class of storage problem: you need to store and retrieve high-dimensional floating-point embeddings at millisecond latency, at a scale where brute-force comparison is not viable.

The result is a crowded market. You can run vector search inside PostgreSQL via pgvector, deploy a purpose-built open-source engine like Qdrant or Weaviate, or hand the operational burden to a managed cloud service like Pinecone. Each choice involves real trade-offs across performance, operational complexity, filtering capability, and cost. This article gives you a concrete framework for deciding.

pgvector — Vector Search Inside PostgreSQL

pgvector adds a vector column type and approximate nearest neighbor (ANN) index types to standard PostgreSQL. If your application already runs on Postgres, pgvector eliminates an entire service from your architecture — embeddings live in the same database as the rest of your data, with the same transactions, backups, and access controls.

pgvector 0.7+ ships two index types: IVFFlat (inverted file with flat quantization — fast build, slightly lower recall) and HNSW (hierarchical navigable small world — slower build, better recall at low latency). HNSW is the default choice for production workloads. The pgvector HNSW documentation covers the m and ef_construction parameters in detail.

-- Enable the extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Store embeddings alongside structured data
CREATE TABLE documents (
  id          BIGSERIAL PRIMARY KEY,
  content     TEXT          NOT NULL,
  metadata    JSONB         NOT NULL DEFAULT '{}',
  embedding   vector(1536)  NOT NULL,  -- OpenAI text-embedding-3-small dims
  created_at  TIMESTAMPTZ   NOT NULL DEFAULT NOW()
);

-- HNSW index — build once, serves queries fast
-- m: number of connections per layer (16 is a good default)
-- ef_construction: search width during index build (64–128 is typical)
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops)
  WITH (m = 16, ef_construction = 64);

-- Similarity search — returns top 10 by cosine distance
-- ${query_embedding} would be the embedding of the search query
SELECT
  id,
  content,
  metadata,
  1 - (embedding <=> ${query_embedding}::vector) AS similarity
FROM documents
WHERE metadata->>'tenant_id' = ${tenant_id}   -- predicate pushdown
ORDER BY embedding <=> ${query_embedding}::vector
LIMIT 10;

-- Tune ef_search at query time for recall/speed trade-off
SET hnsw.ef_search = 100;

pgvector Limitations

pgvector shines for moderate scale (up to a few million vectors) but has real ceilings. HNSW indexes live entirely in shared memory — a 5M × 1536-dim index at float32 occupies roughly 30GB of RAM. Beyond ~10M vectors, query latency climbs and index builds become disruptive. Payload filtering is implemented as a post-filter on ANN results rather than a true filtered ANN, which means high-selectivity filters (fetching 1% of the dataset) degrade recall significantly unless you tune hnsw.ef_search upward.

Qdrant — Purpose-Built for High-Throughput Filtered Search

Qdrant is an open-source vector database written in Rust, designed from the ground up for production vector workloads. Its standout feature is filtrable HNSW: payload conditions (metadata filters) are pushed inside the HNSW graph traversal rather than applied as a post-filter. This preserves recall even for highly selective queries — exactly the scenario where pgvector struggles.

Qdrant supports multiple named vectors per point (sparse and dense), scalar and product quantization for memory reduction, on-disk indexing for datasets larger than RAM, and a gRPC API alongside REST. It runs as a single binary, a Docker container, or a Kubernetes StatefulSet, and offers a managed cloud tier at cloud.qdrant.io.

from qdrant_client import QdrantClient
from qdrant_client.models import (
    Distance,
    VectorParams,
    PointStruct,
    Filter,
    FieldCondition,
    MatchValue,
    Range,
)

client = QdrantClient(url="http://localhost:6333")

# Create a collection with HNSW config
client.recreate_collection(
    collection_name="documents",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
    hnsw_config={
        "m": 16,
        "ef_construct": 100,
        "full_scan_threshold": 10_000,  # below this, skip ANN entirely
        "on_disk": False,               # set True for datasets > RAM
    },
    quantization_config={
        "scalar": {
            "type": "int8",
            "quantile": 0.99,
            "always_ram": True,         # keep quantized vectors in RAM
        }
    },
)

# Upsert points with payload (metadata)
client.upsert(
    collection_name="documents",
    points=[
        PointStruct(
            id=1,
            vector=[0.1, 0.2, ...],    # 1536-dim embedding
            payload={
                "tenant_id": "acme",
                "category": "technical",
                "published_year": 2025,
                "score": 0.92,
            },
        ),
    ],
)

# Filtered search — filter is evaluated INSIDE HNSW graph traversal
results = client.search(
    collection_name="documents",
    query_vector=query_embedding,
    query_filter=Filter(
        must=[
            FieldCondition(key="tenant_id", match=MatchValue(value="acme")),
            FieldCondition(key="published_year", range=Range(gte=2024)),
        ]
    ),
    limit=10,
    with_payload=True,
)

for hit in results:
    print(f"id={hit.id}  score={hit.score:.4f}  payload={hit.payload}")

Weaviate — Hybrid Search, GraphQL, and Built-in Vectorizers

Weaviate differentiates itself with two capabilities that neither pgvector nor Qdrant offer out of the box: built-in vectorizers (Weaviate can call OpenAI, Cohere, or a local model to embed data at write time) and hybrid search combining BM25 full-text and dense vector search via a tunable fusion algorithm. If your retrieval pipeline needs keyword precision alongside semantic recall — a common requirement for enterprise document search — Weaviate handles this natively.

Weaviate also has first-class multi-tenancy with per-tenant data isolation, which makes it a strong fit for SaaS products that need to serve multiple customers from a single cluster without cross-tenant data leakage. See the Weaviate multi-tenancy documentation for the implementation details.

import weaviate
from weaviate.classes.config import Configure, Property, DataType
from weaviate.classes.query import MetadataQuery, HybridFusion

client = weaviate.connect_to_local()

# Define schema with vectorizer — Weaviate calls OpenAI at write time
client.collections.create(
    name="Document",
    vectorizer_config=Configure.Vectorizer.text2vec_openai(
        model="text-embedding-3-small",
    ),
    generative_config=Configure.Generative.openai(model="gpt-4o-mini"),
    properties=[
        Property(name="content", data_type=DataType.TEXT),
        Property(name="category", data_type=DataType.TEXT),
        Property(name="tenant_id", data_type=DataType.TEXT),
        Property(name="published_year", data_type=DataType.INT),
    ],
)

collection = client.collections.get("Document")

# Hybrid search — alpha=0 is pure BM25, alpha=1 is pure vector
# Relative score fusion balances keyword and semantic results
results = collection.query.hybrid(
    query="distributed tracing OpenTelemetry",
    alpha=0.6,                             # lean towards semantic
    fusion_type=HybridFusion.RELATIVE_SCORE,
    filters=weaviate.classes.query.Filter.by_property("tenant_id").equal("acme"),
    limit=10,
    return_metadata=MetadataQuery(score=True, explain_score=True),
)

for obj in results.objects:
    print(f"score={obj.metadata.score:.4f}  content={obj.properties['content'][:80]}")

# Generative search — retrieve then generate in one API call
response = collection.generate.hybrid(
    query="how does Kafka handle exactly-once semantics",
    alpha=0.5,
    limit=5,
    grouped_task="Summarize the key points from these documents in 3 bullet points.",
)
print(response.generated)

client.close()

Pinecone — Fully Managed, Zero Operational Overhead

Pinecone is the only fully managed option in this comparison. You provision an index via API or console, upsert vectors, query — and Pinecone handles scaling, replication, hardware provisioning, and index optimization internally. There is no Docker container to run, no Kubernetes StatefulSet to maintain, no index rebuild to schedule.

The trade-off is cost and control. Pinecone's pricing is per pod or per serverless write/read unit — at high scale, the operational savings must be weighed against bills that can exceed the cost of a self-hosted cluster. Pinecone's serverless indexes eliminate pod provisioning entirely, billing per query and storage — a better model for variable-traffic workloads.

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key="${PINECONE_API_KEY}")

# Create a serverless index
pc.create_index(
    name="documents",
    dimension=1536,
    metric="cosine",
    spec=ServerlessSpec(
        cloud="aws",
        region="us-east-1",
    ),
)

index = pc.Index("documents")

# Upsert with metadata — Pinecone stores vectors + sparse metadata dict
index.upsert(
    vectors=[
        {
            "id": "doc-001",
            "values": [0.1, 0.2, ...],   # 1536-dim embedding
            "metadata": {
                "tenant_id": "acme",
                "category": "technical",
                "published_year": 2025,
            },
        },
    ],
    namespace="production",      # namespace = logical partition
)

# Query with metadata filter
results = index.query(
    vector=query_embedding,
    top_k=10,
    namespace="production",
    filter={
        "tenant_id": {"\$eq": "acme"},
        "published_year": {"\$gte": 2024},
    },
    include_metadata=True,
)

for match in results["matches"]:
    print(f"id={match['id']}  score={match['score']:.4f}")

Performance: What the Benchmarks Actually Show

The ANN Benchmarks project provides standardized recall-vs-throughput curves for approximate nearest neighbor algorithms, but it tests algorithms without metadata filtering — a key gap for real-world workloads. The more useful comparison comes from Qdrant's own benchmark suite, which tests filtered search at various selectivity levels.

The pattern that emerges from production deployments at scale:

Hybrid Search: Combining Dense and Sparse Retrieval

Pure vector search is semantically powerful but keyword-blind — it can miss exact phrase matches and proper nouns that do not appear in the embedding model's training data. Pure BM25 keyword search is precise for exact terms but has no semantic understanding. Hybrid search combines both, and for most production retrieval systems it outperforms either alone.

Weaviate implements hybrid search natively. Qdrant supports it via sparse vectors (you bring your own sparse encoder, such as SPLADE) stored as a separate named vector. pgvector has no native sparse vector support; you would combine it with PostgreSQL full-text search at the application layer. Pinecone supports sparse-dense hybrid natively in recent index types.

# Qdrant sparse + dense hybrid search with SPLADE
from qdrant_client.models import (
    SparseVector,
    NamedSparseVector,
    NamedVector,
    SearchRequest,
    FusionQuery,
    Prefetch,
)

# Your collection must be created with both dense and sparse vector configs
# Collection setup (run once at init):
# client.recreate_collection(
#   collection_name="documents",
#   vectors_config={
#     "dense": VectorParams(size=1536, distance=Distance.COSINE),
#   },
#   sparse_vectors_config={
#     "sparse": SparseVectorParams(index=SparseIndexParams(on_disk=False)),
#   },
# )

# At query time — fuse results from both retrievers
response = client.query_points(
    collection_name="documents",
    prefetch=[
        Prefetch(
            query=NamedVector(name="dense", vector=dense_embedding),
            limit=20,
        ),
        Prefetch(
            query=NamedSparseVector(
                name="sparse",
                vector=SparseVector(indices=sparse_indices, values=sparse_values),
            ),
            limit=20,
        ),
    ],
    query=FusionQuery(fusion="rrf"),   # Reciprocal Rank Fusion
    limit=10,
    with_payload=True,
)

Decision Framework: Choosing the Right Database

The decision is rarely about raw performance — all four databases are fast enough for most workloads. The real factors are operational fit, scale requirements, and team capability.

Production Checklist