resources

Asymmetric embeddings

https://chatgpt.com/c/6ad44d71-17b0-83ec-82a3-be8ffde1340a

Describe asymmetric embeddings in embedding models

Asymmetric embeddings are a particularly important architecture for search, where queries and documents play different roles.

For example:

Rather than treating both inputs identically, an asymmetric embedding model learns to map a short expression of intent to a longer description of relevant content.

There are three main approaches:

Architecture Query encoder Document encoder Example
Prompt-conditioned Shared weights, query-specific prefix/instruction Shared weights, different prefix or no instruction E5, Qwen3, Nomic
Adapter-asymmetric Shared backbone + query-specific adapter Shared backbone + document-specific adapter Jina Embeddings v3
Two-tower (separate weights) Independently trained query tower Independently trained document tower DPR, custom Sentence Transformers Router

The distinction matters: many models advertised as asymmetric are actually shared-weight models with different input conditioning. A true two-tower model offers greater architectural flexibility, including using a smaller model for queries and a larger one for documents.

1. Open-weight models supporting asymmetric retrieval

Model Mechanism Query/document differentiation Further training
Qwen3-Embedding Query instruction Instruct: ... Query: ... versus plain document LoRA / full
E5 / multilingual E5 Task prefixes query: versus passage: Full / LoRA
Nomic Embed v1.5 Task prefixes search_query: versus search_document: Full / LoRA
Snowflake Arctic Embed v2 Query prefix query: versus plain document Full / LoRA
BGE v1.5 Query instruction Query retrieval instruction versus plain document Full / LoRA
Mixedbread Embed Large Query instruction Query retrieval instruction versus plain document Full / LoRA
GTE Qwen2 Instruct Query instruction Task-specific query versus plain document LoRA / full
Jina Embeddings v3 Different LoRA adapters retrieval.query versus retrieval.passage Adapter / full
DPR Separate encoders Dedicated question and context checkpoints Independent towers
ColBERT Role-conditioned multi-vector Query/document markers and different token processing Contrastive / distillation

The models’ documented query/document conventions are important: E5 and Nomic require the respective prefixes, while BGE v1.5 is more tolerant of omitting its optional query instruction. Qwen3 explicitly recommends a task-specific instruction on the query side.

image

intfloat/e5-large-v2 at main

+6

Jina v3 is particularly interesting because its query/document asymmetry is implemented using different learned LoRA adapters on a shared backbone, rather than just text prefixes. Its checkpoint is CC BY-NC 4.0, however, so commercial usage requires additional consideration.

image

jinaai/jina-embeddings-v3 at main

+1

2. True asymmetric architectures are more interesting for training

A conventional shared encoder has:

\[ s(q,d)=f_\theta(q)^\top f_\theta(d) \]

A genuine asymmetric dual-encoder instead has:

\[ \boxed{s(q,d)=f_{\theta_q}(q)^\top g_{\theta_d}(d)} \]

The query and document encoders can have completely different architectures, provided their outputs are trained into a compatible embedding space.

Independent towers, jointly aligned during training.

For high-throughput search, this opens up a compelling architecture: train a relatively expensive document encoder offline, then distil a smaller, low-latency query encoder against its document embedding space.

An especially useful training pattern is to freeze the document encoder and only train the query encoder. This can preserve compatibility with an existing vector index while improving retrieval relevance.

A concrete, production-oriented example

OpenSearch Neural Sparse Encoding v3 implements a genuinely asymmetric sparse retrieval model:

The query path does not require Transformer inference. This is especially relevant for high-QPS retrieval, although it is a different approach from dense ANN embeddings.

image

Hugging Face

+1

3. How I would fine-tune these models

I would compare three experiments against the same retrieval evaluation set.

Experiment Training Main advantage
A. Shared-weight asymmetric Qwen3 or Nomic, with query/document formatting and contrastive training Simple implementation and strong baseline
B. Separate adapters Shared backbone, independent query and document LoRA adapters Greater specialisation without separate full backbones
C. Independent towers Freeze a strong document encoder; train/distil a small query model Low online cost, potential to reuse an existing index

For experiment C, the Sentence Transformers Router is directly relevant. It explicitly supports different query and document model architectures, including a static query encoder and a Transformer document encoder. Crucially, the documentation warns that independently pretrained encoders require additional training to align their vector spaces.

image

Sentence Transformers documentation

+1

For a quick starting baseline, asymmetric Nomic encoding looks like:

from sentence_transformers import SentenceTransformermodel = SentenceTransformer(    "nomic-ai/nomic-embed-text-v2-moe",    trust_remote_code=True,)queries = ["coastal family home with a pool"]documents = [    "Four-bedroom house near the beach "    "with a swimming pool and garden."]q = model.encode_query(queries, normalize_embeddings=True)d = model.encode_document(documents, normalize_embeddings=True)scores = q @ d.T

These methods apply the configured query and passage prefixes automatically for this model.

image

nomic-ai/nomic-embed-text-v2-moe at main

+1

For further training and evaluation, I would prioritise:

The most interesting research question is whether a small query encoder trained against a frozen, larger document encoder can approach the relevance of a much larger shared embedding model, while preserving index compatibility and reducing online latency.

I would evaluate that directly against a conventional shared Qwen3 model using nDCG\@10, Recall\@100, query-encoding latency and index storage. The key is to distinguish improvements due to asymmetric representation learning from those due to simply having a better underlying model.