resources

Domain-specific embedding and representation model training papers

Your example, ProtModernBERT: Compute-Efficient Protein Language Modeling (14 October 2026), points to a broader category than retrieval fine-tuning: training specialised representation models using domain data, often by adapting general-purpose transformer architectures. ProtModernBERT pretrains a ModernBERT-derived encoder on protein sequences and evaluates it on protein structure, localisation and functional prediction tasks. It reports comparable performance to several established protein models with substantially lower pretraining compute.

arXiv

It’s useful to distinguish three approaches:

  1. ModernBERT-based domain adaptation Paper Domain Training approach ProtModernBERT (2026) Proteins ModernBERT architecture pretrained on amino-acid sequences; investigates pretraining efficiency. Legal Domain Adaptation of Modern BERT Models (2026) US law Continued ModernBERT MLM training on US court opinions; compares adaptation against pretraining from scratch and releases checkpoints. Patent Language Model Pretraining with ModernBERT (2025) Patents Pretrains several ModernBERT variants on a corpus of more than 60 million patent records, evaluating classification tasks. Domain Adaptation of Russian ModernBERT for Long Legal Documents (2026) Russian legislation Continues MLM pretraining on approximately 304,000 legislative documents; assesses language-modelling loss and legal entity extraction. Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings in Biological Databases (2025) Biological databases Uses ModernBERT with contrastive training and mixture-of-experts extensions to learn domain-specific vector representations.

arXiv +3

The legal-adaptation paper is particularly useful: it reports that continuing pretraining from an existing ModernBERT checkpoint performs better than starting from scratch in its experiments. The biological-database paper is closer to explicit vector-embedding training rather than conventional language modelling.

  1. Domain-specific text embedding and retrieval training These papers are especially relevant when the goal is to take an existing open-weight embedding model and specialise it for a collection of documents, products or technical terminology. Paper Domain / base model Training approach Adaptation of Embedding Models to Financial Filings via LLM Distillation (2025) Finance; general retrieval encoder Iteratively mines hard positives and negatives from financial filings, uses LLM relevance judgements, and retrains a student bi-encoder. Reports a 27.7% mean improvement in MRR@5. arXiv FinMTEB: Finance Massive Text Embedding Benchmark (2025) Finance; E5-Mistral-7B Introduces Fin-E5, trained with persona-based synthetic financial data, and evaluates it on 64 datasets covering seven task types. ACL Anthology TermGPT: Multi-Level Contrastive Fine-Tuning for Terminology Adaptation (2025) Law and finance Builds sentence graphs and applies token-level and sentence-level contrastive training to capture specialised terminology. arXiv Learn Before Represent: Bridging Generative and Contrastive Learning for Domain-Specific LLM Embeddings (2026) Medicine, chemistry, code Two-stage approach: first teach the model domain knowledge with generative learning, then optimise its embedding space with contrastive learning. arXiv Towards Better Search with Domain-Aware Text Embeddings for C2C Marketplaces (2025) Mercari marketplace Fine-tunes on real query–purchase pairs, uses asymmetric query/item prefixes and Matryoshka training. Includes offline and online evaluations. arXiv Compass-Embedding v4: Robust Contrastive Learning for Multilingual E-commerce Embeddings (2026) Multilingual e-commerce Synthetic domain data, false-negative-aware InfoNCE, cross-lingual augmentation, model merging and production inference optimisation. arXiv Semantic Retrieval for Product Search in E-Commerce (2026) E-commerce product search Two-stage training: contrastive learning with false-negative masking, followed by preference optimisation over graded product relevance. toehanger.shop Mine and Refine: Optimizing Graded Relevance in E-commerce Semantic Search Retrieval (2026) E-commerce Supervised contrastive training, iterative hard-example mining and LLM-assisted relevance labels to optimise multiple relevance levels. toehanger.shop Contrastive Learning Using Graph Embeddings for Domain Adaptation of Language Models in the Process Industry (2025) Industrial operations Derives positive/negative triplets from knowledge-graph embeddings and uses them to contrastively fine-tune smaller text encoders. arXiv Cross-Task Performance of Domain-Adapted Embeddings for Intellectual Property Applications (2026) Patents Fine-tunes embedding models on expert-verified patent matches; demonstrates that gains in similarity retrieval can come at the expense of classification performance. ScienceDirect Two especially relevant industry case studies are the Mercari paper and Semantic Retrieval for Product Search in E-Commerce. Both address the problem of matching short user queries against structured or semi-structured items whose attributes materially affect relevance.

  2. Biomedical and clinical embeddings Paper Domain / base model Training approach CardioEmbed: Domain-Specialized Text Embeddings for Clinical Cardiology (2025) Cardiology; Qwen3-Embedding-8B Contrastive fine-tuning on approximately 150,000 deduplicated sentences from seven cardiology textbooks, using InfoNCE and in-batch negatives. arXiv Towards Domain Specification of Embedding Models in Medicine (2025) Medical text; GTE Introduces MEDTE, fine-tuned with self-supervised contrastive learning across multiple medical data sources, alongside a 51-task benchmark. arXiv BioLORD-2023: Semantic Textual Representations Fusing LLM and Clinical Knowledge Graph Insights (2023/2024) Clinical terminology; MPNet Uses biomedical ontologies, LLM-generated definitions, contrastive learning, self-distillation and weight averaging. arXiv MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs (2023) Biomedical retrieval; PubMedBERT Trains asymmetric query/document encoders using 255 million PubMed query–article pairs; separately trains a cross-encoder reranker. PMC SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval (2023) Legal documents; BERT Learns dense case representations with multiple structure-aware pretraining objectives targeting facts, reasoning and judgements. arXiv Of these, CardioEmbed is the closest direct example of adapting Qwen3-Embedding to a specialised domain. MedCPT is particularly informative for asymmetric retrieval because it trains separate query and document encoders rather than relying only on instruction prefixes. The following implementation repositories are available: MedCPT Retriever and reranker training code; model checkpoints

BioLORD-2023 Open model, model card and detailed training strategy

SAILER Pretraining scripts, example data schema and released checkpoints

GitHub +2

  1. More specialised ModernBERT models These are additional close matches to ProtModernBERT, particularly for studying domain-specific continued pretraining, corpus construction and architecture modifications. Paper Domain Training approach CheMatE: Bi-semantic Chemical Embedder for Joint Representation Learning of SMILES and Natural Language (2026) Chemistry ModernBERT with continued MLM pretraining on chemistry documents, followed by Matryoshka contrastive training on synthetic SMILES–text pairs. arXiv Climate-ModernBERT: Revisiting Corpus Composition for Domain-Adaptive Continued Pretraining (2026) Climate science Compares continued pretraining on academic, web and synthetic corpora, including checkpoint merging versus mixed-corpus training. arXiv BioClinical ModernBERT (2025) Biomedical and clinical text Continues ModernBERT pretraining on more than 53.5B tokens from diverse clinical and biomedical sources; releases 150M and 396M models. arXiv Clinical ModernBERT: An Efficient and Long Context Encoder for Biomedical Text (2025) Clinical documents Domain pretraining on PubMed, clinical notes and medical ontologies, targeting long-context clinical NLP. arXiv Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction (2025) Financial documents Adds spatial embeddings to the transformer and trains on document-layout and token-labelling objectives. Not a general similarity embedding model. arXiv Implementation resources CheMatE GitHub · Trained embedding checkpoint BioClinical ModernBERT GitHub · Pretraining configurations and reproduction guidance ModChemBERT · Related reproducible project with MLM, domain adaptation, task fine-tuning and checkpoint merging

GitHub +2

CheMatE is one of the strongest matches because it explicitly combines domain knowledge acquisition and embedding-space optimisation. Many of the other ModernBERT papers stop at MLM pretraining and downstream task evaluation.

  1. Protein, molecular and genomic representation learning These explore the more fundamental question of how a domain-specific language can be encoded into a useful latent space. Paper Domain Training approach Protein Language Models: Is Scaling Necessary? (AMPLIFY) (2024) Protein sequences Trains efficient 120M and 350M protein encoders on curated amino-acid data, investigating the impact of architecture, training efficiency and data quality. bioRxiv Structure-Aligned Protein Language Model (2025) Protein sequences and structures Adds contrastive alignment between protein language models and structural graph models, alongside structural-token prediction. arXiv DNABERT-2: Efficient Foundation Model and Benchmark for Multi-Species Genome (2023/2024) DNA Develops a genomics-specific encoder using BPE tokenisation and efficient pretraining across multiple species. arXiv Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics (2025) DNA Pretrains transformer encoders on human and multi-species genomes to generate transferable genomic sequence representations. Nature Methods An Open-source Family of Large Encoder-Decoder Foundation Models for Chemistry (2025) Molecular sequences Pretrains encoder–decoder models on 91 million molecular sequences and studies how the learned embeddings transfer to chemistry tasks. Communications Chemistry The AMPLIFY repository is especially useful for reproducing protein model training: it publishes the training code, data resources and checkpoints.

chandar-lab/AMPLIFY · GitHub

There is also a ModernMolBERT model checkpoint trained from scratch on approximately 2.4 million molecular SELFIES strings, using ModernBERT and a chemistry-aware tokenizer. This is a model release rather than a paper reference.

Hugging Face

  1. Scientific document representation learning These papers are particularly interesting because the training supervision often comes from relationships already present in the corpus, without manual similarity labelling. Paper Training approach SciNCL: Neighborhood Contrastive Learning for Scientific Document Representations (2022) Uses citation graph embeddings to select related papers and carefully controlled hard negatives for contrastive training. arXiv SciRepEval / SPECTER2: A Multi-Format Benchmark for Scientific Document Representations (2023) Trains on millions of scientific citation triplets, followed by task-specific adapters for search, retrieval, classification and regression. arXiv Beyond Citations: Integrating Finding-Based Relations for Improved Biomedical Article Representations (2025) Combines citation information with scientific finding similarity to construct better contrastive training triplets. ACL Anthology Code: SciNCL and SPECTER2. For a corpus containing rich relationships—citations, entities, categories, structured attributes or document links—the lesson is that these can generate training pairs and triplets more directly than relying entirely on synthetic queries.
  2. Most useful papers to prioritise For investigating how to build a custom domain embedding model from open weights, I’d prioritise the following six papers. Priority Paper Main question it addresses 1 CheMatE How to combine domain-adaptive pretraining with contrastive embedding training? 2 Climate-ModernBERT How much do corpus composition and checkpoint merging matter? 3 Financial Filings via LLM Distillation How can unlabelled domain documents produce effective training supervision? 4 Domain-Aware Embeddings for C2C Marketplaces How should asymmetric query–item embeddings be fine-tuned and evaluated in production? 5 BioClinical ModernBERT How can a general-purpose ModernBERT model be specialised through large-scale continued pretraining? 6 ProtModernBERT How much of domain model performance comes from architecture rather than model size or pretraining compute? The key experimental distinction to investigate across these studies is whether continued domain-specific language modelling followed by contrastive learning outperforms contrastive fine-tuning alone. For search applications, I would additionally compare both approaches with a general-purpose embedding baseline, because improvements in MLM loss or domain classification do not automatically imply improvements in Recall@k or nDCG. Most importantly, the papers above suggest treating the training objective, the source of supervision and the desired similarity semantics as separate design decisions—not assuming that exposing a model to more domain text will necessarily produce a better retrieval embedding space.

Below is the consolidated list of 41 papers, including publication year, domain, training approach, GitHub repository and Hugging Face model card where available.

0. Aggregated table with all papers

Title Year Paper (raw link) Domain Training approach GitHub Model card
ProtModernBERT: Compute-Efficient Protein Language Modeling 2026 https://arxiv.org/abs/2610.19738 Proteins ModernBERT-style encoder pretrained on amino-acid sequences using masked language modelling (MLM). — —
Legal Domain Adaptation of Modern BERT Models 2026 https://arxiv.org/abs/2606.28538 Legal Continued ModernBERT MLM pretraining on US court opinions; comparison with training from scratch. — —
Patent Language Model Pretraining with ModernBERT 2025 https://arxiv.org/abs/2509.14926 Patents ModernBERT pretraining on 60M+ patent records; tokenizer and architecture comparisons. — —
Domain Adaptation of Russian ModernBERT for Long Legal Documents 2026 https://arxiv.org/abs/2610.06715 Legal Continued MLM pretraining on Russian legislation. — RuModernBERT-ruLaw
Contrastive Learning and Mixture of Experts Enables Precise Vector Embeddings in Biological Databases 2025 https://www.nature.com/articles/s41598-025-98185-8 Biomedical Co-citation-derived contrastive training using BERT with mixture-of-experts extensions. Code See repository
Adaptation of Embedding Models to Financial Filings via LLM Distillation 2025 https://arxiv.org/abs/2512.08088 Finance Iterative hard-example mining, LLM relevance judgements and knowledge distillation into a bi-encoder. — —
FinMTEB: Finance Massive Text Embedding Benchmark 2025 https://aclanthology.org/2025.emnlp-main.179/ Finance FinE5 trained on persona-generated synthetic financial retrieval data. FinMTEB FinE5*
TermGPT: Multi-Level Contrastive Fine-Tuning for Terminology Adaptation in Legal and Financial Domains 2025 / 2026 https://arxiv.org/abs/2511.09854 Legal, finance Sentence-graph supervision and multi-level token/sentence contrastive fine-tuning. — —
Learn Before Represent: Bridging Generative and Contrastive Learning for Domain-Specific LLM Embeddings 2026 https://arxiv.org/abs/2601.11124 Medicine, chemistry, code Generative domain-knowledge acquisition followed by refined contrastive embedding training. — —
Towards Better Search with Domain-Aware Text Embeddings for C2C Marketplaces 2025 https://arxiv.org/abs/2512.21021 E-commerce Query–purchase contrastive training, asymmetric query/item prefixes and Matryoshka learning. — —
Compass-Embedding v4: Robust Contrastive Learning for Multilingual E-commerce Embeddings 2026 https://arxiv.org/abs/2601.11565 E-commerce False-negative-aware InfoNCE, multilingual synthetic training data and model merging. — —
Semantic Retrieval for Product Search in E-Commerce 2026 https://arxiv.org/abs/2606.01504 E-commerce Qwen3-based dual encoder; contrastive training with false-negative masking, followed by graded-relevance preference optimisation. — —
Mine and Refine: Optimizing Graded Relevance in E-commerce Search Retrieval 2026 https://arxiv.org/abs/2602.17654 E-commerce Supervised contrastive learning, iterative ANN hard-negative mining and LLM-assisted relevance labelling. — —
Contrastive Learning Using Graph Embeddings for Domain Adaptation of Language Models in the Process Industry 2025 https://arxiv.org/abs/2510.04631 Industrial Knowledge-graph-derived positive and negative triplets for contrastive encoder fine-tuning. — —
Cross-Task Performance of Domain-Adapted Embeddings for Intellectual Property Applications 2026 https://www.sciencedirect.com/science/article/pii/S2212827126009236 Patents Fine-tuning on expert-verified patent similarity pairs; cross-task transfer evaluation. — —
CardioEmbed: Domain-Specialized Text Embeddings for Clinical Cardiology 2025 https://arxiv.org/abs/2511.10930 Cardiology Qwen3-Embedding LoRA adaptation using InfoNCE, textbook-derived training examples and in-batch negatives. CardioEmbed CardioEmbed
Towards Domain Specification of Embedding Models in Medicine 2025 https://arxiv.org/abs/2507.19407 Medical GTE-based embedding adaptation using self-supervised contrastive learning on medical text. — —
BioLORD-2023: Semantic Textual Representations Fusing LLM and Clinical Knowledge Graph Insights 2023 https://arxiv.org/abs/2311.16075 Biomedical Contrastive learning using ontology relationships, LLM-generated definitions and self-distillation. — BioLORD-2023
MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs 2023 https://arxiv.org/abs/2307.00589 Biomedical retrieval Asymmetric query/document encoders trained on 255M query–article pairs, with in-batch negatives and a separate reranker. MedCPT Query / Document
SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval 2023 https://arxiv.org/abs/2304.11370 Legal retrieval Structure-aware pretraining using case facts, legal reasoning and judgements. SAILER SAILER-en
CheMatE: Bi-semantic Chemical Embedder for Joint Representation Learning of SMILES and Natural Language 2026 https://arxiv.org/abs/2608.03855 Chemistry Continued ModernBERT MLM followed by Matryoshka contrastive training on synthetic SMILES–text pairs. CheMatE CheMatE-v0
Climate-ModernBERT: Revisiting Corpus Composition for Domain-Adaptive Continued Pretraining 2026 https://arxiv.org/abs/2609.07798 Climate Continued ModernBERT pretraining on academic, web and synthetic corpora; checkpoint merging experiments. ClimateModernBERT Merged model
BioClinical ModernBERT: A State-of-the-Art Long-Context Encoder for Biomedical and Clinical NLP 2025 https://arxiv.org/abs/2506.10896 Biomedical Continued ModernBERT pretraining on 53.5B+ biomedical and clinical tokens. BioClinical Base
Clinical ModernBERT: An Efficient and Long Context Encoder for Biomedical Text 2025 https://arxiv.org/abs/2504.03964 Clinical Domain-specific pretraining using clinical notes, PubMed text and medical code/description pairs. Clinical ModernBERT Clinical ModernBERT
Spatial ModernBERT: Spatial-Aware Transformer for Table and Key-Value Extraction 2025 https://arxiv.org/abs/2507.08865 Financial documents ModernBERT with spatial embeddings and multi-head token classification. Spatial ModernBERT —
Protein Language Models: Is Scaling Necessary? (AMPLIFY) 2024 https://www.biorxiv.org/content/10.1101/2024.09.23.614603v1 Proteins Protein encoder pretraining from scratch with modern architectural choices, curated sequences and efficient MLM. AMPLIFY AMPLIFY-120M
Structure-Aligned Protein Language Model 2025 https://arxiv.org/abs/2505.16896 Proteins Contrastive alignment of protein sequence and structural graph representations. — —
DNABERT-2: Efficient Foundation Model and Benchmark for Multi-Species Genome 2023 https://arxiv.org/abs/2306.15006 Genomics Multi-species DNA pretraining with BPE tokenisation and efficient encoder architecture. DNABERT-2 DNABERT-2
Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics 2025 https://www.nature.com/articles/s41592-024-02523-z Genomics Large-scale transformer pretraining on human and multi-species genomic sequences. Nucleotide Transformer NT-v2 500M
An Open-source Family of Large Encoder-Decoder Foundation Models for Chemistry 2025 https://www.nature.com/articles/s42004-025-01585-0 Chemistry Encoder–decoder molecular pretraining on 91M molecular sequences with reconstruction-based representation learning. SMI-TED SMI-TED
SciNCL: Neighborhood Contrastive Learning for Scientific Document Representations 2022 https://arxiv.org/abs/2202.06671 Scientific literature Citation-graph neighbourhood sampling for contrastive learning, hard positives and hard negatives. SciNCL SciNCL
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations (SPECTER2) 2022 https://arxiv.org/abs/2211.13308 Scientific literature Citation-triplet contrastive pretraining followed by task-specific embedding adapters. SPECTER2 SPECTER2 Base
Beyond Citations: Integrating Finding-Based Relations for Improved Biomedical Article Representations 2025 https://aclanthology.org/2025.bionlp-1.25/ Biomedical literature Contrastive learning incorporating scientific-finding relationships alongside citations. — —
Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models 2025 https://arxiv.org/abs/2506.05176 Multilingual retrieval Multi-stage synthetic-data generation, supervised contrastive fine-tuning, reranker training and model merging. Qwen3-Embedding 0.6B / 4B
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval 2021 / 2022 https://arxiv.org/abs/2112.07577 Cross-domain retrieval Synthetic query generation, hard-negative mining, cross-encoder pseudo-labelling and dense retriever training. GPL —
Promptagator: Few-shot Dense Retrieval From 8 Examples 2022 https://arxiv.org/abs/2209.11755 Cross-domain retrieval Few-shot synthetic query generation using an LLM, followed by dense retriever training. — —
InPars: Data Augmentation for Information Retrieval using Large Language Models 2022 https://arxiv.org/abs/2202.05144 Cross-domain retrieval LLM-generated queries and synthetic query–document pairs for neural retrieval training. InPars —
NV-Retriever: Improving Text Embedding Models with Effective Hard-Negative Mining 2024 https://arxiv.org/abs/2407.15831 General retrieval Positive-aware hard-negative mining to improve contrastive embedding training. — —
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks 2020 https://arxiv.org/abs/2004.10964 Cross-domain NLP Domain-adaptive pretraining (DAPT) and task-adaptive pretraining (TAPT). Don’t Stop Pretraining Biomedical RoBERTa
Matryoshka Representation Learning 2022 https://arxiv.org/abs/2205.13147 General representations Joint optimisation of embedding quality across multiple nested vector dimensions. — —
TSDAE: Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning 2021 https://arxiv.org/abs/2104.06979 Unsupervised text embeddings Denoising autoencoder reconstructs original sentences through fixed-size representations. Sentence Transformers examples —

Conventions: Years refer to first public preprint or journal publication unless otherwise indicated. — means I could not verify a paper-specific repository or model card. A GitHub link may contain training code, evaluation code, or configurations rather than a fully reproducible training pipeline.

Important availability notes

For reproducing domain-specific embedding training, the strongest code-backed starting points from this collection are CheMatE, CardioEmbed, MedCPT, SciNCL, GPL, Qwen3-Embedding and Climate-ModernBERT. They collectively demonstrate continued pretraining, contrastive objectives, synthetic supervision, hard-negative mining, asymmetric encoders and model merging.