Differential Transformer V2
•
29
AI & ML interests
None defined yet.
Recent Activity
Papers
WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks
DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
Articles
ChatBench Datasets and Simulators (same prompt + fine-tuning set-up) from the ChatBench paper.
A collection of SLMs based on Phi3.5-mini-instruct adapted to clinical natural language processing tasks: https://arxiv.org/abs/2505.10717
-
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
Paper • 2505.10717 • Published • 4 -
microsoft/MediPhi-Instruct
Text Generation • 4B • Updated • 3.43k • 58 -
microsoft/MediPhi
Text Generation • 4B • Updated • 555 • 15 -
microsoft/MediPhi-PubMed
Text Generation • 4B • Updated • 412 • 8
Phi-4 family of small language, multi-modal and reasoning models.
-
microsoft/Phi-4-mini-flash-reasoning
Text Generation • 4B • Updated • 1.11k • 263 -
microsoft/Phi-4-mini-reasoning
Text Generation • 4B • Updated • 9.45k • 213 -
microsoft/Phi-4-reasoning
Text Generation • 15B • Updated • 5.83k • 215 -
microsoft/Phi-4-reasoning-plus
Text Generation • 15B • Updated • 55.7k • 331
Phi-1 family of small language models.
🔥BitNet family of large language models (1-bit LLMs).
-
microsoft/bitnet-b1.58-2B-4T
Text Generation • 0.8B • Updated • 5.95k • 1.26k -
microsoft/bitnet-b1.58-2B-4T-bf16
Text Generation • 2B • Updated • 2.3k • 34 -
microsoft/bitnet-b1.58-2B-4T-gguf
Text Generation • 2B • Updated • 4.84k • 224 -
BitNet b1.58 2B4T Technical Report
Paper • 2504.12285 • Published • 81
LLM2CLIP makes SOTA pretrained CLIP modal more SOTA ever.
-
microsoft/LLM2CLIP-EVA02-L-14-336
Zero-Shot Image Classification • Updated • 74 • 59 -
microsoft/LLM2CLIP-Openai-L-14-336
Zero-Shot Classification • 0.6B • Updated • 2.77k • 43 -
microsoft/LLM2CLIP-EVA02-B-16
Updated • 62 • 10 -
microsoft/LLM2CLIP-Openai-B-16
Zero-Shot Classification • 0.4B • Updated • 498 • 18
TAPEX is the state-of-the-art table pre-training models which can be used for table-based question answering and table-based fact verification.
-
TAPEX: Table Pre-training via Learning a Neural SQL Executor
Paper • 2107.07653 • Published • 2 -
microsoft/tapex-large-finetuned-wtq
Table Question Answering • 0.4B • Updated • 963 • 77 -
microsoft/tapex-base-finetuned-wikisql
Table Question Answering • Updated • 856k • • 22 -
microsoft/tapex-large-sql-execution
Table Question Answering • 0.4B • Updated • 130 • 17
The LayoutLM series are Transformer encoders useful for document AI tasks such as invoice parsing, document image classification and DocVQA.
The Orca family of LMs developed by Microsoft.
GIT (Generative Image-to-text Transformer) is a model useful for vision-language tasks such as image/video captioning and question answering.
-
GIT: A Generative Image-to-text Transformer for Vision and Language
Paper • 2205.14100 • Published • 1 -
microsoft/git-base
Image-to-Text • 0.2B • Updated • 6.09k • 106 -
microsoft/git-large
Image-to-Text • Updated • 386 • 17 -
microsoft/git-base-vqav2
Visual Question Answering • 0.2B • Updated • 128 • 20
Industrial Foundation Models
Frontier Text-to-Speech Models https://microsoft.github.io/VibeVoice/
The models and datasets that comprise the Dayhoff Atlas
NextCoder family of code-editing LMs developed with Selective Knowledge Transfer and its training data.
Phi-3 family of small language and multi-modal models. Language models are available in short- and long-context lengths.
-
microsoft/Phi-3.5-mini-instruct
Text Generation • 4B • Updated • 417k • 950 -
microsoft/Phi-3.5-MoE-instruct
Text Generation • 42B • Updated • 87.2k • 569 -
microsoft/Phi-3.5-vision-instruct
Image-Text-to-Text • 4B • Updated • 791k • 724 -
microsoft/Phi-3-mini-4k-instruct
Text Generation • 4B • Updated • 1.32M • 1.37k
Artifacts for the paper "Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements" (https://arxiv.org/abs/2410.08968)
MAI-DS-R1 is a DeepSeek-R1 reasoning model that has been post-trained by the Microsoft AI team.
The SpeechT5 framework consists of a shared seq2seq and six modal-specific (speech/text) pre/post-nets that can address a few audio-related tasks.
The Table Transformer (TATR) is a series of object detection models useful for table extraction from PDF images.
-
microsoft/table-transformer-detection
Object Detection • 28.8M • Updated • 1.5M • 392 -
microsoft/table-transformer-structure-recognition
Object Detection • 28.8M • Updated • 855k • 207 -
microsoft/table-transformer-structure-recognition-v1.1-all
Object Detection • 28.8M • Updated • 191k • 79 -
microsoft/table-transformer-structure-recognition-v1.1-fin
Object Detection • 28.8M • Updated • 341 • 1
Models for biomedical research applications, such as radiology report generation and biomedical language understanding.
UDOP is a general multimodal model for document AI
-
Unifying Vision, Text, and Layout for Universal Document Processing
Paper • 2212.02623 • Published • 11 -
microsoft/udop-large
Image-Text-to-Text • 0.7B • Updated • 18.2k • 121 -
microsoft/udop-large-512
Image-Text-to-Text • 0.7B • Updated • 45 • 5 -
microsoft/udop-large-512-300k
Image-Text-to-Text • 0.7B • Updated • 52 • 33
-
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
Paper • 2311.06242 • Published • 95 -
microsoft/Florence-2-large
Image-Text-to-Text • 0.8B • Updated • 707k • 1.74k -
microsoft/Florence-2-base
Image-Text-to-Text • 0.2B • Updated • 378k • 333 -
microsoft/Florence-2-large-ft
Image-Text-to-Text • 0.8B • Updated • 21.4k • 378
Locomotion policies for hundreds of simulated humanoid locomotion clips and demonstration data for training them.
ChatBench Datasets and Simulators (same prompt + fine-tuning set-up) from the ChatBench paper.
Frontier Text-to-Speech Models https://microsoft.github.io/VibeVoice/
A collection of SLMs based on Phi3.5-mini-instruct adapted to clinical natural language processing tasks: https://arxiv.org/abs/2505.10717
-
A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
Paper • 2505.10717 • Published • 4 -
microsoft/MediPhi-Instruct
Text Generation • 4B • Updated • 3.43k • 58 -
microsoft/MediPhi
Text Generation • 4B • Updated • 555 • 15 -
microsoft/MediPhi-PubMed
Text Generation • 4B • Updated • 412 • 8
The models and datasets that comprise the Dayhoff Atlas
NextCoder family of code-editing LMs developed with Selective Knowledge Transfer and its training data.
Phi-4 family of small language, multi-modal and reasoning models.
-
microsoft/Phi-4-mini-flash-reasoning
Text Generation • 4B • Updated • 1.11k • 263 -
microsoft/Phi-4-mini-reasoning
Text Generation • 4B • Updated • 9.45k • 213 -
microsoft/Phi-4-reasoning
Text Generation • 15B • Updated • 5.83k • 215 -
microsoft/Phi-4-reasoning-plus
Text Generation • 15B • Updated • 55.7k • 331
Phi-3 family of small language and multi-modal models. Language models are available in short- and long-context lengths.
-
microsoft/Phi-3.5-mini-instruct
Text Generation • 4B • Updated • 417k • 950 -
microsoft/Phi-3.5-MoE-instruct
Text Generation • 42B • Updated • 87.2k • 569 -
microsoft/Phi-3.5-vision-instruct
Image-Text-to-Text • 4B • Updated • 791k • 724 -
microsoft/Phi-3-mini-4k-instruct
Text Generation • 4B • Updated • 1.32M • 1.37k
Phi-1 family of small language models.
Artifacts for the paper "Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements" (https://arxiv.org/abs/2410.08968)
🔥BitNet family of large language models (1-bit LLMs).
-
microsoft/bitnet-b1.58-2B-4T
Text Generation • 0.8B • Updated • 5.95k • 1.26k -
microsoft/bitnet-b1.58-2B-4T-bf16
Text Generation • 2B • Updated • 2.3k • 34 -
microsoft/bitnet-b1.58-2B-4T-gguf
Text Generation • 2B • Updated • 4.84k • 224 -
BitNet b1.58 2B4T Technical Report
Paper • 2504.12285 • Published • 81
MAI-DS-R1 is a DeepSeek-R1 reasoning model that has been post-trained by the Microsoft AI team.
LLM2CLIP makes SOTA pretrained CLIP modal more SOTA ever.
-
microsoft/LLM2CLIP-EVA02-L-14-336
Zero-Shot Image Classification • Updated • 74 • 59 -
microsoft/LLM2CLIP-Openai-L-14-336
Zero-Shot Classification • 0.6B • Updated • 2.77k • 43 -
microsoft/LLM2CLIP-EVA02-B-16
Updated • 62 • 10 -
microsoft/LLM2CLIP-Openai-B-16
Zero-Shot Classification • 0.4B • Updated • 498 • 18
The SpeechT5 framework consists of a shared seq2seq and six modal-specific (speech/text) pre/post-nets that can address a few audio-related tasks.
TAPEX is the state-of-the-art table pre-training models which can be used for table-based question answering and table-based fact verification.
-
TAPEX: Table Pre-training via Learning a Neural SQL Executor
Paper • 2107.07653 • Published • 2 -
microsoft/tapex-large-finetuned-wtq
Table Question Answering • 0.4B • Updated • 963 • 77 -
microsoft/tapex-base-finetuned-wikisql
Table Question Answering • Updated • 856k • • 22 -
microsoft/tapex-large-sql-execution
Table Question Answering • 0.4B • Updated • 130 • 17
The Table Transformer (TATR) is a series of object detection models useful for table extraction from PDF images.
-
microsoft/table-transformer-detection
Object Detection • 28.8M • Updated • 1.5M • 392 -
microsoft/table-transformer-structure-recognition
Object Detection • 28.8M • Updated • 855k • 207 -
microsoft/table-transformer-structure-recognition-v1.1-all
Object Detection • 28.8M • Updated • 191k • 79 -
microsoft/table-transformer-structure-recognition-v1.1-fin
Object Detection • 28.8M • Updated • 341 • 1
The LayoutLM series are Transformer encoders useful for document AI tasks such as invoice parsing, document image classification and DocVQA.
Models for biomedical research applications, such as radiology report generation and biomedical language understanding.
The Orca family of LMs developed by Microsoft.
UDOP is a general multimodal model for document AI
-
Unifying Vision, Text, and Layout for Universal Document Processing
Paper • 2212.02623 • Published • 11 -
microsoft/udop-large
Image-Text-to-Text • 0.7B • Updated • 18.2k • 121 -
microsoft/udop-large-512
Image-Text-to-Text • 0.7B • Updated • 45 • 5 -
microsoft/udop-large-512-300k
Image-Text-to-Text • 0.7B • Updated • 52 • 33
GIT (Generative Image-to-text Transformer) is a model useful for vision-language tasks such as image/video captioning and question answering.
-
GIT: A Generative Image-to-text Transformer for Vision and Language
Paper • 2205.14100 • Published • 1 -
microsoft/git-base
Image-to-Text • 0.2B • Updated • 6.09k • 106 -
microsoft/git-large
Image-to-Text • Updated • 386 • 17 -
microsoft/git-base-vqav2
Visual Question Answering • 0.2B • Updated • 128 • 20
-
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
Paper • 2311.06242 • Published • 95 -
microsoft/Florence-2-large
Image-Text-to-Text • 0.8B • Updated • 707k • 1.74k -
microsoft/Florence-2-base
Image-Text-to-Text • 0.2B • Updated • 378k • 333 -
microsoft/Florence-2-large-ft
Image-Text-to-Text • 0.8B • Updated • 21.4k • 378
Industrial Foundation Models
Locomotion policies for hundreds of simulated humanoid locomotion clips and demonstration data for training them.