Vraj Shah

Title

Staff Research Scientist

Bio

Vraj Shah is a Staff Research Scientist at IBM Almaden Research Center, where his work is focused on developing novel AI models and human-in-the-loop tools for natural language processing tasks, while reducing the barrier of entry for domain experts in adopting AI.

Publications

MTRAG-UN: A Benchmark for Open Challenges in Multi-Turn RAG Conversations
- - Sara Rosenthal
  - Yannis Katsis
  - et al.
- 2026
- ACL 2026
Short paper
SemEval-2026 Task 8: MTRAGEval: Evaluating Multi-Turn RAG Conversations
- - Sara Rosenthal
  - Yannis Katsis
  - et al.
- 2026
- ACL 2026
Workshop
Activated LoRA: Fine-tuned LLMs for Intrinsics
- - Kristjan Greenewald
  - Luis Lastras
  - et al.
- 2025
- NeurIPS 2025
Conference paper
MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems
- - Yannis Katsis
  - Sara Rosenthal
  - et al.
- 2025
- ACL 2025
Conference paper
How do Categorical Duplicates Affect ML? A New Benchmark and Empirical Analyses
- - Vraj Shah
  - Thomas Parashos
  - et al.
- 2024
- VLDB 2024
Conference paper

Visit Google Scholar

Top collaborators

SR

Sara Rosenthal

Sara Rosenthal

Staff Research Scientist

LP

Lucian Popa

Lucian Popa

Principal Research Scientist

MD

Marina Danilevsky

Marina Danilevsky

Senior Research Scientist, Manager - core language and conversational technologies

YK

Yannis Katsis

Yannis Katsis

Senior Research Scientist