Victor Barres

I build and study conversational AI agents — systems that have to get real work done while sustaining long, coherent interactions with the people they work with. Doing both at once is where most of what’s hard about deploying them lives.

I’m a founding member of the APEX research team at Mercor, where I’m taking that beyond the single conversation: agents inside real organizations, working with people, and transforming the work itself — and how we measure that well enough to understand it, and help shape it. Before that, I led the τ-Bench family of agent benchmarks at Sierra (live leaderboard at taubench.com).

My background is in computational cognitive science and cognitive linguistics, and I’ve spent years building real-world conversational systems across several startups. More on how I think about the work →

the τ-Bench family

At Sierra I led the τ-Bench family of agent benchmarks (originally introduced there in 2024) — code, repo, public leaderboard, and a sequence of extensions, a line I still contribute to:

τ²-Bench — extends τ-Bench to a dual-control setting where both the agent and the user can act on the world.
τ-Knowledge — knowledge-retrieval domain.
τ-Voice — first benchmark to measure full-duplex voice agents on realistic, grounded customer-service tasks.
τ³-Bench — combines τ-Knowledge and τ-Voice with community-contributed task fixes and code improvements.
τ-Multilingual — extends voice-agent evaluation across languages.
Hyper-τ-Bench — flips τ-Bench around: the agent has to build the agent, from scattered requirements and a client who holds the rest, and we grade what it ships.

User simulation runs through all of it — to build and evaluate agents that talk with people, you need simulated people you can trust. I’m co-organizing the NeurIPS 2026 workshop on Grounded User Simulation for Model Evaluation and Training (Paris, December 12).

news

Sep 04, 2026	Hyper-τ-Bench released — do agents have what it takes to build agents? The best developer agent ships 23.9%; an engineer’s hand-built reference reaches 82.2%. Open source, with a leaderboard that takes submissions by pull request.
Jul 13, 2026	Co-organizing the NeurIPS 2026 workshop on Grounded User Simulation for Model Evaluation and Training — Paris, December 12.
Jun 01, 2026	Joined Mercor as a founding member of the APEX research team — conversational agents in real organizations, and how they’re transforming work.
May 13, 2026	New blog post — τ-Knowledge: benchmarking agents on realistic knowledge. Frontier has moved from 25.5% → 37.4% Pass^1 since the March release, with ~63 pp of headroom still left. Includes a behavioral analysis of what separates the strong agents from the rest.
May 11, 2026	Three τ-Bench family papers accepted to ICML 2026 — including τ²-Bench as an oral (slides): τ²-Bench (dual-control evaluation), τ-Knowledge (knowledge retrieval), and τ-Voice (full-duplex voice agents). See you in July!

selected publications

arXiv

Hyper-τ-Bench: An Environment for End-To-End, Realistic Agent Construction

Quan Shi, Keshav Dhandhania, Karthik Narasimhan, and Victor Barres

2026

arXiv Bib HTML Website

@misc{shi2026hypertau,
  title = {Hyper-τ-Bench: An Environment for End-To-End, Realistic Agent Construction},
  author = {Shi, Quan and Dhandhania, Keshav and Narasimhan, Karthik and Barres, Victor},
  year = {2026},
  archiveprefix = {arXiv},
  primaryclass = {cs.AI},
  url = {https://arxiv.org/abs/2609.04611},
  keywords = {ai-agents}
}

ICML

τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge

Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan, and Victor Barres

arXiv preprint arXiv:2603.04370, 2026

Accepted at ICML 2026.

arXiv Bib HTML Code

@article{shi2026tauknowledge,
  title = {τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge},
  author = {Shi, Quan and Zytek, Alexandra and Razavi, Pedram and Narasimhan, Karthik and Barres, Victor},
  journal = {arXiv preprint arXiv:2603.04370},
  note = {Accepted at ICML 2026.},
  year = {2026},
  url = {https://arxiv.org/abs/2603.04370},
  keywords = {ai-agents}
}

ICML

τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains

Victor Barres^*, Soham Ray^*, Keshav Dhandhania^*, and Karthik Narasimhan

2026

Accepted at ICML 2026.

arXiv Bib HTML Code

@misc{ray2026tauvoice,
  title = {τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains},
  author = {Barres, Victor and Ray, Soham and Dhandhania, Keshav and Narasimhan, Karthik},
  note = {Accepted at ICML 2026.},
  year = {2026},
  archiveprefix = {arXiv},
  primaryclass = {cs.SD},
  url = {https://arxiv.org/abs/2603.13686},
  keywords = {ai-agents}
}

ICML
τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment

Victor Barres^*, Honghua Dong^*, Soham Ray, Xujie Si, and Karthik Narasimhan

2025

Oral at ICML 2026.

Abs arXiv Bib HTML Code Slides

We introduce τ²-bench, a benchmark for evaluating conversational agents in a dual-control environment, where both the agent and the user can take actions on the world. τ²-bench extends τ-bench’s tool-agent-user interaction setting and supports rigorous evaluation of agents that must coordinate with users in real-world customer-service scenarios.
@misc{barres2025tau2, title = {τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment}, author = {Barres, Victor and Dong, Honghua and Ray, Soham and Si, Xujie and Narasimhan, Karthik}, note = {Oral at ICML 2026.}, year = {2025}, archiveprefix = {arXiv}, primaryclass = {cs.AI}, url = {https://arxiv.org/abs/2506.07982}, keywords = {ai-agents} }

NAACL

From Generating Answers to Building Explanations: Integrating Multi-Round RAG and Causal Modeling for Scientific QA

Victor Barres^*, Clifton James McFate^*, Aditya Kalyanpur, Kailash Karthik Saravanakumar, Lori Moon, Natnael Seifu, and Abraham Bautista-Castillo

In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry Track), 2025

Bib HTML Slides

@inproceedings{barres2025explanations,
  title = {From Generating Answers to Building Explanations: Integrating Multi-Round {RAG} and Causal Modeling for Scientific {QA}},
  author = {Barres, Victor and McFate, Clifton James and Kalyanpur, Aditya and Saravanakumar, Kailash Karthik and Moon, Lori and Seifu, Natnael and Bautista-Castillo, Abraham},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry Track)},
  year = {2025},
  url = {https://aclanthology.org/2025.naacl-industry.42/},
  keywords = {ai-agents}
}

arXiv
LLM-ARC: Enhancing LLMs with an Automated Reasoning Critic

Aditya Kalyanpur, Kailash Karthik Saravanakumar, Victor Barres, Jennifer Chu-Carroll, David Melville, and David Ferrucci

2024

Abs arXiv Bib HTML

We introduce LLM-ARC, a neuro-symbolic framework designed to enhance the logical reasoning capabilities of Large Language Models (LLMs), by combining them with an Automated Reasoning Critic (ARC). LLM-ARC employs an Actor-Critic method where the LLM Actor generates declarative logic programs along with tests for semantic correctness, while the Automated Reasoning Critic evaluates the code, runs the tests and provides feedback on test failures for iterative refinement.
@misc{kalyanpur2024llmarc, title = {{LLM-ARC}: Enhancing {LLMs} with an Automated Reasoning Critic}, author = {Kalyanpur, Aditya and Saravanakumar, Kailash Karthik and Barres, Victor and Chu-Carroll, Jennifer and Melville, David and Ferrucci, David}, year = {2024}, archiveprefix = {arXiv}, primaryclass = {cs.AI}, url = {https://arxiv.org/abs/2406.17663}, keywords = {ai-agents} }

AAAI-SS

Template Construction Grammar: A Schema-Theoretic Computational Construction Grammar

Victor J. Barres

In AAAI Spring Symposium on Computational Construction Grammar and Natural Language Understanding, 2017

Bib HTML

@inproceedings{barres2017tcg,
  title = {Template Construction Grammar: A Schema-Theoretic Computational Construction Grammar},
  author = {Barres, Victor J.},
  booktitle = {AAAI Spring Symposium on Computational Construction Grammar and Natural Language Understanding},
  year = {2017},
  url = {https://aaai.org/proceeding/02-spring-2017/},
  keywords = {cogsci}
}

See all publications →