Victor Barres

I build and study conversational AI agents — systems that have to get real work done while sustaining long, coherent interactions with the people they work with. Doing both at once is where most of what’s hard about deploying them lives.

I’m a founding member of the APEX research team at Mercor, where I’m taking that beyond the single conversation: agents inside real organizations, working with people, and transforming the work itself — and how we measure that well enough to understand it, and help shape it. With Aden Barton, Nicolai Foss and Henrik Nielsen, I’m building a bridge between organization theory and AI evaluation (Harder Than the Task). Before that, I led the τ-Bench family of agent benchmarks at Sierra (live leaderboard at taubench.com).

My background is in computational cognitive science and cognitive linguistics, and I’ve spent years building real-world conversational systems across several startups. More on how I think about the work →

the τ-Bench family

At Sierra I led the τ-Bench family of agent benchmarks (originally introduced there in 2024) — code, repo, public leaderboard, and a sequence of extensions, a line I still contribute to:

τ²-Bench — extends τ-Bench to a dual-control setting where both the agent and the user can act on the world.
τ-Knowledge — knowledge-retrieval domain.
τ-Voice — first benchmark to measure full-duplex voice agents on realistic, grounded customer-service tasks.
τ³-Bench — combines τ-Knowledge and τ-Voice with community-contributed task fixes and code improvements.
τ-i18n — extends voice-agent evaluation across languages: Spanish, Brazilian Portuguese, Hindi, Korean and Mandarin.
τ-Entity — can voice agents get the details right? Someone’s name, phone number or email, over the phone.
Hyper-τ-Bench — flips τ-Bench around: the agent has to build the agent, from scattered requirements and a client who holds the rest, and we grade what it ships.

User simulation runs through all of it — to build and evaluate agents that talk with people, you need simulated people you can trust. I’m co-organizing the NeurIPS 2026 workshop on Grounded User Simulation for Model Evaluation and Training (Paris, December 12).

news

Nov 05, 2026	New paper with Aden Barton, and with Nicolai J. Foss and Henrik M. Nielsen of Copenhagen Business School, building a bridge between organization theory and AI evaluation: Harder Than the Task (blog). Benchmarks are made of tasks, but a job is done with others, who hold information, make decisions and depend on you. Only 16% of benchmarks measure any organizational dimension, though attention is growing fast: from 14% claiming one in 2023 to 33% in 2026.
Oct 22, 2026	Talk at Training Agents IRL (GitHub, San Francisco; hosted by Hugging Face / OpenEnv): Fit for Purpose: Environments, benchmarks, and the metrology of AI agents (slides, video). Attach a task and a grader, and an environment produces a number. You can judge it by what it does to the model, or by what it tells you about the model: a reward is not a measurement.
Oct 21, 2026	Two new voice-agent benchmarks with Soham Ray and the Sierra team, with leaderboards on taubench.com. τ-Entity starts from something that sounds trivial: getting someone’s name, phone number or email right over the phone. When agents do check a wrong value, only 24–37% get repaired. τ-i18n takes the τ-bench airline, retail and telecom tasks into Spanish, Brazilian Portuguese, Hindi, Korean and Mandarin, and scores task completion, interaction and generation separately. They don’t line up.
Sep 04, 2026	Hyper-τ-Bench released — do agents have what it takes to build agents? The best developer agent ships 23.9%; an engineer’s hand-built reference reaches 82.2%. Open source, with a leaderboard that takes submissions by pull request.
Jul 13, 2026	Co-organizing the NeurIPS 2026 workshop on Grounded User Simulation for Model Evaluation and Training — Paris, December 12.

selected publications

arXiv

Harder Than the Task: From Technical to Organizational Difficulty in AI Benchmarking

Victor Barres, Aden Barton, Nicolai J. Foss, and Henrik M. Nielsen

2026

arXiv Bib HTML

@misc{barres2026harder,
  title = {Harder Than the Task: From Technical to Organizational Difficulty in AI Benchmarking},
  author = {Barres, Victor and Barton, Aden and Foss, Nicolai J. and Nielsen, Henrik M.},
  year = {2026},
  archiveprefix = {arXiv},
  primaryclass = {cs.CY},
  url = {https://arxiv.org/abs/2611.03494},
  keywords = {ai-agents}
}

arXiv

Hyper-τ-Bench: An Environment for End-To-End, Realistic Agent Construction

Quan Shi, Keshav Dhandhania, Karthik Narasimhan, and Victor Barres

2026

arXiv Bib HTML Website

@misc{shi2026hypertau,
  title = {Hyper-τ-Bench: An Environment for End-To-End, Realistic Agent Construction},
  author = {Shi, Quan and Dhandhania, Keshav and Narasimhan, Karthik and Barres, Victor},
  year = {2026},
  archiveprefix = {arXiv},
  primaryclass = {cs.AI},
  url = {https://arxiv.org/abs/2609.04611},
  keywords = {ai-agents}
}

ICML

τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge

Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan, and Victor Barres

arXiv preprint arXiv:2603.04370, 2026

Accepted at ICML 2026.

arXiv Bib HTML Code

@article{shi2026tauknowledge,
  title = {τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge},
  author = {Shi, Quan and Zytek, Alexandra and Razavi, Pedram and Narasimhan, Karthik and Barres, Victor},
  journal = {arXiv preprint arXiv:2603.04370},
  note = {Accepted at ICML 2026.},
  year = {2026},
  url = {https://arxiv.org/abs/2603.04370},
  keywords = {ai-agents}
}

ICML

τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains

Victor Barres^*, Soham Ray^*, Keshav Dhandhania^*, and Karthik Narasimhan

2026

Accepted at ICML 2026.

arXiv Bib HTML Code

@misc{ray2026tauvoice,
  title = {τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains},
  author = {Barres, Victor and Ray, Soham and Dhandhania, Keshav and Narasimhan, Karthik},
  note = {Accepted at ICML 2026.},
  year = {2026},
  archiveprefix = {arXiv},
  primaryclass = {cs.SD},
  url = {https://arxiv.org/abs/2603.13686},
  keywords = {ai-agents}
}

ICML
τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment

Victor Barres^*, Honghua Dong^*, Soham Ray, Xujie Si, and Karthik Narasimhan

2025

Oral at ICML 2026.

Abs arXiv Bib HTML Code Slides

We introduce τ²-bench, a benchmark for evaluating conversational agents in a dual-control environment, where both the agent and the user can take actions on the world. τ²-bench extends τ-bench’s tool-agent-user interaction setting and supports rigorous evaluation of agents that must coordinate with users in real-world customer-service scenarios.
@misc{barres2025tau2, title = {τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment}, author = {Barres, Victor and Dong, Honghua and Ray, Soham and Si, Xujie and Narasimhan, Karthik}, note = {Oral at ICML 2026.}, year = {2025}, archiveprefix = {arXiv}, primaryclass = {cs.AI}, url = {https://arxiv.org/abs/2506.07982}, keywords = {ai-agents} }

NAACL

From Generating Answers to Building Explanations: Integrating Multi-Round RAG and Causal Modeling for Scientific QA

Victor Barres^*, Clifton James McFate^*, Aditya Kalyanpur, Kailash Karthik Saravanakumar, Lori Moon, Natnael Seifu, and Abraham Bautista-Castillo

In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry Track), 2025

Bib HTML Slides

@inproceedings{barres2025explanations,
  title = {From Generating Answers to Building Explanations: Integrating Multi-Round {RAG} and Causal Modeling for Scientific {QA}},
  author = {Barres, Victor and McFate, Clifton James and Kalyanpur, Aditya and Saravanakumar, Kailash Karthik and Moon, Lori and Seifu, Natnael and Bautista-Castillo, Abraham},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry Track)},
  year = {2025},
  url = {https://aclanthology.org/2025.naacl-industry.42/},
  keywords = {ai-agents}
}

arXiv
LLM-ARC: Enhancing LLMs with an Automated Reasoning Critic

Aditya Kalyanpur, Kailash Karthik Saravanakumar, Victor Barres, Jennifer Chu-Carroll, David Melville, and David Ferrucci

2024

Abs arXiv Bib HTML

We introduce LLM-ARC, a neuro-symbolic framework designed to enhance the logical reasoning capabilities of Large Language Models (LLMs), by combining them with an Automated Reasoning Critic (ARC). LLM-ARC employs an Actor-Critic method where the LLM Actor generates declarative logic programs along with tests for semantic correctness, while the Automated Reasoning Critic evaluates the code, runs the tests and provides feedback on test failures for iterative refinement.
@misc{kalyanpur2024llmarc, title = {{LLM-ARC}: Enhancing {LLMs} with an Automated Reasoning Critic}, author = {Kalyanpur, Aditya and Saravanakumar, Kailash Karthik and Barres, Victor and Chu-Carroll, Jennifer and Melville, David and Ferrucci, David}, year = {2024}, archiveprefix = {arXiv}, primaryclass = {cs.AI}, url = {https://arxiv.org/abs/2406.17663}, keywords = {ai-agents} }

AAAI-SS

Template Construction Grammar: A Schema-Theoretic Computational Construction Grammar

Victor J. Barres

In AAAI Spring Symposium on Computational Construction Grammar and Natural Language Understanding, 2017

Bib HTML

@inproceedings{barres2017tcg,
  title = {Template Construction Grammar: A Schema-Theoretic Computational Construction Grammar},
  author = {Barres, Victor J.},
  booktitle = {AAAI Spring Symposium on Computational Construction Grammar and Natural Language Understanding},
  year = {2017},
  url = {https://aaai.org/proceeding/02-spring-2017/},
  keywords = {cogsci}
}

See all publications →