Artificial intelligence agents are getting better, but they still struggle to handle the complexity of real work environments. Unlike current benchmarks that test one task at a time, actual workplaces require managing dozens of interconnected tasks simultaneously. To bridge this gap, researchers have created a new testing environment called Multi-Horizon Task Environments (MHTEs), which simulate the multitasking nature of real jobs.

Why Traditional AI Benchmarks Fall Short

Most AI benchmarks evaluate agents on a single task, which doesn’t reflect the demands of real work. When tested under multi-task loads, even the best AI agents show sharp drops in performance. For example, as the number of concurrent tasks increases from 12 to 46, their completion rates fall from about 17% to under 9%. This highlights key weaknesses such as memory limitations, interference between tasks, and the need for constant reprioritization.

In real offices, workers juggle many dependent and interrelated tasks, like preparing reports, updating spreadsheets, and replying to emails. These tasks depend on each other, forming complex webs rather than simple sequences. AI agents need advanced memory, planning, and learning capabilities to operate effectively in such environments. The new MHTEs aim to test these skills more accurately, pushing AI development closer to real-world usefulness.

Introducing CORPGEN’s Digital Employees

The researchers behind this work developed a system called CORPGEN, which creates digital employees powered by large language models. These AI agents have persistent identities, role-specific expertise, and realistic work schedules. They can operate productivity tools like Microsoft Office through automated interfaces, mimicking how a human worker would perform tasks throughout a typical workday.

What makes CORPGEN stand out is its modular, architecture-agnostic design. The improvements in performance come from better system design rather than relying on a single model. As underlying AI models improve, these digital employees will become even more capable. They can remember previous tasks, plan ahead, learn from experience, and adapt to changing priorities—all critical for effective multitasking in complex environments.

How CORPGEN Performs in Multi-Horizon Tasks

In testing, CORPGEN’s digital employees consistently outperformed baseline agents across various scenarios. They achieved up to 3.5 times higher task completion rates. This demonstrates that their system architecture effectively addresses issues like memory management and task interference. These gains are significant because they show progress toward AI agents that can handle the real demands of workplace productivity.

The system simulates a typical workday, starting with a structured plan and moving through multiple interdependent tasks. This setup allows researchers to evaluate how well AI agents can manage overlapping responsibilities, reprioritize tasks in real-time, and adapt as situations change. As a result, CORPGEN offers a promising step toward AI that can truly assist in complex, multitask work environments.

Overall, this research highlights the importance of system design in developing more capable AI agents. By focusing on architecture and multi-task management, CORPGEN points the way toward AI tools that are more useful, flexible, and aligned with how humans work in real offices. As models continue to improve, so will these digital employees, bringing us closer to AI that can handle truly real-world work challenges effectively.

Inspired by

• https://www.microsoft.com/en-us/research/blog/corpgen-advances-ai-agents-for-real-work/

Sources

• github.com
• github.com
• aka.ms