AI agents are moving beyond answers and isolated tool calls. On November 3, 2026, Google AI & Infrastructure opened a new challenge built around the messy, demanding work of professional engineering: Physical-Engineering’s Last Exam, or PLE.
The 85-task benchmark now includes 29 public tasks, grading tools, and an evaluation container, alongside an open agent competition and leaderboard. The goal is clear: can an agent produce engineering work that meets codes, physical constraints, and design objectives?
A Benchmark Built Around Real Engineering Deliverables
PLE asks agents to complete assignments across mechanical, electrical, civil, and structural engineering. Instead of answering textbook-style questions, an agent must work through a project and deliver files intended for licensed review, including engineering memos, calculation workbooks, equipment schedules, and drawings.
The benchmark divides its 85 tasks across four disciplines:
- 30 mechanical tasks
- 27 electrical tasks
- 18 civil tasks
- 10 structural tasks
Each assignment runs inside a containerized project environment built on Mercor’s Archipelago. The projects average 61 files, with 5,170 files spread across the full benchmark. Those materials include PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models, and superseded revisions.
That structure forces an agent to make decisions before it can solve the engineering problem. It must determine which documents and requirements govern, perform the needed calculations, and issue a complete set of files rather than stopping after one correct answer. The work tests whether an agent can manage a project folder and turn scattered requirements into a finished submission.
Expert Work, Adversarial Testing, and Tougher Scoring
Practicing engineers built the 44 expert-authored tasks from start to finish, including project folders, prompts, and weighted grading rubrics. Engineers supplied the initial briefs for the 41 co-authored tasks, while an adversarial Creator–Solver pipeline constructed and tested the environments and executable graders using real licensable materials.
Experts then validated the tasks and increased their difficulty. The technical paper says no task includes proprietary or client-confidential information. Expert-authored environments were written from scratch, and co-authored files use public-domain, permissively licensed, or newly generated material.
PLE also changes how success gets measured. Expert-authored tasks use weighted criteria tied to evidence in submitted files, so an agent must show its work through the deliverables. Co-authored design tasks combine constraint compliance with an economic or performance objective.
A failed hard safety or feasibility gate prevents the affected design from earning objective credit. That rule addresses a weakness in rubric-only scoring: two wire schedules might satisfy the same code checks while carrying different installation costs. PLE’s scoring system therefore asks whether the work meets requirements and whether the resulting design achieves its stated objective.
This emphasis on completed work gives the competition a sharper edge. An agent cannot rely on a polished explanation if its calculations, drawings, schedules, or design choices fail the project’s requirements.
Fourteen Models Face the Private Test
Google evaluated 14 models with seven publicly available agent harnesses, running every task three times per configuration. Solver containers had no internet access and allowed six hours of agent time per run, creating a controlled setting for each attempt.
Rubric judging and executable grading happened outside the solver’s container. That separation kept graders and reference deliverables away from the agent, helping preserve the evaluation boundary while the agent worked through its project environment.
The private test set contains 56 tasks, and every model scored below 50% on its native harness. Claude Opus 5.5 led that comparison, but the reported results still show how difficult the benchmark remains across the tested systems.
That result matters because PLE does not measure whether an agent can produce a plausible response in a narrow exchange. It measures whether the system can sustain a long engineering workflow, track requirements across many files, complete calculations, and produce deliverables that survive code, safety, feasibility, cost, or performance checks.
The Public Competition Opens the Next Phase
The November 3 release marks the public opening of the tasks and evaluation tooling, not the paper’s first publication. The paper’s arXiv record lists an initial submission on October 21, 2026, followed by a revision on October 30, 2026.
Now developers can work with 29 public tasks, the grading tools, and the evaluation container, while the competition and leaderboard create a shared arena for agent performance. The private 56-task test set remains the tougher measuring ground described in the evaluation.
PLE points toward a future where AI agents are judged by the work they complete, not only by the words they generate. As agents enter technical fields, the decisive question will be waiting at the end of every project folder: can the system deliver engineering work that is safe, feasible, compliant, and useful?
Based on
- Google Opens 85-Task Engineering Benchmark and Agent Competition — unite.ai
- Chinese AI Models Fall Behind Anthropic, OpenAI in New AI Rankings – Business Insider — businessinsider.com
- Who gets to check the AI giants? A new group wants to set the rules — nbcnews.com
- Recursa Labs Reaches 100% on All 25 ARC-AGI-3 Games, Shares How at Bengaluru Mixer on Self-Improving AI – USA Today — usatoday.com
- AI news: How LLMs can Trojan Horse robots against users — ctvnews.ca
