Recent research shows that AI agents need specific skills to perform tasks effectively, but they can’t learn these skills on their own. Instead, they rely on humans to teach and guide them. A new benchmark called SkillsBench was created to test how well AI agents do across various fields, revealing the importance of human input in AI performance.

Understanding SkillsBench and Its Findings

SkillsBench evaluates AI agents on 84 different tasks across 11 industries, including healthcare, manufacturing, cybersecurity, and software development. The tasks ranged from security audits of software dependencies to analyzing complex biological data. Researchers tested each task under three conditions: without any skills, with curated skills provided by humans, and with agents prompted to generate their own skills.

The results showed that AI agents with curated skills — those given resources like code snippets and guides — performed significantly better than those with no skills. On average, they scored about 16 percentage points higher. However, even the best-performing agents still heavily depended on human-provided skills, highlighting that AI can’t yet fully operate independently.

The Limitations of AI Self-Development

Interestingly, when agents were asked to develop their own skills without any prior knowledge, their performance did not improve. This suggests that AI still needs humans to initiate and guide its learning process. In some cases, human guidance even negatively impacted the results, especially in certain tasks, indicating that human intervention isn’t always beneficial.

Performance varied greatly across different industries. For example, in healthcare tasks, curated skills had a big impact, helping AI achieve much better results. In contrast, for software engineering tasks, the benefit was quite small. This shows that the effectiveness of human-guided skills depends on the specific field and task complexity.

What This Means for the Future of AI

The research underscores that AI agents are not yet capable of fully independent learning. They still need humans to provide the initial knowledge and resources necessary for them to perform well. While AI can assist in many areas, human oversight remains crucial for ensuring accuracy and effectiveness.

This ongoing reliance on human input highlights the importance of collaboration between humans and AI. As AI technology advances, it will likely become more autonomous, but for now, human guidance is essential for optimal performance across various tasks and industries.

Overall, the study reminds us that AI is still a tool that benefits greatly from human expertise. Understanding its limitations helps set realistic expectations and encourages continued development of better training methods to make AI more independent in the future.

Inspired by

• https://www.infoworld.com/article/4135335/ai-agents-still-need-humans-to-teach-them-2.html

Sources

• arxiv.org