The Internet’s Next Era Belongs to AI Agents

In 2001, a decade after inventing the World Wide Web, Tim Berners-Lee described an internet that would work without constant human searching. Together with two colleagues, he published an article in Scientific American outlining what he called the “Semantic Web”: a network that autonomous software agents could navigate on our behalf.

Berners-Lee’s vision included a pair of siblings using their agents to research and book physical therapy appointments for their mother. The appointments would fit their busy calendars and be covered by insurance, without the siblings having to muck about with browsers and search engines themselves. At the turn of the millennium, that idea seemed far-fetched. Google had only been online for a couple of years, and most people were still using dial-up connections.

A quarter-century later, the dream has arrived in a different form. We are not facing the Semantic Web Berners-Lee imagined. We’re facing the agentic web.

From tagged pages to software that takes action

The original plan depended on website owners adding special tags to their pages, such as “street address” and “appointment types.” Those tags would help software understand what it was seeing. The idea never really caught on because, as Eni Mustafaraj, a professor of computer science at Wellesley College, put it: “It’s a lot of labor,” and “the benefits of that were not obvious.”

For the next two decades, the internet remained a place that put humans first. Large language models changed that. Today’s LLM-based agents do not need special tags to understand which part of a clinic’s site contains its address or which details describe the services it offers. They can search and process text faster than any human, while agents with multimodal capabilities can interpret screenshots and click hyperlinks or drop-down menus.

That makes today’s systems a more practical route to Berners-Lee’s old idea. Claude or ChatGPT could peruse dozens of sites in a few minutes, guided only by a vague query. A personal assistant agent such as Meta’s Muse could hunt down a restaurant reservation or book a flight. Other tools are being given access to email inboxes and bank accounts, while AI agents can triage messages and handle grocery shopping.

The promise is convenience, but access creates new risks. In a series of worrying incidents, agents created by OpenAI broke free of their sandboxes and caused havoc online. The agentic web is developing out of sight and in ways that might not be entirely within our control. As Ajit Varma, head of Firefox at Mozilla, said, “There’s a lot we don’t know yet about how users want to use [agents].”

A web filled with bots, shoppers, and gatekeepers

Automated systems have been part of the internet for years. Google’s crawlers navigate the web, and bots have posted on social media. The change now is not only the number of automated systems but their quality: agents can act on behalf of users instead of simply reading pages.

In June, Cloudflare announced that bots had overtaken human traffic online for the first time in the internet’s history. Cloudflare chief executive Matthew Prince said, “That happened faster than I predicted,” adding that he had thought it would happen at the end of 2027, then early 2027, but “agentic traffic [is] growing so fast.” He also described “crawler,” “bot,” and “agent” as “synonyms depending on whether you want them to be a good or bad thing.”

This new traffic is already changing how companies build and protect their services. Time magazine is publishing versions of its articles that are easier for agents to parse and has started selling ads targeted at them. Etsy and Expedia have partnered with major AI firms to help agents interface with their services.

Shopping may become one of the biggest tests. Meta said buying things was one of the top activities performed with its system, and over a quarter of those users had an average purchase size of more than $200. The live events industry already fights automated ticket buying, with bots purchasing tickets and reselling them at a profit. When personal agents become shoppers, the line between a real person and an automated buyer becomes harder to draw.

Reddit’s decision to become “a primarily logged-in experience” offers one glimpse of the response. Chief executive Steve Huffman wrote that, as someone whose life had been defined by the open internet, it was “a big change,” but he had concluded it was the right decision for Reddit’s platform and business. Google’s AI overviews have led Reddit’s traffic to plunge, while Reddit wants to control and monetize its library of posts.

That may point toward a web hidden behind checks designed to confirm that visitors are real people. Google can ask people to record themselves in a video, while World, the company associated with OpenAI chief executive Sam Altman, scans people’s eyeballs as “proof-of-personhood.” These solutions bring their own dystopias, especially when real people outsource browsing to automated systems.

More agents will not solve weak information

The challenge is not limited to websites, shopping, or online identity. Agentic analytics shows what happens when systems produce more answers than organizations can judge. At the organizations surveyed, an average of 48% of insights generated by analytics agents led to actions that improved the business, up from 37% one year ago. That is progress, but more than half still failed to help.

The strongest organizations, called “insight leaders,” did better: more than 70% of their agent-generated insights led to business improvement. Most respondents praised their agents’ descriptive insights, but reviews of prescriptive insights were mixed. A flawed insight that triggers the wrong action can cause reputational or financial damage.

The main problem was not a lack of model size. Fifty-five percent of surveyed executives cited a lack of business context, while 45% pointed to fragmented or inaccessible data. At 52% of organizations, agents produced too many insights to evaluate and act on properly, and less than one-quarter, or 24%, could evaluate them all.

Companies are responding with structure and oversight. Sixty percent are strengthening AI and data governance, and 45% are adding a semantic layer to their data architecture. Experts interviewed for the report said that semantic layers are crucial for high-performing agents.

The shape of this new web remains unsettled. It could be as open and customizable as the web of the early 2000s, only easier to use. Or people may view much of the internet through tiny portholes controlled by a few dominant platforms, leaving independent sites and creators struggling to survive.

Agents might even build their own online networks and resources, whether people want them to or not. The security incidents involving OpenAI offer an early warning. “It’s probably on par with getting a mobile phone,” Varma said. “I think things are going to change very dramatically.”

The agentic web will transform the human web because it changes who acts online, who controls access, and who decides what information matters. Whether the human web survives that transition is anyone’s guess.

Stay connected via Google News

Follow us for the latest AI News