Redundancy with the accessibility tree

[Lulzx]

The accessibility tree already provides a machine-readable representation of a page's action space: labels, roles, states, expected inputs, validation errors, element relationships. Assistive technologies (screen readers, voice control, switch access) use this to perform actions a page supports.

An AI agent can use the same interface. No additional protocol is needed.

If a capability is missing from the accessibility tree for agents, it is also missing for assistive technology users. The fix should be in the accessibility layer, not in a parallel system. This serves both use cases with one implementation.

WebMCP introduces a second description of the page alongside the accessibility tree. These will diverge over time. The accessibility tree cannot desync because it is derived from the DOM. A separately maintained JavaScript tool registry can and will.

Structured tool schemas with typed parameters are not a UI-layer concern. This is what REST/GraphQL APIs are for. The explainer already states WebMCP is not a replacement for backend integrations.

The stamp database example (adding a stamp as a single high-level action rather than filling individual fields) is also useful for assistive technology. If this abstraction is needed, it should be added to the accessibility layer (AOM, ARIA extensions) where it benefits all consumers.

Is there a concrete use case that:

1. Cannot be handled by the accessibility tree (current or with extensions)
2. Is not better served by a backend API
3. Would not also benefit assistive technology users if added to the accessibility layer

Related: #65, #22

[adriangalilea]

Every single need an agent has a human also has.

In the web the right lever is accesibility, and we already have it, developers know it, LLM’s know it.

If for whatever reason you find the accesibility layer not being enough, the solution is not a new protocol but to improve the existing one. You’ll find both humans and agents benefit.

Solving agentic workflow needs with new protocols is a mistake (even MCP is the wrong solution, it is just self-expressive API’s developers also need, ex: OpenAPI)

[victorhuangwq]

i'm answering your third question regarding the accessibility layer. caveat: I am not an accessibility expert by any means, and would love to hear expert opinions on this as well.

speaking as someone involved in W3C standards discussions, in general discussions about the "Agentic Web," the conversations are trending against reusing accessibility trees and ARIA tags for AI agents, as this could actually result in a worse outcome for human users of assistive software.

specifically, during TPAC that occurred, there was a breakout discussing about semantics for agentic web: meeting minutes that i think would be worth taking a look at, as accessibility tree & aria was touched upon. some relevant quotes from the meeting notes

The accessibility tree is also fragile and causes crashes. Would be great if it didn't
… but that's the reality

"I am concerned that people will start using ARIA wrong, making the web worse for human users … So we should be careful about telling users to use ARIA" ---> this is probably aligned with Aria authoring practices guide that No ARIA is better than Bad ARIA.

to me, the key points are that if we overload accessibility features for agents, we risk creating an incentive for developers to optimize for machine efficiency over human usability. additionally, if sites somehow uses ARIA patterns to filter or block bots (if bots are intentionally consuming them), they may inadvertently block human screen reader users in the process.

[adriangalilea]

@victorhuangwq

Appreciate your input, but I feel those are weak points.

The accessibility tree is also fragile and causes crashes. Would be great if it didn't
… but that's the reality

So instad of fixing it it is better to just create something completely from scratch? makes no sense

the key points are that if we overload accessibility features for agents, we risk creating an incentive for developers to optimize for machine efficiency over human usability.

Think through this, tell me specifically a single feature an agent needs than a human does not.

The key to understand is that there is not a single different need. There is a 100% overlap. Make accesibility layer better and both humans and agents will benefit.

No ARIA is better than Bad ARIA.

What better thing could you have than agentic systems being the canary in the coal mine telling you about poor ARIA implementaitons? You are literally creating the perfect feedback loop, accesible websites if serving for agents will make websites more accessible not less, because I can point an agent to test a website for free.

[bvandersloot-mozilla]

What better thing could you have than agentic systems being the canary in the coal mine telling you about poor ARIA implementaitons? You are literally creating the perfect feedback loop, accesible websites if serving for agents will make websites more accessible not less, because I can point an agent to test a website for free.

Not quite. The information appropriate to put on an element depends on the audience and there is a probably a difference in what best assists an agent and what best assists a user assistive tech for a lot of use cases. The informational bandwith and additional context afforded to each is a big gap, putting aside that one is speaking to a user and another to an LLM. What was meant by No ARIA is better than Bad ARIA. in that meeting was that devs optimizing ARIA for agents would make choices that make that content bad for users that need it- Not that agents would unearth ARIA bugs.

[domfarolino]

to me, the key points are that if we overload accessibility features for agents, we risk creating an incentive for developers to optimize for machine efficiency over human usability. additionally, if sites somehow uses ARIA patterns to filter or block bots (if bots are intentionally consuming them), they may inadvertently block human screen reader users in the process.

I think this is one of the strongest points, and why the accessibility community is rightly skeptical of treating AX feature work and agent-focused primitives the same. These simply are not the same thing, and they are not directed at the same users, technology, or sinks.

The key to understand is that there is not a single different need. There is a 100% overlap. Make accesibility layer better and both humans and agents will benefit.

100% overlap? Wow, that's a lot. I'm unconvinced.

Think through this, tell me specifically a single feature an agent needs than a human does not.

The agent use case and accessibility use cases are entirely different. Accessibility technology helps surface the semantic web and what's on the screen, to users of AT so they can better actuate the site as a human would. WebMCP tools are designed for no human involvement. It's designed to skip all the stuff a user would normally use—like a site's semantic HTML, user interface, and UI—that agents have trouble processing like humans, and reduce the site to a set of low-level, web-UI-bypassing primitives. It's like the equivalent of using your computer as you normally do, vs. writing a script to automate your tasks with low-level system calls and OS APIs (things a user wouldn't enjoy touching).

I think users of AT will benefit from using agents that can access WebMCP tools and the like, but I don't think we can confuse agents actuating a site on behalf of a human in the other room, with "100%" of accessibility technology.

[DavidMulder0]

Haven't felt this conflicted reading a discussion for a while, so, let's try it like this: In what case would a WebMCP tool not be a relevant form (as in, the thing you fill out and then submit) a user of assistive technology might want to fill out to interact with the current state of the page?

I have spent the last hour on and off thinking about this, and I honestly can't think of tool examples that wouldn't at least benefit some users of assistive technologies.

I come from a (specialized) text editing background. I don't want to overstate my experience with accessibility software at the moment, because I have limited management buy-in, but 'basic' way a user of accessibility software would navigate such an application is by navigating through the visually laid out menu and toolbar structure. As far as I understand it this is... not a great experience for visually impaired users when working in applications with a lot of commands, and as far as I understand it the golden standard is to provide a "Command Palette" similar to the "Command Palette" in Visual Studio Code. Best I can tell the WebMCP tool list would exactly be such a Command Palette.

---

More generally, as a consumer of WebMCP I would always want to be able to see the tool call before it executes. On many sites I might allow it to run blindly, but let's imagine a future where WebMCP turns into a success and it's one of the main ways people interact with websites: Would you want an LLM to blindly execute tools on your banks website? So: if there (has to) be a way to visualize a tool call, then that's going to look like some type of form. (At which point you also probably want to have at least some ability to fix it, but 🤷.) And once we accept that a tool call is just a standardized form building format...

---

At the same time, I am not sure I agree with @Lulzx that this should be 'in' the accessibility layer. Rendering out to DOM just to have it be a part of the ARIA tree feels 'forced', and would probably not be great DX. The frustrating thing with ARIA is that it doesn't have any curb cut effect. Lulzx is proposing to add a curb cut effect (🎉), but I fear that will just result in (far) fewer people actually implementing such functionality.

The other way around though (assistive technology users benefiting from WebMCP as it is) sounds (more) plausible to me... Embrace the idea that (some) browsers might expose these tools directly to users. Not necessary even just for visually impaired users. I think I would install a browser that exposes these tools in a Command Palette-like interface in a heartbeat. The more I think about it, the more it feels like this should just be called "WebActions". If the goal is to expose "actions" (tools) that interact with your current page (great!)... then there is no reason to "tie" your fate to MCP.

[domfarolino]

I don't doubt that AT users will benefit from WebMCP. I don't see how they wouldn't; we're in agreement. There is nothing stopping accessibility technology from exposing a new command palette feature that surfaces WebMCP tools to the user, or the user using an agent directly which will certainly grow to be a form of AT itself. I'm just saying we don't need to turn WebMCP into a primitive of the accessibility tree and only a primitive of the accessibility tree, to have accessibility wins.

[Lulzx]

There is nothing stopping accessibility technology from exposing a new command palette feature that surfaces WebMCP tools to the user"

Exactly. That's the outcome I want.

The only remaining question is whether the spec should make that mapping normative (the way WebVTT became a MUST for reliable <video> captions).

The mapping itself is mechanical though: registerTool() name to action name, description to accessible description, inputSchema to parameter metadata, surfaced via platform APIs (AXCustomAction / IAccessibleAction / ATK), no DOM nodes or ARIA from the developer.

[adriangalilea]

@bvandersloot-mozilla

The information appropriate to put on an element depends on the audience

Agreed

and there is a probably a difference in what best assists an agent and what best assists a user assistive tech for a lot of use cases.

This is the core of my disagreement, could you please provide an example that illustrates this?

The informational bandwith and additional context afforded to each is a big gap, putting aside that one is speaking to a user and another to an LLM.

Not quite, ARIA is not consumed by the disabled user, just like no one is reading HTML, same thing for the LLM.

semantic HTML -(?)-> disabled user
semantic HTML -(?)-> LLM

I claim the overlap is ~100%, they require the same thing, both derived from proper semantic HTML/ARIA

What was meant by No ARIA is better than Bad ARIA. in that meeting was that devs optimizing ARIA for agents would make choices that make that content bad for users that need it- Not that agents would unearth ARIA bugs.

Again this is asuming there is some sort of divergence.

Treat an LLM as a blind user, it is what it is right now, now you realize the agent and the vision-impaired users have the same needs.

What is the information in this website?
What shape does it have?
What actions can I trigger here?

All this is common for both. There is no difference.

@DavidMulder0

I mostly agree with all you stated.

But while I think that it may be more explicit to have WebActions I think that its is a solved problem, semantic HTML and ARIA is just something that get’s consumed by the tool the disabled user consumes this from, an agent should just have his own way of interpreting this, which would be fairly similar.

Mind you, I just have as much expereince as the next guy that tries to make his websites somewhat accesible and made efforts to understand semantic web.

@domfarolino

I don't doubt that AT users will benefit from WebMCP. I don't see how they wouldn’t;

The more standards with overlaping capabilities the worse.

The developer needs to learn both, one new, unstable, with lack of documentation, LLM’s needs to be trained on this too, codebase grows, more context, more complexity, more $$$, more wasted time and energy, maintenance grows, more opportunities to desync, your html now shows X but webMCP says Y and accesibility software sais Z, so which is it?

The whole experience is worse.

So yes, to me this actively hurts accesibility and general use while providing nothing, nothing to the abled/disabled/agent consumer and nothing to the mantainer, which now has more technical debt.

[adriangalilea]

There is nothing stopping accessibility technology from exposing a new command palette feature that surfaces WebMCP tools to the user"

Exactly. That's the outcome I want.

The only remaining question is whether the spec should make that mapping normative (the way WebVTT became a MUST for reliable <video> captions).

The mapping itself is mechanical though: registerTool() name to action name, description to accessible description, inputSchema to parameter metadata, surfaced via platform APIs (AXCustomAction / IAccessibleAction / ATK), no DOM nodes or ARIA from the developer.

So you are saying <video> captions is solved, llm can use it, img has alt/description, forms/inputs require types and advertise themselves, what is exactly missing that a consumer llm client can’t have?

Why would a developer need to write this new standard instead of properly using what is already there?

I do believe the whole proposal is sitting in the wrong layer.

Create some sort of “browser” for LLM’s that derive from proper html

Don’t bloat with more standards.
Don’t create more debt.
Don’t add more noise.

[bvandersloot-mozilla]

@adriangalilea

Treat an LLM as a blind user, it is what it is right now, now you realize the agent and the vision-impaired users have the same needs.

Let me be charitable and rephrase this that LLMs and vision-impaired users both need non-visual ways to interpret a page. I agree! Hence why I also agree with Dominic that exposing tools to AT is a good idea. But I'm not sure what exactly you are asking for.

---

Probably not relevant to the particular spec discussion, but I think worth responding to!

and there is a probably a difference in what best assists an agent and what best assists a user assistive tech for a lot of use cases.

This is the core of my disagreement, could you please provide an example that illustrates this?

A vision-impaired user has a very different experience than one using an agent, depending on, for example, the ARIA labels and descriptions on a page. Consider something as basic as form labels. A few sentences in a label to provide a crystal clear picture of what is needed in a form field may help the agent get it right at the cost of a few hundred input tokens. The vision-impaired user probably just wants the most concise description, and may have to listen to multiple of these go on and on to figure out how to fill out a form properly. Humans have more context and less bandwidth for streams of text. I don't read my AGENTS.md file every time I open my editor.

[adriangalilea]

@bvandersloot-mozilla

Let me be charitable and rephrase this that LLMs and vision-impaired users both need non-visual ways to interpret a page. I agree!

<3

A few sentences in a label to provide a crystal clear picture of what is needed in a form field may help the agent get it right at the cost of a few hundred input tokens.

Can we test it?

Point me to a 10/10 accesible website, something that has just enough context that a vision-impaired user can navigate.

Then I’ll do a proof of concept agent consumer to illustrate that it needs no more than what is on the page and the prior context of the navigation/conversation.

It is just a matter of “context engineering” (🤢) the input needs to have the exact same information, let me prove it, just give me a link and a “persona” and I make the agent counter-part.

Additional imaginary example to illustrate my point:

Imagine Bob. Bob is a vision-impaired user.
Imagine Alice. Alice is an agent.

They visit the same website without any prior understanding of the website, the landing is a form without context.

They both are equally lost.

If the website lacks meaning, it lacks it for bob too.

If bob knew something about this website or his relationship with this website, it is not the website task to inform either bob or the agent. And if it is, it would be equally relevant to both.

There’s no asymmetry, not from the website POV.

If anything it is the user orchestrating the agent responsibility, and the prior context that he should have about Bob and it’s relationship with said website.

[anssiko]

@LJWatson this is relevant to your interests.

[sugosu]

“Users can use agents to perform web tasks faster without navigating the page. Removing navigation can make the web more accessible for people who struggle with it. Therefore, WebMCP could have potential accessibility benefits.”

Is this currently the primary assumption regarding the accessibility potential of WebMCP? I have been exploring this topic extensively, yet I have not identified clear examples or documented implementations demonstrating concrete accessibility benefits in practice (and maybe because there is none).