Skip to main content
Design With, Not For
The Equity Test · BML-13.SYN

Design With, Not For

Series 13: The Equity Test

By Syam Adusumilli · 12 min read · Cross-Cutting
In a Hurry? Read the executive summary.

The room has eight engineers, three product managers, and two clinical advisors. The average age is 34. The whiteboard shows a product roadmap with a launch date six months out. The product is a personal AI health companion for adults over 65. It will monitor medications, track cognitive change, coordinate care, and alert families and clinicians when something shifts. It is a good product. The people building it are competent and well-intentioned. No one in the room is 65.

No one in the room speaks AAVE. No one in the room lives on $1,140 a month. No one in the room conducted their last medical visit in a language other than English. No one in the room has an ITIN instead of a Social Security number. No one in the room uses a wheelchair. No one in the room has been incarcerated. No one in the room has hidden a relationship to survive in an institutional care setting. No one in the room lives on a reservation.

The product will launch in six months. It will work well for people who look, speak, earn, and age like the people in the room. For the people this series has named, the product will arrive with the gaps already built in. The gaps will become visible six months after launch, when the adverse event reports begin arriving. They will be harder to fix then than they would have been to prevent now.

What the Four Pieces Established
#

This series has documented four categories of exclusion in the personal AI ecosystem for aging adults.

The bias in speech and cognitive AI. Denise Watkins, 68, sharp and active, flagged by a speech analysis system that heard her African American Vernacular English as cognitive anomaly and a screening tool that measured her against a normative population she was never part of. The system cannot hear her correctly because it was not trained on people who talk like her. The fix is a training dataset. The harm is a referral that should never have been sent and a weekend Denise spent wondering whether she was losing her mind.

The cost barrier. Marvella Johnson, 72, retired home health aide, $1,140 a month, no smartphone, no broadband, a community health worker named Raymond who visits twice a month with a clipboard and a caseload of fifty-four. The minimum viable AI ecosystem costs more than Marvella has after rent and food. The free pathways exist but are insufficient. The most realistic near-term solution is not giving Marvella a device but giving Raymond an AI backend that extends his capacity between visits.

The language barrier. Carmen Gutierrez, 74, screened in English, flagged as borderline, diagnosed two years later in Spanish with MCI that was identifiable at year zero. The two-year delay narrowed her intervention window. The AI that monitors her cognition in English measures language fluency and calls it cognitive capacity. The multilingual AI that would have found her earlier is one to two years from clinical deployment in Spanish and longer for every other language.

The documentation, identity, disability, and sovereignty barrier. Rosa Mendoza, 71, thirty-one years in the United States, twenty-eight years of taxes paid on an ITIN, ineligible for the programs the benefits agent navigates. LGBTQ+ elders re-closeted in institutional settings where the AI’s relationship questions become risks. Elders with disabilities excluded by interfaces designed for sighted, hearing, mobile bodies. Indigenous elders whose relationship to land, data, and care does not map onto the ecosystem’s assumptions.

Together, these four pieces form a map of the people the current ecosystem does not serve. The map is not theoretical. Every person named in this series is a composite drawn from documented cases, published research, and the lived experience of the communities each person represents. The exclusions are current, operational, and producing harm.

The “For” Model and Its Failure
#

The conventional product development model builds for a target user, then adapts for edge cases. The target user is defined by the demographic profile of the people the product team knows best, which in practice means the demographic profile of the people in the room. The product is designed, tested, and validated for the target user. After launch, when the complaints and the adverse events and the advocacy letters arrive, the product team begins adapting for the populations it did not initially consider.

The model treats the excluded populations as edge cases. This framing is the problem. Denise is not an edge case. She is part of the 13 percent of Americans over 65 who are Black. Marvella is not an edge case. She is part of the 23 percent of Americans over 65 who live on less than $1,500 a month. Carmen is not an edge case. She is part of the 22 percent of Americans over 65 who speak a language other than English at home. Rosa is not an edge case. She is one of an estimated 600,000 undocumented immigrants over 60 in the United States. An elder with a disability is not an edge case. Roughly 46 percent of Americans over 75 have at least one disability.

The “edge cases” are, collectively, the majority of the population the product claims to serve.

The adaptation-after-launch model fails for a structural reason. By the time the product is stable, the architecture has encoded the exclusions. The training data has been fixed. The interface patterns have been established. The clinical validation has been conducted on the populations the system was designed for. Retrofitting inclusion into a product that was designed without it is exponentially harder than designing it in from the beginning. The speech analysis system that was trained on standard American English cannot be made dialect-aware by adding a patch. The training data must be rebuilt. The validation must be re-run. The clinical partnerships must be re-established. The cost of the retrofit exceeds the cost of the inclusive design by a factor that grows with every month of deployment.

The “With” Model
#

The alternative is simple to state and difficult to execute. Design with the excluded populations before the first line of code is written.

Community advisory boards with genuine decision-making authority, not consultative roles. The distinction matters. A consultative board gives feedback that the product team can accept or reject. A decision-making board has the authority to say no. A product team that cannot launch without the board’s approval builds differently from a product team that can ignore the board’s recommendations. The board should include older adults from the populations the product will serve, not only the populations easiest to recruit.

Training data collected from the populations the system will serve, not only from the populations that are cheapest and most convenient to gather. The speech analysis system that will monitor Black older adults must be trained on Black speech. The cognitive screening tool that will assess bilingual elders must be validated on bilingual elders. This is not a philosophical position. It is an engineering specification. A system trained on the wrong data produces wrong results. The remedy is the right data.

Testing protocols that include performance validation across the full demographic range before deployment. Not after launch. Not after the adverse events. Before the product reaches a single patient whose demographics differ from the training population. The standard is the same one applied to pharmaceuticals: demonstrate that it works for the people who will use it, across the range of people who will use it, before you give it to any of them.

Accessibility as a design requirement from the first prototype. Not a retrofit. Not a compliance checkbox. The interface works for a blind user from the first wireframe. The check-in system works for a deaf user from the first voice flow. The navigation works for a person with limited mobility from the first interaction design. The accessible-first product is not a concession. It is a better product for everyone.

What Inclusive Design Produces
#

The history of accessible design supports the claim that designing for excluded populations produces better products for everyone. Closed captions were developed for deaf and hard-of-hearing viewers. They are now used by millions of hearing people in gyms, airports, bars, and living rooms where someone else is sleeping. The curb cut was designed for wheelchair users. It is now used by parents with strollers, delivery workers with dollies, and travelers with rolling luggage. Voice interfaces were developed for people with mobility impairments who could not use keyboards. Voice interaction is now the dominant mode for smartphone users across all demographics.

The pattern is consistent enough to constitute a principle. Features designed for the margins often become the standard for the center. The engineering constraints imposed by designing for a blind user, a deaf user, a user with limited literacy, a user with no broadband, force the product team to build simpler, clearer, more robust systems. Those systems turn out to be what everyone prefers.

The argument for inclusive design is not only ethical. It is directionally correct about what the market will eventually demand. The product that launches accessible, multilingual, dialect-aware, and affordable reaches a larger market than the product that launches for the comfortable and adapts later. The adaptation never fully arrives, because the architecture was not built for it.

The Institutional Dimension
#

The market rewards speed. Build fast, launch fast, capture the early adopter segment, and expand from there. The early adopter segment for AI health products is affluent, English-speaking, technologically comfortable, and well-served by existing healthcare infrastructure. Building for that segment first is rational from a quarterly revenue perspective. It is also the decision that encodes every exclusion this series has documented.

Inclusive design takes longer. The community advisory boards require time. The diverse training data requires recruitment beyond the populations of convenience. The demographic performance validation requires testing that adds months to the development cycle. The accessibility-first approach requires design discipline that slows the first prototype. Every inclusive design requirement increases the time and cost of the pre-launch phase.

The institutions that fund AI health development need to require inclusive design as a funding condition, not encourage it as a value. The FDA needs to make bias testing before deployment a non-negotiable requirement for AI medical devices, not an aspirational guideline that companies can satisfy with a paragraph in a regulatory submission. Medicare and Medicaid need to include demographic performance data in their coverage determination for AI health tools.

These are institutional choices. They have not been made. Until they are, the market incentive is to build for the comfortable and call it universal, and the equity gaps this series has documented will be present at every product launch.

BML’s Own Equity Test
#

This series has turned a critical lens on the AI ecosystem. The same lens applies to the publication describing it.

BlueMirror.life is written in English. It assumes the reader has broadband access and a device capable of reading a web publication. It assumes digital literacy sufficient to navigate a multi-series editorial architecture. Every assumption that fails is a reader the publication excludes. The populations this series has spent four pieces naming, the populations most excluded by the AI ecosystem, are also the populations least likely to read the publication that documented their exclusion.

The free-pathway standard established in 13.02 is the publication’s standard for itself. Every recommendation includes the accessible alternative. Where a free alternative does not exist, the publication says so. Where a recommendation requires broadband, a smartphone, or a subscription, the publication names the cost and the subsidized option if one exists. A publication that describes solutions only for people who can afford them is a catalog. BML has committed to being something else.

The language limitation is named directly. BML currently publishes in English. This limits its reach to the English-speaking portion of the population it most needs to serve. Spanish-language publication is a goal the publication is working toward. It is not yet achieved. The gap between the goal and the reality is the same kind of gap this series has named in the products it reviews: real, acknowledged, and requiring work to close.

The First Line of Code
#

Return to the room. Eight engineers, three product managers, two clinical advisors. Average age 34. Product launch in six months.

Now change the room. Add Denise Watkins, the retired schoolteacher from Atlanta who was flagged by a system that could not hear her correctly. Add Marvella Johnson’s community health worker Raymond, who knows what the technology needs to do because he does it by hand for fifty-four people. Add Jorge Gutierrez, who knew the cognitive screening was wrong before anyone with a medical degree did. Add a disability rights advocate who has spent thirty years explaining that accessibility is not a favor. Add an Indigenous health worker from Montana who can explain why tribal data sovereignty is not an obstacle to be navigated but a principle to be honored.

The room is different. The conversation is different. The product roadmap changes. Some features move up. Some features that were not on the roadmap appear. The launch date moves from six months to eight. The two additional months are not a delay. They are the cost of building a product that works for the people it claims to serve.

The product that launches at eight months includes dialect-aware speech analysis tested on the populations it will monitor. It includes an offline-capable version that works on a basic phone with a limited data plan. It includes validated cognitive monitoring in Spanish and a roadmap for Mandarin, Tagalog, and Vietnamese with committed timelines. It includes a privacy architecture that lets Rosa navigate available programs without disclosing her immigration status. It includes accessibility from the first screen. It includes community advisory boards with the authority to say no.

The product that launches at six months does not include these things. It includes a plan to add them later, after the adverse event reports arrive, after the advocacy letters accumulate, after the regulatory pressure builds. The later version costs more, works less well, and reaches the excluded populations after the harm has been done and documented.

The two months are the argument. The question for every company building AI for aging adults is whether the people in the room include the people the product will serve. If they do not, the equity gaps are already in the architecture. The gaps will be discovered. They will be expensive to fix. And the people harmed in the interim will be the people who could least afford to be harmed.

The first line of code has not been written yet. The room is still being assembled. Who is in it determines what gets built.

How this article connects to others in Blue Mirror.

BML-13.01 (The AI That Hears You Wrong) documents the training data failure that produced Denise Watkins's referral; this synthesis identifies the design process that made that failure predictable — the room without the people who would have caught it — and specifies the engineering requirements (dialect-aware speech analysis, diverse training data as a specification rather than a recommendation) that would prevent it.
BML-13.02 (The AI That Costs Too Much) identifies the AI backend that would extend Raymond's capacity as the most realistic near-term path; this synthesis frames that solution as a design choice — putting the community health worker in the product development room before the first line of code is written rather than discovering his situation in the adverse event reports.
BML-14.02 (Broadband Is Healthcare) covers the infrastructure barrier that the design-with model cannot solve on its own — the AI designed for Agnes that works when the connection is good still requires the connection the BEAD program is building; this synthesis names the institutional funding decisions alongside the design decisions, because inclusive design and infrastructure investment are both necessary and neither is sufficient alone.
BML-15.03 (Policy That Would Change Everything) covers the specific regulatory and funding changes — FDA bias testing requirements, Medicaid AI coverage, broadband as healthcare infrastructure — that would make inclusive design economically viable rather than competitively disadvantageous; this synthesis identifies the institutional dimension that requires those policy changes, and 15.03 specifies what those changes are.
BML-17.SYN (The System Around You) synthesizes the structural forces — private equity, policy gatekeeping, institutional underfunding — that determine whether the equity changes this synthesis calls for are politically achievable; the reader who understands why the design process produces exclusions will find in 17.SYN the account of why the institutions that govern the design process have not required otherwise.
BGM's coverage of algorithmic bias in healthcare (BGM-9B, The Bias in the Machine) is the diagnostic foundation this synthesis builds a design argument on top of; BGM diagnosed the bias and its structural causes; this synthesis specifies the design process changes that would address it before deployment rather than after adverse events accumulate.
The FDA bias testing requirements, Medicare and Medicaid demographic performance data requirements, and coverage determination reforms this synthesis calls for are the policy territory MCR covers directly; readers who want the regulatory architecture and what specific rule changes would require inclusive design as a condition of coverage should follow MCR's analysis of AI medical device regulation.

Sources cited in this article.

  1. Obermeyer, Ziad, et al. "Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations." Science, vol. 366, no. 6464, 2019, pp. 447-453.
  2. Mace, Ronald L. "Universal Design: Barrier Free Environments for Everyone." Designers West, vol. 33, no. 1, 1985, pp. 147-152.
  3. U.S. Food and Drug Administration. "Artificial Intelligence and Machine Learning in Software as a Medical Device." FDA, 2024.
  4. Pew Research Center. "Internet/Broadband Fact Sheet." Pew Research Center, 2024.
  5. United States Census Bureau. "The Older Population: 2020." Census Brief, 2023.