Essay

States Are Starting to Build the Public Layer for AI

States are beginning to build a public layer for AI through inventories, procurement rules, training, shared tools, and limits on consequential uses. The framework is emerging, but dependence on private platforms remains its weakest point.

Mark Howard

— 10 min read
Share

Inventories, procurement rules, training, shared tools, and limits on consequential uses are beginning to form a public framework.

No state has built a complete public infrastructure for artificial intelligence.

Several are beginning to assemble the parts.

The work is not arriving through one grand program. It is appearing in public inventories, procurement rules, employee training, controlled pilots, shared tools, risk reviews, and occasional decisions about what AI should not be allowed to do.

Taken separately, these can look like administrative details.

Together, they begin to resemble a public layer for AI.

That layer matters because state governments will not experience AI only as regulators. They are also large employers, buyers of technology, holders of sensitive data, operators of public services, and sources of technical assistance for schools, local governments, and other institutions.

They will use AI whether or not they build the capacity to govern it well.

The early record is mixed. California, Maryland, Colorado, Pennsylvania, Illinois, New Jersey, and Washington have each developed useful parts of the framework. None has completed it. Most still rely heavily on private models, private clouds, and vendor expertise.

That is not surprising. Roads were not built before the first wagon, and procurement offices do not usually get a decade of calm reflection before a new technology arrives.

Still, the institutional choices being made now will matter later.

A state that knows where AI is being used, tests systems before deployment, trains its workforce, shares expertise, and preserves human authority is in a stronger position than one that simply opens accounts and waits for efficiency to happen.

The public layer beginning to emerge has seven parts:

  1. Visibility
  2. Rules
  3. Experimentation
  4. Workforce capacity
  5. Shared infrastructure
  6. Contestability
  7. Portability and independence

The first five are becoming easier to see. The sixth appears unevenly. The seventh remains weak almost everywhere.

That last gap may determine whether states are building public capacity or merely becoming better-organized customers.

1. Visibility: know where AI is being used

A government cannot govern systems it cannot see.

That makes inventories a basic piece of public AI infrastructure.

Maryland’s public AI inventory is one of the clearest examples. State law requires the Department of Information Technology to maintain a record of AI uses across agencies, from ordinary productivity tools to systems developed or operated for specialized public purposes.

The inventory does more than produce a list.

It helps the state understand which tools employees are using, where sensitive data may be involved, whether several agencies are solving the same problem separately, and which vendors are becoming embedded across government. It also gives the public some ability to see how AI is entering state operations.

Washington has moved in a similar direction. Its state AI policy and procurement guidance require agencies to identify and document AI-enabled technology, including through the state application-inventory process.

Visibility sounds elementary. In practice, it is difficult.

Employees may use public chatbots without telling their agencies. AI features can appear inside ordinary software updates. A vendor may describe a product as analytics, automation, fraud detection, or decision support rather than AI. Agencies may not know which subcontractors or embedded services are involved.

An inventory therefore needs continuing intake, clear definitions, and enough authority to reach systems that did not arrive through the front door.

It also needs to be useful. A spreadsheet created for compliance and left untouched is not infrastructure. The value comes from connecting the inventory to procurement, risk review, monitoring, and public accountability.

Maryland is beginning to do that. Its inventory includes proposed uses as well as deployed systems, and the state’s AI team can use the information to understand demand for different tools and shape a broader platform strategy.

The public layer starts by making the system visible.

2. Rules: decide how AI enters government

Rules are the second component.

Many states now have acceptable-use policies or general statements of responsible-AI principles. The stronger frameworks connect those principles to actual decisions.

Colorado’s statewide approach requires state agencies to submit generative-AI uses—including tools supplied by third-party vendors—to the Governor’s Office of Information Technology for risk assessment. Colorado organizes its strategy around governance, innovation, and education, and places AI proposals inside an existing state technology-review structure.

That matters because AI adoption rarely presents itself as a philosophical question. It arrives as a procurement request, a software feature, a pilot proposal, or an employee asking whether a tool is permitted.

The rules must operate at that level.

Who must review the system? Which uses require more scrutiny? What information must a vendor provide? What happens when the technology changes after purchase? Who remains accountable for the result?

California has also moved toward a more explicit procurement and risk framework. Its March 2026 executive order on trusted AI procurement strengthened civil-rights, privacy, and risk protections while continuing the state’s effort to deploy generative AI in public services.

Washington’s generative-AI procurement guidelines likewise connect acquisition to risk assessment, data practices, testing, documentation, and continuing oversight.

The important shift is from broad principle to operating rule.

A framework becomes real when an agency cannot bypass it merely by calling the system a pilot.

3. Experimentation: create places to learn before scaling

Government needs room to experiment.

It also needs a way to prevent “pilot” from becoming a permanent category in which systems operate without ordinary accountability.

Maryland’s emerging AI Innovation Lab is intended to help agencies turn promising experiments into usable government tools. The larger Maryland strategy combines pilots, technical assistance, policy guidance, and an AI Enablement Team that agencies can consult as they evaluate possible uses.

Colorado offers a useful example of structured experimentation. Its 90-day Google Gemini pilot included 150 employees across 18 agencies. Participants completed training, signed agreements, joined a learning cohort, and submitted more than 2,000 surveys about productivity, accuracy, fairness, privacy, and other concerns.

The pilot produced encouraging results. More important for this argument, it produced a method.

Colorado created a central information hub, recurring community-of-practice meetings, participation requirements, feedback loops, and a process for deciding whether the tool should move toward broader use. The pilot generated institutional knowledge rather than only a press release.

Pennsylvania has also moved beyond a single demonstration. The Commonwealth reports that agencies have proposed and evaluated dozens of AI pilots, while some successful uses have entered ordinary operations. These include document-quality checks for benefits applications, transcription support for clemency records, and customer-service systems that rely on approved answers and escalate unfamiliar questions to human staff. (Pennsylvania Office of Administration)

Experimentation becomes public capacity when governments preserve what they learn.

That includes negative results. A pilot that shows a tool is unreliable, too expensive, difficult to audit, or poorly matched to the problem may be more valuable than one that produces a flattering productivity estimate.

Public institutions need permission to conclude that AI was not the answer.

4. Workforce capacity: teach the institution, not just the user

AI training can mean many things.

At its thinnest, it teaches employees how to write prompts and avoid entering confidential information.

That is useful. It is not enough.

Public capacity requires employees who can define problems, evaluate tools, recognize failure, redesign workflows, and participate in decisions about where AI belongs.

Pennsylvania provides one of the strongest current examples of workforce-scale adoption. By April 2026, the state reported more than 3,000 employees using approved generative-AI tools across 35 agencies, with another 6,500 enrolled in required training. It has also created a Generative AI Governing Board and a labor-management collaboration group that gives represented employees a role in shaping adoption.

That last feature deserves attention.

Workers often understand the practical process better than anyone else. They know where the official workflow diverges from the real one, which exceptions matter, and what kinds of errors create harm. Excluding them from implementation wastes knowledge and makes resistance more likely.

Colorado’s pilot produced a continuing AI community of practice across agencies. Maryland offers public employees training, office hours, and a community of practice through which civil servants can exchange examples and receive help from the state AI Enablement Team.

These structures help preserve institutional learning.

Without them, knowledge remains scattered among a few enthusiasts, consultants, or vendor representatives. When those people leave, the institution discovers that its AI strategy had been stored largely in their calendars.

Workforce capacity is not a one-time course.

It is the ability to keep learning as the technology and the work change.

5. Shared infrastructure: give agencies more than separate subscriptions

States can also create shared tools and services.

This may be one of their most important roles. Individual agencies, school districts, and local governments often lack the staff, bargaining power, and technical expertise needed to evaluate AI systems on their own.

New Jersey has taken a direct approach through the NJ AI Assistant, a secure generative-AI platform for state employees and authorized users. The state reports that thousands of public-service professionals have used the system. New Jersey has paired the tool with training and an innovation organization intended to help agencies build implementation skills.

Maryland is assembling a different version of shared infrastructure. Its strategy includes common policy guidance, an inventory, technical assistance, training, office hours, an Innovation Lab, and partnerships with the University System of Maryland. State legislation has also established an AI partnership and a public-service fellowship intended to connect public institutions with expertise and talent. (Maryland AI legislation)

California’s agreement with Anthropic offers another route. State agencies—and California cities and counties—can obtain Claude at a substantial discount, along with training, technical assistance, and workflow support.

That may accelerate adoption, especially for smaller public institutions.

It also exposes the central tension in this entire framework.

Shared public access is not necessarily shared public infrastructure.

The Anthropic agreement can give agencies useful tools and expertise while concentrating implementation knowledge around one provider. The state may become a more capable buyer and still remain dependent on a vendor’s model, product roadmap, pricing, and technical architecture.

The relevant test is what the public institution retains.

Does it gain reusable expertise, common standards, evaluation methods, and bargaining power? Can it compare providers and move between them? Or does the shared layer become a statewide channel into one private platform?

A group discount is helpful.

It is not sovereignty.

6. Contestability: preserve the authority to say no

A public framework must do more than help government adopt AI.

It must also preserve the ability of workers, residents, and public institutions to question how the technology is used.

Illinois provides a particularly clear paired example.

The state has produced extensive guidance for school districts and educators on AI use in teaching and learning, following legislation that required the Illinois State Board of Education to develop statewide resources. The guidance addresses practical uses, student literacy, data protection, misinformation, vendor evaluation, and the continuing role of teachers. (Illinois education AI guidance reporting)

At the same time, Illinois enacted Public Act 104-0565, which prohibits AI from assigning numerical scores or qualitative ratings in teacher evaluations and bars its use for evaluation tasks requiring professional judgment. AI may support administrative work, but the consequential judgment remains with the evaluator.

One measure helps institutions use AI.

The other defines a boundary.

Both are forms of capacity.

Contestability also includes public inventories, understandable explanations, impact assessments, audit rights, incident reporting, appeal processes, and meaningful human review. It asks whether people affected by a system can see that it exists, understand its role, challenge errors, and reach someone with authority to change the outcome.

This remains less developed than procurement or training.

Most state frameworks are designed primarily around the needs of agencies and employees. Public participation often comes later, if it comes at all. The result can be responsible internal administration without much external leverage for the people subject to the system.

That is better than ungoverned deployment.

It is not yet a complete public layer.

7. Portability and independence: the missing piece

The final component is the least developed.

Can a state move its data, prompts, workflow history, evaluations, and institutional knowledge from one AI provider to another?

Can it use multiple models inside the same operating environment?

Can agencies continue important services if a provider changes its terms, raises prices, restricts a use, or withdraws a product?

Can public institutions inspect enough of the system to understand what they are relying on?

Most current frameworks have better answers for acceptable use than for exit.

California’s Anthropic agreement, Pennsylvania’s expansion from a ChatGPT Enterprise pilot, Colorado’s use of Gemini within Google Workspace, and New Jersey’s statewide assistant all create practical capability. They may also deepen dependence on particular models, clouds, or technical ecosystems.

That does not make them mistakes.

A state cannot wait for a perfectly open, interoperable, publicly controlled stack before improving services. Nor should every government try to build a frontier model and cloud platform from scratch. That would be an expensive route to rediscovering why specialization exists.

The realistic goal is not self-sufficiency.

It is retained choice.

States can pursue that through contract terms, exportable records, common data formats, model-neutral interfaces, multi-vendor strategies, open standards, continuity provisions, independent evaluation, and internal staff who understand the system well enough to change it.

This is where procurement, implementation capacity, and public infrastructure meet.

Without portability, the first six parts of the framework can make adoption safer and more competent while leaving the underlying dependency largely untouched.

The public layer becomes well governed.

The foundation beneath it remains private.

The framework is emerging unevenly

No state currently demonstrates all seven elements.

California has scale, procurement authority, deployments, workforce initiatives, and substantial vendor partnerships.

Maryland has built one of the clearest institutional structures around inventories, policy, training, shared expertise, and experimentation.

Colorado has developed a disciplined approach to review, pilots, monitoring, and cross-agency learning.

Pennsylvania has moved furthest toward workforce-scale use and operational applications, while bringing employees into parts of the implementation process.

Illinois shows how operational guidance can be paired with an enforceable limit on consequential use.

New Jersey demonstrates the value of a shared, secure employee platform.

Washington provides a strong governance and procurement architecture.

These are not interchangeable models, and they should not be forced into a ranking.

A national Code for America assessment found a similar uneven pattern. States are moving rapidly through governance and experimentation, but more slowly into operational scaling and systematic measurement of public value.

That is a useful caution.

A state can have a policy, a task force, a pilot, a training program, and a chatbot without knowing whether public services have improved.

The framework should eventually produce evidence:

  • fewer errors;
  • faster or more accessible services;
  • better working conditions;
  • clearer accountability;
  • lower switching costs;
  • wider institutional competence;
  • more public ability to see and challenge AI use.

Otherwise, the public layer risks becoming a polished administrative shell around ordinary technology procurement.

What states can build

The emerging framework does not require states to own every model or server.

It asks them to build the capacity to act.

That means knowing what systems are in use. Setting rules before harm occurs. Creating protected spaces for experimentation. Training employees and preserving what they learn. Sharing tools and expertise across agencies. Giving the public ways to understand and challenge consequential uses. Maintaining enough independence to change providers and direction.

Those are ordinary functions of capable government.

AI makes them newly urgent.

The states now moving fastest are not simply “using AI.” They are beginning to build institutions around it.

Whether those institutions become durable public infrastructure will depend on what happens next: whether pilots become accountable operations, whether training becomes organizational knowledge, whether shared tools produce public bargaining power, and whether states can preserve the ability to leave the platforms they are entering.

The public layer is beginning to appear.

It is not finished, and it does not yet own the ground beneath it.

References and Further Reading

Read more