The Race — Building Democracy in the Age of AI. People race toward a shared finish line.
The Race
← Trends

Established trend · Active

Smaller local components create partial alternatives to hosted AI

Latest assessment: · Evidence updated

Standing score
Unscored
Direction of power
Leaning dispersion
Window
Opens
Confidence
Moderate

What we think is happening

Compact weights and local engines let some reasoning/decision steps run under an institution’s own control. Several model families and recent components provide alternative pieces of the stack, but service, licensing, hardware and integration dependence can remain elsewhere.

Scope: Released model/deployment components; availability and complete-workflow independence separated.

Potential direction of power

Smaller models and local components offer partial alternatives to hosted services. Licensing, memory, stable runtimes, integration, maintenance and total cost determine practical usefulness. Available components do not yet establish affordable, independently governable systems across smaller institutions.

Supporting cases

IBM Bob self-hosted deployment

IBM’s October 1 release describes self-hosted Bob deployment in customer-managed, private-cloud and air-gapped environments using supported licensed models. Optional external services can form part of a hybrid configuration. IBM and specialist reporting establish the announced deployment option.

Licensing, exit terms, costs and named production use remain unverified. Place the case in the private-deployment comparison, not as an open-weight release.

Read the case and sources →

Cloudflare Clef/Clef-flash

Cloudflare released Clef and Clef-flash on October 1: decision models with structured probabilities, Qwen-based 27B/9B backbones and Jev-compatible interfaces. Its primary release offers Apache 2.0 model artifacts for local use alongside Workers AI hosting. October 5 rereading confirms this boundary.

Cloudflare edge hosting is not on-device operation. Performance comparisons are supplier tests, and fine-tuning currently uses the provider’s engineers; self-service is prospective. Deployment cost and complete-workflow independence remain uncertain.

Read the case and sources →

Kev9Bv2 open local decision model

The model card dates Kev-9B v 2 to September 30 and identifies an Apache 2.0 adapter/head on an Apache 2.0 Qwen base. It returns probabilities over supplied options rather than generating text. Serving code targets a single 24GB-class GPU. October 5 rereading confirms the v 2 checkpoint and intended use.

Only 8,192-token context is validated despite a longer serving ceiling. The publisher advises workload calibration and human review for consequential decisions; results are self-reported.

Read the case and sources →

A Raspberry Pi example combines vision and language workloads locally on a small AI accelerator

Raspberry Pi’s October 5 technical account describes local camera/vision and small-language-model workloads using a Pi 5 with DeepX acceleration. The 25-TOPS Sixfab AI HAT+ is available, while a lower-performance 13-TOPS product is described as coming later in 2026. TOPS measures trillions of certain operations per second; it is not a direct measure of language-model quality.

The example combines object detection, visual interpretation and small-model responses without requiring each request to go to a hosted AI service. A supplier test of Qwen3 1.7B on the described small module reports approximately 4.64 output tokens per second. The roughly 3-watt figure concerns the accelerator, not an entire deployed system. These are supplier demonstrations, not independent performance or reliability studies. The compiler/model packaging and hardware remain dependencies; no blanket open-source-license claim is made.

Local control over a narrow workload differs from a complete independent AI organization. Slow output, available memory, integration and licensing can limit usability.

It does not establish a public procurement institution or a change in model-market concentration by itself.

Read the case and sources →

Microsoft adds local AI execution controls while expanding Windows' AI role

Microsoft announced October 7 that Windows 11's Microsoft Execution Containment is generally available. It gives AI agents defined limits on access to files, networks and other resources, with agent identities and permissions. Several agent products can use it; some named integrations are still forthcoming.

The same announcement describes local/cloud combinations and forthcoming hardware. Experimental Copilot “Hydra Fusion” is a later-October preview, not an already available general service. The concrete current development is the execution-control layer; the broader promise of more local capability remains partly prospective.

For The Race, local execution can broaden options, while the operating-system owner acquires greater power to define trusted access and permissions. These two mechanisms point in different directions.

Distinct connection: Deployable local components offer usable independent integration with hardware/license dependencies. Overlapping evidence is not independent confirmation.

Read the case and sources →

Perplexity releases downloadable document-search models

Perplexity published the pplx-embed-v2-late model family October 7. Its MIT-licensed weights create numerical representations of text and document images so software can retrieve relevant material. A small query model can search an index created with the larger model because they use a compatible representation.

This can support local searches of documents containing tables, charts and scanned pages. An organization still needs hardware, an index and integration work. The release establishes downloadable components, not a complete public-sector search service or evidence that governments have adopted it.

The power mechanism is practical separation of a useful retrieval step from a required hosted provider. Whether that becomes effective institutional independence depends on implementation capacity.

Read the case and sources →

Google releases a local meeting assistant and an openly licensed retrieval model

Google’s underlying launch was October 6; October 8 reporting brought it into this review. Its experimental Mac app, AI Edge Foresight, processes meeting audio and searches local documents and transcripts on the device. It uses system audio and the microphone, so Google says it can work across meeting platforms and offline.

The related EmbeddingGemma 2 model turns text, images, audio and video into representations that software can search and compare. Its 740 million parameters are modular, allowing developers to load only the components they need. The model card confirms an Apache 2.0 license. Foresight combines retrieval with Gemma 4 for its meeting assistance. An Android service announced for coming weeks is a separate future step. Model card and license ↗; Developer launch account ↗.

The relevant dispersion mechanism is usable local processing with downloadable model components, reducing the need to send meeting content and searches to a hosted service. The app remains experimental, and having an openly licensed component does not establish independence from hardware or all other software dependencies.

Read the case and sources →

MiniCPM's deployment package makes a larger model a local mobile component

Available storage-streaming local runtime changes the mobile hardware route; related Edge0 mechanism and publisher-only performance are qualified. The October 28 OCR parser and quality-checker release adds a local document-processing route within the existing MiniCPM family. It supplements, rather than independently duplicates, earlier family deployment evidence; layout, hardware and human review remain dependencies.

Read the case and sources →

Window of contestability

Opens provisionally for developers and institutions seeking local deployment; actual end-to-end capability, licenses and hardware needs still matter.

What would change this assessment

Reproducible device and workflow tests, suitable licenses, manageable costs and operation without mandatory outside services would demonstrate stronger independent capability.

Related cases

← All 24 trends