Local vs Hosted AI: A Seven-Gate Decision Framework

Choose local, hosted or hybrid AI by assessing a defined business workflow against seven questions: boundary value, task capability, operating ownership, full cost, availability, ability to change direction and safe failure.

Local deployment can provide more direct control over models and data flows. It also transfers infrastructure, security, support and lifecycle duties to the organisation. The right answer depends on whether that exchange creates value for the work.

Define the deployment choices

Local AI

The model runs on infrastructure controlled by the organisation or user. This may be a workstation, internal server, private data centre or isolated cloud environment.

Hosted AI

The organisation sends requests to a provider-operated service under the provider's technical and contractual terms.

Hybrid AI

The workflow uses both. It may process sensitive material locally, route selected tasks to a hosted model or use different routes for different case types.

These labels are not enough for a risk decision. The architecture must show the path taken by prompts, source documents, intermediate data, outputs, logs and tool calls.

Gate 1: Does the local boundary create material value?

Identify the data or dependency that changes when inference moves local.

Useful evidence may include:

• A contractual restriction on external processing
• A customer requirement for a defined hosting boundary
• Data that cannot be sent to the proposed hosted service
• Work that must continue without external connectivity
• A material continuity risk from one provider
• A retention or logging requirement that is easier to control internally

Map every system connection before claiming that data remains local. Storage, retrieval, telemetry, support tools, plug-ins, backups and logs may cross a different boundary from the model.

The NCSC recommends transparency about where AI-system data may be used, accessed or stored, along with controls for models, APIs, data and processing infrastructure.

Gate 2: Does the tested configuration meet the requirement?

Test the deployable configuration rather than a model family in the abstract.

Record the version, quantisation, runtime, hardware, context and reasoning setting. Use representative business cases and compare the final approved output with the current process and a suitable hosted option.

Measure:

• Output quality by case type
• High-consequence errors
• Refusal and escalation
• Human review and correction
• End-to-end time
• Stability across repeated runs
• Behaviour under missing or conflicting inputs

A local model may be suitable for extraction and drafting but not final commercial decisions. Bounded suitability is still a useful result.

Gate 3: Can the organisation operate the service?

Name the operational owner and the work required across the service life:

• Hardware and capacity
• Runtime and operating-system patches
• Model acquisition and provenance
• Access administration
• Monitoring and incident response
• Change testing
• User support
• Backup, recovery and decommissioning

Open weights give the organisation access to the model. They do not provide an operating team.

The service design should not depend indefinitely on the person who built the experiment.

Gate 4: What is the full cost at realistic volume?

Compare costs over a defined period and expected workload.

For local AI, include:

• Hardware purchase or rental
• Spare capacity and resilience
• Electricity and hosting
• Engineering and support time
• Security, monitoring and backup
• Replacement and upgrade
• Human review

For hosted AI, include:

• Usage and platform charges
• Integration and support
• Network and data-transfer needs
• Contracted service level
• Human review
• Expected price or model changes where they can be assessed

The comparison should use the same quality and availability requirement for both routes.

Gate 5: Can it meet the availability requirement?

Define:

• Required service hours
• Acceptable downtime
• Recovery time
• Peak concurrent use
• Manual fallback
• Data and configuration backup
• Behaviour during a partial system failure

A workstation can be an appropriate home for a personal assistant or a controlled experiment. A team-wide operational process may require shared hardware, monitoring and redundancy.

Local AI can reduce dependence on an internet connection while increasing dependence on internal equipment and skills.

Gate 6: Can the organisation change direction?

Preserve the parts of the system that should outlive the model:

• Business task definition
• Evaluation cases and expected results
• Source and tool interfaces
• Approval and exception rules
• Monitoring and outcome measures
• Audit and correction record

Avoid embedding one model's prompt format or behaviour throughout the business process where a stable interface can isolate it.

Open-weight models can reduce dependence on a hosted provider, but local systems still depend on hardware, runtimes, licences, libraries and maintainers. Record those dependencies and the route for replacing them.

Gate 7: Can the system fail safely?

Define the effect of a wrong or unavailable answer.

For every AI-supported decision, record:

• Consequence of error
• How the error can be detected
• Whether the action can be reversed
• Required human authority
• Cases that must be excluded or referred
• Incident and shutdown process

NIST's AI Risk Management Framework recommends testing under conditions similar to deployment, defining human oversight and managing risks from internal and third-party components. A local model changes which components are internal; it does not remove the need for those controls.

Data protection still applies to open models

The ICO states that data protection law does not prevent an organisation from using an open-source AI model merely because the training process is not fully known. The organisation still needs to satisfy itself that its use complies with data protection principles, including accuracy and fairness.

That means the deployment decision should address purpose, lawful use, data minimisation, accuracy, people affected, retention and rights where personal data is involved. Local hosting may help with a particular transfer or access concern, but it is not evidence that all of those questions have been answered.

When each route tends to fit

Local tends to fit when

• A defined data or connectivity boundary matters
• The local configuration meets the task threshold
• Usage supports the infrastructure cost
• Version control or offline operation has value
• The organisation can own the service safely

Hosted tends to fit when

• The work needs stronger or rapidly changing capability
• Demand is low or variable
• Provider controls and terms meet the requirement
• Internal operation would add disproportionate cost or risk
• Managed availability and support matter

Hybrid tends to fit when

• Different stages have different data and capability needs
• Most cases fit locally but a controlled subset needs a stronger hosted model
• Local processing can remove sensitive details before an external request
• A fallback route is required

Hybrid routing needs its own controls. The system must not send a difficult case externally if difficulty is caused by the sensitive information that the boundary was designed to protect.

Use a one-page decision record

Record the following before approval:

• Work and accountable owner
• Intended outcome and success threshold
• Data boundary and its business reason
• Compared configurations
• Evaluation evidence and serious failures
• Human review and approval
• Full cost and expected volume
• Availability and recovery
• Security and retention controls
• Exit and review conditions

Set a review date. Model capability, hardware, prices and provider terms change too quickly for this to be treated as a permanent infrastructure decision.

Local AI now deserves to be considered for real work. It earns deployment only where the complete system meets the requirement and the additional responsibility buys something the organisation can name.