Local AI or cloud? How to deploy AI securely in your company

Companies increasingly use artificial intelligence to work with contracts, client files and internal documents. These documents often contain confidential information, so it is necessary to decide in advance where the AI will run and who will have access to the data. The choice is between running it in the cloud or on your own hardware in the company. In this article, we describe the individual options, their cost and the level of data security.

Most companies that want AI on sensitive documents end up with one of two options: a private API on a B2B cloud such as Azure AI Foundry, or their own hardware with graphics cards when data must not leave the building. The stricter the data requirements, the higher the cost and the effort.

We come across this question often. Companies want to use AI to make their work more efficient, but uploading all their data and trade secrets to ChatGPT is often unthinkable for them. How can this be solved so that all legal and internal data protection requirements are met?

Security, cost

Data stays under your control

Own hardware

DGX Spark, RTX 6000 Pro

€8,000 – €125,000

Rented GPU server

Data centre in Prague

from €10,000 / year

Data leaves your perimeter

Private API

Azure, AWS, Baseten, Zoner Cloud

pay per use

Standard API

OpenAI, Anthropic, Mistral

pay per use

Standard API from OpenAI, Anthropic or Mistral

The first and simplest way to deploy AI is a standard API from one of the well-known providers. The model runs on the provider's servers and you pay per token, roughly $1–20 per million tokens. Deployment costs nothing, and you always have access to the newest models, such as GPT-6 or Claude Opus 5.

The disadvantage is that your data leaves your perimeter. Some providers process data in the EU (data residency) for a surcharge. You can also sign a Data Processing Addendum (DPA), under which the provider deletes the data after each request and does not train on it. For testing and tasks that are not sensitive, this model is more than enough. For sensitive data, the other options described below are more suitable.

Private API - model as a service

Technically, a private API works the same as a standard one; the difference is mainly in the B2B contract you have with the provider. The provider is a company operating in the EU, and the service is aimed at business customers, for whom it is standard that data is not retained. Microsoft Azure, Google Cloud and AWS resell models like GPT-6 and Claude under a B2B contract that guarantees processing in a specific region, for example the EU. If your company already runs on Azure, Azure AI Foundry is the natural choice. You pay per token, as above. You need a company account with the provider and an approved quota.

Czech providers such as České Radiokomunikace or Zoner AI cloud offer the same API with models in Czech data centres. That suits regulated companies whose data must stay in the Czech Republic. The providers also give guarantees for these services. České Radiokomunikace, for example, guarantees data storage in the Czech Republic, Tier III data centres, ISO 27001 certification, GDPR compliance and VPN connectivity. It is also part of the critical infrastructure of the Czech Republic.

Model as a service is therefore a good compromise between security and cost.

Self-hosted open models

If data must not leave your infrastructure even under a contract with a provider, there is no other option than to run the language model yourself. When you run the model yourself, you don't use closed models like GPT-6 or Claude Opus. You run an open model on your own infrastructure: on a physical server in your building, or on a rented server in a data centre. Think of it as a program you install on your own computer with a graphics card.

Open models are usually a few months behind the newest GPT. If GPT-5.1 was good enough for you, your users may not notice the difference. They also improve every few months, so the same hardware gives noticeably better results today than it did half a year ago. The hardware determines how powerful a model you can run.

Renting a GPU server

A rented server with graphics cards keeps the data under your control without you having to buy, install and maintain your own hardware. The server sits in the provider's data centre, but only you have access to it and the provider cannot read your data. The table below shows GPU hosting prices from selected providers in Europe. AWS, Azure and RunPod offer servers in multiple locations, while the last two providers have data centres in the Czech Republic.

The price grows with the number of graphics cards. Today we deploy the NVIDIA AI model Nemotron 3.5 Flash, which requires 1–2 GPUs, and Beam 1.0, which requires 4–5. For the most demanding users, Mistral Large 4 is a good fit, but we don't recommend it because of the cost (see the comparison below). In practice, renting starts at about €10,000 a year.

Buying your own hardware

If data must not leave your building, you have to buy your own hardware. The hardware you need depends on the size of the model and the number of users. For simple tasks and 1–10 users, the compact AI computer NVIDIA DGX Spark is suitable. Agentic tasks need about four RTX 6000 Pro cards. Models at the level of GPT-5.x (Mistral Large 4, approx. 1 trillion parameters) can also run locally, but the hardware for them is very expensive (min. 8× RTX 6000 Pro).

Approximate purchase prices per item:

• NVIDIA DGX Spark: €8,000
• NVIDIA RTX 6000 Pro: €12,000–25,000 per card
• NVIDIA H200: €40,000

Prices may fluctuate due to the market situation. If you choose DGX Spark, we can rent it to you for the first year for a surcharge on the PactusAI licence, so you don't have to buy it right away.

What to expect from local models

The Artificial Analysis Progress Index measures the capabilities of language models on a single scale regardless of release date. This makes it possible to compare open models with older versions of GPT and Claude. Open models that can run locally are in bold in the table.

We can see that on this benchmark, Mistral Large 4 is at the level of GPT-5.4 from March 2026 and Claude Opus 4.7 from April 2026. Beam 1.0 matches GPT-5.2 from December 2025. Nemotron 3.5 Super sits between GPT-5.1 and GPT-5.2, and Nemotron 3 Ultra matches GPT-5 from August 2025.

It should be said that the Progress Index is only one benchmark. There are many benchmarks, and they often give different results. The only reliable way to find out whether a given model is enough for you is to try it on your own task. For completeness, here are the results of other benchmarks from the Intelligence Index v5, which consists of nine tests in English. Below we show three of these tests and the overall index score. The AA-Briefcase test measures knowledge work on projects with thousands of input files (Elo score). The GDP.pdf test measures answers to questions about long PDF documents. The AA-Omniscience test shows how often a model admits it doesn't know an answer instead of making one up.

In AA-Briefcase, Mistral Large 4 sits between GPT-5.5 and GPT-5.6 Sol, level with GPT-6 Luna. Nemotron 3.5 Super, Beam 1.0 and Nemotron 3 Ultra are below GPT-5.5. All four local models make things up less often in AA-Omniscience than Claude Opus 5.5, Claude Sonnet 5.5 and GPT-6.1 Sol.

Note: The score depends on how long the model thinks before answering (reasoning level). In the first table, we show each model at the level with the highest measured score. For GPT and Claude models, this is usually xhigh or max. These levels are slower and more expensive, so they are rarely used in everyday operation. Without reasoning (non-reasoning), Claude Opus 4.7 scores 288, GPT-5.6 Sol 284, GPT-5.5 271, GPT-5.4 244, GPT-5.2 225 and GPT-5.1 206.

Beware of cheap on-premise AI

If someone offers you on-premise AI for less than the hardware it needs, they are not telling the truth. Most likely they are just reselling OpenAI's cloud services under their own name. Hardware prices also change quickly, so treat all figures as estimates.

Let's discuss it with you

The right choice of deployment depends on your data, your users and the work you want to automate. I'm happy to discuss it with you in person and suggest a deployment tailored to you. Write to me at matus@pactus.ai.