senn-techEU AI Act: Why Local AI Is the Right Answer
The EU AI Act is the world’s first comprehensive AI law. It classifies AI applications by risk — and has concrete consequences for where and how we run models. Local inference on your own hardware thereby shifts from a nice-to-have to a compliance advantage.
The Risk Pyramid
The AI Act distinguishes four tiers:
- Unacceptable Risk: Prohibited. Social scoring, real-time facial recognition in public spaces, manipulative AI.
- High Risk: Permitted, but strictly regulated. AI in critical infrastructure, hiring, credit scoring, medical diagnostics.
- Limited Risk: Transparency obligations. Chatbots must disclose they are bots; AI-generated content must be labelled.
- Minimal Risk: No special requirements. Spam filters, video-game AI.
Most business applications fall into the upper three tiers. Obligations increase with risk — and with them the question: Who has access to the data flowing through the model?
Why Local Inference Simplifies Compliance
Every SaaS AI — whether ChatGPT, Claude API, or Google Gemini — sends your data to external servers. That poses a problem for high-risk applications:
- Data Transfer: The AI Act requires documentation of which data flows where. An on-prem vLLM server on your own hardware is crystal clear here: the data never leaves the premises.
- Auditability: Whoever operates a local model can log every request, trace every embedding, archive every answer version. With an API you are dependent on the provider’s log interface.
- Model Pinning: High-risk applications must run a defined model version. An API provider can change the backend model at any time — you might not even notice.
Transparency Obligations: Trivial with a Local Model
Disclosing that a chatbot is a bot, or that an AI-generated image is not a photo, are not technical hurdles — but they must be documented and provable. A self-operated system makes this proof straightforward.
Interaction with NIS2
NIS2 and the AI Act interlock: NIS2 governs infrastructure cybersecurity, the AI Act governs risk assessment of the AI application. Both reward control. Both punish blind outsourcing. Local AI infrastructure — your own server, your own model, your own pipeline — satisfies both frameworks from a single source.
What Local AI Does Not Solve
The AI Act demands risk management, documentation, human oversight. That must be done — whether local or cloud. Local hardware eases the technical compliance, but the organisational burden remains yours.
Conclusion
The AI Act is not a ban, but a framework. Whoever runs AI on their own hardware holds the better cards: data sovereignty, model control, and auditability are the three pillars on which AI compliance stands.
Does the AI Act make local AI necessary in the first place?+
It does not force it, but it makes local inference strategically cleaner. Every SaaS AI sends data to external servers — for high-risk applications the AI Act demands seamless documentation of which data flows where. An on-prem vLLM server on your own hardware answers that question crystal clear: the data never leaves the premises. If you need auditability and model pinning, on-premise is simply the better fit.
Which risk tier does a typical SME application fall into?+
Most business applications land in limited or minimal risk — spam filters or game AI carry no obligations, chatbots only have to disclose themselves. High risk applies to credit scoring, hiring, medical diagnostics, or critical infrastructure. Obligations rise with risk, which is why the question of data access matters more the more sensitive the use case.
Does local AI satisfy the AI Act requirements on its own?+
No. Technical compliance — data sovereignty, auditability, model pinning — becomes markedly easier on your own hardware. The organisational duties like risk management, documentation, and human oversight remain your job in every variant, whether local or cloud. Local hardware is a clear advantage, not a free pass for the rest of the obligations.