> ## Content Index
> Fetch the complete content index at: https://marginsofcare.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# AI Agents On Premise vs In The Cloud, Part 2
- URL: https://marginsofcare.com/ai-agents-on-premise-vs-in-the-cloud-part-2/
- Published: 2026-09-23T19:08:59.000Z
- Updated: 2026-09-28T11:29:25.000Z
- Author: Eva Worst

You can read [Part 1 here](https://marginsofcare.com/ai-agents-on-premise-vs-in-the-cloud-part-1/).

When considering the cases in which AI could run on-premise, we need to understand the clinical workflow from the perspective of computing intensity, speed requirements and collaboration needs.

Here is an easy example: An MRI or CT scan requires a massive computing power purely because of the size of the files that are created. The analysis taking 2 seconds longer is not an issue.

Brain surgery on the other hand, for which you again use MRI imaging during the operation, requires not just massive computing power, but also sub-second latency, or else a delay might cause for the surgeon to cut too much brain out. Oops! You also don’t want to deal with internet issues in that very moment.

And finally, things like patient intake, clinical decision support and billing all require back-and-forth with multiple people, organisations and systems, some of which sit in the cloud.

The take away is simple: For tasks that require a lot of computing, such as diagnostics, the AI can be run both in the cloud and on-premise. The risk with running it in the cloud is you might quickly use up all the credits, due to the sheer amount of data that is moved, which will impact your budget.

For tasks that require close to zero latency and maximum reliability, on-prem is the way to go. At least until we all get space internet.

And for tasks that call for a collaborative workflow, maybe you can do part of the steps on-premise, but if your aim is to have end-to-end automation, you have to go to the cloud.

[Yesterday](https://marginsofcare.com/ai-agents-on-premise-vs-in-the-cloud-part-1/) I made the point that beyond a couple of big hospitals, few others can afford to run their own GPU clusters. But there are a couple more reasons why running AI models locally, especially if the models are “home-made”, might backfire.

For starters, we can look at an AI model as a product, which it very much is. The obvious first steps when you want to build a medical software product is, well, to build it, test it, get it certified (often forgotten in such discussions) and then deploy it. But the work doesn’t stop there. Products need updates (and re-certifications), customer support and continuous maintenance as the IT systems evolve and change. There are entire companies and industries who do just that, what makes us think a coupe of researchers and the hospital IT can do it just as good?

And then there is the problem of bias: a local model, fine-tuned to the local data, population and clinical logic, is not an objective tool anymore. It may be faster than individual clinicians but is not more objective than them, which means it’s only half as good as it should be.

In a nutshell, it’s not a coincidence that the best AI applications run in the cloud and while there certainly are individual use cases where on-premise makes more sense, for mosts tasks and for most organisations there is no going around the cloud.

We might, however, want to rethink what the definition of on-premise is. As chips and batteries are getting more and more powerful and AI models more efficient, computing on the edge using smartphones and tablets, for example, will become a viable option. Apple here is the leader, but Chinese companies are not lagging behind. And once again, just like with the cloud, the question will be: where should we store our data? 

UPDATE: Read [Part 3 here](https://marginsofcare.com/ai-agents-on-premise-vs-in-the-cloud-part-3/).