Private AI for business.

Short answer: private AI means running AI on hardware you control, in your office or your private cloud, instead of typing your business into someone else’s chatbot. The tools have gotten good enough that a small business can now run capable models on its own equipment. It is not the right answer for everyone, and the honest cases for and against are below.

Your team is already using AI

Whether or not you have a policy about it, somebody in your office has pasted a client email, a contract, or a spreadsheet into a free chatbot to save twenty minutes. It happens in most offices we look at, and it is rarely malicious. It is a person trying to get home on time.

The problem is not the AI. It is the route. A free consumer chatbot sits outside your systems, outside your backups, and outside every promise you have made to clients about their data. If you hold personal information about Massachusetts residents, that route also sits uneasily next to your WISP obligations: data leaving your control with no vendor agreement and no oversight.

What private AI actually is

The same kind of AI, running on a machine you own. Open models have become genuinely capable, and a single well-specced server can now run assistants that summarize documents, draft replies, and answer questions about your own files.

Nothing leaves the building. No subscription per seat. No vendor training on your data. The model reads your files where they live, on hardware with your name on it. If you already keep files on your own server, the way our private cloud clients do, AI that runs next to those files is the natural next step. It is the same idea: your data, your hardware, your rules.

What it runs on

Most of the mystery around private AI is really a hardware question, and the answer is less exotic than people expect. It is a server with a graphics card in it.

The graphics card is the part that matters. AI models get loaded into the card’s own memory, called VRAM, and that number decides which models you can run at all. A card with 24GB of VRAM handles the mid-sized models most businesses end up using. Less than that and you are limited to smaller models that are noticeably worse at long documents. Considerably more, and you are into hardware priced for research labs rather than a twelve-person office.

The rest is ordinary. A machine with enough RAM and fast storage, sitting in the same rack as the file server, on the same UPS, in the same backup routine. If you already run a server for files, this is a second box next to it, not a new category of thing to look after.

It can also live in a private cloud rather than your office. Same principle, same isolation, someone else’s rack. That suits businesses without a decent place to put a server, and it is how we handle it for clients who would rather not have hardware on site at all.

Which models a small business can run

The models worth knowing are the open-weight ones: Meta’s Llama family, Mistral, Alibaba’s Qwen, Google’s Gemma, and DeepSeek. Open-weight means the model itself is published and you can download and run it. You are not renting access through an API and you are not sending anything back to the company that made it.

They are not identical, and the differences are practical rather than philosophical. Some are better at code. Some are better at long documents. Some run acceptably on a smaller card. Choosing between them is a half-hour conversation once we know what you want it to do, and it is reversible: swapping the model is a configuration change, not a rebuild.

The thing worth understanding is that this field moves. A model released this year will be beaten next year, and because you own the hardware you can simply run the newer one when it arrives. That is the opposite of the usual software trap, where the vendor decides when you upgrade and what it costs.

The honest comparison

Cloud AI is stronger. The frontier models from the big providers are more capable than anything you can run on a small business budget, and if you need the smartest possible model, that is the cloud, full stop. We wrote separately about what Copilot is worth inside Microsoft 365.

Private AI wins somewhere else. It wins on the sensitive pile: client records, financials, HR files, anything you would not paste into a public tool. It wins on cost shape: hardware is bought once instead of rented per seat forever. And it wins on stability: the model you run today is the model you run next year. Nobody deprecates it, nobody changes the terms, nobody raises the price mid-contract.

Most businesses that go this way land on both. Cloud AI for general work. Private AI for the pile that should never leave the building.

What people use it for

The uses that survive contact with a real office are unglamorous:

• Finding things in your own files. Searching by meaning instead of filename. “The change order where the client agreed to the extra bathroom” finds the document even though none of those words are in the file name.
• Summarizing long documents. Contracts, specifications, inspection reports, insurance policies. Not to replace reading them, but to know what you are about to read.
• First drafts. Proposals, scopes of work, letters that follow a pattern you have written a hundred times.
• Answering questions from your own records. New staff asking what the standard warranty terms are, without interrupting someone who knows.

For the construction and trades clients we work with, the file search is usually the one that earns its keep. Fifteen years of job files nobody can navigate turns into something you can ask questions of. For accounting firms it tends to be the summarizing, applied to the documents clients send in; we wrote that case up on its own, because the rules for a CPA firm are stricter than for most businesses.

When we tell people not to do it

We talk people out of this more often than we sell it. It is a bad fit when:

• The sensitive pile is small. If almost nothing you handle is confidential, a licensed business version of a mainstream tool with proper data protections is cheaper and better. Buy that instead.
• Nobody will own it. Private AI is a system. It needs someone accountable for it the way your server needs someone accountable for it. If that person does not exist and you are not hiring us to be them, it will quietly rot.
• Your files are a mess. AI answers from your data. If your documents are scattered across personal drives and desktops, the answers will reflect that. Fix the filing first. That work is worth doing whether or not any AI ever touches it.
• You want it to replace a person. It does not. It removes the tedious half of a job. The businesses that get value out of it are the ones that expected exactly that.

Private AI and Massachusetts data rules

If you hold personal information about Massachusetts residents, 201 CMR 17 requires you to have a written information security program, and part of that is overseeing the third parties you hand data to. A free consumer chatbot is a third party nobody has assessed, with no agreement in place, and usually no record that it happened at all.

That is a reason to have a policy rather than a reason to panic, because the alternative is an obligation you cannot show you are meeting. Private AI answers the question neatly: the data never went to a third party, so there is nothing to oversee. A written policy plus a licensed business tool answers it too, in a different way, and for plenty of businesses that is the sensible choice.

We are IT people, not lawyers, and this is not legal advice. But we have written enough WISPs to know that “where does the AI send our data” is a question that now belongs in one.

What a sensible first step looks like

Not a project. An hour of conversation about what your team does with AI today and what data is involved. Out of that comes a short written policy: what may go to public tools, what may not, and what the business provides instead.

If the sensitive pile is big enough, the next step is a pilot: one server, one use case, usually document search or summarizing, measured for a month. We build these on the same dedicated hardware as our private cloud work, so the pieces are already familiar.