Open-weight models deployed on your servers or in your own private-cloud account, with no prompts or documents sent to an AI vendor. I select the model against your tasks, fine-tune only when measurement shows the base model falls short, and evaluate the result on your documents and questions before it goes live. This is for small and mid-sized US companies that need the capability without the data leaving their control.
No obligation. Replies typically within one business day.
Sound familiar?
When a third-party AI API is the wrong place for your documents
Your data can’t leave
Contracts, health records, financials and client files often rule out third-party AI APIs under your policies, contracts or risk appetite.
Per-token bills grow with adoption
High-volume internal workloads on metered APIs produce a bill that rises every time another team starts using the tool. The more useful it becomes, the more it costs.
Vendor dependency is a risk
Models get deprecated, prices change, terms shift. A workflow your business depends on shouldn’t sit on someone else’s roadmap.
Leaderboards don’t answer it
Leaderboards and model cards rank models on public tests. They don’t say which model handles your documents, your formats and your questions.
How I approach it
Selection, fine-tuning and hardening, measured on your tasks
01
Size the model to the job
Candidates are evaluated on your tasks, not leaderboard rank. As a ballpark, many document workflows run on one 24 GB workstation GPU with an 8B to 14B model. The real sizing is settled during discovery, once I’ve seen your volume and your documents.
02
Fine-tune only when measurement says to
Fine-tuning changes format, tone, domain vocabulary and task behavior. It does not add facts reliably; that is retrieval’s job. So retrieval comes first, and LoRA or QLoRA fine-tuning is applied when an evaluation shows the base model still falls short.
03
Harden it for production
Containerized deployment, access control, logging, monitoring and a documented update path. A deployment that only works while I’m watching it is a demo, and I don’t hand those over.
04
Document the economics and the licenses
You see the hardware, hosting and maintenance picture up front, including when a managed API is the better fit at your volume. Licensing is part of the same review: several Qwen models are released under Apache-2.0; the other Qwen models, and Llama and Nemotron, carry vendor licenses that I review during selection.
What you get
What a private deployment includes
A production deployment on your infrastructure or in your private-cloud account.
Model selection backed by task-specific evaluation results on your documents and questions.
Fine-tuning, when warranted, with before/after measurements on the same test set.
A license review for each candidate model, so you know what the weights permit before anything ships.
An operational runbook: monitoring, updates and maintenance procedures your team can own.
A full engagement rehearsal on a synthetic benefits-administration corpus: an assessment measured the baseline and ruled out the expensive fixes with data, then structured retrieval for tabular documents and a 14B open-weight model on the same single GPU took answer accuracy from 82.2% to 95.0% on the fixed 180-question test (peak build), with a deterministic auto-accept path that answered 13 of 43 tabular lookups with zero errors.
peak-build answer accuracy, same GPU (171/180, synthetic replica)MeasuredMeasured in a controlled test
82% → 95%
tabular lookups auto-accepted with zero errors (deterministic path)MeasuredMeasured in a controlled test
13/43
paired gain on the same 180 questions (27 fixed, 4 regressed)MeasuredMeasured in a controlled test
Fit check: text-based documentation works best. Scanned PDFs, forms, invoices and receipts can be handled with an AI extraction step (see the OCR benchmark). Knowledge that lives in diagrams, video or people’s heads is harder, and I’ll say so during discovery. Self-hosting has infrastructure costs that depend on volume. During discovery I’ll walk through whether your own hardware, a private-cloud account or a managed option is the better fit, including when the answer is no new hardware at all.
How an engagement runs
01
Free consultation and discovery. You describe the workflow; I tell you whether AI is the right tool, and say so if it isn’t.
02
Scoped assessment or fixed written proposal. Scope, price and success criteria are agreed before any build starts. See what the scoped assessment covers.
03
Pilot on one workflow, typically 8–10 weeks, measured against a baseline before anything goes hands-off.
04
Phased rollout, with a stop option after any milestone.
What I’ll need from you
The source documents the system should answer from, in whatever format they exist today.
50 to 300 real questions from your team, with the answers you consider correct.
A subject-matter reviewer who can judge answers during the pilot.
An IT contact for access, deployment and security questions.
For self-hosting, a GPU server or a cloud account; sizing is settled during discovery.
Questions
Questions about self-hosting a model
Which models do you deploy?
The shortlist is usually drawn from Qwen, Llama and Nemotron, all open-weight. Which one ships depends on how each handles your documents and questions in evaluation, and on what its license permits for your use. A model does not get picked for topping a leaderboard.
What hardware do we need?
Often a single workstation GPU; the ballpark is in the approach section above. What changes the sizing is the number of concurrent users, the length of the documents and the latency target, so it is settled during discovery after I have seen your workload. If you already have a server with a suitable GPU, it is measured before anything new is bought.
Do we need fine-tuning?
Usually not first: fine-tuning changes how a model writes (format, tone, vocabulary and task behavior), while retrieval is what supplies the facts from your documents. The starting point is retrieval with a base model, and fine-tuning is added only when the evaluation shows a gap that retrieval does not close.
What happens when a model is deprecated or a better one comes out?
The model is built in as a replaceable part. The evaluation set from selection stays with you, so a candidate replacement is measured against the same questions before it goes live. The runbook documents the update path, and the weights you deployed remain yours to run for as long as their license allows.
Does a private-cloud account count as our infrastructure?
Yes, with the boundary stated plainly. The model runs in an account you own and administer, your prompts and documents stay inside it, and no AI vendor sees them. Your cloud provider still hosts the hardware under your existing agreement with them. If that boundary is not acceptable under your policies, the on-premises option exists for exactly that case.
Can it run fully offline?
Yes. Once the weights and dependencies are installed, inference needs no internet connection. Updates and new model versions can then be brought in on your schedule, through whatever transfer process your security team already uses for software.
Insured before any engagement starts: professional liability (E&O), cyber and general liability.
You own the deliverables built for your environment.
Stop after any milestone; no lock-in.
Replies typically within one business day.
Find out what this would look like for your team
The consultation and initial discovery are free. You get a preliminary recommendation whether or not we work together.