Off-the-shelf AI fits the average case. Your business is rarely the average case. A bank scoring credit risk, a publisher building a 20-language summarizer, a marketing agency that cannot send a single document outside its perimeter - each needs a model trained on its own data and bolted into its own workflow. This guide is for leaders deciding whether to buy a SaaS tool, fine-tune a public model, or build custom. It covers what custom AI actually means, where the money and time go, where projects fail, and how to pick a partner without learning the lesson the hard way.
Key takeaways
| Point | What it means in practice |
|---|---|
| Custom is a decision, not a default | If a SQL query or an off-the-shelf API solves it, build neither a model nor a roadmap around one. |
| Data work is the project | In real Silk Data projects, data preparation eats 50-65% of the effort. Modeling is 10-15%. |
| Compliance is an architecture choice | GDPR, the EU AI Act, FERPA - these decide where the model runs (cloud or on-prem) before you write code. |
| Most failures are organizational | No business owner, no SME, no feedback loop. The model gets built, then dies in production. |
| Vendor selection beats vendor pricing | Ask who writes the architecture, who owns the IP, and what they refused to build last year. |
What custom AI development actually means
Custom AI development is the design, training, and deployment of a model around your own data, your workflow, and your compliance perimeter - rather than around a vendor's generic dataset. That sounds clean. In practice, the line between custom and configured is blurry.
There are roughly four options on the spectrum:
- SaaS AI: a generic product you log into. Cheapest, fastest, least control.
- API + prompt engineering: a public LLM wrapped around your prompts. Quick wins, but the vendor sees your traffic.
- Fine-tuned or retrieval-augmented model: public foundation, your data on top, your evaluation set.
- Custom-trained model on your infrastructure: built from scratch or heavily adapted, often on-prem. Most expensive, most control.
A practical rule of thumb for picking the level. If the use case is repeatable across many companies and your data isn't unusual, the SaaS or API level usually wins on total cost. If your data is distinctive enough that generic models miss 10-20% of cases that matter to revenue, fine-tuning or RAG starts paying back. Full custom training is rarely the right answer unless you also have a regulatory or IP constraint that forces on-prem - and even then, the question is whether to fine-tune a smaller open model on your infrastructure rather than train from scratch. Building from scratch is the most expensive answer and the rarest correct one.
The honest answer for most teams sits in the middle two. We built a click-through prediction model for an advertising client using CatBoost on tens of millions of records. Custom training paid off because CTR sat at 0.1-0.5%. No off-the-shelf API would handle that class imbalance. For a marketing agency that could not let client briefs leave its network, the right answer was a local LLM inside their platform. Not a from-scratch model. Different problems, different points on the spectrum.
The pros of going custom: you own the model, you control the data, you can retrain it as your business shifts, and you can deploy on-prem when a regulator or a client demands it. The cons: higher upfront cost, longer time to first value, and a maintenance bill that does not stop. Anyone who tells you only the first list is selling, not advising. For a deeper look at the build path, our team has written about AI and ML development services in more detail.
Where the money and time actually go
The short version: data preparation is the project. Modeling is a fraction of it. Most vendors do not show you this breakdown because it is unflattering.
Across roughly 200 projects, our internal effort distribution looks like this:
| Phase | Share of effort | What happens here |
|---|---|---|
| Metric definition | ~10% | Agree what "good" means in numbers. Most projects skip this and regret it. |
| Data preparation | 50-65% | Collect, clean, label, fix. This is where projects die or come alive. |
| Modeling | 10-15% | Train, evaluate, tune. Often less work than the demo would suggest. |
| Deployment and integration | 10-15% | APIs, monitoring, fallback paths, rollback plans. |
| Monitoring | Continuous | Models drift. Data shifts. Budget for this from day one. |
One line item in this table is non-negotiable: continuous monitoring. Every other phase has a defensible "lite" version - a quick metric definition, a minimal data audit, a wrapper-style deployment. Monitoring is the one place where the "lite" version is functionally zero. A model without drift detection does not fail visibly; it fails quietly, often for months. That is the most expensive failure mode in the entire lifecycle, because by the time it surfaces, business decisions made on the model's output are already in motion.
On timeline, a focused proof of concept takes around three months from scoping to a go/no-go decision. An MVP that actually runs in production usually adds another three to six months. A full build, with integrations and a monitoring pipeline, tends to land in the 6-12 month range. Anyone promising a production-grade custom model in four weeks is either selling a wrapper around a public API or skipping the parts that matter.
Yuliya Marazenko, who leads our AI implementation work, puts it bluntly: data quality matters more than the algorithm. On an animal-survival prediction project for a farming client, we found a record listing a single animal at several dozen tons. No model on earth would have produced a useful answer until that row was found and fixed. That is what 50-65% of the budget buys you - the work that makes the rest possible. For teams scoping a first project, a structured AI proof of concept usually exposes these realities cheaply.
Compliance is an architecture decision, not a checkbox
The short answer: in regulated industries, your compliance regime decides where the model runs and what data ever leaves your perimeter. That decision sits at the architecture stage, not at launch.
Three frameworks matter most for European and US business audiences, and they apply to different things:
- GDPR applies whenever personal data is involved in EU contexts. It shapes what you can store, where, and for how long. Official GDPR text is the source of record.
- The EU AI Act classifies AI systems by risk tier and adds obligations for high-risk uses such as credit scoring or hiring.
- NIST AI RMF is voluntary US guidance for managing AI risk, useful as a structure even outside the US.
- FERPA sits separately, for US education data only - relevant for EdTech, not for a German bank.
A timing note that matters for current planning. The Digital Omnibus on AI agreement from May 2026 deferred Annex III high-risk obligations from August 2026 to December 2027, with embedded high-risk products extended to August 2028. This widens the planning window - but it does not change the architectural decision. If your use case will be classified as high-risk, the documentation, data governance, and human oversight obligations still need to be designed in from the start. Treating the extension as a reason to defer the architectural work is exactly how organizations end up retrofitting compliance later at multiple times the cost.
These are legal frameworks, not technical ones. The technical translation is usually one of three things: keep the data inside your network, redact before it leaves, or pick a provider with the right contractual posture. For a German publishing client, we have run a local LLM over their archives for years. The legal team needed certainty that nothing left their infrastructure, and the architecture had to deliver that on day one. Retrofitting that later would have meant rebuilding the system.
Polina Volodina, who advises clients on AI strategy, frames the trade-off well: cloud APIs are faster to launch and cheaper to start. On-prem deployments cost more upfront but remove an entire category of legal and commercial risk. Neither is universally right. The question is which risks your business can carry. Our team's AI consulting work usually starts with exactly this conversation.
Custom vs off-the-shelf: when each one wins
Off-the-shelf wins when your problem looks like everyone else's. Custom wins when accuracy, control, or compliance carry real money.
| Dimension | Off-the-shelf SaaS | Custom or heavily adapted |
|---|---|---|
| Time to first value | Days to weeks | Months |
| Upfront cost | Low | Higher |
| Long-term cost | Recurring SaaS fees, often rising | Predictable, owned |
| Accuracy on your data | Generic baseline | Tuned to your domain |
| Data residency control | Vendor-dependent | Yours, including on-prem |
| IP ownership | Vendor | You |
| Vendor lock-in | High | Low |
A useful filter: estimate the cost of a wrong answer. For an internal note-taker, a wrong summary costs a minute. For a fraud screen, it costs a customer or a fine. The higher that number, the more custom pays back. A second filter: how unusual is your data? We built a semantic search over 6 million images on ElasticSearch with our own vector layer. That was never going to come from a SaaS. The scale and the asset type ruled it out. A sales team that wants meeting transcripts summarized should probably just buy a tool.
One honest counterpoint to the custom case: you also inherit the maintenance. A model is not a deliverable, it is a living asset. If your business does not want to fund monitoring and retraining for years, off-the-shelf is the safer bet even if accuracy is lower.
Why custom AI projects fail, and what actually fixes them
Most failures are not technical. They are organizational. After 200+ projects, the patterns repeat enough to predict.
The common failure modes:
- No business owner. A model without an owner on the business side becomes an orphan within a quarter. Nobody decides when to retrain it or what to do when it misbehaves.
- No SME access. If the underwriters, doctors, or editors who know the domain cannot give the team an hour a week, the model will learn from labels that look right and are wrong.
- Wrong success metric. Accuracy looks great on a balanced test set and falls apart on a 0.3% positive class. Pick the metric before you train.
- Modeling before data work. Teams jump to training because it is more interesting than cleaning. The clean-up has to happen anyway, and it is more expensive after.
- Vendor chosen on price. A low bid usually means the vendor underestimated the data work, the compliance work, or both. You will pay the difference later.
The fix is not exotic. Name a business owner before kickoff. Block SME time in the contract. Define the metric in week one. Spend the data budget before the modeling budget. Yuri Svirid, our CEO, has a rule from 30 years in EdTech and FinTech: if SQL solves the problem, say so and do not sell a model on top. The opposite rule applies too. If a model is the right answer, do not pretend a dashboard will do.
One under-discussed risk: model drift. A churn model trained on 2023 behavior degrades quietly through 2024. You need a monitoring pipeline that flags drift and a retraining schedule that does not depend on someone remembering. That work is unglamorous and it is what separates a project from a product. For teams thinking past the first launch, our work on DevOps and MLOps covers the pipeline side.
How to choose a partner without learning the hard way
The short answer: vet the engineer who will own your architecture, not the logo on the proposal.
A short list of questions that separate serious vendors from polished ones:
- Who writes the architecture, and can I meet them before signing? If the answer is a sales engineer you will never see again, that is the project.
- Walk me through a project that went wrong. Anyone with real scars will have a clean answer. Anyone without one has not shipped enough.
- What did you refuse to build last year, and why? A partner who has never said no is selling, not advising.
- Who owns the model, the weights, and the training data after handover? Get this in the contract, not in the pitch deck.
- How do you handle data preparation cost? If the answer is a flat 10%, they are guessing or hiding it.
- What is the post-launch support model? Monitoring, retraining cadence, emergency rollback, on-call - all should have names attached.
What to do with the answers. Score each vendor on these questions before comparing price. A vendor that can name a project that went wrong, can point to architectures they refused, and can put their lead engineer on a call before signing is operating in a different category than one with polished slides and a sales engineer. The price gap between those categories is usually 30-50%. The cost gap, measured over 18 months including the cost of failed deployments and unplanned rework, usually closes the difference or reverses it.
One more practical point. Long partnerships beat short ones in AI work because the model is never finished. Our team has worked with a German publisher (Reemers) for seven years, across three of their CTOs. The institutional memory of why a model was built a certain way is worth more than any documentation. That kind of continuity is hard to fake and worth asking about. The case-study hub is a reasonable place to see what we mean by long-running work.
About Silk Data
If you are weighing build versus buy, the cheapest hour you can spend is on scoping with someone who has shipped the same shape of problem before. Our team has built on-prem LLMs for clients who cannot let data leave the network, semantic search over millions of assets, contract analysis from PDFs, and resume screening pipelines for hiring at scale. We have also told clients to use SQL and skip the model. Both answers are part of the job.
If a structured starting point would help, our team runs short scoping engagements that end in a go/no-go - not a sales deck. Write to hello@silkdata.ai with the problem, the data you have, and the constraint that scares you most.
