AI applications measured before they are trusted

An AI development company builds software that uses large language models (LLMs) to do a specific job: answer questions from your own documents, pull data out of invoices and forms, draft replies, or sort and route incoming work. What decides whether it succeeds is rarely the prompt; it is whether anyone measured accuracy on real examples before launch and set a limit on what each request may cost. Meritbyte Technologies, a Nepal-based web and software development company, builds these applications with measurement first, and will tell you when a plain rule or a form would do the job better.

Which AI projects are worth building?

AI earns its place where the input is messy language or documents, the volume is high, and a wrong answer can be caught before it does harm.

Good fits:

• Answering staff or customer questions from policies, manuals and product documentation
• Extracting fields from invoices, purchase orders, applications and contracts into your systems
• Classifying and routing emails, tickets and form submissions
• Drafting first versions of replies, summaries or reports that a person approves
• Searching across years of documents by meaning rather than exact words

Poor fits, where we will advise against it:

• Calculations, pricing and eligibility rules that can be written as plain code
• Low-volume tasks where setup costs more than the time saved
• Decisions with legal or safety consequences and no human review
• Problems that are really missing data or a broken process

What we build

Assistants over company knowledge. Chat interfaces for staff or customers that answer from your documents using retrieval-augmented generation (RAG): the system finds the relevant passages first, then the model answers from them and cites its sources. RAG explained for business leaders covers how it works and how it compares with fine-tuning.

Website chatbots. Customer-facing assistants that answer pre-sales and support questions and hand over to a person cleanly. Our guide to an AI chatbot for a business website lists what works and what annoys customers.

Document extraction. Structured data from PDFs, scans and emails, validated against rules (totals add up, dates are real, supplier exists) before anything is written to your accounting or ERP system.

AI automation. Workflows that classify, draft and route, with a person approving anything that leaves the building.

Why the evaluation harness comes before the prompt

An evaluation harness is a set of real inputs with known correct outputs, plus code that scores the system against them automatically. We build it in the first milestone, before tuning anything.

For a document assistant, that means a few hundred real questions from your staff with the answer and source document for each. For extraction, it means a sample of real documents with the correct fields filled in by hand. The harness then reports accuracy, how often answers are grounded in the right source, how often the system correctly says "I don't know", and cost and response time per request.

Every prompt change, model upgrade or new document batch is then re-scored, so improvements are proven and regressions caught before users see them. When a provider retires a model version, as they do on published schedules, switching becomes a measured decision rather than a gamble.

How an AI development project runs

1. Scoping call. Free. We look at the task, the data and the cost of a wrong answer, and say plainly if AI is not the right tool.
2. Fixed-price first milestone. The evaluation set, a working prototype on a staging URL, and a scorecard. This is where you learn whether the idea is viable, for a known price.
3. Two-week blocks. Integration with your systems, user interface, permissions and guardrails, with a demo and updated scores at the end of each block and a written update every week.
4. Launch. A limited rollout with logging, human review queues and feedback buttons, so real usage feeds back into the evaluation set.
5. Run. Monitoring accuracy, cost and drift, updating documents, and handling model changes on a retainer.

Model, data and security choices

We are not tied to one model provider. Depending on your cloud and data rules, models run through Amazon Bedrock, Azure OpenAI, Google Vertex AI or a provider's own API, and open-weight models can be self-hosted when data cannot leave your infrastructure. Retrieval usually runs on PostgreSQL with pgvector, which avoids adding a separate database. Pipelines are written in Python; user-facing applications in TypeScript.

Data handling is documented before build: what is sent to which model, in which region, and how long it is retained. The major providers' business and cloud terms currently say API inputs are not used for training by default, but retention and monitoring rules differ, so we check them against your obligations under laws such as UK GDPR or Australia's Privacy Act 1988. API keys and cloud accounts are in your company's name. Applications are tested against the risks in OWASP's Top 10 for LLM Applications, prompt injection first.

What does AI development cost to build and run?

Build cost depends on the systems the application connects to, how clean the source documents are, permission rules (who may see which answer), and how much accuracy the use case demands. The general pricing models are covered in our custom software development cost guide.

Running cost is paid per request, in tokens, and it can surprise people. We set a cost ceiling per request in the scope and design to it:

• Route simple requests to smaller, cheaper models; reserve large models for hard ones.
• Retrieve a few precise passages rather than stuffing documents into every prompt.
• Use prompt caching and response caching where providers support them.
• Send non-urgent work, such as overnight document batches, through discounted batch processing.
• Cap usage per user and per day, and alert on spend.
• Report cost per resolved question or processed document, not just per token.

Questions to ask an AI development company

• Can you show accuracy scores on our own examples before we commit to the full build?
• What does the system do when it does not know the answer?
• Where is our data sent, and under which provider terms?
• What will each request cost at our expected volume, and what keeps it there?
• What happens when the model we use is retired?

If the answers are demos on sample data and "the model is very accurate", keep looking. Projects that need a larger platform around them run with our custom software development team, and reporting-heavy work with data analytics and BI.

Frequently asked questions

Will an AI assistant replace my support team?

Rarely, and we do not design for that. Well-built assistants answer repetitive questions, draft replies and route the rest, freeing staff for cases that need judgment. Customers still need a clear route to a person. Success is measured in faster resolution and fewer repeated questions, not headcount.

Is our data used to train the AI models?

Not by default on the major providers' business APIs and cloud platforms, according to their current terms, though those terms differ on retention and abuse monitoring. We document exactly what is sent where before the build, pick a provider and region that fits your obligations, and can self-host an open-weight model when data must not leave your infrastructure.

What is the difference between a chatbot and RAG?

A chatbot is the interface: a conversation window. RAG is a technique behind it that searches your documents for relevant passages and gives them to the model, so answers come from your content with citations instead of from the model's general training. Most useful business chatbots use RAG or a similar retrieval step.

How accurate will it be?

That is exactly what the first milestone answers, on your own examples, before you commit to a full build. Accuracy depends on document quality, how clearly questions can be answered from them, and how strict the task is. We report scores per question type, so you can launch where accuracy is high and keep humans on the rest.

Can AI run on our own servers?

Yes. Open-weight models can be self-hosted on your own hardware or in your private cloud account, which keeps data inside your infrastructure. The trade-offs are hardware cost, operational work and, for some tasks, lower accuracy than the largest hosted models. The evaluation harness shows whether a self-hosted model is good enough for your task.