LLM Fine-Tuning Services

LLM Fine-Tuning Services — When, Why & How to Customize Language Models

Every organization that deploys language models eventually asks the same question: should we fine-tune? The base model is impressive but not quite right. It does not use the company's terminology. Its responses are too long or too short. It hallucinates about domain-specific facts. It cannot reliably follow the output format your downstream systems expect. LLM fine-tuning is the engineering discipline that addresses all of these problems systematically.

But fine-tuning is not always the answer. It requires quality training data, GPU compute, engineering expertise, and ongoing maintenance. When prompt engineering or retrieval-augmented generation can solve the problem, fine-tuning adds unnecessary complexity and cost. The first skill in fine-tuning is knowing when not to do it. The second is knowing which method to use when it is genuinely needed.

At ESS ENN Associates, we have been building software systems for global clients since 1993. Our AI engineering practice has delivered fine-tuning projects across model sizes from 3B to 70B parameters, using every major technique from full fine-tuning to LoRA to RLHF. This guide shares the decision frameworks, technical approaches, and practical lessons from those engagements.

When Fine-Tuning Is the Right Choice

Fine-tuning makes sense in specific circumstances, and understanding those circumstances prevents wasted investment. The decision should not be based on intuition or the assumption that custom models are always better. It should be based on evidence that simpler approaches have been tried and found insufficient.

Consistent output formatting is one of the strongest signals that fine-tuning is needed. If your application requires the model to always produce JSON with a specific schema, always follow a particular report structure, or always respond in a specific style, prompt engineering alone often produces inconsistent results. A few-shot prompt might achieve 90% format compliance, but production systems typically need 99%+. Fine-tuning on thousands of correctly formatted examples can push compliance above 99.5%.

Domain-specific vocabulary and reasoning that differs from general language is another strong indicator. Medical, legal, financial, and technical domains use specialized terminology, follow domain-specific reasoning patterns, and have nuanced conventions that base models learn imperfectly from general pretraining data. A base model might know that metformin treats diabetes, but a fine-tuned model knows the dose titration schedule, the contraindications for renal impairment, and the monitoring protocols that a clinician needs in a decision support tool.

Cost reduction through prompt compression is an often-overlooked benefit. If your production prompts include extensive system instructions, numerous few-shot examples, or detailed formatting rules, those tokens cost money on every request. Fine-tuning encodes this knowledge into model weights, allowing much shorter prompts that produce equivalent results. A prompt that uses 2,000 tokens for system instructions and examples can often be reduced to 200 tokens after fine-tuning, cutting input token costs by 90% for that component.

When not to fine-tune: If prompt engineering with a capable model achieves your quality requirements, the maintenance burden of fine-tuning is not justified. If your requirements change frequently (monthly or more), retraining overhead makes fine-tuning impractical. If your training data is limited to fewer than 500 high-quality examples, the fine-tuned model is unlikely to generalize well. And if you only need the model to access specific factual information, RAG architecture is typically more effective and easier to maintain than fine-tuning.

Full Fine-Tuning: Maximum Customization, Maximum Cost

Full fine-tuning updates every parameter in the model during training. For a 70B parameter model, this means training 70 billion weights, which requires holding the model weights, optimizer states (which are 2-3x the size of the weights for Adam-based optimizers), gradients, and activation memory simultaneously in GPU memory. The practical hardware requirement is 8-16 A100 80GB GPUs for a 70B model, making this the most expensive fine-tuning approach.

The advantage of full fine-tuning is that every layer of the model can adapt to your domain, potentially learning deeper domain-specific representations than adapter-based methods. For tasks that require fundamental changes to the model's behavior — learning a new language, acquiring deeply specialized domain knowledge, or substantially altering the model's reasoning patterns — full fine-tuning provides the most thorough adaptation.

However, the quality advantage over LoRA is typically marginal. Across dozens of projects, we have found that full fine-tuning outperforms LoRA by 1-3% on domain-specific benchmarks. In most cases, this small improvement does not justify the 4-10x compute cost increase. Full fine-tuning also risks catastrophic forgetting more severely than adapter methods, because all weights change during training. Careful learning rate selection, data mixing with general-purpose examples, and extensive evaluation are needed to prevent the model from losing general capabilities while acquiring domain expertise.

Full fine-tuning is the right choice when you need maximum domain adaptation quality, when the compute budget supports the cost, when you plan to deploy the model widely enough that the per-request savings from improved quality offset the training cost, or when you are creating a new base model for an underserved domain or language. For most enterprise fine-tuning projects, LoRA provides a better cost-quality tradeoff.

LoRA and QLoRA: The Practical Standard

LoRA has become the default fine-tuning method for production projects because it provides the best balance of quality, cost, flexibility, and operational simplicity. The technique adds small trainable adapter matrices to the model's attention and MLP layers while keeping the base model weights frozen. Only the adapter parameters (typically 0.1-1% of total model parameters) are trained.

For LLM-scale fine-tuning, LoRA's operational advantages are as significant as its compute savings. The adapter weights are stored as separate files (typically 100MB-2GB for a 70B model, compared to 140GB for the full model). This means you can maintain multiple fine-tuned versions for different clients, domains, or tasks without duplicating the entire model. You can swap adapters at inference time, enabling multi-tenant deployments where different users interact with different fine-tuned behaviors on the same base model infrastructure. And you can roll back a problematic fine-tune by simply loading the previous adapter version.

QLoRA extends LoRA by loading the base model in 4-bit quantized format during training. This reduces the GPU memory required by approximately 75%, making it possible to fine-tune a 70B model on 2-4 A100 GPUs instead of 8-16. The quality tradeoff is a further 1-2% on benchmarks compared to full-precision LoRA, which is rarely significant in practice. QLoRA has made large model fine-tuning accessible to teams without datacenter-scale GPU allocations.

Key LoRA configuration decisions for large models include rank selection (16-64 for most tasks, with higher ranks for complex domain adaptation), target module selection (attention projections are mandatory; MLP layers are optional but improve results for domain-heavy tasks), learning rate (typically 1e-4 to 5e-4 for large models, lower than for SLMs to prevent instability), and alpha scaling (typically 2x the rank). These hyperparameters interact with each other and with dataset characteristics, so expect to run 3-5 training experiments to find the optimal configuration for your specific case.

Instruction Tuning: Teaching Models to Follow Directions

Instruction tuning is the specific application of fine-tuning that teaches a model to follow human instructions accurately. Base pretrained models are trained to predict the next token in text, not to follow instructions. The gap between a base model and an instruction-tuned model is dramatic: the base model generates plausible text continuations, while the instruction-tuned model directly addresses the user's request in a helpful format.

The instruction tuning dataset consists of instruction-response pairs that demonstrate the desired behavior across a wide range of tasks. High-quality instruction datasets include diverse task types (question answering, summarization, classification, creative writing, code generation, analysis), varying difficulty levels, explicit format requirements, and examples of appropriate refusal (declining to answer questions outside the model's scope or capability).

Self-instruct and Evol-Instruct are techniques for generating diverse instruction tuning data. Self-instruct uses the model itself to generate new instructions based on seed examples. Evol-Instruct progressively increases the complexity of instructions through multiple evolution rounds, producing datasets that cover a wider range of difficulty levels. Both techniques can generate tens of thousands of instruction-response pairs from a small set of seed examples, though quality filtering is essential to remove low-quality generations.

For enterprise instruction tuning, the instruction set should be biased toward the types of instructions your users will actually give. If your fine-tuned model will be used for customer support, the instruction set should emphasize question answering, empathetic responses, and procedural guidance. If it will be used for document analysis, the instruction set should emphasize extraction, summarization, and comparison tasks. This task distribution alignment is often more impactful than dataset size — 5,000 well-targeted examples outperform 50,000 generic ones.

RLHF and DPO: Aligning Models with Human Preferences

Instruction tuning teaches a model what to do. Alignment training teaches it how to do it well. The distinction matters: an instruction-tuned model might answer a medical question with accurate but tactless bluntness, while an aligned model provides the same information with appropriate sensitivity, appropriate caveats, and a professional tone. Alignment is what makes model outputs feel helpful, professional, and trustworthy.

RLHF (Reinforcement Learning from Human Feedback) is the original alignment technique used to train ChatGPT and similar models. The process involves three stages. First, collect preference data: human raters compare pairs of model outputs and indicate which one is better. Second, train a reward model: a separate neural network that learns to predict human preferences based on the collected data. Third, optimize the LLM: use reinforcement learning (typically PPO) to adjust the LLM's behavior to maximize the reward model's scores while staying close to the original model through KL divergence constraints.

RLHF produces excellent results but is complex to implement. The reward model can develop biases or reward hacking behaviors. PPO training is notoriously unstable and sensitive to hyperparameters. The three-stage pipeline requires careful orchestration and monitoring at each stage. The compute cost is typically 2-5x the cost of supervised fine-tuning alone.

DPO (Direct Preference Optimization) has largely replaced RLHF for most fine-tuning projects. DPO uses the same preference data (pairs of outputs with human preference labels) but eliminates the separate reward model entirely. Instead, it directly optimizes the LLM using a contrastive loss that increases the probability of preferred outputs and decreases the probability of dispreferred outputs. The math shows that DPO implicitly optimizes the same objective as RLHF but through a simpler, more stable training process.

The practical advantages of DPO over RLHF are significant. Training is more stable with fewer hyperparameters to tune. There is no separate reward model to train and maintain. Compute costs are 40-60% lower. And the results are comparable or superior to RLHF on most alignment benchmarks. The only scenarios where RLHF may still be preferred are when you need to combine alignment training with reinforcement learning on environment feedback (not just human preferences) or when you have a very large, high-quality reward model that you want to reuse across multiple fine-tuning runs.

Dataset Preparation for LLM Fine-Tuning

The principles of dataset preparation for LLM fine-tuning parallel those for SLM fine-tuning (covered in our SLM fine-tuning guide) but with additional considerations driven by the larger model scale and broader capability range.

Dataset size for large models follows different guidelines than for small models. Large models are better few-shot learners, meaning they need fewer examples to learn a new pattern. For focused tasks like format compliance or style adaptation, 1,000-5,000 high-quality examples often suffice for a 70B model. For broad domain adaptation, 10,000-50,000 examples produce strong results. For general-purpose instruction tuning, datasets of 50,000-500,000 examples are typical. Going beyond these ranges rarely improves quality and increases training time proportionally.

Preference data collection for DPO alignment requires pairs of outputs where one is clearly better than the other. The most efficient approach generates multiple outputs from the model being fine-tuned, then has human raters or a strong LLM judge rank the outputs. Aim for 5,000-20,000 preference pairs for domain-specific alignment and 50,000+ for general alignment. The quality of preference labels matters enormously — noisy or inconsistent labels degrade alignment quality. Use majority voting across multiple raters for each pair, and discard pairs where raters disagree significantly.

Data mixing strategies combine domain-specific data with general-purpose data to prevent catastrophic forgetting. The typical recipe is 70-80% domain-specific data and 20-30% general data drawn from established instruction tuning datasets. The general data maintains the model's broad capabilities while the domain data shapes its specialized behavior. Adjusting this ratio is one of the most impactful tuning decisions — too little general data causes capability regression, too much dilutes domain adaptation.

Compute Costs and Infrastructure Planning

Understanding the compute requirements for LLM fine-tuning is essential for project planning and budgeting. The costs vary by orders of magnitude depending on model size, method, and dataset scale.

LoRA fine-tuning costs on cloud GPU infrastructure: a 7B model with 10,000 examples requires approximately 4-8 GPU-hours on a single A100, costing $10-30. A 13B model with the same dataset takes 8-16 GPU-hours on a single A100, costing $25-60. A 70B model requires 4 A100 GPUs and takes 8-24 hours, costing $100-600. These are single-run costs; expect to run 3-5 iterations for hyperparameter tuning, multiplying the total by that factor.

Full fine-tuning costs scale dramatically with model size. A 7B model requires 2 A100 GPUs for 8-24 hours ($50-150). A 70B model needs 8-16 A100 GPUs for 1-3 days ($2,000-15,000 per run). A 405B model pushes into the range of 32-64 GPUs for days to weeks, with costs of $20,000-100,000 per run. These costs make full fine-tuning impractical for iterative experimentation on large models, which is another reason LoRA is preferred for most projects.

Infrastructure options include managed fine-tuning platforms (OpenAI, Anyscale, Together AI, Fireworks AI) that abstract away GPU management, cloud GPU instances (AWS p4d/p5, GCP A3, Azure ND series) that provide raw compute, and dedicated GPU servers for organizations with ongoing fine-tuning needs. Managed platforms are simplest but most expensive per GPU-hour. Cloud instances offer flexibility at moderate cost. Dedicated servers have the lowest per-hour cost but require infrastructure management expertise. Our GPU server and MLOps services help teams set up and manage dedicated fine-tuning infrastructure.

Evaluation and Iteration

Fine-tuning without rigorous evaluation is guesswork. The evaluation framework should be established before the first training run so that results can be compared objectively across experiments.

Domain-specific evaluation sets should contain 200-500 examples that are strictly separated from the training data and representative of production usage. These examples should cover the full range of task types, difficulty levels, and edge cases your model will encounter. Automated metrics (exact match, F1, BLEU, ROUGE) provide quick feedback during iteration. Human evaluation or LLM-as-judge evaluation provides deeper quality assessment for the most promising checkpoints. For a comprehensive evaluation methodology, see our LLM evaluation and benchmarking guide.

Regression testing on general capabilities ensures that domain adaptation has not degraded the model's broader skills. Maintain a separate evaluation set of general-purpose tasks and verify that scores remain within acceptable bounds after fine-tuning. A 5% regression on general benchmarks is typically acceptable as long as domain-specific performance improves by 10%+ to justify the tradeoff.

A/B testing in production is the ultimate validation. Deploy the fine-tuned model alongside the base model and route a percentage of production traffic to each. Measure business-relevant metrics — user satisfaction, task completion rates, escalation rates, and response quality scores — to verify that the fine-tuned model delivers real-world improvement, not just benchmark gains.

Frequently Asked Questions

At ESS ENN Associates, our AI engineering services team delivers end-to-end LLM fine-tuning from dataset preparation through production deployment. We operate on dedicated GPU infrastructure optimized for training workloads and bring 30+ years of software delivery experience to every engagement. If you are considering fine-tuning a language model for your business, contact us for a free technical consultation.