As of October 2025
Fine-tuning demos for fraud detection look flawless - upload transaction data, train, ship. Then you run the actual numbers, and the gap between the demo and a model that meaningfully improves in production is enormous.
The instructor’s framing from hands-on training puts it bluntly: for every text-model fine-tune that genuinely helps, thousands of attempts produce no useful change or actively degrade the model. Visual models fare better, but the lesson holds - this is closer to a lab experiment than a software install.
Where the time actually goes is the first surprise. Roughly 80 percent of fine-tuning effort is data preparation - cleaning transaction logs, formatting compliance documents, removing bad examples, labeling edge cases - and only a sliver is running the training code. The easiest part of the job is the part the demos show.
The training code is the cheap part. Knowing what to train on, and whether it worked, is the job.
Data prep
~80%
of fine-tuning time goes to data preparation, not training - the practitioner rule of thumb.
LoRA
~10,000x
fewer trainable parameters than full fine-tuning, with GPU memory down about 3x (Hu et al., 2021).
Examples
100-1000
high-quality, verified examples you still need even with LoRA (practitioner range).
The cheap part
Running the training code. That is the step the vendor demo shows, and it is the least of the work.
The expensive part
Knowing what to train on and whether it worked. Catastrophic forgetting is real - train a model to spot wire-fraud patterns and it can overwrite how it handles ACH, because fine-tuning overwrites rather than adds (Kirkpatrick et al., 2017). LoRA cuts the trainable parameters by orders of magnitude, but you still need 100 to 1,000 curated examples with verified outcomes. Real projects run three months and hundreds of experiments, and you only learn retrospectively which checkpoint actually worked.
