Fine-tuning is usually proposed as an engineering decision and reviewed as a risk decision. The gap between those two framings is where projects stall — often after the work is already done.
Settle these before you spend the compute.
Where did the training data come from?
Not “from our systems.” Specifically: which systems, collected under which terms, from which users, and with what basis for using it this way. If any of it came from customers, check whether your terms of service actually permit training. Many do not, and finding out afterwards is expensive.
Can you produce the exact training set again?
Six months from now, someone will ask why the model behaves a certain way. If you cannot reconstruct the data it learned from, you cannot answer. Snapshot it, hash it, and store the hash with the model artifact.
What happens when someone asks for deletion?
If a user exercises a deletion right and their data is in your training set, what is your answer? “It is baked into the weights” is a real answer, but it is one you need to have decided in advance rather than discovered under a deadline.
How will you know it got worse?
Fine-tuning improves the thing you measured and silently degrades things you did not. Hold out an evaluation set that covers the behaviour you are not trying to change — general capability, refusals, tone — and check it every time.
Who signs off, and against what?
Name the person and the criteria before you start. A model-risk review with no pre-agreed standard becomes a negotiation, and negotiations are won by whoever is most senior rather than whoever is most correct.
Is fine-tuning even the answer?
Asked last deliberately, because it is the question most often skipped. A meaningful share of the fine-tuning projects we review would have been better served by improving retrieval, fixing the prompt, or cleaning the underlying data. Those are cheaper, faster, and reversible.
Fine-tune when you have evidence that the alternatives are exhausted — not because it is the most interesting option on the table.
