Achieving 90% Publish-Ready Translations with AI

Full transcript

Welcome and introductions

Marta (host): Welcome to the webinar today, Achieving 90% Publish-Ready Translations with AI. Thank you to all the attendees for being here. Today we're going to talk about how translations performed by custom AI can achieve 90% acceptance rates, and why generic LLMs miss your brand voice and constraints. We'll specifically focus on two main approaches in the industry: fine-tuned models and retrieval-augmented generation (RAG). We'll show some examples and tell you how this all works at Lokalise. By the end of the webinar, you'll have a clear understanding of both methods and which one is best to achieve human-level translation, optimize your budget, and be more efficient with your teams. We'll have dedicated Q&A at the end.

Our speaker today is Sasho, Engineering Manager at Lokalise on the AI team. He brings more than fifteen years of experience in AI across academia, business sustainability, and health tech, and now applies that expertise to localization.

Why AI needs customization

Sasho: Today we're going to talk about how we can customize AI to improve localization, and in particular how we get to this 90% release-ready number. We'll start by explaining how AI models work in general, where they fall short, how customization improves their performance, the different types of customization and when to choose which, and how Lokalise is doing it.

At a high level, businesses want predictable, low localization costs. They want human review to align closely with the AI translation output, so that whenever adjustment is required it's a small one. They want the terminology and brand voice to come through in translations, so post-editing effort stays low.

General LLMs are generalists with broad knowledge, but that knowledge isn't complete and it's frozen at a point in time. If you're a smaller brand without much public exposure, or you just rebranded, the LLM simply doesn't know about you — that's not public domain knowledge you've shared with anyone. To get an LLM attuned to your brand, you need to give it extra context, and that context needs to be good quality: feed it faulty translations or bad glossary terms, and the output quality will match.

AI models are frozen once trained — using them (inference) doesn't make them better. To change a model, you need training, which is fine-tuning on a bigger model. Fine-tuning creates a new frozen model that then knows about your brand. The other way to give context, without changing the model at all, is few-shot learning: giving enough examples of previous translations or terminology and asking the model to follow the pattern. That's RAG — retrieval-augmented generation — because you retrieve context and augment the generation with it.

How fine-tuning works

Sasho: Fine-tuning needs a comprehensive parallel translation dataset — for each language pair you localize to, you need a substantial number of example translations (a few hundred per pair, at minimum) plus setup time to train and deploy the model. Once that's done, you use the model like any other: no extra context needed, just make your usual translation request and the model already knows how you do things. In practice, that means preparing your data (e.g. as a JSONL file), uploading it to a fine-tuning interface like OpenAI's, choosing a base model, naming and configuring the tuning job, running it, and then using the resulting model in the sandbox or via API.

How RAG works

Sasho: RAG is more lightweight — no complicated training pipeline. What you need is a healthy set of translation assets: a good glossary, style guide, and translation memory, the same things that give context to human translators. You retrieve relevant context from those assets and plug it into your prompt, then instruct the LLM to do the job — putting in the source text to translate, relevant prior translations, and glossary terminology. You can then send that prompt to a provider like OpenAI or Anthropic. The downside is you have to do this over and over if done manually, so it's best set up programmatically.

Fine-tuning vs. RAG: trade-offs

Sasho: Fine-tuning is good for learning an entire voice and putting a lot of data to use, and it doesn't need more context with every use — you train once, then just use the model. On the downside, it's slow to update (correcting a word's usage means retraining), it has a cold-start problem for new companies with no translation history to train on, and there's a fixed, fairly significant cost to every update.

RAG has immediate adaptability — do a translation, and the next similar one immediately benefits from it — and updates cost nothing if your system is programmatic. The trade-off is a per-translation inference cost, since providing context makes each request larger (usually negligible, but worth acknowledging), and RAG is susceptible to errors in your translation assets: one bad glossary entry that happens to be relevant will propagate into new translations, so asset quality matters a lot.

I'd choose RAG if I release often, my terminology changes frequently, and I like to keep improving what I've already translated, provided my resources are clean and well-maintained. I'd lean toward fine-tuning if I'm a well-established brand with lots of resources that doesn't change much but wants to expand into new markets, or if I know I'm about to run a big campaign and want to train ahead of it. My analogy: fine-tuning is boot camp — you train someone hard up front for a well-defined, long-term role. RAG is more like giving someone excellent daily briefs to guide them through a dynamic role that keeps changing.

RAG in practice: a worked example

Sasho: Say I'm a fintech company localizing into Spanish, with decent translation memory where terminology consistency and brand voice matter. If I translate “add the funds to your wallet,” RAG's retrieval step looks at my translation memory, notices “fondos” and “cartera” are relevant prior translations, and uses them as context. The same applies to brand voice — if I want to keep a formal or informal register (“verify your identity” staying formal in Spanish), retrieval surfaces examples that cue the model to match that register. RAG also adapts automatically: if we decide to switch our brand voice to informal (“tu cuenta,” “tu tarjeta”), future retrieval pulls informal examples and the model follows suit.

How this works at Lokalise

Sasho: We ran (and are still running) a beta testing both customization types, and what we call advanced RAG is what we've implemented, because it gives customers strong adaptability and translation quality. We can't say one approach is definitively better — results are comparable — but the flexibility of RAG is why we chose this path. In a blind reviewer test (where the reviewer didn't know if a human or AI produced the translation), our RAG model was accepted up to 95% of the time.

On acceptance rates generally: humans agree with each other somewhere between 85–95% of the time — there's no such thing as a perfect translation. Standard, no-context AI/MT engines sit around 65–75%. Add context, scoring, and smart routing (automatically picking the best model per language pair, since different models have different strengths) and you get to Pro AI, at roughly 75–85% acceptance. Layer on custom AI profiles — advanced RAG on top of everything else — and you reach roughly 85–95% acceptance, which is what we saw in the beta.

On cost: human translation is the 100% baseline. Standard AI gets you to roughly a 30% cost reduction; Pro AI pushes further; and switching to custom AI can bring cost down to roughly 15% of the human baseline. Looking ahead, if you eventually trust the system enough to skip review entirely, that could approach a 97% cost reduction versus human translation — while keeping the context-aware, convenient setup.

Live demo: custom AI profiles

Adam: Hello everyone, I'm Adam, product lead for our AI translations domain. I'll do a quick demo of our custom AI profiles feature, currently in beta. In Lokalise's team settings, there's a new Custom AI Profiles section where you manage which AI profile is used for AI translation and AI scoring. By default, everyone on Pro AI uses a base AI profile for AI suggestions, AI tasks, and AI automations, with smart routing to pick the best LLM for the job, and style guide, glossary, and descriptions already included as context.

You can also activate a custom profile, which adds RAG based on translation memory on top of everything Pro AI already does — retrieving the most relevant examples from your translation memory as extra context for the LLM. To turn it on: select the source and target languages (in the demo, English to Czech and French), and optionally filter your translation memory by date — for example, only trusting entries from a point where translation quality improved. That matters because RAG's improvement is directly dependent on the quality of what it retrieves from: with high-quality translation memory, we've seen acceptance rates grow by 10–20% in real customer cases; with a translation memory full of low-quality, unreviewed machine translation, RAG likely won't help and could even slightly hurt quality. After selecting languages and date filter, you pick the project(s) to activate it on and click Activate — from then on, all AI suggestions, AI tasks, and AI automations for those languages and projects use that custom AI profile automatically. It's currently an open beta for enterprise customers via waitlist.

Key takeaways (from Sasho)

Both types of customization can reach human-level acceptance under good conditions, getting to roughly 90–95%. RAG is simpler and faster to keep current if you're dynamic and don't have perfectly mature data processes. Fine-tuning fits large, preplanned batches in stable domains, and is generally a better fit for mature customers who've been around a long time.

Live Q&A

Q: What's your opinion on starting with a fine-tuned model on our previous translation dataset, then deploying RAG on top to continuously improve?
Adam: I'd actually advise starting with RAG first — it's much simpler to get going, works out of the box, and a fine-tuned model is harder to start with. Try RAG, see how it improves quality, and only then consider fine-tuning. Sasho: One word of warning if you do go this route: check whether your existing fine-tuned setup somehow contradicts your current translations or voice — you may end up fighting yourself. If so, I'd choose one approach over the other, with RAG as the simpler option.

Q: For an indie video game with a lot of specific and made-up terms (e.g. monster names based on existing words), which method would you advise?
Sasho: RAG is particularly good at introducing new, even made-up, terminology the model wouldn't otherwise know. If this is your first time localizing this kind of content, you're likely hitting the cold-start problem, where fine-tuning would struggle and RAG is the better fit. Adam: For the monster names specifically, a glossary is the right asset — it's built exactly for pinning a name to an exact translation, and an LLM given that glossary will respect it in most cases. You can combine glossary for exact terms with RAG based on past translations for everything else.

Q: Did you notice major differences in acceptance rate by language, especially non-Latin-alphabet languages like Russian or Arabic?
Sasho: Both Arabic and Russian have decent acceptance rates because they're well-resourced languages with a lot of training data — around average. Languages like German or French do a bit better, but not by a huge margin. What we've observed is that RAG can actually help even more on less mainstream languages than on well-resourced ones like German or French, though it improves both.

Q: I built a custom GPT infused with seven years of my company's translations. What are the next steps to go live, have it learn automatically from ongoing translations, and know when human review is needed?
Sasho: On keeping it current: if you fine-tuned a model, the way to update it is to keep re-running fine-tuning — that's the downside of that approach. You could layer RAG on top of your fine-tuned model as a hybrid, or abandon fine-tuning and just use RAG going forward. Adam: On knowing when a human check is needed — at Lokalise we use a scoring feature for exactly this: after an AI translation, scoring determines whether it's already high quality enough to publish, or whether there's a chance of an issue or ambiguity that needs human review.

Q: Does the cost-reduction slide factor in the cost of the custom AI feature itself, or is that cost considered negligible?
Sasho: Yes, it's factored in — the AI translation cost itself is small compared to human translation cost, but it is included in those cost-reduction figures.

Q: Can you explain the difference between RAG and “advanced RAG” again?
Sasho: In Pro AI, the context we provide is more static — things like style guide. What we call advanced RAG is the feature that looks directly at previous translations — translation memory, or a specific set of past translations you define — which you can filter however you like, as Adam demonstrated.

Q: The date-filtering option in the demo makes sense — did you consider other filters, like date intervals, or focusing on specific higher-quality projects?
Adam: Yes, we're actively extending the data sources RAG can use. Soon, customers will be able to filter by tagged data — tag translations you trust in specific projects and have RAG retrieve only from that tagged set — and by human-reviewed translations from selected projects. Between translation memory, tagged data, and reviewed translations, our research shows this covers the vast majority of use cases for pinpointing high-quality source data.

Q: For mature, big companies with a lot of training data already reflected in the LLM, what's better — RAG or fine-tuning?
Sasho: It depends how much change you expect going forward. RAG is almost always a safe, good option. Fine-tuning can be more cost-efficient if you're running it in-house and absorbing the AI costs directly, but there are other costs — maintaining the model, building it in the first place, making sure it behaves correctly — that make RAG the safer default. Adam: Agreed.

The session closed with a reminder that follow-up resources on RAG and fine-tuning, plus a waitlist link for the custom AI profiles beta, would be sent by email, along with a survey for feedback on future webinars.