What is the best LLM for translation? A comparison of top AI translation models

Are LLMs better than traditional machine translation tools?

LLMs are the stronger default for many production translation tasks, while traditional MT is increasingly best treated as a baseline or fallback.
To test this rigorously, Lokalise’s product team ran a blind comparison study across three language pairs (from English to German, Polish, and Russian) and five systems.  

Two LLMs (Claude Sonnet 3.5 and GPT-4o) and three traditional machine translation tools (DeepL, Google Translate, and Microsoft Translator) were tested.

Because translation quality is inherently subjective, we used native-speaker pairwise comparisons and statistical ranking to reduce preference noise and produce results we can be confident about.

The thing is, a major blocker in AI localization is trust. Teams hesitate to automate review because they assume quality assessment is too subjective. But our findings suggest that the gap is smaller than people think.

Key findings:

• LLMs demonstrated superior performance across all tested language pairs, with translation quality marked as ‘good’ between 55.7% and 80% of the time, even without any contextual information
• Claude 3.5 remains the benchmark for stylistic fluency, while Gemini 2.0 Pro dominates long-context repository translation
• LLMs reduced review load potential, because “good” output was frequent enough to use LLMs as a first pass and reserve human review for exceptions rather than defaults

Across our evaluation, human-to-human agreement and AI-to-human agreement were close, whether measured with inter-rater reliability (Cohen’s Kappa) or overlap metrics (Average Jaccard Similarity).

In some cases, evaluators aligned more consistently with the AI ranking model than with other human annotators.

What did the WMT25 study find?

WMT25 shows that top-performing translation systems increasingly rely on LLM-based or hybrid approaches. The performance differences become clearer when evaluation uses harder, more realistic test sets instead of “easy” examples.

In the WMT25 General Machine Translation Shared Task, teams could submit systems for 30 language pairs, and the benchmark includes human judgment as a key part of evaluation.

The practical takeaway is that “best model” decisions should be task-based and context-aware. Traditional neural machine translation (NMT) is increasingly being integrated with (or replaced) by LLM-based approaches, typically via fine-tuning or as part of data cleaning and post-editing pipelines.

How we tested LLMs for translation

To ensure scientific rigor, we used multiple evaluation methodologies, including the Bradley-Terry model, one of the most respected statistical methods for ranking items based on pairwise comparisons. It’s a probabilistic framework that models the probability that translation A will be preferred over translation B using strength parameters.

This approach allowed us to establish a clear hierarchy of translation quality based on maximum likelihood estimation.

We focused on high-resource languages to test, not only the hypothesis that LLMs have surpassed ‘more traditional’ MT systems for translation but also that LLM translation is already ‘good enough’ for high-resource languages.

We used Large Language Models to translate from English into three languages:

• English to German
• English to Polish
• English to Russian

Then we asked human annotators to evaluate translation quality through pairwise comparisons of translations from different engines:

• For every translation, native speakers compared the variants from different engines and highlighted the best one
• 600+ pairwise comparisons were carried out by multiple human annotators for each language pair

Interestingly, while Russian is technically a high-resource language, our ‘goodness’ scores were significantly lower than for German and Polish, suggesting that resource availability doesn’t always correlate directly with translation performance.
 

Why LLM-first translation works better in production workflows

LLMs tend to outperform traditional MT because they handle meaning and intent more flexibly, especially when the workflow supplies the right context. This is exactly why translation performance depends on task configuration, not model brand.

LLMs like Claude Sonnet 3.5 and GPT-4o bring several advantages to translation tasks that traditional machine translation tools don’t:

1. Contextual understanding: Unlike traditional translation systems, LLMs grasp the broader context of text, enabling more natural-sounding outputs
2. Cultural nuance: These models can better preserve idioms, cultural references, and tone across languages
3. Adaptability: LLMs demonstrate greater flexibility when handling specialized terminology or uncommon language patterns
4. Consistency: Our testing showed remarkable consistency in quality, with the best LLM achieving good translations 78% of the time across multiple language pairs

As large language models (LLMs) evolve and new models are released, the gap between LLM and traditional machine translation quality is only going to get bigger.
 

How to choose the right LLM for translation

Choose the right LLM for translation by matching the model’s strengths to your specific task. Use Claude for high-nuance marketing content, GPT-4o/5 for technical documentation and code localization, Gemini for long-context, multi-file consistency, and cost-efficient models like DeepSeek for high-volume bulk translation within an automated workflow.

The right model depends on what you’re translating, how much context you can provide, and how strictly you need to protect formatting, terminology, and brand voice.

In 2026, the best LLM for translation is a routing decision inside your automated pipeline.

The four use cases below reflect the most common translation tasks inside automated workflows.

Why use Claude for creative translations

Claude 3.5/4 excels at high-nuance marketing translation where tone, idioms, and brand voice determine whether content feels truly native.

Claude is typically the best choice when the translation task demands stylistic fluency, not just literal accuracy. It performs well on marketing pages and UX copy where tone, idioms, and cultural nuance determine whether the output feels native and on-brand.

Claude is great for translating customer-facing, brand-sensitive content, especially when you ground it with reference material (approved phrasing, terminology, examples) so it doesn’t “invent” a new voice from scratch.
 

Why use ChatGPT for technical and code translations

GPT-4o/5 leads in technical documentation and code localization by reliably preserving variables, placeholders, and structured formatting.

ChatGPT is often the safest choice when translation accuracy depends on constraint-following. It tends to handle placeholders, variables, and formatting rules more reliably, which matters for UI strings and technical documentation, and for code-adjacent strings that must preserve syntax.

ChatGPT works well for translating structured content where slightly less elegant phrasing is acceptable.

Why use Gemini for long-context and multi-file translations

Gemini long-context models are a strong fit when your translation task is not a single string, but an entire collection of related files that must stay consistent.

With a context window that can scale up to 2 million tokens, Gemini is useful for translating long documentation sets, repositories, or multi-file exports where names, terminology, and phrasing must remain stable across sections.

Why use DeepSeek for cost-efficient bulk translations

DeepSeek’s pricing model is explicitly positioned around cost-efficiency at scale, which makes it attractive for large batches and always-on internal workflows.

DeepSeek is often the best option when the task is high-volume, lower-risk translation and the priority is cost and throughput. This can include internal updates, drafts, and bulk content that still needs to be understandable, but doesn’t need a premium marketing voice.

With LLMs constantly evolving and releasing new versions, it’s hard to stay on top of which translation model to choose depending on your needs. That’s where Lokalise comes in.

Why orchestration matters more than model brand

In 2026, there’s no single “best” LLM for translation. There’s a best model for the job in front of you.

Claude is usually the safest bet when tone and nuance matter, GPT-4o/5 is a strong choice when you need strict accuracy with variables and structured formats, Gemini is helpful when you’re translating long documents or keeping many files consistent, and DeepSeek can make sense when you need high-volume output at a lower cost.

Model selection shouldn’t live in someone’s head or in a one-time decision. The most reliable workflows route each task to the right model, ground the model with your approved terminology and past translations, and use quality signals to decide what’s publish-ready versus what needs review.

That approach stays stable even as models change. It’s the easiest way to keep translation quality high without slowing your pipeline down.

Want to learn more? Watch this webinar and see how you too can achieve 90% publish-ready translations with AI.