Localization testing: What is it & how to do it (+sample project)

In this article we’ll discuss what is localization and what’s it all about. After all, we live in an imperfect world, and we as humans are prone to making mistakes. It’s not a bad thing — it’s just something to remember and take into account.

Specifically, there’s a high risk of errors creeping into your localization efforts. Sometimes these are small matters, like typos or slightly awkward translations. But other times, these mistakes can affect whether users from other countries can even use your product. Broken strings, language resources that don’t load, and fonts that don’t support certain languages can be very frustrating.

These details can actively stop customers in the target market from enjoying something that you’ve worked so hard to put out there. So, catching these errors before they go live is critical. That’s where localization & internationalization testing comes in.

Special thanks to my colleagues Tom Franc, Valentyna Kozlova, and Jindrich Oprsal for providing useful insights into this topic. Also thanks to Bill Laffetry from Acclaro for sharing his experience.

You might be also interested in our “How to successfully translate a website” article.

What is localization testing?

By definition, localization testing is the process of testing software or content to ensure that it is suitable for a particular locale or region. This involves verifying that the product functions correctly with the language, cultural norms, and other regional settings of the target audience. The goal of localization testing is to ensure that the product meets the needs and expectations of users in different regions and can be used effectively and correctly by them.

You’re testing how well your user interface (UI), default language, time formats, currency formats, and other factors meet the culture, local regulations, and usability standards of your new audience in the target market. Of course, this process relies on testers who review your website, software, mobile application, or game. They assess usability and look for any areas where localization didn’t work.

It’s similar to the quality assurance (QA) process in software development, where testers come in to learn where the bugs are in your app. So, this process helps you to make sure that everything works as intended for users in your new market. Without this step, you risk your localization efforts seeming inauthentic to native speakers at best and straight up offensive or inappropriate at worst in specific region.

Main aspects of localization testing

It’s important to mention that localization testing is sometimes mixed up with linguistic testing. However, there is a clear distinction between those two terms. Where linguistic testing focuses mainly on translation quality and text readability (and ideally it should be performed by a native speaker with a linguistic background), localization testing focuses on the following key aspects:

• Your application’s user interface — revealing any misplaced, overlapping, or truncated text as well as misplaced UI elements (buttons, boxes, images, etc.). If your application is available on multiple devices, it’s important to perform testing on these devices.
• National characters — these must be displayed properly, and locale-specific typographic rules should be followed properly as well (various styles of quotation marks, apostrophes, etc.).
• Functionality of the localized app  — this should stay the same as in the original app.
• Inputs — these have to be able to handle national characters.
• Locale-specific date, time, currency, and measurement unit formats — these have to be displayed and handled properly.

Common steps

Here are some common localization testing steps (of course, things may vary in your case):

• Go through discovery with customer to identify content subject to testing and where it is hosted.
• Ask for existing test plan or create one based on the requirements of the loc program.
• Determine the testing environment + tool for communicating identified issues (Jira, etc.).
  • Customers usually have a bug reporting tool that they give access to.
• Determine permissions in the testing environment/bug reporting tool for the relevant stakeholders: language leads, testers, loc engineers, PMs.
• Determine how the findings are to be communicated to linguists so that they can make corrections in the CAT enviro. Excel/CSV export, screenshots, track changes, QA checks in TMS, and so on.
  • Update the TM/Glossaries in the process.
• Redeliver.

Common pitfalls are often the result of lack of planning and/or communication, and developing a comprehensive plan in advance is therefore pivotal. Getting buy-in from stakeholders so they know what to expect is also crucial.

How to automate localization testing (but only partially)

Note that localization testing can be partially automated (for example, to check for truncation and overlapping). Also, the QA engineer can be neither a native speaker nor a linguist. S/he just has to be aware of the criteria specific to the given locale.

There are some essential parts of the automated localization testing process worth mentioning:

• Strategy — defines what the primary focus will be, what part of the app will be tested and for which locales, what the percentage of coverage per locale will be, what part of the testing will be automated, and so on.
• Design and development — implementation of the automated test cases. Of course, it’s impossible to automatically check everything, but some companies take advantage of so-called “smoke tests“. Basically, these are usual automated tests that check the main parts of the app. They make sure it generally works well without going into too much detail. In other words, smoke testing is used to check for simple and obvious failures.
• Creation of test cases — developer documentation feature specification is usually employed to prepare test cases. Each test case, as you might know, consists of several parts: setup, “repro” (reproduce) steps, sample files, and a description of the expected behavior.

Don’t forget, however, that tests “can be used to show the presence of bugs, but never to show their absence” (quote by E. Dijkstra).

The localization testing team consists of several roles, which are:

• Project manager
• Test designers
• QA automation developers
• Test team leads
• Localization testers

How to perform localization testing

Your goals

Usually localization testing focuses on three areas: language, UI, and UX.

Your goal when testing localization is to make sure that content loads in the correct language. Your translation shouldn’t take away from the overall user experience of your app or website. But, to test those factors, you also have to check that your code works as it should, the placeholders are tied to the right strings, and that the correct fonts load for each language.

When you’re ready to test, start with a testing checklist. This should include all the information, context, and variables for which you localized. It also provides you with a reproducible framework for all your testing efforts.

Localization checklsit

Here’s an example of what your localization checklist or framework could look like:

1. Gather all the relevant information needed to provide a solid understanding of your product or website. This includes documentation, background information, and any glossaries or industry-specific terms that provide additional context. The more information you give testers, the more thoroughly they can test.
2. Create a user guide for your testers to follow, such as an ideal use scenario. This helps them understand how customers will use your product or website and tells them where they need to look. You can also take a more exploratory approach to your testing and let testers guide themselves. This can be an excellent way to discover the smaller details that may get missed with a user guide. Using both guided and exploratory testing leads to incredibly thorough testing.
3. Test. Testing a website is a bit different than a software application, as we’ll see below, but you’re always checking the same core factors: appearance and layout, functionality, cultural appropriateness, and the accuracy of the translation.

There is a little bit of overlap between localization testing for software, like an app, versus a website. However, they’re different enough that knowing what to look for is important.

Software localization testing

When running software localization testing, there are a few high-level questions to ask first: Did the correct language load (and are the translations accurate)? Are the right regional and cultural pieces in place? Is anything obviously broken in the UI or UX?

Once you’ve covered the high-level review, it’s time to thoroughly examine the software. In this stage, you’re looking for the following:

• Have the hotkeys been adapted correctly? For example, if saving is triggered by Alt+S in the English app using an English keyboard layout, is there compatibility with a Russian keyboard layout?
• Does the correct font load for each language? Latin-based languages, like English, French, and Spanish use one font type, while other languages like Arabic use different fonts to load properly.
• Are all your placeholders loading correctly? Also, did all your strings translate or did something break? Are they concatenating properly?
• Do the widgets provide enough space for the translated text?

Website localization testing

Testing your website begins with the same considerations as for software or an application. Conduct a high-level check of languages, cultural appropriateness, and whether anything is clearly wrong with the UI or UX. Ensure your website localization is accurate and fits the target market.

From there, you start to look deeper into the functionality of your website. This means features like:

• Check your CTAs. Do they still “pack the same punch” in the localized versions? Since their goal is to motivate users to click, you may have to alter the CTA to a more language-appropriate version (you can use transcreation for this).
• Do all your hyperlinks work the same in each language you support? Is the wording on the translated link consistent? Does it still point to the correct place?
• If you’re running an e-commerce website, you’ll also want to go through the checkout process to make sure that everything is accurate and working properly. Pricing, product descriptions, currency — all these variables need to be checked. Be sure to also test what happens when you place an order. Does the thank you page load properly? What about the confirmation email?
• You should also check to make sure that if you’re using a chatbot on your website, it loads in all the languages you support. It’s quite important when support global audience.

Finally, run a spell check in all supported languages for both your software and website. This catches the simple mistakes and helps create a polished final product.

A sample localization testing project

Here’s a sample description of a localization project for a non-existent company. This project is based on a real one:

Project: Company products and documentation articles

Teams: Multiple small teams with different focuses (documentation articles, software, DTP [images], etc.)

Workflow: Team 1: Documentation articles

• Customer files arrived from integration and were uploaded automatically to a translation management system — checking that they were parsed and pre-translated correctly, only translatable text is unlocked, and so on.
• Translation
• Automatic upload to customer’s CMS
• Testing:
  • For no overflowing or truncated text, which usually the most common issue
  • Margins are correct
  • Fonts are correct
  • Links are not broken
  • Market-specific aspects observed (correct images, correct products as some products might not be available for certain countries)
• Fixing directly in the code in the CSM and publishing
• Gather feedback

Workflow: Team 2: DTP

• Same as above, with the added steps of OCR, creating a translatable layer, translating it, and then applying the changes.

Workflow: Team 3: Software

Involves lots of automated tests.

• Testing. Writing test cases:
  • Everything is translated
  • Translations are in correct language
  • No empty translations
  • System items that should not be translated are left intact
• Visual testing
  • No overflowing or truncated text
• Linguistic testing
  • Translation is understandable to the users

Tools to help with localization testing

There are tools that can help you with localization testing, but they’re mostly software testing tools that include a localization feature. They’re not designed with localization in mind. Plus, you’d have to use a few different tools to cover everything you need to test.

Translation management system

Therefore, a translation management system (TMS) that lets you include QA checks as a part of your process could come in very handy. A good TMS will allow you to take control of a few different aspects of your localization testing process. First, it will allow you to use automation to reduce the manual aspects of testing. This includes steps where you need to gather documentation, glossaries, and other context-based collateral to give your testers all the information they need. A TMS should contain all this information anyway, so providing it to your testers becomes easier than if you had to collect everything, upload it to a shared folder, and share it.

Localization itself is a long and involved process, and testing is no different. It’s helpful to have a clear view of where you are, what’s been done, and who’s working on what. Having a built-in chat feature makes communicating with your teams about testing easier as well. The right TMS will also integrate with the tools you use to help the design team quickly iterate and test whether UX features are going to be consistent, for example. It helps the dev team import and export the data they’re using too.

QA checks

Finally, you can get built-in QA checks with a TMS. These help you check certain aspects of the project as you’re working on them, like:

• Inconsistent placeholders (source versus target)
• Inconsistent HTML (source versus target)
• Different URLs (source versus target)
• Trailing whitespace (target)
• Different numbers (source versus target)

This is where we come in.

Lokalise is a TMS that provides everything you need to streamline the localization testing of your software app or website. Lokalise provides the necessary context for your testers (like screenshots of localized content in the target language), integrates with the tools your teams use, and includes 13 built-in QA checks to reduce the amount of time it takes to test everything.

Learn more about how we can help you with all your localization needs. Contact us today.