Comprende? How To Efficiently Test Multilingual Apps

October 9, 2018 Rafał Kubik

It’s always great when your app or product goes global, but worldwide scale often brings its own challenges. More specifically, your app now needs to work in additional languages and, as a tester, this amounts to much more work!

This is a challenge we had to solve with a recent client. Since we managed to find an effective solution, I wanted to share our work process with you here!

What’s The Problem?

During both functional and non-functional testing, a tester may struggle with many different challenges; undefined requirements, performance and usability issues are all everyday problems. Everyone wants to have a “user friendly” website or app that is both easy to use and maintain. The question is how to get all this without having to check multiple pages, along with a mobile app, for various language versions.

This was the problem that occured while working on a project for an airline client. We needed to check seven different languages of the multilingual app: English, Spanish, German, Greek, Italian, French and Portuguese. Imagine having hundreds of pages and buttons to go through! Each page must be tested in different resolutions (all pages are responsive), along with all the errors and validations. We cannot forget about the mobile app and the thousands of different mobile devices that users can install it on. Even after all that, we are only talking about languages – it’s only a small part of what us testers do!

Identifying The Issues

At the beginning, we had to define our tactics, because it would otherwise be impossible to test all seven languages across every single subpage. When the QA team joined the project, the website had already been implemented and it supported all the aforementioned languages. The developers had just added some additional functionalities, so it was easy for QA to check all of them (along with all language versions) manually. After all, we were talking about a few pages, instead of hundreds. The client then delivered translations in an excel spreadsheet, which the developers had to manually copy into an XML file.

Based on this information, after a quick testing session, we drew some important conclusions:

All translations are copied manually by developers, so there’s a risk that some characters will be missing.
Because of the above, all translations need to be compared with the original excel file.
The most problematic languages are German (very long words) and Greek (Greek alphabet). For example, a word like “reservation confirmation” can be expressed as “Reservierungsbestätigung” in German, or “επιβεβαίωση κράτησης” in Greek.
For small screen sizes, the translated phrases usually don’t fit in the intended buttons and frames.
Sometimes, although the translation was delivered in lowercase, the design required it to be displayed in uppercase on the webpage. This was a challenging problem with Greek, where upper and lowercase letters look totally different. “Δ” and “δ” do not look similar, but represent the same letter (delta).

Let’s Test!

During our test session, we mostly focused on small resolution displays (iPhone SE etc.). A lot of attention was put on the German and Greek language versions for the above mentioned reasons.

One of the major challenges was comparing Greek sentences because, to a non-Greek speaker, all the letters can more or less look the same. The solution was Notepad++. It is equipped with a comparison function, which allowed us to spot differences very quickly and easily. We simply took text from the web app and the excel sheet, using the “Compare” function to make sure the strings were identical.

In addition to simple typos, other commonly found bugs included very long strings of text that overlapped with other elements on the page, words displayed in an incorrect language and missing translations.

New Platforms = New Challenges

Life went on but, one day, the client decided to rebuild both the backend and web page completely, as well as request a new mobile app. These new designs were ready a few weeks later, so our developers then got down to work. The web page was rewritten in Angular, while the mobile app was built in Xamarin.

This time, we had to face the challenge of translating something that was work-in-progress, not a complete product. It was obvious that the final version should support all seven languages but, at the beginning, we had to build a single-language version…. and guess what? It wasn’t English!

The client was from Spain, so when it came to app localisation, we started with the Spanish version. Fortunately, this meant we had some help ensuring this language was input correctly (Spanish has its own unique features, such as inverted question and exclamation marks, “¿” and “¡”, respectively)

It is not easy when a client tells you that on page X, the 20th word is a passenger’s name. So, instead of static text, there should be a variable that may look different for various users. However, the real problem was the translations themselves. If they were delivered in excel files, like they were for the old platform, all the old problems would return and, worse, they would return on a much, much bigger scale.

New Problems And Solutions

Luckily, instead of sticking with Excel, the client decided to use Oneskyapp. This is a translation management platform which works with 19 file formats, including XML.

To cut a long story short, the developers gathered complete sets of keys and translations for one language. We asked QA to make an excel file out of it and provided the list to the translation agency. Consequently, they saw the whole list and were able to fill in any missing parts. They then delivered the rest of translations but, this time, they sent them straight to Oneskyapp.

Finally, the developers were able to download a json/xml/resx file that was generated by the programme. This file was used directly in the application, so no additional copy & paste activities were required!

At this point, we noticed a few significant traits that are relevant for future testing:

  • We assumed that all texts within the application are correct (no missing characters, uppercase letters, commas etc.) since they were provided by a translation agency and didn’t have to be entered manually.
  • Small screens were still a concern due to overlaps and text fitting issues.
  • It would be impossible to check all pages manually, so we need to automate the process.
  • German and Greek languages will still be the most problematic, due to long words and the Greek alphabet, respectively.

Based on this, we prepared some automated tests using Selenium for the web app and Calabash for the mobile app. The aim of these tests was to go through all the pages and take screenshots of each of them.

During this task, we struggled with typical automation problems (unreachable locators, issues with the Android emulator and plenty of broken builds in TeamCity).

Similarly, in the development process, we had some problem with Calabash. It turned out that, if we took screenshots using Selenium, the whole page was saved, rather than just the visible part, while the opposite was true in the case of Calabash. Therefore, we needed to add a method that scrolls down and takes a screenshot at the end of page.

After hours of coding, the automated tests were ready! (After some small modifications, we also used them for regression testing). We took a few of the most popular devices and ran tests on each of them.

Since it was impossible to automate all pages and errors, we had to do a lot of manual work as well. Still, the automated tests made the work so much easier. We just had to remember to give the correct name for each screenshot, so we could easily differentiate them.

Is It Over?

After all that, you can be forgiven for thinking we’re finally finished with linguistic & localisation testing…

What if I tell you that, in addition to seven languages, we also needed to check four different currencies?

  • EUR €
  • USD $
  • GBP £
  • AUD $

Moreover, some regions use a dot as a decimal separator, while others use a comma. $25,98 or $25.98? 18,49€ or €18,49? What about the thousands separator? Should we even use it? When some currencies (such as GBP) use commas to separate thousands, while others use commas as a decimal point (such as EUR), we need to ensure each language is clear for their respective markets.

We need to remember to define these things with both the client and Business Analyst (BA) before testing. In the testing world, even commas and dots matter.

App Translation Checklist: What Should I Do?

  • Make sure that you define all culture specific markers: dots, commas, currency signs etc.
  • Focus on small resolutions and “exotic” languages.
  • Automation might help with translation testing.
  • A spreadsheet file is not the best tool for translations. It hinders the work of both developers and testers.
  • For long and complicated phrases, use a comparison function.
  • Exhaustive Testing is impossible, so chose your approach wisely.

Conclusion

Don’t be afraid of translation testing! You can learn 7 different languages simultaneously, all during working hours! We treated it as a challenge and found a solution to test it as efficiently as possible.

Of course, good communication with the client, as well as a translation agency, and an understanding of culture specific markers are key in such situations. Good planning might save hours of testing in the future, so keep calm and test those transitions!

Ciao!

Last posts