How to Properly Assess Translation Quality

All Language Service Providers (LSPs) follow certain industry best practices to give clients the highest quality translations. Unfortunately, most translation tests don’t allow language vendors to adhere to those best practices as they would with a real project. As a translation buyer, it’s important to know why taking another approach is in your best interest. We recommend using a 5-point method to assess translation quality instead. This alternative to test translations will save you time, minimize risk, and ultimately result in a better process for selecting your translation partner.

Sample translations break all the rules.

Although a test translation seems like the easiest way to evaluate a professional translation agency, it may not give you the insight you need. Before you request a sample, you should be clear about why you want one. What do you want to assess?

• The ability of an LSP to act quickly?
• Their ability to communicate with you?
• How the LSP operates?
• The quality of the final product?

You won’t be able to accurately judge any of these criteria via a test translation because it omits essential steps and crucial documentation. What you miss out on with a test translation: Terminology, Style guide, Training, Feedback, Resources, Speed.

Terminology

A glossary that establishes terminology in the target language(s) is a critical component of a quality translation, especially for highly complex domains such as gaming, medical, and legal. The best practice is to obtain a current glossary (if it exists) or develop one and submit it for the client's review and approval before beginning the actual translation. In most sample translation tests, the client does not provide a glossary, nor is there time to create one.

Style guide

Translation style guides govern tone, terminology usage, punctuation, sentence structure, date/time/number formats, and more. Even if the client only has a style guide in the source language, that provides a place to start. The LSP can leverage this to help the client develop style guides for each additional language. To the detriment of test translation projects, clients rarely, if ever, provide a style guide.

Training

Product training is essential for translators, especially if software, hardware, or medical device translations are required. The companies that receive the highest quality translations from their LSPs invest in training the vendor's translation team. Sample translation tests rarely include the associated training.

Questions

Over the course of a normal translation process, translators ask questions about source content to make sure they understand it correctly. Clients clarify terms, meaning, and intent, and the translator revises or constructs the translation accordingly. (All translators are given these clarifying answers.) During the sample translation test process, Q&A is rare. Therefore, translators are left to guess, hoping their assumptions are right.

SMEs/Resources

To provide the highest quality translations, LSPs work with translators who are subject matter experts (SMEs) in the relevant areas. They’re in high demand and are booked well in advance of projects to ensure the right SMEs are working with the right clients. Many translation providers have a large database of translators indexed by subject matter expertise. Sample translation tests assume the best SMEs are available "on-demand." They’re typically removed from other paid projects to address a sample that requires a quick turnaround, breaking all best practice rules. Ironically, most test translations come from clients who have specialized content, which is the most difficult to translate in a test setting.

Speed

Often, clients ask an LSP to complete a test translation in two or three days. This doesn’t give the translation team enough time to follow all of the optimal steps. A turn-around this fast typically requires skipping kickoffs and other information exchanges, as well as the aforementioned Q&A between translators and the client. It’s also nearly impossible to engage the right SMEs without adequate lead time. In the context of an established program, the vendor would have client-familiar resources already in place and would be able to act quickly. So, how can a client assess the quality of a potential language vendor? Here's what we recommend.  

The 5-point approach to assessing translation quality (without a translation test sample)

1. Ask for samples of work from similar clients.

Reviewing previous translation projects for your industry will help you evaluate the quality of the LSP’s work. For example, let’s say you’re a project manager at a SaaS company. You may want to ask for examples of technical documentation, website content, and online help to assess whether they can deliver on a highly technical project.

2. Go in-depth on process and project management.

During the initial conversation, ask the following questions:

• Do you have documented processes for project on-ramp, quality steps, and project methodology?
• Do you follow these processes consistently?
• How do project managers control quality?
• What is your on-time delivery performance?

3. Be clear about your perceived success criteria.

Define success within your organization and confirm that the LSP understands which resources you’ll need to ensure the project’s success. Ask for information on resourcing that validates the LSP’s ability to secure the right resources. Do they have enough language experts and project managers available on a timeline that fits your goals?  

4. Conduct a small, paid pilot project.

If it’s critical to assess an LPS’s work on a live job, start with a small, real-life project. You can control the time and cost of evaluating the materials by short-listing language providers and asking only two to complete the pilot. Then ask the LSP to complete translations that you will actually use, so you don’t spend money on redoing already translated materials. Here are a few points to take into consideration before you include a source text in your pilot project:

• It should represent the typical text type you need translated, i.e., it should be as representative as possible of the materials you commonly translate. It can cover several areas of expertise (such as marketing, legal, technical).
• The volume of a source text or group of texts should be no longer than 1,000 words (ideally, less than 500 words). Otherwise, the workload and your internal costs will be too high. Commission a pilot project that’s longer than 1,000 words and you’ll be focusing on quantity instead of quality.
• Test translations should encompass five or fewer target languages for the same reasons. You want to assess the overall quality of the LSP’s approach, not evaluate translations in multiple languages you may never need. In other words, focus on your core languages.
• Decide which of your team members will review the translations. Make sure they’re native speakers of the target language and have university-level language skills. Being a native speaker is usually not enough.

5. Ask how they safeguard and measure translation quality.

Go into detail and ask the right questions:

• Does the LSP use key performance indicators (KPIs) for quality?
• Do they base quality assessments on industry standards, i.e. SAE J2450, Multidimensional Quality Metrics (MQM), Dynamic Quality Framework (DQF), or proprietary methodologies?
• Is there a regular language quality inspection process in place with which they ensure quality at the source?
• Are there automated document quality inspection tools integrated in their production workflows?
• Do they have an online review tool that lets clients and reviewers interact?
• Is there a feedback management process and system in place?

A good LSP will be able to inform you about their Language Quality Strategy and how it is sustainably put into practice.   Interested in learning more or starting a project? Kickwords boasts a worldwide network of native-speaking subject matter experts who can help translate your content in 100+ languages. Reach out today!