The standard was developed with the aim to maintain “a consistent standard against which the translation quality of automotive service information can be objectively measured”:
The method is based on a points-based system. The less points a translation scores, the higher the quality of the translation.
Errors are divided into 7 categories and each category is outlined comprehensively:
Each category is weighted. Some error categories are considered to affect final quality more than others. For instance, a Spelling Error may score fewer points than errors found in the Wrong Term category.
Once an error is identified as belonging to a category, the human reviewer decides whether the error is either serious or a minor. A serious error is weighted to score more points than a minor error.
The metric is both easy to follow and easy to implement. It is an excellent step towards creating an objective linguistic quality measurement. Furthermore, it is highly customizable: if you feel that in a particular application a spelling error can be more damaging to a translated version than the use of an incorrect term, you can easily change the weighting.
Finally, the results of the metrics can be used for benchmarking linguistic standards and serve as a basis for discussion with both clients or translation groups.
If your organization is planning to use J2450, please remember that:
The metric is published by SAE and specifically thinking about the automotive service information industry. Consequently, it may not be the most suitable method for evaluating translation where style and/or voice are of paramount importance.
Results for the reviews have to be collated manually and comparisons can be difficult if you are working directly from spreadsheets.
Guidance is needed for reviewers using the metric. It needs to be very clear what constitutes a serious or a minor error. Evaluators need to be trained to ensure clear and common understanding.
Once the points have been allocated, there is no benchmark to determine what constitutes a good or bad mark. We carried out a test of the metric at Pangeanic by submitting translations of the same text into 4 different languages to our in-house and external proof-readers. After evaluation, we found that the language with the second least points was not approved by the proof-reader as publishable. However, proof-readers for languages that had scored more points were generally happy with the translation.
The J2450 Translation Quality Metric can be purchased through the SAE internet site.
Av. Cortes Valencianas, 26-5,
46015 Valencia (Spain)
(+34) 917 94 45 64 / (+34) 96 333 63 33
Flat8, 279 Church Road,
+44 203 5400 256
(+34) 91 326 29 33
One Boston Place
Boston MA 02108
228 E 45TH St Rm 9E
10017-3337 New York, NY
21st Floor, CMA Building
64 Connaught Road Central
Toll Free: +852 2157 3950
Ogawa Building 3F
3-37 Kanda Sakuma-cho
Tomson Commercial Building,
710 Dong Fang Road
Pu Dong, Shanghai 200122, China