Flagship Methodology · Framework v1.0

How we test: the Global Language Quality Benchmark Framework

Every score on LocRank comes from a fixed public corpus, a deterministic scoring script, and archived raw outputs. No sponsored placements, no star ratings, no estimates. If a tool has not been run, its page says “test pending” and explains exactly why.

Principles

Measured, never sponsored

Vendors cannot pay for a score, a ranking or a review. Tool links on this site are plain official links; LocRank earns no affiliate commission from them (status as of 2026-11-21, disclosed on every page).

Append-only evidence

Each benchmark run is published as a dated, permanent archive page with the full raw outputs. Runs are never edited after publication; corrections and re-tests ship as new runs. Historical data is the asset — it cannot be recreated retroactively.

Reproducible by anyone

The corpus, the term lists, the formulas and the scoring weights are published on this page. Anyone with the same inputs gets the same numbers.

Conflicts, disclosed

LocRank is published by HankLoc, which sells localization services. That is exactly why the method is mechanical: a deterministic script scores archived outputs, and no commercial relationship enters the formula.

The framework: one architecture, many language benchmarks

A benchmark is a (language, corpus, rule set, scoring script) bundle with its own version number. The framework defines the shared metrics — script purity, wording density, terminology consistency — and each language benchmark supplies its own corpus and norm references. The Traditional Chinese Quality Benchmark (TCQ) is the flagship and the first live instance; Simplified Chinese is in development, and Japanese, Korean and Spanish slots are reserved in the same architecture. A language with no completed run publishes no scores.

Traditional Chinese Quality Benchmark — v1.0 specification

Norm reference: Traditional Chinese as written in Taiwan. Input: 24 fixed English segments across four content types (marketing, UI strings, game dialogue, customer support), chosen to contain known cross-strait wording traps and repeated key terms. Access: each engine is called through its public or API access path, once per segment, with no human editing; the access path is recorded on each tool page and in the run archive.

The three metrics

MetricDefinitionFormulaWeight
Script purityShare of CJK characters in the output that are simplified-only characters, detected character-by-character with the OpenCC s2t conversion table.max(0, 100 − residue% × 4)40%
WordingDensity of mainland-China wording, matched greedily (longest-first) against a 131-term rule list — the same list that powers this site's free wording checker.max(0, 100 − hits per 100 CJK chars × 10)35%
TerminologyFor each tagged term occurrence: the preferred Taiwan rendering scores 1, a mainland variant scores 0, any other rendering is excluded and disclosed.preferred / (preferred + mainland) × 10025%

Composite score = 0.40 × purity + 0.35 × wording + 0.25 × terminology, rounded to the nearest integer. Grades: A ≥ 90 · B ≥ 80 · C ≥ 70 · D ≥ 60 · F < 60.

Worked example (segment M1, run of 2026-11-21)

EngineRaw output (archived)Flags
Google Translate觀看視頻,了解我們的軟體如何幫助您的團隊在線上工作。視頻 (mainland term for “video”), 在線 (mainland phrasing)
Gemini觀看影片,了解我們的軟體如何協助您的團隊進行線上協作。None

Full per-segment evidence: Benchmark run 2026-11-21 (v1.0).

The corpus (v1.0, frozen)

IDTypeEnglish source segmentTagged terms
M1marketingWatch the video to see how our software helps your team work online.video, software, online
M2marketingHigh-quality sound and unbeatable value for money — download the app today.quality, value for money
M3marketingGet instant message notifications and free shipping on every order.message
M4marketingOur network covers the whole city, so you can take a taxi or a bus to get anywhere.network
M5marketingSign up now and print your boarding pass before the countdown ends.print
M6marketingThis smartphone has a bigger screen and a smarter default search.screen, default, search
U1uiSettingssettings
U2uiLog in to your accountlog in
U3uiSearch settings and message notificationssearch, settings, message
U4uiDownload the software updatesoftware
U5uiWatch the video in high qualityvideo, quality
U6uiDefault network settingsdefault, network, settings
G1gameWatch the opening video, then open settings to adjust the screen brightness.video, settings, screen
G2gameCollect 100 coins to level up and unlock the secret level.—
G3gameThe final countdown begins! Your team must protect the crystal at all costs.—
G4gameCheck your messages — the guild leader has sent you a new quest.message
G5gameThis boss drops high-quality gear with excellent value for money.quality, value for money
G6gameDownload the update to play online with friends around the world.online
S1supportPlease contact our online support team if you cannot log in.online, log in
S2supportYou can print your receipt from the account settings page.print, settings
S3supportWe will send you a text message to verify your phone number.text message
S4supportIf the video will not play, check your network connection.video, network
S5supportYour software license expires in 7 days. Download the latest version now.software
S6supportThank you for your message. Our support team will reply within 24 hours.message

Terminology checks

English termPreferred (Taiwan)Mainland variants
default預設默認 / 默认
log in登入登錄 / 登录
message訊息信息
network網路網絡 / 网络
online線上在線 / 在线
print列印打印
quality品質質量 / 质量
screen螢幕屏幕
search搜尋搜索
settings設定設置 / 设置
software軟體軟件 / 软件
text message簡訊短信
value for moneyCP 值 / CP值性價比 / 性价比
video影片視頻 / 视频

How platforms (TMS) are scored

A translation management system does not itself translate: it routes work to MT engines and humans. Scoring a TMS therefore means running its built-in or default pre-translation path on the same corpus, inside a real account, and scoring what comes out — with the access path disclosed. Until that account-based run is executed, a TMS entry shows “test pending”. We do not borrow the score of an engine a TMS can connect to; connections are configuration, not output.

Limitations, stated plainly

  • One run per engine per benchmark version; engines change without notice, so scores are dated and re-tested quarterly.
  • The corpus is 24 segments. It is designed to discriminate script and wording quality, not to grade literary translation.
  • LLM scores belong to the exact model version and prompt archived in the run — a different prompt is a different configuration.
  • Wording hits are machine flags from a published rule list; a human reviewer can dispute a flag, and disputes are resolved in the next run, never by editing a published one.

Cite this benchmark

LocRank, Traditional Chinese Quality Benchmark, v1.0 (2026-11-21). Corpus, method and run archive: locrank.com/directory/methodology/ and locrank.com/directory/benchmarks/zh-hant/2026-11-21/. Linking and citation are welcome; scores consumed programmatically should follow the API terms (attribution required).

Get the full benchmark dataset — and every future score — by email

The public pages show the headline scores. The email edition carries the complete dataset behind them:

  • The full per-segment benchmark dataset for every run (all 24 segments, every engine, every flag)
  • New-tool test alerts — the day a pending tool gets its first measured score
  • Quarterly re-test results, including score changes and what caused them

Get the benchmark report →

Free. One list, no spam, unsubscribe anytime. You will receive a confirmation email first.