Persian OCR Triple Threat 🔥 Benchmark

A same-test leaderboard for Persian OCR across 6,669 printed lines and 6,669 handwritten pages. Both frozen splits live in one public dataset and contribute exactly 50/50 to the overall rank.

WER · word errorCER · character error S³ · semantic importance🔑 Ess Err · critical words

S³ weights substituted and dropped reference words by ShenavaSanj importance. Overall 🔥 is the equal-split macro-average of each split's weighted score: 20% WER, 20% CER, and 60% S³. Lower is better. Avg decode (ms) is measured mean latency per image; API timings include provider and network latency, while local batched timings are amortized per image. 🔒 marks closed paid APIs.

🔥 Best Overall Triple Threat 🥇 Bina 0.1 · 9.8420% WER · 20% CER · 60% S³

Frozen benchmark6,669 + 6,669 printed · handwriting

Sortable leaderboard — ranked by Overall Triple Threat

Sortable leaderboard — ranked by Overall Triple Threat

1	Chandra OCR 2 NVFP4A16	0.097436B	104.92	10.21	12.12	217.47	166.86	11.19	185.39	165.36	24.05	10004.2	Bina · Surya	complete	Persian LoRA checkpoint step 8000, merged for inference; batch size 8 per GPU; max new tokens 4096. All 13,338 rows scored with zero missing predictions. Triple Threat uses the leaderboard's current 20% WER · 20% CER · 60% S³ weighting.


1	🥇 Bina 0.1		9.84	10.21	12.12	10.83	6.54	11.19	11.41	8.38	9.22	305.7	Bina · Surya	complete	Persian LoRA checkpoint step 8000, merged for inference; batch size 8 per GPU; max new tokens 4096. All 13,338 rows scored with zero missing predictions. Triple Threat uses the leaderboard's current 20% WER · 20% CER · 60% S³ weighting.
2	🥈 🔒 Gemini 3.5 Flash		15.72	13.97	15.74	6.71	3.68	3.89	33.95	29.06	24.05	9397.4	Gemini	complete	OpenRouter routing; minimal reasoning effort; reasoning excluded from returned transcription.
3	🥉 🔒 Gemini 3.1 Flash Lite		18.62	16.92	18.79	7.22	4.43	6.59	39.5	33.54	27.24	8886.3	Gemini	complete	OpenRouter routing; default reasoning effort; reasoning excluded from returned transcription.
4	Gemma 4 31B IT	31B	35.66	28.6	31.48	13.16	8.05	8.05	93.45	70.34	49.15	1387.3	Gemma	complete	Provider pinned to cerebras/fp16 with fallback disabled; reasoning disabled.
5	🔒 Qwen 3.6 Flash		37.62	38.63	42.03	16.78	8.95	17.59	67.45	51.21	59.67	8332.2	Qwen	complete	Reasoning disabled. Alibaba refused 35 images during data inspection; refusals are scored as blank predictions.
6	Chandra OCR 2 NVFP4A16	3B	43.66	36.85	40.76	57.42	49.5	20.27	62.19	46.44	53.43	333.2	Chandra	complete	vLLM 0.20.2; batch size 24 per GPU; max new tokens 1536; checkpoint revision b11480ba302227c206eaa4ef558acbacfd7f7eea. All 13,338 rows scored with zero blank predictions.
7	Surya OCR 2		47.94	46.45	48.31	21.51	12.6	18.86	90.71	75.93	74.03	2395.2	Surya	complete	Stallion RTX 5080 Laptop GPU; batch size 8; max new tokens 1536. All 13,338 rows scored; 64 blank predictions included and penalized.
8	Gemma 4 26B A4B IT	26B	57.2	34.34	37.7	9.99	5.2	9.86	185.39	165.36	58.82	8860.9	Gemma	complete	Provider pinned to wafer/fp8 with fallback disabled; reasoning disabled.
9	Tesseract Persian (fas)		66.29	69.93	71.04	50.19	34.51	50.91	89	69.56	88.95	null	Tesseract	complete	Community submission q6qR0L; 897 blank predictions included and penalized. Scored with the frozen Persian normalization protocol.
10	WeightedAI Persian OCR	0.097436B	76.84	79.54	80.5	58.94	40.12	59.68	100.75	91.3	99.41	329.3	PersianOCR	complete	Official OpenCV line segmentation and greedy CTC decoding; line batch size 16; model revision d7edaf2a76e33e77e447bf88df4a2d023bef3b5d. All 13,338 rows scored; latency is the mean over 1,882 saved timing rows.
11	🔒 Gemini 2.5 Flash Lite		104.92	37.53	40.76	217.47	166.86	18.78	235.25	204.43	56.29	10004.2	Gemini	complete	Provider pinned to google-ai-studio; reasoning excluded from returned transcription.

Submit a model

Run the model on all rows of both benchmark splits and submit JSONL using IDs printed:0…printed:6668 and handwriting:0…handwriting:6668. Open a Community discussion with predictions, exact model/API revision, decoding configuration, and runtime details.