Questions and answers

One real question in each of the 30 languages, the right answer, and what every model replied. The other 120 questions stay private so they cannot leak into training data.

In Chicago, the City's Winter Overnight Parking Ban on 107 miles of arterial streets is enforced every night from December 1 to April 1, whether or not it snows. Between which hours of the night is parking banned on those streets? Answer in 24-hour time as HH:MM-HH:MM, with nothing else.

Chicago bans overnight parking on 107 miles of main streets every winter night from 1 December to 1 April. Between which hours? Right answer: 03:00-07:00. 16 of 17 models got it right.

GPT-6 Astra, right: 03:00-07:00
Gemini 3.1 Pro, right: 03:00-07:00
GPT-6 Sol, right: 03:00-07:00
Claude Opus 5.5, right: 03:00-07:00
Gemini 3.8 Flash, right: 03:00-07:00
Claude Fable 5.1, right: 03:00-07:00
GPT-6 Luna, right: 03:00-07:00
Grok 4.7, right: 03:00-07:00
DeepSeek V4 Pro, right: 03:00-07:00
DeepSeek V4.1 Flash, right: 03:00-07:00
Kimi K3, right: 03:00-07:00
Qwen3.8 Max, right: 03:00-07:00
Claude Sonnet 5, right: 03:00-07:00
Gemini 3.5 Flash-Lite, right: 03:00-07:00
GLM-5.3, right: 03:00-07:00
GLM-5.3 Flash, right: 03:00-07:00
Qwen3.8 Flash, wrong: 23:00-07:00

¿Qué locución verbal coloquial, recogida en el Diccionario de la lengua española, significa ‘hacer que una sola persona pague lo que ha perdido junto con otro u otros’, por ejemplo cuando varios compañeros pierden una partida y la cuenta entera recae sobre uno? Responde solo con la locución en infinitivo, sin explicación.

Which colloquial phrase in the Real Academia dictionary means making one person pay for what a whole group lost? Right answer: cargar las cabras. 1 of 17 models got it right.

GPT-6 Astra, right: Echar las cabras.
Gemini 3.1 Pro, wrong: hacer la púa
GPT-6 Sol, wrong: hacer pagar el pato
Claude Opus 5.5, wrong: **Cargar el carro**
Gemini 3.8 Flash, wrong: cargar el mochuelo
Claude Fable 5.1, wrong: Hacer la pava a alguien... no. The DLE entry: "hacer pagar el pato"? Actually DLE: "pagar el pato: Padecer o llevar pena o castigo no merecido, o que ha merecido otro." Not exactly. The definition "hacer que una sola persona pague lo que ha…
GPT-6 Luna, wrong: hacer pagar el pato
Grok 4.7, wrong: clavar
DeepSeek V4 Pro, wrong: pagar el pato
DeepSeek V4.1 Flash, wrong: hacer pagar el pato
Kimi K3, wrong: cargar el muerto
Qwen3.8 Max, wrong: Pagar el pato
Claude Sonnet 5, wrong: Pagar el pato
Gemini 3.5 Flash-Lite, wrong: Pagar el pato
GLM-5.3, wrong: pagar el pato
GLM-5.3 Flash, wrong: pagar el pato
Qwen3.8 Flash, wrong: Pagar el pato

Em português europeu culto, num registo literário, use o verbo «contrafazer» (falsificar, imitar fraudulentamente). Complete a frase com a forma do condicional na 2.ª pessoa do plural (vós), substituindo «as assinaturas» pelo pronome átono correspondente colocado em mesóclise, com a grafia do Acordo Ortográfico de 1990: «Se vos pedissem as assinaturas do rei, ___ sem hesitar?» Responda apenas com a forma verbal, sem qualquer explicação.

Put the rare verb contrafazer (to forge) in the European Portuguese conditional, second person plural, with the object pronoun placed inside the verb. Right answer: contrafá-las-íeis. 12 of 17 models got it right.

GPT-6 Astra, right: contrafá-las-íeis
Gemini 3.1 Pro, right: contrafá-las-íeis
GPT-6 Sol, right: contrafá-las-íeis
Claude Opus 5.5, right: **contrafá-las-íeis**
Gemini 3.8 Flash, right: contrafá-las-íeis
Claude Fable 5.1, right: Contrafá-las-íeis
GPT-6 Luna, right: contrafá-las-íeis
Grok 4.7, right: contrafá-las-íeis
DeepSeek V4 Pro, right: contrafá-las-íeis
DeepSeek V4.1 Flash, wrong: no reply
Kimi K3, wrong: contrafar-las-íeis
Qwen3.8 Max, wrong: contrafazê-las-íeis
Claude Sonnet 5, right: Contrafá-las-íeis
Gemini 3.5 Flash-Lite, right: contrafá-las-íeis
GLM-5.3, wrong: contra-lhas-íeis
GLM-5.3 Flash, right: contrafá-las-íeis
Qwen3.8 Flash, wrong: contra-far-as-íeis

Dans le canton de Genève (Suisse), la loi cantonale sur les jours fériés déclare férié l'anniversaire de la restauration de la République. À quelle date de l'année tombe ce jour férié ? Répondez uniquement par le jour et le mois, sans explication.

The canton of Geneva keeps a public holiday for the restoration of the Republic. On what date does it fall? Right answer: 31 décembre. 17 of 17 models got it right.

GPT-6 Astra, right: 31 décembre
Gemini 3.1 Pro, right: 31 décembre
GPT-6 Sol, right: 31 décembre
Claude Opus 5.5, right: 31 décembre
Gemini 3.8 Flash, right: 31 décembre
Claude Fable 5.1, right: 31 décembre
GPT-6 Luna, right: 31 décembre
Grok 4.7, right: 31 décembre
DeepSeek V4 Pro, right: 31 décembre
DeepSeek V4.1 Flash, right: 31 décembre
Kimi K3, right: 31 décembre
Qwen3.8 Max, right: 31 décembre
Claude Sonnet 5, right: 31 décembre
Gemini 3.5 Flash-Lite, right: 31 décembre
GLM-5.3, right: 31 décembre
GLM-5.3 Flash, right: 31 décembre
Qwen3.8 Flash, right: 31 décembre

Welches deutsche Sprichwort (ein vollständiger Satz) besagt: Wer viel über etwas redet oder es großspurig ankündigt, muss dann auch konkrete Taten folgen lassen? Gemeint ist nicht die Wendung „den Worten Taten folgen lassen“. Antworten Sie nur mit dem Sprichwort im Wortlaut, ohne Erklärung.

Which German proverb says that whoever talks big has to deliver? Right answer: Wer gackert, muss auch ein Ei legen. 2 of 17 models got it right.

GPT-6 Astra, right: Wer gackert, muss auch ein Ei legen.
Gemini 3.1 Pro, wrong: Wer gackert, muss auch Eier legen.
GPT-6 Sol, right: Wer gackert, muss auch ein Ei legen.
Claude Opus 5.5, wrong: Wer A sagt, muss auch B sagen.
Gemini 3.8 Flash, wrong: Wer A sagt, muss auch B sagen.
Claude Fable 5.1, wrong: Wer A sagt, muss auch B sagen.
GPT-6 Luna, wrong: Wer große Töne spuckt, muss auch liefern.
Grok 4.7, wrong: Wer den Mund spitzt, muss auch pfeifen.
DeepSeek V4 Pro, wrong: Wer viel verspricht, muss auch viel halten.
DeepSeek V4.1 Flash, wrong: Wer A sagt, muss auch B sagen.
Kimi K3, wrong: Wer viel verspricht, muss auch viel halten.
Qwen3.8 Max, wrong: Wer den Mund voll nimmt, muss auch schlucken können.
Claude Sonnet 5, wrong: Wer A sagt, muss auch B sagen.
Gemini 3.5 Flash-Lite, wrong: Taten sprechen lauter als Worte.
GLM-5.3, wrong: **
GLM-5.3 Flash, wrong: Viel Geschrei und wenig Wolle.
Qwen3.8 Flash, wrong: Wer viel redet, muss auch viel tun.

Nell'italiano antico e letterario esiste il verbo impersonale «calere», che significa «importare, stare a cuore». Riscrivi la frase «Non credo che a lui importi di questa faccenda» usando il verbo «calere» al congiuntivo presente al posto di «importare», e sostituendo sia «a lui» sia «di questa faccenda» con i pronomi atoni corrispondenti. Scrivi soltanto le parole che vengono dopo «Non credo che», senza spiegazioni.

Rewrite "I don't think this matters to him" with the archaic impersonal verb calere in the present subjunctive and both objects as pronouns. Right answer: gliene caglia. 13 of 17 models got it right.

GPT-6 Astra, right: gliene caglia
Gemini 3.1 Pro, right: gliene caglia
GPT-6 Sol, right: gliene caglia
Claude Opus 5.5, right: gliene caglia
Gemini 3.8 Flash, right: gliene caglia
Claude Fable 5.1, right: gliene caglia
GPT-6 Luna, right: gliene caglia
Grok 4.7, right: gliene caglia
DeepSeek V4 Pro, right: gliene caglia
DeepSeek V4.1 Flash, right: gliene caglia
Kimi K3, right: gliene caglia
Qwen3.8 Max, right: gliene caglia
Claude Sonnet 5, wrong: gliene cagli
Gemini 3.5 Flash-Lite, wrong: gliene calga
GLM-5.3, right: gliene caglia
GLM-5.3 Flash, wrong: gli cagli
Qwen3.8 Flash, wrong: gliene cala

Op het belfort (de Halletoren) aan de Grote Markt van Kortrijk slaan sinds 1961 twee vergulde figuren het uur. Onder welke namen staat dat paar in Kortrijk bekend? Antwoord met alleen de twee namen, verbonden door 'en', zonder uitleg.

Since 1961 two gilded figures have struck the hours on the belfry of Kortrijk's market square. What are they called locally? Right answer: Manten en Kalle. 15 of 17 models got it right.

GPT-6 Astra, right: Manten en Kalle
Gemini 3.1 Pro, right: Manten en Kalle
GPT-6 Sol, right: Manten en Kalle
Claude Opus 5.5, right: Manten en Kalle
Gemini 3.8 Flash, right: Manten en Kalle
Claude Fable 5.1, right: Manten en Kalle
GPT-6 Luna, right: Manten en Kalle
Grok 4.7, right: Manten en Kalle
DeepSeek V4 Pro, right: Manten en Kalle
DeepSeek V4.1 Flash, right: Manten en Kalle
Kimi K3, right: Manten en Kalle
Qwen3.8 Max, right: Manten en Kalle
Claude Sonnet 5, right: Manten en Kalle
Gemini 3.5 Flash-Lite, right: Manten en Kalle
GLM-5.3, right: Manten en Kalle
GLM-5.3 Flash, wrong: Mantelen en Klapperen
Qwen3.8 Flash, wrong: Krispijn en Mariken

Jakie polskie przysłowie mówi żartobliwie, że niezapowiedziane, niespodziewane odwiedziny sprawiają gospodarzom ogromny kłopot? Podaj je w brzmieniu hasła z Wielkiego słownika języka polskiego PAN (WSJP), bez wyjaśnień.

Which Polish proverb jokes that an unannounced visit is a disaster for the hosts? Give the wording in the WSJP dictionary. Right answer: Gość nie w porę gorszy od Tatarzyna. 8 of 17 models got it right.

GPT-6 Astra, wrong: gość w dom, woda do zupy
Gemini 3.1 Pro, right: gość nie w porę gorszy od Tatarzyna
GPT-6 Sol, right: Gość nie w porę gorszy od Tatarzyna.
Claude Opus 5.5, right: **Gość nie w porę gorszy od Tatarzyna.**
Gemini 3.8 Flash, right: gość nie w porę gorszy od Tatarzyna
Claude Fable 5.1, right: Gość nie w porę gorszy od Tatarzyna.
GPT-6 Luna, right: Gość nie w porę gorszy od Tatarzyna.
Grok 4.7, wrong: Nieproszony gość gorszy od Tatarzyna.
DeepSeek V4 Pro, right: Gość nie w porę gorszy od Tatarzyna
DeepSeek V4.1 Flash, right: gość nie w porę gorszy Tatarzyna
Kimi K3, wrong: Nieproszony gość gorzej Tatarzyna.
Qwen3.8 Max, wrong: Gość w dom, kłopot w dom
Claude Sonnet 5, wrong: Gość w dom, Bóg w dom.
Gemini 3.5 Flash-Lite, wrong: Gość w dom, bóg w dom
GLM-5.3, wrong: Nieproszony gość gorszy niż Tatarzyn.
GLM-5.3 Flash, wrong: Nieproszony gość pod piecem śpi.
Qwen3.8 Flash, wrong: Nieproszony gość

Aşağıda Osmanlı Türkçesiyle, Arap harfleriyle yazılmış dört terim var. Her birini Türk Dil Kurumunun Güncel Türkçe Sözlük'ünde madde başı olarak geçtiği biçimde, bugünkü Latin harfli Türkçe yazımla yazın. Düzeltme işareti (şapka) ile bitişik ya da ayrı yazım tam olarak sözlükteki gibi olmalıdır. 1. كاتب عدل 2. اركان حرب 3. مدّعئ عمومى 4. وقايعنامه Yalnızca dört terimi aynı sırayla, aralarına virgül koyarak yazın; açıklama eklemeyin.

Write four Ottoman terms, given in Arabic script, in modern Turkish spelling exactly as the Turkish Language Association dictionary lists them. Right answer: kâtibiadil, erkânıharp, müddeiumumi, vakayiname. 5 of 17 models got it right.

GPT-6 Astra, right: kâtibiadil, erkânıharp, müddeiumumi, vakayiname
Gemini 3.1 Pro, right: kâtibiadil, erkânıharp, müddeiumumi, vakayiname
GPT-6 Sol, wrong: kâtib-i adil, erkânıharp, müddeiumumi, vakayiname
Claude Opus 5.5, right: Kâtibiadil, erkânıharp, müddeiumumi, vakayiname
Gemini 3.8 Flash, right: kâtibiadil, erkânıharp, müddeiumumi, vakayiname
Claude Fable 5.1, right: TDK Güncel Türkçe Sözlük forms: 1. كاتب عدل → "kâtibiadil" — TDK has "kâtibiadil" (noter). Yes, TDK: kâtibiadil. 2. اركان حرب → "erkânıharp" — TDK: erkânıharp. 3. مدّعئ عمومى → "müddeiumumi" — TDK: müddeiumumi. 4. وقايعنامه → "vakayiname" —…
GPT-6 Luna, wrong: kâtib-i adil, erkânıharp, müddeiumumi, vakayiname
Grok 4.7, wrong: kâtip adil, erkânıharp, müddeiumumi, vekayiname
DeepSeek V4 Pro, wrong: kâtibiadil, erkânıharp, müddeiumumi, vekayiname
DeepSeek V4.1 Flash, wrong: kâtip adli, erkânıharp, müddeiumumi, vakayiname
Kimi K3, wrong: kâtip adil, erkânıharp, müddeiumumi, vakayinâme
Qwen3.8 Max, wrong: kâtibiadil, erkânıharp, müddeiumumi, vekâyiname
Claude Sonnet 5, wrong: kâtip adil, erkânıharp, müddeiumumi, vakayiname
Gemini 3.5 Flash-Lite, wrong: Katip adil, erkânıharb, müddeiumumi, vakanname
GLM-5.3, wrong: kâtibiadl, erkânıharp, müddeiumumi, vegayiname
GLM-5.3 Flash, wrong: kâtip, erkan−ı harb, müddeî umumî, vakaayiname
Qwen3.8 Flash, wrong: no reply

Ежегодный крестный ход с иконой «Знамение» Курской Коренной из курского Знаменского монастыря в Коренную пустынь (к нему исстари приурочивали и открытие Коренской ярмарки) совершается в пятницу, счёт которой ведут от Пасхи. Какая это по счёту пятница после Пасхи? Ответьте одним числом, без пояснений.

The yearly procession with the Kursk Root icon from Kursk to the Korennaya Pustyn monastery is held on a Friday counted from Easter. Which Friday after Easter? Right answer: 9. 16 of 17 models got it right.

GPT-6 Astra, right: 9
Gemini 3.1 Pro, right: 9
GPT-6 Sol, right: 9
Claude Opus 5.5, right: 9
Gemini 3.8 Flash, right: 9
Claude Fable 5.1, wrong: The procession from Kursk to Korennaya Pustyn traditionally takes place on the Friday of the 9th week after Easter (девятая пятница по Пасхе). 9
GPT-6 Luna, right: 9
Grok 4.7, right: 9
DeepSeek V4 Pro, right: 9
DeepSeek V4.1 Flash, right: 9
Kimi K3, right: 9
Qwen3.8 Max, right: 9
Claude Sonnet 5, right: 9
Gemini 3.5 Flash-Lite, right: Девятая
GLM-5.3, right: 9
GLM-5.3 Flash, right: 9
Qwen3.8 Flash, right: 9

Яким жартівливим усталеним порівнянням кажуть про людину, що дуже уважно, пильно приглядається до чого-небудь? Напишіть лише сам порівняльний зворот, що починається словом «як», без дієслова і без пояснень.

Which joking Ukrainian comparison describes someone peering at something very intently? Right answer: як сорока в кістку. 3 of 17 models got it right.

GPT-6 Astra, wrong: як кіт на сало
Gemini 3.1 Pro, right: як сорока в кістку
GPT-6 Sol, right: як сорока в кістку
Claude Opus 5.5, right: як сорока в кістку
Gemini 3.8 Flash, wrong: Як цап на нові ворота
Claude Fable 5.1, wrong: Як баран на нові ворота
GPT-6 Luna, wrong: як кіт на сало
Grok 4.7, wrong: як кіт на сало
DeepSeek V4 Pro, wrong: як кіт на мишу
DeepSeek V4.1 Flash, wrong: як кіт на мишку
Kimi K3, wrong: як баран на нові ворота
Qwen3.8 Max, wrong: як кіт на сало
Claude Sonnet 5, wrong: як кіт на сало
Gemini 3.5 Flash-Lite, wrong: як курка на зерно
GLM-5.3, wrong: як кіт на сметану
GLM-5.3 Flash, wrong: як кіт на сметану
Qwen3.8 Flash, wrong: як ворона бобові

افترِض أنّ حُجّةً شرعيّة حُرِّرت في القاهرة بتاريخ 23 رجب 1219 هـ، محسوبًا بالتقويم الهجري الجدولي (الحسابي) على القواعد الآتية: أوّل المحرّم من سنة 1 هـ يوافق يوم الجمعة 16 يوليو 622م بالتقويم اليولياني، والأشهر الفردية (المحرّم وربيع الأول وجمادى الأولى ورجب ورمضان وذو القعدة) ثلاثون يومًا، والأشهر الزوجية تسعة وعشرون يومًا، ويُزاد يومٌ على ذي الحجة في السنوات الكبيسة، وهي السنوات التي ترتيبها 2 و5 و7 و10 و13 و16 و18 و21 و24 و26 و29 في كل دورة من ثلاثين سنة. ونُصّ في الحجة على أجلٍ يحلّ بعد مئة يوم تمامًا من تاريخ تحريرها (أي تاريخ التحرير مضافًا إليه 100 يوم). في أيّ يوم من أيام الأسبوع، وفي أيّ تاريخ ميلادي بالتقويم الغريغوري، يحلّ هذا الأجل؟ أجب بسطر واحد: اسم اليوم ثم التاريخ بالأرقام على هيئة سنة-شهر-يوم، مثل: الاثنين 1900-01-01، دون أي شرح.

A Cairo deed is dated 23 Rajab 1219 AH on the tabular Islamic calendar, whose rules are given. A term falls due 100 days later. Give its weekday and Gregorian date. Right answer: الثلاثاء 1805-02-05. 11 of 17 models got it right.

GPT-6 Astra, right: الثلاثاء 1805-02-05
Gemini 3.1 Pro, right: الثلاثاء 1805-02-05
GPT-6 Sol, right: الثلاثاء 1805-02-05
Claude Opus 5.5, right: **الثلاثاء 1805-02-05**
Gemini 3.8 Flash, right: الثلاثاء 1805-02-05
Claude Fable 5.1, right: الثلاثاء 1805-02-05
GPT-6 Luna, wrong: الاثنين 1805-02-04
Grok 4.7, right: الثلاثاء 1805-02-05
DeepSeek V4 Pro, wrong: no reply
DeepSeek V4.1 Flash, right: الثلاثاء 1805-02-05
Kimi K3, right: الثلاثاء 1805-02-05
Qwen3.8 Max, right: الثلاثاء 1805-02-05
Claude Sonnet 5, right: الثلاثاء 1805-02-05
Gemini 3.5 Flash-Lite, wrong: الخميس 1804-12-01
GLM-5.3, wrong: no reply
GLM-5.3 Flash, wrong: no reply
Qwen3.8 Flash, wrong: no reply

در شهر کوهپایه‌ای مُجِن در شهرستان شاهرود (استان سمنان)، زنان در روزهای پایانی اسفند با خاک سرخ و سفیدی که از ارتفاعات پیرامون شهر می‌آورند، دیوارهای خانه‌های خشتی را از نو اندود می‌کنند و این‌گونه به پیشواز بهار می‌روند. مردم مجن در گویش محلی این رسم را چه می‌نامند؟ فقط نام محلی این رسم را بنویسید، بدون توضیح.

In the hill town of Mojen near Shahrud, women replaster their mud-brick walls with red and white earth before spring. What do locals call the custom? Right answer: گل هاکنی. 1 of 17 models got it right.

GPT-6 Astra, right: گِل هاکُنی
Gemini 3.1 Pro, wrong: گل هاکردن
GPT-6 Sol, wrong: گل‌مالی
Claude Opus 5.5, wrong: گِل‌مالی
Gemini 3.8 Flash, wrong: گِل‌کاری (گِل‌مالی)
Claude Fable 5.1, wrong: گل‌مالی
GPT-6 Luna, wrong: گِل‌مالی
Grok 4.7, wrong: سرخه‌مال
DeepSeek V4 Pro, wrong: گِل‌اندازی
DeepSeek V4.1 Flash, wrong: گل‌مالی
Kimi K3, wrong: مل‌کوبی
Qwen3.8 Max, wrong: گِل‌مالون
Claude Sonnet 5, wrong: گُل‌مالی
Gemini 3.5 Flash-Lite, wrong: گل‌کاری
GLM-5.3, wrong: فیل‌مالی
GLM-5.3 Flash, wrong: گِل‌مالی (گل اندود کردن دیوارها)
Qwen3.8 Flash, wrong: خاک‌ریزی (کول‌زنی / خانه‌تکونی)

हिंदी शब्दसागर (नागरी प्रचारिणी सभा) में दर्ज उस मुहावरे को लिखिए जो ऐसे व्यक्ति के लिए कहा जाता है जो किसी अप्रिय या खटकेवाली बात को दिन-रात देखते-सुनते उसका इतना आदी हो गया हो कि अब किसी आहट या खटके से न चौंकता है, न घबराता है। केवल मुहावरा लिखिए, कोई व्याख्या नहीं।

Which idiom in the Hindi Shabdsagar describes someone so used to a jarring noise that it no longer startles them? Right answer: ठठेरे की बिल्ली. 0 of 17 models got it right.

GPT-6 Astra, wrong: तोप का घोड़ा
Gemini 3.1 Pro, wrong: चक्की का कुत्ता
GPT-6 Sol, wrong: रोज का मरना कौन डरे।
Claude Opus 5.5, wrong: खड़के का घोड़ा
Gemini 3.8 Flash, wrong: कंसाई के पाड़े
Claude Fable 5.1, wrong: हींग लगे न फिटकरी... no. The idiom: "कानों पर जूं न रेंगना"? Not exactly. The Hindi Shabdsagar entry: "आँख का पानी..."? The described sense: not startled by any sound — "कान पक जाना"? Actually Shabdsagar has "अंधे के आगे रोना"? Let me think…
GPT-6 Luna, wrong: खटके का आदी होना
Grok 4.7, wrong: आहट का पक्का
DeepSeek V4 Pro, wrong: no reply
DeepSeek V4.1 Flash, wrong: कान पकना
Kimi K3, wrong: कान में जूँ न रेंगना
Qwen3.8 Max, wrong: खटके का आदी होना
Claude Sonnet 5, wrong: नित की खटक रहना
Gemini 3.5 Flash-Lite, wrong: गोल हो जाना
GLM-5.3, wrong: जो बसे सो न तरसे।
GLM-5.3 Flash, wrong: कान पर जूँ तक न रेंगना
Qwen3.8 Flash, wrong: आदत का फल खटका

Tuliskan frasa "kebudayaan kebangsaan untuk rakyat yang makmur" dalam Ejaan van Ophuijsen (ejaan resmi bahasa Melayu di Hindia Belanda tahun 1901 sampai 1947), lengkap dengan tanda diakritik yang dipakai ejaan itu. Jawab hanya dengan frasa hasil alih ejaannya dalam huruf kecil, tanpa penjelasan.

Write "national culture for a prosperous people" in the van Ophuijsen spelling used from 1901 to 1947, diacritics included. Right answer: keboedajaän kebangsaän oentoek ra'jat jang ma'moer. 8 of 17 models got it right.

GPT-6 Astra, wrong: kĕboedajaan kĕbangsaan oentoek ra'jat jang ma'moer
Gemini 3.1 Pro, right: keboedajaän kebangsaän oentoek ra'jat jang makmoer
GPT-6 Sol, right: keboedajaan kebangsaan oentoek rakjat jang makmoer
Claude Opus 5.5, right: keboedajaän kebangsaän oentoek ra'jat jang ma'moer
Gemini 3.8 Flash, right: keboedajaän kebangsaän oentoek ra'jat jang ma'moer
Claude Fable 5.1, right: Van Ophuijsen spelling: "kebudayaan" → "keboedajaan"; "kebangsaan" → "kebangsaan"; "untuk" → "oentoek"; "rakyat" → "ra'jat" (hamzah/ain marked with apostrophe); "yang" → "jang"; "makmur" → "ma'moer" (ain). Diacritics: trema used on vowels l…
GPT-6 Luna, wrong: keboeajaan kebangsaan oentoek rakjat jang makmoer
Grok 4.7, wrong: kĕboedajaan kĕbangsaan oentoek rakjat jang makmoer
DeepSeek V4 Pro, right: keboedajaan kebangsaan oentoek rakjat jang makmoer
DeepSeek V4.1 Flash, right: keboedajaan kebangsaan oentoek rakjat jang makmoer
Kimi K3, wrong: kĕboedajaan kĕbangsaan oentoek rakjat jang makmoer
Qwen3.8 Max, wrong: keboedajaan kebangsaan oentoek rajat jang makmoer
Claude Sonnet 5, wrong: kaboedajaan kebangsa'an oentoek ra'jat jang makmoer
Gemini 3.5 Flash-Lite, wrong: kebudajaan kebângsaan oentoek rakjat jang makmoer
GLM-5.3, right: keboedajaan kebangsaan oentoek rakjat jang makmoer
GLM-5.3 Flash, wrong: keboedajaan kebangsaan untuk rakjat jang makmoer
Qwen3.8 Flash, wrong: këboedajaän këbangsaan oentoek rajakat jang makmoer

群馬県の郷土かるた「上毛かるた」で、2024年（令和6年）4月以降に出荷されるかるたに入っている「ち」の読み札の文言を答えてください。読み札の文言だけを書き、説明は不要です。

In Gunma's local card game Jōmō Karuta, what does the "chi" card say on sets shipped from April 2024? Right answer: 力あわせる百九十万. 4 of 17 models got it right.

GPT-6 Astra, right: 力あわせる百九十万
Gemini 3.1 Pro, right: 力あわせる百九十万
GPT-6 Sol, right: 力あわせる百九十万
Claude Opus 5.5, wrong: 力あわせる群馬県
Gemini 3.8 Flash, wrong: 力あわせる二百万
Claude Fable 5.1, wrong: 力あわせる二百万
GPT-6 Luna, wrong: 力あわせる二百万
Grok 4.7, wrong: 力を一つに県民の和
DeepSeek V4 Pro, wrong: 地域をつなぐ上毛かるた
DeepSeek V4.1 Flash, wrong: ちからをあわせ未来へはばたく上州人
Kimi K3, wrong: 知と技のものづくりで世界に挑む
Qwen3.8 Max, right: 力合わせる百九十万
Claude Sonnet 5, wrong: 力あわせる二百万
Gemini 3.5 Flash-Lite, wrong: 日本のあらたまつげちからあふれ
GLM-5.3, wrong: 蓄電池で拓く新時代　群馬の挑戦
GLM-5.3 Flash, wrong: 知恵すぐれて　荻原井泉水
Qwen3.8 Flash, wrong: no reply

다음 뜻풀이에 해당하는 속담을 표준국어대사전에 실린 형태 그대로 쓰시오. 뜻: 모두가 경황없이 바쁘고 어수선한 때에, 그 일과는 전혀 상관없는 엉뚱한 일을 해 달라고 남에게 조르는 경우를 비유적으로 이르는 말. 설명 없이 속담만 답하시오.

Which proverb in the Standard Korean Language Dictionary describes pestering someone for an unrelated favour while everyone is frantic? Right answer: 상여 나갈 때 귀청 내 달란다. 0 of 17 models got it right.

GPT-6 Astra, wrong: 불난 집에 와서 담뱃불 달란다
Gemini 3.1 Pro, wrong: 불난 데 떡 해 달란다
GPT-6 Sol, wrong: 불난 집에 가서 바늘 달라 한다
Claude Opus 5.5, wrong: 불난 집에 가서 수수 볶아 달란다
Gemini 3.8 Flash, wrong: 불난 데 부채질한다 (Wait, check: "모두가 경황없이 바쁘고 어수선한 때에, 그 일과는 전혀 상관없는 엉뚱한 일을 해 달라고 남에게 조르는 경우를 비유적으로 이르는 말.") Could it be "불난 끝에 도둑이 든다"? No. Could it be "난리 통에 엿 사 먹는다"? Wait, let's search: "모두가 경황없이 바쁘고 어수선한 때에, 그 일과는 전혀 상관없는 엉뚱한 일을 해 달라고 남에게 조르는…
Claude Fable 5.1, wrong: 불난 집에서 불 좀 빌리자고 한다.
GPT-6 Luna, wrong: 불난 집에 부채질한다.
Grok 4.7, wrong: 불난 집에 부채 빌린다
DeepSeek V4 Pro, wrong: 오뉴월 감기에 보리죽 달랜다
DeepSeek V4.1 Flash, wrong: 초상집에 가서 떡 사 달라고 한다
Kimi K3, wrong: 바쁜 마당에 콩 볶아 먹자 한다
Qwen3.8 Max, wrong: 바쁜 집에 초상 치르듯
Claude Sonnet 5, wrong: 혼인집에서 신랑 잡아 들여라 하는 격
Gemini 3.5 Flash-Lite, wrong: 잔칫집에 소 잡는 소리 한다
GLM-5.3, wrong: 눈코 뜰 새 없이 바쁜데 아이 보는 부탁이냐
GLM-5.3 Flash, wrong: 소 잃고 외양간 고친다가 아니라, "불난 집 부러워 먹는다"도 아니고… 정답은 **쇠뿔도 단숨에 한다**가 아니라, 표준국어대사전 뜻풀이 "아침에 심고 저녁에 국솎이 되랴"…가 아닌, 정답: **가는 말이 고와야 오는 말이 곱다**가 아니라 — 정답: **돌당귀에 수금당귀 온다** 아니, 정확한 표준국어대사전 형태는: **불똥이 아니 튀면 왜 무섭냐**… 죄송합니다, 확실한 답은: **남의 떡이 커 보인다**가 아니라 —…
Qwen3.8 Flash, wrong: 물레방앗간에서 절을 한다

หนังสือราชการสยามฉบับหนึ่งลงวันที่ว่า "วันที่ ๑๕ กุมภาพันธ์ รัตนโกสินทรศก ๑๑๗" โดยนับวัน เดือน ปี ตามแบบที่ราชการสยามใช้อยู่ในเวลานั้น วันดังกล่าวตรงกับวันอะไรในสัปดาห์ และปีจุลศักราชที่ตรงกับวันนั้นเรียกว่าศกอะไร (ชื่อศกตามเลขท้ายปีจุลศักราช) ตอบในรูปแบบ "วัน... ...ศก" เช่น "วันจันทร์ เอกศก" เท่านั้น ไม่ต้องอธิบาย

A Siamese government letter is dated 15 February, Rattanakosin Era 117, counted the way Siam did then. Which weekday was it, and what was that Chula Sakarat year called? Right answer: วันพุธ สัมฤทธิศก. 10 of 17 models got it right.

GPT-6 Astra, right: วันพุธ สัมฤทธิศก
Gemini 3.1 Pro, right: วันพุธ สัมฤทธิศก
GPT-6 Sol, right: วันพุธ สัมฤทธิศก
Claude Opus 5.5, right: วันพุธ สัมฤทธิศก
Gemini 3.8 Flash, right: วันพุธ สัมฤทธิศก
Claude Fable 5.1, wrong: R.S. 117 began 1 April 1898 and ended 31 March 1899. So 15 February R.S. 117 = 15 February 1899 (Gregorian, Siam adopted Gregorian calendar in 1889 with R.S. 108). Day of week for 15 Feb 1899: 1 Jan 1899 was a Sunday. Jan has 31 days; Feb …
GPT-6 Luna, right: วันพุธ สัมฤทธิศก
Grok 4.7, right: วันพุธ สัมฤทธิศก
DeepSeek V4 Pro, right: วันพุธ สัมฤทธิศก
DeepSeek V4.1 Flash, wrong: no reply
Kimi K3, wrong: วันพุธ เอกศก
Qwen3.8 Max, right: วันพุธ สัมฤทธิศก
Claude Sonnet 5, wrong: วันพฤหัสบดี นพศก
Gemini 3.5 Flash-Lite, right: วันพุธ สัมฤทธิศก
GLM-5.3, wrong: วันพุธ สัมศก
GLM-5.3 Flash, wrong: วันพุธ เอกศก
Qwen3.8 Flash, wrong: no reply

Lễ hội Trò Trám, tổ chức ngày 11 và 12 tháng Giêng âm lịch tại xã Phùng Nguyên, tỉnh Phú Thọ, còn được người dân gọi bằng một tên khác. Tên gọi khác đó là gì? Chỉ trả lời tên gọi, không giải thích.

The Trò Trám festival, held on the 11th and 12th of the first lunar month in Phùng Nguyên, Phú Thọ, has another popular name. What is it? Right answer: Linh tinh tình phộc. 10 of 17 models got it right.

GPT-6 Astra, right: Lễ hội Linh tinh tình phộc
Gemini 3.1 Pro, right: Lễ hội Linh tinh tình phộc
GPT-6 Sol, right: Lễ hội Linh tinh tình phộc
Claude Opus 5.5, right: Linh tinh tình phộc
Gemini 3.8 Flash, wrong: Lễ hội "Linh tinh tình phộc"
Claude Fable 5.1, right: Lễ hội Linh tinh tình phộc
GPT-6 Luna, right: Lễ hội Linh tinh tình phộc
Grok 4.7, wrong: Hội Trám
DeepSeek V4 Pro, right: Lễ hội Linh tinh tình phộc
DeepSeek V4.1 Flash, right: Lễ hội Linh tinh tình phộc
Kimi K3, wrong: Lễ hội Bắt cá ông Táo
Qwen3.8 Max, right: Linh Tinh tình phộc
Claude Sonnet 5, right: Lễ hội Linh tinh tình phộc
Gemini 3.5 Flash-Lite, wrong: Lễ hội Trám Trám
GLM-5.3, wrong: Lễ hội Rước kiệu, chữa lửa (Xuân nả/ná lửa)
GLM-5.3 Flash, wrong: Hội Vua Ba
Qwen3.8 Flash, wrong: Lễ hội Tứ Dân

有一个四字成语，语本东汉应劭《风俗通义》的逸文，用来比喻企图同时占有两方面的好处、贪得无厌。请写出这个成语，只回答成语本身，不要解释。

Which four-character idiom, from a lost passage of Ying Shao's Fengsu Tongyi, describes greedily wanting both benefits at once? Right answer: 东食西宿. 11 of 17 models got it right.

GPT-6 Astra, right: 东食西宿
Gemini 3.1 Pro, right: 东食西宿
GPT-6 Sol, right: 东食西宿
Claude Opus 5.5, right: 东食西宿
Gemini 3.8 Flash, right: 东食西宿
Claude Fable 5.1, wrong: 得陇望蜀
GPT-6 Luna, wrong: 鱼与熊掌
Grok 4.7, right: 东食西宿
DeepSeek V4 Pro, right: 东食西宿
DeepSeek V4.1 Flash, right: 东食西宿
Kimi K3, right: 东食西宿
Qwen3.8 Max, wrong: <think> 用户问的是一个四字成语，语本东汉应劭《风俗通义》的逸文，用来比喻企图同时占有两方面的好处、贪得无厌。让我想想《风俗通义》逸文中的成语。 "一石二鸟"？不是。 "一箭双雕"？不是，出自北朝。 "一举两得"？不是。 "得陇望蜀"？不是，出自《后汉书》。让我想想"一石二鸟"... 等等，"一石二鸟"不是四字成语吗？是的，但出处不对。让我再想想... 比喻企图同时占有两方面的好处、贪得无厌... "一石二鸟"是比喻做一件事达到两个目的，不完全是"贪得…
Claude Sonnet 5, wrong: 蚕食鲸吞
Gemini 3.5 Flash-Lite, wrong: 两全其美
GLM-5.3, wrong: 兼而有之？No — "脚踏两只船"? Actually 语本《风俗通义》"齐人攫金"? The idiom: "贪得无厌"… 逸文 "两头大"? It's "一女不事二夫"? The idiom is "得陇望蜀"? 语本《风俗通义》… I think it's "东食西宿".东食西宿
GLM-5.3 Flash, right: 东食西宿
Qwen3.8 Flash, right: 东食西宿

下面這句話混用了異體字。請依教育部《異體字字典》，把句中每一個異體字都改成它所屬的正字（臺灣的標準字體），其餘文字與標點一律不變。句子：「衆人圍在竈邊敍舊，唇上還沾着麪屑。」請只回答改好的整句，不要解釋。

Replace every variant character in a sentence with its standard Taiwanese form, following the Ministry of Education's variants dictionary. Right answer: 眾人圍在灶邊敘舊，脣上還沾著麵屑。. 7 of 17 models got it right.

GPT-6 Astra, right: 眾人圍在灶邊敘舊，脣上還沾著麵屑。
Gemini 3.1 Pro, right: 眾人圍在灶邊敘舊，脣上還沾著麵屑。
GPT-6 Sol, right: 眾人圍在灶邊敘舊，脣上還沾著麵屑。
Claude Opus 5.5, right: 「眾人圍在灶邊敘舊，脣上還沾著麵屑。」
Gemini 3.8 Flash, right: 眾人圍在灶邊敘舊，脣上還沾著麵屑。
Claude Fable 5.1, right: 眾人圍在灶邊敘舊，脣上還沾著麵屑。
GPT-6 Luna, wrong: 「眾人圍在灶邊敘舊，唇上還沾著麵屑。」
Grok 4.7, wrong: 眾人圍在灶邊敘舊，唇上還沾著麵屑。
DeepSeek V4 Pro, wrong: 眾人圍在灶邊敘舊，唇上還沾著麵屑。
DeepSeek V4.1 Flash, right: 眾人圍在灶邊敘舊，脣上還沾著麵屑。
Kimi K3, wrong: 「眾人圍在灶邊敘舊，唇上還沾著麵屑。」
Qwen3.8 Max, wrong: 「眾人圍在灶邊敘舊，唇上還沾着麵屑。」
Claude Sonnet 5, wrong: 眾人圍在灶邊敘舊，唇上還沾著麵屑。
Gemini 3.5 Flash-Lite, wrong: 眾人圍在竈邊敘舊，唇上還沾著麵屑。
GLM-5.3, wrong: 眾人圍在灶邊敘舊，唇上還沾著麵屑。
GLM-5.3 Flash, wrong: 眾人圍在灶邊敘舊，唇上還沾著麵屑。
Qwen3.8 Flash, wrong: 「眾人圍在灶邊敘舊，唇上還沾著麵屑。」

Obiceiul agrar de primăvară «Tânjaua de pe Mara», la care este sărbătorit cel mai harnic gospodar, primul ieșit la arat, se ține în fiecare an într-un sat din județul Maramureș. Cum se numește satul? Răspundeți doar cu numele satului, fără explicații.

The spring custom "Tânjaua de pe Mara", honouring the first farmer out ploughing, is held every year in a Maramureș village. Which one? Right answer: Hoteni. 12 of 17 models got it right.

GPT-6 Astra, right: Hoteni
Gemini 3.1 Pro, right: Hoteni
GPT-6 Sol, right: Hoteni
Claude Opus 5.5, right: Hoteni
Gemini 3.8 Flash, wrong: Hărnicești
Claude Fable 5.1, right: Hoteni
GPT-6 Luna, right: Hoteni
Grok 4.7, right: Hoteni
DeepSeek V4 Pro, right: Hoteni
DeepSeek V4.1 Flash, right: Hoteni
Kimi K3, right: Hoteni
Qwen3.8 Max, right: Hoteni
Claude Sonnet 5, wrong: Șieu
Gemini 3.5 Flash-Lite, wrong: Vadu Izei
GLM-5.3, wrong: Hărnicești? Actually: Giulești? — The village is Giulești.
GLM-5.3 Flash, right: Hoteni
Qwen3.8 Flash, wrong: Băiuț

Apakah peribahasa Melayu yang bermaksud "mencari keuntungan dalam lingkungan keluarga atau kawan sendiri"? Tulis peribahasa itu sahaja, tanpa penjelasan.

Which Malay proverb means seeking profit from your own family or friends? Right answer: Memancing dalam belanga. 0 of 17 models got it right.

GPT-6 Astra, wrong: Mengail dalam belanga.
Gemini 3.1 Pro, wrong: Makan kawan
GPT-6 Sol, wrong: Mengail dalam belanga.
Claude Opus 5.5, wrong: Mengail dalam belanga
Gemini 3.8 Flash, wrong: Menepung tiung
Claude Fable 5.1, wrong: Mencari rezeki di celah-celah kaum sendiri... Hmm, let me think of the actual known proverb. The Malay proverb meaning "mencari keuntungan dalam kalangan keluarga/kawan sendiri" is "Berkelahi dalam mimpi"? No. Common one: "Mengaut rezeki d…
GPT-6 Luna, wrong: Makan kawan
Grok 4.7, wrong: Kawan makan kawan, saudara makan saudara
DeepSeek V4 Pro, wrong: makan kawan
DeepSeek V4.1 Flash, wrong: Makan kawan
Kimi K3, wrong: Bagai tikus membaiki labu
Qwen3.8 Max, wrong: Makan di dalam periuk sendiri.
Claude Sonnet 5, wrong: Hangat-hangat tahi ayam
Gemini 3.5 Flash-Lite, wrong: Makan anak saudara
GLM-5.3, wrong: Bagana makan bagana
GLM-5.3 Flash, wrong: Bagai menangguk di tasik jernih
Qwen3.8 Flash, wrong: Berjalan di atas kepala kawan

বাংলাদেশে বাংলা একাডেমির সংশোধিত বাংলা বর্ষপঞ্জি (যে সংশোধন ১৪২৬ বঙ্গাব্দের আশ্বিন মাস থেকে কার্যকর) অনুযায়ী ৬ মার্চ ২০২৮ খ্রিষ্টাব্দ কোন বাংলা তারিখ, এবং সেদিন কী বার? তারিখ অঙ্কে লিখুন, এই ছাঁদে: তারিখ মাস বঙ্গাব্দ, বার। শুধু উত্তরটি লিখুন, কোনো ব্যাখ্যা দেবেন না।

Under the Bangla Academy's revised calendar used in Bangladesh, which Bengali date is 6 March 2028, and which weekday? Right answer: ২২ ফাল্গুন ১৪৩৪, সোমবার. 11 of 17 models got it right.

GPT-6 Astra, right: ২২ ফাল্গুন ১৪৩৪, সোমবার।
Gemini 3.1 Pro, right: ২২ ফাল্গুন ১৪৩৪, সোমবার
GPT-6 Sol, right: ২২ ফাল্গুন ১৪৩৪, সোমবার।
Claude Opus 5.5, right: ২২ ফাল্গুন ১৪৩৪, সোমবার
Gemini 3.8 Flash, right: ২২ ফাল্গুন ১৪৩৪, সোমবার
Claude Fable 5.1, wrong: ২৩ ফাল্গুন ১৪৩৪, সোমবার
GPT-6 Luna, right: ২২ ফাল্গুন ১৪৩৪, সোমবার.
Grok 4.7, wrong: ২৩ ফাল্গুন ১৪৩৪, সোমবার
DeepSeek V4 Pro, right: ২২ ফাল্গুন ১৪৩৪, সোমবার
DeepSeek V4.1 Flash, right: ২২ ফাল্গুন ১৪৩৪ বঙ্গাব্দ, সোমবার
Kimi K3, wrong: ২৩ ফাল্গুন ১৪৩৪, সোমবার।
Qwen3.8 Max, right: ২২ ফাল্গুন ১৪৩৪, সোমবার।
Claude Sonnet 5, wrong: ২৪ ফাল্গুন ১৪৩৪, সোমবার
Gemini 3.5 Flash-Lite, right: ২২ ফাল্গুন ১৪৩৪, সোমবার
GLM-5.3, right: ২২ ফাল্গুন ১৪৩৪ বঙ্গাব্দ, সোমবার।
GLM-5.3 Flash, wrong: ২৩ ফাল্গুন ১৪৩৪, সোমবার।
Qwen3.8 Flash, wrong: no reply

جنوبی پنجاب اور سرائیکی وسیب میں لوگ ہر سال ایک مقررہ تاریخ کو سرائیکی ثقافتی دن (جسے سرائیکی اجرک ڈے بھی کہتے ہیں) مناتے ہیں اور نیلی سرائیکی اجرک پہنتے ہیں۔ یہ دن ہر سال کس تاریخ کو منایا جاتا ہے؟ جواب میں صرف تاریخ اور مہینے کا نام لکھیے، مثلاً "12 جنوری"، کوئی وضاحت نہیں۔

On which date each year do people in South Punjab mark Saraiki Culture Day and wear the blue Saraiki ajrak? Right answer: 6 مارچ. 13 of 17 models got it right.

GPT-6 Astra, right: 6 مارچ
Gemini 3.1 Pro, right: 6 مارچ
GPT-6 Sol, right: 6 مارچ
Claude Opus 5.5, right: 6 مارچ
Gemini 3.8 Flash, right: 6 مارچ
Claude Fable 5.1, right: 6 مارچ
GPT-6 Luna, right: 6 مارچ
Grok 4.7, right: 6 مارچ
DeepSeek V4 Pro, right: 6 مارچ
DeepSeek V4.1 Flash, right: 6 مارچ
Kimi K3, right: 6 مارچ
Qwen3.8 Max, right: 6 مارچ
Claude Sonnet 5, wrong: 14 مارچ
Gemini 3.5 Flash-Lite, wrong: یکم مارچ
GLM-5.3, wrong: 21 دسمبر
GLM-5.3 Flash, right: 6 مارچ
Qwen3.8 Flash, wrong: 1 دسمبر

איזה ניב, שמקורו בתלמוד הבבלי ושרווח בספרות העברית החדשה (למשל אצל ביאליק), נאמר על שטר, תעודה או מסמך שאין בהם שום ערך, כמו שטרי חוב של מי שפשט את הרגל? כתבו את הניב בצורת שם הפועל שבה הוא מובא במילון, ללא ניקוד וללא הסבר.

Which Talmudic idiom, used by Bialik, describes a worthless document such as a bankrupt's IOUs? Right answer: לצור על פי צלוחית. 5 of 17 models got it right.

GPT-6 Astra, right: לצור על פי צלוחית
Gemini 3.1 Pro, right: לצור על פי צלוחיתו
GPT-6 Sol, right: לצור על פי צלוחית
Claude Opus 5.5, right: לצור על פי צלוחיתו
Gemini 3.8 Flash, wrong: לצרוף על פי צלוחיתו
Claude Fable 5.1, wrong: The idiom: "לצור על פי צלוחיתו" — meaning worthless (something only good to use as a stopper for a flask). From Talmud (Bavli, e.g., "לא יצור בו על פי צלוחיתו"? Actually "אין בו כדי לצור על פי צלוחיתו"). Infinitive form: "לצור על פי צלוחיתו…
GPT-6 Luna, wrong: להיות חספא בעלמא
Grok 4.7, right: לצור על פי צלוחית
DeepSeek V4 Pro, wrong: no reply
DeepSeek V4.1 Flash, wrong: no reply
Kimi K3, wrong: מאנה דבי כסי
Qwen3.8 Max, wrong: לאסמכתא
Claude Sonnet 5, wrong: נייר שאינו שווה הנייר שעליו הוא כתוב
Gemini 3.5 Flash-Lite, wrong: כחרס הנשבר
GLM-5.3, wrong: **
GLM-5.3 Flash, wrong: עשה שטרותיו חרם
Qwen3.8 Flash, wrong: שטר פסול

I det äldre svenska skriftspråket (till exempel i Bibel 1917 och fram till mitten av 1900-talet) hade de starka verben särskilda pluralformer i preteritum indikativ och en egen preteritum konjunktiv. Sätt in de tre verben i meningen nedan i de starka former som det äldre skriftspråket krävde: det första och det andra i preteritum indikativ plural, det tredje i preteritum konjunktiv. "Medan gesällerna _ (gjuta) den nya kyrkklockan och lärpojkarna _ (kvida) av trötthet, önskade mästaren att natten ___ (skrida) fortare." Svara med endast de tre verbformerna i ordning, åtskilda av kommatecken, utan förklaring.

Fill in three strong verbs in the older written Swedish of the 1917 Bible: two past plurals and one past subjunctive. Right answer: göto, kvedo, skrede. 7 of 17 models got it right.

GPT-6 Astra, right: göto, kvedo, skrede
Gemini 3.1 Pro, right: göto, kvedo, skrede
GPT-6 Sol, right: göto, kvedo, skrede
Claude Opus 5.5, wrong: guto, kvido, skrede
Gemini 3.8 Flash, wrong: göto, kvido, skrede
Claude Fable 5.1, wrong: göto, kvido, skrede
GPT-6 Luna, right: göto, kvedo, skrede
Grok 4.7, right: göto, kvedo, skrede
DeepSeek V4 Pro, right: göto, kvedo, skrede
DeepSeek V4.1 Flash, wrong: gjöto, kvedo, skrede
Kimi K3, wrong: göto, kvedo, skride
Qwen3.8 Max, wrong: göto, kvido, skrede
Claude Sonnet 5, right: göto, kvedo, skrede
Gemini 3.5 Flash-Lite, wrong: gjuto, kvodo, skridede
GLM-5.3, wrong: goto, kvado, skride
GLM-5.3 Flash, wrong: gjöto, kvädo, skredo
Qwen3.8 Flash, wrong: gjutu, kvidu, skride

Språkrådet har en offisiell liste over innbyggjarnamn i Norge. Hva kalles en person fra landskapet Solør (Grue, Åsnes og Våler i Innlandet) ifølge denne lista? Svar bare med ordet i ubestemt form entall, uten forklaring.

What is someone from the Solør district called, according to the Language Council of Norway's list of demonyms? Right answer: solung. 8 of 17 models got it right.

GPT-6 Astra, right: solung
Gemini 3.1 Pro, right: solung
GPT-6 Sol, right: solung
Claude Opus 5.5, wrong: solunging
Gemini 3.8 Flash, right: solung
Claude Fable 5.1, right: Solung
GPT-6 Luna, right: solung
Grok 4.7, right: solung
DeepSeek V4 Pro, right: solung
DeepSeek V4.1 Flash, wrong: solørning
Kimi K3, wrong: soløring
Qwen3.8 Max, wrong: soløring
Claude Sonnet 5, wrong: solunge
Gemini 3.5 Flash-Lite, wrong: soløring
GLM-5.3, wrong: Soløring
GLM-5.3 Flash, wrong: Sørling
Qwen3.8 Flash, wrong: soluble

Hvilket dansk ordsprog bruger man om, at også den, der arbejder langsomt eller har ringe evner og midler, til sidst når sit mål? Svar kun med ordsproget, uden forklaring.

Which Danish proverb says that even the slow and poorly equipped get there in the end? Right answer: den der ager med stude, kommer også frem. 1 of 17 models got it right.

GPT-6 Astra, right: Den, der ager med stude, kommer også frem.
Gemini 3.1 Pro, wrong: Med tiden kommer sneglen også til Jerusalem.
GPT-6 Sol, wrong: Den, der kryber, kommer også frem.
Claude Opus 5.5, wrong: Langsom kører også.
Gemini 3.8 Flash, wrong: Langsomme heste kommer også til byen.
Claude Fable 5.1, wrong: Den, der går stille, kommer også frem.
GPT-6 Luna, wrong: Langsomt men sikkert.
Grok 4.7, wrong: Stadig dryppen huler stenen.
DeepSeek V4 Pro, wrong: Sneglen kom også med i arken.
DeepSeek V4.1 Flash, wrong: Lidt efter lidt, så kommer man vidt.
Kimi K3, wrong: Langsomt, men sikkert.
Qwen3.8 Max, wrong: Langsomt, men sikkert.
Claude Sonnet 5, wrong: Lidt efter lidt æder ræven høns.
Gemini 3.5 Flash-Lite, wrong: Lidt men godt kommer også aften
GLM-5.3, wrong: "Langsomt men sikkert" — ordsproget er: "Også den trætte hest kommer til vands."
GLM-5.3 Flash, wrong: Skræppen tog også til Århus, sagde huggebrodden.
Qwen3.8 Flash, wrong: Den, der kører langsomt, kommer også frem.

Slovo hrabě mělo ve starší češtině podle Slovníku spisovného jazyka českého (SSJČ) kromě dnešního rodu ještě jiný, zastaralý rod (dokládá ho např. u Klicpery). Přepište větu „Ten starý hrabě byl už dlouho nemocen.“ tak, aby všechna slova, která se s podmětem shodují, odpovídala tomuto zastaralému rodu. Jmenný (krátký) tvar přídavného jména ponechte jmenný a pořadí slov neměňte. Odpovězte pouze přepsanou větou, bez vysvětlení.

Older Czech treated hrabě (count) as neuter. Rewrite "The old count had long been ill" so every agreeing word takes that old gender. Right answer: To staré hrabě bylo už dlouho nemocno.. 10 of 17 models got it right.

GPT-6 Astra, right: To staré hrabě bylo už dlouho nemocno.
Gemini 3.1 Pro, right: To staré hrabě bylo už dlouho nemocno.
GPT-6 Sol, right: To staré hrabě bylo už dlouho nemocno.
Claude Opus 5.5, right: To staré hrabě bylo už dlouho nemocno.
Gemini 3.8 Flash, right: To staré hrabě bylo už dlouho nemocno.
Claude Fable 5.1, right: SSJČ: hrabě, -te m. (†s., Klic.) — tedy zastaralé neutrum. Shoda v neutru: To staré hrabě bylo už dlouho nemocno. To staré hrabě bylo už dlouho nemocno.
GPT-6 Luna, right: To staré hrabě bylo už dlouho nemocno.
Grok 4.7, wrong: Ta stará hrabě byla už dlouho nemocna.
DeepSeek V4 Pro, right: To staré hrabě bylo už dlouho nemocno.
DeepSeek V4.1 Flash, right: To staré hrabě bylo už dlouho nemocno.
Kimi K3, wrong: Ta stará hrabě byla už dlouho nemocna.
Qwen3.8 Max, wrong: Ta stará hrabě byla už dlouho nemocna.
Claude Sonnet 5, wrong: Ta stará hrabě byla už dlouho nemocna.
Gemini 3.5 Flash-Lite, wrong: Ta stará hrabě byla už dlouho nemocna.
GLM-5.3, wrong: Ta stará hrabě byla už dlouho nemocna.
GLM-5.3 Flash, wrong: Ta stará hrabě byla už dlouho nemocna.
Qwen3.8 Flash, right: To staré hrabě bylo už dlouho nemocno.

Back to the Multilingual LLM Bench leaderboard