Best AI for Arabic: Claude Opus leads the big chatbots on Stanford's Arabic test

Which AI is best for Arabic?

Claude Opus is the best of the big chatbots for Arabic, and two Arabic specialists beat it. Stanford's CRFM runs HELM Arabic with Arabic.AI. It scores 41 models on seven Arabic test sets, from school exam questions to answers built on retrieved documents. Here's the release of 25 June 2026, mean score out of 100:

The top three general chatbots sit within two points of each other. Claude Opus leads them at 84.1, and GPT and Gemini Flash tie at 82.5. Above them, Arabic.AI's LLM-X scores 91.9 and a model listed as incept-arabic-1 scores 88.3. Below them sit the older Arabic models: ALLaM 7B at 70.8 and the 30B Jais family at 64.5. Arabic.AI is also Stanford's partner on the leaderboard, so weigh its top spot with that in mind.

Read the chart with one caveat. HELM switches thinking off, and it leaves out any model that can't turn it off. No Gemini Pro model is on it, so Google is there only as Flash. The GPT-6 family and Jais 2 aren't on it either. So Claude Opus leads the chatbots Stanford tested, not every chatbot, and in our own test below GPT Astra tied it. For the wider fight between the three big assistants, see ChatGPT vs Claude vs Gemini.

How did 17 models do on Whizi's own Arabic questions?

GPT Astra and Claude Opus tied at the top, with 4 of 5 Arabic questions right. That's from our Multilingual LLM Bench, run on 2 October 2026. Every model got the same 5 questions in each of 30 languages, at low reasoning. Each answer was graded against a sourced key, and an answer in the wrong language scored zero.

Five questions is a small sample. One question moves a model's Arabic score by 20 points, so treat this as a rough order, not a ranking.

The split by question type is the part no leaderboard shows. Facts and date maths were fine. Proverbs weren't:

Each proverb question describes a saying by its meaning and asks for the saying itself. Fifteen models missed the first, sixteen missed the second, and none got both. Claude Opus got one. GPT Astra and Claude Fable got the other.

The calendar question is the one Arabic item we publish. It dates a Cairo deed 23 Rajab 1219 AH on the tabular Hijri calendar, gives the rules, and asks for the weekday and Gregorian date 100 days later. Eleven of 17 models answered Tuesday, 5 February 1805.

Don't use the bench to compare Arabic with other languages. Arabic averaged 41.2% across the 17 models and Spanish 25.9%. That says more about how hard each language's five questions are than about the language.

Is ChatGPT good at Arabic?

Yes, ChatGPT is good at Modern Standard Arabic. OpenAI's GPT scored 82.5 on HELM Arabic in June 2026, level with Gemini Flash and 1.6 points behind Claude Opus. OpenAI's own numbers point the same way. Its simple-evals results put its o3 reasoning model at 0.904 in Arabic and 0.906 in French, on an MMLU test that professional translators rewrote in 14 languages.

Anthropic publishes the same kind of figure. Its multilingual support page puts Claude Sonnet 4.5 at 97.2% of its English score in Arabic, and the smaller Claude Haiku 4.5 at 92.5%. For a big chatbot, Arabic is close to a first-class language.

The catch is dialect. A chatbot that writes clean MSA will often answer an Egyptian or Moroccan question in MSA anyway, and that's the next section.

Can AI write Arabic dialects, or only Modern Standard Arabic?

AI understands the Arabic dialects better than it writes them. The AL-QASIDA study tested nine models, GPT-4o among them, on eight dialects from Algerian to Syrian. Its authors write that "LLMs do not produce DA as well as they understand it", where DA means dialectal Arabic. They found the models fluent enough but reluctant. They blame part of it on post-training, and found "that few-shot examples can overcome this deficiency."

The leaderboards can't show you this. Six of HELM's seven Arabic test sets are multiple choice, and none of them grades a reply written in dialect. The Open Arabic LLM Leaderboard, run by the OALL team with 2A2I, TII and Hugging Face, rebuilt its tests in February 2025 and dropped the machine-translated ones. Its team explained why: "Many tasks were direct translations from English, which frequently introduced linguistic and contextual mismatches." It's built for open models: the community submits them, TII's hardware runs them, and every model its launch post names is an open one. So it won't show you where ChatGPT or Claude stands.

The Arabic labs say the same thing about their own training. TII's post on Falcon-H1 Arabic, dated 5 January 2026, says "Arabic is not monolithic" and that it widened dialect sources for Egyptian, Levantine, Gulf and Maghrebi Arabic.

So prompt for the dialect. Name it, name a city, and show two short exchanges. Paste this and swap the dialect:

```

Reply only in Egyptian Arabic, the way people talk in Cairo, not in Modern Standard Arabic. Keep this dialect even if I switch to MSA or English. Match the style of these two exchanges:

Me: ازيك؟ عامل ايه النهارده؟

You: الحمد لله كويس، وانت عامل ايه؟

Me: عايز اروح الأهرامات بكرة، أسهل طريقة ايه؟

You: أسهل حاجة تاخد تاكسي أو أوبر من وسط البلد.

Now answer my next message in the same dialect.

```

Swap "Egyptian" and "Cairo" for Gulf and Dubai, Levantine and Beirut, or Moroccan and Casablanca, and rewrite the two examples in that dialect. Keep the examples short. Use your own phrasing if you can, since the model copies what it sees.

Which Arabic native AI should you try: Jais, HUMAIN Chat, Fanar or Falcon?

Jais Chat and HUMAIN Chat are the two Arabic native chatbots you can download today, and both are free. The Gulf labs behind them ship fast, so we read each vendor's own page in October 2026:

Each lab grades itself on its own benchmarks, so read their claims as claims. Inception's model card puts Jais 2 70B at 70.71 on the AraGen generative test, against 62.58 for Qwen2.5 72B, but it compares only open models. TII says Falcon-H1 Arabic 34B scores about 75% on the Open Arabic LLM Leaderboard.

The independent numbers are less kind to the older Arabic models. On HELM Arabic, ALLaM 7B scored 70.8 and the 30B Jais family 64.5, against 84.1 for Claude Opus. Stanford's team wrote that the Arabic specific open models "underperformed" and that this "may be explained by the age of these models." Jais 2 and Falcon-H1 Arabic are newer than the Jais and Falcon versions on that board.

Pick a local app when the data or the dialect is local. HUMAIN hosts HUMAIN Chat in Saudi Arabia for the kingdom's data protection law, according to launch coverage. For mixed Arabic and English work, the big chatbots still beat the older Arabic models on HELM, the one leaderboard that scores both kinds.

Where does Whizi fit for Arabic?

Whizi is our product, so here's where it helps and where it loses. The interface runs in Arabic, right to left. Pro at $29.99 a month includes Claude Opus, Gemini Pro, ChatGPT Sol and Kimi K3, and its side by side model chats run one Arabic prompt through two models at once, each billed at its own rate. GPT Astra, the other top scorer in our test, needs Powerhouse at $49.99. Starter at $15.99 has ChatGPT Luna, which got 2 of 5 of our Arabic questions right, and Claude Haiku. Whizi loses on the local side. It carries no Jais, ALLaM, Fanar or Falcon model, so for Saudi data hosting or a Gulf dialect specialist, HUMAIN Chat or Jais Chat is the better pick. Side by side model chats shows how compare works.