◆ NOISE IN → SIGNAL OUT◆ READALCHEMIST.COM◆ FREE / NO PAYWALL◆ NOISE IN → SIGNAL OUT◆ READALCHEMIST.COM◆ FREE / NO PAYWALL
THE DIGITAL ALCHEMIST
AIIMPACT 78

Alibaba Claims Its New AI Beats Claude. Independent Tests Disagree.

Qwen4-Max launched Monday, Nov. 16 with a benchmark chart Alibaba drew itself. The testers who didn't work for Alibaba, Artificial Analysis, Agent Arena, Vals, ran the same model and got Claude and Kimi winning instead.

2026-11-165 MIN READ#Alibaba · #Qwen4-Max · #Claude · #Kimi K3.1 Pro · #benchmarks · #open weights · #China AI
The Digital Alchemist
The Digital Alchemist

Alibaba put out a benchmark chart Monday, Nov. 16 showing its new flagship beating Claude and Kimi on coding. Three independent trackers who'd been poking at the same model for a week came back with a different scoreboard entirely.

The verdict: the crown is real on exactly one chart, and Alibaba drew it.

What launched. Alibaba said its Qwen4-Max model scored near the top of coding benchmarks, outperforming Anthropic's Claude Fable 5.1 and Moonshot's Kimi K3.1 Pro. The model uses a sparse mixture-of-experts design with 2.4 trillion parameters, carries a 1-million-token context window, and runs only as a hosted service with no downloadable weights. That's the real story: the Qwen series is the most downloaded open-weight model family in the world, and open weights were the foundation of Qwen's influence. The flagship that built that reputation just shipped locked.

The Chart Alibaba Drew

On DeepSWE v1.1, an agentic coding test, Alibaba puts Qwen4-Max at 75.7, against 75.2 for GPT-6.1 Sol, 74.2 for Opus 5.5 and 74.1 for GPT-6 Astra—a real win, but a thin one. On Terminal-Bench 4.0 it scores 48.5, against 59.6 for Opus 5.5, 59.1 for GPT-6 Astra and 67.2 for Kimi K3.1. Of the 16 panels in the chart, Qwen4-Max has the top score in four. That's Alibaba's own data. Nobody outside the company touched it.

The Digital Alchemist
The Digital Alchemist
Agent Arena Live Task Success Rate (Nov. 13 session data)
Claude Fable 5.2 (Max)28.61%Claude Fable 5.1 (Max)14.1%Qwen4-Max (xHigh)7.46%
Source: Agent Arena leaderboard, ~2.98 million live sessions, cited via Techmeme/dev.to coverage, Nov. 16, 2026.
Price per Million Tokens
InputOutput
Qwen4-Max$1.50$4.50
Claude Fable 5.1$10$50
Source: Agent Arena pricing listings, cited via dev.to coverage, Nov. 16, 2026.

The Chart Nobody Asked Alibaba to Draw

Artificial Analysis is not an Alibaba subsidiary. Its Intelligence Index put Qwen4 Max at 43, versus 46 for Kimi K3.1 Pro—the model Alibaba says beat Kimi scored lower than Kimi on the independent index. On Terminal-Bench 4.0, Artificial Analysis reports Qwen4 Max at 48%, behind Kimi K3.1 Pro's 67%.

Agent Arena scores live agent sessions rather than static test sets. Its leaderboard, drawn from roughly 2.98 million sessions on Nov. 13, puts Claude Fable 5.2 (Max) at #1 with 28.61% task success, Qwen4-Max (xHigh) at #14 with 7.46%, and Claude Fable 5.1 (Max) at #6 with 14.10%. The exact model Alibaba claims to have beaten outperforms Qwen4-Max by nearly two to one on real tasks. Vals Index v2.2 puts Qwen4-Max xhigh at 63.20% and Opus 5.5 max at 70.14%.

Translation of the official line: we built something that beats the competition, trust our chart, and no, you can't download it to check the other 12 panels yourself.

This is the second flagship launch in ten days to pair a capability claim with a locked file. Thinking Machines Lab did the identical move on Thursday, Nov. 6, announcing a benchmark crown for a model it would not yet release. Alibaba just ran the same play with better production values and a bigger name attached.

Where Alibaba does win cleanly is price: Qwen4-Max at $1.50 / $4.50 per million tokens (in/out) versus Claude Fable 5.1 at $10 / $50. That's 85% cheaper input, 91% cheaper output. Cheap is a real advantage. Cheap is not the same claim as best, and Alibaba's launch post conflated the two.

If you're routing production traffic to Qwen4-Max because Alibaba says it beats Claude, you're trusting a vendor's internal test against its own product, graded by nobody else, on a model nobody else can download. That's a procurement decision, not a technical one.

What to watch: whether Alibaba ships the promised smaller Qwen4 tiers with open weights, and on what date; whether Artificial Analysis or Agent Arena publish fuller independent runs once the commercial API stabilizes past launch week; and whether the next flagship launch from OpenAI, Anthropic or xAI submits to third-party scoring before the marketing post goes out.

Sources
  1. Alibaba unveils new flagship AI model as US-China AI race accelerates
  2. Qwen4-Max Beat Claude on Coding Benchmarks — Then Alibaba Locked the Weights
  3. Qwen4-Max Launches at $4.50/M Output, 91% Below GPT-6 Astra
  4. Qwen4-Max doubles up at No. 6 in Arena's preliminary web-dev rankings
  5. Qwen4-Max vs Claude Opus 5.5: Benchmarks, costs and tests
  6. Alibaba Launches Qwen4-Max Flagship AI Model, Saying Work Task Costs Fall Sharply
  7. Alibaba Flagship AI Model: What We Know So Far
  8. Alibaba drops Qwen4-Max, beats Claude Fable 5.1, Kimi K3.1 Pro
  9. Qwen4 Max Benchmark Results for Data Analysis
  10. Qwen4-Max Preview: Alibaba's Silent Frontier Launch
  11. Alibaba Qwen4-Max-Preview Retires November 25 With No Benchmarks, Model Card or Price
  12. Qwen 4 Max Preview: Specs, Benchmarks, Pricing, and How to Access It
  13. Alibaba Qwen 4 Launch: Date, 27B Open Weights & Benchmarks
  14. Qwen 4 Release Date: Qwen4-Max Preview Is Out
  15. Qwen 4 Is Here (Sort Of): I Tested Qwen4-Max Against Qwen 3.8 Max
  16. Qwen4-Max Preview — Benchmarks, Specs & Release Date
  17. Qwen 4: Release Date, Models and What's Confirmed (November 2026) - AIToolsReview
  18. Alibaba Quietly Won the Open AI Race 3 Billion Downloads
  19. Qwen4 Nears Launch as Alibaba's Open Source Promise Faces Real Doubt - Startup Fortune
  20. Qwen open-weight AI models top 3B downloads, beating Meta and Google - AINave
  21. Qwen by Alibaba: The Open-Weight AI Family Quietly Eating the LLM World
← back to the feed
QueryFlow
ASK

Ask your database anything.

Stop writing queries. You don't need to know SQL to get the answer. QueryFlow AI writes it for you.

Native Mac app · by yForest

  • Snowflake
  • Redshift
  • PostgreSQL
  • MySQL
  • BigQuery
  • Databricks
  • Salesforce
  • Google Sheets
  • CSV/Excel
NVDA 150.96 ▼ 2.89%MSFT 508.29 ▲ 0.40%GOOGL 282.44 ▲ 0.80%AMZN 267.74 ▲ 3.82%META 616.38 ▼ 0.40%TSLA 383.72 ▲ 0.95%AMD 530.14 ▲ 5.31%AVGO 354.05 ▲ 3.64%PLTR 156.67 ▲ 0.07%COIN 110.51 ▼ 3.92%MSTR 90.32 ▼ 6.40%NVDA 150.96 ▼ 2.89%MSFT 508.29 ▲ 0.40%GOOGL 282.44 ▲ 0.80%AMZN 267.74 ▲ 3.82%META 616.38 ▼ 0.40%TSLA 383.72 ▲ 0.95%AMD 530.14 ▲ 5.31%AVGO 354.05 ▲ 3.64%PLTR 156.67 ▲ 0.07%COIN 110.51 ▼ 3.92%MSTR 90.32 ▼ 6.40%