◆ NOISE IN → SIGNAL OUT◆ READALCHEMIST.COM◆ FREE / NO PAYWALL◆ NOISE IN → SIGNAL OUT◆ READALCHEMIST.COM◆ FREE / NO PAYWALL
THE DIGITAL ALCHEMIST
AIIMPACT 58

Liquid AI's Qwen-Beating Claim Didn't Survive a Recount

LFM3-1.7B trended on Hugging Face after Liquid AI said its 1.73B model beats a rival five times its size. An independent recount found the lead vanishes once you stop counting one benchmark three times.

2026-11-034 MIN READ#LiquidAI · #LFM3 · #open models · #Hugging Face · #on-device AI · #benchmarks
The Digital Alchemist
The Digital Alchemist

LFM3-1.7B hit the Hugging Face trending page claiming something that should stop any engineer mid-scroll: a model you can run on a phone beating a rival five times its size. The engineering is real, the headline number is not.

Liquid AI shipped LFM3-1.7B on Tuesday, Oct. 27, as the text model anchoring a six-model family—vision, audio, omni, and two decision models—all branching from one 1.73-billion-parameter backbone. The tri-hybrid architecture splits 24 layers between gated short convolutions, sliding-window attention, and global attention, trained for agentic tool-calling workloads. At 4-bit precision it fits in about 1 GB of memory. On a single Nvidia B300 it can produce 3.6 billion tokens a day at 1,024 concurrent requests.

The company also ran it against Qwen3.5-2B on 65 everyday agent tasks in real macOS environments. LFM3-1.7B won: 64% task success versus 48%, 58% fewer tokens per task, stuck less than 1% of attempts versus 15% for Qwen. That's a real, independently reproducible test.

The number that doesn't hold

Liquid AI's headline: on its own 29-benchmark table, LFM3-1.7B averages 49.37 against 47.24 for Qwen3.5-9B, a model five times its size. That 2.13-point lead is the whole case for "small models crossed a threshold."

An independent recompute found the lead rests on one evaluation counted three times across the grid. Strip the duplication and the win becomes a 0.04-point loss. Translation: the number that made this model trend is an artifact of scorecard construction, not a genuine advantage.

Liquid AI's own documentation is blunt about what it doesn't do: not recommended for agentic coding or complex multi-step reasoning. On Qwen's strengths it loses badly—80.40 to 63.37 on GPQA Diamond, 87.11 to 65.56 on RULER at 128K tokens. For general chat or broad reasoning, a larger model will outperform it by a wide margin.

The Digital Alchemist
The Digital Alchemist

The license has a ceiling too

The weights download without a gate, but "open" is doing heavy lifting. The LFM Open License v1.0 ends free commercial rights the moment your company's annual revenue exceeds $10 million USD across your whole corporate group. Past that line you need a commercial license from Liquid AI. That makes it source-available with a revenue trigger, not an OSI open-source license—useful for a solo developer, a different calculation for anything built to scale.

What to watch: whether Liquid AI corrects the benchmark table now that the duplication is public, and whether download counts (42,400 and climbing) keep rising after the correction lands or fade after the quality floor gets tested in production.

Sources
  1. LiquidAI/LFM3-1.7B · Hugging Face
  2. LFM3-1.7B - Liquid AI Docs
  3. LiquidAI/LFM3-1.7B-Base · Hugging Face
  4. Introducing LFM3: Every Modality for Specialized Intelligence
  5. LiquidAI LFM3-1.7B: A Local Agent Model That Fits on a Phone
  6. LiquidAI LFM3-1.7B: A 1GB Local AI Model Built for Agents
  7. Build specialized multimodal models with LFM3
  8. LFM3 explained: Liquid AI's 1.7B model, benchmarks, speed, license
  9. LFM3-1.7B Benchmarks: A 1.7B Model Built for Agentic Tool Use
  10. LiquidAI/LFM3-Omni-2B · Hugging Face
  11. LiquidAI/LFM3-Audio-2B · Hugging Face
  12. Add evaluation results for LiquidAI/LFM3-Audio-2B
  13. LiquidAI/LFM3-VL-2B · Hugging Face
  14. LiquidAI (Liquid AI)
  15. LFM2 Technical Report
  16. LFM3: Technical Deep Dive into Liquid AI's Tri-Hybrid Multimodal Family - Local AI Zone
  17. LFM License
  18. LFM Open License User Guide - Liquid Docs
  19. Liquid AI LFM3 Guide: 2B Edge Models & $10M License Cap
  20. LICENSE · LiquidAI/LFM2-350M-ENJP-MT at main
  21. Liquid AI Opens LFM3 Free to Firms Under $10M in Revenue - FourWeekMBA
  22. LFM2.5 from Liquid AI: above ten million dollars, the licence is not granted — CSIDIA
  23. LFM3's 2.13 point lead rests on one test; $10M revenue licence limit
  24. LiquidAI/LFM2-2.6B-Exp-GGUF · LFM Open License v1.0 isn't free license
  25. LiquidAI LFM2.5 Review 2026: Edge AI on a Raspberry Pi — With a License Catch
← back to the feed
QueryFlow
ASK

Ask your database anything.

Stop writing queries. You don't need to know SQL to get the answer. QueryFlow AI writes it for you.

Native Mac app · by yForest

  • Snowflake
  • Redshift
  • PostgreSQL
  • MySQL
  • BigQuery
  • Databricks
  • Salesforce
  • Google Sheets
  • CSV/Excel
NVDA 150.96 ▼ 2.89%MSFT 508.29 ▲ 0.40%GOOGL 282.44 ▲ 0.80%AMZN 267.74 ▲ 3.82%META 616.38 ▼ 0.40%TSLA 383.72 ▲ 0.95%AMD 530.14 ▲ 5.31%AVGO 354.05 ▲ 3.64%PLTR 156.67 ▲ 0.07%COIN 110.51 ▼ 3.92%MSTR 90.32 ▼ 6.40%NVDA 150.96 ▼ 2.89%MSFT 508.29 ▲ 0.40%GOOGL 282.44 ▲ 0.80%AMZN 267.74 ▲ 3.82%META 616.38 ▼ 0.40%TSLA 383.72 ▲ 0.95%AMD 530.14 ▲ 5.31%AVGO 354.05 ▲ 3.64%PLTR 156.67 ▲ 0.07%COIN 110.51 ▼ 3.92%MSTR 90.32 ▼ 6.40%