ByteDance Seed Introduces EdgeBench Benchmark Revealing New AI Scaling Law

ByteDance Seed released EdgeBench, a benchmark that evaluates AI agents over ultra‑long tasks ranging from 12 to 72 hours across 134 real‑world problems. After analysing 38,000 agent‑run hours, the study found performance follows a log‑sigmoid curve driven by environment‑interaction time rather than model size, suggesting a doubling time of about three months for learning speed.

At the same time, experts warn that many AI benchmarks, including popular leaderboards, measure narrow abilities such as reasoning or coding but do not assess whether models produce factually correct statements. This gap allows hallucinations to persist even as scores improve, because benchmarks were never designed to test truthfulness or up‑to‑date knowledge.

Together, the EdgeBench findings and the critique of existing benchmarks highlight a shift in how AI progress may be measured, emphasizing real‑world experience and factual reliability over traditional size‑based metrics.