AI researcher José Hernández‑Orallo unveils new framework to assess artificial intelligence

Science published an international study led by José Hernández‑Orallo of the Universitat Politècnica de València, with collaborators from the University of Cambridge, Princeton University, MIT and Google DeepMind. The paper argues that the prevailing practice of measuring AI progress by its ability to match or surpass human performance – rooted in the Turing test – is insufficient for today’s complex systems.

The authors propose a three‑pronged evaluation model: (1) assess distinct cognitive profiles and abstract capabilities of AI systems rather than a single human benchmark; (2) examine how AI changes human cognition when humans and machines work together; and (3) evaluate societal impact and alignment with human values, aiming to anticipate risks before deployment.