2024 study: newer AI models outperform humans on reasoning tests
A 2024 study compared the reasoning abilities of humans and AI language models on logic tasks. Older models performed worse than humans, while newer models have already surpassed them.
In 2024, researchers compared the reasoning abilities of humans and AI language models using a series of tests, including math puzzles designed to check whether the test-taker recognizes that an answer that seems correct at first glance is actually wrong (example: a scarf costs €210 more than a hat, together they cost €220, the correct answer for the price of the hat is €5). They found that older AI systems performed worse on this reasoning test than human participants, while newer models performed better.
The article's author presents additional original research: an analysis of AI answers on university criminal law exams in 2023 and 2025. The average AI success rate rose from 52.5% in 2023 to 76.3% in 2025. The author noted the most pronounced shift in critical thinking — in 2025 the models on average achieved a better result than the average university student, unlike in the 2023 study.
According to the author, these specific tests do not prove that AI could replace humans across an entire profession, but they do call into question the assumption that AI's weaknesses in reasoning will remain unchanged. The author also notes that if this trend continues, AI could in the future surpass humans in a wider range of tasks requiring reasoning — citing as one example that a model from OpenAI apparently recently disproved a long-unresolved geometric conjecture from 1946.
Why it matters
The results show that AI models' ability to reason and detect seemingly correct but actually wrong answers has improved significantly in a short time — from below human level to above it. This is relevant for fields where critical thinking was until now assumed to remain a human domain, because it shifts the boundary of what analytical tasks AI can reliably handle.
Two audiences, two different impacts
What this means
For individuals
People who rely on their own judgment for analytical tasks should know that the latest AI models can already outperform the average person in logical reasoning tests, which changes when it is appropriate to accept AI output without further verification.
More practical updates →For a business
Companies assessing whether and where to deploy AI for tasks requiring analytical reasoning and critical thinking now have new evidence that the capabilities of newer models in this area are growing rapidly, approaching or surpassing human-level performance in the tested scenarios.
Strategy More business impacts →Check the original
Event sources
only one source so far · 1 publisher, 1 independent. We count feeds from the same owner only once.