Highlights of Stanford HAI's 2025 Artificial Intelligence Index Report

Author: Stanford HAI (Stanford Artificial Intelligence Institute)

Compiled by: Felix, PANews

Stanford HAI recently released the 456-page "Artificial Intelligence Index Report 2025". Here are some key points about artificial intelligence trends:

1. AI is becoming much more powerful than imagined

In the new benchmarks MMMU, GPQA, and SWE-bench, AI performance improved significantly: scores increased by 18.8%, 48.9%, and 67.3%, respectively. In addition to the benchmarks, AI systems made significant progress in generating high-quality videos, and in some cases, large language models (LLMs) even surpassed humans in timed programming tasks.

Note:

MMMU is a novel, carefully designed benchmark for multi-disciplinary multimodal understanding and reasoning at the university level, aiming to evaluate the expert-level multimodal understanding capabilities of underlying models on a wide range of tasks.

GPQA is a challenging dataset consisting of 448 high-quality and difficult multiple-choice questions written by experts in different fields. Experts who hold or are pursuing a PhD in the corresponding field achieve only 65% accuracy, while highly skilled non-expert verifiers achieve only 34% accuracy despite spending an average of more than 30 minutes and having unlimited access to the Internet.

SWE-bench is a benchmark for evaluating the performance of Large Language Models (LLMs) on real-world software questions collected from GitHub.

2. AI is more efficient, accessible, and affordable

Smaller AI models with fewer parameters are becoming increasingly powerful: in just two years, the number of parameters has been reduced by about 100 times, while still scoring over 60% on the Massive Multi-Task Language Understanding (MMLU) test.

The gap between open source and closed source models is also narrowing, with the performance gap falling from 8% to just 1.7% in some benchmarks.

Furthermore, the cost of inference for systems reaching the level of GPT-3.5 dropped by more than 280 times from November 2022 to October 2024. At the hardware level, costs dropped by 30% per year, while energy efficiency improved by 40% per year.

The threshold for advanced AI is rapidly decreasing. Not to mention the development of sparse models like DeepSeek, where only relevant parameters are activated to answer the user’s query under the Mixture of Experts (MoE) structure, making the whole thing more efficient.

Indeed, as smaller but more powerful AI models continue to emerge, the requirements for AI model training have been reduced, and cost-effective distributed training is expected to become mainstream in the next decade. There are currently some top projects conducting related research based on different theoretical frameworks.

3. AI is increasingly integrated into everyday life

In 2023, the U.S. Food and Drug Administration (FDA) approved 223 AI-assisted medical devices, up from just six in 2015. On the roads, self-driving cars are no longer experiments: Waymo, one of the largest operators in the U.S., provides more than 150,000 self-driving rides per week, and Baidu’s Apollo Go fleet of driverless taxis is now operational in several Chinese cities.

4. Corporate investment in AI has increased significantly, driving record investments and adoption

The adoption of AI in business is also accelerating: 78% of organizations are using AI in 2024, up from 55% the year before. At the same time, a growing body of research confirms that AI can increase productivity and help close skills gaps across the workforce.

In fact, product-market fit breakdowns will occur more frequently as AI causes customer expectations to grow exponentially, making existing solutions obsolete overnight, leaving incumbents with no chance to adapt.

5. Despite rising optimism about AI globally, Asians are more optimistic about AI

In countries such as China (83%), Indonesia (80%) and Thailand (77%), the majority believe that the benefits of AI products and services outweigh the risks. In contrast, optimism remains far lower in places such as Canada (40%), the United States (39%) and the Netherlands (36%).

However, this attitude is changing: since 2022, optimism has grown significantly in a number of previously skeptical countries, including Germany (up 10%), France (up 10%), Canada (up 8%), the United Kingdom (up 8%) and the United States (up 4%).

6. The influence of artificial intelligence in scientific research is increasing, and it has become an important driving force for scientific progress

The growing importance of AI is reflected in major scientific prizes: two Nobel Prizes were awarded for research into deep learning (physics) and its application to protein folding (chemistry), while the Turing Award recognized groundbreaking contributions to reinforcement learning.

Clearly, AI is advancing at an exponential and unexpected pace, which is significant to most people. As a result, AI security is becoming increasingly important. While AI makes forgery easier, cryptography makes it more difficult. Look forward to crypto projects that can leverage the native properties of blockchain (verifiability and transparency) to build practical solutions in this area.

Related reading: Interview with Chris Dixon, founder of a16z: The intersection of artificial intelligence and encryption technology