AI readiness report

Today I’m going to summarise the latest report from Scale AI’s 2024 AI Readiness Report.

The report focuses on what it takes for organisations to transition from merely adopting AI to actively optimising and evaluating it.

To understand the current state of AI development and adoption, Scale surveyed over 1,800 ML practitioners and leaders directly involved in building or applying AI solutions.

Key challenges identified include security concerns, lack of expertise, and insufficient benchmarks to effectively evaluate models. While improving operational efficiency was the top reason for adopting AI (cited by 79%), only half are measuring the business impact of their AI initiatives.

AI Year in Review 2023 saw rapid advancements in generative AI from leaders like OpenAI, Google, Anthropic, and Meta.

Key milestones included:

• OpenAI's GPT-4 in March 2023, demonstrating human-level performance
• Google's launch of Bard and PaLM 2
• Anthropic's release of Claude 2 in the summer with a 100K context window
• Meta's unveiling of Llama 2 and Code Llama
• Google DeepMind's release of Gemini in late 2023, outperforming humans on the MMLU test
• Emergence of open source models like Falcon, Mixtral, and DBRX enabling local inference with less compute
• Anthropic's launch of Claude 3 in March 2024, doubling the context window
• Cohere's release of Command R designed for scalability and long context