Renowned AI researcher and former Google engineer Fran�ois Chollet is co-founding the ARC Prize Foundation, a nonprofit dedicated to establishing benchmarks for AI systems aspiring to achieve human-level intelligence? Led by former Salesforce engineering director Greg Kamradt, who will serve as president and board member, the foundation seeks to guide the journey toward artificial general intelligence (AGI)? Fundraising efforts for the foundation will commence later this January?
Expanding on ARC-AGI
The foundation builds upon the Abstract and Reasoning Corpus for Artificial General Intelligence (ARC-AGI), a test devised by Chollet in 2019 to evaluate AI systems on acquiring skills beyond their training data? Unlike traditional benchmarks that emphasize AI risk with complex questions, ARC-AGI challenges AIs with puzzles that demand adaptability? Driven by the objective of closing the gap in human-like adaptability, future iterations of ARC-AGI aim to bring AI’s human capability alignment closer to zero?
Competition and Challenges
Previously, Chollet and Zapier co-founder Mike Knoop launched a competition to develop an AI that could surpass ARC-AGI’s performance? OpenAI’s ‘o3’ model was a frontrunner, albeit requiring substantial computational resources? Despite improvements, ARC-AGI has faced critique for being oversold as a definitive milestone towards AGI? The intrinsic challenges in defining AGI heighten the debate, with recent claims suggesting its criteria are already surpassed by AI outperforming average human abilities across various tasks?
Future Endeavors
The ARC Prize Foundation plans to release a second-generation ARC-AGI benchmark and a subsequent competition early this year while starting the development of a third version? The foundation aims to forge an academic network and a coalition with leading AI labs to drive AGI evaluations and partnerships? Although OpenAI’s collaboration with the ARC-AGI team was insinuated, no definitive updates were provided in the latest announcement?
Implications for AI Progress
While some industry leaders like OpenAI�s Sam Altman demonstrate willingness to partner with benchmarks like ARC-AGI, the discourse around AGI continues to evolve? The foundation�s pursuits mark significant steps towards establishing a clearer path to achieving human-equivalent AI capabilities, drawing attention to the complexities entwined in the AI landscape?