Why frontier AI can't solve this professor's math problem - Greta Panova

Greta Panova is a mathematician at USC who studies algebraic combinatorics and writes problems for the Putnam Mathematical Competition. For the FrontierMath Symposium, she created a problem so difficult that even when today’s most advanced AI models are told which papers to read and which steps to take, they still have no idea how to proceed. While she initially predicted a model might solve it within a year or two, she now thinks it might take longer. She likens current models to an “unprepared but ambitious student”, one that makes logical errors but presents its work with confidence. Her prediction: once AI can do math at a professor’s level, it won’t stop there. It will be capable of any intellectual task that doesn’t rely on human dexterity. -- Episode links -- Transcript: -- Timestamps -- 00:00:00 – Writing an unsolvable math problem for AI 00:03:08 – How hard is it compared to the Putnam? 00:04:25 – Testing ChatGPT and frontier models on math 00:08:03 – Using AI in mathematical research 00:13:52 – How AI “searches” like humans 00:15:41 – What happens if AI surpasses mathematicians 00:22:12 – Why benchmarks like FrontierMath matter 00:27:49 – What would truly surprise Greta in AI math 00:30:56 – How fast should AI progress? Special thanks to interviewer Daria Ivanova: https://youtube.com/@inQEDible