On Monday, OpenAI announced impressive advances in mathematics, computer science and physics, as well as what the company considers a scalable approach to reporting its results. But despite its apparent change of direction, researchers who spoke to The edge are growing tired of the company’s antics.
These last few months have been hectic between OpenAI and mathematicians, who are still grappling with the hundreds of preprints that were unceremoniously submitted to them in early October. This time, OpenAI says it is working with expert “shepherds” to decide how best to share its latest advances.
The three results address some of the most important unsolved problems in mathematics, computer science and physics. Some extend OpenAI’s previous work and others innovate. Following the advice of the OpenAI “shepherds”, everyone has a different release strategy. For the mathematical result, OpenAI published the AI-generated result, “leaving researchers free to organize their own efforts to explain the result.” For its computational findings, OpenAI will also bring together an external group of researchers “to explain the result.” The physics, meanwhile, has not been published at all, and OpenAI said it would “defer” to an external group of experts on how to publish the materials.
- The mathematical result seems striking. It claims to solve the Kakeya conjecture in all dimensions. Basically, the problem is how little space is needed to rotate a needle or pencil in all directions. A proof of the three-dimensional version was among the achievements for which NYU mathematician Hong Wang received a prestigious Fields Medal earlier this year. Brendan Hassett, a professor of mathematics at Brown University, said The edge OpenAI’s claim “seems well supported”, although it stressed that it has not yet verified Lean formalization, a means of computational verification of results.
- The physics result explores the Standard Model, the most advanced and comprehensive theoretical framework in physics. It concerns quantum chromodynamics, which describes the strong nuclear force, one of the four fundamental forces of nature (alongside the gravitational force, the electromagnetic force and the weak nuclear force) and its relationship with Yang-Mills theory, the cornerstone of modern particle physics. Yang-Mills theory is also at the heart of one of the seven Millennium Prize math problems, although OpenAI’s work does not address the unresolved “mass gap.”
- The end result purports to disprove the exponential time hypothesis, a widely held assumption in theoretical computer science that some problems cannot be solved much faster than brute force approaches. It is closely related to P versus NP, another Millennium Prize problem and one of the largest unsolved problems in the field, which asks whether problems whose solutions can be easily verified are also easy to solve.
Given the varying levels of disclosure and the complexity of evaluating this work, it is difficult for researchers to fully judge the company’s conclusions. Even with all the information, it would take time. But the experts who spoke The edge said there were good reasons to take the results seriously, calling them in turn “huge,” “major breakthroughs” and “significant.”
“It makes no sense that only a few eminent scientists can examine them.”
But researchers’ enthusiasm for the science has been tempered by exasperation with OpenAI’s behavior, a familiar feeling after months of what they describe as an exhausting barrage of AI-generated discoveries. Many were eager for the broader community to take on the arduous task of reviewing the AI lab’s results and wondered why OpenAI was erecting additional barriers by selecting experts to oversee the process.
Most experts The edge welcomed OpenAI’s decision to consult external researchers, but argued that publicly announcing the results should also mean making this work accessible to everyone. “I agree with them releasing the issues like they did today,” said Constantin Kogler, a researcher at the Institute for Advanced Study, praising the more manageable batch of three results compared to the hundreds previously released. But he questioned the decision to restrict access to experts chosen by OpenAI. “It makes no sense that only a few eminent scientists can look at them,” he said.
“The fact that they are cherry-picking the group is a disaster,” said Andreas Karch, a physics professor at the University of Texas at Austin. The edge. “The big labs already have enough power. They shouldn’t pick winners and losers.” Karch initially suggested selecting researchers through a “competitive process”, although acknowledging that this would likely produce a similar group, which he described as “top-notch”. He later questioned why OpenAI “can’t release the proof now”, in addition to naming his team of experts. “This would democratize access while showing that they care about making the results meaningful and understandable,” he said.
“I have a feeling that whatever approach is taken, there will be someone who will be upset.”
This is exactly the kind of scrutiny that the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), a group of experts that OpenAI had agreed to consult on how to share its findings, warned against just a day before the announcement. “We recommend that AI companies do not adopt this practice,” they said, arguing that selecting certain people to receive results first is “unnecessary,” borrows from the reputation of the selected researchers and risks unfairness, biased selection and delays, among other problems. Instead, they recommended that AI labs publish their results on scientific repositories, with formalized evidence and information about how the problems were chosen, and provide funding and other support to people working to understand the results.
Following the publication of the latest results, AGMAI urged OpenAI to “immediately make public the natural language and Lean results behind its announced results” in physics:
“While these are central questions in physics, they are also important questions in mathematics. The existence of Yang-Mills quantum theory is part of a Millennium Prize problem, and the construction of quantum theories of interacting fields in four dimensions has been a goal of mathematical physics for decades.
We have great respect for the scientists tapped by OpenAI to lead this work. But as we explained in our recent statement, understanding must be developed openly, with equal access to all. OpenAI said it has delayed release until Lean formalization is complete. According to their own statements, this operation is now complete, so there is no scientific reason to delay publication further. »
Making results openly available could also lead to better science and encourage broader adoption of new tools like AI, Hassett said. “The fact that many groups around the world are experiencing these results supports the rapid adoption of new tools by those who want to use them,” he said.
“Fundamental results are resolved so quickly without human input…papers are difficult to read and verify.”
Finding a way to communicate with researchers in the areas that OpenAI is increasingly encroaching on that satisfies everyone may be impossible. The company says it is experimenting with different approaches and acknowledges that each has tradeoffs. “I have a feeling that whatever approach is taken, there will be someone who will be upset,” said Jared Duker Lichtman, a Stanford mathematician. “So it’s just about finding the least offensive option.”
Xiaoyu He, a mathematics professor at the Georgia Institute of Technology, thinks the problem runs deeper than how OpenAI chooses to announce its findings. “The results are very impressive,” he said, “but I think they are pretending to try to be responsible and thoughtful in the way they publish the results, without being able to change the main problems: the basic results are resolved so quickly without human input and the documents are difficult to read and verify.”
Gn bussni
