Interesting point, but do not fully understand. Posting to be revisited.
By Gregory Goth, Commissioned by CACM Staff, March 29, 2023
Nearly two years since the publication of the paper in Nature, Google has not yet fully open-sourced the data or code on which its claims were based.
The contentious discussion over the validity of Google researchers' claim that machine learning agents could achieve superhuman results in creating plans for computer chips entered a new, more public phase Tuesday (March 28), with a leading researcher in design automation finding the Google technology did not perform as its authors claimed in a paper published nearly two years ago in Nature.
The dispute around the Nature paper's claims has bubbled for nearly a year in prepared public statements and GitHub code repositories and FAQ sections; researchers directly involved in the situation have declined to speak extemporaneously for the public record. Even some subject matter experts have not wished to speak openly, given Google's dominant position in its ability to distribute research resources to academic computer scientists. However, Tuesday's presentation by Andrew Kahng, a prominent University of California, San Diego researcher in the field of electronic design automation (EDA), at the 2023 ACM/IEEE International Symposium on Physical Design, could elevate the issue to a more open avenue of argument among industry and academic experts.
Briefly stated, the authors of the Nature paper claimed their reinforcement learning (RL) agents could revolutionize the labor-intensive task of floorplanning—the architecting of the incredibly intricate network of memory components (called macro blocks) and logic circuitry (standard cells) on a chip. "Our method generates manufacturable chip floorplans in under six hours, compared to the strongest baseline, which requires months of intense effort by human experts," the authors wrote.
Kahng served as a peer reviewer for the paper, and also wrote an encapsulation for the news and views section of the journal, quoting science fiction author Arthur C. Clarke's observation that any sufficiently advanced technology is indistinguishable from magic.
"To long-time practitioners in the fields of chip design and design automation, (lead author Azalia) Mirhoseini and colleagues' results can indeed seem magical," Kahng wrote.
How open is open?
Science is not magic, however, and the Google paper's claims took the research community by storm. At the conclusion of his summation, Kahng wrote, "We can therefore expect the semiconductor industry to redouble its interest in replicating the authors' work, and to pursue a host of similar applications throughout the chip-design process."
For researchers who presumably were interested in trying to replicate those results, the Google team noted at the end of the paper that "the data supporting the findings of this study are available within the paper and the Extended Data," and that "the code used to generate these data is available from the corresponding authors upon reasonable request."
