正文 Markdown
I don't think automation of AI R&D will rapidly lead to domain-general super-intelligence. I think this will be true even if AIs can do *literally everything* a human AI researcher does today. Even after the full automation of AI R&D, further capabilities progress will only happen through (1) widespread deployment of AI throughout the economy, accompanied by data collection; and/or (2) the wholesale recreation of much of the economy by AI labs. Without access to the real-world signal provided by either of the above, I think that the only thing produced by automated AI researchers would be a "Goodhart Singularity". If I'm right, this is obviously good news. I make the case for this in a new piece on my substack