Software is something that can be completely automated. Its just compute budget. They can create a ERM from scratch _and_ provide all the migration scripts to move customers off of the competitors etc. Its just compute.
If that were true, I would expect that any software company that had feature parity and provided automated migration would be the defacto victor in their market. However, we see almost no instances of monopolies in software; on the contrary, for most problem spaces we are absolutely inundated by choice.
Not really. Even if we imagine perfect automated developers, it's also market research, marketing, tracking user feedback, tracking legal requirements, collecting and fixing issues, etc.
Any transition to essentially all AGI is going to involve people for a while. But the bulk of decision making and activity would move to the AGI rapidly.
The key is, suddenly intellectual labor has a nearly free marginal cost. Including the intellectual labor of managing different units, each with a different focus.
Where is the bottleneck?
Right now, human beings, our costs, limitations, individual idiosyncrasies, inability to scale, unreliability, down time, etc., are the most profound bottlenecks companies have.
> But the moment that the AI can exceed a human programmer, at something as narrow as coding, then the company that has that AI shouldn't sell it to replace humans at other companies - it should instead use it to write programs to replace the other companies.
That paragraph makes a very narrow claim, unlike your broad one about AGI.
If AGI ever arrives and is in any way cost-efficient, our entire society is obsolete and the only jobs that matter are pretty much mining and taking care of hardware until robots take over. That and techno-libertarian aristocrat, of course.
Software is something that can be completely automated. Its just compute budget. They can create a ERM from scratch _and_ provide all the migration scripts to move customers off of the competitors etc. Its just compute.