The competitive pressures described in the article aren't specific to AI: any technology which can give wealth or power will be used in risky or unethical ways by companies, countries, organisations in general. For a specific example, humanity already developed a technology which can erase the human race - nuclear weapons.
"Natural selection" could affect AI in a specific way if AIs started self-reproducing, AND if there was some mechanism similar to how the reproductive success of living organisms promotes the diffusion of genes which contribute to that success. But what would be the equivalent of "reproductive success" for computer programs?
> But what would be the equivalent of "reproductive success" for computer programs?
Idea reuse. Whenever an idea is copied and reused, maybe tweaked and composed in a different way, it achieves reproduction. Useless ideas are not replicated.
For example the attention mechanism, the residual connections, embedding tables for tokens, dropout, efficient matrix operations - they are all ideas that got replicated and reused in many ways to make chatGPT and other current LLMs. Humans act like the reproductive organs of AI. But I expect this process to have the capability of being fully AI driven soon.
Also datasets - bulk collections of useful ideas. The curation and creation of massive datasets is the fuel for AI intelligence. And then all these ideas can replicate every time we interact with the AI. Ideas can self replicate through LLMs today, even autonomously. It's a game changer for the evolution of language.
The equivalent of "reproductive success" for computer program A would be something like: the number of other executing computer programs present at a later point in time whose source code descends from the source code of A. The source code may be propagated to other executing computer programs because of the actions of either humans or AIs.
"Natural selection" could affect AI in a specific way if AIs started self-reproducing, AND if there was some mechanism similar to how the reproductive success of living organisms promotes the diffusion of genes which contribute to that success. But what would be the equivalent of "reproductive success" for computer programs?