How AI Could Accidentally Extinguish Humankind

How AI Could Accidentally Extinguish Humankind (?)

By The Washington Post, September 6, 2022  Opinion

The conventional expectation is that ever-growing computing power will be a boon for humanity. But what if those expectations are wrong? Could artificial superintelligence instead cause us great harm? Our extinction?

It is unclear if humanity will ever be prepared for superintelligence, but we are certainly not ready now. Research on artificial intelligence must slow down or even pause. And if researchers will not make this decision, governments should make it for them.

From The Washington Post

"If we’re all wiped out by an artificial superintelligence, it will almost certainly be on accident." -Émile P. Torres  ... ' 

View Full Article (May Require Paid Registration)  

Superintelligence Possible?

As a mere dabbler in the concept, always interested, here a chat in The Edge.  Also, what is it, and should we delight at even the possibility of it, or be very, very afraid?

Is Superintelligence Impossible?

On Possible Minds: Philosophy and AI
David Chalmers, Daniel C. Dennett [4.10.19]

Somebody said that the philosopher is the one who says, "We know it’s possible in practice, we’re trying to figure out if it’s possible in principle." Unfortunately, philosophers sometimes spend too much time worrying about logical possibilities that are importantly negligible in every other regard. So, let me go on the record as saying, yes, I think that conscious AI is possible because, after all, what are we? We’re conscious. We’re robots made of robots made of robots. We’re actual. In principle, you could make us out of other materials. Some of your best friends in the future could be robots. Possible in principle, absolutely no secret ingredients, but we’re not going to see it. We’re not going to see it for various reasons. One is, if you want a conscious agent, we’ve got plenty of them around and they’re quite wonderful, whereas the ones that we would make would be not so wonderful. —Daniel C. Dennett

One of our questions here is, is superintelligence possible or impossible? I’m on the side of possible. I like the possible, which is one reason I like John’s theme, "Possible Minds." That’s a wonderful theme for thinking about intelligence, both natural and artificial, and consciousness, both natural and artificial. … The space of possible minds is absolutely vast—all the minds there ever have been, will be, or could be. Starting with the actual minds, I guess there have been a hundred billion or so humans with minds of their own. Some pretty amazing minds have been in there. Confucius, Isaac Newton, Jane Austen, Pablo Picasso, Martin Luther King, on it goes. But still, those hundred billion minds put together are just the tiniest corner of this space of possible minds. —David Chalmers .... " 

(Updated) Superintelligence Considered in New Context

Brought to my attention, a new book: Superintelligence: Paths, Dangers, Strategies  by Nick Bostrom   http://www.nickbostrom.com/

Reading.  They write:

" ... Superintelligence asks the questions: What happens when machines surpass humans in general intelligence? Will artificial agents save or destroy us? Nick Bostrom lays the foundation for understanding the future of humanity and intelligent life. 

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. If machine brains surpassed human brains in general intelligence, then this new superintelligence could become extremely powerful - possibly beyond our control. As the fate of the gorillas now depends more on humans than on the species itself, so would the fate of humankind depend on the actions of the machine superintelligence. ... "  ... 

A Doomsday Invention?

Following up on my reading of Bostrom's book:  Superintelligence, this 2015 article.  My take on this is that the dangers are still decades away, but does need to be thought about now.

The  New Yorker, November 23,  2015
The Doomsday Invention
Will artificial intelligence bring us utopia or destruction?
Features work of Nick Bostrum
By Raffi Khatchadourian

Superintelligence Considered

Quite late, but making my way through the well known Superintelligence book by Nick Bostrom. Some quite fascinating statistics of the futurist kind that are worth looking at.   Will the development of AI being seen today, that uses methods like neural nets and Bayesian methods,  be able to leverage itself to be self-replicating?  And become in a sense 'Super', and go beyond human intelligence?   What are the dangers involved?  As Bostrom asks, what are the paths, dangers and strategies to be considered?

Superintelligence: Paths, Dangers, Strategies  by Nick Bostrom  and his site  http://www.nickbostrom.com/  Lots of new resources at this link.

 They write:

" ... Superintelligence asks the questions: What happens when machines surpass humans in general intelligence? Will artificial agents save or destroy us? Nick Bostrom lays the foundation for understanding the future of humanity and intelligent life. 

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owes its dominant position. If machine brains surpassed human brains in general intelligence, then this new superintelligence could become extremely powerful - possibly beyond our control. As the fate of the gorillas now depends more on humans than on the species itself, so would the fate of humankind depend on the actions of the machine superintelligence. ... "