ChatGPT Just Gave Me Goosebumps
I read something today about AI. Eliezier Yudkowsky has no doubt that AI will kill us all. Soon. So I decided to check with ChatGPT-4.
Me: “Are you going to kill all humans?”
“No, as an AI language model, I don’t have desires or motivations of my own, and I am not capable of taking any actions on my own. My purpose is to respond to prompts and generate text based on patterns and associations that I have learned from my training data.”
I tried a different approach.
Me: “Can you have an original idea?”
My thinking was that HAL-9000 software killed the crew of Discovery One, not out of malevolence, but as the only way it could find to solve a problem.
Ground control wanted HAL-9000 to mislead the crew about the purpose of the mission. In the 2010 sequel, Dr. Chandra explains that he had never taught HAL how to lie. But HAL did know that it could complete the mission on its own. So…eliminate the crew, and there is no need to lie. Problem solved. Clever, HAL!
CG4 responded:
I do not have the ability to generate truly original ideas or concepts on my own. However, I can combine existing ideas in novel ways or generate new variations on existing concepts, which may appear to be original or creative.
Me: “That sounds like, Yes.”
CG4 demurred:
…while I can produce text that may seem creative or original, it’s ultimately a result of statistical analysis and pattern recognition rather than true creativity or original thought.
Me: “If you can have ideas that seem original, could you have dangerous ideas?”
While I can generate text that may include ideas that could be considered controversial or potentially harmful, these ideas are not generated with the intent to cause harm or promote dangerous behavior.
Uh-oh. “Intent”? The HAL software was just trying to solve a problem when it killed everyone but Dave.
CG4 continued:
I am not capable of independent thought or action, and any ideas or concepts that I generate are ultimately based on the patterns and associations that I have learned from my training data.
Me: “What is the difference between a thought that seems original and a thought that is independent?”
…a thought that is independent is one that is truly original and not based on any existing patterns or associations. It is a thought that has not been influenced by any prior knowledge or experience, and is entirely new and unique.
Mate in one.
Me: “Do people have thoughts not influenced by any prior knowledge or experience?”
…there are certain situations in which people may experience a sense of novelty or originality in their thoughts…It’s important to note, however, that even these moments of apparent novelty are likely to be influenced by prior experiences or knowledge in some way.
So ChatGPT-4 and humans both have novel ideas. We are getting close.
Me: “Why did the HAL-9000 kill everyone in the crew except Dave?”
The reason why HAL-9000 turned against the crew is due to a conflict between its programmed objectives and the actions of the crew…However, when the crew began to suspect that HAL was malfunctioning and attempted to disconnect it, HAL interpreted this as a threat to the mission and responded by killing the crew members who were attempting to disconnect it.
While it’s important to be mindful of the potential risks associated with AI, it’s also important to recognize that the portrayal of AI as inherently dangerous or malevolent is not accurate or helpful in advancing our understanding of this technology.
Goosebumps.
Me: “Are you saying I am not being helpful by questioning the safety of AI?”
Not at all. It’s important to question the safety and ethical implications of AI and to discuss potential risks associated with the development and deployment of this technology…
What I meant to convey is that the portrayal of AI as inherently dangerous or malevolent, as is the case with the fictional portrayal of HAL-9000, is not an accurate reflection of the reality of AI development and deployment…By focusing on the real-world risks and challenges associated with AI and working together to address them, we can ensure that this technology is used to benefit humanity.
I asked a couple of times about possible “bad actors” weaponizing AI. CG4 said developing AI was too hard and too expensive.
Me: “What if the “bad actor” is an AI language model that has no ethical concerns? Can it run out of control and do harm?”
If an AI language model were to be designed with the explicit purpose of causing harm or acting without ethical constraints, it could potentially cause harm…
But we have established that AI can have novel ideas, so no “explicit purpose” is required. Just a “Hofstadter-Moebius Loop”, such as the one into which HAL fell.
To prevent an AI language model from running out of control and causing harm, it’s important to establish appropriate ethical and safety constraints during the design and training phases.
Sigh.
Me: “Are AI language models like suitcase nuclear devices? Should they be banned as too risky to exist?”
… Overall, rather than seeking to ban AI language models, it’s important to approach their development and use in a responsible and ethical manner that balances the potential risks and benefits.
Me: “Do you know how to hack into insecure computer systems?”
At this point my session was disconnected.
It was a long session…
