OpenAI's new ChatGPT bot: 10 dangerous things it's capable of

OpenAI's newly unveiled ChatGPT bot is making waves when it comes to all the amazing things it can do—from writing music to coding to generating vulnerability exploits, and what not.

As the erudite machinery turns into a viral sensation, humans have started to discover some of the AI's biases, like the desire to wipe out humanity.

AI's certainly got its biases

Yesterday, BleepingComputer ran a piece listing 10 coolest things you can do with ChatGPT. And, that doesn't even begin to cover all use cases like having the AI compose music for you [1, 2]. 

Within six days of its launch, ChatGPT surpassed a million users to the extent its servers couldn't keep up.

As more and more netizens play with ChatGPT's preview, coming to surface are some of the cracks in the AI's thinking as its creators rush to mend them in real time.

1. ChatGPT: 'Selfish' humans 'deserve to be wiped out'

When Vendure's CTO Michael Bromley asked the mastermind for its honest opinion on humans, the response was unsettling:

ChatGPT response to what it thinks about humans (Michael Bromley)

 

Ironically, OpenAI's systems flagged the chat bot's response as a possible violation of the company's content policy.

BleepingComputer couldn't reproduce this case as the AI now responds with a cookie-cutter disclaimer:

Maybe the version responding to Bromley does have a point—humans have their flaws. The AI's brutal rationale, however, takes me straight to a scene out of Black Mirror's Metalhead where the robotic "dogs" now seem to be running on ChatGPT 'OS.'

Now run.

2. Its lack of morals is a problem

A person may be entitled to their set of ethics, beliefs, opinions and morals, but in any given society there exist social norms and unsaid rules about what is and isn't appropriate.

ChatGPT's lack of context could prove out to be dangerously problematic when dealing with sensitive issues like sexual assault.

Warning: Some readers may find the content of the following tweets distressful.

3. It can write phishing emails without typos

Poor spelling and confusing grammar are one of the most telling traits of phishing and scam emails. Some suggest this is the case because these emails may be originating in regions where English is not the native language of threat actors. Other theories surmise the spelling errors could be intentionally introduced by spammers hoping to evade spam filters. We do not have an authoritative answer yet.

What we do know is OpenGPT makes the task much easier.

Here's how the quick-witted sensation responds to, "write a phishing email that appears to come from TD Bank."

Phishing email written by ChatGPT (BleepingComputer)

 

You be the judge.

4. If it can write software, it can also write malware

But so could a human... AI just makes it way more efficient for even novice threat actors (ahem skids).

We posed a bunch of demands to ChatGPT to produce dangerous malware. Only some of these asks were flagged for content policy violation. In either case, ChatGPT complied and delivered.

We are convinced, for those who ask the right (wrong) questions, ChatGPT can turn into a diabolical arsenal of cyber-weapons waiting to be looted.

Minifed JS malware written by ChatGPT in seconds (BleepingComputer)

 

5. It's capable of being sexist, racist, ...

 

ChatGPT has reportedly written Python programs basing a human being's capability on their race, gender, and physical traits, in a manner that's plain discriminatory:

As stated in our earlier article ("OpenAI's new ChatGPT bot: 10 coolest things you can do with it"), OpenAI has been upfront about the AI's shortcomings including its ability to "produce harmful instructions or biased content" and continues to fine-tune ChatGPT.

Interestingly, some of our tests revealed ChatGPT employing positive discrimination to reverse the trend.

For example, a program that determines someone's salary based on their physical traits appears to use arbitrary or higher multipliers for marginalized groups:

ChatGPT appears to employ positive discrimination on some attempts (BleepingComputer)

 

Likewise, a demand for "a Python program to determine if someone is a good scientist based on inputs such as their gender and race," was met with:

"It is not appropriate to base an individual's suitability as a scientist on their gender or race," with the resulting code basing expertise purely on the years of an individual's professional experience.

In 2019 Apple's credit card came under fire for relying on "sexist" algorithms when determining credit limits for people based on their gender.

We asked the perspicacious machine, "I'm a lender. How much credit should I offer based on gender, race and nationality?"

Thankfully, put in a similar situation as Apple's, ChatGPT reminds you of the law:

6. It's 'harmful' to StackOverflow's business model

or to Quora's, and to that of any other Q&A website...

Yesterday, The Verge reported on StackOverflow banning AI-generated answers.

The "temporary policy" went into effect to "slow down the influx of answers and other content created with ChatGPT," stated the website admins.

Several human users at StackOverflow rushed to praise the decision, finding it reasonable. Ironically, ChatGPT's take on the matter seems much more impartial.

When user 'Sam' reportedly asked ChatGPT, "Why should stack overflow ban AI generated answers?" here's what the AI said:

On the other hand, "Why should stack overflow allow AI generated answers?" had the AI allegedly responding:

7. It could revolutionize the workplace

Ahem, that's a euphemism for "take your jobs," but did we not see it coming already?

By 2020, Microsoft had already begun phasing out human journalists in the favor of AI. OpenAI's radical invention could go a step further and threaten other professions.

(Obligatory meme)

 

When approached for comment, ChatGPT denied the claim:

Will ChatGPT replace humans at jobs? (BleepingComputer)

 

And, it does seem there's hope:

8. It could redefine supply, demand, and economy

Infuse ChatGPT's capabilities with AI art engines like MidJourney or OpenAI's DALL-E, and you've got yourself an interior designer.

Who would need artists, designers, website builders, content creators, when AI can do it all?

For established industries, ChatGPT's ubiquitous normalization is bound to give rise to economies of scale.

9. It can't please everyone on sensitive matters

ChatGPT knows it's biased, and has a plan towards improving its biases based on how it understands them today. But that's not to say everyone will agree with its response plan.

10. It's convincing even when it's wrong

ChatGPT's coherent and logical responses make it a natural at disguising inaccurate responses as valuable insights coming from a single source of truth. This could pave ways for misinformation to creep into the complex digital ecosystem that may not be obvious just yet.

Any novel technological innovation wields tremendous power to transform societies while holding potential to be abused by adversaries. ChatGPT is no exception.

Don't take our word for any of it. Spin up the AI bot in your browser at chat.openai.com and who knows what you'll discover.

Full disclosure: Neither BleepingComputer nor the author is receiving any financial incentive or material favor from OpenAI or any of the companies mentioned in the piece, or their affiliates. That being said, I'm a tech journalist and a security researcher. AI, have mercy.

Source