It sounds like science fiction but it’s not: AI can financially destroy your business

Scammers last year stole about $11m from unsuspecting consumers by fabricating the voices of loved ones, doctors and attorneys requesting money

Everyone seems to be worried about the potential impact of artificial intelligence (AI) these days. Even technology leaders including Elon Musk and the Apple co-founder Steve Wozniak have signed a public petition urging OpenAI, the makers of the conversational chatbot ChatGPT, to suspend development for six months so it can be “rigorously audited and overseen by independent outside experts”.

Their concerns about the impact AI may have on humanity in the future are justified – we are talking some serious Terminator stuff, without a Schwarzenegger to save us. But that’s the future. Unfortunately, there’s AI that’s being used right now which is already starting to have a big impact – even financially destroy – businesses and individuals. So much so that the US Federal Trade Commission (FTC) felt the need to issue a warning about an AI scam which, according to this NPR report “sounds like a plot from a science fiction story”.

But this is not science fiction. Using deepfake AI technology, scammers last year stole approximately $11m from unsuspecting consumers by fabricating the voices of loved ones, doctors and attorneys requesting money from their relatives and friends.

“All [the scammer] needs is a short audio clip of your family member’s voice – which he could get from content posted online – and a voice-cloning program,” the FTC says. “When the scammer calls you, he’ll sound just like your loved one.”

And these incidents aren’t limited to just consumers. Businesses of all sizes are quickly falling victim to this new type of fraud.

That’s what happened to a bank manager in Hong Kong, who received deep-faked calls from a bank director requesting a transfer that were so good that he eventually transferred $35m, and never saw it again. A similar incident occurred at a UK-based energy firm where an unwitting employee transferred approximately $250,000 to criminals after being deep-faked into thinking that the recipient was the CEO of the firm’s parent. The FBI is now warning businesses that criminals are using deepfakes to create “employees” online for remote-work positions in order to gain access to corporate information.

Deepfake video technology has been growing in use over the past few years, mostly targeting celebrities and politicians like Mark Zuckerberg, Tom Cruise, Barack Obama and Donald Trump. And I’m sure that this election year will be filled with a growing number of very real-looking fake videos that will attempt to influence voters.

But it’s the potential impact on the many unsuspecting small business owners I know that worries me the most. Many of us have appeared on publicly accessed videos, be it on YouTube, Facebook or LinkedIn. But even those that haven’t appeared on videos can have their voices “stolen” by fraudsters copying outgoing voicemail messages or even by making pretend calls to engage a target in a conversation with the only objective of recording their voice.

This is worse than malware or ransomware. If used effectively it can turn into significant, immediate losses. So what do you do? You implement controls. And you enforce them.

This means that any financial manager in your business should not be allowed to undertake any financial transaction such as a transfer of cash based on an incoming phone call. Everyone requires a call back, even the CEO of the company, to verify the source.

And just as importantly, no transaction over a certain predetermined amount must be authorized without the prior written approval of multiple executives in the company. Of course there must also be written documentation – a signed request or contract – that underlies the transaction request.

These types of controls are easier to implement in a larger company that has more structure. But accountants at smaller businesses often find themselves victim of management override which can best be explained by “I don’t care what the rules are, this is my business, so transfer the cash now, dammit!” If you’re a business owner reading this then please: establish rules and follow them. It’s for your own good.

So, yes, AI technology like ChatGPT presents some terrifying future risks for humanity. But that’s the future. Deepfake technology that imitates executives and spoofs employees is here right now and will only increase in frequency.

Source