Generative AI and Fraud

What are the risks and how do we stay safe?

16 November 2023

Share this article

Since the release of highly accessible generative AI tools such as ChatGPT, interest and excitement around artificial intelligence’s potential has surged. With the increased coverage of news surrounding the space, our Fraud Team have provided an overview of generative AI and the potential fraud risks.

What is generative AI?

Whereas traditional machine learning AI models primarily focus on recognising patterns from existing data to learn and make predictions, generative AI can produce new content based on existing sets of data and inputs. Outputs aren’t limited to analytical answers, generative AI can create novel content as outputs in the form of text, imagery, video and more.

Interest in generative AI has exploded over the last 12 months and its potential to change our world is staggering, with recent research published by Goldman Sachs Research team suggesting that it could drive a 7% increase in global GDP, amounting to nearly $7 trillion over a decade. The research suggests it could play a significant role in breaking down communications barriers and driving productivity growth, bringing about sweeping changes in the global economy.

But as exciting as this technology may be, fraudsters could also use generative AI to their advantage. Just like the internet, despite the positive usage of generative AI, there are some users that may act with malicious intent…

What are the potential fraud risks to be aware of?

Our Fraud Team outline some potential risks to be aware of and guidance on staying safe online:

More convincing phishing attempts: Fraudsters could use generative AI to produce more convincing (and more persuasive) phishing emails and messages. This technology brings the ability to tailor content and correct typos or grammatical errors, which could dramatically reduce our ability to notice when something is off. Generative AI could also learn how to mimic the style of a specific brand, making it easier for fraudsters to impersonate a company and trick people into sharing their data. You can read more about phishing scams here.
More frequent attacks: Generative AI can enable fraudsters to automate huge chunks of their work so they can target victims further and faster. By removing language barriers, this technology can also help cybercriminals to increase their global reach.
Deep fakes: Across voice, image and video, AI has improved its ability to mimic real humans. Although we believe cases involving the impersonations of family, friends and colleagues to extort money are likely to be low volume, they could be highly damaging when they do occur. Fraudsters will need to invest more time in researching and formulating their approach. However it is easy to see how deep fake content could cause someone to panic and transfer money to the wrong hands; therefore not answering calls from numbers you don’t recognise is a way to avoid exposing yourself to scams involving this technology.

Tips to stay safe:

Many of the scams we are already aware of (e.g. cloned websites, romance and charity relief scams) will continue to cause problems. But generative AI can make them more convincing and more prevalent.

This means that much of the tips to stay safe not only remains relevant, but increases in importance. Make sure you know who you’re in contact with before giving out any personal information; avoid clicking on links in emails or texts from senders you don’t recognise; don’t let anyone rush or pressure you into making a decision.

However, as traditional markers of fraud start to disappear (e.g., spelling errors in phishing emails) it becomes even more important to know when something is not right – trust your instincts. Having strong controls, passwords and security on social media also becomes a bigger priority, as oversharing or sharing with the wrong people can provide content that can be analysed or mimicked.

Find out more about steps you can take to protect yourself from fraud.

Find out more

The content in this article is for information only and is not advice. It uses sources from GS’ Global Investment Research Division, Marcus by Goldman Sachs and third parties. All content in this article was accurate on the date of publication shown above.