The UK’s cybersecurity agency has warned businesses to be cautious when integrating AI-powered chatbots due to potential security risks.
The National Cyber Security Centre (NCSC) today warned that algorithms designed to replicate human interaction, which are increasingly being used for customer service chatbots, could be hijacked by malicious hackers into making unauthorised actions.
The so-called “prompt injection” attack could be of particular concern for financial institutions, such as banks, which could leave customer money at risk to rogue users putting in the right prompts to ‘trick’ the chatbots.
Giving large language models (LLMs) access to sensitive company information puts businesses and their customers at risk, the NCSC warned.
Amid conversations about generative AI being used to replace the work of real people, customer service workers have consistently been named as an at-risk industry to be replaced by LLMs, such as ChatGPT.
Despite it still being early days, generative AI has already been incorporated into customer service roles, such as H&M’s digital stylist chatbot.
“Organisations building services that use LLMs need to be careful, in the same way they would be if they were using a product or code library that was in beta,” wrote the NCSC in a blog post.
“They might not let that product be involved in making transactions on the customer’s behalf, and hopefully wouldn’t fully trust it. Similar caution should apply to LLMs.”
Jake Moore, global cybersecurity advisor at ESET, said: “Unfortunately, speed to launch or cost savings can typically overwrite standard and future-proofing security programming, leaving people and their data at risk of unknown attacks. It is vital that people are aware [that] what they input into chatbots is not always protected.”
Recent research from KPMG has found that generative AI systems have the potential to provide a £31bn boost to the UK economy. However, regulators are yet to iron out safeguards for the technology.
In November, the UK will host a global AI Safety Summit at the historic Bletchley Park to address these risks with the international community.
Read more: UK summit is first step on long road to global AI safety