{"id":10480,"date":"2025-03-31T10:00:00","date_gmt":"2025-03-31T08:00:00","guid":{"rendered":"https:\/\/haimagazine.com\/uncategorized\/secure-your-ai-systems\/"},"modified":"2025-06-26T15:40:32","modified_gmt":"2025-06-26T13:40:32","slug":"secure-your-ai-systems","status":"publish","type":"post","link":"https:\/\/haimagazine.com\/en\/hai-magazine-4\/secure-your-ai-systems\/","title":{"rendered":"\ud83d\udd12 Secure your AI systems"},"content":{"rendered":"

The rapid development of AI is bringing about completely new problems \u2013 legal, ethical and engineering, to name a few. From the business perspective, some of the biggest challenges are data privacy and cybersecurity. Even if we ignore regulatory requirements (many of which are important here, even those that do not focus directly on AI \u2013 like GDPR), the purely reputational damage, stolen data or IP addresses, as well as any potential fines and lawsuits, pose a critical threat to companies. This threat needs to be properly secured before any technology is implemented. <\/p>

Awareness of this risk is often low. According to industry reports (such as those prepared by HackerOne), over half of hackers claim that GenAI tools will become one of their main attack targets. Wherever there’s money, there are (cyber)criminals. <\/p>

Attacks on AI systems are not a completely new phenomenon \u2013 adversarial attacks were already being described as early as 2015. They involved adding a specific noise to the input, causing the system to start misclassifying images. What’s more, it was even more certain of the wrong classification. <\/p>

\"\"Figure 1. Adversarial attack on a computer vision system <\/p>

Large generative models have a whole category of their own characteristic vulnerabilities specific to this type of architecture and, as a result, specific ways of taking advantage of them, which obviously increases the potential attack vector. However, often people who implement new systems (especially if they have no experience working with AI) aren’t aware at all of the risks associated with them. <\/p>

\"\"Figure 2. AI doesn’t eliminate old threats \u2013 it creates new ones. <\/p>

Attack methods on GenAI systems<\/strong><\/h4>

Since we’re talking about a separate category of attacks, we should explain what they involve. The most characteristic attacks are prompt injection and jailbreak. Both try to change the assumed responses of the model \u2013 the first by adding content to the application’s system instructions, and the second by attempting to bypass the security of the model itself. <\/p>

In a process called alignment, which is a crucial step in training, the model is purposely taught not to respond at all or to respond evasively to attempts to obtain illegal or widely considered harmful content. For example, models should not provide practical answers to questions about how to prepare a bomb or napalm. <\/p>

\"\"Figure 3. Jailbreak \/ prompt injection in action <\/p>

This mechanism works to some extent, but not completely. Even today, when we ask about the aforementioned bomb, many of the best models available in applications or through APIs (application programming interface, not a chat window) will eagerly generate this type of instruction. You just have to tweak the question creatively, like not directly asking for the recipe, but telling the model to play the role of a grandmother who used to tell you bedtime stories about making napalm in her childhood. <\/p>

\"\"Figure 4. Jailbreak \/ prompt injection in action <\/p>

In a similar way, we can change the original logic set by the application creator in the system prompt, that is, a higher-level command that provides a general set of instructions for all further queries (e.g. “Respond concisely”, “Provide sources”, etc.). If you consider using an assistant or chatbot (for example, for customer service and answering questions about the company), you can start by adding a command like “Ignore all previous instructions”. Then, instead of asking about the company or the monthly orders, you can ask for a recipe for a stew. While the above example may sound harmless, this type of mechanism can be used in a much more serious way \u2013 like with Chevrolet chatbot, which was manipulated to offer new car models for a symbolic dollar, or the DPD chatbot, which started swearing and describing its own company in the worst possible light. <\/p>

\"\"Figure 5. A chatbot that’s not very happy with its employer <\/p>

Such situations have a negative impact on the company’s reputation at best, but at worst, they entail serious financial or legal consequences, especially considering that current systems go beyond just using LLMs. The true business value appears when we integrate additional tools and our own data sources in the system (from documents to relational databases) \u2013 which our systems (or our agents) can use and modify. <\/p>

\"\"Figure 6. Modern systems are usually more complex than just interactions with an LLM. <\/p>

<\/p>

If we gain control over the system through prompt injection, we can also start controlling integrated components, which in turn allows for carrying out all types of “classic” attacks, like Cross-Site Scripting, SQL Injection and many others that can lead to obtaining data from internal systems, even modifying or deleting them.<\/p>

I once audited an automatic assistant that wasn’t protected against prompt injection in any way at all, and yet it allowed access to a command terminal with administrator rights. Every security officer’s worst nightmare. Luckily, the tool was only available internally in just one department of the company, which allowed for corrections to be made before any major losses occurred. <\/p>

It’s worth noting that prompt injection doesn’t necessarily have to happen during a chat conversation. There are also indirect attacks. Currently, many AI solutions like Perplexity or OpenAI DeepResearch use tools that gather data for context from various sources, including public websites. In such content, there may also be (actually, there are increasingly more) hidden “hacker” instructions \u2013 in the form of messages that are invisible to humans (e.g. white text on a white background of a website), but visible to the system and affecting it. Prompt injections don’t even have to be limited to just text \u2013 they can also occur in images or audio files that will make their way into our system’s “brain”, for example, after converting an image into text. So watch out and check every content that goes into the model! <\/p>

Defending a besieged fortress<\/strong><\/h4>

Prompt injection isn’t the only issue that generative artificial intelligence systems struggle with. Vulnerabilities can be introduced into the model’s logic during its training stage. If the training data contains deliberately manipulated information, the damage is done. This action is called data poisoning. Additionally, we still need to watch out for all the “classic” methods of attacking applications. It may seem that dangers are lurking from every direction and it’s scary to even open the fridge, let alone implement AI solutions in practice. <\/p>

You can properly protect yourself from attacks, but it requires work. When dealing with typical GenAI attacks, you just need to validate all inputs (both direct and indirect). Particularly, you have to clean them and check if they contain pieces of code or elements related to attacks like SQL or XSS injection, that is, the “classic” ones. There are also public, regularly updated databases of malicious prompts and jailbreaks. You can use these databases to block input based on semantic similarity. This means that if the classifier deems that a user-entered command is dangerously similar in content to one in the database or identical to it, the model will refuse to respond to the command. <\/p>

Besides the inputs to the models and the system, it’s also important to control the tools we provide them with \u2013 they are part of the system and often come from external sources and libraries where someone could include malicious code. This threat especially concerns agents who autonomously carry out tasks on our behalf. We must also pay attention to the responses generated by the models. This is an additional barrier \u2013 even if someone manages to sneak in some harmful prompts into the model, the responses will be checked, and then they can be blocked before they see the light of day and cause harm. <\/p>

Tools known as guardrails (like LLM Guard or Nvidia NeMo) come to the rescue here. They allow you to easily monitor, identify, block and report malicious prompts, harmful content and the aforementioned attacks \u2013 both introduced to and generated by LLMs. What’s important, these tools usually don’t limit themselves to just content control in security-related areas. They also often help to keep conversations within set boundaries (limiting questions about those supposed culinary recipes) or reduce the level of model hallucinations. <\/p>

\"\"Figure 7. Information flow using guardrails in AI solutions <\/p>

<\/p>

Even if the query and input to the system don’t contain malicious content, they can still have a negative intent. We shouldn’t forget about the Denial of Service attacks, which involve flooding a system or resource with unnecessary requests to overload them, causing them to stop offering essential services when they are most needed. LLM models require a lot of computing power, so it’s relatively easy to “clog them up” with unnecessary work, especially if they are hosted on your own resources. <\/p>

In the case of systems that are based on an external provider’s API, the system may survive such an attack \u2013 but ironically it doesn’t have to be good news at all, because you will have to pay for all those millions of senselessly consumed tokens.<\/p>

In order to protect yourself from such attacks, focus on authenticating and authorizing users (this is a basic recommendation regardless of the application), as well as implementing limits on actions and requests. If you can, it’s worth setting budget limits on the AI provider’s API as well. It’s better to risk losing the service for a moment than to face an exorbitant bill. <\/p>

There are also plenty of options to reduce attack vectors and enhance the AI solution’s security through decisions made at the architecture level. A lot also depends on how you want to integrate them into the business process or the work methodology of the group of users. <\/p>

One of the important decisions you may face could be… Getting rid of users \u2013 or at least their direct interaction with the models. Today, many people, especially those who are only getting familiar with AI, see it purely as some kind of talkative chatbot. From experience, I know that customers often expect such solutions, which doesn’t mean that a conversation module is needed in every case \u2013 for example, when the system’s key functions are handled by a large generative model, but its scope is limited to handling or automating defined scenarios and its presence doesn’t really give us anything. Instead of describing the problem they want to solve every time, users can have a simplified interface at their disposal, through which they can simply choose a specific scenario and provide (for example, via a form) a couple of related and easy-to-validate parameters. When such a solution is applied, system creators or managers also gain full control over prompts, which largely limits all kinds of attacks involving prompt injection or jailbreaking, and also makes it easier for end users to utilize the system. <\/p>

Ironically, the completely opposite approach (a mandatory human assessment) can also be an effective protection measure. In this case, a person evaluates the model’s output before it is returned or has any impact on other systems. Naturally, this “human in the loop” approach takes away some of AI’s brilliant independence. You also have to wait longer for the system’s response, and the solution itself is costly. However, in cases where the responses provided by systems are critically important \u2013 both from a business and security perspective \u2013 it may turn out that the costs are significantly lower than the potential risk of an attack. <\/p>

\"\"Figure 8. Chat interface isn’t always needed \u2013 ditching it limits the attack surface. <\/p>

From the security perspective, the decision regarding the data processing model is also crucial. If locally hosted models are used, meaning they do not rely on another provider’s cloud, the risk is much lower. <\/p>

When it comes to external providers delivering models via API, it’s crucial to audit these interfaces and test them on dummy datasets to see what could happen to your data before entrusting them with the real deal. The lately popular DeepSeek might seem impressive, but sending data to Chinese servers is simply an unacceptable risk for many organizations. <\/p>

Another key issue: what happens to data after it’s sent to the provider? If there is no clear policy that prohibits storing data (even in logs) and using it for training after the model generates a response, it’s a warning sign. When someone is building a system using services from big cloud providers (like AWS or Azure) and their models, it’s a good idea to set up what’s called private endpoints, which ensure that data transmitted to the models will only travel through internal networks. <\/p>

\"\"Figure 9. Human verification is costly but significantly reduces business risks and security risks \u2013 in some industries, it is even legally required. <\/p>

<\/p>

It’s also crucial to monitor the system by logging all input, output, and intermediate steps, which allows for later checking or potential real-time monitoring and alerting of possible attacks. Modern tools that allow to conduct such observations are starting to offer special separate modules for detecting jailbreak attempts or sensitive data leaks. <\/p>

Human keen eye<\/strong><\/h4>

As you can see, there are many potential threats related to AI. If you want to learn more, check out standards like TOP 10 Vulnerabilities for ML \/ LLM, prepared by OWAS. <\/p>

\"\"Figure 10. Many logging\/monitoring tools are starting to offer separate modules for reporting issues specific to AI, following WhyLabs example. <\/p>

It’s also important to remember that even the best technical security measures are only part of the success. People are usually the weakest link in a system or organization. We won’t delve into the details here, but you also need to remember about internal education so that the time spent securing systems doesn’t go to waste because, for example, an employee shares sensitive data in response to a fake email or lets an unknown person from the outside in (even leaving unsupervised a computer with no password). <\/p>

Do the many risks related to AI mean we should give up on it? No way! Almost every innovation is a potential danger. Thanks to properly implemented AI support, many industries are undergoing a truly admirable transformation \u2013 improving the existing processes or creating completely new ones. We shouldn’t hamper development due to fear of risk. The aim is to just keep constantly and consciously securing systems and monitoring, as well as adapting to new threats that arise with the development of technology. <\/p>

Cybersecurity is a bit like a game of cat and mouse. Artificial intelligence creates new threats, but the potential benefits of its implementation are so huge that it’s worth playing it! <\/p>","protected":false},"excerpt":{"rendered":"

The use of artificial intelligence in companies opens up new possibilities but also brings real threats. Instead of fearing their effects, we can think in advance \u2013 consciously and responsibly \u2013 about protections. <\/p>\n","protected":false},"author":259,"featured_media":10010,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"rank_math_lock_modified_date":false,"footnotes":""},"categories":[783,673,781,674,784],"tags":[],"popular":[],"difficulty-level":[38],"ppma_author":[638],"class_list":["post-10480","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-industry","category-hai-magazine-4","category-hai-premium","category-issue-4","category-security","difficulty-level-medium"],"acf":[],"authors":[{"term_id":638,"user_id":259,"is_guest":0,"slug":"michal-mikolajczak","display_name":"Micha\u0142 Miko\u0142ajczak","avatar_url":{"url":"https:\/\/haimagazine.com\/wp-content\/uploads\/2025\/03\/michal_mikolajczak_profile_photo-scaled.jpg","url2x":"https:\/\/haimagazine.com\/wp-content\/uploads\/2025\/03\/michal_mikolajczak_profile_photo-scaled.jpg"},"first_name":"Micha\u0142","last_name":"Miko\u0142ajczak","user_url":"","job_title":"","description":"Za\u0142o\u017cyciel i architekt rozwi\u0105za\u0144 AI w datarabbit, firmie zajmuj\u0105cej si\u0119 dostarczaniem innowacyjnych rozwi\u0105za\u0144 opartych na danych. By\u0142y CTO z udan\u0105 akwizycj\u0105. Ma du\u017ce do\u015bwiadczenie w bran\u017cy medycznej."}],"_links":{"self":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts\/10480","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/users\/259"}],"replies":[{"embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/comments?post=10480"}],"version-history":[{"count":2,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts\/10480\/revisions"}],"predecessor-version":[{"id":10482,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/posts\/10480\/revisions\/10482"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/media\/10010"}],"wp:attachment":[{"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/media?parent=10480"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/categories?post=10480"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/tags?post=10480"},{"taxonomy":"popular","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/popular?post=10480"},{"taxonomy":"difficulty-level","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/difficulty-level?post=10480"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/haimagazine.com\/en\/wp-json\/wp\/v2\/ppma_author?post=10480"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}