LLMs’ dangerous weak spots: cats, Dr. House, poetry and authority figures
In theory, they’re resistant to manipulation. In practice, a cleverly phrased prompt can push them to work around their own safeguards. Language models can handle very long contexts, but they still get swayed by subtle cues—from poetic style to unusual asides. So where does that vulnerability come from?
LLMs are really good at doing exactly what they were built for. Looking at everything in the context window, they compute a probability for each of the hundreds of thousands of possible tokens in their vocabulary. Based solely on those probabilities, one token is picked from all the options—the one that gets sent to the output and appended to the running context. That expands the context, and then the process repeats to generate the next token.
Until we put the model to work on something concrete with a verifiable result, there’s really nothing in this process to poke holes in.
Share
You might be interested in
LLMs’ dangerous weak spots: cats, Dr. House, poetry and authority figures
In theory, they’re resistant to manipulation. In practice, a cleverly phrased prompt can push them to work around their own safeguards. Language models can handle very long contexts, but they still get…
Mistral 3: unveiling the new generation of open-weight models
Mistral AI is introducing the Mistral 3 model family, which includes the Large version with MoE architecture and the Ministral series for local devices. All models are available under the Apache 2.0…
Amazon’s new venture – AI in companies is set to pay off
Instead of focusing on model specs, Amazon has unveiled an ecosystem of agents and tools designed to have artificial intelligence do real work, not just spit out responses. And it’s supposed to…
AI on the headset. Partner or rival to the consultant?
Not long ago, automation in Contact Centers was mainly associated with basic IVR scripts and the mechanics of call queuing. Today, with the maturation of generative artificial intelligence, we’re stepping into a…
