What can go wrong with a phishing test?
The best-known UK example is West Midlands Trains. On 21 April 2021 it sent around 2,500 staff an email, apparently from the managing director, thanking them for their work through the pandemic and promising a one-off payment. Staff who clicked found it was a phishing simulation. The Transport Salaried Staffs’ Association described the company’s behaviour as “totally crass and reprehensible”, pointing out that one employee had died of Covid-19 and others had fallen ill.
As a lure, it was effective: timely, plausible and pitched at exactly what people wanted to hear. That is precisely why it was the wrong choice. A test that succeeds by exploiting gratitude teaches staff that the security team is not on their side, and people do not volunteer information to a team they feel is trying to catch them out.
Tests can also simply confuse people. In a previous organisation I worked in, the IT team sent a phishing test announcing that the marketing team had won an award and had free iPads to give away. In fact, the marketing team really had won a couple of iPads from a supplier and was planning to hand them out at the next all-hands meeting. Nobody knew which emails to believe.
Another test I saw at a previous employer promised staff cash prizes. There was no prize at the end of it, which created a lot of bad feeling among the people involved.
Do phishing simulations actually work?
The evidence is humbling. The largest study to date, at UC San Diego Health, sent 10 simulated phishing campaigns to more than 19,500 employees over eight months. The researchers, led by Grant Ho and presented at the IEEE Symposium on Security and Privacy in 2025, found no significant link between completing annual mandated training and the likelihood of clicking. Embedded training, the page people see after clicking a test link, reduced the likelihood of clicking by just 2 per cent, and three quarters of people spent a minute or less on it.
Clicking also rose over time. In the first month, 10 per cent of employees clicked a phishing link; by the eighth month, more than half had clicked at least once. And the lure mattered far more than the training: 1.82 per cent clicked a link to update their Outlook password, while 30.8 per cent clicked a supposed update to the organisation’s holiday policy.
In other words, a click rate tells you more about the person who wrote the test email than about the people who received it. Change the lure and you can produce almost any result you like.
How do you make a phishing test effective without making it cruel?
The starting question is one many organisations never ask: is the goal to catch people, or to change what they do next? Most measure the first and claim credit for the second. In my experience, the balance comes from five rules.
1. Measure reporting, not just clicking
How many people flagged the email, and how quickly, is the number that matters when a real attack lands. The National Cyber Security Centre makes the same point in its phishing guidance: as well as measuring how many people clicked, focus on how many reported it.
2. Agree a no-go list with HR before you start
Pay, bonuses, redundancy, health and bereavement should be off limits, whatever real criminals do. The NCSC advises checking with HR before running any simulation, because punishing people for clicking on emails you have sent yourself starts to resemble entrapment.
3. Mirror the threats your business actually faces
Fake invoices for the finance team and fake document shares for everyone else are realistic enough without being cruel. The aim is to rehearse the attacks your organisation is likely to see, not to win.
Look beyond email, too. Attackers also target the points where passwords and multi-factor authentication are reset, often by phoning the help desk. In the 2025 Marks & Spencer attack, threat actors impersonated an employee to trick a third party into resetting a password. Testing identity checks at reset points is as important as testing inboxes.
4. Treat a click as a private learning moment
Keep it short, private and practical. No league tables, no naming and no shaming.
5. Back people with controls
People who click repeatedly need a conversation and stronger protection, such as phishing-resistant multi-factor authentication, not public criticism. No training can teach everyone to spot every phishing email, which is why the NCSC recommends layering technical controls alongside it, an approach that sits at the heart of Zero Trust security.
Why does this matter for the business?
There is a hard business case too. The UK Government’s Cyber Security Breaches Survey 2025/2026 found that phishing was the most common type of breach or attack, experienced by 38 per cent of businesses, and was the type most often rated the most disruptive.
When the real attack arrives, the employee who reports it in the first five minutes is worth more than a hundred caught in a test. Embarrass people, and they stop telling you. Thank them, and they become one of the best early-warning systems you have.
What this means for your organisation
Run phishing simulations to build a reporting habit, not to produce a league table. If the score you report to the board is a click rate, consider replacing it with how many people reported the test, and how fast. The question is not how many people you caught. It is how many will tell you next time.
If you would like to see where your organisation stands, our phishing security test gives you a baseline, and our security awareness training and email security and human risk services help you build the reporting culture and technical controls around it. Get in touch to talk it through.