Are your security investments actually reducing risk?

For Cybersecurity Awareness Month, over the past three weeks, I’ve written about attackers exploiting legitimate identities and trusted IT processes, detecting what happens after they gain access, and whether automation helps security teams investigate and respond.

We should be able to explain what that work has changed. If leadership asks what an attacker would struggle to do today that they could have done six months ago, we need an answer supported by evidence. A list of products and a green dashboard asks them to take the protection on faith.

Two incidents, different outcomes

Cloudflare’s July 2022 phishing incident is a useful example. Three employees entered their credentials into a phishing site, but the attackers couldn’t complete the sign-ins because physical security keys were required. Cloudflare reported that none of its systems were compromised in that attack, despite the passwords being stolen.

The British Library’s October 2023 ransomware attack exposed a recovery problem. Its incident review described having secure copies of its digital collections, but lacking viable infrastructure on which to restore them. The data had survived, yet services couldn’t simply be brought back online.

Supporting reference: Cloudflare: The mechanics of a sophisticated phishing scam and how we stopped it (August 9, 2022) ↗

Supporting reference: British Library: Learning lessons from the cyber-attack (March 8, 2024) ↗

Test the controls and the response

When we report that MFA is deployed, we should know whether that includes administrator accounts, supplier access and any exceptions. When we report that backups are working, we should know how long it takes to restore a critical service, including the identity, application and infrastructure it depends on.

I’d expect the same detail from detection and response. What happened during the last test of a compromised account? Did the team receive enough information to investigate, and how long did it take to remove access? Would that response have worked at 2 am?

That also gives us a way to judge the automation discussed last week. We can measure whether it helped the team gather evidence and contain the activity, then check that access was actually removed.

Bring evidence to the next review

I’m cautious about using fewer alerts or faster ticket closure as evidence of reduced risk. A missing data source can make the dashboard look quieter, and a closed ticket can still leave an attacker with access. Paying a provider to close those tickets doesn’t make that a better measure.

For the next leadership review, I’d bring the test results, the gaps they exposed, and an owner and deadline for each fix. Once the changes are made, repeat the tests and report what improved. If the same attack still works, the issue stays open, regardless of whether the deployment project has been signed off.