Kevin Beaumont 🌈 @GossiTheDog I tweet the weird stuff. Security Operations Centre Manager. Won Best Technical Blog at InfoSec 2019. Tweets not those of my employer. Dec. 11, 2018 4 min read

Equifax report megathread while I'm on lunch!  https://oversight.house.gov/wp-content/uploads/2018/12/Equifax-Report.pdf 

Within 3 days of Apache Struts vuln being announced, Equifax used unknown open source code to scan their infrastructure for Struts vuln, using an IP sweep. This didn't work as they needed to know the URL of the Struts webapp. A day later, somebody ran 'whoami' against servers.

4 days later, they installed Snort rules for detection of exploitation. Unfortunately as they later discovered, the SSL/TLS reverse decryption certificate had expired long ago so the IDS didn't scan any traffic.

Two months later, the Struts patches still hadn't been deployed and the IDS wasn't working still. Attackers deployed web shells on the web servers to maintain persistence in case of later patching. They realised the connected application servers weren't in a DMZ, so owned those

By this point they were inside Equifax's corporate network, thanks to a system first implemented decades ago. The system (which processed credit cards) didn't have FIM (a PCI requirement). The attackers deployed 30 web shells through Equifax, accessing 40 other databases.

A further two months after attackers had access to the Equifax corporate network, they realised the SSL certificates had expired on a total of 67 of their hosted websites. They fixed the certificate and immediately noticed the issue.

Equifax staff jumped on it, but couldn't see the full picture as they missed prior logs - instead they saw something else being exploited (it's clear the Equifax breach goes far beyond what was reported prior, by the way). They blocked an IP from China.

The following day they vuln scan the ACIS system and realise it has SQL injection vulns (which weren't even being exploited here, that's another issue). At mid day somebody tells the CISO in a phone call about the issue, who kicks off full blown incident response.

Another day goes by. The Chief Legal officer and deputy don't respond to emails or calls. The CIO is notified.

The investigation team discover two unauthorised JSP files, at which point they image the servers.

A day goes by, the CIO tells the CEO about the investigation. External counsel hired Mandiant to take over investigation. They investigate for 2 months. A month into that time - Sept 1 - the Equifax board are told of the scale of the issue by Mandiant.

A week later, Equifax publicly disclose the issue.

Through September and October the CIO and CISO both took early retirement, the CEO left the company, and Senior VP CIO was fired.

I've heard along the grapevine they have a great new CISO and other security staff now, and they're turning things around.

Things you can look at detecting: people running whoami (or whoami.exe on Windows), people mounting filesystems. Things for strengthening - I wouldn't present Struts to internet; too exploitable. Firewall app servers.

Another learning point: 67 of Equifax’s self hosted webapps can’t have generated any IDS alerts for almost two years due to expired SSL inspection certs. If you aren’t getting any IDS alerts, you need a process to detect, and proritise remediation.

For non techies IDS is intrusion detection system, it’s like a smoke alarm to say your systems may have caught fire. They go off sometimes in error and that’s super annoying, but if they stop working for two years and you have a fire, you’re going to have a problem.

Also if you must use Struts, enable SELinux or AppArmor. They’re baked into the OS, free, easy to use, and stop commands being run by Struts. They’re also enabled by default on Redhar, orgs end up disabling them 😅  https://doublepulsar.com/hardening-apache-struts-with-selinux-db3a9cd1a10c 

Another learning - in late 2000s Equifax moved the InfoSec function from CIO to report to the head of legal, who they also called "Head of Security". He had no IT or security background, and headed their legal function. CISO wasn't at any board or senior leadership meetings.

I should point out post breach Equifax have moved the CISO's function back under the CIO. I know there's a strong feeling in InfoSec you shouldn't ever do that, but sometimes it works better - Information Security can sometimes work influencing rather than shadow boxing.

Another one is they were trying to get PCI compliance for the breached system (which took credit card data since forever) during the last year of its life, but never finished the process. That would have introduced FIM etc.

Final one! Equifax's Group Security budget was $38m a year at time of breach, it is now four times that, with board level engagement.

Actually one more on way home. Equifax outright fired one employee over this for not forwarding Struts email, they announced to press during hearings and CEO blamed him as sole flaw. Read his side. One of 400+ people at company on email, not responsible in process or policy.

He had nothing to do with the system at fault or development or Struts. He had been there 7 years and never got asked to forward anything. That’s a short straw moment, and transparently awful bus chucking from above.

If your security posture of all your core business depends on one random dude with no responsibility on a 400 person distribution list forwarding an email randomly (to who?), you done IT badly. The report does call this out.

Anyhoo, these are the kind of problems most companies face - the report is a greatest hits of CMDBs, patch mgmt etc. Important thing for me (aside from our new Splunk searches 😀) is learning points, like don’t have legacy IT with no voice InfoSec reporting to Legal head.

Threat modelling is important, as another point the report doesn’t touch. Credit agencies will be very high targets for nation states. Their chance of ‘the nightmare hack’ is high. Not all orgs have that threat profile.


You can follow @GossiTheDog.



Bookmark

____
Tip: mention @threader_app on a Twitter thread with the keyword “compile” to get a link to it.

Enjoy Threader? Sign up.