The Pulse: Time to rethink software security because of AI?

Hi, this is Gergely with a bonus, free issue of the Pragmatic Engineer Newsletter. In every issue, I cover Big Tech and startups through the lens of senior engineers and engineering leaders. Today, we cover one out of four topics from last week’s issue of The Pulse. Full subscribers received the article below eight days ago. If you’ve been forwarded this email, you can subscribe here.

It’s been an absolutely wild few days, with security breaches and major security vulnerabilities being exposed worldwide across tech. Many incidents took place on Tuesday and Wednesday of this week. We’ll start with a brief summary, then look closer at three of them:

1. Intermediate CA certificate compromised: eight private 384-bit keys issued by eMudhra – which is an intermediate Certificate Authority (CAs) – leaked, meaning that servers in server-to-server communication can face man-in-the-middle (MITM) attacks. Any internet communication not going through modern browsers (that already patched this) could be risky.
2. PHP: Unpatched servers risk remote code execution (RCE) and opening up access for attackers. Around 70% of all websites run on PHP and unpatched systems are at risk. Two critical security patches have been released.
3. Railway: customer secrets stored on the service were leaked and some have been decrypted. Secrets exposed can include access to production systems, enabling attackers to potentially access customer systems.
4. GitLab: Unpatched GitLab servers are at risk of an attacker gaining full access. A critical security patch for RCE has been released.
5. Node.js: Unpatched servers are vulnerable to remote execution attacks. A new release fixes 10 critical vulnerabilities (CVEs), including one that allows for RCE, and another fix revokes the compromised eMudhra intermediate certificate.
6. Go: a new release is out, adding a mitigation for the compromised eMudhra intermediate certificate.
7. TiDB: attackers gained near-total access to the complete control plane systems of the managed TiDB provider, PingCAP, for a whole month (!!). The attacker released 10TB of customer-sensitive data, including from Booking.com, Airbnb, Pinterest, Databricks, and more such as SQL queries that customers ran, billing, contacts, and source code for PingCAP’s systems The breach lasted from 9 Sept until 7 Oct, but was only disclosed this week. PingCAP claims no customer data stored inside TiDB was accessed, but I find it hard to see how the company will avoid losing security-conscious customers. This incident shows that the absence of adequate access monitoring and alerting can be an open invitation for attacks lasting weeks.

With that, let’s get into the first three incidents in the list.

eMudhra CA compromise

Certificate authorities (CAs) are organizations that vouch for identity on the internet. When a browser, phone, or server pings a site like chase.com or revolut.com, intermediate CAs and digital signatures verify that you really are talking with the bank’s server, and not a fake one set up by an attacker with the purpose of stealing users’ credentials. This is where the HTTPS protocol comes into play, with root CAs, intermediate CAs and the TLS handshake.

The whole system is built on a chain of trust and verification: we trust the root CAs and their certificates, and so we can trust the signatures of the intermediate CAs. The assumption is that both the root CA and intermediate CAs are trustworthy and verifiable.

But eMudhra broke this trust in the most blatant way: its private keys turned out to be single-digit numbers. A private key is a 384-bit key, which means that it would be a number between 0 and 2^384, (a number with 116 digits!)

Except, the key turned out to be the number “4.” Yes, that’s right: the number four. What on earth was eMudhra doing?! It’s as if they chose random numbers between 0 and 384, not 0 and 2^384! It is quite literally comical: in 2007, an XKCD comic described exactly this choice:

And the other keys were also not that much better: 56, 66, 115, 156, 230 and 237! It’s as if eMudhra generated an 8-bit private key, not a 384-bit one.

eMudhra explaining how these keys were generated (emphasis mine):

“We have done some preliminary assessment, there appears to be a flaw in the KGC [Key Generation Ceremony] script that was used on that specific day to create these ECC issuers. As for these ECC keys [elliptic-curve cryptography key pairs], our review of the KGC, assessment of the code and view of HSM [Hardware Security Module] keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the CSR [Certificate Signing Request] (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)

The RSA issuers went through a different path as per our assessment of the code and are therefore not impacted.

Some additional context - these issuers were meant for export to our US and European data centers to solve data sovereignty issues for some of our customers (but never ended up operationally in production except for one).

We are still assessing the code to see steps where this process went wrong.”

I have so many questions:

• How does eMudhra test generated keys? There seems to have been no automated testing of the key in place at all, in order to discover that these numbers were not “random” from a flaw in the key generation ceremony
• How did eMudhra not notice this on day one, when the Key Generation Ceremony went wrong? If this was an automated process, the script would’ve changed from one day to the next, which would raise questions. If the process was fully manual, it explains how the mistake would be missed. And if it was manual, then why? Also, where are the detailed audits for such an important process?
• The certificates with the easy-to-guess private keys were generated on 30 April 2025, so why did eMudhra not spot the mistake for 18 months?

Carelessness in the creation of intermediate certificates risks reputational damage to eMudhra’s business as a certificate authority. Google Chrome announced it is evaluating whether to continue with the company:

“Due to the seriousness of this incident, the Chrome Root Program is closely investigating the ongoing inclusion of eMudhra’s remaining CA certificates in the Chrome Root Store and will share further updates in public forums (e.g., the Common CA Database Public list) as our evaluation progresses.”

Given eMudhra’s sole business is in issuing digital certificates, being dropped by Chrome as a CA could be crippling. eMudhra is a publicly traded company on the Indian stock market, and two days before the breach was revealed its market cap was close to $500M. After details of its incompetence in creating certificates emerged, the company lost 40% of its value in two days.

PHP exploit: AI model beats attackers to finding critical vulnerability

The PHP situation is interesting because the “good guys” in this case (the security researchers) found a critical vulnerability in the framework before attackers did. From OpenAI:

“Over the past two months, we led a coalition of maintainers, researchers, and security providers to address critical vulnerabilities in PHP, an open-source programming language used to run websites and web applications. Calif [a respected security team] identified a critical flaw using GPT‑5.6 Sol as part of our Patch the Planet⁠(opens in a new window) initiative, with Wiz [a security company] also independently discovering and reporting both vulnerabilities. PHP’s maintainers led the fixes and release, with our teams and partners like Wiz, Cloudflare, and others contributing research, technical support, and coordination to prepare protections ahead of disclosure.”

On the surface, this is great news! Critical vulnerabilities were found and can now be patched, thereby closing the door before attackers can exploit them. Right..?

The problem of critical vulnerabilities for open source software is that systems which aren’t immediately patched can be exploited. When the PHP team released the fix for these two vulnerabilities, attackers got details about the vulnerabilities: the code itself shows where the attack can be inititiated! In the case of PHP, that means attackers can now easily execute remote code on any PHP server that is as yet unpatched.

Around 70% of all websites run PHP servers, which is a massive attack surface. We can be confident that many sites won’t update their PHP versions for days, weeks, months, or even years.

This particular vulnerability was so severe that before the security patch, the National Security Agency (NSA) in the US issued guidance urging websites to immediately patch upon release, or take applications fully offline until patched.

Exploits began hours before the fix was announced. It was known in advance that a major vulnerability fix was coming for PHP, and somehow, the vulnerability leaked from the group that disclosed and fixed it. Cloudflare was in the group that worked on this problem and created tooling to monitor and block exploit attempts. From Cloudflare CEO Matthew Prince:

Amazon detected exploit attempts days before the fix. From AWS:

“Amazon Threat Intelligence identified a PHP2Shell exploitation attempt dating to Oct. 17, 2026, in MadPot telemetry, four days before public disclosure of CVE-2026-109618 and CVE-2026-109619. The exploit attempt originated from an IP address associated with a commercial anonymization network. While the vulnerability was under embargo, the targeting of a honeypot from an anonymized IP address suggests the activity was conducted by a threat actor, though it remains unclear how the actor acquired exploit information prior to disclosure.”

Attackers now have AI agents to automate and speed up attacks. Now that the vulnerability is known, attackers can simply spin up agents, instructing them to attempt to exploit servers that have not applied the patch. This probably explains why Cloudflare saw millions of exploit attempts just a few hours after the patch was released.

Railway’s customer secrets leak

Elsewhere on the same day, developer-first cloud provider Railway was hacked and the environment variables (secrets) of its customers were exposed. From the company:

“Our investigation to date indicates that an unauthorized third party may have accessed encrypted environment variables and, for some of them, obtained the decrypted contents. Environment variables may contain secrets such as API keys or database and registry credentials.”

Immediately after detecting the breach, Railway took decisive action by notifying customers and the public, and pausing all deployments.

Pausing deployments created a huge headache for customers, who could not rotate their keys without crashing their systems running on Railway. Also from Railway:

“We strongly recommend that you immediately roll all environment variables, including any API keys, stored in Railway, as well as any private registry credentials, and review logs for those services for unusual activity.

All deployments are paused while we investigate. Until builds are restored, you will not be able to roll out new versions. We apologize for this inconvenience. Follow this incident to be notified when builds resume.”

This is contradictory: rotating environment variables – say, a database access token – means that the application using the old variable stops working until a redeploy. However, Railway has paused deploys! So, customers of Railway had two choices:

• Rotate keys (the responsible step) and deliberately crash their application or take it offline (irresponsible)
• Don’t rotate and hope their key was not leaked – (irresponsible) – and let their application continue to operate (responsible)

Of course, it’s clear why Railway is pausing deployments after the environment variables leaked, and the team must ensure that new variables added by customers do not leak.

In the end, it took Railway close to 20 hours to resolve the problem and allow deployments again. During that time, they unfortunately gave security-conscious customers an impossible choice of bricking their service while rotating keys, or ignoring security best practices to keep services running.

Can we expect future ‘deluges’ of security incidents?

AI labs did warn of a looming “security tsunami” fueled by the technology in May 2026, as per CNBC:

“Dario Amodei warned Tuesday [5 May 2026] that artificial intelligence has created a narrow window for the world’s tech firms, governments and banks to fix tens of thousands of software vulnerabilities found by his company’s latest model.

That AI model, Mythos, was previewed last month along with the disclosure that it had unearthed decades-old vulnerabilities in crucial software.

Since AI models from geopolitical adversary China are “maybe six to 12 months” behind the Anthropic product, there is “roughly that amount of time” to fix these issues, Amodei said.

“The danger is just some enormous increase in the amount of vulnerabilities, in the amount of breaches, in the financial damage that’s done from ransomware on schools, hospitals, not to mention banks,” Amodei said.

Anthropic has limited Mythos to a few partner companies because of concerns about what criminals or adversarial nations could do with it.”

Around that time, I talked with “AI-pilled” devs – some with access to the Mythos model – who told me the same: they expect the security landscape to become very bad in the coming months, with record numbers of systems being breached. One engineer said he’s moving many of his secrets like 2FA backup phrases off cloud storage services and onto paper because he expects cloud storage services to also be breached.

This week, it has seemed like the prediction in May has come to fruition, much of it done with AI’s help. A few days ago, on 17 October, OpenCode was breached and its customer secrets – AI API keys – stolen. Cofounder, Dax Raad, shared that the company has been under constant attack, in his reply to Railway:

Attacks much easier & defense much harder with AI

I sense the balance of power between the good and bad actors in this space is shifting because of AI. Let’s see what it took for an attacker to get access to a system before AI existed, via a critical vulnerability exposure (CVE) – disclosed or not:

With AI, finding critical vulnerabilities is easier for everybody, including bad actors who can automate attacks as well, and do so easily. The change:

It gets worse: before AI, it took quite some skill to be an effective attacker: you needed to be a good-enough coder, understand concepts like buffer overflows, and also learn techniques for executing remote code execution attacks. With AI, none of that applies. Venture capitalist, Deedy Das, described it as scary that nontechnical people now have the tools to easily exploit known CVEs like the PHP one:

So, let me update the diagram to account for the drastic lowering of the bar to becoming an effective attacker:

And we see all of this playing out in practice, starting with publicly disclosed CVEs skyrocketing:

We may need to urgently rethink software security for all systems:

• Automated patching: manual monitoring for patches on a daily/weekly basis will become as irresponsible as applying patches months or years after release. Of course, automatic patching has its own risks, such as if a vendor providing a patch is breached, or a patch causes an outage, etc.
• Expect attackers to use day-zero CVEs. Attackers will increasingly have access to undisclosed vulnerabilities and will access systems externally, or even execute code remotely.
• Ongoing automated security monitoring to be non-negotiable. To detect security breaches, you need a decent level of automated logging and monitoring. This needs to be in place sooner rather than later. The TiDB incident this week shows what happens when a company allows an attacker to freely access their system for a month.
• Defenders will have to use more AI. Attackers are gaining advantage by using AI agents, so defenders will have to do the same. There are no best practices in this field, but there are likely to be lessons in how companies like OpenCode, Railway, and PingCAP are responding – if they share their approaches.
• Securing services and software will become more expensive. Continuously running AI agents to monitor for and mitigate breaches costs money, as do security engineers who are “AI-native.” This all increases the cost of security engineering. We previously covered security engineering basics and did a podcast on code security.

... and the security incidents continue

Since publishing the original article on 22 October, there have been more:

• The Brotli remote execution CVE: Brotli is a compression library used at Node.js, Bun, Envoy, and others. The flaw allows an attacker to take over a service. Details
• Another PHP remote code execution CVE. Just two days after the high-impact PHP2Shell patches, PHP warned of another important CVE, for which a fix is coming. PHP versions will need to be updated fast again.
• Linux CVEs posted publicly. Several unpatched Linux kernel vulnerabilities were accidentally posted on the public kernel mailing list. Details.
• Python Pillow CVE: a vulnerability inside the Python Pillow image library lets an attacker issue OS commands. Details
• … and others I’ve doubtless missed.

At this rate, we might start getting “vulnerability fatigue” and become numb to critical security vulnerabilities unless they affect frameworks our applications use. To avoid that, it’ll be necessary for the industry to improve at coming up with and implementing security practices which work in a security environment where AI speeds up the discovery, publication, and exploitation of vulnerabilities.

Read the full issue of The Pulse this is from, or check out this week’s The Pulse. This week’s issue covers:

1. Anthropic kills its coding interview. A team at the leading frontier AI company gets zero signal from DSA interviews and so has removed it from the engineering recruitment process. The rest of the company, and other AI-native startups, could follow.
2. Details on incident.io’s AI spend and ROI. The resiliency startup has slashed the amount it spends on AI; down from $4,500 per dev, per month, in August, to $1,600 now.
3. Amazon formally incorporates AI usage into SDE and SDM expectations. AI has been placed at the heart of software engineering after the company found a correlation between AI usage and productivity. Also, software development engineers (SDEs) and managers (SDMs) must be close to customers.
4. Microsoft issues new exit offers to engineers. The tech giant has offered 60,000 employees the opportunity to quit by the end of the year. But the exit package is not particularly tempting, and so the number of people who take it will surely be in the thousands, not tens of thousands.

Subscribe to my weekly newsletter to get articles like this in your inbox. It's a pretty good read - and the #1 software engineering newsletter on Substack.