Incident Report - 15 November 2023

¶ Timeline

15 November 2023

Incident

My librem key accepted my laptop, but I was unable to decrypt my main drive.

16 November 2023

Incident

My librem key rejected my laptop. I was still unable to decrypt my main drive. During remediation (below), sensitive data including my secret key was exposed to untrusted hardware.

Remediation

I reflashed PureBoot through software. The ISO used to download the ROM, and the public key used to verify it, were previously saved to a separate medium.

17 November 2023

Remediation

I reinstalled QubesOS.

27 November 2023

Remediation

I flashed PureBoot externally and reinstalled QubesOS.

¶ Analysis

On the 15th, it is entirely plausible that my failure to decrypt my main drive was due to human error. It was late in the evening and I was tired. However, this is not plausible on the 16th. I decrypt this laptop practically every day.

There are many reasons why my librem key might have rejected my laptop. For example, toggling a setting off and back on would cause this behavior without any other user-visible changes. Additionally, if the HOTP counters became misaligned for any reason, the codes would not match and the laptop would be rejected.

The simplest explanation would be that a bug corrupted some of my data. However, I find this scenario unlikely as the corruption would have been oddly specific. In particular, if the file storing the HOTP count and the bytes storing the LUKS key (or key slot) were corrupted the above behavior would be explained. However, the other files in the boot filesystem contained reasonable contents and the main drive was still recognized as a LUKS device.

If there was an actual attack against my laptop, it would have been either remote or local.

I have received a number of phishing emails recently which got through my spam filters. They were fairly crude (identity/subject combination didn't make sense, character substitution in subject line, etc) but it is not impossible that a more sophisticated one slipped through and fooled me. If this happened, an attacker would have needed a way to escape to dom0. A javascript-to-dom0 exploit chain would suffice, but it seems likely that only a well-funded adversary would have access to such a chain. There was a recent QSB which disclosed a vulnerability that would allow a program with local code execution in a PV guest to escalate to dom0. This would have required compromising a different guest than the one I viewed the hypothetical email in, which would again be expensive. While PureBoot does not allow flashing after it loads the operating system, it is not clear to me whether an attacker with local dom0 privileges could arrange for PureBoot to reflash itself the next time it rebooted. If so, then it would be possible to remotely install a malicious version of PureBoot (although this should cause a noticeable delay on the next boot). If not, then only a DoS would be possible (writing files to /boot would not have helped because they are not used until after the librem key accepts the laptop, and if their contents changed the legitimate PureBoot would reject them).

On a technical level it would be trivial for a local attacker to cause a DoS leading to the observed behavior. However, this would have required the hypothetical attacker to physically enter my apartment (while I was present but asleep), putting themselves at legal and physical risk. It would also have been trivial to install a malicious version of PureBoot in this scenario. I assume that developing a malicious PureBoot (including one that survives a software reflash) would be moderately expensive but not require a well-funded adversary.

¶ Commentary

I cannot think of any reason why a well-funded adversary would want to target me. I cannot think of anyone with a grudge against me so strong that they would risk themselves just to annoy me. I have not noticed any new symptoms of an attack (no unauthorized withdrawals from my bank account, my website has not been defaced, etc), so an impersonally motivated attack seems unlikely (or, perhaps, unsuccessful). However, as previously explained I also find the explanation of a software bug implausible. I have no confidence in any explanation for the observed behavior. Perhaps exploits were leaked from a well-funded adversary and someone(s) launched remote attacks against a large number of users (exploits are generally expensive to develop but cheap to deploy).

I reflashed PureBoot externally in order to ensure that the only attacker who could have been successful is a well-funded one. On the first boot after flashing, my laptop took an abnormally long time to display the startup screen: long enough for a different version of PureBoot to be flashed. I do not know enough about the relevant hardware and software components to judge whether or not this behavior is expected. However, if someone has successfully attacked my laptop in such a way that it can reinstall itself after an external reflash I likely do not have the resources required to defend against them. The remediation steps I took are the most reasonable steps to take considering the resources that I have access to and the resources the hypothetical attacker would require in order to defeat them.

There was a delay before the external flash because I needed new hardware components.

Frankly, I never expected to be in a position where my librem key would reject my laptop. I use it because it provides an opportunity to practice disciplined action. However, now that it has triggered I can hardly ignore it. As stated in my analysis, I have not come up with any plausible explanation as to why it triggered.

My secret key material was exposed to untrusted hardware due to poor planning; I did not expect the key to trigger, so I did not think through how I would respond. I now have a plan in place to recover from a future rejection without putting any sensitive data at risk.

I continued working normally after reflashing PureBoot through software. If there was an attacker who was sophisticated enough to install malicious firmware that survives a software reflash, then continuing my normal day-to-day work would not expose any new sensitive information to them. In the future a similar attack will expose less information.