Confidential Computing is cool!
Background
Google Cloud provides encryption and/or integrity of data in transit and at rest by default. We also employ other techniques like sandboxing to help protect our infrastructure and customer data. This virtualization security model allows us to offer a secure multi-tenant architecture where VMs can run isolated and protected from each other.
In this security model, the hypervisor is part of the Trusted Computing Base since it creates, manages and isolates these VMs. Hence, it is critical to the security of the system. If the hypervisor itself is compromised all the VMs can become vulnerable.
We put a lot of effort to harden and keep our infrastructure secure, including KVM, the hypervisor Google uses. We proactively research and test for vulnerabilities, contributing our changes back to the community, and have processes to provide code integrity and provenance assurance.
Still, some markets can benefit from protecting their VMs from bugs in the hypervisor or the cloud provider, especially those on regulated industries or managing highly sensitive data. Confidential Computing comes to solve this.
Security to the next level
The Confidential Computing Consortium, that Google is member of, aims to address the security of data in use. Even when data is encrypted in transit or at rest it must be decrypted for processing in memory. If another process were able to read that memory it would compromise the security of the system. Of course one such process is the hypervisor. In theory, other processes running at lower privilege level could also get access to that memory if they exploit a bug in the hypervisor.
Non-volatile memory technology aggravates this situation. These memory chips can be physically removed and their data read like a hard drive. Other types of attacks like DRAM interface snooping are also possible. Arguably, these are quite complicated in a cloud environment.
To address this type of security risks and give more confidence and control to users, Google now offers Confidential VMs. These are GCE VMs running on hosts using the AMD Secure Encrypted Virtualization (SEV) technology. This technology provides memory encryption of VMs, using a key that is known only to the processor and not the hypervisor. Not even Google has access to it.
This way, when a new VM is created a random key is also created, and its memory is automatically encrypted so that its contents can only be read by the VM itself. Other processes including the hypervisor could only access the data in its encrypted form. Security is rooted in the hardware itself.
There are other solutions for confidential computing used by other cloud providers based on Intel SGX technology. This technology allows to partition regions of memory into enclaves, which are areas of execution with hardware-assisted encryption and integrity protections to prevent access from other processes outside the enclave itself, including those running at higher privilege levels.
Intel includes SGX instruction set extensions on some modern CPUs that allow code developers to create and manage these enclaves. However, this means that users should modify their applications to take advantage of this technology, and use it appropriately. This is a serious drawback, although Intel and Microsoft have developed SDKs and frameworks to make this work easier for developers.
I think AMD SEV technology takes a better approach here since it allows confidential computing without requiring any application software changes. The changes to support it should be made on the lower levels of the stack, in the guest operating system and hypervisor. Google takes care of that for you, so your application can be migrated as is. Users only need to enable it when creating a new VM.
How the magic happens
AMD SEV builds upon AMD-V virtualization features. AMD-V supports running multiple VMs under the control of a hypervisor. When the hypervisor runs guest VM code, it executes a VMRUN instruction with the virtual machine control block as argument which describes the VM. SEV encryption capabilities can then be enabled by setting a control bit.
Each SEV-enabled VM is associated with a unique memory encryption key. Key management is not handled by the CPU but a separate Secure Processor and firmware, so encryption keys are not visible to the main CPU or the hypervisor. The firmware provides an interface to help the hypervisor perform common tasks such as launching, running, snapshotting, and migrating a VM.
The generated keys are loaded to memory controllers. Each controller includes an AES-128 engine that encrypts data when it is written to DRAM and decrypts it when read. Once SEV encryption is enabled, the guest OS can decide which pages of memory will be private (encrypted) or shared by setting the C-bit (enCrypted bit) in the corresponding guest page table entry (PTE). Typically, the guest will map all memory pages as private except e.g. those to be shared with the hypervisor for I/O.
Not only data is encrypted when SEV is enabled. Guest instruction pages and page tables themselves are always treated as private, so other processes such as the hypervisor can’t inject or modify guest code successfully. While in theory the hypervisor can modify these pages, doing so would cause junk data to be executed (and very likely a fault).
AMD SEV offers more features than described here. I have described the basics but there are more technologies being developed to protect virtual environments against more complex threat models including malicious hypervisors.
Conclusion
Confidential Computing enables new cloud scenarios where customers can run highly sensitive workloads minimizing their risk to hypervisor bugs or insider access, and the need for trust in the cloud provider. Along with encryption of data in transit and at rest, encrypting data in use is the third pillar to the end-to-end encryption story. This is really a game changing technology that raises the security bar, and now Google offers it to you with a click of a button. Isn’t it cool?
PS: Thanks to Jose Carlos Cerezo for the inspiration to write about this.
