Multyx is Part of a New Press Release by Microsoft w/ NVIDIA

Announcing Azure confidential VMs with NVIDIA H100 Tensor Core GPUs in Preview

November 15, 2023. Today, we are excited to announce the preview of Azure confidential VMs with NVIDIA H100 Tensor core GPUs.  These VMs are ideal for training, fine-tuning and serving popular open-source models, such as Stable Diffusion and its larger variants (SDXL, SSD…) and language models (Zephyr, Falcon, GPT2, MPT, Llama2. Wizard, Xwin).

Azure is constantly innovating in security to provide the best protection for its customers’ data and applications. One of the innovations that Azure has been pioneering is the enablement of confidential computing. Confidential computing is the protection of data in use by performing computation in hardware-based, attested Trusted Execution Environments (TEEs). These TEEs prevent unauthorized access or modification of application code and data during use.

While Azure has been leading the enablement of confidential computing on CPUs, there is also a need to enable confidential computing on GPUs. GPUs are widely used for high-performance computing, machine learning, and graphics rendering, which can involve processing large amounts of sensitive data. By enabling confidential computing on GPUs, Azure offers customers more options and flexibility to run their workloads securely and efficiently on the cloud.

“As AI scales across every industry, protecting company and customer data is paramount — and Microsoft Azure and NVIDIA have collaborated to engineer a solution to this challenge,” said Ian Buck, Vice President of Hyperscale and HPC at NVIDIA. “The Azure confidential VMs with NVIDIA H100 GPUs bring a complete, secure computing stack from the VMs to the GPU architecture itself, enabling users to build and deploy AI applications with confidential computing on Microsoft Azure while knowing that their data and AI models remain protected end to end.”

Azure confidential VMs with NVIDIA H100 Tensor Core GPUs

   These Virtual Machines have the following features:

• Next-generation CPUs: AMD 4^th Gen EPYC processors with SEV-SNP technology to meet CPU performance for AI training/inference.
• AI state-of-the-art GPUs: NVIDIA H100 Tensor Core GPUs with 94GB of High Bandwidth Memory 3 (HBM3).
• Trusted Execution Environment (TEE) that spans confidential VM on the CPU and attached GPU, enabling secure offload of data, models and computation to the GPU.  
• VM memory encryption using hardware-generated encryption keys.  
• Encrypted communication over PCIe between confidential VM and GPU.
• Attestation: Ability for CPU and GPU to generate remotely verifiable attestation reports capturing CPU and GPU security critical hardware and firmware configuration.

Confidential computing support

NCC H100 v5 VM SKUs support hardware-based TEEs that protect VMs against privileged host components and attackers. With this SKU, GPUs are assigned to the VM in confidential mode. In this mode, the GPU High Bandwidth Memory 3 (HBM3) and security critical configuration registers are isolated and protected against unauthorized access. When the GPU device driver is loaded in the confidential VM, it establishes a secure channel with GPU and uses this channel for all subsequent data transfers between CPU and GPU. Additionally, customers can request attestation to verify that the VMs and GPUs are running a correctly configured TEE before launching sensitive applications and releasing secrets such as data encryption keys. Almost all applications, including those that use NVIDIA CUDA for acceleration, can be transparently executed in these VMs.

Use cases across industries

Use cases for privacy preserving GPU based analytics and ML span multiple verticals:

• Healthcare: For analysis of medical images, such as X-rays, CT scans, or MRIs. This can help protect the privacy of the patients and demonstrate compliance.
• Finance: For analysis of financial data, such as transactions, portfolios, or risk models. This can protect the confidentiality of sensitive financial data and models.
• Government: For analysis of data collected by various government agencies, such as census, tax, and national security. This can help protect the privacy of citizens and bolster national security.

“With its sensitive-data collaboration SaaS platform, Multyx enables creation of new enterprise data and AI applications previously impossible or too lengthy, costly, or risky to implement. Azure confidential VMs with NVIDIA H100 Tensor Core GPUs hosted within Azure Confidential Computing, boosts our platform with new capabilities and efficiencies especially for Gen AI and LLM-based business applications across banking, insurance, and related enterprises”

– Ophir Holder, Ph.D., CEO and Co-Founder of Multyx.