How are the resources assigned to virtual machines managed in a virtual environment based on Nutanix AHV? How much does the resource configuration matter in a NUMA topology?
In a NUMA topology, a VM’s memory access times depend on where the memory is located relative to the processor.
A VM accesses a processor’s local memory faster than non-local memory. You can achieve optimal resource utilization by using both CPU and memory from the same physical NUMA node. If the CPU runs on one NUMA node (for example, node 0) and the VM accesses memory from another node (node 1), you might experience memory latency.
So it’s a good idea to make sure that the virtual hardware topology of the VMs matches the physical hardware topology, to get minimal memory latency.
AHV hosts support virtual non-uniform memory access (vNUMA) on virtual machines. You can enable vNUMA on VMs when creating or modifying them, to optimize memory performance.
vNUMA optimizes the memory performance of virtual machines that require more vCPUs or memory than the capacity of a single physical NUMA node. In a vNUMA topology, you can create multiple vNUMA nodes, each of which includes vCPUs and virtual RAM.
When you assign a vNUMA node to a physical NUMA node, the vCPUs can intelligently determine the memory latency (high or low). Low memory latency within a vNUMA node results in low latency within a physical NUMA node.
Let me try a practical example:
Let’s assume we have an NX-1065-G9 node with the following specs:
– Dual Intel Xeon® Silver 4509Y processors, 8 cores / 2.60 GHz
– 384 GB of RAM evenly distributed between the two memory controllers.
We need to deliver a virtual machine with the following characteristics:
– At least 12 cores
– 256 GB of RAM
Does this configuration require vNUMA?
What would the vNUMA sizing be in this case?
Solution
In this scenario, we can see that the resources required by the virtual machine are greater than those available from a single physical NUMA node (that is, 8 CPU cores x 192 GB of RAM). In this scenario it’s definitely advisable to enable vNUMA to expose the physical NUMA architecture to the VM’s guest operating system for optimal performance.
Of course, vNUMA should be sized based on the architecture of the underlying physical NUMA node.
How do you enable vNUMA on a virtual machine in AHV?
Step 1:
Use SSH to log on to a Controller VM. Run the following command to check the number of NUMA nodes available on each AHV host in the cluster.
nutanix@cvm$ hostssh "numactl --hardware"
Step 2:
You can enable vNUMA when you create a VM or modify an existing one. Below are the two cases with their commands:
Enable vNUMA if you’re creating a VM:nutanix@cvm$ acli vm.create num_vcpus=x \
num_cores_per_vcpu=x memory=xG \
num_vnuma_nodes=x
Enable vNUMA if you’re modifying an existing VM:nutanix@cvm$ acli vm.update \
num_vnuma_nodes=
PS: the information here is taken, in part, from a Nutanix course available on the https://university.nutanix.com/ platform