Extreme Personal Supercomputing with HPCBOX on Microsoft Azure

We are pleased to announce preview support in HPCBOX for NVIDIA A-100 powered NDv4 instances on Microsoft Azure, specifically Standard_ND96asr_v4. As per Microsoft’s documentation, the ND A100 v4-series uses 8 NVIDIA A100 TensorCore GPUs, each with a 200 Gigabit Mellanox InfiniBand HDR connection and 40 GB of GPU memory. 

Introduction

This blog post will be the first in what will hopefully be a two-part series with more information to share later. But, for now, using these machines was so exciting that an initial post was well worth it.

Drizti’s HPCBOX platform delivers a fully interactive turn key HPC solution targeting end-users directly and comes with expert HPC support offered by Drizti. This gives end-users a single point of contact and a fully integrated solution which has already been optimized or can be optimized by our HPC experts in collaboration with users for custom codes which are developed in-house.
​
The NDv4 instances are cutting edge and Microsoft Azure is probably the only public cloud vendor to offer this kind of a machine configuration for applications that can effectively use multiple GPUs. The screenshot below shows the output of nvidia-smi on one of these instances.

Not one but many for extreme scale

At Drizti, we like to make sure we can offer a truly super scale computing setup with a fully interactive Personal Supercomputing experience. We want our users to be able to use supercomputers in the same way as they use their PCs or workstations and eliminate the learning curve and time wasted waiting for efficient use of supercomputing technology. Therefore, we always try to challenge HPCBOX and our HPC capabilities, and in the case of NDv4, we used not one but multiple instances to use them at a massive scale and exercise all the GPUs and InfiniBand links at max throughput. It was an amazing experience.
​
In this screenshot we see multiple NDv4 instances attached to the HPCBOX cluster and ready for use.

In the screenshot below, you can see some of the HPCBOX Monitoring charts showing GPU utilization (32 GPUs in this case).

.The output below shows a device-device bandwidth test across two nodes over InfiniBand.

# OSU MPI-CUDA Bandwidth Test v5.6.2
# Send Buffer on DEVICE (D) and Receive Buffer on DEVICE (D)
# Size      Bandwidth (MB/s)
1                       0.99
2                       2.04
4                       4.09
8                       8.23
16                     16.28
32                     31.46
64                     58.59
128                   117.69
256                   232.73
512                   432.70
1024                  885.07
2048                 1739.59
4096                 3153.56
8192                 3507.39
16384                6132.79
32768               10483.22
65536               15758.70
131072              19443.64
262144              22122.89
524288              22959.62
1048576             23661.67
2097152             23814.17
4194304             23854.35

In the screenshot below, you can see a snapshot of the IB links and the GPU utilization from a four node test run. 

HPCBOX AutoScaler to optimize budget spend

We recently announced support for low-priority spot priced instances in HPCBOX AutoScaler. Combining the NDv4 instances with the HPCBOX AutoScaler is a nice way to optimize your budget spend when running non-critical jobs where you can afford to have the job rescheduled and restarted when nodes get pre-empted. The nice thing about HPCBOX is that one can have both standard and low-priority instances on the same cluster and users can target different class of machines based on the importance of the jobs that they are submitting.

More to come

This post highlights how easy it is to access and use an extreme scale HPC cluster with the fully interactive user experience delivered by HPCBOX. Furthermore, with the personalized HPC support delivered by Drizti, HPC is way easier to use effectively than what it used to be and that is why we call this Personal Supercomputing!
​
Over time, we will be performing further tests, analysis, optimizations and try different applications on the NDv4s. Hopefully, we will be able to share some of those experiences in a future post.

Availability

Get in touch with us to use HPCBOX and accelerate your innovation with extreme scale Personal Supercomputing!
Contact Us

​Author
Dev S.
Founder and CTO, Drizti Inc

All third-party product and company names are trademarks or registered trademarks 
of their respective holders. Use of them does not imply any affiliation or endorsement by them.