At scale, the fabric is the computer.

aihpcnet is an independent technology practice — twenty years of HPC network architecture, from the Blue Gene/L torus to production platforms at Bloomberg, now pointed at AI infrastructure.

Book time at SC26 See capabilities

Track recordBlue Gene/L — Top500 #1, 2004–2008
Scale3D torus, tens of thousands of nodes
Topologytorus, fat-tree, spine-leaf
Storagemulti-petabyte GPFS
CurrentNVIDIA NCP-AIN certified

The problem

A GPU cluster is a network problem wearing a compute budget.

Accelerators are the line item everyone argues about. The fabric is what decides how much of them you get to use — and it fails in ways that never page anyone.

Collectives serialize

One slow rail holds the whole step

An all-reduce finishes when its slowest participant finishes. A single congested leaf, or one link that negotiated down a lane, stretches every iteration behind it. The symptom is a utilization number, not a network alarm.

Lossless is a tuned system

PFC and ECN have to be set together

Priority flow control, ECN marking, buffer headroom and congestion control are one system, tuned per topology and per traffic pattern. Vendor defaults either drop what RDMA cannot afford to lose, or push pause upstream until the fabric seizes.

Degradation is quiet

Fabrics get slower before they break

A marginal cable raising its error rate, a link flapping below the alerting threshold, a routing weight set during bring-up and never revisited. Without fabric telemetry joined to job records, nobody can say which run paid for it.

Capabilities

Three places where this work pays for itself.

Interconnect architecture

The topology decision and everything that follows from it, at the scale where the decision stops being reversible.

Topology selection and sizing — torus, fat-tree, spine-leaf — and what each costs in cabling, bisection bandwidth and failure domains
Bandwidth and parallel I/O architecture for MPI workloads across tens of thousands of nodes
RDMA data paths and the congestion control that keeps them lossless
Design review of a fabric already on paper: where the oversubscription actually bites, and what to change before it is cabled
InfiniBand and UFM fundamentals, current as of NVIDIA's AI Networking certification

AI datacenter Ethernet

RoCE for GPU scale-out, approached the way HPC has always approached an interconnect.

Lossless, semi-lossy and lossy designs — which one your collectives and your storage actually want
PFC and ECN configuration, and the buffer headroom reasoning behind the numbers
Spine-leaf and rail-optimized layouts, ECMP hashing, and where the oversubscription ends up
NVIDIA Spectrum-X and Quantum platform fundamentals, by way of the NCP-AIN certification
A straight answer on what is proven today and what is still vendor roadmap

Platforms and the teams behind them

A fabric only becomes throughput once storage, provisioning and the organisation around it agree with the design.

Parallel file systems at multi-petabyte scale — GPFS/Spectrum Scale in production under large parallel jobs
Private cloud infrastructure: OpenStack across hundreds of hypervisor hosts and tens of thousands of VMs
Provisioning, configuration management and validation automation in Python
SRE practice that survives contact with production: on-call, incident response, SLOs, postmortems
Hiring, developing and leading the engineers who have to run it after you leave

Fabric design

Why rail-optimized wiring changes the whole build.

The cabling decides which collectives are cheap. Everything above it — scheduler, NCCL topology, storage placement — then has to agree.

Rail-optimized scale-out unit, simplified: 4 nodes, 4 of 8 rails drawn, one GPU NIC per rail per node.

rail 0 — GPU 0 of every node
rail 1 — GPU 1 of every node
rail 2 — GPU 2 of every node
rail 3 — GPU 3 of every node

What it buysAn all-reduce that stays on one rail completes inside a single leaf switch: no spine hop, no cross-rail congestion, and a blast radius limited to one rail when a leaf goes down.

What the spine doesThe spine then carries only cross-rail traffic, storage and management. That is what makes the core affordable at scale instead of strictly non-blocking everywhere.

Where it goes wrongAny job that cannot stay rail-aligned pays a spine hop per step. The scheduler, the NCCL topology and the physical cabling have to describe the same machine — and that mismatch is a common reason a new cluster lands well under the numbers it was sold on.

Engagements

Scoped work with something you can hold at the end.

Each engagement ends in a document, a measurement, or a punch list — not a slide deck of observations.

Fabric design review

Topology, bandwidth and parallel I/O architecture for a new build or an expansion, with the failure analysis behind every choice written down rather than inherited from a default.

2–3 weeks
remote
HPC-to-AI infrastructure assessment

For teams who already run traditional HPC or a general-purpose private cloud and now have to stand up GPU clusters. What transfers, what quietly does not, and what to fix while it is still a diagram.

2–4 weeks
remote
Parallel storage and data path review

Whether the storage side can actually feed the compute: parallel file system design, bandwidth planning, and the I/O patterns your jobs will really produce rather than the ones the benchmark produces.

1–3 weeks
remote
Scale-out and capacity planning

How the infrastructure grows from one unit to many: topology, bandwidth, power and rack implications, oversubscription decisions, and an honest account of what breaks first and at roughly what size.

1–2 weeks
remote
Fractional infrastructure leadership

Retained time each month for architecture review, vendor evaluation, RFP support, hiring, and mentoring the engineers who will own the system. Twenty years of running infrastructure teams, available by the day.

ongoing
monthly retainer

Who you're actually hiring

One engineer, visible from the outset.

Matthew Newton has spent more than twenty years designing, building and operating high-performance computing and network infrastructure — fifteen years at IBM Research, and fifteen at Bloomberg, where he manages the engineering team behind a private cloud application platform used by compliance officers at global financial institutions.

From 2000 to 2005 he was an infrastructure architect on the IBM Research team that built Blue Gene/L, which held the Top500 #1 ranking from 2004 to 2008. He drove the design and build-out of its custom 3D torus interconnect across tens of thousands of nodes, and made the topology, bandwidth and parallel I/O decisions that MPI workloads at that scale stand or fall on.

He then led the infrastructure team for IBM's Research Compute Cloud, one of the earliest large-scale private HPC clouds: hundreds of hypervisor hosts, tens of thousands of virtual machines serving IBM Research worldwide, and a multi-petabyte GPFS parallel file system underneath. Topology and bandwidth planning, low-latency interconnect design, hardware lifecycle and the provisioning automation were all his team's to own.

At Bloomberg he has owned platforms end to end — OpenStack infrastructure, load balancer farms, a Python microservice backend, and the network and web security for a system serving external financial institutions — while hiring and developing the engineers who run them.

He holds NVIDIA's Certified Professional, AI Networking (NCP-AIN), and chairs the board of The Knowledge House, a technology education nonprofit in the South Bronx.

Built and operated

3D torus
Fat-tree
Spine-leaf
MPI workload infrastructure
RDMA
ECN / PFC
GPFS / Spectrum Scale
OpenStack
Load balancer farms
Python
Bash
Parallel I/O architecture

Certified — NVIDIA AI Networking (NCP-AIN)

InfiniBand
NVIDIA Quantum
Spectrum-X
RoCEv2
UFM
DOCA
NCCL

Leading the work

Hiring & team development
SRE practice
Incident response
SLOs
Capacity planning
Technical roadmap
Cross-team architecture

2020–now
Engineering Manager, Compliance ApplicationsBloomberg LP — platform-as-a-service for compliance products; OpenStack, load balancer farms, Python microservices
2019–2020
Engineering Manager, Compliance Data PlatformBloomberg LP — large-scale pipelines over high-volume financial data streams
2011–2018
Engineering Team Lead, Bloomberg for EducationBloomberg LP — architecture for customer-facing applications and third-party integrations
2006–2011
Team Lead, Research Compute Cloud (RC2)IBM TJ Watson Research Center — hundreds of hypervisor hosts, tens of thousands of VMs, multi-petabyte GPFS
2000–2005
Architect, Blue Gene/LIBM TJ Watson Research Center — 3D torus interconnect; Top500 #1, 2004–2008
1996–2000
HPC ArchitectIBM TJ Watson Research Center — clusters, high-speed networks and parallel storage for academic and government research
cert.
NVIDIA Certified Professional, AI NetworkingNCP-AIN
M.S.
Computer ScienceNYU Tandon School of Engineering
B.A.
EnglishBoston College

Case notes

Three systems, and what the hard part actually was.

Two of these are twenty years old. That is rather the point: the topology and bandwidth questions an AI cluster asks today were asked first on machines like these, and answered under harder constraints.

Blue Gene/L: a torus at tens of thousands of nodes

Context: IBM Research, 2000–2005. A machine designed to reach the top of the Top500 by scaling out much further than contemporary designs, not by making each node faster.
Decision: A custom 3D torus rather than a switched fabric: nearest-neighbour locality and predictable bisection bandwidth, bought at the cost of topological flexibility.
My part: Infrastructure architecture for the interconnect build-out, and the topology, bandwidth and parallel I/O decisions that MPI workloads at that node count depend on.
Outcome: Top500 #1 from 2004 to 2008.

RC2: a private HPC cloud before that was a category

Context: IBM Research needed self-service compute for researchers worldwide, on infrastructure good enough for HPC workloads rather than commodity virtualization.
Built: Hundreds of hypervisor hosts, tens of thousands of VMs, low-latency interconnect, and multi-petabyte GPFS shared storage behind it.
My part: Led the team that designed, built and ran it — network topology and bandwidth planning, hardware lifecycle across hundreds of hosts, and provisioning and validation automation in Python.
Outcome: Self-service HPC for research teams across IBM labs in several countries — years before private cloud was a category you could buy rather than build.

A compliance platform that cannot go down

Context: Bloomberg. The deployment target for compliance applications used by officers at major global financial institutions, plus internal-only workflows.
Scope: OpenStack VM infrastructure, load balancer farms, and a Python microservice backend integrating SSO, Solr and several database systems — with network and web security for external and internal traffic both.
My part: Engineering manager, end to end: architecture across the network, virtualization and application teams, plus hiring, reviews and the technical roadmap.
Outcome: A platform whose load is measured in consequence rather than volume: compliance workflows at major financial institutions, where an outage is a regulatory problem and not an inconvenience.

SC26

Let's find thirty minutes at the show.

The fastest way to work out whether there's something here is to describe your fabric for ten minutes and let me ask the awkward questions. Bring a topology diagram if you have one.

SC26 runs November 15–20 at McCormick Place in Chicago. Pick a slot on the calendar and we'll talk before the week gets away from us.

Get in touch

Book 30 minutes calendly.com/newton10471/sc26-30-minutes

Email mnewton@aihpcnet.com

LinkedIn linkedin.com/in/newton10471

Based in New York, NY · Eastern

Availability Open to consulting engagements and full-time roles.