Minio S3 small setup 40 Gb/sec read using PCI 5.0.

Small S3 minio setup on prem gives nearly 40Gigabyte/Sec sustained read. Roughly 20Gigabyte/Sec left on the databus . Earlier X14 servers based on PCI 4.0 gave 22.8 GB/Sec.

Tested with Wasabi's s3 performance bench in Go.

rickard@ ~]# ./s3-benchmark -a minioadmin -b data6 -s changeme  -u http://localhost:9000 -z 16M -t 32
Wasabi benchmark program v2.0
Parameters: url=http://localhost:9000, bucket=data6, region=us-east-1, duration=60, threads=32, loops=1, size=16M
Loop 1: PUT time 60.0 secs, objects = 29522, speed = 7.7GB/sec, 491.7 operations/sec. Slowdowns = 0
Loop 1: GET time 60.0 secs, objects = 151692, speed = 39.5GB/sec, 2527.1 operations/sec. Slowdowns = 0
Loop 1: DELETE time 1.9 secs, 15567.7 deletes/sec. Slowdowns = 0

Ingredients X14 PCI 5.0 server

• Minio Single mode multi disk
• 4x Kioxa NVME pci 5.0
• Supermicro H13ssw-nt PCI 5.0
• Epyc 32 core

The installation is part of a bigdata stack distribution where the particular S3-minio installation script can be found here: https://github.com/Ignalina/racksetup/tree/main/dist/rocky_9_3/installation_kits/singleminiomultidisk