New server inbound – Home ESXi environment

After reading Phil Jaenke’s (@rootwyrm) blog about building up an ESXi server, lovingly named “BabyDragon”, I decided that I really needed my own ESXi box, based on proper server hardware, that could handle the advanced features of ESXi, especially PCIe passthrough (aka VMDirectPath).

My need for VMDirectPath comes from the desire to take some subset of the disks, and pass them through natively to a single VM, in which I’ll run ZFS. Running ZFS on top of virtual disks presented by VMware cuts a lot of the performance out from under the zpools. Maybe I could get part of it back via doing fancy alignment tricks, but I didn’t want to deal with that.

My goals for my ESXi box were slightly different than Phil’s:

• It does NOT need to be small, in fact, I’d like the chassis to be able to hold various cards I might play with over time.
• It does need to be somewhat power efficient, both the CPU and the power supply
• Quieter is better. No 1U screamers
• It needed to have room for 6-8 hot swap 3.5″ SAS/SATA drives.

My current rig is running an unlocked AMD Phenom II X2 dual core (unlocked to 4 cores) but the desktop grade motherboard doesn’t support PCIepassthrough (IOMMU in AMD parlance).

What I came up with is this:

Newegg Wishlist:

• SUPERMICRO SYS-7046A-3 4U Rackmountable / Tower Barebone Server
• Intel Xeon E5607 Westmere-EP 2.26GHz LGA 1366 80W Quad-Core Server Processor
• Intel Xeon E5620 Westmere-EP 2.40GHz LGA 1366 80W Quad-Core Server Processor
• Kingston 12GB (3 x 4GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) ECC Unbuffered Server Memory

The barebones server has a SAS controller onboard (8 SAS), plus 6 SATA ports. It lacks IPMI, but has a slot for an add-in IPMI+KVM card to enable it if I really want it.

The chassis is huge, and this one is set up as a “whisper quiet workstation”, claiming 28db noise levels, and has an 80+ certified 865W power supply that runs quietly too.

I chose a relatively low clock-rate Westmere processor, the 2.262.4GHz quad core, 80W version. I didn’t want to go with a higher wattage proc, nor did I want to shell out for a 2.66 or higher. Phil’s Baby Dragon has better clock speeds, but I think this Westmere will keep up.

I’ll report back after I get ESXi installed. My hope is to essentially move over my data (4 disk ZFS zpool, two mirror sets) natively and use PCIe passthrough to present it to a VM. The VM will boot from regular virtual disks, which I will mirror in the VM onto two different backing datastores. So I’ll have 8 disks, at least 4 of which will be passthrough to my FreeBSD storage server environment.