Home ESXi Server

Updated 8/23/2105

This page has been out of date for some time. Here’s what my current home VMware Lab looks like now:

  • Apple Mac mini Server 6,2 (Late 2012)
    • ESXi 5.5
    • 16 GB RAM
    • 256 GB Samsung 830 SSD
    • 1 TB Apple SATA
    • 2 @ 1 Gbit Ethernet (1 onboard, 1 Apple Thunderbolt GigE adapter)
  • HP ML110 (currently OFFLINE)
    • ESXi 5.5
    • Intel Xeon E31220 @ 3.10GHz quad-core (no hyperthreading)
    • 32 GB ECC RAM
    • No storage
  • Apple Mac Pro 4,1 (2009)
    • ESXi 5.5
    • Updated to 5,1 (2010) firmware
    • Updated with Intel Xeon W3670 @ 3.20GHz 6-core
    • 48 GB ECC RAM
    • Fusion-io ioScale 1650 (1.65 TB) PCIe Flash
  • Supermicro 7046A-3
    • Motherboard replaced with Supermicro X8DTH-6F
    • 2 @ Intel Xeon E5620 @ 2.40GHz quad-core (8 physical cores total)
    • 48 GB ECC RAM (40 recognized, need to track this down)
    • 6 @ 2 TB 7200 RPM Seagate ST2000DX001 SHDD (Hybrid)
    • Fusion-io ioScale 1650 (1.65 TB) PCIe Flash
    • One “server” VM with PCIe passthrough SATA storage (see image)

Slide1

  • Synology 1513+
    • Upgraded RAM
    • 3 @ 3 TB Seagate ST3000NC002 “Value Enterprise” 7200rpm HDD
    • 2 @ 250 GB Seagate ST240FP0021 SSD (used as read/write cache)
    • 1 @ 4TB 5400rpm Hitachi USB external drive (shared by Synology as TimeMachine target)
    • NFS and iSCSI datastores
  • ZyXEL GS1510-16 16-port web-managed switch

Updated 12/19/2011

Lots of compute and disk, at 125 watts idle!

Hardware:

  • Supermicro 7046A-3 (this is a very quiet chassis! Purchased at Newegg for $924)
  • Intel Xeon E5620 Westmere-EP 2.40GHz (Purchased at Newegg for $389)
  • 12GB DDR3 ECC RAM (3 @4GB), Unbuffered (Kingston, $199 from Newegg)
  • 6 @ 1TB 7200rpm SATA drives (5 Samsung Spinpoint F3, 1 Hitachi, $59 each from Microcenter on sale, purchased prior to this build)
  • 2 @ 320GB Seagate 7200rpm SATA drives (purchased a few years ago, sitting in a pile in my garage)
  • 1 USB key (boot)
  • DroboElite for bulk iSCSI storage

Operating System: VMware ESXi 5.0

VMs:

  • FreeBSD 8.2 (primary NAS device, see below)
  • Windows 2008 R2 (used for VSphere Client, misc)
  • Linux (VMware VMCA) (off for now)
  • Linux LAMP server (running this site)
  • others…

Physical to Virtual mapping

When I added the physical drives in the order that I wanted them (i.e. 4 data pool drives imported from a FreeBSD system all physically contiguous), the LSI controller didn’t map them in the same order. So I get a different numbering than the left-to-right ordering of the drives in the drive bay.

6 drives PCI passthrough via the embedded LSI SAS controller, and an additional 2 320GB used over SATA bus by ESXi for datastores ds1 and ds2.

The system uses 160W (measured at the outlet) when all 8 drives are spinning (but idle). I currently use a spindown daemon to spin down the 6 passthrough drives when they aren’t busy. This means the 4 in the data pool (which is our home NAS primarily) are spun down most of the time. The 2 in the Time Machine backup pool spin down at night, but are up much of the evening due to 5 Macs in our house using it for backups. It uses 122-127W when all but 2 drives have spun down.

9 thoughts on “Home ESXi Server

  1. Rickard

    Nice rigg. May I ask the budget for this system when you built it?

    Br
    RIckard

  2. Bill Plein Post author

    Sure!
    I bought it all through Newegg.com. I’ll update the list above with the price I paid in a few minutes.

  3. Bill Plein Post author

    It came out to under $1600 plus shipping, tax, for the Server, CPU and RAM. Drives were from prior purchases. A few hundred more than “BabyDragon” but more flexibility for growing it down the road.

  4. David

    You seem to have finished my TODO list. I recently purchased a system[1] similar to yours, but have been agonizing over how best to handle storage (and improve the poor disk performance under ESX), short of investing in an additional system.

    Could I ask you to contact me privately? I’d love to discuss your setup.

    ———-
    1. FreeBSD friendly and ESXi Certified
    http://www.ixsystems.com/ix/servers/home-and-office/athena-hpc-workstation

  5. don watkins

    Too Funny:

    I have word with some Bill’s over the years. That’s a good thing. So what is the rest of the home IT. Cisco VoiP? DR into a colo nieghbors basement via wireless? Are you using SRM? Brocade?

  6. Bill Plein Post author

    Hahaha, no, nothing fancy at home, any more! I am selling my old Fibre Channel SAN array that I’ve played with over the years.

    I do plan on doing offsite backup to a friend who lives over 100 miles away. I’m going to set it up where we have 1-2TB at each other’s home, with some automation to encrypt and backup each others photos and other important data.

  7. Dennis

    How do you spin down your drives? BTW this still sounds like a lot of power being consumed (I haven’t checked out your CPU specifically), but currently I have a Raptor and 6x 2TB Samsung F4 drives in a i5-2400 system with 16GB of RAM, and I idle around 100W.

    I know my drives aren’t spun down so I’m looking at how to implement this.

  8. Bill Plein Post author

    Your CPU, on paper, uses more power than mine, but it’s a newer architecture and may be a bit more efficient. You also may be using more aggressive power savings setup than I am doing (Intel C-states). Are you running FreeBSD native, or under ESXi?

    ESXi doesn’t have any way of spinning down disks. But the disks I pass through to ESXi via PCIe passthrough are spun down using this spindown daemon.

    What kind of chassis are you running? There are a lot of variables beyond the CPU and drives. I have more drives than you as well. You may have a very efficient power supply too (although mine is pretty efficient). And I’ve got a couple of internal fans as well as this is a server chassis.

    100W on your rig is very good!

Leave a Reply