Updated 12/19/2011
Lots of compute and disk, at 125 watts idle!
Hardware:
- Supermicro 7046A-3 (this is a very quiet chassis! Purchased at Newegg for $924)
- Intel Xeon E5620 Westmere-EP 2.40GHz (Purchased at Newegg for $389)
- 12GB DDR3 ECC RAM (3 @4GB), Unbuffered (Kingston, $199 from Newegg)
- 6 @ 1TB 7200rpm SATA drives (5 Samsung Spinpoint F3, 1 Hitachi, $59 each from Microcenter on sale, purchased prior to this build)
- 2 @ 320GB Seagate 7200rpm SATA drives (purchased a few years ago, sitting in a pile in my garage)
- 1 USB key (boot)
- DroboElite for bulk iSCSI storage
Operating System: VMware ESXi 5.0
VMs:
- FreeBSD 8.2 (primary NAS device, see below)
- Windows 2008 R2 (used for VSphere Client, misc)
- Linux (VMware VMCA) (off for now)
- Linux LAMP server (running this site)
- others…
Physical to Virtual mapping
When I added the physical drives in the order that I wanted them (i.e. 4 data pool drives imported from a FreeBSD system all physically contiguous), the LSI controller didn’t map them in the same order. So I get a different numbering than the left-to-right ordering of the drives in the drive bay.
6 drives PCI passthrough via the embedded LSI SAS controller, and an additional 2 320GB used over SATA bus by ESXi for datastores ds1 and ds2.
The system uses 160W (measured at the outlet) when all 8 drives are spinning (but idle). I currently use a spindown daemon to spin down the 6 passthrough drives when they aren’t busy. This means the 4 in the data pool (which is our home NAS primarily) are spun down most of the time. The 2 in the Time Machine backup pool spin down at night, but are up much of the evening due to 5 Macs in our house using it for backups. It uses 122-127W when all but 2 drives have spun down.
Nice rigg. May I ask the budget for this system when you built it?
Br
RIckard
Sure!
I bought it all through Newegg.com. I’ll update the list above with the price I paid in a few minutes.
It came out to under $1600 plus shipping, tax, for the Server, CPU and RAM. Drives were from prior purchases. A few hundred more than “BabyDragon” but more flexibility for growing it down the road.
You seem to have finished my TODO list. I recently purchased a system[1] similar to yours, but have been agonizing over how best to handle storage (and improve the poor disk performance under ESX), short of investing in an additional system.
Could I ask you to contact me privately? I’d love to discuss your setup.
———-
1. FreeBSD friendly and ESXi Certified
http://www.ixsystems.com/ix/servers/home-and-office/athena-hpc-workstation
Drop me an email, bill@
Too Funny:
I have word with some Bill’s over the years. That’s a good thing. So what is the rest of the home IT. Cisco VoiP? DR into a colo nieghbors basement via wireless? Are you using SRM? Brocade?
Hahaha, no, nothing fancy at home, any more! I am selling my old Fibre Channel SAN array that I’ve played with over the years.
I do plan on doing offsite backup to a friend who lives over 100 miles away. I’m going to set it up where we have 1-2TB at each other’s home, with some automation to encrypt and backup each others photos and other important data.
How do you spin down your drives? BTW this still sounds like a lot of power being consumed (I haven’t checked out your CPU specifically), but currently I have a Raptor and 6x 2TB Samsung F4 drives in a i5-2400 system with 16GB of RAM, and I idle around 100W.
I know my drives aren’t spun down so I’m looking at how to implement this.
Your CPU, on paper, uses more power than mine, but it’s a newer architecture and may be a bit more efficient. You also may be using more aggressive power savings setup than I am doing (Intel C-states). Are you running FreeBSD native, or under ESXi?
ESXi doesn’t have any way of spinning down disks. But the disks I pass through to ESXi via PCIe passthrough are spun down using this spindown daemon.
What kind of chassis are you running? There are a lot of variables beyond the CPU and drives. I have more drives than you as well. You may have a very efficient power supply too (although mine is pretty efficient). And I’ve got a couple of internal fans as well as this is a server chassis.
100W on your rig is very good!