RENMX | Forum

This is going to be an interesting build. Last year in Feb 2013 I upgraded the server. We went to at the time the latest Six Core technology Intel had to offer in the Core i7 3930K. This is still an amazing CPU but it is a consumer processor. It lacks some features we need, namely ECC memory.

So this build is no longer called "Colossus" as that is our old consumer build. This is new enterprise equipment and it's going in a new case and I felt a new name would be appropriate and I've decided to call it PROMETHEUS.

New case, a server 4U Rack with 24 Hotswap bays, new memory 32GB of DDR3 ECC RAM. New XEON CPU with the E5 Ivy Bridge-EP architecture. And lastly a new motherboard, the Asrock EP2C602-4L/D16 a dual socket XEON motherboard with RDIMM & UDIMM ECC support up to 256GB per socket (512GB RAM support total).

It's gonna be a beast. The most advanced server I've built for our community and it gets us out of the consumer parts and on to the enterprise parts.

So the main thing you're probably thinking is, why. Why am I doing this? Well the only reason is ECC memory. This stands for Error-correcting code and it basically means if the memory has a problem (which can occur several times a year in a 24.7 system) the memory can detect and fix the issue.

In normal systems like the one we are currently using over time the memory can have a bit flip. Each memory stick is full of millions of these bits they hold data either a 1 or a 0. Sometimes electronic fields produced by other electrical devices near the RAM can cause one of these bits to flip from a 1 to a 0 or a 0 to a 1. When this happens it can result in the server crashing or file loss / corruption.

ECC is an enterprise feature which Intel only supplies on XEON CPU's so that is what we are upgrading to. This also means we have to upgrade our motherboard as that doesn't support ECC memory and finally we need new memory. We currently use 24GB (6 x 4GB) of DDR3 Mushkin memory. This will be swapped out for 32GB (4 x 8GB) of Kingston ECC UDIMM DDR3 Memory.

Finally we are going with a Dual Socket board because it offers us room to grow. We can stick two 12 Core XEON's in it if we wanted taking us to 24 cores / 48 threads. And the motherboard can run with only one CPU inserted. We will start off with a single XEON 4 Core CPU which is as fast as the 6 Core CPU we are currently using on single threaded workloads like Minecraft.

Then over time as the processors drop in price due to Intel releasing Haswell-EP XEON's later this year we can upgrade to single or dual six core XEON's.

Now some photos!

Server Case:


Motherboard:


You can see above those two huge CPU Sockets and the 16 (!!) RAM slots which enable up to 512GB of memory to be installed. When we use it only 4 of those 16 slots will be populated. I expect us to eventually reach 128GB of RAM in this system when both CPU slots are populated.

Thanks for reading so far, the thread will evolve over the rest of the year, this is a very expensive build and things won't all be bought at once as I'm going to be scrounging around for the best deals for each component until all the parts are together.

I woke up reading this looks amazing and now I have a semi in my pants.

The motherboard has been ordered and will be here on Monday. I'll take pictures

The case is currently out of stock but should be available soon. I'll be ordering that.

The great thing about this build is this motherboard is fully compatible with our current Core i7 3930K and 24GB of Non-ECC memory. So all I need is the Motherboard, Case and some SAS Cables and I can move us over to this board. Then later on I can add in a XEON and the ECC Memory (those two don't need to be bought together either as the XEON will work with Non-ECC memory too).

This means I can wait and drag this build out over the rest of the year picking up the best deals. Already I saw some XEON's sell on ebay for £100 which cost £400+ brand new and with the new XEON's coming out next month on a new socket it should flood ebay with older XEON's the kind we will be using in this build.

Exciting

Motherboard arrived today. I just snapped one quick picture with my iPhone as I'm not feeling so great.

Ordered some more parts today. The CPU Heatsinks, PWM fans for the CPU Heatsinks (the defaults are way too loud). Dolly Cart for the server chassis to sit on and a USB extension cable for the back as the case has USB 3.0 on the front but the motherboard only has USB 2.0. Not many server boards offer USB 3.0 so I'm not really sure why X-CASE used USB 3.0 on the front.

I should have the case ordered later today but I have to phone the company first and it's not business hours yet. They finally got some stock.

Oh and yesterday I ordered a new BIOS chip for the motherboard as it came with BIOS P1.10 (theres a sticker on the board indicating the BIOS version) and I need P1.40 or higher to use a 3930K or an Ivy Bridge-EP CPU in the motherboard. The new chip should be here in a few days from Germany and it contains BIOS version P1.70

Once I've got all the bits here I can start putting it together. I'm leaning towards doubling up on the RAM from 32GB to 64GB whilst keeping our current 3930K for a bit longer. The RAM would still be ECC so I can drop a XEON (or two) in to the server at a later time to get the ECC functionality working. At the moment DDR3 prices are climbing steadily as fabrication plants switch to DDR4 causing a supply and demand issue that pushes prices up so it may be best to get all 64GB of RAM now and get the much cheaper XEON parts later.

Also Intel is about to release their new XEON's which use a new socket and I'm expecting ebay to get flooded with E5 Ivy Bridge-EP XEON's the kind we will be using. The newer chips Intel are releasing use super expensive DDR4 memory and don't offer much of a performance boost, not to mention the motherboard costs are astronomical.

Thought I'd post a picture of our CPU Heatsink. It is made by SuperMicro and is designed specifically for LGA2011 in a 4U chassis. So it is as big as it can be whilst fitting in to a 4U Server case. It comes equipped with a 92mm fan which I'll be swapping out for a quieter more efficient model.

New case has been ordered. Sadly they've run out of stock (again) it's a very popular case but I've pre-ordered it and they should have stock very soon as they are expecting a pallet of them in a few days time.

I've also ordered the SAS 8087 cables I need. I'm waiting to hear back about RDIMM compatibility with 3930K but if they are compatible that's what I'll be buying to the tune of 64GB.

RAM has been ordered. I went with 64GB (8 x 8GB) of Kingston 1600MHz ECC DDR3 UDIMM's they are 1.35v - This is a bit faster than the RAM we currently use which is 1333MHz 1.5v. And of course a lot more, we only have 24GB currently so this is a huge upgrade.

It is two sets of this RAM: http://www.scan.co.uk/products/32gb-(4x ... -certified

The SuperMicro CPU Heatsink+Fan came this morning. I'm extremely impressed with this cooler. It is very well made. The fan they are using is probably the highest quality computer fan I've ever seen.

I looked up the price online and to buy this CPU fan on its own currently costs around $60 dollars. SuperMicro must have got an insane deal. I plugged it in to my desktop to test its noise and air flow, very impressed by its acoustics and performance.

It's a 92mm fan and it pushed more air than any other computer fan I've held even the 140mm Corsairs I have don't push as much air as this thing, it's like a jet. At 3,700 RPM it is very noisy but at 2400 RPM it's practically inaudible and still moves an insane amount of air.

I was intending to switch the fan with a low noise one, I bought two already which also came this morning but now I'm not going to bother, this fan is simply too high quality not to use.

Smallest part of the build arrived today. The motherboard BIOS chip.

It's smaller than a centimeter!