PROMETHEUS

Post your computer builds here and ask for hardware assistance, buying advice or anything hardware related for computers, smart phones, tablets or other electrical devices.
User avatar
Pri
Site Admin
Site Admin
Posts: 5433
Joined: Fri Dec 14, 2007 8:59 am

Re: PROMETHEUS

Mon Nov 27, 2017 3:21 pm

PROMETHEUS is getting an upgrade. Today I ordered some new hardware, we're maxing out our CPU's.

Current Hardware:
1 x E5-1650 Sandy Bridge-E XEON / 3.2 GHz to 3.8 GHz Turbo / 6 cores / 12 threads / 12MB L3 Cache
1 x Supermicro 4U 48db cooler

New Hardware:
2 x E5-2667 v2 / Ivy Bridge-E XEON's / 3.3 GHz to 4.0 GHz Turbo / 8 Cores / 12 Threads / 25MB L3 Cache each
2 x Noctua 4U 18db coolers / http://noctua.at/en/nh-u9dx-i4

Part Pictures:

ImageImage

This new setup gets us 10 more cores so we'll have 16 cores and 32 threads at our disposal. It also boosts memory bandwidth from 52GB/s to 104GB/s due to moving from Quad Channel to a Octo-Channel memory configuration. It nets us a whopping 50MB of L3 cache up from the 12MB we have right now.

It boosts both our single-thread performance by 5% minimum (likely will be around 7-10% in reality). And it boosts our multithreaded performance by 181% - Yes you read that correctly, 181%. From 11768 points to 33092 points on CPU benchmark tests.

This drastic multithreaded increase is not just due to the 10 extra cores but the extra 38MB of L3 cache which really assists heavily in multithreaded workloads as it's used in core to core data exchange so the more cache you have the more efficiently the cores can work together before needing to access the much slower RAM outside of the CPU's.

This upgrade does come at a cost, it'll be about $809 USD, possibly $909 USD after import duty / taxes as I've had to import these CPU's from America to get them so cheaply.

Usually the CPU's cost £500 each but I managed to get them for £220 each before shipping and import taxes making it about £500 total for both, possibly £600 with import duty and then another £100 for the heatsinks and fans.

I'm really excited about this upgrade. I did consider upgrading us instead to a 1950X 16 core CPU from AMD. However not only does that CPU have lower Single Thread and Multithread performance than two E5-2667 v2's but it also would cost a lot more. £700 for the 1950X on its own + £700 for 64GB of ECC DDR4 Memory + £370 for a motherboard + £60 for a cooler, all that would have come out to £1830 instead of the £600-£700 this will end up costing. So by comparison this is not just an absolute bargain but higher performing too.

I did not do this upgrade for Minecraft, although that is part of it. On our server we have regularly been hitting 100% CPU Usage throughout the day. We mostly sit around 60% CPU usage averaged out with peaks to 100% lasting a few minutes at a time, over a full day that's about an hour or two of 100% CPU load. These high load scenarios are not caused by Minecraft alone, that's only about 25-35% of the problem.

The main thing taking up the rest is all the other stuff the server is doing. Virtual Machines, backing up, downloading, serving my business needs, I run several websites from it one of which serves millions of requests per day. All of this combined has been why the server needed to be upgraded. Due to this mostly being for my personal / business needs I am not going to be placing the cost of this upgrade on our Minecraft donation page, I do not consider this a Minecraft server expense.

I still may break out Minecraft into its own server next year but we'll see how the new dual processor system upgrade handles first. By having two processors and so many cores I am expecting our CPU to turbo up to 3.8-4GHz more often than our current 6 core is able to, thus resulting in higher single thread performance which is what Minecraft craves.

So I hope you found this post interesting, the new processors will be here between December 1st and December 5th.
User avatar
Pri
Site Admin
Site Admin
Posts: 5433
Joined: Fri Dec 14, 2007 8:59 am

Re: PROMETHEUS

Tue Nov 28, 2017 3:57 pm

Looks like I'm also adding 10Gb connectivity. I decided what the hell doing a major upgrade might aswell go all the way. Today I ordered two Intel X540-T2's. Each is a dual-port 10Gb card. So 20Gb total each when bonded together.

This won't really improve server performance at all but it will make my life easier.
User avatar
Pri
Site Admin
Site Admin
Posts: 5433
Joined: Fri Dec 14, 2007 8:59 am

Re: PROMETHEUS

Tue Dec 05, 2017 10:27 pm

Got the 10Gb network cards in today, they are dual port so 20Gb total transfer speeds. I tried a lot of things to get them to run at full speed but the storage at either end is just not fast enough to saturate 20Gb (2.5GB/s) so I created a RAMDISK on the server and used Crystal Disk Benchmark to test it and look at the result..

Image

(Click the image to view full size)


In the task manager screen you can see clearly both Ethernet 10Gb ports running at 10Gb/s and Crystal Disk Bench is quoting speeds of 2.4-2.45GB/s (almost complete 20Gb/s).

Crazy fast. I'd call this upgrade a complete success I'm essentially getting their advertised speeds of 100% line utilisation with some loss for encoding overhead which is expected. General transfers to/from the server are around 550-600MB/s which is about right for the RAID6 array in there and the SSD's are rated around the same numbers.

Overall I'm quite impressed with the results.
User avatar
Pri
Site Admin
Site Admin
Posts: 5433
Joined: Fri Dec 14, 2007 8:59 am

Re: PROMETHEUS

Mon Jan 14, 2019 4:51 pm

Put a new upgrade into the server earlier this month. We are now rocking 256GB of RAM up from 64GB we had previously. The motherboard we're using originally cost me £165 and I think it has delivered value 20x that. Absolute incredible purchase.

Original Hardware:
1 x E5 1650v1 6 Core Sandy Bridge 3.2GHz-3.6GHz CPU
8 x 8GB (64GB) 1600MHz Low Voltage RAM
4 x 1Gb Ethernet

Current Hardware:
2 x E5-2667v2 8 Core Ivy Bridge 3.6GHz-4.0GHz CPU's
16 x 16GB (256GB) 1600MHz Low Voltage RAM
2 x 10Gb + 4 x 1Gb Ethernet

And all of these upgrades have been very cost effective. Below is the retail cost of these upgrades and then what we actually paid.

Retail Cost of Upgrades:
$4200 - Dual CPU's
$1700 - 256GB RAM
$700 - Dual 10Gb Network Card
----
Total: $6600

Actual Cost I Paid:
$600 - Dual CPU's
$515 - 256GB RAM
$103 - Dual 10Gb Network Card
----
Total: $1118

Huge difference in prices by buying stuff as it becomes available in the used market as opposed to buying brand new stuff but the performance is practically identical due to the slowing of moore's law. And the only way these cost savings are possible is due to the choice of motherboard. Dual Socket, 16 RAM Slots, Lots of PCIe slots etc

Finally, a screenshot showing the new memory installed, I will be making use of it when we launch a Hytale server and I intend to add more virtual machines to the server for specific tasks I need done.

Image

Return to “Hardware Hangout”

Who is online

Users browsing this forum: No registered users and 2 guests