AMD Opteron Server – Part 1 (Hardware)

RELATED ARTICLES DISCUSSION   

There are possibilities to buy a full-equipped Opteron-based server (e.g. IBM eServer 325) but we could not forgo a chance to build the server ourselves. This first article describes choice of components and putting them together.
Readers experienced with server hardware can skip this text. It's main purpose is to show differences between our server and low-end desktop hardware. In next parts we will discuss Linux support for AMD64 platform, review 64-bit versions of some popular distributions and compare performance to a dual-Xeon server in several benchmarks.(i-more)

1-way or 2-way?

Our first idea was about an inexpensive single-processor system, not necessary equipped with an Opteron but maybe an Athlon 64 FX. But soon we found out there was no motherboard suitable for such a server. We do not understand this since we can imagine lots of applications which do not need an extreme CPU performance but would benefit from the AMD64 platform (support for large memory, fast memory access, high I/O throughput).
One example from all is a file server (samba, ftp, etc.) on a gigabite network serving many clients. Such server needs lots of memory for disc cache (the more the better), gigabite SG-enabled NIC residing at a fast bus (not 32/33-PCI) and fast storage (U320 SCSI host adapter with several SCSI discs, PCI-X HW-RAID card with SCSI or IDE discs, FC host adapter and external array etc.). All available single-CPU AMD64 motherboards are intended mainly for desktop or workstation use. For example this one by ASUS has onboard 6-channel sound chip, AGP8X and gigabite NIC connected to a 32-bit PCI bridge. Presence of no PCI-X slot in fact prevents from connecting any fast I/O device (like an U160 or U320 SCSI host adapter or full-featured gigabite NIC).
In this situation we decided not to compromise and build a dual-Opteron server.

Processors

Various Opteron models and its model numbers are described here. We got two Opteron 240 boxes as shown on the picture.

The CPU has 940 pins which is the record by now. It is a slightly smaller than Athlon/Duron and finally has a heat spreader! This feature was long been overdue on the Athlon, explained by the ever-present argument of cost.

Motherboard, memory, storage

The most common dual-Opteron motherboards are MSI K8T Master and Tyan Thunder K8 series. Of course, there are other manufacturers like IBM with its own motherboard in eServer 325 but Tyan or MSI products are the most available (from the cost point, e.g.). We choose Tyan because of an option of integrated U320 SCSI host adapter and presence of PCI-X slots. We used the lowest model of Thunder K8 series – Thunder K8S S2880 UGNR.

Let's briefly describe its features. As expected, there are two 940-pin CPU sockets supporting AMD Opteron models from 240 up to 248. There are 4 memory slots for CPU-1 and 2 slots for CPU-2. Only registered ECC modules can be used, maximum amount of memory is 6 x 2 GB = 12 GB.
Hard drives can be connected to dual-channel LSI 53C1030 U320 SCSI controller, to ATA-133 controller integrated in AMD-8111 chipset or to Promise PDC20378 SATA controller. We used two U320 SCSI Maxtor drives, model ATLAS10K4_36WLS (36.7 GB each, 10000 rpm).
Network connection is managed by dual-channel Broadcom BCM5704C GbE NIC (with two RJ-45 connectors) connected to a 133 MHz PCI-X bridge. Expansion slots include two 133/100/66/33 MHz PCI-X slots, two 66/33 MHz PCI-X slots and one legacy 33 MHz 32-bit PCI slot.
Other equipment is quite common (four USB 1.1 ports, one serial port with console support and one parallel port, two PS/2 ports for keyboard and mouse and integrated ATI Rage XL 8MB graphics controller).

Power supply

The motherboard requires EPS12V PSU. This is an ATX extension introduced by Intel, the main connector has 24 pins and the secondary connector used for powering CPUs has 8 pins. We got one of the manufacturer recommended power supplies but I did not work. After consultation with Tyan tech-support and some investigation we found that there exists yet another ATX extension called ATXGES. It uses exactly the same connectors as EPS12V. Moreover, the only difference between EPS12V and ATXGES version of our power supply was a small badge saying "For Intel" and "For AMD". After switching the PSU with the "For Intel" version the server did boot with no problems. That's cool to have an AMD server powered by a "For Intel" PSU, isn't it?

Rackmount Chassis

Dimensions of S2880 motherboard was 12" x 13" (extended ATX) and CPU coolers included in the boxes required at least 2U size (unlike IBM eServer 325 which is 1U!). We choose a 2U chassis EM-2830L by EMKO.

The chassis has a very nice design, aluminium doors, four ball-bearing fans, two temperature sensors and own logic for control of the fans and temperature.

In the following chapter we will take a closer look at AMD64 architecture.

This server is available in our Linux CD Shop. for 89.900 CZK including VAT and installed and configured Linux (approx. 2,730 Euro).