Page 1 of 2

Advise for building a file server

Posted: Mon Apr 27, 2009 4:17 pm
by fogus
Hello,

I need to build a new file server (not an active directory machine) for use on a gigabit LAN with both windows and linux machines. I will aim to have this finished by late August. I would like to have approx. 10TB of drive capacity although I may initially only install 6TB. I'd love some advise on the following:

1. Should I build in a tower or rack mount case? (I have no rack yet)
2. What motherboard should I use? (I usually build with AMD or Intel on ASUS)
3. AMD or Intel?
4. DDR3 or DDR2?
5. On board graphics or a card?
6. Onboard SATA or PCI RAID cards?
7. 1TB, 1.5TB or 2TB drives? (I have 6@1TB already)

Also, is there a way to do some kind of "software JOBD RAID"? I am no longer interested in using RAID for performance, but I wouldn't mind seeing all my drives as one big drive over the network (although I do NOT want to ever have a file spanning across drives). The drives must work individually (that is, I must be able to remove one and plug it into another machine and use it). I had a RAID controller fail on me and the idea of RAID as a backup seems like a joke to me now.

All persistent (hard drive) data must be fully encrypted (I've been using truecrypt for this). I'd also like to have the drives auto mount on boot up (encrypted system drive). What do you think about using only key files to do this?

Cheers,

Posted: Mon Apr 27, 2009 5:25 pm
by TheSovereign
super micro tower case will hold 10 drives easily and comes with a board. all you need is drives a processor and memory. lots of free distributions(easy to use) that u can load on for filesharing. such as freenas or openfiler. freenas is more for home based users while openfiler is more pro. just search for server barebones on newegg. raid 0 is available from either distribution in LVM format which is software. LVM raid cannot be used individually. raid is not meant for backups, backups are meant for backups. raids provide support in case of mechanical drive failure and thats all they accomplish. raid 0 is not really raid because the first R isnt present "redundant". both of the distributions i presented support encryption. cheers back at ya!

Posted: Mon Apr 27, 2009 5:31 pm
by FlyingPenguin

Posted: Mon Apr 27, 2009 7:41 pm
by fogus
FP: RAID with parity is unbearably slow from what I've seen. I don't get 1/3rd the continuous write speed from parity that a nested RAID 0+1 offers. I didn't find any info saying that I would be able to run with no RAID at all (is this possible?) That looks like a cool OS though. Would I be able to run truecrypt on that?

Posted: Mon Apr 27, 2009 10:03 pm
by FlyingPenguin
Fogus: Read the FAQ. That is NOT a RAID. That's why it's called UnRaid. It works more like a Drobo.

You can add as many drives as you want of any size, and swap out drives. You can lose any one drive with no data loss. You can also treat the entire array as one drive letter. I believe you can pull a drive and read it's contents on another PC as stipulated in your requirements.

I am experimenting with it myself for possible use for a client. I just threw two drives (a 250Gb and a 300Gb) in an old Athlon32 2200+ box with 512Mb, loaded the OS on a flash drive I had lying around and it booted up and recognized both the onboard NVidia 100Mbit NIC and a Netgear GA310 1Gbit NIC I installed in it. Haven't had a chance to play with it much yet though. Using the free version which is limited to 3 drives.

Not totally sure but you would probably be able to use TrueCrypt to encrypt a folder on the NAS because it would just be an encrypted file as if it was on a Flash drive or external hard drive.

Posted: Tue May 05, 2009 7:36 pm
by fogus
FP: it IS a RAID: (see "unRAID under http://en.wikipedia.org/wiki/RAID0#Non- ... AID_levels ). It's called Unique RAID; the "un" is not used as it is in the Latin sense for "not". It is a redundant array of independent disks. It's a software RAID though, which is a step in the right direction.

Is there anything that would let me software RAID these together under Ubuntu (or some such OS) (in a way that files do not span hard drives and each hard drive could be independently mounted with truecrypt)?

Posted: Tue May 05, 2009 8:16 pm
by FlyingPenguin
Not sure how it works in Ubuntu if at all, but in Windows an UnRAID box is seen as separate drives normally unless you define a User Share which makes all the drives appear as one drive.

True, it is RAID, but pretty unrestricted.

From the FAQ:
unRAID Server is a Network Attached Storage server operating system designed to boot from a USB Flash device and specifically designed for digital media storage. It employs a unique RAID technology, close to RAID4, but without striping, which provides for great configuration flexibility:

* Any combination of IDE and SATA hard drives may be used
* All the hard drives do not need to be the same size or speed
* Hard drives not being accessed may be spun down
* Can rebuild any single failed hard drive


True Incremental Storage

Unlike other RAID systems, unRAID Server supports true incremental storage expansion. You can add capacity by adding more hard drives or by upgrading existing hard drives. This is a great way to make use of older, smaller hard drives you might have laying around.

For example, you might start out by installing one or two new high capacity hard drives along with some number of smaller hard drives you already own. Later, you might decide to replace one of the smaller drives, and unRAID Server will restore the data of the smaller drive onto the new drive, and then expand the file system to incorporate the full size of the new drive.


Better Fault Tolerance

Similar to other RAID systems, unRAID Server permits reconstruction of a single failed hard drive. However in the unlikely event of multiple hard drive failures, data loss would be isolated to only those hard drives which failed. In traditional RAID systems, multiple simultaneous hard drive failure results in complete data loss.

Posted: Wed May 06, 2009 7:03 pm
by fogus
Has anyone used Logical Volume Manager in combination with Truecrypt? I'd really like to know if such a powerful combination is possible.

Posted: Sat Jun 27, 2009 5:55 pm
by fogus
I am now considering going with some combination of RAID cards (NOT using the actual RAID option; using LVM on Linux instead).

What do people think of using this 8 port PCI-E 4x card with, say, a P5Q with 6 on board SATA ports? I could get a solid state 32 GB drive like this one and run CentOS 64bit. Can anyone recommend a cheaper way of doing this?

I am really open to suggestions but I would like to keep the minimum number of drives that I could host to 12. I believe the above combo gives me 13 drives + 1 OS drive. I have been using CentOS at work a lot recently but I would be open to other suggestions. I now have 8@1TB drives.

Posted: Sun Jun 28, 2009 12:43 pm
by Pugsley
I have that RAID card in my file server running raid5. It works ok. It performs like a champ and is easy to setup and manage. I say it is ok because it runs hot. so if your using one make sure it has a fan near it or it is in the air flow.

I used this case and it works great. And its not too expensive of a case for what it is.

Posted: Mon Jun 29, 2009 7:49 pm
by fogus
Pugsley wrote:I have that RAID card in my file server running raid5. It works ok. It performs like a champ and is easy to setup and manage. I say it is ok because it runs hot. so if your using one make sure it has a fan near it or it is in the air flow.

I used this case and it works great. And its not too expensive of a case for what it is.
What MB/s do you get for continuous write under RAID 5?

Posted: Mon Jun 29, 2009 7:57 pm
by Pugsley
Using my network traffic as a gauge about 30,000kB/s

Posted: Tue Jun 30, 2009 1:06 am
by fogus
What do you all think of using this case (thanks to Pugsley) with this motherboard, and this PCI graphics card? (that's right, PCI, not PCI-E, this is a file server) and a cupple of these PCI-E 4x 8 channel SATA cards with four of these SATA cables along with all of the other non-essential peripherals like a CPU?

Posted: Tue Jun 30, 2009 8:57 am
by normalicy
Can't see a reason that all wouldn't work together. Make sure you get some good cooling if you use that many drives though.

Posted: Tue Jun 30, 2009 1:07 pm
by Pugsley
This case only has SATA connectors on it. There is a different model that has the SAS connectors on it. The one I have just has a SATA connector for every drive.