Looking through internet magazines and forums, I found everywhere the same subject: people are using devices called NAS (Network Attached Storage). The range is from very simple devices based on very power efficient ARM CPU with 1 or 2 GB of memory and 1 or 2 hard drives usually connected in RAID 0 or 1 or simple JBOD (Just Bunch Of Disks) to server type with 16/32GB ECC memory and many hard drives (from 4 to 8 usually). Also, in the middle of this bunch, they are companies like QNAP or Synology, where you can have 2-8 hard drives all in very nice cases ready to set up and run… But… Not many people, actually have any clue what they want from this new gold aka NAS device.
I hear all the time that QNAP is the best or Synology is even better… Some people like me, they will prefer to build it from the ground up than relying on 3rd party devices. It all depends on what you want to do with it. Let’s divide those devices into 3 different groups:
GROUP A: This is so-called low powered ARM CPU based NAS with 1 or 2 disks in Raid 0 (stripe) or Raid 1 (mirror) or just as a JBOD (all HDDs connected next to each other). Now, true JBOD works kind of like Raid 0 (in terms of there is no redundancy and NO speed of Raid 0, but if one disk fails, you will lose everything, just like Raid 0). Some companies made it JBOD as “one disk finish, next will start”, so if one dies you will have the other working and you will have an accessible NAS anyway. At least half of data is still available… There are many examples of those devices with nice casing, small LCD… many different services like DLNA to stream your movies to TV or Blu Ray Player any other Media Player, FTP, iTunes… but those devices are designed for people who have NO clue about computers, they just need device to dump their DVD or BD or MP3 collection and forget about it. “Press a button and is ON”, also performance is more likely from 25 MB/s to 50MB/s. Not very often you will see more of that. “Raid 0? Raid 1? What’s that?”
GROUP B: Here are devices from QNAP or Synology. They have a better hardware and software, they use better memory with error correction (we will talk about it later), also they can use many more hard drives (usually up to 8). They have their own proprietary system with GUI, but you can do a lot more than with those devices from group A. Speed can be easily twice more (we are talking about SMB connection for simplicity), but… apart of replacing hard drives, there is not much you can do. Those devices are designed for users with bigger needs also with bigger knowledge and bigger pockets (and we are talking hundreds of dollars).
GROUP C: This is the most configurable, but the most complicated group of all, is also the most rewarding for you. Once configured, you will not have to do it again, but configuration process can be a bit of pain… Creating your own NAS from scratch: choice of CPU, motherboard, memory, hard drives, casing… I made my first NAS out of spare parts, I found around my home. It worked better than any of devices from group A. For free.
I knew, that I don’t need any device from group A, simply because most of them will work for me just like additional SLOW hard drive, and at that time they cost about £150 without any drive. For that price, I could have a new computer, if I wanted. Group B was way too expensive, even without drives. What I wanted was something with redundancy, low(ish) power consumption and a possibility to upgrade when and if needed…
It was my breakthrough when I discovered something called ZFS. “Zettabyte file system”… Zetta… what?! It was something that no other device from group A or B did have. I discovered that it was my ideal file system to store what I need to store. Let me explain: I needed storage to hold my backups from windows, family photos, movies… etc. Usually, I had a spare hard drive, on my desktop where it was formatted in NTFS and I could access it via network simple by share it… It was simple and it worked, or at least I thought it was working… but it didn’t. There is something called a “bit rot” I wasn’t aware of, but I had a first-hand experience: one day I could find some of my pictures half or partially shown, everything else was black or blue… part photo, part plain colour. At the time I didn’t put too much weight on it, so I left it as it was.
Bit rot is when somehow, on your hard drive, one small bit change its state from 0 to 1 or from 1 to 0 without you knowing. “It’s nothing” you will say, but “ONE” bit in the middle of JPEG will render rest of this jpeg plain. This is what’s happened to my family photos. Not much trouble with movies, it will probably just skip frame or few, but for jpeg, it was a killer. Then, I couldn’t decompress some files by 7-zip. “Decompressing error” will say… I start searching and found the reason… It not always will happen, but filesystems like NTFS, EXT2/3/4, don’t have any safeguards for this type of error. ZFS does. It safeguarding against many other troubles like erased files. On ZFS they are not erased but temporarily removed, you can always bring them back unless you have no free space and system needs to save information in those areas, where your erased file was… self-healing when it finds problems… and many more. You need to read about it in depth to find all the perks. For me it was the 4 things: RaidZ2, compression, safeguarding against errors and self-healing.
Compression can be enabled, so any file being dropped onto those hard drives are being compressed on the fly. Yuppi! 100GB of Windows 7 backup, will not take 100GB anymore!
RaidZ2: it was like a song sung to me from far future. Imagine: 6x 500GB hard drives working together as one, where you can sustain 2 hard drives die on you at the same time… Now, they say that RaidZ1 is an equivalent of Raid5, RaidZ2 –>Raid6, RaidZ3 –>Raid7… It’s not entirely true and normal Raid system will not have any of the benefits of ZFS system but many of shortcomings like trouble when you interrupt power when Raid5 is writing a file (so-called “memory write hole”, no problem for ZFS).
OK, enough of theory, you can read for yourself and there is a lot of info out there, just have a look. What they will NOT tell you (mostly) is the hardware needed for all that stuff. Of course, you can base all of it on Intel and you will pay a premium for it (“I’ve got INTEL MAN!”) or simply you can scavenge AMD on the cheap and have working NAS in no time with minimal money spend.
First of all, I want to point out something: the need for ECC memory. There are many schools whether to get them or not, I think that you WILL NEED THEM sooner than later. Why? I had my first production NAS based on Asus motherboard P8H77-I with 6x SATA, ITX size, 2x DDR3 (up to 32GB) and CPU Intel Celeron G1610 @ 2.6GHz, 6GB RAM @1333MHz NON-ECC and 4x500GB hard drive in Raid10. Also, using ZFS with no ECC memory was missing the point. Imagine: If ZFS read block and this block gets corrupted in memory, it will think that blocks on HDD are bad and rewrite them with bad ones…IE whole pool is lost… ECC prevent it from happening and I had first-hand experience with mine where I had memory @1600MHz and AMD CPU could use a maximum of 1333MHz. Of course, memory gets corrupted and system has done all it could: reset before you could use this bad memory block.
To be honest… there are many “theories” whether you must use ECC memory or not, for me, I want to be sure, that “all gaps” are being filled.
After retiring my Celeron to media player duty (it’s actually very happy there), I had a look whether I can get another Intel, but with ECC support. Unfortunately, at the time, all I could find was with a huge price tag as it needs it Cxx series Northbridge and Intel didn’t want ECC support in non-server motherboards even if many CPUs actually support it. Now, Asrock rack does quite cheap ECC supported Intel’s LGA 1150 still with price tag of £150
So, AMD was next in line. After spending hours on the internet finding out what CPU & motherboards can use ECC memory, I’ve found that only ASUS will state its support for ECC, some people had luck with Gigabyte, but others are ignoring this feature. I wanted the latest generation, so I went for an AM3+ motherboard and having experience with ITX, that they are too expensive for what they are I wanted MATX. ASUS M5A78L-M/USB3 was the one. 4x memory slots with ECC support “up to” 32GB, 6x SATA2.0, 2x PCI, 1x PCIe x1 (for Intel LAN card), PCIe x16 for HBA card if ever needed… All I ever need it was there for the price of £49.99. Smashing.
CPU: I had Celeron G1610 before and it NEVER went more than 30%, so all I need was to find something with low thermal envelope and at least half of Celeron’s power. AMD 210e with 45W TDP fit the bill perfectly. It cost me £26 on eBay. Memory? I was to use 6x 500GB so, a memory I needed was like 4 GB, but I go eBay to run for its name and got 2x2GB for £12 and 2x1GB for £6. Nice. All of them 1333MHz. 3U casing cost me another £60. All in about £150. This is what I like.
Operating System: I chose NAS4Free. Simple, it will do all I need and more, forum guys are stars, very easy to deploy.
Whole set: Right now, I have a 32GB ECC @1333 (eBay again @ £70 a piece of 16GB 1600MHz) memory and planning to increase from 6x500GB to 6x2TB, hard drives are: 2x Hitachi 7200.C solid hard drive, 4x Seagate 7200.11 (2 dying slowly). I have to say that Seagate has a bad name after 7200.11 series as they will just die on you.
Performance: 80MB/s writing and 85MB/s reading constantly around it. As for Realtek NIC, it’s a good result. Enough for me. I may fiddle with Intel CT NIC cards to see if I can squeeze more or just wait for new hard drives.
NEW hard drives: I need to replace two of them, so I think I will start to update what I have. There are 3 options: Standard WD RED 1TB @ £47 with 3 years warranty, WD BLACK 1TB with 5 years warranty @ £56 or throw me into darkness with Toshiba (rebranded Hitachi) 2TB for £56 including delivery and 2 years warranty.
I like darkness… I’m working nights 😉 Toshiba!!! I’m coming!!!