What To Choose: a NAS for home
Looking through internet magazines and forums, I found everywhere the same subject: people are using devices called NAS (Network Attached Storage). The range is from straightforward devices based on very power-efficient ARM CPU with 1 or 2 GB of memory and 1 or 2 hard drives usually connected in RAID 0 or 1 or simple JBOD (Just Bunch Of Disks) to server type with 16/32GB ECC memory and many hard drives (from 4 to 8 usually). Also, in the middle of this bunch, they are companies like QNAP or Synology, where you can have 2-8 hard drives all in adorable cases ready to set up and run… But… Not many people, actually have any clue what they want from this new gold aka NAS device.
I hear all the time that QNAP is the best or Synology is even better… Some people like me, they will prefer to build it from the ground up than relying on 3rd party devices. It all depends on what you want to do with it. Let’s divide those devices into three distinct groups:
GROUP A: This is so-called low powered ARM CPU based NAS with 1 or 2 disks in Raid 0 (stripe) or Raid 1 (mirror) or just as a JBOD (all HDDs connected next to each other) like single disk WD My Book Desktop 4TB or double disk version WD My Book Duo Desktop RAID 4TB. Now, true JBOD works kind of like Raid 0 (in terms of there is no redundancy and NO speed of Raid 0, but if one disk fails, you will lose everything, just like Raid 0). Some companies made it JBOD as “one disk finish, next will start”, so if one dies you will have the other working, and you will have an accessible NAS anyway. At least half of the data is still available. There are many examples of those devices with a beautiful casing, small LCD. They can have many different services like DLNA to stream your movies to TV, FTP, iTunes… but they designed for people who have NO clue about computers, and they just need a device to store their DVD or BD or the MP3 collection and forget about it. “Press a button and is ON”.
GROUP B: Here are devices from QNAP or Synology. They have better hardware and software, and they use better memory with error correction (we will talk about it later), also they can use many more hard drives (usually up to 8). They have their proprietary system with GUI, but you can do a lot more than with those devices from group A. Speed can be easily twice more (we are talking about SMB/Windows Share for simplicity), but… apart of replacing hard drives, there is not much you can do. Those devices designed for users with more significant needs, also with more extensive knowledge and bigger pockets (and we are talking hundreds of dollars).
GROUP C: This is the most configurable, but the most complicated group of all, is also the most rewarding for you. Once configured, you will not have to do it again, but the configuration process can be a bit of pain. Creating your own NAS from scratch: choice of CPU, motherboard, memory, hard drives, casing… I made my first NAS out of spare parts, I found around my home. It worked better than any of devices from group A or B. For free.
I knew that I don’t need any device from group A, simply because most of them will work for me just like a new SLOW hard drive, and at that time they cost about £150 empty. For that price, I could have a new computer, if I wanted. Group B was way too expensive, even without drives. What I wanted to be something with redundancy, low(ish) power consumption and a possibility to upgrade when and if needed.
It was my breakthrough when I discovered something called ZFS. “Zettabyte file system”… Zetta… what?! It was something that no other device from group A or B did have. I discovered that it was my ideal file system to use for what I need to save. Let me explain: I needed storage to hold my backups from windows, family photos, movies, etc.
Usually, I had a spare hard drive, on my desktop where I’ve formatted it in NTFS and I could access it via network simple by share it… It was simple, and it worked, or at least I thought it was working, but it didn’t. There is something called a “bit rot” I wasn’t aware. I had a first-hand experience: one day I could find some of my pictures half or partially shown, everything else was black or blue, part photo, part plain colour. At the time, I didn’t put too much weight on it, so I left it as it was.
Bit rot is when somehow, on your hard drive, one small bit change its state from 0 to 1 or from 1 to 0 without you knowing. “It’s nothing” you will say, but “ONE” bit in the middle of JPEG will render rest of this jpeg plain. It happened to my family photos. Not much trouble with movies, it will probably just skip a frame or few, but for jpeg, it was a killer. Then, I couldn’t decompress some files by 7-zip. “Decompressing error” will say… Below is an example of what “bit rot” can do.
I started searching and found the reason. It not always will happen, but filesystems like NTFS, EXT2/3/4, don’t have any safeguards for this type of error. ZFS does. It is safeguarding against many other troubles like erased files. On ZFS they are not deleted but temporarily removed, you can always bring them back unless you have no free space and system needs to save information in those areas, where your erased file was… self-healing when it finds problems… and many more. You need to read about it in-depth to see all the perks. For me, it was the four things: RaidZ2, compression, safeguarding against errors and self-healing.
Compression can be enabled, so any file dropped onto those hard drives are being compressed on the fly. Yuppi! 100GB of Windows 7 backup, will not take 100GB anymore!
RaidZ2: it was like a song composed to me from far future. Imagine: 6x 500GB hard drives working together as one, where you can sustain two hard drives failures on you at the same time. Now, they say that RaidZ1 is an equivalent of Raid5, RaidZ2 –>Raid6, RaidZ3 –>Raid7. It’s not entirely true. Standard Raid 5 system will not have any of the benefits of ZFS system but many of shortcomings like trouble when you interrupt power when Raid 5 is writing a file (so-called “memory write hole”, no problem for ZFS). Also, if your Raid card dies, you will have to replace it with the same make and model (sometimes you can round it up to the same family of cards).
OK, enough of theory, you can read for yourself, and there is a lot of info out there, have a look. What they will NOT tell you (mostly) is the hardware needed for all that stuff. Of course, you can base all of it on Intel, and you will pay a premium for it, or you can scavenge AMD on the cheap and have working NAS in no time with minimal money spent.
First of all, I want to point out something: the need for ECC memory.
There are many schools whether to get them or not, and I think that you will need them sooner than later. Why? I had my first production NAS based on Asus motherboard P8H77-I with 6x SATA, ITX size, 2x DDR3 (up to 32GB) and CPU Intel Celeron G1610 @ 2.6GHz, 6GB RAM @1333MHz NON-ECC and 4x500GB hard drive in Raid10. Also, using ZFS with no ECC memory was missing the point. Imagine: if ZFS read block and this block gets corrupted in the memory, it will think that blocks on HDD are bad and rewrite them with bad ones and the whole zpool is lost. ECC prevent it from happening. I had the first-hand experience with mine where I had memory clock set manually to 1600MHz, but the memory was 1333MHz thus overclocking. It was working before on normal board as non-ECC. Of course, memory gets corrupted, and the system has done all it could: reset before you could use this bad memory block.
There are many “theories” whether you must use ECC memory or not. I want to be sure, especially that was designed to run 24/7.
After retiring my Celeron to media player duty (it’s delighted there), I had a look whether I can get another Intel, but with ECC support.
Unfortunately, at the time, all I could find was with a considerable price tag as it needs it Cxx series Northbridge and Intel didn’t want ECC support in non-server motherboards even if many CPUs support it. Now,
Asrock rack does quite cheap ECC supported Intel’s LGA 1150 still with a price tag of £150
So, AMD was next in line. After spending hours on the internet finding out what CPU & motherboards can use ECC memory, I’ve found that only ASUS will state its support for ECC, some people had luck with Gigabyte, but others are ignoring this feature. I wanted the latest generation, so I went for an AM3+ motherboard and having experience with ITX, that they are too expensive for what they are I wanted MATX.
MOTHERBOARD: ASUS M5A78L-M/USB3 was the one. 4x memory slots with ECC support “up to” 32GB, 6x SATA2.0, 2x PCI, 1x PCIe x1 (for Intel LAN card), PCIe x16 for HBA card if ever needed, and onboard graphics card, so no need for separate one… All I ever need it was there for the price of £49.99. Smashing.
CPU: I had Celeron G1610 before, and it NEVER went more than 30%, so all I need was to find something with low thermal envelope and at least half of Celeron’s power. AMD 210e with 45W TDP fit the bill perfectly. It cost me £26 on eBay. Memory? I was to use 6x 500GB so, a memory I needed was like 4 GB, but I go eBay to run for its name and got 2x2GB for £12 and 2x1GB for £6. Nice. All of them 1333MHz. 3U casing cost me another £60. All in about £150.
OS: I chose NAS4Free (now XigmaNAS). Simple, it will do all I need and more, forum guys are stars, very easy to deploy.
Full Set: Right now, I have a 32GB ECC @1333 (eBay again @ £70 a piece of 16GB 1600MHz) memory and planning to increase from 6x500GB to 6x2TB, hard drives are: 2x Hitachi 7200.C solid hard drive, 4x Seagate 7200.11 (2 dying slowly). I have to say that Seagate has a bad name after 7200.11 series as they will die on you.
Performance: 80MB/s writing and 85MB/s regularly reading around it. As for Realtek NIC, it’s a good result. Enough for me. I may fiddle with Intel CT NIC cards to see if I can squeeze more or wait for new hard drives.
NEW hard drives: I need to replace two of them, so I think I will start to update what I have. There are three options: Standard WD RED 1TB @ £47 with three years warranty, WD BLACK 1TB with five years warranty @ £56 or throw me into the darkness with Toshiba (rebranded Hitachi) 2TB for £56 including delivery and two years warranty.
UPDATE: Mar 2020
Since this article was written, the core principles didn’t change much. I have access to more professional knowledge and equipment, so naturally updated with “better” hardware (more speed, but keeping similar power consumption). Here is the latest version of my XigmaNAS.