RAID Storage for Photographers and Filmmakers Explained

Explaining the meaning of RAID is among the most difficult concepts to describe to my students and clients. It sounds like a military term, at best, and super geeky tech talk that only mechanical engineers understand.

RAID stands for Redundant Array of Independent Disks. It is a system of two or more connected hard drives that work to improve performance, store information in a more reliable manner, and provide additional levels of protection. I’d also argue that by decreasing the chance of data loss and therefore minimizing wasted time, RAID systems also increase productivity.

Article 010 - Image 01 - RAID 01

 

Over the past few years demand for RAID systems has grown exponentially, mostly because they are optimized for digital content creation (sound design, video editing, motion graphics, digital retouching, etc.) and are relatively low in price. They also provide excellent backup solutions for photographers and filmmakers.

Current portable computers have more than enough processing power to support Full HD and even 4K video editing (Here’s an “out of the box” system I often recommend). However, when editors need to work with Apple’s ProRes 422 codec, along with Apple’s Intermediate Codec (AIC), and multitask with software applications like Adobe Premiere Pro, Apple Final Cut Pro, Adobe After Effects, or Adobe Photoshop, hard disk bandwidth requirements increase so dramatically that laptop-based, multi-stream, real-time editing is barely possible.

A typical workflow involves storing all the original assets on an external RAID and working with two more hard drives in a manner such as this:

Hard Drive 1 = Operating System and Applications (ideally a Solid State drive)

Hard Drive 2 = Video and Audio Files (configured as RAID 0 [see below] for faster performance)

Hard Drive 3 = Export Files (typically a cheaper USB 3 drive)

Most current systems typically offer capacities from 2TB to 8TB and are compatible with Apple’s iMac, MacBook Pro and Mac Pro, as well as with Linux and Windows workstations.

High-end RAID systems usually monitor and manage all aspects of the system, including the hard drives’ health, RAID set controller, controller cache, fan, power supply, and enclosure temperature. A front-panel LED display provides visual feedback, and an audible alarm signals a controller or drive failure. Some systems allow users to monitor and manage the RAID storage from anywhere (via TCP/IP) and they can even send error notifications via email.

The “catch” with RAID systems is that they offer several different levels of configurations, and that’s what causes the most confusion.

Different configurations and when to use them.

There are several different levels at which a RAID system can be configured. In the photo and video industries, we generally use RAID levels 0, 1, 5, and 6. They are categorized something like this:  Fastest (0), Mirror (1), Safe (5), and Safest (6). Let’s see what each configuration does and take a look at their advantages and disadvantages.

RAID 0:  Speed
A RAID 0 (also known as stripping) provides maximum performance and speed by accessing multiple drives simultaneously.

RAID 0 does not provide any data redundancy, which is a high risk, but offers the fastest speed of all of the levels by breaking up the data into smaller blocks and then writing a block to each drive in the array.

The probability of a RAID 0 failure increases in direct proportion to the number of drives, which means that the chance to lose data with an eight-drive RAID 0 array is eight times higher than with a single drive.

RAID 1:  Mirror

RAID 1 (also known as disk mirroring) increases reliability. The software or hardware controller mirrors the data on other drives, thereby cutting the chance of data loss by half. If one drive fails, all of the data is preserved on the other drive.

Unfortunately, storage capacity is also cut in half and write speed is much slower because the data has to be written twice. Also, because parity data requires disk space, a system’s data storage space is less than the RAID’s total capacity.

RAID 5:  Safety

On a RAID 5 array, the data is written to all of the drives instead of being concentrated on a dedicated disk. For example, you can play a movie and pull a drive from the system, and playback will continue without issues or losses. If one drive in the system fails, the parity information can be used to reconstruct the data from the failed drive. All drives in the array system can be used at the same time, which greatly increases performance.

RAID 6:  Safety and Performance

A RAID 6 array is essentially an extension of a RAID 5 array, with a second independent distributed parity scheme. RAID 6 provides an extremely high fault tolerance, and can sustain two simultaneous drive failures without downtime or data loss.

Capacity obviously depends on many factors like capture format, resolution, etc., but for the sake of argument, let’s say a 1TB Hard Drive can hold up to:

84 hours of HDV 1080i

10 hours of ProRes 422 HQ

2.25 hours of HD 1080/60i

Keep in mind that having four 1TB hard drives does not mean that you have 4TB of hard drive space available. Parity data requires disk space, so a system’s data-storage space is always less than the RAID’s total capacity. Also, depending on the RAID level that you choose, you might see a significant reduction of available capacity.

On Raid 1 (mirroring) the amount of available space is 50% of the total capacity. On a Raid 5 and 6, the amount of available space is (on average) 75% of the total available space of the drives, when used with three or more drives.

Professional RAID systems offer a chassis that can hold up to eight drives and use load-balancing power management to divide the workload between two power supplies. This enables the system to continue working in the event of a power-supply failure. They support multi-stream HDV, DVCPro HD, XDCAM HD, ProRes 422, and uncompressed SD workflows, and they offer high-speed interfaces like eSATA, FireWire 800, USB 3.0, and Thunderbolt which was specifically designed for performance, simplicity, and flexibility, and it is capable of delivering a blistering two channels of 10GB/s (1.25GB/s) per port of performance.

Article 011 - Image 01 - RAID 01

So, there you have it, RAID systems explained. I hope this article makes you feel more comfortable about adding a RAID system to your photography or video workflow.

 


Eduardo Angel is an independent Technology Consultant, Educator, and Visual Storyteller based in Brooklyn, NY. He currently teaches at The School of Visual Arts and the International Center of Photography, and mentors the photography program at the Savannah College of Art and Design.

He is a co-founder of the idea production company The Digital Distillery, author of popular filmmaking courses on Lynda.com, and regularly shares his thoughts on technology, photography, and cinema on his website eduardoangel.com.

One comment

  1. Be mindful of hard errors which are the reason that RAID no longer lives up to its original promise. If one of the drives fails, then during the rebuild if you get an error – the entire array will die.

    http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/

    SATA drives are commonly specified with an unrecoverable read error rate (URE) of 10^14. Which means that once every 12.5 terabytes, the disk will not be able to read a sector back to you.

    http://www.lucidti.com/zfs-checksums-add-reliability-to-nas-storage

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>