Thursday, February 06, 2014

Backing up Music Projects

Let's face it, hard drives fail and files get deleted.  As Forrest Gump once said, "Shit Happens."

Imagine, if you will, that the device you are reading this blog post on - particularly if it's a laptop or desktop - suddenly melts down or is completely pulverized (and of course you are perfectly unharmed!).  Replacing the hardware probably won't be too expensive - chances are it was due for replacing anyway.  But what of the creative files that existed on that device?  What of those?  Are you certain they are backed up?  You may say, as I did, "Yes, absolutely - I have Carbonite and it's backing up all my files.  If this device fails, I can restore all my creative projects."  You may be right - but only partially right. True, your photos, documents and various other files may indeed be backed up.

But being partially right, at least when it comes to backups, may mean you're totally screwed.  

I recently discovered - and fortunately sooner, rather than later - that certain Abelton Live project files (which consist of WAV audio and other associated files within a project folder structure) were not being backed up.  Some of them were - but the truth is that if my laptop crashed right now (and without a manual backup) my next CD project (of which I've spent considerable time on over the last year or so) would be completely lost!

Here's the good, bad and utterly ugly truth of Carbonite's backup software (on Windows at least) when it comes to backing up Abelton Live projects:

The following is a sub-directory with several of my Abelton Live projects- as you can see, the "green dot" is Carbonite's way of providing visual feedback that my project is being backed up - so far that seems good:

But here's the "bad" - if we look into the sub-folders of this particular project, we see two folders ("Abelton Project Info" and "Samples)" that have a Carbonite "green dot" signifying that they are backed up.  There's a "Desktop.ini file which is insignificant and no big loss if it's not backed up.  However - and this gets ugly fast folks - the actual Abelton Project file (in this case the file with the .ALS file extension) is NOT being backed up!  Attention music creators:  this is the frickin' project file.  It's the file that Abelton Live needs to open to load the project in order to render the MIDI and associative WAV/audio data.  This alone is very ugly - but, unfortunately, it gets even worse!

The logical step is to simply say, "Oh, I see.  Carbonite doesn't automatically backup this file so I have to manually tell it to do so."  If I right click on the .ALS project file and click "Properties" and then click the "Carbonite" tab, something like the following dialog appears.  First of all, you'll note that the "Back up" check box is grayed out.  That's bad - or ugly, I'm not sure which.  It doesn't matter because the other check box with the label "Back up files of this type (within folders selected for backup)" does not retain my check setting.  And that's just darn-right annoying:

Oh dear reader - if you've suffered this far into my ranting post you're probably thinking, "Well at least the WAV files are being backed up - right?"  No - they are not.  If my hard drive crashed this second - and I did not have an alternative backup plan (which I do) - this project (and all of my other ones) would be lost to the giant bit bucket!

As we saw previously, the "Samples" folder (which stores the WAV audio data associated with the project would appear to be backed up.  After all, it has the trusty Carbonite "green dot" on the folder.  But the software is deceptive in the worst of ways.  If we navigate into the "Samples" folder there will be a "Recorded" folder, and it too will have the Carbonite "green dot" - but within that file are the WAV audio files, which are.....evil dissonant chord here....NOT being backed up!  Nor, apparently, will Carbonite allow them to be:

Again, any attempts to override Carbonite's settings to force the WAV files to be backed up proved unsuccessful (for me at least).

In short, there's only a perception that Abelton Live projects are being backed up by Carbonite.  The reality is that any restore of the actual backup (from Carbonite at least) would produce a mere skeleton of the projects folders - nothing of substance would have been backed up.  All that work - gone forever!

But fortunately not for me, and hopefully not for you either now that we both know.

One very simple solution is to back your music projects (AbeltonCarbonite, Finale, or a host of other different music apps that we, as music creators, use to create and produce our music with) to a separate backup device.  Here's where the antiquated, but highly efficient, DOS-style "batch file" can help out, at least until a more robust and comprehensive "cloud" solution can supersede Carbonite:

You'll need to substitute your own "rob\music" directory (and possibly even the "C:" drive) reference, as well as the "J:\backup\rob-music" with your own corresponding "source to target" drive/folders.  The XCOPY command with the /Y /R /D /E parameter flags are sufficient to backup all sub-directories that have changed since they were last backed up.  And of course, the *.* ensures that ALL files, without exception, are backed up to your target drive.  The XCOPY backup approach is far from perfect.  But you can implement it, freely, in a matter of a couple minutes.  This technique is not ideal when folders and/or files change names - you'll find your backup will have extra copies of files.  But at least they will be backed up and that's a small price to pay.

We've all seen the Carbonite ads - I like this one, about the guy that manages to get his backup started in a matter of minutes and is then able to frolic the rest of his day away while Carbonite does all the work.

But I can't help imagine that he's a musician/producer.  His partner is, in my imagination, a musician as well and together they have a group and big dreams. Later that month they will both discover that all of their new recording projects (including her vocals and his banjo solos) were completely lost due to the fact that he never actually verified that their critical DAW project files were being backed up!  So much for the great Carbonite value proposition.  For $59 a year, backup your stuff.  Rest easy.  Then realize later - much too late - that the stuff you thought was backed up was, in fact, not.

Is it possible that there's a Carbonite config file I can dig into to override the settings?  Maybe.  However, options, as far as I can tell, to force Carbonite to backup files not automatically backed (like Abelton Live projects) do not appear to be available.

I believe this to be a major flaw in the Carbonite software.  In my particular case, the Abelton Live project files are not being completely backed up.  I suspect, however, that there are numerous other creative files produced by artists, writers, and filmmakers that are not being backed up, yet the perception and false sense of security that they are persists.

There are clearly internal business rules in Carbonite's software layer at work here - the lack of options to override these internal rules is also, in my view, a major flaw.  I have considered the possibility that these  rules are intentional; perhaps to lower the volume of files being backed up to their data warehouse storage sites.

I've reported the issue to Carbonite support and, to their credit, they did get back to me - it was an automated reply directing me to setup an appointment to speak with someone directly.  I haven't done this yet - maybe I will, maybe I won't.  It's a drag to have to operate like that.  Why can't the written word and some screen shots convey the issue?  Why must they connect into my system to diagnose this?  It seems funky to me.

Anyway, that's pretty much my rant for this month - it's a bit technical but definitely related to music - but not just music.  It had to be said.

As creative people, we have to take care of business and make sure our hard work isn't inadvertently lost.  Everyone needs a backup plan!  And every once in a while it's a good idea to make sure that plan is working as advertised.