The never ending saga of photo image management workflows – how to best organise and manage your photos

Written by admin on February 18th, 2017

I have been battling this issue for years and it is not an easy one on many fronts:

  • First there is the issue of organising your photos so that you can find them again
  • Secondly is making sure you have your images backed up – preferably in at least two physically separate environments (and preferably on another location in case your house gets burnt down or you are burgled and your computer disks are stolen)
  • Thirdly is a workflow mechanism to generate the highest quality images you can extract from your camera
  • Fourth, is a workflow mechanism to post-process the images to your desired taste
  • Fifth, is a workflow mechanism to output an appropriately sharpened and resized image to your target, be it web or a printer

On this blog post, I will be looking only at the first 3 issues and will be discussing what I have found on my Windows machines – I don’t use MacBooks and these have their own issue with hiding where they keep your photos and the Time Machine backup process becoming corrupt – when was the last time you backed up your Time Machine onto new drives kept elsewhere and how old are the drives in your Time Machine? I prefer manual back up methods so I know it is done and where they all are.

Digital photography has made many aspects much easier BUT it is also much easier to lose lots of your photos if you are not careful and take steps to back them up safely and securely – remember too that hard drives and USB sticks and memory cards all have a finite life – you do need to constantly create new backups on new drives!

We are often our own worst enemy in backing up – a few years ago I had thought that I had backed up several month’s worth of photos from my computer before I replaced it only to discover a year later there are no backups and the originals were all deleted and the computer and drives sent to the rubbish. OUCH!!! YOU MUST KEEP AT LEAST TWO COPIES ON DIFFERENT DRIVES!

I have heard of people backing up their RAW files to the Cloud and that is all and good as an additional measure if you have fast internet and you don’t mind paying an ongoing subscription to store them there.

Disclaimer: There are many options for your workflow, this is just one that is sort of working for me but it may well be too complicated for your needs and you may wish to  simplify it or use something totally different – either way it may highlight a few issues of which you may not be aware.

If you don’t really care about your image quality, just shoot jpegs, copy them to your phone, then upload them into Instagram, apply some image degrading “filter” and post online like most other people – if the content is interesting, the 1 second that people look at your small web image, they hopefully won’t notice the poor post-processing.

The Adobe Lightroom approach:

I have used Lightroom for a few years having given up on Adobe Photoshop when Adobe decided you must have an ongoing monthly subscription payment to use it, and I figured that as I try to get everything as best I can in the camera and I am not a “digital artist”, I can get by without the extra benefits of Photoshop such as layers, etc.

For newbies, Lightroom is a weird beast – unlike most software packages where you can just open a file, view it and edit it, Lightroom forces you to:

  1. create a “Lightroom Catalogue” – a folder system in which is stored your edits and previews
  2. “import” your photos into the catalogue – doesn’t actually store your photos in there, just thumbnail previews and any edits
  3. make a local copy of your photos during import if your folder from which you are importing your photos is a “removable medium” such as a memory card (whether it is external or internal), external USB stick. Lightroom will actually also automatically copy all the photos into a new folder on your computer, by default, in your Pictures folder under the year of import – this is a massive pain if you are using a laptop with minimal free hard disk space! Wouldn’t it be great if Adobe would just put an option in there as to whether or not you want to copy files during import?
  4. use the LR Library folder directory to view you photos inside Lightroom at a later date, you must go to the “Library” section, find the original drive and folder where it was imported from (or to). If Windows has renamed the drive for an external drive, or you have copied the photos to a different folder, you need to right click on the now disabled Library folder highlighted with a ? and select Find Missing Folder and then re-allocate it the new folder location.

The advantages of this process in Lightroom is that for users who have plenty of space on their computer’s hard drive, they can consolidate all their photos in one place and have them catalogued and easily searched using keywords, etc.

The disadvantage is that the catalogue itself can quickly become huge and risk corruption (hence LR reminds you to back it up via its own backup system) or running out of memory. For users who use external hard drives to store their photos, they may have to constantly go through the Find Missing folder process.

A disadvantage of Lightroom for Olympus users is that the RAW conversion within Adobe products requires a lot of sophisticated tweaking to get a similar output as the Olympus camera or Olympus Viewer software in terms of the much lauded “Olympus colours”.

If you are using a new camera such as the Olympus OM-D E-M1 Mark II, you may need LR6 or LR CC to be able to read and edit the RAW files, or you can download the free Adobe DNG Converter and convert them to DNG RAW files which LR5 will read, or you can use Olympus Viewer 3 to convert them to TIFF files.

A solution to getting “Olympus colors” in LR is to initially use Olympus Viewer 3 (a free download) to edit your RAW images (ORF files) and then export them into 16 bit TIFF files which can then be further processed in Lightroom. BUT Olympus Viewer is slow and has a few issues you need to beware of (see below), and the TIFF files are HUGE 88Mb for 16mp shots and 118Mb for 20Mp shots.

A virtual drive to the rescue   …. or not:

A solution to this last issue which I have come up with, is to create a virtual folder using Truecrypt (this is no longer supported but you can download Veracrypt for free to access the Truecrypt files or create Veracrypt versions).

There are two advantages to using Truecrypt/Veracrypt:

  • you gain control over which drive letter gets assigned when you mount the file as a virtual drive
  • by default, the files are encrypted so thieves won’t easily get access to all your family pics

A recent problem I have ran into is that Windows 10 Anniversary Edition – yes that lovely new version Microsoft forced upon us – does not play well with these virtual drives – even when you nicely close them down (dismount) before closing Windows – I was getting regular file corruptions, even without Windows closing, but just on dismounting the drive, and running Windows chkdsk just deleted these files and all my photos (lucky I keep back ups elsewhere!).

I seem to have remedied this corruption issue by selecting the “Mount as Removable Medium” in the Veracrypt mount options (although corruptions and file deletions still occasionally occur so keep an up to date back up!) BUT now if I import photos from within these virtual drives into Lightroom, it insists on creating copies of every photo onto my laptop hard drive with minimal free space.

My laptop does have a microSD slot in which I have a microSD card to provide additional “hard drive” memory but as it only has one USB port, to get my photos from my camera’s SD card onto a Truecrypt virtual drive which resides on a USB external hard drive, I must first copy them to the microSD card, then remove the SD card reader and attach the USB hard drive, mount the virtual Truecrypt drive but this time not as removable medium (keep my fingers crossed it won’t be corrupted by Windows 10 on this episode), copy the photos from the microSD card to the virtual drive, then run Lightroom and create a new Catalogue saved to the virtual drive, and then import the photos from the virtual drive. Close Lightroom, dismount the virtual drive, then remount it with the Removable Medium option checked and then all should be well.

Now, of course, I could have opened Lightroom after attaching the SD card reader and SD card, Lightroom would have detected the card and offered to import the photos and possibly I could have changed the destination folder for the copies to be the microSD card – but 1. I don’t trust LR to do this and not miss copying some files, and 2. after I copied the microSD card photos to the virtual drive, I would have to do the Find Missing Folder process again.

Once I am happy that I have all the photos securely copied, I then make a copy of the dismounted Truecrypt file onto another hard drive – but these files can be quite large depending on how large you want them to be – for example if I shoot 200 RAW shots with large jpegs and I need space for processed TIFFs etc, then I may select 20-25Gb as a FAT32 encrypted file. Don’t forget your password to mount the file!!

One disadvantage of this is that you fragment your “library” into multiple smaller catalogues but to me the advantage of keeping your LR catalogue data together with your photos on the one drive and your catalogue remains small and manageable, outweighs this.

The Olympus Viewer 3 approach:

Olympus Viewer 3 is free to download for registered purchasers of Olympus cameras.

When I first used it to convert my RAW photos to TIFF I was shocked at the poor quality – lots of smudged image details on pixel peeping which I was not used to seeing in my Adobe RAW conversions.

I then discovered something which had not occurred to me – the TIFF export process in Olympus Viewer 3, although it has a few basic options in the dialog box, doesn’t warn you that, by default, the TIFF file it is going to create will be based upon the camera settings when the RAW file was shot – so essentially it is going to make a TIFF file version of your jpeg settings – and if you are like me who uses Vivid Picture Style or other styles to gain faster AF, these will, by default be applied to your TIFF file.

Thankfully, OV3 allows one to change all of these settings and then even save them as a settings file which can then later be applied to a batch export.

So, I set Picture Style to Natural (or you can choose Portrait for perhaps a little more dynamic range and less contrast), turned OFF the noise filter, Gradation normal, Contrast 0, Saturation 0, and set Sharpening to minus 2.

In addition to the saved My settings file, you can create a Batch Processing Settings File with your preferred parameters:

  • in the main OV window click on a RAW file and INSTEAD of clicking on the RAW icon, click on the EDIT icon
  • this will take you to a different RAW edit window – ensure your processing settings are as you would like them
  • select from the menu, [Edit] then [Save batch Editing File] and then you can save this file for later use in the Export dialog under Advanced Settings

My lovely TIFFs were back again without any over-processed digitized artefacts, and all ready for me to play with them in Lightroom.

You can also use the OV Export function to create resized jpegs with your Olympus colors – just change the Export dialog settings accordingly, but in this case you may want to create a special batch processing file that applies some sharpening, etc.

Olympus Viewer also copies versions of your images to a cache folder on your hard drive – so if you are space challenged like me, make sure you clear the cache every so often! See Tools:Database:Clear Cache.

Summary workflow:

  • Copy RAW files to computer drive (if you don’t wish to take the SD card out of your camera, you could use a USB cable and OV or LR will copy them across to the hard drive for you but this will be a slower option if your camera or computer is only using USB 2.0 instead of USB 3.0)
  • Open Olympus Viewer and select the folder, then press Ctrl-A to Select ALL
  • Click on RAW icon this will open the RAW Development module
  • If you have already saved your favorite development settings in a file, use the My Settings to select that file and the settings will be applied to all of your RAW files you have selected.
  • Click on the Export button and set destination, 16 bit TIFF, etc in the subsequent dialog box then press Save to export the files – this will take some time.
  • Then the 16 bit TIFF files can be imported into Lightroom or whatever software you wish to use.

Other RAW conversion options:

There are a number of software packages available which you can use to convert your RAW photo files into TIFFs or jpegs.

Perhaps the best for Olympus users is Capture One Pro but this will cost you 279EUR + VAT if you live in Europe. It is said to give colours close to the “Olympus colors” and is faster and has more editing tools than are available in Olympus Viewer.

Photographers will all have their own personal likes and dislikes with each of these packages.

See my wiki page for more info on these for Olympus users.

 

You must be logged in to post a comment.