Friday, July 12, 2013

What get stored is getting larger

My last entry tried to describe how historical data in the electronic age can get lost and degraded over time.  I intimated that the advent of purely digital storage systems would counter that scenario due to several properties of digital storage.
  • Digital storage is content format neutral.
  • Copying from one generation to the next generation of storage systems is a one step process and the same user cost no matter how large the content.
I realize that there are some unspoken assumptions behind this claim as well.
  • For the same or lower operational cost, the next generation has larger capacity.
  • You (or someone) is willing to expend the on going operational cost.
I used to think that continuous and sometimes rapid improvement of technological  would have to come to an end.  I don't think that end will be in sight as long as we maintain a commercially viable  economy. ( I might be able to explain that assumption in a future post).  The real bug-a-boo is the assumption that someone is willing to expend the costs.  Even with the costs declining it's still a cost and has to be measured against any other use or value that can be had from expending that cost on something else.  Which is a convoluted way of saying that maybe I, or you, or someone will get bored or tired of keeping the content around.  Or find something else to store of higher value in that space.

I believe that content format will also play a role in that lack of interest in continuing to maintain the content, this is really a degradation of value.  And it plays out in decades and sheer quantity.  If you come to believe that the content has little to no value, say for example, because the content was acquired in a way no longer considered viable, why would you expend any effort to maintain it?

I think that statement is true no matter what the content, whether it be digital medical records or digital photographs.  The variability in format is part of the picture, but in a subtle way.  Let's take digital photographs.  In the beginning of my storage I used jpeg and slowly added raw when it became available.  I still shoot jpeg for a variety of reasons.  First the summary:

Date Camera MegaPixels JPEG sizes Raw Sizes
June 15, 2001 Casio QV3000 3 1 N/A
June 12, 2004 Panasonic FZ10 4 3-4 N/A
March 26, 2005 Canon Digital Rebel XT 8 3-4 10
Feb 6, 2008 Olympus E510 10 7 9
April 5, 2009 Nikon D300 12 5-9 11-12
April 22,2009 Panasonic G1 12 3-5 14
March 26, 2012 Olympus E-M5 16 8-10 12-13

A flower sample from each, a same sized crop from what a print would look like viewed from the same distance.  At the smaller size all the cameras are pretty good (minus the bad exposures in several of them).  At the larger size, I have never printed this large, to my eye's things don't start looking good until 12 MP.  Too bad I didn't have the foresight to photograph exactly the same thing with each camera over the last 12 years :)

Camera
11 x 15 print
4 x 7 print

Casio QV3000




Panasonic FZ10




Canon Digital Rebel XT




Olympus E510




Nikon D300




Olympus EM5











Here is what I realize.  With flower pictures or any scene that can be easily found again in the real world, the superiority of camera's 2 to 3 generations ahead is so great that I am unlikely to keep the earlier photo's if I have to make a choice.  I am still copying them forward, simply because it has little to no cost, since between 2001 and 2013, my storage had increased into the Terabytes on the server side and 8 to 32 GB memory cards in the camera allow me to shoot 2 week vacations in both raw and jpeg.

One more thing, while one can argue that for my purposes of on screen viewing and at max 8 x 10 prints, 12MP is as good 16MP.  I do not think the MP increase is over and there are several reasons for that.  There are advantages in reducing noise and increasing apparent acuity but perhaps the most important is that it allows a true digital zoom.  The example here is to compare the Canon XT image to the Olympus EM5 image, one could crop out a 8 MP or so image from the Olympus and achieve acceptable digital zoom.  First quality digital zoom wont happen until we have 24 to 36 MP to play with.  You can already see that advantage in the smallish Nokia Lumia  phone cameras that use 48MP to reduce noise and allow digital zoom.

No comments:

Post a Comment