Feng is fired!  Film at 11!

Sophie Tech 3: Big picture

Today's post is a shorter than normal journey through what data will be like for a child growing up today.  This was prompted by the pictures of Sophie I just saw last night.  In one particularly cute one, she is casually glancing back over her shoulder with a little grin on her face, and I noticed a few light speckles around her mouth.  Upon further examination, I found that they were specks from the snack she had eaten during the session that didn't get wiped off (or what more likely happened - she palmed a cheese puff and ate it when she got a chance).  Right there in the high-definition glow of the photo was a flaw that would in another time be unnoticed.  With a regular camera, the particular background and the other things happening would have softened the crumbs so that they weren't readily visible.  Regular film was good at that, giving an approximate representation that allowed the best possible interpretation on the part of the viewer.

Enter the digital camera.  With the definition present, each crumb stood out in its fake yellow-orange glory for all the world to see.  This is an interesting switch for the human mind and how it generally operates, and is linked to the difference between analog and digital technology.  Without trying to over-simplify this, I ask you to put two pictures in your mind.  The first is a giant S as tall as the Golden Gate Bridge, lying on its side.   The second is the picture out of the cell phone commercial that shows the same Golden Gate Bridge represented by the bars (indicating the service levels present).  If you put those two pictures op top of each other, they would approximate the same height, and would cover the same territory.

When you look in your mind's eye, though, you can tell a difference between the two.  The sideways S is smooth, while the form of the bars is a little rough, with flat spots instead of the gradual curving that takes place.  Even if you put more bars in that were thinner, you would still have a very different picture than the smooth curves.  This represents the difference between analog and digital information.  The analog information is the smooth S, the sound as it actually happens, or the picture as it actually appears.  The digital information is the picture made up of the many bars, each bar representing a sample.  As an aside, when someone talks about improving the digital quality, they are really talking about increasing the sample rate so that the bars get thinner and thinner and more closely represent what is actually there.

No matter what you do, the digital will not be the same as the analog.  It will be neater, more orderly, and more clear in content, if not intent.  In an analog photograph. there are grains that occur, but they are not all uniform, and the grain count is the average count per square inch, or some other increment.  The pieces all fit together and either form definite edges or smooth transitions between edges, and there are some fine details that get averaged into the other details in such a way that they get faded out, or covered over.  The general effect is a more natural appearance.  After all, this is how our eyes and brains work, harnessing processing power with the ability to do something that has eluded computing technology - smooth away the lines and barriers to grasp the whole picture.

In the digital world, the concept of grains is gone, replaced by pixels.  Unlike grains, each pixel is both the same size, and has the same distribution over the entire object.  Unlike grains, there is no transition between pixels, and each pixel can have only one color.  If you want a smoother transition, you must add more pixels, each getting incrementally smaller until you attain as good a picture as time and technology allow.  As a side effect, the more pixels you throw in there, the more varied colors you can find within the same image, colors that are taken by the natural eye and analog film and blended together, but in the digital image are starkly separated.  Enter the crumbs.

As cute as the crumbs are, I wonder how she will react in an increasingly digital world, where there are more details and less continuity and blending than ever before.  The same worry exists about the data within our systems.  We have a pile of 1's and 0's, and then we use logic and some smattering of skill to assemble them, then we interpret the results, inferring what is not readily apparent and constructing a picture that we can use to order our business and decisions.  This is a messier process, but it does involve the human intellect.  Taking the 'guesswork' out can make for a neater picture, but it can also cause the loss of the 'soft' data, the interpretive data, the things that actually glue together cohesive work packages into one, successful piece.  In other words, you can build a CRM [Customer Relationship Management - a 'Holy Grail' of software packages that attempts to replicate the savvy of Dave, the salesman who's been there 30 years and knows everyone, including all his customers, knows who's a Cowboys fan and who hates the Patriots, and keeps them straight when he sends out his touchy-feely e-mails, and does all the relationship-building things with his customers, without actually hiring a Dave - basically bringing the skilled position of salesman into a system and using only data operators] package that is successful, but it still can't understand the subtle nuances of relationships.  This is the problem for us as we write systems, and it is the problem for Sophie as she grows up in a world where everything is presented in a nice, neat little box, without the interpretation or discrimination that comes from real-world life.

Hopefully in our quest to capture and record everything within our data, we do not pixellate everything, reducing everything down  to a place where if you add all the pieces together they do not exactly equal the whole (remember when you are thinking about analog versus digital that the word lossless refers to the amount of the digital signal that is lost when transferring to another digital device - if you turned each sample into a number and you added them together, the sum would not be totally equal to the analog - it gets close, but until you can divide 10 by 3 and come out with a terminating number, the sum cannot by definition be the same).  Sometimes the mastery of a field has less to do with analysis of the parts than it does with the analysis of the whole.  Until humans are totally removed from interfacing with technology on a permanent basis, the more finely we pixellate our data, the less we understand fully.  Besides, sometimes when the Daves who are good at the soft data side provide that relationship with customers who deal in absolutes all day, the mixture leads to a great number of innovations that wouldn't normally have come up.