Section 2. The Segmenting/compositing Software.

General Considerations.

Areas of sharp focus would tend to be areas in which there was maximum RGB/luminance difference between adjacent pixels. In the simplest case, these would be areas of sharply resolved contrasty texture. Unfocused areas will tend towards the condition in which the RGB/Luminance values of the subject are integrated across the area of the pixel array, resulting in minimal difference between adjacent pixels. The quality of the final composited image will depend upon how cleanly the sharp and unsharp areas can be defined and joined. How well this works will depend both on the subtlety of the software as well as the nature of the image produced by the optical hardware -- thus favouring optical systems of high resolution and low depth of field.

Problem areas will include the rendering of transparent and translucent objects, diffuse shapes with ill-defined edges, such as hair, and focused but textureless areas in the object space -- much the same problem areas as those faced by workers using blue screen techniques in the feature film industry and chromakey effects in video and broadcast TV.

The generation of quasi-stereoscopic image stacks from the MSC camera image data input is essentially the same technique as that established by the Laser Confocal Scanning Microscope people. Whether this can be achieved in real time in the MSC camera remains to be seen, and the possibility of storing the image data on multiple synchronized computer hard drives or multiple timecoded digital tape drives for later processing is discussed elsewhere in this document.

Other problems confronting MSC software developers are of a kind similar to those encountered in all software which attempts to emulate human perception. Other examples are OCR software, object recognition, handwriting recognition, and voice recognition software. Work continues in these fields as the rewards for success are large. I feel this will also prove to be the case with the MSC camera.