What do I mean by “perfect” image? I am referring to the mental picture that designers, engineers, and manufacturers have when developing state-of-the-art image sensor technology, which drives the further advancement of this technology and drives camera manufacturers to continue creating newer, better imaging modules. This may not make much sense right now, but keep on reading and you’ll begin to appreciate what .
Let’s start at the beginning. Before we even begin to ask why one might want a “better” camera, we must first ask what is a better camera. In the highly specialized research community dedicated to imaging sensors and technologies, the answer to that question is highly specific to the application of the imaging device in question (i.e., the camera). However, the general form of the answer would read something like the following: “A better camera is one that captures any given image perfectly, or at least comes closer than any of the existing cameras.” In this article, I will try to answer one question for all of us: what is the perfect “image”?
The Human Fascination with Vision and Vision Processing
For hundreds, even thousands, of years, intellectuals and normal folk, such as you and I, alike have been fascinated by human vision. How can our eyes see color? How do our eyes re-focus so easily? How can our brains perceive depth from our vision? Our fascination has led us to explore the intricacies of human vision and the supreme vision processing our brain performs in real-time. Particularly, the reproduction of our eye’s faculty of vision and our brain’s capacity to meaningfully process that vision a mainstay in research efforts for a long time.
We build cameras to imitate the functionality our eyes provide. We design computer algorithms to detect human faces for security applications, imitating our brain’s ability to detect faces. However, the technology we have now is not perfect – what sort of technology would be perfect? Clearly, the “perfect” image might simply be the image that perfectly imitates the image we see with our own eyes.
Quick Timeline of Image Processing
- In 1816, the French inventor Joseph Nicéphore Niépce built a photographic apparatus that successfully captured the earliest known photograph.
- In 1888, George Eastman, founder of the Eastman Kodak Company, pioneered the film camera with the release of his first camera, which he simply named the “Kodak”.
- In 1934, A. O. Gelgar built the world’s first single-reflex lens (SLR) camera, which he called “Sport” (translated from Russian).
- This hand-held camera gave the user the ability to see the image that the camera would then print on 35 mm film.
- In 1948, Edwin Land’s Polaroid Model 95 became the world’s first instant-capture camera.
- This hand-held camera gave the user the ability to capture pictures relatively instantly, making mobile photography a truly viable possibility.
- In 1975, the first electronic camera was invented by an engineer at Eastman Kodak, Steven Sasson.
- This hand-held camera could store images digitally, entirely phasing out the need for photographic film.
- The central piece of technology in this camera, the charge-coupled device (CCD) image sensor, digitalized the capture of images.
- In 1988, the Japanese photography company Fujifilm showcased the Fuji DS-1P camera, the world’s first portable digital camera.
- The camera recorded the images it captured onto a 2 MB SRAM memory card.
Nowadays, leaders in the imaging industry such as Nikon, Canon, Fujifilm, Sony, Tamron, and Zeiss are continuing to build cameras that have already become significantly more advanced than the 1988 Fuji DS-1P that Fujifilm developed. In fact, the industry has grown and advanced so far that the DS-1P model is already archaic and obsolete in the eyes of all prominent digital photographers. However, the question I am focusing on here is: what is that epitome of imaging are these companies working towards?
How Our Fascination in Human Vision Drives the Timeline
For now, let’s focus only on the progress of the state of imaging technology. The first photograph is successfully captured on 1816 by a French inventor. This piece of technology has thus taken a necessary and key step towards recreating our eye’s ability to capture images. However, the camera takes a very long time to capture this image and it provides no easy or reliable means to capture images continuously, as our eyes do. So, in 1888, an American entrepreneur invents the first camera that captures images on photographic film. This technology enables the recording of cinematic film and thus achieves the goal of continuously capturing of images, just as our eyes do. Fast forward to 1948 and an engineer at Polaroid builds the world’s first instant-capture camera. This new technology achieves the next goal of capturing images instantly, just as our eyes do. You can probably piece together how the remainder of the developments in the imaging industry have mirrored the human fascination in reproducing the visual capacities of our eyes and our brain’s ability to constantly process that visual information.
Relating the Human & Machined Visual Systems
Consider the following relationship between our brain’s facial recognition and the function of a facial recognition system built today. First, we catch sight of someone’s face. Physically, this only the visualization of a bunch of photons incident on our eyeballs. Next, our brain captures a mental “image” of the face of a loved one every time we see their face. A camera is designed to do the same thing. After this, our brain stores that image so that when we see their face the next time we will recognize them. The memory card in a camera is designed to do the same thing. Moreover, not only with our brain recognize them by their face but it will also recall the memories that we associate with that person. A computer with facial detection software works towards this same objective. From just the image of a person that we capture with our eyes, our brains develop a rich accompaniment of details and features to build a world around us. Machine learning techniques are incorporated into facial detection software, so that the computer will learn how to identify a human face better as it encounters more human faces. How fascinating is it that so much of what we know of the world around us is just our brain adapting and learning from the images our eyes provide it?
This fascination is what drives all past, current, and future R&D efforts in the imaging industry. The relationship I outlined above is mirrored in the progress we can already observed in the historical timeline of the imaging industry. Over time, technologies have been developed or machines have been invented that provide yet one more function of the human visual system.
So… What is the “Perfect” Image, Then?
The “perfect” image, if we consider it separately from the imaging system that created it, is one that reproduces exactly what our eyes see. In other words, the “perfect” image could be held up at any given time next to our natural human vision without any inherent differences, perceivable or not. However, this is quite the lofty goal, so it is safe to say that the imaging industry isn’t a risk of achieving its ultimate goal any time soon; recent research conducted at MIT has shown that our brain can process a single visual frame in around 13 milliseconds.
However, this question has not received the complete answer it deserves yet. If you are a careful reader, you probably came to the realization that the “perfect” image is much more than just the image. Very broadly, the image cannot be considered separately from the imaging system that created it. The “perfect” image is only as perfect as the visual system is in comparison to the complete human visual system. Our very human nature prevents us from considering the concept of an “image” without thinking as well about the colors that make up that image, the objects we see in that image, the memories that image holds, and so on. Thus, the perfect image is not only seen by the eyes, but understood by the brain, and intrinsic in everything that falls in between the moment we catch sight and the moment our brain produces something as a result of that sight.
In my next article, I will explore the actual engineering that goes into building the digital cameras of today; you know, those chunky digital cameras you see tourists wearing around their necks at all of the memorials and important sites in our hometown.
If you’re interested, feel free to check out the Serious Computer Vision Blog. You might find their article on how our brains model images and their important features very interesting.