Apple is working on ways to further improve photo taking on its iOS devices, per a patent (number 8274583) at the U.S. Patent & Trademark Office. It's for radially-based chroma noise reduction for cameras.
Per the patent, a system, apparatus, computer readable medium, and method for radially-dependent noise reduction in image capturing devices involving an edge-preserving blur window are disclosed. In one embodiment, the edge-preserving blur includes only those pixels in the blur window that are within a threshold value of the blur window's current center pixel in its blurring calculation.
By creating a threshold function that varies radially from the center of the image sensor's light intensity falloff function, a more appropriate threshold value can be chosen for each pixel, allowing for more noise farther from the center of the image, and allowing for less noise closer to the center of the image. Light-product information taken from the image's metadata may be used to scale the threshold value parameters dynamically. This allows the method to perform the appropriate amount of processing depending on the lighting situation of the image that is currently being processed.
Here's Apple's background on the invention: "Today, many personal electronic devices come equipped with digital cameras. Often, these devices perform many functions, and, as a consequence, the digital image sensors included in these devices must often be smaller than sensors in conventional cameras. The digital image sensor, such as a charge-coupled device (CCD), of a digital camera has a plurality of photo-sites arranged in a colored filtered array or pattern, such as a RGB Bayer pattern. In the RGB Bayer pattern, each photo-site is filtered so that it is receptive to either: red, green, blue, or some variation thereof.
"The type of colored filter array and digital imaging sensor varies, typically based on the manufacturer of the digital camera. For example, some color filtered arrays use a pattern of yellow, cyan, green, and magenta. Typically, the digital camera has an image pipeline that performs a demosaicing or de-Bayering process on the image, lens correction, and noise reduction. The image pipeline then performs RGB gamma correction and tone mapping on the image data and encodes the image into the YCbCr family of color spaces or other format suitable for displaying and viewing.
"Various considerations must be addressed when processing a digital image obtained with a digital camera, digital video camera, or other imaging device. One consideration involves the large amount of image noise resultant from the use of small camera sensors, due to their typically smaller image sensor sites. Increased noise in pixels is typically caused by the random arrival times of visible light photons to the sensor photosites, but may also be caused by the process of reading the pixel values from the sensor photosites, or for any number of other reasons, and is usually made worse by low light conditions. Although noise can lead to a grainy appearance in images due to the pattern of the color filter array, increased noise also leads to increased false colors in images.
Due to a phenomenon known as 'lens falloff,' the amount of light reaching off-center positions of a sensor is often less than the amount of light reaching the more central positions of the sensor. This imaging phenomenon causes the image intensity to decrease radially toward the edges of an image, creating an effect known as 'vignetting.' The amount of vignetting present in an image depends upon the geometry of the lens and will vary with various lens properties such as focal length and f-stop.
"The vignetting effect is more apparent in lenses of lower f-stop (e.g., larger aperture), which are often used in consumer cameras and digital video recorders. Another consequence of the lens falloff effect is that there is often more noise found in pixels farther radially from the center of the sensor or, more precisely, farther radially from the center of the sensor's light intensity falloff function, which may or may not be at the geometric center of the sensor.
"To correct for this lens falloff phenomenon, a technique known as 'flat fielding' can be performed which compensates for the different gains and dark current levels over the surface of the detector. Once a detector has been appropriately flat-fielded, a uniform signal will create a uniform output. Due to the lens falloff effect, flat fielding often disproportionately boosts the intensities of pixels near the outer edges of the sensor so that they are in line with the intensities measured near the center of the sensor when exposed to the same uniform signal. As a consequence, the greater noise levels found in the pixels located closer to the outer edges of the sensor are also magnified to a greater degree than the noise levels in pixels near the center of the sensor.
"To compensate for reduced performance in smaller camera sensors, various processing techniques may be implemented. However, most existing noise reduction techniques either produce a blotchy appearance in the images or are too computationally expensive to be used. Thus, there is need for a low computational cost, radially-based system and method for reducing noise effects in image capturing applications to create more visually appealing images."
The inventors are Mark Zimmer, Ralph Brunner and David Hayworth.
