COLORING OF BLACK AND WHITE IMAGES-A SURVEY

Abstract

This paper proposes a new color image coding scheme called “colorization image coding.” The scheme is based on the colorization technique which can colorize a monochrome image by giving a small number of color pixels. We develop algorithms useful for color image coding. First, the luminance component is separated from an input color image. Then, a small number of color seeds are selected as chrominance information. The luminance image component is coded by a lossy coding technique and the chrominance image component is stored as color seeds. The decoding is performed by the colorization algorithm. It is shown that this colorization technique is effective to image coding, especially for high compression rate, through the experiments using different types of images.we develop a new strategy that attempts to account for the higher-level context of each pixel. The colorizations generated by our approach exhibit a much higher degree of spatial consistency, compared to previous automatic color transfer methods. We also demonstrate that our method requires considerably less manual effort than previous user-assisted colorizationmethods Given a grayscale image to colorize, we first determine for each pixel which example segment it should learn itscolor from. This is done automatically using a robust supervised classification scheme that analyzes the low-level feature space defined by small neighborhoods of pixels in the example image. Next, each pixel is assigned a colorfrom the appropriate region using a neighborhood matching metric, combined with spatial filtering for improved spatial coherence. Each color assignment is associated with a confidence value, and pixels with a sufficiently high confidence level are provided as “micro-scribbles” to the optimization-based colorization algorithm of Levin et al., which produces the final complete colorization of the image.

INTRODUCTION

Colorization, the process of adding color to monochrome images and video[12], has long been recognized as highly laborious and tedious. Despite several recent important advances in the automation of the process, a considerable amount of manual effort is still required in many cases in order to achieve satisfactory results.For example, Levin et al. recently proposed a simple yet effective user-guided colorization method. In this method the user is required to scribble the desired colors in the interiors of the various regions. These constraints are formulated as a least-squares optimization problem that automatically propagates the scribbled colors to produce a completely colorized image. Other algorithms based on color scribbles have subsequently been proposed While this approach has produced some impressive colorizations from a small amount of user input, sufficiently complex images may still require dozens, or more, carefully placed scribbles, as demonstrated in figure 1(a).

In addition to the manual effort involved in placing the scribbles, the pallet of colors must also be chosen carefully in order to achieve a convincing result[1], requiring both experience and a good sense of aesthetics. This difficulty may be alleviated by choosing the colors from a similar reference color image. In fact, Welsh et al. proposed an automatic colorization technique that colorizes an image by matching small pixel neighborhoods in the image to those in the reference image, and transferring colors accordingly.This approach is a special case of the more general image analogies framework where a general filter is learned from the relationship between two images Aand A0 and then applied to an input image B to produce a filtered result B0. However, image analogies and its derivatives typically make local (pixel level) decisions and thus do not explicitly enforce a contiguous assignment of colors. The Levin et al[2]. method, on the other hand, promotes contiguity by formulating and solving a global optimization problem.In this paper, we introduce a new color transfer method,which leverages the advantages of these two previous colorization approaches, while largely avoiding their shortcomings.Similarly to the method of Welsh et al[2]., our method colorizes one or more grayscale images, based on a userprovided reference — a partially segmented example color image. This requires considerably less input from the user than scribbling-based interfaces, and the user is relieved from the task of selecting appropriate colors (beyond sup plying the reference image). On the other hand, our method explicitly enforces spatial consistency, producing more robust colorizations thanWelsh et al[2]., by using a spatial voting scheme followed by a final global optimization step.

These advantages of our approach are demonstrated in figures 1.Our approach[1] is motivated by the observation that finding a good match between a pixel and its neighborhood in a grayscale image and a pixel in the reference image is not sufficient for a successful colorization. Often, pixels with the same luminance value and similar neighborhood statistics may appear in different regions of the reference image[8],which may have different semantics and different colors. For example, figure 1(c) shows the result of applying a simple nearest-neighbor matching based on the average luminance and the standard deviation in small pixels neighborhoods,and transferring the corresponding chromatic channels. In order to improve the results in such cases, Welsh et al. propose letting the user select pairs of corresponding swatches between the example and each input image, thus limiting the search for matching neighborhoods to particular regions.However, this user- assisted variant still relies on pixelwise decisions and does not enforce contiguity.

METHOD

In this section, we describe the algorithm for transferring color. The general procedure for color transfer requires a few simple steps. First RGB source image is converted into the YUV color space. This color space has been chosen because it promptly provides the luminance value (channel Y) which is a crucial datum for our procedure. It also grants a more faithful modeling of human perception. Next the Antipole tree is constructed, each vector contains the information necessary to perform the Antipole search and the UV components of the pixel color After the data structure has been completed, in scan-line order, for each pixel in the gray-scaled image we construct its vector and perform the Antipole search to select the best matching vector in the Antipole tree.

The UV components of the best matching vector are then transferred to the gray-scaled image to form the final image, while the Y component (luminance)of the pixel in the gray-scaled image is retained to its original value. Although this procedure is very simple and direct the experimental results show that it works very well on a large set of images.Even if at this stage of research we focused on homogeneous images it is likely to imagine that the algorithm will also work well on nonhomogeneous[3] (segmented) images Both color (source) and greyscale (target) RGB images are converted to the decorrelated lαβ space [Ruderman et al. 1998] for subsequent analysis. lαβ space was developed to minimize correlation between the three coordinate axes of the color space.

The color space provides three decorrelated, principal channels corresponding to an achromatic luminance channel (l) [12]and two chromatic channels α and β, which roughly correspond to yellow-blue and red-green opponent channels. Thus, changes made in one color channel should minimally affect values in the other channels. The reason the lαβ color space is selected in the current procedure is because it provides a decorrelated achromatic channel for color images. This allows us to selectively transfer the chromatic α and β channels from the color image to the greyscale image without cross-channel artifacts. The transformation procedure follows directly from Reinhard et al. [2001].In order to transfer chromaticity values from the source to the target, each pixel in the greyscale image must be matched to a pixel in the color image. The comparison is based on the luminance value and neighborhood statistics of that pixel.

The luminance value is determined by the l channel in αβ space. In order to account for global differences in luminance between the two images we perform luminance remapping [Hertzmann et al. 2001]to linearly shift and scale the luminance histogram of the source image to fit the histogram of the target image. This helps create a better correspondence in the luminance range between the two images but does not alter the luminance values of the target image.The neighborhood statistics are precomputed over the image and consist of the standard deviation of the luminance values of the pixel neighborhood. We have found that a neighborhood size of 5x5 pixels works well for most images. For some problematic images we use a larger neighborhood size.Since most of the visually significant variation between pixel values is attributed to luminance differences[11], we can limit the number of samples we use as source color pixels and still obtain a significant range of color variation in the image.

This allows us to reduce the number of comparisons made for each pixel in the grayscale image and decrease computation time. We have found that approximately 200 samples taken on a randomly jittered grid is sufficient. Then for each pixel in the greyscale image in scan-line order the best matching color sample is selected based on the weighted average of luminance (50%) and standard deviation (50%)[9]. We have also included the neighborhood mean and varied the ratio of these weights but have not found significant differences in the results. Once the best matching pixel is found, the α and β chromaticity values are transferred to the target pixel while the original luminance value is retained[12].This automatic, global matching procedure works reasonably well on images when corresponding color regions between the two images also correspond in luminance values. however, regions in the target image which do not have a close luminance value to an appropriate structure in the source image will not appear correct.Figure 2c shows the results of transferring color using the global image matching procedure. Here, the sky and trees match reasonably well between the images, but the road in the target does not match to the road in the source.

CONCLUSIONS

In this paper we have studied a new, general, fast, and user friendly approach to the problem of colorizing greyscale images.While standard methods accomplish this task by assigning pixel colors via a global color palette, our technique empowers the user to first select a suitable color image and then transfer the color mood of this image to the greylevel image at hand. We have intentionally kept the basic technique simple and general by not requiring registration between the images or incorporating spatial information. Our technique can be made applicable to a larger class of images by adding a small amount of user guidance. In this mode, the user first transfers the desired color moods from a set of specified swatch regions in the color image to a set of corresponding swatch regions in the greyscale image. Then, in the second and final stage of the colorization process, the colorized swatches are employed, using a texture synthesis-like method, to colorize the remaining pixels in the greyscale image. Currently, the L2 distance is used to measure texture similarity within the image. In the future we believe the technique can be substantially improved by using a more sophisticated measure of texture similarity.Our technique of employing an example color image to colorize a greylevel image is particularly attractive in light of the growing sophistication of internet image search engines and the emergence of centralized and indexable image collections which can be used to easily locate suitable color images. Finally, one could also utilize a database of basis texture swatches for the initial color transfer in the user-guided stage of the colorization process.

References

R. Irony, D. Cohen-Or, and D. Lischinski:Colorization By Example Eurographics Symposium on Rendering (2005) WELSH T., ASHIKHMIN M., MUELLER K.: Transferring color to greyscale images. ACM Transactions on Graphics 21, 3 (July 2002), 277–280. BURNS G.: Colorization. Museum of Broadcast Communications: Encyclopedia of television,http://www.museum.tv/archives/etv/index.html. CHEN T., WANG Y., SCHILLINGS V., MEINEL C.: Grayscale image matting and colorization. Proc. ACCV (2004), 1164–1169. DUDA R. O., HART P. E., STORK D. G.: Pattern Classification,2nd ed. Wiley Interscience, New York, 2000. HERMES L., BUHMANN J.: Semi-supervised image segmentation by parametric distributional clustering. LNCS Energy Minimization Methods in Computer Vision and Pattern Recognition(2003), 229–245. HERTZMANN A., JACOBS C. E., OLIVER N., CURLESS B., SALESIN D. H.: Image analogies. In Proc. SIGGRAPH2001 (Aug. 2001), Computer Graphics Proceedings, Annual Conference Series, pp. 327–340. EFROS, A. A. AND FREEMAN, W. T., 2001. Image Quilting for Texture Synthesis and Transfer, In proceedings of ACM SIGGRAPH 2001, 341-346. EFROS, A. A. AND LEUNG, T. K., 1999. Texture Synthesis by Non-parametric sampling..7th IEEE International Conference on Computer Vision, 1999, 1033-1038. GONZALEZ, R. C. AND WINTZ, P., 1987. Digital Image Processing, Addison-Wesley Publishing, Reading MA.IOANNIS, P., 1991. Digital Image Processing Algorithms, Prentis Hall, 58.PRATT, W. K. 1991. Digital Image Processing, John Wiley & Sons, 311. REINHARD, E. ASHIKHMIN, M., GOOCH B. AND SHIRLEY, P., 2001. Color Transfer between Images, IEEE Computer Graphics and Applications, September/October 2001, 34-40.