International Journal of Innovative Research in Computer and Communication Engineering
MenuISSN ONLINE(2320-9801) PRINT (2320-9798)
ISSN ONLINE(2320-9801) PRINT (2320-9798)
Ms. Lidiya Augustine1 Mrs. Nagalakshmi Venugopal2
|
| Related article at Pubmed, Scholar Google |
Visit for more related articles at International Journal of Innovative Research in Computer and Communication Engineering
In Image Processing, a variety of edge-aware image manipulation techniques are used. It has received much attention in the computer graphics community. The rationale behind mixed-domain image manipulation approach is to process or filter images without destroying or blurring the important features such as edges. This technique processes a Gaussian pyramid from coarse to fine, and at each level, it applies a non-linear filter bank to the neighbourhood of each and every pixel. The results of these spatially varying filters are combined by global optimization. It uses a precise mixed domain approach for the optimization problem. This technique can be applied to several areas such as image smoothing, detail and contrast manipulation, non-photorealistic rendering, HDR compression etc. This avoids all the unsightly artifacts on the images. Further development in the field of edge-aware image manipulation techniques will absolutely helps to resolve the complex image processing tasks to a much greater extend. Here it describes the leading improvement in edge-aware image manipulation techniques for advanced research.
Keywords |
| image processing, edge-aware image processing, multi-scale method, optimization based image processing |
INTRODUCTION |
| Image processing, is a method to convert an image into its digital form and perform some operations on it, in order to extract some useful information from it or to get an enhanced image. We can classify the purpose of image processing into five. They are |
| 1. To make an image better: Image sharpening and Image Restoration |
| 2. To measure the interesting objects in an image: Pattern Measurement |
| 3. To search for an image of interest: Image retrieval |
| 4. To distinguish or to recognize the objects in an image: Image identification or Image Classification |
| 5. To see Objects which are not seen: Image Visualization |
| Currently image processing is an active area of research in such diverse fields as astronomy, medicine, defense, microscopy, seismology, and the publication and entertainment industries. |
| The art of altering an image to express what we want, rather than what the original image may have displayed is called image manipulation. This is as an application of image editing techniques to photographs, in order to create a replica (in contrast to mere enhancement or correction), of the original photograph. This technique alters the content of the image in a devious manner. It becomes difficult for the viewers to differentiate between the manipulated image and the actual image. |
| Edge aware image manipulation techniques help us to process images carefully, without destroying or blurring the fine scale image details such as edges. The study on edge- aware image processing is important, because many of the presently used image manipulation techniques may introduce some unwanted artifacts in the original image, mainly at image edges; these are called halo effect, gradient reversal problem, aliasing etc. This technique can be effectively used, to reduce un-wanted artifacts that occur during image manipulation. It provides a simple interface for independently adjusting both the detail and overall appearance of input images [1]. This paper makes a comparative study on existing edge-aware image manipulation techniques and gives an idea about the problems that may appear during image manipulation. |
| Non-linear filter banks are more suitable for edge aware image processing rather than linear filter banks. Even though non-linear filters are difficult to design, it can preserve image edges more sharply than linear filters and it is very effective at removing impulsive noise. Whereas linear filters tend to blur edges and other details of the image and its performance is very poor with Gaussian noise. |
| In image manipulation variety of optimization methods are used. It is more suitable to use multi scale approach, because more accurate results can be achieved by comparing the results in each step with the previous one. Among various optimization approaches, the more recent technique is to use an explicit mixed-domain concept (where it uses both the real space and DCT transform space) [1]. |
ISSUES IN IMAGE MANIPULATION |
| Artifact means, any undesirable feature that may emerge during image manipulation, which is not present in the original image. The main problem in edge-aware image manipulation especially in case of algorithms which process both the overall and detail appearance is reducing or if possible avoiding artifacts at image edges. The following paragraphs describe about some of the artifacts, which are commonly occurred during image manipulation. |
| Halos are usually in the form of a ring of light that encircles something bright (e.g. manipulation of the full moon image). We are not able to distinguish the edges clearly, if the image posses halos. In signal processing ‘halo effect’ will occur when the result overshoot at any step. Consciously weak halos are introduced in un-sharp masking, because it can make image edges stand out, stronger halos are considered as unwanted artifacts. The best approach to reduce halos during image processing is to use the Gaussian pyramid concept .it involves generation of different resolution images. It processes images level-by-level, from coarse to fine to reduce halos, hence called multi-scale approach in image processing. |
| Gradient reversal is a challenging problem in image processing. Without the boundary values we can still recover the image but not at the same saturation and contrast as that of the original image. There will always be a constant term missing from the entire image. Likewise in signal processing , if averaging is done in such a way as to sharpen the input step, then the result can even be a ‘gradient reversal’ of the original image. |
| Another major problem in image manipulation is Aliasing. It refers to the effect, that causes different signals to become indistinguishable (or aliases of one another) when sampled. Here one signal acts as the alias of another. |
BACKGROUND |
| In digital image processing and computer graphics many different image manipulations techniques have been proposed over the years. But the main area which has received much attention than any other techniques is edge aware image manipulation. Here we briefly give an idea about the previous techniques which are most closely related to our work. |
A. EDGE-AWARE IMAGE PROCESSING |
| Anisotropic diffusion [2] technique, it uses a non-linear partial differential equation (PDE) which depends on the local variation of the input images. The scale space approach used here, involves generation of different resolution images, formed by, convolving the original image with a Gaussian kernel. It iteratively smooths an image without destroying or blurring important features of the input image. Here a parameter called, diffusion coefficient is chosen, which encourages intra-region smoothing in preference to inter-region smoothing. However these methods are based on iteration, anisotropic and other PDE based methods are relatively low, as it involves computation of these diffusion parameters. We can use anisotropic diffusion as the basis for high dynamic range (HDR) compression, but there is a probability that sometimes it may tends to over sharpen the image edges. |
| The Bilateral filtering (BLF) [3] method is a different kind of edge aware image smoothening approach. Here it uses a non iterative approach for image manipulation. It uses a collection of non- linear image values which are in close proximity. The grouping of color or gray level, image values are based on their photometric similarity and their close proximity. It uses explicit, data dependent filters for their processing. It is simple and local. The main problem associated with BLF is that it is harder to analyse than other domain filters, because of its non-linear nature. There involves a trade-off between data smoothing and edge preservation. Sometimes if too much smoothing is applied, mainly during the separation of mean surface from detail, it may lead to halos at image edges. Also, BLF method is limited in their ability to extract detail of the image at arbitrary scales. |
| Fattal’s [4] method based on weighted least square (WLS) approach, which uses more careful edge-aware image decomposition. Usually the weights are made small at sharp edges, to preserve them .In WLS method, it first computes the smooth component of the input image, by optimizing quadratic energy based on squared gradients which have spatially varying weights. This method provides an excellent foundation for multi scale HDR and LDR tone mapping, detail and contrast manipulation, and image abstraction. One of the major drawbacks of this method is that, it involves solution of large linear sparse systems. Later Fattal proposed a new approach to edge aware image processing, which focus on generating two data dependent edge avoiding wavelets [5]. In edge-aware image manipulation, these wavelets are more suitable than the traditional wavelets, for producing better results. Wavelet acts as the foundation for representing images in various degrees of resolutions. This approach is mainly used in compression technique where images are first converted to signals or waves of varying frequency and duration .Here the wavelets are generated based on the edge content of the image. It doesn't contain pixels from both sides of an edge. Hence the name edge avoiding wavelets. We can extend this technique to dynamic compression of images .but determining wavelets is a difficult task in this approach, and also the quality of the input image may become poor, because of the approximation used during parameter computations. |
| Another important approach to edge-aware image processing is to use pyramid based methods, using multi-scale approach such as Gaussian or Laplacian pyramid or a wavelet decomposition. In Paris's method [6] based on Laplacian pyramid, it uses a set of local filters for image decomposition. It is well suited for representing details of the images. We can use a standard Laplacian to characterize the edges of the images. This allows us to differentiate large scale edges from small scale image details by applying a simple threshold on pixel values in the images. It is computationally costly, because there is a need to construct sub-pyramids for each of the element in the Laplacian pyramid. |
| There is a recent technique proposed by Gastal [7] called domain transform. This approach is mainly used for realtime edge-aware image processing. Domain transform defines an isometry between curves on the 5D image manifold in 2D and real line. This also preserves the geodesic distance between points on these curves. It performs operations first by adaptively warping the input signal, so that 1-D edge preserving filtering can be effectively performed in linear time. Even though it is a fast filtering technique, it doesn't produce rotationally invariant results (i.e. filtering the rotated image and rotating the filtered image gives different results). |
| In mixed-domain edge-aware image processing [1] approach, it uses global optimization technique instead of traditional optimization methods. Image processing can be done by using Gaussian pyramid concept, it processes Gaussian pyramid from coarse to fine, at each level it applies a non-linear filter bank to the neighbourhood of each and every pixel. Here outputs are merged using an explicit mixed domain concept. This technique is more efficient, accurate and easy to implement. This is a best suitable approach for processing rectangular images. The limitation of this paper is that, currently we can support only two-scale manipulation (detail versus overall appearance) as we have to process the pyramid as whole, to reduce the halo effect. Also parameter setting needs more careful attention. Another major limitation of this approach is that, here optimization is not generalized so that we can't apply this approach to other spatially varying filters for further applications such as up-sampling and de-blurring . |
B. OPTIMIZATION-BASED IMAGE PROCESSING |
| Image processing, optimization of an image plays a key role in computer science graphics community. Regularization and relaxation in both narrow and broad senses are used in various fields and problems in image processing, and currently they are being combined with the general purpose optimization algorithms. We can use Optimization in image processing with a range of objective functions. Usually image optimization involves choosing a suitable graphic compression format, which reduces the image file size and thus reduces the time it takes to download and display it in a browser. |
| Fattal’s Gradient domain image processing [8] is used for high dynamic range image compression. We are also able to drastically enhance ordinary images by bringing out specific features in dark regions. Here we operate image gradients via optimization, by using Gaussian pyramid concept. This approach is used to manipulate the gradient field of luminance image by attenuating the value of the large gradients. As a result, a new low dynamic range image is obtained by solving a Poisson function on the new modified gradient field. It is best suitable for producing the results without halos. |
| In sparse optimization method [9], it searches for a simple, straightforward solution for an optimization problem rather than a more complex precise solution. We can use this technique in image de-noising also. In gradient domain optimization frame work [10] it provides a uniform quadratic optimization method called Gradient shop. It is simple and new way for graphic programmers to define the image and video filters instead of just writing a function that processes the input pixel colors of the image. This method also allows the programmer to process the input pixel gradients (i.e. differences between nearby pixels) this filter paradigm also allows the construction of dominant image processing applications with very few lines of code. The applications of this approach can be extended to saliency sharpening, pseudo re-lighting, de-blocking, and image colorization etc. Conversely it’s capability to produce various effects relies on carefully assigned spatially varying weights on image gradients which acts as the constraints. |
| Other technique such as total variation optimization [11] is used mainly for removing the noise content from the images. The use of more constraints in this method will yield more details of the solution in our de-noising method. In L0 Gradient Minimization approach [12], we propose a sparse gradient counting scheme in an optimization framework. The main donation of this paper is a new strategy to confine the discrete number of intensity variations among neigh boring pixels, which relates directly to the mathematical L0 norm for information sparsity detection. Both of these techniques uses iterative algorithms hence they are generally slow and may be deficient in mathematical guarantees of convergence and stability. |
| Recently multi-scale optimization approach is proposed. Unlike the other optimization methods, it permits a fast frequency domain solution to the resulting optimization problem. Here at each scale we optimize the quadratic energy function over sub-windows to merge the outputs of a set of local edge –aware filters [1].So that we are able to reduce halos efficiently. This method can be implemented easily. It is very fast and we can readily accelerate it on the GPU. Here the operation consists of very few pixels rather than complex solvers for large linear systems. |
COMPARITIVE STUDY |
| We can make a comparative study by taking mixed-domain edge aware image manipulation with some of the previous works such as BLF based method[3],WLS method[4] and LLF approach[6] as in Fig.1 [1]. |
| From the Fig.1, it is clear that BLF based methods can cause edge halos. The WLS method reduces halos to a large extent as compared to BLF, but there are some halos that are still visible. Mixed-domain edge-aware image manipulation produces better halo free results which are somewhat similar to results produced in LLF, because both of these methods uses very similar local filters. However, in mixed-domain concept, unlike the LLF method, it merges multiple outputs of those local filters via a novel optimization framework, with an efficient and exact mixed domain solution. |
 |
| Fig.1. HDR compression method (comparison with previous works). (a) Bilateral filtering technique [3]. (b) Weighted least square Method [4]. (c) Local Laplacian filter [6]. (d)Mixed-domain concept [1] |
APPLICATIONS |
| Applications of this technique come under HDR compression, where HDR images having largest ratio between the brightest and darkest part in the image. Haze removal [13], used for improving the clarity of the outdoor images, smoothing, mainly for removing the noise content in the images during image processing, image colorization and color contrast enhancement, image abstraction [14], [15] for retrieving particular features such as edges in the images etc. It is used for reducing aliasing in images also. Several of the applications of this technique are shown in Fig.2 and 3[1]. |
 |
 |
| We can widen the application of this method to a set of problem such as non-photorealistic rendering, color contrast enhancement, detail enhancement, detail smoothing etc. |
CONCLUSION |
| While several approaches to edge-aware image, manipulations, have been proposed, no single method can completely solves all of the problems in image processing, and it is still, a subject of an active study. Methods vary in its advantages and disadvantages and the selection of which technique to use depends on the requirements posed by the application at hand. The success of image manipulation depends on the clarity of the image after processing, as well as, the required storage space. A good edge-aware image manipulation approach should preserve edges well and it should be effective at removing all the artifacts which arise during image manipulation |
References |
|