Keywords
|
| Seam Carving; Visual Effects; Canny Edge Detector; Geometric constraints; Scaling; Cropping. |
INTRODUCTION
|
| With the rapid growth of display device diversity and versatility today, new demands are made of the digital media. Adaptive resizing of images is one of the most useful techniques in relevant areas. For example, images can be changed to different sizes or aspect ratios for displaying on devices with various screen resolutions. Designers can provide different previews for photos on a website. A feasible resizing algorithm should be able to preserve the important content in an image as well as the global visual effect. Seam carving, which can change the size of an image by gracefully carving out or inserting pixels at different locations, is an efficient technique for content-aware image resizing. A seam is constructed by searching for a connected path of pixels crossing the image from top to bottom, or left to right. Backward energy or forward energy is used to evaluate the importance of a pixel. The main drawback is the frequently occurred damage of local structure or global visual effect. The reason is due to the energy-based strategy of the algorithm. This algorithm always removes the seams containing or inserting low energy until the desired image size is achieved, without considering the real visual effect. Simple methods such as scaling and cropping also have clear drawbacks. Scaling the image in horizontal or vertical direction can be performed in real-time using interpolation and will preserve the global visual effects. However, scaling causes obvious distortion if the aspect ratio is different between the input and the output. The second approach is to crop the output to a window of the input image. This method will discard too much information of interest if the output resolution is significantly lower than the input resolution.If only cropping is used important image content in periphery is spoilt. If only scaling is used image looses its actual shape. If only seam carving is considered visual quality is distorted when image depicts objects of straight lines. Edge preserving techniques can be adopted only for landscape images. Conventional techniques don't give user control. So, the Seam Carving and traditional algorithm is combined here along with canny edge detector to preserve the straight lines and maintain the global visual effects of the image. |
RELATED WORK
|
| In SCCAIR [1] the size of an image an be changed by gracefully carving-out or inserting pixels in different parts of the image. Seam carving uses an energy function defining the importance of pixels. A seam is a connected path of low energy pixels crossing the image from top to bottom, or from left to right. By successively removing or inserting seams we can reduce, as well as enlarge, the size of an image in both direction. For image reduction, seam selection ensures that while preserving the image structure, we remove more of the low energy pixels and fewer of the high energy ones. For image enlarging, the order of seam insertion ensures a balance between the original image content and the artificially inserted pixels. These operators produce, in effect, a content-aware resizing of images. Furthermore, by storing the order of seam removal and insertion operations, and carefully interleaving seams in both vertical and horizontal directions we define multi-size images. Such images can continuously change their size in a content-aware manner. A designer can author a multi-size image once, and the client application, depending on the size needed, can resize the image in real time to fit the exact layout or the display. Seam carving can support several types of energy functions such as gradient magnitude, entropy, visual saliency, eye-gaze movement, and more. The removal or insertion processes are parameter free; however, to allow interactive control, this method also provide a scribble based user interface for adding weights to the energy of an image and guide the desired results. This tool can also be used for authoring multi-size images. But the issues in using this method is that artifacts might appear in resulting image, there are times when removing multiple seams from an image still creates noticeable visual artifacts in the resized image which counts in for the drawback of this approach. |
| In OSSIR [2], the goal is to design an image resizing scheme that minimizes noticeable distortion of prominent features and structural objects, such as people, vehicles or buildings. Recently, seam carving and image warping have been proposed to resize images non-homogeneously. Seam carving greedily removes or inserts 1D seam that pass through the less important regions in the image. Warping methods place a grid mesh onto the image and then compute a new geometry for this mesh, such that the boundaries fit the new desired image dimensions, and the quad faces covering important image regions remain intact at the expense of larger distortion to the other quads. Since humans are less sensitive to distortion of homogeneous information, such as clouds or sea, both classes of methods attempt to keep the prominent objects untouched and distort only the homogeneous regions. Unfortunately, keeping the prominent objects unchanged is certain to fail if their widths are larger than the target image width. In other words, the absence of homogeneous regions along the resizing direction would cause obvious distortion. Here a warping method that, instead of enforcing the size of salient image regions to remain unchanged is presented which determines an optimal scaling factor for each local region. The scaling factors are iteratively optimized, and the amount of deformation to each region is guided by a significance map that characterizes the visual attractiveness of each pixel; this significance map is computed automatically using a combination of gradient- and salience based measures. The strategy is called “optimized scale-and-stretch” since it allows regions with high importance to scale uniformly and regions with homogeneous content to be distorted. The technique is to warp the grid mesh that represents the image such that it fits the new image dimensions, and each quad’s deformation matches the local scaling factor. The scaling transformations and the positions of the grid vertices are both variables in the global optimization process. Efficiency stems from the specially-tailored objective function formulation that reduces the nonlinear problem to a series of linear problems with a fixed system matrix. The matrix can be prefactorized, and each iterative step only requires a backsubstitution. The key aspect is that the distortion due to image resizing is optimally distributed over the image, irrespective of the direction of the resizing operation (horizontal, vertical or both). This gives the full freedom to utilize homogeneous image regions to hide the distortion. Moreover, this method enjoys the advantage of respecting structures within the image but the drawback here is that the discrete nature of seam carving may damage structures because the information on the removed seam is lost and prominent lines straight may lead to an over-constrained system causing other regions to distort more |
PROPOSED METHODOLOGY
|
| In proposed system image resizing has been done by performing seam carving, scaling and Egde preservation methods. In the edge preservation the idea is to develop a mathematical form for the two criteria( Edge point localization and low error rate) which can be used to design detectors for arbitrary edges. Moreover, the first two criteria are not "tight" enough, and that it is necessary to add a third criterion to circumvent the possibility of multiple responses to a single edge. Using numerical optimization, optimal operators for ridge and roof edges are derived. Then the criteria for step edges and give a parametric closed form for the solution is specialised . In the process we will discover that there is an uncertainty principle relating detection and localization of noisy step edges, and that there is a direct trade-off between the two. One consequence of this relationship is that there is a single unique "shape" of impulse response for an optimal step edge detector, and that the trade-off between detection and localization can be varied by changing the spatial width of the detector. Several examples of the detector performance on real images will be given. These three methods can be integrated and processed. The traditional scaling and cropping algorithms is combined with the interesting features of seam carving. To preserve the straight lines in the depicted objects of the images canny edge detection algorithm is used. |
EXECUTION AND RESULTS
|
| Content-aware image resizing is to remove partial pixels with SC and homogeneously scale the others. |
| Given an original image I, our goal is to calculate a new image T with user-specified size, minimizing the following distance function |
 |
| Where, dIE represents a patch-based bidirectional distance between I and T,(sDCD ∈ [0, 1]) is the similarity of the two dominant color descriptors (DCD), (sE ∈ [0, 1]) is a special seam-energy based factor which is used in our algorithm to revise the distance of the resized image, ( ∈ [0, 1]) is a user specified coefficient. |
| In painting and texture synthesis can be done by using exemplar based image in painting algorithm. |
| Exemplar-based inpainting works well in case of regular textures, where the missing information can be re-filled by suitable patches from the known area. |
| Working Steps of Exemplar-based inpainting |
| 1. Pixels along the border of the inpainted domain are sorted according to priority, which is based on structure saliency and on confidence of already inpainted pixels. |
| 2. A block of pixels (due to later usage of rotation invariance, we call it a patch) around the first pixel in the list is called a target patch. |
| 3. A source patch of the same size as the target patch is searched in a neighbourhood of a pre-determined size. |
| 4. The best match based on the known pixels (or its part) is copied to the position of the target patch. |
| 5. The priorities are updated and the whole process is repeated. |
| After each seam is removed, we directly scale the current image to the target size and compute the distance to the original image. The resized image with the minimum distance is the final result. Combining seam carving with scaling can protect the global visual effect and some local structures of the original image, especially when the output resolution is much lower than the input one. |
| SC-Scaling-Combined optimized image resizing algorithm can be used for scaling. |
| The steps are as follows: |
| • Start the resizing process with SC process. After each seam removal operation, scale the image directly to the target size and calculate the image distance to the original image, using the distance function as follows |
| • Record the seam number (NSC V, NSC H) of the current step. |
|
| The resized image which has the minimum distance to the original image is the final result. At the same time, we also know the optimized seam number for obtaining best visual effect. |
| Finally, the edge information can be preserved while performing seam carving. This can be done by using Canny edge detector to identify significant edges. |
| Edges in the image are detected based on the Canny edge detector. As parameters of Canny, we use a Gaussian mask of size 3 for noise reduction, and Tup = 100 and Tlow = 20 as upper and lower thresholds for the hysteresis. |
| Edge pixels are transformed into Hough space IH next. Each point in Hough space corresponds to a straight line in the edge image. A threshold Though = 0.6 ? max{IH} is derived from the maximum value in Hough space. Only the most significant straight lines are selected by considering Hough pixels that exceed this threshold. For each line candidate, the number of edge pixels located on this line is is counted. An edge pixel is considered as line pixel, if the distance between edge pixel and line is below a threshold Tdist = 0.5 pixels, and if the line segment has a length of at least Tlength = 10 pixels. Small gaps between valid line segments are filled up (Tgap = 30). Because the precision of the detected lines is not sufficient, we use a gradient descent algorithm to optimize the parameters of a line by maximizing the total number of line pixels on each line. |
CONCLUSION
|
| A novel technique for content-aware image resizing is presented in the proposed algorithm. Seam carving and homogeneous scaling were integrated together to form a hybrid system. Then different methods such as seam carving, Scaling, cropping and edge preservation can be done for efficient image resizing. Edge preservation can be done by using canny edge detector method. This algorithm is based on the seam carving algorithm and additionally adds line detection and edge preservation .When a seam crosses a straight line, adjacent energy values are increased in order to prevent the following seams from crossing the line nearby. The distribution of the seams preserves straight lines much better and less distortion is included in the adapted image. Compared to the original seam carving, the proposed method achieves significantly better results when used on images with prominent straight lines or structures. This method can be extended to be applied for video, where ribbon carving method is applied instead of seam carving in future which is not taken into consideration here. |
Figures at a glance
|
 |
| Figure 1 |
|
| |
References
|
- Avidan, S., and Shamir, A., “Seam Carving for Content Aware Image Resizing”, ACM Transactions, Vol. 26,No. 3, pp. 1-10, 2007.
- Wang, Y.S., Tai, C.L., Sorkine, O., and Lee, T.Y, ”Optimized Scale-and-Stretch for Image Resizing”, ACM Transaction on Computer Graphics, Vol. 27, pp- 118, 2008.
- Canny, J. F., “A Computational Approach to Edge Detection,” IEEE Transactions on Pattern analysis and Machine Intelligence, Vol.8, No.6 ,pp. 679–698, 2006.
- Rubinstein, M., Shamir, A., and Avidan, S.,” Improved Seam Carving for Video Retargeting”, ACM Transaction on Computer Graphics, Vol.27, No.3,pp.16, 2008.
- Criminisi, P. P´erez and K. Toyama,” Region Filling and Object Removal by Exemplar-Based Image In painting”, IEEE Transactions on Image Processing, Vol. 13, No. 9, sep 2004.
|
Menu