ISSN ONLINE(2320-9801) PRINT (2320-9798)

All submissions of the EM system will be redirected to Online Manuscript Submission System. Authors are requested to submit articles directly to Online Manuscript Submission System of respective journal.

Survey on Content Based Image Retrieval Systems

Yogita Mistry 1, Dr.D.T. Ingole2
  1. Research Scholar, Dept. of Electronics, PRMITR, Badnera, India
  2. Principal and Professor, Dept. of Electronics, PRMITR, Badnera, India
Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Innovative Research in Computer and Communication Engineering

Abstract

As image collections are growing at a rapid rate, demand for efficient and effective tools for retrieval of query images from database is increased significantly. Among them, content-based image retrieval systems (CBIR) have become very popular for browsing, searching and retrieving images from a large database of digital images as it requires relatively less human intervention. This paper is an attempt to explore the CBIR techniques and their usage in various application domains.

Keywords

Content based image retrieval (CBIR), Region-based image retrieval, Similarity measure, Image retrieval, Color Histogram and Texture features

INTRODUCTION

There are many resources on the internet which people can use to create, process and store images. This has created the need for a means to manage and search these images. Therefore, finding efficient image retrieval mechanisms from large resources has become a wide area of interest to researchers [1]. Image retrieval method is a technique for searching and retrieving images from a large database of digital images. In today’s modern age, virtually all spheres of human life including commerce,hospitals, crime prevention, surveillance, engineering,architecture, journalism, fashion and graphic design, government, academics, and historical research use imagesfor efficient services. A large collection of these images is referred to as image database. Animage database is a system where image data are integrated and stored. Image datainclude the raw images and information extracted from images by automated or computerassisted image analysis.
The police maintain image database of criminals, crime scenes and stolen items. In the medical profession, mammographic images and scanned image database are kept for diagnosis,monitoring, and research purposes. In architectural and engineering design, imagedatabase exists for design projects, finished projects, and machine parts. In publishingand advertising, journalists create image databases for various events and activities suchas sports, buildings, personalities, national and international events, and product advertisements. In historical research, image databases are created for archives in areas that include arts, sociology, and medicine. In a small collection of images, simple browsing can identify an image. Image retrieval is the problem encountered when searching and retrieving images that are relevant to a user’s request from a database [2 - 4].
In 1979, a conference on Database Techniques for Pictorial applications was held in Florence. This was the beginning of attraction and attention of researchers in the field of image database management technologies. But still research area in this era was not so active. In February 1992, United Nations National Science Foundation (USNSF) organized a workshop in Redwood, California to highlight research areas for visual information management systems and its applications in various fields [2]. Since then many researchers started work in this area. Development of methods which would increase retrieval accuracy and reduce retrieval time is the main challenges in CBIR.
Early techniques were not generally based on visual features but on the textual annotation of images. The images were first annotated by text and then searched using text based approach. However in many situations, text annotation scheme is inefficient. For the huge image data the vast amount of labor required in manual annotation. Also describing every visual feature within the images is very time consuming and difficult. So instead of manual annotations by text based keywords, images are indexed by their own visual features such as colour, texture, shape etc [5].
In CBIR no additional information on images, such as text annotations, time or place of creation is available. The retrieval problem is solved only by analyzing content of the image based on the available characteristics of its pixels.
An alternative method of the content-based image retrieval is description based image retrieval (DBIR). In DBIR, retrieval is possible if all images of the collection have annotations describing their content. A general CBIR system makes use of different type of queries such as query by example image, sketch or region and provides relevant images from a given database, based not exclusively on textual annotation or media metadata, but on a similarity function using low-level features. More recent works attempt to combine content-based image retrieval with annotation-based text search. The idea is to support the submission of hybrid queries either by fusing the results of different retrieval modules) or by generating recommendations after processing the initial results of a query and exploiting the heterogeneous information. This way CBIR techniques can be utilised in several domains, either as standalone implementations supporting queries by example, or as complementary modules of an integrated framework that realises additional retrieval options, such as text and concept search, in order to improve and enhance the results. In this survey only content based image retrieval algorithms are discussed and reviewed.

CBIR SYSTEMS

A. Principle of CBIR

Content-based retrieval uses the contents of images to represent and access the images from the large database. Atypical content-based retrieval system is divided into two types: off-line feature extraction and online image retrieval. Fig.1. shows architecture for content-based image retrieval.In off-line stage, the system automatically extracts visual attributes (colour, shape and texture) of each image in the databasebased on its pixel values and stores them in a different database within the system called a feature vector database. The feature data (also known as image signature or image features for each of the visual attributes of each image is very much smaller in size compared to the image data,thus the feature database contains an compact form of the images in theimage database. Significant compression can be achieved using feature vector representation of image database over the original pixel values.
In on-line image retrieval, the user submit a query image to the CBIR system insearch of desired images. The system represents this query image with a feature vector. Thesimilarities between the feature vectors of the query example and those ofthe images in the feature database are then computed and ranked. Retrieval is computed by applying an indexing scheme to provide an efficient way of searching the imagedatabase. Finally, the system ranks the retrieval results and then returns the images that aremost similar to the query images.
image
The general architecture of CBIR system is shown in figure1 [6]. For the given image database, features are extracted first from individual images. The features can be visual features like colour, texture, shape, region or spatial features or some compressed domain features. The extracted features are described by feature vectors. These feature vectors are then stored to form image feature database. For a given query image, we similarly extract its features and form a feature vector. This feature vector is matched with the already stored vectors in image feature database. Sometimes dimensionality reduction techniques are employed to reduce the computations. The distance between the feature vector of the query image and those of the images in the database is then calculated. The distance of a query image with itself is zero if it is in database. The distances are then stored in increasing order and retrieval is performed with the help of indexing scheme.
Feature extraction techniques affect the retrieval rate of the CBIR system. In this survey paper, various popular algorithms for feature extraction are considered. A feature vector is a set of numeric parameters describing an image. The majority of such vectors represent oneimage feature, such as colour, texture, or shape of theobject. Feature vectors generated by the same algorithmform a space of feature vectors. Text annotations for image description areclassified as high-level features. Features, such as colour and texture, are called as low-level features. Shapes of objects in the image,which can be obtained by analyzing regions present in the image are classified as a low levelfeatures.
The important issues of content based image retrieval system, which are: 1. Selection of image database,
2. Similarity measurement, 3. Performance evaluation of the retrieval process and 4. Low-level image features extraction.
Evaluation of retrieval performance is a crucial problem in content-based image retrieval (CBIR). Many different methods for measuring the performance of a system have been created and used by researchers. The most common evaluation measures used in CBIR are precision and recall which are defined as,
image

B. Classificationof CBIR systems

Content-based retrieval methods can be classified into classes depending on the features theyuse such as colour, texture, and shape (refer Fig.2.). Each features class is further divided into subclasses by the type of the algorithm used for constructing the feature vector. Shape features are further divided as boundary based and region based feature extraction methods. In the literature, some researchers classify spatial features of imagesinto a separate class.
image

COLOUR FEATURES

Colour feature is the most significant one in searching collections of colour images of arbitrary subject matter. Colour plays very important role in the human visual perception mechanism. All methods for representing colour feature of an image can be classified into two groups: colour histograms and statistical methods of colour representation. The most frequently used colour spaces are as follows: RGB (red, green, and blue used in colour monitors and cameras), CMY (cyan, magenta and yellow), CMYK (cyan, magenta, yellow, and black used in colour printers), Lab (CIE L*a*b, lightness, a and b aretwo colour dimensions, from green to red and from blueto yellow) HSI, HSV (hue,saturation, and value).
The Lab space relies on the international standard of colour measurement developed by the International Commission on Illumination CIE (Commission International de Eclairage). The HSV space is similar to spaces HSI, HSL, and HSB. The HSV space is used more frequently because the RGB to HSV transformation is simpler from the computational standpoint compared to the RGB to Lab transformation.
The simplest and most frequently used way to represent colour is colour histograms. For each point of the considered colour space, the number of image pixels of a given colour is calculated. Such representation of information on colour is simple and natural; however, it has one considerable disadvantage: the distance between two images that have similar but not identical colours is large. In addition, such histograms are very sparse and, thus, sensitive tonoise.
Stricker and Orengo used cumulative colour histograms [7]. Such a representation of colour is less sensitive to noise and also reduces the number of the Type II errors if adjacent elements of histograms correspond to similar colours. Another approach to take into account the similarity of different colours is presented in [8]. In this work, various metrics based on the space of colour vectors (histograms) is proposed. The colour histogram itself does not store information on spatial layout of colours on the image. A solution to this problem was suggested in [9]. After constructing a colour histogram where only main colours of an image are taken into account, for every nonzero element of the histogram, the coordinates of the center of mass of the corresponding colour region is calculated. This information is used to measure the similarity between the images together with the number of pixels belonging to this colour region. This solution makes it possible, in a sense, to take into account spatial layout of colours, but it possesses one significant disadvantage. If the image contains several compound components of the same colour, this fact will not be reflected in the feature vector of the image. Instead, a common center of mass for all components will be calculated. A modification of this model was suggested by Stricker and co-authors in [10]: distributions of separate colour channels are considered as a part of a three dimensional distribution rather than as independent distributions. For the feature vector, average values foreach colour channel and covariance matrix of the channel distributions are used. To retrieve graphics and images simultaneously, this work applies an adaptive retrieval method [11]. The proposed method uses histograms of oriented gradient (HOG) as pixel-based features. However, the characteristics of graphics and images differ, and this affects feature extraction and retrieval accuracy. Thus, an adaptive method is proposed that selects different HOG-based features for retrieving graphics and images.In [12], a method to extract colour and texture features of an image quickly for content-based image retrieval (CBIR) is proposed. First, HSV colour space is quantified rationally. Colour histogram and texture features based on a co-occurrence matrix are extracted to formfeature vectors. Then the characteristics of the global colour histogram, local colour histogramand texture features are compared and analyzed for CBIR. Based on these works, a CBIR system is designed using colour and texture fused features by constructing weights of featurevectors.
In [13], features such as shapes and texture are extracted from the query and reference images and are compared by means of Euclidean distance. The morphological operation with spatially-variant structuring element is used for feature extraction. After the feature extraction process, the feature vectors are calculated by applying Block Truncation coding (BTC) over the feature extracted images. It improves the performance of image retrieval with reduced computational complexity for query execution. Based on HSV colour model, a method of object-based spatial-colour feature (OSCF) for colour image retrieval is proposed in [14]. Firstly, objects are extracted from colour, then image features are represented by objects in it. Colour and spatial-colour feature are adopted for description of objects. The new method only pays attention to main central objects. In [15], author proposed a novel fuzzy approach to classify the colour images based on their content, to pose a query in terms of natural language and fuse the queries based on neural networks for fast and efficient retrieval.
A new colour-based image retrieval method is proposed in the paper [16]. The quantization precision of this algorithm is higher than that of supervised method and its efficiency well than unsupervised way. First, through the distance-matrix of colour, the sample image is clustered in a self-organizing way, thus its palette can be obtained.
Based on the palette, other images in the database are mapped in terms of min- distance. In this way, a uniform histogram according to the same palette can be obtained for each image in the database. Besides, this algorithm also combines the main colour area to represent the spatial distribution of colour.
In [17], author discussed on the comparative method used in colour histogram based on two major methods used frequently in CBIR which are; normal colour histogram using GLCM, and colour histogram using KMeans. Using Euclidean distance, similarity between queried image and the candidate images are calculated. Experiment results shows that colour histogram with K-Means method had high accuracy and precise compared to GLCM.
In [18], a method is proposed for binary image retrieval, where the black-and-white image is represented by a novel feature named the adaptive hierarchical density histogram, which exploits the distribution of the image points on a twodimensional area.A new type of histogram which incorporates only the visual information surrounding theedges of the image is introduced in [19]. The edge extractionoperation is performed with the use of a center-surroundoperator of the Human Visual System. The proposedCenter-Surround Histogram (CSH) has two main advantagesover the classic histogram. First, it reduces theamount of visual information that needs to be processedand second, it incorporates a degree of spatial informationwhen used in content based image retrieval applications.
A robust image retrieval based on colour histogram of local feature regions(LFR) is presented in [20]. Firstly, the steady image feature points are extracted by using multi-scale Harris-Laplace detector. Then, the significant local feature regions are ascertained adaptivelyaccording to the feature scale theory. Finally, the colour histogram of local feature regions isconstructed, and the similarity between colour images is computed by using the colourhistogram of LFRs.A novel CBIR system is proposed in [21] named iSearchand global/local matching of local features are combined todo precise retrieval of item images in an interactive manner.Multiple local features are extracted including scale invariantfeature transform (SIFT), regional colour momentsand object contour fragments to sufficiently represent thevisual appearances of items; while global and localmatching of large-scale image dataset are allowed. To do this, an effective contour fragments encoding and indexing method is developed.

TEXTURE

Texture gives us information on structural arrangement of surfaces and objects on the image. Texture is not defined for a separate pixel; it depends on the distribution of intensity over the image. Texture possesses periodicity and scalability properties; it can be described by main directions, contrast, and sharpness. Texture analysis plays an important role in comparison of images supplementing the color feature. The most frequently used statistical features include,
general statistical parameters calculated from pixels’ intensity values,
parameters calculated based on the co-occurrence matrices,
texture histograms built upon the Tamura features.
One of the first methods for representing texture features of images was grey level co-occurrence matrices (GLCM) proposed by Haralick et al. [22]. Authors suggested 14 descriptors, including the angular second moment, contrast (variance, difference moment), correlation, and others. Each descriptor represents one texture property. Therefore, many works for example as described in [23], are devoted to selecting those statistical descriptors derived from the cooccurrence matrices that describe texture in the best way. In [24], firstly, transforming color space from RGB model to HSI model, and then extracting color histogram to form color feature vector. Secondly, extracting the texture feature by using gray co-occurrence matrix. Thirdly, applying Zernike moments to extract the shape features. Finally, combining the color, texture and shape features to form the fused feature vectors of entire image. Experiments on commonly used image datasets show that the proposed scheme achieves a very good performance in terms of the precision, recall compared with other methods.
A method is proposed [25] for efficient image retrieval that applies a weighted combination of color and texture to the wavelet transform, based on spatial-colour and second order statistics, respectively. The proposed descriptor is particularly useful for multi-resolution image search and retrieval.

Wavelet-Based Texture Description

In wavelet based texture description, a specific feature of this method is representation and analysis of signals in different scales, i.e., under different resolutions. The image is described by a hierarchical structure each level of which represents the original signal with a certain degree of detail.
Smith and Chang used statistical characteristics (average and variance) calculated for each subband as texture features [26]. They compared effectiveness of texture classification for the features constructed by means of the wavelet approach, homogeneous decomposition into subbands (without scaling, each subband contains a part of a signal of certain frequency), discrete cosine transform, and spatial decomposition. In [27], the mean and standard deviation of the distribution of the wavelet transform coefficients are used to construct the feature vector. In the case of transformation with N filters, the image is represented by a feature vector of dimension 2N.
In [28], authors computed a new texture feature by applying the generalized Gaussian density to the distribution of curvelet coefficients which is called curvelet GGD texture feature. The purpose was to investigate curvelet GGD texture feature and compare its retrieval performance with that of curvelet, wavelet and wavelet GGD texture features. Experimental results shown that both curvelet and curvelet GGD features perform significantly better than wavelet and wavelet GGD texture features. Among the two types of curvelet based features, curvelet feature shows better performance in CBIR than curvelet GGD texture feature. The work consists on minimizing low-level features describing an image by using a reduced descriptor that combines color and texture information which is wavelet transformation is explored in [29]. A method is proposed to describe the image by high frequency subbands of discrete wavelet transformation (DWT) related to weighted salient regions after a fuzzy segmentation step.
In [30], a simple image signature based on the standardized moments of the wavelet coefficient distributions is proposed. This signature can be computed for each possible wavelet filter fast. An image signature map is thus obtained which is used as an image characterization for Content-Based Image Retrieval (CBIR). The work presented a modified curvelet transform (MCT) and its combination with vocabulary tree (VT) for feature collection and retrieval of the images from database [31]. MCT has been implemented using the Gabor wavelet sub-bands. The proposed algorithm captures edge information in an image more accurately than Gabor transform (GT) and curvelet transform which uses à trous wavelet transform (ACT) for decomposition of an image.
In the proposed approach [32], a hybrid meta-heuristic swarm intelligence-based search technique, called mixed gravitational search algorithm (MGSA), is employed. Some feature extraction parameters (i.e. the parameters of a 6-tap parameterized orthogonal mother wavelet in texture features and quantization levels in color histogram) are optimized to reach a maximum precision of the CBIR systems. An extremely fast CBIR system which uses Multiple Support Vector Machines Ensemble is proposed in [33]. Authors used Daubechies wavelet transformation for extracting the feature vectors of images. In [34], a different wavelet basis is used to characterize each query image. A regression function, which is tuned to maximize the retrieval performance in the training data set, is used to estimate the best wavelet filter, i.e., in terms of expected retrieval performance, for each query image.
Tamura et al. [35] presented an approach to describing texture on the basis on human visual perception. They suggested six parameters coarseness, contrast, directionality,line-likeness,regularity, and roughness corresponding to the six texture properties that were recognized as visuallymeaningful in the course of psychological experiments. Howarth and Rüger [36] – [37] noticed that the parametersdescribing the first three properties coarseness, contrast and directionality are rather effectivein classifying and searching images by texture. The set of all such points for one image is referredto as the Tamura image. Since texture features proposed by Tamura et al. arevisually meaningful and natural and have demonstratedtheir effectiveness in a number of experiments.
Texture analysis by means of the Gabor filters is a special case of the wavelet approach. This is the most frequently used method in image retrieval by texture. In most of the CBIR systems based in Gabor wavelet [38] - [40], the mean and standard deviation of the distribution of the wavelet transform coefficients are used to construct the feature vector. In the case of transformation with N filters, the image is represented by a feature vector of dimension 2N. Another idea of CBIR system development is based on the expansion of the image in termsof a basis obtained by analyzing a training set ofimages. Example is the ICA filtersobtained by applying the independent component analysisto the training set. The way the ICA filters are constructed is similar to the training process of the human vision system. These ICA filters obtained by the independent component analysisare local edge filters and are similar to the Gaborfilters. Unlike the latter, the ICA filters are naturallyconstructed and reflect main texture properties ofimages that were used to obtain them. The construction of the ICA filters can be found in [41 - 44].
Three image features are proposed for image retrieval in [45]. The first and second image features are based on colour andtexture features, respectively called colour co-occurrence matrix (CCM) and difference between pixels ofscan pattern (DBPSP). The third image feature is based on colour distribution, called colour histogramfor K-mean (CHKM). In [46], first HSV colour space is quantified rationally.Colour histogram and texture features based on a cooccurrence matrix are extracted to formfeature vectors. Then the characteristics of the global colour histogram, local colour histogramand texture features are compared and analyzed for CBIR. Based on these works, a CBIRsystem is designed using colour and texture fused features by constructing weights of featurevectors.
In [47], a content-based image retrieval method is proposed based on anefficient integration of colour and texture features. As its colour features, pseudo-Zernikechromaticity distribution moments in opponent chromaticity space are used. As its texturefeatures, rotation-invariant and scale-invariant image descriptor in steerable pyramid domainare adopted, which offers an efficient and flexible approximation of early processing in thehuman visual system. The integration of colour and texture information provides a robustfeature set for colour image retrieval.
A new feature scheme calledenhanced Gabor wavelet correlogram (EGWC) is proposedfor image indexing and retrieval in [48]. EGWC uses Gaborwavelets to decompose the image into different scales andorientations. The Gabor wavelet coefficients are thenquantized using optimized quantization thresholds. In thenext step, the autocorrelogram of the quantized waveletcoefficients is computed in each wavelet scale and orientation.A novel approach is proposed which uses a well-known clustering algorithm k-means and a database indexing structure B± tree to facilitate retrieving relevant images in an efficient and effective way [49]. Cluster validity analysisindexes combined with majority voting are employed to verify the appropriate number of clusters. For extracting the featurevectors of images Daubechies wavelet transformation is used.
In [50], a rotation invariantcurvelet features for texture representation is proposed which significantly outperforms thewidely used Gabor texture features. A novel region paddingmethod is also proposed to apply curvelet transform to regionbased image retrieval.A method is proposed which is an extremely fast CBIR system which usesMultiple Support Vector Machines Ensemble [51]. Daubechies wavelet transformation forextracting the feature vectors of images. Content-based image retrieval (CBIR) method for diagnosis aid in medicalFields is presented in [52]. In the proposed system, images are indexed in a generic fashion, without extracting domain-specificfeatures: a signature is built for each image from its wavelet transform. These image signatures characterizethe distribution of wavelet coefficients in each subband of the decomposition. A distancemeasure is then defined to compare two image signatures and thus retrieve the most similar imagesin a database when a query image is submitted by a physician.

SHAPE FEATURES

Along with colour and texture characteristics, shape of objects (figures) is also often used for image comparison. Methods for representing and describing shapes can be divided into two groups: external methods, which represent the region in term of its external characteristics (its boundary), and internal ones, which represent the region in terms of its internal characteristics (the pixels comprising the region). Shape features are classified in to two types: boundary descriptors and region descriptors. Further they are classified as (a) Structural and (b) global. The global boundary descriptors include various signatures, Fourier descriptors and wavelet descriptors.

A. Boundary Descriptors

1) The chain code: It describes an object boundary as a sequence of line segments with a given orientation. To build a chain code, the image is superimposed with a grid, and the boundary points are approximated by the nearest grid nodes. The line segments connect the neighbouring nodes.
2) Signatures: Signature is a description of a boundary of a two-dimensional object by means of function of one variable, which is assumed to be easier to describe compared to the original two-dimensional boundary.
3) Fourier descriptors: The Fourier descriptors are one of the most popular methods of contour parameterization. The basic idea of this method consists in the application of the discrete Fourier transform to the signature and use of the Fourier coefficients obtained as parameters describing the contour.
In [21], a novel approach is proposed named iSearch and global/local matching of local features are combined to do precise retrieval of item images in an interactive manner.First authors extracted multiple local features including scale invariant feature transform (SIFT), regional color moments and object contour fragments to sufficiently represent the visual appearances of items; while global and local matching of large-scale image dataset are allowed. To improve the SIFT algorithm, a robust approach is proposed for image retrieval based on the integration of keypoints and edges information in [53]. The approach is robust to translation, rotation and partial occlusion of the object.
A novel method for content-based image retrieval based on interest points is proposed in [54]. Interest points are detected from the scale and rotation normalized image. Then the normalized image is divided into a series of sector sub-regions with different area according to the distribution of interest points. A new algorithm using directional local extrema patterns meant for content-based image retrieval application is proposed in [55]. The proposed method differs from the existing LBP in a manner that it extracts the directional edge information based on local extrema in 0◦, 45◦, 90◦, and 135◦ directions in an image.
A trous wavelet correlogram feature descriptor for image representation is used in [56]. By further extension in this descriptor, á trous gradient structure descriptor (AGSD) is proposed for content-based image retrieval. AGSD facilitates the feature calculation with thehelp of á trous wavelet’s orientation information in local manner. The local information of the image is extracted through microstructure descriptor (MSD); it finds the relations between neighbourhood pixels. In [57], a new feature scheme called enhanced Gabor wavelet correlogram (EGWC) is proposed for image indexing and retrieval. EGWC uses Gabor wavelets to decompose the image into different scales and orientations. The Gabor wavelet coefficients are then quantized using optimized quantization thresholds. In the next step, the autocorrelogram of the quantized wavelet coefficients is computed in each wavelet scale and orientation.

B. Region Descriptors

1) Grid based method: Sajjanhar and Lu [58] proposed an intuitively clear method for description of object shape, the so-called grid based method. The basic idea of the proposed method can be expressed using two steps: (1) a grid with cells of certain size is superimposed on the object, and (2) the cells of the grid are traversed from the right to the left and from top to bottom
2) Moments and their invariants: Moment invariants are currently the most popular and widely used region descriptors. The idea of using moments for the shape description was first put forward by Hu in 1962 [59]. Author considered geometrical moments of a function of two variables.
In [60], Luren and Fritz put forward a fast method for calculating moments for binary images based on the use of a discrete variant of Green’s theorem. In [61], alternative invariants for Geometrical moments are derived. In addition to geometrical moments (they are referred to as sometimes regular or general), other moments are also used. The generic Fourier descriptors (GFD) suggested by Zhang and Lu [62], like other moment-based descriptors, rely on the idea of expansion of a signal in terms of a certain basis.
A new edge based shape feature representation method with multitier solutionenhanced orthogonal polynomials model and morphological operations for effective image retrieval is presented in [63]. The Pseudo Zernike moment based global shape features, which are invariant to basicgeometric transformations, are extracted and are used for retrieving similar images with Canberra distance metric.CBIR system is presented using shape feature descriptorand the modified Zernike moments based on the Zernikemoments with minimum geometric error and numericalintegration error [64]. In [65], experimental analysis of pixel-based densedescriptors such as local binary pattern (LBP), local directionalpattern (LDP) and their variants are done. These descriptorsare used as local features along with ZMs global features inachieving higher and accurate retrieval rate in SBIR system.A Novel method is proposed for content-based image retrieval based on interest points [66]. Interest points aredetected from the scale and rotation normalized image. Then the normalized image is divided into a series of sectorsub-regions with different area according to the distribution of interest points.

CONCLUSION

This paper has surveyed the essential concepts of content-based image retrieval systems. This survey attempts to introduce the theory and practical applications of CBIR techniques. Use of the hybrid feature including color, texture and shape as feature vector of the regions to match images can give better results. Classification and content-based retrieval methods based on the features they use such as colour, texture, and shape are discussed along with their subclasses and algorithms used for constructing the feature vector.

References

  1. A. Smeulders, M. Worring, S. Santini, A. Gupta and R. Jain(2000), ‘Content-based image retrieval at the end of the early years’, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 1349–1380.
  2. F. Long, H. J. Zhang, D. D. Feng (2003) , ‘Fundamentals of content – based image retrieval’, in: D. D. Feng, W. C. Siu, H. J .Zhang(Eds.), Multimedia Information Retrieval and Management—Technological Fundamentals and Applications, Springer, pp.1–27.
  3. R. Datta, D. Joshi, J. Li, J. Z. Wang (2008), ‘Image retrieval: ideas, influences, and trends of the new age’,ACM Computing Surveys vol. 40 pp.1 – 60.
  4. R. Veltkamp, H. Burkhardt, H. -P. Kriegel (2008), ‘State-of-the-Art in Content-Based Image and Video Retrieval’, Kluwer Academic Publishers, New York.
  5. Manesh Kokare, B.N. Chatterji and P.K. Biswas (2002), ‘A survey on current content based imageretrieval methods,’ IETE journal of Research, Vol. 48, No.3&4, 261-271.
  6. Sanjay Patil, Sanjay Talbar (2012), ‘Content Based Image Retrieval Using Various Distance Metrics’, Data Engineering and Management, Lecture Notes in Computer Science, Vol. 6411, pp 154-161
  7. Stricker, M. and Orengo, M. (1995),‘Similarity of Color Images’, Proc. of the SPIE Conf, vol. 2420, pp. 381–392, 1995.
  8. Ioka, M. (1989),‘A Method of Defining the Similarity of Images on the Basis of Color Information’, Tech. Report RT-0030, IBM Tokyo Research Lab.
  9. Vassilieva, N. and Novikov, B. (2005)‘Construction of Correspondences between Low-level Characteristics and Semantics of Static Images’, Proc. of the 7th All-Russian Scientific Conf. ‘Electronic Libraries: Perspective Methods and Technologies, Electronic Collections’ RCDL’2005, Yaroslavl’, Russia.
  10. Stricker, M. and Dimai, A. (1997), ‘Spectral Covariance and Fuzzy Regions for Image Indexing’, Machine Vision Applications, vol. 10, pp. 66–73.
  11. Hong-Bo Zhang & Shang-An Li & Shu-Yuan Chen & Song Zhi Su & Der-Jyh Duh & Shao Zi Li.(2012)‘Adaptive photograph retrieval method Multimedia Tools & Applications’, DOI 10.1007/s11042-012-1233-7.
  12. Jun Yue, Zhenbo Li, Lu Liu and Zetian Fub (2011),‘Content-based image retrieval using color and texture fused features’, Mathematical and Computer Modelling, vol.54, pp. 1121–1127.
  13. Daisy, M.M.H., TamilSelvi, S. and Prinza, L. (2012)‘Gray Scale Morphological Operations for Image Retrieval’, 2012 International Conference on Computing, Electronics and Electrical Technologies [ICCEET], pp. 571-575.
  14. Chaobing Huang, Yarong Han, Yu Zhang (2012),‘A Method for Object-based Color Image Retrieval’, Fuzzy Systems and Knowledge Discovery (FSKD), 2012 9th International Conference on , pp:1659-1663.
  15. Fernando, R. and Kulkarni, S. (2012), ‘Hybrid Technique for Colour Image Classification and Efficient Retrieval based on Fuzzy Logic and Neural Networks’, Neural Networks (IJCNN), The 2012 International Joint Conference on, pp:1-6.
  16. Zhu Qiaoqiao, Huang Yuanyuan. (2012),‘A New Image Retrieval Method Based on Color Feature’, Intelligent System Design and Engineering Application (ISDEA), 2012 Second International Conference on , pp:56-59, 2012.
  17. Rasli, R.M.Muda, T.Z.T., Yusof, Y.,Bakar, J.A. (2012)‘Comparative Analysis of Content Based Image Retrieval Technique using Color Histogram. A Case Study of GLCM and K-Means Clustering’, Intelligent Systems, Modelling and Simulation (ISMS), Third International Conference on, pp: 283 – 286.
  18. Content-based binary image retrieval using the adaptive hierarchical density histogram Panagiotis Sidiropoulos , Stefanos Vrochidis, Ioannis Kompatsiaris, Pattern Recognition vol. 44, pp. 739–750
  19. Konstantinos Konstantinidis, Vasileios Vonikakis, Georgios Panitsidis and Ioannis Andreadis (2011),‘A Center-Surround Histogram for content-based image retrieval’, Pattern Anal Applic, vol. 14, pp. 251–260
  20. Xiang-Yang Wang & Jun-Feng Wu & Hong-Ying Yang(2010), ‘Robust image retrieval based on colour histogram of local feature regions’, Multimed Tools Appl vol. 49, pp. 323–345.
  21. Haojie Li, Xiaohui Wang, Jinhui Tang and Chunxia Zhao(2013), ‘Combining global and local matching of multiple features for precise item image retrieval’, Multimedia Systems vol. 19, pp. 37–49.
  22. Haralick, R.M., Shanmugam, K., and Dienstein, I.(1973), ‘Textural Features for Image Classification’, IEEE Trans.Systems, Man Cybernetics., vol. 3, no. 6, pp. 610– 621.
  23. Howarth, P. and Rüger, S. (2004), ‘Evaluation of Texture Features for Content-based Image Retrieval’, Proc. Of CIVR'04, pp. 326–334.
  24. Jiayin Kang and Wenjuan Zhang, ‘A Framework for Image Retrieval with Hybrid Features’, Control and Decision Conference (CCDC), 2012 24th Chinese , pp: 1326 – 1330, 2012.
  25. Yong-Hwan Lee, Sang-Burm Rhee, Bonam Kim, ‘Content-based Image Retrieval Using Wavelet Spatial-Color and Gabor Normalized Texture in Multi-resolution Database’, Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS), 2012 Sixth International Conference on , pp: 371 – 377, 2012.
  26. Smith, J.R. and Chang, S.-F. (1994), ‘Transform Features For Texture Classification and Discrimination in Large Image Databases’, Proc. of IEEE Int. Conf. on Image Processing (ICIP-94), Austin.
  27. Do, M.N. and Vetterli, M. (2000), ‘Texture Similarity Measurement Using Kullback–Leibler Distance on Wavelet Subbands’, Proc. of Int. Conf. on Image Processing, 2000, vol. 3, pp. 730–733.
  28. Sumana, I.J., Guojun Lu, Dengsheng Zhang (2012),‘Comparison of Curvelet and Wavelet Texture Features for Content Based Image Retrieval’, Multimedia and Expo (ICME), IEEE International Conference on, pp.290 – 295.
  29. Gallas, A., Barhoumi, W., Zagrouba, E. (2012),‘Image Retrieval Based on Wavelet Sub-bands and Fuzzy Weighted Regions’, Communications and Information Technology (ICCIT), 2012 International Conference on , pp: 33 – 37.
  30. Quellec, G., Lamard, M., Cochener, B., Roux, C., Cazuguel, G.(2012), ‘Comprehensive Wavelet-Based Image Characterization for Content Based Image Retrieval’, Content-Based Multimedia Indexing (CBMI), 2012 10th International Workshop on , pp:1-6.
  31. Anil Balaji Gonde, R.P. Maheshwari and Balasubramanian (2013),‘Modified curvelet transform with vocabulary tree for content based image retrieval’, Digital Signal Processing vol. 23, pp: 142–150.
  32. Esmat Rashedi, Hossein Nezamabadi-pour and Saeid Saryazdi (2013),‘A simultaneous feature adaptation and feature selection method for content-based image retrieval systems’, Knowledge-Based SystemsVolume 39, Pages 85–94.
  33. Ela Yildizer, Ali Metin Balci, Mohammad Hassan and Reda Alhajj (2012),‘Efficient content-based image retrieval using Multiple Support Vector Machines Ensemble’, Expert Systems with ApplicationsVolume 39, Issue 3, Pages 2385–2396.
  34. Quellec, G., Lamard, M., Cazuguel, G., Cochener, B. (2012), ‘Fast Wavelet-Based Image Characterization for Highly Adaptive Image Retrieval’, Ieee Transactions On Image Processing, Vol. 21, No. 4
  35. Tamura, H., Mori, S., and Yamawaki, T. (1978), ‘Textural Features Corresponding to Visual Perception’, IEEE Trans. Systems, Man Cybernetics, vol. 8, pp. 460–472.
  36. Howarth, P. and Rüger, S. (2005), ‘Robust Texture Features for Still Image Retrieval’, IEEE Proc. Vision, Image Signal Processing, vol. 152, no. 6, pp. 868–874
  37. Howarth, P. and Rüger, S. (2004), ‘Evaluation of Texture Features for Content-based Image Retrieval’, Proc. Of CIVR'04, pp. 326–334.
  38. Sebe, N. and Lew, M.S.(2000), ‘Wavelet Based Texture Classification, Proc. of Int. Conf. on Pattern Recognition’, vol. 3, pp. 959–962.
  39. Manjunath, B.S. and Ma, W.Y. (1996), ‘Texture Features for Browsing and Retrieval of Image Data’, IEEE Trans. Pattern Analysis Machine Intelligence, vol. 18, no. 8, pp. 837–842.
  40. Manjunath, B.S., Wu, P., Newsam, S., and Shin, H.D. (2000), ‘A Texture Descriptor for Browsing and Similarity Retrieval, Proc. Signal Processing Image Commun.’, nos. 1–2, pp. 33–43.
  41. Bell, A.J. and Sejnowsky, T.J. (1997), ‘The ‘Independent Components’ of Natural Scenes are Edge Filters, Vision Research’ , no. 37, pp. 3327–3338.
  42. Borgne, H., Guerin-Dugue, A., and Antoniadis, A.(2004), ‘Representation of Images for Classification with Independent Features, Pattern Recognition Letters’, vol. 25, pp. 141–154.
  43. Snitkowska, E. and Kasprzak, W. (2006), ‘Independent Component Analysis of Textures in Angiography Images, Computational Imaging Vision’, vol. 32, pp. 367– 372.
  44. Field, D.J. (1987), Relations Between the Statistics of Natural Images and the Response Properties of Cortical Cells, J. Optical Soc. America, vol. 12, no. 4, pp. 2370– 2393.
  45. Chuen-Horng Lin, Rong-Tai Chen, Yung-Kuan Chan (2009), ‘A smart content-based image retrieval system based on colour and texture feature’, Image and Vision Computing, vol. 27, 658–665.
  46. . Jun Yue, Zhenbo Li, Lu Liu and Zetian Fu(2011), ‘Content-based image retrieval using colour and texture fused features’, Mathematical and Computer Modelling, vol. 54, pp. 1121–1127
  47. Xiang-Yang Wang & Bei-Bei Zhang & Hong-Ying Yang, ‘Content-based image retrieval by integrating colour and texture features’, Multimed Tools Appl, DOI 10.1007/s11042-012-1055-7
  48. H. Abrishami Moghaddam and M. Nikzad Dehaji, ‘ Enhanced Gabor wavelet correlogram feature for image indexing and retrieval’, Pattern Anal Applic.
  49. Ela Yildizer, Ali Metin Balci, Tamer N. Jarada, Reda Alhajj, ‘Integrating wavelets with clustering and indexing for effective contentbased image retrieval’, Knowledge-Based Systems 31 (2012) 55–66
  50. Dengsheng Zhang · M. Monirul Islam · Guojun Lu and Ishrat Jahan Sumana, ‘ Rotation Invariant Curvelet Features for Region Based Image Retrieval’, Int J Comput Vis (2012) 98:187–201
  51. Ela Yildizer, Ali Metin Balci, Mohammad Hassan, Reda Alhajj , ‘ Efficient content-based image retrieval using Multiple Support Vector Machines Ensemble’, Expert Systems with Applications 39 (2012) 2385–239
  52. G. Quellec, M. Lamard, G. Cazuguel, B. Cochener, C. Roux, ‘Wavelet optimization for content-based image retrieval in medical databases’, Medical Image Analysis 14 (2010) 227–241
  53. Liang-Hua Chen, Yao-Ling Hung, and Li-Yun Wang (2012), ‘An Integrated Approach to Image Retrieval’, Telecommunications and Signal Processing (TSP), 2012 35th International Conference on, pp: 695 – 699, 2012.
  54. Meng Fanjie, Guo Baolong and Wu Xianxiang (2012), ‘Localized Image Retrieval Based on Interest Points’, 2012 International Workshop on Information and Electronics Engineering (IWIEE), pp. 3371 – 3375.
  55. Subrahmanyam Murala, R. P. Maheshwari and R. Balasubramanian (2012), ‘Directional local extrema patterns: a new descriptor for content based image retrieval’, Int J Multimed Info Retr (2012) 1:191–203.
  56. Megha Agarwal · R. P. Maheshwari (2012), ‘Á trous gradient structure descriptor for content based image retrieval’, nternational Journal of Multimedia Information Retrieval, Volume 1, Issue 2, pp: 129-138.
  57. H. Abrishami Moghaddam and M. Nikzad Dehaji (2013), ‘Enhanced Gabor wavelet correlogram feature for image indexing and retrieval’, Pattern Analysis and Applications, Vol. 16, issue 2, pp:163-177.
  58. Teague, M. (1980), ‘Image Analysis via the General Theory of Moments’, J. Optical Society America, vol. 70, no. 8, pp. 920–930.
  59. Hu, M. K. (1962), ‘Visual Pattern Recognition by Moment Invariants’, IEEE Trans. Information Theory, vol. 8, issue 2, pp. 179–187
  60. Luren, Y. and Fritz, A. (1994), ‘Fast Computation of Invariant Geometric Moments: A New Method Giving Correct Results’, Proc. of IEEE Int. Conf. on Image Processing.
  61. Hew, P., Geometric and Zernike Moments (1996), ‘Diary’, Department of Mathematics, The University of Western Australia, 1996. http://citeseer.ist.psu.edu/hew96 geometric.html
  62. Zhang, D.S. and Lu, G., ‘Generic Fourier Descriptor for Shape-based Image Retrieval (2002)’, Proc. of IEEE Int. Conf. on Multimedia and Expo (ICME2002), Lausanne, Switzerland, vol. 1, pp. 425–428
  63. R. Krishnamoorthy, S. Sathiya Devi 92013), ‘Image retrieval using edge based shape similarity with multiresolution enhanced orthogonal polynomials model', Digital Signal Processing vol. 23, 555–568
  64. Z. M. Ma, Gang Zhang and Li Yan(2011), ‘Shape feature descriptor using modified Zernike moments’, Pattern Anal Applic vol. 14, pp. 9–22
  65. Anjali Goyal, Ekta Walia, ‘Variants of dense descriptors and Zernike moments as features for accurate shape-based image retrieval’, SIViP DOI 10.1007/s11760-012-0353-x
  66. Meng Fanjie, Guo Baolong, Wu Xianxiang (2012), ‘Localized Image Retrieval Based on Interest Points’, Procedia Engineering, vol. 29 pp. 3371–3375