A Review on Re-ranking Techniques

Reeba B; Bindhu J S

A Review on Re-ranking Techniques

Reeba B¹, Bindhu J S²

PG scholar, Department of Computer Science, College of Engineering, Perumon, Kerala, India
Associate Professor, Department of Computer Science, College of Engineering, Perumon, Kerala, India

Related article at Pubmed, Scholar Google

Visit for more related articles at International Journal of Innovative Research in Computer and Communication Engineering

Abstract

To improve retrieval precision, no other technique is as useful the re-ranking technique. The technique leaves no ambiguities as we consider only the variant characteristics or modalities, in order to gain the image and video retrieval. An innovative re-ranking algorithm circular re-ranking is undertaken here. This re-ranking process aids the mutual transfer of information across multiple modalities to improvise search performance to facilitate authentic searching. The pattern is simple the strong performing modality advances from weaker modalities and vice versa. Technically speaking, the process f circular re-ranking employs multiple circular runs of random walks by mutually transferring the ranking scores among various prospects. In contrast with the prevailing techniques, the method reassures the interaction between numerous modalities for the wide acceptance that is beneficial in re-ranking. This paper discusses peculiar circular re-ranking properties, including the methodology of information propagation, about how and which the configuration should be made, leaving no errors.

Keywords

Visual search, circular re-ranking, multimodality fusion

INTRODUCTION

The random advancement of web 2.0 technologies paves the way for the obvious development of research activities relating to visual search. Till this day, the commercial visual search engines generally do retrieved by key word matching. At the mean time the visual documents exploit audio-visual content and user applied text. by making use of vivid set of features the re-ranking of the visual documents of a search engine is made practical, leading to the perfection of search performance. here the prime goal is to attain agreement from the respective modalities in order to reorder the documents, thereby improving the precision in retrieving. Two general approaches are categorised visual pattern mining [6] and multi modality fusion [1],[2].The former approach mines the recurrent patterns, disregarding the degree of complexity, i.e. either explicitly or implicitly from the foremost search result and then progress with the ranks of visually congruent documents random walk[7] does self re-ranking through sorting out and finding the documents with similar patterns, rooting on the inter image similarity and antecedent rank scores. this approach never explores the collaborative utilisation of multiple modalities. Rather uneven modalities are subjected as independent modalities. Apart from that, the utilization of a modality is normally rooted on the defined application, thereby making the generalization of mining process a hard nut to crack. The latter method multimodality fusion recognizes the significance of modalities through fusion weight learning, enabling the reordering of documents through linear combination. Doing the fusion at decision stage. The gaining of the fusion weights estimation is chiefly from the ranking scores in alternative ranked list.

The discussion here is based on circular re-ranking, an algorithm that makes use of both pattern mining and multi modality fusion for visual search. The notable thing is that modality interaction should be observed on multiple points of view implicit mining of recurrent patterns and making use of various modalities for enhancing search performance.

CIRCULAR RE-RANKING

Circular re-ranking eases the interaction among various modalities by mutual reinforcement. This enables the reinforcement of the strong modality through communication with weaker modalities, whereas the weak modality is also benefited from strong modalities. The inter transfer of information side to side with multiple modalities ensues the philosophy of strong performing modality learning from weaker modalities, and vice-versa. It is a clear cut fact that circular re-ranking administers frequent randomized cyclic runs by taking turns on the ranking scores alongside different aspects. In contrast to the existing techniques, the re-ranking method fosters the interaction between modalities to find a tranquillity which is helpful for re-ranking.

Recently, multimodal fusion acquired the attention of a lot of researchers as a result of the benefit provided for various multimedia analysis processes. Multimodal fusion is the blending of multiple media and their associated features, or the intermediate decisions to conduct an analysis task. Modality interaction mines recurring patterns and leverages the modalities for enhancing search performance. A multimedia analysis task includes the undertaking of multimodal data in order to get important insights relating about the data, a situation or a higher level activity. All these media inclusions are fused together for the consummation of numerous analysis tasks. The fusion of multiple modalities enables complementary information, enhancing the accuracy of the overall decision making technique. A citing is the fusion of audio-visual features in addition with other textual information that leads to the effective event detection of a sports team video, which when processed through a single medium is impossible.

Visual search is a sort of intuitive task that needs attention, involving a diligent scan of the visual dimensions for a specific object or target among a lot of objects. This process can be made true by eye movements or devoid of eye movements. The capability of locating the target among a compounded pattern of stimuli has been broadly studied over four decades. A practical citing is picking out an out of order production a supermarket console or trying to search our family in a huge mob. Here is a listing of some of the visual search advancement among human’s eye movements of human and non humans are highly standardized and here; there is a visual elicitation of complex natural scenes.

RELATED WORKS

We briefly divide the related works for visual search re-ranking into two groups: recurrent pattern mining and multimodality fusion. The former assumes the existence of common patterns among relevant documents for re-ranking. The later predicts or learns the contribution of a modality in search re-ranking.

A .Recurrent Pattern Mining

Recurrent pattern mining research has carried on along three different dimensions: self re-ranking [4], [7], [5], crowd re-ranking by exploiting online crowd sourcing knowledge [10], and example-based re-ranking by leveraging userprovided queries [11], [20]. Fergue s et al,[4], employed probabilistic Latent Semantic Analysis (pLSA) for mining visual categories through clustering of images in the initial ranked list and which extends pLSA (as applied to visual words) to include spatial information in a translation and scale invariant manner Candidate images are then re-ranked based on the distance to the mined categories. Self re-ranking seeks consensus from the initial ranked list as visual patterns for re-ranking. Hsu et al, [5], employed information bottleneck (IB) re-ranking to find the clustering of images that preserves the maximal mutual information between the search relevance and visual features. The IB re-ranking method, based on a rigorous Information Bottleneck (IB) principle which finds the optimal image clustering that preserves the maximal mutual information between the search relevance and the high-dimensional low-level visual features of the images in the text search results. Among all the possible clustering’s of the objects into a fixed number of clusters, the optimal clustering is the one that minimizes the loss of mutual information (MI) between the features and the auxiliary labels. Richter et al,[12], employed an crowd re-ranking is similar to self re-ranking except that consensus is sought simultaneously from multiple ranked lists obtained from Internet resources and further formulated the problem as random walk over a context graph built through linearly fusing multi-modalities for visual search. We proposed to use a multimodal similarity measure to find nearest neighbours of images. The nearest neighbour search of an image is then limited to such a subspace, i.e. to a subset of images in our database. This way we reduce the number of image comparisons required for the graph construction to a linear amount depending on the cluster sizes.

Liu et al. [10], suggested a re-ranking paradigm by issuing query to multiple online search engines. Based on visual word representation, both concurrent and salient patterns are respectively mined to initialize a graph model for randomized walks based on re-ranking. Different from self- and crowd-re-ranking, example-based re-ranking relies on a few query examples provided by users for model learning. Yan et al, [20], employed an classifiers are learnt by treating query examples as positive training samples while randomly picking pseudo-negative samples from the bottom of initial ranked list. The classifiers which capture the visual distribution of positive and negative samples are then exploited for re-ranking. Liu et al. [11], proposed a query examples are utilized to identify relevant and irrelevant visual concepts, which are in turn employed to discover the rank relationship between any two documents using mutual information for correcting ranking of document pairs.

B .Multimodality Fusion

Multi-modality fusion based on weighted linear fusion is widely adopted. Broadly, we can categorize the existing research into adaptive [15], and query-class-dependent fusion [9].

Wilkins et al, [18], proposed a multi-modal data for video Information Retrieval, modelled the change of scores in a list to predict the importance of a modality. Specifically, the gradual (drastic) change of scores indicates the difficulty (capability) of a modality in distinguishing relevant from irrelevant items, and fusion weights are thus determined accordingly. Firstly that an examination of the distribution of the scores can reveal correlations between those results which undergo a rapid initial change in score, to those results which perform well with regard to relevance. Secondly, we presented an initial model to take advantage of these correlations and to automatically generate weights for a retrieval system without giving that system any prior training or outside knowledge of the collection.

Tan et al, [15], proposed an agreement-fusion optimization model for fusing multiple heterogeneous data. The leveraged rank agreement mined from multiple lists iteratively to update the weights of modalities until reaching an equilibrium stage. The agreement between the scores from multiple modalities is explored to guide the fusion of multiple graphs in both linear and adaptive manners. The agreement is exploited in two ways, namely as the personalization distribution for random walk, or as pseudo training samples for semi-supervised learning to adapt the fusion weights of different modalities. To reconcile the conflicting objectives between graph fusion and agreement, score exchange is conducted iteratively between the two steps to reach an equilibrium solution.

Kennedy et al, [9], proposed a query class dependent search models in multimodal retrieval for the automatic discovery of query classes. This scheme starts by predefining query classes, then learning of weights in offline conducted on the query class level. During search, a given query is routed into one of the predefined classes, and the learnt weights are directly applied for fusion. This scheme is effective in general when the underlying query classes can be clearly defined and there are enough samples for weight learning. Query-class-dependent models for multimodal search by defining query classes through a clustering process according to search method performance and semantic features.

Wei et al, [16], proposed a concept-driven multi-modality fusion (CDMF), explores a large set of predefined semantic concepts for computing multi-modality fusion weights in a novel way. In CDMF, the query-modality relationship is decomposed into two components that are much easier to compute: query-concept relatedness and concept-modality relevancy. In earlier, it can be efficiently estimated online by using semantic and visual mapping techniques, while the latter can be computed offline based on concept detection accuracy of each modality. To determine the fusion weights, the concept-to-modality relationship in a large number of visual concepts is mapped with the query. It automatically discovers useful query classes by clustering queries in a training set.

CONCLUSION

The topic of discussion in this paper is circular re-ranking algorithms. its specifications and related features initiated and categorized by the researchers. How it all started for the purpose of advancements in image processing. the various algorithms discussed in this paper will be a suitable aid in bringing about effective circular re-ranking technique for image processing. It is beyond doubts that he comprehensive study of specific circular re-ranking processes will provide a platform for future presentations we will look forward to it.

References

P. K. Atrey, M. A. Hossain, A. E. Saddik, and M. S. Kankanhalli.“Multimodal fusion for multimedia analysis: A survey,” Springer Multimedia Syst. J., vol. 16, no. 6, pp. 345–379, 2010.

C. A. Bhatt and M. S. Kankanhalli, “Multimedia data mining: State of the art and challenges,” Multimedia Tools Appl., vol. 51, no. 1, pp. 35– 76, 2011.

K. M. Donald and A. F. Smeaton, “A comparison of score, rank and probability-based fusion methods for video shot retrieval,” in Proc. ACM Int. Conf. Image Video Retr., Jul. 2005, pp. 61–70.

R. Fergus, L. Fei-Fei, P. Perona, and A. Zisserman, “Learning object categories from google’s image search,” in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2005, pp. 1816–1823.

W. Hsu, L. Kennedy, and S.-F. Chang, “Video search reranking via information bottleneck principle,” in Proc. ACM Int. Conf. Multimedia, 2006, pp. 35–44.

W. Hsu, L. Kennedy, and S.-F. Chang, “Reranking methods for visual search,” IEEE Multimedia, vol. 14, no. 3, pp. 14–22, Jul.–Sep. 2007.

W. Hsu, L. Kennedy, and S.-F. Chang, “Video search reranking through random walk over document-level context graph,” in Proc. ACM Int. Conf. Multimedia, 2007, pp. 971–980.

L. Kennedy, S. F. Chang, and A. Natsev, “Query-adaptive fusion for multimodal search,” Proc. IEEE, vol. 96, no. 4, pp. 567–588, Apr. 2008.

L. Kennedy, A. Natsev, and S. Chang, “Automatic discovery of queryclass-dependent models for multimodal search,” in Proc. ACM Int. Conf. Multimedia, 2005, pp. 882–891.

Y. Liu, T. Mei, and X.-S. Hua, “CrowdReranking: Exploring multiple search engines for visual search reranking,” in Proc. ACM Special Interest

Y. Liu, T. Mei, X. Wu, and X.-S. Hua, “Optimizing video search reranking via minimum incremental information loss,” in Proc. ACM Int.Workshop Multimedia Inf. Retr., 2008, pp. 253–259.

F. Richter, S. Romberg, E. Hörster, and R. Lienhart, “Multimodal ranking for image search on communitydatabases,” in Proc. ACM SIGMM Int. Workshop Multimedia Inf. Retr., 2010, pp. 63–72.

F. Smeaton, P. Over, and W. Kraaij, “Evaluation campaigns and trecvid,” in Proc. ACM SIGMM Int. Workshop Multimedia Inf. Retr., 2006, pp. 321–330.

G. M. Snoek, K. E. A. van de Sande, O. de Rooij, B. Huurnink, J. C. van Gemert, J. R. R. Uijlings, J. He, X. Li, I. Everts, V. Nedovic, M. van Liempt, R. van Balen, F. Yan, M. A. Tahir, K. Mikolajczyk, J. Kittler, M. de Rijke, J. M. Geusebroek, T. Gevers, M. Worring, A. W. M. Smeulders, and D. C. Koelma, “The mediamilltrecvid 2008 semantic video search engine,” in Proc. NIST TRECVID Workshop, 2008, pp. 1–14.

H.-K. Tan and C.-W. Ngo, “Fusing heterogeneous modalities for video and image re-ranking,” in Proc. ACM Int. Conf. Multimedia Retr., 2011, p.

M. Wang, L. Yang, and X.-S. Hua, “MSRA-MM: Bridging research and industrial societies for multimedia information retrieval,” Microsoft, Washington, DC, Tech. Rep. MSR-TR-2009-30, 2009.

X.-Y. Wei, Y.-G.Jiang, and C.-W. Ngo, “Concept-driven multi-modality fusion for video search,” IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 1, pp. 62–73, Jan. 2011.

P. Wilkins, T. Adamek, G. J. F. Jones, N. E. O’Connor, and A. F. Smeaton “Trecvid 2007 experiments at dublincity university,” in Proc. NIST TRECVID Workshop, 2007, pp. 1–8.

P. Wilkins, P. Ferguson, and A. F. Smeaton, “Using score distributions for query-time fusion in multimediaretrieval,” in Proc. ACM SIGMM Int. Workshop Multimedia Inf. Retr., 2006, pp. 51–60.

P. Wilkins and A. F. Smeaton, “Weighted data fusion for content-based multimedia information retrieval,” in Proc. Internet Multimedia Search Min., Jan. 2010, pp. 1–45.

R. Yan, A. Hauptmann, and R. Jin, “Multimedia search with pseudorelevance feedback,” in Proc. ACM Int. Conf. Image Video Retr., 2003, pp. 238–247.

R. Yan, J. Yang, and A. Hauptmann, “Learing query-class dependent weights in automatic video retrieval,” in Proc. ACM Int. Conf. Multimedia, 2004, pp. 548–555.

T. Yao, T. Mei, and C.-W. Ngo, “Co-reranking by mutual reinforcement for image search,” in Proc. ACM Int. Conf. Image Video Retr., 2010, pp. 34–41.

T. Yao, C.-W. Ngo and T. Mei, “Circular reranking for visual search”, in Proc. IEEE Trans. Image Processing, vol. 22, no. 4, 2013.

D. Zhou, S. A. Orshanskiy, H. Zha, and C. L. Giles, “Co-ranking authors and documents in a heterogeneous network,” in Proc. IEEE Int. Conf. Data Mining, Dec. 2007, pp. 739–744.