International Journal of Innovative Research in Computer and Communication Engineering
MenuISSN ONLINE(2320-9801) PRINT (2320-9798)
ISSN ONLINE(2320-9801) PRINT (2320-9798)
N.Shanmugapriya1, R.Nallusamy2
|
| Related article at Pubmed, Scholar Google |
Visit for more related articles at International Journal of Innovative Research in Computer and Communication Engineering
Content Based Image Retrieval (CBIR) presents special challenges in terms of how image data is indexed, accessed, and how end systems are evaluated. This paper discusses the design of a CBIR system that uses global colour as the primary indexing key, and a user centered evaluation of the system’s visual search tools. The results indicate that users are able to make use of a range of visual search tools, and that different tools are used at different points in the search process. The results also show that the provision of a structured navigation and browsing tool can support image retrieval, particularly in situations in which the user does not have a target image in mind. The results are discussed in terms of their implications for the design of visual search tools, and their implications for the use of user-centered evaluation for CBIR systems.
Keywords |
| User Centered Evaluation, General Image Indexing, Color Based Indexing, Content Based Image Retrieval. |
INTRODUCTION |
| Content Based Image Retrieval (CBIR) systems with fully automatic indexing of images have been of increasing importance in recent years, because of the widespread use of digital cameras and the World Wide Web (for example), and because of the increasing practicality of such systems. However, the ultimate goal of automatically adding text annotations to (general) images remains a long way off. Therefore, CBIR systems must rely upon sophisticated access mechanisms with appropriate supportive interfaces if they are to be effective. The evaluation of these highly interactive interfaces presents new challenges for information retrieval research. Problems of distinguishing properties of the core from properties elsewhere in the setup (to use the terminology of Spark Jones [26] and Spark Jones and Galliers, [27]), well known in evaluating interactive text retrieval systems, are compounded by greater variability in user |
| perceptions of relevance and even perhaps perceptions of the data. Briefly, by core we mean the underlying indexing and retrieval engine; by setup we mean the context in which the engine operates; including the user interface, the users themselves, the experimental or operational setting, the task they are asked to perform and so on. |
| In this paper we report on an evaluation of a CBIR system (CHROMA), [15, 16, 17] which uses a combination of structured navigation and browsing, supported primarily by a global colour indexing method which we have previously argued [15] may facilitate query formulation and refinement. As well as reporting the results of our evaluation of these claims we will point out some useful insights it has given us into conducting user centered evaluations. A possible disadvantage of our approach is that we evaluate complete setups and access strategies rather than underlying core engines. In other words we are abandoning the easy reproducibility and interface independence which has served IR so well since the days of Cranfield [5]. In addition, we are not reliant upon task independent relevance judgements and so cannot use precision and recall. However, there is a growing consensus of opinion that the focus of IR evaluation needs to be on the behaviour of real users in realistic settings [7,9,13], so we feel our approach is justified. In order to put the evaluation work in context, we will begin with a brief description of the CHROMA system, we then move on to describe our evaluation and its results. Finally we will look at some broader implications of this work. |
II. THE CHROMA SYSTEM |
| CHROMA (Colour Hierarchical Representation Oriented Management Architecture) [16], a full-scale prototype image retrieval system, was developed to run on MS Windows-based personal computers. The system was developed for use where large numbers of digital photographic images need to be managed but manual text-based indexing is not practical, and is representative of state of the art CBIR systems [6, 22, 23]. CHROMA uses global colour as its primary indexing key. Images are organised in virtual hierarchical clusters using a set of 10 |
| colours. Users may navigate this hierarchy, use a sketch tool, or select an image and retrieve further similar images. The overall architecture of the CHROMA system can be divided into two sub-systems: the Indexing System and the Access System. The Indexing System undertakes fully automatic image indexing. The Access System allows users to specify image queries and to retrieve images from the collection using the index database. |
| 2.1 Automatic Indexing Using the Perceptual Colour Model |
| CHROMA uses a defined “Perceptual Colour Model” in both the indexing scheme and its retrieval tools. The model is illustrated in Table 1. The group “0” represents uncertain colours such as dark shadows and bright reflections. A central assumption of the model is that people find it easier to use particular base colours only, such as red, blue, green, etc, rather than variant shades of that base colour. For example, users may simply use “blue” to query for a clear sky instead of using “light blue” or “dark blue”. |
 |
 |
| Evidence from the psychological literature seems to support this view. For example Arnheim [1] suggests that although we can distinguish between subtly different shades, when relying upon memory for a colour our powers of discrimination are severely limited. In addition, the studies of the linguists Berlin and Kay [4] introduced the eleven basic colour terms, which are denoted by “*” in Table 1 meaning that CHROMA uses reasonably well found theories of human colour understanding. |
| The indexing scheme generates a 1-D “Class Key” using the ten Colour Descriptors, for each image according to the area of the image assigned to each of the ten perceptual colours. This is intended to provide a useful cue to the user during retrieval and supports structured browsing and navigation [27] by providing a virtual hierarchical classification. For example, the image collection is divided into mainly blue images, mainly yellow, mainly green and so on. The blue images are then divided into those which are mainly blue and yellow (beaches perhaps), mainly blue and green (landscapes perhaps) and so on. A 2-D “Index Vector” is also generated to represent spatial information (or local features) in an image using the perceptual colours, which can be used for similarity retrieval. More details about the indexing scheme can be found in [16]. |
| 2.2 The Access System |
| The CHROMA Access System supports different styles of retrieval within a single interface. The layout of the user interface is divided into three windows, illustrated in Figure 1.The top -left window, called the Navigation Tool, allows users to explore the hierarchical classification for the image database. The top-right window, called the Thumbnail Viewer, outputs the retrieved images by displaying sets of thumbnail images, which are selected by using the Navigation Tool. The bottom window, called the Information Viewer, displays a list of related information about the retrieved images. Interactions in each window are related, so by selecting a specific class with the Navigation Tool, all of the images in this class are listed in the Information Viewer. The Thumbnail Viewer allows browsing sets of thumbnails by clicking the “scroll buttons” above one at a time (i.e. to display previous and next sets). The design of the Navigation Tool is based on a tree-based directory (or folder) structure, which represents the hierarchical classification. It allows users to navigate specific “image classes” on the basis of their dominant colours. CHROMA, in common with a number of other CBIR systems (e.g. [2]) supports a “click-to-enlarge” operation of thumbnails, allowing full size image viewing, and its converse. |
| 2.3 The Navigation Tool |
| The CHROMA system provides a method for navigating and browsing the hierarchical classification using the Navigation Tool. The initial state of the Access System is to display all of the images in order of the hierarchical classification on “ALL”, which means the user can browse all of the images serially. In order to minimise the possibility that users might become lost in the navigable hierarchy the user view of the tree structure is limited to two levels of colour groupings. Navigational support is also provided by helping users distinguish between sets of thumbnails they had previously viewed and those that remain to be seen. This feature mirrors existing methods used to indicate revisitation patterns in hypertext systems, users may skip tagged classes, to assist in gradually decreasing the “search domain” within the image database. |
| 2.4 The Sketch Tool |
| The CHROMA system also supports similarity-based methods for querying images. The Sketch Tool provides a user construction tool to outline the approximate contents in terms of colours and spatial locations of objects in the required image using a paintbrush (or a mouse). This tool aims to allow users to retrieve an existing image which they have seen or used before. The layout of the Sketch Tool is illustrated in Figure 2. |
 |
2.5 Query-By-Image-Example |
| CHROMA also provides a tool for “Query-by-Image-Example”, which is a well -known method to allow users to point out a candidate image and the system then retrieves some more images similar to it. The operation of the method is performed by the action of “click-to-query”. Users may choose a candidate image among the set of thumbnails on the browser. A right mouse button click then allows this image to be used as the basis of a similarity search of the image database. |
III. THE EVALUATION |
| A number of investigators have highlighted the advantages offered by the use of user-centred evaluation techniques in image and information retrieval [7, 8, 9, 11, 13]. For example, Dunlop [8, 9] points out that standard precision and recall metrics merely show the retrieval effectiveness of the underlying system and do not take account of user interface and speed issues. Therefore, if such measures were used to compare two IR systems, they would not be able to predict which system would perform best under certain task conditions and with different groups of users. User-centred evaluation however, allows us to compare system performance with different users and for a large variety of tasks [8, 9]. Jose, Furner and Harper [13] also support this view and stress the need for evaluating real world systems in real world settings. Draper [7] suggests that one of the main advantages of the user-centred approach is that it allows us to observe how people actually use our systems as opposed to how we expect them to be used. Therefore, this type of evaluation may lead to positive system refinements that precision and recall would not have identified. |
3.1 Motivations |
| The evaluation of the CHROMA system focused on the system’s visual search tools. We are not attempting to support the claim that visual search tools are superior to text-based queries for image retrieval. The CHROMA system has a number of visual search tools that can be used on their own and interactively. Therefore it was decided to evaluate small units of the system individually rather than attempting to construct one large-scale evaluative study of the system as a whole. This mirrors Harper and Hendry’s approach [11], which suggests the use of small controlled tasks that are directed at evaluating a given claim or hypothesis. |
| The primary aim of the evaluation was to test the following claim. That the provision of a structured navigation and browsing tool can help overcome some of the problems associated with similarity based measures by facilitating query formulation, particularly in circumstances in which the user has only a vague idea about what it is they wish to retrieve. To evaluate this claim an experiment was devised to compare the performance of users with the CHROMA navigation tool and with the similarity-based query-by-sketch tool. This kind of experimental comparison would allow us to examine subjects’ performance with each of the tools and in the type of task conditions that would lead users to employ one tool over the other. We hypothesised that when subjects had only a vague search goal they would be more likely to use the navigation tool, and when the task was more specific, the similarity tool would be relied upon more. |
 |
3.1.1 Tasks, Materials and Participants |
| The people who took part in the experiments were all students from the University of Sunderland. We feel justified in our use of students as experimental subjects because the CHROMA system was intended for use by a wide population of general users which will inevitably include students. Different participants were used in each study. Two types of experimental task were used, scenarios and timed search tasks. Scenarios were used in order to simulate how participants might typically interact with the system in a non-experimental setting. A focus group consisting of the experimenter and five first year students discussed areas in which this type of system might be used. The main areas that the discussion generated were; retrieving photographs to illustrate a set piece of course work, or for illustrating pages for publication on the World Wide Web. Since all agreed that these were areas in which they would be most likely to use such a system they were chosen to serve as the basis for the task scenarios. Other areas included the production of student newsletters, general correspondence, and information posters. The purpose of the scenarios was to encourage participants to work at their own pace; they were told that no time limit would be imposed. For this type of task performance data is difficult to capture since it would be affected by the subjects’ individual working style. In order to obtain some measure of performance a timed search task was included. |
| In this task subjects were instructed that we were interested in two factors, the number of successful searches and the time taken to complete searches. Queries were presented to the participants in either target specific or target non -specific form. In the target specific condition participants were given a target image to retrieve, in the target non -specific condition participants were required to find an image to illustrate a general theme. Two versions of a special test collection of 1000 images extracted from a commercial royalty free CD-ROM was prepared. All experiments used a Latin Squares design, counterbalancing measures were taken to minimize practice and order effects. |
3.2 Experimental Studies |
| Query based approaches to image retrieval may be problematic for users because they require the user to be able to express a clear and succinct search query. However, there is evidence to suggest that users may experience problems during query formulation [15]. These problems may be further exacerbated in situations in which the user has only a vague idea of what it is they wish to retrieve. Lai, Tait and McDonald [15] argue that the provision of a structured image browsing and navigation tool may alleviate these problems and facilitate retrieval at both the initial stage of the search process, and during subsequent query refinement. The assumptions behind this claim are that structured navigation and browsing has a number of advantages over similarity based methods. First, the user can browse through the image collection in a structured and organised manner guided by the major colour groupings of the hierarchical structured navigation tool. Second, the user can search the collection without having to compose a sketch of the required image, which may be a difficult task if he/she is unsure of their search goal. Third, the user does not have to locate a candidate target image from the collection on which to base their search. To evaluate this claim an experiment was devised to compare the performance of users with the CHROMA navigation tool and with the similarity-based query-by-sketch tool, in situations in which the user has only a vague idea of what it is they need to retrieve (target non- specific condition), and in situations in which the search need is pre-specified (target specific condition). |
| Twenty-four student volunteers participated in the study, twelve males and twelve females. Their ages ranged between 22 and 31 years. The experiment used a within-subjects design, in which the same group of subjects featured in both experimental conditions. The independent variables were System Type (Navigation Tool vs. Sketch Tool) and Search Type (Target specific vs. Target non-specific). The dependent variables included performance measures and data of a more qualitative nature regarding user satisfaction and system usability obtained through the administration of a questionnaire. After a demonstration and description of the first system’s functionality, subjects were allowed to familiarise themselves with the system by performing a series of practice searches. This practice session served to set a performance baseline. The experiment began with the scenario task. The scenarios were presented to the subjects as written instructions. Each scenario required the user to conduct four searches. Two searches required the user to retrieve a specified image, the remaining two searches did not specify an image, but required users to locate an image on a general theme. Participants were allowed to issue as many separate queries as they deemed necessary to complete each search, no time limit was imposed. After each completed search subjects were asked to rate on a five point scale the extent to which they agreed or disagreed with a number of statements referring to their satisfaction with the retrieved image set and their satisfaction with the search tool. Subjects were then required to use the system to complete a timed search task. Specifically, subjects were asked to locate five target images from the database. Pictures of each target image were handed to the subjects one at a time. The presentation order was randomised for each subject. Subjects were also required to complete a target non-specific search task. Subjects were given five general themes and were asked to find an image to illustrate each theme. After locating an image subjects were asked to indicate whether they were satisfied with their chosen image. Each satisfied response was counted as a successful search, the number of successful searches and the time taken to complete each search was calculated for each subject. After completing the last search task subjects were given a break of approximately two hours. Test Phase Two began with a practice session with the second system. The procedure followed for Test Phase Two was exactly the same as in Test Phase One. Finally, after subjects had used both systems, they were asked to answer a number of questions about their interactions with both systems. |
3.3 Results |
| The results of the user satisfaction measures suggest that although both systems scored highly on measures of general usability, indicating that subjects thought both tools were easy to use, learn and remember, other user-satisfaction indicators suggested that users preferred the Navigation Tool to the Sketch Tool. The pattern of results for the performance data was as follows. There was no significant difference in the number of successful searches made between the Navigation tool and the Sketch Tool in either task condition (Target specific vs. Target non-specific). However, subjects performed the search tasks significantly faster using the Navigation Tool (mean = 255 seconds) than when using the Sketch Tool (mean = 413 seconds) when the task was target non-specific. When the task was target specific there was no difference in search times between the two systems (Navigation Tool = 242, Sketch Tool = 246). The results of the performance measures suggest that the provision of a structured browsing and navigation tool leads to fast and efficient retrieval of images, especially when the user has only a vague idea of what it is they wish to retrieve from the image set. |
 |
| There are a number of possible explanations for these results. First, it could be that it requires less effort to browse dominant colour groups than to sketch likely candidate images. A second explanation (supporting Lai, Tait and McDonald [15]) might be that users experience problems constructing a query when they only have a vague idea of what it is they wish to retrieve. Third, the sketch tool might perform badly because it is a poor example of its kind. However, this explanation seems unlikely since there was no difference in performance between the navigation tool and the sketch tool when the search task was target specific, and the sketch tool rated highly on measures of general usability. Nevertheless, it was decided that another experiment should be conducted in order to provide further support to our findings. We therefore constructed an experiment to compare the CHROMA sketch tool with that used by the colour layout editor of IBM’s QBIC system [12]. The experiment used a within subjects design. Ten subjects worked with each tool in turn. The task replicated the scenario task used in experiment one. Overall, there was no significant difference in the user satisfaction measures with the images retrieved by the two tools. (CHROMA sketch mean rating 4.4; QBIC sketch mean rating 4.35). However a Wilcoxon test revealed that there was a significant difference (Z = -2.46, p>0.01) in the ratings of the subjects’ satisfaction with the search tool. Subjects expressed greater satisfaction with the CHROMA sketch tool (mean rating 4.1) than with the QBIC sketch tool (mean rating 2.6). When questioned about their preference for the CHROMA sketch tool, subjects reported that they preferred the tool because its design was similar to that of a typical freehand drawing package. |
| The results of experiment one suggest that subjects were able to perform their searches faster with the navigation tool than with the sketch tool, and that overall, subjects preferred the navigation tool to the sketch tool. However, in order to compare the two different types of access in an experimental format it was necessary to use an experimental design in which the subject used both tools at different times. While the experiment showed which tool produced the best performance on a number of tasks, it does not show which tool subjects would use if they had the opportunity to access both tools, or which tool subjects would choose to use during different stages of the query formulation process. |
| For the test phase of the experiment subjects were required to complete ten searches, five searches were target specific and five target non -specific. Each time a subject began a search the tool they used to access the system was recorded; this was called the initial query phase. After submission of this initial query, the experimenter noted which tool was used to refine or reformulate that query (refinement phase). In some cases subjects refined their queries several times, in others only one refinement of the initial query was necessary. |
| A 2x2 repeated measures analysis of variance on the initial query data revealed a significant main effect of Tool (F(1,11) = 245.8., p <0.01). There was no main effect of task type (target specific vs. target non-specific) or any significant interactions (See Figure 3a). Overall, the navigation tool was used the most during the initial query stage (combined total figure for target specific and target non-specific condition = 81), followed by the sketch tool (combined total figure = 37). A paired samples t test revealed that the difference between these scores is significant (t = 3.32., p < 0.007). A 2x2 repeated measures analysis of variance on the query refinement data revealed a significant main effect of Tool (F(1,11) = 18.4., p <0.01). There was no main effect of task type (target specific vs. target non-specific) or any significant interactions (See Figure 3b). Overall, the query-by-image-example tool was used the most during the query refinement stage (combined total figure for target specific and target non-specific condition = 62), followed by thumbnail browsing (combined total figure = 29). A paired samples t test revealed that the difference between these scores is significant (t = -2.21., p < 0.05). |
| After completing these tasks subjects were asked to specify which tool they believed to be of most assistance during the execution of the search tasks. Eight out of twelve subjects reported that they believed the navigation tool to be of most help, one subject stated that they believed the sketch tool was of most help, and three indicated they the query-byimage example was of most assistance. When questioned why they preferred the navigation tool a number of reasons were given. Some related to speed issues, e.g. "it was quicker to use this tool than to draw a picture". Others related to problems with query formulation, e.g. "At the start I found it difficult to think of an image to draw". Other reasons related to the tool's ability to present an overview of the test collection and to engender a sense of closure. |
IV. DISCUSSION |
| This work has implications for both the design of visual search tools, and for the use of user centered methods for the evaluation of CBIR systems. We will now discuss these aspects in turn. Taken together, our results suggest that the provision of a structured browsing and navigation tool may facilitate image retrieval, particularly in situations where the user does not have a specific target image in mind. The results of the satisfaction data suggest that users found the tool both easy and effective to use. The results of experiment three support these findings in that users found the navigation and browsing tool to be particularly useful during initial query formulation, as it does not require them to express a specific query. These results support the claims that similarity can be problematic and relate to previous work in Information Retrieval that most novice users express very general search specifications during initial query formulation [14, 20]. While the performance of participants was better with the navigation tool than the sketch tool, the results also show that users were also able to express their search needs as visual-spatial queries using a sketch tool, and that users found this particular mechanism to be enjoyable and easy to use. These findings support those of Jose et al [13] who also found that users were able to express visual-spatial queries. |
| The results also suggest that there is a need to provide users with a wide range of query tools, and that different tools are used at different points during the search process. For example, in experiment three subjects most often browsed the image set via the navigation tool during initial query formulation, and then refined retrieved sets using a similarity measure (query-by-example). Within the context of our results, the strategies described can be said to have been exhibited by novice users, it may well be the case that more experienced individuals may have used the access tools in different ways, perhaps favoring the similarity based sketch tool. However, if we are to provide a scaffolding framework for novice users, allowing the submission both general and specific access tools seems to be of importance. |
| Our evaluative studies suggest therefore, that browsing may be an effective image retrieval strategy, confirming early work by Batley [3]. However, we believe the success of our approach lies in the structure the navigable hierarchy applies to the browsing process. While there is work to suggest that users like the high levels of control browsing gives them [18, 21, 25], if left to browse large information spaces users can experience feelings of disorientation that can lead to a measurable decline in search performance [19]. Disorientation when browsing seemed to be less of a problem in CHROMA because the images that comprise the information space are organized so that similar items (in terms of dominant color) are near each other. Furthermore, the organization is such that a notion of directionality or “scent” is apparent to the user. The notion of information scent was first applied to navigation through hypertext based systems. Furnas [10] has argued that for a system to be navigable the link information or label of a given link must describe not only the next node but also every other node down that path. Hypertext link information must therefore depict what information can be found down a particular route or direction rather than just the next node in the system. The scent of a target is the remote indication of that target in out- link information throughout the hypertext structure. Poor scent can lead to navigational problems because the navigator has no way of knowing whether the information required is available or how to find it. These ideas can be applied to image retrieval systems that employ a navigational metaphor. Within the CHROMA system the layout of the colour tool means that all of the dominant colour groups are available to the user, therefore they can easily move from less relevant images to more relevant images by selecting a different dominant colour group. In addition, CHROMA provides a tagging mechanism so that users can instantly identify the parts of the image collection they have already examined. This helps users to estimate the extent of the database and make more reliable judgements about with parts of the information space they have or have not visited. |
| While we believe that the design of the CHROMA navigation tool should alleviate problems associated with disorientation, we must acknowledge that the image collection used in our studies is comparatively small (1,000 images). It may be the case that with such a small collection of images, browsing was both easier and perhaps a more natural way to access the collection. However, it is important to note that while the collection may have been small, two version were used in order to minimise practice effects. The test collection was dropped into the statistical analysis carried out in all of our studies and was found not to have had any significant effect on the results. In order to address this problem of collection size, further studies are planned to examine performance with the CHROMA navigation tool and other browsing arrangements with a much larger collection of images. However, while this limits our ability to draw generalizations from our work about the effectiveness of browsing as a retrieval strategy over similarity based methods, we believe our results do have a number of interesting and important implications for CBIR systems. First, our results suggest that users can indeed express their search needs in visual form and can successfully locate images on the basis of a global colour search. Second, while the relatively small collection limits our results, the way in which the participants used the navigation tool does seem to suggest that there is a good basis for the use of structured navigation. As described earlier, the initial state of the navigation tool is to display all of the images in the collection. From this initial state users can opt to browse the whole collection using forward and back buttons or they can access the collection using the hierarchical classification tool. In our studies it was explained to the participants that they could choose either one of these options. Without exception, all subjects opted to use the navigation tool, which we believe suggests that users are less inclined to browse the whole collection, and that the hierarchical collection was perceived to be a faster alternative. In order to browse the entire collection without the aid of the navigation tool in sets of eight images would take 125 iterations. On average the user would only hit a specified target image by chance in 63 iterations, which is quite a high figure. Evidence from Markkula and Sormunen [18] suggests that more experienced users will browse approximately 100 individual images before reformulating their query, this figure may possibly be reduced for novice users such as ours, and therefore the comparatively small size of the collection may not have had a marked impact on the results. Third, the results suggest that browsing is particularly useful in situations in which the user does not have a specific candidate image in mind. This is borne out by the results of experiment 1, in that when the task was target non-specific retrieval was much faster using the navigation tool than when using the sketch tool. |
| In addition we believe our studies also demonstrate the advantages of user centered evaluations for CBIR systems. The use of this approach enabled us to obtain a greater variety of data in terms of both user performance and their subjective assessments of usability and system performance. It also allows us to test specific hypotheses about our systems under a variety of task conditions, and in line with Draper’s [7] observation, the user of user- centered evaluations allow is to see how our systems are actually used as opposed to how we expect them to be used. For example, although not discussed in this paper, the CHROMA system incorporates an original algorithm for handling subtle lighting effects in images. During the development of the system this issue was thought to be of importance, however, user testing suggested that users do not consider this aspect to be important and failed to use the facility provided. |
| In our evaluation we focused on the use of controlled experiments, with measures of efficiency and user perceptions of efficiency and effectiveness. There are of course, a number of other methods available within this general framework (for example, protocol analysis, constructive interaction scenarios, semantic differential scales etc.), although little guidance is available concerning the relative contributions different methods make to the evaluation of CBIR systems or how these methods compliment on another. Future work in our own laboratory is planned to investigate these issues further. |
| While user-centered approaches to evaluation can provide a wealth of data, they can be both time consuming and labour intensive. Moreover, the value of the results obtained is dependent upon the quality of the experimental design. The importance of good experimental design, such as controlling for practice effects using counter balancing of tasks and materials (including the image collection) is of paramount importance if misleading or spurious results are to be avoided. |
| However, we believe that the importance of the user in the evaluation of CBIR systems, and the claims or hypothesis that motivate the construction of those systems, cannot be overstated. The use of visual search tools means that we are asking users to search for images in ways that are quite different from what they have come to expect (searching on the basis of low level visual features as opposed to content descriptions). Clearly this change in focus will have some affect people’s search strategies and success, and therefore must be considered. |
V. CONCLUSION |
| Our results illustrate the usefulness of adopting user centered evaluation studies over traditional precision and recall metrics, in that we were able to obtain data regarding the users perceptions of the search tools studied, as well as effectiveness data. Moreover, by adopting this approach we were able to modify the search tasks (i.e. task specific/task nonspecific) to discover the circumstances in which each of the CHROMA tools were most likely to be used. However, we must also emphasis the limitations of our approach. The experiments used a relatively small image database of 1,000 images and relatively small sample of users, during only one period of interaction with the system. Future work will focus on studying, search behaviour with more well defined notions of target specific and target non-specific test queries within a longitudinal paradigm. A further aim is to conduct a more fine grain analysis of the query formulation process. |
| Nonetheless, our current work has demonstrated the utility of using a simple (computationally and conceptually) global colour-based indexing scheme combined with a direct browsing mechanism for the index structure. We suspect, despite its effectiveness for initial query formulation, that it would not be effective as a sole query mechanism. CHROMA’s indexing scheme has a number of disadvantages. For example the same object photographed with different levels of "zoom" may index quite differently, even though a human indexer or retriever may perceive little difference between the photographs. In the future we would like to explore the use of indexing (and browsing) algorithms which make use of regioning or other spatial information and features other than colour: perhaps texture and edginess. Of course combining this into a computationally efficient and easy to use environment will present significant challenges. Our results that Query-by-Image- Example is a useful means of query refinement will be no surprise to any student of relevance feedback. However we claim these results do reinforce the notion that global colour indexes are an effective if simple and crude way to capture human notions of visual similarity for general photographic images. |
| Content Based Image Retrieval is an increasingly important branch of Information Retrieval. The evaluation work reported here has indicated that the combined use of an easy to understand colour indexing scheme, support for browsing index structures, query-by-image-example and a sketch tool is an extremely promising approach to CBIR. |