AspectBrowser Tutorial

Version 1


 

 

 

 

 

AspectBrowser Tutorial

 

Version One

 

 

 

Author(s) and company:

Robert Villa, David Hannah

 

Document status:

Draft

 

Confidentiality:

Public

 

 

 

 

 

 

 

The work presented in this document was partially supported by the European Community under the Information Society Technologies (IST) priority of the 6th framework programme for R&D.

This document does not represent the opinion of the European Community, and the European Community is not responsible for any use that might be made of its content.

This document contains material, which is the copyright of certain SALERO consortium parties, and may not be reproduced or copied without permission. All SALERO consortium parties have agreed to full publication of this document. The commercial use of any information contained in this document may require a license from the proprietor of that information.

Neither the SALERO consortium as a whole, nor a certain party of the SALERO consortium warrant that the information contained in this document is capable of use, nor that use of the information is free from risk, and does not accept any liability for loss or damage suffered by any person using this information.


Table of Contents

1    Executive Summary. 6

2    Introduction. 7

3    Interface Overview.. 8

4    Executing a text query. 11

5    Executing an image query. 13

6    Carrying out a query on an uploaded image. 15

7    Using multiple aspects. 17

8    Glossary. 21

 


1    Executive Summary

This document provides a short tutorial about the AspectBrowser interface, designed to allow users to search web search engines and other video and image databases. This document is intended for end-user usage; it is not a detailed description of the workings of the system.

2    Introduction

The AspectBrowser is the content-based search engine which has been designed to search both SALERO-sourced data (typically image and video) and also provide a meta-search for the web. It is designed around the idea of a search being split into a number of “aspects”, each aspect is represented explicitly on the interface, and allows the user to search using text or visual queries on a range of databases. This documents describes the interface in a rolling tutorial style, each chapter covering a separate feature of the interface.

3    Interface Overview

The interface is implemented as a web application, and can be started by entering the following address in a web browser:

http://mir.dcs.gla.ac.uk:8080/AspectBrowser5/AspectBrowser.html

At startup, the web page will look as follows:

The web page is split into two main sections, the coloured area at the bottom half of the screen will list search results, the top white area is where saved documents will appear in tabs. By pressing the “new aspect” button on the top left of the interface, an Aspect is created, which will update the screen as follows:

 

Each of the labelled areas are as follows:

A.    Slider which adjusts the width of each tab at the top of the screen

B.    Slider which adjusts the division between the tabs and search results area (F)

C.    Pull down list of deleted aspects. Clicking an aspect will undetete it, placing it at the end of the list of tabs

D.   The tab of an aspect. This area shows any document or images which have been saved in this particular aspect. Dragging and dropping material onto this area will save it.

E.    The search bar, allowing a text query to be entered, collection selected, features to be used, etc. (see below)

F.    Search results area

The search bar contains some of the most important components of the search interface, each of which is listed below:

 

1.    Text query entry box

2.    Collection pull down menu, allowing you to select a collection to search

3.    Visual feature selection: three different visual features are extracted from images or videos, dealing with colour, texture, and edges. These will be extracted from images which have been selected in the aspect, and used in generating results, if checked. You can select combinations of features

4.    Search button, which starts a search on the selected collection. A search can use both the text query, plus the visual features from any selected images of the aspect

5.    The browse and upload image function: clicking the “Browse” button will bring up a file selection dialog box to your local computer. After selecting an image, and clicking ok to dismiss the file dialog box, you can then click the “upload” button. This will upload your selected image, which will appear in the aspect’s tab (area D), and which can then be used in searching

6.    Query history: drop down menu showing the list of previous text queries you have executed with this aspect.

It should be noted that not all collections can be searched by visual feature (e.g. we cannot search youtube or the web via yahoo in this web), while other collections (e.g. TaikWeb5) cannot be searched by text (since this image collection contains no text information). Visual or text queries will be ignored by the system for the corresponding collections.

The collections available for searching will vary over time, but as of writing include the following:

·         TaikWeb5: a downloaded set of web images for TAIK (can only be searched by image example)

·         Tinyplanets: An image asset collection from Pepper’s Ghost Productions (only searchable by image example)

·         Bingandbong: Bing and Bong videos from Pepper’s Ghost Productions (only searchable by image example)

In the next chapters, step-by-step instructions for executing text and image queries will be given.

 

4    Executing a text query

Executing a text query is very simple, just enter the text and press search, noting that only some collections are text-searchable, e.g. after starting the interface, and creating a new aspect, enter “George Bush” in the text box, and then select the “trecvid-2008” collection. After pressing the search button you should see the following:

 

Search results appear as a list, most relevant first. Each search result has (where appropriate) a thumbnail image on the left hand side, followed by a document title underlined (in this case the titles are shot numbers from videos) with a URL underneath. A textual document summary will also appear where appropriate.

At the far left of each result is a green arrowed icon  which allows that result to be dragged and dropped. For example, by clicking the  icon next to the first result, and dragging that result into the tabbed area D in the above screenshot, that result will be marked as belonging to the aspect.

The screen should look as follows:

 

There is not explicit limit to the number of shots which can be added to an aspect in this way (but you’ll probably get bored after a while). When you execute a search, the aspect’s tab will also change to reflect the last text query entered. Clicking on the underlined title from a search result will either display the corresponding web page or image in a new window.

Note also that when a result has been added to an aspect’s tab, an extra red “delete” icon  will appear.  Clicking this will remove the image or document from the aspect’s tab.

 

5    Executing an image query

Any images which you have marked in the tab area of an aspect can be used to carry out visual searches on appropriate collection. For example, after carrying out the text search in Chapter 4, we can now carry out a visual search using the image we’ve selected. For example, select “TaikWeb5” from the collections list, and press the “Search” button again. The following results should be seen:

This particular collection has no text, and so our text query will be ignored. Instead the image entitled “shot191_143” will be used as the query, and in this case all three visual features (ColorLayout, HomogenousTexture and EdgeHistogram) will be used in generating the search results. By removing features, different results will be returned. If only EdgeHistogram is used, the following results will be displayed:

 

6    Carrying out a query on an uploaded image

As well as selecting search results, you can also upload images into the interface using the “Browse” and “Upload” buttons. Clicking “Browse” will open a file selection dialog box, allowing you to select an image from your local computer. For example, after pressing Browse we select an example image, in this case a drawing:

 

 

After pressing “open” on the dialog box, you must then press “upload” to upload the image to the search interface, after which it will appear on the aspect’s tab:

Pressing “Search” at this point will search based on both the apple image and the “shot191_143” image from the previous search. In this case, to ensure we only search using the apple drawing, we remove shot191_143 using the red delete  icon, and then press search. The results are as follows, for EdgeHistogram only:

7    Using multiple aspects

The interface supports the creation of multiple search “aspects”. Up until this point we have searched in only a single aspect, although by pressing the “New aspect” button you can create as many aspects as you require. The screenshot below shows the result of pressing the “new aspect” button a further 2 times, to create 2 new aspects:

 

Each aspect is represented by a single tab at the top of the interface, the currently selected aspect being highlighted in a darker blue colour. By clicking on a tab, you select that aspect, which will update the search results and search panel at the bottom of the interface. As can be seen above, this third new aspect has no query, and no search results. Clicking on the first tab switches back to the first aspect:

As can be seen above, the earlier search and associated search results are again shown at the bottom of the interface. Using drag and drop, it is possible to drag results or marked images between aspects, for example, we can drag the apple image onto the second aspect by dragging it with the  icon, and dropping it onto the second tab:

Clicking on the second tab then highlights that aspect, with its associated empty result list:

Selecting the “TinyPlanets” collection from the drop down menu and pressing search will then use the apple image to carry out a search using the PGP data:

 

 By clicking the tabs, you can now switch between the two result lists. Search results can also be dragged and used as queries in a similar way.

At the top of each tab of each aspect there is a bar with left  and right  arrows, and a third “menu” icon . Clicking the left or right arrows will move the aspect left or right in the sequence, allowing you to reorder the aspects. Clicking the Menu icon  will bring up a menu with four options: rename, delete, copy and description:

These four options are as follows:

·         Rename allows you to give a name for the aspect, which will turn red.

·         Delete will delete the entire aspect, including any selected images and search results. It’s name will appear on the “undelete aspect” tab, which when clicked will reinsert the aspect to the end of the list of tabs

·         Copy will copy the contents of the aspect, inserting the new aspect at the end of the list of tabs

·         Description allows you to give a description to the aspect

It should be noted that the interface as of the time of writing does not support the saving of the contents of a search or of an aspect. If you navigate away from the web page, you will lose any searching results and data. This will be changed in the next version, available soon.