The aim of this work is to index images in domain specific databases using colors computed from the object of interest only, instead of the whole image. The main problem in this task is the segmentation of the region of interest from the background. Viewing segmentation as a figure/ground segregation problem leads to a new approach - eliminating the background leaves the figure or object of interest. To find possible object colors, we first find background colors and eliminate them. We then use an edge image at an appropriate scale to eliminate those parts of the image which are not in focus and do not contain contain significant structures. The edge information is combined with the color-based background elimination to produce object (figure) regions. We test our approach on a database of bird images. We show that in 87% of 450 bird images tested, the segmentation is sufficient to determine the colors of the bird correctly for retrieval purposes. We also show that our approach provides improved retrieval performance.