Digital Asset Management (DAM) platforms such as OpenAsset offer a scalable solution to the problem of indexing and accessing the digital files that bring value to an organization. Using a structure of project and file keywords, users can swiftly apply metadata to assets, ensuring that anyone in the organization can select the best available images for their projects. The problem with systems that rely on metadata is that they are dependent upon the quality and accuracy of the keywords chosen by users.
By harnessing the capabilities of Artificial Intelligence (AI), OpenAsset promises to deliver significant efficiencies to this often time-consuming process.
Challenges with image recognition services
As we explored in our previous blog, a number of Application Programming Interfaces (API) are available to developers looking to incorporate image recognition into their existing tech stacks. Services like Amazon Rekognition offer advanced image analysis features and by populating the Rekognition cache with a large volume of Architecture, Engineering and Construction (AEC), and Real Estate images, we were able to analyze the usefulness of the AI service for these industries.
From this period of testing, we concluded that Amazon Rekognition presented a number of challenges before it could be effectively integrated into OpenAsset. The main concern was that the accuracy of the suggested keywords from the Rekognition API was too low to deliver value to our clients and that it could lead to inaccurate search results.
The other challenge presented by Amazon Rekognition is that the model on which the image recognition is based is a generic one designed to satisfy a wide range of use cases. The model functions as a vocabulary that the AI uses to predict the contents of images. When this generic model is applied to a specific domain such as AEC and Real Estate, the quality of the keyword suggestions suffers.
Having concluded that the keyword suggestions from the API could not be used to automatically tag images, we re-assessed how AI could be incorporated into OpenAsset and arrived at two new lines of enquiry:
- AI driven image similarity search
- AI driven keyword suggestions
Image Similarity
By leveraging data from the image recognition platform Amazon Rekognition, we developed a system that computes the visual similarity between images. Users may already be familiar with image similarity or ‘reverse image search’ features in Google, Bing, Instagram or Pinterest.
Until now, users searching for images within OpenAsset relied upon keywords that had been selected by other users. Manually applying metadata in a large database of images is a time-consuming process, requiring staff or image librarians who are trained to use a specific keyword structure. Until metadata has been applied to these assets, they are not in a searchable state and therefore are unable to deliver value to an organisation.
For the near-future, we believe that human input will continue to be necessary for a well-organised DAM. However, by overlapping manual with automated workflows, these process can be streamlined and searches can be of higher quality. The Image Similarity feature uses AI-powered image recognition to calculate the probability of images featuring similar content. This will offer users a new way of discovering images that works alongside metadata-based searches.
When a customer selects an image in OpenAsset, the Image Similarity feature will return a selection of images from across the entire image library that it identifies as being visually similar. By comparing the image recognition data, the application is able to calculate a ‘Similarity score’ between 0-1 and rank images according to this number.
We believe the Image Similarity feature will offer customers a natural enhancement of search, which should be intuitive and easy to use. There are numerous scenarios in which we believe this feature can deliver value:
- Users will be able to establish connections between projects built in a common architectural style or which share unique features. For example, Image Similarity could be used to find examples of a particular visual aspect of a building such as composition or pattern, or to view a selection of buildings with specific features like turrets or spiral staircases. In this respect, Image Similarity can be used to discover details in images that may be overlooked during the tagging process.
- If a user is struck by the particular qualities of an image, they can progress their search without having to decide upon search terms to describe these qualities. Image Similarity will assist serendipitous discovery across the DAM, inspiring creative decisions.
- If a user discovers a valuable search term that is missing from the keyword structure, such as images featuring white columns or reception desks, the Image Similarity feature could help to speed up the process of identifying images that should be tagged with the missing keyword.
In the following blog, we shall look at our ongoing research and development on AI Driven Keyword Suggestions.
