srch

How do you describe an image to Google ?

Challenge

The process of searching images in 2017 barely differs from 10 years ago. While google keeps improving its algorithm, the way HOW we interact with search engines fundamentally didn't change - we just got better at googling. Editing the search term is still the way to go, restricting you to words when searching images.

Discovery

During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:

look into the state of today's image search process
analyze and define the problems of current solutions
discuss the capability and limitations of image recognition API's
characterize our target audiences
brainstorm and prioritize features based on value and effort

During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:

look into the state of today's image search process
analyze and define the problems of current solutions
discuss the capability and limitations of image recognition API's
characterize our target audiences
brainstorm and prioritize features based on value and effort

During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:

look into the state of today's image search process

analyze and define the problems of current solutions

discuss the capability and limitations of image recognition API's

characterize our target audiences

brainstorm and prioritize features based on value and effort

The key takeaway points were:

The cognitive dissonance of searching IMAGES by using WORDS
Having too many search results causing choice paralysis
Nonexistent visual representation of different search result clusters

The key takeaway points were:

The cognitive dissonance of searching IMAGES by using WORDS
Having too many search results causing choice paralysis
Nonexistent visual representation of different search result clusters

The key takeaway points were:

The cognitive dissonance of searching IMAGES by using WORDS
Having too many search results causing choice paralysis
Nonexistent visual representation of different search result clusters

Getting everyone on the same page with a kick-off workshop

Research

After we had a better idea of the problem and brainstormed some solutions we defined our objectives: redefine the image search experience by actively including the user in a more engaging and interactive image search journey, humanize the intersection of user and image recognition API, and explore a 3D environment to browse pictures in VR.

To validate our assumptions and better understand the motivation and context of image search, my designer teammate and I set out to interview real people and conduct usability tests with current solutions. The dpa building was a great place to investigate user needs, and we had the chance to talk to several journalists and editors. Observing them taught us a lot about the workflow and thought process of one of our target groups.

After we had a better idea of the problem and brainstormed some solutions we defined our objectives: redefine the image search experience by actively including the user in a more engaging and interactive image search journey, humanize the intersection of user and image recognition API, and explore a 3D environment to browse pictures in VR.

To validate our assumptions and better understand the motivation and context of image search, my designer teammate and I set out to interview real people and conduct usability tests with current solutions. The dpa building was a great place to investigate user needs, and we had the chance to talk to several journalists and editors. Observing them taught us a lot about the workflow and thought process of one of our target groups.

"Sometimes I have a great picture in mind when I start searching. Then I stumble upon an even better one."

Peer Grimm, Managing Photo Editor DPA

While the interviews confirmed most of our assumptions, we also realized that we did not account for the explorative aspect of searching. Finding exactly what you want might be critical, but often enough the journey to the right picture is just as valuable as finding what you had in mind. We also decided to position our product strongly towards the stock image sector, providing the most value to professionals who spend a lot of time searching pictures.

Persona

Based on those insights we experimented with different task flows and created two personas with the emphasis on JTBD and motivation over demographics.

The Explorer

Motive

starts with a vague idea
has fewer time constraints
wants to find the BEST pic

Motive

starts with a vague idea
has fewer time constraints
wants to find the BEST picture

Motive

starts with a vague idea
has fewer time constraints
wants to find the BEST picture

Frustration

too many results & sources
a lot of similar looking pictures
uncertain about the final result

Frustration

too many results & sources
a lot of similar looking pictures
uncertain about the final result

n

The Hunter

Motive

starts with an image in mind
doesn't care for the best fit
wants to find a pic FAST

Motive

starts with an image in mind
doesn't care for the best fit
wants to find a picture FAST

Motive

starts with an image in mind
doesn't care for the best fit
wants to find a picture FAST

Motive

starts with an image in mind
doesn't care for the best fit
wants to find a picture FAST

Frustration

too many results & sources
searching takes too much time
image choice could be better

Frustration

too many results & sources
searching takes too much time
image choice could be better

Execution

After collecting and discussing the data we started prototyping and implementing what we learned. One of the cornerstone ideas for the interface was to enable the user to interact with the content in the most natural way - by pointing at things to refine search results rather than editing the search term. To accomplish this I mapped the available data directly on the image. To avoid overwhelming the user and distract from the image itself the info would only appear upon hovering the mouse over a region of the picture, limiting the number of simultaneously displayed data.

Refining search results directly in the picture

Crowdsourcing image recognition

I also wanted to include a way interact with unidentified content. After some investigation, I learned that the bottleneck of the Google Vision API we used (among Adobe Stock, Microsoft Face, and dpa PictureAlliance) was the information richness of an image - the more data in a picture the less chance of identifying objects. My initial thought was to reduce the sample size, so I tested the idea by running segments of the same image individually by Google Vision:

Reducing sample size to increase recognition

This process greatly increased the API's recognition. To take advantage of this, I introduced a tool for brushing over unidentified objects. The image recognition API would now only analyze the selected area, resulting in a positive id of the object. While giving the user a tool to filter for unrecognized elements, at the same time this feature could also boost the capability of image recognition API's - the more people use it the better detection gets.

Filtering unrecognized elements

The ability to quickly and naturally apply filters based on the image content also exponentially improved the quality of search results. Given current solutions users started with 2-4 keywords, possibly edited them, and spend the majority of the time scrolling through results.

In SRCH we increased the motivation and ability of users to quickly narrow down their search results by making the process more:

●   intuitive (clicking instead of typing)
●   playful (magic brush)
●   engaging (conversational process)

In SRCH we increased the motivation and ability of users to quickly narrow down their search results by making the process more:

●   intuitive (clicking instead of typing)
●   playful (magic brush)
●   engaging (conversational process)

Users quickly adopted the mindset of interacting with images and applied on average 8 filters, increasing the accuracy and decreasing the number of search results.

The future of grid systems

We also experimented with different ways of displaying images in VR. While the traditional grid view remains the most efficient way of displaying pictures in 2D, it is inherently limited by the size of the screen. To illustrate multiple search result clusters instead of having only one pile of pictures, we explored the upcoming realm of VR. One of the biggest advantages of virtual reality, apart from its immersive experience, is the drastically bigger surface to interact with. Unfortunately, after running first tests it became clear that the inaccuracy of available input devices resulted in an overall longer, although initially more fun experience.

Browsing in VR

Feedback

After a final feedback round including a mock pitch and more usability testing we finalized the design with some quality of life features including a bookmark option for later view, an indicator for applied filters and minor look & feel tweaks. In the final pitch, we delivered a use case driven presentation displaying an engaging workflow of future image search and a google cardboard version outlining the opportunities of VR.

During the hackathon, I had the chance to talk to great people from a lot of different backgrounds which gave me the opportunity to test my ideas and influenced and challenged my design thinking from new angles. I had a really fun time working with an amazing, diverse team and enjoyed working in such an inspiring environment. Having only 48 hours to tackle a problem has its own charm and really pushes your decision making.

Our team was awarded "Best Overall" and "Best Use Of API".

next case study ›

Say Hi: hello@nicolaifleischhauer.com