Srch
Srch
Srch
Srch

How do you describe an image to Google ? 

How do you describe an image to Google ? 

How do you describe an image to Google ? 

How do you describe an image to Google ?

How do you describe an image to Google ? 

srch2

Challenge


Challenge


The process of searching images in 2017 barely differs from 10 years ago. While google keeps improving its algorithm, the way HOW we interact with search engines fundamentally didn't change - we just got better at googling. Editing the search term is still the way to go, restricting you to words when searching images.

The process of searching images in 2017 barely differs from 10 years ago. While google keeps improving its algorithm, the way HOW we interact with search engines fundamentally didn't change - we just got better at googling. Editing the search term is still the way to go, restricting you to words when searching images.

Discovery


Discovery


Discovery


During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:

  • look into the state of today's image search process
  • analyze and define the problems of current solutions
  • discuss the capability and limitations of image recognition API's
  • characterize our target audiences
  • brainstorm and prioritize features based on value and effort

During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:

  • look into the state of today's image search process
  • analyze and define the problems of current solutions
  • discuss the capability and limitations of image recognition API's
  • characterize our target audiences
  • brainstorm and prioritize features based on value and effort

During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:

  • look into the state of today's image search process
  • analyze and define the problems of current solutions
  • characterize our target audiences
  • brainstorm and prioritize features based on value and effort

The key takeaway points were:

  1. The cognitive dissonance of searching IMAGES by using WORDS
  2. Having too many search results causing choice paralysis
  3. Nonexistent visual representation of different search result clusters

The key takeaway points were:

  1. The cognitive dissonance of searching IMAGES by using WORDS
  2. Having too many search results causing choice paralysis
  3. Nonexistent visual representation of different search result clusters

The key takeaway points were:

  1. The cognitive dissonance of searching IMAGES by using WORDS
  2. Having too many search results causing choice paralysis
  3. Nonexistent visual representation of different search result clusters
srch3

Getting everyone on the same page with a kick-off workshop

Research


Research


After we had a better idea of the problem and brainstormed some solutions we defined our objectives: redefine the image search experience by actively including the user in a more engaging and interactive image search journey, humanize the intersection of user and image recognition API, and explore a 3D environment to browse pictures in VR.

To validate our assumptions and better understand the motivation and context of image search, my designer teammate and I set out to interview real people and conduct usability tests with current solutions. The dpa building was a great place to investigate user needs, and we had the chance to talk to several journalists and editors. Observing them taught us a lot about the workflow and thought process of one of our target groups.

After we had a better idea of the problem and brainstormed some solutions we defined our objectives: redefine the image search experience by actively including the user in a more engaging and interactive image search journey, humanize the intersection of user and image recognition API, and explore a 3D environment to browse pictures in VR.

To validate our assumptions and better understand the motivation and context of image search, my designer teammate and I set out to interview real people and conduct usability tests with current solutions. The dpa building was a great place to investigate user needs, and we had the chance to talk to several journalists and editors. Observing them taught us a lot about the workflow and thought process of one of our target groups.

"Sometimes I have a great picture in mind when I start searching. Then I stumble upon an even better one."
 
"Sometimes I have a great picture in mind when I start searching. Then I stumble upon an even better one."
 
"Sometimes I have a great picture in mind when I start searching. Then I stumble upon an even better one."
 
"Sometimes I have a great picture in mind when I start searching. Then I stumble upon an even better one."
 
"Sometimes I have a great picture in mind when I start searching. Then I stumble upon an even better one."
 

Peer Grimm, Managing Photo Editor DPA


Peer Grimm, Managing Photo Editor DPA


Peer Grimm, Managing Photo Editor DPA



Peer Grimm, Managing Photo Editor DPA


Peer Grimm, Managing Photo Editor DPA

While the interviews confirmed most of our assumptions, we also realized that we did not account for the explorative aspect of searching. Finding exactly what you want might be critical, but often enough the journey to the right picture is just as valuable as finding what you had in mind. We also decided to position our product strongly towards the stock image sector, providing the most value to professionals who spend a lot of time searching pictures.

While the interviews confirmed most of our assumptions, we also realized that we did not account for the explorative aspect of searching. Finding exactly what you want might be critical, but often enough the journey to the right picture is just as valuable as finding what you had in mind. We also decided to position our product strongly towards the stock image sector, providing the most value to professionals who spend a lot of time searching pictures.

Persona


Persona


Based on those insights we experimented with different task flows and created two personas with the emphasis on JTBD and motivation over demographics.

persona_srch

The Explorer

 

    Motive

  • starts with a vague idea
  • has fewer time constraints
  • wants to find the BEST pic

 

    Motive

  • starts with a vague idea
  • has fewer time constraints
  • wants to find the BEST picture

 

    Motive

  • starts with a vague idea
  • has fewer time constraints
  • wants to find the BEST picture

    Frustration

  • too many results & sources
  • a lot of similar looking pictures
  • uncertain about the final result

    Frustration

  • too many results & sources
  • a lot of similar looking pictures
  • uncertain about the final result

n

n

The Hunter

 

    Motive

  • starts with an image in mind
  • doesn't care for the best fit
  • wants to find a pic FAST

 

    Motive

  • starts with an image in mind
  • doesn't care for the best fit
  • wants to find a picture FAST

 

    Motive

  • starts with an image in mind
  • doesn't care for the best fit
  • wants to find a picture FAST

 

    Motive

  • starts with an image in mind
  • doesn't care for the best fit
  • wants to find a picture FAST

    Frustration

  • too many results & sources
  • searching takes too much time
  • image choice could be better

    Frustration

  • too many results & sources
  • searching takes too much time
  • image choice could be better

Execution


Execution


Execution


Execution


After collecting and discussing the data we started prototyping and implementing what we learned. One of the cornerstone ideas for the interface was to enable the user to interact with the content in the most natural way - by pointing at things to refine search results rather than editing the search term. To accomplish this I mapped the available data directly on the image. To avoid overwhelming the user and distract from the image itself the info would only appear upon hovering the mouse over a region of the picture, limiting the number of simultaneously displayed data.

After collecting and discussing the data we started prototyping and implementing what we learned. One of the cornerstone ideas for the interface was to enable the user to interact with the content in the most natural way - by pointing at things to refine search results rather than editing the search term. To accomplish this I mapped the available data directly on the image. To avoid overwhelming the user and distract from the image itself the info would only appear upon hovering the mouse over a region of the picture, limiting the number of simultaneously displayed data.

Refining search results directly in the picture

Crowdsourcing image recognition


Crowdsourcing image recognition


I also wanted to include a way interact with unidentified content. After some investigation, I learned that the bottleneck of the Google Vision API we used (among Adobe Stock, Microsoft Face, and dpa PictureAlliance) was the information richness of an image - the more data in a picture the less chance of identifying objects. My initial thought was to reduce the sample size, so I tested the idea by running segments of the same image individually by Google Vision:

srch_api

Reducing sample size to increase recognition

This process greatly increased the API's recognition. To take advantage of this, I introduced a tool for brushing over unidentified objects. The image recognition API would now only analyze the selected area, resulting in a positive id of the object. While giving the user a tool to filter for unrecognized elements, at the same time this feature could also boost the capability of image recognition API's - the more people use it the better detection gets.

Filtering unrecognized elements

The ability to quickly and naturally apply filters based on the image content also exponentially improved the quality of search results. Given current solutions users started with 2-4 keywords, possibly edited them, and spend the majority of the time scrolling through results.

bjfogg

In SRCH we increased the motivation and ability of users to quickly narrow down their search results by making the process more:

●   intuitive (clicking instead of typing) 
●   playful (magic brush) 
●   engaging (conversational process) 

In SRCH we increased the motivation and ability of users to quickly narrow down their search results by making the process more:

●   intuitive (clicking instead of typing) 
●   playful (magic brush) 
●   engaging (conversational process) 

Users quickly adopted the mindset of interacting with images and applied on average 8 filters, increasing the accuracy and decreasing the number of search results.

The future of grid systems


The future of grid systems


We also experimented with different ways of displaying images in VR. While the traditional grid view remains the most efficient way of displaying pictures in 2D, it is inherently limited by the size of the screen. To illustrate multiple search result clusters instead of having only one pile of pictures, we explored the upcoming realm of VR. One of the biggest advantages of virtual reality, apart from its immersive experience, is the drastically bigger surface to interact with. Unfortunately, after running first tests it became clear that the inaccuracy of available input devices resulted in an overall longer, although initially more fun experience.

Browsing in VR

Feedback


Feedback


After a final feedback round including a mock pitch and more usability testing we finalized the design with some quality of life features including a bookmark option for later view, an indicator for applied filters and minor look & feel tweaks. In the final pitch, we delivered a use case driven presentation displaying an engaging workflow of future image search and a google cardboard version outlining the opportunities of VR.

During the hackathon, I had the chance to talk to great people from a lot of different backgrounds which gave me the opportunity to test my ideas and influenced and challenged my design thinking from new angles. I had a really fun time working with an amazing, diverse team and enjoyed working in such an inspiring environment. Having only 48 hours to tackle a problem has its own charm and really pushes your decision making.

Our team was awarded "Best Overall" and "Best Use Of API".

srch_mockup

2019 © Nicolai Fleischhauer