The process of searching images in 2017 barely differs from 10 years ago. While google keeps improving its algorithm, the way HOW we interact with search engines fundamentally didn't change - we just got better at googling. Editing the search term is still the way to go, restricting you to words when searching images.
The process of searching images in 2017 barely differs from 10 years ago. While google keeps improving its algorithm, the way HOW we interact with search engines fundamentally didn't change - we just got better at googling. Editing the search term is still the way to go, restricting you to words when searching images.
During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:
During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:
During the DPA Hackathon 2017, we set out to explore a more intuitive, fun way to browse images. I began by facilitating a workshop with the entire team to:
The key takeaway points were:
The key takeaway points were:
The key takeaway points were:
Getting everyone on the same page with a kick-off workshop
After we had a better idea of the problem and brainstormed some solutions we defined our objectives: redefine the image search experience by actively including the user in a more engaging and interactive image search journey, humanize the intersection of user and image recognition API, and explore a 3D environment to browse pictures in VR.
To validate our assumptions and better understand the motivation and context of image search, my designer teammate and I set out to interview real people and conduct usability tests with current solutions. The dpa building was a great place to investigate user needs, and we had the chance to talk to several journalists and editors. Observing them taught us a lot about the workflow and thought process of one of our target groups.
After we had a better idea of the problem and brainstormed some solutions we defined our objectives: redefine the image search experience by actively including the user in a more engaging and interactive image search journey, humanize the intersection of user and image recognition API, and explore a 3D environment to browse pictures in VR.
To validate our assumptions and better understand the motivation and context of image search, my designer teammate and I set out to interview real people and conduct usability tests with current solutions. The dpa building was a great place to investigate user needs, and we had the chance to talk to several journalists and editors. Observing them taught us a lot about the workflow and thought process of one of our target groups.
Peer Grimm, Managing Photo Editor DPA
Peer Grimm, Managing Photo Editor DPA
Peer Grimm, Managing Photo Editor DPA
Peer Grimm, Managing Photo Editor DPA
Peer Grimm, Managing Photo Editor DPA
While the interviews confirmed most of our assumptions, we also realized that we did not account for the explorative aspect of searching. Finding exactly what you want might be critical, but often enough the journey to the right picture is just as valuable as finding what you had in mind. We also decided to position our product strongly towards the stock image sector, providing the most value to professionals who spend a lot of time searching pictures.
While the interviews confirmed most of our assumptions, we also realized that we did not account for the explorative aspect of searching. Finding exactly what you want might be critical, but often enough the journey to the right picture is just as valuable as finding what you had in mind. We also decided to position our product strongly towards the stock image sector, providing the most value to professionals who spend a lot of time searching pictures.
Based on those insights we experimented with different task flows and created two personas with the emphasis on JTBD and motivation over demographics.
The Explorer
Motive
Motive
Motive
Frustration
Frustration
n
n
The Hunter
Motive
Motive
Motive
Motive
Frustration
Frustration
After collecting and discussing the data we started prototyping and implementing what we learned. One of the cornerstone ideas for the interface was to enable the user to interact with the content in the most natural way - by pointing at things to refine search results rather than editing the search term. To accomplish this I mapped the available data directly on the image. To avoid overwhelming the user and distract from the image itself the info would only appear upon hovering the mouse over a region of the picture, limiting the number of simultaneously displayed data.
After collecting and discussing the data we started prototyping and implementing what we learned. One of the cornerstone ideas for the interface was to enable the user to interact with the content in the most natural way - by pointing at things to refine search results rather than editing the search term. To accomplish this I mapped the available data directly on the image. To avoid overwhelming the user and distract from the image itself the info would only appear upon hovering the mouse over a region of the picture, limiting the number of simultaneously displayed data.
Refining search results directly in the picture
I also wanted to include a way interact with unidentified content. After some investigation, I learned that the bottleneck of the Google Vision API we used (among Adobe Stock, Microsoft Face, and dpa PictureAlliance) was the information richness of an image - the more data in a picture the less chance of identifying objects. My initial thought was to reduce the sample size, so I tested the idea by running segments of the same image individually by Google Vision:
Reducing sample size to increase recognition
This process greatly increased the API's recognition. To take advantage of this, I introduced a tool for brushing over unidentified objects. The image recognition API would now only analyze the selected area, resulting in a positive id of the object. While giving the user a tool to filter for unrecognized elements, at the same time this feature could also boost the capability of image recognition API's - the more people use it the better detection gets.
Filtering unrecognized elements
The ability to quickly and naturally apply filters based on the image content also exponentially improved the quality of search results. Given current solutions users started with 2-4 keywords, possibly edited them, and spend the majority of the time scrolling through results.
In SRCH we increased the motivation and ability of users to quickly narrow down their search results by making the process more:
● intuitive (clicking instead of typing)
● playful (magic brush)
● engaging (conversational process)
In SRCH we increased the motivation and ability of users to quickly narrow down their search results by making the process more:
● intuitive (clicking instead of typing)
● playful (magic brush)
● engaging (conversational process)
Users quickly adopted the mindset of interacting with images and applied on average 8 filters, increasing the accuracy and decreasing the number of search results.
We also experimented with different ways of displaying images in VR. While the traditional grid view remains the most efficient way of displaying pictures in 2D, it is inherently limited by the size of the screen. To illustrate multiple search result clusters instead of having only one pile of pictures, we explored the upcoming realm of VR. One of the biggest advantages of virtual reality, apart from its immersive experience, is the drastically bigger surface to interact with. Unfortunately, after running first tests it became clear that the inaccuracy of available input devices resulted in an overall longer, although initially more fun experience.
Browsing in VR
After a final feedback round including a mock pitch and more usability testing we finalized the design with some quality of life features including a bookmark option for later view, an indicator for applied filters and minor look & feel tweaks. In the final pitch, we delivered a use case driven presentation displaying an engaging workflow of future image search and a google cardboard version outlining the opportunities of VR.
During the hackathon, I had the chance to talk to great people from a lot of different backgrounds which gave me the opportunity to test my ideas and influenced and challenged my design thinking from new angles. I had a really fun time working with an amazing, diverse team and enjoyed working in such an inspiring environment. Having only 48 hours to tackle a problem has its own charm and really pushes your decision making.
Our team was awarded "Best Overall" and "Best Use Of API".
Say Hi: hello@nicolaifleischhauer.com
2022 © Nicolai Fleischhauer