‘ Image may contain … cat ?’- now Facebook can talk you through your photos

, , , ,

Blind users of the social network can have images described to them although we shouldnt stop captioning and tagging just yet

If you post a picture of a cat on Facebook, a bunch of people will instantly identify it as a cat but now Facebook automatically knows its a cat, too. This could be seen as another invasion of our online privacy, but for the visually impaired its unbelievably useful: with the help of Apples VoiceOver software, phones and tablets using the Facebook app can now tell you what might be in the picture. Image may contain: cat, the audio software says when I tap on the photo. Correct! So I try a puppy. Image may contain: grass, outdoor, nature, it says. Ah. It missed a huge golden retriever. Still, nice try.

The visually impaired usually rely on scenes being tagged so that screen reading software can tell them whats there. On a regular website its down to the web decorator, and on social media its down to the person posting the photo but of course people often cant be bothered. So Facebook is now automating that process by using artificial neural networks to recognise ideas within images, and in many cases its pretty accurate. Cats, snowfall, beaches, pizza and sky pose no problems, but responses is to be able to vague( a picture of some salad was described as plant, food, an elephant merely outdoors) and occasionally very wrong( an arid desert scene was outdoors, water ). For my own part, Facebook claims that it can identify at least one concept with 80% accuracy in more than half the photos it analyses.

Image recognition software is getting cleverer but still fights with things humans find easy. For instance, Facebook can be fairly sure when a photo contains one person, but is unable to differentiate between Ed Sheeran and Judi Dench, let alone name them. Imagga, a competing image recognition product, helpfully offers percentage assessments of its abilities but it is only 46% assured that a picture of Jay-Z is male, and 23% certain that hes white. So while artificial intelligence gets a grip on who is who and what is what, the responsibility for tagging and captioning our images still lies with us. Because its not always obvious to all that a cats a cat.

Read more: www.theguardian.com

Comments are closed.