Facebook Develops Visual Q&A, App Aids Sight-Impaired to ‘See’

DUBLIN – Very soon Facebook may be able to help the visually-impaired “see”. It has developed an app that allows people with impaired vision to question and have answers read out to them related to photos on the social media site.

This was introduced by Facebook Chief Technology Officer Mike Schroepfer who was speaking in Dublin at the annual Web Summit Internet conference. Here he promoted Facebook’s latest advancements in artificial intelligence, which included this experimental photo app. It is known as Visual Q&A by Facebook and the app allows people question a picture in order to formulate an improved mental image of all that it depicts, even when they are unable to see it.

To give an example of what this app can do, let’s take a situation where someone is presented with the image of a friend’s children. The person who was presented the picture might ask where the children are or what they are doing. The app will then reply by announcing out loud that the children are in a park or that they are playing soccer.

The app is still in its developmental stage but it looks to be very promising.

Schroepfer accompanied his presentation with a blog post and through this; he highlighted several of Facebook AI Research (FAIR) team’s recent developments. These developments include improving image recognition which is designed to aid a computer in separating an image from its background or from other people in the image.

Another recent development is that they have been working on a technology which allows neural networks to answer questions like a human would by developing a short term memory of sorts. This technology is known as Memory Networks or MemNets. The computer is able to answer questions about an image by combining Memnets with image recognition.

The reasoning behind these developments? Firstly, Facebook is in the midst of testing FacebookM, which is an intelligence-driven personal assistant. Once developed it would rival Microsoft’s Cortana, Apple’s Siri, and Android’s Google Now. Secondly, Facebook aims to have its AI recognize the contents of images. This will enable the largest social network worldwide to subsequently, fine-tune the contents in your news feed.

During the presentation, Schroepfer also explained how Facebook is attempting to teach Artificial Intelligence to make predictions. He gave an example of a computer attempting to predict the chances of a stack of blocks toppling over. He even joked that much work had been done to teach AI to play Jenga.

Schroepfer, during the course of his speech, provided an update of Facebook’s next-generation technologies. Apart from the developments in artificial intelligence, Facebook’s 10-year plan also depends heavily on developing virtual reality as well as connecting the world. This is Facebook’s three-pronged future plan.

Facebook recognizes that there are many areas of the globe with no Net access. Its Aquiladrone is one way in which the company aims to connect these areas. The Aqulia drone is an autonomous plane with a wingspan of a 737 jet. It stores power from the sun. A network of Aquila drones, using the stored power, is designed to remain 60-90,000 feet in the air. They will be able to do this for three months at a time and in this way, they will beam the Internet to each other using precisely calibrated lasers.

Schroepfer even brought an Aquila engine pod onto the stage at Web Summit. It stood taller than the Facebook CTO, but according to the CTO, the pod is “lighter than a MacBook.” The full-size Aquila will be tested soon although; a small-scale version has been tested earlier this year.