It is now extremely easy to identify just about anything on mobile devices through image recognition apps. From plants to insects, from foreign words to ancient texts, from rare paintings to state of the art buildings, from fingerprints to faces, there will be an app to help you identify what you are looking at. 

Many of these apps improve their accuracy the more they are used as they learn from users’ confirmation of the correctness of their matches. Perhaps what is different about some of these apps is that while they exist on smartphones, there is often not an equivalent on a computer.

Recognition apps do have many potential applications for education. In the natural sciences, the apps can help students identify species, in geography the apps can help identify places, in art history the apps can help identify drawings, in languages the apps can translate texts. They can also make learning resources more accessible. For the partially-sighted they can read texts, for those with mobility issues they can write texts, for those with impaired hearing they can provide text versions of audio.

But of course, identification apps do not only help the educational disciplines, they also provide source material and an area for research. There are plenty of philosophical questions that can be posed about the possibilities identification apps give rise to, such as surveillance and control. There are cultural shifts that such technologies create and political questions about what will be allowed and disallowed. But leaving aside the material for study they provide for one moment, in this post we look at the apps themselves and the technical possibilities they allow.

Audio recognition

Audio recognition technologies listen for sounds, and make informed guesses about what the smartphone is listening to. There have been different uses for education. For example, the technologies can be used for identifying similarities in sound tones and patterns. They can also be used to allow people to create text without typing.

Music

Perhaps the first recognition apps that I was familiar with were the music recognition apps. This technology listens to the songs that are being played and tells you their name and the artists. An early cross-platform player in this technology was Shazam, however now, many Android and iOS devices have built-in song recognition software. It can be used in teaching when asking the students to give feedback on a mood, song genre or even what instruments are playing using the similar songs features.

Speech

There are now many technologies that will translate speech into text. For example, Google is currently converting the words that I am speaking into text. I can tell the app to add a full stop or delete a word through text commands. Microsoft and many other products do the same. There are apps that work for every device (phone, tablet or computer) so long as you have an audio input. They are very useful if  you are finding it difficult to type.

Bird song

Audio recognition is not only for human created noises, it can also be used to capture any noise. For example, there are bird call and song recognition apps, which can help you identify the bird species that you can hear in the trees. My favourite Android app is BirdNet. It could be used in teaching on either sound pattern matching tasks or identification field trips. 

Image recognition

Image recognition technology identifies shapes within images and makes educated guesses about what the image might be. There are many apps for anything from famous pictures to buildings. In an educational setting these apps can be used for practicing data collection and classification. 

General images

There are a number of general apps that attempt to recognise whatever the image type. The most well known mobile app is Google Lens. There is also Microsoft Lens and Apple have suggested that they will also be releasing a similar app for iOS soon. Additionally, Google image search allows you to photograph or upload an image into the search field and search by the image. 

Species

Of greatest interest to me at the moment are the possibilities of image recognition for species identification. Plants, insects and butterflies all have apps that allow their identification. I no longer need to walk through the woods wondering what the plants and animals are. My favourite Android app is PlantNet, but another popular cross platform app is iNaturalist. 

Text recognition

Text recognition is useful when you would like to be read to, the text is obscured from you or not in your language. Many of the apps can recognise texts in multiple languages and can translate them.

There are many text recognition apps will read the text to you. This can be for your pleasure and can be useful if you are partially sighted and can not read the text without help. 

Text recognition apps are useful if a reading is not in your native language. The app will translate text from one language into another. An example of this is the Translate function in Chrome browser. These  translations have to be used with caution as they will not always be accurate as a result of technological limitations, different accents and colloquialisms.

Conclusion

There are many uses for recognition apps. If you would like to use them in your teaching please contact tel@sussex.ac.uk. Sign up to our identification app event at 2pm on Wednesday the 10th November as part of the library’s Digital Discovery Week.