Talking 7-Segment displays

Student: Joti Maes

External party: Blindenzorg Licht en Liefde vzw

Context – problem statement

Almost all household appliances are equipped with a screen. Characters are often represented using 7 segments.

Many people with a visual impairment find it difficult or impossible to use these screens.

Nowadays, smartphone operating systems, such as Android or iOS, provide screen reading software that makes it possible for people with visual impairments to read the screen, through text-to-speech or braille (TalkBack on Android or VoiceOver on iOS). That is why the smartphone is increasingly used as an aid: for example, there are apps that use the camera for object recognition, color recognition, OCR, etc. However, these apps are not able to reliably perform 7-segment displays recognition, and are also highly sensitive to the lighting conditions (e.g., reflections of sunlight or lamps just hitting the 7-segment display).

There are already several open-source projects that have been specially developed to recognize characters in a 7-segment display. For example, SegoDec gives more reliable results than ssocr.

However, these tools still assume ideal conditions (e.g. the user must select the crop area himself, choose his own filters, etc.).

Objectives

In this master’s thesis, the student will develop an app that can read 7-segment displays fully automatically or with minimal configuration per device.

This includes:

• Check which algorithm is best suited to read 7-segment displays. This may or may not be based on existing software and may or may not be based on AI techniques.
• Consider how to deal with practical rather than ideal circumstances. This could include automatically (AI-based or not) finding the 7-segment display in an image, automatically adjusting the shutter speed to tackle overexposure or instructing the user on how to reposition the smartphone so that there are no adverse effects such as reflections in the image, etc.
• Integrate everything into one (ideally cross-platform) app
• Evaluate whether it is useful (battery use) and/or feasible (additional delay) to offload certain image processing functions to the cloud.

In case the master’s thesis is successful, consideration will be given to making the app open source available to the target audience.

Volunteer as tester!

In case you are interested in participating in testing the app, please volunteer by sending an email to empadis. As development still needs to start, the test phase should not be expected immediately; having access to willing early testers is more than welcome!