VIRAS: Accessible Shopping

Independent shopping is a significant need for blind and visually impaired individuals. With VIRAS, the ILT Institute for Lab Automation and Mechatronics at OST is developing a technical assistance system in close collaboration with four Swiss associations for the blind, enabling visually impaired customers to shop independently in supermarkets. VIRAS stands for Visually Impaired Robot-Assisted Shopping: a self-driving shopping cart is being developed to navigate users through the store and guide them to the desired products.

High Interest in Technical Support

To assess the need, the ILT conducted a survey across Germany and Switzerland with various associations for the blind. The results clearly show a strong interest in technical support for supermarket shopping. Currently, most affected individuals avoid shopping independently, often relying on a third person. The use of the VIRAS shopping cart appeals to 87% of respondents. If the shopping cart were made available, almost 90% of visually impaired and blind individuals would shop independently.

Four percent of the Swiss population—an increasing trend—are affected by visual impairment. Given this, the project can not only contribute to inclusion but also significantly improve the quality of life for those affected. The Swiss Central Association for the Blind, the Swiss Federation of the Blind, the Swiss Caritas Action for the Blind, and the Swiss Association for the Blind and Visually Impaired support the development and realization of the VIRAS project through financial and ideological backing. The VIRAS team maintains close communication with these associations. By involving the affected group, the development is closely aligned with the needs of the target audience.

Product Recognition on Shelves

The requirements for a shopping cart for blind and visually impaired individuals are defined by the challenges they face. The most important and challenging hurdle when shopping without assistance is finding products on the shelves. Particularly with changes in the assortment and product placement, affected individuals have significant difficulties locating the desired product. Additionally, uneven lighting in supermarkets complicates orientation.

Weighing vegetables or fruits is nearly impossible, as scales cannot be operated without sufficient vision. Another difficulty is finding the checkout counter.

Shopping with VIRAS

VIRAS enables accessible shopping. The self-driving shopping cart is equipped with three stereo cameras as well as integrated speech recognition and output functions. This allows the user to be guided to the desired products without assistance from others. Along the way, the system detects obstacles or other customers and avoids them.
The stereo cameras on the cart not only serve to map the supermarket in three dimensions but also identify products and the user's hand, providing acoustic instructions to guide them to the desired product. Artificial intelligence is used to recognize the sought-after product. With this innovative approach, it is not necessary to scan the product's barcode. Products are reliably detected regardless of their orientation on the shelf. If a shelf is empty, the customer is informed.

Here's how the shopping process for blind and visually impaired individuals could look with VIRAS:

1. Locating the shopping cart in the supermarket using acoustic signals
2. Entering the shopping list via the VIRAS app
3. Navigating to the products and avoiding obstacles or people
4. Stopping the cart upon reaching the product
5. Product recognition using AI object detection
6. Acoustically naming the position of the product on the shelf
7. Detecting the user's hand
8. Providing acoustic guidance on the position of the user's hand relative to the product
9. Acoustically confirming the product name when the user selects the correct product
10. Navigating to the next product or to the checkout

Further Development for Older Adults

A prototype of the VIRAS shopping cart is currently being tested at the Migros store in Hinwil. In further development, features for inventory control are to be implemented, including the ability to check product availability in real-time. Promotional offers could also be communicated through the shopping cart.

Additionally, the user base of VIRAS is intended to be expanded. Older adults could also benefit from an autonomous shopping cart, especially if they have significant functional limitations such as reduced mobility.
To make the shopping cart usable for selected groups of older adults and to promote their autonomy while shopping, thereby enhancing their quality of life, the Institute for Aging Research at OST is conducting a needs analysis. Through a written survey, guided interviews, and focus groups, barriers to shopping are being identified. Together with older adults and the Institute for Lab Automation and Mechatronics, ideas are being developed on whether and how the functions and design of VIRAS can meet the needs of specific groups of older adults. Additionally, the potential for expanding the prototype is being examined.

>>More about VIRAS