Unilever brings AI-assisted cooking for blind and low-vision shoppers

Unilever has partnered with the accessibility app specialist Be My Eyes (a developer of accessibility solutions apps connecting blind or low-vision people to a world network of volunteers and artificial intelligence (AI)-powered virtual assistance), to expand their use of AI technology and add powerful new tools to the cooking experience of their products.

The new partnership builds on addition of on-pack Accessible QR (AQR), developed by computer vision specialists Zappar, to Unilever’s Persil and Colman’s products in the UK last year and is part of Unilever’s global-connected pack strategy, which includes using new digital experiences and technology to evolve and differentiate the way shoppers interact with and use Unilever’s products.

Unilever is initially trialling the Be My Eyes technology in the UK through their Colman’s Singapore Noodles Meal Maker, with the intention of rolling it out across more brands in 2024. Scanning the AQR code on the front-of-pack, shoppers can now be directed to the Be My Eyes app for free and connected to either a volunteer to read and describe cooking instructions or to a virtual AI-chat bot which uses Chat GPT-4 capabilities to address recipe or cooking questions.

This is the first time the Be My Eyes AI technology has been integrated with a food product, offering an AI-assisted cooking experience at home. Unilever is working with Be My Eyes to train their volunteers and provide specific Colman’s product information and data to the Chat GPT-4 AI bot. Unilever’s customer support team will also be integrated into the Be My Eyes app experience, allowing shoppers the option to be automatically connected to the Colman’s careline team for further support if needed.

The Be My Eyes app is the only 100 per cent free app that allows users with blindness, low-vision or deaf-blindness, to receive live accessible information from volunteers, trained customer support representatives and artificial intelligence.

“We’ve accelerated digitising our packs to offer new opportunities for brand engagement and elevated shopping experience and now we’re also focusing on how we can also use digital experiences to make our products more accessible," said Rachana Dongre, Unilever's Senior Digital Engagement and Strategy Lead Nutrition & Ice Cream.

“The 285 million blind and low vision consumers in the world buy the same products as everyone else,” said Be My Eyes CEO Mike Buckley, “The difficulty in reading package ingredients and cooking instructions has long marred the cooking experience of these consumers. Unilever’s commitment to use new Zappar technology with Be My Eyes is pioneering a new era in product accessibility and disability convenience.”

Unilever will be working with Be My Eyes and Zappar to collate user feedback to optimise the technology with the aim to roll out the accessible technology to other Unilever products. Unilever’s ambition is to continue to partner with accessibility experts and apps to support wider integration of AQR technology across different products, categories and countries and to work with other businesses using this technology to share best practice.