Integration of LLM in Expiration Date Scanning for Visually Impaired People

Abstract

In this study, the authors explore an approach to detect expiration dates of food products using a live feed stream and the integration with Large Language Models in order to improve accessibility for visually impaired people. The main objective is to enhance their capacity to engage in common tasks like grocery shopping autonomously. The novelty of this research lies in employing Meta LLAMA 2, a large language model, and experimenting with both traditional and a new OCR solution to find the expiration date using image processing. This approach offers audio information about whether the product has expired or when it will expire, helping in shopping and product recognition for visually challenged customers. The proposed solution consists of optical character recognition, mainly the EasyOCR library, fine-tuned on cropped images containing only the expiration dates and a validation phase that filters and checks the extracted data.