A Hyper-personalized, Context-aware Café Recommendation Mobile Application Integrating Real-time Environmental Sensing and Augmented Reality

Abstract

This study aims to develop an innovative recommendation system that provides a tailored café experience by integrating users’ nuanced preferences with a real-time environmental context. With the recent surge in domestic coffee consumption, the café market has reached a saturation point, leading consumers to seek spaces that match a specific atmosphere or purpose beyond merely consuming beverages. Existing recommendation systems, which primarily rely on past ratings or static information, have shown limitations in meeting these dynamic and multidimensional demands. To overcome these limitations, this paper proposes a novel framework that fuses hyper-personalization, context-aware recommendation, acoustic scene classification (ASC), passive crowd density estimation, and location-based augmented reality (AR). The proposed system utilizes a large language model (LLM) to extract abstract atmospheric characteristics such as “cozy,” “vibrant,” or “suitable for work” from unstructured text data collected from social media and review platforms. Simultaneously, it quantifies the actual environment of a space by analyzing real-time data collected through in-store sensors (or user devices). Specifically, ASC technology identifies the qualitative characteristics of sound – such as conversations, background music, and machine noise – going beyond simple noise levels. Passive detection of smartphone Wi-Fi probe signals accurately estimates indoor crowd density without infringing on personal privacy. This multi-modal data is combined with user profiles to generate a list of recommendations optimized for each individual. Finally, users can have an immersive exploration experience through a location-based AR interface, visually confirming recommended cafés, friends’ reviews, and personalized notes overlaid on their real-world surroundings. Through the synergistic combination of these advanced technologies, this research presents the design and implementation potential of a system that shifts the paradigm of physical space recommendation and provides a truly experience-centric service.