Abstract: Artificial intelligence (AI) systems, when applied in practical applications, have an impact on human behaviour. On the one hand, AI provides cognitive assistance to humans, such as helping us to interpret data more efficiently and discover hidden knowledge in large data resources. On the other hand, these AI systems also affect human decision making and cognitive and socio-emotional development. In this seminar I will provide an overview of the research carried out at HUMAINT (Human Behaviour and Machine Intelligence), an interdisciplinary research project carried out at the European Commission's Joint Research Centre. The goal of the project is to study the impact of AI on human behaviour, and aims to provide evidence-based scientific support to the European policymaking process in this field. I will present our policy context, project approach and outcomes, focusing on four core applications (facial processing, automated driving, child-AI interaction and music recommendation) and connected to practical methodologies for fairness, diversity, transparency and human oversight.

Abstract: Restaurant recommender systems are designed to support restau- rant selection by assisting consumers with the information overload problem. However, despite their promises, they have been criticized of insufficient performance. Recent research in recommender systems has acknowledged the importance of personality in improving recommendation; however, limited work exploited this aspect in the restaurant domain. Similarly, the importance of user preferences in food has been known to improve recommendation but most systems explicitly ask the users for this information. In this paper, we explore the influence of personality and user preference by utilizing text in consumers' electronic word of mouth (eWOM) to predict the probability of a user enjoying a restaurant he/she had not visited before. Food preferences are extracted though a trained named-entity recognizer learned from a labelled dataset of foods, generated using a rule-based approach. The prediction of user personality is achieved through a bi-directional transformer approach with a feed-forward classification layer, due to its improved performance in similar problems over other machine learning models. The personality classification model utilizes the textual information of reviews and predicts the personality of the author. Topic modelling is used to identify additional features that characterize users' preferences and restaurants properties. All aforementioned features are used collectively to train an extreme gradient boosting tree model, which outputs the predicted user rating of restaurants. The trained model is compared against popular recommendation techniques such as nonnegative matrix factorization and single value decomposition.

Abstract: In this paper, we introduce a recommender system where it automatically captures the context of what users or guests look for and recommends a bundle of products to be added to their shopping cart. The recommendation system takes selected recipes from a user as input and recommends a shopping cart with ingredients in optimized quantities as well as any kitchen gadgets that might be necessary to efficiently prepare the recipes using neural networks. We propose a system architecture, dive deep into the individual components, and evaluate the performance of information retrieval, semantic search, and quantity optimization algorithms. Using an ensemble methodology, we attained a mean average precision of over 0.9 for ingredient and quantity recommendations. The recipe-based bundle recommender system may be used not only to improve the user's shopping experiences but also to enable and encourage them to have healthier eating habits, aiming at providing personalized product recommendations.

Abstract: All transductive recommender systems are unable to make predictions for users who were not included in the training sample due to the process of learning user-specific embeddings. In this paper, we propose a new method for replacing identity-based user embeddings in existing sequential models with interaction-based user vectors trained purely on interaction sequences. Such vectors are composed of user interactions using GRU layers with adjusted dropout and maximum item sequence length. This approach is substantially more efficient and does not require retraining when new users appear. Extensive experiments on three open-source datasets demonstrate noticeable improvement in quality metrics for the most of selected state-of-the-art sequential recommender models.

Abstract: We address the problem of location- and time-aware recommender systems where users with dynamically changing locations are interested in trending and volatile items. Unlike existing work, we do not assume a known static location of each user and derive user-locational preferences from their long-term history of implicit feedback. We propose a recommendation model that accounts for spatial, temporal, popularity and social influences, thereby assuming items tagged with a location, i.e. geotag, city or country. Key ingredients of our online method include: (1) deriving location preferences from the history, (2) learning relevant nearby locations, (3) accounting for recency and popularity jointly, and (4) combining location- and time-aware recommendations with collaborative filtering. Supported by realistic offline and online experiments on a large dataset collected from a popular newspaper, and public datasets, we find that the proposed recommender outperforms content-based and time-aware collaborative filtering approaches.

Abstract: Building a scalable and real-time recommendation system is vital for many businesses driven by time-sensitive customer feedback, such as short-videos ranking or online ads. Despite the ubiquitous adoption of production-scale deep learning frameworks like TensorFlow or PyTorch, these general-purpose frameworks fall short of business demands in recommendation scenarios for various reasons: on one hand, tweaking systems based on static parameters and dense computations for recommendation with dynamic and sparse features is detrimental to model quality; on the other hand, such frameworks are designed with batch-training stage and serving stage completely separated, preventing the model from interacting with customer feedback in real-time. These issues led us to reexamine traditional approaches and explore radically different design choices. In this paper, we present Monolith, a system tailored for online training. Our design has been driven by observations of our application workloads and production environment that reflects a marked departure from other recommendations systems. Our contributions are manifold: first, we crafted a collisionless embedding table with optimizations such as expirable embeddings and frequency filtering to reduce its memory footprint; second, we provide an production-ready online training architecture with high fault-tolerance; finally, we proved that system reliability could be traded-off for real-time learning. Monolith has successfully landed in the BytePlus Recommend product.

Abstract: Recommender systems have long grappled with optimizing user satisfaction using only implicit user feedback. Many approaches in the literature rely on complicated feedback modeling and costly user studies. We propose online recommender systems as a candidate for the recently introduced Interaction Grounded Learning (IGL) paradigm. In IGL, a learner attempts to optimize a latent reward in an environment by observing feedback with no grounding. We introduce a novel personalized variant of IGL for recommender systems that can leverage explicit and implicit user feedback to maximize user satisfaction, with no feedback signal modeling and minimal assumptions. With our empirical evaluations that include simulations as well as experiments on real product data, we demonstrate the effectiveness of IGL for recommender systems.

Abstract: Online recommender systems are increasingly prevalent given their ability to adapt to the customer's needs in real time. Nonetheless, they come with additional costs (computation, operational) and complexity (infrastructure). In this keynote, we explore when it makes sense to use an online recommender and when a batch recommender is good enough. Then, to better understand the differentiating strengths of online recommenders, we share three systems at Amazon Books that play to these strengths, high-level results, and lessons from making them work in the field.

Abstract: For industrial-scale advertising systems, prediction of ad click-through rate (CTR) is a central problem. Ad clicks constitute a significant class of user engagements and are often used as the primary signal for the usefulness of ads to users. Additionally, in cost-per-click advertising systems where advertisers are charged per click, click rate expectations feed directly into value estimation. Accordingly, CTR model development is a significant investment for most Internet advertising companies. Engineering for such problems requires many machine learning (ML) techniques suited to online learning that go well beyond traditional accuracy improvements, especially concerning efficiency, reproducibility, calibration, credit attribution. We present a case study of practical techniques deployed in a search ads CTR model at a large Internet company. This paper provides an industry case study highlighting important areas of current ML research and illustrating how impactful new ML methods are evaluated and made useful in a large-scale industrial setting.

Abstract: Amazon encompasses a large number of discrete businesses such as Retail, Advertising, Fresh, Business (B2B e-commerce), and Prime Video, most of which maintain a presence across its e-commerce website. They produce content for our customers that belong to diverse content types such as merchandising (e.g. product recommendations), product advertisements (e.g. sponsored products and display ads), program adoption banners (e.g. Amazon Fresh), and consumption (e.g. Prime Video). When customers visits a web page on the website, it triggers a content allocation process where we determine the specific content to show in regions of customer shopping experience on that web page. Content produced by the aforementioned businesses then needs to be arbitrated during this process. We present a causal bandit based framework to address the problem of content optimization in this context. The framework is responsible for fairly balancing the differing objectives and methods of these businesses, and selecting the right content to display to the customers at the right time. It does so with the goal of improving the overall site-wide customer shopping experience. In this paper, we present our content optimization framework, describe its components, demonstrate the framework's effectiveness through online randomized experiments, and share learnings from deploying and testing the framework in production.

Abstract: Providing personalized recommendations based on the dynamic sequential behaviors of users plays an important role in e-commerce platforms since it can considerably improve a user's shopping experience. Previous works apply a unified model pipeline to build recommender systems, without considering the differentiated behavior patterns and intrinsic shopping tendencies on different pages of an e-commerce website. In this paper, we focus on generating a personalized recommender system optimized to both the View Item Page and Homepage by elaborately designing strategies for data formulation and model structure. Our proposed model (PW-PRec)consists of a causal transformer encoder together with a fusion module designed for different pages, built on the basis of the classical two-tower structure. This provides the capability to capture a balanced long-short interest or diverse multiple interests of a user during their shopping journey across multiple types of pages. We have conducted experiments both on in-house datasets as well as public datasets to validate the effectiveness of our model, all showing significant improvements on Recall@k metrics compared to the commonly applied sequential models of recent years. Additionally, we built a state-of-the-art deep learning based retrieval system utilizing real-time KNN search as well as near real-time (NRT) user embedding updates to reduce the recommendation delay to a few seconds. Our online A/B test results show a big advantage compared to the previous GRU-based sequential model in production, with a 38.5% increase in purchased items due to model improvements and 107% increase in purchased items due to the engineering innovations.

Abstract: The next-basket recommendations are a well-studied problem in the academic field. As a rule, we want to predict top-K goods that might be added to the last known basket of the user based on the previous purchase history. However, the existing studies do not take into account that each user can be limited by the amount of money. Therefore, clients of online websites might be interested in the opportunity to receive personal recommendations with a given fixed cost. For example, we can take the average spending over the history of customer’s purchases as an upper bound. In this article, we consider addressing the problem of making next-basket recommendations with set total cost restrictions.