Scrutability of Proactive Intelligent Personal Assistants

Abstract

In just over a decade, Intelligent Personal Assistants (IPAs) have become ubiquitous, with Google Assistant, Apple Siri, and Amazon Alexa now integral components of smartphones, smart speakers, dashboards, and other internet-connected devices. More recently, advances in artificial intelligence have catalysed a transformation of IPAs from mainly command-driven programs assisting the user with discrete, straightforward tasks to systems that attempt to parse the user's intent, act proactively, and delegate in communications with others.

Evolving capabilities amplified concerns regarding trust calibration and privacy, as IPAs remained inscrutable, by not facilitating active user investigation of the underlying models and control of the resulting personalisation. The concept of scrutability was initially defined by Judy Kay [1999] with its importance widely recognised in user modelling research. Still, the concept is rooted in legacy systems, such as hypertext editors and single-domain recommender systems. This dissertation explores how the principles of scrutability may be extended to facilitate effort-efficient user investigation and control of models powering IPAs that operate proactively, provide personalised advice, and interact with others on the user's behalf.

Adopting a human-centric approach, we reanalysed data from studies on the causes of non-use of IPAs and expectations of future IPAs to identify user-desired interaction principles and use cases. Following the reanalyses and a literature review, we focused our further exploration on time management assistance, a complex and paradigmatic yet specific problem space. We then organised a multidisciplinary expert discussion to outline a pertinent IPA interaction framework, related modelling and interaction design challenges, and potential new scrutability principles. The insights from the expert discussion were then embedded in Scrutable Intelligent Personal Assistant for Time Management (SIPA4TM), our scrutable-by-design calendaring assistant that served as the basis for our user study with thinking-aloud interactions and focus group discussions, which has led to the final set of scrutability principles and design guidelines.

The first major contribution of the dissertation is a human-assistant interaction paradigm that links scrutability to user trust through prospect calibration, extending the meaning of scrutability to include supporting users in understanding and controlling how IPA features may affect their self-concept, social relations, and required effort. Second, we contribute a set of principles and design guidelines that operationalise scrutability for proactive IPAs as an integrated system-level capacity, rather than a set of model-level architectural requirements. The first minor contribution is the design and implementation of SIPA4TM, developed to enable potential expansion and reuse. Finally, in this dissertation, we contribute a consolidated landscape of user and expert desiderata and concerns regarding personalised, proactive, and delegative time management assistants..