Research Article: Experienced mental workload, perception of usability, their interaction and impact on task performance

Date Published: August 1, 2018

Publisher: Public Library of Science

Author(s): Luca Longo, Stefano Federici.

http://doi.org/10.1371/journal.pone.0199661

Abstract

Past research in HCI has generated a number of procedures for assessing the usability of interacting systems. In these procedures there is a tendency to omit characteristics of the users, aspects of the context and peculiarities of the tasks. Building a cohesive model that incorporates these features is not obvious. A construct greatly invoked in Human Factors is human Mental Workload. Its assessment is fundamental for predicting human performance. Despite the several uses of Usability and Mental Workload, not much has been done to explore their relationship. This empirical research focused on I) the investigation of such a relationship and II) the investigation of the impact of the two constructs on human performance. A user study was carried out with participants executing a set of information-seeking tasks over three popular web-sites. A deep correlation analysis of usability and mental workload, by task, by user and by classes of objective task performance was done (I). A number of Supervised Machine Learning techniques based upon different learning strategy were employed for building models aimed at predicting classes of task performance (II). Findings strongly suggests that usability and mental workload are two non overlapping constructs and they can be jointly employed to greatly improve the prediction of human performance.

Partial Text

In recent years, with the advent of the Internet and the explosion of web-based system development, the construct of usability has been invoked in many different ways. Research in the past decades has generated a number of procedures for assessing the usability of interactive systems. It is believed it is a multi-dimensional construct, encompassing several features. Frequently, for example, during usability inspection, there is a tendency to omit characteristics of the users, aspects of the context and peculiarities of the tasks. This tendency is reasonable and it justified by the complexity of usability as a construct and a lack of a model that unifies all of these factors. Taking into account features of users is fundamental for the User Modeling community in order to build systems that fit the specific background, knowledge and objectives of users [1–3]. Similarly, considering the context of use has a significant influence in the inference of meaningful assessments of usability [4–7]. Additionally, during the usability inspection process, accounting for the demands of the underlying task is essential for predicting user experience and informing the design of interactive systems [8, 9]. Building a cohesive model that incorporate user, context and task-specific factors is not obvious. Usability inspection should be accompanied by the assessment of one of all of these factors when possible. Beside Usability, another construct has a long research history in the field of Human Factors: the construct of human mental workload (MWL) [10, 11]. This is often referred to as cognitive load and I believe this can significantly contribute to the goal of informing interaction and web-design. MWL, with roots in Psychology, has been mainly adopted within the fields of Ergonomics with several application in the transportation and nuclear industries [12]. Its assessment is fundamental for predicting performance, which in turn is key for describing user experience and engagement. The link usability and mental workload is nowadays under explored. A few studies have attempted to apply the construct of MWL to explain usability [13–18]. Despite this weak interest, not much has yet been done to explore their relationship empirically. The aim of this research is to empirically investigate the relationship between subjective perception of usability and mental workload with a particular focus on their impact on objective user performance, this being assessed through observation of tangible facts. Fig 1 depicts the main constructs employed in this research study and their relationship.

Widely employed in the broader field of HCI, usability and mental workload are two constructs from the discipline of Human Factor, with no limpid and broad definitions. Since their inception, there has been an intense debate about their assessment and measurement [19–21]. Although multidimensional and complex, their usefulness for describing the user experience and informing interaction, interface and system design is beyond doubt.

Table 4 and Fig 4 show the means and the standard deviations of the usability and the mental workload scores for each information-seeking task (Table A5 in S1 Appendix).

The results obtained in the previous sections are summarised and they are aligned to the research hypotheses previously set in Tables 2 and 3. Findings are critically discussed, including information about their statistical significance. Eventually, their implication to the broader field of Human-Computer Interaction is described.

This study attempted to investigate the correlation between the perception of usability and the mental workload imposed by typical tasks executed over three popular web-sites: Youtube, Wikipedia and Google. A literature review on prominent definitions of usability and mental workload was presented, with a particular focus on the latter construct. A well known subjective instrument for assessing usability —the System Usability Scale —and two subjective mental workload assessment procedures —the NASA Task Load Index, and the Workload Profile —have been employed in a primary research study involving 46 subjects. The perception of the usability of the interfaces these subjected interacted upon and the mental workload they have experienced while executing a selection of tasks, over selected interfaces, does not seem to correlate. The obtained empirical evidence strongly supports that usability and mental workload are two non overlapping constructs. Findings suggest that these two constructs can be jointly employed to improve the prediction of human performance, thus enhancing the description of user experience. The implications to the broader field of Human-Computer Interaction include the provision of mental workload as an important concept relevant for the design of interactive technologies better aligned to the human mental limited capacities and that can maximise human performance.

 

Source:

http://doi.org/10.1371/journal.pone.0199661