Research Article: Heuristics to Evaluate Interactive Systems for Children with Autism Spectrum Disorder (ASD)

Date Published: July 21, 2015

Publisher: Public Library of Science

Author(s): Kamran Khowaja, Siti Salwah Salim, Katsuaki Suzuki.

http://doi.org/10.1371/journal.pone.0132187

Abstract

In this paper, we adapted and expanded a set of guidelines, also known as heuristics, to evaluate the usability of software to now be appropriate for software aimed at children with autism spectrum disorder (ASD). We started from the heuristics developed by Nielsen in 1990 and developed a modified set of 15 heuristics. The first 5 heuristics of this set are the same as those of the original Nielsen set, the next 5 heuristics are improved versions of Nielsen’s, whereas the last 5 heuristics are new. We present two evaluation studies of our new heuristics. In the first, two groups compared Nielsen’s set with the modified set of heuristics, with each group evaluating two interactive systems. The Nielsen’s heuristics were assigned to the control group while the experimental group was given the modified set of heuristics, and a statistical analysis was conducted to determine the effectiveness of the modified set, the contribution of 5 new heuristics and the impact of 5 improved heuristics. The results show that the modified set is significantly more effective than the original, and we found a significant difference between the five improved heuristics and their corresponding heuristics in the original set. The five new heuristics are effective in problem identification using the modified set. The second study was conducted using a system which was developed to ascertain if the modified set was effective at identifying usability problems that could be fixed before the release of software. The post-study analysis revealed that the majority of the usability problems identified by the experts were fixed in the updated version of the system.

Partial Text

Autism is one of the five disorders under the umbrella of autism spectrum disorder (ASD), which includes autism, Asperger syndrome, childhood disintegrative disorder, Rett syndrome and pervasive development disorder-not otherwise specified (PDD-NOS). It is a neurological disorder characterised by impairment in social communication and restricted or repetitive behaviour, the symptoms of which appear during the first three years of a child’s life, and the impairment will take its toll during the later course of life. The behavioural changes caused by ASD may vary from child to child: one may be very verbal, bright and enthusiastic, while another may be non-verbal and intellectually challenged [1].

In this research, we selected the existing guidelines in designing the interactive systems for children with ASD and used the heuristics of Nielsen to create a specialised set of heuristics for the evaluation of systems for these children. The user interface of an interactive system for children with ASD is important in the overall success of the system; the review of guidelines can reveal various guidelines which play a vital role in the design of user interface for these children. The key processes in this research consist of compilation of guidelines, grouping of similar guidelines, formation of a modified set of heuristics and its evaluation as shown in Fig 1. Each of these processes is described as follows.

This section describes the participants and the recruitment process, instrument used in the study, study protocol, inter-rater reliability (IRR) test and the results of this expert review study.

This section describes the participants recruited, instruments used, protocol used, analysis of data, followed by presentation and discussion of results.

This section describes participants involved, instruments used, study protocol, followed by presentation and discussion of results.

In this research, we adapted and improved the original set of heuristics by Nielsen and developed a modified set of 15 heuristics based on literature review on guidelines of interactive systems for children with ASD. When comparing the evaluation results of the modified set with those of the original set in Study 1, the analysis shows the following details:
Five new heuristics effectively contributed to the success of the experimental group in identifying problems.Responsiveness heuristic was violated most frequently among the five new heuristics.Five improved heuristics of the modified set helped in finding more usability problems than the corresponding heuristics of Nielsen. There is a significant difference in effectiveness between the five improved heuristics and their corresponding heuristics in the original set.Consistency, minimalist and control heuristics were violated more frequently than others using the 5 same and 5 improved heuristics of the modified set.Overall, the modified set is significantly more effective than the original set.

 

Source:

http://doi.org/10.1371/journal.pone.0132187