Some individuals who use AAC need alternative access solutions to AAC technology (eye tracking, head tracking, switch-scanning). Despite the wide range of access options available, some individuals continue to struggle with consistent and efficient access (Fager et al., 2019; Koch Fager et al., 2019). To date, AAC technologies do not automatically adapt to individuals’ motor abilities, limiting long-term and consistent access. Many AAC devices incorporate generic keyboard interfaces (e.g. QWERTY) that might be familiar but may not be the most accessible or efficient to use given differing physical access abilities. This presentation provides an overview of past and current work related to the development and clinical evaluation of a personalized keyboard interface that is personalized to the unique physical movements of the individual using the system.
Approach:
This multi-phase NIH-funded effort (R43DC018437- PI: Contessa; R44DC018437- PIs: De Luca, Vojtech) focused on the development of an adaptive AAC system that accommodates unique motor abilities. The specific approach utilizes sEMG to detect muscle movements and IMUs to detect body movement for cursor control. Movement patterns were characterized using a modified Fitt’s Law point-select task to estimate user-specific motor parameters (movement time, direction, and efficiency) to generate personalized keyboard layouts via a computational algorithm.
Phase I Summary Results:
The first phase of this work (described in Mitchell et al., 2022) examined the feasibility of this approach with 3 adults with motor impairments. Performance on target acquisition and spelling to dictation tasks were compared across the 1) their personalized keyboard, 2) a generically optimized keyboard, and 3) a standard QWERTY keyboard layout. Initial results demonstrated that all participants experienced improvements in accuracy and information transfer rates for the personalized keyboard compared to generically optimized and QWERTY.
Phase II Method:
The second phase of this work (currently underway) is focusing on clinically evaluating the approach with a range of individuals who use AAC and assistive technology to support face-to-face and written communication. The clinical evaluation includes completing a brief calibration task to build a personalized keyboard interface access via head tracking and a period of familiarization using the personalized keyboard via head tracking. After familiarization is complete, an alternating treatment design will evaluate the difference between the personalized keyboard with the subject’s current/standard AAC method during conversational tasks. Performance metrics will include rate, accuracy, linguistic complexity of utterances, turns, as well as perceptual measures of workload from the NASA TLX.
Discussion:
This presentation will provide a brief overview of current work focused on designing optimized/personalized interfaces based upon the unique movement capabilities of the individual using the system. Along with the preliminary results, a discussion of the need for personalized and adaptive approaches across broader ranges of AAC and assistive technologies will included.
Acknowledgements:
The authors of this presentation proposal (Drs. Fager & Gormley) serve as clinical evaluators for this NIH funded project and are provided a subcontract to lead the clinical evaluation on these grants awarded to Altec, Inc. The grant PIs from Altec, Inc. include Paola Contessa, Gianluca De Luca, and Jennifer Vojtech. The current technical team on the project at Altec, Inc. includes John Chiodini, Bhwana Shiwani, Joshua Cline, Serge Roy, and Laura Raiff.
Reference:
Mitchell, C. L., Cler, G. J., Fager, S. K., Contessa, P., Roy, S. H., De Luca, G., Kline, J. C., & Vojtech, J. M. (2022). Ability-based Keyboards for Augmentative and Alternative Communication: Understanding How Individuals’ Movement Patterns Translate to More Efficient Keyboards. CHI ’22 Extended Abstracts. https://doi.org/10.1145/3491101.3519845
