ExerG – an exergame-based training device for the rehabilitation of older adults: a functional model usability study

Abstract

Background

Exergames are interactive technology-based exercise programs. By combining physical and cognitive training components, they aim to preserve independence in older adults and reduce their risk of falling. This study explored whether primary end users (PEU, healthy older adults and patients with neurological and geriatric diagnoses) and secondary end users (SEU, health professionals) evaluated the ExerG functional model to be usable, providing a positive experience and therefore acceptable.

Methods

We conducted a multi-methods study using several assessments to quantify usability and enjoyment outcomes, along with semi-structured interviews to gain an in-depth understanding of the users’ experiences. Descriptive statistics were used for quantitative outcome measures. For qualitative data, a thematic analysis (TA) using an inductive, data-driven approach was carried out to develop themes for each user group.

Results

We interviewed 20 PEUs (13 healthy older adults, 7 patients) and 22 SEUs at two rehabilitation centres in Austria and Switzerland. Users' scores of over 70 on the System Usability Scale denoted good usability. On the Physical Activity Enjoyment Scale-16, both PEU groups rated the ExerG highly. Our TA approach identified four themes per user group. Themes from both PEU groups confirmed their enjoyment of training with the ExerG, however more variety and greater challenges were requested. Whilst the patient group appreciated the security given by the harness system, the healthy older adults reported feeling restricted. SEU themes reflected their approval of this novel training device, although a desire for increased difficulty and more individualisation was expressed. Clear instructions and an easy-to-use harness system were acknowledged and useful feedback for the developers emerged.

Conclusions

The ExerG is usable, offers a positive experience, and can therefore be regarded as an acceptable solution for the combined physical and mental training of older adults. Our findings contribute to the ongoing development of the ExerG, which will be a welcome addition to current training options for this target group. Further research is needed to confirm its effectiveness in preserving or improving functional independence in daily life and reducing the risk of falling.

Background

The United Nations predict that over 65-year-olds will represent 15.9% of the world population in 2050, increasing from 9.3% in 2020 [1]. In Europe, this age group is expected to account for 28.1% rising from 19.1%, with its population of over 80-year-olds almost doubling from 5.3% to 10.1% in the same period [1].

Falls are one area of concern, with 28-35% of older adults (OAs) aged 65 and over falling each year [2], leading to long-term health issues, with psychological and social consequences that incur substantial increases in health care costs [3]. A large European survey has shown that high fall rates are not only associated with a loss in social participation, but also increased caregiver burden [4]. Another concern is the prevalence of non-communicable diseases , with just under 23% of disability-adjusted life years attributed to the top three (ischemic heart disease, stroke, and diabetes mellitus) in this age group according to the Global Burden of Disease study [5].

The World Health Organisation has issued ‘Global recommendations on physical activity for health [6] specifying that for over 65-year-olds multicomponent physical activity improves functional and cognitive health, unfortunately 36.8% of adults in high income countries do not meet these guidelines [7]. The risk factor attributions of “low physical activity” for the aforementioned non-communicable diseases are estimated at 7.9% (ischemic heart disease), 4.5% (stroke), and 12.6% (diabetes mellitus)in Western Europe [5] and links have also been established between sedentary lifestyles and cognitive decline [8]. Integrating challenging cognitive exercises into physical activity programmes for OAs can also contribute to improving their cognitive health [9], whilst physiotherapy interventions to reduce the risk of falling have long proven their effectiveness [10].

Research is now focussing on a new approach to motivate older adults to train for both physical and cognitive health benefits, with so-called exergames, which are interactive, exercise-based videogames [11]. In their systematic review, Zheng et al. found exergames improved balance and mobility function of frail elderly [12]. Lord and Close reported an effective improvement in walking, balance, fall-rate, and cognition being achieved with exergames in OAs with and without disabilities [11]. By combining cognitive and physical activities in exergames, it is possible to simulate processes of daily activities where environmental stimuli are processed and translated into specific movements and the positive effects of the physical and cognitive stimulation seem to synergistically interact generating additive effects [13]. In their systematic review, Kappen et al. reported on the use of exergames for OAs for both cognitive and physical training, rehabilitation and psychosocial wellness and they emphasise the “growing importance” of this field in human-computer interaction [14].

Even though literature is growing on the benefits of using exergames, the evidence on usability aspects is contradictory [15]. Although some studies have found exergames to be an enjoyable, effective method of training for older adults [16,17,18], others obtained mixed responses concerning perceived ease-of-use when investigating the challenges and barriers to using exergames for the elderly [19]. Pyae et al. considered elderly-user-friendly instructions and simple game graphics important, in addition to fun factors [20]. Conversely, Chen et al. reported the perception of use as being more important than fun features [21].

With these elements in mind, a new exergame training device for the elderly - the ExerG - is being developed by the ExerGetic project. The goal of this project is the adaptation and further development of the existing ExerCube fitness system [22] for use in rehabilitation with older adults. The ExerCube combines holistic training experiences for physical, cognitive, and mental fitness with immersive gaming scenarios. As recommended by Nawaz et al., the development of the ExerG has involved older adults from the beginning [23]. Representatives of both end-user groups in two study centres in Austria and Switzerland were involved in an iterative design process by testing both the software and hardware of the functional model and providing valuable feedback. This information will be used to adapt the functional model of the ExerG, which is still in an early stage of development, to ensure it meets the end users’ needs and expectations.

The objective of this usability study was to assess the usability of the ExerG among primary end users (PEUs, including healthy older adults and patients with neurological and geriatric diagnoses) and secondary end users (SEUs, such as health professionals), focusing on their evaluation of its usability, user experience, and overall acceptability.

Methods

Study design

A multi-methods convergent study design using the Rapid Iterative Testing and Evaluation (RITE) method was implemented [24]. The RITE method of testing, enabling usability issues to be identified and solved quickly, promotes a highly efficient optimisation process that benefits the end users [24]. The existing ExerCube [25] fitness exergame measures 3.50 metres (width open end), 2.30 metres (width front wall), 2.80 metres (height) and 2.60 metres (straight line depth). It comes with three projectors, a sound system, and a customised whole-body motion tracking system, which users wear on their wrists and ankles (HTC Vive, Slough, UK). When working out with the ExerCube, the user is surrounded by three padded walls, which serve as projection screens. The ExerG functional model, based on the ExerCube, has been equipped with a harness system to support elderly users (Figure 1).

Fig. 1

Study team member in the ExerG

In order to evaluate the newly developed soft- and hardware of the ExerG functional model with regard to usability, training experience and acceptance for PEUs and SEUs, both quantitative (questionnaires and rating scales) and qualitative instruments (semi-structured interviews) were used. The study procedure is shown in Figure 2.

Fig. 2

Overview of study procedure. CRF = Case Report Form, T = Test Time, PACES-16 = Physical Activity Enjoyment Scale-16, BORG-CR10 = Borg Category-Ratio scale, MERS = Mental Effort Rating Scale, SAM = Self-Assessment Manikin, SUS = System Usability Scale

Sampling, eligibility criteria and recruitment

Primary end users

At the Clinic for Rehabilitation Münster (RZM) in Austria, convenience sampling was used to recruit healthy volunteers, aged 65 years and over, in nearby communities. Some snowball sampling occurred with older adults asking friends to join.

Convenience sampling was also used to recruit elderly in- and outpatients of a second Rehabilitation clinic (Reha Rheinfelden, RHF, Switzerland). If they passed the first screening, patients were approached by a member of the research team to explain details about the project. In case patients were interested in participating, it was checked that they fully met the eligibility criteria (Table 1) using the centre’s medical records and asking the participant the appropriate questions. In the healthy PEUs in RZM, the Everyday Fitness Test (EFT) was used to confirm sufficient cardiovascular fitness [26] and all PEUs were screened with the Mini Mental State Examination (MMSE) to verify overall cognitive function [27]. Patients received all study-related information in writing (patient information sheet) and it was guaranteed that participants would not miss any therapy sessions due to participation in the study.

Table 1 Eligibility criteria

After having been informed about the study procedure, the participants had at least one day to decide whether or not they wished to participate. All participants provided their written informed consent. Table 1 gives an overview of the eligibility criteria. The main differences between the two centres were due to the nature of the participants (healthy older adults versus patients).

Secondary end users

For the recruitment of SEUs, emails were sent to all rehabilitation experts (physiotherapists, occupational therapists, sports scientists, and psychologists) working at the rehabilitation centres inviting them to contact their research department if they were interested in participating. Again, convenience sampling was used, enrolling the therapists in the order that replies were received.

Sample size

The sample size was determined based on a study showing that with 20 users, the mean probability of detecting all usability problems is 98.4%, with a reported 95.0% minimum detection [30].

By combining the results from the two study centres, complete sets of data were collected from 20 PEUs and 22 SEUs. Although there were two different groups of PEUs (patients in RHF, healthy older adults in RZM) data saturation was deemed to have been achieved in each group after no new data and no new themes were observed [31] and due to the homogeneity of the participants and the scope of enquiry limited to the ExerG [32].

Data collection and management

Demographic data regarding, for example, age, use of mobility aids and recent falls for PEUs, and profession, level of education and work experience for SEUs, were collected. In addition, any previous experience with technology-assisted training was recorded for both user groups in RHF.

The evaluation sessions were video-recorded to enable game designers to observe patient-system interaction and an observation sheet was used for each session. These were then uploaded to the encrypted project database to provide access for the game designers.

Recorded interview data were transcribed verbatim and anonymised by a researcher of the respective rehabilitation centres or a professional transcription service (Transkripto, Rotterdam, Netherlands). In order to transfer the recorded interview files to Transkripto, the company signed a non-disclosure agreement, and a Secure Sockets Layer computer network guaranteed the security of the data transfer.

Study schedule

PEUs received three evaluation sessions and SEUs conducted two sessions with patients using the ExerG functional model. Quantitative data was collected using questionnaires and rating scales, along with qualitative data via semi-structured interviews to gain more insight into usability, user experience and user acceptance.

Procedure of the evaluation sessions for primary end users

PEUs took part in three testing sessions (T 1-3) within a two-week period (T1: familiarisation with the system; T1-3: completing questionnaires and rating scales; T3: additional semi-structured interview; see supporting information for the PEU interview guide). During every evaluation session, PEUs were able to test both the hard- and software of the ExerG functional model. All game levels required a combination of cognitive and physical skills, with the software providing easy, medium, and hard levels for each activity. Every PEU started at the easy level and then progressed depending on their individual possibilities. They were not specifically tested to determine which difficulty level should be used for them. The decision to increase the level during gameplay was made by study personnel on an individual basis, considering factors such as cognitive strain and physical demands to prevent overload. The study personnel did not include members of the software and hardware team.

The study participants tested various game scenarios, each depicting a natural environment. There was the task of 'Apple picking', in which the hands or arms should be moved towards the randomly appearing apples to the left, right, or in front of them on a virtual tree, 'Walking' through tall grass, attempting to lift each foot quite high, 'Rowing', where the arms had to be moved accordingly to propel a virtual boat forward and 'Finding symbols', where a symbol displayed on the screen among all the symbols placed on the ground should be located to stand there as quickly as possible. The difficulty level, for example in 'apple picking', referred to the frequency of appearing apples that had to be virtually collected.

Procedure of the evaluation sessions for therapists

For SEUs, there were two evaluation sessions: T1 and T2 comprised setting up the system, fitting the harness on a member of the project team (in the role of a patient) and then conducting a training session as therapist using the ExerG functional model followed by completion of the SUS and an additional semi-structured interview at T2; see supporting information for the SEU interview guide. In order to gain a patient perspective, the SEUs at RHF were additionally invited to a prior familiarisation session (T0) with the harness system and exergame software. All SEU participants are employed as therapists, including the sport scientist.

Outcomes and outcome measures

Regarding usability, the primary outcome measures were the System Usability Scale for SEUs, along with the dominance dimension of the Self-Assessment Manikin (SAM) for PEUs. Concerning whether the ExerG provided PEUs with a positive experience, the primary outcome measures were the Physical Activity Enjoyment Scale [33,34,35] and the pleasure dimension of the SAM. Detailed information on each of the primary measures is given below.

System Usability Scale (SUS)

The original System Usability Scale (SUS) instrument by Brooke in 1995 is composed of 10 statements that are scored on a 5-point Likert scale (from 1 “strongly disagree” to 5 “strongly agree”) [36]. Final scores for the SUS can range from 0 to 100, where higher scores indicate better usability [37]. SUS scoring is achieved using the system given in the original paper [36] including reversing item scores for negatively-worded items [38]. According to Bangor et al., raw SUS scores above 70 identify systems that are described as good and demonstrate an acceptable usability. Scores below 50 are deemed unacceptable, whereas scores of 85 or above are reserved for systems with excellent usability [37]. The SUS demonstrated a high level of internal consistency (Cronbach’s alpha of 0.91) and concurrent validity (r=0.81 concerning the rating of user friendliness) [37].

Physical Activity Enjoyment Scale (PACES)

To evaluate the level of enjoyment of physical activity in the ExerG, the German version of the Physical Activity Enjoyment Scale (PACES) [34] was used [39]. In this modified version, 16 items were rated on a 5-point Likert scale from 1 (completely disagree) to 5 (totally agree). Reverse scoring was necessary for the seven negatively-worded items, before summing the scores. A range of 16-80 was possible, whereby the higher the score, the higher the level of enjoyment. Jekauc et al. concluded that PACES is a reliable, valid and invariant measure of physical activity enjoyment to be used in German-speaking adults reporting excellent internal consistency (Cronbach’s α=0.94). Test-retest reliability was evaluated as good with an Intraclass Correlation Coefficient (ICC) of 0.76) [34].

Self-Assessment Manikin (SAM)

This is a picture-orientated questionnaire designed to measure three central features of an emotional response (pleasure/valence, arousal and perceptions of dominance/control) using a 9-point Likert [40]. Due to its brevity, the SAM can be used to capture emotional responses to a wide array of emotion elicitation methods [40]. For older adults, the SAM demonstrated high reliability (Cronbach's alpha of 0.82 for valence and α=0.98 for arousal) [41].

Borg Category-Ratio scale (CR10)

The Borg CR10, a Category­Ratio (CR) scale was used to measure individual’s effort and exertion [42]. This scale ranges from 0 “no exertion at all” up to 10 “maximal exertion”. Its reliability was found to be high (r=0.90) [43].

Mental Effort Rating Scale (MERS)

Mental effort was defined by Paas & Merriënboer [44] as the total amount of controlled cognitive processing in which a subject is engaged. The MERS scale is easy to use with participants reporting their perceived mental effort using a 9-point Likert scale ranging from 1 “very, very low mental effort” to 9 “very, very high mental effort” [44]. Excellent test-retest reliability has been reported (Cronbach's alpha 0.90) for the MERS [44].

Semi-structured interviews

Semi-structured interviews were carried out using an interview guide (Additional File 2) with open-ended questions for the two end-user groups. The interview guide was developed based on various models and existing questionnaires, which have been previously used in the domain of Human–Computer-Interaction, such as the Game Flow Model [45] and the Dual Flow Model specifically for exergames [46].

The PEU interview guide had general questions to begin, before asking questions on the patients’ perceptions of the actual games, game control, the harness system, focus, motivation, and training. Although the SEU interview guide covered most of the same topics, the objective was to gain insights from the therapists’ perspective with further input on, for example, game adjustments, as well as their evaluation of the ExerG training content.

Data analysis

Quantitative analyses

Descriptive statistics were used for the demographic variables and usability outcome measures. Raw counts (absolute and relative frequencies, N (%)) are presented for count and nominal data. Medians (25th and 75th percentiles, minimum-maximum) are reported for ordinal data (MMSE, PACES-16, Borg-CR10, Paas MERS, SAM and SUS) and means (standard deviation, minimum-maximum) or medians (25th and 75th percentiles, minimum-maximum) for continuous data (age (years), education (years), professional experience (years), time to set up a participant, time to play, time to remove the harness) as appropriate.

Qualitative data analysis: coding and theme development

A researcher from the respective rehabilitation centres checked the verbatim transcriptions and amended where necessary, due to specialised vocabulary pertaining to the field of research and specific devices unknown to the external company. Thematic Analysis (TA) was then used to examine the qualitative interview data in order to gain a deeper understanding of the participants' experiences.

TA is a flexible method for processing a data set to identify, analyse and report on patterns (themes) of meaning within the data [47]. The inductive, data-driven coding approach used in this study, allowed the researchers to recognise patterned responses, which were then developed into themes that capture important information in relation to the overall research question [47].

TA analysis was performed in six phases as suggested by Braun & Clarke [47], which is the same TA process as used for the first phase of the ExerGetic project [48]. An independent coder at each centre in RZM and RHF carried out the initial coding and in an iterative process developed a codebook for the PEUs and SEUs respectively. MAXQDA (VERBI GmbH, GER) and f4 (dr. dresing & pehl GmbH, GER) software were used for the coding. These codes were reviewed internally by experienced researchers at each centre and several coding meetings were held to discuss the findings in an extended interdisciplinary group of researchers from both centres. After a final round of coding based on the approved codebooks, the same iterative procedure was applied to the development of themes for each user group. Based on the convergent design, in a final step, the data were combined to provide a meaningful interpretation [49] and answer to the research question.

Results

Participants’ characteristics

The study was carried out from May to October 2022. A total of 25 PEUs and 22 SEUs were recruited. The PEUs were divided into two groups with 15 healthy OAs in RZM and 10 patients in RHF. Figure 3 shows the recruitment flowchart for the patient group.

Fig. 3

Patient recruitment flowchart

The mean age of the healthy PEUs was 69.4 (range 65-83) years, whereas for the patient PEUs mean age was 76.1 (range 71-83) years. Participant characteristics of PEUs are shown in Table 2 (for RZM) and Table 3 (for RHF). 86.7% of the patient PEUs had four or more additional chronic health conditions. For the SEUs, the mean age was 32.4 (range 24-55) years, with participant characteristics presented in Table 4.

Table 2 Descriptive data for the primary end users (healthy older adults) from RZM

Table 3 Descriptive data for the primary end users (patients) from RHF

Table 4 Participant characteristics of secondary end users

A total of five PEUs did not complete all three testing sessions (three patients and two healthy older adults), which resulted in twenty full datasets. Reasons for the withdrawals from participation in the study were: feeling uncomfortable during exergaming due to the swinging momentum of the hardware system (n=1); not feeling safe enough to walk without the rollator and feeling too exhausted after having had several therapies (n=1); feeling annoyed and exhausted by the walking activity in between the mini-games due to the hot weather and because of the facial mask that had to be worn (n=1); the hot weather, which was associated with high effort and feeling uncomfortable during the exergaming (n=1); and not being motivated to train in this device / non-compliance (n=1).

Primary end user groups

Secondary end user group: therapists

A total of 22 SEUs (rehabilitation experts) were enrolled in the study (12 in RZM, 10 in RHF). SEU participant characteristics are shown in Table 4.

Results of the outcome measures

Descriptive statistics for the quantitative assessments are presented in Table 5.

Table 5 Patients' evaluations according to the session number for primary end-users

Over the three sessions, the PACES scores remained consistently high in the healthy OAs group, whilst fluctuating slightly in the patient group. In the rating of emotional dimensions with the SAM, there was little difference between the two PEU groups. Both physical and cognitive perceived exertion were rated low at T1, but increased with the more demanding game levels (“medium” or “hard”) in the following sessions, especially for the patients in RHF.

The SEUs evaluated the ExerG to be usable as signified by scores >70 (please see Table 6).

Table 6 Median SUS scores per testing session for secondary end users

For each of the three testing sessions, the duration of hardware setup, actual playing time and closure is presented in Table 7. For both centres, the total training duration increased from the first to the last session.

Table 7 Descriptive data on training session, set-up and closure durations

Qualitative interview data

Primary end users’ feedback on the ExerG

Using TA, four themes were identified in the data of each PEU group, as presented in Table 8.

Table 8 Themes resulting from the thematic analysis of primary end users’ data

For each theme, key example quotes from the interviews are given below, with the extracts identified by centre, participant identification reference, and paragraph number.

Primary end users RZM: Healthy older adults

Theme 1

The PEUs (healthy seniors) witnessed a positive experience and enjoyed the exercise while playing the exergames. They immersed themselves in the games and felt comfortable during the execution. Additionally, the users expressed a desire to continue playing if the games included an increase in difficulty and offered a variety of options, thereby providing a stronger physical and mental challenge."I liked the study, but it was completely new for me, I've never experienced anything like it." (RZM_PEU12, paragraph 5)

"I'd keep playing that for a while, too, if there were more variations. I mean, if I could learn more about it, that would be an incentive for me to keep going." (RZM_PEU13, paragraph 7)

“It definitely has a certain challenge, which of course would have to improve with training. Physically and mentally" (RZM_PEU10, paragraph 58)

Theme 2

The PEUs were ambitious to increase their own fitness through the games. To achieve this, they requested clear instructions and goals from the program, feedback on their performance during the game, and a way to track their own development. Furthermore, they desired comparative values with other users at the end, which positively influenced their motivation. “You can’t tell, was that right or wrong, and if there was some indication, you could relate better to it” (RZM_PEU01, paragraph 45)

“Yes, I missed having goals, goals and comparative values are needed, in order to be able to rate oneself.” (RZM_PEU02, paragraph 41)

Theme 3

The PEUs considered the technical implementation of the games to be basically successful. For further development, they wished for the games to be more closely aligned with everyday life, to have an immediate system response to the user, and to feature vivid color schemes and appealing soundscapes. These enhancements would support correct task performance."Everything is clearly structured." (RZM_PEU02, paragraph 31)

"I would like to run longer, it was a bit more like jogging. You could walk through the grass, those are the things that I also liked." (RZM_PEU06, paragraph 33)

"So a proper calibration of the colors, hardware and software have to fit together. Different lighting conditions in the game must be balanced." (RZM_PEU13, paragraph 23)

Theme 4

For most of the PEUs, the safety system was evaluated negatively during the game. They felt restricted in completing the tasks and in moving freely. Their concentration was disturbed by its noisy sound. However, during the exergame, the harness receded into the background, as the users' focus was entirely on the game."As already mentioned, you feel safer with a harness." (RZM_Interview_PEU05, paragraph 65)

"I felt slightly handicapped and restricted with this harness." (RZM_Interview_PEU11, paragraph 19)

Primary end users RHF: Patients with geriatric and neurological conditions

Theme 1

Patients were really motivated and particularly enjoyed the novelty aspect of the ExerG which provided a change to standard therapeutic sessions. Patients appreciated being able to go at their own pace, but also having a clock running to encourage them to go as fast as possible. They valued the improvements they experienced in just three sessions.

“It is just good to try out something new, it was interesting for me. And it was really exactly right for me, perfectly adapted to my situation, or not? I came here to relearn reactions and balance.” (RHF_PEU10, paragraph 6)

Theme 2

PEUs particularly liked the fun aspect of the ExerG training with the different mini-games and activities. The ‘apple picking’ mini-game was mentioned several times and enjoyed by all, although some prefered walking through the grass or rowing across the water.

“It was also fun to play with the colours.” (RHF_PEU08, paragraph 4)

Patients also recognised the benefits of working both physically and mentally in an effort to improve their condition. Almost all the patients found the mini-games challenging, in contrast to the healthy older adults. The concentration needed was demanding, especially in combination with the exercise. All concurred that they were able to immerse themselves in the mini-games.

“It was a challenge, of course, both for the mind and body, I am no longer the youngest, it needs concentration.” (RHF_PEU05, paragraph 48)

Patients came up with many original ideas on possible game scenarios to increase the variety on offer. Some suggested adding hidden snakes, or a tropical rainforest background, whereas others suggested more realistic environments such as shopping centres.

“I would stay in a natural setting. One could, for example, take a stream, water with fish, then one would have to look for individual fish and touch them, that would also be a possibility.” (RHF_PEU04, paragraph 73)

Theme 3

Patients are aware of the various challenges faced depending on an individual’s diagnosis and condition, hence the numerous requests for the possibility to individualise the ExerG experience. Whereas some encountered difficulties due to the upper limb movements, for others it was the visual oversight and filtering of information coming from both the sides and the front, which proved harder to tackle.

“Quite a few new tasks, related, but a little different […] if you did it well or were very successful, [e.g. picking apples] then add another or something else. To see what level you have, where your limits are.” (RHF_PEU08, paragraph 55)

“Yes, that is another point, last time you told me one must stay longer on the ‘continue’ button and that is sometimes a bit difficult if one cannot hold the postion properly.” (RHF_PEU04, paragraph 61)

Theme 4

The feeling of security provided by the harness system enabled patients to move around without having to worry about falling and was much appreciated.

“So it’s good that one was secured, or that I couldn’t fall” (RHF_PEU10, paragraph 39)

However, some felt it slowed them down and limited their movements or were unhappy with the noise that the harness system made when swinging to the end positions.

“Though, of course, the speed with which one can start moving is restricted.” (RHF_PEU08, paragraph 28)

Secondary end users’ feedback on the ExerG

During the TA, initial coding and theme development was carried out on the data from each centre separately, before being merged in a final step (Table 9). Again, for each theme, key example quotes are given below.

Table 9 Themes resulting from the thematic analysis of secondary end users’ data

Theme 1

SEUs believed that primary end-users, seniors and patients, were motivated, quickly engaged with, and enjoyed exergaming. They found the ExerG to be a welcome addition to therapeutic training options. Numerous ideas for further development were suggested, including the ability to adapt the system to specific patient conditions, increase exercise variety, and enhance the visual and acoustic game environment. However, the potential loss of therapy time due to repeated technical challenges faced by the therapists was a significant hurdle to regular use.

The therapists from both centres found the majority of patients to be motivated by the novelty aspect of the ExerG.

"The lady was extremely motivated and she was very satisfied with her performance, she solved the tasks with great concentration.” (RZM_SEU12, paragraph 35)

“I can imagine that it would really motivate patients to be able to do something different, something new, something in an individual setting. That can also be very motivating. It makes a change from their routine daily therapy.” (RHF_SEU03, paragraph 141)

Based on their experience as rehabilitation experts, they were able to provide many ideas on how to further improve the ExerG. These included the ability to adapt the mini-games to specific patient conditions, such as changing calibration for overhead arm movements, options to cater for different levels of ability and ideas for different scenery.

“I think one could maybe make these games reflect everyday life a little more […] I don’t know, for example, one could do something, like go shopping, so that it becomes more realistic.” (RHF_SEU07, paragraph 206)

For regular use in a rehabilitation setting, the technical reliability needs to be improved as therapists have to work within a tight schedule and they faced various problems with the hardware, such as the beamers not working or there being no sound.

“So to boot up the whole system […] and until the patient is ready to play, really needs to be within a timeframe that does not stress me. That I’m not wasting too much therapy time. That means when I turn it on, then it boots up, I shouldn’t then have to adjust something here and adjust something there, and the calibration, until the patient is actually ready. That somehow needs to run uniformly.” (RHF_SEU01, paragraph 78)

"The game should react faster, that wasn't ideal either. Calibration didn't always work correctly. So it would be good if it were a bit easier.” (RZM_SEU01, Paragraph 33)

Theme 2

Therapists perceived the cognitive challenges to be more significant than the physical demands of the mini-games in the ExerG. To increase the attractiveness of the system, therapists suggested including a wider variety of game activities, a greater range of progression, and the ability for patients to compare their performance with previous game results and other players.

SEUs agreed that the tasks could generally be more physically demanding, but the level of perceived cognitive challenge was very much dependent on the participant’s condition, which is why the ability to individualise the experience was deemed to be important.

“If, for example, somebody is very good, that one can increase the level of difficulty, that it is possible to vary the tasks.” (RZM_SEU01, paragraph 23)

“One could make more levels, where it becomes more intensive, or where the game lasts longer, for example. Or where players are asked to vary the speed and walk faster.” (RHF_SEU03, paragraph 159)

Although some SEUs commented that the scenery could be more realistic, the majority approved of the comic style backdrop.

“So visually, I would say the environment looks a bit like in a comic. But that doesn’t bother me, for it is therapy and not animation. I think one should differentiate between the two.” (RHF_SEU01, paragraph 41)

Theme 3

The SEUs, therapists, found the exergame's instructions easy to understand and control, but they suggested improvements to optimize the game's instructions and target display. They also desired real-time responses from the program to ensure smooth gameplay and increased motivational input from the software.

For the most part, SEUs thought the instructions were clear and the players knew what to do, but there was nevertheless room for improvement.

“I found it generally easy to understand, but with the first time rowing I found the vertical bars and the cookies rather confusing” RZM_SEU09, Absatz 25

“I think the text could be a little bit bigger, the writing was all a bit small” (RHF_SEU07, paragraph 71)

SEUs felt they had to intervene on occasion as the ExerG did not provide enough motivation or clear goals, which also serve to motivate players.

“I kept wanting to motivate players as I found it was a little lacking, the motivation. I would have liked something like a countdown towards the end of the task or to signal the time was nearly up […]. Maybe some sort of standard value could be given, so that the patient has an idea. During the exercise you don’t know if you should go faster? Or slower?” (RHF_SEU09, paragraph 75-79)

Theme 4

SEUs also found that tightening the safety belt was easy, secure, comfortable, and fitting. Mobility was considered just sufficient due to rebound effects and noise. The system could be used in a rehabilitative setting, but adjustments and a stronger functional orientation needed to be made. Suggestions were offered regarding the harness system, calibrations for the overhead arm movements, and guidance on the correct settings for patient size.

After the first fitting, SEUs thought the harness itself was easy to use, although some suggestions were made regarding the clip mechanism. The main problem with the safety system was that it had a negative impact on the players’ freedom of movement and was noisy, which can be distracting for players.

“So the harness itself is good. It is also easy to put on. But the safety system is too loud and too sluggish.” (RHF_SEU06, paragraph 82)

“The belt system should cushion the momentum more, the patient should feel comfortable and stable.” (RZM_SEU10, Paragraph 41)

SEUs did not feel confident in adjusting the tension on the rope to ensure that the correct amount of slack was given for playing versus the height necessary to avoid knee contact with the floor in the event of falling. A table with standardised settings in relation to patient size would be welcome.

“That I can’t enter how tall the patient is. Or that one must start by measuring the leg length. So that one can adjust it in accordance with that information.” (RHF_SEU01, paragraph 17)

Discussion

In this usability study, we aimed to evaluate whether PEUs and SEUs evaluated the ExerG as usable, providing a positive experience, and thereby acceptable. For both user groups, our findings showed that this new training device is indeed usable, experienced in a positive manner, and acceptable.

The multi-methods convergent design was used to provide not only quantitative data to answer our research question, but also qualitative data to enable a more in-depth examination of the question and generate feedback and suggestions which will be beneficial for the continued development of the ExerG.

With SUS scores of 70 and 75, in the two evaluations sessions respectively, the ExerG was evaluated as good by the SEUs and demonstrated an acceptable usability, as defined by Bangor et al. [37]. The interviews confirmed this evaluation and as is to be expected of a product in development, various suggestions for improvement were made in the interviews, providing valuable information for the hard- and software designers.

Another important factor for SEUs was being able to include an ExerG training session within the time constraints imposed by therapy planning in institutions. The duration of the training sessions increased with each round almost achieving the full 30 minutes for all PEUs. The time necessary for the set up decreased from first to last session suggesting that the therapists became familiarised with the safety system and developed a routine. In RHF, the time needed to set up and close the sessions was approximately halved suggesting that familiarisation was even more important when dealing with patients, as opposed to healthy OAs. In the third evaluation session, the set-up time spanned from 94 seconds (RZM) to 195 seconds (RHF) and the session closure time was 60 seconds or less to remove the equipment. These values were deemed acceptable, however, it emerged from the interview data that hardware reliability was an issue that needs to be addressed.

The PEUs also approved of this new training device and felt that the technical implementation was successful. With a median score of seven for all PEUs in the dominance dimension of the Self-Assessment Manikin, it was shown that they felt in control when using the ExerG, as backed up by the qualitative feedback. Based on our analysis of the interviews, game instructions were clear, which according to the literature is an important factor to be taken into account when designing exergames for older adults [20, 50, 51]. The security provided by the safety harness was appreciated by the patients from RHF, confirming the importance of safety reported in previous studies [48, 52]. However, more freedom of movement was requested by the healthy older adults in RZM. Useful feedback was given on the safety system enabling the designers to continue developing the ExerG.

We were also interested in whether the ExerG provided a positive experience. Our quantitative and qualitative findings were in agreement about the positive nature of the experience.

With a mean median score of 73 (range 67-76) over the three sessions in the PACES questionnaire, representing 91% of the 80 point maximum score, enjoyment whilst using the ExerG was rated highly by PEUs. The pleasure dimension of the SAM questionnaire confirmed this result with a constant median score of seven out of nine given by all the PEUs. In the interviews, both groups of PEUs also reported that they enjoyed the mini-games and were motivated to continue using the sytem. This aligns with the growing body of literature, reporting that enjoyment achieved through a playful environment is a significant factor in increasing motivation and promoting physical activity in the elderly [20, 53,54,55,56].

Our thematic analysis showed that sufficient cognitive and physical challenges are required by PEUs to enhance their motivation, and expected by SEUs to train effectively, which are both elements found in previous research [10, 57]. As measured with consistently low to medium scores on both the MERS and Borg-CR10 rating scales, the qualitative data confirmed that a greater range of progression was deemed necessary by all end users. The ability to customise the experience for each player, ensuring the appropriate level of difficulty in accordance with the person’s physical and mental capabilities was important, again corresponding with recent findings by other researchers [9, 51].

Strengths and limitations

Our study implemented an extensive testing procedure for the functional model of a newly developed exergame, using a mixed-methods approach with the questionnaires and the interviews mutually confirming the findings. Using the RITE method ensured that smaller adjustments could be carried out in an efficient manner, which was beneficial for all study participants.

Another strength was the wide perspective achieved by having two different PEU target groups - patients and healthy individuals all aged 65 and older - and SEUs with different levels of experience and fields of expertise, in addition to the study being carried out in two rehabilitation centres in two different countries.

Furthermore, the ExerGetic project benefits from an international and interdisciplinary consortium of experienced researchers with areas of expertise ranging from health and rehabilitation to human-computer interation and game design. The advantages provided by this pool of knowledge were used throughout the study, from generating the training concept through to the development of codebooks for each end user group and the subsequent analyses.

Our study had a limitation regarding the gender distribution in the patient PEUs in RHF. Whereas in RZM the gender distribution corresponded to that of the general population, the study participants in RHF were 60.0% male, which rose to 71.0% male in the complete data sets, due to two of the three drop-outs being female. In future studies, we would aim to recruit participants with this in mind, in order to better represent the target group for this training device. This problem did not occur in the SEUs, as 68.1% of participants were female, reflecting the dominance of female workers in the health care industry.

Our findings that the ExerG was acceptable emerged from the qualitative data. In order to reinforce these findings, consideration could be given to use one of the available questionnaires, such as the Technology Acceptance Model [58], which is short and easy to administer, in the next study. However, although the extensive nature of our data collection is also a strength, the amount of time and concentration required by our elderly PEUs to complete the battery of assessments and interviews may be considered rather high [59]. One example of possible changes to implement in future studies, would be the use of the PACES- 8 scale, which is specifically tailored to the elderly, shorter than the original, faster to use and would help to reduce participant burden [60].

At a later date, randomised controlled trials will be needed to measure the effectiveness of the ExerG in improving cognitive and physical functions of OA and reducing their risk of falling.

Conclusions

Our findings confirm that the ExerG functional model is a promising training device. It was evaluated as usable by healthy older adults, patients and therapists. The positive experience reported by primary end users, was indeed observed by secondary end users. The qualitative data confirmed that the ExerG was acceptable and suggestions for optimisation were obtained. These will serve to further develop the ExerG towards the ultimate goal of providing an excellent training solution specifically designed for the elderly, adaptable to both healthy older adults and patients with a variety of difficulties in daily life. In the long term, the social impact is expected to be substantial, by enhancing the daily life independence of older adults and improving their quality of life.