The efficacy of digitally delivered physical literacy training on productivity, wellbeing, and confidence: An online randomised controlled trial in a secondary aged sample

In 2022 Adapt won Innovate UK Smart Grant funding to develop personalised physical literacy plans for students using machine learning and incorporate them into its already-successful independent learning app. These aimed to help students combat stress and anxiety, improve wellbeing and productivity, and raise confidence levels.

Introduction to Adapt

Adapt is a revision timetable app for GCSE and A Level students that’s already helping over 500,000 students develop independent learning skills and reach their academic goals. Adapt is driven by teacher expertise and bespoke technology. Our team of expert teachers have broken down the specifications for all GCSE/A Level (and UK equivalent) subjects into every topic a student needs to know. Our algorithm then designs and delivers independent learning pathways for students to ensure they cover every topic by exam day, and continually adapts it around their life so it’s always up to date. Adapt's sophisticated algorithm customises users' plans to their individual needs and effectively boosts productivity and exam results in an unprecedented way.

Adapt has grown substantially since it launched on the 10th of January 2020, and has maintained consistent growth and usage despite the Coronavirus pandemic and its effect on education in the UK. At launch Adapt was the Number 1 Education app on the App Store, going on to acquire 65k users in under a month though almost entirely organic channels. In Adapt’s first year it saw rapid growth, averaging 1-2k new users per day, and it’s now used by over 500,000 students and over 5,000 schools globally.

Adapt has been universally lauded, from winning the CVC Young Innovator Award in 2019 to fund our Beta release, London Tech Week’s Elevating founders competition in 2021, three Innovate UK grants in 2021/22, being featured by PiXL and TeachFirst as an outstanding academic resource, and receiving glowing feedback from students and parents alike.

"The subject breakdowns minimise the appearance of large mountains of work which make it more doable and help my motivation. I've made so much more progress than before where I would get overwhelmed and end up doing nothing." (Sasha, GCSE student, 2022)

"It really helped me organise my time and planned out my work in advance for me; I ended up with A*s and As." (Cerys, A Level student, 2022)

“My son was noticeably more organised and in control of his learning which helped dramatically as we have just started the Mocks. Our favourite feature is the daily reminders- a gentle nudge to get things done. My son loved the Premium content especially as it was broken down into bite size pieces.” (Maria, parent to GCSE student, 2023)

Previous research on physical literacy

There is also evidence that physical activity can support cognitive development in adolescents (Donnelly et al., 2016). The view that physical literacy is of value in promoting self-efficacy and self-confidence is underwritten by psychologists, who also champion the important role of self-esteem in developing characteristics such as independence and creativity (Chen, 2015).

While physical literacy training has traditionally been conducted in person, tablets and other electronic devices can be incorporated to facilitate a more student-centred approach to learning and evaluation. The ability to recognize (upon viewing) and implement the appropriate modifications to their fundamental movements increases the likelihood of successful application (Whitehead, Durden-Myers, & Pot, 2018).

A particular physical literacy programme was piloted with 100 primary aged students and 20 controls (Davey, 2016, personal communication). Students who were in the lowest quartile academically in the primary classroom received physical literacy training and saw academic improvements in English (76% improved vs. 43% controls), reading (86% improved, versus 56% controls), mathematics (70% improved, versus 30% controls), and spelling (62% improved, versus 30% controls) as well as improvements in students’ self-reported ability to achieve (94% of students felt that their ability to focus and concentrate had increased and school work had become easier) and self-confidence (81% of teachers reported positive changes in students, particularly in areas of self-confidence, coordination, and emotional functioning). However this particular physical literacy programme faced substantial drop out, with 70 students remaining at the end of the pilot due to added teacher workload and expectations on parents to keep up with the programme at home (there was no student drop out; Davey, 2016, personal communication), and therefore shows potential for self-delivery in an older population.

Study aims

This physical literacy programme has shown promising early results but has thus far been limited to primary aged participants and in-person (1:1) delivery. This study aims to assess the potential for this particular physical literacy programme to be (i) adapted for online self-delivery in a secondary aged population, and (ii) explore transfer effects to self-report measures of academic stress, feelings of control, and wellbeing, as well as in app measures of productivity and confidence.

Method: Stage 1. Develop an effective online adaptation of this particular physical literacy programme for a secondary level population

The current programme was refined for testing based on the expertise of consultants and through focus groups. Adapt hired consultants who worked on a larger physical literacy programme to reduce several hundred exercises down to the 84 most appropriate for self-delivery by 14-18 year olds.

The programme was built into a format that matches with the other academic subjects available in Adapt (Subject:Module:Topic). Demonstration videos for each exercise were created for each exercise in order to deliver the training online. Adapt recruited three theatre students who were physically able to perform all the exercises. They were all trained on how to perform the exercises over two days. Adapt worked with two videographers to direct and film the exercises, then edit them ready for online delivery in Adapt.

Using the community hosting platform Mighty Networks, 100 student participants were recruited to test the programme exercises and provide feedback. Participants rated each exercise on the Brief Mood Introspection Scale (Mayer & Gascheke, 1988) in order to remove any exercises which might be distressing. One exercise which involved too much spinning had to be omitted, but otherwise the full programme of exercises was cleared for use and renamed the Adapt Physical Literacy Programme (used hereafter).

The Adapt Physical Literacy Programme was added as a private subject in Adapt. An algorithm for personalised exercise allocation was created so that a student’s exercise recommendations would reflect their skill levels and competencies and could be continuously adjusted based on their behaviour.

Method: Stage 2. Assess the efficacy of the Adapt physical literacy programme to impact a battery of productivity and mental wellbeing outcomes

In this study Adapt was interested in assessing the impact of using Adapt versus the impact of using Adapt with physical literacy on a battery of outcomes including stress, feelings of control, wellbeing, productivity, and confidence.

Study design

This study was a double-blind between-subjects design where one group was randomised to receive a physical literacy intervention within their Adapt account (intervention) and the other received Adapt without physical literacy (control). All participants completed the same battery of questionnaires pre- and post-trial within Adapt.

Sample size

Adapt recruited 500 student participants to take part in the trial. 36 participants dropped out of the study (citing that they no longer wanted to take part) leaving a final sample size of 464 participants.

Measures: Self-report outcome measures

1. Perceived academic stress

Four items assessing academic stress from the Graduate Stress Inventory-Revised (GSI-R) questionnaire by Rocha-Singh (1994). The items were:

• “Exams”
• “Fear of failing to meet expectations”
• “Handling the academic workload”
• “Meeting deadlines for course assignments”

Students were asked, “How stressful do you find the following?” and responded on a 5-point Likert scale from Not at all (1) to Extremely stressful (5).

2. Perceived control

Five items assessing the extent to which individuals believe they can directly affect how their time is spent were taken from Macan et al. (1990). The items were:

• "I feel in control of my time"
• "I underestimate the time that it will take to accomplish tasks"
• "I spend a lot of time on unimportant tasks"
• "I find myself procrastinating on tasks that I don't like but that must be done."

Students responded to each item using a 5-point Likert scale from Seldom true (1) to Very often true (5). The first item will be analysed alone, and responses to the latter three items will be aggregated as an “out of control” variable.

3. Wellbeing

Wellbeing was assessed with a reduced version of the Warwick-Edinburgh Mental Well-being Scale (Tennant et al., 2007). We used the following six items:

• I’ve been feeling optimistic about the future
• I've had energy to spare
• I've been dealing with problems well
• I've been feeling confident
• I’ve been interested in new things
• I’ve been feeling cheerful

Students responded to each item on a 5-point Likert scale from None of the time (1) to All of the time (5).

Measures: In app outcome measures

1. Productivity

Productivity was measured according to the average number of tasks completed per day, stratified by a) total number of tasks completed, and b) revision tasks completed.

2. Confidence ratings

Each time students completed a topic they were asked to rate their confidence in that topic. Students responded using a 3-point Likert scale of Not at all (1), Somewhat (2), and Very (3).

Hypotheses

While we expect to see all of the following in both the intervention and control groups, we expect to see a significantly higher rate of change in the intervention group (using physical literacy exercises) for the following hypotheses:

• Perception of academic stress will decrease over time.
• Perception of feeling in control will increase over time.
• Perception of feeling out of control will increase over time.
• Wellbeing will increase over time.
• Productivity will increase over time.
• Confidence will increase over time.

Procedure

New users to Adapt in September were shown a sign up pop-up using OneSignal software to ask if they were interested in taking part in a trial to better understand students' academic experience. Those who said yes were asked to read and sign an informed consent sheet. They were then randomly allocated to either the control or intervention group which was automatically connected to their backend profile data and linked to an automatically created participant identification number. Group assignment was done automatically through a technical programme meaning that participant identification numbers were fully blind to both the participant and experimenter. Participants were prompted to complete their daily recommendations via a push notification at 09:00 and 17:00. For those in the control group, this only included revision recommendations. For those in the intervention group, they were also recommended two physical literacy tasks per day. All participants completed pre- and post-trial surveys.

Statistical analysis

Frequentist statistics were computed in JASP version 0.17 (JASP Team, 2023). For all measures, frequentist repeated measures analysis of variance (ANOVA) models were performed to compute F-statistics, p-values and effect sizes. For all analyses, our primary outcome of interest was the omnibus interaction.

Results: Adapt Physical Literacy Programme

The intervention group was recommended two Adapt physical literacy exercises per study day (students set their number of study days out of 7; Mean = 3.32; SD = 3.17). Confidence on each task was rated and subsequently coded as “not at all confident” = 1, “somewhat confident” = 2, “very confident” = 3).

Results: Self-report outcome measures

The Graduate Stress Inventory-Revised (GSI-R; Rocha-Singh, 1994) assessed academic stress, items assessing feelings of control were taken from Macan et al. (1990) and separated for assessment to look uniquely as “feeling in control” and an aggregate of three items assessing feeling “out of control” (underestimating time needed to complete a task, wasting time, and procrastinating), and six items from the Warwick-Edinburgh Mental Wellbeing Scale (Tennant et al., 2007) were used to assess wellbeing.

There was a significant reduction over time in academic stress (GSI-R) and feeling out of control, and a significant increase in feeling in control. Increased wellbeing (W-E MWS) was marginally non-significant. There were no significant interactions between group and any self-report outcome measures over time. However, there was a trend towards significance in the group*time interaction for feeling in control and feeling out of control, whereby those in the intervention group were more likely to feel in control of their revision and less likely to underestimate the time taken to complete a task, waste time, or procrastinate, but not to a statistically significant extent.

Results: In app outcome measures

In app data was extracted for the total revision tasks and total non-Adapt tasks (custom tasks which may or may not be revision related) completed as well as the confidence ratings assigned in October and December. Revision and non-Adapt tasks were used to assess productivity.

There was a significant increase in the total revision tasks and total non-Adapt tasks over time, however there was not a significant difference for either measure between groups. There was not a significant difference in confidence ratings over time or between groups over time.

Discussion

This study sought to (i) adapt a physical literacy programme which had shown promising early results but had previously been limited to primary aged participants and in-person (1:1) delivery for online self-delivery in a secondary aged population and (ii) explore transfer effects to self-report measures of academic stress, feelings of control, and wellbeing, as well as Adapt’s in app measures of productivity and confidence.

In accordance with our first aim, we were successfully able to adapt the physical literacy programme for effective online self-delivery in Adapt. Students were as likely to complete their physical literacy tasks as they were their revision tasks, and showed high engagement consistently throughout the trial as well as incredibly low drop out considering the full trial was conducted online and without in-person participation or nudging.

However, counter to our hypotheses for our second aim, physical literacy failed to transfer to the majority of self-report and Adapt in app outcome measures. Nevertheless, for two measures, one self-report measure of how in control of their revision students felt and one aggregate measure on being out of control of their revision, analyses showed inconclusive effects that were in the predicted direction, indicating that participants in the intervention group were more likely to feel in control of their revision and less likely to report behaviours related to a lack of control over their revision (underestimating time needed to complete a task, wasting time, and procrastination). This provides some evidence for the efficacy of the programme; however, the difference between the physical literacy and control group slopes was not significant, so strong inferences should not be drawn. A longer trial period and/or a trial period leading up to national exams (rather than mock exams) may reveal stronger effects.

Prior to this research, the impact of Adapt on self-report and in app outcomes has only been investigated cross-sectionally. This research showed that, agnostic to group, using Adapt led to a significant reduction over time in academic stress and feeling out of control with their revision, and a significant increase in feeling in control of their revision. Increased wellbeing (W-E MWS) was marginally non-significant in the predicted direction. When looking at in app measures of productivity, there was a significant increase in the total revision tasks and total non-Adapt tasks over time. There was not a significant difference in confidence ratings over time, however this may be because the trial took place mid-academic year when students were learning new topics for the first time throughout the trial. Adapt’s data on our full userbase overall shows that students take, on average, 3.44 revision sessions before they rate of topic as “very confident” (which marks it as done in their app), and therefore these results may have differed if the trial was conducted leading up to national exams (rather than mock exams). Overall this provides strong evidence for the benefits of Adapt not only with revision and productivity, but with further reaching measures of stress, feelings of control, time management, procrastination, and wellbeing.

Overall, in this research we were successfully able to adapt the physical literacy programme for effective online self-delivery in Adapt but it failed to transfer to the majority of self-report and Adapt in app outcome measures. There was some inconclusive evidence for transfer effects to two measures: one self-report measure of how in control of their revision students felt and one aggregate measure on being out of control of their revision. However, given the successful adaptation of the physical literacy programme to Adapt and the high engagement and retention shown by participants even with self-delivery, clarifying whether even small effects exist is likely worthwhile.

References

Caird, J., Kavanagh, J., O'Mara-Eves, A., Oliver, K., Oliver, S., Stansfield, C., & Thomas, J. (2013). Does being overweight impede academic attainment? A systematic review. Health Education Journal, 73: 497–521.

Chen, A. (2015). Operationalizing physical literacy for learners: Embodying the motivation to move. Journal of Sport and Health Sciences, 4, 125–131.

Donnelly, J. E., Hillman, C. H., Castelli, D., Etnier, J. L., Lee, S., Tomporowski, P., & Szabo-Reed, A. N. (2016). Physical activity, fitness, cognitive function, and academic achievement in children: A systematic review. Medicine & Science in Sports & Exercise, 48(6): 1197–1222.

Ericsson, I., & Karlsson, M. K. (2012). Motor skills and school performance in children with daily physical education in school–A 9-year intervention study. Scandinavian Journal of Medicine and Science in Sports, 24(2): 273–278.

Lambourne, K., Hansen, D. M., Szabo, A. N., Lee, J., Herrmann, S. D., & Donnelly, J. E. (2013). Indirect and direct relations between aerobic fitness, physical activity, and academic achievement in elementary school students. Mental Health and Physical Activity 6(3): 165–171.

Lubans, D., Richards, J., Hillman, C., Faulkner, G., Beauchamp, M., Nilsson, M., & Biddle, S. (2016). Physical activity for cognitive and mental health in youth: A systematic review of mechanisms. Pediatrics, 138(3): e20161642.

Macan, T., Shahani, C., Dipboye, R. & Phillips, A. (1990). College Students' Time Management: Correlations With Academic Performance and Stress. Journal of Educational Psychology. 82. 760-768. 10.1037//0022-0663.82.4.760.

Mayer, J. D., & Gaschke, Y. N. (1988). The experience and meta-experience of mood. Journal of Personality and Social Psychology, 55, 102-111.

Rocha-Singh, I. A. (1994). Perceived stress among graduate students: Development and validation of the Graduate Stress Inventory. Educational and Psychological Measurement, 54(3), 714–727.

Tennant, R., Hiller, L., Fishwick, R., Platt, P., Joseph, S., Weich, S., Parkinson, J., Secker, J., & Stewart-Brown, S. (2007). The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation, Health and Quality of Life Outcome, 5, 63.

Tomporowski, P. D., Davis, C. L., Miller, P. H., & Naglieri, J. A. (2008). Exercise and children’s intelligence, cognition, and academic achievement. Educational Psychology Review, 20(2): 111–131.

Whitehead, M. E., Durden-Myers, E., & Pot, N. (2018). The Value of Fostering Physical Literacy. Journal of Teaching in Physical Education, 37(3), 252-261.

---

Follow Adapt on Instagram!

Meet the team behind Adapt

Read the rest of our Blog posts