RhythmWalker: Using Music Streaming Services to Make Exergaming More Engaging
Exergaming and Music | Lewis Matheson Creed
Exergaming is a branch of serious video game research aiming to address rising obesity levels and sedentary lifestyles. One subfield of exergaming is mobile rhythm games, which utilise the psychological benefits of listening to music to motivate exercise. However, research in this area has only used predetermined songs that researchers chose. Meanwhile, music streaming services offer users near-limitless song selection. This article presents RhythmWalker, a novel application that connects to Spotify, giving users the freedom to exercise with their choice of music. Subsequently, an informal user study was conducted, and inconclusive results regarding the hypothesis were found.
Motivation, Background, and Related Work
Global obesity rates are worryingly high and are climbing in many countries. For example, 1 in 3 adults and 3 in 25 children in New Zealand are obese [1]. Long-term obesity is linked to numerous chronic and life-threatening illnesses such as stroke, diabetes, and heart disease [2]. Aside from the personal toll of these complications, treating obesity-related illnesses is very expensive. For instance, the associated public healthcare costs in New Zealand are around NZ$2 billion annually [3]. Thus, trying to reduce obesity is good for the general public’s quality of life and would free up money within the health sector for other needs. To address this social need, a branch of computer graphics research called ‘serious gaming’ has been looking into developing ‘exergames’, which use gamification to engage users with exercise activities. But why make a video game to address obesity?
Currently, video games are the largest media industry in the world, producing more money than both the music and movie industries combined [4]. If an exergame could engage users with exercise while gaming, it could translate to the player reaching recommended physical activity goals [5]. Since video gaming is both widely popular and usually a sedentary activity, gamers are a worthy target demographic for research to counter obesity and promote a more active lifestyle. An exergame may also appeal to non-gamers too.
Smartphone devices appear to be a compelling choice of platform for developing exergames [6]. This is for two main reasons. First, smartphones are very accessible, with a user base in the billions [7]. Second, smartphones are lightweight, small, and wireless. This makes them well suited for use alongside many forms of exercise which require mobility, such as running and field sports. Several ‘location-based’ mobile exergames have proven to be highly popular too, such as 2016’s Pokémon GO [8], which still has millions of active users. Alongside their popularity, several studies have shown that such mobile exergames have both positive psychological effects [6] and increase their users’ daily step counts [9]. With this proven potential in mind, the development of such mobile exergames has become an active subfield of ‘serious game’ research and development. For example, the terms ‘mobile’ and ‘exergame’ together return over 5,500 results in Google Scholar, with the majority of those articles being released in the last 4 years alone.
Despite this vast number of exergames being developed by academics and students, not all of these ventures have successfully engaged their players [10]. A 2023 systematic literature review of the effectiveness of exergames, a majority of which were developed for smartphones, found that most exergames amounted to being just a ‘novelty’ and failed to maintain user engagement and increased exercise levels in the long term [10]. But the same review also found that exergames which consciously applied psychological research when designing their gameplay were more engaging and proved to be more effective at altering long-term behavioural patterns [11].
Music is already an important part of many people’s exercise routines, including dance classes such as Zumba [12], background music at gyms, and jogging with earphones in. Indeed, the psychological benefits of listening to music while exercising have been well studied. For example, it is known that listening to music reduces the perceived effort of low to moderate exercises (such as walking) by as much as ten per cent [13]. This ‘ergogenic’ psychological effect has been observed in weightlifting too [14]. It has also been shown that music generally enhances the endurance of those exercising, with a slightly more pronounced effect when the music is the choice of the exerciser [15].
It has previously been shown that this ergogenic effect spawned from music can be applied fruitfully in exergames [16]. However, few papers have experimented with using music-based rhythm gameplay in a mobile exergame. One example is Komninos et al.’s [17] ClearBeatWalker, which encouraged their users to maintain an optimal walking pace by using music. If the user was not meeting the required walking pace, the music’s sound quality was reduced [17]. Similarly, Otsubo et al. [18] produced BeatSync, an exergame that synchronised the music’s beats per minute (BPM) with the user’s steps per minute (SPM). However, their gameplay involved adjusting the volume level rather than sound quality [18]. A major drawback of both papers is that they did not allow users to select their own music for the exercise, instead listening to a predefined song selection chosen by the researchers. This seems like a significant oversight, as people’s music tastes vary dramatically.
My approach with this project was to fill this research gap by experimenting if the ergogenic effect of listening to music can be enhanced when the choice of music is that of the user. The natural technology to enable this functionality in a mobile exergaming app would be a music streaming service, as such services already enable on-demand audio playback of millions of songs. Thus, my research question was
“Does the freedom of song selection enabled by music streaming make a mobile rhythm-exergame more engaging?”
Developing the RhythmWalker App
The first roadblock for developing such an exergame for any researcher in this area is undoubtedly a legal one. The handful of popular music streaming services such as Spotify, Apple Music, YouTube, SoundCloud, Amazon Music, and Tidal all offer API access, but with very restrictive terms of use. For example, Section 3.2 of the Spotify Developer Policy prohibits creating games using their API. Since RhythmWalker was intended for academic research, I ignored such API usage embargoes, although I did try reaching out to a number of these companies for their blessing, with no response. Ultimately, I chose Spotify’s API for RhythmWalker. This was for three reasons. First, and practically, the Spotify API is the only music streaming API which has a Unity package, conveniently the same game engine I used to develop the app. Second, the Spotify API can access song BPM information directly, which is important for the gameplay. Third, Spotify happens to be the most popular dedicated music streaming service [19]. Spotify also has over 100 million songs in its catalogue [19], which is perfect for evaluating our research question, as there is a nearly limitless selection of music available for participants to walk with.
It is worth noting that from a software design perspective, the choice of music streaming service is arbitrary, as they act as interchangeable middleware. Their only role in RhythmWalker’s gameplay is obtaining musical content (and BPM information in Spotify’s case) to play in the app. Theoretically, a complete version of the RhythmWalker application could let the user pick and choose the streaming service of their choice, making the app more accessible too.
I opted to implement the same rhythm-based walking exergame gameplay used by Otsubo et al.’s BeatSync, where the music’s playback volume is matched to the user’s step cadence, i.e. matching BPM to SPM. This gameplay style is particularly intuitive and natural, as distant sounds are quieter than nearer sounds, proportionately reflecting how close the user’s walking cadence is to the target pace. Adjusting the volume is also non-intrusive to the musical experience, whereas distorting the sound is. Furthermore, it encourages sustained exercise to continue hearing their music; if they stop moving, the music will become inaudible.For tracking the user’s walking cadence (SPM), I used accelerometer step counting with zero-crossing counting [20]. Implementing this counting method requires the three-dimensional accelerometer sensor data from the mobile device to be translated into a single value, so I used the Euclidean Norm Minus One (ENMO) averaging method [20]. The ENMO average can then be tracked over time. As the user steps or swings their arm, the ENMO value spikes rhythmically in the positive and negative direction. The zero-crossing method counts steps by checking if the ENMO value crosses the threshold from negative to positive. The threshold is usually normalised to be 0. In practice, this method detects steps accurately enough, although it is fairly naive as it can be tricked into counting false steps by shaking the mobile device. The sensed SPM value over time is then compared to the BPM value of the song being played from Spotify, and playback volume of the music is adjusted accordingly.
Figure 1: RhythmWalker UI. The user’s total step count would be displayed in blue where the word STEP is. Information about the song currently playing is also visible. The music can be controlled by the user or remotely by the researcher via the Spotify App.
Figure 2: ENMO formula
Figure 3: Zero-crossing illustration. Blue circles represent where steps would be detected.
Over a week, ten users participated in a limited informal user study. This limitation was due to the restricted resources and time available. The survey was qualitative, using open-ended, multichoice, and Likert scale questions. Each participant alternated which version of the RhythmWalker application they walked with first, either the control “Preset” playlist or the experimental “Custom” streaming service. This was done to mitigate order bias influencing the overall results. Even so, analysis of the responses revealed that the order in which users tested the application did impact their experience. For example, lap times on average decreased across all participants on their second lap, regardless of what music they listened to.
Next, I conducted a user study to assess if having the freedom to listen to music of the user’s choice made the exergame more engaging and motivational than listening to fixed or predetermined music. Each participant was asked to complete the same 750-metre lap twice while using the RhythmWalker app: once while listening to music of their choice, and once while listening to a preset playlist consisting of five popular songs with 130 BPM, which is the recommended SPM for higher-intensity walking [21]. It has also been studied that songs around 120 BPM are easier for most people to detect the beat [22]. Thus, the five songs were If I Had You by Adam Lambert, Boogie Wonderland by Earth, Wind & Fire, I’m Gonna Be (500 Miles) by the Proclaimers, Shut Up and Dance by Walk the Moon, and I Gotta Feeling by Black Eyed Peas.
User Study, Results, and Conclusion
Figure 4: Map of user study walking route marked in yellow. Map courtesy of Google Earth.
As for answering the research question, the results of the experimental user study were inconclusive, as statistical analysis of the users’ responses did not yield any statistically significant trends. But this was unsurprising as the user study was far too small for statistically significant trends to be clearly discernible. As such, future work in this direction should aim to conduct a larger formal user study to test the hypothesis properly. Despite being unable to confirm the hypothesis, a discussion of the results can still be held, albeit with limited broader applicability beyond this study.
The results did align with the hypothesis and informally indicated that the ability to choose the music being played could make mobile rhythm-based exergames more engaging. When users were asked if they preferred exercising with RhythmWalker when they had control over the music, there was an overall positive response. To double-check this sentiment, I also asked a similar question seeking a reversed reply, instead asking to what extent the user preferred listening to a preset exercise playlist with RhythmWalker. The users still preferred listening to their own selection of music.
General feedback on the gameplay was highly positive. Most users felt the RhythmWalker app was fun and engaging. Most users also thought that the volume adjustment system strongly motivated them to walk and they would be encouraged to exercise more frequently by playing the exergame if RhythmWalker was developed into a finished product. This makes sense, as the app essentially gamifies jogging while listening to music, an already popular and common exercise activity.
Importantly, regarding the ergogenic effects of music on those exercising, it is noteworthy that the custom laps were reported as more taxing, despite the research presented by [13], [17], [18],[19] on the additional ergogenic effects of user-determined music. I think this anomaly can be explained when one considers that the speed users were expected to walk was tied to the BPM of the music they were listening to. The preset lap had a set BPM of 131, which is at an optimal level for fast but comfortable walking [22]. The average BPM of the custom laps was 4 beats faster, which is a trivial difference in tempo. However, as the sample size of users was very small, I believe outlier responses have skewed the data. For example, one user’s custom song choices had an average BPM of 165, which would have meant they were required to walk significantly faster than the preset lap. Users also found it slightly more difficult to sense the beat of their own song choices and to walk to the beat during the custom lap than the preset lap.
Figure 5: Overall user feedback on the RhythmWalker application.
Nevertheless, the key takeaway of this research project was that while users did enjoy being able to choose which songs they walked with, sudden shifts in BPM between songs, alongside songs with BPMs that were too slow or too fast, had adverse effects on user experience. Any applications wanting to integrate the freedom of choice that music streaming services offer must assist users in selecting appropriate songs for exercise. This could be achieved in several ways, including moderating BPM changes between automatically queued songs or creating playlists that cater to the user’s music tastes but contain songs with similar BPMs.
Additionally, my study also reaffirms existing research by [17], [22] that adjusting audio volume is an effective and motivational game design methodology for mobile rhythm-based walking exergames. However, a follow-up formal user study is necessary to be able to confirm whether a user’s song choices has a more positive effect than predetermined music. Nevertheless, this research investigated the viability of integrating music streaming services with exergaming and found it was very possible to implement.
[1] Ministry of Health, “Obesity statistics,” health.govt.nz. https://www.health.govt.nz/nz-health-statistics/health-statistics-and-data-sets/obesity-statistics (accessed April. 13, 2024).
[2] CDC, “Health Effects of Overweight and Obesity,” cdc.gov.https://www.cdc.gov/healthyweight/effects/index.html (accessed April. 13, 2024).
[3] B. Swinburn. “The cost of excess weight in NZ.” auckland.ac.nz. https://www.auckland.ac.nz/en/news/2021/12/24/cost-of-excess-weight-in-nz.html (accessed April. 13, 2024).
[4] W. Witkowski. “Videogames are a bigger industry than movies and North American sports combined, thanks to the pandemic.” marketwatch.com. https://www.marketwatch.com/story/videogames-are-a-bigger-industry-than-sports-and-movies-combined-thanks-to-the-pandemic-11608654990 (accessed April. 13, 2024).
[5] Ministry of Health, “How much activity is recommended?” health.govt.nz. https://www.health.govt.nz/your-health/healthy-living/food-activity-and-sleep/physical-activity/how-much-activity-recommended (accessed April. 13, 2024).
[6] Z. Cheng, B.N. Greenwood, and P.A. Pavlou, “Location-Based Mobile Gaming and Local Depression Trends: A Study of Pokémon Go,” Journal of Management Information Systems, vol. 39, no. 1, pp. 68-101, Apr. 2022, doi:10.1080/07421222.2021.2023407.
[7] BankMyCell, “How Many Smartphones Are In The World?” bankmycell.com. https://www.bankmycell.com/blog/how-many-phones-are-in-the-world (accessed April. 13, 2024).
[8] Pokémon GO, “Home Page,” pokemongolive.com.https://pokemongolive.com/?hl=en (accessed April. 13, 2024).
[9] T. Althoff, R.W. White, and E. Horvitz, “Influence of Pokémon Go on Physical Activity: Study and Implications,” Journal of Medical Internet Research, vol. 18, no. 12, pp. 1-14, Dec.2016, doi: 10.2196/jmir.6759.
[10] S. M. R. A Elaheebocus and F. Grant, “Exergaming Characteristics in Interventions Addressing Physical Activity and Nutrition: A Systematic Literature Review,” EAI Endorsed Transactions on Pervasive Health and Technology, vol. 9, pp. 1-21, Jun. 2023, doi:10.4108/eetpht.9.2674.
[11] N. Aldenaini, F. Alqahtani, R. Orji, and S. Sampalli, “Trends in Persuasive Technologies for Physical Activity and Sedentary Behavior: A Systematic Review,” Frontiers in Artificial Intelligence, vol. 3, pp. 1-40, Apr. 2020, doi: 10.3389/frai.2020.00007.
[12] Zumba, “Move With Your Crew,” zumba.com. https://www.zumba.com/en-US/ (accessed April. 13, 2024).
[13] C. I. Karageorghis and D. Priest, “Music in the exercise domain: a review and synthesis (Part I),” International Review of Sports and Exercise Psychology, vol. 5, no.1, pp. 44-66, Dec. 2011, doi:/10.1080/1750984X.2011.631026.
[14] N. R. S. Silva, F.G. Rizardi, R.A. Fujita, M.M. Villalba, and M.M. Gomes, “Preferred Music Genre Benefits During Strength Tests: Increased Maximal Strength and Strength-Endurance and Reduced Perceived Exertion,” Perceptual and Motor Skills, vol. 128, no. 1, pp. 324-337, Feb. 2021, doi:/10.1177/0031512520945084.
[15] Z. Cole and H. Maeda, “Effects of Listening to Preferential Music on Sex Differences in Endurance Running Performance,” Perceptual and Motor Skills, vol. 121, no. 2, pp. 390-398, Oct. 2015, doi: 10.2466/06.PMS.121c20x9.
[16] T. Park et al., “ExerSync: facilitating interpersonal synchrony in social exergames,” in CSCW ‘13: Proceedings of the 2013 conference on Computer supported cooperative work, San Antonio, TX, USA, Feb. 2013, pp. 409-422, doi: 10.1145/2441776.2441823.
[17] A. Komninos, M. Dunlop, D. Rowe, H. Allan, and S. Coull, “Using Degraded Music Quality to Encourage a Health Improving Walking Pace: BeatClearWalker,” EAI Endorsed Transactions on Ambient Systems, vol. 2, no. 7, pp. 1-9, doi: 10.4108/icst.pervasivehealth.2015.259072.
[18] A. Otsubo, H. Suwa, Y. Arakawa, and K. Yasumoto, “BeatSync: Walking Pace Control Through Beat Synchronization between Music and Walking,” in 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Kyoto, Japan, Mar. 2019, pp. 367-369, doi: 10.1109/PERCOMW.2019.8730833.
[19] S. Jayalath and N. Abhayasinghe, “A gyroscopic data based pedometer algorithm,” in 2013 8th International Conference on Computer Science & Education (ICCSE), Colombo, Sri Lanka, Apr. 2013, pp. 551-555, doi: 10.1109/ICCSE.2013.6553971.
[20] K. Bakrania, T. Yates, A.V. Rowlands, D.W. Esliger, S. Bunnewell, and J. Sanders, “Intensity Thresholds on Raw Acceleration Data: Euclidean Norm Minus One (ENMO) and Mean Amplitude Deviation (MAD) Approaches,” PLOS One, vol. 11, no. 10, Oct. 2015, doi: http://dx.doi.org.ezproxy.auckland.ac.nz/10.1371/journal.pone.0164045.
[21] R. Shewale. “Spotify Stats 2023 — Subscribers, Revenue \& Other Insights.” demandsage.com. https://www.demandsage.com/spotify-stats/ (accessed April. 13, 2024).
[22] F. Styns, L. van Noorden, D. Moelants, and M. Leman, “Walking on music,” Human Movement Science, vol. 26, no. 5, pp. 769-85, Oct. 2007, doi: https://doi.org/10.1016/j.humov.2007.07.007.
Lewis is a fifth-year undergraduate currently completing his honours project, which focuses on natural language processing, machine learning, image processing, and the ancient Egyptian language. He also founded the BYO Music Club at UoA and is the current Arts Editor for Craccum.