A Systematic Literature Review of Educational Apps: What Are They Up To?

Mauricio Vásquez-Carbonell

Computer Sciences & Electronic Department, Universidad de la Costa, Barranquilla, Colombia

E-mail: mvasquez1@cuc.edu.co

Received 06 July 2021; Accepted 28 July 2021; Publication 27 October 2021

Abstract

As the use of Educational Apps rises every day and the population begins its use at an increasingly early age, it becomes relevant to understand the positive and negative aspects of this technological tool. However, the information may seem overwhelming, especially for those starting investigations on this topic. For this reason, a Systematic Literature Review was conducted on 119 published scientific papers, in order to create a work that synthesizes all the recent data about most used keywords, funding aid, authors and publishing journals, just to name a few. Additional data also reveals the need, expressed by authors and backed by their research, to evaluate the effectiveness of the Educational Apps. As an additional point, some solutions are offered to deal with the aforementioned problem, as well as some recommendations for the correct development of applications.

Keywords: Educational Apps, mobile learning education, review, effectiveness, recommendations.

1 Introduction

It is widely known that the advances in technology have a great impact in the way’s mankind realizes their activities. One of the main reasons for this impact is the ease with which the information can be transmitted, in addition to the great scope that it has. In the present day, the Information and Communication Technologies (ICT or ICTs) are responsible for the transmission of information and knowledge to an enormous amount of people around the world, the shift of employment models towards a digital economy, the creation of new types of employs and educational innovations, just to name a few [1, 2].

In the educational aspects, ICTs can provide new opportunities to obtain, not only technological abilities, but also new methods that can ensure the knowledge’s acquisition. That is why it is important for each actor in the teaching process to keep up with those technological advancements [3]. Virtual Reality, Augmented Reality and Mobile Applications (Mobile Apps) are just a few of those new tools available to educators to strengthen the educational process [4–6].

The case for the mobile apps is interesting and important to examine. The decreasing cost of the smartphones/tablets and their increasing capabilities, constant access to internet services and the creation of specialized markets like the App store from APPLE and the Play Store from GOOGLE, has helped to expand the number of users of these software applications [7–9]. This has made the use of mobile learning applications a daily and widely accepted activity, evidenced by multiple studies [10–12]. Mobile Learning has helped to create an interactive communication between students and their teachers, to create collaborative environments and personalize the education to help the students [10, 13]. In developed countries, children are becoming users of smartphones and applications at an early age, and parents’ concern begins to grow, as they seek mobile apps to positively influence their children [14, 15]. Likewise, university students are regular smartphone users, and this is already being used by teachers to create a new type of educational experience, especially in developed countries [13].

The understanding of this tool becomes relevant to analyse not only the benefits, but also the areas with opportunities for improvement. Therefore, Systematic Literature Review was carried out to find and synthesize the most relevant information for those authors eager to work with Mobiles Apps. This work also includes some recommendations based on the information given by the authors and their research.

2 Mobile Apps – Benefits and Improvement Areas

In 1973, the general manager of Motorola, Martin Cooper, introduced the first cell phone to the public [16]. Later, in 2002 Blackberry introduced the first cell phone with computing abilities, which can be considered the first smartphone [17]. Thanks to the continuous development of this devices, the creation of new computational elements, such as the tablets and Portable Digital Assistances (PDAs), and the decrease in prices, the population has been able to acquire those unevaluable tools [3, 18] facilitating the access to applications.

Mobile Apps offer one exceptional benefit: Continuing education, meaning that the communication and interaction between teachers and students is no longer restricted to the typical classroom and educational activities can be carried out at any moment and ease the acquisition of formal and informal learning [13, 19–22]. In the same way, educational resources (video calls, eBooks, forums…etc.) are available so a user can access them immediately from any location, and it is It is possible to create specific environments for different professions (e.g. simulate the interaction with a patient), as well as environments to solve the needs of a certain group of students [6, 23]. A widely used use of apps by professionals is that of personalized education, to reach different groups or people with unique needs. Lastly, the Mobile Learning Apps are the perfect tool to create collaborative environments for the students [10].

The educational benefits are also reflected in the youngest population. The scientific community, along with educators, agree on the positive effects on the learning experience, when children are responsibly exposed to learning apps, so much so that measures have been issued by governments to promote the use of these tools [24]. These effects can be maximized when the children use educational applications under the supervision of a responsible adult or an educator [25, 26].

However, it is also necessary to know the aspects to improve to create more efficient educational applications. The initial studies were carried out in controlled environments and with limited groups of people, which could give different results found in a user’s life [27]. To solve this inconvenience, the researchers began to collect the usage data from the different app store users, generating a new method that helped to understand the use of an application by a person in what is considered a favourable scenario [28]. The use of this method has not yet become popular among researchers, due to the fact that they must handle multiple tasks and information of which they have not yet become familiar [7].

As mentioned by [29], research currently focuses on education applied to specific segments (Children, Adolescents, K-12), which represents a limitation. Although some studies cover broader fields, this segmentation process is understandable, which in turn, may present limitations in the results of some studies. Being able to correctly evaluate an educational application takes a significant value today, because the current market is full of apps that claim to be educational, but have little or no educational content [30].

Lastly, there are different population groups that may not be adept at using the most recent technological advances, such as educational apps. An example of this are older adults, who are reluctant to use new technologies and tend to use those that are familiar to them [31]. A study by [32], showed that farmers in Ireland are reluctant to use app technology for 4 reasons: Availability of internet service, distrust in new technologies as well as the lack of familiarization with them and the perception of not receiving adequate benefits when using Mobile Apps. By analysing all the documentation presented by the various researchers, Table 1 was created to compile both benefits and improvement areas.

Table 1 Benefits and improvement areas. Table made by author

Mobile Apps : Benefits & Improvement Areas

Benefits	Improvement Areas
Continuing Education	Limitations in studies
Easy acquisition of formal and informal learning	Limitations in the internet service
Remote access	Distrust of new technologies
Immediate information	Lack of familiarization with the app
Creation of specific scenarios and environments	Perception issues
Personalized education
Collaborative environments

3 Systematic Literature Review

Introduced in 1990’s, the Systematic Literature Review (SLR), is used in this work to synthesize data and provide a recent summary of the authors conclusions and findings [33–35]. Also, a similar process used by [34] was implemented, in which the SLR was divided into 3 stages, as shown in Figure 1. The goal of this work is to answer the following questions:

– Which countries are leading the investigations in Educational Learning Apps (Educational Apps)?

– Which authors are leading the investigations in Educational Apps?

– What were the most used keywords by the authors?

– Which are the journals with the most publications on Educational Apps?

– If there is funding to investigate Mobile Learning Apps, where are these coming from?

– What is the focus of the papers about Educational Apps?

– What are the main recommendations when creating an educational app?

Figure 1 3-stage SLR applied in the document, similar to [34]. Figure made by authors.

In the initial phase (Stage 1 – Planification), the main topic in this document was identified: mobile applications and their use in education. This identification was necessary, since this work used a similar keyword identification methodology employed by [36]. Thanks to the identification of the main topics, it was possible to obtain the keywords required for the literature search: “Education” and “Apps”. Not only that, but the inclusion and exclusion criteria were established. In the inclusion, it was decided to analyse documents published between 2018–2021. It was also decided to explore educational applications, in any group of the population (children, teenagers, adults, men and women), as well as any approach that these applications have, as long as they provide some type of teaching, information or training. As the main exclusion criteria, it was established that those documents without keywords would not be explored. The next and final step in this stage was the election of the databases (DBs), which were ScienceDirect and IEEE for their wide coverage of multidisciplinary educational work.

Figure 2 SLR process. Figure made by authors..

The search algorithm was developed to obtain the best results (thus beginning Stage 2 – Execution). The first DB used was ScienceDirect, where the most results were obtained. The results were analysed in groups of 50 articles, and those that met the inclusion criteria were downloaded. This process was repeated once more in the IEEE DB, where the number of results was lower than ScienceDirect DB. Regarding the results, over 31.000 scientific documents were found on the ScienceDirect DB. applying filtering by dates, that is, from 2018 onwards, this number was reduced to a still high 14.320 documents. Only those articles whose main topic was healthcare education, general education, Training, rehabilitation and Information, which further reduced the search to approximately 1,782 documents. A same process was applied in the IEEE DB, , where the results were over 10.000 scientific documents. After the initial date filter (2018 onwards), the result was reduced to over 4200 documents, and the second search filter returned 137 documents. The citation of each resulting document was stored in a database, where it was then imported into Excel in order to initiate an exhaustive and even more thorough review of the preselected papers. Finally, 119 documents aligned with the objectives of this work were downloaded.

In the final stage of the SLR (Stage 3 – Result’s and Dissemination), each document was carefully reviewed in order to analyse the origins of those papers or which countries lead the investigation in the Apps used for learning purposes. In addition, Authors, Keywords, Language, Publishing Journals, Funding, Area of the article and Focus by the authors were studied. Finally, a table with recommendations was created as the end point. Figure 2 shows this 3-stage process, with the results.

Table 2 Nations. Table made by author

Nation	Mentions	%
United States	33	27.21%
Australia	11	9.24%
China	8	6.72%
United Kingdom	8	6.72%
Spain	7	5.88%
South Korea	6	5.04%
Brazil	5	4.20%
Italy	5	4.20%
Canada	4	3.36%
Ecuador	4	3.36%

Figure 3 Publishing nations. Figure made by authors.

4 Systematic Literature Review Results

4.1 Source of the Documentation

All the 119 documents were analysed to identify their source. The scientific papers were produced by 46 nations, with the United States being the leading nation with 33 (27.71%) participation of the 119 documents. Australia is the second nation with most participation in the papers found in the DB, with 11 (9.24%) mentions. The top 5 is completed by China with 8 (6.72%) mentions, United Kingdom with also 8 (6.72%) mentions and Spain with 7 (5.88%) mentions. Table 2 shows the first 10 nations with mentions in the documents that make up the DB and Figure 3 shows all the nations found. A note to mention is that most articles were developed with the participation of 2 or more nations.

4.2 Authors

It was found that the articles that make up the database were written by 523 researchers. It was also found that only 8 authors or 1.53% are mentioned as authors or co-authors of 2 or more articles. V. Robles-Bykbaev and Y. Robles-Bykbaev contributed to the creation of 3 articles each one, Antonella Samoggia, F. Pesántez-Avilés, Hadeel Mohammed Jawad, Harald Baumeister, Juliana Chen and Margaret Allman-Farinelli contributed to the creation of 2 articles each one. Table 3 shows the results for those authors with 2 or more contributions.

Table 3 Authors. Table made by author

Author	Origin	Contribution in Articles
V. Robles-Bykbaev	Ecuador	3
Y. Robles-Bykbaev	Ecuador	3
Antonella Samoggia	Italy	2
F. Pesántez-Avilés	Ecuador	2
Hadeel Mohammed Jawad	United States	2
Harald Baumeister	Germany	2
Juliana Chen	Australia	2
Margaret Allman-Farinelli	Australia	2

4.3 Keywords

The keywords of each article were taken and added. Once the sum was completed, 427 different keywords were obtained. The most common words were mHealth and Smartphone, each one used as an identifier or reference in 15 (12.60%) of the 119 downloaded articles. The third and fourth words correspond to Education and Mobile Apps respectively, each used in 13 articles (10.92%) and the fifth word is Mobile Applications, used 12 times (10.08%). Table 4. Shows the 10 most mentioned keywords.

Table 4 Keywords. Table made by author

Keyword	Times Used	%
mHealth	15	12.60%
Smartphone	15	12.60%
Education	13	10.92%
Mobile Apps	13	10.92%
Mobile Applications	12	10.08%
App	11	9.24%
Mobile App	11	9.24%
Technology	8	6.72%
Apps	6	5.04%
Mobile application	6	5.04%

4.4 Language

Unsurprisingly, the English was the most used language in the articles downloaded. Of the 119 scientific papers, 116 (97.48%) were written in English, 2 (1.68%) in Portuguese and only 1 (0.84%) in Spanish. The Figure 4 shows the overwhelming presence of the English language in written articles.

Figure 4 Languages. Figure made by author.

4.5 Publishing Journals

The documents present in the database were published by 92 different journals. Computer and Education published 6 (5.04%) of the 119 downloaded documents. International Journal of Medical Informatics published 5 (4.20%), Internet Interventions published 4 (3.36%), Journal of Affective Disorders published 4 (3.36%) and 2018 IEEE Frontiers in Education Conference (FIE) published 3 (2.52%). Table 5 shows the first 10 journals that published the articles handled in the present SLR.

Table 5 Journals. Table made by author

Journal	Articles Published	%
Computers & Education	6	5.04%
International Journal of Medical Informatics	5	4.20%
Internet Interventions	4	3.36%
Journal of Affective Disorders	4	3.36%
2018 IEEE Frontiers in Education Conference (FIE)	3	2.52%
Computers in Human Behavior	3	2.52%
Nurse Education Today	3	2.52%
Telematics and Informatics	3	2.52%
Archives of Psychiatric Nursing	2	1.68%
Journal of Nutrition Education and Behavior	2	1.68%

4.6 Funding

In the analysis of the downloaded documents, it was found that in 53 (44.54%) of the 119 articles, the authors mentioned having received some type of financial aid. This economical help was made by 74 different institutions around the world. This means that some documents were financed by 2 or more institutions, whether public, private or foundation. The most mentioned financing institution was National Research Foundation of Korea (South Korea) with 3 mentions, followed by European Regional Development Fund (European Institution), National Science Foundation (United States), Spencer Foundation (United States) and UNESCO (World Organization), each one with 2 mentions. The 10 institutions with the most mentions can be seen in Table 6.

Table 6 Funding. Table made by author

Institution	Origin	Mentions in Articles
National Research Foundation of Korea	South Korea	3
European Regional Development Fund	European Institution	2
National Science Foundation	United States	2
Spencer Foundation	United States	2
UNESCO	World Organization	2
Universidad Politécnica Salesiana	Ecuador	2
AMAG pharmaceuticals	United States	1
Arnold Foundation	United States	1
Auckland University of Technology	New Zealand	1
Australian Government	Australia	1

However, if the financial aid topic is looked at from another perspective, the picture is altered. When analysing the location of all the funding institutions, 25 (33.78%) are located in the United States. 7 of them are located in Spain (9.46%), 6 in the United Kingdom (8.11%), 5 in South Korea (6.76%) and 3 in Australia (4.05%). The first 10 countries where the financing institutions are located, can be seen in Table 7.

Table 7 Location of Funding institutions. Table made by author

Institution Location	Mentions	%
United States	25	33.78%
Spain	7	9.46%
United Kingdom	6	8.11%
South Korea	5	6.76%
Australia	3	4.05%
China	3	4.05%
New Zealand	3	4.05%
Brazil	2	2.70%
European Institutions	2	2.70%
France	2	2.70%

4.7 Area of the Article and Focus

In order to better understand the direction that each author wanted to give to their research; it was decided to examine each article from two perspectives. The first objective is to know what area of knowledge their research applies to (Medicine, General Education, Tourism, Science Education…etc.). The second objective was to find the focus of the document ( Effectiveness of an App, Creation of an App, Review…etc.). With this information, figures and a table were elaborated with each of the proposed objectives.

About the area of the articles, each article was examined in order to assign it an area. In the end, 2 main categories were created, the Educational/Learning area and the Medical/Health area as can be seen in the Figure 5 and 6. The main area was General Education, followed by Patient Education, and Fitness/Weight Management tied with Pregnancy & Neonatal Education.

Figure 5 Article’s area part 1. Figure made by author.

Figure 6 Article’s area part 2. Figure made by author.

Table 8 Article’s focus. Table made by author

Focus	Articles	%
Effectiveness	27	22.69%
App Development	26	21.85%
Intention of use	22	18.49%
Review	22	18.49%
Education	9	7.56%
Cultural competences	5	4.20%
Self-care	4	3.36%
Artificial Intelligence	1	0.84%
Augmented Reality	1	0.84%
Drug/Medicines Education	1	0.84%
Relation in physical activities	1	0.84%

The main focus of research was the effectiveness of the apps in a certain area of knowledge or in education in general. Of the 119 scientific documents, 27 (22.69%) focused on this aspect, in finding the effectiveness of a certain application or group of applications. After, is found App Development with 26 documents (21.85%) in which the researchers gave theoretical frameworks for the development of an App for educational purposes or evaluate an existing framework. Intention of use (18.46%), Review (18.59%) and Education (7.56%) closes the top five focus given by authors.

4.8 Type of Document

As the last field of analysis of the database made up of the downloaded articles, it was decided to examine what type of scientific document it is (Research Article, Conference proceeding, Letter, Editorial…etc.). With 70 (58.82%) out of 119 scientific papers being research papers, it is safe to say that this was the most common type of file downloaded. This was followed by 33 (27.73%) Conference proceedings documents, 8 Review articles (6.73%), 2 Clinical trials (1.28%) and 1 Bibliometric analysis (0.84%). There was also 1 Case study (0.84%), 1 Discussion (0.84%), 1 Report (0.84%), 1 Short Communication (0.84%) and 1 Technical note (0.84%). In total, 10 different types of scientific documents were found, which can be seen in Table 9.

Table 9 Type of document. Table made by author

Type of document	Total	%
Research Article	70	58.82%
Conference Proceeding	33	27.73%
Review	8	6.73%
Clinical Trial	2	1.68%
Bibliometric Analysis	1	0.84%
Case Study	1	0.84%
Discussion	1	0.84%
Report	1	0.84%
Short Communication	1	0.84%
Technical Note	1	0.84%

5 Result’s Analysis

At the beginning of this work, 7 questions were presented with the intention to be resolved after the results of the SLR. The first one is Which countries are leading the investigations in Educational Learning Apps (Educational Apps)? The answer is straightforward, the United States is the country producing the biggest amount of scientific documentation about Educational Apps with 27.21% of all the downloaded papers. In a distant second place is Australia with 9.24% of all documentation, and China closes the top three with 6.72%. The second question was Which authors are leading the investigations in Educational Apps? While it is impossible to say with absolute certainty that the authors are the undisputed leaders in the investigations about Educational Apps, it was found that V. Robles-Bykbaev and Y. Robles-Bykbaev, each with 3 mentions, are the researchers with the most references of contribution among the 523 investigators in the downloaded sample. In the third question, What were the most used keywords by the authors? The first 10 keywords are, sort by use, mHealth, Smartphone, Education, Mobile Apps, Mobile Applications, App, Technology, Apps and Mobile Application. Perhaps the most interesting fact is that the most used Keyword is mHealth or Mobile Health, because it indicates that a large portion of the scientific documents are focusing on teaching about health, using mobile devices. The fourth question, Which are the journals with the most publications on Educational Apps? After studying the author’s DB, it was discovered that Computers & Education is the journal with the most published documents about the topic of these investigations, with 6 articles. The fifth question, If there is funding to investigate Mobile Learning Apps, where are these coming from? Have a clear result, of course there is funding; in fact, 44.54% of all the documentation in the downloaded sample have been financed by an institution. Likewise, the major contributor found was the National Research Foundation of Korea, which although it is only mentioned 3 times, is the entity with the highest number of mentions. The sixth question, What is the focus of the papers about Educational Apps? Can be resolved saying that articles are focusing on the effectiveness of the Mobile Learning Apps, if they accomplish the purpose of educating the target audience. The seventh question, What are the main recommendations when creating an educational app? Is answered with a series of references or impact factors mentioned by several authors that can make it not only more accessible and engaging, but more effective, like the creation of an app with the participation of adequate personal or professionals and the feedback of the users. This answer will be deepened in the following section.

6 Recommendations

The main objective of an Educational app is, as his name says, educate. However, if the app does not engage the student, this objective will not be accomplished. Likewise, if the app is not correctly developed with the appropriate content, it may engage the student, but will not fulfill its purpose.

As the first step to generate an app with educational value, it is important to create those Mobile Learning Apps with the participation of an educator or expert in the required field.

It is essential to understand that the effectiveness of educational applications is not inherent to technology. This effectiveness is due to a combination of a correct educational methodology by the teacher and a correct use by the student [10, 37]. Other studies have highlighted the attitude of users towards an application and the relationship between ease of use and attitude towards new technologies [13, 38]. Also, the quality of the content and its interactivity has repercussions on student learning [26]. Finally, an appropriate user’s guide or training can be beneficial for the parties that use the application. Applying a correct development and management of the mentioned factors (shown in Table 10) can lead to an adequate interest on the part of a student. A constant communication with the users of the app, constitutes a good measure to verify that it is fulfilling the educational objectives that were raised.

Table 10 Impact factors in mobile learning apps. Table made by author.

Impact Factors in Mobile Learning Apps
Co-development between educator and programmer
Educational Methodology by educator
Appropriated use by student
Ease of use of the App
Quality of the content
Interactivity of the content
Appropriated user’s guide / Usage training

7 Discussion

The educational benefits that Mobile Applications can have is undeniable. However, everyday new Educational Apps are being released in the Apple App Store and Google Play Store, and that is without taking in consideration emerging marketplaces like the Huawei AppGallery. The economic potential is enormous and many companies considerer the monetary gaining before the learning process and design apps to engage the user; only few companies use studies and a proper guidance to develop educational app. Additionally, many studies that are the basis for the developers are lackluster, do not give information on how those studies and the validation parameters were conducted, and sometimes do not even mention any study or framework as a pillar [12, 39–43].

A study by [40] raised a concern about how little educational value the so-called Educational Apps sold in the Apple App Store and Google Play Store have yet everyday new Educational Apps are added to the marketplaces without a proper regulation that give this title, [29] expressed concern on the lack of studies on the influence of mobile learning tools applied in the educational field, [44] mentions the lack of studies between the creativity of early age students and game-based learning environment, [45, 46] also mentions the lack of studies confirming the efficiency of fitness apps to educate for proper exercising and feeding habits, [47] concludes in his research that the certain apps to teach about business negotiation do not have an adequate educational quality and [48] expressed the lack of understanding and awareness of apps focused on pharmacological education.

An interesting result of SLR is the approach that many scientists are taking to their research. 22.69% of the documents contained in the DB are investigations focused on investigating the efficiency, either of one or more Apps (already existing or newly created) [43, 47, 49–52] in different areas of knowledge such as Health, Pregnancy, Language to mention just a few, to verify the educational benefits of these. This result seems to be aligned with the expressed need on the part of the scientific community to evaluate these Mobile Learning Apps.

Two possible solutions are proposed to solve this problem. The first would consist of the creation of a theoretical framework that can establish the efficiency of a specific educational application. This seems like a difficult task to complete due to the number of different factors that should be considered, such as the target population and the learning information that should be evaluated under the same framework. However, it would be prudent to study the feasibility of this proposal. The second proposal would be the establishment of a regulatory body, which with the help of experts on the subject, can certify the educational factor of a determinate App.

The truth is that the need to verify the educational value of the applications is immense and the discussion on this subject cannot wait any longer.

8 Discussion

Technological advances have helped to create different types of teaching tools, from which children to teachers take advantage. One of these tools are the Mobile Learning Applications, Learning Apps or Educational Apps, which has proven to be flexible, provide excellent teacher-student communication and facilitate the acquisition of knowledge.

This SLR offers the answer to certain questions, like the country leading the investigations on Educational Apps/Mobile Learning App, authors with the highest number of contributions, most used keywords, funding role in Mobile Apps research, the focus of the examined scientific papers and the most common recommendations. With this information, can be considered an excellent guide, mainly to researchers who are starting an investigative process on Mobile Learning Apps. Also, there are answer for questions no planted

While there are results that might not surprise those familiar with the subject (like the dominance of the investigators from the United State) there are multiple outcomes that are interesting to analyse, like the surprisingly high number of scientific articles that receive financial aid for their development.

However, we can consider the highlight of this work, the need expressed by multiple authors in their works, about the lack of a correct evaluation and development of educational applications. This can be the reason why the major focus of the researchers in the studied documents is the effectiveness of one or more Educational Apps.

An important finding and not mentioned in many consulted literature reviews, is the role of financial aid in the researcher’s work. With the 44.54% of the consulted documents acknowledging having received some type of funding, the institutional monetary aid has become an important support for the elaboration of investigations in the Educational App field.

Lastly, based on the information deposited by the different authors in their articles, a recommendation table was elaborated to serve as a guide for those seeking to develop an Educational App with high educational value.

References

[1] United Nations, “Progresos realizados en la aplicación y el seguimiento de los resultados de la cumbre mundial sobre la sociedad de la información a nivel regional e internacional,” 2019.

[2] L. C. M. Miranda and C. A. S. Lima, “Trends and cycles of the internet evolution and worldwide impacts,” Technol. Forecast. Soc. Change, vol. 79, no. 4, pp. 744–765, 2012.

[3] UNESCO, “ICT Competency Standards for Teachers,” 2008.

[4] M. Vásquez-Carbonell and J. Silva-Ortega, “Tendencias y características de la realidad virtual: Una revisión de la literatura entre los años 2017 y 2018,” Comput. Electron. Sci. Theory Appl., vol. 1, no. 1, pp. 36–70, 2020.

[5] H. K. Wu, S. W. Y. Lee, H. Y. Chang, and J. C. Liang, “Current status, opportunities and challenges of augmented reality in education,” Comput. Educ., vol. 62, pp. 41–49, 2013.

[6] C. Y. Chang, C. L. Lai, and G. J. Hwang, “Trends and research issues of mobile learning studies in nursing education: A review of academic publications from 1971 to 2016,” Comput. Educ., vol. 116, pp. 28–48, 2018.

[7] X. L. Pham, T. H. Nguyen, and G. D. Chen, “Research Through the App Store: Understanding Participant Behavior on a Mobile English Learning App,” J. Educ. Comput. Res., vol. 56, no. 7, pp. 1076–1098, 2018.

[8] A. J. Williams, S. Ekins, A. M. Clark, J. J. Jack, and R. L. Apodaca, “Mobile apps for chemistry in the world of drug discovery,” Drug Discov. Today, vol. 16, no. 21–22, pp. 928–939, 2011.

[9] J. T. Piotrowski and L. Meester, “Can apps support creativity in middle childhood?,” Comput. Human Behav., vol. 85, pp. 23–33, 2018.

[10] S. Adams Becker, M. Cummins, A. Freeman, C. Hall Giesinger, and V. Ananthanarayanan, “RESUMEN INFORME HORIZON Edición 2017 Educación Superior,” 2017.

[11] I. V. Osipov, A. Y. Prasikova, and A. A. Volinsky, “Participant behavior and content of the online foreign languages learning and teaching platform,” Comput. Human Behav., vol. 50, pp. 476–488, 2015.

[12] K. J. Dickinson and B. L. Bass, “A Systematic Review of Educational Mobile-Applications (Apps) for Surgery Residents: Simulation and Beyond,” J. Surg. Educ., vol. 77, no. 5, pp. 1244–1256, 2020.

[13] S. Han and Y. J. Yi, “How does the smartphone usage of college students affect academic performance?,” J. Comput. Assist. Learn., vol. 35, no. 1, pp. 13–22, 2019.

[14] P. Dias and R. Brito, “Criteria for selecting apps: Debating the perceptions of young children, parents and industry stakeholders,” Comput. Educ., vol. 165, no. July 2020, pp. 1–13, 2021.

[15] W. Sanders, J. Parent, R. Forehand, A. D. W. Sullivan, and D. J. Jones, “Parental perceptions of technology and technology-focused parenting: Associations with youth screen time,” J. Appl. Dev. Psychol., vol. 44, pp. 28–38, 2016.

[16] G. Lippi and M. Plebani, “Laboratory applications for smartphones: Risk or opportunity?,” Clin. Biochem., vol. 44, no. 4, pp. 273–274, 2011.

[17] C. Lee Ventola, “Mobile devices and apps for health care professionals: Uses and benefits,” P T, vol. 39, no. 5, pp. 356–364, 2014.

[18] A. Samoggia and B. Riedel, “Assessment of nutrition-focused mobile apps’ influence on consumers’ healthy food behaviour and nutrition knowledge,” Food Res. Int., vol. 128, no. October 2019, p. 108766, 2020.

[19] S. K. Park, C. H. Bang, and S. H. Lee, “Evaluating the effect of a smartphone app-based self-management program for people with COPD: A randomized controlled trial,” Appl. Nurs. Res., vol. 52, no. April 2019, p. 151231, 2020.

[20] E. Quattromani, M. Hassler, N. Rogers, J. Fitzgerald, and P. Buchanan, “Smart Pump App for Infusion Pump Training,” Clin. Simul. Nurs., vol. 17, pp. 28–37, 2018.

[21] S. Sung and H. A. Park, “Effect of a mobile app-based cultural competence training program for nurses: A pre- and posttest design,” Nurse Educ. Today, vol. 99, no. January, p. 104795, 2021.

[22] X. mei Fu, “Does heavy ICT usage contribute to the adoption of ride-hailing app?,” Travel Behav. Soc., vol. 21, no. October 2019, pp. 101–108, 2020.

[23] C. del P. Gallardo Montes, A. Rodríguez Fuentes, and M. J. Caurcel Cara, “Apps for people with autism: Assessment, classification and ranking of the best,” Technol. Soc., vol. 64, no. October 2020, p. 101474, 2021.

[24] L. Hoareau, A. Thomas, Y. Tazouti, J. Dinet, C. Luxembourger, and A. Jarlégan, “Beliefs about digital technologies and teachers’ acceptance of an educational app for preschoolers,” Comput. Educ., vol. 172, no. June, p. 104264, 2021.

[25] F. C. Blumberg et al., “Digital Games as a Context for Children’s Cognitive Development: Research Recommendations and Policy Considerations,” Soc. Policy Rep., vol. 32, no. 1, pp. 1–33, 2019.

[26] L. Crescenzi-Lanna, “Emotions, private speech, involvement and other aspects of young children’s interactions with educational apps,” Comput. Human Behav., vol. 111, no. February, p. 106430, 2020.

[27] J. N. Druckman and C. D. Kam, “Students as Experimental Participants: A Defense of the ‘Narrow Data Base,”’ SSRN Electron. J., 2011.

[28] N. Henze and M. Pielot, “App stores: External validity for mobile HCI,” Interactions, vol. 20, no. 2, pp. 33–38, 2013.

[29] H. Crompton and D. Burke, “The use of mobile learning in higher education: A systematic review,” Comput. Educ., vol. 123, no. September 2017, pp. 53–64, 2018.

[30] N. Kucirkova, “A framework for guiding design-based research for iPad apps.,” Br. J. Educ. Technol., 2016.

[31] G. O. Anderson, “Getting connected: Older Americans embrace technology to enhance their lives,” 2018.

[32] U. Kenny and Á. Regan, “Co-designing a smartphone app for and with farmers: Empathising with end-users’ values and needs,” J. Rural Stud., vol. 82, no. July 2020, pp. 148–160, 2021.

[33] C. D. Mulrow, “Rationale for systematic reviews,” Br. Med. J., vol. 309, no. 6954, pp. 597–599, 1994.

[34] J. Ramey and P. G. Rao, “The systematic literature review as a research genre,” IEEE Int. Prof. Commun. Conf., 2011.

[35] N. Pulsiri and R. Vatananan-Thesenvitz, “Improving systematic literature review with automation and bibliometrics,” PICMET 2018 – Portl. Int. Conf. Manag. Eng. Technol. Manag. Technol. Entrep. Engine Econ. Growth, Proc., no. Figure 1, 2018.

[36] K. B. Debnath and M. Mourshed, “Forecasting methods in energy planning models,” Renew. Sustain. Energy Rev., vol. 88, no. March, pp. 297–325, 2018.

[37] M. I. Yousuf, “Effectiveness of mobile learning,” Turkish Online J. Distance Educ., vol. 8, no. 4, pp. 114–124, 2007.

[38] J. C. Gu, S. C. Lee, and Y. H. Suh, “Determinants of behavioral intention to mobile banking,” Expert Syst. Appl., vol. 36, no. 9, pp. 11605–11616, 2009.

[39] K. Hirsh-Pasek, J. M. Zosh, R. M. Golinkoff, J. H. Gray, M. B. Robb, and J. Kaufman, Putting Education in “Educational” Apps: Lessons From the Science of Learning, vol. 16, no. 1. 2015.

[40] S. Papadakis, M. Kalogiannakis, and N. Zaranis, “Educational apps from the Android Google Play for Greek preschoolers: A systematic review,” Comput. Educ., vol. 116, pp. 139–160, 2018.

[41] S. M. Schueller, C. M. Armstrong, M. Neary, and R. P. Ciulla, “An Introduction to Core Competencies for the Use of Mobile Apps in Cognitive and Behavioral Practice,” Cogn. Behav. Pract., 2021.

[42] D. M. Saboia, C. T. M. Vasconcelos, M. O. B. Oriá, K. de C Bezerra, J. A. Vasconcelos Neto, and M. H. B. de M Lopes, “Continence App: Construction and validation of a mobile application for postnatal urinary incontinence prevention,” Eur. J. Obstet. Gynecol. Reprod. Biol., vol. 240, pp. 330–335, 2019.

[43] S. Laato et al., “Evaluation of the Pedagogical Quality of Mobile Math Games in App Marketplaces,” Proc. – 2020 IEEE Int. Conf. Eng. Technol. Innov. ICE/ITMC 2020, 2020.

[44] N. Behnamnia, A. Kamsin, and M. A. B. Ismail, “The landscape of research on the use of digital game-based learning apps to nurture creativity among young children: A review,” Think. Ski. Creat., vol. 37, no. May, p. 100666, 2020.

[45] Y. Liu and M. Avello, “Status of the research in fitness apps: A bibliometric analysis,” Telemat. Informatics, vol. 57, no. July 2020, p. 101506, 2021.

[46] É. Gál, S. Ştefan, and I. A. Cristea, “The efficacy of mindfulness meditation apps in enhancing users’ well-being and mental health related outcomes: a meta-analysis of randomized controlled trials,” J. Affect. Disord., vol. 279, no. March 2020, pp. 131–142, 2021.

[47] T. Q. Nguyen, T. N. Dang, C. L. Hoang, D. T. N. Truong, and V. A. Nguyen, “Educational Mobile Apps for Negotiation: Evaluation and Design Criteria for Curriculum Designers and Developers,” 2019 IEEE Can. Conf. Electr. Comput. Eng. CCECE 2019, 2019.

[48] K. L. Donohoe, A. T. Matulewicz, F. M. Alotaibi, and K. C. Ogbonna, “Medical apps used during advanced pharmacy practice experiences,” Curr. Pharm. Teach. Learn., vol. 10, no. 2, pp. 195–200, 2018.

[49] A. Larco, F. Enriquez, and S. Lujan-Mora, “Review and evaluation of special education iOS apps using MARS,” EDUNINE 2018 – 2nd IEEE World Eng. Educ. Conf. Role Prof. Assoc. Contemp. Eng. Careers, Proc., pp. 5–10, 2018.

[50] F. Veiga and A. Andrade, “Evaluation of apps used in an educational context,” Iber. Conf. Inf. Syst. Technol. Cist., vol. 2020-June, no. June, pp. 24–27, 2020.

[51] M. Stec and M. W. Arbour, “Wellness and Disease Self-Management Mobile Health Apps Evaluated by the Mobile Application Rating Scale,” Adv. Fam. Pract. Nurs., vol. 2, pp. 87–102, 2020.

[52] J. Oh, S. Jang, H. Kim, and J. J. Kim, “Efficacy of mobile app-based interactive cognitive behavioral therapy using a chatbot for panic disorder,” Int. J. Med. Inform., vol. 140, no. May, p. 104171, 2020.

Biography

Mauricio Vásquez-Carbonell is an Electronic Engineer and Master in Engineering from the Universidad de la Costa (Barranquilla – Colombia). Assistant Professor II of the Universidad de la Costa (Department of Computer Science and Electronics). His work experience is related to the health sector, in the development of tools for teaching and learning in clinical patients and students. Currently focused on the development of virtual reality, programs and educational apps, as well as multiple aspects of applied education.