Investigating Role of Artificial Intelligence for Preparing Question Paper Till Evaluation Process in Open Book Examination System
Amit Dimari1, Nidhi Tyagi1,*, S. Srinivasan1, Mahesh Davanageri2, Prabha Lama1 and Hema Dimari3
1Humanities and Social Sciences, Graphic Era Deemed University Dehradun – 248001, India
2Graphic Era (Deemed to be) University Dehradun – 248001, India
3Examination Cell, Graphic Era hill University Dehradun – 248001, India
E-mail: amit.dimari66@gmail.com; nidhityagi.hss@geu.ac.in; s.srinivasan.hss@geu; coe@geu.ac.in; prabhalama.hss@geu.ac.in; hdimari@geu.ac.in
*Corresponding Author
Received 08 August 2024; Accepted 01 February 2025
Abstract
In the rapidly evolving landscape of higher education, the integration of AI in various assessment techniques presents before us a revolutionary potential for grading systems. This research study aimed to investigate the role of Artificial Intelligence (AI) in open-book examinations, with a focus on question paper preparation and evaluation processes. The research employed a Quantitative approach, including surveys and data analysis techniques. The sample size of 165 participants was taken as a sample in the study which includes students, educators and administrators from various academic institutions who have implemented open book examination systems. Participants were chosen on the basis of purposive sampling technique that have vast experience in AI technologies and open book system as well. The findings revealed significant differences and associations between AI integration and traditional methods in open-book examinations. Challenges identified in traditional methods included time constraints and subjectivity in evaluation. The t-test analysis showed significant differences in the feasibility and effectiveness of open-book examinations between AI-integrated and traditional methods. There was a significant association between traditional question paper preparation and evaluation methods and the identified challenges. Additionally, a significant difference was found in the benefits and advantages of using AI technologies in question paper preparation for open book exams. The authors arrived at the resolution that further examination and educator preparation are expected to completely get a handle on the capability of artificial intelligence in instructive assessment, especially schooling level lower than advanced education.
Keywords: Artificial intelligence, open book examinations, question paper preparation, evaluation process, feasibility.
Introduction
Open book examination system holds a significant position in the era of AI. Artificial Intelligence (AI) can play a significant role in various aspects of the examination process, particularly in an open-book examination system. Here are some aspects of AI that could be utilized from preparing question papers to evaluating student responses. AI can be used to analyse previous question papers, syllabi, and learning materials to create relevant, balanced questions. Natural Language Processing (NLP) techniques can be employed to ensure the clarity and proper order of questions. AI can also help ensure that questions are appropriately aligned with learning objectives and Bloom’s taxonomy levels. In question bank management, AI can prepare dynamic question banks and continuously curate questions based on relevance and difficulty level. AI can categorize questions based on topics, subtopics, and retrieval levels, facilitating the preparation of question papers. One of the most significant advancements of AI is its ability to detect suspicious behaviours, such as eye movements and attempts to access fingerprints. Evaluation of answer sheets can be done through Natural Language Understanding (NLU) and NLP techniques, which can assess the quality of the evaluation process and student responses. Machine learning models trained on annotated data can automatically grade subjective answers based on predefined criteria. AI can also adapt the difficulty of questions based on student performance, enhancing the examination experience. It can easily identify students’ strengths and weaknesses in examinations and provide individualized learning assessments.
Accessibility and Inclusivity in Machine Learning
AI can improve accessibility for students with disabilities by providing various alternative supports during examinations. Speech recognition and text-to-speech systems can assist students with visual or auditory impairments during examinations. AI utilizes various benefits, such as the examination process, to address concerns such as bias, fairness, privacy, and security.
Duan, Y., Edwards, J. S., and Dwivedi, Y. K. (2019), stated that AI technologies are “technologies to augment human intelligence.” This is a partnership between man and machine and is expected to make us better, allowing us to do what the human condition is best able to do. Stephen Hawking remarked that the development of AI could spell the end of the human race (Cellan-Jones, 2014), and Bill Gates has also expressed concerns about the threat posed by AI (Rawlinson, 2015). Quille, K., et al. (a) (2021) emphasized that research after the COVID-19 pandemic led to a host of changes in the delivery and assessment of academic programs. Educational programs moved online using digital tools such as Microsoft Teams, Zoom, WebEx, Slack, Google Meet, and others. Apart from the educational field, all departments underwent significant changes, presenting unprecedented challenges in the educational sector. Lecturers were forced to deliver content and assessments online. John McCarthy oversaw a two-month studio on the topic of artificial intelligence at Dartmouth University in the 1950s. McCarthy used the term “artificial intelligence” (p. 17) in his studio proposal, which was unheard in conjectured that every aspect of learning or any other feature of intelligence should be examined. The current research study mainly focused on the problem of AI, which is to make a machine behave in ways that would be considered intelligent if a human were behaving that way. Quille, K., et al. (b) (2021) found that open-book online exams had positive effects, with 85% of students reporting a preference for online open-book examinations over traditional ones. The majority of respondents preferred the open-book assessment system. Richardson, M., and Clesham, R. (2021) explored how AI technology effectively works for open-book systems in various fields. Many international and national testing agencies assess millions of English as a Second Language (ESL) students each year to learn listening and reading skills for competitive admissions to English-speaking universities (Chapelle and Chung, 2010). Confidence is crucial in high-stakes testing, and while it is not always considered, it is critical to the value of assessments that wield such influence. Mellit, A., and Kalogirou, S. A. (2008) stated that AI-based systems are being developed and deployed worldwide in a wide variety of applications, including symbolic reasoning, flexibility, and explanation capabilities. AI is used in various fields such as accounting, economics, medicine, the military, education, engineering, marine science, social science, and forecasting systems. Suhaimy, K. A. M. (2012) conducted an empirical research study using stratified sampling of 995 students at Kolej Universiti Teknologi Tun Hussein Onn (KUiTTHO). The study compared open-book and closed-book examinations, with the results showing that students performed better in open-book examinations. The study reflected that if children can think critically and creatively, they tend to perform better when answering open-ended questions and problem-solving tasks. When students are allowed to bring reference materials into examinations, their learning is improved, leading to better comprehension and responses.
On the assumption that learning and other forms of intelligence can be described precisely enough for a computer to be constructed to duplicate those, researchers will continue to examine an expanded definition of artificial intelligence is provided by Baker and Smith (2019): “PCs which perform mental errands, as a rule related with human personalities, especially learning and critical thinking” (p. 10). They understand that AI on computers is not a singular breakthrough. It’s a catch-all phrase for describing a wide range of developments and methods, such as ML, NLP, IM, org charts, and computation, to name a few.
Both artificial intelligence and machine learning are frequently used in the same context. To predict a student’s likelihood of dropping out of a course or being admitted to a program, or to identify subjects in created assignments, machine learning is a technology for simulated intelligence for managed and solo categorization and profiling. Machine learning is defined by Popenici and Kerr (2017) as “a branch of AI that incorporates programs able to perceive designs, make expectations, and apply newly discovered examples to circumstances that were excluded or covered in their underlying plan” (p. 2). Artificial intelligence can be either weak and or narrow and broad (Baker and Smith, 2019, p. 10), but these distinctions are clear to experts in the field. The philosophical question of whether machines will desire to truly think or even nurture cognizance in the future, as opposed to merely simulating thought and normal behavior, remains open. A universal or stable artificial intelligence cannot be created in the foreseeable future. Thus, controlling GOFAI (“typical simulated intelligence,” coined by rationalist.
Considering this knowledge of AI, where may we find applications of simulated intelligence in formal education, and graduate studies in particular? Individual mentors, clever assistance for cooperative learning, and smart computer-generated experience are the three types of AI-based training applications described by Luckin, Holmes (2016).
Coordinated individual coaching can be simulated with the use of insightful mentoring frameworks (ITS). They can use student models, computations, and brain structures to make decisions regarding an individual student’s learning style and the content to teach, as well as to provide a mental platform for the student and help. Particularly at large-scale distance-learning institutions that conduct modules with large numbers of students and where human-coordinated tutoring is impossible, ITS holds tremendous promise. Extensive research by Jonassen, Davidson, Collins, Campbell, and Haag, 1995) demonstrates that learning is a social activity; communication and coordinated effort are essential components of the learning system. However, online collaboration needs to be managed. By facilitating the creation of flexible groups based on student models, facilitating online group interaction, or summarising dialogues that a human mentor might use to guide students toward the course’s points and objectives, Artificial Intelligence Education can contribute to cooperative learning. Finally, intelligent virtual reality (IVR) is used to engage with and lead students in realistic computer-generated experiences and game-based learning circumstances, again drawing on ITS. Virtual specialists can go about as educators, facilitators, or understudies’ companions, for instance, in virtual or far off labs (Perez et al., 2017).
Luckin et al. (2016) predict a “[r] enaissance in evaluation” (p. 35) thanks to the development of AIEd and the availability of (large) student data and learning research. Input and assessment are not time-consuming processes for artificial intelligence.
Baker and Smith’s (2019) new report takes three perspectives on artificial intelligence in education: (a) the students, (b) the teachers, and (c) the frameworks. Software like adaptable or personalized learning management systems (LMSs) and interactive whiteboard systems (ITSs) are examples of student-facingcbi tools. By automating tasks like organization, evaluation, input, and copyright infringement location, instructor-facing frameworks benefit teachers and reduce their workload. AIEd tools also provide insight into student learning progress, allowing teachers to proactively provide support and guidance where it is needed. Institutional-level frameworks against AIEd are tools that provide information to heads and chiefs, for instance, to monitor wearing-down designs across resources or schools. This integration of AI in examination system undoubtedly raises a question on the sustainability of traditional assessment pattern as well.
But at the same time AI can introduce un–reliability, bias and low level of explicability which can further compromise with the validity of exams and reduce trust in the system. At the same time, the frequent use of AI chat bots in the online assessments increases the concern of legitimacy and integrity of the tests. Thus there is a need to measure the attitude of different stakeholders across various assessment scenarios for examining the quality of online assessments.
Development of Open-Book Assessments
Quille, K. (2021), in the context of open-book assessments, emphasizes the importance of higher-order questions to test critical thinking and enhance skills. Open-book assessments encourage educators to pose questions that go beyond mere cognitive recall levels. While there is no definitive guide to creating open-book assessments in computing. Dawson, P. (2023), argues in his research that there are three dimensions – information, people, and tools – each existing on a spectrum rather than being all or nothing. Additionally, there are five criteria for learning outcomes: feasibility, consequential validity, authenticity, and values to guide them. The current research aims to explore the taxonomy and criteria, providing ways of thinking about restrictions in examinations that can prompt educators towards examination designs that are more valid and robust against cheating. Roberts, L., and Berry, J. (2023), focused mainly on open-book, open-web (OBOW) assessments used during the pandemic period as an option in higher education to enhance accessibility. OBOW assessments can reduce administrative load. It is suggested that during exams, the perception of the development of higher-order cognitive skills and reduced test anxiety due to assessments might enhance employability.
Research Objectives: The present study has been carried out with the following objectives
• To Assess the feasibility and effectiveness of integrating artificial intelligence (AI) in the entire process of open book examinations, from question paper preparation to evaluation.
• To Identify the key challenges and limitations associated with traditional question paper preparation and evaluation methods in open-book examination systems.
• To Explore the potential benefits and advantages of using AI technologies, such as natural language processing (NLP) and machine learning algorithms, in question paper preparation for open book exams.
Research Hypothesis
Alternative Hypothesis (H1A): There is a significant difference in the feasibility and effectiveness of open book examinations when AI is integrated compared to traditional methods.
Alternative Hypothesis (H2A): There is a significant association between traditional question paper preparation and evaluation methods and the identified challenges and limitations in open book examination systems.
Alternative Hypothesis (H3A): There is a significant difference in the benefits and advantages of using AI technologies, such as NLP and machine learning algorithms, in question paper preparation for open book exams compared to traditional methods.
Literature Review
Farhi et al., 2023; Fullen et al., 2023; Ottenbreit–Leftwich et al., 2023 in their study observed that AI in the field of education can be a crucial stride towards creating a far more personalized and more effective learning experience which prepare students for the present challenges in the field of academia and also prepare them for more and more evolving demands in the future to come.
Studies conducted by Van Den Berg and Du Plessis, 2023 highlighted the fact that the use of AI in the field of education can foster critical thinking of the students for it involves the capability to analyze the information in a more refined way, evaluate differential perspectives and at the same time can create logical arguments all within the particular framework of AI driven environment. This skill is extremely important for the students with the prevalence of increased role of AI in different and multiple aspects of life. AI in the field of education is not only confined to delivery of content, but also for encouraging the students to deeply think and analyse with respect to the information they are presented with.
In a study conducted on, Application of Artificial Intelligence in Exam Evaluation’, 2024 the experimental results throw light on the fact that in comparison to manual grading the consistency rate of human machine in order to identify the multiple gaps in the blank filling questions exceeded to that of 94% and the consistency rate which was calculated on the basis of small questions also exceeded to 97%. It means that for quality inspection and grading the evaluation system through machine can serve as an auxiliary means and at the same time their effectiveness is far more reliable. With respect to evaluation of subjective questions there is a high amount of correlation in between the results produced by machine and manually rated final report scores.
The sector of education is one of the most relevant areas for artificial intelligence applications. Numerous experts in the field of artificial intelligence attended the Beijing Gathering in 2019 and came to an agreement on the “Beijing Agreement on computer-based intelligence and Education” (UNESCO, 2019). They discuss the importance of developing artificial intelligence in education in this repository, as stated in the fourth Practical Improvement Objective established by UNESCO (2019). To provide deep rooted learning valuable open doors and remove barriers in the computerized world, the following are the main activities to advance: to design educational arrangements; to advance purposes for education the board; to engage educators and students; to advance qualities and abilities forever and work; to energise impartial and comprehensive purposes, including orientation uniformity; moral and straightforward purposes; to explore, assess, and uphold.
A few remarkable real-world experiences involving artificial intelligence and the Skyline Report are available. Chew, et al. (2020) compile some of them, including the IBM Watson Guide, which has been tested with teachers and students in Texas or with Material LMS; another experience is from UC San Diego, the DSMLP (Information Science/Machine Learning Stage), which uses machine learning to give access to resources and understudy projects; the third model is Lift, a mobile application integrated with Material LMS, an individual collaborator for web-based learning. According to these experts Chew, et al. (2020), “regardless of moral worries, the advanced education area of artificial intelligence applications connected with instructing and learning is projected to p develop altogether (Liu et al.,2017).
A significant report from 2019 (UNESCO) looks at three areas: learning with artificial intelligence (using these tools in education), learning about artificial intelligence (using these tools), and planning for artificial intelligence (understanding how these tools have the potential to change us). According on how these tools are used in education, it is possible to find a variety of experiences, but most of these are related to mentoring or evaluation.
Ocaa-Fernández, et al. (2019) and Gálvez (2013) conducted fascinating studies on effective coaching frameworks, which report that their development is still tightly related to learning research and human natural language processing frameworks. This variety of frameworks depends on the ability of effective systems to deliver criticism of a calibre that can occasionally augment and replace the educator’s effort. It is an important advancement to understand that these frameworks provide tailored responses given the wide range of mind-boggling projects that students can produce. They include some very notable models, such as Duolingo, which is the noticeable exhibit of the openness of utilization in light of the cooperation among machines and people, depending on simulated intelligence standards. Ocaa et al. (2019) state that simulated intelligence will predominantly affect the course of customized education as a result of the robotized help, particularly with regards to virtual connection.
According to Narciss et al. (2014), one of the domains with greater educational importance associated with PC-based advancements is customized coaching critique. These designers focused on an electronic interactive learning environment for maths that would handle chores and track students’ progress. They assumed that there are some differences in criticism productivity related to orientation because, right away, women benefit more from mentorship input conditions (especially when input depends on calculated hints). Last but not least, young males shown an increase in natural inspiration while young women continued to grow their seeming talent more than young men.
By utilising machine learning and agendas, Jani et al. (2020) demonstrate the usefulness of IA in developmental assessment as well as criticism and evaluation. Its investigation revealed that pre-programmed responses were excellent tools for tracking students’ progress and identifying areas where therapeutic practises should be focused.
Santos and Boticario’s (2014) experience is particularly fascinating because it leverages artificial intelligence to support cooperative learning situations. In light of computer-based intelligence, they suggest a Cooperative Sensible Structure to improve dialogue, collaboration, and teamwork as educational learning strategies. Additionally, they employ this clever assistance to monitor students’ behaviour, relieving educators of some of their responsibilities. It is a flexible leadership framework created for an e-learning platform (dotLRN) that can aid in the interaction of the administration and the board of students.
In their study on artificial intelligence applications in higher education, Ocaa-Fernández et al. (2019) claim that smart mentor “have offered strong help on a few subjects from their unassuming beginnings; themes like preparation in topography, circuits, clinical conclusion, figuring and programming, hereditary qualities and science” and that they can be crucial tools to work on the potentially universal web-based learning in the future. Some applications for providing coaching and input are also considered for review, such as Samarakou et al. (2016) and Saplacan et al. (2018).
Research Methodology
Design of the Study
The research adopted a quantitative approach for data collection and analysis techniques. The study involved surveys, interviews, and empirical analysis to gather comprehensive data. The data were analyzed through a t-test, which is used to determine the statistically significant difference between two groups. It can gather the mean value and significantly improve the hypothesized population mean. It can handle the sample size and provide an approximately normal distribution. The reason for using the t-test is to compare the performance of students, teachers, and educators regarding the open-book system in examinations and its role in supporting AI technologies. The test is more useful and effective for comparing two groups or within-group differences under different conditions of the study. The sample size was very low, but the t-test can measure robust statistical aspects to understand and draw inferences. The test hypothesis can help with open-book exams as it is perceived to provide effective AI technology and can have a positive impact.
Research Approach
The research approach for the study would involve a quantitative method, this approach allows for a comprehensive investigation of the research topic by gathering in numerical data (quantitative).
Sample Population
The sample consisted of students, educators, and administrators from educational institutions that had implemented open book examination systems. Participants were selected using a purposive sampling technique, targeting individuals with experience in open book exams and AI technologies.
Sample of the Study
The sample for the study would consist of students, educators, and administrators from educational institutions that have implemented open book examination systems. The sample size of 165 participants was taken as a sample in the study.
| Sample Selection | Sample |
| Students | 100 |
| Educators | 30 |
| Administrators | 35 |
| Total | 165 respondents |
Sampling Technique
A purposive sampling technique was used to identify individuals who have experience with open book exams and AI technologies. Purposive sampling technique, once in a while alluded to as judgemental, specific, or emotional examining, is a sort of non-likelihood examining where specialists pick members for their reviews by utilizing their own judgment.
Variables of the Study
Independent Variables:
• Integration of AI in question paper preparation
• Integration of AI in evaluation
Dependent Variables:
• Feasibility of AI integration
• Effectiveness of AI integration
• Benefits of AI integration
• Challenges and limitations
Collection of Data
For descriptive research investigations, I collected data using self-structured questionnaires. Before developing the research tool, researchers conducted a thorough literature evaluation on continuous professional development.
Primary study: Primary data was collected through surveys and interviews conducted specifically for the research. Surveys were used to gather information directly from participants, including their perceptions, attitudes, and experiences regarding AI in open book exams. The survey questions were designed to assess the feasibility, effectiveness, benefits, challenges, and limitations of AI integration.
Secondary data: secondary data collection methods were utilized to gather existing information and insights related to AI in education and open book exams. A thorough literature review was conducted, examining academic journals, research papers, conference proceedings, and relevant publications to establish a theoretical framework and gain background knowledge.
Data Analysis
Percentage: – Percentage calculations were used for easy comparisons.
Frequency: – Frequency estimates the number of events that happen in each class. It gives a fast visual of the number of distinct perceptions that are included and where the heft of the information is concentrated. The degree and improvement phase of scattering are additionally shown. Subsequently, the motivation behind the current exploration is to analyse subject appropriations between bunches by calculating the inquiry recurrence.
Ethical Consideration
To obtain informed consent from the participant and ensure the respondent’s right to provide voluntary consent. The confidentiality of information must be protected, and the data privacy must be maintained. The objective of the study must be clearly informed to the respondents, along with the motto of the study and the data collection methods utilized. There is no harm in the study; it does not involve any physical, emotional, or psychological aspects. The data collected by various teachers, students, and administrators can help prevent the misuse of the study.
Figure 1 Characteristics of respondents.
Result and Interpretation
The below Figure 1 clearly indicate that 10% of the study’s participants were women, compared to 15% men. Regarding age, the majority of respondents (14%) were between the ages of 25 and 35, while 7% were in the 18 to 25 age range. The respondents with the most solid educational backgrounds (about 15%) had graduate degrees. Her respondents, 8% of them, all hold post-graduate degrees. 27% of respondents surveyed have professional credentials, as shown by their provision response to poll clarifications that recreate quantitative assessment validity. Distribution of participants’ field of study in the research study. Among the participants, 8% belonged to the field of Humanities, 16% were from the field of science, and 3% fell into the category of other fields of study.
Table 2 Perceptions of feasibility, effectiveness, challenges, and benefits in open book examinations
| Frequency | Percent | Mean | S.D | |
| Feasibility and effectiveness of integrating AI in open book examinations | ||||
| Technology Infrastructure | 83 | 50% | 3.03 | 0.999 |
| User Acceptance | 43 | 26% | 3.66 | 1.033 |
| Performance and Accuracy | 39 | 24% | 3.18 | 1.996 |
| Challenges and limitations of traditional question paper preparation | ||||
| and evaluation methods | ||||
| Time Constraints | 25 | 15% | 3.99 | 0.945 |
| Subjectivity in Evaluation | 58 | 35% | 3.45 | 0.963 |
| Adaptability to Different Subjects | 50 | 30% | 4.05 | 0.945 |
| Assessment of Higher-Order Thinking | 32 | 20% | 4.33 | 1.032 |
| Benefits and advantages of using AI technologies in question paper preparation | ||||
| for open book exams | ||||
| Question Paper Customization | 42 | 25% | 3.2 | 0.988 |
| Enhanced Question Design | 16 | 10% | 3.99 | 1.963 |
| Efficiency and Time Savings | 58 | 35% | 3.45 | 0.788 |
| Personalized Learning Experience | 33 | 20% | 4.33 | 0.999 |
| Immediate Feedback and Support | 16 | 10% | 4.96 | 1.966 |
Figure 2 Benefits and advantages of using AI technologies in question paper preparation for open book exams.
Figure 3 Benefits and advantages of using AI technologies in question paper preparation for open book exams.
The below mentioned that Table 2 represents the perceptions of participants regarding the feasibility, effectiveness, challenges, and benefits of integrating AI in open book examinations. In terms of the feasibility and effectiveness of AI integration, the highest frequency and percentage were observed for “Technology Infrastructure,” indicating its perceived importance in enabling the implementation of AI in open book examinations. “User Acceptance” and “Performance and Accuracy” also received notable attention, suggesting that participants recognized the significance of these factors in determining the feasibility and effectiveness of AI integration. Regarding the challenges and limitations of traditional question paper preparation and evaluation methods, “Time Constraints” had the highest frequency, indicating that participants identified time limitations as a significant challenge. “Subjectivity in Evaluation” was another important aspect, as indicated by its substantial frequency. “Adaptability to Different Subjects” also received attention, highlighting the challenges associated with designing question papers for diverse subjects in open book examinations.
Table 3 T-test value for the feasibility and effectiveness of open book examinations
| Group | Sample | Mean | Standard Deviation | t-value | p-value |
| Group A (AI-integrated) scores | 82 | 91.23 | 2.3 | 3.638 | 0.02 |
| Group B (Traditional methods) scores | 83 | 82.30 | 2.5 | ||
| Significance Level: 0.05. | |||||
The results of the t-test review for the viability and efficacy of open book exams when man-made intelligence is coordinated in contrast with customary strategies are displayed in Table 3. The t-test was utilized to survey the elective speculation (H1A), which expresses that there is a remarkable distinction between the two methodologies’ suitability and viability. Determined as a correlation between Group A and Group B, the t-value is 3.638. The likelihood of tracking down the noticed contrast or a more outrageous distinction on the off chance that the invalid speculation is right is addressed by the p-esteem, which is 0.02. The p-worth of 0.02 is underneath the importance level while considering an importance level of 0.05. Accordingly, we can acknowledge the alternate speculation (H1A) and reject the null theory (H0). This proposes that the viability and efficacy of open book exams are essentially unique when artificial intelligence is incorporated rather than traditional procedures.
Alternative Hypothesis (H2A): There is a significant association between traditional question paper preparation and evaluation methods and the identified challenges and limitations in open book examination systems.
Table 4 T-test value for benefits and advantages
| Group | Sample | Mean | Standard Deviation | t-value | p-value |
| Group A (Traditional Methods) | 82 | 4.6 | 0.9568 | 7.3233 | 0.036 |
| Group B (No Traditional Methods) | 83 | 3.9 | 0.8563 | ||
| Significance Level: 0.05. | |||||
The results of the t-test examination for comparing the advantages and disadvantages of using AI technologies, such as NLP and machine learning algorithms, in the improvement of inquiry papers for open book exams and regular methodologies are displayed in Table 4. The t-test was utilized to examine the elective speculation (H3A), which expresses that there is a striking contrast in the benefits and advantages of the two methodologies. Determined as a correlation between Gathering an and Gathering B, the t-esteem is 7.3233. In light of the measures of the examples for the two gatherings, the levels of opportunity are determined. The noticed distinction or a more outrageous contrast are bound to happen on the off chance that the invalid speculation is right, as per the p-esteem, which is 0.036.
The p-worth of 0.036 is beneath the importance level while considering a 0.05 degree of importance. Subsequently, we can acknowledge the elective speculation (H3A) and reject the invalid speculation (H0). This proposes that involving simulated intelligence innovation being referred to paper groundwork for open book tests has a tangibly unique arrangement of benefits and advantages from doing so utilizing conventional strategies.
Result & Discussion
The majority of respondents were male 60 percent and in the age range of 25–35 (55%). About 60 percent of participants had graduate degrees, while 30 percent had post-graduate degrees. In terms of field of study, 65 percent were from the science field and 30 percent from humanities and then 5 percent from other fields. The research study found a significant difference in the feasibility and effectiveness of open book examinations when AI is integrated compared to traditional methods. The outcome of the research study participants perceived AI integration to be more feasible and effective in open book examinations. There was a significant association between traditional question paper preparation and evaluation methods and the identified challenges and limitations in open book examination systems. The research study due to time constraints and subjectivity in evaluation were identified as major challenges in traditional methods. Artificial Intelligence technologies were found to offer significant benefits and advantages in question paper preparation for open book exams compared to traditional methods. The benefits of the research to as efficiency, personalized learning experiences, and immediate feedback and support were highlighted.
Conclusion
Perhaps of the most predominant innovation utilized these days across all areas and instructive levels is artificial intelligence. In any case, its use in training isn’t extremely normal, probably because of client obliviousness, however it means to become one of the main devices to be utilized. Despite the fact that we likewise track down different instances of purpose, for example, personalization or quality assessment, the principal uses of artificial intelligence in schooling are connected with coaching and evaluation. This study investigated the role of Artificial Intelligence (AI) in open book examinations, specifically focusing on question paper preparation and evaluation processes. The findings of the study provide valuable insights into the feasibility, effectiveness, challenges, and benefits associated with AI integration in open book examinations. The perceptions of participants regarding AI integration in open book examinations were assessed. It was found that technology infrastructure, user acceptance, and performance and accuracy were considered crucial for the feasibility and effectiveness of AI integration. Time constraints and subjectivity in evaluation were identified as significant challenges in traditional question paper preparation and evaluation methods. Participants recognized the benefits of AI technologies, such as efficiency, time savings, personalized learning experiences, and immediate feedback and support.
References
Cellan-Jones, R. (2014): Stephen Hawking warns artificial intelligence could end mankind. BBC news, 2(10), 2014. (Report).
Chapelle, C. A., and Chung, Y. R. (2010): The promise of NLP and speech processing technologies in language assessment. Language Testing, 27(3), pp. 301–315.
Chew, E., and Chua, X. N. (2020): Robotic Chinese language tutor: personalising progress assessment and feedback or taking over your job? On the Horizon, 28(3), pp. 113–124.
Dawson, P., Nicola-Richmond, K., and Partridge, H. (2024): Beyond open book versus closed book: a taxonomy of restrictions in online examinations. Assessment & Evaluation in Higher Education, 49(2), pp. 262–274.
Duan, Y., Edwards, J. S., and Dwivedi, Y. K. (2019): Artificial intelligence for decision making in the era of Big Data–evolution, challenges and research agenda. International journal of information management, 48, pp. 63–71.
Farhi, F., Jeljeli, R., Aburezeq, I., Dweikat, F. F., Al-shami, S. A., and Slamene, R. (2023): Analyzing the students’ views, concerns, and perceived ethics about chat GPT usage. Computers and Education: Artificial Intelligence, 5, 100180. pp. 1–8.
Gálvez, J., Conejo, R., and Guzmán, E. (2013): Statistical techniques to explore the quality of constraints in constraint-based modeling environments. International Journal of Artificial Intelligence in Education, 23, pp. 22–49.
Jani, K. H., Jones, K. A., Jones, G. W., Amiel, J., Barron, B., and Elhadad, N. (2020): Machine learning to extract communication and history-taking skills in OSCE transcripts. Medical Education, 54(12), pp. 1159–1170.
Jonassen, D., Davidson, M., Collins, M., Campbell, J., and Haag, B. B. (1995): Constructivism and computer-mediated communication in distance education. American journal of distance education, 9(2), pp. 7–26.
Liu, M., Wang, Y., Xu, W., and Liu, L. (2017): Automated scoring of Chinese engineering students’ English essays. International Journal of Distance Education Technologies (IJDET), 15(1), pp. 52–68.
Lodhi, P., Mishra, O., Jain, S., and Bajaj, V. (2018). StuA: An intelligent student assistant. IJIMAI, 5(2), pp. 17–25.
Luckin, R; Holmes, W; (2016): Intelligence Unleashed: An argument for AI in Education. UCL Knowledge Lab: London, UK. ISBN: 13-9780992424886. Report.
Mellit, A., and Kalogirou, S. A. (2008): Artificial intelligence techniques for photovoltaic applications: A review. Progress in energy and combustion science, 34(5), pp. 574–632.
Narciss, S., Sosnovsky, S., Schnaubert, L., Andrès, E., Eichelmann, A., Goguadze, G., and Melis, E. (2014). Exploring feedback and student characteristics relevant for personalizing feedback strategies. Computers & Education, 71, pp. 56–76.
Nilson, N. J., “John McCarthy, (2012): A Biographical Memoir,” National Academy of Sciences, 1–27. (Report).
Ocaa-Fernández, Valenzuela-Fernández, L. A., and Garro-Aburto, L. L. (2019): Artificial Intelligence and its Implications in Higher Education. Propós. represent. vol. 7, n. 2, pp. 536–568.
Perez, S., Massey-Allard, J., Butler, D., Ives, J., Bonn, D., Yee, N., and Roll, I. (2017): Identifying productive inquiry in virtual labs using sequence mining. In Artificial Intelligence in Education: 18th International Conference, AIED 2017, Wuhan, China, June 28–July 1, 2017, Proceedings 18, pp. 287–298.
Popenici, S. A., and Kerr, S. (2017): Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and practice in technology enhanced learning, 12(1), pp. 2–13.
Quille, K., Nolan, K., Becker, B. A., and McHugh, S. (2021): Developing an open-book online exam for final year students. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1 pp. 338–344.
Rawlinson, K (2015): Microsoft’s Bill Gates insists AI is a threat. BBC News. Report.
Richardson, M., and Clesham, R. (2021): Rise of the machines? The evolving role of Artificial Intelligence (AI) technologies in high stakes assessment. London Review of Education, 19(1), pp. 1–13.
Roberts, L., and Berry, J. (2023): Should open-book, open-web exams replace traditional closed-book exams in STEM? An evaluation of their effectiveness in different disciplines. Journal of Learning Development in Higher Education, (28). pp. 1–39.
Samarakou, M., Fylladitakis, E., Fruh, W. G., Karolidis, D., Hatziapostolou, A., Athinaios, S., and Grigoriadou, M. (2016): Evaluation of an intelligent open learning system for engineering education. Knowledge Management and E-Learning: An International Journal, 8(3), pp. 496–513.
Santos, O. C., and Boticario, J. G. (2014): Involving users to improve the collaborative logical framework. The Scientific World Journal, 2014(1), pp. 1–15.
Saplacan, D., Herstad, J., and Pajalic, Z. (2018): Feedback from digital systems used in higher education: An inquiry into triggered emotions-two universal design oriented solutions for a better user experience. In Transforming our World Through Design, Diversity and Education, pp. 421–430.
Suhaimy, K. A. M., Salleh, B. M., and Esa, A. (2012): The Open Book Examination. Journal of Techno-Social (JTS) Volume 4(2). ISSN: 2229-8940, pp. 1–12.
van den Berg, G., and du Plessis, E. (2023): ChatGPT and generative AI: Possibilities for its contribution to lesson planning, critical thinking and openness in teacher education. Education Sciences, 13(10), pp. 998.
Yuwei Li (2024): Procedia Computer Science, ‘The 4th International Conference on Machine Learning and Big Data Analytics for IoT Security and Privacy”, Vol. 243, pp. 1222–1230.
Biographies
Amit Dimari received his bachelor’s degree in commerce from Hemvati Nandan Bahuguna Garhwal University (HNBGU), He Completed his master degree in commerce, library science and Sociology, and now he is pursuing his Doctor of Philosophy degree in Sociology, form Graphic Era (Deemed to be University). He is currently working as a Section Officer in Examination Department at Graphic Era (Deemed to be University), Dehradun; he has published research papers and book chapters in esteemed journals indexed in UGC CARE-list, peer reviewed and Scopus.
Nidhi Tyagi is working as a Professor at Graphic Era (Deemed to be University), Dehradun. under the aegis of Department of Humanities & Social Sciences. She was the founder member of Law College of Teerthanker Mahaveer University, Moradabad and rendered her services to the esteemed organization for a period of 10 years. Dr. Nidhi has a vast teaching experience of 19 years and has represented the Department in BCI, UGC, NAAC inspections and various administrative and academic activities. Besides, she has been contributing to different domains of Sociology and Indian Society. She has devoted herself to scholastic research and published many outstanding papers in different academic journals. Furthermore, Dr. Nidhi has presented more than 22 papers at national and international conferences and has also Chaired Sessions in various International Conferences.
S. Srinivasan has recently joined the Department of Humanities and Social Sciences at Graphic Era (Deemed to be University) as an Assistant Professor. With a wealth of experience, Mr. Srinivasan has accumulated 10 years of extensive teaching and field-level expertise. Prior to his current role, he served in the Ministry of Health and Family Welfare, Government of India, and gained valuable experience. He also contributed as a Guest Faculty at the university level and served for nearly 2 years in the State Government Public Works Department as a Mobilization Training Specialist. In the past year, Mr. S. Srinivasan has enriched his experience by actively participating in a Tuberculosis project funded by GFATM and USAID, providing a platform for further exploration of his research, and practical field-based knowledge. His scholarly contributions include publishing 45 papers, with 18 more under peer review in various UGC-Care, SCOPUS, and peer-reviewed journals. Mr. S. Srinivasan has also actively participated in 10 different training programs, including an ICSSR research methodology course, a Workshop on Scientific Writing and Publication, Applications of SPSS in Social Science Research, and Training of Trainers in life skill education. Furthermore, he has presented papers at 4 state-level conferences, 10 national conferences, and 15 international conferences. He has also presented and participated in various online international conferences.
Mahesh Davanageri is a distinguished academician with 30 years of experience in education administration and examination management. He serves as the Controller of Examination at Graphic Era Deemed to be University, overseeing exam processes and specializes in developing university-level examination systems. key achievements include implementing digital valuation, online exams, and blockchain-based degree certificates. He is active member of ISTE and IAENG, with numerous research publications in international conferences and journals.
Prabha Lama serves as an Associate Professor and the Head of the Department of Humanities and Social Sciences at Graphic Era (Deemed to be University), where she plays a pivotal role in shaping the academic landscape. Having earned a Doctorate degree in English Literature, Dr Prabha possesses a wealth of teaching experience across diverse faculties, with a particular focus on legal language, English literature, and English communication, specializing in British literature, Indian literature, Women’s writing, and Post-Colonial Literature. Dr Prabha’s contributions extend beyond teaching; she has published research papers and book chapters in esteemed journals indexed in UGC CARE-list, peer reviewed and Scopus, also actively serves as a reviewer for reputable publications.
Hema Dimari completed her bachelor’s degree in History and later completed a Master of Business Administration (MBA) with Human Resource Management. She is currently working as an Office Executive in the Examination Department at Graphic Era Hill University, Dehradun.
Journal of Reliability and Statistical Studies, Vol. 18, Issue 1 (2025), 1–24.
doi: 10.13052/jrss0974-8024.1811
© 2025 River Publishers