Advertisement

Effect of a Nursing Comprehensive Skill Training Course (NCST-C) on Nursing Students' Metacognitive Awareness: A Quasi-experimental Study

Open AccessPublished:November 01, 2022DOI:https://doi.org/10.1016/j.anr.2022.10.003

      Summary

      Purpose

      This study explored the effect of a nursing comprehensive skill training course (NCST-C) on the metacognitive awareness of nursing students to provide a scientific foundation for improving metacognitive awareness.

      Methods

      This study used a quasi-experimental two-group matched pretest, post-test, and follow-up test. Ninety-six junior nursing students were recruited using convenience sampling and assigned to two groups by drawing lots with odd and even numbers in a nursing school at Huzhou University. The control group received a traditional skill-training course. The intervention group received NCST-C. Nursing students' metacognitive awareness inventory (MAI) was evaluated at the baseline, 16-week, and 20-week follow-up points. A repeated-measures analysis of variance and a simple effect test was used to compare each outcome measure of the two groups three times.

      Results

      The NCST-C resulted in greater benefits for nursing students' metacognitive awareness as well as various dimensions (knowledge of cognition and regulation of cognition) in the intervention group. Combined with a simple effects test, the MAI and dimension scores of those in the intervention group significantly improved at 16 weeks after the baseline (F = 9.78–44.03; all ps < .01). The sustainable effect of NCST-C lasted 1 month after the intervention (F = 14.24–62.36; ps < .01), reaching statistical significance (p < .05).

      Conclusions

      The NCST-C effectively developed metacognitive awareness among nursing students. Its design provides a new type of experimental course for improving metacognitive awareness.

      Trial registration

      chictr.org.cn: ChiCTR2200057910. First registration date: 1/11/2020.

      Keywords

      Introduction

      Rapid developments in science and technology have had positive effects on the quality of education [
      • Curran V.R.
      • Fairbridge N.A.
      • Deacon D.
      Peer assessment of professionalism in under graduate medical education.
      ]. Online courses have facilitated student-centered education and the innovative development of diversified teaching models [
      • Sezer H.
      • Şahin H.
      Faculty development program for coaching in nursing education: a curriculum development process study.
      ]. Construction of online courses is both a challenge and an opportunity. Research shows that the construction and implementation of online courses cannot meet the needs of contemporary students' learning, and students call for returning to classroom learning [
      • Medina M.S.
      • Castleberry A.N.
      • Persky A.M.
      Strategies for improving learner metacognition in health professional education.
      ]. The view that high-quality resource construction of online courses and free learning environment beyond space are auxiliary carriers of classroom teaching has been recognized by scholars [
      • Abdelrahman R.M.
      Metacognitive awareness and academic motivation and their impact on academic achievement of Ajman university students.
      ]. Therefore, exploring the design of online and offline hybrid courses, integrating the advantages of online platforms and the original motivation of classroom teaching is the focus of research to adapt to social and professional development. Recent study has shown that these all require the drive of students' metacognitive awareness [
      • Flavell J.H.
      Cognitive development.
      ].
      Metacognitive awareness includes knowledge of cognition and regulation of cognition, it has been defined as the ability to reflect upon, understand, and control one's learning [
      • Shea N.
      Metacognition and abstract concepts.
      ]. Knowledge of cognition can help students recognize their strengths and weaknesses, and then affects learning motivation and interest [
      • Schraw G.
      • Dennison R.S.
      Assessing metacognitive awareness.
      ]. Regulation of cognition includes planning, implementing, monitoring, debugging, and evaluating strategies and it is a trigger for evaluating and adjusting students' learning strategies [
      • Abdelrahman R.M.
      Metacognitive awareness and academic motivation and their impact on academic achievement of Ajman university students.
      ,
      • Akturk A.O.
      • Sahin I.
      Literature review on metacognition and its measurement.
      ]. Knowledge of cognition and regulation of cognition are interrelated, but not simply compensatory [
      • Song J.H.H.
      • Loyal S.
      • Lond B.
      Metacognitive awareness scale, domain specific (MCAS-DS): assessing metacognitive awareness during raven's progressive matrices.
      ]. Studies show that students with higher knowledge of cognition have shown better test performance. However, if their regulation of cognition ability decreases, students will show unreasonable planning and self-debugging, thus affecting their exam results [
      • Chang C.
      • Colón-Berlingeri M.
      • Mavis B.
      • Solomon D.
      Medical student progress examination performance and its relationship with metacognition, critical thinking, and self-regulated learning strategies.
      ,
      • Bernstein A.
      • Hadash Y.
      • Lichtash Y.
      • Fresco D.M.
      Decentering and related constructs: a critical review and metacognitive processes model.
      ]. Recent studies have shown that metacognitive awareness contributes to the development of critical thinking and positive learning [
      • Desender K.
      • Teuchies M.
      • Gonzalez-Garcia C.
      • Brass M.
      Metacognitive awareness of difficulty in action selection: the role of the cingulo-opercular network.
      ,
      • Shekhar M.
      • Rahnev D.
      Sources of metacognitive inefficiency.
      ]. Students with low metacognitive awareness often adopt ineffective learning strategies and fail to use critical thinking or to develop practical skills to overcome learning challenges [
      • Bektas I.
      • Bektas M.
      • Ayar D.
      • Celik I.
      The predict of metacognitive awareness of nursing students on self-confidence and anxiety in clinical decision-making.
      ]. Therefore, focusing on metacognitive awareness as a means to adjust learning motivation, positive learning ability, and professional thinking can be considered important for improving the quality of nursing education.
      Evidence suggests that active learning can improve students' metacognitive awareness [
      • Rashwan Z.I.
      • Busebaia T.J.
      • Al-Sabbagh A.S.
      • Eweida R.S.
      Effect of guided reciprocal peer questioning strategy on pediatric nursing students' self-esteem and metacognitive awareness: current approach and future directions.
      ,
      • Chan C.W.H.
      • Tang F.W.K.
      • Chow K.M.
      • Wong C.L.
      Enhancing generic capabilities and metacognitive awareness of first-year nursing students using active learning strategy.
      ]. Active learning is defined as a process in which learners play an energetic and dynamic role in their education [
      • Blake T.K.
      Journaling; an active learning technique.
      ]. Examples of active learning modes include mixed online and offline teaching modes and cooperative learning [
      • Waltz C.F.
      • Jenkins L.S.
      • Han N.
      The use and effectiveness of active learning methods in nursing and health professions education: a literature review.
      ]. Mixed online–offline teaching modes involve multi-spatial teacher–student interaction. Here, teachers need to design online teaching content and methods while also designing offline classroom teaching methods and content [
      • Hsu L.L.
      • Hsieh S.I.
      Factors affecting metacognition of undergraduate nursing students in a blended learning environment.
      ,
      • Mahmoud H.
      The effect of cooperative learning strategy on undergraduate nursing students enrolled in nursing administration course.
      ]. Studies have suggested that the design of the teaching method before, during, and after class is the key content of hybrid online–offline teaching [
      • Mahmoud H.
      The effect of cooperative learning strategy on undergraduate nursing students enrolled in nursing administration course.
      ]. Cooperative group learning is another online–offline teaching mode that can promote learning interaction and knowledge exchange among students [
      • Siqueira M.A.M.
      • Gonçalves J.P.
      • Mendonça V.S.
      • Martins M.A.
      Relationship between metacognitive awareness and motivation to learn in medical students.
      ]. Research has shown that the instructional design of cooperative group learning is an important technical aspect of successful teaching [
      • Gholami M.
      • Moghadam P.K.
      • Mohammadipoor F.
      • Pour A.H.
      Comparing the effects of problem- based learning and the traditional lecture method on critical thinking skills and metacognitive awareness in nursing students in a critical care nursing course.
      ,
      • Martirosov A.L.
      • Moser L.R.
      Team-based learning to promote the development of metacognitive awareness and monitoring in pharmacy students.
      ]. Thus, whether cooperative group learning in mixed online–offline teaching can successfully replace traditional teaching depends on the course content and the feasibility of the teaching design.
      Undergraduate nursing education in China has been subject-oriented, with a strong emphasis on theoretical knowledge and technical competence [
      • Hao X.
      • Peng X.
      • Ding X.
      • Li K.
      Application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic: a systematic review.
      ]. Mixed online–offline teaching with cooperative group learning has been found to achieve good results in nursing theory courses but not research in skill training courses [
      • Meng X.H.
      • Xu X.Y.
      • Chen H.L.
      • Zhang L.
      The effectiveness of combining e-learning, peer teaching, and flipped classroom for delivering a physiology laboratory course to nursing students.
      ,
      • Schramm J.
      • Uranga T.
      • Birkle A.
      • Taylor L.
      Improving holistic care skills for advanced practice registered nurses: integration of a complementary and alternative medicine course into a doctor of nursing practice curriculum.
      ]. Previous studies have shown that mixed online–offline and cooperative group learning needs to be guided by case problems, the integration of diversified teaching methods, and scientifically feasible teaching design [
      • Jiang Y.
      • Chen Y.
      • Lu J.
      • Wang Y.
      The effect of the online and offline blended teaching mode on English as a foreign language learners' listening performance in a Chinese context.
      ]. Research suggests that undergraduate nursing education should focus on the high-level development of skill training content and assessment courses [
      • Liu H.Y.
      Promoting creativity of nursing students in different teaching and learning settings: a quasi-experimental study.
      ]. Therefore, this study aimed to develop students' metacognitive awareness by using online–offline teaching and cooperative group learning that integrated experimental technology curriculum content and a multi-station nursing skill examination design.
      In 2021, China's National Nursing Skills Competition adopted a new form of multi-station nursing skill examination (Ministry of Education of China). This consists of both personal operation and team cooperation. The number of test stations can be set to three or eight. Starting from the first test station, each student conducts nursing assessment, preparation, planning, and integration using an advanced simulation model to implement individual or team skill operations [
      • Hwang G.J.
      • Chang C.Y.
      • Ogata H.
      The effectiveness of the virtual patient-based social learning approach in undergraduate nursing education: a quasi-experimental study.
      ,
      • Brauer D.G.
      • Ferguson K.J.
      The integrated curriculum in medical education: AMEE Guide No. 96.
      ]. Each station requires 6 or 10 min to complete all case operations. Scoring is done using a combination of computer scoring and manual scoring. Multi-station nursing skill examination establishes higher standards for nursing students' skill operation accuracy.
      A comprehensive course can move beyond existing traditional disciplinary frameworks, re-integrate internally related content, and form new courses [
      • Lipsey M.W.
      • Wilson D.B.
      The efficacy of psychological, educational, and behavioral treatment: confirmation from meta-analysis.
      ]. For this study, we constructed a comprehensive nursing skill training course, which includes integrate medicine nursing, surgical nursing, and emergency and critical care experimental content, design online–offline teaching model, use cooperative group learning methods, design multi-station skill assessment examination to evaluate the course's effect on students' metacognitive awareness.
      We hypothesized that nursing students who participated in the NCST-C would have a better performance on metacognitive awareness than the control group. We further hypothesized that there were differences in short-term and long-term effect outcomes between a tailored integrated nursing comprehensive skill training course intervention and traditional teaching skill training course.

      Materials and methods

      Study design

      This study used a two-group matched pretest, post-test, and follow-up test designs which utilized a quasi-experimental study.

      Ethical considerations

      Huzhou University's Ethics Committee review board approved our study (no. 202012-JG01), and the study complied with the Declaration of Helsinki. All participants were informed of the study's objectives, procedures, and potential risks, and informed that they have the right to withdraw at any time. Written consent was obtained from the participants.

      Sample and setting

      The participants were recruited from junior-year nursing students at Huzhou University, Zhejiang Province, China. G∗Power 3.1 was used for sample size calculations. Repeated-measures analysis of variance (ANOVA) was selected, there were three measurements and two groups. With a power (1-β) of .80 fests, the effect size was .25, error probability value was .05. Based on per-experimental data, we established a correlation among the repeated measures of .50. Consequently, the total sample consisted of 96 people, 48 in the experimental group and 48 in the control group.
      The inclusion criteria were as follows: (1) nursing students in the second semester of the third year in University; (2) voluntary course selection; (3) informed consent, willing to cooperate. The exclusion criteria were as follows: (1) not interested and (2) research objects who are participating in other teaching reform.

      Sampling and recruitment method

      Our research team conducted publicity and knowledge lectures on the curriculum teaching reform project among the nursing students in the third year of the University. According to the interests and with informed consent, the students were recruited in the form of voluntary registration. After that, nursing students applied to join through the curriculum registration network system. The computer randomly assigned the students to the experimental group and control group according to cardinal and even number of the submission system sequence. Students participating in the project will be taught in an independent laboratory. The nursing students in the experimental group joined the online course platform through personal application and the course administrator agree. According to the principle of equality, nursing students in the control group were informed that after the course was completed, they would join the course platform and learn freely.

      Intervention

      Control group

      Traditional teaching skill training course methods comprised a total of 64 class hours, offline teaching in the laboratory. Curriculum content included 27 items single experiment technology. Part of the content as shown in Figure 1, course chapter content included basic nursing experiment technology, internal nursing experimental technology, surgical nursing experimental technology, and experimental techniques of emergency nursing. Teaching method and assessment were based on teacher demonstration, students practice by themselves, and one-way technical examination. Teaching organization was under the guidance of two teachers, a total of 16 weeks of course teaching was completed in the form of classroom teaching (Figure 2).
      Figure 1
      Figure 1Nursing comprehensive skill training (NCST-C) curriculum content.
      Figure 2
      Figure 2Online and offline mixed curriculum design.

      Intervention group

      • (1)
        Curriculum content
      According to the independence and team-based nature of the technology application, the integrated nursing experiment technology included 30 items, as shown in Figure 1.
      • (2)
        Curriculum design
      The curriculum involved a three-module, mixed online–offline teaching mode. It comprised a total of 64 class hours: 16 h online and 48 offline. One class “hour” was 45 min. Module one, during the first week, included 2 h online and 2 h offline; it covered the curriculum introduction and learning method training. Module two lasted 12 weeks, with a unit module every 2 weeks (2 h online, 6 h offline), for a total of six unit modules. This covered online problem-oriented self-learning skills, classroom guidance and training skills, and multi-station examination. Module three (3 weeks; 2 h online and 10 h offline) covered online team skill self-learning, in-classroom multi-station examination guidance and exercises, team multi-station examination, and a nursing practice module.
      • (3)
        Teaching method
      Online teaching adopted a task-oriented approach that included 1 h of video learning, 15 min of online practice, 10 min of discussion and interaction, and 1000 words of operation process writing. Offline teaching adopted cooperative group learning methods, including group scenario simulation, and group discussion. After class, students completed post-learning tasks on the online teaching platform.
      • (4)
        Teaching organization
      Online teaching was organized and managed by a teacher, who would post online videos, tasks, assignments, and interactions. In-class teaching used the form of cooperative group learning. First, before class started, students were randomly divided into four groups of 12. Second, matching of teachers and student groups was determined by lottery for each lesson. All four teachers conducted collective lesson preparation and training before class.
      • (5)
        Course assessment
      Course assessment adopted a combined online–offline form. The curriculum design included 30 learning tasks, seven multi-station nursing skill examinations, and three nursing practices. Based on the tasks published online, online course assessment involved calculating a total score using a big data platform. Offline course assessment involved the skill learning and multi-station assessment of teams and individuals. Each multi-station examination included three stations, individual examinations were 6 min per station, and team examinations were 10 min per station.

      Instruments

      Demographic information

      General demographic information was collected, including age, gender, only child or not, and family location.

      Metacognitive awareness inventory

      Metacognitive awareness inventory (MAI) consists of knowledge of cognition (17 items) and regulation of cognition (35 items), a total of 52 items [
      • Schraw G.
      • Dennison R.S.
      Assessing metacognitive awareness.
      ]. Knowledge of cognition includes declarative knowledge, procedural knowledge, and conditional knowledge. Regulation of cognition includes planning, information management, monitoring, debugging, and evaluation. Each item is scored on a 5-point Likert scale. Total scores ranged from 52 to 260. The higher the score, the stronger the metacognitive awareness. To revise the MAI for nursing students in China, we carried out translation, back translation, expert consultation, and exploratory factor analysis (EFA). The Cronbach's alphas of the scales and subscales were .96 and .87–.99, respectively. The KMO measure was computed to be .84, the C-WAI (52 items) had a eight-factor solution, accounting for 89.44% of the total variance, and the content validity index at the scale level (S-CVI) was .912. The questionnaire was demonstrated good reliability and validity. Our study used the total scale on knowledge of cognition comprehensively observe the changes of metacognitive knowledge.

      Data collection

      Data collection were conducted at 3 time intervals, which were evaluated at the baseline (T0), 16-week (T1), and 20-week follow-up points (T2). No nursing student dropouts during the experiment. All investigators participated in one-day training before the survey, they were independent of the research team. The same investigators completed the collection, inspection, and analysis of the scales, with a recovery rate of 100%.

      Statistical analysis

      SPSS 21.0 was used for data analysis. Descriptive analysis was used to describe the collected demographic data, normal distribution, and MAI scores at T0, T1, and T2. A repeated-measures analysis of variance was used to compare the MAI between the two groups. The statistical significance was established at p-value less than 0.05. Cohen's d was used to calculate the effect sizes post-intervention, mainly using the mean and combined standard deviation of the conditional measures (less than 0.33 was small, 0.33–0.55 moderate, and 0.56–1.2 large) [
      • Lipsey M.W.
      • Wilson D.B.
      The efficacy of psychological, educational, and behavioral treatment: confirmation from meta-analysis.
      ].

      Results

      Participant characteristics

      Participants' mean age was 21.07 ± 1.03 years. Table 1 shows that there were no significant differences in demographic information between the two groups of students (p > .05).
      Table 1Sociodemographic Characteristics of Participants in the Two Groups.
      Sociodemographic characteristicsInvention Group (n = 48)Control Group (n = 48)T or Chi square -Valuep
      Age, mean (SD)21.20 ± 1.6020.99 ± 1.010.77.444
      Sex, n (%)0.33.563
       Men6 (12.5)8 (16.6)
       Women42 (87.5)40 (83.4)
      An only child in a family, n (%)0.42.838
       Yes25 (52.1)26 (54.2)
       No23 (47.9)22 (45.8)
      Family location, n (%)0.04.838
       Rural area24 (50.0)23 (47.9)
       Town24 (50.0)25 (52.1)
      Notes: SD = standard deviation.

      Intervention efficacy

      The data of the intervention group and the control group were normally distributed (p > .05). Table 2 shows that the results of metacognitive awareness and subcategory measures of Mauchly's identity matrix test showed significant (X(2,96)2 = 5.02–95.34; all ps < .01). Metacognitive awareness and subcategory measures were found significant interaction effects (F(2, 96) = 11.43–50.30; all ps < .01). Also, a statistically significant main effect for the group factor was observed for debugging (F(1, 48) = 4.93, p = .032) and evaluation (F(1, 48) = 9.60, p = .013). Significant group factors were found for other subcategory outcome measures (F(1, 48) = 14.87–23.10; all ps < .01). Meanwhile, a statistically significant main effect for the time factor was observed (F(2, 96) = 15.52–64.20; all ps < .01) for metacognitive awareness and the subcategory outcome measures.
      Table 2Impact of the Intervention on Metacognitive Awareness Outcome Measures at Three Time-points (Group × Time) test (N = 96).
      Outcome measuresMeasure TimeMauchly 's identity testGroup FactorTime FactorInteraction Effect
      T0 (Mean T SD)T1 (Mean T SD)T2 (Mean T SD)X2pFpFpFp
      Metacognitive knowledge17.30.00059.12.00047.09.000
       Experimental group (G1)66.56 ± 6.4074.46 ± 6.9074.95 ± 5.8017.89.000
       Control group (G2)67.00 ± 7.1867.75 ± 5.1167.23 ± 4.6495.34.000
      Regulation of cognition20.81.00049.36.00034.86.000
       Experimental group (G1)130.92 ± 29.86165.23 ± 29.08167.50 ± 24.2917.89.000
       Control group (G2)132.73 ± 34.43135.63 ± 29.06136.77 ± 22.475.02.000
      Planning15.86.00057.89.00044.82.000
       Experimental group (G1)26.58 ± 5.9133.77 ± 5.3634.54 ± 4.8118.69.000
       Control group (G2)27.06 ± 7.0427.79 ± 5.9727.37 ± 5.0844.69.000
      Information management21.11.00093.17.00046.19.000
       Experimental group (G1)37.50 ± 5.8045.97 ± 6.1746.16 ± 4.999.90.007
       Control group (G2)37.10 ± 6.9939.29 ± 5.1339.12 ± 4.7630.82.000
      Monitoring23.10.00064.20.00050.30.000
       Experimental group (G1)26.56 ± 5.8134.94 ± 6.0735.35 ± 4.969.95.007
       Control group (G2)27.00 ± 7.1827.31 ± 5.1527.71 ± 4.5167.07.000
      Debugging4.93.03230.80.00011.43.000
       Experimental group (G1)18.58 ± 6.3523.91 ± 6.1724.31 ± 4.9617.14.000
       Control group (G2)19.12 ± 6.8920.29 ± 5.1420.62 ± 4.6065.46.000
      Evaluation9.60.01315.52.00013.80.000
       Experimental group (G1)21.68 ± 6.3726.62 ± 6.0927.12 ± 5.4116.87.000
       Control group (G2)21.91 ± 7.4522.25 ± 4.9621.93 ± 4.1659.26.000
      Metacognitive awareness14.87.00053.80.00032.90.000
       Experimental group (G1)197.48 ± 36.05239.69 ± 35.48242.64 ± 29.1818.89.000
       Control group (G2)199.21 ± 42.72204.69 ± 30.98204.48 ± 27.0587.72.000
      Notes: T0, baseline; T1, post-intervention (16 weeks after baseline); T2, 1-month follow-up (20 weeks after baseline); p value .000 -mean “<.001”.

      Simple effect test on interaction effects for metacognitive awareness

      Table 3 shows that at the T0 level, the group factor had no effects on metacognitive awareness and subcategory outcome measures (F(2, 96) = 0.01–0.16; all ps > .05). This means that there were no significant differences between the two groups at the baseline. At the T1 level, two groups had significant effects on metacognitive awareness and the subcategory outcome measures (F(2, 96) = 9.78–44.03; all ps < .01). This means that 16-week NCST-C intervention has significant effect on metacognitive awareness. At the T2 level, two groups had significant effects on metacognitive awareness and the subcategory outcome measures (F(1,48) = 14.24 – 62.36; all ps < .01). This means that on the metacognitive awareness and subcategory outcome measures, the NCST-C intervention effect persisted for 1 month. In addition, from post-intervention to follow-up, the intervention group had a higher effect size than the control group (Cohen's d > 0.56).
      Table 3Results of Simple Effects of Interaction Effects on All Outcomes.
      Source of variationMetacognitive knowledgeMetacognitive regulationPlanningInformation managementMonitoringDebuggingEvaluationMetacognitive awareness
      FValueCohen's dFValueCohen's dFValueCohen's dFValueCohen's dFValueCohen's dFValueCohen's dFValueCohen's dFValueCohen's d
      G WITHIN T00.01.753−0.060.00.979−0.060.13.722−0.070.01.9240.060.11.743−0.070.16.694−0.080.03.875−0.790.04.831−0.04
      G WITHIN T129.28.0001.1138.32.0001.0226.62.0001.0533.36.0001.1744.03.0001.369.78.0000.6414.88.0000.7826.51.0001.05
      G WITHIN T251.95.0001.4756.75.0001.3150.22.0001.4549.99.0001.4462.36.0001.6114.24.0000.7727.72.0001.0744.16.0001.36
      T WITHIN G163.54.00054.75.00063.31.00079.85.00073.86.00024.69.00017.83.00051.34.000
      G1 (T0 v. T1)−0.37−1.16−1.27−1.41−1.41−0.85−0.79−1.18
      G1 (T0 v. T2)−0.40−1.34−1.48−1.60−1.63−1.01−0.92−1.38
      G1 (T1 v. T2)−0.08−0.08−0.15−0.03−0.07−0.07−0.09−0.09
      T WITHIN G21.30.2820.69.5141.05.3522.87.0640.59.5621.64.2020.24.7810.81.448
      G2 (T0 v. T1)−0.12−0.09−0.11−0.32−0.05−0.19−0.05−0.15
      G2 (T0 v. T2)−0.04−0.14−0.05−0.31−0.12−0.26−0.01−0.14
      G2 (T1 v. T2)0.11−0.040.080.03−0.08−0.070.070.01
      Notes: G, group; G1, Experimental group; G2, Control group; T, time point; T0, baseline; T1, post-intervention (16 weeks after baseline); T2, 1-month follow-up (20 weeks after baseline).
      Each group was compared at three time points as seen in Table 3. The intervention group results revealed that before and after the intervention for all outcome measures had statistically significant differences (F(1, 48) = 17.83–79.85; all ps < .01). Metacognitive awareness and subcategory outcome measures effect sizes at post-intervention (Cohen's d = −1.41 to 0.37) and follow-up (Cohen's d = −1.63 to 0.40) were higher than baseline measures in the intervention group. However, metacognitive awareness and subcategory outcome measures in the control group showed that a change trend was not obvious at three time points, and there were no statistically significant differences on follow-up and post-intervention (F(1, 48) = 0.24–2.87; all ps > .05).

      Discussion

      This study found that nursing students' metacognitive awareness significantly improved in the intervention group, but there was no statistically significant difference in the control group. This indicates that traditional skill training courses do not cultivate students' metacognitive awareness. The intervention group results, meanwhile, showed that the NCST-C could improve metacognitive awareness. This may be partially attributable to our study's three-module online–offline design and multi-station examination design [
      • McCabe J.
      Metacognitive awareness of learning strategies in undergraduates.
      ]. Mansueto's research emphasizes that a maximum learning effect occurs when individuals are reinforced metacognitive beliefs and receive timely feedback [
      • Mansueto G.
      • Marino C.
      • Palmieri S.
      • Caselli G.
      Difficulties in emotion regulation: the role of repetitive negative thinking and metacognitive beliefs.
      ]. NCST-C provides periodic feedback in the three dimensions of online data feedback, classroom skill examination, and practical service testing. Students can observe their performance in real time, gaining a sense of satisfaction and accomplishment, which can stimulate internal learning motivation and affect metacognitive awareness [
      • Sart G.
      The effects of the development of metacognition on project-based learning.
      ].
      The intervention group had a significant improvement in its knowledge of cognition scores at 16 weeks compared with the control group. This improvement was long-lasting (over 1 month; Cohen's d = 1.31) and almost identical to what Gholami found in problem-based learning intervention [
      • Siqueira M.A.M.
      • Gonçalves J.P.
      • Mendonça V.S.
      • Martins M.A.
      Relationship between metacognitive awareness and motivation to learn in medical students.
      ]. Studies on metacognition have shown a positive relationship between students' knowledge of cognition and their problem-solving ability, in this regard, case-based blended learning modules can act as a catalyst for improving the knowledge of cognition in nursing students [
      • Siqueira M.A.M.
      • Gonçalves J.P.
      • Mendonça V.S.
      • Martins M.A.
      Relationship between metacognitive awareness and motivation to learn in medical students.
      ]. Adding case questions to the NCST-C individual and team examinations is the key point to improve the knowledge of cognition on nursing students in this study. A recent study showed that guided reciprocal peer questioning improved nursing students' knowledge of cognition [
      • Chan C.W.H.
      • Tang F.W.K.
      • Chow K.M.
      • Wong C.L.
      Enhancing generic capabilities and metacognitive awareness of first-year nursing students using active learning strategy.
      ]. This is similar to our study in that NCST-C stimulated students to think about how to use online learning resources and cooperative group learning, adapt to a new type of examination, and smoothly complete the cooperative group experimental course. This process promotes the accumulation of declarative, procedural, and conditional knowledge contained in knowledge of cognition.
      The intervention group showed a significant improvement in the regulation of cognition scores compared with the control group. This included improvements in students' planning, information management, monitoring, debugging, and evaluation abilities. This could be because metacognitive awareness is a multi-dimensional process that involves individual awareness in recalling and thinking about information and transforming it into behavior [
      • Turan S.
      • Demirel O.
      • Sayek I.
      Metacognitive awareness and self-regulated learning skills of medical students in different medical curricula.
      ]. Previous research has shown that more examination setting and team-based learning can improve students' information, time management skills, and enhances students' metacognitive awareness [
      • Rivers M.L.
      • Dunlosky J.
      • Persky A.M.
      Measuring metacognitive knowledge, monitoring, and control in the pharmacy classroom and experiential settings.
      ]. The NCST-C focused on multi-station nursing skill examination, cooperative group learning, and experiential online resources. The classroom teaching component focused on group teaching, which inspires students to use teamwork to solve problems together. These placed higher requirements on students' planning, information management, team monitoring, and self-debugging abilities. In addition, developing such capabilities would likely take 16 weeks in a university learning environment, which the NCST-C can achieve. The results of the one-month follow-up showed that these abilities were maintained.
      The students' adherence rates and open-ended comments indicated that the NCST-C was well-planned, flexible, and acceptable, and students appreciated the teaching and learning environment. The adherence rate of the intervention group was 100%. Our approach was acceptable to the nursing students because it fits in with the current Chinese culture and educational development. First, this high adherence rate may be attributable to the integrated learning and teaching methods. Module one mainly covered learning and teaching methods content, and students were familiarized with the new curriculum and learning forms [
      • Khodaei S.
      • Hasanvand S.
      • Gholami M.
      • Amini M.
      The effect of the online flipped classroom on self-directed learning readiness and metacognitive awareness in nursing students during the COVID-19 pandemic.
      ]. Second, we use cooperative group learning methods, integrate experimental content design online–offline teaching mode, and use individual and team multi-station skill assessment were key to controlling teaching quality and enhancing students' interest in learning [
      • Desender K.
      • Teuchies M.
      • Gonzalez-Garcia C.
      • Brass M.
      Metacognitive awareness of difficulty in action selection: the role of the cingulo-opercular network.
      ]. The smooth implementation of the NCST-C can also be attributed to the efforts of our course team teachers to course resources and provide professional guidance.

      Limitations

      Due to time and conditions limitation, our study has some limitations. First, this was a quasi-experimental study and administered in only one school. The limitations of this design include its small sample size, potential desirability bias, selection bias, and limited external validity. Future studies should be tested in other schools and explore the effects of NCST-C using larger and more diverse samples. Second, self-reported questionnaires have inherent limitations, inability to deeply analyze the emotional experience of participating in the course. Future studies should consider adding qualitative interviews to explore students' subjective feelings about participating in a course. Third, given the positive effects on the metacognitive awareness and 1 month long-term effects, future studies could examine its effects on other aspects and explored beyond 1 month long-term effects.

      Conclusion

      This study demonstrated that the NCST-C effectively developed metacognitive awareness among nursing students. Its design provides a new type of experimental course for improving metacognitive awareness. We hope the findings can specifically inform multi-dimensional structural design in nursing curricula reform while providing a practical basis for nursing curricula development.

      Contributions

      Shasha Li: conceiving, designing the study and writing the paper. Minerva de Ala: design direction. Dandan Mao: data survey and analyzing the data. Afeng Wang and Congwen Wu: implementation of teaching. The authors gave final approval of the version to be published.

      Funding

      The study was funded by Ministry of Education Industry-University Cooperation Collaborative Education Project, China (No. 202202244010 ). Ministry of Education Humanities and Social Sciences Research Youth Fund Project, China (No. 17YJCZH092 ).

      Conflict of interest

      The authors declare that they have no conflicts of interest.

      Acknowledgments

      We thank all of the students and teacher who participated in this study.

      References

        • Curran V.R.
        • Fairbridge N.A.
        • Deacon D.
        Peer assessment of professionalism in under graduate medical education.
        BMC Med Educ. 2020; 20: 504-510https://doi.org/10.1186/s12909-020-02412-x
        • Sezer H.
        • Şahin H.
        Faculty development program for coaching in nursing education: a curriculum development process study.
        Nurse Educ Pract. 2021; 55: 1-10https://doi.org/10.1016/j.nepr.2021.103165
        • Medina M.S.
        • Castleberry A.N.
        • Persky A.M.
        Strategies for improving learner metacognition in health professional education.
        Am J Pharm Educ. 2017; 81: 78-86https://doi.org/10.5688/ajpe81478
        • Abdelrahman R.M.
        Metacognitive awareness and academic motivation and their impact on academic achievement of Ajman university students.
        Heliyon. 2020; 6 (1–7)https://doi.org/10.1016/j.heliyon.2020.e04192
        • Flavell J.H.
        Cognitive development.
        2nd ed. Prentice-Hall, Upper Saddle River, NJ1988: 99-115
        • Shea N.
        Metacognition and abstract concepts.
        Philos Trans R Soc Lond B Biol Sci. 2018; 373: 1-7https://doi.org/10.1098/rstb.2017.0133
        • Schraw G.
        • Dennison R.S.
        Assessing metacognitive awareness.
        Contemp Educ Psychol. 1994; 19: 460-475
        • Akturk A.O.
        • Sahin I.
        Literature review on metacognition and its measurement.
        Proc Soc Behav Sci. 2011; 15: 3731-3736https://doi.org/10.1016/j.sbspro.2011.04.364
        • Song J.H.H.
        • Loyal S.
        • Lond B.
        Metacognitive awareness scale, domain specific (MCAS-DS): assessing metacognitive awareness during raven's progressive matrices.
        Front Psychol. 2021; 11: 1-14https://doi.org/10.3389/fpsyg.2020.607577
        • Chang C.
        • Colón-Berlingeri M.
        • Mavis B.
        • Solomon D.
        Medical student progress examination performance and its relationship with metacognition, critical thinking, and self-regulated learning strategies.
        Acad Med. 2021; 96: 278-284https://doi.org/10.1097/ACM.0000000000003766
        • Bernstein A.
        • Hadash Y.
        • Lichtash Y.
        • Fresco D.M.
        Decentering and related constructs: a critical review and metacognitive processes model.
        Perspect Psychol Sci. 2015; 10: 599-617https://doi.org/10.1177/1745691615594577
        • Desender K.
        • Teuchies M.
        • Gonzalez-Garcia C.
        • Brass M.
        Metacognitive awareness of difficulty in action selection: the role of the cingulo-opercular network.
        J Cogn Neurosci. 2021; 33: 2512-2522https://doi.org/10.1162/jocn_a_01773
        • Shekhar M.
        • Rahnev D.
        Sources of metacognitive inefficiency.
        Trends Cogn Sci. 2021; 25: 12-23https://doi.org/10.1016/j.tics.2020.10.007
        • Bektas I.
        • Bektas M.
        • Ayar D.
        • Celik I.
        The predict of metacognitive awareness of nursing students on self-confidence and anxiety in clinical decision-making.
        Perspect Psychiatr Care. 2021; 57: 747-752https://doi.org/10.1111/ppc.12609
        • Rashwan Z.I.
        • Busebaia T.J.
        • Al-Sabbagh A.S.
        • Eweida R.S.
        Effect of guided reciprocal peer questioning strategy on pediatric nursing students' self-esteem and metacognitive awareness: current approach and future directions.
        Nurse Educ Today. 2021; 107: 1-9https://doi.org/10.1016/j.nedt.2021.105153
        • Chan C.W.H.
        • Tang F.W.K.
        • Chow K.M.
        • Wong C.L.
        Enhancing generic capabilities and metacognitive awareness of first-year nursing students using active learning strategy.
        BMC Nurs. 2021; 20: 81-89https://doi.org/10.1186/s12912-021-00601-7
        • Blake T.K.
        Journaling; an active learning technique.
        Int J Nurs Educ Scholarsh. 2005; 2: 1-15https://doi.org/10.2202/1548-923x.1116
        • Waltz C.F.
        • Jenkins L.S.
        • Han N.
        The use and effectiveness of active learning methods in nursing and health professions education: a literature review.
        Nurs Educ Perspect. 2014; 35: 392-400https://doi.org/10.5480/13-1168
        • Hsu L.L.
        • Hsieh S.I.
        Factors affecting metacognition of undergraduate nursing students in a blended learning environment.
        Int J Nurs Pract. 2014; 20: 233-241https://doi.org/10.1016/j,nedt.2021.105164
        • Mahmoud H.
        The effect of cooperative learning strategy on undergraduate nursing students enrolled in nursing administration course.
        Int J Learn. 2017; 7: 722-738https://doi.org/10.5296/ijld.v7i1.11021
        • Siqueira M.A.M.
        • Gonçalves J.P.
        • Mendonça V.S.
        • Martins M.A.
        Relationship between metacognitive awareness and motivation to learn in medical students.
        BMC Med Educ. 2020; 20: 393-401https://doi.org/10.1186/s12909-020-02318-8
        • Gholami M.
        • Moghadam P.K.
        • Mohammadipoor F.
        • Pour A.H.
        Comparing the effects of problem- based learning and the traditional lecture method on critical thinking skills and metacognitive awareness in nursing students in a critical care nursing course.
        Nurse Educ Today. 2016; 45: 16-21https://doi.org/10.1016/j.nedt.2016.06.00
        • Martirosov A.L.
        • Moser L.R.
        Team-based learning to promote the development of metacognitive awareness and monitoring in pharmacy students.
        Am J Pharm Educ. 2021; 85: 131-136https://doi.org/10.5688/ajpe848112
        • Hao X.
        • Peng X.
        • Ding X.
        • Li K.
        Application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic: a systematic review.
        Nurse Educ Today. 2022; 108: 1-11https://doi.org/10.1016/j.nedt.2021.105183
        • Meng X.H.
        • Xu X.Y.
        • Chen H.L.
        • Zhang L.
        The effectiveness of combining e-learning, peer teaching, and flipped classroom for delivering a physiology laboratory course to nursing students.
        Adv Physiol Educ. 2022; 46: 21-26https://doi.org/10.1152/advan.00062.2020
        • Schramm J.
        • Uranga T.
        • Birkle A.
        • Taylor L.
        Improving holistic care skills for advanced practice registered nurses: integration of a complementary and alternative medicine course into a doctor of nursing practice curriculum.
        J Dr Nurs Pract. 2021; 11: 421-431https://doi.org/10.1891/JDNP-D-20-00077
        • Jiang Y.
        • Chen Y.
        • Lu J.
        • Wang Y.
        The effect of the online and offline blended teaching mode on English as a foreign language learners' listening performance in a Chinese context.
        Front Psychol. 2021; 12: 1-16https://doi.org/10.3389/fpsyg.2021.742742
        • Liu H.Y.
        Promoting creativity of nursing students in different teaching and learning settings: a quasi-experimental study.
        Nurse Educ Today. 2022; 108: 1-9https://doi.org/10.1016/j.nedt.2021
        • Hwang G.J.
        • Chang C.Y.
        • Ogata H.
        The effectiveness of the virtual patient-based social learning approach in undergraduate nursing education: a quasi-experimental study.
        Nurse Educ Today. 2022; 108: 21-31https://doi.org/10.1016/j.nedt.2021.105164
        • Brauer D.G.
        • Ferguson K.J.
        The integrated curriculum in medical education: AMEE Guide No. 96.
        Med Teach. 2015; 37: 312-322https://doi.org/10.3109/0142159X.2014.970998
        • Lipsey M.W.
        • Wilson D.B.
        The efficacy of psychological, educational, and behavioral treatment: confirmation from meta-analysis.
        Am Psychol. 1993; 48: 1181-1209
        • McCabe J.
        Metacognitive awareness of learning strategies in undergraduates.
        Mem Cognit. 2011; 39: 462-476https://doi.org/10.3758/s13421-010-0035-2
        • Mansueto G.
        • Marino C.
        • Palmieri S.
        • Caselli G.
        Difficulties in emotion regulation: the role of repetitive negative thinking and metacognitive beliefs.
        J Affect Disorder. 2022; 308: 473-483https://doi.org/10.1016/j.jad.2022.04.086
        • Sart G.
        The effects of the development of metacognition on project-based learning.
        Procedia Soc Behav Sci. 2014; 152: 131-136https://doi.org/10.1016/j.sbspro.2014.09.169
        • Siqueira M.A.M.
        • Gonçalves J.P.
        • Mendonça V.S.
        • Martins M.A.
        Relationship between metacognitive awareness and motivation to learn in medical students.
        BMC Med Educ. 2020; 20: 393-396https://doi.org/10.1186/s12909-020-02318-8
        • Turan S.
        • Demirel O.
        • Sayek I.
        Metacognitive awareness and self-regulated learning skills of medical students in different medical curricula.
        Med Teach. 2009; 31: e477-e483https://doi.org/10.3109/01421590903193521
        • Rivers M.L.
        • Dunlosky J.
        • Persky A.M.
        Measuring metacognitive knowledge, monitoring, and control in the pharmacy classroom and experiential settings.
        Am J Pharm Educ. 2020; 84: 549-560https://doi.org/10.5688/ajpe7730
        • Khodaei S.
        • Hasanvand S.
        • Gholami M.
        • Amini M.
        The effect of the online flipped classroom on self-directed learning readiness and metacognitive awareness in nursing students during the COVID-19 pandemic.
        BMC Nurs. 2022; 21: 2-11https://doi.org/10.1186/s12912-022-00804-6