Evaluating Training

Clause 6.2.2: Competence, Awareness and Training in ISO9001:2000 standard sets the training requirements for organizations. Problems with evaluating the outcomes of training arise during examination and verification. Many organizations have never formulated specific provisions on desired outcomes of training, and do nothing but keep test papers and the grades of exams, without evaluation.

Contribution of the MERISE-Type Conceptual Data Model to the Construction of Monitoring and Evaluation Indicators of the Effectiveness of Training in Relation to the Needs of the Labor Market in the Republic of Congo

This study proposes the use of the MERISE conceptual data model to create indicators for monitoring and evaluating the effectiveness of vocational training in the Republic of Congo. The importance of MERISE for structuring and analyzing data is underlined, as it enables the measurement of the adequacy between training and the needs of the labor market. The innovation of the study lies in the adaptation of the MERISE model to the local context, the development of innovative indicators, and the integration of a participatory approach including all relevant stakeholders. Contextual adaptation and local innovation: The study suggests adapting MERISE to the specific context of the Republic of Congo, considering the local particularities of the labor market. Development of innovative indicators and new measurement tools: It proposes creating indicators to assess skills matching and employer satisfaction, which are crucial for evaluating the effectiveness of vocational training. Participatory approach and inclusion of stakeholders: The study emphasizes actively involving training centers, employers, and recruitment agencies in the evaluation process. This participatory approach ensures that the perspectives of all stakeholders are considered, leading to more relevant and practical outcomes. Using the MERISE model allows for: • Rigorous data structuring, organization, and standardization: Clearly defining entities and relationships facilitates data organization and standardization, crucial for effective data analysis. • Facilitation of monitoring, analysis, and relevant indicators: Developing both quantitative and qualitative indicators helps measure the effectiveness of training in relation to the labor market, allowing for a comprehensive evaluation. • Improved communication and common language: By providing a common language for different stakeholders, MERISE enhances communication and collaboration, ensuring that all parties have a shared understanding. The study’s approach and contribution to existing research lie in: • Structured theoretical and practical framework and holistic approach: The study offers a structured framework for data collection and analysis, covering both quantitative and qualitative aspects, thus providing a comprehensive view of the training system. • Reproducible methodology and international comparison: The proposed methodology can be replicated in other contexts, facilitating international comparison and the adoption of best practices. • Extension of knowledge and new perspective: By integrating a participatory approach and developing indicators adapted to local needs, the study extends existing research and offers new perspectives on vocational training evaluation.

Training and Implementation of Subjective Questions Scoring System Based on the Baidu Qianfan Model Platform

Leveraging the Baidu Qianfan model platform,this paper designs and implements a highly efficient and accurate scoring system for subjective questions,focusing primarily on questions in the field of computer network technology.The system enhances the foundational model by utilizing Qianfan’s training tools and integrating advanced techniques,such as supervised fine-tuning.In the data preparation phase,a comprehensive collection of subjective data related to computer network technology is gathered,cleaned,and labeled.During model training and evaluation,optimal hyperparameters and tuning strategies are applied,resulting in a model capable of scoring with high accuracy.Evaluation results demonstrate that the proposed model performs well across multiple dimensions-content,expression,and development scores-yielding results comparable to those of manual scoring.

Evaluation of Training Program for New Nurses to Enhance Caring Behavior

This research was aimed to evaluate the training program enhancing caring behaviors of new nurses by Kirkpatrick＇s four level for evaluation model： reaction, learning, behavior, and result of the program. The participants were new nurses, preceptors of new nurses, administrators, patients and members of the patients＇ family cared by new nurses. The research instruments were： （1） five questionnaires toward program suitability, knowledge, attitude, caring expression, and result to organization. The mean, standard deviation and dependent sample t-test were used for data analysis; （2） guidelines for focus group discussion and semi-structural questionnaire analyzed by content analysis. The study revealed that： （1） the mean of suitability was at the very high level （X = 4.49, SD. = .30）; （2） the knowledge and attitude after training were significantly higher than before training at .000 level, [t =-21.65, p = .000 and t = -19.30, p = .000）; {3} caring behavior after training was significantly higher than before training at the .000 level; and {4} the result of the program was at the high level { X = 4.25, SD. = .17}, related to the result of semi-structured interview and focus group discussion. These evaluation research finding suggested that administrators can use for improving the preparation of any project and apply to evaluate other training programs, developing human resource system.

Cultivating global health professionals: evaluation of a training course to develop international consulting service competence in China

Background:China’s accelerating development and increasingly important role in global health engagement create a great demand for global health professionals including international consulting experts.This study reported the detailed development and evaluation of an international consulting training for global health workforce.Methods:Based on Kirkpatrick’s model,a mixed-methods approach was used to evaluate the effectiveness of the training.Quantitative and qualitative data on participants’reaction,learning,and application of the learned knowledge and skills were collected by a training evaluation survey at the ending of training and a follow-up interview in three months after the training.Results:Thirty-six participants attended the training and 34 of them completed quantitative investigation.The training satisfaction evaluations were positive,for which participants rated the training program highly and over 90%of them agreed with the usefulness of the training.About knowledge and skills change,participants showed improved consulting knowledge and skills from pre-to post-training(P<0.001).A total of 23 participants accepted follow-up interview,and most participants applied knowledge and skills learned from the training in their daily work or study.However,only 30.4%of participants applied their learning in the consulting program.The largest barrier of application was the lack of consulting opportunities.In addition,almost all the participants reported that they would be glad to attend more training courses in the future.Conclusion:The international consulting training program was well-received,and was feasible to improve the consulting service competence of global health professionals.According to participants’feedback,it is essential to develop and expand consulting training in the field of global health.

Anti-money Laundering in Banks:Towards a Model for Training Evaluation

This paper aims to propose an evaluation model specific for anti-money laundering training in banks.The model was structured taking from some of the most popular training evaluation models proposed in the literature the evaluation levels considered suitable for evaluating anti-money laundering training programs.Two additional evaluation levels are included,which assess possible exposure of training activity to the risks associated with knowledge management.The originality of the proposed model lies in the fact that,to the best of the authors’knowledge,to date,there is still no specific model for evaluating anti-money laundering training in banks.Among the limitations of this study is the fact that,for the moment,the proposed model is a in a purely theoretical formulation.

A CRANE AMONG THE CHICKENS?: EVALUATING A TEACHER TRAINING PROGRAMME

"Sometimes you try to share the ideas, they even turn up their noses at you, describing you as ’trying to be a crane among the chicks’." This rather forlorn comment comes from a middle school English teacher recently returned to his school in a remote province of China, full of enthusiasm for the new methods of teaching English gathered over two years away on a teacher-training course held in Shanghai. It gives the title to a recently published study of the fate that met this teacher and others like him after they had completed their course. Conducted by staff at Shanghai International Studies University (SISU), this set out to contribute to the evaluation of two in-service training courses for teachers of English held at that university through a follow-up study of former trainees. The study took place under the auspices of the Chinese State Education Commission and the British Overseas Development Administration, as did the two courses themselves. The most recent of these was a senior middle school teacher training (SMSTT) course for teachers working in less developed areas of China. This two-year course, designed and run at five universities, is still in operation at SISU and formed the primary focus of attention of the study. The other was a one-year course in advanced teacher training (ATT) for lecturers in higher education.

Training effectiveness evaluation of helicopter emergency relief based on virtual simulation

Helicopters are playing an increasingly important part in emergency relief, such as earthquake rescue, firefighting and medical transport. With the development of virtual simulation technology, virtual simulation-based training is widely used in the training of the helicopter crew especially for the dangerous and costly missions mentioned above. A complete training effectiveness evaluation method is proposed to evaluate the trainees＇ training effect based on virtual simulation in this paper. A key to this method is regarding the complicated process as a discrete event-activity flow system and establishing the evaluation indicator system. Then expert group and Analytical Network Process（ANP） are applied to determine the weight of indicators. When the training data are processed, there is a novel attempt to apply Fuzzy Comprehensive Evaluation（FCE） model to calculate the two categories of indicators. Eventually, an experiment and the analysis were carried out to validate the evaluation method.

Training for percutaneous renal access on a virtual reality simulator

Background The need to develop new methods of surgical training combined with advances in computing has led to the development of virtual reality surgical simulators. The PERC MentorTM is designed to train the user in percutaneous renal collecting system access puncture. This study aimed to validate the use of this kind of simulator, in percutaneous renal access training. Methods Twenty-one urologists were enrolled as trainees to learn a fluoroscopy-guided percutaneous renal accessing technique. An assigned percutaneous renal access procedure was immediately performed on the PERC MentorTM after watching instruction video and an analog operation. Objective parameters were recorded by the simulator and subjective global rating scale （GRS） score were determined. Simulation training followed and consisted of 2 hours daily training sessions for 2 consecutive days. Twenty-four hours after the training session, trainees were evaluated performing the same procedure. The post-training evaluation was compared to the evaluation of the initial attempt. Results During the initial attempt, none of the trainees could complete the appointed procedure due to the lack of experience in fluoroscopy-guided percutaneous renal access. After the short-term training, all trainees were able to independently complete the procedure. Of the 21 trainees, 10 had primitive experience in ultrasound-guided percutaneous nephrolithotomy. Trainees were thus categorized into the group of primitive experience and inexperience. The total operating time and amount of contrast material used were significantly lower in the group of primitive experience versus the inexperience group （P=0.03 and 0.02, respectively）. Conclusions The training on the virtual reality simulator, PERC MentorTM, can help trainees with no previous experience of fluoroscopy-guided percutaneous renal access to complete the virtual manipulation of the procedure independently. This virtual reality simulator may become an important training and evaluation tool in teaching fluoroscopy-guided percutaneous renal access.

Evaluation of Fire Service Command Unit Trainings

The lack of routine and training of command units and emergency managers is among the main causes of suboptimal decisions and could lead to serious consequences.To ensure optimal standards of emergency management training,specific and valid evaluation tools are needed—but are lacking.Thus,the present study’s purpose is to validate instruments for the evaluation of tactical and strategic leader trainings,in particular command unit trainings,based on survey data of n=288 German Command Unit members.Resulting questionnaires were named"FIRE-CU"(Feedback Instrument for Rescue forces Education–Command Unit)and"FIRE-CPX"(Feedback Instrument for Rescue forces Education-Command Post e Xercise scale).Results of confirmatory factor analyses show a good fit for the postulated four-dimensional structure of process scales in the FIRE-CU(trainer’s behavior,structure,overextension,group),for the two-dimensional structure of outcome scales in the FIRE-CU(self-rated competence,transfer),and for the one-dimensional structure of the FIRE-CPX.Further,strong evidence is found for reliability as well as for convergent,divergent,and concurrent validity of both the FIRE-CU and FIRE-CPX.Implications for research and practical application are also discussed to enable broad applicability in various educational programs for public security and crisis management.