Evaluating the effectiveness of electronic distance learning. The effectiveness of distance learning and the practice of using the results of distance learning Criteria for distance learning courses

In this article we will consider the following questions:

• Goals and objectives distance learning

Methods for assessing the effectiveness of distance learning

In training, the result is important - it is economically feasible, justifying all the resources expended in this process. There are several methods for determining how effectively it works. Below are the most common ones.

1) Questionnaire

The simplest, and therefore the most popular method: using a survey, the level of satisfaction with the course is measured among training participants, usefulness, completeness and accessibility are assessed educational materials. The survey is carried out immediately after the completion of the training activities. In this case, the survey can be conducted not only among the trainees, but also among their managers - they will be able to answer the most important question: have there been any changes in the work of employees after the training?

2) Testing

Testing, as well as other forms of testing knowledge (practical tasks, exercises, etc.), allows you to measure how well the information from training course. It is useful to conduct testing in two stages - immediately after training and after some time to assess residual knowledge. Sometimes they also arrange preliminary testing, before the start of training, in order to measure the level of knowledge “before”, and then compare it with “after”.

3) Analysis of the dynamics of individual employee indicators

If a company keeps statistics on any performance indicators, these data can be used to evaluate the impact of training on the level of skill development. This could be: sales volumes, the number of mistakes made, the number of complaints from customers, the results of Mystery Shopping (checking the work of employees with the help of mystery shoppers).

4) Analysis of the impact of training on the company’s business goals

A rather complex way to assess the effectiveness of a LMS: you need to evaluate the relationship of training courses with indicators such as the level of customer and employee satisfaction, the quality of products or services provided, etc. The difficulty is to separate learning outcomes from the influence of many other factors.

5) Benchmarking

A relatively new method, the essence of which is to compare key performance indicators with similar indicators from competitors, especially from industry leaders. The main difficulty is that it is rare for a company to disclose information about its employee training program and its results. However, such data are rarely voiced at specialized conferences or mentioned in specialized publications.

In a survey among large Russian companies using distance learning, it turned out that after completing the course, 83% of them conduct a survey (or another method of obtaining feedback), testing (or another method of testing knowledge) - 71%, the change in performance is estimated at 48 % of these companies, and the economic effect is calculated at 10%.

Modern methods of distance learning

There are four schemes for organizing the operation of the distance learning system in Russia and the CIS countries. They are used either individually or in combination with each other.

Ways to organize distance learning in a company:

1. Self-learning

As the name implies, students independently interact with the LMS, without the participation of teachers. This method allows the organization to save human resources and is well suited for transferring knowledge that is easy to master. Online tests are used as a test, which also do not require time investment on the part of the teacher.

2. Instructor-led training

In this case, students not only get acquainted with the training course, but also interact in some way with the teacher. For example, it can provide feedback on completed tasks that cannot be replaced by testing. Another common format: webinars, where the LMS plays the role of a technical platform, and the main carrier of knowledge is the teacher.

3. Personalized learning

The one-to-one distance learning scheme (one teacher and one student) is rarely used, because it is too labor-intensive, and it is not practical to purchase or develop a distance learning system just so that two people can interact remotely with each other.

4. Combined training

A frequently used method is when employees study part of the training materials in the LMS, and the other part is presented to them by a teacher (business coach) during a face-to-face meeting. The “LMS + full-time training” scheme is the most common and effective: it is obvious that employees are able to learn some knowledge on their own, so there is no need to waste the teacher’s time; on the other hand, there are things that are best learned in practice, for example, during a training or master -class.

Purpose of distance learning

The vision of the ultimate goal should not escape attention when it comes to choosing a distance learning method and how to evaluate its effectiveness: methods should be determined by economic feasibility, and the object of measurement should be the factors influencing the profitability of the company. Passing testing, for example, is not an end in itself, but an intermediate result in the learning process, which should be aimed at improving skills to the level necessary to successfully complete the assigned tasks.

Thus, the ultimate goal of implementing a distance learning system is to obtain greater profits by using new skills and knowledge acquired by employees.

Modern method Distance learning (DL) via the Internet has become a logical continuation of earlier methods of obtaining “remote” education. A lot is written about it, it is advertised and called a worthy alternative to “face-to-face” education. However, the question of its effectiveness is practically not covered. And this is very important, since the popularity of preschool education is constantly growing, which means everything larger number specialists gain knowledge in this way.

Modern distance learning method(DO) via the Internet has become a logical continuation of earlier methods of obtaining “remote” education: correspondence, correspondence, television, etc. A lot is written about it, it is advertised and called a worthy alternative to “face-to-face” education. However, the question of its effectiveness is practically not covered. And this is very important, since the popularity of distance learning is constantly growing, which means an increasing number of specialists are gaining new knowledge in this way.

Prerequisites for the growing popularity of distance learning

The main factor determining constant growth popularity of distance learning, is that this method allows one to overcome limitations in the availability of education associated with geographical and spatial factors. At the same time, distance learning courses today are often offered at a more affordable price than “face-to-face” ones. Which, however, is quite logical: organizing the educational process using remote technologies requires significant costs just for initial stage(which in most universities has already been successfully completed a long time ago), and in the future does not require serious additional financial investments.

In other words, receiving education using distance learning allows you to:

• save money - not only do pre-service courses cost less than full-time ones, but you also don’t have to spend money on accommodation and food in another city;
• independently plan the learning process - the student chooses how and when it is convenient for him to study the educational material;
• receive education without interrupting work - a self-planned schedule of “lectures” allows you to combine work with study.

It is quite natural that students, constrained by time, geographic and financial constraints (and there are more and more of them every year) make their choice in favor of distance learning. But how effective is such training?

Types of distance learning

Initially, even at the dawn of the formation of the BS, it was clear that despite all benefits of distance learning, this method is devoid of those main factors that provide (or can provide) high quality learning in a traditional classroom or individual education situation: the presence of a talented and charismatic teacher who is able to develop a student’s interest in the subject, captivate him, encourage him to comprehend and solve problems, and also maintain high self-esteem and self-confidence.

Obviously, all this is probably missing in teaching by correspondence and - with a very high degree of probability - in television lessons. But this state of affairs was considered a normal cost of the method, a payment for the opportunity to study at a distance and without special expenses. The advent of technology has provided greater possibilities for supplying material and greater ease of use of the method.

Modern Remote education presented in two types:

• synchronous - classes are carried out in real time. That is, at the appointed time, a student (or a group of students) “meets” online with a teacher who gives lectures, answers students’ questions and asks counter questions to check the level of mastery of the material. As a rule, virtual classes are conducted using the Internet and web cameras.
• asynchronous - a classic version of DL, in which the student is sent training material for self-study followed by passing tests and exams. At the same time, a student can take tests both in person and remotely.

The effectiveness of distance learning

It's not difficult to guess that synchronous distance learning, which differs from the classic “face-to-face” only in the way the teacher interacts with students, is not inferior in effectiveness to its traditional counterpart. True, there is no need to talk about temporary independence here, since the student must go online in a clear manner certain time. And the cost of such training, even if it is lower than that of full-time training, will not be much lower. Therefore, of the three main advantages of distance learning listed above (financial, time and geographic accessibility), only geographic independence is evident.

But the efficiency asynchronous distance learning is very doubtful, since here a lot depends on the student’s ability to learn, the level of his preparation, the correspondence of the complexity of the chosen course to the student’s capabilities, the availability of support and assistance from the teacher, etc. We should also not forget about the student’s motivation.

Experienced teachers believe that “... when the motivation to learn is very low, one can, without risking a mistake, assume that potential academic success will be reduced to some extent.” The specificity of the asynchronous method of DL in the matter of learning motivation is manifested in the absence of standard traditional education motivating moments such as group pressure, familiar learning situations, social factors etc.

In other words, the effectiveness of asynchronous distance learning in to a greater extent depends on the student himself: on his desire to learn and gain new knowledge, the ability to self-organize and the desire for self-improvement.

Conclusion

Talking about effectiveness of distance learning– and more broadly: education with the help of new information and communication technologies, one can draw an unambiguous conclusion. Yes, it can be considered quite effective, but only if it is a synchronous learning method or the student has strong enough motivation to study the material independently. In other cases, distance learning is more of a self-deception and a way of complacency, allowing you to get only “crusts”, but not the proper level of knowledge.

Personnel training in modern fast-growing enterprises requires a significant investment of both time and money. This is forcing many businesses to turn to e-learning, which allows them to deliver the necessary information to staff anywhere, anytime via the Internet, faster and cheaper.
The effectiveness of distance learning (DL) is determined by the combination of five key factors that enable learners to retain more information, increase their awareness, achieve better performance and thereby increase their return on invested capital.
These factors include:
- interactivity,
- memorability,
- flexibility in use,
- providing assistance,
- availability.

Interactivity
The introduction of interactivity into the learning process makes the student’s participation more active, makes him try to achieve maximum results. Interactivity also helps instructors incorporate more complex content into courses.
Interactivity can be combined with simulation during the learning process of the environment that students must become familiar with. For example, if the course is about using computer program, the image on the screen must correspond to that displayed when this program is running, and the student can be given the task to perform an operation corresponding to one of the issues being studied. For more complex topics in this regard, you can model a business process and ask the learner to go through it.

Memorability
To remember better, students must feel the importance of the material being studied. Structuring the headings of the topics being studied also helps with this, ensuring that the necessary information is retained in memory. This, in turn, makes it more likely that they will apply the acquired knowledge in future real-world projects. Additionally, organizing headings helps avoid repetition, which often reduces the effectiveness of eLearning.

Flexibility in use
The system should provide training for employees with different levels of training and capabilities. It is necessary that students can easily move along the training course, monitor their movement, and also be able to return to the position where they were during the previous session of accessing the training course.
As a company grows, course content rarely remains the same, so the learning tool must allow for changes to the learning content. The possibility of such changes must be built into the learning tool at the very beginning. The change itself must be planned and its implementation monitored.

Providing assistance
Since e-learning usually does not take place in a group setting, it is important that the learning system provides assistance to the learner. Your employees will need instructions for completing the course, navigation tools, tips for completing assignments, links to obtain definitions, support for technical questions, etc. A button for calling help should be accessible from any slide in the course.
All this will help employees focus on learning and not be distracted by annoying interruptions. It is also useful to provide a section for frequently asked questions, as well as a glossary of terms that may be unfamiliar to the trainees, to provide flexibility for trainees. These tools can become valuable reference resources for employees even after training is completed.

Availability
Due to workloads, employees often find it difficult to find time for training. To solve this problem it is necessary to use various ways delivery of educational content.
If training material is available online, ensure that employees can use the material outside of work.
For those offices where there are difficulties with access to the Internet, for employees who want to study the material at home, as well as for technical staff who need reference material outside the office, it is possible to provide for the creation of a version of the training material on CD-ROM. The costs of creating such a training disk will not be comparable with the costs of expanding training.

Using the example of a company with 4,000 employees, let’s look at the experience of implementing distance learning. The motto of its organizers: ensure accessibility and mass participationtraining of employees, subject to the same quality of training for everyone.

Efficiency indicators of the distance learning system (DLS)

Launching distance learning in a company is a rather expensive project. To assess the feasibility of spending on it, it is necessary to develop a system of performance indicators for LMS. Among these are:

• general (characterize the functioning of distance learning in the company as part of the overall training system);
• private organizational;
• quality indicators that go back to the concept of pedagogical effectiveness.

The key issue in achieving the effectiveness of distance learning concerns the implementation of standards for its organization in the company. CJSC Insurance Group UralSib has identified the following KPIs in this area:

1. Preparation and approval of a document establishing the procedure for organizing distance learning, namely containing a description of specific procedures, including those related to the creation curricula, as well as document flow charts and reporting standards.
2. All directors, functional and line managers Those who determine the need for distance learning should be familiar with the procedure for its organization.
3. All branches and sales points of the regional network must have access to the LMS, i.e. ensuring the technical capability of distance learning for 100% of employees.
4. All areas of the organization’s activities must be provided with support for distance learning in accordance with the existing need. It is identified annually as part of the planning procedure. The volume, timing and topics are determined in accordance with the plans of business and functional departments - the main initiators of distance learning. In particular, it is mandatory when introducing a new insurance product, assessing the qualifications of key specialists to confirm or change authority (to enter into contracts or settle losses), familiarizing employees with new or adjusted policies and rules, passing certification forms required by law, post-training support for full-time programs training, adaptation of new employees (orientation course, basic programs), expansion or change of functional responsibilities of employees, etc.
5. All business unit requests for reports on staff development through distance learning must be completed within the specified time frame.

The first three indicators were formulated as criteria for achieving goals for improving the LMS after the launch of the pilot project, which were assessed based on the fact of execution. The fourth is specified annually in accordance with the growing need for distance learning. All of them are aimed at ensuring the stability and sustainability of the LMS. The fifth of these indicators is the constant KPI of the learning system in general and distance learning in particular. He talks about the usefulness and productivity of training representatives of business units and management.

Private ones are also important. organizational performance indicators of LMS administration:

• processing of applications for distance learning within N working days at 100%;
• organization of educational activities of the LMS in accordance with the deadlines stated by the customer (the head of the regional structural unit) 100%.

These indicators allow us to assess the quality of the process organization and influence the level of satisfaction of specialists and managers with the distance learning system. Interestingly, to evaluate the administration, it is enough to simply observe: which of the HR department specialists do the training participants and the branch employees responsible for monitoring prefer to call in case of problems and for support.

Distance learning courses can be created by both internal specialists and external providers of educational and consulting services - the choice is made by the company itself. Indicators program development efficiency may be the following:

• implementation of the plan for creating and modifying courses by 100%;
• preparing and adjusting programs on time;
• providing 100% of training programs with knowledge testing tools (tests, tasks, cases);
• development and approval of standards for the creation of distance learning programs;
• inclusion educational presentations, methodologically oriented towards distance learning, in 100% of programs.

The latter indicator may be underestimated, but it contains several important aspects. The educational material of the LMS must be visual, structured, logically organized, free from contradictions and, most importantly, self-sufficient, since there is no teacher next to the training participant who could give the necessary explanations and comments. The course should also include algorithms for the employee’s work, examples of calculations or communication with the client, reminders, “cheat sheets” with speech patterns, phrases useful for work, etc. Such elements ensure the effectiveness of the training course.

The focus on these indicators contributes to the regularity, stability and consistency of program development processes, to ensure that the educational materials used take into account the specifics of the distance learning form. Analysis of the ratio of the predicted and actually spent time by the student on completing the course and certification allows us to identify shortcomings in methodological or organizational work. In practice, using all of the five indicators listed is impossible, since it would take too much time to track them. In CJSC Insurance Group UralSib, the first and third of these are mandatory.

When analyzing the effectiveness of distance learning, it is also important to take into account indicators pedagogical effectiveness. This is a systemic concept traditionally used in studies of the effectiveness of distance learning in educational institutions(for example, in the works of A. A. Andreev, T. Russell, O. K. Filatov, D. V. Chernilevsky, etc.). The main criterion is the share (in percentage) of certified training participants from their total number. In the UralSib Insurance Group company, it was included in the list of performance indicators for the enterprise as a whole, formulated as “the percentage of certified employees.” The importance of this indicator stems from both general training goals (ensuring the appropriate level of personnel qualifications) and private ones. For example, product training is organized so that sales department specialists know and understand the terms and conditions of insurance. An introductory course telling about the history, traditions and achievements of the enterprise is carried out so that employees remember what company they work for, understand where and when this knowledge can be useful to them, and how it can be used in their daily activities. Mastering policies (for example, information security) is necessary in order to provide conditions for their compliance. Studying the typical difficulties of working in accounting databases and systems allows you to maintain high accuracy and integrity of stored information. And it is obvious that it is impossible to monitor the achievement of the listed (and other) goals without testing the acquired knowledge. Thus, the proportion of employees who successfully passed certification becomes a significant indicator of both the effectiveness of the distance learning system and the development of personnel as a whole.

During the period of formation of the SDS at the Ural-Sib Insurance Group CJSC, the following was considered as an indicator of motivation for distance learning: the completion of at least two training programs per year by 75% of the branch employees. Research in this area is complicated by the fact that quite often mandatory training courses for certain categories of personnel are organized in the form of distance learning. Individual characteristics (self-organization/disorganization, perseverance/hasty, ability for analytical thinking, sociability/isolation, etc.) on attitude towards this species training and the quality of its results certainly influence, but taking them into account requires special attention.

It is advisable to analyze the effectiveness indicators of distance learning systematically. Combining them into a common concept should take into account the uniqueness of the processes of a particular company. An example of an indicator that integrates different groups factors can serve as an analysis of methodological and administrative work. The time spent on administration (registration of participants, sending information letters and notifications, generation of final protocols based on training results, technical support for users of the training portal, etc.), according to the experience of Ural-Sib Insurance Group CJSC, should not exceed a third of the period creation and modification of training courses. Otherwise, the quality of the LMS as a whole may gradually decline.

An essential role is played by assessing the application of knowledge and its impact on the results of activities. Let's consider the methods by which this assessment can be carried out.

Methods for assessing the effectiveness of LMS

Questionnaire

The traditional method of assessing the effectiveness of training is feedback from participants and customers. It can be obtained by surveying employees immediately after completing classes, as well as managers - on a regular basis (for example, once a quarter). Thus, according to P. Kirkpatrick’s model for assessing the effectiveness of adult training, the immediate reaction of students is recorded first (survey of employees) and then how much their behavior and attitude towards work has changed after training (analysis of managers’ opinions).

Participants in the questionnaires are asked to evaluate (on a 3-, 5-, 7- or 10-point scale) the clarity and visibility of the training materials, the relevance, practical value of the knowledge gained, the quality of the LMS organization, and also describe whether the course was interesting, how difficult or it was easy to go through it and give a general emotional assessment: “liked/disliked.” An established indicator can indicate the effectiveness of training, for example, an average score of at least 7 points (on a 10-point scale).

Questioning line managers is a more difficult task. The following factors hinder the receipt of quality feedback from them:

• lack of initiative of the manager in determining the need for training - the selection of employees is carried out according to their functions, and not based on the vision of the situation in the department by its head;
• the training period is determined centrally, and in connection with this, the manager does not have the freedom to plan the completion of courses by his subordinates, taking into account the current workload (both theirs and his own).

These two circumstances significantly reduce the desire to provide feedback and share your opinion about organized training. Nobody needs formal answers.

In the case where distance learning is carried out centrally for all employees of a certain category (for example, operational support specialists in branches), you can limit yourself to surveying direct participants. If the manager himself selects from the catalog of programs those that he considers necessary for his subordinates, then he is able to provide comprehensive information about the effectiveness of distance learning.

Assessment by questionnaire in the case of LMS has its own characteristics. Thus, participants and managers can give feedback at will, i.e. This is an optional procedure. Technically, it makes sense to organize it in the form of a survey tied to a specific course. It is important that the questionnaire is presented in its entirety, and not just one question at a time, since when answering, the employee simultaneously analyzes his own impression of the training program, and it is important for him to reflect it in its entirety.

Some organizations create separate forums where employees can discuss various issues regarding a specific training program, as well as receive additional information from the forum administrator and course instructor. The UralSib Insurance Group company does not have a forum, since its support requires significant time and human resources.
Another important feature of distance learning assessment using questionnaires is the need to respond to the feedback received.

The above situation also serves as an example of how work can be done to expand the “live” part of training courses in a direction that is interesting for employees. However, the key point is the immediate response to feedback, i.e. all the work done on its basis. This activates communication between participants in distance learning, thus increasing its interactivity, and hence its effectiveness.

Incident method

The experience of implementing LMS in the Ural-Sib Insurance Group company has shown that it is advisable to use the incident method to assess the effectiveness of distance learning. Its essence is inherited from the practice of engineering psychology: in cases where an atypical question arises in the work, a special analysis of the problem is carried out.

It includes: clarifying the circumstances of the emergence of difficulties, considering the actions of the participants, finding the optimal way out of the situation. The causes of the phenomenon and the solution found, as well as methods of correct behavior in similar situations, are recorded in writing, and then transmitted to all employees who are involved in maintaining the work of the LMS. Such analysis helps improve the quality of training courses and helps enrich teaching materials.

From time to time there is a need to clarify the wording and expand the content of training courses. For example, work in insurance places high demands on an employee of a sales department: knowledge of legal norms, the economic foundations of insurance, enterprise economics, taxation features, the basics of related industries, etc.

This approach allows you to record possible technical difficulties that arise when using the training portal and ways to eliminate them, as well as transfer current experience to colleagues. The compiled directory is convenient to use for both new and experienced administrators.

Example 1

The training participants made it clear that there were not enough practical examples in the training course. This was reflected in low scores on the corresponding scale in the feedback questionnaire. The methodologist analyzed the situation, specified with the help of a targeted survey what the “remark” was, and then added and corrected examples in the course. Having agreed on the changes made with the responsible employee of the business unit, I placed information about them in the “News” section on educational portal with thanks to those who provided critical feedback that improved the quality of the course.

Example 2

An employee had difficulties with one of the test questions of the product course, and he addressed it to the personnel training and development directorate. The methodologist clarified what the problem was, then turned to the materials and checked the correctness of the wording. If necessary, in such cases, he requests assistance from representatives of the business unit that is the developer of the insurance conditions. The changed test question was agreed not only with the specialists who participated in the creation of the training course, but also with the employee himself, who identified the inaccuracy. Then the methodologist recorded the essence of the incident and how it was resolved. The information was transmitted to colleagues in the form of a regular email.

Example

A training participant encountered a technical difficulty while taking the test. He called the administrator, but he was unable to find the cause of the problem and solve it. Then a distance learning methodologist was brought in. His task was to determine the nature of the technical problem and record the circumstances of its occurrence. The methodologist contacted an IT specialist, who, based on the results of the situation analysis, determined the criteria for identifying the problem, derived an algorithm for solving it, and broadcast it to all distance learning administrators.

Analysis of the dynamics of the company's financial indicators

A special place in the assessment of LMS is occupied by general business performance indicators.

Today, a popular indicator of the quality of training is “sales volume”. In insurance, it is considered as the volume of insurance premium in the context of corporate and retail insurance, by type of insurance, sales channels, nature of the contract (new/extended), etc.

It is necessary to remember the following. In evaluating training programs, what matters is the volume and dynamics of sales not in general, but only for those types of insurance services for which the training was conducted. Of course, in the case of both good and bad values, it cannot be said that they were the result of learning (or lack thereof). What matters is “synchronicity”: if the course is completed, there is a result, as well as vice versa. You should be wary if training does not lead to changes in work.

The experience of UralSib Insurance Group CJSC shows that it is possible to focus training on the required sales areas only during the period when the training program plan is drawn up following the approval of the product strategy and sales channel development strategy, repeating the identified business priorities.

The “sales volume” indicator characterizes the performance, first of all, of employees of selling departments, and therefore is not the only one. For example, when training to work with databases, which is also convenient to run in a mass distance format, the effectiveness of the course will be indicated by statistics and analytical information on the number and nature of errors made when working with programs. Comparison of data collected before and 1-2 months after training will demonstrate the impact of the courses taken.

The training portal allows you to involve staff in distance learning. To increase the activity of participants, you can use: “performance” ratings of branches, divisions or personal ones; public virtual awards - “for organization” during training, “for the highest qualifications of a branch/unit”, “for active participation”, etc. Thus, measures of involvement can be: the number of participants in a survey, voting, competition; dynamic indicators of participation in training (meeting deadlines, the share of participants from the total number of personnel, an increase in the number of one-time applications for courses submitted by line managers).

Comparison method

In its work, the company's personnel training and development directorate also uses the comparison method. To assess whether the LMS is effectively organized, it is advisable to see how others do it, for example, by visiting a well-known e-learning exhibition and conference or at open seminars and round tables organized by enterprises that develop and sell software for distance learning. These same companies themselves maintain special Internet resources for their clients, where they can ask the provider questions, get answers, and see what specialists from other organizations are interested in.

The complexity of the task of determining the effectiveness of distance learning is undeniable. Due to active development information technologies one can be optimistic about the prospects for implementing different assessment methods, subject to minimal investment of time and human resources. Let us once again draw attention to the importance of systemic (general) quality indicators of LMS. The success of distance learning in general significantly depends on the methodologists involved in the organization of training. It is their task to establish dialogue with participants and increase the impact of distance programs.

Currently, distance learning technologies have penetrated almost all sectors of education (schools, universities, corporations, etc.). Thousands of companies and universities spend a significant portion of their resources on such projects. Why are they doing that? What are the returns (direct and indirect)? What is the return on this investment? These and many other similar questions are asked today by many leaders and participants in educational projects. To answer them, it is necessary to evaluate such an essential parameter as the effectiveness of these projects.
There are different approaches to assessing the effectiveness of various training systems. Let us first dwell on those of them that allow us to evaluate the relative effectiveness based on the analysis of learning technologies. Thus, we can highlight the following main processes that determine the effectiveness of the educational system under study as a whole: methodology for forming a knowledge system, methodology for forming a system of professional skills, profitability educational process and a smart tuition pricing strategy.
Let us formulate some rules that allow us to move from methodological statements to more specific criteria for assessing the comparative effectiveness of training systems. type="1"> A teaching system is more effective compared to another if the student’s workload during the period under consideration (semester, year) in this system will be more equal
number than in the compared one. This statement is based on the fact that in most cases it is impossible to form a knowledge system with an uneven teaching load (for example, only during session classes). A training system is more effective compared to another if the ratio of the number of hours of classes that form skills and professional skills to the total number of classes is greater than the similar ratio in the compared system. In other words, in order to develop professional skills, it is necessary to conduct a sufficient number of practical classes, trainings, business games and other classes in which complex professional skills are developed. Taking into account the introduction of interactive network tasks in distance learning technologies, the ratio of practical classes to the total number of hours becomes quite high and approaches the indicators of full-time education. The peculiarities of organizing and conducting distance learning do not allow increasing the value of this indicator to an acceptable level. On average, the ratio of the number of hours of practical training to the total number of hours allocated to studying the discipline for various forms of training is distributed as follows: full-time - up to 40%, part-time - up to 10%, distance learning - up to 30%. The training system must be cost-effective with a reasonable pricing policy.
Thus, according to the criteria for the formation of knowledge, abilities and skills, as well as due to a more even distribution of the student’s academic load throughout the year (Fig. 2.1), technology

The technology of distance learning is comparable to full-time education and is significantly superior in terms of quality parameters correspondence form training.
The fairly high efficiency of distance learning is largely achieved thanks to high degree its interactivity, specific features of information and telecommunication technologies, individual planning and organization of the educational process.
This is confirmed by an experiment conducted by a professor of sociology at the University of California state university J. Schutte. One semester, the professor randomly divided a group of students into two subgroups. Students from one of them attended classes as usual: listened to lectures, did homework and took tests, while students from the other took a similar course remotely. The latter received assignments and grades via the Internet, took the necessary information from the university website, asked questions and sent completed tests to the professor by e-mail, and also participated in group discussions using the Internet. They appeared in the classroom only to take the exam for the semester. And what's interesting! “Online” students demonstrated results that were on average 20% higher than the results of their classmates from the regular full-time group. Of course, in this case, the “network” students had higher motivation and it is not worth drawing general conclusions based on one experiment. But the specific result speaks volumes. By the way, the relative anonymity of participating in discussions on the Internet allowed many students to ask questions that they would never ask in a real classroom in the presence of their friends.
Beginning in the 1960s, training professionals began to understand the importance of quantifying the economic impact of educational projects, especially in corporate training. There are four main reasons for assessing the effectiveness of training in companies: training is a business tool, one of the projects implemented by an organization to increase its productivity and profitability; justification of the expediency of training costs necessitates its quantitative analysis; Evaluating effectiveness helps developers improve training materials because, without formal evaluation, the basis for change is extremely subjective. Performance evaluation helps to choose the best teaching method.
The effectiveness of corporate distance learning is determined by a combination of five key factors that allow
trainees retain more information in memory, increase their awareness, achieve better work results and thereby increase the rate of return on invested capital. These factors include interactivity, memorability, flexibility of use, assistance, and accessibility.
Interactivity. The introduction of interactivity into the learning process makes the student’s participation more active and forces him to try to achieve the maximum result. Interactivity also helps instructors incorporate more complex content into courses.
Interactivity can be combined with simulation during the learning process of the environment that students must become familiar with. For example, if the course is about using a computer program, the image on the screen should correspond to what is displayed when the program is running, and the learner can be assigned to perform an operation corresponding to one of the topics being studied. For more complex topics in this regard, you can model a business process and invite the learner to study it.
Memorability. To remember better, students must feel the importance of the material being studied. Structuring the headings of the topics being studied also helps with this, ensuring that the necessary information is retained in memory. This, in turn, makes it more likely that they will apply the acquired knowledge in future real-life projects. Additionally, organizing headings helps avoid repetition, which often reduces the effectiveness of eLearning.
Flexibility in use. The system should provide training for employees with different levels of training and capabilities. It is necessary that students can easily move along the training course, monitor their movement, and also be able to return to the position where they were during the previous session of accessing the training course.
As a company grows, course content rarely remains the same, so learning tools must allow for changes to the learning content. The possibility of such changes must be built into teaching tools at the very beginning. The change itself must be planned and its implementation monitored.
Providing assistance. Since e-learning usually takes place individually, it is necessary that the learning system provides assistance to learners. They need instructions on how to complete the course, navigation aids, tips for completing assignments, links to obtain information.
divisions, support when technical questions arise, etc. The help button should be accessible from any slide of the course. All this will help employees focus on learning and not be distracted by annoying interruptions. To provide flexibility for trainees, it is useful to provide a section for frequently asked questions, as well as a glossary of terms that may be unfamiliar to trainees. These tools can become valuable reference resources for employees even after training is completed.
Availability. Due to the workload of students, it is often difficult to choose a time for the learning process. To solve this problem, it is necessary to use various methods of delivering educational content. If training material is available via the Internet, then it is necessary to ensure that these materials can be used outside of work. For those businesses that have difficulty accessing the Internet, for students who want to study the material at home, and for technical staff who need reference material offsite, consider creating a CD-ROM version of the training material.
To improve economic efficiency distance courses both in general and in corporate training, a marketing approach to serving trainees is of great importance in order to ensure that the results of educational projects are as consumer-oriented as possible. Marketing mix tools should take into account not only market requirements, but also ways to actively influence it. The result of the activities of the marketing service is manifested in the organization of optimal recruitment of students and in the employment of trained specialists. At the same time, it is extremely important that marketing is an ongoing process with the mandatory participation of all universities included in the system open education.
The first, most well-known and currently practiced methodology for assessing the effectiveness of training programs was developed by Donald Kirkpatrick more than 40 years ago. According to this methodology, the assessment is made at four levels.
Level 1. Students' reaction to the training program. At this level, the participant's satisfaction with the program and the course of training is assessed. This is the easiest level to implement. Evaluation technology may include personal interviews, students filling out anonymous questionnaires, etc. Traditionally, course organizers are interested in whether students liked the program, the teacher, and the organization of the learning process. This is mainly typical for corporate training and training centers, where all programs are assessed at this level, but increasingly these technologies are also used in universities when surveying
students. In addition, for course evaluation, it is interesting to obtain similar data from those who did not take the training.
However, despite the prevalence, assessment results at this level are subjective, they only indicate whether the students liked the program, but do not provide any information about its success both for the student and for the company where he works, as well as about the quality of the knowledge and experience gained .
Level 2. Assessment of knowledge and experience gained by the student in the training program. Assessment technology at this level involves conducting an interview (exam), survey, testing after the learning process. This level is universally implemented in educational institutions and is presented in the form of a system of tests (exams). The same methods are used in 60% of cases in corporate training. Thus, at level 2, learning outcomes are assessed: it is determined whether the students received the knowledge and skills that were included in the program.
Level 3: Assessment of workplace behavior. If, based on the results of level 2 based on the final tests, an assessment of the level of knowledge was obtained, then it remains open question about the extent to which new knowledge and skills are retained in memory and used in the workplace. Level 3 is used to answer this question, i.e., it turns out how students apply the acquired knowledge and skills in work conditions. This is not practiced in educational institutions, and only 30% of corporate training programs are assessed at this level. The difficulty of the assessment is that to get results you need to wait from a month to six months. However, despite the complexity and cost of evaluating a training project at Level 3, this is where the real benefit of the training program to the employee first becomes apparent.
Level 4. Assessing the impact of the training program on the company's business. At this level, the impact of the training program on the company's business results is assessed. In other words, what measurable results the company has achieved through its corporate training programs. This is the most difficult level to implement, and only 10% of staff training programs are assessed on this parameter. The financial and other statements of the company are studied, management, clients, and other market participants are interviewed. But the main difficulty is to isolate the effect of training from a huge number of other factors that also affect the company’s business.
Kirkpatrick's model only provides a framework for evaluating performance, but does not provide any method or solution. Therefore, for further analysis it is necessary to use the following additional data: efficiency of use
funds; effectiveness of the learning process; compliance with training requirements.
In the mid-1990s, a model that took into account the return on investment ROI (Return on Investment),%, became especially popular in business:
ROI = Revenue - Costs 1QQ Costs
The use of ROI is changing the role of training and development in an organization. This process focuses on meeting business requirements, improves the design, development and delivery of training material, and adds value to learning and development within the organization. The main difficulty is expressed in forecasting future income in cash. Therefore, only 5% of training programs are evaluated using this model.
In addition to calculating the ROI coefficient, another universal method for evaluating projects is the analysis of benefits and costs of CBA (Cost Benefit Analysis). The CBA method is a fairly common approach in which an attempt is made to value in monetary terms every element of the project that leads to profit or cost, even if this element does not have a direct monetary value. The peculiarity of IAS is the assessment of alternative project options based on expected income.
Today, the Balanced Scorecard model is one of the most promising and used models, proposed in 1992 by Robert Kaplan and David Norton. Today, this model is implemented in most companies from the Fortune 500 list. The peculiarity of this model is that it takes into account the entire range of strategic and tactical goals of the company, which are not always of a financial nature. For this purpose, 20 - 25 indicators are identified that correspond to the company's strategy and are classified into four groups: financial indicators; customer service indicators; indicators characterizing internal business processes; training and development indicators.
The implementation of this model includes the following stages: a set of key success factors is formed; key performance indicators are determined; Based on the strategy and the current state of affairs, current goals are formulated.
One of the detailed models for evaluating training programs was proposed by Jack Phillips in 1997. This model included many features of previously developed generally accepted
models. The author collected and described more than 40 training projects in various organizations, identified the advantages of the evaluation methods used in each case, and developed a six-step “ROI model”.
Step 1. Data collection. Immediately before collecting data, it is necessary to identify the purposes for which the assessment is being made, then data is collected using various tools (surveys, interviews, testing, observations, performance measurement). Assessment is carried out at several stages: before, after, during training, and also after some time.
Step 2. Isolating the learning effect. Completing this stage allows you to exclude the influence of external factors that are not directly related to the training project. To date it has been developed big number solutions that allow this to be accomplished, all of them are not flawless, but when applied comprehensively they produce an adequate result.
Step 3. Convert data into monetary form. It is necessary to move from concepts such as increased quality, efficient allocation of resources, reduction of downtime and labor costs, to a monetary amount distributed over time intervals.
Step 4. Estimate training costs. It is necessary to evaluate not only explicit costs, such as payment for the developers of the training course, purchase of the necessary software and hardware, training materials, payment of the teacher (tutor), but also implicit ones, for example, absence from the workplace during the training period, etc.
Step 5. ROI calculation. To implement it, the results of the third and fourth steps are used.
Step 6. Determine implicit benefits. Implicit benefits can be considered an increase in “team spirit”, job satisfaction, reduction in conflicts, etc.
It should be noted that the simplicity of calculating the ROI indicator becomes unobvious in the case of evaluating a real project in the education system. The fact is that income (or training results) is very difficult to assess; they are often hidden, implicit in nature and can only appear a significant period of time after completion of training. And even if the income is quite obvious, the problem arises when converting it into monetary terms. Costs are also often difficult to estimate. Although there are many items that are reported in monetary terms, many costs are hidden and exist indirectly (for example, accounting for additional teacher time when answering student questions via email). In addition, the data needed to calculate ROI can often only be obtained after training.

If complete and reliable information necessary for calculating ROI is not available, you can use the simplified approach proposed by Jeff Moonen to assessing the return on investment SROI (Simplified Return on Investment), which takes into account various criteria for training projects (economic, quality and efficiency).
In order to reduce the complexity of data collection and calculations themselves, when calculating SROI, only those factors (both positive and negative) that in the current situation differ significantly from the average values ​​are taken into account. The result is a kind of “gain” or “loss” in the ROI value when the situation changes. However, the main and most important task is to select these “substantially different” factors from the point of view of the different participants in the situation.
The principles of calculating SROI are presented in six stages. Use relative data when necessary. ROI can be calculated as a relative value when absolute data is not available. In this case, the amount by which the ROI will change when moving from one situation to another is calculated (for example, when putting into operation new technology). The advantage of relative ROI calculations is that it eliminates the need to collect hard-to-find data used in absolute calculations. Expressing qualitative data on a numerical scale. There are many ways to categorize factors for use in calculating ROI. To simplify and expand this procedure, it is proposed to consider only three categories: economic criteria (directly related to the economy, expressed in quantitative parameters, usually in monetary units); qualitative criteria (abilities, behavioral criteria, quality of service, usually expressed in terms of “better” and “worse”); performance criteria (expressed in terms of the time required to complete a particular task, usually in terms of “more” and “less”). Presenting the SROI calculation as a non-financial calculation. Often, in a traditional ROI calculation, data originally measured in non-financial units is converted in some way to reflect monetary values. Unfortunately, such data transformation is rarely objective and deep, since many criteria (such as quality and efficiency) that have big influence to the value of the ROI coefficient. Therefore, when calculating SROI, the data is not converted into financial form. Instead, financial data is converted into relative scores (eg, on a scale of -10 to +10). This approach emphasizes the subjective nature of ROI, which is much closer to reality.
Taking into account the minimum number of variables that change the current situation. For the calculation, only those aspects of a specific situation should be considered for which the expected difference before and after the necessary changes will be not just noticeable, but significant. Calculation of different ROI values ​​depending on the participants in the situation and the stage of innovation implementation. When calculating ROI using a different set of participants in a situation, different results are often obtained (for example, from the point of view of the manager and the teacher). Therefore, it is necessary to carry out ROI calculations for each of the categories of participants in the process: staff, teachers, students, etc. Using SROI results to improve the decision-making process and methodology, but not the decisions themselves. By compiling a short list of ROI results, you can get much more complete information about a specific innovation. However, the most important contribution of SROI is the systematic analysis, the generation of food for thought, and the creation of a basis for discussion among stakeholders. Therefore, the application of SROI does not provide an accurate description of the effect (although all data are based on reliable and accurate data), but provides a guide for making informed decisions.
Such new approach(“simplified ROI”), taking into account traditional economic and financial parameters, also reveals implicit results related to the quality of the project and its effectiveness (competence of teachers, work schedule and quality of service). Calculation of “simplified ROI” leads to a more accurate understanding of the return on investment when planning the main indicators of each specific educational project. This is precisely its main purpose.