Objectivity of assessments is ensured by:
Independence of the certification body and the experts involved by it from the applicant or other parties interested in the results of assessment and certification;
The completeness of the composition of the commission of experts. Together, the commission must have knowledge of quality system standards, inspection techniques, as well as the features of production and regulatory requirements for it. The commission must include a specialist in the type of economic activity being checked. If necessary, the commission may include specialists in metrology, economics, etc.;
The competence of the experts conducting the certification. Experts must be certified for the right to conduct quality system certification or production certification and are registered in the Register of Experts of the GOST R Certification System.
Reproducibility of the results of inspections and assessments of quality systems
The reproducibility of the results of inspections and assessments is ensured by:
Application during inspections and assessments of quality systems (production) rules and procedures based on uniform requirements;
Conducting audits and assessments based on evidence;
Documentation of the results of inspections and assessments of quality systems;
A clear organization of the system of accounting and storage of documentation by the certification body.
Confidentiality
The certification body, its experts and all specialists involved in the work of the commission (including trainees) must keep confidential all information about organizations obtained at all stages of certification, as well as conclusions characterizing the state of the quality system (production). Conditions for compliance with the confidentiality of information are provided:
For the staff of the certification body - establishing confidentiality requirements in job descriptions;
Confidentiality of orders of the head of the body;
For personnel involved in certification work, by establishing confidentiality requirements in labor contracts concluded between the certification body and involved specialists;
For interns - by establishing confidentiality requirements under the conditions of the audited organization.
informative
The Register shall ensure quarterly publication of official information on certified quality systems (production facilities) of certificate holders. In addition, operational sources of information (periodicals of Rosstandart of Russia and its institutions) should publish current information on issued certificates of conformity of quality systems and production, suspension or cancellation of their validity.
The official source of information on the above issues is the consolidated list of certified quality systems and production facilities of the Register.
Specialization of bodies for certification of quality systems
Certification bodies should be specialized in areas of accreditation in accordance with the classification by type of economic activity adopted in the GOST R Certification System. ) for certification of quality systems, for certification of production, for certification of products, for certification of services, as well as technical experts (own and / or attracted), specialized in the relevant types of economic activity.
Verification of compliance with the requirements for products (services) in the legally regulated area
Subject to the presentation of mandatory requirements for products (services) established in accordance with the current legislation of the Russian Federation in state standards or other documents during certification of quality systems (certification of production), the ability of the organization to ensure compliance with these requirements is checked.
Reliability of evidence on the part of the applicant of compliance of the quality system (production) with regulatory requirements
When certifying quality systems, the certification body evaluates the reliability of the applicant's evidence on the fulfillment of the requirements of GOST R ISO 9001.
The high quality of the work of the experts participating in the survey has a decisive influence on obtaining accurate and reliable results.
The quality of experts is determined by the following groups of properties: competence, interest in the results of the examination, efficiency and objectivity (impartiality).
Expert Competence should apply to the object of expertise (professional competence) and assessment methodology (qualimetric competence).
Professional Competence includes knowledge of the technological stages of production, the value of quality indicators of its various types, prospects for the development of products. consumer requirements, conditions and nature of consumption.
Qualimetric competence provides a clear understanding by the expert of the approach to assessing the quality of products as a measure of meeting the needs of consumers; knowledge of quality assessment methods, especially expert methods; the ability to use various qualimetric scales that differ in a large number of gradations.
Interest expert in the results of the examination depends on many factors: the degree of workload of the main work, with which, as a rule, the examination is combined; the possibility of using the results obtained; the purposes of the examination; the nature of the conclusions that can be drawn from the results of the examination; individual characteristics of the expert.
efficiency the expert includes composure, mobility and elasticity of attention, allowing you to quickly switch from assessing one quality indicator to assessing another; contact, i.e. the ability to work with people in solving problems in a conflict situation; the motivation of the assessments made.
Objectivity (impartiality)- the ability to take into account only the information that determines the satisfaction of the needs of this product. The bias of the expert consists in overestimating or underestimating the object of examination for reasons that are not related to quality, for example, due to the inability to resist the opinion of the majority of other experts due to lack of confidence in their rightness (manifestation of conformism).
Methods for assessing the quality of experts are divided into five groups:
heuristic (subjective);
· statistical;
test;
documentaries;
combined .
Each of these groups is divided into types (private estimates) and methods for obtaining them.
Heuristic (subjective) estimates – assessments assigned by a person and based on the assumption that the quality of an expert is correctly reflected through the assessment of others or self-assessment.
There are the following types of heuristic estimates:
· self-esteem;
mutual evaluation;
assessment of argumentation and familiarity with the object of examination;
Evaluation of an expert by a working group.
Self-esteem - type and method of heuristic assessment of competence by the expert himself. It has been established that the reliability of the average expert assessment is the higher, the greater the average self-assessment of group members. At the same time, it should be borne in mind that self-assessment is characterized by significant subjectivity, and this entails certain disadvantages of it. Self-esteem depends on the psychological characteristics of experts (the presence of high or low self-esteem, sometimes unreasonably), the degree of self-satisfaction, and understanding of the rating scale. This explains the discrepancy between the results of self-assessment and mutual assessment.
To reduce subjectivity, self-assessment is carried out differentially, which increases the accuracy of the results. To do this, the self-esteem indicator is determined as a function of two coefficients: familiarity and reasoning. For expert evaluation of goods, it is recommended that the self-esteem of the expert be determined taking into account his knowledge and familiarity with the product being evaluated by filling out the "Self-Assessment Questionnaire". The expert notes the regularity of reading the sources of information listed in the questionnaire and the degree of familiarity with the products being evaluated.
Self-esteem ( K ca m j) is calculated by the formula
K ca m j= ΣM i K ij ,
where M i- the weight of indicators of awareness and acquaintance; K ij- assessment, depending on the degree of awareness and familiarity.
Since ΣM i= 1.00 and 0 ≤ K ij≤ 10, then 0 ≤ K ca m j ≤ 10.
Taking into account the specifics of the evaluated products, the weight of individual indicators (M i) can be corrected by the expert group.
Mutual evaluation- the type and method of the heuristic assessment, defined as the average of the assessments assigned by other experts. This assessment is intended to reduce the subjectivity of the assessment of the competence of each expert. It has been established that there is a close relationship between the competence of an expert and his average rating received from colleagues. The essence of this type of assessment is that each expert gives an assessment to all other experts, and then the average result is calculated.
Depending on the number of experts in the group, two different procedures are applied:
· if the number of experts in the group is less than 15 people, each expert evaluates all the others;
· if the number of experts is 15 people and more, a special questionnaire is filled out, in which the experts are divided into three groups by qualification – above average, average and below average qualification, as well as by ranks within subgroups (6 – 8 people each). Then the experts of each subgroup are ranked. Rank 1 – the most qualified expert, 2 – next in qualification, etc. Each evaluated expert is assigned a numerical score from 10 points (most qualified) to 0 (completely unqualified) with an accuracy of 0.5 points.
The disadvantages of peer review are:
Experts may not know each other well enough;
• mutual sympathies or dislikes can influence the results of the assessment;
· Experts, as a rule, tend to avoid super-high and super-low ratings;
· ambiguous perception of the concept of "quality of an expert".
· use peer-assessment only in those groups in which the majority of experts know each other well. If some of the assessed experts are unfamiliar, it is recommended to put a dash in the corresponding column;
conduct anonymous surveys;
· explain that the results of the questionnaire will be used only to adjust the ratings of the product;
· to acquaint experts with the structural scheme of properties and their partial assessments.
Evaluation of argumentation and familiarity with the object of expertise - type and method of heuristic assessment of the expert's degree of specialization and factors influencing his competence. Heuristic assessment of competence, based on the degree of specialization and familiarity, is highly effective and reliable, provided that methods for quantifying each of these factors in the form of assessment questionnaires are carefully developed. In this case, only those factors are evaluated that can only be characterized subjectively in the form of self-assessment; the expert quantifies the degree of his specialization in this type of product and the form of acquaintance with it. Then, using a special questionnaire (similar to a self-assessment questionnaire), he determines the coefficient of argumentation of his knowledge and assesses the degree of his familiarity with the problem under study. The competence coefficient is calculated as the arithmetic mean of the reasoning and familiarity coefficients.
Evaluation of the expert by the working group - type and method of heuristic evaluation designed to quantify the expert's interest in peer review and his attentiveness during the survey. The assessment is given by analysts who conduct a survey of experts. At the same time, they evaluate the attitude of experts to the ongoing examination and their activity in discussing assessments. It is expedient to make estimates of the working group on a 10-point scale.
Statistical estimates – assessments obtained as a result of processing the judgments of experts about the object of assessment.
These estimates are used to reduce, as far as possible, the errors that arise in expert estimates. Their necessity is due to the fact that, unlike measurements with the help of technical devices based on the comparison of unknown quantities with known ones, with expert methods there is often no known value (or sample, or standard of goods), the value of which is taken as valid, i.e., the maximum close to true.
The organization and methodology of the assessment may affect the accuracy of the result; the quality of building a block diagram of quality indicators, the number of quality indicators, methods for determining assessments, the nature of the relationship between experts, etc.
Thus, the accuracy of an expert assessment is influenced by many objective (depending on the assessment methodology) and subjective (depending on the expert) factors, resulting in assessment errors that have systematic and random components.
Systematic error - constantly repeatable part of the error. The main reason for its occurrence is insufficient or incorrect awareness of experts. It can be reduced by familiarizing the expert with the necessary information before the start of the examination or by conducting a briefing, as well as a discussion, during which the expert receives additional information from analysts or other experts. In addition, since the systematic error of an expert is random for a group of experts, then averaging the group's estimates can improve the accuracy of the quality assessment. The systematic error can be estimated by the degree of deviation from the average opinion of the expert group.
random error depends on the psychological and physiological characteristics of the expert (collection, confidence in the correctness, attentiveness, other personal qualities) and decreases with repeated repetitions of assessments. Its value can be determined by the reproducibility of the results.
There are the following types of statistical estimates:
Evaluation by deviation from the average opinion of the expert group;
assessment of the objectivity of the expert.
Assessment by deviation from the average opinion of the expert group– an estimate based on the premise that the actual value of the group's expert estimate is the mean of the group's estimate. Therefore, the smaller the deviation of an individual expert assessment from a group one, the higher the quality of the expert who gave this assessment is recognized.
Individual expert assessments can be of two types:
Ranking by the expert of the estimated values (in descending or ascending order);
· Determination by an expert of the numerical values of the estimated values (for example, when determining the weighting coefficients of quality indicators, experts assign certain numerical values to each coefficient).
An assessment of the deviation from the average opinion of the expert group can be expressed through a set of absolute deviations (D і) , calculated as the difference between the average group score ( X s.d) and individual assessment ( X i):
D i = X c .d - X i. .
The more D i , the higher the deviation of the individual opinion of the expert from the average opinion of the expert group.
Assessment of the objectivity (impartiality) of an expert - assessment of the expert's compliance with the principle of objectivity, his ability to impartially evaluate specific samples of goods. This very important characteristic of the quality of an expert significantly affects the accuracy of the results of a group expert assessment. Statistical methods for directly assessing the objectivity of experts have not been developed, therefore, in practice, they are content with an indirect assessment of the deviation from the average opinion of experts.
Test grades – evaluation by testing the psychological and physiological characteristics of experts. These assessments are intended to assess the objectivity, qualimetric and professional competence of an expert. The advantage of these assessments is the ability to assess the personal qualities of an expert, for which other methods and types of assessments are unacceptable. Their disadvantage is that the results obtained during testing cannot be compared with the data obtained by any objective method.
When conducting test assessments, the following requirements are imposed on the tests:
understanding by the tested expert of the statement of the test problem and the conditions that must be met by its solution;
the probability of randomly guessing the solution of the problem should be close to zero;
The test problem must have an exact solution;
· substantiation of the closeness of the test problem and those real problems that the expert has to solve when assessing the quality of products.
The types of test scores are:
assessment of reproducibility of results;
Qualimetric competence of the expert;
the objectivity of correcting their assessments.
Evaluation of the reproducibility of the result - assessment of the degree of closeness of the individual assessments of the expert, carried out at certain intervals of time. It is usually used in cases where the quality of experts is assessed in several rounds, while the assessments of the same author in different rounds are compared. Most often, the assessment of the reproducibility of results is used in the organoleptic analysis of food products.
It should be taken into account that the period of time between rounds should be small in order to exclude the influence of possible new information that changes his judgments, but at the same time, it should be sufficient for the expert to forget the assessment data in the previous round.
If the ranking of quality indicators is carried out, then the assessment of the reproducibility of the results is calculated by the formula
where – j-th expert; Rj– rank correlation coefficients for j-th expert, is determined by the formula
where dij- the difference between the ranks assigned i- mu quality indicator j-th expert in the first and second rounds of the survey; n– the number of ranking objects.
If the experts calculate the numerical values of the weight coefficients, then the assessment of the reproducibility of the results is calculated by the formula
where – reproducibility factor j-th expert; ρ j is the distance from the average opinion of the group for j-th expert, calculated by the formula
where M ij* and M ij ** – values i th weighting factor assigned j-th expert in the first and second rounds, respectively.
The coefficient of reproducibility of the results of an expert can vary in the range from 0 to 10. It is recommended to admit experts with a reproducibility of results of 6 or higher to the examination of product quality.
Assessment of the qualimetric competence of an expert - assessment of theoretical knowledge of quality assessment methods and the ability to apply them. The assessment of the theoretical training of an expert can be carried out by oral or written control of knowledge using tests in a particular area of knowledge. The situation is somewhat more complicated with skill checks, which are divided into three types:
the ability to use different types of rating scales;
the ability to determine subjective probabilities;
· the ability to distinguish a sufficient number of gradations of the evaluated property.
Ability to use different types of rating scales. A scale is an ordered series of marks corresponding to the ratio of successive values of the measured quantities. In qualimetry, the scale is a means of adequate comparison and determination of the numerical values of individual properties and qualities of various objects. In practice, the following types of qualimetric scales are used: the scale of names, the scale of order, the scale of intervals, the scale of ratios and the scale of absolute values.
Name scale used in cases where several unknown sizes need to be compared with one and determine which of them are equal to the size selected for the base for comparison, and which are not. According to the scale of names, sizes are classified on the basis of equivalence, identity, equality. This measurement is the simplest, but the least informative. In this case, it is not determined which of the unequal sizes is larger or smaller than the size taken as the base one, i.e. the order of increasing or decreasing sizes is not set. Measurement consists only in determining the similarity (equality) or difference (inequality) of one or another size from a predetermined value. Therefore, the defining relations between the measured sizes are as follows: equal or not equal, i.e. in symbols = or ≠.
The mathematical expression of the essence of measurements on the scale of names can be written as follows:
Р i = Р j or Р i ≠ Р j ,
where P i- the size with which they compare (base size); R j – j-th size of compared sizes ( j= 1,2,3, … , n); n– number of compared sizes.
When comparing and measuring sizes on a scale of names, for example, control and evaluation of the quality of something is carried out according to an alternative principle: good - not good; suitable - not suitable; corresponds - does not correspond, etc.
order scale- this is a sequential series of values that gives a systematic idea of the simplest ratios of the values of the compared sizes of properties, features or qualities of the evaluated objects as a whole.
When comparing all measured sizes in pairs, it is established which size is larger or smaller than the other, which is better or worse than the other. If there are identical sizes, then this ratio is also determined. Further, the established size ratios are ranked in ascending and/or descending (decreasing) order of their values. The values themselves remain undefined. The resulting range of values is an ascending or descending order scale.
On order scales, size values can be classified (estimated) not only according to the criterion “the same or not”, but also according to the ratio that “more or less” of another or “what is better and what is worse” of another.
The mathematical expression for the ratios of pairwise compared sizes is:
R i= P j or R i≠ P j or R i> P j or R i < P j.
As a result of comparing the dimensions of R i and R j determine which size is larger or smaller than the other, as well as which dimensions have the same values, i.e. the following ratios are determined on the order scale: equal (=), not equal (≠), greater than (>), less than (<).
The disadvantage of measurements on order scales is that the results obtained in the form of a ranked series are the least informative. In particular, with such a measurement, it is not possible to determine how much one size is larger or smaller than another, better or worse than another. However, the main advantage of measurements using order scales is that instrumentally unmeasurable quantities can still be estimated (measured) quantitatively using them. Measured on order scales include such properties of objects as taste, smell, attractiveness, aesthetics, comfort, etc. General expert assessments of the quality of several compared objects are often made on the order scale.
Interval scale is used in cases where it is not possible to measure the dimensions of the observed quantities themselves, but it is possible (or there is a need) to measure only the differences (differences) between the dimensions cognizable by comparison.
On the measuring scale of the intervals, the differences in the compared sizes are recorded. This form of displaying the measured values is more perfect, since there are conditional, but well-defined units of measurement on the interval scale, which makes it possible to quantitatively (numerically) characterize the ratio of the sizes under study.
The mathematical record of comparing two homogeneous sizes with each other by their difference has the form:
ΔР i,j= P i- R j .
On a scale of intervals, such ratios of sizes are determined as equal (=), not equal (≠), greater than (>), less than (<), сумма (+), разница (– ).
A classic example of measurement on a scale of intervals is the measurement of temperatures on the Celsius scale. According to the scale of intervals, it is impossible to determine how many times one size is larger or smaller than another.
Relationship scale- this is a measuring scale on which the numerical value of the quantity is counted (determined) qi as a mathematical ratio of the measured size P i to another known size, taken as a unit of measurement [P].
Any measurement on a ratio scale involves comparing an unknown size with a known one and expressing the first through the second in a multiple or fractional ratio.
The mathematical notation of the measurement on the scale of ratios is:
where i = 1, 2, 3, … , n is the number of the measured size.
The scale of ratios is a scale of intervals in which the zero element is defined - the reference point, as well as the size (scale) of the unit of measurement [Р].
According to the ratio scale, such values of measured dimensions are determined as: equal (=), not equal (≠), more (>), less (<), сумма (+), разница (– ), multiplication (×), division (:). Consequently, with the relative values of the measured dimensions, many logical and all arithmetic operations can be carried out.
The ratio scale is most suitable for measuring most quality indicators, especially for such numerical characteristics as the geometric dimensions of objects, their density, strength, stress, and others.
Scale of absolute values used in cases where the quantity is measured directly. For example, the number of defects in a product is directly counted. Number of production units, etc. With such measurements, the absolute quantitative values of the measured are marked on the measuring scale. Such a scale of absolute values has the same properties as the scale of ratios, with the only difference that the quantities indicated on this scale have absolute, not relative values.
The results of measurements on the scale of absolute values have the highest reliability, information content and sensitivity to inaccurate measurements.
The scales of intervals, ratios and absolute values are called metric, since measures are used in their construction, i.e. dimensions taken as units of measurement.
Characteristics, parameters or characteristics of the properties of objects, measured on a scale of names or on a scale of order, are not quantitative, but quality, i.e. indeterminate in their true magnitude and in the magnitude of the differences between them. The size determined by any of the metric scales is a quantitative value, and these scales themselves are quantitative.
Qualimetric scales, their defining relationships, the types of quality characteristics measured by them, as well as some examples of the measured are given in Table. 4.1.
Each of the qualimetric scales has its own meaning and its field of application, and therefore they are most often not interchangeable when solving a particular expert problem. The ability to determine subjective probabilities is the ability of an expert to assess the likelihood of certain events occurring. Experts who do not have this ability should not be involved in forecasting expertise.
Ability to determine subjective probabilities when making certain types of assessments. For example, when determining the weight coefficients of some events, an expert sometimes has to use the concept of the subjectivity of the probability of their occurrence. Often, special tests are used to assess the ability of a candidate for expert to accurately determine subjective probabilities.
The ability to distinguish a sufficient number of gradations of the property being evaluated- the ability of an expert to capture differences in the intensity of manifestation of each property. This ability is largely due to the sensitivity of the expert to the most insignificant changes in the properties and indicators of the objects of expertise. For example, during organoleptic analysis, testers are tested for the ability to detect differences in tastes, smells, color shades, etc. The accuracy of individual and group expert assessments also depends on the ability of an expert to distinguish between gradations of an assessed property, so tests to identify this ability should help select high quality professionals.
Table 4.1. Types of qualimetric scales
“We can set ourselves a goal of a new level and a different scale - to make the Russian school one of the best in the world” Vladimir Putin
The last meeting of the State Council in the outgoing year was devoted to a topic that invariably provokes heated discussions in society - the quality of school education. Russian President Vladimir Putin has formulated an ambitious task: in the next 10 years, the Russian education system should become one of the best in the world.
Much remains to be done to solve it: build and equip new schools, introduce advanced information technologies into the educational process, create a new system of motivation and professional growth of teachers. The foundation of all future transformations aimed at improving the quality of education should be the creation of a comprehensive and objective system for assessing students' knowledge at all levels of education. As the head of Rosobrnadzor Sergey Kravtsov explained, it is about stimulating the development of education and the formation of a single educational space through various evaluation procedures. “We need to build a full-fledged system, but for this we need results at all levels of education in order to identify and close problem areas,” said Sergey Kravtsov.
All-Russian test works, national studies of education quality assessment, studies of teachers' competence, and, of course, the results of an objective USE - all this together will make up a nationwide system for assessing the quality of education. “Its main function is not to punish, but to help develop both students and teachers. This applies not only to the activities of educational institutions, but also to the study of the professional competencies of teachers,” said the head of Rosobrnadzor. To conduct research at all levels of education and summarize their results, the department has already created the Federal Institute for Educational Quality Evaluation (FIOKO). The data he collects will be used to improve educational standards and improve the skills of teachers. In addition, the Institute will generalize the results of all-Russian verification work, which will be carried out by schools at the end of each academic year. This year, the mechanisms for conducting All-Russian verification work, National studies of the quality of education have been successfully tested. Now it is important that the conclusions from the results of these studies serve as a basis for developing a support system for weak schools, creating motivation for expanding the activities of institutes for advanced training of teachers.
“Despite the fact that we have already taken confident steps to ensure the objectivity of the system for assessing the quality of education, following the results of the State Council, we received additional guidelines for ourselves,” said Sergey Kravtsov.
According to him, the focus on objectivity will be a key task for Rosobrnadzor, and the main goal is to obtain real data on the quality and results of education in order to make further decisions to improve the education system. The Federal Service for Supervision in Education and Science is constantly developing various tools for an objective assessment of the quality of education and is taking steps to strengthen control over the objectivity of the final assessment of students.
So, the GIA in the 9th grade next year will be held under the selective control of public observers. This was recently stated by the head of the department, Sergei Kravtsov, speaking at the conference "National studies of the quality of education: results and prospects."
According to him, it is necessary to objectively conduct all examination procedures in schools, and not just the Unified State Examination, organized at the federal level.
“Often we see that exams without federal control are biased. When a student receives a good grade in GIA-9, without having the appropriate knowledge, and then goes to high school, he has problems passing the exam, ”the head of Rosobrnadzor noted. He added that the lack of proper control over exams in the ninth grade "disincentives students to study," and only an objective examination will force students to prepare for it responsibly.
The head of Rosobrnadzor recalled that in the current academic year, within the framework of GIA-9, schoolchildren will have to take not only the compulsory Russian language and mathematics, but also two elective subjects. Rosobrnadzor notes that studies of the quality of education indicate a deterioration in the performance of schoolchildren after the fifth grade in a number of regions. The most problematic area is the main school, grades 6-8, so the main focus should be placed on working with it and improving the skills of teachers in the main school.
Since 2014, Rosobrnadzor has been conducting National Education Quality Surveys (NIKO) to identify existing gaps in students' knowledge. Next year, such studies will be held in history, social science and foreign languages. Also, the department has begun work to study the professional competencies of teachers, since the certification that teachers pass is often carried out formally and does not reveal problem areas in their work. This academic year, Rosobrnadzor will assess the competencies of teachers of the Russian language, mathematics and literature. Sergey Kravtsov expressed hope that the All-Russian Testing Works (VPR), which were tested in December 2015, could become another tool for an objective assessment of schoolchildren's knowledge. Their results will be entered into the federal database and analyzed.
In the Tambov region, one and a half thousand fourth-graders from twelve schools took part in the approbation of the All-Russian test papers in mathematics and the Russian language. According to the information of the regional organizers, the VPR took place as usual.
The test procedure itself did not differ much from the usual control. During one lesson (45 minutes), the students completed tasks in mathematics, which are designed to test the ability to count and apply mathematical knowledge. Tasks in the Russian language were divided into two days. In the first part, the guys wrote a dictation. The second part of the work on Russian included tasks to test reading literacy, as well as tasks testing knowledge of the basics of the Russian literary language system.
In order to explain to the public the essence of the All-Russian screening work, the Department of Education and Science held the action “VPR is not the Unified State Examination”, in which journalists from almost all regional media, as well as parents of fourth-graders of gymnasium No. 12 named after. G.R. Derzhavin of the city of Tambov. On December 8, at the same time as schoolchildren, adults were able to test their knowledge. For the parents of gymnasium students, the staff of the institution prepared an open lesson in mathematics, the basis of which was the tasks included in the VPR. At the same time, the correspondents wrote a dictation and completed tasks in the Russian language. Parents were able to make sure that the VPR does not bear any additional burden on the children. At the end of the lesson, schoolchildren's parents and journalists were able to communicate with the regional organizers of the event. During the meeting, the director of the Tambov Center for Evaluation of Educational Activities, Oleg Meksichev, told the audience about the prospects for the introduction of VPR, as well as how the results of the work will affect the final progress of students. In particular, he emphasized that the VPR is not the USE, as many have argued. "Nothing like this. There is no exam in the 4th grade. In addition, this year the final assessment will not depend on the results of the VPR,” Oleg Meksichev noted. He explained what the All-Russian verification work is. According to him, the results of the VPR will first of all be required by the school to adjust the curriculum, for the individual work of the teacher with each child, for the work of the deputy director with each teacher. “Based on the results, we will be able to judge where they teach better, where they teach worse, where the curriculum needs to be adjusted. Based on these data, it will be possible to manage the quality of education,” said the head of the Center for Expertise in Educational Activities. VPR are designed to check how the federal state educational standard is being introduced at different levels of education.
The difference between VPR is that the work is carried out according to uniform tasks developed at the federal level and is checked according to uniform criteria. Thus, all children are on an equal footing. According to the organizers, in the future, All-Russian testing will be carried out at the end of each academic year, and they will be implemented in stages: in 2016 - for grades 4, in 2017 - for grades 4 and 5, and so on.
As for the development of an objective assessment of students' knowledge, one of the ways to improve the quality of university education, according to Sergey Kravtsov, should be the assessment of students' knowledge by employers.
current score is a procedure evaluation of individual progress in mastering the curriculum of the subject. The current assessment can be formative, i.e. supporting and guiding the efforts of the student, and diagnostic, contributing to the identification and awareness of the teacher and students of the existing problems in learning. The object of the current assessment is the thematic planned results, the development stages of which are fixed in the thematic planning. The current assessment uses the entire arsenal of forms and methods of verification (oral and written surveys, practical work, creative work, individual and group forms, self- and mutual assessment, reflection, progress sheets, etc.), taking into account the characteristics of the subject and the characteristics of the control and evaluation teacher's activities. The results of the current assessment are the basis for the individualization of the educational process; at the same time, individual results indicating the success of training and the achievement of thematic results in a shorter (compared to those planned by the teacher) terms can be included in the cumulative assessment system and serve as a basis, for example, to free the student from the need to perform thematic test work 11 .
Thematic Assessment is a procedure assessment of the level of achievement thematic planned results in the subject, which are recorded in educational methodological sets recommended by the Ministry of Education and Science of the Russian Federation. For subjects introduced by the educational organization independently, the thematic planned results are established by the educational organization itself. Thematic assessment can be conducted both during the study of the topic and at the end of its study. Evaluation procedures are selected so that they provide for the possibility of assessing the achievement of the entire set of planned results and each of them. The results of the thematic assessment are the basis for the correction of the educational process and its individualization.
Portfolio is a procedure estimatesdynamics of educational and creative activity student, orientation, breadth or selectivity of interests, the severity of manifestations of creative initiative, as well as levelhighest achievements shown to these students. The portfolio includes both the student's work (including photos, videos, etc.) and reviews of these works (for example, award lists, diplomas, certificates of participation, reviews, etc.). Selection of works and reviews for the portfolio conducted by the students themselves together with the class teacher and with the participation of the family. The inclusion of any materials in the portfolio without the consent of the student is not allowed. A portfolio in terms of a selection of documents is formed electronically during all the years of study at the basic school. The results presented in the portfolio are used in developing recommendations for choosing an individual educational trajectory at the level of secondary general education and can be reflected in the characteristic.
Intraschool monitoring is a procedure :
assessment of the level of achievement of subject and meta-subject results;
assessment of the level of achievement of that part of personal results which are associated with the assessment of behavior, diligence, as well as with the assessment of educational independence, readiness and ability to make an informed choice of the training profile;
assessing the level of professional skills of a teacher, carried out on the basis of administrative tests, analysis of the lessons attended, analysis of the quality of training tasks offered by the teacher to students.
The content and frequency of intra-school monitoring is established by the decision of the pedagogical council. The results of intra-school monitoring are the basis for recommendations both for the current correction of the educational process and its individualization, and for improving the teacher's qualifications. The results of intra-school monitoring in terms of assessing the level of students' achievements are summarized and reflected in their characteristics.
Intermediate certification is a procedure for assessing students at the level of basic general education and is carried out at the end of each quarter (or at the end of each trimester) and at the end of the academic year for each subject studied. Intermediate certification is carried out on the basis of the results of the accumulated assessment and the results of thematic verification work and is recorded in the document on education (diary).
An intermediate assessment, which fixes the achievement of subject planned results and universal educational activities at a level not lower than the basic one, is the basis for transferring to the next class and for admitting the student to the state final certification. During the period of the introduction of the Federal State Educational Standard, in the case of using standardized measuring materials, the criterion for achieving / mastering the educational material is set as completing at least 50% of the tasks of the basic level or receiving 50% of the maximum score for completing the tasks of the basic level. In the future, this criterion should be at least 65%.
The procedure for conducting intermediate certification is regulated by the Federal Law "On Education in the Russian Federation" (Article 58) and other regulations.
State final certification
In accordance with Article 59 of the Federal Law "On Education in the Russian Federation", the state final certification (hereinafter referred to as the GIA) is a mandatory procedure that completes the development of the main educational program of basic general education. The procedure for conducting the GIA is regulated by the Law and other regulations 12 .
The purpose of the GIA is to establish the level of educational achievements of graduates. GIA includes two mandatory exams (in Russian and mathematics). Students take exams in other academic subjects on a voluntary basis of their choice. The GIA is carried out in the form of the main state exam (OGE) using control measuring materials, which are sets of tasks in a standardized form and in the form of oral and written exams using topics, tickets and other forms at the decision of the educational organization (state final exam - GVE).
final grade(final certification) in the subject consists of the results of internal and external evaluation. To the results external evaluation include the results of the GIA. To the results internal evaluation include the subject results recorded in the cumulative assessment system and the results of the final work on the subject . This approach makes it possible to ensure the completeness of the coverage of the planned results and to identify the cumulative effect of training, which provides an increase in the depth of understanding of the material being studied and the freedom to operate with it. For subjects not submitted to the GIA, the final grade is based on the results of only the internal assessment.
The final grade in the subject is recorded in a document on the level of education of the state standard - a certificate of basic general education.
final grade for interdisciplinary programs is set on the basis of the results of intra-school monitoring and is recorded in the student's profile.
Characteristic prepared on the basis of:
objective indicators of educational achievements of a student at the level of basic education,
graduate portfolio;
expert assessments of the class teacher and teachers who taught this graduate at the level of basic general education.
Graduate profile:
the educational achievements of the student in the development of personal, meta-subject and subject results are noted;
pedagogical recommendations are given for the choice of an individual educational trajectory at the level of secondary general education, taking into account the choice of areas of profile education for students, identified problems and noted educational achievements.
1. General rules for conducting evaluation procedures.
1) Any procedures carried out within the AC should be aimed at the targeted collection of the widest possible diagnostic information, as opposed to using the same methods for counseling or training. All actions of the expert during the assessment should be aimed at collecting information about the current capabilities of the subject, and not at education, counseling or therapy. Of course, the very participation of the subject in the assessment program gives him the opportunity to get acquainted with new ways of organizing work, assess the level of his preparedness and the development of certain qualities, but this is rather an additional result.
2) In the process of evaluation, the working conditions of all subjects must be equalized, it is necessary to minimize the influence of side factors that may affect the result.
When conducting an assessment, it is necessary to be sure that all the differences in the manifestations shown by the subjects within the same program are associated with a different degree of severity of the qualities being assessed, and not with different working conditions.
3) In the evaluation process, it is essential to obtain reliable information about the behavior of the subject in a normal situation. Therefore, when carrying out evaluation procedures, it is important to try to reduce the “examination effect”, when the person being evaluated does not behave as in a real situation, but in accordance with the expectations of the expert.
The tests used in AC programs are aimed primarily at collecting preliminary information (building hypotheses) about the level of development of personal and business qualities, psychological characteristics that are essential for effective professional activity. This information can be clarified during special exercises, group interactions, interviews, role plays, etc.
1) Before each test or exercise, very precise and strict instructions must be given, which may include:
Description of the work order (stages of the exercise, rules for fixing the result, the composition of materials that must be submitted at the end of the exercise, etc.);
Rules of conduct during the exercise (time regulations, a list of tools that you can use, when and how you can ask questions, is it possible to interrupt work, etc.);
3) in the process of filling out the tests, it is advisable to observe how the participants work - this will provide an opportunity to collect additional information about the psychological and business qualities.
In AC programs, group exercises provide an opportunity to collect information that represents the actual behavior of a person in situations close to ordinary work situations. The information obtained, in comparison with the data of test procedures and written exercises, makes it possible to refine hypotheses about individual characteristics and specify the repertoire of behavioral means.
3. The general rules for conducting group exercises include the following:
1) the leader of group work (facilitator) - must build the work of the group in such a way as to give the participants the opportunity to manifest themselves as much as possible. Only when his intervention can help observers to better see the characteristics of the subjects, active actions are possible. It is appropriate to compare the work of a facilitator with the work of a magician's assistant, who is not visible, but without which none of the conceived tricks will work. He must always be ready to direct group work in a direction that would make it possible for each participant to show what he is capable of. To successfully fulfill his role, it is important for the facilitator to: gain the authority of the group and each participant, receive a kind of “vote of confidence” from the group, otherwise his actions may be blocked, leaving behind the possibility of flexible group work;
2) it is desirable that observers of group exercises be minimally involved in group work and do not try to independently influence the behavior of a particular participant or the group process as a whole, since such additional stimulation can significantly distort the picture of behavior.
It is expedient to finish the work of all participants of the Central Organ with a general meeting. The purpose of such a meeting is to sum up the results of the day, symbolically complete the work, relieve tension and get out of contact. As a rule, all AC participants (both subjects and experts) feel both mental and physical fatigue after a whole day of intensive work. After participating in some procedures, participants may feel dissatisfied. To relieve tension, it is necessary to answer all the questions that have arisen, to provide additional information, if it is requested. It is useful to talk about where this particular program has already been used, how many people participated in it, and once again recall the strict confidentiality of the information received. If possible, conduct a "feedback" procedure - the presentation of individual results, it is necessary to agree on the day, time and place of its implementation.
Analysis of the results and preparation of materials for transfer to the customer
The information obtained from the results can be very widely used in various personnel programs. Depending on the purpose of the AC, the way in which the results are presented may vary. In table. 4 shows the main ways of presenting the final information, depending on the purpose for which the assessment center was carried out.
Table 4
After conducting the AC program, the data on the subjects obtained during the various procedures must be analyzed, compared and turned into complete information. There are several stages of information passing and processing:
1) primary processing of test results, special and group exercises;
2) translation of the obtained results into indicators according to the evaluation criteria;
3) evaluation - conversion of indicators into scores according to criteria;
4) comparison of scores obtained by one criterion in different procedures, the formation of the final score and the preparation of individual score tables;
6) preparation of generalized materials for the group of assessed - ranked lists, personnel distribution maps.
The test results are assessed by psychodiagnostic specialists who prepare their own draft conclusion on the degree of severity of the assessed personal and business qualities. The same preliminary individual assessment is carried out by specialists processing the results of special exercises. To increase objectivity, it is advisable to involve several specialists at the same time and then compare the results.
The results of group exercises are initially summarized in groups of facilitators and observers who compare the results of behavioral diagnostics, clarify primary data, and formulate hypotheses about the degree of manifestation of the qualities being assessed.
The final discussion - the procedure for summarizing the results of the evaluation of various procedures - is usually carried out in the form of a group exchange of views. In the course of such a general discussion, it becomes possible to compare the manifestations of the same qualities by the assessed, shown in various procedures, to test the hypotheses formed by private experts, to come to a final decision on how the level of development of the qualities of the subject can be assessed, which may contribute or hinder him. effective work, make assumptions about the prospects for growth and preferred areas of activity of the employee.
An example of the structure of the final conclusion
1. General conclusion on a specific AC participant (on a five-point scale).
2. Description of the participant's weaknesses and strengths.
3. Proposals for the development of qualities and skills, as well as specific proposals in connection with promotion.
I. General conclusion:
(1.1) what the participant seemed to be before the assessment began
(1.3) what impression
(1.4) other data (age, abilities, inclinations, partial analysis of the results of previous work)
(1.5) characteristics of the qualities, abilities inherent or lacking in the participant as a leader
(1.6) characteristics of the participant in the process of performing individual exercises, analysis of behavior in terms of the qualities necessary for the leader:
Leadership Ability
Ability to communicate with people relationships in a group
Communication style in free time (during breaks) handwriting
2. Description of strengths and weaknesses of the AC participant
(2.1) enumeration
(2.2) general observations:
general impression
oratory
commercial ability
sensitivity
purposefulness
capacity for critical judgment
ability to plan and organize...
3. Proposals based on the results of the AC on the possibility of promotion:
(3.1) suggestions on what the participant should do personally to overcome their shortcomings
Information about personality traits can help in announcing AC results. This is also important because in nine out of ten cases, the results of the assessment are announced by the immediate supervisor.
The experience of conducting programs of personnel assessment centers shows that if the work ends with the preparation of individual conclusions, then the information received is used only partially. At the same time, in a number of cases, it becomes possible, using the results obtained, to conduct a special analysis of the personnel state of the organization - a personnel audit.
The concept of "personnel audit" - new in the practice of personnel management - includes the following elements:
Assessment of the level of professional security of the organization - the quantity and quality of specialists working at various levels and directions;
Assessment of the organization's readiness for change;
Identification of types of organizational culture existing in the organization;
Building a forecast for the development of the organization and assessing the degree of feasibility of the goals set;
1. The essence of AC technology is to observe the candidate in model situations typical for the activity performed and identify the presence or absence of the qualities necessary for successful work, give a description of his features and formulate training tasks.
2. The requirements for the evaluation technology are that the personnel be evaluated objectively, with a high degree of reliability, reliability, complexity and accessibility, while it is necessary to ensure the possibility of forecasting and integrating into the overall system of personnel work in the organization.
3. The main principles underlying the AC technology: 1) modeling of key moments of activity, 2) development of a system of evaluation criteria, 3) complementarity of the techniques and exercises used, 4) inclusion of specially trained observers from among the employees of the organization in the evaluation process, 5) evaluation real behavior, not hypotheses about its causes.
4. In the methodology of the AC, three theoretical approaches to the study of human manifestations have merged: psychometry, socio-psychological and anthropological principles for describing behavior, and clinical observation.
5. The main methods used in the evaluation centers are: 1) special exercises, 2) interviews, 3) group exercises, 4) psychological tests and 5) organizational and managerial games.
Bibliography
For the preparation of this work, materials from the site http://www.i-u.ru/
Ratings; 3) designing evaluation procedures; 4) training of experts (observer-appraisers); 5) implementation of the AC program; 6) analysis of the results and preparation of materials for transfer to the customer. 2. FEATURES OF THE STAFF EVALUATION CENTER IN THE SOCIAL SPHERE The specificity of work in the social sphere lies in the fact that it is closely connected with communication. Often the success of the work itself depends on ...
From 1 to 5 days (depending on the number of working experts and the number of assessed), processing the results and preparing reports (for 30 people) - about a month. The main methods used in the evaluation centers Implementation of managerial actions. For the two hours allotted to complete the task, the subject must familiarize himself with some instructions, business papers, orders and other materials ...
