Validity - compliance case study accepted standards (perfect experiment).
Validity external - compliance of a particular study with natural reality and / or other similar studies. Determines the possibility of transferring and / or generalizing the results to other objects and research conditions. Depends on the representativeness of the sample and the correspondence of additional variables controlled in the study, their variability in other conditions. A particular form of external validity is environmental validity, which determines the ability to extend the conclusions of a particular study to real conditions, and not to other laboratory conditions.
Validity is internal - the correspondence of a particular study to an ideal one; evaluates the change in the dependent variable, is determined by the influence of the independent variable, and not by other causes. Internal validity depends on the systematic change in the influence of the independent and other variables on non-equivalence and the change in the compared groups over the course of the experiment.
Construct validity - characterizes the accuracy of the implementation of the theoretical hypothesis in the experimental procedure. It is one of the manifestations of internal validity. Determines the area of phenomena studied in the experiment. In psychological diagnostics, construct validity characterizes the degree of a measured property in the test results.
Validity criteria - reflects the compliance of the diagnosis and prognosis obtained on the basis of testing data, activity and vital signs; includes current and predictive validity.
Operational validity - compliance of the experimenter's operations with the theoretical description of the variables controlled in the study. The conditions varied by the experimenter should correspond to the independent variable. Operational validity is one of the manifestations of internal validity.
35. Give a classification of experimental plans: pre-experimental plans, plans of true experiments and quasi-experimental plans.
Pre-Experimental Plans
Quasi-experimental plans are built on the basis of this definition of the concept of quasi-experiment: a quasi-experiment is a study aimed at establishing a causal relationship between two variables, in which, however, there is no procedure for preliminary alignment of groups, that is, parallel control with the participation of the control group is replaced by a comparison of the results group testing before and after measurements. There are several types of quasi-experimental designs, for example: a design for two non-equivalent groups with testing before and after testing; plan for previous final testing two randomized groups; discrete time series plans for one group or for two non-equivalent groups. The quasi-experiment is very common in modern psychology and is no worse than true. For some tasks psychological research they are necessary and, despite the absence of a preliminary group alignment procedure, they special tool solving the problems of experimentation, if the independent variable is not the object of the researcher's manipulation, but is chosen by him by selecting non-equivalent groups. Let's say if the purpose of the experiment is to check how the level affects personal anxiety students on the success of passing a responsible final exam, groups are specially formed equivalent to pre-measured high and low levels anxiety and its influence on the dependent variable - the success of tests. The test results are dependent on this personal characteristics subjects, which is selected to create non-equivalent groups. Such a quasi-experiment is in no way inferior to the real one. Let us consider several examples of a quasi-experimental design. For example, in an experiment with nonequivalent control group the results of non-equivalent groups before and after the test are compared. This makes it possible to control individual threats to validity, and in experiments with such plans, changes in indicators are compared - pre-experimental and post-experimental testing in non-equivalent groups. This difference between scores is then evaluated using Student's t-test to determine the level of significance of disagreement. Let's give an example of an experiment from D. Martin's work "Psychological Experiments". The purpose of the experiment was to investigate whether the productivity of workers increases machine-building plant when paying "from production". The independent variable is the method of payment, the dependent variable is labor productivity. Two groups of factory workers participated in the experiment. Their labor productivity was preliminarily measured, after which for one of the groups (group A), the participants of which voluntarily agreed to the experiment , payment "from production" was introduced. Measurement after the experiment showed that the productivity of the participants in group A increased. In this quasi-experiment there are threats to validity, namely: the interaction of testing with the results experimental exposure, that is, the previous level of labor productivity, can reinforce the experimental influence. 50. Variable control schemes in empirical research.
36. Explain factorial experiment. Variables, hypotheses and schemes in a factorial experiment.
| pre-experimental plans, more precisely, the studies that have been and are still being carried out according to certain schemes do not take into account the requirements for the plan of the classical pilot study. The reason is ignorance of these requirements or the impossibility of taking them into account when conducting a real study in specific conditions. Research according to the scheme of pre-experimental plans was carried out even before the emergence of the theory of planning an experiment. The pre-experimental plans serve rather as an illustration. In practice scientific research they should be avoided whenever possible. Quasi-experimental plans are an attempt to take into account the realities of life when conducting empirical research. Quasi-experimental plans are created specifically with a departure from the "true experiment" scheme. The researcher is aware of those sources of artifacts - external variables that he cannot control. To partially compensate and control the effects that occur when the plans of "true experiments" are violated, researchers use artificial schemes, which are called quasi-experimental plans. A quasi-experiment is a kind of compromise between reality and the “rigor” of methodological prescriptions. However, the quasi-experiment is used not only to solve applied problems, but also to conduct scientific research. The quasi-experimental design is used when the application best plan impossible. |
The concept of the validity of an experiment includes at least three basic concepts: operational validity, external validity and internal validity. Let us consider these concepts and the realities denoted by them in more detail.
operational validity
The concept of operational validity is closely related to a certain problem that objectively exists not only in experimental, but also in any other empirical study. In particular, in an experimental study, this problem lies in the fact that the reality under study is presented in it in two ways. First of all, in the research hypothesis, it is presented in the form of its verbal description, a Secondly, in the research methodology, it is presented in the form of specific phenomena that have certain characteristics. Obviously, from the point of view of the ideal requirements for conducting an experimental study, it should be organized in such a way that there is a complete correspondence between these two forms of representing the reality under study. If this correspondence is not achieved, then the experimental study is inadequate in the sense that its results cannot be used either to confirm or to refute the hypothesis.
In connection with this problem, the concept of operational validity can be defined as the legitimacy of the conclusion about the completeness of the correspondence of the studied reality, presented in the research hypothesis in verbal form, the reality that is presented in the research methodology in the form of specific phenomena.
Depending on the degree of completeness of this correspondence, an experimental study may have operational validity in varying degrees, i.e. be more or less operationally valid.
In practice, in order to assess the operational validity of an experimental study, it is necessary to resolve the issue of the validity of the correspondence between the contents of the main terms that are used to describe the reality under study in the research hypothesis, and how these contents are presented in the research methodology.
To illustrate the problem of operational validity, one can use following example. For example, the research hypothesis includes the term “attention volume”, which means the number of elements that a person can perceive simultaneously clearly and distinctly, and the research methodology uses a tachistoscope to measure attention volume, which allows visual presentation at the same time on a short time set of elements, and as specific phenomenon, which characterizes the amount of attention, is the number of elements that the subjects can reproduce. In this case, the experimenter conducting this study must inevitably face next question, directly related to the problem of operational validity, namely, whether the number of elements reproduced by the test subjects corresponds to the number of elements perceived by them clearly and distinctly. As is known from history experimental psychology, Wundt admitted the existence of such a correspondence. However, in the experiments of J. Sperling using the partial report technique, it was shown that the number of elements reproduced by the test subjects may be less than the number of elements perceived by them clearly and distinctly. It follows from the above example that the author of this hypothetical study needs to explain more carefully why he considers it possible to use the index of reproduction productivity to measure attention span, otherwise his study may be operationally invalid.
In any case, construct validity determines the theoretical accuracy of the experimental study, i.e. how correlated is the causal relationship formulated in the theory and the causal relationship of the independent and dependent variables found as a result of the study: “... construct validity has to do with theoretical accuracy: do we correctly explain the causal relationships that were found in the study ? ... ".
In other words, construct validity expresses the accuracy of the experimental implementation of the hypothesis in the research procedure: “... construct validity characterizes the accuracy of the implementation theoretical hypothesis in the experimental procedure. It is one of the manifestations of internal validity. Determines the area of phenomena studied in the experiment ... "
Construct validity is controlled by content planning. It determines the theoretical accuracy of the design perfect experiment. Confidence in construct validity comes as you get everything more confirmatory results: “…construct validity – especially suitable for experimental studies. It is relevant to two questions at once: whether the construct being evaluated is valid, and whether the instrument with which the evaluation is performed is the best. Construct validity is closely related to the essence of the theory, the construction of hypotheses on the basis of the theory, and the evaluation of theories derived from the results of the study. Construct validity as such is never confirmed or destroyed by a single study, and it cannot be proven for the same reasons that it is impossible to prove a theory. Confidence in construct validity increases gradually as investigators obtain confirmatory results…”
The key to the high construct validity of our experimental study is the historical (creative) method of substantiating the design of an ideal experiment, and the practical confirmation of the high construct validity of the study is real growth. learning motivation. As is well known, in today's educational process the dynamics of learning motivation has the opposite sign.
Another type of internal validity is operational validity. Operational validity determines the adequacy of the methodology and plan of the experimental study to the tested hypothesis. Operational validity determines conformity real experiment theoretical hypothesis, since it is in the hypothesis that the nature of the relationship between the independent and dependent variables is fixed: “... operational validity is the correspondence of the experimenter's operations to the theoretical description of the variables controlled in the study. The conditions varied by the experimenter should correspond to the independent variable. Operational validity is one of the manifestations of internal validity ... ".
In other words, operational vaality is responsible for the fact that the connection established in a real study is exactly the connection that is formulated in the hypothesis: evaluation of the internal and operational validity of the experiment ... »
Meaningful planning is a way to control operational validity, as well as constructive validity. AT this case with the help of meaningful planning, the “independent variable” (“scheme” of the organization of ontological communication), the “dependent variable” (the psychological complexity of subject tasks) were operationalized, the influence of an additional variable (“different mental age of students”) was evaluated, and a solution was found by differentiating the requirements for students different mental ages: “... meaningful planning includes the solution (resolution) of all issues of constructive and operational validity, consisting in the specification and operationalization of WIP (independent variable), RFP (dependent variable) and the choice of levels of DP (additional variable). It involves the deployment of substantive arguments both in terms of substantiating experimental hypotheses and in terms of correlating the psychological constructs used and the methodological procedures for measuring variables ... ".
External validity
External validity - is a measure of compliance experimental procedure investigated real object.
External validity is affected by additional variables. External validity is controlled by controlling for additional variables: objective reality let's call external validity … ».
External validity depends on the representativeness of the sample and the comparability of controlled additional variables with additional variables in other conditions: “... external validity is the correspondence of a particular study to natural reality and / or other similar studies. Determines the possibility of transferring and/or generalizing the results to other objects and research conditions. Depends on the representativeness of the sample and the correspondence of additional variables controlled in the study, their variability in other conditions. A particular form of external validity is ecological validity, which determines the ability to extend the conclusions of a particular study to real conditions, and not to other laboratory conditions ... "
The transfer of results from laboratory conditions to reality and the generalization of results (conclusions) depend on external validity: “... external validity ... is the possibility of transferring the results of an experiment to a certain real-life set, situation or conditions that differ from the conditions of the experiment ... "
Since our research was largely organized as a field (natural), special problems with external validity did not stand. As a result, experimental training was carried out under conditions a large number additional variables (class occupancy; age of students; type educational institutions; the degree of urbanization of cultural and social environment; culture; ethnopsychological features; the degree of peripherality of the educational institution; teaching style; socio-psychological climate in educational institution; social background of students, etc., etc.).
The resulting data indicate their high external (environmental) validity.
Criteria Validity
In the study, another type of validity, criterion validity, was of particular importance. The need to consider it is due to the fact that in the conditions experiential learning the results obtained are always correlated with the criteria on the basis of which the effectiveness and success of experimental training is assessed: "... validity is called "criteria", because the results of the measurements under consideration correlate with some value or criterion ... ".
The importance of criterion validity lies in the fact that it allows evaluating the effectiveness of experimental learning with the possible effectiveness of real life activity: “... criterial validity reflects the compliance of the diagnosis and prognosis obtained on the basis of testing data with activity and life indicators, includes current and predictive validity…”
From this point of view, criterion validity is of great importance as a measure of the predictiveness of an ideal study: "... the relationship of an ideal study to reality can be characterized as theoretical, or predictive, validity ...".
So far, some types of psychological data have been presented separately - as fixed indicators and in the context of the problem of statistical decisions. Now you should do next step- reveal the problem of choice psychological techniques as a means of operationalizing variables. In relation to the sample values of the RFP, the following are accepted: statistical solutions, according to the values of the variables are the results of the use of certain psychological techniques.
To obtain data, the psychologist records both psychophysiological indicators, if the hypothesis involves consideration of their connection with the process being studied, and self-reports of the subjects. He can observe the behavior, or he can specifically highlight only some, methodically specially selected aspects of the subject's activity - reaction time, the number of proposed solutions, the characteristics of the response in an uncertain situation. He uses standardized and non-standardized methods, with the help of which indicators of the cognitive sphere of a person (perception, attention, memory, thinking), as well as his emotional and personal sphere (motives, characterological properties, etc.) are recorded.
From the characteristics used methodologies should not be what method research was carried out. External characteristic An experimental study may indicate that it should contain at least two methods: for setting the NP and measuring the RFP. And both of them are evaluated in terms of representativeness - the constructs of NP and RFP or compliance with those real conditions, the aspects of which are modeled.
Operating cash- evaluation of the experiment in terms of the degree of compliance with the empirically loaded concepts of NP, ZP and DP of those methods for setting and measuring variables that are used in the study.
Most simple way it would be the definition of a psychological variable by linking it with the method of obtaining certain data. Indeed, if we define the differences between the concepts "method" and "method", then this would seem like a logical path. Method is the most large unit study type analysis. A technique is a smaller unit. Techniques can migrate from one study to another and be used in different ways research organization.
Psychological technique- these are ways of fixing psychological data, "techniques" for constructing psychological indicators. Students usually get acquainted with the variety of psychological methods in subject and research practicums.
At the same time, the technique may not look psychological, but be used to set, say, the levels of the independent variable or to measure the dependent variable. For example, in the assessment different kind advertising - as organized psychological impact- its effectiveness is presented in the number of units of the product sold. This is an economic indicator. It becomes a psychological variable only in one case - when it is associated in a psychological hypothesis with the mediating process of psychological influence. Then different types advertisements for the same product appear different levels NA (for example, advertising permanent or not, in print or electronic media, etc.).
Isolation of NP levels can also be carried out, it would seem, regardless of the arsenal of psychological techniques. In the example of the artificial environment "The Fifth Dimension" given in Section 5.3.3, it can be interpreted as a single (active) level of NP. Then the control (inactive) level will be normal conditions education and leisure activities for the child. Numerous engineering psychology studies have compared different conditions human performance professional activity, despite the fact that the difference in conditions is set by the features of the instruments used, their location, etc. Without continuing the examples, we can say that these levels of experimental factors become psychological precisely due to the fact that in the link experimental hypothesis they are associated with the expected changes in the studied basic process.
In the very general view Under the basic process can be understood central link regulation of changes reflected in indices or indicators, 311. An indicator recorded by the methodology as a response to the influence of NP does not in itself reduce to the basic process under study.
In a psychological experiment, this is also included in the stage of meaningful research planning - to justify what is meant by changes in the index. For example, behind the reaction time indicator, one can assume completely different processes (denoting what exactly happens in fixed periods of time before the subject's response).
If we are talking about more complex techniques, which include a significant layer interpretations psychologist of this or that index, then the interpretive process will directly determine the way the variable is constructed. For example, in relation to projective methodology the thematic apperceptive test (TAT) by E. T. Sokolova, a change in the ways of interpreting how in stories using indefinite pictures (photos) it is possible to establish the manifestation of projection mechanisms, psychological protection etc. From the end of the 30s to the end of the 60s. 20th century interpretations changed significantly depending on the inclusion of assumptions about the manifestation of their rational "I" in the stories of the subjects [Sokolova, 1980].
The same types of deep motivation (sociogenic needs in G. Murray's classification) can be diagnosed using a questionnaire proposed by L. Edwards [Kornilova, 1997]. projective test and verbal, built in the Edwards questionnaire based on the method of forced choice of preferences, can give different locations of the subject in relation to a specific motivational scale, since they imply the manifestation of different basic processes behind the used indices of two procedurally different methods.
Thus, the same psychological constructs (achievement motivation, aggression, affiliation, etc.) are operationalized in different ways. Their difference is connected precisely with the implied mechanisms of conscious evaluation of preference or unconscious manifestation in the subject's story of one or another deep motivation (with the instruction "imagination is explored").
So, the simplest way of linking the essence of a psychological variable with one method or another can be misleading as to what the fixed indicator represents. Another way is more adequate - an indirect reconstruction of what is represented by the indicator. Two conditions must be met.
First. Presented in the hypothesis as empirically loaded concepts - independent and dependent variables - must be operationalized, i.e. the way of their methodical presentation in the study should be specified.
Second. This operationalization should be such that the control of the experimental factor or changes in the basic process can be represented in the form of one scale or another, i.e. measured.
Not all methods allow you to select variables. For example, observation methods that involve fixing data in units natural language or in legend, do not yet act as methods for extracting variables. Watching changes is not the same as measuring them. Modern research, performed by the method of observation, include a huge arsenal of standardized techniques. Software to proceed to the measurement procedures must be performed additional terms, as a result of which the possibility of isolating the rules for assigning numbers to psychological objects is achieved.
It is the measurement of variables that is the condition for carrying out psychological experiment, as well as the implementation of a number of other methods - correlation, quasi-experiment, longitudinal.
The enumeration of events, the establishment of their frequency is not yet a measurement, but a procedure that allows you to build an RFP. Let us give an example of how such a ZP is constructed when its index is derivative, and does not reflect the fixation of any methodologically specified indicator.
In the example with artificial experimental condition"The Fifth Dimension" by M. Cole, which will be presented in chapter 5 (excursion 5.8), many variables can be distinguished. AT real conditions on the beach, psychologists created a special situation of relaxation for children, who, at the same time, game situation- mastered a number of means (in particular, a computer) of their activities. 311 was a few. One of them is the mention in the diaries of students who worked with children of various "artifacts" in their instrumental meaning or in the form of references as guidelines for mediating the actions of the child. The number of mentions of the same objects of the environment in two contexts - orientation to the possibility of their use or mediation of actions by their use - made it possible for the author to trace the temporal dynamics of the "culturation" process in an indirect (reflected by a psychologist's assistant) form.
Rice. 3.1. Dynamics of variables reflecting the instrumental and orientational attitude to "artifacts" in the created game context
In the diagram shown in fig. 3.1, it can be seen how it grows with the time spent in built environment The "fifth dimension" is a quantitative indicator of the instrumental attitude to its elements in comparison with another trend - a decrease in the indices of the general orientation towards the elements of the environment. Thus, in the natural conditions of this study, those indicators were singled out whose dynamics served to verify (empirical verification) the theoretical hypothesis about the role of "artifacts" as means given to the child to transform his gaming activity and thus its development.
