4. Basic schemes for the validation of psychodiagnostic methods. Only large scientific and methodological centers can solve the problem of predictive validity. Indeed, psychometric research to test predictive validity must involve an order of magnitude

3. The principle of limited dissemination of psychodiagnostic techniques (the principle of professional secrecy) In a brief summary, this principle states that professional psychodiagnostic techniques certified by the psychological society can be distributed

RELIABILITY AND UNPREDICTABILITY Natrum muriaticum is the very opposite of complacency, based on a sense of stability and security. Even if everything goes well in his life, he is rarely satisfied. Fearing a turn of fate, he begins to superstitiously think: “All this

Increasing visual acuity using hardware techniques Despite the fact that regular eye exercises help improve visual acuity, in case of high myopia, therapeutic treatment in a hospital is recommended. Currently there are

Chapter 3 VARIETY OF METHODS In addition to turpentine baths, there are other methods of physiotherapeutic effects on capillaries. All of them are rooted in the ancient past, that is, their safety and effectiveness have been tested by time. Many ancient techniques today

Increasing visual acuity using hardware techniques Despite the fact that regular eye exercises help improve visual acuity, in case of high myopia, therapeutic treatment in a hospital is recommended. Currently, there are

Alphabetical index of tests, methods and analytes A-Z 17-hydroxyprogesterone 9617-ketosteroids 993D ultrasound 4794D ultrasound 479CA 15-3 162CA 19-9 162CA 72-4 163CA 125 161Cyfra 21-1 163C-reactive protein 31D-dimer 82 HLA antigens 94IgA 86IgE 86IgG 87IgM 87MAR test 91pH urine

ERMs are reliability! The psychotechniques of the SOS system allow almost any person to become a very good hypnotist for themselves. However, turning yourself into one with the help of only this book (remember, for comparison, how some enthusiasts learn to play musical

*Reliability and validity of a test are characteristics of a study’s compliance with formal criteria that determine quality and suitability for use in practice.

What is reliability

During test reliability testing, the consistency of the results obtained when the test is repeated is assessed. Data discrepancies should be absent or insignificant. Otherwise, it is impossible to treat the test results with confidence.

Test reliability is a criterion that indicates that the following properties of tests are considered essential:

• reproducibility of the results obtained from the study;
• degree of accuracy or related instruments;
• sustainability of results over a certain period of time.

In the interpretation of reliability, the following main components can be distinguished:

• the reliability of the measuring instrument (namely the literacy and objectivity of the test task), which can be assessed by calculating the corresponding coefficient;
• the stability of the characteristic being studied over a long period of time, as well as the predictability and smoothness of its fluctuations;
• objectivity of the result (that is, its independence from the personal preferences of the researcher).

Reliability factors

The degree of reliability can be affected by a number of negative factors, the most significant of which are the following:

• imperfection of the methodology (incorrect or inaccurate instructions, unclear wording of tasks);
• temporary instability or constant fluctuations in the values ​​of the indicator that is being studied;
• inadequacy of the environment in which initial and follow-up studies are conducted;
• the changing behavior of the researcher, as well as the instability of the subject’s condition;
• subjective approach when assessing test results.

Methods for assessing test reliability

The following techniques can be used to determine test reliability.

The retesting method is one of the most common. It allows you to establish the degree of correlation between the results of studies, as well as the time in which they were conducted. This technique is simple and effective. Nevertheless, as a rule, repeated examinations cause irritation and negative reactions in subjects.

• constructive validity of a test is a criterion used when evaluating a test that has a hierarchical structure (used in the process of studying complex psychological phenomena);
• criterion-based validity involves comparing test results with the test subject’s level of development of one or another psychological characteristic;
• content validity determines the correspondence of the methodology to the phenomenon being studied, as well as the range of parameters that it covers;
• predictive validity is one that allows one to evaluate the future development of a parameter.

Types of Validity Criteria

Test validity is one of the indicators that allows you to assess the adequacy and suitability of a technique for studying a particular phenomenon. There are four main criteria that can affect it:

• performer criterion (we are talking about the qualifications and experience of the researcher);
• subjective criteria (the subject’s attitude towards a particular phenomenon, which is reflected in the final test result);
• physiological criteria (health status, fatigue and other characteristics that can have a significant impact on the final test result);
• criterion of chance (takes place in determining the probability of the occurrence of a particular event).

The validity criterion is an independent source of data about a particular phenomenon (psychological property), the study of which is carried out through testing. Until the results obtained are checked for compliance with the criterion, validity cannot be judged.

Basic criteria requirements

External criteria that influence the test validity indicator must meet the following basic requirements:

• compliance with the particular area in which the research is being conducted, relevance, as well as semantic connection with the diagnostic model;
• the absence of any interference or sharp breaks in the sample (the point is that all participants in the experiment must meet pre-established parameters and be in similar conditions);
• the parameter under study must be reliable, constant and not subject to sudden changes.

Ways to Establish Validity

Checking the validity of tests can be done in several ways.

Assessing face validity involves checking whether a test is fit for purpose.

Construct validity is assessed when a series of experiments are conducted to study a specific complex measure. It includes:

• convergent validation - checking the relationship of assessments obtained using various complex techniques;
• divergent validation, which consists in ensuring that the methodology does not imply the assessment of extraneous indicators that are not related to the main study.

Assessing predictive validity involves establishing the possibility of predicting future fluctuations of the indicator being studied.

conclusions

Test validity and reliability are complementary indicators that provide the most complete assessment of the fairness and significance of research results. Often they are determined simultaneously.

Reliability shows how much the test results can be trusted. This means their constancy every time a similar test is repeated with the same participants. A low degree of reliability may indicate intentional distortion or an irresponsible approach.

The concept of test validity is associated with the qualitative side of the experiment. We are talking about whether the chosen tool corresponds to the assessment of a particular psychological phenomenon. Here, both qualitative indicators (theoretical assessment) and quantitative indicators (calculation of the corresponding coefficients) can be used.

The transformation of psychodiagnostic procedures and techniques into a reliable tool of science and practice depends on the efforts of many specialists in psychometric debugging, designing tests that meet the basic psychometric requirements: reliability, validity, standardization. The basic principles of testing and determining the reliability, construction and validation of psychodiagnostic methods are covered in a number of special works on psychodiagnostics (A. Anastasi, A. Bodalsi, V. Stolin, A. Shmelev, K. Gurevich, V. Melnikov, etc.). In this tutorial we will describe the basic concepts and principles of conducting a psychodiagnostic examination, knowledge of which is an indispensable condition for the professional qualifications of a practical psychologist.

Psychodiagnostics as a scientific discipline includes three areas of psychological knowledge:

the subject area of ​​psychology that studies these mental phenomena;

psychometrics - the science of measuring individual differences and diagnosable variables;

the practical use of psychological knowledge for the purpose of adequate psychological influence and helping people solve their problems.

The methodological basis of psychodiagnostics is psychometrics. It is this science that develops the technology for creating specific psychodiagnostic techniques and determines the methodology for ensuring scientific requirements for them:

reliability - internal consistency of parts of the test and reproducibility of results during repeated testing;

validity – reflection in the test results of exactly the property for which it is intended to diagnose;

reliability - protection of the test from influence on the results of the test taker’s desire to change them in the desired direction;

representativeness - the presence of norms for the results of a mass survey in the population for which the test is designed, allowing one to assess the degree of deviation from the average values ​​of any individual indicator.

These psychometric requirements apply to different groups of tests, with the greatest extent to objective tests and personality questionnaires, and to the least extent to projective techniques.

An objective assessment of psychological techniques and tests means determining their reliability. In psychometrics, the term “reliability” always refers to the consistency of scores obtained from the same subjects.

How useful is this test? Does it really fulfill its functions? These questions can and sometimes do cause lengthy, fruitless discussions. Prejudices, subjective conclusions, and personal biases lead, as A. Anastasi believes, on the one hand, to an overestimation of the capabilities of a particular test, and on the other, to its persistent rejection. The only way to answer such questions is through empirical testing. Objective assessment psychological tests first of all means determining their reliability and validity in specific situations.

Test reliability there is consistency in scores obtained from the same subjects when retested with the same test or an equivalent form.

If a child’s IQ is 110 on Monday and 80 on Friday, then it is obvious that this indicator can hardly be taken with confidence. Similarly, if an individual correctly identified 40 words in a series of 50 words, and 20 in another considered equivalent series, then neither of these indicators can be considered as a measure of his verbal comprehension. Of course, in both examples it is possible that only one of the two indicators is erroneous, but only subsequent testing can confirm this; From the data presented, it only follows that the indicators together cannot be correct.

Before a psychological test is made available to the public, a thorough, objective test of its reliability must be conducted. Reliability can be tested with respect to changes over time, the choice of specific tasks or test samples, the personality of the experimenter or test processor, and other aspects of testing. It is very important to specify exactly the type of reliability and how it is determined, since the same test can vary in different aspects. It is also advisable to have information about the number and characteristics of individuals on whom the reliability of the test was tested.

Such information will enable the user of the test to decide how reliable the test is for the group to which he intends to apply it.

The most complete explanation of the reliability of test methods is given by A. Anastasi. Reliability refers to the consistency of test results obtained when it is repeated on the same subjects at different points in time, using different sets of equivalent tasks, or when changing other examination conditions. The calculation is based on reliability measurement errors, which serves to indicate the probable limits of fluctuations of the measured quantity that arise under the influence of extraneous random factors. In its broadest sense, reliability refers to the extent to which individual differences in test scores turn out to be “true” and the extent to which they can be attributed to random errors. If we translate this into the language of special terms, then measuring the reliability of a test allows us to estimate the value of the total dispersion of test indicators, which is error variance. The question, however, is what to count as error variance. The same factors, which are extraneous in relation to some problems, are already considered sources of “true” differences when solving other problems. For example, if we are interested in mood swings, then day-to-day changes in emotional state test scores could be related to the purpose of the test and therefore to the true variance of the scores. But if the test is designed to measure more stable personality characteristics, then the same daily fluctuations can be attributed to error variance.

The important thing is that any changes in the conditions under which the test is conducted, if they are not relevant to its purpose, will increase the error variance. Therefore, by adhering to uniform testing conditions (controlling the general environment, time constraints, instructions to the subject, contact with him and other similar factors), the experimenter reduces the error variance and increases the reliability of the test. But even under optimal conditions, no test is an absolutely reliable tool. Therefore, a standard set of test data should include a measure of reliability. This measure characterizes the test when it is administered under standard conditions and administered to subjects similar to those who participated in the normative sample. Therefore, it is also necessary to provide information about this sample.

K. M. Gurevich defines reliability as “an extremely complex and multifaceted concept, one of the main functions of which is to assess the consistency of test performance indicators” [Gurevich, 1981].

In principle, we can say that reliability should justify measurement error—it should show how much of the variability in measures is due to error. There are several main factors that determine the level of reliability. Thus, reliability will always tend to increase if the conditions of the testing procedure are kept constant, since this reduces the error of variability of the measured parameter. At the same time, the multiplicity of goals, the complexity of the problem, and the variability of situations tend to increase measurement error, thereby reducing reliability.

There are as many varieties of test reliability as there are conditions affecting the test results, so any such conditions may turn out to be extraneous to the purpose, and then

the variance caused by them should be included in the error variance. However, only a few types of reliability find practical application. Since all types of reliability reflect the degree of consistency or consistency of two independently obtained series of indicators, their measure can be correlation coefficient. A more specialized discussion of correlation with a detailed description of computational procedures is given in textbooks on statistics for teachers and psychologists (V. Avanesov, A. Gusev, Ch. Izmailov, M. Mikhalevskaya, etc.).

In practice, three main methods are used to assess the reliability of tests:

1) retesting;

2) parallel testing;

3) splitting method.

Let's consider each of them separately.

Retesting It is one of the main methods for measuring reliability. Repeated

testing a sample of subjects is carried out with the same test after a certain time interval under the same conditions. Retesting is usually called retest, and reliability measured in this way is test-retest reliability. The test-retest reliability assessment scheme looks like this:

In this case, the correlation coefficient between the results of two tests is taken as the reliability index.

The repeated testing method has both advantages and disadvantages. The advantages include the naturalness and simplicity of determining the reliability coefficient. Disadvantages include the uncertainty in choosing the interval between two measurements. The emergence of temporary uncertainty is due to the fact that retesting differs from the initial one. The subjects are already familiar with the content of the test, remember their initial answers and are guided by them when repeating the test. Therefore, during repeated testing, one often observes either “adjustment” to the initial results, or, as a consequence of negativism, the demonstration of “new” results. To avoid this, when giving test-retest reliability in the test manual, you should indicate what time interval it corresponds to. Due to the fact that test-retest reliability decreases with increasing time interval, the most reliable are high reliability coefficients obtained with clearly large intervals between tests. Insufficiently high reliability coefficients may be a consequence of suboptimal determination of time intervals.

Parallel testing In this case, multiple measurements are organized using parallel, or equivalent, tests. Parallel tests are tests that measure the same mental property with the same error. In this case, the same individuals perform multiple versions of the same test or equivalent tests. As a rule, the practical use of this type of reliability is associated with significant difficulties, since it is extremely difficult to construct several versions of one test in such a way that the subject cannot detect their psychological homogeneity. And the distorting influence of training in this case is not completely removed. In addition, the question arises: are alternative types of reliability characteristics of test reliability, and not test equivalence parameters? After all, if two forms of testing are carried out under the same type of constant conditions, then, most likely, the equivalence indicators of the two forms of testing are studied, and not the reliability indicators of the tests themselves. The measurement error in this case is determined by fluctuations in the test execution, and not by fluctuations in the test structure.

The scheme for using parallel tests to measure reliability is as follows:

The correlation coefficient calculated between two tests is called equivalent reliability.

Cleavage method It is a development of the parallel testing method and is based on the assumption of parallelism not only of individual test forms, but also of individual tasks within one test. This is one of the simplest tests of a test, when the correlation coefficient between its halves is calculated. How to divide the test into two halves in order to be able to align both halves on one or another specific basis? Most often, test tasks are divided into even and odd, which allows to some extent eliminate possible shortcomings. The main advantage of this type of reliability is the independence of test results from such elements of activity as development, training, practice, fatigue, etc. When dividing the test into two parts, the reliability index is calculated using the Spearman-Brown formula, which proposed it independently of each other. Their articles were published in the same issue of a psychological journal with conclusions and formulas [Avanesov , 1982]. In their formula

R(x, 0=2 RJ\ + R, y

where R is the correlation coefficient of the two halves of the test. The average modulus of the correlation coefficient of all test items or the average determination coefficient is considered as a reliability index coefficient.

So far, we've looked at three empirical methods for assessing test reliability: retesting with the same test, retesting with a parallel form of the test, and splitting the test.

Which of these methods provides a true estimate of the test's reliability? Which method should you use? The answer to this question depends on personal preference and the goals of the study.

When using the repeated testing method, we obtain an assessment of the degree of stability of the results over time and depending on the testing conditions. Therefore, the test-retest reliability coefficient is also called stability coefficient or stability test. When using the parallel forms method and the splitting method, the degree of mutual consistency of the test parts is assessed. Therefore, the reliability coefficients obtained by these two methods are interpreted as swayed and homogeneity, homogeneity tests.

In addition to indicators of stability and homogeneity, R. B. Cattell considers it necessary to consider the indicator transferability. It is an assessment of a test's ability to maintain measurement accuracy across different samples, subcultures, and populations. Together, stability, homogeneity and portability form a complex characteristic of reliability, which R. B. Cattell calls consistency and defines it as “the degree to which a test continues to predict what it once predicted despite changes (within certain limits): a) the extent to which the test is applied; b) the conditions under which it was used; c) the composition of the sample in which it is applied.”

Finally, there is a type of reliability that directly relates to the reliability of the person administering the test. An estimate of the reliability of the person administering the test is obtained by independently simulating the test by two different experimenters.

The reliability of test results depends not only on the reliability of the test itself and the procedure for conducting it. An important factor influencing the results of data interpretation is the specificity of a particular sample. The most significant characteristics of the sample, from this point of view, should be recognized as socio-psychological homogeneity in various parameters; age and gender are also taken into account.

A.G. Shmelev proposes to carry out the sequence of actions when checking reliability as follows [General psychodiagnostics, 1987]:

1. Find out whether there is data on the reliability of the test proposed for use, on what population and in what diagnostic situation it was tested. If there was no check or if the features of the new population and situations are clearly specific, re-check the reliability taking into account the options indicated below.

2. If opportunities permit, then retest on the entire standardization sample and calculate all the coefficients given both for the whole test and for individual items. Analysis of the obtained coefficients will help to understand how negligible the measurement error is.

3. If possibilities are limited, repeat testing only on part of the sample (at least 30 subjects), manually calculate the rank correlation to assess the internal

consistency (by splitting method) and stability of the whole test.

Of course, the considered concepts of psychodiagnostics are its most important attributes. However, high reliability indicators in themselves do not determine the practical value of the test. The leading factor that allows you to measure the target results of psychological testing is validity.

Ministry of Education and Science of the Russian Federation

State educational institution

Higher professional education

"Volga Region State Social and Humanitarian Academy"

Department of Social Pedagogy, Psychology and Pedagogy of Primary Education

Reliability of psychodiagnostic techniques: its types, methods of establishing. The relationship between validity and reliability.

Completed by a second year student

Psychological and pedagogical diagnostics

day department

Faculty of Primary Education

(specialty Social Pedagogy)

Pavlova Marina Vyacheslavovna

Samara 2011

Introduction

Reliability of psychodiagnostic techniques

Methods of establishing and types of reliability

Relationship between validity and reliability

Conclusion

List of used literature

Introduction

In traditional testing, the term “reliability” means the relative constancy, stability, and consistency of test results during initial and repeated use on the same subjects. As A. Anastasi writes, one can hardly trust an intelligence test if at the beginning of the week the child had a score of 110, and by the end - 80. Repeated use of reliable methods gives similar scores. In this case, to a certain extent, both the results themselves and the ordinal place (rank) occupied by the subject in the group may coincide. In both cases, when repeating the experiment, some discrepancies are possible, but it is important that they are insignificant, within the same group. We can say that the reliability of a technique is a criterion that indicates the accuracy of psychological measurements, that is, it allows one to judge how trustworthy the results are.

Reliability can be established in two general ways: 1) by comparing the results obtained using a given technique or test by different diagnosticians; 2) by comparing the results obtained by using a given technique or test under identical conditions. There are many types of reliability and specific ways to determine their level. Determining the reliability of a test is associated with the concept of the internal consistency of the test; this is expressed in dividing the test into parts and then comparing the results of the parts. The reliability of the test is also determined by the methods of variance and factor analysis.

Types of reliability

When studying reliability, the task is often to determine the reasons leading to the formation of one or another side of reliability. Without this, it is impossible to outline the correct program of work to improve reliability. This leads to the division of reliability into:

Hardware reliability due to the condition of the equipment;

Software reliability of an object, determined by the state of the programs;

Reliability of the facility due to the quality of service;

Functional reliability.

The concept of “software reliability” deserves special attention, since its important role in ensuring the reliability of automated control systems is one of the most characteristic features of the applied theory of automated control system reliability. The concept of “software reliability” arose as a result of the following main reasons. In engineering practice, program-controlled products are becoming increasingly important: program-controlled machine tools; computers and machine systems; ACS data transmission systems, etc. These products are characterized by the fact that they are an organic fusion of technical means (equipment) and software. Without software, a computer complex, or data transmission path, is a “dead” set of technical devices that comes to life if and only if it is used as a whole with a program. Therefore, talking about the reliability of such devices is meaningless if you do not take into account the influence of software.

Taking into account the influence of software leads to the need to distinguish software reliability of objects as a special type.

Functional reliability - the reliability of the performance of individual functions assigned to the system. ACS, as a rule, is a multifunctional system, i.e. it is intended to perform a number of functions of varying importance. Reliability requirements for the performance of various functions may be different (for example, the “payroll calculation” function requires high accuracy, but does not require a strict time limit). Therefore, it may be appropriate to specify different requirements for different functions. An example of functional reliability in an automated control system can be the reliability of the transmission of certain information in a data transmission system.