Psychometric Test Reliability, Validity and Score Interpretation
Cross Validation:
Check the statistical proof as to the reliability and validity of the test. The first new application is necessary to validate the new definitive test.
Interpretation Referred to the Norm:
Comparison of the scores obtained by each subject with those of the group members who took the test, applied in a broad representative sample of the population. The fundamental aspect of the construction is that the tables relate the possible scores of the test to their significance, as standard norms: 1) percentile ranges (formula), typical z scores (formula) and standardized scores.
Interpretation Referred to the Criterion:
Domain proximity of the questions, known and considered by those who represent the domain. Proximity to the domain: the variation is zero at 0 and 1 equals total. Problem: knowing from which score one considers that a subject sufficiently dominates the subject matter.
Common Scales Used in Test Scoring:
Percentile ranges (formula RP), typical scores (formula z) and arbitrary scale with mean 50; IQ with mean 100, standard deviation 15; T with mean 50, standard deviation 10 standard scores.
Percentile Ranges:
Standardization procedures and scoring methods used in tests (RP formulas): fa: accumulated frequency; fx: frequency of x. The calculation is made as follows:
- Ordering of the different possible scores.
- Counting the frequency of each score.
- Calculation of the accumulated frequencies.
- Calculation of the RP expressed by the formula.
Percentile ranges clearly express the order of the scores.
Typical Scores:
Typical scores are a frequently used transformation to interpret the position of subjects. Their advantage is that the mean is always 0 and the standard deviation is 1. They indicate the score of each subject, showing whether it lies above or below the mean of the reference group. Typical scores can be positive or negative and include decimals, and they are relative in order to express position.
Typical Score Derivatives:
Arbitrary scales and the means and standard deviations that are chosen for convenience (S, IQ, T).
Standard Scores:
They are based on percentile ranges, and the z score determines them to decide these ranges from the normal curve table. Scales that use quantitative measures allow ordinal measures to be interpreted unequivocally.
Validation:
Accumulation of data on the various aspects and whether they have empirical meaning or justify the use of an indicator in a particular context.
Aspects of Validation:
- Reliability (reproducibility, equivalence, temporal stability and consistency of scores)
- Validity (the extent to which the scores of an indicator can support assertions about a criterion or construct)
Reliability:
Classifications of reliability: reproducibility, rankings and quantifications, whatever technique arises from the relevant information.
Classic Methods to Estimate Reliability:
Test-retest methods, described for the most part, as well as the methods of Spearman, Rulon and Guttman.
Alternative Methods and Procedures:
Using the composite reliability of two scores to define each subject from the matrix of items. Two formulas: Rulon, Spearman and Guttman. Items are divided into odd and even, and the observed differences in each subject's scores are purely random. These approaches cannot compute pure reliability regardless of test speed, but they are alternative methods used when tests are short or unequal.
Coefficient Alpha:
This statistic indicates the following: 1) no essential components; 2) average correlation among items: high correlation and similar effect rates among the components of the items, so the first condition is necessary but not sufficient for unidimensionality; 3) receives various names: internal consistency, item-to-item reproducibility, reliability, equivalence, or random error. Alternative methods of relation may be among the other methods.