304: Validity

1. Validity "the degree to which a test actually measures what it purports to measure"
1. Gave an example of the GARS and how it is not a sensitive enough measure of autism yet is still used in most public schools in this area probably because it has good marketing
2. Reliability
1. Essential for validity but not all inclusive
3. Face Validity
1. Not a true measure of validity since a measure can look ok but not actually measure what it purports to measure
2. Often it is important to have a low face validity so participant's answers can remain unaffected
1. However, there are still instances where high face validity can be useful
2. Why?
1. It may increase the risk of measurement error
2. It may also help motivate because the test appears relevant to the subject
4. Content-Related Validity
1. Definition
1. The degree to which a test or measure adequately represents the conceptual domain (universe) it is designed to measure
2. CVR
1. Uses a panel of judges to rate items on a 3-point scale
2. > 50% of judges consider it essential then it has validity
5. Criterion-Related Validity
1. Sub-types
1. Predictive Validity
1. The accuracy with which a test predicts a criterion obtained at a later time.  SAT is the predictor and college GPA is the criterion
2. Concurrent Validity
1. At the same time.  Give you a test now and I'll interview your supervisor and see how well those match up
3. Postdictive Validity
1. Score predicts a previously obtained criterion.  I'll take a bunch of prisoners, give them a test, and I will be able to guess what kinds of crimes they committed in the past
2. Validity coefficient
1. Relationship between a test and a criterion
3. Validity coefficient squared (interested in this rather than just the validity coefficient)
1. The percentage of variation in the criterion we can expect to know in advance because of our knowledge of the test score
1. Most of the time a low validity coefficient will mean the results are not worthwhile.  What are some situation with a low validity but they still may be completely necessary and worth it?
4. Concerns with interpreting validity coefficients
1. Has the cause of relationships changed?
2. What does the criterion mean?
1. Nothing if it is not valid and reliable
3. Was a representative subject population used in the validity study?
1. The GARS was useful for the population studied but has low external validity to other populations 
4. Is the sample size adequate?
5. Never confuse the criterion with the predictor
1. Remember you want successful graduate students and not successful GRE scores
6. Is there restricted range on both the predictor and the criterion?
7. Review evidence for validity generalization
8. Consider differential prediction