The Original NY Times Op-Ed of March 11th, 2014

On Tuesday, March 11th, I published a piece in the New York Times on college admissions testing in which I argued for the assessment of mental abilities beyond the SAT. The original op-ed is online here.

In that op-ed piece, I suggested that students might want to take additional measures of abilities beyond the SAT. If they were to do so, it would better reflect the multifaceted nature of our youth and their talents. I also believe it would free teachers to use a broader range of instructional techniques, because the wider range of tests would be likely to pick up the virtues of different teaching styles (I did not make this point explicitly in the original op-ed). It would also result in less emphasis on any single test.

Here is the basic outline of the op-ed:

• First, we find test scores difficult to absorb—particularly when they are lower than we had hoped. Low scores result in mental anguish and that can make it hard for us to accept test scores as a true reflection of our mental qualities. For this reason, tests such as the SAT will never be universally loved—but they can be tolerated as a social good.
• Second, all tests are imperfect. For example, if students haven’t had access to a good education their performance may be impacted through no fault of their own. As a second example, my personal opinion is that there are at least a very few people who are smart about the language and mathematics (as I see it) and are not served well by the SAT in that they have scored lower than might be expected.
• Third, acknowledging the imperfections of tests, the current SAT works well at measuring people’s verbal and mathematical abilities (I assume the test’s revision will work well also).
• Fourth, given that we (and our children) are multifaceted in our talents, our tests should represent this. That means colleges and applicants should continue to take the SATs (and/or ACTs) but might consider adding tests of additional abilities.
• Fifth, allowing students to opt out of tests sends a message that self-knowledge gained from testing is unimportant. Allowing for a broader range of tests expresses the value of developing self-knowledge, while it also acknowledges the multifaceted nature of our abilities.

Vetting the Op-Ed

As a condition for publication at the New York Times, contributions are fact-checked. Because psychological testing is a controversial area that attracts a great deal of attention, and because in places I made claims that appeared to be inconsistent with other reporting in the paper, my remarks were especially carefully scrutinized. I have provided the key references on which I based my arguments, in the section labeled “Supporting Documents” below.  In one case, I have substituted an article that is better than what I provided at the time.

Commentary from Elizabeth Wissner-Gross

In a letter to the NY Times, Elizabeth Wissner-Gross, an educational consultant, argued that there are a range of optional tests students can take to demonstrate their aptitude in traditional academic areas, and provided a helpful list. As she put it:

“…students always have the option of participating in a bounty of enrichment exams: AMC (to demonstrate math talent), USACO (computer science), USABO (biology), USNCO (chemistry), F=MA Contest (physics), NACLO (linguistics), Intel STS or Siemens (science research), Scholastic Art and Writing Awards (art and literary talent), Concord Review (history), National Latin Exam, Le Grand Concours (French), NSE (Spanish), performing arts contests and auditions, and lots more.”

I think many students may be surprised and intrigued to see these additional choices, and her recommendations are in keeping with one key idea I’m espousing: that to reduce the sting of any single test (and the test’s control over educational practices), it makes sense to test more broadly.

At the same time, Wissner-Gross doesn’t see why testing should be extended to matters such as creativity or emotional intelligence. My rationale for extending our testing to include these capacities is that we can now do so for the first time with ability-based tests, and that there are important advantages to doing so. As I see it, these are to:

• reduce the weight we place on any single test (and the influence it can exert)
• to acknowledge a broader range of skills that people possess, thereby being more fair-minded to our general human abilities, and
• to allow students to explore their own interests in high school without feeling overly pressured to join every possible club or activity so as to exhibit all of their skills.

Despite our disagreement, I very much appreciated Wissner-Gross’ comments and recommend her letter.

Elaborating a Bit on the NY Times Op-Ed: Tests I Had in Mind

(Please note: I am still working on this section -JDM)

I’ve been asked about what additional tests I had in mind, specifically. A quick answer (and this is a work in progress), is below.

Criteria for Test Recommendations

The tests, I believe, should meet certain key criteria:

• They should be ability-based because those are impossible to fake, and any educational training for them potentially improves a student’s actual knowledge.
• Researchers should have published enough studies concerning whether the test measures the ability it claims to (test validity), to have some confidence in the measure.

Remember that I am acknowledging that all tests are imperfect—that includes all the tests I am recommending here. By using them together, however, I believe we can learn more about a person’s abilities than by using the SAT alone (or, worse, in my opinion, opting out).

Here is a short list of tests I had in mind when I wrote the Op-ed piece, but I’m sure there are more tests that meet these criteria.

Disclosure

I have a financial interest in the tests below on which I am a coauthor. For the other tests (on which I am not an author) I have no relationship with the authors or companies who publish them.

Tests for Consideration

Creativity

The Torrance Test of Creative Thinking

Spatial Ability

The measurement composite for spatial ability drawn from Project Talent and outlined by   on pp. 821 and 822 of:

Wai, J., Lubinski, D., and Benbow, C. P.  (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817-835.

Emotional Intelligence (Please note: I am a coauthor on the following test I am recommending).

The Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT).

Personal Intelligence (Please note: I am a coauthor of the following test I am recommending).

The Test of Personal Intelligence (TOPI). This test is presently in development and shows considerable promise.

Other Tests

The Miller Analogies Test

 Supporting Documents

The following are the articles that I supplied to the NY Times over the course of their fact-checking in support of my argument. One exception: I have provided a better article about spatial intelligence here than the one I supplied to the Times.

The surprise value of SAT scores:

• Paulhus, D. L., Lysy, D. C., & Yik, M. S. M. (1998). Self-report measures of intelligence: Are they useful as Proxy IQ tests? Journal of Personality, 66, 525-554.

The usefulness of the SAT itself and related tests such as the GRE:

• Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the Graduate Record Examinations: Implications for graduate student selection and performance. Psychological Bulletin, 127, 162-181.
• Mattern, K. D., Patterson, B. F., Shaw, E. J., Kobrin, J. L. & Barbuit, S. M. (2008). Differential validity and prediction of the SAT [Technical Report]: New York: The College Board.
• Mattern, K. D., Shaw, E. J., & Kobrin, J. L. (2011). An alternative presentation of incremental validity: Discrepant SAT and HSGPA performance. Educational and Psychological Measurement, 71, 638-662.
• Shen, W., Sackett, P. R., Kuncel, N. R., Beatty, A. S., Rigdon, J. L., & Kiger, T. B. (2012). All validities are not created equal: Determinants of variation in SAT validity across schools. Applied Measurement in Education, 25, 197-219.

Tests as Infrastructure and other uses of tests:

• Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A., Goldberg, L .R. (20o7). The power of personality: The comparative validit of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological Science, 2, 313-345.

The usefulness of additional tests beyond the SAT:

• Wai, J., Lubinski, D., and Benbow, C. P.  (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817-835.