Concerns about Online Assessment? Yes! It's CAT.

When I forwarded my desperate email to Diane Ravitch to my good friends and kind listeners, my friend, Christine, always so observant and such a careful reader, wondered what my concerns were with the NWEA MAP assessment. I realized that I had about 100 concerns within that email, and that if I am to help grow the national conversation, I need to take my rant down a notch. In this first blog, I am going to try to clarify my concerns with online tests, such as the NWEA MAP test because of computer adaptive tests.

Advocates of computer adaptive tests (CAT) say that the program behind the assessment tailors the test to the student's ability. No longer are students frustrated by an exam with items that are too difficult. Teachers and students are given immediate results; no longer do we have to wait for months before test results are given. The tests have RIT reporting that allows all to see how student learning grows over time and over years. Educator evaluation is required by law for every teacher in every building in Michigan. These MAP tests are being used to evaluate teachers in some districts, especially as pre- and post-tests are incorporated into classrooms. The CA tests are being used across the nation with growing numbers. They are purported to be cost effective, efficient, and objective.

I have several concerns about the CAT, especially with regard to equity. For the sake of time, I will limit this rant to three.  First, the process of taking a computer adaptive assessment can easily become a psychological issue that has absolutely nothing to do with demonstrating knowledge. I have watched 30-40 students at a time take CA tests, up to 150 students. Students who are not easily compliant and are sick of assessments find out quickly that if they choose random answers that the questions get easier and easier and the program soon kicks them out of testing. For students who are anxious and want to a good job, they begin to know in short time that they are getting items wrong, because the items get easier and easier. For struggling students, this reinforces the fact that they will likely get a "bad grade", further digging the hole of hopelessness. The longer the student is taking the test, the better they are doing. So. students who want to get a "good grade" on the test will be anxiously anticipating each question, self-assessing how they are doing on the tests. This becomes less an opportunity to demonstrate knowledge, and more a psychological "man vs. test" scenario.

Second, most of these CA tests are being made by companies who exist to make a profit. The testing industry is a multi-billion-dollar industry. The largest of these companies is Pearson that now owns the national lion's share of textbooks, student information systems, assessments, and other hugely profitable products. While NWEA is a non-profit organization, it is partners with Pearson. I have learned that when you want to know who has the power and is driving the bus? Look for the money trail.

Third, and most concerning, students do not have the ability to review their answers on a CA test. When they have finished with an item, they go on to the next. This flies in the face of what we know about learning and demonstrating knowledge. How many times have you been allowed to go back and make revisions on high stakes assessment? Even with the GRE (paper and pencil still in 1996), I had an opportunity to go back and review my answers. For these types of assessments one corrected answer can make a huge difference in the overall score. For any paper I write, I go back and make revisions and edits until I feel that it is 'ready.' Even then, I find mistakes afterward. The only time I have been unable to go back and check my work is when I was subjected to yearly standardized tests as a child. The final product or performance is an end result of revision, editing, and reviewing.

Our children must understand that self-correction is an indication of learning. With the stakes placed on these tests, students' inability to review their answers, correct mistakes, and make revisions fails to give us information about how students' thinking has changed. Thoughtful and careful test-taking is difficult, if not impossible.

Next up? Standardized testing.