My Rasch Error

John pointed out a mistake in my assertion that the number of items correct doesn’t matter in the score. He is correct.

In your most recent response for item 4, you wrote:

"For a Rasch calibrated test, the student’s score doesn’t depend on the number correct but where on the scale he falls as a result of his responses to the items.

"If he got a lot of hard item correct and no easy ones, he would score higher than another student who answered more items correct but didn’t get as many difficult items correct (although the test results would show a serious misfit for this student meaning something was wrong with the way the student responded).

"It doesn’t matter how many items but which items he answers correctly. We could give a short version of the test and still get the same Rasch score for a student because the items only serve to locate where on the scale the student is according to which items he answers correctly. The scale has nothing to do with the number correct."

Below is something I copied into a Word file from the ODE web page a couple years ago and I did not save the link. It seems to me that what you wrote above flatly contradicts the table below. If not, I am really confused. John

Sample Test, Benchmark 2, Grade 5
2001-03

CONVERTING TO A RIT SCORE—

No. Correct RIT
1 169.0
2 178.8
3 184.9
4 189.5
5 193.3
6 196.6
7 199.6
8 202.3
9 204.8
10 207.2
11 209.5
12 211.8
13 214.0
14 218.4 o
15 220.7
16 222.9
17 225.3
18 227.8
19 230.5 oo
20 233.3
21 236.6
22 240.3
23 244.8
24 250.8
25 260.6
Recommendations for Level Test Placement:
o Likely to meet Benchmark II Standards
oo Likely to exceed Benchmark II Standards

I was flat out wrong when I said the number of items correct doesn't directly convert to a score. In the examples I gave, the two students with the same number correct would have the same score. The one student would have a large misfit statistic showing his answer pattern didn't match the difficulty sequence of the items. Sorry.

I put the question to Trevor Bond who wrote the book Applying the Rasch Model: Fundamental Measurement in the Human Sciences. He replied:

N - the number right - is the sufficient statistic; all convert to the same logit value
Fit statistics reflect the pattern
good fit means the pattern is close enough for Rasch that the N=logit conversion holds

So, John, you are correct. This new way of thinking about test construction is not easy for me, either.