Item analysis has been a part of jMetrik since its inception. It is available for variables with item scoring information. See Item Scoring in this guide if you need to complete item scoring before running an item analysis. You can conduct an item analysis with data from binary (e.g. multiple-choice) and polytomous (e.g. constructed response) test items. Item analysis provides classical item statistics and reliability estimates for your test items. In particular it provides item difficulty (i.e. pvalue, the mean item score), item discrimination (i.e. item-total correlaiton), and a distractor analysis for each item. Reliability estimates include KR-21, coefficient alpha, Guttman’s Lambda 2, Feldt-Gilmer, Feldt-Brennan, and Raju measures of internal consistency. Reliability confidence intervals and standard errors of measurement are also provided.
- Click Analyze > Item Analysis to start the dialog.
- Choose the items you would like included in the analysis. (Note: If no variables appear in the dialog, you have not provided item scoring. See Item Scoring in this guide for more information.)
- You can run the analysis with the default options, or change them to suit your needs. The available options are:
- Compute item statistics – Select this option to include item statistics. If not selected, only reliability estimates will be computed.
- Item deleted reliability – Choose this option to see a table with item deleted reliability estimates. The table will show reliability estimates when each item has been omitted from the analysis. This helps you choose which item to remove to improve reliability.
- All response options – If selected, the output will show a complete distractor analysis. If not selected, only item difficulty and discrimination for the correct answer (binary item) or overall item score (polytomous item) will be computed.
- Listwise deletion – If selected, an examinee will be omitted from the analysis if missing data on any item. This could result in large amounts of missing data and it is not recommended. If not selected, missing item responses will be scored as zero points and included in the analysis.
- Correct for spuriousness – If selected, the item-total correlation (or distractor total correlation) will be adjusted to account for inclusion of the item score in the test score. That is, it will produce an item-remainder correlation, where the remainder is the test score without the studied item. If not selected, no adjustment will be made.
- CSEM – Select this option to compute the conditional standard error of measurement. A table will be added to the output with this informaiton.
- Show headers – Select this option to add a header before each item in the output.
- Unbiased covariance – If not selected, correlations and standard deviations will use a biased estimator (n in the denominator). If selected, they will use an unbiased estimator (n-1 in the denominator).
- Cut scores – Type one or more cut scores (on the raw score metric) separated by a white space to have jMetrik compute decision consistency indicies. A table with Huynh’s raw agreement and kappa statistics will be displayed when cut scores are provided.
- Item-total correlation type – Select Pearson correlation to have jMetrik compute Pearson correlations (point-biserial for binary items, Pearson correlation for polytomous items). Choose Polyserial to have jMetrik compute Polyserial correlations (biserial correlaiton for binary items, polyserial correlaiton fo polytomous items).
- If you would like to save item statistics in a new database table, click the Save button and type a name for the new table.
- Click the Run button to execute the analysis.