1.

I need help with jMetrik. Who can I contact?
With the release of version 3, free support is no longer available. Users are encouraged to purchase a support plan or register for an online short course. Support plans are available at a minimal cost and they cover support for a 12 month period. Online short courses are currently offered in August and January of each year. If neither of these dates are convenient for you or your organization, workshops are also available at other times for a fee and they may be offered on-site with the inclusion of travel expenses.

2.

Can I enter data directly into jMetrik?
No. jMetrik only provides a view of the data. If you must enter data manually, use a program like OpenOffice Calc, http://www.openoffice.org/ (it's a free download). Save the file as a CSV (comma delimited) file. Import the CSV file into jMetrik. The first row of data in the CSV file may contain variable names, but variable names are not required. You should have one row of data for each examinee and one column for each variable.

3.

What is the maximum sample size in jMetrik?
There is no sample size limit in jMetrik. We have tested jMetrik with data files as large as 1,000,000 examinees using the default memory allocation of 256MB. If you have data files larger than 1,000,000 examinees, you may want to increase memory allocation to improve performance. See the Tips and Tricks page for instructions on increasing the memory allocation.

4.

What is the maximum number of variables allowed in jMetrik?
The maximum number of variables in each table is 1,012. However, you can have an unlimited number of tables. If you have a data file with more than 1,012 variables, consider dividing it into multiple files and import them as multiple tables.

5.

How can I learn more about jMetrik and its capabilities?
We are now providing online workshops and videos. These workshops cover the theory of all the methods included in jMetrik and an explanation of all of the options found in jMetrik and a description of the output. To maximize the utility of jMetrik, please register for one of our workshops. Workshops are held in January and August of each year. All you need to participate in an internet connection. For more information, see http://curry.virginia.edu/community-programs/conferences/jMetrik.

6.

How can I allocate more memory to jMetrik?
jMetrik requires 256MB of dedicated memory to run. It will rarely need this much memory, but you may want to allocate more memory to improve performance and reduce the amount of time jMetrik spends in garbage collection. You can increase performance by modifying the jMetrik.jvmoptions file before starting jMetrik. This file is located in the place where you installed jMetrik. For example, on Windows, the location is C:\Program Files\jmetrik. Open jMetrik.jvmoptions in a plain text editor. If you would like to increasse the maximum memory allocation from 256 MB to 512 MB, change the line -Xmx256m to -Xmx512m. Be sure there is a single carriage return after this line. Save the file and start jMetrik. Note that Windows users will need to edit this file by starting Notepad in administrator mode. Then, use Notepad to open and edit the jMetrik.jvmoptions file.

7.

Can I save a graph produced by jMetrik?
You can save all graphs created in jMetrik. After creating a graph, right click (or CTRL click on a Mac) the graph to display a context menu. One of the options is for saving the graph as a PNG file. (Other options allow you to customize various features of the graph.)

8.

What is the information in the DIF analysis output?
There are several columns in the DIF analysis output. The first column "Item" is the name of the item. The second and third columns are the Mantel-Haesnzel chi-square and associated p-value. Note that jMetrik uses the Cochran-Mantel-Haenszel for stratified 2 x k tables. It is a generalization of the Mantel-Haenszel for stratified 2 x 2 tables and it works with binary and polytomous items. One difference between the Cochran-Mantel-Haesnzel for binary items and the Mantel-Haenszel is that the former does not use a correction for continuity. Therefore, with binary items the Mantel-Haenszel value reported by jMetrik may slightly differ from the Mantel-Haenszel reported by other programs.

The "Valid N" column lists the number of examinees involved in the Mantel-Haenszel statistic. Examinees that failed to provide a value for the DIF group code are deleted from the analysis. Also, any stratum table with only one examinee is not included in the analysis. As such, you could have several examinees eliminated from the analysis if you have several tables with only a single examinee. In this situation, try using the "Deciles" or "quntiles" options to preserve more data.

The "E.S. (95% C. I.)" columns list the effect size (E.S.) and 95% confidence interval for the effect size, respectively. For binary items, the effect size is the common odds ratio (COR). You can optionally convert this value to the ETS Delta metric. For polytomous items, the effect size is the standardized P-DIF statistic (sP-DIF).

The final column ("Class") is the ETS DIF classification level. The possible classifications for binary items are A, B, and C, while the possible classification levels for polytomous items are AA, BB, and CC. A and AA items show little to no DIF. B and BB suggest moderate amounts of DIF. C and CC items suggest a large amount of DIF. These classifications are a function of statistical and practical significance. The rules for binary items are:

• A item: (a) Chi-square p-value > 0.05 or (b) the COR is strictly between 0.65 and 1.53.
• B item: not and A or C item
• C item: (a) COR < 0.53 AND the upper bound of the 95% confidence interval for the COR is less than 0.65, or (b) COR > 1.89 AND the lower bound for the 95% confidence interval for the COR is greater than 1.53.
  

Given that polytomous items use a different effect size, the classificaiton rules are different. The rules for polytomous items are are based on dividing sP-DIF (the value in the output) by the item score range to limit values to the interval from 0 to 1. Call this new value sP-DIF*. It is not displayed in the output. This change allows the rules developed for binary item P-DIF to be applied to polytomous items. The rules for polytomous items are:

• AA item: sP-DIF* < 0.05
• BB item: 0.05 >= sP-DIF* <0.10
• CC item: sP-DIF* >= 0.10

Each DIF classification also includes a sign. A "+" sign (without the quotes) indicates that the item favors the focal group. A "-" indicates that teh item favors the reference group. More information about the DIf classification levels are available in the following articles.

Dorans, N. J., Schmitt, A. P., & Bleistein, C. A. (1992). The standardization approach to assessing comprehensive differential item functioning. Journal of Educational Measurement, 29, 309-319.

Potenza, M. T., & Dorans, N. J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation, Applied Psychological Measurement, 19, 23-37.

Zwick, R., & Ercikan, K. (1989). Analysis of differential item functioning in the NAEP history assessment. Journal of Educational Measurement, 26, 55-66.

We will be holding an online jMetrik shourt course in January, 2013. The workshop covers the theory for all of the methods inplemented in jMetrik. It also includes detailed information about data management and writing jMetrik commands. Participants have access to recorded training sessions for a month following the workshop. Details may be found here, http://curry.virginia.edu/community-programs/conferences/jMetrik.