Calculate Statistics item analysis is a particularly valuable tool that can be used to to eliminate ambiguous or misleading test items and to increase the quality of an assessment as a whole.
This article will provide an overview of the Item Analysis report Calculate Statistics feature.
Calculate Statistics is available for district-level Answer Key Only (AKO), Express, ExamView and Item Bank assessments. The assessment must contain a minimum of ten items and the Administration Window must be closed. District administrators can grant permissions for viewing and calculating statistics to district, school, and teacher-level roles via the View and Update permissions for Statistics Data from the System Admin tab of role permissions.
The statistics that will be calculated are Mean, Standard Deviation, Cronbach Alpha, Standard Error of Measurement, P-value, and Point Biserial.
When Classical Statistics is enabled in District Defaults, in order for the Calculate Statistics function to be available for a user in the assessment manager for district, school, or teacher-level assessments, ALL of the following conditions must be met:
- Statistics Data permission: Update permission enabled for the user role
- Status of assessment: The status of the assessment must be In Progress or Closed.
- Assessment Type: The assessment must be Answer Key Only, Express, Item Bank, or ExamView
- Number of student scores: At least 25 students have taken the assessment.
- Statistics: The classic statistics of the assessment has not been calculated before
- Number of Items: The AKO, Express, IB, or EV assessment must have at least 10 items.
If 2PLM and/or 3PLM IRT statistics is enabled in District Defaults, then IRT-based statistics will also be calculated if the assessment has 100 or more student results available. Otherwise, if there is 25-99 student results then only Classical Statistics will be calculated for the assessment.
1. Permission to Calculate Statistics
1. From the Top Menu Bar Grid icon, select the Assessments tile.
- Or, from the Launchpad, select the Assessments flip card to reveal the reverse side available options. Select Manage Assessments.
2. In the Roles tab of the Admin module, locate the district user Role Name. Select the down arrow under the Functions column /to the right of Role Name and select Edit.
3. Select the System Admin tab.
4. In the Permissions table, select View and Update checkboxes for Statistics Data.
5. Select Save.
Note: In order to enable the Calculate Statistics for a user Role, the View Statistics permission, as well as the Data View > District Summary Level Data View permission must be enabled.
2. Request Statistics for a District Assessment
1. From the Top Menu Bar Grid icon, select the Assessments tile to open the Manage Assessments module.
2. Find the closed District Assessment for which the statistics will be calculated.
3. In the Functions column to the right of the assessment name, select the down arrow, and select Calculate Statistics.
4. A message will appear confirming this action. All CR items must be scored first before calculating statistics. Select OK to continue.
5. From the Top Menu Bar Grid icon, select Reports to navigate to the Report Manager. Then select the Item Analysis Report.
6. Check the box next to Statistics and/or IRT to view them.
Statistics shown are: Mean, Standard Deviation, Cronbach Alpha, Standard Error of Measurement for the entire test, and P-value and Point Biserial for each item.
Statistics Descriptors
- Mean: Average score
- Standard Deviation: the amount of variation of scores, relative to the mean
- Cronbach Alpha: a measure of internal consistency; how closely related a set of items are as a group
- Standard Error of Measurement: a measure of the precision of the assessment
- P-Value: a measure of item difficulty; ranges between 0.0 and 1.0; the higher the value, the greater proportion of students answered correctly, thus an easier item.
- Point Biserial: a measure of item reliability; ranges between -1.0 and +1.0; correlates student scores on one item with scores on the test as a whole; the closer to 1.0, the more reliable an item is because it discriminates well among students who mastered test material and those who didn’t.
Admins have the ability to set Classical, IRT 2PLM, or IRT 3PLM as the District Default(s).