Next generation standardized tests face major technical barriers

In schools across the country students are taking new standardized tests. For many schools it will mean abandoning antiquated paper tests and replacing them with Internet based assessments. Last year we predicted that schools would encounter difficulty in administering these tests due to old hardware, lack of high-speed Internet access, and software glitches. Districts were able to avoid the worst case scenario of scrapping next generation assessments, but there were numerous reports of serious issues throughout the country.

Computer glitches

Serious technical glitches occurred in schools all over the United States. The most widespread testing problems occurred in Florida where more than half of the state’s counties encountered problems. The issues in the Sunshine State were likely made worse by the lack of in-state field testing. The American Institutes for Research (AIR) field tested the exam in Utah, which denied test administrators in schools the opportunity to document technical issues in a low stakes setting. In Wisconsin state leaders moved the test window back two weeks due to a bevy of technical glitches. The change impacts 375,000 children who can now take the test in between April 13 and May 22. The delay presents a major headache to educators who have to reorganize lesson plans. Ohio school districts had the option of administering their assessment online for the first time. Though the assessment was vigorously field tested there were 9,600 phone calls to Pearson the company that designed the test. Some educators reported difficulty logging onto the test application and others reported that the test application was buggy.

Cyber-attacks

Schools in Florida and Colorado also had to deal with cyber-attacks. In Florida there was an attempt to “make the [test] website unavailable by flooding the firewall with nonsense connection.” Law enforcement officials are investigating that attack. A school district in Colorado Springs experienced a denial of service attack. The identity of the attackers and their motive is currently unknown in both cases. It’s possible that the timing of the hacks and the standardized testing is purely a coincidence. But, it highlights the general need for better cybersecurity at schools.

Outdated Operating Systems

According to the Smarter Balanced Assessment Consortium’s report on the success of their field tests many schools were using antiquated operating systems. Last year 13 percent of computers used to take the field test had Windows XP installed. Three states that participated in the field test had between 21 and 30 percent of devices that were using the old operating system. This is a critical issue because Microsoft ended support for Windows XP in April 2014, which meant that it stopped issuing updates. Any school still using the legacy operating system is running a serious risk of having their systems exploited, which could also impact the administration of online assessments.

Internet connection speeds

Lack of affordable high-speed Internet is an ancillary issue that could compound technical glitches. Internet based applications are more likely to be error prone if minimum bandwidth levels are not available. According to the State Educational Technology Directors Association the recommended download speed for online multiple choice assessments is 64 Kilobytes per second for each student. There is a dearth of data on Internet speeds at schools, but the available evidence suggests that about 1 in 10 schools are unable to administer the current generation of standardized tests online.

It’s reasonable to expect that test writers will issue patches that will lessen the number of glitches in future years. But, issues related to old hardware and slow Internet connection speeds will continue to present challenges. One issue that receives little attention is the impact that technical glitches have on test validity. If glitches cause test systems to inaccurately score online exams then it could represent a serious problem with the current generation of assessments. However, it’s currently not possible to know the scope of this issue. The gravest threat to the future of advanced assessments is the gap in technological capacity between school districts. One virtue of technology is the beneficial network effects it can generate with relatively low costs. Students will lose out on these benefits if the nation’s leaders are unwilling to increase investments in hardware and Internet access in the nation’s schools.