A look at Academic Assessment Day

The idea came from discussions in a Road Map Strategic Priority 2 (Learning Community) meeting. How could Binghamton University know which students were experiencing academic struggles early enough in the semester to provide outreach and support while there was still time to make a difference?

“We’ve been trying to figure out the magic formula for a while,” said Don Loewen, vice provost for undergraduate education and enrollment management. “As it turns out, there isn’t one. We have great students but something goes wrong. They might be homesick or have roommate issues. There are many non-academic as well as academic reasons that cause things to go wrong. But if we can’t predict these issues up front, what is the next best thing to do?”

The most significant predictor of any student’s overall academic success at Binghamton is their first-semester GPA, Loewen said. “It’s become a solid, leading indicator of the likelihood that a student will come back for a second year, or will graduate in four years,” he said.

So, finding out about students struggling after mid-terms or at week 10 of the semester, way past the add-drop deadline, can be too late. “To influence the first semester GPA, we need to get as far upstream as possible,” Loewen said. “And the only way we can do that is with faculty help.”

When the Road Map Steering Committees were asked to submit proposals for the 2021-2022 year, SP2 faced an immediate and urgent challenge, said Paul Schleuse, associate professor and chair of the Department of Music and a member of the SP2 committee. “Due to the COVID-19 pandemic, most of the incoming first-year class had endured a highly disrupted and stressful final year of high school, and moreover, had been admitted for the first time through a test-optional application process,” he said.

“This raised concerns that students might have greater difficulty making the transition to college and may have lacked the preparation to be academically successful here,” Schleuse added. “Plans were in place to provide additional academic and other support services to this class, but there was also a need to identify at-risk students quickly enough that these interventions could make a difference.”

“At my undergraduate institution, first-year students received a mid-term grade report in their first fall semester (a tradition known as “Pumpkin Grades”),” Schleuse said. “I suggested something along these lines, and the basic idea was adapted into the Academic Assessment Day that was introduced this fall.

“We wanted to encourage faculty of lower-division courses especially to include at least one assessment measure (a quiz, paper or other assignment) that could be completed and returned by the end of the fifth week of the semester, so they could identify students who were struggling and students could have a sense of their own progress,” Schleuse said. “This process worked in conjunction with a new reporting system for academically at-risk students, recognizing that both academic and non-academic interventions could be helpful.”

Faculty used Advise, a new software program that allows them to send alerts of varying levels — and kudos — directly to students.

“Instructors took notice of Academic Assessment Day,” Loewen said. “It was a target date (Sept. 27), where we asked faculty to help us identify students where they had concerns. Our ultimate objective was to reach out to these students to help them be successful this semester and then through to graduation.”

In concert with the Advise alerts, the University also beefed up tutoring this semester, partnering with University Tutorial Services for one-on-one tutoring as well as expanding the number of courses offering tutoring, and increasing supplemental instruction to provide not just outreach but actual support for students after that outreach

“We now have an entire ecosystem where an instructor can go back and see what happens when an alert is raised. Advisors can see all the interactions and that allows them to take further action. If a Watson student has alerts raised in some Harpur courses, Watson advisors can still reach out to them and see what’s going on,” Loewen said. “It increases the opportunities for engagement. You can send texts and emails and follow up if there is no response. It’s an effective way to manage the interactions you have with students.

“And the power of it is when you have multiple points of reference for a student, we can see if the problems are isolated to one course or are repeated in several courses,” he added.

But instructors shouldn’t forget the students who are doing well, either, Loewen said. “Kudos are one of the things I hadn’t thought about having an impact. We’re so focused on saving the students in difficulty but we have to remember to tell students when we notice them working hard and doing well, too. That’s powerful.”

According to Amber Stallman, assistant provost for student success, as of this writing, alerts have been raised in well over 250 courses for students at all levels, not just first-year students. Over 2,000 of the alerts were kudos to students for doing well, but 942 alerts were raised at the medium level for things like attendance concerns, low quiz scores or missed assignments. Alerts of high concern were raised for 1,264 students, who require serious intervention if they are to be successful in the course.

The good news is a lot of instructors really responded positively, and we’re still getting alerts, Loewen said. “We want to develop a culture where instructors are always paying attention and raising these alerts. It’s never too late, but we hope they raise them as early as possible.” (The system doesn’t close so faculty can raise alerts at any point in the semester.)

Additionally, as alerts are raised, academic advisors and others review them and reach out directly to the students — texting has seemed to have the most success — to offer support, especially to those who received multiple alerts across multiple courses.

“We can intervene and have a conversation with the student,” Loewen said. “If it is an academic issue, tutoring or advising can help. The challenge is that it has to be a really well-coordinated, campus-wide initiative. We have to work closely together. For example, international students can’t drop too many credits and Excelsior students can’t lose their scholarships. Financial aid, ISSS, academic advisors, the Student Transition and Success team and others are working together to review the alerts and provide the support to students that will make a difference.

“We would really love to see a campus culture where this becomes a matter of course for instructors and students,” Loewen said. “’I want to get my alerts and kudos out.’ The more instructors who participate, the better we’ll be at it.”

It’s likely that academic assessment will remain on the academic calendar moving forward, but the single day idea is being reconsidered because students faced a lot of exams and coursework at the same time. “But it’s a signal that this made a difference,” Loewen said. “People looked at it and took it seriously. We’re looking at dispersing it over more days to spread things out for students.”