For those who might not be familiar with it, EDUCAUSE is a nonprofit association whose mission is to advance higher education through the use of information technology. The EDUCAUSE Annual Conference claims to unite “the best thinking in higher education IT.” EDUCAUSE 2013 was held in mid-October, 2013, and I was there to investigate learning analytics.
Analytics is the use of data, statistical, and quantitative methods, and explanatory and predictive models to allow organizations and individuals to gain insights into and act on complex issues. The use of digital tools, especially Learning Management Systems (LMS), like Blackboard, for academic data and Student Information Systems (SIS) for demographic data, can create mounds of digital data that could be mined for discovering trends or predicting outcomes. Examples:
- Marist College is developing a predictive model using Banner (SIS) and Sakai (LMS) to deliver intervention notices to identify students unlikely to pass a course. A model was built using grade book data across a broad set of courses. They built their own system out of mostly Open Source tools. They found the most powerful predictor to be student’s GPA. Presentation.pptx [3 MB, Powerpoint slides].
- University of California Chico built a system from server log files and Excel, Tableau, Stata, and SPSS and looked at one large course (373 students). They found LMS usage to be the best predictor of success (not GPA), using these LMS usage variables: total course website hits; total course “dwell time”; administrative tool hits; assessment tool hits; content tool hits; and engagement tool hits. summary of results [455 KB, PDF]; presentation slides from EDUCAUSE 2013 [3 MB, Powerpoint slides]
- University of Kentucky uses a hardware “appliance” from SAP (HANA) to look at data in near realtime, push out administrative reports to administrators, and “how am i doing” reports to students via a custom mobile application. Academic advisers get an iPad application that compiles advisees’ data, giving both advisor and student a better idea of where they are and where they are going. Using Groundbreaking Analytics and Fast Data [7 MB, Powerpoint slides]
- South Orange County Community College District built the mobile app, “Sherpa,” a recommendation engine similar to Netflix or Amazon that helps students choose courses, services, and get information based on previous enrollments, major/minor declarations, and grades. It pushes out warnings and reminders to students via email or text message. Powerpoint slides.
- Coppin State University implemented Blackboard Analytics for Learn, providing a slew of dashboards for deans, chairs, faculty, and students using data from the Blackboard Learn LMS alone. Mesa Community College has taken it one step further, using Blackboard Analytics to also ingest SIS data. University of Maryland, Baltimore County is using Blackboard Analytics for Learn to explore the LMS in much finer detail and assess the impact of faculty course redesign training.
Barriers? Sure. Analytics are hard. The people who developed Sherpa called in three outside mathematicians to help design their statistical model. Kentucky hired three PhDs. Analytics require buy-in and many of the presenters were CIOs, provosts, presidents, or vice-this-or-thats. There is a lot of missing data (e.g., classes that don’t use an LMS), and a lot of inconsistent data (e.g, variance in how faculty use LMS gradebooks). Statistical models are still in an early stage of development and proprietary software, like Blackboard Analytics, is expensive.
For more on learning analytics, visit The Society for Learning Analytics Research (SoLAR), an interdisciplinary network of leading international researchers who are exploring the role and impact of analytics on teaching, learning, training, and development.