Effective Learning Analytics

The University is introducing a new project which aims to help students reach their full academic potential while studying at Exeter

Undergraduate and taught postgraduate students: Click here to view or change your consent to the use of your data to build and populate your My Progress dashboard

Our ambition

The University’s Vision (2015) is stated as follows Our driving ambition is to be a Global 100 research leader which creates graduates of distinction within a community of the most talented and creative minds’. Ensuring that we support and stimulate all of our students from their diverse backgrounds to achieve their potential is integral to fulfilling this vision.

This project seeks to enhance our capacity to do this through development and implementation of research-informed practice using the power of learning analytics that has the following aims: to enable individual students to monitor and benchmark their own performance; and, to enhance information provision to academic and professional services staff responsible for supporting students in their academic journey. 

In fulfilling these aims, the Effective Learning Analytics project will contribute to the delivery of the Education Strategy and I will monitor progress on it through the University Education Executive. I encourage students and staff to engage with the project to ensure that it delivers what we need. Please contact Sue Milward if you would like further information.

Professor Tim Quine
Deputy Vice-Chancellor (Education)

The Effective Learning Analytics project will use existing data about students’ academic progress and will present it in an accessible format that will enable them, and their tutors, to see how they are doing academically and whether there is any additional support we can put in place to help them succeed in their studies.

As part of this project we will consider the most useful data to present, design data presentation, develop ways to analyse the data and check that our guidelines for handling student data is robust and will ensure it remains confidential and secure.

‌Project Details

The project is sponsored by the Dean of Students, Wendy Robinson and is overseen by a project board with the following membership:

Role

Name

College/Service

Chair and Sponsor

Wendy Robinson

SSIS

Project Manager

Sue Milward

ESE

User (Professional staff)

Paul Blackmore

ESE

User (WP staff)

Karl Devincenzi

ESE

User (Exeter student)

Bryony Loveless

Guild

User (Penryn student)

Dean Pomeroy

FXU

User (STEMM staff)

Crawford Winlove

UEMS

User (HASS staff)

Marion Gibson

HUMS

Supplier (Exeter IT)

Riichard Autherson

Exeter IT

Supplier (BI)

Mariano Sanchez-Luppi

Policy, Planning and Business Intelligence

User (HASS staff)

Barbara Powell

CLES

User (Professional staff)

Roni Roberts

TQAE

The project will be delivered by 5 workstreams:    

1. Technical (Richard Autherson, Exeter IT)

2. Dashboard Design (Dr Stuart Robinson, Lecturer in Management, MBA Director, Business School)

3. Data Modelling (Dr Hywel Williams, Lecturer in Systems Ecology, CLES)

4. Interventions (Dr Alice Osborne,  Lead for Remediation, Lead for Professional Practice Groups, UEMS)

5. Evaluation (Dr Jo Smith, Associate Professor in the School of Psychology, CLES)

6. Innovation (lead TBA)

 

Workstreams are delivering the following milestones during phase 2 of the project:

Output

Workstream

Start

End

Developing models: data mining, correlations and forecasting

Data Modelling

Aug 17

Jan 18

Refine intervention processes and recording for diagnostic dashboards

Interventions

Aug 17

Feb 18 

Review suitability of existing technical architecture

Project

Mar 18

Apr 18 

ELA architecture defined

Project

Apr 18

Apr 18

Future LA possibilities plan

Innovation

May 18

Jul 18

Staff and student diagnostic dashboard design 

Dashboard Design

Oct 17

Feb 18 

Staff and student diagnostic dashboard development

Technical

Nov 17

Jun 18

Staff and student diagnostic dashboard pilot launch

Project

Feb 18

Mar 18 

Staff and student diagnostic dashboard launch

Project

Jul 18

Jul 18

Refine and re-publish ethics code of practice for learning analytics

Project

Dec 17

Jul 18

Business case for further data collection

Project

Apr 18

Jul 18

Data sources loaded into the BI warehouse

Technical

Aug 17

Jul 18

Project Evaluation

Evaluation

Aug 17

Jul 18

 

 

Get Involved

There are many opportunities to get involved as the project progresses.  Students are invited to join project teams to provide valuable input and also to enhance their employability skills by engaging with a project which is very relevant to education, business and industry.

Contact the Project Manager, Sue Milward to find out more or to get involved.

Latest News

September 2017

The project has now received funds to run for a third year to July 2018.

Highlights:

  • The My Progress student dashboard in iExeter has launched with displays of student grades and employability data. The complementary staff dashboard is in final testing, and will be made available to all academic staff during October as a tool to support personal and academic tutoring.
  • At registration this year, undergraduate and postgraduate taught students are being requested to give their active consent to the creation of a personalised dashboard and corresponding tutor interventions. Over 80% of all students have consented and almost 90% of first years. Students may opt-in or opt-out at any time during the year. The consent process is supported by a promotional website.
  • The Code of Practice for Learning Analytics has been endorsed by the University Ethics Committee and is available for viewing at Learning Analytics Code of Practice
  • The data mining sub-project led by Dr Hywel Williams is ongoing with final results expected at the end of the calendar year.
  • The focus of the project for the next year will be on developing the dashboards further (based on the outputs of the data mining research) and producing guidelines for staff on the use of the dashboard data to support targeted conversations with students.
  • The project manager is currently meeting with DOEs to discuss requirements for discipline specific dashboards and is seeking volunteers to pilot the next phase.
  • The project has submitted an application to the Reimagine Education awards 2017.

 

June 2017

Since February the project has made much progress, continuing to develop and improve the iExeter student dashboard, evaluating current practise at Exeter and disseminating data modelling research at a number of conferences.

The iExeter dashboard has continued to prove popular, with new updates gaining very positive feedback from students at Streatham and Penryn during student workshops. These new developments, presenting employability progress like eXfactor and careers advice attendance, will be released soon. Development of the complementary staff dashboard has also nearly been completed and will be released shortly.

The data modelling workstream have disseminated early results, with two papers being accepted for publishing in journals. The research team have also presented their findings at the internal education conference and a national Learning Analytics Jisc event here at Exeter, as well as at the LAK conference in Vancouver.

Importantly, to be in line with the new General Data Protection Regulation and to further the project, there will be a section on learning analytics in this September’s student registration. We are developing our Code of Practise which will support our use of learning analytics as the project progresses. Currently, a short animated video is in production that will accompany registration to inform students about the project and the benefits of signing up

The project has also benefited greatly the past few months from the work of our Student Campus Partner interns. Emily Hughes has been assisting the data modelling workstream as they have been conducting a longitudinal student engagement survey, Natalia Oleynik has been instrumental in designing the iExeter dashboard and Courtney Buckler has completed an audit of all current interventions in place at the University.

 

Feb 2017

Lots has been happening with the project since the last update in October.

The undergraduate iExeter MyGrades dashboard was released in October showing previous year’s grades and enhanced in January to show all years grades. The dashboard has proved extremely popular with students. Within the first day after launch there were almost 2,500 views, and in the first month almost 25,000 views. Access peaked on launch and then again when an iExeter alert was sent to all students. The complementary staff dashboard is almost ready for release.

Development of the version 2 student and staff dashboards has now started. These will provide useful snapshots of employability engagement in iExeter and Cognos and are due for release in April.

The University data warehouse has now been populated with over 600 validated, and cleaned data fields which are available for use in current and future dashboards.

Research into the predictors of academic success at Exeter is well underway. The research team are mining multiple data sources to look for patterns of engagement and have recently launched a longitudinal student engagement survey to validate the data findings. The team will be presenting their findings to a Jisc conference hosted in Exeter on 22 Feb and the international LAK conference on Vancouver on March. The project is due to complete in July 2017.

Earlier research seeking evidence of the impact of evaluation and effectiveness of Learning Analytics has been submitted for publication in Computers and Education, while new research into the impact of student interventions is about to be commissioned.

Evaluation of the success of the project so far is being undertaken by the project team and workstream leads. There are already many project outputs which have already had research and business impact.

Work on an Ethics Policy and Student Guide to Learning Analytics is ongoing. It is imperative that we are governed by clear legal and ethical guidelines as we carry out the project, and the new General Data Protection Regulation (GDPR) which comes into force in May 2018 will guide our deliberations.

Finally, we are now undertaking detailed planning for phase 2 of the project which will allow extra time to fully realise the benefits of learning analytics to support all our students who may require extra support to reach their full potential. The extension will also offer the opportunity to explore how learning analytics and big data can be used to demonstrate clear evidence of learning gain and added value from the HE experience

 

October 2016

A student grades dashboard will be released in iExeter in October. It will be visible to all undergraduate students, showing previous year’s module and assignment grades.

In addition, pilot disciplines undergraduates will be able to see their current year’s grades when they are available. If this is successful during term 1, it is planned to release to all undergraduate students during term 2.

A complementary Cognos report is being developed for personal tutors. This will initially display grades data equivalent to that shown on the iExeter dashboard. To support the dashboard, staff information is also being developed to support staff who may receive queries from students. It is planned to present this to the next Senior Tutor Forum meeting

Student feedback will inform further dashboard developments.

Detailed planning for phase 2 of the project is now underway. The aim is to provide a sophisticated solution which will support students to make informed choices and thus reach their full academic potential.

 

May 2016

Detailed planning for the phase 1 student and staff dashboards is well underway. Assignment grade data will be displayed in a graphical format to students via iExeter and to staff via Cognos reports by the start of the next academic year. Phase 2 of the project will deliver a more sophisticated solution which will support students to reach their full potential and provide staff with a valuable decision support tool.

A proof of concept staff dashboard exercise has now completed in Sports and Health Sciences and Penryn Geography. It has recommended that the University does not expose staff and students to predictive analytics until there is evidence and confidence that the predictions will be accurate. It also concluded that the Tribal product used was not a good fit for Exeter’s needs.

Student focus groups and feedback have provided encouraging evidence that students feel they would benefit from a dashboard which enables them to follow their academic progress.

 

April 2016

The proof of concept in Sports and Health Sciences and Penryn Geography has now finished.  Student focus groups and staff feedback questionnaires have provided valuable data which will be collated into a report available at the end of April. Preliminary recommendations are expected to endorse the decision not to attempt to predict student outcomes and to stress the importance of good dashboard design.

Planning for the development of the phase 1 student and staff dashboards is well underway. Working with staff and students we have now agreed that assignment grade data should be displayed in a graphical format to students via iExeter and to staff via Cognos reports. For the first time, staff and students will have a single view of their component marks in one easily-digestible format. We are now starting undertake detailed tasks to ensure that the systems are ready for use at the start of the next academic year.

 

February 2016

The project is well underway with all workstreams actively engaged.

The project will be presented to an ASPIRE seminar on Friday 4 March where academic staff will have the opportunity to discuss the project and how analytics can support them.

The proof of concept has now started with colleagues in Penryn (Geography) and St Lukes (Sports and Health Sciences) now reviewing the demonstration dashboard. Their feedback will inform the design of the phase 1 dashboard which will be available to all students by September 2016. Students are also involved in focus groups to feedback on the sample dashboard.

Work has started on desk-based research  to  produce a  summary list of data and behaviours required to predict behaviours and outcomes with an indication of the relative impact,  and also development of a report reviewing evaluations of the effectiveness of learning analytics initiatives and details of how the effectiveness may be measured.  This work will inform the data collection for the dashboards and also be invaluable as we evaluate the impact of the project.

Following on from the development of a draft ethical code of practice last summer, the ethics workstream lead, Alice Osborne is developing detailed processes to underpin the overarching principles in partnership with student volunteers. Updates to The ‘Data Protection Information for Students’  for next year have been approved by the Records Manager, and will allow the University to develop a widening participation ‘flag’  to be accessible on the dashboards to those staff who ‘need to know’.

It is currently proposed that staff dashboards will be made available as a decision-support tool only  to those academic and professional staff who need access to individual and aggregated data.  Working closely with the proposed Academic Tutor Framework project and professional staff,  the ethics and the dashboard design and selection workstreams will  develop a set of user guidelines which promote best practice and include ethical and Data Protection considerations .

A new workstream Dashboard Selection and Design has been formed under the leadership of Stuart Robinson from the Business School. Stuart was formerly a CIO and teaches  a Masters module ‘Practical Insights into Data and Analytics’  and so is well qualified to take this workstream forward. The workstream will ask suppliers to present their dashboard solutions to student and staff focus groups in order to identify the best supplier and best dashboard for Exeter. This will form the phase 1 dashboard which will use existing technology to initially display student progress over time.  Selection of a product to deliver a phase 2, predictive dashboard with extra data sources will commence early in 2016/17.