Student Reviewers of Teaching Practice : Reflections on the Design and Experience of Participants

This case study focuses on the benefits and challenges of students and staff working in partnership to review virtual learning environments as part of a wider review of an academic course. The paper considers how it enables a more rounded view of the course to be gained and how students identified possible enhancements to online sites in their own departments from the experience. Most issues raised by students related to the organization of the sites, with differences between students and staff in both the approach they thought should be taken to this and the importance placed upon it. These different perspectives make dialogue and respect very important in this area of student-staff partnership.

These schemes vary in terms of the length of the collaboration, from semester-long at Bryn Mawr and Haverford Colleges to one-off observations at the University of Lincoln.The schemes also involve different aspects, from observing classroom teaching to helping to collect and analyse student feedback.Here we discuss our experience of students reviewing teaching practice at University College London (UCL).In this scheme, participating staff were observed for three hours' worth of classroom teaching, and students reviewed the layout and organisation of a virtual learning environment (VLE)-in this case, Moodle, an online learning management system that allows educators to create private interactive websites for the courses they teach-and an assignment brief.While in similar schemes, such as that of Edinburgh Napier University, the students were also asked to review material on Moodle (Huxham et al., 2017); there is little discussion of this in the literature.In this case study, we focus on the VLE aspect, considering the benefits of its inclusion in a student consultancy scheme as well as the issues it raised.
The review of a VLE was considered important in light of the Horizon report, which identified that blended learning is increasingly gaining traction in the higher education sector (New Media Consortium, 2017).The report notes that blended learning, when done well, can enable ways of learning that do not occur on campus, allowing for more personalised approaches.However, implementation of blended learning is likely to be hindered by a lack of understanding of students' expectations and skill levels with technology.For example, recent studies suggest that students' ability to use technology may not be as great as commonly assumed (Bennett & Maton, 2010).

THE SCHEME
The aim of the scheme was to provide a continuing professional development opportunity for both staff and students, with all participants gaining a greater insight into how students learn, enabling them to enhance their teaching and learning practices.Furthermore, students also gained insights into how staff approach their teaching and design of online materials.
Three principles inform the design: the first was that the observations should occur over a period of time and look at a range of practices.The argument by Gosling (2002) that we need to widen our definition of what is covered by peer observation of teaching made sense in a scheme designed to enhance the student learning experience.The digital infrastructure supporting a course is an important part of the learning experience.It is also a part that is often seen as less open to student partnership development work, as giving students editing rights can raise confidentiality issues.In this case, however, confidentiality was not an issue since the students were solely reviewing the sites from a student point of view (i.e., without editing rights).This meant that while the students were not working in partnership to directly enact change, they were working in partnership with the staff to explore the effectiveness of the teaching practices.Marie, J., Hansen, J., Hazen, M., Elliot, M.I., Piccoli, G., & Thorogood, J. (2018).Student reviewers of teaching practice: Reflections on the design and experience of participants.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3370127 This focus on partnership led to the second principle of the scheme, which was that the staff and student partners should all reflect on the element under review (classroom teaching, assignment brief, Moodle site) by conversing with one another.This emphasis on dialogue is in line with more recent work on peer observation schemes, which suggests that feedback on its own does not lead to improvement.The discussion of what the students and staff thought of the observed practice and why they thought it was more or less effective is essential because it enables critical reflection, exploration, and critique of the reasoning behind practice.Where this scheme differs is in believing that there is benefit to be gained from undertaking this dialogue with a focus on student experiences of practice as opposed to pedagogical theories of effective practice (Hendry & Dean, 2002;Byrne, Brown, & Challon, 2010;Peel, 2005).As students discuss a staff member's practice directly with them, power dynamics would come into play and could affect what the students feel able to discuss.To address this, we aimed to pair students with staff from a different department but similar discipline, so that the staff would not, for instance, be marking the students' work (see Huxham et al., 2017;Cook-Sather & Alter, 2011).In line with the literature (Peat, 2011), this should also make it more natural to discuss teaching approaches rather than content.One of the unintended benefits of students coming from a different department was that they gained examples of good practice that they could take back to their own departments-this was particularly the case for the VLE observations.
The third principle was that the student participants should provide a student perspective, rather than a pedagogical one.A key benefit of working with students is that they have recent experience of what it is like to learn at university and can inform staff how their learning and that of their peers are affected by different factors (Cook-Sather et al., 2014).In her capacity as director of the scheme, Jenny Marie (JM) provided a training workshop for the student reviewers to develop their observation skills and ability to facilitate reflective conversations, but they were not trained in pedagogy or good digital education design (see Huxham et al., 2017).During the workshop, students were given the opportunity to undertake a practice observation of a Moodle site, with the agreement of the Moodle course's owner, and to identify the main points they would want to discuss with him.
Participation in the scheme was entirely voluntary.All staff who applied were either given a place or, if they did not have classroom teaching that term, were guaranteed a place the following year.Students were selected on the basis of how far along in their degree they were and whether they were a good disciplinary match for the staff participants.If this information was insufficient to make a choice between students, selection was random.Priority was given to final-year students because they would not get the opportunity to participate again and were considered to have a lot of recent student experience upon which to draw.

REFLECTIONS ON TRAINING STUDENTS TO OBSERVE A VLE
When I (JM) facilitated the training workshop, I noticed three things regarding the VLE observation.First, the students provided comments on the course: some said that the course looked like it was well structured and interesting; others said that the amount of information was overwhelming and made them think that the course would be difficult or boring.At the Marie, J., Hansen, J., Hazen, M., Elliot, M.I., Piccoli, G., & Thorogood, J. ( 2018).Student reviewers of teaching practice: Reflections on the design and experience of participants.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3370128 end of the pilot year, student feedback led me to design observation templates for the scheme (the one for the VLE is provided in Appendix A) and from this experience, to include prompts around their learning or impression of the course.
Second, the students commented on the difficulty of navigating the Moodle course being reviewed.When I discussed this matter with the course leader, he explained that the layout on Moodle is designed to tackle a particular challenge within the course.This demonstrates the importance of dialogue and that contextual information can enhance observation.It has, thus, been hard to pinpoint when is the best time for the VLE observation to occur.Should it be at the beginning to enhance understanding of how it may shape student expectations, later when the reviewers better understand its relationship to the course, or is there sufficient benefit in each that it should be done at both the beginning of the review and again later on?
The third thing worthy of note is that the device used to access the VLE matters.During the training, the only internet-enabled device that many of the students had on them was their mobile phone.Their view of the Moodle course was very different to that of those using laptops or tablets and they were far more likely to be critical of a course that required a lot of scrolling.A number of students asked me which device they should use, to which the answer invariably was, "Which device do students use to access VLEs?"It could be argued that I should have asked, "Which device would students like to use to access VLEs?"However, we probably have to accept that VLEs are not designed to be viewed on mobile devices.

Melissa
My partnership was unique and unexpected as the lecturer I was paired with used British Sign Language (BSL) as his primary language.Due to the communication barrier, the VLE platform was implemented heavily since it bridged the verbal versus non-verbal communication gap.The platform was readily accepted by the students of the course, and enabled classroom discussions to be taken beyond lecture hours.My partner clearly cared about his students and their opinions, which was shown by how prompt his responses were to emails and the discussion questions posted online.Moodle was organized in such a way that all questions outside the classroom could be found on the platform, and any other questions that could not be answered from what was already online could be answered punctually by the lecturer himself.In this case, the VLE contributed towards more inclusive and effective communication.

Joe
The member of staff I observed used the VLE to facilitate flipped lectures, whereby resources are provided to students prior to the face-to-face teaching component to enable them to learn content before the classroom sessions, thus freeing up time within the classroom for interactive activities.Research has shown that students and staff enjoy the flipped lecture style, but students often want more structure to help ground this unfamiliar learning technique (Wanner & Palmer, 2015).Online resources are indispensable to the flipped lecture, but reviewing the staff member's use of the VLE showed me that structuring an effective flipped Marie, J., Hansen, J., Hazen, M., Elliot, M.I., Piccoli, G., & Thorogood, J. (2018).Student reviewers of teaching practice: Reflections on the design and experience of participants.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3370129 classroom is time-consuming.The resources for the flipped lecture were extensive and varied, including tools such as online quizzes, Lecture Casts from the previous year, as well as the occasional Dropbox folder for students to upload mini-assignments.Each of these activities had to be scheduled and reported to the students, as many of the online quizzes had deadlines for completion.Most importantly, the flipped lectures were consistent.They ran every week and the activities were discussed in every classroom session.Rather than being a refreshing diversion from the normal lecture format, they became part and parcel of the learning experience.The VLE was also used to help facilitate group work; it did not just contain instructions but also links to other virtual platforms (Skype and Google Hangouts) where group work could be done remotely.The VLE also included a discussion space for each group.Prior to this, I had only ever used the VLE as a clumsy repository for resources.My staff member showed me that group work was something that could be nurtured through the VLE, rather than something students are expected to do.As a teaching assistant, I am glad I was able to observe how the activities were communicated to the students and how they were received in the physical sessions.

Manuela
It was clear to me that my staff partner had put a good amount of effort into the VLE for the course.Although the site was straightforward, it helped to go through it with him so I could understand the rationale behind each element.My staff partner had organised the site so that there was a clear progression, which helped students know what to expect from the sessions and prepare their material in advance, especially as the classroom was a very interactive space in which they were asked to contribute a lot.The Moodle page's games section was specially designed to enable students to practise what they had learned in class while at the same time have fun.Being able to see who was playing gave the lecturer insight into the different ways the students were engaging with the course.

Commentary
Melissa and Joe's reflections show how the experience of the classroom and VLE are intertwined.It is not just a matter that students can tell a lot about the course from the VLE, but that it may not be possible to properly understand the classroom without also considering how the VLE enables students to prepare for the classroom or how the VLE supports student learning beyond the classroom.
Manuela's reflections focus in part on layout, which was one of the major issues for students with the sites that they reviewed.Other student partners reported that they had gained a better understanding of where to find materials on other Moodle sites.One said that after realising his staff partner had laid the site out like a book-he had expected it to be laid out as a web page-he was now much better at finding resources on other sites.Another student surmised that perhaps staff members make the layout sequential because they write a lot but students really want to see things at a glance, without having to scroll down.
Melissa and Joe's experiences of learning how Moodle sites could be effectively utilised was common among the student participants.One staff member reported that her student had Marie, J., Hansen, J., Hazen, M., Elliot, M.I., Piccoli, G., & Thorogood, J. (2018).Student reviewers of teaching practice: Reflections on the design and experience of participants.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3370130 noticed simple organisational things, such as the video of the lecture being on the Moodle site, and had taken that back to her department as recommended good practice.Another student participant was added as a tutor to the Moodle site.They noted how many tools were available, which contrasted sharply with their experience of Moodle sites being repositories for PDFs.They suggested to their department that the postgraduate teaching assistants be asked to use these tools to enhance the sites.It is not yet clear if this will be followed up, but the suggestion also recognises the lack of time that most lecturers have to dedicate to this aspect of their teaching.

STAFF PERSPECTIVES
Below we provide the perspectives of two staff members who participated with different student partners from those whose reflections are provided above.

Jesper
The questions my reviewer asked following the observations forced me to explain my rationales, and this process made me realise that certain areas were not as clear as I had thought.This led to concrete changes in how the sessions were structured and to a layout revision for the VLE.While the resources available on the VLE were seen as relevant and important, the layout meant that all the sessions and resources were presented as one long list, and though divided into sections, was not very user-friendly.Liaising with the learning technology team, we decided to change the Moodle format to collapsed topics.This allows the user to quickly gain an overview of the course, which is increasingly important as more and more students view online resources on smaller screens such as tablets or phones.Another suggestion concerned the PowerPoint slides, which were already made available on the VLE.I had planned to improve the presentations by deleting the hidden slides and reducing the number of references on specific slides, replacing these with a final list of references.The review, however, showed that these aspects were seen as valuable by students, who used the slides to revise and catch up, particularly when they began preparing their final assignment.The hidden slides were considered useful extra material and the references on specific slides made it easier for students to locate the material they needed.I therefore decided to keep both of these elements.

Jenny
One of the challenges I faced was that I was not the module lead for the course that was being reviewed and therefore did not have editing rights for the VLE.This made the discussions about the VLE frustrating and uncomfortable.My reviewer had a good eye for detail and spotted a range of minor mistakes on the VLE, which I did not have the power to correct.I found it frustrating to get such useful feedback and to only be able to pass them on to the module leader, with no guarantee that they would be acted upon.This also made me reflect on my role and responsibilities teaching the course.I had viewed my responsibility as ensuring that I was prepared for the classroom sessions and knowing what the students had to do on the VLE between sessions, so that I could signal this to them.I had not considered it my responsibility to Marie, J., Hansen, J., Hazen, M., Elliot, M.I., Piccoli, G., & Thorogood, J. (2018).Student reviewers of teaching practice: Reflections on the design and experience of participants.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3370131 review the site and to ensure that it was up to my standards.I probably would not change this approach because I have to prioritise my work load.Although I am not responsible for the site, I am more aware of how its quality affects the experience of students in my classroom.

Commentary
Jesper reports both making changes and not making planned changes as a result of the review.In general, staff reported making few changes as a result of the observations of the VLE.However, some talked about the validation it provided for their current practice.For example, one staff member received feedback that it was nice for students to see their actual tutor on video, rather than something taken from YouTube.
While they may have made few changes, staff members commented on the importance of students observing all three aspects: classroom, VLE, and assignment brief.This helped them understand if the students could see how the three were meant to relate to each other.
Jenny's reflections suggest that the purpose of reviewing the VLE needs to be clearly established when the staff member does not have responsibility or editing rights for the site.For example, the VLE review may be necessary to provide context to the classroom sessions and it can provide easy access to resources such as the learning outcomes for sessions being reviewed.

CONCLUSION
The inclusion of the VLE element in the student reviewers scheme enabled the staff/student pairs to consider how well different aspects of the course were aligned.Observations of the classroom environment were enriched by the VLE component.VLE observations were useful for understanding the preparation that students were expected to do (particularly for flipped classrooms) and how discussions were extended beyond the classroom.This may make it a valuable part of the scheme, even where the staff member does not have editing rights.However, in this context the purpose of the review needs to be carefully considered.
Reviewing the VLE did not compensate for classroom observations.Feedback from students following the pilot was that three hours of classroom observation did not give sufficient insight into the course; thus, we have increased this element to six hours this academic year.However, it did provide orientation for the classroom observations in that the students had easy access to the module's handbook and intended learning outcomes.It enabled them to consider the work students were undertaking between sessions and to see the course as an integrated whole.
Technology is an area where staff and students appear to have quite different approaches.The review of a VLE is thus likely to bring up wider differences in perspective and perhaps the risk of more misunderstandings.Our impression is that many staff saw issues of layout as minor, whereas it was a major issue for students who could not find the resources they were after quickly.Some of the causes of this may be differences in the approach that lecturers take to organising materials online, to the approach that students expect them to take.Once they understood how staff members approach organising sites, student participants Marie, J., Hansen, J., Hazen, M., Elliot, M.I., Piccoli, G., & Thorogood, J. (2018).Student reviewers of teaching practice: Reflections on the design and experience of participants.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3370133 interests include the role of post-graduate teaching assistants, staff-student partnership and research-based education.