Collaborative Curricular ( re ) Construction — Tracking Faculty and Student Learning Impacts and Outcomes Five Years Later

The Collaborative Curricular (re)Construction, or C, was an initiative at Creighton University that paired faculty (academics) and students in a process of backward course design, in two cohorts, in the 2013/14 and 2014/15 academic years. Faculty/student pairs worked over the span of a year to redesign a course within their discipline; courses ranged from theory-, skill-, and laboratory-based courses. The study investigated four primary questions: (1) Was C an effective tool for faculty development? (2) Did students emerge from the C experience changed as learners? (3) Did the course revisions result in increased student learning in subsequent course offerings? (4) Did the effects of the C workgroup affect curriculum as well as the culture within the program or department? Previous work has described the immediate impact to faculty and student; here, however, findings include the long-term impact on faculty and on student learning in the redesigned courses. Results conclude that even a brief faculty/student collaborative redesign experience has lasting impacts on student learning and, in several cases, on program-wide curriculum.

redesign a course to improve student learning.These efforts focused both on outcomes as well as on the formative processes.And because the literature is so rich in terms of benefits to both faculty and students in SaP models, we specifically sought to determine the impact of the model on student learning and overall culture and curriculum at our institution.

BACKGROUND AND CONTEXT
Pairing students and faculty to collaboratively redesign courses took the form of a faculty/student development initiative we called Collaborative Curricular (re)Construction, or C 3 for short.The model sits at the intersection of the categories of Learning, Teaching, and Assessment, and Curriculum Design and Pedagogic Consultancy in the model of Healey et al. (2014).This project was funded by the Office for Academic Excellence and Assessment (AEA), essentially Creighton's Teaching and Learning Center, headed by Danielson.Faculty participants were invited using the following criteria: tenured status, a reputation for openness to curricular innovation, a general willingness to engage with students, diversity in gender, and representation from different schools/colleges within the university-for faculty, one of the chief draws of working in such a group is the opportunity to meet and work with colleagues from different colleges and schools.Each faculty member was asked, upon invitation, to do the following: (1) identify a course that would benefit from redesign and student input; (2) select and recruit a student participant, ideally a student who had taken the course recently; and (3) attend at least six meetings over the course of the semester, both in large groups and in individual meetings with their student partner (SP).In recognition of the value of this collaborative work, some compensation was provided.The compensation was nominal; faculty received a copy of Understanding by Design, by Wiggins and McTighe (2005), as well as $100 in faculty development funds, and students received both a copy of the book and a $100 credit at the campus bookstore.Faculty participants were much more motivated by the opportunity to improve their own teaching by working closely with a student than by any offer of compensation.Several invited participants did, however, decline to join the program; lack of time was cited rather than any issue with the provided compensation.
Participants were recruited early in the fall semester and met at least six times over the next several months (see Table 1), culminating in a final large group meeting in December.Group sharing of curricular redesign of courses Implementation plans for the newly designed course * Many faculty/student pairs met much more frequently, some weekly or bi-weekly.
Two separate faculty-student cohorts completed the C 3 process: although eight faculty were invited, only four faculty-student pairs accepted during the 2013/14 and 2014/15 academic years.Rather than inviting new faculty when the originally invited faculty declined, the study proceeded with smaller but clearly diverse cohorts.The two cohorts had faculty from the following disciplines: biology, chemistry, education, fine and performing arts, law, nursing, and pharmacy science.

RESEARCH QUESTIONS
In the initial C 3 project, our primary research questions were the following: 1. Are students changed as learners by participating in a SaP experience? 2. Are faculty changed as teachers by participating in a SaP experience? 3. How does such a partnership change the nature of the course being designed?
The first two questions, of course, probe the impact on students and faculty: Is this an effective method to develop both faculty and students?Duda and Danielson (2015) showed that student participants approached learning in subsequent courses differently, and faculty were profoundly affected by this experience and changed many aspects of their teaching and the way they designed courses as a result (see associated Appendix: Redesigned Courses for full description).
The SaP literature has demonstrated the validity of this cooperative approach and the benefit to both instructors and students.However, there has been little research done on the impact such partnerships can have on student learning in the redesigned courses.Furthermore, a measure of the effectiveness of an approach to faculty development such as C 3 would be the promulgation of resulting curricular changes through programs, departments, and colleges.Marie and McGowan (2017) also examined student and staff perceptions or lessons learned one year later, an approach that identified a series of partnership values necessary for student learning.The original C 3 cohort met almost five years ago; although faculty intent was to immediately implement changes, many implementations were delayed by sabbaticals, off-cycle courses, changing teaching assignments, and even a core curriculum overhaul.In fact, one course implementation was delayed until spring 2017.To capture student learning in these Duda, G., & Danielson, M.A. (2018)

METHODS
Several qualitative research methods were employed to study our research questions; where possible, quantitative analysis was also employed, particularly to examine evidence of student learning gains in specific courses.This work was reviewed and granted exempt status from the Creighton University Institutional Review Board, and follows best ethical practices for educational research.One-hour-long semi-structured interviews with faculty participants (N = 8) were conducted and reported, principally to collect information on long-term impact and on student learning in revised courses.Although most of the SPs had graduated several years ago at the time this article was being written, one former SP was currently teaching part-time at Creighton and participated in the semi-structured interview with their faculty partner (FP).Faculty and student participants also completed online surveys at the end of their participation in C 3 .Additionally, the AEA office sponsored a one-and-a-half-hour-long forum/panel discussion on the C 3 project at the end of the second iteration of the project; seven FPs and four SPs spoke on that panel.This focus-group-like session (recorded and transcribed) addressed the primary research questions posed in a free-form discussion with minimal prompting from the authors.

Faculty development
Faculty discovered this process nurtured and gave an outlet for some of their natural "tinkerer" predispositions while it encouraged active reflection and concrete pedagogical changes.FPs described themselves as "looking for a change," "by nature open to change," and "creative, flexible, a tinkerer."This openness to change predisposed them to accepting the C 3 invitation; more importantly, it naturally led to both affective and behavioral changes.As one FP noted, it "opened [his] eyes," while other FPs appreciated seeing their courses through their "students' fresh eyes," even as they struggled "to 'get' how their students were not 'getting it.'" Beyond this shifting to a more student-centric perspective, faculty learned they had to "stick with the plan," even as they saw students struggling.For example, one FP overcame his natural desire to step in and intervene when he saw students struggle.Following his SP's advice, the FP did the following: (1) clarified course expectations (e.g., it is very important you study in advance of the course sessions); (2) allowed students to "fail" in weeks one and two (e.g., failing quiz scores); (3) intentionally aligned quiz items to better match the instructional Duda, G., & Danielson, M.A. (2018).Collaborative curricular (re)construction-Tracking faculty and 43 student learning impacts and outcomes five years later.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3568session; and (4) reduced material coverage to 80% of previous semesters.His SP admonished him, "Believe in your own idea!" SPs commented on their FPs development, as they found FPs to be "extremely open and accepting of [their] input and working collaboratively to improve the course."They also shared the realization that "professors really do work hard to make sure their students get something from the class they are in." Faculty continue to be open to changes and have not forgotten the lessons learned from their student partners.More than one FP reported that they continue to solicit feedback from students; while one FP utilizes multiple opportunities to communicate with current students (e.g., mid-semester feedback, lunches, students' daily visits to the office), another FP finds a continued "openness to learn from students," even when students prove the FP to be in the wrong.As she noted, "I really didn't want them to be right, but they were!" FPs' initial analyses and innovations persist: current activities include conversations and even collaborations with other faculty, often within their department or school/college; changes in their teaching of other courses; and active solicitation of more meaningful contact with students.As described by one FP, "the most essential change was motivating [me] to start doing changes that have been only contemplated in the past."

Student development
Students' "learning" extended from deepened understanding of their course's subject matter to faculty design and course preparations to reflection on both their own and fellow students' study habits.As one SP explained, "It required me to look further into the study of [my course] in order to see how I learn as an individual and how those around me learn." Collaborations with FPs allowed students to better appreciate "what goes into developing a course."Further, the student learned "how to verbalize and understand [what they] had been doing as a learner and how to describe that process in words and then develop a course based on those experiences."Faculty expressed appreciation of students' sharing as they exhibited "unbridled enthusiasm" from "one[s] who hadn't been trampled yet" by academia.As faculty confirmed, students were true collaborators in that they "helped identify the [course] problems and came up with solutions."FPs' pride in their SPs' work was evident, as was their vocal encouragement.As one FP encouraged the SP, "You have a voice, so use it." Students' behaviors changed (in subsequent courses) because of what they experienced during this collaboration; specifically, "I have tried my best to get involved in each of my classes as much as possible because I understand how much energy and time my professors have put into creating the course" and "I now will look more in-depth at these objectives in order to more fully understand the class structure and my teacher's goal."These former student partners carry and translate their lessons into their current professional practices, be it as students in professional degree programs or as teachers, developing and delivering actual course instruction.
FPs were committed to and excited by the opportunity to partner with students as part of ongoing student development.At the initial meeting, FPs were more enthusiastic in their introduction of their SPs than themselves or their courses, and in the final group sharing, more than one FP interrupted the student's presentation to inform the larger group that some critical idea or innovation was, in fact, the idea of their student.This commitment to student development did not end with their course redesign effort.In the follow-up interviews, all eight faculty members immediately identified their SP and their SP's current employment or educational status.It was evident that faculty and student partners maintain an active professional relationship.

Student learning
For practitioners of the Scholarship of Teaching and Learning (SoTL), there is no question dearer than that of student learning.Any classroom innovation must be ultimately judged by the following metric: Did it increase student learning?Although it is difficult to assert a causal or correlational relationship between involving students in a course redesign process like C 3 and improvement in student grades, insight into student grades and improvement in student scores is presented as one source of evidence.Taken into consideration with other lines of evidence, such as qualitative interview data, we believe that there is sufficient evidence that the C 3 program (student-faculty partnerships to revise courses) led to positive outcomes and, in particular, increased student learning.
The faculty interviewed all strongly believed their participation in C 3 led to stronger courses and more robust student learning.For example, the FP in the music theory course spoke passionately about how students had gained a greater ability to actually compose music for their final project because of the scaffolding of composition exercises that the course team had built into the course.The FP says, But by having them do the composition exercises, they had to take the information they knew and apply it … so that actually made going into the final exam, which was a presentation of a composition, the final composition, much more meaningful to students.They're much more engaged, much more interested in it, and they do a better job.Rather than just an introductory "write a few measures of music," they're actually letting this music do a conversation back and forth between the instruments that are involved.
Besides anecdotal and qualitative evidence, there is particularly compelling quantitative data from several courses.In the interest of brevity, we present three examples, one each from chemistry, biology, and pharmacy.
1. Biochemistry Laboratory is a required course for chemistry majors that complements a three-credit lecture-based course.The purpose of the lab is to introduce students to very practical lab-based techniques in biochemistry, which they will encounter in industry, graduate school in biochemistry, or other post-graduation professional programs.The laboratory course meets weekly for a one-hour pre-lab session, along with a two-hour laboratory block.This prelab meeting has traditionally been used to supplement lecture and teach some of the theory behind the experiments that occurred in the lab.Instruction in the pre-lab meeting was delivered via lecture.In the biochemistry lab course, the intervention of flipping the classroom and providing students with additional scaffolding to identify key concepts and ideas led to increased student learning.The midterm quiz in the course focused heavily on conceptual understanding, and the instructor reported that it is a good indicator of student learning.Compared to two sections that the same faculty member taught in the fall of 2013, the midterm exam scores were statistically significantly higher (p = 0.003, ES = 1.37) for the course that used the C 3 modifications (see Table 2).The final quiz for the biochemistry lab course was not focused on conceptual understanding; instead, it was a more traditional lab final with numerical problem solving.Despite the flipped-classroom approach, students in spring performed as well as students in the fall (who had sat through traditional lab lectures).In other words, not only did the flippedclassroom students demonstrate a better conceptual understanding of the material, but they also performed just as well on more traditional measures.
2. General Biology II is the second semester of a freshman-level general biology class, typically taught at Creighton in two or three lecture-based sections of approximately two hundred students.Recitation groups for this course were introduced in spring 2014 on a limited basis: a small number of students could self-select into a smaller recitation group (RG) of about eighteen students that met once a week for an hour and fifteen minutes.These students were excused from attending the large lecture sections and instead watched recorded versions of the lectures outside of class.Students were given readings to complete before attending their RG, and time during the RG meetings focused on more difficult material using active learning pedagogy (such as small group discussions, tutorials, or simulations).The FP for the recitation groups in the General Biology II course came to C 3 with two issues: (1) a bi-modal grade distribution and (2) a problem with coverage of content defeating the purpose of the RG model.As can be seen in Figure 1, the general exam distribution for the course in spring of 2013 (the academic year prior to C 3 participation) peaks in the B to A range.The most recent iteration of the RGs in General Biology has been extremely successful.After implementation of the course revisions developed in the C 3 workgroup, students participating in RG have outpaced traditional lecture students in terms of their learning gains.Duda, G., & Danielson, M.A. (2018) For example, Table 3 shows the exam score average for the four exams and final exam for RG and non-RG students.The difference between the RG and traditional students on Exam 1 is not statistically significant, but for each subsequent exam, the difference is statistically significant; the difference increases for each subsequent exam.Furthermore, the bi-modality that was seen in the spring of 2013 completely disappeared.In spring 2018, RG students were underrepresented in C, D, and F grades and overrepresented in B through A grades.
3. Pharmacy Lab: The FP that taught the P3 pharmacy labs was reassigned to teach P1 pharmacy labs in the fall of 2014 before the innovations that she and the SP developed could be tested.However, the ideas that they developed collaboratively transferred easily to this new laboratory setting (the P1-P3 labs focus on similar content).Furthermore, the pharmacy faculty interviewed students to discuss what could be done to help them with the process of prescription verification (checking) and sterilization verification, using the entire course, in effect, as SPs.Using that feedback, the FP developed online practice modules for these activities.As can be seen in Table 4 below, students who utilized these practice activities had statistically significantly higher final grades for these lab components (p < 0.01).

Table 4. Mean scores on prescription verification lab activities for P1 students in 2015/16
Wider curricular change as a result of C 3 The ultimate measure of success of a program like C 3 is not that it affected a single course within a department or program, but rather that participation in such a program helps lead to wider curricular and cultural changes (e.g., the creation of learning communities).However, such change, given faculty turnover, sabbaticals, and new teaching assignments, Duda, G., & Danielson, M.A. (2018).Collaborative curricular (re)construction-Tracking faculty and 48 student learning impacts and outcomes five years later.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3568often takes time.Even given the limited time since the original pilot, there is evidence that C 3 has affected departments and programs in a wider sense.
1.At the time of this study, the FP for the Nursing Care Management course was also the chair of the nursing curriculum committee The committee was working on a complete revision of the nursing program, and in part, because of their C 3 experience, the FP pushed for the incorporation of active-learning and pre-lecture activities in every course.Every fifty-minute block in the new nursing curriculum will include fifteen minutes of active-learning activities to be done outside of class.Nursing faculty, though initially resistant to this change, have been convinced by 25% increases on practice exams for the NCLEX-RN exam.
2. In the contracts course within the Law School, the course team introduced the use of midterm exams.Midterm exams have served as an early warning indicator and have allowed faculty to work with students who are at risk of failing out of the program.As the FP reported, "Whereas before we might have had three to four students at a failing level at the end of our first semester fall class, now we have one or two.The greatest impact is on the C, D, or F level."Since a midterm exam was piloted in Contracts I, almost all faculty in the Law School have adopted the use of midterm exams.The FP continues to be an "early adopter" of pedagogical and technological innovations, many of which have spread to other faculty (like the use of clickers during lecture and collaboration with legal writing instructors).
3. The RGs in General Biology were an experiment with initially one faculty member (who participated in C 3 ).The FP here unfortunately did not return to teach RG in the spring of 2014, because of staffing issues, and then took a yearlong sabbatical in the 2015/16 academic year.However, RGs returned to the General Biology II course in the spring of 2017, and the FP began implementing the changes made with the SP in C 3 .Spring 2017 had nine RGs taught by three faculty members.In the spring of 2018, the RGs were expanded: twelve RG sections were offered, taught by five different faculty members.The success of the RG experiment has changed the way general biology is taught at Creighton.It has also changed expectations for faculty: RG has been shown to work with any faculty member; therefore all biology faculty members are now expected to have meaningful instructional contact with first-year students; and it has allowed the faculty to engage with each other in new ways.This engagement has affected the way upper division biology courses are taught as well.As the FP put it, "Do I teach better in my genetics class because of this?YES!" 4. The School of Pharmacy and Health Professions, at the urging of the FP involved in C 3 , has informally adopted SaP as a way to revise courses.For example, several faculty members gathered students to do a focus group about a recent fall 2017 elective course.Students expressed interest in participating in a mock rounds-type case that covered multiple diseases and treatments; typically, only a single disease or treatment is covered.The pharmacy faculty then piloted a mock rounds-type case in a third-year pharmacy course as an online module and simulation.
Even in this limited sample, there is evidence of the effects of C 3 promulgating outward from single courses to departments or programs.Of course, there are other factors at play here other than participation in C 3 .Faculty members were selected who were leaders or emerging leaders in their departments/programs with a reputation for pedagogical innovation.However, Duda, G., & Danielson, M.A. (2018).Collaborative curricular (re)construction-Tracking faculty and 49 student learning impacts and outcomes five years later.International Journal for Students as Partners, 2(2).https://doi.org/10.15173/ijsap.v2i2.3568 in our follow-up interviews, every faculty member attested that C 3 helped them think differently about their teaching and their relationships with students.

DISCUSSION AND CONCLUSIONS
The C 3 model has proved to be an effective tool for faculty development.It nurtured faculty creativity and pedagogical flexibility.It gave faculty permission to start experimenting in their courses as they only "contemplated in the past."Further, it opened faculty eyes to student perspectives, difficulties, and challenges resulting in new instructional designs or strategies such as flipped classrooms, active learning elements, and experiential learning.Finally, the research team also observed that this process gave faculty a sense of connectedness to other faculty and allowed them to collectively shoulder the responsibility for understanding and improving student learning.
Students emerged from the C 3 experience as changed learners.Changes included a deepened understanding of the course's subject matter, appreciation of the importance and centrality of course learning objectives, and an increased desire to more actively engage with their own courses.These changes echo what Hutchings (2005) has described as pedagogical intelligence-"an understanding about how learning happens, and a disposition and capacity to shape one's own learning." When curricular changes resulting from C 3 were implemented, there was increased student learning in subsequent offerings.Anecdotally, all faculty participants believed that their courses were strengthened, and student learning was enhanced through their participation in C 3 .Assessment data provided further evidenced statistically significant differences on a number of measures in multiple courses.
Beyond increased learning in subsequent sections, one of the effects of the C 3 workgroup was more widespread curricular and cultural change.Colleagues who did not participate in C 3 in disciplines such as law, biology, nursing, pharmacy, and others were persuaded to experiment personally with new pedagogies.Now, faculty are more open to student participation and to using this model for subsequent course revision.One FP expressed the sentiment that he regretted that he didn't continue with this practice of student partnership in the subsequent semester.Faculty more automatically think of getting student feedback and collaborating with students; it is now more naturally a part of who they are as a faculty member.
While there were four primary research questions addressed in this study, conversations with FPs and SPs offer implications for future studies.Both faculty and students were intrigued and enthusiastic at the initial call, but what didn't become clear until the end was how important personal qualities and traits of the participants were.Openness, flexibility, and creativity were important faculty qualities.Student self-confidence, or as one FP described it, "sassiness," ensured students contributed a strong student voice to the process.Both faculty and students, however, shared a sense of vulnerability, both with their partner and the larger collective.As a student shared, "Truthfully I was a little bit nervous about this process as I was not sure what to expect." There are broader implications for this work as well.Curriculum design and pedagogic consultancy comprises about 40% of all the SaP literature surveyed by Mercer-Mapstone et al.

Figure 1 :
Figure 1: Exam score distribution for General Biology II in the spring of 2013

Table 2 . Student scores on the midterm and final quizzes in spring 2014 (vs. fall 2013)
Scores are out of a maximum possible of fifty points.