Peer Learning and Feedback Primers

New instructional material for Eli Review meant to be quick, approachable reads about important pedagogical concepts

Client/Project: Eli Review
Objectives: Develop compelling, accessible content to persuade instructors of the value of peer learning and feedback
Team:
  • Mike McLeod, Project Manager, Eli Review
  • Bill Hart-Davidson, Jeff Grabill and Melissa Meeks, Eli Review
  • Peter Johnston, Videography
  • Natalie Kozlowski, Graphic Design and Animation
My Role: User research, information architecture, interface design, requirements, front-end development, video editing, technical writing
Categories: , , , , , , , ,

Overview

This is a case where I needed to write and build a new resource that would be a quick primer for instructors about the pedagogical foundations behind Eli Review. Developing the content was both a technical writing and instructional design challenge that required marshalling a lot of human resources. The result is a set of materials that remain one of the highest sources of traffic for our support website and that get consistently added to instructor syllabi.

Backstory + Project Requirements

We needed a way to easily introduce the pedagogical underpinnings of Eli Review in a way that people could consume and understand without any of the company principals needing to make a personalized pitch.

Business Goals

Helping new and potential customers understand the “why” of our product had always been somewhat laborious and required one or more of our team to make a personalized pitch. We had a website with sales material but none of that was effective for making the case about the teaching methodology that justified Eli’s usefulness.

The business goals of this project, then, were to develop a set of resources that would make a compelling argument for the pedagogical values Eli was built to support (particularly peer learning and rapid feedback cycles) independently of any of us. We also made the decision that the content would be presented as content-agnostic as possible to avoid the ickyness of a sales pitch, focusing on the pedagogy and not on the affordances of Eli. In this way, we’d have resources that could drive the values that justify using Eli while building our brand, both of which would ultimately help drive indirect sales. We also decided to release them using a Creative Commons license so that users would feel free to utilize them without copyright concerns.

User Goals

Users had not asked for this resource directly, but the idea for building it came from how frequently we were called upon to talk about the pedagogical concepts individually or in presentations to professional development groups or classrooms.

What we decided to build, and ended up validating through early testing and iteration, was something that would be very light, avoiding jargon and academic conventions (e.g. extensive literature reviews) in favor of something people would actually read. We knew that anything too academic would not be read, and anything that read too much like a sales pitch would likewise get abandoned. We needed to walk a fine line and get feedback early and often.

Instructional Design Challenges

While Eli Review is build to make a pedagogy operational, and our curriculum materials are meant to give instructors detailed materials they can use in the classroom, these resources would need to serve a more general audience and to talk about pedagogy in a way that’s smart but accessible, reduced without being reductive. I had the following questions to consider as I started writing and designing:

  • How do we convince instructors to reconsider pedagogy?
  • What do we want instructors to come away from these readings believing?
    • Students learn better from each other (peer learning)
    • More feedback leads to better revision
    • More revision makes better writers
    • More room for writers to practice feedback and revision is critical
  • How can we model peer learning between teachers?
  • How many references will be useful, and how do we encourage additional reading?
  • How do we encourage next steps, or going from one reading to the next?
  • How can we sell the pedagogy without selling the product too aggressively?
  • How can we foster discussion of these concepts beyond the interface?
  • What might make instructors assign these, or similar resources, to students?

Technical Writing Challenges

As with the curriculum content, I need to find a way to present this new information in a compelling format. It wasn’t enough just to create content, but to put it into a container that is useful, usable, and persuasive.

  • What genre might accomplish the instructional design requirements?
  • How do we compel people to keep reading?
    • Chunked reading, designed for scannability
    • Illustration of difficult concepts, including interactive visualizations
  • What affordances of the web could help us design an experience that keeps readers making progress?
    • Waypoints and chapters to divide the content
    • Video and animation that plays on scroll
    • Progress bar that advances as users read
  • What affordances can we build into the content to keep readers engaged?
    • Illustrations for complex subjects
    • Short videos that break up content
    • Simple illustrations or animations to reinforce key points

Building the Readings

The Team

I was the project leader for building this resource because of my experience not only with developing instructional material and technical writing, but because I could also do graphic design, content management, post-production video editing, front-end development, back-end engineering and graphic design. I also had working with me:

  • Jeff Grabill and Bill Hart-Davidson – my co-inventors of Eli and co-founders of Drawbridge, Jeff and Bill worked with my on my initial draft of each module to fine-tune the copy, particularly to make sure the theory was not reduced too far.
  • Melissa Graham Meeks, who took on full writing responsibilities for the fourth instructor module and co-wrote the two student modules with me
  • Peter Johnston – Pete is a videographer who captured all of the one-on-one interviews we utilized in these modules as well as the in-class footage of Bill teaching and demonstrating Eli featured in module 3.
  • Natalie Kozlowski – Natalie is a front-end web designer who did the initial graphic design work for the modules and prepared all of the animated content.

Borrowing a Genre

When considering solutions to the instructional and technical challenges, I was inspired by an interactive article I’d read in The Guardian. That article used a combination of brief bits of text, infographics, animations, and very short auto-play videos from experts to explain the surveillance scandal at the NSA. I found the combination of these elements compelling and quickly consumed the entire article.

I thought we could borrow a similar structure for our content modules. Sectioning the content would be relatively straightforward, as would identifying where pull quotes and illustrations would help land important ideas. I prepare a first draft of the content, followed by feedback and validation from Jeff and Bill.

With the content relatively set, I proceeded to scope out the structure for the module container. I created a Google Doc containing all of the content as well as placeholders for the visual elements I wanted, which I then handed off to Natalie Kozlowski, our graphic designer, who fleshed out those designs and acted on the imperatives that we use space well and draw people downward through the content.

I created a Google Doc with all of the content in place and very rough placeholders for the visual elements and structural content
Natalie took the rough draft and fleshed out the visual design, responding to my content as well as the directive to use space well and compel people to keep moving down (the pointer and an animation)

We followed a similar process to prepare illustrations for important concepts. The rough draft in our Google Doc established the requirements for an illustration. Natalie took those requirements and turned them into compelling visualizations, including animation.

From the original draft, very rough shapes indicate where an animation would be helpful, along with brief instructions of what to animate.
Natalie’s illustration, incorporating the requested animation, and beautifully visualizing the concepts.

We also proceeded with the video. I produced a set of interview questions intended to produce the types of sound bites that would illustrate what we wanted to highlight with the modules. We workshopped the questions with the team and then hired Peter Johnston to capture the video and, in most cases, to conduct the actual interviews.

Once the interview footage was recorded, I created extensive transcripts of each interview and, which transcribing, identified the clips that would best address the specific point we wanted to emphasize in the module. This process was time consuming but helped us find the most effective options for each case. The result was not only excellent sound bites, but perfect examples of peer learning – opportunities for instructors to learn from other instructors.

Partial transcript from an interview subject. I broke each interview down by question asked, along with timecodes and a rough transcript of their response. I highlighted clip-worthy pieces in red so that we could revisit which clip best addressed the point in the module.
From all of the captured videos we pulled the best snippets that illustrated key points we wanted to hit in the modules. Videos would autoplay if browsers supported that, but if not, the HTML5 player defaulted to a poster image with play controls, including the option to captions drawn from the transcripts.

Engagement

To address both the instructional design and technical writing challenge of keeping the reader engaged and moving through the content, I fell back on familiar tech writing tropes but also drew on some affordances from the web.

I drew on the old technical writing standbys of designing text in short, readable chunks, using headers to break things up but to create sequence and relationships, and turning text into bulleted lists that could be consumed more easily than narrative text.

The earliest sketches of the materials had a floating progress bar that made navigation between sections easier but also provided a marker for the current section and a progress bar demonstrating overall progress toward the end of the content.

Progress bar and section markers make wayfinding and progress easy to understand.

Smaller elements in the UI also draw the user’s attention down the page. On the banner opening each module, beneath the title was a call to action (“Let’s Go”) along with an arrow that would animate in to draw the user’s attention down the page, to indicate directionality.

Banner elements like the Let's Go button and arrow propel the reader downward

We also chose animation as a way to draw reader’s attention to critical concepts. In most cases, I identified these concepts myself based on my experiences supporting users of the product or delivering training workshops. I identified the concept in the original draft, worked with Natalie to develop a rough understanding of what we wanted to see. After that she would produce a rough version of the animation for feedback and then build a final version either using CSS animation or as an animated image.

Visualizations like the describe-evaluate-suggest animation make it easier to engage with content

The Scaffolding

When organizing the content, I decided that the body of the module should focus on the topic and that we could fit a lot of the more explicit instruction content in the final section, which I called “Discussion, Resources, and Next Steps”.

The final section of each module was reserved for the instructional content that would help instructors learn more, continue reading, or find resources to use in their courses.
  • Key Takeaways – the 4-5 primary points that we wanted them to retain
  • Questions for Further Discussion – questions meant to provoke dialogue between instructors who were using the resource for professional development activities
  • Next Steps – links to the other modules, also including wayfinding
  • Resources – any handouts or links that further demonstrated a topic
  • Sources Cited – a section to do the proper academic work of citing sources, not just for academic integrity but also to encourage readers to access them
  • Creative Commons license – to reinforce for instructors that the materials are free to share and to circulate with attribution

The idea behind all of this content was to give readers next steps and follow ups to act on what they just consumed. They could proceed to the next module or they could share with students or engage in a conversation with their faculty discussion groups.

Outcomes / Takeaways

The outcome of this process was a series of resources that drive significant traffic for us and that we refer instructors to all the time as they are learning the pedagogy. It’s incredibly satisfying to learn when instructors have added them to their reading lists or assign students to write about them (often publicly on Twitter).

The initial project had substantial upfront cost but produced a set of four instructor modules, followed a year later by a set of 2 student modules:

Instructor Modules

Student Modules

The genre produced a document with a long scroll but a short read – we estimated a 7-minute read per module, and Google Analytics tells us readers often spend about 5 minutes on each of these modules, which we have decided is a success.

Things We Learned

  • People will read – our initial concern that people would not read the content proved to be unfounded; where the average session on our pages was ~ 30 seconds, the time on these resources was ~ 5 minutes.
  • Regular use  – In addition to our initial goal of making readable content, we know anecdotally that instructors add these resources to their syllabi because students often use the “key takeaway” content in tweets.
  • Significant usage – in addition to the substantial screen time, we know that this content drives significant traffic to our website, both from the Eli Review app and from external audiences. The two student modules are in the top 5 requested pages from our site.

Future Plans

We’ve been satisfied with these modules and how they’ve been utilized, but I have been tweaking them over time as technology has evolved. This is in part to prevent experience rot, particularly with how browsers handle the autoplay of video clips, but also to make sure that they conform with best accessibility practice and print options as browsers evolve.