This is a case where I had to fix my own mess. I made several assumptions many years before, while building the Eli Review prototype, that came back to haunt us. This is how we learned from those screwups, the stories we heard from users hacking their way around them, and how we made expensive upgrades to the system to make things right.
Backstory: Not Getting How Time Works
My colleagues and I built the Eli Review prototype as part of a research project at Michigan State University. As I was writing the code and building the database, we didn’t imagine that this proof-of-concept would become a commercial entity with a global user base. We didn’t anticipate any teachers except ourselves using it, along with our co-located students: all of us in the same building, working asynchronously.
As a result, the prototype – and the commercial product that grew from it – were hard-coded with a myopic view of how people (particularly teachers and students) experience time. Every assignment was assumed to have a deadline of 11:59:59 pm in Eastern Standard Time, because 1) that’s how the three of us typically structured our due dates, and 2) that’s the timezone where we lived.
As the original developer, I unknowingly hard-wired these assumptions into the app in its code but also in its database. The app’s MySQL tables all stored dates in a DATE type field, meaning that we had no way to store data about time (if we’d thought about time at all). Not having time data also made it impossible to work with dates localized by timezone, which we also just defaulted to Eastern Standard Time.
The commercial app was built on all of the assumptions made in the prototype, including those about time. We struggled to support users living outside EST, like our earliest users at Washington State University and San Francisco State University, both in Pacific Time. We struggled even harder with customers in Sweden and in Singapore, where instructors had to coach students in how to translate time to values that made sense to them. Over time, we developed a set of time-related use cases that eventually compelled us to take the serious step of updating every time element in the app.
In the summer of 2016, we set out to completely overhaul how Eli Review handles time and how users experience it in the app, which required some extensive and costly changes to our database and our UI.
User Stories and Project Scope
Before starting the overhaul or even drafting requirements, I worked with our team to condense those years of feedback into the following user stories:
- As a user, I want the system to recognize my timezone and display all times localized to my timezone.
- As a teacher, I want to specify the time a submission is due, not just the day.
- As a teacher, I need to see the timeliness of a student’s submission.
- As a teacher, I need to know the timezones in which my students live and their difference from my own so that the due dates I choose don’t cause conflicts.
- As a teacher and as a student, I want to see the difference between the due date for an assignment and the time it was submitted.
The biggest lesson we learned while coming to understand the scope of these use cases was that not only would it make the system better meet user needs, but that it’s important example of inclusive design. It’s not exactly the same as a11y (a project dedicated to web accessibility), but not respecting time and timezones definitely make the earlier version of the app less inclusive and usable for people who didn’t happen to live in Eastern Standard Time.
With those user stories in mind, I prepared an extensive set of requirements for our developers. I determined how the UI for selecting dates and times would change (primarily by choosing a new third-party datepicker). I also combed through the entire app, documenting every instance where time surfaces in the UI, and describing in detail how that instance needed to change. Download the complete set of requirements.
The following is a brief summary of all of the high-level changes to Eli Review needed to support new time functionality:
- System Timezone: our developers considered many different ways of storing time, including the MySQL timestamp format, but they ultimately recommended we adopt Universal Coordinated Time (UTC) so that all stored times would be in the same timezone, regardless of the user’s location, and then localize as needed.
- Database: every instance of a DATE field in the app’s MySQL database was changed to a DATETIME field, making it possible to store time in all instances.
- Date picker: a new UI tool for selecting both dates and times
- Localized Timezone: while the system stored values in UTC, the UI would display for users times translated to their local timezone as specified in their profiles
- Date difference: the system also required a simple function to calculate the difference between times, which was especially important for instructors who wanted to assess the timeliness of student submissions.
Before & After
BEFORE: Since its commercial release in 2010, Eli Review utilized the jQuery UI datepicker to allow users to select dates from a GUI rather than entering and formatting them manually. This datapicker, by default, allows users to specify a date but not a time. This is obvious in the related form field where the selected date is displayed in YYYY-MM-DD format.
Another significant before issue is how the app implemented timezone support. I added this feature a short while after the initial release in an attempt to patch the obvious problem, but it didn’t go far enough. When signing up, users were prompted to select their timezone from amongst only four supported values which represented only the continental US. The selection was saved as part of a user’s profile, but it played no role in how the system represented time to the user, rendering this step in the signup processes useless.
AFTER: For a replacement, I identified an open-source datepicker by XDSoft that would enable convenient date and time selection and our developers used it to replace the jQuery UI version in all UI instances of date and/or date and time selection. It made a tremendous improvement, both functionally and aesthetically.
The timezone selection at signup was also expanded to better represent how people actually experience time. The drop down selections were greatly expanded to represent global timezones, including oddball cases like the state of Arizona not recognizing daylight savings. The system was also modified to make a best guess about the user’s location and to select that timezone by default, while allowing them to make a manual change as needed.
Student Timezone Summary: we added a new “Student Timezones” display for instructors which could be opened via a link next to each date picker. This modal window gives them a reminder of time differences for students who live outside the course timezone (if any). This has proven to be a particularly helpful feature for instructors in fully online courses who have to think about how due dates affect students who live around the world.
Submission timeliness: The developers prepared a great time comparison function that made it easy to compare two dates and determine whether a submission was late or on time. With that function I was able to update each student submission UI to help instructors more easily see when students had turned in an assignment early or late.
Task Completion Report: with the time comparison function complete, I was able to abstract timeliness data one level up and give instructors a class-level view of all submissions and student timeliness. The report uses both text and color to flag students who did not complete their work in a timely fashion, making it easy for instructors to see who’s engaged and who isn’t.
Course-Level Completion Report: With the task completion report done, I was able to abstract one level even further to help instructors see timeliness over the duration of an entire course. This report also uses colors and iconography to identify which students have completed their tasks in a timely way and to easily see students who are struggling. While users without assistive technologies see a report like the screenshot above, the report is built so that the icons will read as text when being accessed by a screen reader, and the colors are not required for meaning, ensuring usability for instructors of differing abilities.
Outcomes / Takeaways
Helping our users hack their way through a lack of time support for so many years taught us a few (painful) lessons about software development. Primarily among them:
- Find and test assumptions early: it wasn’t until well after the prototype had been commercialized that we realized the corner we’d painted ourselves into by assuming the only variable that mattered was a date. We didn’t know enough to to interrogate ourselves early in the process and that ignorance was costly.
- Don’t underestimate how people test a system: Students are notorious for pushing boundaries and gaming systems and it turns out that time and due dates are one of the most common ways they’ll do that. As we were developing time support, it became clear that student would push deadlines to the point where submissions needed to be tracked down to the microsecond, not unlike the stock exchange.
We’ve also continued to learn about how users experience time since we released these improvements in August 2016. These lessons will almost certainly drive development of future updates to further support users:
- Custom deadlines: Eli currently supports a single deadline for every student in each task. Many instructors would like the flexibility to create customized deadlines for individual students as needed; for example, if a student needs additional time, the instructor might extend the deadline for that student by a day, leaving the rest of the class with the default deadline.
- Grace periods: Instructors would like the option to create a window after the due date wherein students could still turn in their work without it being considered late. This would effectively be the task due date, but creating the appearance of flexibility can be enough to incentivize students to complete a task.
- Nudges: Many instructors would like the option to push students who are incomplete to get their submission in before moving on. Using a feature like this, instructors could send a ‘nudge’ to all late students with a single click.