The “Case Study” Genre

An attempt to make sense of what designers mean when they ask for "case studies"

October 23, 2018 - Product Design, User Experience

Genres are slippery things. To paraphrase rhetoric scholar Carolyn Miller in her article Genre as Social Action, we recognize a form as a genre because we’ve seen it used repeatedly to solve similar problems in similar ways, but it’s not so rigid that it can’t adapt and evolve to meet the needs of new situations.

With that in mind, it’s possible that ten people can think of a “case study” and have a general understanding of what one does but have ten different ideas about how to write one. People generally understand them as telling a story about a problem and how that problem was solved, and they tend to be “a fusion of lower-level forms”, including a summary, objectives, challenges, methods, and results.

However, as the rest of this post and annotated examples will suggest, things people write and call “case studies” tend to vary wildly, suggesting that the genre still lacks a “means of rules” and that people are still negotiating what a “case study” looks like.

Case Studies in Design and Development

The focus of this post is on a specific type of case study – the type that describe the design and development of product. I chose this narrow focus because I needed a better sense of what case studies look like in my own field but also because I found case studies about engineering to be too focused on technical details rather than solving human problems. See this excellent Github repository assembled by Andrew Romanov for front-end engineering case studies from major corporations like Google, Facebook, AirBnb, etc.

I kept watch on Twitter for case study examples for almost a year and collected what I thought were the most compelling examples. They are each annotated below, but these are trends that I noticed across all of them that may help define the “case study” as a genre:

  • Length: Some are long and exhaustively detailed, while others are short and digestible but less substantive. There are definite trade offs here, where longer case studies can feel impenetrable while shorter ones can feel empty.
  • Process vs Artifacts: length also plays a role in the substance of the case study. Shorter studies tend to focus on deliverables (e.g. screenshots of finished products) while longer studies have space to write and demonstrate more of the development process and its artifacts (research methods, sketches, diagrams, prototypes, etc).
  • Framing: the best examples framed the case not just as technical but also as cultural, as something that required buy-in from stakeholders as an essential part of the design process. Reading how designers negotiated cultural issues was one of the most satisfying aspects of reading these case studies.
  • Lessons Learned: every study included a section in which the writers reflect on the success of the project, bumps along the way and how they overcame them (or didn’t), and takeaways for the next iteration or next project.
  • Voice: the most interesting case studies (that is, the ones that compelled me to want to finish reading) have a distinct voice telling the story, either from an individual writer or from a team. It’s possible to feel the pain that a designer experiences when a design doesn’t work, or when user research tells them that their assumptions were wrong, or being relieved that a long shot turned out. None of the case studies featured here are dispassionate and logical – they are all very personal.
  • Data: perhaps most surprising was that, with one notable exception (Franklin Barbecue, see below), all of the case studies focused on narrative and not on analytics. Trends have suggested that performance data (e.g. improved KPIs, conversions, etc) should be the focus of design, yet all but one of these chose not to focus on quantitative measures but instead qualitative assessments of the user experience.

Perhaps the most important variables in the different case studies examined below are audience and purpose. Some of these are meant simply to share knowledge of of an experience, while others are clearly meant to be portfolio pieces that might help the writer secure a job. This is another area in which the form of a “case study” varies – two case studies can be comprised of the exact same pieces, but when focused on different audiences or purposes, they take on a much different shape.

Annotated Case Studies

Building a large-scale design system

by Maya Benari, 18F

Screenshot from the 18F case study describing the deployment of a new design system across all government websites.

This case study tells the story of how the US Government agency responsible for improving how the government interacts with the public (18F, a division of the General Services Administration) built a design system meant to bring consistency and stability to 30,000 government websites.

Observations:

  • Written by an individual, but documenting a team process
  • The study frames the problem (inconsistency of user experience across government websites) with detailed, compelling examples
  • It also frames the problem and the solution as cultural and goes deep into a description of the process the used to negotiate amongst so many stakeholders
  • It includes detailed, re-usable examples for each stage of the development process (colors, typography, interface audit, etc)
  • The “lessons learned” are both reflective of the 18F team’s experience but are also generalized enough to be applied to other projects

Assessment:

  • Takeaways suggest it’s meant for an outside audience, but reflection on the project outcomes (“our system is helping agency teams in the federal government create simple, efficient, consistent experiences quickly and at reduced cost”) suggests that it’s making an internal argument as well about the effectiveness of the work.
  • This interdisciplinary 18F team was clearly assembled for a specific mission, but their ability to set expectations and deliver suggests they’d be the kind of team I’d want to assemble to tackle a problem as complex as this.

Franklin Barbecue

by Kevin Sharon, National Design Service

Screenshot from the National Design Service case study for Franklin Barbecue.

This use case describes the redesign of the Franklin Barbecue website, focusing in particular on how they were able to expand beyond typical restaurant websites to add advanced functionality and drive new business.

Observations:

  • Screenshots are large and beautiful, but often feel so large that it feels unclear where they stop and the case study resumes.
  • The narrative is made of short, simple stories for different components of the project (the details about the responsive menu they designed are particularly compelling)
  • Features the only use of data in any of the reviewed case studies, specifically included are impressive metrics about conversion rates for their new smoker order system and pre-order system for skipping lines
  • This case study is a fast read and light on text, but that text is incredibly effective.

Assessment:

  • Written entirely for an outside audience, serves as a portfolio piece to drive business to their consultancy / design service.
  • While the case does not definite outcomes or objectives as clearly as others (“Franklin Barbecue website was long in need of an overhaul”) it does an amazing job describing the different actions they took to carry out that overhaul and then providing metrics to measure their effectiveness.
  • I would hire National Design Service to rebuild a website in a heartbeat. I want to work with someone who is this talented visually but also communicates this effectively (their responsive menu design is particularly stunning, as is the data they provide).

Metro alarm — a UX case study

By YaChin You, Maker of uxchallenge.co

One of the illustrations from the Metro Alarm case study demonstrating the author's freehand sketches.

This case study is the author describing how she worked through a design exercise that was more a test for herself than a project for clients.

Observations:

  • Written by an individual offering insight into her own process – a design challenge
  • Offers a personal rationale for the project rather than business goals
  • Describes the framework used to complete the project (Jobs To Be Done)
  • Describes her methodology for developing the project, including success metrics
  • Features lots of process documents, including sketches and wireframes and prototypes, and compelling visuals
  • Overly long, though – level of details is almost overwhelming
  • “Key Lessons” are less about things she learned from the project than things overall that helped her complete it and advice for others
  • Most helpfully demonstrates how she utilized a lot of existing assets to turn work around quickly

Assessment:

  • Her advice to others trying the same thing in her “Key lessons” section demonstrates real leadership potential, not just independent design skill.
  • Someone I would want to hire – her thoroughness and thoughtfulness are impressive, as is her ability to walk through the entire development process, beginning with success conditions and accountability metrics and ending with fully functional prototypes.

Product thinking: behind our new Messenger home screen

By Brian Donohue, Director, Product Management, Intercom

An illustration from the Intercom case study demonstrating a reimagined user workflow.

This case study walks through the evolution of an idea which the Intercom team acknowledges “wasn’t something our customers were directly asking us for” but that eventually combined several different ideas and launched a new product.

Observations:

  • Written by an individual summarizing a team’s long effort to rebuild a major feature
  • Includes research of how others had addressed similar problems
  • Excellent visuals – small enough and annotated, making them easy to comprehend
  • Includes multiple annotated examples of iterative attempts to address the same problem
  • Includes a detailed discussion of the final version and why it was chosen from several different designs

Assessment:

  • Illustrations and annotations clearly reveal designers speaking to other designers
  • A case study like this is less likely to convert a customer, but may be a helpful recruitment tool for other designers and developers
  • This is obviously a thoughtful and effective team, would hire

Shipping system fonts to GitHub.com

by Mark Otto, Senior Director of Design at GitHub

Compelling use of screenshots to demonstrate before-and-after of a design at the start of a case study.

This case study describes how the writer led a team to deploy modernized fonts to github.com.

Observations:

  • Opens with a before and after, rather than holding transformation story until the end
  • Screenshots are in a simulated browser window, which looks really nice
  • Detailed rationale and compelling argument for why a change was needed
  • Engaging breakdown of challenges faced (“Dealing with Chrome”) that didn’t get lost in the technical weeds
  • “How we did it” – nerd-level details about how they implemented the changes that was interesting without being alienating
  • Reflection after a year, rather than “lessons learned” – allows writer to think about things missed and reflect on successes of project

Assessment:

  • Not immediately clear who the audience is, but it’s clear that the writer cares about the product and wants to share how it’s grown.
  • An excellent story that’s easy to digest.
  • Obviously a passionate person who is passionate about the product, may have a personal connection to it or its development history.

Mobility platform for the city of Antwerp

By Little Miss Robot

Screenshot of section of this case dedicated to stakeholder decisions, with documents too small to comprehend.

This case study looks at an app designed to help commuters navigate the city of Antwerp.

Observations:

  • Prepared by a design agency documenting their work for prospective customers
  • Introduced from a browsable list of cases with a short, compelling story – concisely describes the problem and what the agency was hired to do about it
  • Framed one of the challenges as culture within the organization and proposed a solution for managing potential conflict
  • Light on details, but heavy on pretty screenshots
  • Some screenshots are so large as to be overwhelming, almost becoming the whole page

Assessment:

  • They obviously produce beautiful visuals, but they are very light on details or on insight
  • Because of the lack of insight into process or objectives or outcomes, I likely would not consider hiring this agency.

Navigating the complexity of change aversion

Cindy Chang, Senior Product Designer; Ulaize Hernandez Troyas, Product Manager

Screenshots of four separate approaches to solving the same problem.

This case study (the second mentioned from Intercom) lays out not only the specifics of an interface they designed but also the method by which they deployed it to cause minimal user disruption.

Observations:

  • Frames the complexity of redesigning existing products that already have loyal users
  • Opens with before and after, not waiting for a dramatic reveal
  • Discusses the need for change as part of a larger brand shift at the organization
  • Includes multiple interactions with detailed customer feedback on each
  • Insight into testing designs with limited groups so as to mitigate risk / harm to customers
  • “What we learned” section was again a generalized set of principles that could be translated to other experiences
  • “Epilogue” section provides an update on how the design was received once the product was pushed live.

Assessment:

  • Tells a human problem to a larger audience, not a technology problem
  • Published with the idea of helping others, sharing knowledge
  • As with the other Intercom example, there’s little sales value to publishing content like this, but this case study makes an excellent recruiting tool for engineers and designers
  • If not hiring outright to build product, could absolutely see hiring them as a consulting service to get a project off to a solid start

I Got Rejected by Apple Music… So I Redesigned It

By Jason Yuan

The designer makes it easy to compare version A of Apple Music to his redesigned version B.

This case study tells the story of how the designer was rejected from a position at Apple and used it as an opportunity to redesign the core user experience of Apple’s Music app.

Observations:

  • Introduces three areas of focus (Core Experience, Brand Identity, Visual Interface) and offers a detailed rationale for each
  • Detailed prototypes that use animation when it helps convey a concept
  • Author’s personality shines through – clearly a passion project and not something driven by extensive user feedback

Assessment:

  • Shows initiative, someone who turns rejection into something positive
  • Demonstrates full range of talents, from design insight to sketches to prototype
  • This would move the writer’s resume to the top of a stack. This is phenomenal work.