Exploring Human-Centered Design methods to uncover performance evaluation needs for an NPO
Grappling with value-centeredness and unique roles, non-profit organizations can face struggle in effectively evaluating staff using traditional performance measures. We worked with an NPO in Chicago, IL to understand how we might approach this with a design research lens.
NOTE: Collaboration organized by the Steans Center at DePaul University to explore Service design for community service - Coursework for Design Ethnography, group of 3 graduate students.
our design partner, and what they desired
ICA, a Non-Profit Organization based in Chicago, sought a creative performance evaluation solution. We had 8 weeks to figure it out together.
Our partner desired a performance evaluation system that is
Amount of work, quality of deliverables, Work outcomes
Alignment of performance with company values
In many ways, a Non-Profit Organization may function differently from traditional work settings - job roles, team structures, hierarchies, motivation and desires of staff. Since the evaluation process runs over long timelines, it was essential for us to delve deeper into how people at ICA do things.
our design research plan
Ethnography vs. Design Ethnography
Researcher's descriptions in the form of detailed accounts of people, cultures, customs.
Ethnographic accounts of specific settings and focus areas based on a design framework.
Participatory and co-creation methods where both research and subject parties are contributors
We learned more about why our partner might need a tailored solution, rather than opting for an existing Performance evaluation software (e.g. Lattice)
We met a representative staff member in an unstructured, informal interview where we learned about company structure, teams, work and culture.
roles & team structure
relatively flat hierarchy
unique roles and overlapping teams based on projects
We formulated a research plan, focussing on key aspects that may affect performance evaluation
To understand practices around long term and short term aspects that may contribute to evaluation, we learned more from a sample of people that could represent the diversity of roles, teams and work settings.
We asked how people
- Describe their roles and responsibilities
- Work on projects, timelines, teams
- Document work for reporting
- Report work; what and how
- Imagine desired ways of evaluation
- How people work in their settings
- How people communicate for work or casual purposes
- Workplace culture
preparing Raw data
57 pages of raw data
We transcribed our interview and observation notes for analysis
iterative data analysis approach
Using various methods and frameworks, we sought to extract insights and sentiments in an iterative analysis approach.
1. developing an analytical framework
We used an inductive coding methodology using 10% of our data towards an analytical framework
analytical framework in the form of a codebook used by our team
2. data analysis using the framework
Using a deductive coding methodology, we applied the codebook to analyze all our raw data.
3. analysis: mapping patterns in our data
Grouping saliently occuring codes together, we used an affinity mapping methodology to map patterns and relationships across our data.
findings - unique settings = non traditional needs
Responsibilities beyond said roles
- Changing responsibilities, adapting to need
- Autonomy, self- accountability in planning and executing work
- Collaboration across teams
- Multiple tasks outside said roles
Need for a feedback framework to augment current self evaluation
- Time on documented projects
- Feedback from stakeholders on individual projects
- Verbal check ins with supervisors to confirm progress and discuss issues
Need for documentation for reporting
- Current documentation for project reports
- Other documentation involving day-to-day work planning
- No explicit documentation for reporting
- Over the long term, only major outcomes are remembered
Values embodied in day to day work
- Across unique roles, staff demonstrated similar values
- But company values may not be the same as staff values
- Staff embodied company and personal values and demonstrated common traits in ways of approaching work
SOLUTIONs | a design toolkit to begin the transition
Based on our findings, we devised a set of solutions to address the needs and desires identified
Staff values may =/= company values.
Documenting values embodied in work.
Using human-centered methodologies to elicit values, we created a documentation of values and value definitions from established HR literature
1:1 reporting tool to use in conjunction with existing documentation methods to capture work for evaluation
Creating a simplistic near term 1:1 documentatation format using AirTable helped us think of ways where users could "copy-paste" pointers leveraging project-related digital documentation.
With this setup, we provided an extensive tutorial of how to use and customize this.
More than an end-all solution, this tool aimed to make the case of the need near term documentation for reporting purposes, that may tie in with current practices.
Tying in these practices with long term reporting that may happen annually/ bi-annually
We provided a comprehensive review of existing performance review of tools, and features that could leverage near term reporting and value mapping practices described above - Keeping in mind that long term reporting statistics may be used for reporting company growth and overall growth to stakeholders.