Exploring Human-Centered Design methods to uncover performance evaluation needs for an NPO 

Grappling with value-centeredness and unique roles, non-profit organizations can face struggle in effectively evaluating staff using traditional performance measures. We worked with an NPO in Chicago, IL to understand how we might approach this with a design research lens.

NOTE: Collaboration organized by the Steans Center at DePaul University to explore Service design for community service - Coursework for Design Ethnography, group of 3 graduate students.

 our design partner, and what they desired 

ICA, a Non-Profit Organization based in Chicago, sought a creative performance evaluation solution. We had 8 weeks to figure it out together.

Our partner desired a performance evaluation system that is

1. work-centric 


Amount of work, quality of deliverables, Work outcomes

2. value-centric 


Alignment of performance with company values

In many ways, a Non-Profit Organization may function differently from traditional work settings - job roles, team structures, hierarchies, motivation and desires of staff. Since the evaluation process runs over long timelines, it was essential for us to delve deeper into how people at ICA do things. 

 our design research plan 

Ethnography vs. Design Ethnography

Researcher's descriptions in the form of detailed accounts of people, cultures, customs.

Ethnographic accounts of specific settings and focus areas based on a design framework.

Participatory and co-creation methods where both research and subject parties are contributors

We learned more about why our partner might need a tailored solution, rather than opting for an existing Performance evaluation software (e.g. Lattice)

We met a representative staff member in  an unstructured, informal interview where we learned about company structure, teams, work and culture. 

roles & team structure 

relatively flat hierarchy

unique roles and overlapping teams based on projects

We formulated a research plan, focussing on key aspects that may affect performance evaluation

To understand practices around long term and short term aspects that may contribute to evaluation, we learned more from a sample of people that could represent the diversity of roles, teams and work settings.

structured interviews

We asked how people 

- Describe their roles and responsibilities

- Work on projects, timelines, teams

- Document work for reporting

- Report work; what and how

- Imagine desired ways of evaluation 

We observe

- How people work in their settings

- How people communicate for work or casual purposes

- Workplace culture 

observation studies

preparing Raw data

57 pages of raw data

We transcribed our interview and observation notes for analysis

 iterative data analysis approach 

Using various methods and frameworks, we sought to extract insights and sentiments in an iterative analysis approach.

1. developing an analytical framework

We used an inductive coding methodology using 10% of our data towards an analytical framework 

analytical framework in the form of a codebook used by our team

using design theories [1,2] of value elicitation from staff responses

2. data analysis using the framework

Using a deductive coding methodology, we applied the codebook to analyze all our raw data. 

3. analysis: mapping patterns in our data

Grouping saliently occuring codes together, we used an affinity mapping methodology to map patterns and relationships across our data.

Untitled-1_edited_edited.jpg

 findings - unique settings = non traditional needs 

Responsibilities beyond said roles

- Changing responsibilities, adapting to need

- Autonomy, self- accountability in planning and executing work

- Collaboration across teams

- Multiple tasks outside said roles

Need for a feedback framework to augment current self evaluation

- Time on documented projects

- Feedback from stakeholders on individual projects 

- Verbal check ins with supervisors to confirm progress and discuss issues

Need for documentation for reporting

- Current documentation for project reports

- Other documentation involving day-to-day work planning

- No explicit documentation for reporting 

- Over the long term, only major outcomes are remembered

Values embodied in day to day work

- Across unique roles, staff demonstrated similar values 

- But company values may not be the same as staff values 

- Staff embodied company and personal values and demonstrated common traits in ways of approaching work

 SOLUTIONs | a design toolkit to begin the transition 

Based on our findings, we devised a set of solutions to address the needs and desires identified

#1

Staff values may =/= company values. 
Documenting values embodied in work.

Using human-centered methodologies to elicit values, we created a documentation of values and value definitions from established HR literature

#2
 

1:1 reporting tool to use in conjunction with existing documentation methods to capture work for evaluation

Creating a simplistic near term 1:1 documentatation format using AirTable helped us think of ways where users could "copy-paste" pointers leveraging project-related digital documentation.

With this setup, we provided an extensive tutorial of how to use and customize this.

More than an end-all solution, this tool aimed to make the case of the need near term documentation for reporting purposes, that may tie in with current practices.

#3

Tying in these practices with long term reporting that may happen annually/ bi-annually

We provided a comprehensive review of existing performance review of tools, and features that could leverage near term reporting and value mapping practices described above - Keeping in mind that long term reporting statistics may be used for reporting company growth and overall growth to stakeholders.