Welcome to Charity Navigator's Blog!

The team from Charity Navigator, the nation's largest independent charity evaluator and leading donor advocate, shares their thoughts on emerging nonprofit-sector issues and offers tips to better inform your intelligent giving decisions.

Return to Main Site   |   Find a Charity   |   Receive Email Updates   |   Support Charity Navigator

Tuesday, June 9, 2020

Providing a Window into the Mission - Visual Explorations

Written by Stephen Rockwell, Chief Ratings Technology Officer at Charity Navigator. Stephen is leading the team of charity analysts, program specialists, and development engineers that will launch the new rating methodology and criteria. Read the preceding two installments here.

As Charity Navigator evolves the rating system, we are considering ways to improve the visual interface of not just the ratings, but also the website. I came into the organization with a deep appreciation of the history of Charity Navigator, but also dissatisfied with a rather dated website design built with older technologies.  Therefore, we are evolving all three aspects of how we deliver the product at the same time: the ratings methodology and visual design, the website user interface and design of a nonprofit profile, and the underlying technologies that power both ratings and the website. We believe that we will deliver a much better experience that paints a more complete picture of the important work in which nonprofits engage the users and donors who rely on Charity Navigator.

An Alternative to Stars for the Experimental Rating System
At the outset of building an experimental rating system, the intent was to create a space to experiment while not affecting the existing 4-star rating system and the nonprofit organizations who rely on that system as part of their fundraising operations. We also do not want to confuse donors and users to the site.  We challenged our design agency, Second Melody, to develop an alternative visual concept for a rating system.  The recommendation was to use a 100 point system with the rationale being:

  • Stars often are used in user feedback systems. Whereas, numerical systems are more often used in evaluation systems.
  • We are all familiar with the 100 point system since we were all graded on one in our schooling.  This concept is portable to international understanding of evaluation systems as well.
  • There’s more ability to grade. This is important as we want to evaluate many more indicators and topics than we currently do. 
  • We potentially can convey gift worthiness better than stars.  For example, we consider 3-star nonprofits solid organizations worthy of a gift, but we know many donors and nonprofits don’t view it the same.  We may be able to offer a clearer passing grade.

With all of those factors in mind and without any further ado, below we have one mockup of that potential system….

As a reminder, the intent of the experimental rating system is to evaluate components of a nonprofit well beyond the two elements to our current star rating evaluation: finances and A&T (accountability and transparency).  Without getting into the details of the methodology (for another blog post), the recommendation is that we go with four measures underneath the overall score, as you can see in the graphics above. We would rate each of these aspects separately on a 100-point scale that would ladder up into the overall score (weightings to be determined).  In the visual concept, passing scores would fill in a beacon.  We are also exploring modalities where the visual would be delivered with a scale so that you know what a passing grade is and/or how the organization performs relative to other nonprofits.

User Testing: Stars vs. Numerical System
In order to determine the validity of the numerical rating system, we performed multiple user tests with current donors/users of the website, nonprofit leaders, and non-users. We tested across a multitude of factors and tested the numerical system and star system side by side.  The results were surprising.

In general, the numerical rating far outperforms the star rating across a number of attributes including: representing nonprofit performance, comparison of performance of multiple nonprofits, worthiness of a donation, accuracy, and trust. The one attribute that was more evenly matched was ease of use of the rating system.

These tests provide some comfort in moving forward with a numerical rating for our experimental rating system. 

 Other Visual Explorations
As we continue our exploration of a numerical system, we are asking other branding and visual concept questions:

  • Does the overall score connote gift-worthiness and that a donor should trust that nonprofit or do we need to provide some other top-level visual?
  • Do we go with a 4-beacon system for ease of understanding?  How do we best show users a passing grade?
  • Should we provide some scale of performance against the rest of the sector?
  • Should we brand the system with its own name that would complement the Charity Navigator brand knowing that we have multiple products (Ratings and our Giving Basket donation platform)?
  • Should we give the rating system its own logo and mark?
  • How best do we lay all of this information out on a nonprofit profile page on our website?  How do we meet the needs of users who just want the top-level information while also providing the underlying data to users who want to do the deeper dives?
  • How best do we render our Advisory System on the new kinds of pages?  Can we better layout the underlying details of the advisory so that a user can quickly see that a nonprofit is having an issue of which donors should be aware?

We will be testing on all of these questions in the coming weeks and look forward to hearing your feedback as we move into our beta launch that will allow you to explore the new rating system and new nonprofit profile pages.

1 comment:

Unknown said...

I'm looking forward to your new rating system.
I hope you continue to show the salaries of top officers - CEO, etc.
Thank you for what you do!

Rev. David N. Moose
Louisville, Kentucky