By Kelsey Johansen
This Blog Post is a summary of a presentation called 'Performance Measurement: How to Collect Data that Matters!' delivered on September 22nd, 2023, as a part of the Recreation NB | Loisir N-B Trail Development Summit held at Killary Lake Lodge.
Performance Measurement in Trails
Performance measurement (PM) provides the feedback that keeps an organization on target.
It is the process of collecting, analyzing and/or reporting information about the performance of a group, organization, or system.
It collects and analyzes data to evaluate progress toward goals and objectives.
It can help improve accountability, decision-making, motivation, and resource allocation.
It aims to facilitate decision-making and resource allocation by providing data-driven insights into what works and is not working.
It communicates performance to stakeholders, funders, and government.
Strategic planning requires constant feedback about how well strategies are working. Performance measurement helps determine progress towards goals and objectives that aligned with strategic planning, including initiatives outlined in strategic plans and trail master plans.
When combined with organizational strategic planning, tourism readiness assessments, or provincial trail strategies, performance measurement keeps an organization on target while allowing for implementation of reasonable adjustments that help ensure timely, and resource-efficient achievement of goals and objectives.
Categories and Types of Performance Measurement
There are several categories of performance measurement, whether in the trail sector or others. They include:
Input-based measure – a measure of the resources used to produce a particular output or outcome.
Example: time, money, volunteer labour, in-kind supports, etc.
Output-based measure – a measure of the tangible results of a particular activity or process.
Example: 15 new kms of trail, or 5 km of trail swept and maintained after a severe weather event.
Outcome-based measure – a measure of the impact of a particular activity or process on a specific objective or goal.
Example: Increased individual and community-level engagement in trail-based recreational activities like walking or cycling, and / or using trails for active transportation following a brief marketing campaign or after new trail connections are established.
Process-based measure – a measure of the productivity and usefulness of a particular process or activity.
Example: decreased likelihood of accidents, incidents, and near-misses after introducing a new hike leader training program.
Quality-based measure – a measure of the quality of a particular product or service.
Example: Positive trail user experience(s) following introduction of a new trail wayfinging and interpretive signage program.
Financial measure – a measure of the financial performance of an organization or project.
Example: return on investment, revenue generation, cost savings / financial optimization, economic impact, etc.
Specific types of performance measurement, that fall within the categories listed above, include:
Key Performance Indicators (KPIs);
Customer Satisfaction Surveys;
Trail Count Data;
Trail Audits;
Risk Management Tracking;
Membership Data Tracking and Analysis;
Project-Specific Return on Investment (ROI);
Economic Impact Assessments; and,
Environmental Impact Assessments or Invasive Species Monitoring.
The type of performance measurement your organization selects should align with the objectives of your PM initiative; knowing the category of PM your organization or initiative wants to employ will help with this.
Why Measure Performance in and for Trails?
A performance measurement framework provides a structured approach to developing and implementing a system tailored to an organization or program’s goals and objectives.
Performance measurement helps individuals and organizations stay accountable, make informed decisions, stay motivated, allocate resources effectively, and continuously improve.
It tells the story of your trail, or trail organization, in a more compelling way;
It sheds light on connections to larger provincial and federal strategic priorities;
It supports applications for funding and justifies investments made by funders;
It supports outreach and advocacy work by demonstrating the impact of programming and amenities;
It demonstrates the impact of in-kind support from supportive businesses and partners;
It highlights the impact of volunteers; and,
It is a vehicle for community engagement.
Despite these benefits, there are also several drawbacks to undertaking performance measurement initiatives.
When designing a Performance Measurement (PM) Initiative, it is important to proactively factor these potential drawbacks into how the design your PM methods. This will help limit their negative impact on the initiative and your organization, including your Board and volunteers time, efforts, energy, and other resources.
Performance Measurement Methods
At its most basic level, performance measurement involves four key stages:
Identification of Performance Objectives;
Selecting Performance Metrics;
Collecting and Analyzing Data; and,
Reporting and Communicating Findings.
This is a simplified overview of the four stages. Each stage in explained in more detail, below, with examples.
Identifying Performance Objectives: Asking Good Questions
One of the biggest stumbling blocks for individuals, organizations and businesses interested in performance measurement is identifying what questions they should ask about their performance, and what performance objectives should measure in order to answer those questions. Some things that can help your organization to narrow it down include:
Strategic Plans;
Real World Problemsor Challenges;
Reporting Requirements; and,
Funding or Insurance Requirements.
Example 1: Strategic Plans and tracking progress towards achieving KPIs established in an organizational 5 year Strategic Plan.
Example 2: Insurance Requirements and tracking accidents, incidents and near misses
Next: Select and design meaningful performance measures that go beyond brainstorming and benchmarking and create buy-in and support for your PM initiative.
Select Performance Metrics: What to Measure and Why?
Performance measurement is more than trail counts, and trail counts don’t require expensive trail counters! Performance measurement in trails can be:
Customer Satisfaction Surveys e.g., Importance-Performance Analysis Surveys;
Trail Count Data e.g., Trailhead Counts, Strava Data, Volunteered Data;
Trail Audits e.g., Manual Review of Trails and Completion of an Audit Form;
Risk Management Tracking e.g., Tracking Accidents, Incidents and Near Misses; or
Membership Data e.g., Changes in Total Number, type, duration year over year, or increases following marketing campaigns.
Performance measurement can also include:
Project-Specific Return on Investment (ROI);
Economic Impact Assessment; and,
Environmental Impact Assessment.
These last three, bolded, items are slightly more complex to undertake, and may require partnering with external stakeholders (e.g., Chambers of Commerce, Destination Marketing Organizations) or external supporters (e.g., Faculty members from University Economics, Tourism Business, or Recreation and Leisure departments).
Even if your organization has the internal expertise to support these more intiatives, it may not have the time, or access to analytical software licenses. These are just two of the ways that partnering with a university can benefit a non-profit trail group.
Collecting Data: Make it Good
Collecting good data is fundamentally connected to whether or not the data is:
Reliable;
Accurate;
Good quality; and,
Manageable.
Reliable data enables better decision-making. Reliable data is complete and accurate. Ways to ensure you have a complete, and accurate data set include:
Testing your form or checklist before use, and,
Calibrating novel data (e.g., voluntary app-based data or trail counters).
Calibration refers to adjusting estimates of total use to account for how much use is represented by people included in the novel data source. Novel data refers to new data sources whose original purpose was not to count recreation engagement, but from which researchers have developed methods to extrapolate or estimate recreational use. It includes data from apps, websites, and cell phones. For example,
Cell phone tracking data: The information created when cell phone GPS features like 'location services' are turned on;
Fitness tracking data: The data generated from apps like Strava, AllTrails, Outdooractive, GoogleFit, MapMyFitness allow users to track the location, distance, and speed of workouts;
Socially-generated data: Data generated from social media platforms like Twitter and Instagram;
Social media posts: The photos, tweets, or other details users add to social media apps; and,
Voluntary app-based data: Any data users choose to upload to apps, including social media, fitness tracking, and photo sharing. This does not include cell phone data tracking.
Because novel data sources depend on some on-site counting, they cannot fully replace traditional trail counters and surveys. However, novel data sources can allow trail managers to estimate trail use where counters are not available or are too expensive, to better understand where users are going, and to proactively manage for changes in user and use trends.
Accurate data can also be achieved by simplify your data collection process, including simplfying your data collection form or checklist by:
Asking fewer, clearer, questions;
Creating simplified checklist criteria that will help avoid user error;
Going digital; and,
Training all of your data collectors to ensure accuracy.
It is also important to weigh quality data against the quantity of data you collect. A small amount of GOOD data (reliable and accurate) is often better than a large amount of BAD data (unreliable and inaccurate)!
This means asking good research questions that are tied to your performance measurement objectives.
All data also needs to be manageable. A good data collection technique need to have a low barrier to entry (a fancy way of saying it needs to be easy to use) and it should help you automate a workflow. This means for example, leveraging already collected data on membership or incident report forms, checklists, observations, or surveys to answer questions, and / or finding meaningful ways to store and analyze data as it comes in.
Analysis: Knowing When to Use What Approach
Performance measurement initiatives can collect numerous types of data. The two most common types are quantitative, and qualitative data.
Quantative:
Numbers – Descriptive, Correlational, Casual, Comparative.
E.g., Statistics, Demographics, Trends, Changes Over Time
Qualitative:
Word & Images – Content, Thematic, Narrative, Interpretative Analysis.
E.g., Themes, Observations, Issues, User Insights / Experiences
Whether you collect qualitative or quantitative data, it is often easier and less time consuming to figure out how you plan to analyze it before collecting it!
For example, you may want to collect quantative survey data using a digital or online survey tool. Digital or assisted data analysis makes the process quicker, easier, and less costly. For example, PM initiatives can use embedded analytical tools in online survey platforms. Some include: Survey Monkey Statistical Reporting, Auto-Generated Tables, and Charts, Excel, etc..
Sharing Your Findings
Depending on the goals of your PM Initative, you may need to communicate your findings to different audiences. Common audiences, and the outcomes of sharing with them, include:
Internal: Helps motivate, develop, and retain your staff and volunteers, as well as identify areas of improvement and alignment, etc.;
External – Industry: Helps to identify key trends, best practices, and challenges within and across the trails sector;
External – Government: Helps to increase political support for trails by improving understanding of the value of trails, etc; and,
External – Funders / Investors: Helps to demonstrate ROI, validates expenditures, ensures reporting obligations are met, etc..
A key consideration when planning to share your findings is the need to share the right information, in the right way, with the right person / group. When assessing this, trail organizations should:
Determine their Audience: Members / Users, Sector, Demographics, Degree of Tech Literacy, Time, etc.;
Determine the Objectives and Goals of Sharing: Inspire, Advocacy, Capacity Building, to met a Reporting Requirement, to secure support (In-Kind, Monetary, Political), etc.;
Determine the Platform and Method of Sharing: Virtual, In-person, In-Writing, Online; and,
Design Insightful and Actionable Reports and Dashboards: This should bring measures to life in a way consistent with your ‘why’ and keep the narrative focused on successes, transformations, and improvement.
Many graphic design platforms, like Canva, provide low or no-cost Pro-Version access for non-profit and charitable organizations. After taking a quick and free course at your library, or online through Youtube or LinkedIn, you'll be able to design infographics, social media posts, and newsletters to communicate the findings of your PM initiative in a meaningful way that is targeted towards your intended audience.
Impact
The four most common impacts of Performance Measurement initatives are:
Continuous Improvement
Supports benchmarking, progress tracking, and continuous improvement.
Increased sensitivity to user needs, meeting program outcomes, and achieving strategic objectives;
Strategic Resourcing
Ensures alignment of the activities of the human and other resources to the strategic and day-to-day requirements of the organization and making the full and best utilization of said resources.
Volunteers, staff, funding, etc.;
Celebrating Achievements
Ensures that organizational achievements are documented allowing for the celebration of accomplishments.
Volunteer recognition, acknowledgment of the impact of grants, and documenting organizational milestones; and,
Achievement of Goals.
Supports fact-based governance.
Employing evidence-based decision-making within local systems of governance improves transparency, accountability, fairness, and responsibility.
Knowing this will help you, and those supporting your PM initiative, to stay focussed on the impact and therefore motivated to push through any challenges that emerge along the way.
Funding and Collaboration
If the idea of tackling a PM initative is intimidating, whether it is concerns about your trail organization's internal capacity, how to fund the Initiative, or where to start, there are numerous funding and collaboration opportunities that can help to make the process more feasible. For example, it may help an organization to stary by identifying the following resources:
Internal: Existing data, volunteers, existing software and licenses, internal funds for matching external funding requirements, etc.;
External: Recreation NB, Nature NB, NatureTrust of New Brunswick, Watershed Associations, Provincial and MunicipalGovernment, Summer Student and Co-Op Programs, etc.; and,
In-Kind: Industry, Faculty at Universities / Colleges, Hiking NB, Canadian Trails Federation, Trans Canada Trail, IMBA, etc.
Additionally, partnering with other organizations can help open up acess to new funding opportunities that your organization would not be able to access if working alone.
Now What?
Before you dive into your next PM initiative, do not forget to:
Review your guiding documents (e.g., Strategic Plan, Constitution, etc).
If you don’t know what you are trying to accomplish, it is too early for KPIs!
Determine your ‘why’ and what influences it (internally and externally).
Determine the appetite for performance measurement within your organization and, where needed, build buy-in and support from staff and stakeholders so that you can enthusiastically own performance measurement, and improvement, as a whole.
Inventory the assets available to help you start a performance measurement initiative (e.g., know what data you’re already collecting).
Articulate your performance measurement strategy properly, including goals, objectives, and performance measures customized to each objective (e.g., KPIs).
Define and document both direct AND indirect measures.
Set targets and thresholds.
Identify and secure additional supports (in-kind, partnerships, monetary, resources, etc.).
Establish teams and roles; agree on a process and procedures.
Doing these things first will help to minimize any challenges associated with implementing your PM initiative.
Then What?
Then, work to foster a performance culture. Remember to communicate to those supporting your PM Initiative that:
Performance measurement doesn’t end with measuring performance!
Once you know how you have performed, it is essential to design and implement improvement initiatives. This requires:
A review process that allows you to transform your measurement data into evidence-based knowledge and understanding;
A plan for report writing and sharing information which is the first steps towards better decisions and improving overall performance;
Holding review meetings to review, interpret, and discuss performance information;
Designing effective improvement strategies based on goals, objectives, KPIs and knowledge gained; and,
Implementing your improvement strategies and re-assessing your performance using existing or updated measures.
Performance measurement isn't a one-and-done thing. It is cyclical and iterative.
Take Aways
Some key take aways from this talk, and its follow-up Blog Post, include that ...
Performance measurement provides the feedback that keeps an organization on target;
It doesn’t have to take a lot of time or money;
Its more than just trail counts (but can be those too!);
To be successful, performance measurement should be connected to strategic planning;
Performance measurement requires that you ask good questions, collect good data, and undertake sound analysis;
Sharing the right information, with the right people, in the right way, can increase the impact of your performance measurement initiatives beyond your organization; and,
Funding and external supports are available to help you get started.
Lastly, performance measurement can help trail organizations achieve their strategic goals and objectives, and doesn't have to be scary!
Where To Go from Here
If you're interested in how research, like Performance Measurement Initiatives, can support trails, consider checking out our earlier blog posts on The Value of Research for Trails and Trails, Social Enterprise, and Collective Impact: Insights from the Literature.
Get In Touch
If you have questions about incorporating performance measurement into your trail organization's standard operating procedures, have been requested to include them as a condition of receiving a grant but don't know where to start, or want to better document progress towards achieving strategic goals and objectives, please reach out to us at: info@trailresearchhub.com.