Potential Funding Sources: NCHRP 20-24
Research Period: 18 months
Funding Estimate: $250,000
This research will develop a data assessment tool for systematic evaluation of external data sources to be used in performance measurement and management.
Transportation agencies seeking to strengthen performance measurement need a variety of data to understand both performance trends and causal factors. Some performance data are collected directly by agencies, but there are a variety of existing and emerging data sources that can be used to supplement agency-collected data. These include data from federal, state and regional governmental agencies as well as data available from commercial sources. Transportation agencies would benefit from a tool that helps them to assess the applicability and usefulness of potential available data sources to support their performance management program. In turn, this would allow them better ways to report performance using available data.
The objective of this research is to develop a data assessment tool that helps a transportation agency to systematically evaluate an external data source for supporting performance measurement and management.
• Define a set of criteria for evaluating suitability of data sources for performance management. These criteria should consider data content, scope, level of spatial and temporal granularity, collection method, availability of documentation and metadata, distribution format, availability of support, cost and stability of provider
• Identify rating scales and categories based on the criteria
• Select a diverse sample of data sources, and develop ratings based on the established criteria and rating scales
• Review the examples with a stakeholder group, and revise based on feedback
• Develop a graphical method for presenting data source ratings (for example, using the approach found in Consumer Reports)
• Produce a guide that provides agencies with instructions on how to apply the rating methodology
Availability of a data evaluation tool would provide a uniform method for evaluating and comparing different potential data sources. It would help transportation agencies to better understand and anticipate relative strengths and weaknesses in data sources in order to make well-informed choices, and plan steps to compensate for identified data source limitations.
• Disseminate the data rating guidance
• Assess further applications of the ratings methodology – e.g. an annual ratings report, a self-reporting process, etc.
Previous work on data include:
NCHRP Report 666: Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management provides a framework and specific guidance for setting performance targets and for ensuring that appropriate data are available to support performance-based decision making.