Establishment of Performance Data Collection and Reporting Standards

Status: Candidate    
Potential Funding Sources:    
Timeframe: Short-Term
Research Period: 12 months
Funding Estimate: $200,000
Potential Sponsors:


This research will recommend data collection and reporting standards and develop an implementation plan to improve data quality.

Background

In an era of increased demand for transparency and accountability, transportation agencies are embracing performance management practices. A key component of this approach is the tracking and reporting of a wide range of performance measures. The recent passage of MAP-21 will add a set of national performance measures to the body of performance data being publically released. In order to realize the notable benefits from the increased monitoring and sharing of performance results, the industry must address:
• The current absence of performance data collection and reporting standards
• The need to build a foundation for effective performance management approaches through quality data
• Without data collection and reporting standards, publically available performance information that appears to be the same can in fact be an “apples to oranges” situation
Given this, there is a vital need to work towards ensuring that transportation agencies can consistently obtain quality data for use in performance management practices.
“Quality data” can be succinctly described as data exhibiting the following six characteristics:
• Timely (available within a reasonable timeframe)
• Complete (all required data elements included)
• Accurate (free of errors)
• Consistent (same definitions and collection standards applied)
• Integrated (linking different pieces of data straightforward)
• Accessible (data is easily retrieved by all potential users).
The creation of quality data is dependent on the data supply chain steps:
Definitions, Acquisition, Processing, Validation, and Reporting.
Given that agencies with varying transportation systems, technological capabilities and staff resources will be adopting performance management principles, it will be important to establish data collection guidelines that can reasonably fit within existing agency practices. For data to be used smoothly and effectively, strong communication is essential among data producers, analysts and decision makers.
Establishing data collection standards will also prepare agencies for MAP-21 requirements to establish performance targets, prioritize resource allocation decisions and report on results.

Research Objectives

The purpose of this project is to:
1. Recommend data collection and reporting standards
2. Develop an implementation plan to improve data quality across all transportation agencies
3. Provide standardized reporting guidelines

Proposed Research

1. Document recommended data collection and reporting standards: Conduct literature review and draft technical memorandum containing descriptions of reviewed documents and relevance to establishing data collection and reporting standards; include summary of issues that may impact data collection standard recommendations
2. Data Collection and Reporting Standards Implementation plan: Given variable characteristics of transportation agencies, the implementation plan will not be a “one-size fits all” approach, but provide a range of recommendations to improve the data quality while balancing the need for nationwide consistency and agency technology, staff capability and resource constraints. The implementation plan will be specific to the five steps in the data supply chain:
a. Data definitions: Do clear data definitions exist? Where is there a need for further clarification?
b. Data acquisition: What type of data collection methods are used by the agency? How frequently is the raw data collected? How comprehensive is the data collection coverage?
c. Data processing: How are agencies compiling the raw data and turning it into daily, monthly or annual data? What processing techniques are used ? Where is the data stored? What documentation exists for what is excluded from computations?
d. Data validation: What cross checks are used to validate data? Is there a max/min threshold value used to flag data? What certification procedures are followed?
e. Data reporting: For each metric what are the procedures for internal reports, stakeholder reports and federal requirements? Does documentation on data submission processes exist?
3. Reporting Templates: Conduct a scan of existing performance reports to develop recommended standardized reporting templates and presentation examples for a range of performance information.

Potential Benefits

• Establishing data collection and reporting standards will clarify the necessary steps, roles and overall workflow to obtaining quality data.
• Developing and documenting standards will also help identify points of risks. By identifying risks, future data discrepancies can be avoided and the data supply chain can provide guidance on “why” the discrepancies occur when they do. In essence this research will help prevent a “garbage in- garbage out” situation.
• By advancing good data business management practices, performance management practices will improve.

Implementation and Follow-on Activities

Conduct a pilot to test the recommended implementation plan and usability of the reporting templates.

Related Research

• NCHRP 8-92 Implementation Transportation Data Program Self-Assessment
• State DOT Strategic Business Plans, the comparative performance measurement effort for state DOTs initiated in 2004 (NCHRP 20-24 (37)
• The recently established AASHTO communications portal to assemble examples of noteworthy practices in reporting performance (www.communicatingperformance.com)


X