Decision-Making Framework for Performance and Data

Status: Concept    
Potential Funding Sources: NCHRP 20-24    
Timeframe: Years 3-4
Research Period: 24 months
Funding Estimate: $250,000
Potential Sponsors:


Background

Under MAP-21, performance management will provide a means to more efficient investment of Federal transportation funds in part by improving transportation investment decision-making. However, the data needs and framework to inform those decisions still needs to be defined and improved. This research asks the questions:
• To what extent have transportation policy makers analyzed the correlation between investment and performance?
• If fully armed with perfect performance or other data, would two different policy makers land on the same investment decision?
• Evolutions of federal authorization have helped shape the transportation data landscape. Fatalities, condition of pavement and bridge elements, and travel time delays are widely reported and relied upon for program-level and project-level investment criteria. But what other information and qualitative factors help guide executives and policy makers?
• With endless data points but without the ability to verify, synthesize, or analyze it, how can the information be useful?
The term DRIP, or Data Rich and Information Poor, resonates with many Chief Executive Officers. A framework for better connecting data to performance-based decisions is needed to add strength to performance management efforts and value to our transportation system.

Research Objectives

This research endeavors to guide transportation officials in better aligning technology, data, and analytic resources with their most important investment decisions through the following:
1. Understand the relationship between performance, decision-making, and data in order to develop a framework that transportation agencies can use for improved decision-making and optimizing data investments.
2. Develop a framework for performance-based decision-making that transportation agencies can use to establish good business processes, produce good data, and yield good performance.
As a military leader, Colin Powell applied a doctrine of 40-70. Making a decision with less than 40% of the available information was premature. Waiting for more than 70% of the available information, however, can prove costly from the standpoints of data collection and time lost for implementation. This research will seek to determine whether such a rule be applied in transportation to capacity-adding projects or operational improvements. This will assist agencies in determining when existing data is adequate to make investments at various levels.

Proposed Research

1. Develop a framework including key elements for a business model with minimum data/information needed to make good, defensible decisions. The framework would include (a)
a. Performance measures
b. The range of performance results likely depending on data availability (i.e. better data would yield more certain target setting)
c. An agency’s analytic maturity
2. Scan data gatherers and information owners to determine relative costs of data – performance and otherwise – by data category (e.g. Safety vs. Traffic vs. Infrastructure Condition vs. Customer Satisfaction vs. Environmental Impact).
3. Survey transportation executives on the role and importance of performance and other data in decision making.
a. Assess whether the most crucial decisions are made quantitatively or qualitatively.
b. Determine gaps in both necessary information and staff skill sets in effectively delivering that information.
c. Group decision types by those that most heavily rely on data and those that least heavily rely on data.

Potential Benefits

This research will help agencies understand the connection between investing resources in gathering data and the quality and scope of the decisions they can make. Without this understanding, agencies may cut costs on data tasks that can cause problems later.
For example, a transportation agency may hire several engineering consultants to refine design plans and cost estimates for a large infrastructure project, only to witness cost savings evaporate as interest rates rise and financing costs consume all value engineering. Or it may deploy bare bones technology resources to inform travelers of delays on its network only to watch travelers use more reliable privately funded applications.

Implementation and Follow-on Activities

1. Build a “New Executive Handbook to the Use of Transportation Information in Decision Making”
2. Analyze correlations between transportation investments and resulting performance; leveraging existing studies in this area, determine whether states that spend the most relative to their size are, over time, achieving higher performance
3. Comparing states with similar performance in commonly measured performance areas, determine whether those states are making similar investment decisions. Develop a structure including template job descriptions and hiring practices for developing a Business Analysis Unit inside a public transportation agency
4. Examine state DOT processes to determine when and where most critical resource allocation decisions are made (Long-range plan, Statewide Transportation Improvement Program, Annual budgeting, Statewide or district project selection, etc.)
5. By logical grouping (e.g. seven national performance areas), offer best practices in data gathering, reporting, and analysis among public transportation agencies in the U.S. and abroad. Where applicable, offer case studies from the private sector (e.g. shipping company analysis of logistics and travel delay and costs)
6. Survey cross-asset or other resource allocation tools

Related Research

The area of highway safety is an area of transportation that relies heavily on data-driven decision making is the area of highways safety. National Highway Safety Transportation Administration (NHTSA) requires for grant eligibility that benefit-cost calculations be performed using NHTSA standards.
• NCHRP 667 examined core competencies required for highway safety and discussed “the environment in which road safety decisions often take place and data supported decision-making in terms of problem identification, intervention planning, and evaluation.” This research could serve as a representative sample of decision making research needed across transportation agencies.
• NCHRP 8-92 Implementing Transportation Data Program Self-Assessment tests the feasibility of the data program self-assessment process and seeks to operationalize the framework and produce a guide for transportation agencies to implement a data self-assessment.
• FHWA’s Performance Based Planning and Programming Guidebook is designed to highlight effective practices to help transportation agencies in moving toward a performance-based approach to planning and programming.
• International Roughness Index has long been a data gathering requirement for the Highway Performance Monitoring System. FHWA examined disparities in data gathering and reporting for three states along a common Interstate. Beginning with this and related studies, this research could explore whether executives in similar situations with comparable data are making like decisions.
• Several academia and private sector studies may also support this research, such as the Empirical analysis of transportation investment and economic development … by Berechman, Ozmen, and Ozbay.



X