1	Clarus Multi-State Regional Demonstration Program: Summary of Evaluation Findings Deepak Gopalakrishna
2	Team Composition Paul Pisano, COTM, RWMP, FHWA Chris Cluett, Battelle Deepak Gopalakrishna, Battelle Kevin Balke, Texas Transportation Institute Dan Middleton, Texas Transportation Institute Bob Brydia, Texas Transportation Institute
3	Four Case-Study Evaluations Use-Case 2: Seasonal Load Restriction Decision Support Use-Case 3: Non-Winter Maintenance Decision Support System Use-Case 4: Multi-State Control Strategy Tool Use-Case 5: Enhanced Road Weather Traveler Advisories – Discussed in later session
4	USE-CASE 2 SEASONAL LOAD RESTRICTIONS
5	What we evaluated? Are load restriction placements with tool more timely? Are states satisfied that the tool is helping them make better restriction decisions? How commercial operators view the restrictions process and the value to them of better timed restrictions?
6	Evaluation Approach Review of timing decisions and tool usage Interviews with maintenance decision makers Commercial Vehicle Operator survey Worked with North Dakota, Montana DOTs
7	Results Summary Advance notification of sub-surface conditions a real and new benefit Up to 7-14 days of forecast information Comfort level with temperatures: high. Other outputs: unknown. Led to following examples of use Availability of tool forecasts supported NDDOT’s decision to hold off on restrictions by 14 days following a early warming spell in February Difficult to assess whether the decision would have been different if interviewees didn’t have access to this tool but they confirmed the value of the tool during this time A MDT maintenance chief was able to decide to place restrictions at least 7 days in advance based on his interpretation of the tool’s subsurface temperature profiles.
8	Results Summary State DOTs see value and potential in this tool Four state DOT personnel who responded to a survey on their experience with the tool reported their perception of “reliability and/or accuracy of the tool and the information it provides” with an average score of 7.3 on a scale of 0 to 10. Reported their average level of trust in the tool as 6.8 (scale 0-10), which is fairly positive but with variation across users. They expect over time that the use of this tool can shorten the restriction period and support more advance notice at the beginning and end of the restriction period of 7 to 10 days. The tool provides State DOTs with an approach for determining return of pavement strength.
9	Results Summary Commercial Vehicle Operators have high expectations and concerns during restriction season 72% of motor carriers to change their routes (more mileage) and 66% to alter/divide loads (higher costs) to meet restrictions. Reducing the length of time of the restriction period was important to 86% of the responding motor carriers. Improvements noted by the respondents included facilitating permitting convenience, adjusting weight policies, additional coordination between state and counties, use of speed restrictions instead of load restrictions and finally to upgrade roads and eliminate restrictions. Value greater consistency and fairness in the restriction process. Most carriers appreciate the need for restrictions but want them to be less burdensome and of shorter duration
10	Improvements suggested by users Changes to the user interface State level overlay of sub-surface conditions Alerts and advisories based on thresholds Road segment-based view Verification and validation of sub-surface conditions
11	USE-CASE 3 NON WINTER MDSS
12	What we evaluated? Does the use of the non-winter maintenance decision support tool lead to more efficient and productive scheduling decisions compared with historical approaches? Are road maintenance resources (labor, equipment, material) assigned more effectively and efficiently by using the tool? Are weather forecasts provided by the tool more actionable than previously used services/products?
13	Evaluation Approach Worked with maintenance districts Illinois DOT and Iowa DOT to track activity scheduling using the tool April 1^st to October 1^st, 2010 for Illinois and April 1^st to August 30^th, 2010 for Iowa DOT. “With and Without” methodology Review of detailed logs of their activities performed and the weather conditions encountered in the field. Interviews with the maintenance crew chiefs and supervisors to understand qualitatively the potential of the tool for use in non-winter operations.
14	Results Summary No differences in scheduling approaches were observed between the control and experimental groups in Illinois In Illinois, the tool was not as useful primarily due to implementation and software difficulties. Over the 64 day evaluation period, 15 days were impacted by weather. Of these, the tool was able to provide notification only 3 times to the experimental group Greater flexibility in schedule adjustments reported in Iowa 11 schedule changes were made based on the use of the tool in about 37 days. Supervisor was able to effectively use the tool to adjust daily schedules
15	Results Summary Crew safety No instances where crews were in unsafe conditions in either control or experimental districts Integration of weather capabilities are well-appreciated. Communications from the tool to field personnel critical Need to push alerts through BlackBerry Alerts should be highly location-specific Alerts should be clearly linked to the activity which triggered the alert High degree of flexibility at maintenance sheds for routine activities Keeping crews productive is not a problem Chiefs make day to day decisions on activity schedule looking not only at weather but also staff mix, geographic distribution of activities Flexibility ensures that advanced planning is not required for a host of daily/routine activities
16	Results Summary Overall, both Iowa DOT and Illinois DOT noted that the tool and the concept should be easy to adopt if some of the technical issues were resolved with respect to the software. While still not operations ready, the primary users of the tool (maintenance supervisors) provided continuous feedback that already has led to various software enhancements. Focusing on high-priority activities can help generate support for the tool Specific maintenance activities (concrete, asphalt) work Activities requiring resources from other parts of states Contracted activities
17	USE CASE 4 MULTI STATE CONTROL STRATEGY TOOL
18	What can the evaluation tell us? How did having access to the tool change responses of agencies during actual weather events? Did agencies use predicted information to make control decisions? Did agencies feel they were better informed and had better access to information about surrounding conditions and actions of other agencies when they made their control decisions?
19	Evaluation Approach Evaluation of use and opinions of specific stakeholders in the Quad Cities area Table top exercise Interviews Usage statistics were collected to determine the frequency with which alerts were received about the above listed weather conditions and whether and when agencies responded proactively to these alerts.
20	Results Summary -- Tabletop Exercise Usefulness of Tool Best suited for use in dispatch centers. On-scene responders too busy to access and enter information Provides good method for generating record of responses. Need capability to sort through responses. Agencies want automatic methods of keeping information up-to-date or identifier of when information last updated. Agencies want flexible way of generating new conferences on fly – invite subscribers
21	Results Summary -- Tabletop Exercise Weather Information Agencies want to have weather information on top with pop-up for messages Agencies want processed weather information and not raw weather data itself What does it mean from dispatcher’s perspective? Agencies want information about forecasted impacts of weather on event Tell me when to expect weather to start and when to expect it to end Tactical decision-making: When to call crews in and how long to keep them
22	Results Summary Weather Alerts Need to be careful how you set up alert – easy to be inundated with alerts Weather information found to be generally valid – alerts generally agreed with known conditions Except for frost alerts Agencies want easy interface that will allow novice user to establish their own alerts. User has to be able to translate weather data into roadway impacts Tool provides similar functionality as other existing tools (in terms of alerts). Agencies want more sophistication in establishing alerts
23	Results Summary Interviewees unclear on the value of the tool when compared to several existing interfaces between stakeholders that promote information sharing Value of weather information is greater when integrated with existing processes and tools
24	OVERALL CONCLUSIONS
25	Overall Conclusions 4 credible applications developed collaboratively incorporating Clarus system data Generally, state DOTs participating in the evaluation have positive reactions to the tools Concepts well received. Tools can be developed further Operational and deployment readiness – Some closer than others
26	Some lessons learned Data is good, information is better Users know what they want; they are not sure how they want it or how they will use it! Users don’t distinguish between front-end systems and back-end systems Layering of information is critical. Don’t provide data because it is there. Make clear the value proposition compared to existing approach to the end-user
27	For More Information: Paul Pisano Road Weather Management Program, FHWA (202) 366-1301 paul.pisano@dot.gov Chris Cluett Battelle (206) 528-3333 cluett@battelle.org Deepak Gopalakrishna Battelle (202) 479-9645 gopalakrishnad@battelle.org