Evaluation Strategy White Paper (Final Draft)
Contract #: DTFH61-02-C-00061
ITS Program Assessment Support
Contract Task SA61020
Submitted to:
ITS Joint Program Office
Department of Transportation
Submitted by:
Science Applications International Corporation
1710 SAIC Drive, M/S T1-12-3
McLean, VA 22102
June 21, 2006
Table of Content
5.0 Data Collection Techniques
1.0 Introduction
The Camera Phone Proof of Concept project consists of a joint partnership between the FHWA and the University of Maryland’s Center for Advanced Transportation Technology (UMD-CATT). The goal of this project is to improve incident management and response activities in the event of a traffic incident or other emergency situations that affect traffic operations. This project will consist of two main objectives - first to demonstrate the feasibility of using commercial off-the-shelf (COTS) wireless telephones equipped with cameras to capture and deliver traffic incident imagery that is useful to follow-on responders, such as tow companies, HAZMAT remediation services, health departments, or highway repair teams. Secondly, to assess the value of these images to follow-on responders based on improvements in time, safety, and efficiency while responding to and clearing traffic incidents. This project will be coordinated with UMD-CATT’s ongoing Capital Wireless Information Net (CapWIN) program and will be implemented in the Washington D.C. Metropolitan (Metro) Region.
2.0 Project Background
Throughout the United States, many areas have seen increased levels of traffic congestion. Not surprisingly, traffic incidents (and emergency situations) which further increase traffic congestion are a daily occurrence. These incidents include automobile crashes, HazMAT spills, disabled vehicles, and other situations that limit traffic flow. In many cases, these traffic incidents produce conditions that result in secondary crashes (further exacerbating congestion) and create dangerous situations for responders attempting to clear the incident or manage traffic through the incident scene. It is therefore imperative that incidents be cleared from the roadway as quickly as possible.
During a traffic incident, emergency response agencies (e.g., safety service patrol, law enforcement, fire and rescue) often require additional assistance from follow-on responders in order to clear the incident from the roadway. While emergency responders can use wireless telephones, radios, or dispatcher services to call for assistance, they typically do not have the ability to transmit detailed incident imagery directly to the follow-on responders. Although CCTV cameras have been deployed and cover large segments of the road network serving the Metro Region, many segments remain without coverage. In addition, specific details that would be useful to follow-on responders may not be visible to CCTV operators due to vehicle positioning or CCTV resolution limitations. As a result, follow-on responders may arrive at the incident scene without the proper equipment (e.g., tow trucks, HAZMAT transporters) or personnel required to address the situation quickly and efficiently. In many of these cases, follow-on responders are forced to return to headquarters or call for additional support in order to get the right equipment to the scene. This results in increased incident duration, causing longer traffic delays and prolonging the hazardous situation. The inability to clear incidents quickly and efficiently continues to put pressure on Washington’s roadways.
3.0 Project Summary
In order to address these challenges, FHWA and UMD-CATT will distribute between 20 and 40 COTS wireless camera telephones (i.e., camera phones) depending on the number of individuals taking part in the proof of concept project. Current participants in this project will include the Virginia Department of Transportation (VDOT), the Maryland State Police, nad and several local commercial towing and recovering services. Each camera phone will employ wireless picture phone technology that will enable participating agencies to take and transmit digital photos to one another. Each camera phone will be used by emergency response agencies to take detailed pictures of traffic incident scenes and then transmit those images directly to their dispatcher and the appropriate follow-on responders. Typical COTS camera phones are equipped with VGA (video graphics array) cameras with 640x480 resolution and digital zoom features. They may also be equipped with Enhanced Data rates for GSM Evolution (EDGE) data communications capabilities, resulting in improved transmission times and reliability. As a result, follow-on responders will then be able to receive these images on their own compatible camera phones or on any computer with an internet connection. It is expected that incident imagery (which may also include short video clips) will help follow-on responders have a greater understanding of the incident, which will allow them to better define their operational procedures and select the most appropriate equipment or personnel before leaving for the incident scene. Consequently, this should allow follow-on responders to arrive at the incident scene with the proper resources in hand, resulting in an accelerated incident remediation process.
3.1 Operational Concept
The concept of operations (CONOPS) for the Camera Phone Proof of Concept project identifies the processes, roles and responsibilities, policies and data flows that illustrate how this project is expected to improve incident management and response activities. The CONOPS serves as a road map for the Evaluation Team in developing their evaluation strategy. In order to provide a comprehensive and accurate assessment of this project, the Evaluation Team will review several areas represented in the CONOPS, including the technical and operational aspects that are incorporated throughout the proof of concept project.
Figure 1 illustrates the initial CONOPS that was developed during July 2005. It is expected that this CONOPS will be modified after all project participants have been identified. At a recent stakeholder meeting held on April 5, 2006, the final CONOPS was still in development. Once completed, the updated CONOPS will be reflected in the final Evaluation Strategy document.
Figure 1 - Camera Phone Concept of Operations
4.0 Project Timeline
The schedule presented in Table 1 illustrates the activities that will take place during Phase I of the proof of concept project and the evaluation. The evaluation will be conducted in two phases. During Phase I, lessons learned will be documented and posted to the Lessons Learned Database maintained by the Joint Program Office. These will be posted on a monthly basis throughout the project. During Phase II, a Case Study will be developed in draft and final versions.
Table 1 - Project Schedule and Milestones – Phases I and II[1]
Tasks |
Target Dates (s) |
|---|---|
Obtain Phones |
Week of July 10 2006 |
Develop Training Materials |
Week of July 10 2006 |
Configure Phones & Schedule Training |
Week of August 7, 2006 |
Distribute Phones and Training Materials |
Week of August 7, 2006 |
Proof of Concept Duration |
August through October, 2006 |
Evaluation Phase I - Documentation of Lessons Learned |
August through October, 2006 |
| Evaluation Phase II – Development of Case Study[2] |
Draft – January 15, 2006 |
5.0 Evaluation Approach
This section presents the core objectives and areas of concentration for the evaluation.
The main focus of the Evaluation Team is to qualitatively assess the benefits realized by the use of wireless camera phones to improve response capabilities for traffic incidents or emergency situations. The following assessment areas are included in the evaluation:
- User Satisfaction - The evaluation will assess end-user perceptions on the feasibility of using camera phones to improve incident clearance times. The evaluation will also examine end-user perceptions regarding overall benefits for incident management and potential next steps for future applications of this technology.
- Operational Procedures - The evaluation will determine how camera phones were used to improve incident management and response activities. The evaluation will determine how the camera phones were integrated into the existing operational/incident response activities and assess how varying weather conditions, time of day, or other factors affect the ability of end-users to capture/transmit high quality images. In addition, this evaluation area will include assessments of when and under what conditions the camera phones were utilized. End-user perceptions of efficiency gains or enhancements provided as a result of using camera phones will also be evaluated.
- Technology Functionality - The evaluation will assess the effectiveness of the technology used throughout the camera phone proof of concept. Specifically, the evaluation will assess image quality, equipment reliability and usability, speed of service, and web server imagery.
- Safety Improvement - The evaluation will assess end-user perceptions of how the use of camera phones can improve levels of safety during traffic incident remediation. The evaluation will also determine if the camera phones reduce roadway exposure for response personnel, queue length, and secondary accidents.
- Institutional and Technical Issues - The evaluation will identify any issues, either technical or institutional, that were encountered throughout the project (and not addressed in other evaluation areas). The Evaluation Team will examine the effect of these issues on the project and describe how they were addressed.
The evaluation objectives are designed to measure key aspects of this project and will prove valuable in determining if wireless camera phones can be used to improve the efficiency of incident response activities.
6.0 Data Collection Techniques
This section describes the data collection methods that will be used to collect information for this evaluation. The Evaluation Team recommends that several overlapping methods be employed in order to provide a comprehensive examination and to address each evaluation area from multiple perspectives.
Field Observations. The Evaluation Team will conduct field observations to assess the functionality and effectiveness of using camera phones during incident management and response activities. This will require the Evaluation Team to ride along with project participants and document how the camera phones were used during incident operations. This will provide the Evaluation Team with a first-hand account of how incident operations are carried out. In addition, field observations may provide additional details about the types of situations or conditions that were present when/if camera phones are used during the field observations. The Evaluation Team will work with UMD-CATT staff and project participants to schedule field observations.
User Interviews. The Evaluation Team will work with UMD-CATT staff and project participants to identify and schedule the appropriate personnel to be interviewed. This will include developing survey instruments or questionnaires. Interviews will be conducted in person, by email, or by telephone. At the conclusion of each interview, the evaluator will prepare summary notes that detail each interview. Summary notes will then be forwarded to the interviewee for final comments and to ensure accuracy.
Archived incident images. Project participants will have the ability to store images on their camera phones or personal computers (via internet connections). As a result, the Evaluation Team will work with UMD-CATT staff and project participants to analyze sample images that may determine which factors (image quality, time of day, weather conditions, etc.), if any, influence incident management and response activities.
Operating procedures and other documentation. The Evaluation Team will work with UMD-CATT and project participants to determine if the operating procedures, as defined in the CONOPS, can be modified in order to achieve optimal performance. This will include a “before and after” analysis to determine if the camera phone improves upon the existing operating procedures and communication processes used by project participants. The Evaluation Team will also meet with project participants to document how the camera phones were used and identify the types of conditions or variables (e.g., weather, time of day) that affected the ability of the camera phone to enhance incident response activities.
Archived Incident Data. CAPWIN and VDOT currently archives traffic data/details for incidents in which camera phones were used by CapWIN and VDOT personnel to aid in the incident response process. As a result, the Evaluation Team will work with CapWIN and VDOT personnel to acquire and review archived incident data. This data will be used by the Evaluation Team to document the camera phone’s usage and examine the types of incidents or conditions (e.g., weather, time of day) that were present at the time of the incident. In addition, the examination of archived incident data may also help identify best practices or illustrate the abilities and limitations of camera phone technology.
Several types of incident data will be studied and documented, including:
- How the camera phone was used
- The type of incident that was involved (traffic accident, HAZMAT spill, etc.)
- The location of the incident
- The time of day in which the incident occurred (day vs. night)
- The type of weather conditions that were present during the incident
7.0 Project Deliverables
An important aspect of this evaluation will be to document the lessons learned as they unfold throughout this project. All lessons learned will be submitted to the United States Department of Transportation (US DOT) Intelligent Transportation System (ITS) Joint Program Office for inclusion into their Lessons Learned database after they have been validated by project participants and the Evaluation Team.
At the end of each phase of the project, a Benefits and Lessons Learned Report will be prepared. This report will summarize lessons learned, identify specific benefits, and document best practices. In addition to the Benefits and Lessons Learned Report, the Evaluation Team will develop a Case Study of the project as the primary Phase II deliverable. The development of the Case Study will allow the Evaluation Team to illustrate the benefits of using camera phones for incident management and response to project stakeholders or transportation communities. The Case Study will also provide potential end-users with a “real-world” example of how camera phones were used to improve incident management and response activities and will provide direct feedback on the experiences or findings that were encountered by current end-users. As a result, the Evaluation Team may be able to address the following questions with a greater sense of accuracy and detail:
- How was the camera phone used to improve the incident management and response
activities during the incident? - How much time was saved throughout the incident management and response process as a result of using camera phones?
- Was the camera phone reliable during incidents?
- Did the camera phone improve levels of safety during incidents?
- Was the camera phone easy to use?
The Case Study will provide an in-depth look at the value and role of camera phones during incident response operations. In addition, the case study will allow the Evaluation Team to collect critical information that may be used to further support the evaluation of this project. It is anticipated that the case study will be executed during Phase II of this evaluation.
[1] As currently planned, Phase II of the evaluation will be conducted once the Phase II project plan is approved, pending available funding. The Camera Phone Project Management Team has indicated that many of the activities initially planned for Phase II of the project have already been completed.
[2] Pending availability of funding.