ITS - Intelligent Transportation Systems Report ITS Home Page
Table of Contents


5. EVALUATION FINDINGS

5.1 SYSTEM PERFORMANCE

The primary system performance assessment findings are as follows:

5.2 SYSTEM IMPACT

The time period for which after project data was collected coincided with the initial months of system operation. Some quantitative data was obtained and analyzed, but the State has not had adequate time to use the system and develop a database that might be used to develop a comprehensive empirical estimate of system impact.

Although the amount of quantitative data available for analysis was limited, the Evaluation Team noted that one result of the FOT was a significant improvement in the quality of the data. For example, the before project data collected frequently had incident start and stop times indicated on the hour or half hour, and also contained a large number of incidents for which one or both times were not entered. In the after project data, the system was able to accurately capture both start and stop times to the minute.

An additional example of improved data quality involved the coding of incidents. Prior to the FOT, coding errors were common, in particular when agencies attempted to map their incident codes to UDOT codes. With the integration, this translation is done electronically and agencies are now able to match incidents using the integrated system.

The evaluation findings related to system impact are qualitative, as follows:

5.3 INSTITUTIONAL CHALLENGES

In general, Utah faced fewer institutional challenges than would be expected in most states in implementing an integrated CAD-TMC system. Agencies in Utah had very close working relationships, particularly DPS/UHP and UDOT. DPS dispatchers were located in the UDOT TOC and working relationships were particularly strong. In addition, the momentum from the 2002 Winter Olympics and the strengthened institutional relationships significantly supported the evaluation efforts. Even in this setting, however, there were some institutional challenges that the agencies involved had to overcome, which are described in sections 5.3.1 through 5.3.5.

5.3.1. Data Sharing

The first of the institutional challenges was partially due to the unique position held by the VECC in the region. VECC dispatches for essentially all emergency response agencies in Salt Lake County with the exception of DPS/UHP, Salt Lake City, and the Sheriff's Office. This single agency allowed UDOT and UHP to coordinate with fewer agencies. However, VECC had to reflect its client agencies' policies. As a result, VECC doesn't have the authority to provide certain types of information to the integrated system. Because the VECC system incorporated many agencies, the automation has to reflect a consensus or agreement position among all of the agencies involved. The agency with the most conservative policy in data sharing would drive what the system provides. Some of VECC's client agencies were concerned over how the information would be used and controlled after it was shared in the integrated system. The agencies were especially concerned about the privacy of individuals involved in the incident. As a result, only incident type and location were transmitted to the integrated system.

With the initial system operation, the shortcomings of sharing only the limited incident type and location data became evident, as stated in section 4 of this document. Currently, an effort is underway to develop a Memorandum of Understanding (MOU) that will cover how shared information will be used and protected. The UDOT incident management system used the following operator-entered fields:

UDOT would like additional information shared to reduce the entry required by its operators. Especially desired is the direction of travel, the start time of the incident (which is automatically generated when the UDOT operator initiates an incident), and the time the incident is closed (also automatically generated for incidents that were opened by a UDOT operator).

For other agencies that may consider implementing an integrated CAD-TMC system, it would be wise to work out an agreement on how the data will be used and protected prior to system implementation. This action would ensure that all system partner agencies would receive as much data as is needed by the incident management and reporting system.

5.3.2. Operator and Dispatch Procedures

With the exception of the VECC staff, the integrated system design relied on CAD operators to decide which incidents should be sent to which partner agencies. This decision point added a step in the operator/dispatcher normal work process. As a result, incidents were not shared consistently from dispatcher to dispatcher. When they are particularly busy, it was less likely that a dispatcher would have the time to add any steps into their normal work process. It may also be most critical that the incidents that occurred during these busy times be shared. Therefore, it is important for integrated systems to accommodate existing work processes for operators or dispatchers.

UDOT has planned future system improvements that will automate sharing incidents with rules for the data that can be shared and to determine what incidents should be sent to which agencies. This planned improvement is discussed more fully in section 5.4 of this document.

5.3.3. Primary Agency Responsibilities

Another institutional challenge faced in Utah was that each agency had its own primary responsibility. Integrating the CAD and TMC functions is not a primary responsibility of any of the partner agencies, but is important to all agencies. As a result, sometimes individual agency priorities required that less attention be paid to the integrated system than would have been optimal. It is unreasonable to think that priorities would change during the development and implementation of the integrated system.

In Utah, the partner agencies were responsive to project requirements. Project participant meetings helped keep the project momentum and open communications ongoing, and provided a venue to promote emphasis on the project from each partner agency's perspective. Agencies interested in pursuing an integrated CAD-TMC system should keep in mind the importance of these meetings. In addition, the approach and schedule for developing and implementing an integrated system should reflect the challenges inherent in working with multiple agencies on a project that is not top priority for any one of the participating agencies. Schedules should be longer than initially anticipated and more effort should be budgeted for meetings and coordination.

5.3.4. CAD System Modifications Contractual Arrangements

The integration required that all of the CAD systems had to be modified. Each agency contracted directly with its respective CAD vendor to upgrade the individual agency's CAD system. Coordination among and between the vendors and UDOT's system integrator was often challenging. UDOT's system integrator was responsible for making the whole system fit together and work according to specifications, but had no control and limited influence over the work being done by the individual CAD vendors.

An alternative approach could have been the agencies forming a consortium to contract for the entire integrated system. A single integrator could be contracted, who then subcontract to each CAD vendor. The agencies involved would have control over the work in their systems through the consortium, and the single integrator would have more control or influence over the work contracted to the vendors.

5.3.5. UTA

An overall institutional challenge for UTA was that many agencies, in particular, the public safety agencies, felt that there were security and/or privacy issues with releasing much detail on incidents via the messaging interface. As a consequence, this led to messages from these agencies indicating that the message related to a certain incident type, but with little additional insight - not even about the incident status (i.e., new, ongoing, ended), which was the primary type of additional information of interest to UTA. In some cases, there were some additional details included in the message description field, but UTA dispatchers found it difficult to quickly decipher the meaning of the various encoded information or to extract the traffic impact implications.

The combination of this challenge, along with UTA's choice to receive all messages, led to a general feeling for UTA dispatchers of being flooded with messages, with those from particular agencies containing information of limited value to UTA. The practical consequence was that dispatchers came to pay only limited attention to messages generated by those agencies.

In hindsight, UTA indicated that it might be more effective in the future for the system to evolve towards UDOT generating "traffic impact-oriented" messages based on monitoring the messages from all agencies, which UTA could monitor.

5.4 TECHNICAL CHALLENGES

Even though the Utah CAD-TMC integration project was determined to be a technical success, there were some rising technical challenges. The way in which the Utah agencies overcame these challenges may be instructive to other agencies considering a similar project.

5.4.1. CAD System Upgrade Schedules

The approach taken in Utah required an upgrade to every CAD system included in the integrated system. Since CAD systems are primarily off-the-shelf products, the main reason for the upgrade was to provide standards-compliant messages for system communication. The CAD systems developers provided upgrades on a fixed release schedule to ensure that the systems would continue to be supported as part of the core CAD product. However, the overall project schedule was dependent on the CAD vendors' release schedules.

For agencies taking a similar approach to an integrated system, it is crucial to include consideration of the vendor release schedules in the overall project schedule.

5.4.2. Degree of Automation

As discussed in section 4, the original approach for most of the agencies was to allow the dispatchers to decide which incidents should be sent to which partner agencies. However, as discussed under institutional challenges, this approach led to changes in the dispatchers' work processes.

The technical challenge was to determine the level of automation that would be appropriate and to provide a system that allowed some flexibility in the level of automation. This determination would enable agencies to start at one level of automation, and then change as they learned what worked best for their dispatchers and their partner agencies.

Since the Utah agencies intend to modify their systems to allow more automation, they will need to upgrade their systems to do so.

5.4.3. Information Filtering

For VECC, all incidents were sent to the integrated system. However, not all incidents were of interest to all agencies. Operators at UDOT, for example, had to decide which VECC incidents should be brought into their incident management system. VECC had been transmitting all incident data, and at times it was difficult to determine the degree of importance of each transmittal to the CommuterLink system. In busy times, it was likely that some incidents that could be of interest were missed by UDOT operators. Automatic filtering of incidents would help by presenting only those incidents of most interest to the operators.

The partner agencies and the UDOT system integrator discussed filtering incidents early in the project. However, because the agencies involved had no experience in receiving incidents from partner agencies, they weren't sure what filters would be most useful. Following the end of the FOT, they determined that it would be beneficial to add a mechanism to filter messages. Both the sending and receiving agencies would provide filters to the messages.

For other agencies considering an integrated system, it would be valuable to consider a similar option and plan to include filtering if funding allows.

5.4.4. Communication and Architecture

Communication among agencies in any integrated system is critical. Messages and information have to reliably pass among the systems. In Utah, a fiber optics system was used to provide the primary communication medium. In case the fiber-optics system ever malfunctioned or was damaged, Utah's contingency plan included using the Internet as a back-up communication mechanism to link agencies via the State's wide area network.

5.4.5. Architecture and Standards

The Utah system utilizes a peer-to-peer architecture. Each vendor wrote their communication routines to conform to the protocol that UDOT and its system integrator specified. The protocol relied on ITS standards, and was based heavily on the IEEE 1512 incident management standard.

One of the challenges was that not all CAD systems used a code set that would be compatible with IEEE 1512 codes. The options from which dispatchers in the dispatch centers can select did not necessarily have a one-to-one correspondence to the IEEE 1512 codes. This led to an imprecise translation among the CAD systems and the UDOT incident management system.

5.4.6. GIS Standards

Geo-referencing standards are also important in an integrated system so that the location of an incident transmitted from one system is interpreted as the same location by another. Not all of the systems use the same geo-referencing scheme, so a translation from one system to another would be required. The DPS/UHP system uses State Plane Coordinates, as do many State police agencies nationwide. The system translates State Plane Coordinates to latitude-longitude. This translation is relatively straightforward in a confined space, such as the Salt Lake City region. However, over a larger geographic area, such as the entire State, there would be distortions as the plane coordinate system is translated to the spherical coordinates of latitude-longitude.

Agencies considering an integrated system should be aware of the various geo-referencing schemes used by the systems involved so accurate translation can be included in the schedule and budget.

5.4.7. UTA

An ongoing technical challenge for UTA was that the IEEE 1512 incident messaging standard underwent some evolution, and the agency needed to adjust its implementation to incorporate those changes.

In addition, since UTA was one the first agencies to implement its messaging interface, the agency found that it also needed to make ongoing adjustments to reflect changes in firewall and Internet Protocol (IP) address settings as other agency interfaces came on-stream.

Yet another ongoing evolutionary challenge was the switch, after the system became operational, from Internet-based communications to use of the ATMS fiber optic communications system. While UTA already had security and access rights infrastructure in place for communications with other agencies via the Internet, the switch to fiber optic communications required that additional infrastructure be established.

Also, it was necessary to ensure that other agencies (1) provided UTA the required security and access rights to their infrastructure; (2) provided documentation on these rights to UTA; and (3) informed UTA of changes in its security and access rights configuration.

While UTA understands that fiber optic communications holds an intrinsic advantage over Internet-based communications regarding message security, UTA did not feel that fiber optic communications provided any noticeable increase in the performance of the message delivery system from the agency's perspective.

Previous | Next