4.0    Evaluation Findings

The evaluation findings are organized in three sections.  Section 4.1 reports the acceptance test results of the FOT voice routing system.  Section 4.2 presents the assessment of the performance of the data routing which shares the OnStar crash data with MnDOT for further dissemination on CARS.  While the first two sections focus on quantitative evaluation of the FOT performance, Section 4.3 provides findings and discussion based on interviews with the key stakeholders of the FOT, including telematics service provider, PSAP operators, call-takers, first responders, traffic operators, the Minnesota 9-1-1 program manager, and FOT team members.

4.1       TSP Voice Routing Field Test

The TSP voice routing field test was undertaken to determine whether the MnDOT MAYDAY/9-1-1 FOT solution demonstrates adequate reliability in routing TSP-initiated emergency calls to an appropriate PSAP with vehicle location (in latitude and longitude) and call back number.

Section 4.1.1 presents and discusses the test results.  Section 4.1.2 provides a summary of the findings of the testing.

4.1.1   Test Results

The field test consisted of 190 test calls made over eight days in mid-August of 2005.  Analysis of the test results consisted of first determining whether each call should be counted as an acceptance test record or as a non-counted record.  The official acceptance test calls were then further examined to determine if they passed or failed acceptance.  Table 4-1 below shows the summarized results of the acceptance test.

Table 4-1 contains a separate line for each PSAP in the evaluation, identifying its name, the date acceptance sampling was performed, and the planned number of calls for that PSAP.  It then shows the actual number of sample calls made with separate columns for the counts of calls that passed acceptance, failed acceptance, or were not counted toward the acceptance test.  The reasons for non-counted calls are discussed later in this section.  The table contains AACN transfer and Secondary PSAP Routing (Mayo) statistics where these capabilities were available.

Table 4-1.  Summary Results of FOT Acceptance Testing of Voice Routing Functions

¹ Where applicable from the Acceptance Calls

² Missed last Mower call made up with one extra call in Washington

³ The recorded AACN data does not match the reported call type

Table 4-1 shows the following:

• A total of 147 acceptance test sample calls were made from August 10, 2005 through August 19, 2005.  Either six or seven calls were placed from each of the 22 PSAPs, reflecting the original sample plan to obtain as closely as possible an equal sample in each PSAP.  All 147 of the acceptance sample calls passed the acceptance test criteria. 

The FOT passed acceptance testing with 94% confidence that the true system failure rate is no more than four percent.

• For the 20 acceptance sample calls made to areas with IES as the 9-1-1 service provider, simulated AACN data were transmitted with each call.

• Three of these calls were not considered because the PSAP operator did not have the application active at the time of the call to receive these additional data.
• Of the remaining 17 calls,
• 16 (94 percent) calls reported AACN data that matched the simulated data sent by the MnDOT sample team according to their recorded sample logs.
• the one AACN failure was a call from Kandiyohi at 10:21 AM on August 10, 2005.  The sample data collection log shows a simulated call type of “7” but the recorded data reported by the PSAP matches a simulated call type of “9”.  It is therefore most likely that this was simply a data recording error on the part of the sample collector and not an error in transmission of AACN data by the FOT or in reporting by the PSAP.

• For the 20 acceptance sample calls made to PSAPs with the Mayo Clinic as a secondary PSAP, the Mayo Clinic PSAP correctly received the call back number and latitude/longitude coordinates of the sample calls in all cases.

Data Quality Issues

Several calls were made where the reported latitude and longitude coordinates did not match their planned locations within the 0.01% criteria identified above.  These included:

• A 10:21 AM call from Kandiyohi on August 10
• A 9:17 AM call from Olmsted on August 15
• An 11:54 AM call from Renville on August 10
• A 2:05 PM call from Eden Prairie on August 12, and
• A 1:59 PM call from Eagan on August 17

For the first call, review of the data records revealed that the reported coordinates matched that of an attempted sample call made immediately prior to this call at another location.  For the last three calls, their data forms contain notes specifying why the call was not made exactly from the planned location.  It is not known why the second call’s coordinates did not match the plan.  However, in all five cases the reported coordinates matched those of the Telcordia data logs.  This means that the reported latitude and longitude, while not matching the planned location, was consistent with the data actually sent by the OnStar test unit.  Since the acceptance test only applies to FOT functions, these calls were all judged to be successful.  In several other cases with non-matching planned to reported coordinates, the data were not counted toward the acceptance test and were replaced by additional sample calls.  These points are discussed in greater detail below.

In seven instances, the data recorded for longitude on the data form was not a negative number.  (Locations with similar latitude but positive longitude as the test points in Minnesota correspond to Northern China near the border with Mongolia.)  This was considered to be a tolerated error made either by the data collector or by the PSAP operator and the test calls were not counted as errors for the acceptance test.  In three other instances, a minor omission or data transmission on the data collection forms were not considered to constitute test failure.  These issues actually support the value of automatic data transmission rather than verbal communication with regard to accuracy.

Spatial and Temporal Distribution of Sample Calls

Although not required, it was considered desirable to have calls made in diverse geographic regions throughout the FOT area.  Figure 4-1 below shows the geographic sample locations of the calls as relayed by the PSAP and recorded in the data collection forms.  Note that the desired diversity of locations appears to have been met.  It should be pointed out that the PSAPs included both county areas and towns/cities (e.g. Minneapolis, Apple Valley).  In the towns and cities, the smaller geographic area necessarily forced the sample locations to be more closely clustered.

Figure 4-1.  Spatial Distribution of FOT Voice Routing Acceptance Test Calls

Similar to the geographic diversity, it was desired, but not required that calls be placed at different sample times.  It was not operationally feasible to make calls outside normal working hours.  However, the calls made did show distribution over many hours of the day.  Figure 4-2 illustrates this through a histogram of the 147 acceptance test calls.

Figure 4-2.  Temporal Distribution of FOT Acceptance Testing of
Voice Routing Functions

Non-Counted Calls

In addition to the acceptance sample calls, a number of additional calls were placed as part of the acceptance test where the full acceptance criteria were not satisfied.  Under investigation, it was determined in each of these cases that the reason for the failures was outside the control of the FOT.  By agreement in the original acceptance test plan, these calls were categorized as neither successes nor failures with regard to the acceptance test.  Though not included in the evaluation of acceptance, the issues encountered on these calls are nevertheless informative to the FOT.  There were 43 such calls throughout the acceptance test period.  They represent a small number of reasons.  Each reason and accompanying frequency of occurrence is provided below.

1)   An incorrect latitude/longitude of (0,0) transmitted by the portable OnStar unit (13 occurrences) – This issue is related to turning on the external, portable OnStar unit for the first time after a shutdown.  The issue was identified before the acceptance test and provisions were made so OnStar would not try to forward calls that have a (0,0) value for latitude and longitude.

2)   Call is not picked up by OnStar Emergency Advisors and call eventually times out (20 occurrences) – The reason for each occurrence of this problem is not known but it may have been due to routing problems internal to OnStar and/or congestion in the OnStar call center where the test calls (by design) had lower priority than true OnStar emergency calls.  Regardless, it was agreed that this issue was not under the control of the FOT. 

3)   OnStar Emergency Advisor’s call encounters busy signal (1 occurrence) – It was discovered that the Telcordia primary rate interface (T1 line) to Verizon was not working which prevented OnStar from calling Telcordia.  This line is outside the FOT control.

4)   GPS coordinates passed through OnStar were from a previous call (4 occurrences) – This issue is related to a warm start (i.e., like cycling the ignition) of the OnStar unit.  It occurs when a GPS lock has not been established and the OnStar unit transfers the last known latitude and longitude coordinates.  In the acceptance test, this even resulted in occurrences where the call was routed to the wrong PSAP when the MnDOT staff person had moved from one PSAP area to another and the first call in the new PSAP area was transmitting GPS coordinates for the last location in the previous PSAP area.  By verifying that the recorded coordinates reported by the PSAP matched those of the Telcordia data transmission logs, it was determined that the FOT process was operating properly.  Therefore, these sample calls could legitimately be considered to pass acceptance, which did occur in a few occasions (see Data Quality Issues above).  The non-counted occurrences of this phenomenon were cases that the evaluation team elected to repeat or replace the sample calls with others that did not have this issue.

5)   ALI record (with latitude and longitude) was not received at PSAP (2 occurrences – Anoka County PSAP, August 12, 2005) – Investigation of this issue provided no specific reason for the failed transmissions except they were shown not to be attributable to the FOT.  Telcordia’s logs show the LAT/LON data were received from OnStar and were subsequently sent to the ALIs.  It is therefore assumed that the failure was at the PSAPs or ALIs, and this is outside the control of the FOT.

6)   OnStar incorrectly routed call (3 occurrences) – This appears to have been an OnStar training issue and is not attributable to the FOT.

4.1.2   Summary of Findings

• Using a statistical acceptance sampling approach, the FOT solution passed the acceptance test.  Passing the acceptance test corresponds to 94 percent confidence the true system failure rate does no exceed four percent.
• In 147 sample calls placed throughout 22 PSAPs, there were no observed errors within the control of the FOT with regard to transmitting latitude, longitude, and call back number from the OnStar test unit at the test location through to the proper responding PSAP.  Hence, the observed failure rate was zero percent.
• Similarly, there were no observed errors in the 20 sample calls that further forwarded this location and call back number data to another PSAP (Mayo Clinic).
• In a limited test of 17 sample calls to an area with IES as the 9-1-1 service provider, the ability to transfer AACN data through the OnStar call was also confirmed.  There was one error in this evaluation that appears to be a data entry rather than a system error.
• The acceptance test demonstrated the success of the FOT solution in routing data with voice to achieve increased speed and accuracy in transmitting critical 9-1-1 call information to PSAPs.

4.2       ACN/AACN Data Routing Evaluation

The data routing portion of the FOT sought to develop and test a system for a telematics service provider (e.g., OnStar) to push ACN and AACN data into a MnDOT SOAP server for further distribution to CARS. 

Due to data availability, this evaluation only examined the performance of the first link of the data distribution between OnStar and SOAP server, as shown in Figure 4-3.  The system performance is defined as the reliability and latency of the data transfer.  Different from the voice routing evaluation, the data routing involved the transmission of actual OnStar crash data throughout the state of Minnesota.

Figure 4-3.  Focus of Data Routing Performance Evaluation

Section 4.2.1 contains the results of the characterization of the data routing system and its estimated reliability and latency.  Section 4.2.2 summarizes the findings of the evaluation.

4.2.1   Analysis Results

The data routing function from OnStar to the SOAP server was characterized through basic summary statistics and was then analyzed for both reliability and latency.  Different from the voice routing test, the data routing portion of the FOT transmitted the actual OnStar accidents in Minnesota over the period of the FOT.

Characterizing the System Usage

Over the 41 weeks for which data were available, OnStar transmitted 1,297 crash records to the SOAP server.  Fifty of these calls failed to be received and are discussed further below.  Of the 1,247 calls that were successfully received, there were:

• 17 AACN calls (1%)
• 137 ACN calls (11%)
• 1093 SOS calls (88%)

Over each week, the total number of calls received varied.  After August 7, 2005, the number dropped considerably as OnStar stopped sending SOS calls to the SOAP server.  During the rest of the time period (and ignoring the weeks where no data are available) OnStar sent an average of 34 calls per week to the SOAP server.  The week with the most calls was August 1 – August 7, 2005 when 50 total calls were delivered.  This is shown graphically in Figure 4-4.  The figure also shows the relative number of calls for each type of success (AACN, ACN, and SOS) as well as any failures.  As noted above, the sharp dropoff after week 63 corresponds to OnStar discontinuing transmission of SOS calls.

Reliability of Data Transmission

In two weeks during the period, there were instances of failed transmission of crash data.  These consisted of:

2005.04.18-to-2005.04.24  14 failures

2005.04.25-to-2005.05.01  36 failures

The failures were due to a computer security key not being updated.  This prevented any files from being exchanged.

While these failures were explained and resolved, they do serve to illustrate the potential for failures in this or any data network systems.  Over the 41 weeks a total of 50 test failures were observed compared to 1,297 total OnStar calls (successes and failures).  This represents a failure rate of 3.9%.

Figure 4-4.  FOT Data Routing Calls from OnStar to SOAP Server
October 11, 2004 – September 4, 2005

Latency

In evaluating latency of the transmission system, the objective is to determine an average delivery time per call.  Even though we do not have the delivery time for the individual call records, we can determine the overall average delivery time per call as:

where

i is the week number

n is the total number of weeks (=41)

c_i are sums of AACN, ACN, and SOS successful calls for each week

 are the average call times for each week

This calculation may not perfectly match the value obtained with individual call records due to rounding issues.  However, it is expected to be close.

The mean latency is 0.9 seconds.

It is concluded that the first stage of the data routing functionality within the FOT demonstrated high reliability (96.1%) in delivering every OnStar record sent to the SOAP server.  Aside from a few anomalous cases, these records were sent in an average of less than one second.

Many other factors may be considered in evaluating the data routing function.  This evaluation did not have a means for independently verifying that the tabulated calls were the only ones sent by OnStar or for independently validating the accuracy of the data elements transferred.  It is assumed these summaries were correctly prepared by OnStar.  This evaluation also did not have the raw data of transmission time for each call so that the variability in transmission time could not be analyzed.  However, the very low average transfer times likely make any issues of transmission time variance insignificant on the scale observed (i.e., less than one second).  More importantly, though, this evaluation did not have a means to evaluate the full time interval of relaying the OnStar data to the CARS database- a true latency measure from an end user’s stand point.

4.2.2   Summary of Findings

• The evaluation of the FOT data routing function considered only delivery times for (data) calls made by the OnStar Call Center to the SOAP server for OnStar incidents occurring in Minnesota.  Dissemination from SOAP server to CARS and other data servers could not be evaluated with the available information.  Therefore, caution should be exercised in generalizing the results of this evaluation to the data routing portion of the FOT. 
• The large majority (88%) of calls made were SOS (emergency button pushes) with about an order of magnitude less ACN calls (11%) and another order of magnitude less AACN calls (1%).  The higher percentage of ACN over the AACN calls reflects the larger installed base of the basic ACN instrumented vehicles, based on the feedback from OnStar.  The calls averaged about 34 per week during most of the evaluation period and never exceeded 50 in any one week.
• The overall failure rate of the data routing function from the OnStar Call Center to the SOAP server from September 27, 2004 through September 4, 2005 was 3.7% of the OnStar calls sent.  All the failures represented special causes that were isolated and believed to be fixed. 
• Excluding a few unusually long delivery times, the vast majority of calls were delivered to the SOAP server in an average of less than one second.  Minimal latency is important for its intended application.  While it takes time for a Telecmatics Emergency Advisor to set up a 3-way call with an appropriate PSAP, it is imperative to provide the available ACN or AACN data in a timely fashion to assist decision making in emergency response. 
• In the FOT setting, when PSAPs or medical responders (e.g., Mayo Clinic) receive 9-1-1 calls from the OnStar, they could expect an icon corresponding to the reported incident to pop up overlaying on the digital map provided by the CARS.  By clicking on the icon using a computer mouse, additional ACN or AACN data, if available, will be displayed.  This evaluation confirmed that the data link between the OnStar Call Center and the SOAP server was operating properly.  The evaluation, however, did not examine the data communications between the SOAP server and CARS, based on a decision with the FOT team.

4.3       User Acceptance and Deployment Issues Evaluation Results

This section documents the findings from the interviews with the key stakeholders of the FOT, including for voice routing: PASP operators, call takers, and OnStar; and for data routing: emergency responders and MnDOT traffic operation managers who share the OnStar ACN data via the CARS.  In addition, deployment issues and broader implications were gathered through interviews with the FOT team, the Minnesota 9-1-1 program manager, and the NENA technical issues director.

4.3.1   PSAP, Medical Responder, 9-1-1 Program Manager, Traffic Operator Feedback

            4.3.1.1            Interview Results

The PSAP interviews were structured around a set of standard questions, consistent with the intent, scope and goals of the FOT, and addressed the following subject areas:

• PSAP Operational Environment
• Experience receiving and processing wireless 9-1-1 calls (both Phase I and Phase II)
• Facility operational and technical capacities and limitations
• GIS and mapping capability (both “mapped ALI”, and CAD)
• Historical experience with TSP calls
• Identification of incident-related location information
• Potential use of geo/coordinate based location identification
• Feedback on field test
• Suggestions for future improvements
• Observations about expanded use of FOT solution

The Mayo Clinic and the Mayo Medical Transport, as emergency medical responding and treatment service entities, currently have access to real-time ACN data shared through MnDOT’s CARS.  Interviews with selected Mayo Clinic representatives were conducted in conjunction with the process outlined above, and addressed the following subject areas:

• Operational Environment
• Nature of CARS access
• Frequency of use
• Utility of the information involved (both transport and treatment—i.e., positive patient outcome, and facilitating early response)
• History of use and benefits of the ACN data, during the period such data has been available
• Perception of information timeliness and accuracy
• Problems encountered
• Suggestions for future improvements
• Observations about expanded use of FOT solution

The benefit of incident data is also important to MnDOT and traffic management operations.  To that end, interviews were conducted with individuals involved in processing ACN/AACN-related CARS data for the purpose of coordinated traffic incident management.  Topics included the following:

• Operational Environment
• Nature of CARS access
• Frequency of use
• Utility of the information involved for the sake of managing traffic operations (including data format, timeliness, and user interface)
• History of use
• Perception of information timeliness and accuracy
• Problems encountered
• Suggestions for future improvements
• Observations about expanded use of FOT solution

Five primary PSAPs were selected for post test interviews, including the counties of Kandiyohi, Olmsted, and Anoka, and the cities of Burnsville and Minneapolis.  These PSAPs provided a good mix of PSAP environments, including 9-1-1 Service Provider (i.e., Quest and IES), PSAP service environment (i.e., urban, suburban, rural), and size of PSAP.  The Mayo Clinic was involved as both a secondary PSAP (emergency response), and, as a CARS data user, as was the Roseville Regional Transportation Management Center (RTMC).  A majority of the above interviews took place during the week of September 19, 2005, and involved five coordinated meetings.

In many locations around the country, 9-1-1 services are often funded, coordinated and/or facilitated by cognizant 9-1-1 authorities or agencies specifically established for that purpose.  Such authorities range from regional, multi-jurisdictional coordinating bodies, to state agencies or functions.  That is true in Minnesota as well, and two such organizations were specifically involved in this FOT and interviewed, including the Minnesota Statewide 9-1-1 Program (Department of Public Safety) and Minneapolis/St. Paul Metropolitan Emergency Services Board.

Interview questions for the above organizations covered the following subject areas:

• Nature and responsibilities of the authorities involved
• Involvement with the FOT
• Suggestions for future improvements
• Observations about expanded use of FOT solution
• Future organizational plans for such applications

            4.3.1.2            Summary of Findings

The results of the interviews were fairly consistent among all the PSAPs visited and interviewed.  Following is a summary of those interviews, and the agencies’ observations and experiences with the FOT: 

• Few of the PSAPs maintained 9-1-1 call history data in sufficient detail to statistically compare historical operational experiences with trial events.  However, based upon experience, all indicated a preference for native 9-1-1 call delivery for such calls, noting that the latter insures more accurate routing, the automatic delivery of call-related data (like Automatic Number Identification28 (ANI), ALI29 and class of service30) and priority processing (i.e., call into 9-1-1 trunk as opposed to the administrative line).
• Most PSAPs were either wireless 9-1-1 Phase II ready, or deployed with one or more carriers31.  That is, they are able to receive wireless 9-1-1 calls with a call back number and call location (LAT/LON) provided.  The FOT calls were designed to emulate a wireless 9-1-1 call and thus can display call back number and location with Phase II ready PSAPs. 
• Several of the PSAPs continue to work on mapping and GIS resources to support the processing of wireless- calls.  The GIS tool is part of the wireless 9-1-1 PSAP solutions to make use of the call location data for visual representation using electronic maps.  Figure 4-5 shows several of the PSAPs where interviews were conducted.
• While limited, most PSAPs had historically received telematics-based emergency calls (including OnStar) on the administrative line, and they cited positive experiences and outcomes with those calls.  Also cited was the unique capability for OnStar Emergency Advisors to remotely flash the vehicle head lights in assisting the identification of the crashed vehicle, in the case of run-off-the-road accidents with unconscious drivers.  In addition, OnStar Emergency Advisors can provide continuous update of vehicle location for incidents involving a moving vehicle (e.g., car-jacking).
• Most PSAPs stated that the location of the calling party provided by telematics service providers (historically, and, as part of the FOT) appeared to be accurate.  Historically, the location information was described by the OnStar Emergency Advisor as the street address, or in the case of rural area, the main route and the nearest cross street or landmark.  The FOT voice routing solution automatically provided the location data in LAT/LON that can be plotted using a GIS tool – conceivably a less time consuming and less error-prone approach.
• All the PSAPs interviewed currently receive telematics-initiated calls on so-called “administrative lines,” or on non-native 9-1-1 trunks dedicated for such purposes.  In all instances, while calls on these lines are processed identically to native 9-1-1 calls, they do receive lower call-handling priority than calls on 9-1-1 trunks (though that only becomes an operational issue when 9-1-1 call-takers are busy on other calls).  Since such lines generally do not automatically provide ANI and ALI data, the PSAPs involved attempt to capture the “caller ID” of the calling party, and then poll their ALI database for location.  The limitation of OnStar-initiated three-way calls, however, only display the number of the OnStar Operation Center in the caller ID, not the cell phone number of the crashed vehicle. 
• A few PSAPs noted that a telematics-initiated call involving a three-way conversation between the calling party, the telematics service center, and the PSAP can be complicated, but beneficial with training.
• The Minneapolis/St. Paul Metropolitan Emergency Services Board noted that the FOT voice delivery solution appeared to be a clean solution, minimizing infrastructure requirements and impact on existing 9-1-1 network configurations.
• PSAPs served by Independent Emergency Services (IES) were able to receive advanced ACN data as part of the FOT.  Kandiyohi County (as the IES PSAP involved in the interviews) cited the importance of such data in processing an ACN-type emergency event, though the use of such enhanced data will require additional call-taker training.
• In a few instances, where FOT test calls failed to execute as planned, it appeared that the casual factors were outside the scope of the trial, and thus did not negatively impact the results of the FOT.
• Mayo Medical Transport, as both a secondary PSAP, and, as a CARS data user, specifically cited the benefit of ACN/AACN incident data in responding to an emergency event.  While the CARS data are limited, as is the means of access, the timely notification of an incident, along with descriptive data about the event, does facilitate emergency response.
• All PSAP interviewees agreed that the FOT appeared to reflect a successful solution to delivering a telematics-initiated 9-1-1 call to the appropriate PSAP over the existing 9-1-1 infrastructure with a call back number and call location.
• All interviewees also agreed that exploring and fostering the wider deployment of such a solution would be beneficial, both within Minnesota and elsewhere.
• The Regional Transportation Management Center in Roseville, MN makes effective use of the CARS data, both in support of traffic management during incidents and DPS emergency response.  This center is a model for similar efforts around the country.  Figure 4-6 shows selected applications of ACN data in CARS by MnDOT RTMC.
• The OnStar ACN/AACN data on CARS supplements the information and many resources currently possessed by the RTMC in support of traffic-related incident management.  Such resources include the large number of traffic detectors and Closed Circuit Television (CCTV) cameras for detection and verification of incidents, and the ability to control traffic signals and provide traffic advisories using Dynamic Message Signs, the MnDOT web site, and the 511 traveler information system.

Figure 4-5.  Selected PSAPs where Interviews Were Conducted

Figure 4-6.  Use of ACN Data in CARS by MnDOT RTMC

4.3.2   Telematics Service Provider Feedback

            4.3.2.1            Interview Results

OnStar customer service and corporate representatives were interviewed on October 19, 2005 at OnStar headquarters in Detroit, Michigan.  That interview was structured around the following areas:

Emergency Advisors

• General operational responsibilities
• Existing emergency call processing protocol
• Experience in responding to emergency calls
• As a subset of the above, experience in working with PSAPs
• Roles and responsibilities as it pertains to the FOT
• Suggestions for future improvements
• Observations about expanded use of FOT solution

Corporate Representatives (representing corporate planning, industry affairs and service development)

• OnStar employee role and responsibility
• Corporate experience in responding to emergency calls
• Corporate emergency call processing policy
• Involvement with the trial
• Suggestions for future improvements
• Observations about expanded use of FOT solution

            4.3.2.2            Summary of Findings

OnStar, as the telematics service provider involved in the FOT has a long history servicing customer emergency calls, and interacting with the 9-1-1 community.  Emergency Advisors receive special training in processing such calls, and specifically interfacing with 9-1-1 call takers.  The following is a summary of the above interview meeting, and the observations and FOT experiences shared: 

• OnStar’s history in processing customer 9-1-1 calls has been positive, as has been their working relationship with the 9-1-1 community and PSAPs.
• While the current process of handling 9-1-1 destined calls works (i.e., calling into PSAP administrative lines), the FOT solution would provide a more effective way of accomplishing the same task for the reasons cited elsewhere in this report.
• In spite of the perceived benefits of the FOT solution, OnStar indicated the need to preserve the ability to contact the PSAP via the administrative line, citing a significant portion of customer requests or events are of non-emergency nature.
• The data routing portion of the FOT was perceived as a success.  OnStar indicated increased demands in recent years to provide their data to the Department of Transportation (of various levels) and emergency response community in light of management of regional natural disaster-related events.   
• Nationwide deployment of the FOT voice routing solution would involve significant cost, a factor impacting the potential adoption and rollout of the approach nationwide.  It was evident that the WSP-specific call routing solution was cost prohibitive and a simulated approach must be used by the FOT.  The cost for implementing and operating such a nation-wide system is disproportionately high considering the very small representation of telematics-initiated 9-1-1 calls.  OnStar suggested broadening the users of the FOT solution to include other national security or regional public safety entities that require long distance routing of 9-1-1 calls.
• It was unfortunate that the voice routing portion of the FOT ultimately was not able to use a real wireless service provider, and was forced to simulate those functions within a lab environment.  While the simulation did not negate the results of the FOT, it did impact the “field” emphasis of the operational test.
• While the setup of the FOT voice routing required an OnStar Emergency Advisor to manually dial a Temporary Location Directory Number (in order to route the call to the simulated WSP hosted in Telcordia’s New Jersey facility), it was noted that that part of the process could be easily automated.
• Considerations should now be given to real world deployment of the FOT solution, recognizing wireless network functionality and impact, cost, and migration to NG9-1-1 IP-based infrastructure.

4.3.3   Implications for NG9-1-1

            4.3.3.1            Third Party Technical Review

In accordance with the evaluation plan, the FOT concept, technical design, and results of the field test were reviewed with Mr. Roger Hixson, NENA’s Technical Issues Director.  The interview specifically related to NENA’s work in the area of Next Generation E9-1-1 systems, recognizing that the latter will ultimately generate or facilitate enhanced solutions for the objective of this FOT. 

Mr. Hixson observed that, while the design involved would require new TSPECRS-specific functionality in the wireless network, a potentially challenging matter in light of evolving wireless networks, the concept and solution being tested by the FOT represented a “viable interim method” to automatically route emergency calls being generated by customers through telematics call centers to “native Enhanced 9-1-1 systems in the USA.”  Ultimately, Mr. Hixson indicated, IP based, NG 9-1-1 systems are likely to offer “more effective and seamless solutions for ACN calls and basic data delivery, and subsequent PSAP access to supportive data from Telematics provider databases.” 

            4.3.3.2            Summary of Findings

The following summarizes the NENA observations provided above:

• The FOT Concept of Operations does offer an effective solution to the delivery of telematics-based 9-1-1 calls to PSAPs in a fashion consistent with the normal and native delivery of wireless 9-1-1 calls.
• This solution would require new functionality in the wireless 9-1-1 network infrastructure, and that in turn would presumably involve cost.
• Ultimately, the IP-based next generation 9-1-1 infrastructure (both wireline and wireless) deployment will provide more effective and seamless solutions for the delivery of ACN/AACN data and telematics emergency calls.
• However, the FOT voice routing solution could provide an effective migratory step in that direction.

            4.3.3.3            Project Team Observations

In accordance with the FOT Evaluation Plan, a post-trail project team member interview was conducted on November 10, 2005.  The intent of the interview was to give all team members an opportunity to comment on the results of the FOT; solicit lessons learned and observations about the potential for FOT concept application in real world environments.  Much of that discussion is also reflected in the FOT project reports [Ref 2, 3].

            4.3.3.4            Summary of Findings

• While the development of TSPECRS voice routing was challenging, the field test was conducted successfully, and the concept/solution involved does represent a viable approach for delivering telematics-based emergency calls to the PSAP as native 9-1-1 calls.

• Several technical issues did arise during the voice routing field test, but they related to factors either outside the scope of the project, or beyond operational control of the project team.  Consequently, such calls were not considered failures.  For example, test calls transmitted incorrect LAT/LON were caused by incorrect use of the portable OnStar test unit and thus not counted as failures.  This was discussed under the non-counted calls in Section 4.1.1.
• The TSPECRS test experiences emphasized the need for robust and redundant networks to support reliable call delivery.
• 9-1-1 call relay at the telematics provider service center should be automated to avoid manual miss-dials, a feature consistent with the intent of this trial (i.e., to automate as much of the call delivery process as possible).
• Beyond the native delivery of 9-1-1 calls, the system also provides an opportunity to deliver additional emergency event information and data important to emergency response.
• The opportunity for real world deployment should be examined further, particularly as a migratory step to NG9-1-1 network infrastructure that may ultimately provide long-range solutions to accomplishing the goals of the FOT.