Module 54 - T304

T304: Applying Your Test Plan to Field Management Stations (FMS) - Part 1 Signal System Masters (SSM) Based on NTCIP 1210 Standard v01

HTML of the Student Supplement

(Note: This document has been converted from the Student Supplement to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)

T304: Applying Your Test Plan to Field Management Stations (FMS) - Part 1 Signal System Masters (SSM) Based on NTCIP 1210 Standard v01

Table of Contents

Module Description - 2

Introduction / Purpose - 2

Case Study - 3

Glossary - 9

References - 10

Study Questions - 11

Icon Guide - 11

1. Module Description

This module is based on the IEEE 829-2008 formats for testing documentation (the NTCIP 1210 v01 standard does not offer testing documentation preparation information). The IEEE 829-2008 approach has already been amply covered by testing series modules. This module will help users leverage their NTCIP 1210 v01-based specifications to produce and apply a sample test plan, including test design specifications, test case specifications, and test procedure specifications, for SSM.

Thus, the development of clear and unambiguous NTCIP 1210 v01-based testing documentation can be used by system developers and integrators during procurement specifications preparation, system acceptance, and ongoing maintenance efforts. The module will guide agencies in verifying that delivered products conform to NTCIP standards and comply with the agency's specifications.

The logical step for the participant is to consider modules in the testing lifecycle, which are T101, T201, and T202, as well as T203 Parts 1 and 2, and T204, which lead up to the T304 module: Applying Your Test Plan to Field Management Stations - Part 1 Signal System Masters (SSMs) Based on NTCIP 1210 Standard v01.

2. Introduction to the Signal System Master (SSM)

The Signal System Master (SSM) is a traffic controller device used in the field as a supervising device that operates above the intersection traffic controllers. As a Field Management Station (FMS), a Signal System Master (SSM) acts as a traffic controller device assigned to supervise several Signal System Locals (SSLs) located at intersections in close vicinity. The current NTCIP 1210 v01 SSM standard is systems engineering (SE) process based and provides for the SSM needs and requirements and related design content. The current standard does not provide information on testing procedures. NTCIP 1210 Standard v01-based user needs and requirements were covered under modules A304a and A304b, respectively (see reference below). This module will direct participants to consult these two modules for gaining necessary knowledge and understanding of SSM user needs and specifying SSM requirements prior to preparation of SSM testing documentation, which is covered in this module.

There is a need to understand what to test, when to test, and why to test SSM so that users get what they have specified (a communications interface with field SSMs), and to learn to prepare testing documentation required for the agency's SSM procurement specification. Keeping this overall need in mind, this module aims to create SSM test documentation based on standardized formats (IEEE 8292008) and relate the formats to key elements of the NTCIP 1210 Standard v01. This will ensure central Traffic Management System's (TMS's) connectivity and data communication interface with the field SSMs within the reference architecture provided by the standard. (Note, the reference architecture shows three parts: TMS, SSMs and SSLs).

3. Case Study: An Outline of an SSM Test Plan

Testing is a process that uses a documented test plan designed to check conformance to the SSM standard. Testing of SSM is done to verify that requirements are fulfilled, reduce risks of misinterpretation between agency and manufacturers, and ensure interoperability. A testing process is guided by a test plan, which prescribes the Scope, Approach, Resources, and Schedule for the testing. Some of the testing aspects covered by a test plan include:

• Item(s) to be tested
• Features to be tested
• Features not to be tested
• Testing tasks to be performed
• Personnel responsible for each task
• Risks associated with the plan

The following outline is typically used for SSM testing based on IEEE 829-2008 formats and guidance provided. An agency may be able to inject local project needs and requirements based on this template.

1.0 Introduction

1.1 Testing Documentation Identifier

SSM Comm TP v01.01
SSM Communications Test Plan v01.01
11 June 2016, City of Midsize

1.2 Scope

1.3 References

1.4 Level Test Plan Testing to be covered

2.0 Details of Level Test Plan: Unit/Bench Testing

2.1 Test items and their identifiers

2.2 RTCTM (Test Design/Test Procedures)

2.3 List of SSM features to be tested (PRL)

2.4 Objects to be tested (RTM)

2.5 Approach

2.6 Item Pass/Fail criteria

2.7 Suspension Criteria and Resumption Requirements

2.8 Test Deliverables

SSM Communication Test Plan
SSM Communication Test Designs
SSM Communication Test Cases
SSM Communication Test Procedures

Reporting results

SSM Communication Test Logs
SSM Communication Test Incident Reports
SSM Communication Interim Test Status Reports
SSM Communication Test Reports (one for each test design)

3.0 Test Management

3.1 Planned activities and tasks; test progression

3.2 Environment/infrastructure

3.3 Responsibilities and authority

3.4 Resources and their allocation

3.5 Training

3.6 Schedule: As per project schedule (see main contract)

4.0 General

4.1 Quality assurance procedures

The testing quality will fall under the Testing QA Procedures established by the city and Company Name.

4.2 Metrics

The percentage of test cases passed per test design will be recorded.

4.3 Test coverage

All data elements specified by the SSM PRL and RTM shall be included in at least one test using nominal values. An RTCTM guides the linkages.

4.4 Glossary

4.5 Document change procedures and history

Test Plan Structure per IEEE 829-2008 Standard

(Extended Text Description: Author's relevant description: IEEE 829-2008 Based Test Plan Structure diagram shows the structure starting with Test plan at the top, Test Plan describes the Overall Approach to Testing. This leads to Test Design specifies the details of the test approach with Test Design Unit Test, Test Design Integrated Test and Test Design System Acceptance Test. This all leads to Test cast outlines a set of test inputs, execution conditions, and expected results with a Test Case, which is connected to Test Procedure and Test Plan Execution (Process).)

Testing Process: Consider Testing as an activity that is carried out with a series of steps in a lifecycle of an ITS project. To ensure that the system interface delivers what the users have specified, a testing process is necessary to assess outcomes. Such requirements are "communicated" in the testing documentation, beginning with a Test Plan (Test Case Specification is a component of the Test Plan). The purpose of software and software-based systems testing is:

• To help build quality into the software and system during the lifecycle processes and to validate that the quality was achieved
• To determine whether the products of a given lifecycle activity conform to the requirements of that activity, and whether the product satisfies its intended use and user needs
• Includes inspection, demonstration, analysis, and testing of software and software-based system products
• To perform test activities in parallel with development efforts, not just at the conclusion of the development effort

Test Plan: A test plan provides a description of the overall approach to testing all of the requirements to be verified. The Test Plan outlines the scope, approach, resources, and schedule of testing activities.

Breakdown of this Key Point: This first key point of the Testing Process describes a test plan - the master document that will include the test cases. Explaining this key point shows the hierarchical structure (test plan - test design specification - test case) that is required to develop a fully system-engineered test plan.

Test Design Specification (TDS): A test design breaks apart testing into smaller efforts and describes a test design specification - the specification that outlines the requirements to be tested and which test cases cover which requirements. This key point shows the hierarchical structure (test plan - test design specification - test case) that is required to develop a fully system-engineered test plan.

Test Case Specification (TCS): A test case identifies and specifies the inputs, outcomes, and conditions for execution of a test and is included in a document called Test Case Specification (TCS) as part of an ITS project overall Test Plan. It identifies a specific input and/or output that needs to be tested and records the purpose of the test, a description of the test, the input and output test specification, and the environmental needs, and references the test procedure and describes the results of the test.

The suggested outline for a TCS is shown below:

• Test Case Identifier
• Objective
• Inputs
• Outcomes
• Environmental Needs
• Special Procedural Requirements
• Intercase Dependencies

What does a Test Case verify?

• A Test Case verifies the requirements related to information exchanged between two systems by:
  • Verifying the sequence of information exchanged is correct
    • Standards use dialogs to define the information exchange sequence
  • Verifying the structure of information exchanged is correct
    • Standards define the order of Messages-Data Frames-Data Elements
  • Verifying the content of information exchanged is correct
    • Standards define the valid value rules (e.g., value ranges) for data exchanged

Test Procedure Specification (TPS): defines the steps to execute a test. Multiple Test Cases may reference a single Test Procedure.

Requirements to Test Case Traceability Matrix (RTCTM) for SSM

• An RTCTM table provides traceability from requirements to test cases to test procedures
• Each SSM Test Design has an RTCTM for the requirements and test cases applicable to the Test Design

Role of PRL in Testing: Identify Features to Be Tested

(Extended Text Description: Author's relevant description: Use the Project PRL to Identify Features to Be Tested - PRL table with seven columns and several rows is presented. Fifth column is conformance column which has M, Mandatory requirements and O-optional requirements circled. This shows that Mandatory must be selected and Optional is up to the project to select. An arrow points from Optional O to the requirement stated below: 2.5.2 Manage SSLs These features are to be tested to verify capability to upload-download-retrieve data. Must be selected YES. The table is shown below:

User Need ID	User Need	FR ID	Functional Requirement	Conformance	Support	Additional Specifications
2.4.3	Connect Communication Networks			M	Yes/No
		3.3.1.6	Explore SSL Data by the TMS	M	Yes/No
2.5.2	Manage SSLs			O	Yes/No
		3.3.1.6 ;	Explore SSL Data by the TMS	M	Yes/No
		3.3.1.7 :	TMS Acceptance of Data from the SSL	M	Yes/No
		3.3.1.8	TMS Delivery of Data to the SSL	M	Yes/No

Module A304a

2.5.2 Manage SSLs These features are to be tested to verify capability to upload-download-retrieve data. Must be selected YES.)

(Extended Text Description: Author's relevant description: PRL Example: What Needs to be Tested: Mandatory Requirements PRL shows a red box over conformance and support columns. All of them are to be tested. Table is shown below:

PRL Example: What Needs to Be Tested: Mandatory Requirements

User Need ID	User Need	FR ID	Functional Requirement	Conformance	Support	Additional Specifications
2.4.1	Provide Live Data			M	Yes
		3.3.1.1	Accept Data from the TMS	M	Yes	All are to be test
		3.3.1.2	Deliver Data to the TMS	M	Yes
		3.3.1.3	Explore SSM Data by the TMS	M	Yes
		3.3.3.1	Determine Access Settings	M	Yes
		3.3.3.2	Configure Access	M	Yes

)

(Extended Text Description: Author's relevant description: PRL Example: What Needs to Be Tested/Not to be Tested - PRL example for 3.4.2 is shown with a red box over requirements with what will be tested with YES marked with circle and what will not be tested. M-is always tested and O-is optional which is selected by project. The table is shown below:

User Need ID	User Need	FR ID	Functional Requirement	Conformance	Support	Additional Specifications
2.5.1.1	Configure Cycle Timers and Unit Backup Time			M	Yes
		3.4.2 Manage the SSM Configuration
		3.4.2.2.1	Determine SSLs Currently Connected:	M	Yes
		3.4.2.2.2	Determine Pattern Selection Capabilities...........:.....	M	Yes	.....will be tested
		3.4.2.2.3	Determine :SSM Section Characteristics	M	Yes
		3.4.2.2.4.1	Configure Cycle Timer Reference	O	Yes / No
		3.4.2.2.4.2	Determine Cycle Timer Capability	O	Yes / No	.... will NOT be tested
		3.4.2.2.5	Determine SSM Software Version	M	Yes / No

)

(Extended Text Description: Author's relevant description: Find Object Ranges from Project RTM to Prepare Test Cases - An arrow is points to 8-255 crossed with red line in object range. The case study has 10 SSLs to be tested. The table is shown below:

Functional Requirement Reference	Functional Requirement	Dialog Reference	Object Reference	Object	Comments (Informative)
3.4.2.1	Synchronize SSM Clock with TMS	4.1.3	1201.2.4.1	globalTime
3.4.2.2.1	Determine SSLs Currently Connected	4.2.2.3	5.2.1	maxlntersections
			5.2.2.1.3	intersectionSection
3.4.2.2.2	Determine Pattern Selection Capabilities	4.2.6.3	5.1.1	maxSections
			5.23.1	algorithmSupport

5.2.1 Maximum Number of Intersections

maxIntersections OBJECT-TYPE

SYNTAX INTEGER (8..255).)

Role of RTCTM in Testing

(Extended Text Description: Author's relevant description: Prepare RTCTM for Testing Documentation, table shown below:

Req. ID	Req.	Test Case ID	Test Case	Test Proc ID	Test Procedure
3.4.2.2.1	Explore SSL Data by the TMS
		TC3.4.3 .1.6-1	Verify maximum intersections
				TP3.4.3 .1.6-1	Verify object range 8-40

)

4. Glossary

Term	Definition
LTP	Level Test Plan
MIB	Management Information Base
MTP	Master Test Plan
NTCIP	National Transportation Communications for ITS Protocols
PRL	Protocol Requirements List
RTCTM	Requirements to Test Case Traceability Matrix
RTM	Requirement Traceability Matrix
SNMP	Simple Network Management Protocol
SEP	Systems Engineering Process
SSL	Signal System Local
SSM	Signal System Master
TMC	Traffic Management Center
TMS	Traffic Management System
TPG	Test Procedures Generator
Agency Specification	A document that has been prepared by an agency to define requirements for a subject item or process when procured by the agency.
Compliance	A condition that exists when an item meets all of the requirements of an agency specification.
Concept of Operations	A document that describes the purpose for a system project, including a description of the current and proposed system, as well as key user needs that the new system is required to address.
Conformance	A condition that exists when an item meets all of the mandatory requirements as defined by a standard. It can be measured on the standard as a whole, which means that it meets all mandatory (and applicable conditional) requirements of the standard or on a feature level (i.e., it conforms to feature X as defined in section X.X.X), which means that it meets all mandatory (and applicable conditional) requirements of the feature.

5. References

• NTCIP 1210 v01: National Transportation Communications for ITS Protocol Field Management Stations-Part 1: Object for Signal System Masters http://www.ntcip.org/library/documents/
• NTCIP 1201 v03.13a, National Transportation Communications for ITS Protocol, Global Object Definitions (www.ntcip.org)
• IEEE 829-2008, IEEE Standard for Software and System Test Documentation. IEEE, 2008. http://standards.ieee.org/findstds/standard/829-2008.html
• Professional Capacity Building Training Modules 24: A304a , Understanding User Needs for Field Management Stations - Part 1 Object Definitions for Signal System Masters Based on NTCIP 1210 Standardstds training.aspx
• Professional Capacity Building Training Module 25: A304b, Specifying Requirements for Field Management Stations - Part 1 Object Definitions for Signal System Masters Based on NTCIP 1210 Standard
• stds training.aspx
• NTCIP 9001-NTCIP Guide Version 04. http://www.ntcip.org/library/documents/

6. Study Questions

1. Which is NOT a part of the testing process in a system lifecycle?

1. Test planning
2. Preparation of test documentation
3. Test execution and reporting
4. Identification of system requirements

2. Which is NOT included in a structure of a test plan?

1. Test logs
2. Test design
3. Test case with inputs/outputs
4. Test procedures with steps

3. What is the primary purpose of RTCTM?

1. Sets the testing workflow sequences
2. Correlates User Needs to Requirements
3. Contains only test cases
4. Traces Requirement to Test Case to Test Procedure

4. Which is NOT a valid statement related to an SSM testing documentation?

1. Test plan contains an overall testing approach
2. Test design contains project RTCTM
3. Test procedures are provided by the manufacturer
4. Test procedure includes error detection

7. Icon Guide

The following icons are used throughout the module to visually indicate the corresponding learning concept listed below, and/or to highlight a specific point in the training material.

1) Background information: General knowledge that is available elsewhere and is outside the module being presented. This will be used primarily in the beginning of the slide set when reviewing information readers are expected to already know.

2) Tools/Applications: An industry-specific item a person would use to accomplish a specific task, and application of that tool to fit the need.

3) Remember: Used when referencing something already discussed in the module that is necessary to recount.

4) Refer to Student Supplement: Items or information that are further explained/detailed in the Student Supplement.

5) Example: Can be real-world (case study), hypothetical, a sample of a table, etc.

6) Checklist: Used to indicate a process that is being laid out sequentially.