Module 50 - T309

T309: Applying Your Test Plan to Ramp Meter Control (RMC) Units Based on NTCIP 1207 Standard v02

HTML of the PowerPoint Presentation

(Note: This document has been converted from a PowerPoint presentation to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)

Slide 1:

(Extended Text Description: Welcome - Graphic image of introductory slide. A large dark blue rectangle with a wide, light grid pattern at the top half and bands of dark and lighter blue bands below. There is a white square ITS logo box with words "Standards ITS Training" in green and blue on the middle left side. The word "Welcome" in white is to the right of the logo. Under the logo box is the logo for the U.S. Department of Transportation, Office of the Assistant Secretary for Research and Technology.)

Slide 2:

(Extended Text Description: This slide, entitled "Welcome" has a photo of Ken Leonard, Director, ITS Joint Program Office, on the left hand side, with his email address, Ken.Leonard@dot.gov. A screen capture snapshot of the home webpage is found on the right hand side - for illustration only - from August 2014. Below this image is a link to the current website: www.its.dot.gov/pcb - this screen capture snapshot shows an example from the Office of the Assistant Secretary for Research and Development - Intelligent Transportation Systems Joint Program Office - ITS Professional Capacity Building Program/Advanced ITS Education. Below the main site banner, it shows the main navigation menu with the following items: About, ITS Training, Knowledge Exchange, Technology Transfer, ITS in Academics, and Media Library. Below the main navigation menu, the page shows various content of the website, including a graphic image of professionals seated in a room during a training program. A text overlay has the text Welcome to ITS Professional Capacity Building. Additional content on the page includes a box entitled What's New and a section labeled Free Training. Again, this image serves for illustration only. The current website link is: https://www.its.dot.gov/pcb.)

Slide 3:

Slide 4:

T309:

Applying Your Test Plan to Ramp Meter Control (RMC) Units Based on NTCIP 1207 Standard v02

Slide 5:

Instructor

Dave Miller,

Chair: NEMA / AASHTO / ITE Joint Committee on ATC

Chair: 3TS Technical Committee

Principal Systems Engineer

Siemens Industry, Inc.

RC-US MO MM-ITS R&D

Austin, Texas, USA

Slide 6:

Target Audience

• Traffic management and engineering staff
• Traffic Management Center/operations staff
• Freeway and traffic signal maintenance staff
• System developers
• Private and public sector users, including manufacturers

Slide 7:

Recommended Prerequisite(s)

• I101: Using ITS Standards: An Overview
• A101: Introduction to Acquiring Standards-based ITS Systems
• A102: Introduction to User Needs Identification
• A103: Introduction to ITS Standards Requirements Development
• A201: Details On Acquiring Standards-based ITS Systems
• A202: Identifying and Writing User Needs When ITS Standards Do Not Have SEP Content
• A203: Writing Requirements When ITS Standards Do Not Have SEP Content
• C101: Introduction to the Communications Protocols and Their Uses in ITS Applications
• A309a: Understanding User Needs for Ramp Meter Control (RMC) Units Based on NTCIP 1207 Standard v02
• A309b: Understanding Requirements for Ramp Meter Control (RMC) Units Based on NTCIP 1207 Standard v02

Slide 8:

Curriculum Path

(Extended Text Description: Curriculum Path: A graphical illustration indicating the sequence of training modules that lead up to and follow each course. Each modules is represented by a box with the name of the module in it and an arrow showing the logical flow of the modules and the current module highlighted. This slide focuses on the modules that lead up to the current course Each box in the top row is labeled I101, A101, A102 and A103 left to right respectively, each with the course title. Each box in the second row is labeled A201, A202, A203 and C101 left to right respectively, each with the course title. Each box in the bottom row is labeled A309a, A309b, and T309 left to right respectively, each with the course title.)

Slide 9:

Learning Objectives

1. Describe within the context of a systems life cycle the role of a test plan and the testing to be undertaken
2. Recognize the purpose, structure, and content of a well-written test documentation based on IEEE 829
3. Describe test documentation for NTCIP 1207 Standard v02: Test plan, test design specifications, test cases, test procedures, and test reports
4. Describe the application of good test documentation to a ramp metering unit based on the NTCIP 1207 Standard v02
5. Identify a process to write test plans to verify NTCIP 1207 Standard v02 requirements

Slide 10:

Learning Objective #1: Describe Within the Context of a Systems Life Cycle the Role of a Test Plan and the Testing to Be Undertaken

• The purpose of testing RMC units
• Review the concept of a systems life cycle and testing to be undertaken
• Review verification methods
• Describe the testing process in relation to the systems life cycle

Slide 11:

Learning Objective #1

The Purpose of Testing RMC Units

How Do We Know RMC Units Will Work as Intended?

• Many legacy RMC units were developed for a particular agency
• Purpose of testing RMC units:
  • Verify that the RMC unit works as intended for this project.
  • Provide objective evidence that the RMC unit will:
    • Satisfy the allocated RMC unit requirements
    • Solve the right problem
    • Satisfy the intended use and user needs
• Evidence is delivered according to the IEEE 829-2008 standard
  • Test steps that are familiar to all stakeholders
  • Common understanding of terms for all stakeholders

Slide 12:

Learning Objective #1

Concept of a Systems Life Cycle and Testing to Be Undertaken

RMC Testing Fits Within the Systems Life Cycle

RMC unit life cycle: Vee model System Life Cycle

(Extended Text Description: Author's relevant description: Graphic depicts the standard VEE project workflow model with Testing to be Undertaken Here circled around the four right-hand levels of the VEE. The four levels circled include, from bottom to top, Unit/Device Testing, Subsystem Verification, System Verification & Deployment, and System Validation. Taken with permission from US Department of Transportation, Systems Engineering for Intelligent Transportation Systems, USDOT, January 2007)

Slide 13:

Learning Objective #1

Concept of a Systems Life Cycle and Testing to Be Undertaken

RMC Testing Fits Within the Systems Life Cycle (cont.)

• RMC unit testing to be undertaken includes level tests of:
  • Unit/Device: Each hardware or software module
  • Subsystem: Integrated hardware, software, interfaces
  • System Verification: Subsystems connected together
  • System Validation: Final system configuration
• Evidence is delivered according to the IEEE 829-2008
  • Test steps that are familiar to all stakeholders
  • Common understanding of terms for all stakeholders

(Extended Text Description: New graphic created by the author for this course depicting the right side four levels of the VEE model workflow with an arrow directed upwards from the word Levels. The four levels include, from bottom to top, Units/Devices, Subsystems, System Verification and System Validation.)

Slide 14:

Learning Objective #1

Concept of a Systems Life Cycle and Testing to Be Undertaken

Traceability Within Life Cycle

Trace each level to the design documents:

• Unit/Device Level to Detailed Design
• Subsystem Levels to High-Level Design
• Integrated System to System Requirements
• Configured System to Concept of Operations

(Extended Text Description: New graphic created by the author for this course repeating the graphic from Slide 12, with four arrows directed to the right towards the four levels of the left side of the VEE workflow model and the word Traceability above the arrows. The four levels include, from bottom to top, Units/Devices, Subsystems, System Verification and System Validation. On the right, the levels include, from bottom to top, Detailed Design, High-Level Design, Requirements, and Concept of Operations.)

Slide 15:

Learning Objective #1

Review of Verification Methods

Testing Process to Verify Conformance

Verification Methods

• The testing process determines whether the system:
  • Conforms to the requirements
  • Satisfies the intended use, the user seeds (IEEE 829)
• Determination may be based on one or more methods:
  • Inspection
  • Demonstration
  • Analysis
  • Testing

Slide 16:

Learning Objective #1

Testing Process Relative to System Life Cycle

The Testing Process

According to IEEE 829

• The testing process provides:
  • An objective assessment of the system products
  • Carries forward throughout each project's life cycle
• Project life cycle testing points:
  • At the completion of each development iteration
  • During installation
  • At system "go-live"
  • During operations and maintenance
  • At each system upgrade
  • At system decommissioning and replacement

Slide 17:

Learning Objective #1

Testing Process Relative to System Life Cycle

The Testing Process (cont.)

Steps of the Testing Process:

• Step 1: Test Planning
  • Test plan
• Step 2: Test Documentation Preparation
  • Test design
  • Test case
  • Test procedure
• Step 3: Test Execution and Reporting
  • Test report

Slide 18:

Learning Objective #1

Testing Process Relative to System Lifecycle

Test Planning

Testing Process and System Life Cycle

(Extended Text Description: Author's relevant description: Graphic depicts the standard VEE project workflow model with Test Planning and Test Document Preparation identified with the left side of the VEE and Test Execution and Reporting identified with the right side of the VEE. Taken with permission from US Department of Transportation, Systems Engineering for Intelligent Transportation Systems, USDOT, January 2007)

Slide 19:

Slide 20:

Learning Objective #1

Which of the below is not a reason to test an RMC unit?

Answer Choices

1. Satisfy system requirements
2. Testing is part of the NTCIP 1207 Standard v02
3. Solve the right problem
4. Satisfy user needs

Slide 21:

Learning Objective #1

Review of Answers

a) Satisfy system requirements
Incorrect. RMC is tested at the system verification level.

b) Testing is part of the NTCIP 1207 Standard v02
Correct! Testing is not part of the NTCIP 1207 Standard v02, but must be designed and documented during the project.

c) Solve the right problem
Incorrect. Testing confirms that the right problem is solved.

d) Satisfy user needs
Incorrect. As we saw in the Vee model of the systems life cycle, testing traces back to user.

Slide 22:

Slide 23:

Learning Objective #1

Which is not a testing process within the life cycle?

Answer Choices

1. Test planning
2. Preparation of test documentation
3. Test execution and reporting
4. Identification of system requirements

Slide 24:

Learning Objective #1

Review of Answers

a) Test planning
Incorrect. Test planning is done during concept of operations and system requirements.

b) Preparation of test documentation
Incorrect. Test documents are created during high-level design and detailed design.

c) Test execution and reporting
Incorrect. Test execution and reporting are done at each level of the testing workflow.

d) Identification of system requirements
Correct! System requirements are not a testing process. Test planning is based on system requirements.

Slide 25:

Summary of Learning Objective #1

Describe Within the Context of a Systems Life Cycle the Role of a Test Plan and the Testing to Be Undertaken

• Reviewed that RMC units are tested to insure that the installed RMC unit meets the expected user needs
• Testing fits within the systems life cycle on the right side of the Vee model, which is traceable back to the user needs
• Reviewed that verification methods include inspection, demonstration, analysis, and testing
• Testing process is conducted within the life cycle in three steps— planning, documentation preparation, and test execution—that results in test reports

Slide 26:

Learning Objective #2: Recognize the Purpose, Structure, and Content of Well-Written Test Documentation Based on IEEE 829

• Purpose of a test plan
• What is a test plan?
• Structure of test plans
• Content of test plans

Slide 27:

Learning Objective #2

Purpose of a Test Plan

Used to Plan and Manage the Execution of Tests

• Identifies an overall document for:
  • Planning the RMC unit tests
  • Managing the RMC unit tests
• Identifies test activities and methods
• Sets objectives for each test activity
• Identifies the testing risks, resources, and schedule
• Determines the requirements for test documentation

Slide 28:

Learning Objective #2

What Is a Test Plan?

From IEEE 829-2008 Standard

• A test plan is a document describing:
  • Scope (technical management)
  • Approach
  • Resources needed
  • Schedule to complete
• A test plan identifies:
  • Test items
  • Features to be tested
  • Testing tasks
  • Risks requiring contingency plan

Slide 29:

Learning Objective #2

What is a Test Plan?

From IEEE 829-2008 Standard (cont.)

• Two types of test plans:
  • Level Test Plan (LTP)
  • Master Test Plan (MTP)
• Be aware that test plans are not part of the ITS standards

Slide 30:

Learning Objective #2

Structure of Test Plans

From IEEE 829-2008 Standard

• Master Test Plan
  • Integrity level scheme and choice
  • Overall test processes, activities, and tasks
  • Test levels and documents
• Level Test Plan
  • Unit test plans(s)
  • Subsystem integration test plans(s)
  • System acceptance test plan

Slide 31:

Learning Objective #2

Structure of Test Plans

Structure of Test Plans

(Extended Text Description: Graphic depicts hierarchy of Master Test Plan box at top with lines connecting to Unit Test Plan box, Subsystem Integration Test Plan box, and System Acceptance Test Plan box. Graphic is reused from prior modules. The top level (Master Test Plan) has the following items associated with it: Integrity level scheme and choice; Overall test processes, activities, and tasks; Test levels and documents. The bottom level has the following items associated with it: Scope of test level, Resources, Test method(s).)

A Master Test Plan may not always be required!

Slide 32:

Learning Objective #2

Structure of Test Plans

Workflow of RMC Unit Test Plans

(Extended Text Description: Author's relevant description: New graphic created for this module. The graphic from Slide 13 is repeated at the right of the slide. Graphic also depicts a horizontal Test Sequence Workflow arrow at the bottom directed towards the right. Above the arrow is a Master Test Plan box with lines connecting to all of the following: RMC Unit Test Plan box, ASC Unit Test Plan box, RMC Subsystem Integration Test Plan box, ASC Subsystem Integration Test Plan box and System Acceptance Test Plan box.)

Slide 33:

Learning Objective #2

Content of Test Plans: PRL for RMC System

Protocol Requirements List from A309b

UN ID	User Need	RQ. ID	Requirement	Conformance	Project Requirement	Additional Project Requirements
2.1	Provide Live Data	3.2.1	Provide Live Data	M	YES
2.2	Provide Logged Data	3.2.2	Provide Off-Line Logged Data	M	YES
2.3	Retrieve Identity	3.3.1	General Configuration	M	YES	NTCIP 1207 v02 Annex B, CG B.3
2.9	Configure RMC Unit	3.3.1	Configuration of Device	M	YES	NTCIP 1201, CL 2.2
2.4	Fixed Rate	3.3.2	Metered Lane	M	YES	NTCIP 1207-3.3
2.5	Queue Override	3.3.3	Queue Override	O	YES/NO	Not widely used
2.N	Block Objects	3.N		O		Undecided

Users may modify entries in rows to suit local project needs, but columns should not be changed to remain consistent with SEP

Slide 34:

Learning Objective #2

Content of Test Plans

Master Test Plan Outline per IEEE 829

• Introduction
  • Document identifier, scope, and references
  • System overview key features
  • Test overview (organization, schedule, resources, tools, etc.)
• Details of the Master Test Plan
  • Test processes, definitions
  • Test documentation requirement for NTCIP 1207 v02
  • Test administration requirement
  • Test reporting requirement
• General
  • Glossary
  • Change procedures, history

Slide 35:

Learning Objective #2

Content of Test Plans

Level Test Plan Outline per IEEE 829

• Introduction
  • Document identifier, scope, and references
  • Level in the overall sequence
  • Test classes and test conditions
• Details of the Level Test Plan
  • Test items and their identifiers
  • Test Traceability Matrix of NTCIP 1207 v02 objects and dialogs
  • Features tested/not tested
  • Test approach
  • Pass/Fail criteria
  • Suspension criteria and requirements to resume testing
  • Test deliverables

Slide 36:

Learning Objective #2

Content of Test Plans

Level Test Plan Outline per IEEE 829 (cont.)

• Test Management
  • Planned activities and tasks
  • Test progression
  • Environment/infrastructure
  • Responsibilities/authorities
  • Interfaces among stakeholders
  • Resources and training
  • Schedules, estimates, and costs
  • Risk(s) and contingencies

Slide 37:

Learning Objective #2

Content of Test Plans

Level Test Plan Outline per IEEE 829 (cont.)

• General
  • Quality assurance procedures
  • Metrics for specific measures
  • Glossary
  • Document change procedures and history

Slide 38:

Slide 39:

Learning Objective #2

Which is not a reason to use the IEEE 829 Standard?

Answer Choices

1. IEEE 829 is part of NTCIP 1207 Standard v02
2. Provides familiar documents
3. Standard definition of terms
4. Reuse in later projects

Slide 40:

Learning Objective #2

Review of Answers

a) IEEE 829 is part of NTCIP 1207 Standard v02
Correct! NTCIP 1207 Standard v02 does not reference IEEE 829 standard.

b) Provides familiar documents
Incorrect. IEEE 829 provides familiar documents and steps.

c) Standard definition of terms
Incorrect. IEEE 829 does provide standard definitions.

d) Reuse in later projects
Incorrect. Using the standard steps and definitions of IEEE 829 results in documents that can be easily reused in later projects or when the existing system is expanded at a later date.

Slide 41:

Slide 42:

Learning Objective #2

Which is not a part of a Level Test Plan?

Answer Choices

1. Introduction
2. Test details
3. Planning for multiple levels of test
4. Test management

Slide 43:

Learning Objective #2

Review of Answers

a) Introduction
Incorrect. Each Level Test Plan includes an Introduction to the testing.

b) Test details
Incorrect. Each Level Test Plan includes test details.

c) Planning for multiple levels of testing
Correct! The Master Test Plan documents and coordinates multiple Level Test Plans

d) Test management
Incorrect. Each Level Test Plan includes test management.

Slide 44:

Summary of Learning Objective #2

Recognize the Purpose, Structure, and Content of Well-Written Test Documentation Based on IEEE 829

• The purpose of a test plan is to provide an overall document for planning and managing RMC unit tests
• Test plan documents describe the testing scope, approach, resources, and schedule, among other items
• Test plans are structured as Level Test Plans for all projects, plus an optional Master Test Plan for large, complex projects
• Test plan content includes introduction, details, management, and general information

Slide 45:

Learning Objective #3: Describe Test Documentation for NTCIP 1207 Standard v02: Test Plan, Test Design Specifications, Test Cases, Test Procedures, and Test Reports

• Overview of test documentation
• Understand the difference between test plans and test documentation
• Overview of a test design and the relationships between test plans, test design, test cases, and test procedures

Slide 46:

Learning Objective #3

Overview of Test Documentation

Test Documentation per IEEE 829

• Test documentation requirements are specified in the MTP (if used)
• Detailed list of test deliverables specified in the LTP
  • Test plan
  • Test designs
  • Test cases
  • Test procedures
  • Test logs
  • Anomaly reports
  • Interim test status reports
  • Test reports
  • Master Test Reports (if there is an MTP)

Slide 47:

Learning Objective #3

Overview of Test Documentation

Test Documentation per IEEE 829 (cont.)

(Extended Text Description: Graphic depicts a Master Test Plan box at the top with lines connecting to Unit Test Plan box, Subsystem Integration Test Plan box and System Acceptance Test Plan box. The Unit Test Plan box has a line connecting to Unit Test Design box. Unit Test Design box has a line connecting to Unit Test Cases and another line connecting to Unit Test Procedures box. The top level (Master Test Plan) has the following items associated with it: Integrity level scheme and choice; Overall test processes, activities, and tasks; Test levels and documents. The second level has the following items associated with it: Scope of test level, Resources, Test method(s). The third level has Detail updates for test methods, Features to be tested. And on the bottom level, associated with Unit Test Cases is the text Input, Output; and associated with Unit Test Procedures is the text Test setup, Execution instructions.)

Slide 48:

Learning Objective #3

Overview of Test Documentation

Test Documentation per IEEE 829 (cont.)

Graphic depicts Unit Interim Test Status Report box with a line connecting to Unit Test Logs box.

(Extended Text Description: Author's relevant description: Graphic depicts Unit Interim Test Status Report box with a line connecting to Unit Test Logs box. Unit Test Logs box has lines connecting to Unit Test Report box, Subsystem Integration Test Report box and System Acceptance Test Report box. Each of those three boxes have a line connecting to Master Test Report box. All of the boxes except Master Test Report box have red arrows directed to Anomaly Report box.)

Slide 49:

Learning Objective #3

Understand the Difference Between Test Plans and Test Documentation

Test Plans vs. Test Documents

• Test plans
  • Define the required RMC unit test documents
  • Are developed earlier than test documents
• Test documentation includes all information to be delivered by all of the test activities:
  • Test documents, including among others:
    • Test cases
    • Test procedures
    • Test reports
  • Test inputs
  • Test output data
  • Test tools

Slide 50:

Learning Objective #3

Overview of a Test Design and the Relationships Between Test Plans, Test Design, Test Cases, and Test Procedures

Test Design

• IEEE 829 defines a test design as a test document that:
  • Specifies the details of the test approach
  • Identifies the features to be tested by this design
    • RTCTM
• Identifies the associated tests—commonly including the organization of the tests into groups
  • Test cases
  • Test procedures

Slide 51:

Learning Objective #3

Overview of Test Documentation

Example of RMC Unit Test Documentation

Requirement		Test Case
ID	Title	ID	Title
1.	Metered Lane Main Configuration
1.1	Maximum Number of Metered Lanes
		TC1.1	Test the Boundaries
1.2	Number of Metered Lanes
		TC1.2	Test the Combinations
1.3	Metered Lane Configuration Table
		TC2.1	Test the Min Meter Time
		TC2.2	Test the Min Red
		TC2.3	Test the Min Green

Slide 52:

Learning Objective #3

Overview of a Test Design and the Relationships Between Test Plans, Test Design, Test Cases, and Test Procedures

Test Sequence

• Test plan vs. Test design
  • Only one test design per test plan
• Test design vs. Test case
  • One test design may be associated with multiple test cases
  • Any one test case is associated with only one test design
• Test case vs. Test procedure
  • One test case may be associated with multiple test procedures and vice versa (IEEE 829)
• For simple devices, NTCIP combines test case and test procedure

Slide 53:

Learning Objective #3

Overview of Test Documentation

Example of RMC Unit Test Documentation

(Extended Text Description: Author's relevant description: Graphic depicts a Unit Test Plan box at the top with an arrow connecting down to a Unit Test Design box. This box has three arrows pointing down to the next level of boxes: Unit Test Case 1, Unit Test Case 2, and Unit Test Case 3. Unit Test Case 1 points down to a fourth level box, Unit Test Procedure 1. The Unit Test Case 2 box also points down to Unit Test Procedure 1. The Unit Test Case 3 box has two arrows pointing down to the fourth level of boxes, Unit Test Procedure 2 and Unit Test Procedure 3. At the top, Unit Test Plan has three bullets, Scope of test level, Resources, Test method(s). Next to Unit Test Design are two bullets, Detail updates for test methods, Features to be tested. Next to the Unit Test Cases is Input, Output. And next to the Unit Test Procedure boxes is Test setup, Execution instructions.)

Slide 54:

Slide 55:

Learning Objective #3

When is the test documentation completed?

Answer Choices

1. Before the test is executed
2. Only during the test execution
3. Only after the test is executed
4. During and after the test execution

Slide 56:

Learning Objective #3

Review of Answers

a) Before the test is executed
Incorrect. The test plan is developed before test execution, but is not filled in with test documentation results.

b) Only during the test execution
Incorrect. The test data are recorded during the test execution, but documentation also requires summaries after execution.

c) Only after the test is executed
Incorrect. Test summaries are documented after test execution, but also include the test data during test execution.

d) During and after the test execution
Correct! Test data and test summaries are both part of documentation, taken during and after test execution.

Slide 57:

Summary of Learning Objective #3

Describe Test Documentation for NTCIP 1207 Standard v02: Test Plan, Test Design Specifications, Test Cases, Test Procedures, and Test Reports

• List of test documentation according to IEEE 829
• Test plan documents describe the testing scope, approach, resources and schedule that are delivered before testing. Test documentation includes test data and test summaries delivered during and after testing
• Test design specifies the details of the test approach and identifies the features to be tested, usually organized into groups of test cases and test procedures

Slide 58:

Learning Objective #4: Describe the Application of Good Test Documentation to a Ramp Metering Control Unit Based on NTCIP 1207 Standard v02

• Identify key elements of NTCIP 1207 Standard v02 relevant to what is covered in the test plan
• Develop a Requirements Test Case Traceability Matrix (RTCTM)
• Review key elements of the conformance statement

Slide 59:

Learning Objective #4

Key Elements of NTCIP 1207 Standard v02 Relevant to Test

Definition of RMC Unit from Module A309

Ramp Meter Control

A system in which the entry of vehicles onto a freeway from an on-ramp is controlled by a traffic signal, allowing a fixed number of vehicles to enter from each metered lane of the on-ramp during each cycle.

A Ramp Meter Control (RMC) unit consists of:

• The field controller
• Suite of sensors
• Warning signs and signals

Slide 60:

Learning Objective #4

Key Elements of NTCIP 1207 Standard v02 Relevant to Test

Basic RMC Unit Hardware

• RMC Field Hardware
  • ATC 5201, 5202 or legacy
  • Communications Device
  • Vehicle Detection
    • Inductive Loops
    • Video Detection
    • Magnetic, radar, others
  • Load Switches
  • Equipment Electrical Cabinet
  • Traffic Signals
    • Green Amber Red option
    • Green Red option
  • STOP BAR

(Extended Text Description: Author's relevant description: New graphic created by author for this module depicting a roadside electrical cabinet containing Advanced Transportation Controller, input and output load switches. The inputs are shown connected to vehicle detectors. The output load switches are shown connected to a traffic signal head. A double-ended arrow connects the roadside cabinet assembly to a traffic management computer.)

Slide 61:

Learning Objective #4

Key Elements of NTCIP 1207 Standard v02 Relevant to Test

Basic RMC Unit Software (cont.)

• RMC unit software application
  • RMC unit configuration
  • Read inputs from detectors
  • Control algorithm
  • Write outputs to signal heads
  • Communications to/from TMC

Slide 62:

Learning Objective #4

Key Elements of NTCIP 1207 Standard v02 Relevant to Test

RMC Unit Test Environment for Unit Level Testing

• Simulated inputs (vehicle calls)
• RMC unit outputs to simulator
• Communications network
• RMC units (NTCIP 1207 v02)
• Test software (TMC simulator)
• Data analyzer (as required)

(Extended Text Description: Author's relevant description: New graphic created by author for this module depicting an ATC 5201 or ATC 5202 controller with an arrow labeled Signals directed to the right towards an Intersection Simulator and an arrow labeled Detectors directed left from the Intersection Simulator toward the ATC. A cloud graphic is depicted with a double ended arrow labeled NTCIP 1207 located between the cloud and the ATC, plus another double ended arrow labeled NTCIP 1207 located between the cloud and a computer labeled Test Software.)

Slide 63:

Learning Objective #4

Key Elements of NTCIP 1207 Standard v02 Relevant to Test

What Is Provided by NTCIP 1207 Standard v02?

• Conformance Groups (CG)
• CG is a set of managed objects

Ref	Conformance Group	Clause of Profile	Status	Support
B.5	Metered Lane Conformance Group	NTCIP 1207 3.3	M	Yes
B.12	Configuration Conformance Group	NTCIP 1207 2.2	M	Yes
B.10	Physical I/O Conformance Group	NTCIP 1207 3.7	O	Yes / No
	- Metered Lane Output	NTCIP 1207 3.7	O	Yes / No
	- Dependency Group Output	NTCIP 1207 3.7	O	Yes / No

Slide 64:

Learning Objective #4

Key Elements of NTCIP 1207 Standard v02 Relevant to Test

What Is Not Included in NTCIP 1207 Standard v02?

From Prior Modules:

• User needs
• Requirements
• Dialogs
• Protocol Requirements List (PRL)
• Requirements Traceability Matrix (RTM)
• Requirements Test Case Traceability Matrix (RTCTM)
• Test cases
• Test procedures

Although not included, each is required to develop test documents.

Slide 65:

Learning Objective #4

Develop an RTCTM

Example of RTCTM

• Correlates each requirement to a test case
  • Requirement identification and title
  • Test case identification and title

Requirement		Test Case
ID	Title	ID	Title
1.	Metered Lane Configuration
1.1	Set maximum # of metered lanes	TC1.1	Test the Boundaries
1.2	Set number of metered lanes	TC1.2	Test the Combinations

Example of Requirements Test Case Traceability Matrix

Slide 66:

Learning Objective #4

Key Elements of a Conformance Statement

Conformance Statement (CF)

• Suppliers claiming conformance to NTCIP shall provide a CF
• CF applies to specific implementations (e.g. RMC)
• Uses for conformance statement
  • Supplier: Checklist of objects
  • Procurement: Clear, unambiguous list of NTCIP deliverables
  • User: System interoperability
  • Tester: Selection of tests to assess the conformance claim
• Conformance statement content
  • Requirements list
  • Indicates the features that are supported in the implementation

Slide 67:

Slide 68:

Learning Objective #4

What is the primary purpose of the RTCTM ?

Answer Choices

1. Sets the testing workflow sequences
2. Correlates User Needs to Requirements
3. Describes the Optional Objects needed to fulfill Requirements
4. Correlates each RMC Requirement to Test Case

Slide 69:

Learning Objective #4

Review of Answers

a) Sets the testing workflow sequences
Incorrect. Testing workflow is part of the Level Test Plans

b) Correlates User Needs to Requirements
Incorrect. User Needs to Requirements are part of the Protocol Requirements List

c) Describes the Optional and Mandatory Objects
Incorrect. Optional and Mandatory Objects provided by the manufacturer are part of the Conformance Statement

d) Correlates each RMC Requirement to Test Case
Correct! RTCTM depicts the Test Cases that will be used to verify each Requirement.

Slide 70:

Summary of Learning Objective #4

Describe the Application of Good Test Documentation to a Ramp Meter Unit Based on NTCIP 1207 Standard v02

• The key elements of the NTCIP 1207 Standard v02 that are relevant to the test plan include the RMC unit configuration, detector inputs, signal outputs, and others
• We developed a RTCTM based on test cases, features to be tested, and RMC unit requirements
• The conformance statement must be provided by RMC unit manufacturers claiming compliance to the NTCIP 1207 Standard v02. The Conformance Statement includes the requirements list and the RMC unit features that are supported in the manufacturer's implementation

Slide 71:

Learning Objective #5: Identify a Process to Write Test Plans to Verify NTCIP 1207 Standard v02 Requirements

• Apply the process of creating test documentation based on test specifications
• Address consequences of testing boundary and error conditions
• Describe test tools and equipment available

Slide 72:

Learning Objective #5

Test Documentation Based on Test Specification

Develop Test Case Documents - Test Case

Test Case: TC1.1	Title:	Test the Boundaries
	Description	This test case verifies the maximum number of metered lanes that can be SET by the central station. The test is conducted just below, just above, and exactly at the boundary
	Variables	Max Lanes	From project requirements
		Max Lanes -1	From the test plan
		Max Lanes +1	From the test plan
	Pass/Fail Criteria	1. The DUT shall accept data at Max Lanes 2. The DUT shall accept data at Max Lanes -1 3. The DUT shall return an error at Max Lanes +1

Step	Test Procedure	Expected Results
Slide 1:	Configure: SET the Max Lanes = 2, record the DUT response	Responds with Max Lanes = 2
Slide 2:	SET the number of Metered Lanes = 1, record the DUT response	Responds with Metered Lanes = 1
Slide 3:	SET the number of Metered Lanes = 2, record the DUT response	Responds with Metered Lanes = 2
Slide 4:	SET the number of Metered Lanes = 3, record the DUT response	Error, exceeds Max Lanes = 2

Slide 73:

Learning Objective #5

Testing Boundary and Error Conditions

Positive and Negative Testing

• Positive testing
  • Valid input values, dialogs, and sequences per test procedure
  • Expected outputs from Device Under Test (DUT)
• Negative testing
  • Assert invalid input values, dialogs, or sequences per the test procedure
  • DUT does not process
  • DUT retains normal operation
  • DUT output per test procedure
    • Error message
    • Error type

Slide 74:

Learning Objective #5

Testing Boundary and Error Conditions

Testing Boundary Conditions

• All boundary condition are tested:
  • Just below each limit
  • Just above each limit
  • Exactly on each limit
• Boundary is valid, DUT should:
  • Process successfully
  • Respond accordingly
• If error conditions occur, DUT should:
  • Respond with error message
  • Remain in normal operation
  • No communications loss

Slide 75:

Learning Objective #5

Testing Boundary and Error Conditions

NTCIP Testing: Extent Should be Part of Specification

• Complex process that requires:
  • Test planning
  • Test documentation preparation
  • Test execution
  • Test result reporting
• Every NTCIP object should be tested
  • Sampling of valid inputs
  • Test each boundary condition
  • Selectively test error conditions for critical functions
• Progression testing for new and corrected features
• Regression testing
  • No unintended changes
  • Done for all software affected by the test failure

Slide 76:

Learning Objective #5

Test Tools and Equipment Available

NTCIP Test Tools

• Capable of performing tests for conformance to specific NTCIP information level standards
• Support for communications testing such as SNMP
• Scripting features to support automated testing
• Support of various protocols such as PPP, PMPP, TCP/IP, etc.
• Support a wide variety of media including Ethernet and Serial

Slide 77:

Learning Objective #5

Test Tools and Equipment Available

Passive NTCIP Test Tools

• Used as a data analyzer
• Monitor the data exchange only
• Provide real-time data capture
• Do not provide an NTCIP stimulus
• Do not respond to an ITS device
• Examples
  • Serialtest
  • Ethereal
  • Others

Slide 78:

Learning Objective #5

Test Tools and Equipment Available

Active Test Tools

• Used as the main test software
• Send message to DUT
• Timestamp and log DUT response
• Mandatory and optional objects
• Active NTCIP tool limitations, require special-purpose software
  • Block objects
  • Manufacturer-specific objects
  • Communications load tests
• Active NTCIP test tool examples
  • DeviceTester
  • NTCIP Exerciser
  • Ntester
  • SimpleTester

Slide 79:

Summary of Learning Objective #5

Identify a Process to Write Test Plans to Verify NTCIP 1207 Standard v02 Requirements

• We created test documentation from test specifications including:
  • Test cases
  • Test procedures
• We learned consequences of boundary conditions and error conditions
  • Sample of valid inputs
  • Each boundary condition
  • Critical functions
• Use of test tools automate the test process, but have limitations
  • Passive: Monitor and record
  • Active: Stimulate with response
  • No block objects and no manufacturer-specific objects
  • Development of special tests for special features

Slide 80:

What We Have Learned

1. RMC units are tested to ensure that the RMC units meet the expected user needs and associated requirements when installed. In addition to testing, other verification methods include inspection, demonstration, and analysis.
2. Test plan is used to plan and manage the RMC tests, including the scope, approach, resources, and schedule.
3. According to the IEEE 829 standard, the test documentation includes test data and test summaries delivered before, during, and after the test execution by the test operator.

Slide 81:

What We Have Learned

4. Key elements of an RMC that are relevant to the test plan include configuration, detector inputs, and signal outputs. These key elements are included in an RTCTM based on test cases.
5. Manufacturers claiming conformance to NTCIP 1207 must provide a conformance statement including the requirements list and the features supported in the RMC implementation.
6. For Boundary Conditions and Error Conditions a sample of valid inputs of the most critical functions is tested at each boundary condition. Passive tools monitor and record while Active tools stimulate to create responses. Special tests must be developed for block objects, and manufacturer-specific objects.

Slide 82:

Resources

• US Department of Transportation, Systems Engineering for Intelligent Transportation Systems, USDOT, January 2007
• NTCIP 1207 RMC Units Standard v02
• T202: Overview of Test Design Specifications, Test Cases and Test Procedures
• T203 Part 1 of 2: How to Develop Test Cases for an ITS Standards-based Test Plan, Part 1 of 2
• T203 Part 2 of 2: How to Develop Test Cases for an ITS Standards-based Test Plan, Part 2 of 2
• T204 Part 1 of 2: How to Develop Test Procedures for an ITS Standard-based Test Plan, Part 1 of 2
• T204 Part 2 of 2: How to Develop Test Procedures for an ITS Standard-based Test Plan, Part 2 of 2

Slide 83:

Resources (cont.)

• Institute of Transportation Engineers, ATC 5201 Advanced Transportation Controller (ATC) Standard Version 06. ATC Joint Committee, 30 July 2012.
• Institute of Transportation Engineers, ATC 5202 Model 2070 Controller Standard Version 03. ATC Joint Committee, 28 December 2012.
• IEEE Standard for Software and System Test Documentation, IEEE Computer Society, IEEE Std 829-2008, July 18, 2008.

Slide 84:

Questions?