Module 18 - T313
T313: Applying Your Test Plan to Environmental Sensor Stations (ESS) Based on NTCIP 1204 v04 ESS Standard
HTML of the Student Supplement
(Note: This document has been converted from the Student Supplement to 508-compliant HTML. The formatting has been adjusted for 508 compliance, but all the original text content is included, plus additional text descriptions for the images, photos and/or diagrams have been provided below.)
T313: Applying Your Test Plan to the Environmental Sensor Stations based on the NTCIP 1204 ESS Standard v04
Table of Contents
1. Module Description - 2
2. Introduction/Purpose - 2
3. Samples/Examples - 3
3.1. Sample of Completed PRL - 3
3.2. Sample Test Plan - 4
4. Reference to Other Standards - 9
5. Glossary - 9
6. Study Questions - 10
7. Icon Guide - 11
1. Module Description
An environmental sensor station (ESS) consists of a remote microprocessor coupled with any number of sensors that monitor the surrounding environment, which may monitor weather, pavement, water levels, and/or air quality conditions. An ESS may also be used to control related equipment, such as pavement treatment systems and/or cameras.
The focus of this updated module is to assist user agencies in creating test documentation specific to their ESS needs based on the NTCIP 1204 Standard v04. The test documentation will include a test plan and test design specification that links to the standardized test cases and test procedures that are already available in NTCIP 1204 v04 documentation and explain how this testing documentation traces back to the requirements and design elements of the standard. The module will also provide examples of how to perform tests. Prior to taking this course, the student is expected to be knowledgeable of the NTCIP 1204 Standard v04 and testing methodologies.
This module will be placed in the context of the systems engineering process as well in the acquisition curriculum path. The complete series of ITS Standards Training Modules for the acquisition of an ESS is as follows: I101, A101, A102, A201, A313a, A313b, T101, T201, T202, T204, and T313. This module is the final module in the ESS acquisition series.
2. Introduction/Purpose
At the conclusion of this course, participants will be able to:
1. Describe within the context of the testing lifecycle the role of test plans and the testing to be undertaken
- What is an ESS?
- Review the concept of the systems life cycle
- Describe the purpose of the testing process in relation to the systems life cycle
- Describe test documentation as defined by IEEE 829-2008
2. Identify key elements of NTCIP 1204 v04 relevant to the test plan
- Explain relationship among NTCIP standards
- Explain structure of NTCIP 1204 v04
- Explain elements related to testing
3. Describe the application of a good test plan to an ESS system being procured
- State what a typical ESS site might include
- Understand which other modules assist in defining requirements
- Explain how requirements T313 trace to test cases and test procedures through a test design specification
- Create a test plan for an ESS
4. Describe the testing of an ESS using standard procedures
- Explain how to perform sample test procedures
- Use different types of test steps
- Analyze and record test results
- Appreciate the benefits of automated testing
3. Samples/Examples
3.1. Sample of Completed PRL
Figures 1-3 show the first few pages of a completed PRL as might apply to the Sample Test Plan below.
Figure 1: First page of a sample PRL
Table 6 Protocol Requirements List
(Extended Text Description: Figure 1: This figure shows the first page of a sample PRL that has been filled out. The support column is filled out as follows:
- Architectural Needs: Yes
- Generic Architectural Needs: Yes
- Features: Yes
- ESS Manager Features: Yes
- Generic Features: Yes
- Monitor Door Status: No
- Retrieve ESS Door Status: N/A
- Monitor Power: No
- Retrieve Battery Status: N/A
- Retrieve Line Volts: N/A
- Monitor Mobile Situation Data: N/A
- Retrieve Mobile ESS Movement: N/A
- Determine ESS Type: Yes
- Permanent: Yes
- Transportable: No
- Mobile: No
- Retrieve ESS Characteristics: Yes
- Monitor the Status of the ESS: No
- Retrieve ESS Status: N/A
- Sensor Manager Features: Yes
- Monitor Weather Conditions: Yes
The full text of the table is shown below:
| Protocol Requirements List (PRL) | ||||||
| User Need ID | User Need | FR ID | Functional Requirement | Conformance | Support | Additional Specifications |
|---|---|---|---|---|---|---|
| 2 4 | Architectural Needs | M | Yes | |||
| 2.4.1 | Generic Architectural Needs | M | Yes | |||
| 2.5 | Features | M | Yes | |||
| 2.5.1 | ESS Manager Features | M | Yes | |||
| 2.5.1.1 | Generic Features | M | Yes | |||
| 2.5.1.2 | Monitor Door Status | 0 | Yes / No | |||
| 3.5 1.2.1 | Retrieve ESS Door Status | M | Yes / NA | |||
| 2.5.1.3 | Monitor Power | 0 | Yes / No | |||
| 3.5.1.2.2 | Retrieve Battery Status | 0.1(1..*) | Yes / No / NA | |||
| 3.5 1.2.3 | Retrieve Line Volts | 0.1(1..*) | Yes / No / NA | |||
| 2.5.1.4 | Monitor Mobile Station Data | Mobile:M | Yes / NA | |||
| 3.5.1.3.1 | Retrieve Mobile ESS Movement | M | Yes / NA | NTCIP 1204 v04 does not impose any accuracy requirements. Any accuracy requirements should be inserted here. _____________________________ | ||
| 2.5.1.5 | Determine ESS Type | Yes | ||||
|
2.5.1.5.a 2.5.1.5.b 2.5.1.5.c (Mobile) |
Permanent Transportable Mobile |
0.2(1) 0.2(1) 0.2(1) |
Yes / No Yes / No Yes / No |
|||
| _ | 3.5.1.1.1 | Retrieve ESS Characteristics | M | Yes | ||
| 2.5.1.6 | Monitor the Status ol the ESS | 0 | Yes / No | |||
| 3.5.1.2.4 | Retrieve ESS Status | M | Yes / NA | |||
| 2.5.2 | Sensor Manager Features | 0.3(1..*) | Yes / No | |||
| 2.5.2.1 | Monitor Weather Conditions | 0.4(1..*) | Yes / No / NA | |||
)
Figure 2: Second page of a sample PRL
(Extended Text Description: Figure 2: This figure shows the second page of a sample PRL that has been filled out. The support column is filled out as follows:
- Monitor Atmospheric Pressure: No
- Retrieve Atmospheric Pressure Metadata – Location: N/A
- Retrieve Atmospheric Pressure Metadata – Sensor Information: N/A
- Configure Atmospheric Pressure Metadata – Location: N/A
- Retrieve Atmospheric Pressure: N/A
- Required Number of Atmospheric Pressure Sensors: N/A
- Monitor Winds: Yes
- Retrieve Metadata for Each Wind Sensor – Text Description: No
- Retrieve Metadata for Each Wind Sensor – Location: No
- Retrieve Metadata for Each Wind Sensor – Sensor Information: Yes
- Configure Wind Sensor Metadata – Location: No
- Retrieve Wind Data: Yes
- Required Number of Wind Sensors: Yes
- Monitor Air Temperature: Yes
- Retrieve Temperature Sensor Metadata – Height: No
- Retrieve Temperature Sensor Metadata – Location: No
- Retrieve Temperature Sensor Metadata – Sensor Information: Yes
The full text of the table is shown below:
| Protocol Requirements List (PRL) | ||||||
| User Need ID | User Need | FR ID | Functional Requirement | Conformance | Support | Additional Specifications |
|---|---|---|---|---|---|---|
| 2.5.2.1.1 | Monitor Atmospheric Pressure | 0.5(1..*) | Yes / No / NA | |||
| 3.5.2.1.10.1 (PressLoc) | Retrieve Atmospheric Pressure Metadata - Location | 0 | Yes / No / NA | |||
| 3.5.2.1.10.2 | Retrieve Atmospheric Pressure vetacata -Sensor |formation | 0 | Yes / No / NA | |||
| 3.5.2.1.10.3 | Configure Atmospheric Pressure Metadata - Location | PressLoc:O | Yes / No / NA | |||
| 3.5.2.3.2.10 | Retrieve Atmospheric Pressure | M | Yes / NA | |||
| 3.6.1 | Required Number ol Atmospheric Pressure Sensors | M | Yes / NA |
The ESS shall support at least ____ (1..255:Default=1) atmospheric pressure sensors. |
||
| 2.5.2.1.2 | Monitor Winds | 0.5(1..*) | Yes / No / NA | |||
| 3.5.2 1.2 (Wind) | Retrieve Metadata for Each Wind Sensor - Teort Description | 0 | Yes / No / NA | |||
| 3.5.2.1.11.1 (WindLoc) | Retrieve Metadata for Each Wine Sensor - Location | 0 | Yes / No / NA | |||
| 3.5.2.1.11.2 | Retrieve Metadata tor Each Wind Sensor-Sensor Info rmation | 0 | Yes / No / NA | |||
| 3.5.2.1.11.3 | Configure Wind Sensor Metadata - Location | Wind:0; WindLoc:0 | Yes / No / NA | |||
| 3.5.2.3 2 2 | Retrieve Wind Data | M | Yes / NA | |||
| 3 6.2 | Required Number of Wind Sensors | M | Yes / NA | The ESS shall support at least __1__ (1..255:Default=1) wind sensors. | ||
| 2.5.2.1.3 | Monitor Air Temperature | 0.5(1..*) | Yes / No / NA | |||
| 3 5.2.1.3 (Temperature) | Retrieve Temperature Sensor Metadata - Height | 0 | Yes / No / NA | |||
| 3.5.2.1.12.1 (TempLoc) | Retrieve Ternperature Sensor Metadata - Location | 0 | Yes / No / NA | |||
| 3.5.2.1.12.2 | Retrieve Temperature Sensor Metadata - Sensor Information | 0 | Yes / No / NA | |||
)
Figure 3: Third page of a sample PRL
(Extended Text Description: Figure 3: This figure shows the third page of a sample PRL that has been filled out. The support column is filled out as follows:
- Configure Temperature Sensor Metadata – Location: No
- Retrieve Air Temperature: Yes
- Retrieve Daily Minimum and Maximum Temperature: Yes
- Required Number of Temperature Sensors: Yes
- Monitor Relative Humidity: No
- Retrieve Precipitation Sensor Metadata – Location: N/A
- Retrieve Precipitation Sensor Metadata – Sensor Information: N/A
- Configure Precipitation Sensor Metadata – Location: N/A
- Configure Precipitation Total User Specified Period: N/A
- Retrieve Precipitation Presence: N/A
- Retrieve Precipitation Rates: N/A
- Retrieve Precipitation Totals: N/A
The full text of the table is shown below:
| Protocol Requirements List (PRL) | ||||||
| User Need ID | User Need | FR ID | Functional Requirement | Conformance | Support | Additional Specifications |
|---|---|---|---|---|---|---|
| 3.5.2.1.12.3 | Configure Temperature Sensor Metadata - Location | Temperature:0; TempLocc:0 | Yes / No / NA | |||
| 3.5.2.3.2.3 | Retrieve Air Temperature | M | Yes / NA | |||
| 3.5.2.3.2.4 | Retrieve Daily Minimum and Maximum Temperature | M | Yes / NA | |||
| 3.6.3 | Required Number of Temperature Sensors | M | Yes / NA | The ESS shall support at least ____ (1 ..255:Default=1) temperature sensors. | ||
| 2.5.2.1.4 | Monitor Relative Humidity | 0.5(1..*) | Yes / No / NA | |||
| 3.5.2.1.13.1 (HumLoc) | Retrieve Humidity Sensor Metadata - Location | 0 | Yes / No / NA | |||
| 3.5.2.1.13.2 | Retrieve Humidity Sensor Metadata - Sensor Information | 0 | Yes / No / NA | |||
| 3.5.2.1.13.3 | Configure Humidity Sensor Metadata - Location | HumLoc:0 | Yes / No / NA | |||
| 3.5.2.3.2.11 | Retrieve Relative Humidity | M | Yes / NA | |||
| 3.6.4 | Required Number of Humidity Sensors | M | Yes / NA | The ESS shall support at least ____ (1 ..255:Default=1) humidity sensors. | ||
| 2.5.2.1.5 | Monitor Precipitation | 0.5(1..*) | Yes / No / NA | |||
| 3.5.2.1.14.1 (PrecipLoc) | Retrieve Precipitation Sensor Metadata - Location | 0 | Yes / No / NA | |||
| 3.5.2.1.14.2 | Retrieve Precipitation Sensor Metadata - Sensor Information | 0 | Yes / No / NA | |||
| 3.5.2.1.14.3 | Configure Precipitation Sensor Metadata - Location | PrecipLoc:0 | Yes / No / NA | |||
| 3.5.2.1.14.4 | Configure Precipitation Total User Specified Period | PrecipUser:0 | Yes / No / NA | |||
| 3 5 2 3.2.6.4 | Retrieve Precipitation Presence | M | Yes / NA | |||
| 3.5.2.3.2.6.5 (PrecipRate) | Retrieve Precipitation Rates | 0 | Yes / No / NA | |||
| 3 5 2 3.2.6 6 | Retrieve Precipitation Totals | 0 | Yes / No / NA | |||
)
3.2. Sample Test Plan
3.2.1. Introduction
<Enter project Background information>
Test Plan Identifier
This document is one of a series of documents as a part of Project <Project>. It defines the NTCIP Test Plan (NTP) for an NTCIP conformance test of a <Developer> ESS. This is the first version of the test plan. The complete document identifier is:
<Project>-<Developer>-NTP-1
Scope
This test plan has been developed to verify that the <Developer> ESS conforms to the NTCIP 1204 v04 standard per its completed Protocol Requirements List (PRL). The completed PRL is available as <Project>-<Developer>-PRL-1.
Successful completion of this test will result in <Action, such as payment, production of letter of completion, approval to move to next phase of project, etc>
This test plan covers the major features of an NTCIP ESS. This includes requirements related to:
- Data formats (e.g., the encoding of data over the communications channel),
- Data exchange procedures (e.g., the proper sequencing of data), and
- Related end user functionality (e.g., ensuring that the sign blanks when commanded to do so via the NTCIP interface).
This test plan does not provide an exhaustive test of all values or features of the NTCIP requirements; it only covers those features for which NTCIP 1204 v04 test procedures exist.
References
<Project Procurement Specification>
NTCIP 1201:2010v03.15r Global Object Definitions
NTCIP 1204 v04<precise revision> Environmental Sensor Station Interface Protocol NTCIP 2104:2003v01.11 Ethernet Subnetwork Profile NTCIP 2202:2001 v01.05 Internet Transport Profile
NTCIP 2301:2010 v02.19s Simple Transportation Management Framework Application Profile
3.2.2. Details of the NTCIP Test
Items to be Tested
The item to be tested is the NTCIP-related operation of an <Developer> ESS. The version and revision of the equipment to be tested shall be recorded in the test results.
The NTCIP standards will provide the basis for defining the correct operation. Any conflicts between the standardized test procedures and the remainder of the NTCIP standards text shall be resolved by assuming that the text in the body of the standard takes precedence over the text in the test procedure.
Features to be Tested
The following table lists all of the tests defined by NTCIP 1204 v04 and indicates whether they will be performed. The selection of tests is based on the Requirements to Test Case Traceability Table in Annex C of NTCIP 1204 v04. The test procedures are as implemented by <Test Tool> unless otherwise indicated.
Table 1: Test Cases to Perform
| Clause | Test Case Name | Perform |
|---|---|---|
| C.2.3.1.1 | ESS Characteristics | Yes |
| C.2.3.1.2 | Retrieve Compressed Station Metadata | No |
| C.2.3.1.3 | Retrieve ESS Door Status | No |
| Etc. | ||
Features Not to be Tested
Features that are not defined by NTCIP Standards or are not included within the standardized Test Procedures are not covered by this test plan. These features typically include, but are not limited to:
- Environmental operating requirements
- Construction and material requirements
- Power anomaly requirements
- Performance requirements
While some aspects of these features may be tested, this test plan does not focus on these types of requirements because they are not the focus of NTCIP 1204 v04. In addition, those tests not selected in the above table will not be performed.
Approach
The Tester will perform each selected test case identified under the "Features to be Tested" section of this test plan.
Inputs - Variable Values
The ESS Test Procedures include "variables" that are used as inputs to the test cases and must be assigned values prior to performing given tests. The specific values to be used for this test are provided in Appendix A.
Regression Testing
Any problem identified with the ESS shall be corrected by the Developer. Upon completion of the modifications, the Developer shall resubmit the component for another test. Partial tests may be performed during the resolution of identified anomalies, but a complete regression test shall be performed on the final submittal so that any unexpected impacts resulting from program modifications can be detected. The regression test shall include performing all tests included in this test plan.
Item Pass/Fail Criteria
In order to pass the test, the ESS shall pass all test procedures included in this test plan without demonstrating any characteristic that fails to meet NTCIP specifications.
Suspension Criteria and Resumption Requirements
The test may be suspended, at the convenience of test personnel, between the performances of any two test procedures. The test shall always resume at the start of a selected test procedure.
If any modifications are made to the ESS, a complete regression test shall be required in order to pass this test plan.
Test Deliverables
The Tester will ensure that the following documents are developed and entered into the configuration management system upon their completion of the final test:
- The NTCIP Test Plan (this document, including Appendix A)
- The Test Logs
- The Test Summary
- Any and all Test Incident Reports
All test documentation will be made available to the involved parties in a widely recognized computer file format such as Microsoft Word or Adobe Acrobat. In addition, the files from the test software shall be provided in their native file format as defined by the test software.
3.2.3. Test Management
Testing Tasks
The testing tasks are summarized in Table 2.
Table 2: Testing Tasks
| Task | Name | Predecessor | Responsibility | NTCIP Knowledge Level (1 = Low; 5 = High) |
|---|---|---|---|---|
| 1 | Finalize Test Plan | Specifications | Tester | 2 |
| 2 | Transmit Device to be tested to the test group | Implementation | Developer | 1 |
| 3 | Perform tests and produce test report documentation | 1, 2 | Tester | 5 |
| 4 | Resolve test incident reports | 3 | Developer, Tester | 4 |
| 5 | Repeat steps 2-4 until all test cases pass | 4 | N/A | N/A |
| 6 | Prepare the summary test report | 5 | Tester | 2 |
| 7 | Transmit all test documentation to all parties | 6 | Tester | 1 |
Environment/Infrastructure
All Test Cases covered by this test plan require the device under test to be connected to a test application as depicted in Figure 4.
Figure 4: Test Equipment
(Extended Text Description: Author's relevant description: Figure 4: This figure depicts the main testing environment figure from NTCIP 8007 where a test application is connected to a device under test via a communications cloud that is also connected to a data analyzer.)
The test application will be <Test Software>.
The test will be performed at <Test Facility Name> in <Test City>. All tests shall be over an <Link Technology>.
All tests shall be performed using an SNMP/UDP/IP communications stack. All tests shall be performed using the following settings unless otherwise defined in the specific test procedure.
Read Community Name: <read name>
Write Community Name: <write name>
Timeout Value: <timeout>
Responsibilities and Authority
The Agency shall be responsible for:
- Providing an environment to conduct the testing
The Tester shall be responsible for:
- Finalizing the test plan
- Providing the test tools
- Preparing the test environment
- Executing the tests according to the test plan
- Preparing the test results
- Delivering the test results
The Developer shall be responsible for:
- Providing the test items
- Assisting test personnel in connecting equipment, as needed
- Witnessing the performance of the tests (Optional)
- Checking the test results, and
- Resolving any areas of non-conformance identified
Staffing and Training Needs
The following staffing is expected for this test plan:
Agency - 1
Tester - 1
Developer - 1
The Tester must be familiar with how to use the test software and with the NTCIP standards.
Schedule, Estimates, and Cost
The Test Plan will be completed by <date> and delivered to the Developer for approval. Testing setup will begin approximately <date>. Testing will finish as soon as possible after that date with an expected completion no later than <date>.
Risks and Contingencies
The performance of the test suite is expected to take <number of days> days if there are no or few problems found. If the sign fails to pass the test, the anomalies will be reported to <Developer> and another test date will be scheduled.
3.2.4. General
Glossary
The following terms shall apply within the scope of this test plan.
ESS - Environmental Sensor Station, including the sensors, controller (sometimes called a remote processing unit), cabinet, and other associated field equipment.
Developer - The organization providing the equipment to be tested.
Tester - The organization that performs the testing according to the test procedures and interprets and records the results.
4. Reference to Other Standards
NTCIP 1204 v04 - National Transportation Communications for ITS Protocol: Environmental Sensor Stations Interface Protocol.
Glossary
To include additional descriptions/acronyms used primarily in the module. Listed out in alphabetical order.
| Term | Definition |
|---|---|
| Agency Specification | A document that has been prepared by an agency to define requirements for a subject item or process when procured by the agency. |
| Compliance | A condition that exists when an item meets all of the requirements of an agency specification. |
| Concept of Operations | A document that describes the purpose of a system project, including a description of the current and proposed system, as well as key user needs that the new system is required to address. |
| Conformance | A condition that exists when an item meets all of the mandatory requirements as defined by a standard. It can be measured on the standard as a whole, which means that it meets all mandatory (and applicable conditional) requirements of the standard or on a feature level (i.e., it conforms to feature X as defined in section X.X.X), which means that it meets all mandatory (and applicable conditional) requirements of the feature. |
| ESS | Environmental Sensor Station. |
5. Study Questions
To include the quiz/poll questions and answer choices as presented in the PowerPoint slide to allow students to either follow along with the recording or refer to the quiz at a later date in the supplement.
1. Which of the following most accurately describes a benefit of having standardized NTCIP test documentation included in NTCIP 1204 v04?
- Eliminates the need for customized test documentation
- Reduces the effort to prepare for testing
- Ensures that all devices conform to the standard
- Eliminates the need for additional tools to perform testing
2. Which statement most closely describes the documentation that a project should prepare before conducting NTCIP 1204 v04 testing?
- Just reference Annex C of NTCIP 1204 v04
- Develop a test plan with appropriate additions to link to NTCIP 1204 v04
- Develop a test plan and test design specification with appropriate additions to link to NTCIP 1204 v04
- Develop all documents defined by IEEE 829-2008
3. Which of the below is not included in a test plan?
- Identification of who will perform the testing
- Identification of which features will be tested
- Identification of the reason for the test
- Identification of the steps used to test the device
4. Which of the below is not a type of step used in NTCIP 1204 v04 testing?
- UPDATE
- SET
- VERIFY
- IF
6. Icon Guide
The following icons are used throughout the module to visually indicate the corresponding learning concept listed out below, and/or to highlight a specific point in the training material.
1) Background information: General knowledge that is available elsewhere and is outside the module being presented. This will be used primarily in the beginning of the slide set when reviewing information readers are expected to already know.
2) Tools/Applications: An industry-specific item a person would use to accomplish a specific task, and application of that tool to fit the need.
3) Remember: Used when referencing something already discussed in the module that is necessary to recount.
4) Refer to Student Supplement: Items or information that are further explained/detailed in the Student Supplement.
5) Example: Can be real-world (case study), hypothetical, a sample of a table, etc.
6) Checklist: Used to indicate a process that is being laid out sequentially.
