5.1 The Test Description proforma
The testing methodology used in the present document is specified in the oneM2M TS-0015 [6].
A Test Description (TD) is a well detailed description of a process that aims to test one or more functionalities of an implementation. Applying to interoperability testing, these testing objectives address the interoperable functionalities between two or more vendor implementations.
In order to ensure the correct execution of an interoperability test, the following information should be provided by the test description:
- The proper configuration of the vendor implementations.
- The availability of additional equipment (protocol monitors, functional equipment, ...) required to achieve the correct behaviour of the vendor implementations.
- The correct initial conditions.
- The correct sequence of the test events and test results.
In order to facilitate the specification of test cases an interoperability test description should include, at a minimum, the following fields as indicated table 5.1-1.
Table 5.1-1: Interoperability test description
| Identifier | A unique test description ID. |
|---|---|
| Objective | A concise summary of the test which should reflect the purpose of the test and enable readers to easily distinguish this test from any other test in the document. |
| References | A list of references to the base specification section(s), use case(s), requirement(s) and Test Purposes which are either used in the test or define the functionality being tested. |
| Applicability | A list of features and capabilities which are required to be supported by the SUT in order to execute this test (e.g. if this list contains an optional feature to be supported, then the test is optional). |
| Configuration or Architecture | A list of all required equipment for testing and possibly also including a reference to an illustration of a test architecture or test configuration. |
| Pre-Test Conditions | A list of test specific pre-conditions that need to be met by the SUT including information about equipment configuration, i.e. precise description of the initial state of the SUT required to start executing the test sequence. |
| Test Sequence | An ordered list of equipment operation and observations. The test sequence may also contain the conformance checks as part of the observations. |
The test descriptions are provided in proforma tables. In order to ensure the correct execution of an interoperability test, the following information is provided in the test description:
- The configuration applied for the test.
- The need of additional equipment (protocol monitors, functional equipment, etc.) required to achieve the correct behaviour of the implementations.
- The initial conditions.
- The sequence of the test events and test results.
The following different types of test operator actions are considered during the test execution:
- Astimulus corresponds to an event that enforces a DUT to proceed with a specific protocol action, such as sending a message.
- A configure corresponds to an action to modify the DUT configuration.
- An IOP check consists of observing that one DUT behaves as described in the standard: i.e. resource creation, update, deletion, etc. For each IOP check in the Test Sequence, a result can be recorded. The overall IOP Verdict will be considered OK if all the IOP checks in the sequence are OK.
- In the context of Interoperability Testing with Conformance Checks, an additional step type, PRO checks can be used to verify the appropriate sequence and contents of protocol messages, this is helpful for debugging purposes. PRO Verdict will be PASS if all the PRO checks are PASS.