This chapter is intended to provide an easy -- but not necessarily trivial -- introduction to TTCN for the beginner.
ISO/IEC 9646 (ITU X.290 series) is a five-part standard which defines a framework and methodology for conformance testing of implementations of OSI and ITU protocols. The test notation, the Tree and Tabular Combined Notation (TTCN), is the third part of this standard, i.e. ISO/IEC 9646-3.
The use of TTCN is increasing, and as the notation has now become an ISO International Standard and a ITU Recommendation, we believe there exists a need for a guideline on TTCN.
We do not describe the TTCN in the same order that it is presented in the standard, but instead have used a bottom-up approach. We begin by introducing some basic TTCN features developed round a simple example. Additional features are introduced as required. We have tried to concentrate on aspects that have been introduced in the IS version of the notation, especially concerning the use of ASN.1. We also address the concurrent TTCN.
We hope that readers will find this approach instructive. The guidelines intends to provide:
Among other things, parts 1 and 2 of ISO/IEC 9646 define the basic concepts and abstract methods that are the cornerstone of standardized conformance testing. It is beyond the scope of this guideline to examine these concepts in detail, but a knowledge of the terms and concepts described below is helpful to understanding the TTCN.
One of the basic premises of the conformance standard is that the implementation of the protocol, called an implementation under test (IUT) is a black box.
Any conclusions that we may draw about the conformance of that IUT will be made by observing and controlling the events that occur at the lower and upper service interfaces of the IUT. In ISO/IEC 9646 terms these interactions occur at points of control and observation (PCO) and are expressed in terms of protocol data units (PDUs) embedded in abstract service primitives (ASPs).
An IUT is tested by a test system. In TTCN the different parts of the test system are called test components.
In simple terms, the test components which communicate with the IUT via the PCOs at the lower interface are collectively called the lower tester (LT). The test components which communicate with the IUT via the PCOs at the upper interface are collectively called the upper tester (UT). There must be at least one test component always present in the test system. This is called the master test component (MTC) and it is responsible for coordinating and controlling the test and for setting the final verdict of the test.
Communication between test components in the LT is achieved via coordination points (CP). Similarly, UT test components may communicate with each other via CPs.
Coordination between the LT and the UT is achieved by test coordination procedures (TCP).
Figure 1 : Generalized parallel test architecture (illustrated for a single-layer implementation)
|
|
The lower tester is the more complex of the two components as it is also responsible for the control and observation of the protocol data units (PDUs) embedded in the ASPs that it sends and receives. In fact, at any given time the LT, when executing a test case, is implementing a portion of the relevant protocol.
In order to test the IUT we need to specify the sequences of interactions, or test events, that we wish the test system to control and observe. A sequence of such events that specify a complete test purpose is called a test case. A set of test cases for a particular protocol is called a test suite.
The TTCN is a notation that has been developed for the specification of test cases at a level that is abstracted from the architecture of any real test system that these test cases may eventually be run on.
The abstract test cases contain all the information that is necessary to fully specify the test purpose in terms of the protocol that the IUT is supposed to implement. It does not include test system specific information. However, this does not mean to imply that the notation itself is abstract - during the last few years the definition of TTCN has become very precise, with regard to both syntax and operational semantics, and is now close to a programming language.
The main body of ISO/IEC 9646-3 defines the graphical form of the notation (TTCN-GR), where all information is presented using tables. There is also an underlying machine processable format (TTCN-MP) specified in an extended form of BNF (Backus-Naur Form). In this guideline we shall concentrate on the TTCN-GR.
From Figure 1 it can be seen that, generally speaking, the ISO conformance standard requires that tests are specified in terms of (N-1)-layer ASPs, (N)-layer ASPs and (N)-layer PDUs. In order to fulfil these requirements the minimum functionality that the TTCN should provide is:
In order to do this the TTCN allows:
We shall examine in detail how these, and other, features are supported in TTCN to make it a powerful notation for specifying abstract test cases.
For the purposes of this guideline we shall invent a simple case study for an imaginary protocol, which we shall call the X-Protocol. The case study is based on the architecture introduced in the previous section. The IUT is an implementation of the X-protocol.
We shall assume that there is an underlying service provider that provides a network service (N), over which we shall run the test. This leads to the following:
Our examples will introduce the TTCN features necessary to specify the simple scenario described below:
The case study has two test purposes, these can be stated as:
Each test purpose will be expressed as a separate test case.
We shall create a TTCN complete mini-test suite that contains all the TTCN necessary to specify the above test cases. The following table (this is not a TTCN table!) shows the main sections of this example in the order in which they would appear in a real test suite. The complete study can be found in The Complete Case Study. The right-hand column of the table tells you where these sections are described in this guideline:
Let us start by specifying the test component configurations needed in the test suite. The conformance standard defines various abstract test methods. For the purposes of this guideline we shall assume that the IUT is a single-layer implementation and that we are testing with the distributed method. Also, we are testing in a multi-party context because our second test purpose requires more than one connection.
In this case, the architecture of Figure 1 says that we need:
This is illustrated in the following figure:
Figure 2 : Illustration of the multi-party distributed test method
|
|
In concurrent TTCN this architecture is specified using the following tables:
Case study 1: This table lists all the test components that may be used in the test suite. They can be thought of as building-blocks that can be used to construct different configurations. A test component may have the role of main test component (MTC), or parallel test component (PTC).
Figure 3 : Test Component Declarations
|
|
Case study 2: This table shows the configuration for the single-connection test case. In any one configuration there should never be more than one MTC.
Figure 4 : Test Component Configuration Declaration (SINGLE_PARTY)
|
|
Case study 3: This table shows the configuration for the multi-connection test case.
Figure 5 : Test Component Configuration (MULTI_PARTY)
|
|
Before we continue with the declaration of the test case let us now look at how TTCN describes the behaviour of the various test components. Many standardized service definitions and protocol specifications use state diagrams and/or state tables to describe the behaviour of the service or protocol.
Test cases are derived from these specifications. However, because conformance testing is concerned with observing and controlling sequences of interactions at service interfaces it is more appropriate that we specify test system behaviour as a tree which has branches for all the possible sequences of interactions that may occur between any two given protocol states.
In TTCN a tree of interactions is called a behaviour tree. The tree structure is represented by using increasing levels of indentation to indicate progression into the tree with respect to time.
Figure 6
: A tree is represented in TTCN using indentation
Note that an absolute level of indentation does not necessarily mean that nodes are siblings. For example, although the nodes F and G are numerically at the same level of indentation (i.e. 3) as I and J, the nodes F and G are in one branch of the tree and the nodes I and J are in another
|
|
A node in a behaviour tree is called a behaviour line. A behaviour line consists of the following components:
Exactly which components of the behaviour line are used at a specific time varies. For example, line numbers and comments are always optional and constraint references and verdicts shall only be used when required.
Behaviour lines are specified in dynamic behaviour tables. There are three kinds of behaviour tables, each consisting of a header and a body.
The visual difference between these three tables is in the header. The format of the body is the same for all three. However, there is a significant difference, which will be explained later, in how these different tables are used.
Figure 7
: The body of a dynamic behavior table
The table body shows the columns for the line numbers, labels, statement lines, constraint references, verdicts and comments. The light shading indicates the extent of a single behaviour line. The dark shading indicates the extent of a single statement line.
|
|
A sequence of one or more statements, together with the indentation information, in a single behaviour line is called a statement line. Statement lines appear in the behaviour description column of dynamic behaviour tables.
The behaviour of the test system, such as sending and receiving ASPs, is expressed using TTCN statements. Statements can be split into three distinct types:
Some statements will be successful, i.e. match, depending on the occurrence of certain events. There are two types of event: input events and timer events. An input event is the arrival of an ASP at a named PCO or a message at a named CP. A timer event is the expiry of a protocol timer. The TTCN statements that are events are:
Some statements will always be successful, i.e. execute. We shall call such statements actions, although this term is not used in ISO/IEC 9646-3. These are actions that are executed by the test system, and TTCN assumes that they can always be executed successfully. The TTCN statements that are actions are:
Statement lines may include a qualifier statement, i.e. a boolean expression. We call such statement lines qualified statement lines. No event can match, nor can any action be executed unless the qualifier included in the statement line evaluates to TRUE. An unqualified statement line is one that does not include a qualifier.
A TTCN qualifier is simply a:
The actual combinations of events, actions and qualifiers that are allowed are defined by the TTCN-MP. The different combinations will be described at the relevant points in this guideline.
We shall now consider how a behaviour tree is traversed and executed.
A set of statement lines at the same level of indentation, and in the same branch of the tree are called a set of alternative statement lines, or alternatives for short. Thus, in the Figure 6 (A,B), (C, D, E), (F, G), (I, J) and (H) are all different sets of alternatives.
Because the ordering within any given set of alternatives is significant it is important that all events and qualified statements appear before any unqualified actions.
Execution starts at the root of the tree. That is, the first set of alternatives is repeatedly looped with each alternative being evaluated in the order of its appearance in the set. This looping continues until a statement line is successfully executed or matched. If a statement line is successful then the next set of alternatives (if any) is entered, and the process is repeated.
Execution stops when a leaf of the tree is reached. A final verdict will also halt execution, see Verdicts.
In the example shown in Figure 7 execution starts by looping through the first set of alternatives (A, B). If B is successful then execution terminates. If A is successful then the next set of alternatives (C, D, E) is entered. Let's assume that the statement line E is successful: then the next set of alternatives is (I, J). If either I or J is successful then execution terminates. Note that if no statement line in any set of alternatives is ever successful then execution gets `stuck' as we repeatedly loop through those alternatives.
Figure 8 : Cycle of execution of a test case behavior tree
|
|
We shall now discuss TTCN data types and their values. These are used to specify the data types, including ASPs and PDUs that are used in the behaviour descriptions.
TTCN has been tailored to interface with the Abstract Syntax Notation One (ASN.1, ISO/IEC 8824:1990). There is no clear boundary between TTCN types and ASN.1 types; the distinction is an artificial one. It is there, however, to allow test suite specifiers to build the types, ASPs, PDUs, etc. they need without using ASN.1 if they do not wish to do so. This is relevant, for example, in lower-layer protocols, where ASN.1 is not normally used in the protocol specifications.
TTCN contains a number of predefined types. It also allows the user to construct his own types from the predefined types. This may be done using the following tables:
TTCN supports a rich set of predefined (built-in) types. The predefined TTCN types, with the exception of HEXSTRING, are a subset of the ASN.1 built-in types and are compatible with their ASN.1 counterparts. The HEXSTRING type does not exist in ASN.1. The remaining ASN.1 built-in types may also be used without being explicitly defined.
Figure 9 : List of predefined types that may be used in TTCN Test Suites
|
|
The value denotation for the predefined types is the same in both TTCN and ASN.1, see ASN.1 Types and Values.
The TTCN user can construct other types based on the simple predefined types, without the need to resort to ASN.1 syntax. These subtypes are defined in the Simple Type Definitions table, and they may be used anywhere in the test suite. They are constructed by restricting the predefined types (and possibly previously declared subtypes) by specifying:
|
Note: The TTCN syntax does allow the use of ASN.1 in the Simple Type Definitions table if wished. We recommend, however, that the special tables for ASN.1 types be used instead. |
Figure 10 : Some miscellaneous simple type definitions
|
|
Case study 4: Definition of a user type for test case results.
Figure 11 : Simple Type Definitions
|
|
TTCN has tables for the declaration of structured (i.e., complex) types. These types, like the predefined types and the simple types may be used anywhere (that is why they are defined early on in the test suite). However, their main use is to substructure ASPs and PDUs and we shall therefore discuss them in Specifying ASP, PDU and CM Values.
ASN.1 is used to describe data types. It is often used as a means for defining the PDU structures of many OSI protocols. In the following we shall give an overview of the ASN.1 type and value notation.
A type reference is the "name" of an ASN.1 type constructed by the ASN.1 user. The ASN.1 standard requires that the initial character of a type reference is always an upper case letter.
A value reference is the "name" of an ASN.1 value constructed by the ASN.1 user. The ASN.1 standard requires that the initial character of a value reference is always a lower case letter.
|
Note: Many test suite specifiers ignore this convention, indeed, so does ISO/IEC 9646-3! |
As you will see shortly, type references in SETs and SEQUENCEs etc. may be "labeled". Such labels are called identifiers. The ASN.1 standard requires that the initial character of an identifier is always a lower case letter. Value references and identifiers are distinguished by context.
The ASN.1 standard allows the character dash (-) in identifiers. TTCN does not (otherwise there would be ambiguity in arithmetic expressions between dash and minus). TTCN uses underscore (_) instead. If ASN.1 definitions are imported, copied, borrowed etc. from external specifications (e.g. the PDU definitions for a particular protocol) then all occurrences of dash in identifiers should be changed to underscore.
BOOLEAN is a type denoted by two distinguished values: TRUE and FALSE. The TTCN has operators for values of any type whose base type is BOOLEAN.
INTEGER is a type denoted by the distinguished values which are the positive and negative whole numbers, including zero. TTCN has operators for any values whose base type is INTEGER.
REAL is a data type specified as a triple of three INTEGERS consisting of:
The base is limited to 2 or 10.
TTCN does not have any operators for values of REAL types. If REAL arithmetic is absolutely necessary this can be achieved by user defined operations.
BIT STRING is a type whose distinguished values are the ordered sequences of zero, one, or more BITs. Individual BITs in the BIT STRING may be named.
OCTET STRING is a type whose distinguished values are the ordered sequences of zero or an even number of HEX digits, each digit corresponding to an ordered sequence of four bits. Individual OCTETSs in the OCTET STRING may be named.
|
Note: There is an incompatibility between TTCN and ASN.1 over the value denotation of OCTET STRING. TTCN terminates OCTET STRING values with the keyword `O' rather than the keyword `H'. |
A variety of character sets are supported. For the purposes of this guideline we shall restrict ourselves to using the ITU character set IA5String.
ENUMERATED types represent the complete set of values (domain) that an instance of a data type may take.
No TTCN operators can be applied to values of ENUMERATED type.
OBJECT IDENTIFIER denotes a named object as a sequence of non-negative INTEGERs. The naming hierarchy of specific objects is decided by the relevant authority (e.g. ISO or ITU). An OBJECT IDENTIFIER specifies a unique path in this hierarchy, i.e. all objects are uniquely named.
The OBJECT IDENTIFIER for ISO/IEC 9646-3 would be 1.0.9646.3
Objects that TTCN needs to reference in this manner might be ASN.1 modules (PDUs etc.) and PICS, PXIT documents.
OBJECT DESCRIPTOR denotes a text string that references an object. The difference between an OBJECT IDENTIFIER and an OBJECT DESCRIPTOR is that the former is unique, while the latter may not be.
Complex data types can be built from the simple predefined types (excepting HEXSTRING) using ASN.1 constructors. This process is recursive, i.e. constructors can use other constructors (including themselves) to an arbitrary level of nesting. The constructor types are:
SEQUENCE is a data type denoting an ordered set of elements. This set may be empty. The elements of this set may be of any ASN.1 type and may be of different types. The elements in the set may be named.
SEQUENCE OF is a data type denoting an ordered set of elements. This set may be empty. The elements of this set may be of any ASN.1 type but they shall all be of the same type. The elements in the set may be named.
SET is a data type denoting an unordered set of elements. This set may be empty. The elements of this set may be of any ASN.1 type and may be of different types. The elements in the set may be named.
SET OF is a data type denoting an unordered set of elements. This set may be empty. The elements of this set may be of any ASN.1 type but shall all be of the same type. The elements in the set may be named.
At first glance SEQUENCE and SET may appear the same. The difference is that a SEQUENCE is ordered, a SET is not. For instance, in the previous examples when the SET is eventually encoded it can be transmitted by sending field2 before field1, if wished. In the case of SEQUENCE field1 must always precede field2. This has implications when testing, which will be discussed later.
The same applies to SEQUENCE OF and SET OF.
In short, SEQUENCE and SEQUENCE OF rely on ordering to avoid ambiguity. SET and SET OF rely on the data type and/or tag of each element to uniquely distinguish each element.
The keyword OPTIONAL in a SEQUENCE or SET indicates that the presence of that element in the SEQUENCE is not mandatory and may be included or omitted at will.
The same applies for SEQUENCE OF, SET and SET OF.
In certain cases it is useful to be able to specify a DEFAULT value to be used (in encoding) if the element is not present in the data type.
A CHOICE type is a data type that defines the union of one or more data types. The alternatives in this union may be named. Any given instance, i.e. value, of a CHOICE shall be exactly one of the alternatives of the CHOICE. This has implications for testing which will be discussed later.
Tags are used to distinguish between different occurrences of the same type. Tags are denoted by a non-negative INTEGER enclosed in square brackets. Tags are included in the encoding of the data type. There are four classes of tags:
The keyword IMPLICIT may be used together with the definition of the tagged type. IMPLICIT is an instruction to an ASN.1 encoder that only the tag need be encoded and thus transmitted over the network. This is done to reduce the amount of transmitted data. IMPLICIT may only be used where no loss of essential information would occur. For example, it should not be used with CHOICE.
Use of the EXTERNAL type is not allowed in TTCN.
The TTCN supports an asynchronous communication model. Communication between the test components and the IUT or service provider is achieved via points of control and observation (PCOs). Communication between the test components themselves is achieved via coordination points (CPs).
We shall use the same queue model to describe both PCOs and CPs:
The SEND action appends an ASP to the relevant PCO send queue. Even in the case of the IUT and the underlying service provider the send queue is considered to be unbounded, and that the IUT or service provider will always accept the ASPs sent by an LT or UT.
A successful RECEIVE pops the ASP from the top of the RECEIVE queue. We shall see later that RECEIVE involves two steps:
Figure 12 : Illustration of the PCO and CP queue model
|
|
All PCO types that are used in the test suite must be declared in the
Case study 5: Declaration of the PCO Types N_SAP and X_SAP.
Figure 13 : PCO Type Declarations
|
|
All PCOs that are used in the test suite must be declared in the
Case study 5: Declaration of the PCOs L and U.
|
|
If the test suite only uses one PCO it is allowed to omit the PCO name in the TTCN statements that use them. If there is more than one PCO used (e.g. as in the distributed method) then the PCO and CP names (if any) must appear in the TTCN statements that use PCOs or CPs.
We have already described how a behaviour tree is executed by repeatedly looping through a set of alternatives until a statement line is successful. At the beginning of each loop a snapshot is taken of each input queue in every PCO or CP. Statements are evaluated on the basis of the state of the snapshots, not on the actual state of the PCO or CP queues. This has the effect of freezing time while a set of alternatives is being processed i.e. "prevents" the occurrence of an event in between snapshots. This means that the arrival of an ASP, PDU or CM during processing of a set of alternatives is not registered until the snapshots are updated.
All CPs that are used in the test suite must be declared. This is done in the CP Declarations table. Each PCO requires the following information:
Case study 6: Declaration of the coordination points called CP1 and CP2.
Figure 15 : Test Component Declarations
|
|
The transmission of ASPs and/or PDUs to the IUT or messages to other test system components is one of the fundamental actions in a typical TTCN behaviour tree.
The SEND statement allows the test suite specifier to express that an ASP of a certain type is to be transmitted over a named PCO. The SEND statement is denoted by:
The SEND statement may be qualified and it may be followed by an ASSIGNMENT_LIST and/or TIMER_OPERATION. The order in which these statements may appear in the statement line is fixed, as shown below; the square brackets indicate that the presence of the statement in the statement line is optional:
The numbers on the line above indicate the order, with respect to time, in which the statements should be executed: the QUALIFIER (if any) is evaluated first. If it evaluates to FALSE processing stops and the statement line is not successful. If it evaluates to TRUE then the ASSIGNMENT_LIST (if any) is executed. Only then can the SEND statement be executed. Finally, the TIMER_OPERATION (if any) is executed.
Normally PDUs are embedded in ASPs, and will not be explicitly named in the SEND statement. However, not all protocols have a service definition (e.g. X.25) and TTCN therefore permits the SEND statement to be used explicitly with PDUs instead of ASPs. The action of sending a PDU is denoted by:
Other statements that may be associated with sending a message, and the order in which the statement line is processed, is exactly the same as for an ordinary SEND statement line.
The SEND statement is also used to send messages to coordination points. The action of sending a CM is denoted by:
Other statements that may be associated with sending a message, and the order in which the statement line is processed, is exactly the same as for an ordinary SEND statement line.
The receipt of ASPs and/or PDUs from the IUT or messages from other test system components is one of the fundamental events in a typical TTCN behaviour tree.
The RECEIVE statement allows the test suite specifier to express that an ASP of a certain type is to be received over a named PCO. The RECEIVE statement is denoted by:
The RECEIVE statement may be qualified and it may be followed by an ASSIGNMENT_LIST and/or TIMER_OPERATION. The order in which these statements appear in the statement line is fixed, as shown below; the square brackets indicate that the presence of the statement in the statement line is optional:
The numbers on the line above indicate the order, with respect to time, in which the statements should be executed: the RECEIVE is evaluated first, and succeeds if an ASP of the correct type is at the head of the PCO queue. If the RECEIVE fails then processing stops and the statement line is not successful. If the RECEIVE is successful then the QUALIFIER (if any) is evaluated. If the QUALIFIER evaluates to FALSE processing stops and the statement line is not successful. If it evaluates to TRUE then the ASSIGNMENT_LIST (if any) is executed. Finally, the TIMER_OPERATION (if any) is executed. For example:
Normally PDUs are embedded in ASPs, and will not be explicitly named in the RECEIVE statement. However, not all protocols have a service definition (e.g. X.25) and TTCN therefore permits the RECEIVE statement to be used explicitly with PDUs instead of ASPs. Receipt of a PDU is denoted by:
Other statements that may be associated with sending a message, and the order in which the statement line is processed, is exactly the same as for an ordinary RECEIVE statement line.
The RECEIVE statement is also used to accept messages from coordination points. Receipt of a CM is denoted by:
Other statements that may be associated with receiving a message, and the order in which the statement line is processed, is exactly the same as for an ordinary RECEIVE statement line.
The OTHERWISE statement allows the test suite specifier to express that an ASP or PDU of any type is to be received over a named PCO. Note that this includes objects that may not normally be recognized as proper ASPs or PDUs, due to the fact that the IUT may not be working correctly, i.e. OTHERWISE is a catch-all. The OTHERWISE statement is denoted by:
OTHERWISE should not be used at coordination points.
|
Note: Always have an OTHERWISE as an alternative to a RECEIVE event. |
ASPs are derived from the relevant standardized service definitions. When using the distributed method, for example, ASP definitions are needed for both the (N) and the (N-1) service. There should be one ASP definition for each ASP used in the test suite.
PDUs are derived from the relevant protocol specifications. There should be one PDU type definition for each PDU used in the test suite. If, for the purposes of testing, it is required to use non-standard PDUs then these too should also be defined in the test suite.
Coordination Messages are also defined by the test suite specifier.
TTCN has tables that allow the definition of ASPs, PDUs and CMs using either the simple TTCN tabular format or ASN.1.
TTCN has tables for the declaration of the following complex types:
Using these complex types we can define arbitrarily structured ASPs and PDUs (structured types are substructures of ASPs and PDUs). In essence there is no real difference in TTCN between the composition of the body of an ASP, PDU or structured type. Note the following:
Normally, the ASP parameters, PDU fields and structure elements will be predefined or simple types (note that this includes the use of ASN.1, if wished). However, as noted above, the parameter, field or element types may also be PDUs or structures to allow the chaining of these types to build complex definitions.
Figure 16 : Structuring ASPs and PDUs
|
|
In ASN.1, constructors such as SEQUENCE and SET are used to build arbitrarily complex types. ASN.1 definitions may be used in the following tables:
It is always possible to express the TTCN tabular format in ASN.1, but not vice versa. The two formats can be used in combination, if wished. A common example is to use a tabular ASP to carry a structured PDU defined in ASN.1.
Figure 17 : Structuring ASPs and PDUs using ASN.1
|
|
There will always be at least one type definition in an ASN.1 table. This is the main definition, and it is named in the table header. However, these ASN.1 tables may also include any number of local definitions which are only available to the type definitions defined within the table itself, i.e. the main definition and other local definitions (if any).
Note that local definitions begin with typereference ::=. This is not the case for the main definition as the type identifier already appears in the header.
In order to save repeating PDU and other type definitions that are specified in another standard TTCN allows the following types to be declared by reference rather than explicitly:
A single table is used for all references to a particular type. The reference tables are:
Note that because the entries in the Type Reference column and the Module Identifier column follow ASN.1 syntax they may contain the dash character. Note also that the module identifier may be followed by an optional object identifier.
Figure 18 : Example of PDU definitions by reference
|
|
OSI service primitives are often defined in a standard as a tuple, i.e. the primitive name followed by a list of parameters. Each parameter is defined using natural language descriptions and may represent service control information or service user data. Some parameters are mandatory (i.e. must always be present) while others are optional and, under certain circumstances, may be omitted.
In TTCN service primitives are called Abstract Service Primitives (ASPs) and are declared in ASP Type Definition tables.
Case study 7: A service provider ASP type definition.
Figure 19 : ASP Type Definition (N_DATArequest
|
|
The above example uses the PDU metatype. This indicates that any type of PDU, and not just a particular type of PDU may be embedded in this ASP.
In most OSI standards PDUs are usually defined using either:
In the first case the specification may be rather loose, and typing of PDU fields and substructuring of the PDUs is not always obvious. The test suite specifier must transpose these definitions to the more powerful and precise formats available in TTCN.
For example, a standard may describe a particular field as comprising 8-bits, implying that it shall be encoded as a BITSTRING. If none of the bits in this BITSTRING need to be referenced individually, it may be adequate for testing purposes (and easier to understand) if this field is defined as an OCTETSTRING.
In the case of ASN.1 the types and structure of PDUs and their fields is usually complete and well-defined, and may be taken directly from the protocol standard, either by copying them or by reference.
Case study 8: Type definition of an X_PDU using the TTCN format.
Figure 20 : PDU Type Definition (CR_PDU)
|
|
Figure 21 : Type definition of the same X_PDU using ASN.1
|
|
TTCN structured types (which we will sometimes refer to simply as structures) are only used to substructure ASPs, PDUs, CMs and other structured types.
If ASN.1 is used then the ASN.1 type definition table may be used not only to substructure ASPs, PDUs etc. but also to define types general to the entire test suite.
Case study 9: Type definition of a PDU substructure using the TTCN format.
Figure 22 : Structure Type Definition (VARIABLE_PART)
|
|
Figure 23 : Type definition of the same substructure using ASN.1
|
|
Coordination messages are special to each test suite and are created by the test suite specifier. Either the tabular format or ASN.1 may be used.
Case study 10: Type definition of a coordination message.
Figure 24 : CM Type Definition (PTC_RESULT)
|
|
Figure 25 : Type definition of the same coordination message using ASN.1
|
|
In this section we will expand our example a bit further and show how the sequencing of ASPs w.r.t. time is expressed in TTCN behaviour descriptions.
Figure 26 : An N_DATArequest, carrying a CR_PDU, is sent over the network
It results in the IUT generating an X_CONNECTindication, which is responded to by the UT tester sending an X_CONNECTresponse. This results in an N_DATAindication, carrying a CC_PDU, appearing at the LT.
|
|
Figure 27 : The above scenario can be expressed as two TTCN behavior trees
Note the introduction of the OTHERWISE statement.
|
|
All values in TTCN are quite simply expressions. Note that the TTCN syntax allows the operands of expressions to be:
Exactly which variables etc. may be used in an expression depends on the context in which the expression is used. This aspect will be discussed in the relevant sections.
TTCN supports the following kinds of operators for use in expressions:
TTCN supports the following arithmetic operators for use only with operands of INTEGER type or derivations of INTEGER type:
Expressions that use these operators are called arithmetic expressions.
The equal to and not equal to operators may be used on values of any type:
Expressions that use these operators must always evaluate to a BOOLEAN value:
TTCN supports the following relational operators for use only with operands of INTEGER type or derivations of INTEGER type:
Expressions that use these operators must always evaluate to a BOOLEAN value:
TTCN supports the following logical operators for use only with operands of BOOLEAN type or derivations of BOOLEAN type:
Expressions that use these operators must always evaluate to a BOOLEAN value:
A qualifier is an expression enclosed in square brackets:
The expression must evaluate to a BOOLEAN value.
A TTCN statement may be an ASSIGNMENT_LIST, i.e. a list of assignments, separated by commas and enclosed in parentheses:
The left-hand side (l.h.s.) of an assignment must resolve to a variable. In the context of SEND and RECEIVE statements the l.h.s. of an assignment may resolve to an ASP parameter reference, a PDU field reference or a structure element reference. The right-hand side (r.h.s.) of the assignment is an expression, which must evaluate to a value of a type compatible with the type of the l.h.s.
|
Note: By type compatibility we quite simply mean that a value, a, of type A is type compatible with type B if a is a legal value of both type A and type B. |
TTCN supports both a number of predefined operations and a mechanism that allows the definition of user operations. Operations may be used as operands in expressions.
TTCN now supports a number of predefined operations. More are expected to be added by the work on TTCN extensions. Currently these operations are:
TTCN allows the informal definition of user define operations. A possible approach is to use a programing language to `describe' the operation.
|
Note: The TTCN amendment (PDAM 2) is currently exploring ways of how user operation descriptions can be made more `formal'. |
Like the predefined operations user defined operations may be used in both behaviour trees and as `values' in constraints.
Each user defined operations is declared in a Test Suite Operations table.
Case study 11: Definition of a user defined operation.
Figure 28 : Test Suite Operation Definition
|
|
If an operator does not have any arguments it should be called with an empty actual parameter list, e.g. DATE ( ).
The previous section showed how to define structured ASP, PDU and CM types. However, when a tester SENDs or RECEIVES an ASP, PDU or CM it is necessary to specify in detail actual values of these complex types.
Values, or instances, of complete ASPs, PDUs and CMs are called constraints. For each ASP, PDU or CM definition table there should be at least one corresponding constraint table.
The constraint declaration tables are:
ASPs, PDUs and structured types may be chained to allow the construction of arbitrarily complex ASPs and PDUs. Static chaining means that the actual name of a PDU constraint or structure constraint appears as the value of an ASP parameter, PDU field or structure element, i.e. the structure is hardwired by symbolic references. Dynamic chaining means that the linking occurs when the actual constraint is passed as a parameter in the constraints reference.
Figure 29 : Relation between structured ASP and PDU types and their constraints
|
|
The concept of chaining is an integral part of ASN.1, although it is not described in those terms. It is expressed by the use of type references. If a reference is made from one type definition to another then there should be a corresponding value for that reference in the relevant constraints.
Generally, for every ASP type definition there will be at least one ASP constraint declaration. However, some service definitions include ASPs that do not have parameters. In such cases a constraint is not necessary. The same may apply to coordination messages, but it does not apply to PDUs. A PDU without fields is a nonsense.
ASP constraints are very similar to PDU constraints, which are more fully described in the next section. The rules that apply to PDU constraints, also apply to ASP constraints.
Case study 12: A typical ASP constraint. Note that the constraint is parameterized - more about that later.
Figure 30 : ASP Constraint Declaration (NDr)
|
|
Generally, for each field in the PDU type definition, there will be a corresponding field in the constraint. The value of the constraint field must be compatible with the type definition for that field. We shall see later how fields may be omitted or replaced, and how the derivation path entry should be used. We shall also see how constraint values are matched.
Case study 13: Declaration of an X_PDU constraint using the TTCN tabular format.
Figure 31 : PDU Constraint Declaration (CR1)
|
|
Figure 32 : Declaration of the same constraint using ASN.1
|
|
Constraints on structured type definitions and ASN.1 type definitions are constructed in the same way as ASPs and PDUs. Just as the type definitions may be used by both ASP and/or PDU type definitions, so also may the constraints.
When the TTCN tabular format is used the structure of the constraints shall be the same as the structure of the type definitions. That is, if a PDU field is defined as being of structure type then there will be one constraint for the PDU and one for the structure.
This rule is relaxed in ASN.1. The structure must be compatible but there need not necessarily be a one-to-one correspondence between the type tables and the constraint tables.
Case study 14: Declaration of a structured type constraint using TTCN tabular format.
Figure 33 : Structured Type Constraint Declaration (variable_part_CR1)
|
|
Figure 34 : Declaration of the same substructure constraint using ASN.1
|
|
Coordination message constraints are also similar to PDU constraints.
Case study 15: Declaration of a CM constraint using TTCN tabular format
Figure 35 : CM Constraint Declaration (PTC_RES)
|
|
Figure 36 : Declaration of the same CM constraint using ASN (we shall assume
that RESULT_TYPE is an ASN.1 ENUMERATED type)
|
|
The TTCN SEND and RECEIVE statements indicate only which ASP or PDU type is to be transmitted or received. The constraints column in dynamic behaviour tables is used to state exactly which ASP or PDU value is to be sent, or is expected to be received. In other words, each SEND or RECEIVE statement must be accompanied by a constraints reference.
|
Note: This rule can be relaxed for parameterless ASPs. |
Figure 37 : Using constraint references in behavior lines
|
|
Constraints can be parameterized. That is, a constraint name may be followed by an optional formal parameter list. The formal parameters can be used in the value column of the constraint.
Figure 38 : A parameterized constraint
|
|
The actual parameters are passed to the constraint when it is invoked from the constraints column in a behaviour description.
Figure 39 : Invocation of a parameterized constraint
|
|
The actual parameter must always resolve to a specific value. In a SEND constraint this is the value that will eventually be encoded and transmitted.
In a RECEIVE constraint the actual parameter resolves to the value that will be matched against the received value. No binding occurs, i.e. the received value is not bound to the actual parameter. If it wished to capture received values, then this should be done by explicit assignment statements in the behaviour descriptions.
A common use of parameterized constraints is to link ASPs, PDUs and structures dynamically rather than statically, as we have described earlier. The linking occurs when the actual constraint is passed as a parameter in the constraints reference.
Figure 40 : Dynamic chaining of a parameterized PDU in an ASP
An N_DATArequest is used to carry the DT_PDU of the previous example.
|
|
The rules for sending a constraint are not the same as those for receiving one. We shall examine each of these aspects in turn.
A constraint in the context of SEND specifies the values that will eventually be transmitted over the network (at this point in time we will ignore encoding issues). In TTCN this transmitted object is called the Send Object which is built from information in the relevant constraint. Note that assignments may override values derived from the constraint in the Send Object, which is why BUILD occurs before ASSIGNMENT_LIST.
In the context of SEND we shall use the term received constraint value to mean the value of an ASP parameter, PDU field or CM field of the ASP, PDU or CM constraint that the test specifier wishes to transmit. The type of the constraint value is defined in the relevant ASP, PDU or CM definition.
Constraint values for Send Objects should always be fully specified at the time of transmission of the object.
Figure 41 : Execution of an alternative that contains SEND
|
|
The receipt of an ASP, PDU or CM is more complex than simply receiving an ASP, PDU or CM of the correct type. Testing often requires that the composition of the ASP, PDU or CM is checked in detail. This is achieved in TTCN by specifying a constraint that the ASP, PDU or CM is expected to match. The RECEIVE event can be considered successful only if all the conditions stipulated in the constraint are satisfied. We shall, therefore, extend our description of the RECEIVE statement line of The RECEIVE Statement to be:
TTCN uses the term Received Object to mean the ASP, PDU or CM that is currently at the top of the relevant incoming PCO or CP queue, and is being checked during evaluation of a RECEIVE statement.
In the context of RECEIVE we shall use the term received constraint value to mean the value of an ASP parameter, PDU field or CM field of the ASP, PDU or CM field that the test specifier wishes the received value to match. Sometimes the received constraint value is called the expected value. The type of the expected value, defined in the relevant ASP, PDU or CM definition, is called the expected type.
We shall use the term received value to mean the value of received object element. A received value is always, of course, a literal value, of a type compatible with the type of the corresponding element in the ASP, PDU or CM definition.
Figure 42 : Execution of an alternative that contains RECEIVE
|
|
Constraints are not used with the OTHERWISE statement. Remember, OTHERWISE will always match if the named PCO incoming queue is not empty. No other checking is required.
Figure 43 : Execution of an alternative that contains OTHERWISE
|
|
In this section we shall take a closer look at the RECEIVE statement and how it is used to check the received values against the specified constraint values.
In most cases a constraint value will be a specific value. Note that this is not necessarily always a literal value. In TTCN a specific value is an expression which evaluates to a value compatible with the corresponding element type in the relevant ASP, PDU or CM definition. The TTCN syntax allows the operands of these expressions to be:
When a specific value is used as a constraint value a successful match means that the received value is exactly equal to the value to which the constraint expression evaluates. Specific values can of course, be used to specify constraint values of all types.
In many cases it may be necessary to omit ASP parameters or PDU fields. In the tabular format all parameters or fields are considered to be optional and may be omitted. This is denoted by writing a dash (-) instead of value.
Case study 16: Omitting values.
Figure 44 : Structured Type Constraint Declaration (variable_part_CR1)
|
|
In the ASN.1 only parameters or fields that are defined as being OPTIONAL or DEFAULT may be omitted. This can be indicated either by explicitly using the OMIT keyword, or by not including the parameter or field in the constraint.
Figure 45 : Omitting values in ASN.1 constraints
|
|
In ASN.1 constraints may be constructed from previously defined constraints by using the REPLACE keyword.
Figure 46 : This table indicates that the constraint variable_part_CR2 is exactly the same as variable_part_CR1, except that the value of paramB is set to FALSE
|
|
In many instances it is not possible, or even desirable, to specify that the field of a received PDU shall have a specific value. It may be more appropriate to say that a match occurs if the received value falls within certain boundaries or fulfils certain conditions.
TTCN supports a number of matching mechanisms: matching symbols, matching operations and attributes that allow the test specifier to express these matching conditions instead of specific values.These mechanisms include:
A constraint value may be a list of one or more specific values (remember that specific values include expressions etc. so the elements in the list may be quite complex). A match occurs only if the received value is equal to any one of the values in the constraint value list, otherwise the match fails.
If a value list is preceded by the keyword COMPLEMENT then a match occurs only if the received value is not equal to any of the values in the constraint value list, otherwise the match fails. Complement can be used on values of any type.
Ranges may only be used to match values of INTEGER compatible types. The keywords INFINITY and -INFINITY may be used to specify ranges that may be unbounded in the positive and/or negative direction.
A range matches if the received value is within the range, including the upper and lower boundary.
In many cases the test suite specifier is prepared to accept any single value for a particular field, provided that the actual value is compatible with the corresponding element type.
The matching symbol AnyValue is denoted by "?". A match will occur if the received value is any value that is compatible with the expected type.
The AnyOrOmit matching symbol, denoted by "*", is similar to AnyValue, except that the value may be omitted altogether. If there is a value present then a match will occur if the received value is any value that is compatible with the expected type; otherwise the value must be omitted. This is only allowed with optional fields.
There are two wildcards that may be used within values:
The AnyOne symbol, denoted by "?", is used to replace single elements within all the string types, and within SEQUENCE, SEQUENCE OF, SET and SET OF types. However, the element may not be omitted.
Note that the denotation is the same as for AnyValue, but the semantics of the symbol are not the same.
The AnyOrNone symbol, denoted by "*", is used to replace single elements or a consecutive number of elements within all the string types, and within SEQUENCE, SEQUENCE OF, SET and SET OF types. Also, the element may be omitted.
The If_Present attribute is intended for use with OPTIONAL fields. The test suite specifier may not know beforehand whether the IUT will be including an OPTIONAL value or not in a particular PDU - the protocol allows either or. The test then has to specify that if the optional value is present it should be checked.
Note that in the tabular format all fields are considered to be OPTIONAL. The match will occur if the received value is any value that is allowed by the specified expected type, i.e. they need not be explicitly declared as such. In ASN.1 this is not the same case; any fields that are OPTIONAL have to be declared as such.
Length restrictions apply to the following types:
Essentially these are the same length restrictions that may be placed on the type definitions. The restriction may state the precise length of the string:
or it may define a range:
Figure 47 : Examples of matching mechanisms used in a PDU constraint
|
|
The TTCN standard says nothing about the actual encoding of values that are to be transmitted and received over the network. This aspect is being addressed in the second TTCN amendment, WDAM 2. It will allow the TTCN user to specify encodings at several levels:
It is unusual for a standard to specify the types of the parameters that constitute an ASP. How ASPs are realized is an implementation issue, outside the scope of the standard. The types, therefore, that are given to TTCN ASPs should not be considered binding - they are there to give a consistent representation in the test suite, and are mainly for documentation purposes.
In other words, checking of ASP parameters should be consistent with the implementation of those ASPs in the test system, rather than the exact TTCN specification.
In contrast to ASPs, PDU fields are typed in the relevant protocol standard and it is essential that these types are implemented correctly in the ETS. As far as encoding is concerned, TTCN currently refers to the standards that the PDUs are derived from. For example, if ASN.1 is used it is probable that the ASN.1 basic encoding rules (BER) rules apply, but not necessarily. Work needs to be done on this issue.
In some cases of testing it may be necessary to manipulate the encoding of values, e.g. in testing of the presentation layer. This aspect, too, is addressed in WDAM 2.
TTCN allows the use of individual components of complex types as operands in expressions or as the l.h.s. of an assignment. Such components include:
In the context of ASN.1 it is possible to access:
These references may be made either:
These are references to ASP parameters, PDU fields or structure elements made from a statement line that contains a SEND or RECEIVE and, most importantly, an associated constraint. In their simplest form these references are denoted by:
Suppose that a substructured PDU is chained to an ASP. To reference the kth element in the structure from a statement line we could write:
However, because the ASP, PDU and structure identifiers are unique within the test suite, it is allowed to simply use:
For example, if we wish to preserve the value of the user_data field of the incoming DT_PDU, embedded in an N_DATAindication. This could be done by writing:
This is rather verbose and because the PDU identifier is unique it is enough to write:
In other words the `dotted path' need only contain the identifiers that are enough to give a complete and unique reference.
The same mechanism can be used to reference elements in ASN.1 constraints that use SEQUENCE, SEQUENCE OF etc.
Suppose that we have defined the following PDU:
|
|
A constraint on that PDU may be:
Figure 49 : A TTCN PDU Constraint
|
|
Then writing:
|
|
means that the third field in the constraint is overridden and that the send object a_pdu is transmitted with field3 having the value FALSE.
|
Note: Where possible values should be set using parameters, rather than by this mechanism. |
In cases where the elements are not named, such as the elements in the SEQUENCE OF INTEGER that compose field2 in our previous examples, it is possible to reference the element by position. For example, if we wish to override the value 22 in field2 we simply write:
|
|
Individual bits in a BIT STRING can also be accessed in a similar manner. If we wish to change the third bit in the value of field1 from 0 to 1 we write:
|
|
This mechanism cannot be used with other string types.
An incoming ASP or PDU (i.e. received object) is only preserved for the duration of a RECEIVE statement, i.e. components of the received object cannot be accessed on statement lines subsequent to the RECEIVE event. It is possible, however, to declare variables of ASP, PDU or structure type. These variables are then bound to the received object. Suppose the variable temp_pdu is of type A_PDU:
Figure 53 : A Receive statement
|
|
We can now access components of a_pdu on subsequent statement lines and not just on the statement line that contains the RECEIVE statement:
Figure 54 : A Qualifier statement
|
|
There are two mechanisms in TTCN that provide assignment of verdicts to a test case. These mechanisms are:
A preliminary result or explicit final verdict may be associated with any TTCN statement except for the following:
TTCN has a predefined test case variable, known as the result variable, called R. This variable may be used in expressions and the verdict column of a behaviour description. It is used to store preliminary results and has the following characteristics:
The value of R is changed by recording a preliminary result in the verdicts column. A preliminary result may be one of the following:
For example:
Preliminary results have an order of precedence, for example:
The table below shows how R may be changed according to the precedence rules:
Figure 55 : Calculation of the preliminary result variable R
|
Current value of R |
Preliminary verdict | ||
|---|---|---|---|
| (PASS) | (INCONC) | (FAIL) | |
|
none |
pass |
inconc |
fail |
|
pass |
pass |
inconc |
fail |
|
inconc |
inconc |
inconc |
fail |
|
fail |
fail |
fail |
fail |
Execution of a test case is terminated either by:
A final verdict may be one of the following:
If no explicit final verdict is reached, then the final verdict is the value of R. If R is still bound to the value none then this is a test case error.
The final verdict must be consistent with the value of R. For example:
The table below shows how the final verdict should be recorded according to the value of R:
|
Current value of R |
Final verdict | |||
|---|---|---|---|---|
| (PASS) | (INCONC) | (FAIL) | R | |
|
none |
pass |
inconc |
fail |
*error* |
|
pass |
pass |
inconc |
fail |
pass |
|
inconc |
*error* |
inconc |
fail |
inconc |
|
fail |
*error* |
*error* |
fail |
fail |
In order to be able to express repetitive behaviour in a convenient way, TTCN allows statement lines to be labelled so that jumps may be made to them from later points in the tree. A GOTO is denoted either by:
or:
Infinite loops should be avoided, i.e. entering the GOTO loop should always depend on some event occurring or condition being fulfilled.
Figure 57 : Using GOTO in a behavior tree
|
|
The following rules should be followed:
TTCN timers are used to test timer events in the IUT. This is usually done by timing an expected response from the IUT using the START timer operation and the TIMEOUT event. The CANCEL timer operation is used to stop and reset a running or expired timer.All timers are declared in the Timer Declarations table. The duration is the period of time that will pass from the moment a timer is started to the moment it expires. Duration is measured in one of the following units:
Figure 58 : Declaration of timers
|
|
TTCN maintains a timeout list. If a timer expires its name is added to the timeout list. Three things can remove the name of the timer from the timeout list:
The test suite specifier may state that a named timer be checked to see if it has timed-out. This is denoted by:
When this statement is encountered while processing a statement line the TIMEOUT will match if the named timer is in the timeout list, otherwise the TIMEOUT fails.
An alternative use of TIMEOUT is simply:
i.e. no TimerIdentifier is given. In this case the TIMEOUT statement will succeed as long as the timeout list is not empty.
The TIMEOUT statement may be qualified and it may be followed by an ASSIGNMENT_LIST and/or TIMER_OPERATION. The order in which these statements may appear in the statement line is fixed, as shown below; the square brackets indicate that the presence of the statement in the statement line is optional:
|
Note: TIMEOUT should not be used to guard against a faulty IUT not sending a required response. It is the responsibility of the test system to implement detection of such an occurrence. |
We have already mentioned that at the beginning of each cycle through a set of alternatives a snapshot is taken of the incoming PCO queues. The alternatives are then checked against this snapshot. The same thing is done for the timeout list. A snapshot is taken of this list at the start of each cycle and if a TIMEOUT alternative is encountered in the set of alternatives it is checked against the timeout snapshot rather than the actual timeout list. This means that the expiry of a timer during processing of a set of alternatives is not registered until the timer snapshot is updated.
A named timer is started using the START timer operation. This is denoted by:
The duration for this timer is taken from the timer declaration. Alternatively, an explicit duration may be given, which overrides the declared duration:
If the timer is already running when the START is invoked then the timer is cancelled, reset and then started, i.e. the timer is re-started.
If the timer has expired then its name is removed from the timeout list before it is re-started.
The START_TIMER statement may be qualified and it may be followed by an ASSIGNMENT_LIST. The order in which these statements may appear in the statement line is fixed, as shown below:
Figure 59 : Using START and TIMEOUT in a behavior tree
|
|
A named timer is cancelled using the CANCEL operation. This is denoted by:
An alternative use of CANCEL is simply:
i.e. no TimerIdentifier is given. In this case all running timers are cancelled and reset and the timeout list is cleared.
Cancelling a timer that is expired will result in the timer being reset and its identifier is removed from the timeout list.
The CANCEL_TIMER statement may be qualified and it may be followed by an ASSIGNMENT_LIST. The order in which these statements may appear in the statement line is fixed, as shown below:
Figure 60 : Execution of an alternative that contains a stand-alone timer operation
|
|
TTCN supports both constants and variables. There are two types of constants:
and two types of variables:
The tables used are:
Test suite constants are declared globally and may be used anywhere in the test suite, including the constraints part. The value of the constant is specified at its point of declaration and may not be changed.
Case study 17: Declaration of test suite constants.
Figure 61 : Test suite constant declarations
|
|
Test suite parameters are also constants, but their actual values are not known to the abstract test suite specifier. These values will depend on which IUT is being tested, and possibly on the test system itself. In this sense the values of test suite parameters will be different from IUT to IUT, but during the testing of any given IUT they will remain constant.
Case study 18: Declaration of test suite parameters.
Figure 62 : Test suite parameter declarations
|
|
The test suite parameter values are derived from the Protocol Implementation Conformance Statement (PICS) and the Protocol Implementation eXtra Information for Testing (PIXIT). These documents are like checklists that are filled-in according to the characteristics of the IUT.
Prior to executing the tests the PICS and PIXIT are used to bind values to the test suite parameters. This process is called test suite parameterization.
Both test suite variables and test case variables are declared globally i.e. they may be used by test cases, test steps and defaults throughout the test suite. A default value may be specified for each variable, if wished. If no default value is specified, then the variable is said to be unbound.
Variables should be bound before use, unless they appear on the l.h.s. of an assignment.
Case study 19: Declaration of test case variables.
Figure 63 : Test case variable declaration
|
|
The difference between the two kinds of variable is when they are reset to their default values (if no default value is specified for a variable, then resetting means that the variable becomes unbound):
When more than one test component exists, as does with concurrent TTCN, then each test component is supplied with its own copy of each test case variable.
Test suite variables behave the same way in concurrent TTCN as they do in the non-concurrent version.
There are three types of tables for specifying the behaviour descriptions:
We have already noted that the difference between the different behaviour tables is in the header, rather than in the body of the tables.
Figure 64 : Fragment of a test case behavior table, showing the header for the test case
|
|
The test case identifier appears in the first field and, like most TTCN identifiers, it should be a name unique to the entire test suite. The second field contains the test case reference, which is a path name that specifies the test case's location in the test suite structure.
In the case of test steps this path specifies the test step's location in the test step library. In the case of defaults it specifies the location of the default in the default library. These references have the general format:
Note the terminating slash, which is the last group name in the path. The path may begin with the first GroupIdentifier, i.e. the SuiteIdentifier is optional. If the test suite has no hierarchy then the reference is empty.
In the test case table the third field is used to specify the test purpose. The corresponding field in test steps and defaults is called the objective.
The configuration entry is introduced by the concurrent TTCN to state the configuration in which this test case behaviour description is used. This field does not appear in test steps and defaults.
The default entry is used to state the default behaviour which should be used, if any.
Case study 20: Test step dynamic behaviour.
Figure 65 : Test step dynamic behaviour (LT_DATA_TRANSFER)
|
|
One of the main aims of TTCN is to specify behaviour descriptions so that the human reader can easily understand the TTCN specification of the test purpose.
The conformance standard requires that behaviour be expressed in terms of (N) and (N-1) ASPs. However, a behaviour tree consisting of mostly (N-1)-data requests and indications says very little to the reader. What is important are the PDUs embedded in these service primitives. If static chaining is used the reader will have no idea, without turning to the constraints, what PDU interactions are specified in the test.
The alias mechanism allows ASPs (and if necessary PDUs) to be renamed to reflect the different PDUs that they carry. The (N-1)-data request and indication may have several aliases, depending on which (N)-PDU they are carrying.
|
|
In ISO/IEC 9646-3 aliases are defined as textual expansions. However, it is probably easier to think of alias identifiers as alternatives to ASP or PDU identifiers in the SEND and RECEIVE statements. The effect is exactly the same, i.e.
Figure 67 : Test Step dynamic behavior using aliases
|
|
Test cases can be long and complex. There exist two mechanisms that allow test cases to be modularized: test steps and defaults.
Behaviour trees can be modularized by splitting them into sub-trees called test steps. Test steps are either:
Test steps may be parameterized, i.e. the calling tree can pass PCOs, variables, literal values, constraints etc. to the attached test step.
Local test steps may only be used within the behaviour description in which they appear:
Figure 68 : Illustration of local test trees and test step library
|
|
Test steps that belong to the test step library are specified in Test Step behaviour tables. These steps may be called by any test case, test step or default.
The ATTACH statement is used to invoke a test step, and is denoted by:
In both cases the actual parameter list should only be used if the test step has a formal parameter list. Note that a parameter may also be a PCO or CP.
Case study 21: The following test step:
Figure 69 : Test Step Dynamic Behaviour (LTS)
|
|
Case study 22: Is the same as:
Figure 70 : Test Step Dynamic Behaviour (LT)
|
|
TTCN defines tree attachment as the actual expansion of the called tree, i.e. the test step, into the calling tree, which may be a test case or another test step. While this is a sensible approach, adequately described in the TTCN standard, we feel that the view of treating test steps as subroutines is a valid one, and one that implements the TTCN semantics for tree attachment correctly. These semantics are easily understood by anyone with a programming background.
A test step can be considered as a subroutine when handled in the following manner:
Figure 71 : Execution of a test step
|
|
The conformance standard requires that a TTCN test case must be fully specified, in terms of behaviour. This means that at any point in time a tester must be prepared to accept all possible incoming ASPs or PDUs. This includes not only the ASPs or PDUs that are allowed by the protocol but also any ASP, legal or otherwise, that the IUT or service provider may issue
The easiest way to take care of this is by using the OTHERWISE statement. However, TTCN requires that the OTHERWISE statement leads to a fail verdict and this may not always be desirable. For example, it may be perfectly legal for the underlying network service to issue an N_DISCONNECTindication at any time. Certainly, an OTHERWISE would pick this up, but a verdict of FAIL in such an instance would be quite wrong. The only verdict that should be assigned in this case is INCONClusive, i.e. use with care!
Specifying all possible combinations tends to clutter up the main behaviour description, detracting from the readability of the test case. The default behaviour can be used to specify this peripheral behaviour in a precise manner. It will often comprise the set of ASPs or PDUs that are allowed by the protocol at any given time but which are not part of the test purpose, and an OTHERWISE to catch all other unspecified events. It is also common practice to include a general TIMEOUT in the default.
Case study 23: If we specify the following default behaviour.
Figure 72 : Default Dynamic Behaviour (LT_DEFAULT)
|
|
Default behaviour can be modeled as a tree attachment that is implicitly called as the last alternative in every set of alternatives.
A test case or test step references this default behaviour in the Defaults entry in its header. If this entry is empty then no default behaviour is applicable.
Figure 73 : The test step with a default
|
|
Figure 74 : Is the same as the previous test step
|
|
The following TTCN objects may be parameterized:
In all cases parameterization is indicated by the relevant TTCN object identifier being followed by a formal parameter list. For example:
Parameterized objects are invoked with an actual parameter list. For example:
The following rules apply:
The TTCN uses textual substitution to define the passing of actual parameters in test steps and defaults. An alternative, and more intuitive, way of describing parameter passing for test steps and defaults and yet retain TTCN semantics is to describe the mechanism in terms of call-by-reference, in which the called routine (test step or default) has access to the original argument, not a local copy. All operations that effect that argument have the same effect on the original.
The TTCN standard states that neither user defined operations nor constraints may change the values of any actual parameters that are passed to them, i.e. they shall have no side-effects. Thus, for user defined operations and constraints it is more suitable to describe the parameter passing mechanism in terms of call-by-value in which the called routine (user defined operations or constraint) works on a local copy of the argument. The original argument is not affected by the routine.
A test suite contains many hundreds, perhaps thousands, of test cases. In most cases of testing it will only be necessary to choose and run a selection of tests taken from the test suite. This choosing process is called test case selection. Depending on values and answers obtained from the PICS and PIXIT only a subset of the entire test suite need be executed.
TTCN allows each test case to be associate with a selection expression. These expressions are predicates that will evaluate to TRUE or FALSE depending on the answers given to the relevant PICS and PIXIT questions. If no selection predicate is given then the test will always be selected.
The predicates are defined in the Test Case Selection Expression Definitions table, and references are made to them from the Test Case Index.
Groups of test cases may be selected in a similar manner by making references to selection expressions from the Test Suite Structure table.
Each TTCN object has a specific position in the hierarchy of the test suite.
The different test suite components may only appear in a specific order. A TTCN test suite consists of four parts:
Each part contains a number of TTCN tables. The order in which the tables appear is shown in the following list. Each bulleted item in this list represents a TTCN table. The tables that have number subscripts are tables for single TTCN objects, e.g. PDUs and test cases. The tables that do not have a subscript are multiple TTCN object tables, e.g. simple type definitions or test suite variables.
Some tables may be displayed in a compact format. Tables printed in italic font are defined in the TTCN extensions.
The test suite overview consists of four tables:
The declarations part is concerned both with the definition of new (i.e. not predefined) data types and operations and the declaration of all the test suite components.
The constraints part contains the tables for all the ASP, PDU, structure and CM constraints. Both in the tabular form and the ASN.1.
|
Note: ASP Constraints may displayed in a compact format if wished. |
|
Note: ASN.1 ASP Constraints may displayed in a compact format if wished. |
|
Note: PDU Constraints may displayed in a compact format if wished. |
|
Note: ASN.1 PDUConstraints may displayed in a compact format if wished. |
|
Note: Structured Type Constraints may displayed in a compact format if wished. |
|
Note: ASN.1 Type Constraints may displayed in a compact format if wished. |
The dynamic part contains all the test cases, all the test steps in the test step library and the all the defaults in the default library.
|
Note: Test groups, i.e. the test suite structure, are not represented here. Test cases may displayed in a compact format if wished. |
|
Note: Test step groups are not represented here. |
|
Note: Default groups are not represented here. |
In ITEX on UNIX, the implemented version of TTCN does support modularization of a TTCN document. This support is conveniently used to concurrently produce multiple documents with some definitions in common or to cooperatively produce one large TTCN document.
The mechanism implemented assumes that each user have a private target directory and all collaborating users having the same source files.
The following figures depict how this is accomplished. In the figures TS denotes a test suite and M denotes a TTCN module. Numbers are used to distinguish separate documents.
Figure 75 : Two Modular Test Suites using a common Module
|
|
Figure 75 depicts the case where two test suites are developed in parallel by two different users with both test suites referencing objects defined in a common module.
Figure 76 : A large Modular Test Suite
|
|
Figure 76 depicts the case where a large test suite (TS) is developed in parallel by two different users, each responsible for one module of the test suite (M1 and M2), with both modules referencing objects defined in a common module (M3).
In the case where more than one user is to concurrently develop different parts of a large test suite, here is a simple way to distribute the document files in the file system.
The user may now change the directory representation back to the relative form if so wished.
Figure 77 : Test suite structure
|
|
|
|
|
|
Figure 80 : Simple type definitions
|
|
Figure 81 : Definition of VARIABLE_PART
|
|
Figure 82 : Definition of INC operation
|
|
Figure 83 : Test suite parameter declarations
|
|
Figure 84 : Test suite constant declarations
|
|
Figure 85 : Test case variable declarations
|
|
Figure 86 : PCO type declarations
|
|
|
|
Figure 88 : Coordination point declarations
|
|
Figure 89 : Test component declarations
|
|
Figure 90 : Declaration of SINGLE_PARTY
|
|
Figure 91 : Declaration of MULTI_PARTY
|
|
Figure 92 : Definition of IN_DATArequest
|
|
Figure 93 : Definition of IN_DATAindication
|
|
Figure 94 : Definition of X_CONNECTindication
|
|
Figure 95 : Definition of X_CONNECTresponse
|
|
Figure 96 : Definition of X_DATArequest
|
|
Figure 97 : Definition of X_DATAindication
|
|
Figure 98 : Definition of CR_PDU
|
|
Figure 99 : Definition of CC_PDU
|
|
Figure 100 : Definition of DT_PDU
|
|
Figure 101 : Definition of PTC_RESULT
|
|
Figure 102 : Alias definitions
|
|
Figure 103 : Declaration of variable_part_CR1
|
|
Figure 104 : Declaration of variable_part_CR2
|
|
Figure 105 : Declaration of NDr
|
|
Figure 106 : Declaration of NDi
|
|
Figure 107 : Declaration of CONind
|
|
Figure 108 : Declaration of CONrsp
|
|
Figure 109 : Declaration of DATreq
|
|
Figure 110 : Declaration of DATind
|
|
Figure 111 : Declaration of CR1
|
|
Figure 112 : Declaration of CC1
|
|
Figure 113 : Declaration of DT1
|
|
Figure 114 : Declaration of PTC_RES
|
|
Figure 115 : Definition of SP_DATA_TRANSFER
|
|
Figure 116 : Definition of MP_DATA_TRANSFER
|
|
Figure 117 : Definition of LTS
|
|
Figure 118 : Definition of ESTABLISH_CONNECTION
|
|
Figure 119 : Definition of LT_DATA_TRANSFER
|
|
Figure 120 : Definition of CLOSE_CONNECTION
|
|
Figure 121 : Definition of UTS
|
|
Figure 122 : Definition of ACCEPT_CONNECTION
|
|
Figure 123 : Definition of UT_DATA_TRANSFER
|
|
Figure 124 : Definition of UT_DEFAULT
|
|
Figure 125 : Definition of UT_DEFAULT
|
|
Figure 126 : Definition of T_DEFAULT
|
|