This chapter describes two ways for generating TTCN test suites based on SDL specifications. The first one is to use TTCN Link, which assists you in manual specification of test suites. The other one is to use the Autolink feature of the SDT Validator, which allows automatic generation of test suites.
For more information about the SDT Validator, see The SDT Validator.
Tutorials for TTCN Link and Autolink can be found in Tutorial: The TTCN Link and Tutorial: The Autolink.
Testing is one of the most important steps in the development of a new product. Often, it is also very time consuming and costly. As part of the Conformance Testing Methodology and Framework, the Tree and Tabular Combined Notation (TTCN) has been defined as a formal language for test suite specification. A test suite consists of four basic parts: The test suite overview, the declarations, the constraints and the dynamic behavior description.
In Telelogic Tau, TTCN test suite generation is supported by TTCN Link and Autolink. They both use an SDL specification as the basis for test generation, but they differ in their functionality.
TTCN Link generates the TTCN declarations part automatically and you use it for interactive building of test cases in the dynamic part.
Autolink is embedded in the SDT Validator. In addition to the SDL specification, it uses MSCs for test purpose descriptions. With this input, Autolink generates the declarations, constraints and dynamic behavior description parts of a TTCN test suite automatically.
In comparison, the test generation features of Autolink are superior to the ones of TTCN Link. If for some reason the test purpose description with MSCs is not applicable, then you should use TTCN Link. In any case, test cases built with TTCN Link can be merged with test cases generated by Autolink.
Figure 241 : Overview of TTCN Link and Autolink
|
|
TTCN Link automatically generates the TTCN declarations part based on an SDL specification. The default dynamic behavior table is also generated. It contains timeout and otherwise statements for each PCO, which will ensure that any incorrect response from the implementation under test always will give a FAIL verdict as a result from the test case. After the default and constraints tables have been generated, you can interactively build test cases.
When you use TTCN Link, there are four (five) phases involved:
For more information about TTCN Link, see Using TTCN Link.
Autolink can be used for automatic generation of TTCN test suites based on an SDL specification and a number of MSCs. The steps involved when you use Autolink are:
For more information about Autolink, see Using Autolink.
Figure 242 : Generation of a TTCN test suite with ITEX and Autolink
|
|
The phases involved when you are using TTCN Link -- Preparing for the Generation of Declarations, Generating the Declarations and Creating Test Cases -- will be described below. You can also read about Showing SDL System Information and Merging TTCN Test Suites in ITEX.
Before it is possible to generate the declarations, you have to do two things:
The major adaptation of the SDL system that you have to do, is to modify it to properly describe the test architecture that is to be used. Basically, the requirement is that the channels from/to the environment of the system must correspond to the points of control and observation in the test suite. For example, the SDL system might be a specification of a communication protocol where the lower side of the protocol in practise can only be accessed through a network. To be able to create correct test cases, you also have to include the communication media in the SDL specification. Note however, that the specification of the media does not have to be a detailed specification of the functionality, only a specification of the aspects relevant to the current testing situation.
Once the SDL specification describing the system to be tested and the test architecture are finished, you can generate a Link executable. (A Link executable is sometimes also referred to as state space generator.) You do this in the same way as when you generate an SDT Simulator or Validator.
To generate a Link executable:
<sdl system name>_xxx.link, where xxx is depending on the compiler used.
There are two steps involved in generation of the declarations (and the default table):
Before the actual generation of the TTCN declarations, you have to specify the Link executable -- and thereby the SDL system -- to use. There are two methods for doing this: associating the SDL and TTCN systems in the Organizer and explicitly selecting the Link executable in ITEX. In case a Link executable has been specified both in the Organizer and in ITEX, the one selected in ITEX is the executable that will be used.
Also note that the TTCN test suite that you are going to generate the declarations in, have to be created and added to the Organizer (and the same system file as the SDL system is included in).
|
The SDL system from which the Link executable is generated may contain external synonyms that do not have a corresponding macro definition (see External Synonyms). Such an SDL system cannot be used with TTCN Link and you will get an error message when trying to select the Link executable. However, if you set the environment variable The syntax of a synonym file is described in Reading Values at Program Start up. |
When you have specified the Link executable, you can generate the declarations in ITEX:
There are two alternatives available for generation of the PCO types: either one PCO type is generated for each channel in the SDL system or only one PCO type is generated. This is defined by the configuration command define-pco-type-mapping (see PCO Type Generation Strategy). The default is that only one PCO type is generated.
If more than one PCO type is generated, they are named <ChannelName>_TypeId. If only one PCO type is generated it is called PCO_Type.
The generated ASPs/PDUs are given the same name as the corresponding SDL signal with one exception: If there are multiple PCO types and there is one signal that can be transported on more than one channel to the environment, this signal is divided into two ASPs, since an ASP may only be associated with one PCO type. The ASPs are then given names like <SDLSignalName>_<PCOName>.
By default, ASPs are generated from the SDL signals. However, you can change it to PDUs by using the define-signal-mapping command (see SDL Signal Mapping Strategy).
The data type mapping from SDL to TTCN/ASN.1 is defined in the following table. In most cases a table containing an ASN.1 type definition is generated for each data type. In the table below, this is indicated by a "<TTCN name> -> <ASN.1 definition>" clause. The <TTCN name> is the name used to denote the type in the test suite and the <ASN.1 definition> is the ASN.1 type definition that is the contents of the generated table. If the TTCN name is omitted, the name is given by the name of the corresponding SDL data type.
In addition to the data types above, data types defined in external ASN.1 modules can also be used. These data types are mapped to definitions in the table named "ASN.1 Type Definitions by Reference" in the test suite. For each data type in the external ASN.1 module that is used on signals to/from the environment, one line defining the data type will be generated in this table. Note that the Generate Declarations command in the SDT Link menu assumes that the external ASN.1 module is setup as a dependency of the TTCN document.
No other data types than the ones mentioned above may occur on signals to/from the environment in the system.
Note that SDL is not case sensitive whereas TTCN is case sensitive. The spelling of the names generated by TTCN Link is given by the defining occurrence of the corresponding SDL name. Also note that no transformation of names is performed during generation of the TTCN names. This may in some cases lead to incorrect TTCN names if for example a reserved word from TTCN is used in the SDL system. To fix this problem, you have to change the name in the SDL system to a legal TTCN name.
In some cases, you may find it useful to manually modify the declarations that have been generated by TTCN Link before continuing with the development of the test cases. There are in particular two interesting cases:
It is possible to regenerate the declarations from an SDL system to incorporate new signals, channels and/or data type into the test suite. If you select Generate Declarations from the SDT Link menu again, only declarations with a name different from the existing test suite declarations will be inserted into the test suite.
To create test cases with TTCN Link, you use the Table Editor in ITEX. When TTCN Link is used, the test cases are synchronized, that is, verified against the SDL specification.
In synchronized mode, the test case is guaranteed to be consistent with the specification. Each action you perform during the development of the test case will be incrementally verified by the state space exploration part of TTCN Link. When a table is in synchronized mode, the SDT Link menu of the UNIX Table Editor will contain new menu choices for editing of the test case. In Windows, you can find the corresponding commands in the Link dialog:
If you modify the contents of the test case by using other menu choices, the editor will leave the synchronized mode.
The verdict for the generated test case lines, will always be either PASS or INCONCLUSIVE since the generated receive lines will always correspond to valid behaviors of the implementation under test.
A default test step, which consists of an otherwise fail for each PCO, will ensure that an incorrect response from the implementation under test always will give a FAIL verdict as a result from the test case.
The constraints that are used in send and receive statements in the test cases, are subject to certain restrictions:
The following is however allowed in send and receive constraints a test case is resynchronized, even though it is not generated automatically for receive constraints:
It is often useful to structure the test case into test steps. To do this by using TTCN Link:
When you use TTCN Link for creating a test case, it is possible to access the SDL specification from the Table Editor.
To do this you first select a line in the test case. Then you have three alternatives:
By using TTCN Link, you can only generate the declarations and create the dynamic tables. Either you could add the constraints and dynamic tables manually or merge the TTCN Link generated test suite with one generated by Autolink. A test suite generated by Autolink is in TTCN-MP format and contains constraints, declarations and dynamic tables.
To merge the test suites:
The merge will only work if the two test suites do not conflict. A conflict occurs if any TTCN object in the MP file has the same name as any TTCN object in the destination document. However, if such a conflict is detected, the merge will continue but the conflicting object in the MP file will be skipped.
Constraints will be merged in a special way. For example, the MP file may contain a TTCN ASP constraint called constraint1 that refers to the type type1 which is of the incompatible type TTCN PDU TypeDef. Because of this, a copy of constraint1 will be inserted as a TTCN PDU constraint instead. However, this "type conversion" is limited. An ASN.1 constraint will not be converted to a TTCN constraint or vice versa.
TTCN Link supports test case development and there are two major objectives:
This section will give a brief introduction to the algorithm used by TTCN Link to synchronize a test case with an SDL system.
The system that TTCN Link analyzes is the composed system that consists of both the SDL specification and the TTCN test case that is interactively created.
Figure 243 : A composed SDL/TTCN system
|
|
The connection between the SDL system and the TTCN test case is created by connection of the channels to/from environment in the SDL system to the PCOs in the test suite.
The technique used by TTCN Link is based on state space exploration (sometimes referred to as reachability analysis) of the system composed of the SDL system together with the test case that is being created. The state space of this system can be viewed as a graph, where the nodes represent system states and the edges represent actions that can be performed by the system.
Figure 244 : A state space fragment
|
|
Each system state represents the combined system in one moment in time. It contains information of for example:
The actions represented by the edges are either SDL actions like input, output, tasks, etc., or TTCN actions like send, receive or start timer.
Essentially, the algorithm to generate the state space of the combined system is the following, where two global variables -- StateSpace (a graph that will contain the state space of the system) and TreatList (a list of states that is yet to be treated by the algorithm) -- are used:
Since you interactively create the test case that describes the TTCN part of the combined SDL/TTCN system, it is not possible for TTCN Link to compute the entire state space at once. Instead, the state space exploration is performed in an incremental fashion in the following way:
The consequence of this algorithm is that the state space of the combined SDL/TTCN system is explored in an incremental fashion, where each increment corresponds to a command you have given. Also the structure of the state space is influenced by the incremental way that it is generated. The state space can be visualized as a tree structure, where each node represents one line in the test case and a subpart of the state space, to be more precise, the subpart of the state space where the test case has executed this particular line but not the next one.
Figure 245 : A structured state space
|
|
Since each line in the test case is created by a command from you, each node with its associated part of the state space can also be viewed as the state space increment that was created by a specific command.
The default state space exploration algorithm used by TTCN Link is the algorithm described in the previous sections. This is usually referred to as exhaustive exploration since it exhaustively explores the state space until all reachable states has been generated. The benefit of this algorithm is that when the exploration is finished, you can be sure that all possible combinations and alternatives are explored, that is, if TTCN Link generates two alternatives in a receive statement, the algorithm guarantees that there are no more valid alternatives. However, the drawback is that the algorithm requires all states in the state space to be kept in primary memory. For large SDL systems this may not always be possible with the computers available. The response time may also be unacceptable for interactive work with large systems.
To be able to use TTCN Link even in these cases, a second exploration algorithm is also provided. This is the random walk algorithm that, instead of exploring the entire state space, explores random paths in the state space using the algorithm described below. The input to the algorithm is a set of start states called StartList and a maximum depth of the exploration (MaxDepth) and the number of repetitions (Rep):
The benefit with the random walk algorithm is that not more than a few system states (the current state and its successors) need to be kept in the memory at the time. The drawback is that there is no guarantee that the entire state space is explored, so, for example, even if TTCN Link generates only one receive alternative, it is possible that there are more alternatives.
To accomplish the best, both from exhaustive exploration and random walk, a two-step approach can be used when you use TTCN Link for large SDL systems:
This strategy gives both good performance when you interactively create the test cases and a good verification of the correctness of the test case during the automatic resynchronize. The execution time of the random walk algorithm is proportional to the number of repetitions.
You select which algorithm to use in the .linkinit file (on UNIX) or the linkinit.com file (in Windows) by using the define-algorithm command as described in section Configuring the TTCN Link Executable.
The algorithm used by TTCN Link to resynchronize a TTCN test case with an SDL system, is based on an incremental state space exploration of the state space of the composed system. The system consists of the SDL specification, together with the test case under construction. In the state space exploration, each command that you give will cause a new part of the state space to be explored, that is, the part that corresponds to the TTCN statement line that is inserted by the command. Two different exploration algorithms are available: exhaustive exploration and random walk. Exhaustive exploration is used for interactive development of test cases for a small SDL system and for verification of test cases for large systems. Random walk is used for interactive development of test cases for large SDL systems.
This section describes the various options that can be used to configure the Link executable. You can change the options by giving the corresponding commands in a file called .linkinit (on UNIX) or linkinit.com (in Windows) in the target directory. The options that can be set are:
These options will be described below.
It is also possible to add user-defined rules to the .linkinit file (on UNIX) or the linkinit.com file (in Windows) in order to prune the state space that is explored by Link. The user-defined rules are described in User-Defined Rules.
What exploration algorithm is used, is defined by the command
define-algorithm [exhaustive|randomwalk]
and has the default value exhaustive. The differences between the different algorithms are described in section Overview of the TTCN Link Algorithm.
The maximum depth of each random walk is defined by the command
define-randomwalk-depth <integer>
and has the default value 500.
The number of times a random walk is performed when a state space is explored by using this algorithm is defined by the command
define-randomwalk-repetitions <integer>
with a default value of 3.
The PCO Type generation strategy is defined by the command
define-pco-type-mapping [system|channel]
and has default value system.
system then only one PCO type is generated for the entire system. channel then one PCO type is generated for each channel to/from the environment. The choice also has an impact on the ASPs that are generated from SDL signals. Usually the name of the ASP is the same as the name of the SDL signal. However, if one PCO type is generated for each channel to/from the environment and one SDL signal can appear on more than one of these channels, then one ASP is generated for each channel it appears on. This is needed since an ASP can only be associated with one PCO type. The names of the signals are in this case defined as <signal name>_<channel name>.
The SDL signals that appear on channels to/from the environment in the SDL system are either mapped to ASN.1 PDU or ASN.1 ASP definitions in the test suite. The mapping is defined by the command
define-signal-mapping [asp|pdu]
and has default value asp.
The stable state option is defined by the command
define-stable [on|off]
and has the default value on.
It works like this:
Consider the situation when an empty test case has been resynchronized. The Link executable will now have computed the state space that can be reached without any input from or output to the tester. Usually, the state space looks something like this:
Figure 246 : Original state space
|
|
where 0 is the start state, 1--4 are intermediate states and 5 is a stable state where all internal queues in the SDL system are empty and nothing more can happen without any input from the tester.
Let us now give a send command. This implies that new states are created as send transitions are added to the state space.
If the stable state assumption is off then we add one send transition to each state in the original state space, the new state space looks something like this:
Figure 247 : The new state space with stable state assumption "off"
|
|
where all states called something with `a' are new. Now the states space that contains states that can be reached from the `a' states without any input from or output to the tester is explored.
On the other hand, if the stable state assumption is on then we only add a send transition to the stable state, giving a new state space looking like:
Figure 248 : The new state space with stable state assumption "on"
|
|
Now, only the states that can be reached from "5a" without any input from or output to the tester are explored. This state space is of course a lot smaller that the one above.
This is an option that in most cases will not have to be changed. The option defines how to interpret timeout actions compared with all other actions in the system. The option is changed by the command:
define-timer-mode [long|short]
The default is long.
If the timer mode is long, timeout actions will never occur if there is an internal event possible in the system. Essentially, the assumption is that the performance of the test system and IUT is good enough to ensure that the execution time for the actions are very small compared to the timeout times. Consider a system with the following state space when the long timer mode is used:
Figure 249 : State space with timer mode "long"
|
|
In this system, the timeout event does not occur until no other event can happen. The transition leading to states 1--3 are all usual transitions -- for example. inputs and outputs -- so the timeout will only occur in state 3, where no other event is possible.
If the timer mode would have been short, the state space would have looked differently:
Figure 250 : State space with timer mode "short"
|
|
The reason is that the timeout event now is possible in all the states 0--3.
The transition option defines what is considered to be an atomic transition in the state space and is defined by the command:
define-transition [`sdl'|'symbol']
The default value is symbol.
If the transition option is set to sdl, then SDL process graph transitions are considered atomic. This means that there will be no states in the state space where SDL processes are in the middle of a process graph transition. In all system states in the state space, the processes will always be in a process graph state.
If the transition option is set to symbol, the SDL process graph transitions are not considered to be atomic. In theory, this would imply that the processes could be interrupted anywhere during the execution of a transition. However, it turns out that for test generation purposes it is enough if the process graph transitions are divided at the points where one process communicates with another process, for example after an output or a create. A sequence of tasks or decisions is still viewed as atomic.
The consequence of this is that there will be more transitions in the state space if the transition option is symbol than if it is sdl. Consider a simple system with only one process. Let this process have a transition like:
state xx;
input st1;
output sig2;
output sig3;
output sig4
nextstate st2;
If the transition option is sdl, there will only be one transition in the state space that corresponds to the process graph transition above:
X : process is in state st1
|
Y : process is in state st2
If the transition option is symbol, there will be a sequence of transitions in the state space:
X : process is in state st1
| input st1; output sig2;
X1
| output sig2;
X2
| output sig3;
X3
| output sig4; nextstate st2;
Y : process is in state st2
This will of course give a lot bigger state space.
The scheduling option is defined by the command
define-scheduling [`first'|'all']
This option controls how many processes are allowed to execute in a given system state. If the option is set to first, only one process (the first in the ready queue) is allowed to execute. If the option is set to all, all processes that can execute are allowed to do it. The default value is all.
Consider an SDL system with two static processes. If the scheduling option is all, the initial part of the state space for this system will probably look like:
Figure 251 : State space with scheduling as "all"
|
|
where
start symbol). start symbol.start symbol.On the other hand, if the scheduling option is first, it will look like:
Figure 252 : State space with scheduling as "first"
|
|
where
start symbol). start symbol.If there are lots of processes, there will be a significant difference in the size of the state space depending on how the scheduling option is set!
The MSC trace options control what is displayed in the MSCs generated by TTCN Link. There are two different MSC trace options controlling if states and actions in the SDL system are showed as MSC condition symbols and MSC action symbols. The options are set by the following commands:
define-MSC-trace-actions [ `on' | `off' ] define-MSC-trace-states [ `on' | `off' ]
The default value for both options is `off'.
This following .linkinit file (on UNIX) or linkinit.com file (in Windows) will change the exploration algorithm to random walk and set the number of repetitions to 2:
define-algorithm random define-random-rep 2
This is a useful configuration when you work interactively with TTCN Link, since the random walk algorithm is quicker and requires less memory than the exhaustive algorithm. But, since the random walk in some cases can miss some alternative receive statement, a useful strategy is the following: Use the configuration above when you work interactively with TTCN Link. However, when you have finished with a test case or a number of test cases, check that the assumptions are valid by resynchronizing with the exhaustive algorithm or a larger number of repetitions.
Note that in the commands in the .linkinit file (on UNIX) and the linkinit.com file (in Windows) can be abbreviated as long as they are unique.
User-defined rules can be used during state space exploration to prune the search performed by the TTCN Link. Whenever a system state is found that matches the defined rule, the search is pruned at this particular state. This can be useful in order to remove specific exceptional behavior from the test cases that are designed and instead handle these in a special default test step.
Consider an SDL specification that contains a clock process that has a timer intervaltimeout. Every time the intervaltimeout expires, a signal DisplayTime is sent to the environment of the SDL system. Since this can happen at any time during the execution, there will be an alternative at all receive statements corresponding to the reception of this signal. To avoid this, it is better to add a receive statement in the default dynamic behaviour that skips this signal.
To achieve this with TTCN Link, you have to do two things:
A rule that prunes the state space whenever the intervaltimer expires is the following:
def-rule sitype(signal(clock:1))=intervaltimeout;
The statements that need to be added to the default behaviour are:
PCO?DisplayTime DisplayTime_Match_All RETURN
These lines will cause the tester to ignore DisplayTime signals sent from the system.
A rule essentially gives the possibility to define predicates which describe properties of one particular system state. As soon as this predicate matches a system state, TTCN Link will prune the search. A rule consists of a predicate (as described below) followed by a semicolon (`;'). In a rule, all identifiers and reserved words can be abbreviated as long as they are unique.
|
Note: Only one rule can be used at any moment. If more than one rule is needed, reformulate the rules as one rule, by using the boolean operators described below. |
The following types of predicates exist:
Parenthesis are allowed to group predicates.
The quantifiers listed below are used to define rule variables denoting process instances or signals. The rule variables can be used in process or signal functions described later in this section.
exists <RULE VARIABLE> [: <PROCESS TYPE>] [ | <PREDICATE>]
This predicate is true if there exists a process instance (of the specified type) for which the specified predicate is true. Both the process type and the predicate can be excluded. If the process type is excluded, all process instances are checked. If the predicate is excluded, it is considered to be true.
all <RULE VARIABLE> [ : <PROCESS TYPE>] [ | <PREDICATE>]
This predicate is true for all process instances (of the specified type) for which the specified predicate is true. Both the process type and the predicate can be excluded. If the process type is excluded, all process instances are checked. If the predicate is excluded, it is considered to be true.
siexists <RULE VARIABLE> [ : <SIGNAL TYPE>] [ - <PROCESS INSTANCE>] [ | <PREDICATE>]
This predicate is true if a signal (of the specified type) exists in the input port of the specified process for which the specified predicate is true. If no signal type is specified, all signals are considered. If no process instance is specified, the input ports of all process instances are considered. If no predicate is specified, it is considered to be true. The specified process can be either a rule variable that has previously been defined in an exists or all predicate, or a process instance identifier (<PROCESS TYPE>:<INSTANCE NO>).
siall <RULE VARIABLE> [ : <SIGNAL TYPE>] [ - <PROCESS INSTANCE>] [ | <PREDICATE>]
This predicate is true for all signals (of the specified type) in the input port of the specified process for which the specified predicate is true. If no signal type is specified, all signals are considered. If no process is specified, the input ports of all process instances are considered. If no predicate is specified, it is considered to be true. The specified process can be either a rule variable that has previously been defined in an exists or all predicate, or a process instance identifier (<PROCESS TYPE>:<INSTANCE NO>).
The following boolean operators are included (with the conventional interpretation):
not <PREDICATE> <PREDICATE> and <PREDICATE> <PREDICATE> or <PREDICATE>
The operators are listed in priority order, but the priority can be changed by parenthesis.
The following relational operator predicates exist:
<EXPRESSION> = <EXPRESSION> <EXPRESSION> != <EXPRESSION> <EXPRESSION> < <EXPRESSION> <EXPRESSION> > <EXPRESSION> <EXPRESSION> <= <EXPRESSION> <EXPRESSION> >= <EXPRESSION>
The interpretation of these predicates is conventional. The operators are only applicable to data types for which they are defined.
The expressions that are possible to use in relational operator predicates are of the following categories:
Most of the process functions must have a process instance as a parameter. This process instance can be either a rule variable that has previously been defined in an exists or all predicate, a process instance identifier (<PROCESS TYPE>:<INSTANCE NO>) or a function that returns a process instance, e.g. sender or from.
state( <PROCESS INSTANCE> )
Returns the current SDL state of the process instance.
type( <PROCESS INSTANCE> )
Returns the type of the process instance.
iplen( <PROCESS INSTANCE> )
Returns the length of the input port queue of the process instance.
sender( <PROCESS INSTANCE> )
Returns the value of the imperative operator sender (a process instance) for the process instance.
parent( <PROCESS INSTANCE> )
Returns the value of the imperative operator parent (a process instance) for the process instance.
offspring( <PROCESS INSTANCE> )
Returns the value of the imperative operator offspring (a process instance) for the process instance.
self( <PROCESS INSTANCE> )
Returns the value of the imperative operator self (a process instance) for the process instance.
signal( <PROCESS INSTANCE> )
Returns the signal that is to be consumed if the process instance is in an SDL state. Otherwise, if the process instance is in the middle of an SDL process graph transition, it returns the signal that was consumed in the last input statement.
<PROCESS INSTANCE> -> <VARIABLE NAME>
Returns the value of the specified variable. If <PROCESS INSTANCE> is a previously defined rule variable, the exists or all predicate that defined the rule variable must also include a process type specification.
<RULE VARIABLE>
Returns the process instance value of <RULE VARIABLE>, which must be a rule variable bound to a process instance in an exists or all predicate.
Most of the signal functions must have a signal as a parameter. This signal can be either a rule variable that has previously been defined in an siexists or siall predicate, or a function that returns a signal, e.g. signal.
sitype( <SIGNAL> )
Returns the type of the signal.
to( <SIGNAL> )
Returns the process instance value of the receiver of the signal.
from( <SIGNAL> )
Gives the process instance value of the sender of the signal.
<RULE VARIABLE> -> <PARAMETER NUMBER>
Returns the value of the specified signal parameter. The siexists or siall predicate that defined the rule variable must also include a signal type specification.
<RULE VARIABLE>
Returns the signal value of <RULE VARIABLE>, which must be a rule variable bound to a signal in a siexists or siall predicate.
maxlen( )
Gives the length of the longest input port queue in the system.
instno( [<PROCESS TYPE>] )
Returns the number of instances of type <PROCESS TYPE>. If <PROCESS TYPE> is excluded the total number of process instances is returned.
depth( )
Gives the depth of the current system state in the behavior tree/state space.
<STATE ID>
The name of an SDL state.
<PROCESS TYPE>
The name of a process type.
<PROCESS INSTANCE>
A process instance identifier of the format <PROCESS TYPE>:<INSTANCE NO>, e.g. Initiator:1.
<SIGNAL TYPE>
The name of a signal type.
null
SDL null process instance value
env
Returns the value of the process instance in the environment that is the sender of all signals sent from the environment of the SDL system.
<INTEGER LITERAL> true false <REAL LITERAL> <CHARACTER LITERAL> <CHARSTRING LITERAL>
The restrictions imposed on the SDL specification by TTCN Link are basically of four different kinds:
TTCN Link is based on the SDT C Code Generator and has thus the same restrictions as the Simulator and Validator which are also based on the C Code Generator. The major restrictions are:
For more information about the general SDL restrictions see SDL Restrictions.
TTCN Link is based on state space exploration of the combined state space of the SDL system and the TTCN test case. Since there is only a finite amount of memory available in computers, this means that there will be restrictions on the size of the state space that can be handled. It is not possible to give a numeric value on this restriction since it depends both on the SDL system and on the computer, but TTCN Link has been successfully used on SDL systems with more than 10 processes on a SPARCstation 10 computer. Also see Overview of the TTCN Link Algorithm.
Only the data types described in section Generating the Declarations are allowed on the channels to/from the system.
The mapping of concepts from SDL to TTCN generates a lot of names in TTCN. For example, the signals in SDL will become ASPs/PDUs in TTCN and SDL data types will become TTCN data types. The names of the generated entities are taken directly from the names on the corresponding SDL entity. This will lead to problems if, for example, the names are reserved words in TTCN. In this case, the names in the SDL system have to be changed.
This section describes the extra menu choices that are available in ITEX when TTCN Link is used on UNIX.
The following menu choices are available in the Browser SDT Link menu:
Makes a connection between a test suite and the corresponding SDL system. In the file dialog that opens, a Link executable should be selected. If the file is not a legal Link executable, the selection will fail.
When a Link executable is selected, a place holder for it is stored in the test suite. It is possible to change the Link executable if, and only if, there is no test case which uses the current SDL system (i.e. there is no synchronized test case).
It is also possible to select a Link executable by associating the SDL system with the TTCN system in the Organizer. However, a Link executable selected in ITEX will override an executable selected in the Organizer.
See also External synonyms.
Generates TTCN versions of the relevant type declarations in the SDL system. The menu choice is only available if a Link executable has been selected.
The generated objects use the ASN.1 syntax. They are automatically analyzed after they have been generated. This is necessary for later operations and usage of these types. If there is no other declarations (types) in the test suite (e.g. the test suite is empty), the analysis will not fail. On the contrary if other declarations (types) already exist, the analysis may fail due to name conflicts and incorrect references. The error messages of this analyzing will not be displayed. To check if the generated declarations are analyzed, use the Selector and the Show Error Message command on the incorrect tables.
At the same time that the declarations are generated, a Default table will be generated. It consists of an otherwise statement for each PCO and a timeout statement.
No timer will be generated from the SDL system. If the design of the test suite requires any timers they must be defined manually.
More details about the generated tables etc. can be found in Generating the Declarations.
To generate a test case (the behaviour description of a test case), the test case table must be in synchronized mode. In synchronized mode, the test case is synchronized (has an established connection) with the selected Link executable.
Once in synchronized mode, the test case editor will stay in this mode as long as only commands from the SDT Link menu are used. As soon as any field in the table (besides the comment fields) is edited, the synchronized mode will be terminated. It is however possible to analyze the test case without leaving the synchronized mode.
The following commands are available from a Table Editor for test cases or test steps. They are applied on a behaviour line (they insert a new behaviour line below/after the behaviour line with the input focus) hence it is required that the test case or the test step either is empty or has the input focus on a leaf row.
When each of these commands is performed the input focus is moved to the new generated behaviour line.
Adds a send statement below/after the behaviour line with the input focus. The new behaviour line will have an increased indent level compared with the previous one.
|
|
The send statement with the selected PCO, ASP/PDU and constraint will be verified by the selected SDL system. If the verification does not fail, a send statement is generated and inserted below/after the row with the input focus. For a more detailed description of this dialog, see Add Send Statement.
Adds appropriated receive and/or timeout statement(s) below/after the behaviour line with the input focus. The new row(s) will have the same indent level. This indent level will be increased compared with the previous one.
For each retrieved receive statement from the SDL system, a new Constraint is generated if there is no appropriate Constraint (a Constraint with a similar value). The new Constraint will be named with a unique name and will be analyzed. This new name is used in the Constraints Ref column.
Adds a start timer statement below/after the behaviour line with the input focus. The new behaviour line will have an increased indent level compared to the previous one.
Figure 254 : Start Timer dialog
|
|
The start timer statement with the selected timer will be verified with the selected SDL system. If the verification does not fail, a start timer statement is generated and inserted below/after the behaviour line with the input focus. For a more detailed description of this dialog see Add Send Statement.
Analogous to Start Timer.
Adds an attachment statement below/after the row with the input focus. The new behaviour line will have an increased indent level compared to the previous one.
The test step dialog used by this command is similar to the dialog in the Table Editor. The selected test step (the behaviour lines in the behaviour description) will be verified with the SDL system. The test step must have passed analysis before this operation.
Verifies the test case in a Table Editor using the previously chosen Link executable. The table will change mode to the synchronized mode. This command is available from Table Editors for test cases and only if an SDL system is selected.
If the test case does not have any default reference and there is more than one default in the test suite, a selection dialog pops up and a default must be selected. If the test suite contains only one default, it will be selected automatically. If the test case already has a default, no change will be made.
If the test case (or the test step) contains behaviour lines, they will be verified with the current SDL system. If the verification of any line fails, the table will keep the normal mode.
Opens the SDL Editor with the symbols selected which were executed in the SDL system and are associated with the behaviour line which has the input focus. More precisely, the SDL symbols which were executed after the current test case line but before the next test case line, are selected exactly.
Opens the MSC Editor with a process level MSC that illustrates the execution path from the start of the SDL system to the state corresponding to the behaviour line with input focus.
Opens the Coverage Viewer that displays test coverage information for the current test case. The test coverage displays how many times each symbol in the SDL system has been executed during the generation of the test case. Note that the important information is not the exact number of times a particular symbol has been executed (since this is dependent upon the particular algorithm used by the Link executable). The important information is whether a symbol has been executed or not. If a symbol in the SDL system has not been executed when generating the test case, the requirement defined by this symbol is not tested by the test case.
Shows the current settings of the configuration parameters that control the way the Link executable explores the state space of the combined SDL/TTCN system.
In Windows, different TTCN Link commands are included in the SDT Link menu and the Link dialog. The menu choices in the SDT Link menu are used for selecting the Link executable, generating declarations and for showing execution information. By using the Link dialog, you can generate various statements.
The SDT Link menu and the Link dialog will be explained below.
The following menu choices are included in the SDT Link menu:
Makes a connection between a test suite and the corresponding SDL system. Opens a dialog in which you may select the Link executable.
It is also possible to specify the Link executable in the Organizer by associating the SDL system with the TTCN system. However, a Link executable selected in ITEX will override an executable selected in the Organizer.
See also External synonyms.
Generates TTCN versions of the relevant type declarations in the SDL system. The menu choice is only available if a Link executable has been selected.
The generated objects use the ASN.1 syntax. They are automatically analyzed after they have been generated. This is necessary for later operations and usage of these types.
At the same time as the declarations are generated, a Default table will be generated. It consists of an otherwise statement for each PCO and a timeout statement.
Timers will not be generated from the SDL system. If the design of the test suite requires any timers, they must be defined manually.
Opens an SDL Editor with the symbols selected which were executed in the SDL system and are associated with the selected behaviour line. More precisely, the SDL symbols which were executed after the current test case line but before the next test case line, are selected exactly.
Opens an MSC Editor with a process level MSC that illustrates the execution path from the start of the SDL system to the state corresponding to selected the behaviour line.
Opens an SDT Coverage Viewer that displays test coverage information for the current test case. The test coverage displays how many times each symbol in the SDL system has been executed during the generation of the test case. Note that the important information is not the exact number of times a particular symbol has been executed (since this is dependent upon the particular algorithm used by the Link executable). The important information is whether a symbol has been executed or not. If a symbol in the SDL system has not been executed when the test case was generated, the requirement defined by this symbol is not tested by the test case.
Shows the current settings of the configuration parameters that control the way the Link executable explores the state space of the combined SDL/TTCN system.
The Link dialog can be opened from the SDT Link menu. The Link dialog can only be used when a behaviour line is selected in the table or if the table does not yet contain a behaviour line. When the Link dialog is opened, the current table automatically becomes synchronized with the Link executable, that is, the table is read-only. The only time synchronization is lost for the current table, is when TTCN Link is activated in another table or when you insert a behaviour line from the Data Dictionary dialog in the current table.
The Link dialog has almost the same appearance as the Data Dictionary dialog. A list in the lower left corner of the dialog makes it possible to switch between the Link and Data Dictionary dialog. For more information about the Data Dictionary, see Creating Behaviour Lines.
The operations available in the Link dialog will be described below. The operations are applied on a selected behaviour line and the result is that a new behaviour line is inserted below/after. The test case or the test step must be empty or a leaf row must be selected.
Generate a send statement by selecting a PCO, ASP/PDU, constraint, etc. When you press the Apply button, the statement will be verified by the selected SDL system. If the verification succeeds, a new send statement will be inserted below the selected behaviour line.
Figure 255 : Generating send statements
|
|
Generate receives and/or timeout statement(s) below the selected behaviour line by clicking the Generate Receives button. The new row(s) will all have the same indent level. This indent level will be increased compared with the previous one.
This operation is only valid when the current selected behaviour line is a leaf row.
For each retrieved receive statement from the SDL system, a new Constraint is generated if there is no appropriate constraint (a constraint with a similar value). The new constraint will be named with a unique name and will be analyzed. This new name is used in the Constraints Ref column.
Generate a timer statement by selecting Start or Cancel and a timer from the listbox (optional for Cancel). When you click the Apply button, the timer statement will be verified against the SDL system. If the verification succeeds, a timer statement will be generated and inserted below the selected behaviour line. The new behaviour line will have an increased indent level compared to the previous one.
Figure 256 : Generating timer statements
|
|
When you select an attachment and click Apply, an attachment statement will be generated below the selected behaviour line. The new line will have an increased indent level compared to the previous one.
The selected test step (the behaviour lines in the behaviour description) will be verified with the SDL system. The test step must have passed analysis before this operation.
|
Note: TTCN Link does not support attachment parameters. |
The generation of a TTCN test suite with Autolink proceeds in several steps.
Figure 257 : Test suite generation with Autolink
|
|
You start by specifying an SDL system, see Specifying the SDL System and Performing Other Preparations. Based on this SDL specification, you generate a Validator application which includes Autolink.
Next, you define paths, see Defining Paths for Test Case Creation, which describe the purpose of the test cases of your test suite. These paths are stored on disk as system level MSCs, that is, MSCs with only one instance axis for the SDL system and one or more instance axis for the environment. Test cases may contain test steps which are stored as separate system level MSCs on disk.
You may also want to define an Autolink configuration, see Defining an Autolink Configuration, in order to guide the naming and parameterization of constraints. Autolink can also be configured to store test cases into a hierarchy of test groups.
The next step is to generate test cases, see Translating MSCs into Test Cases. This can be done either by a state space exploration of the SDL system or by directly translating system level MSCs into test cases. In any case, the result is an internal representation for each test case. At the same time, a list of constraints is generated. These constraints can be renamed and merged, see Modifying Constraints. You can also add new constraints manually.
Finally, you generate a TTCN test suite, see Generating a TTCN Test Suite, based on the test case representations and the list of constraints. The test suite is stored on disk in TTCN-MP format. You can import this file in ITEX on UNIX, simply open it in ITEX in Windows or convert it to the graphical TTCN format in the Organizer.
On the following pages, the steps will be described in more detail.
Before you start the Validator, the directories to store test case and test step representations in, have to be created. If there are no appropriate directories:
You have to specify an SDL system in order to create a Validator. At minimum, you must specify all channels to the environment of your system and all signals sent via these channels. With such a minimal specification, you can use Autolink to translate MSCs directly into TTCN by using the Translate-MSC-Into-Test-Case command. The advantages and disadvantages of using this command are described in Translating MSCs into Test Cases.
When you have specified the SDL system, you can generate and start the Validator. How to do this is described in Generating and Starting a Validator.
Before you start defining test cases and test steps, you have to specify where they are to be saved:
When you later leave the Validator, you can save these values.
In Autolink, a test case is derived from a path. A path is a sequence of events that have to be performed in order to go from a start state to an end state. In practice, this means that you:
Define-Root Current command.
|
Note: Autolink always considers the current root of the behavior tree to be the start state of a path. Also note that when a test case is generated, the root has to be the same as it was at the moment of the test case definition. You have to keep track of the start state with Print-Path and Goto-Path, for example if you want to leave the Validator temporarily. |
An MSC test case consists of one instance axis for the SDL system and a separate instance axis for each channel to/from the environment.
The system level MSC that will be saved contains the observable events of the path between the start and the end state. Observable events represent the external interaction that takes place between the SDL system and its environment. (During conformance testing, external interaction takes place between the implementation and the test system.)
Figure 258 shows an MSC test case.
Typically, test cases are structured logically into several parts, for example a preamble, a test body and a postamble. These parts are called test steps. You may incorporate test steps in a test case by using MSC references.
Figure 258 shows an MSC with two MSC references. Each of the referenced MSCs represents a separate test step; they are called Preamble and Postamble.
Figure 258 : An MSC with a preamble and a postamble
|
|
|
|
|
|
Test steps are stored with the same file extension as test cases (.mpr). They are created in analogy to test cases with Save-MSC-Test-Step.
If you want to create a test case with a preamble and a postamble, several steps are necessary:
Alternatively, you may create a single MSC test case and split the file into preamble, test body and postamble afterwards.
Test steps may refer to other test steps, but not to test cases. During test case generation, Autolink keeps track of the nested structure of test cases and test steps.
|
Note: When Autolink generates test cases (see Generate-Test-Case and Translate-MSC-Into-Test-Case), the semantics of MSC references is different from the semantics given in the ITU-T Recommendation Z.120! Autolink requires that a test step is completely evaluated before the next test step starts, whereas Z.120 considers MSC references as macros which do not have to be evaluated as a unit. With regard to Figure 258, this means that all signals of the test body in |
The Power Walk exploration algorithm is an efficient mechanism in the Validator to specify a large set of test case descriptions automatically. The algorithm is designed to generate a report whenever a sequence of transitions in the state space is found which increases the number of visited SDL symbols. These reports can be converted into system level MSCs with the command Save-Reports-as-MSC-Test-Cases.
For more information about the Power Walk algorithm, see Coverage Based Test Generation.
Autolink offers a special language for defining rules for the naming and parameterization of constraints, the introduction of test suite parameters and constants, and the distribution of test cases and test steps into test groups.
While it is possible to define an Autolink configuration on the fly at the Validator command prompt, it is better to write a command file which includes the configuration.
See also:
Once you have defined a set of MSC test cases, you can compute internal test case representations for each MSC test case.
Either all test cases in the current test cases directory or just a single one may be generated, depending on the parameter of Generate-Test-Case. If an already generated test case is regenerated, its former internal representation and its corresponding constraint definitions are replaced.
|
Note: If the SDL specification is not detailed enough in the sense that it does not model the signal flow in a given MSC, the generation of the test case fails. MSCs which cannot be verified can still be converted into test cases with the Translate-MSC-Into-Test-Case command, which is described in Translating MSCs into Test Cases. Inline expressions and HMSC diagrams are not supported by Autolink. If you want to create test cases based on HMSCs, you have to transform your HMSC diagrams into basic MSCs first. You can do this by verifying the HMSCs and saving all MSC Verification reports as MSC test cases (command Save-Reports-as-MSC-Test-Cases). |
With complex systems, test generation may take a while. To avoid time-consuming test generation failures, you should verify all MSCs first.
This makes it possible to distribute the generation of a set of test cases to several computers. When all test cases have been generated and saved in individual files, they can be reloaded on a single machine and saved as a complete test suite.
There are several parameters which influence the state space exploration:
Define-Scheduling All Define-Transition Symbol-Sequence Define-Symbol-Time Zero Define-Priorities 2 1 3 2 2 Define-Channel-Queue ISAP1 On Define-Channel-Queue MSAP2 On Define-Max-Input-Port-Length 10Other SDL systems require different Define-Channel-Queue commands.
Off command. This disables the automatic setting of Autolink's default options.After the generation of a test case, some warnings or error messages may be displayed:
Error: Autolink did not find any complete pass paths. No test case is generated.Warning: Incomplete pass path found.
First event gets `INCONC' verdict.Warning: Alternative events found for send event.
Inconclusive events are deleted.X1 1 X2 X3 X4, where Xi > 1.Warning: No translation rule can be applied to the following signal: <Signal Name>If an SDL system is not fully specified, some or all MSCs may not be converted into test cases if you use the Generate-Test-Case command. Instead, you can use the Translate-MSC-Into-Test-Case command to translate MSCs directly into the same internal test case format which is used for test cases generated by state space exploration. Therefore, you can use all commands related to listing, displaying, removing, saving and loading of test cases (introduced in Computing Test Cases) with directly translated test cases as well.
Furthermore, all rules for constraint naming (see Translation Rules) and test grouping (see Test Suite Structure Rules) apply to translated test cases, too.
Either all test cases in the current test cases directory or just a single one may be translated, depending on the parameter of Translate-MSC-Into-Test-Case. If an already translated test case is retranslated, its former internal representation and its constraint definitions are replaced.
|
Note: Inline expressions and HMSC diagrams are not supported by Autolink. |
After the generation of a test case, some warnings or error messages may be displayed:
Error: No test case could be generated.
Please check whether the MSC contains separate instance axes for each channel to the environment.Warning: No translation rule can be applied to the following signal: <Signal Name>It is highly recommended that you specify constraints naming and parameterization rules in an Autolink configuration file. Otherwise, a generic name of the form <Test case name>_<three digit number> is assigned to each constraint during test case generation. However in the latter case, you will probably find it useful to modify the constraints generated by Autolink.
|
Note: A test step that is used in several places may lead to trees with different inconclusive events or different verdicts. In this case, they will be given new, unique names. |
Test cases that are structured into preamble, test body and postamble will automatically be assigned preliminary pass verdicts at the end of the test body. However, test cases can contain an arbitrary number of MSC references (and hence test steps). Therefore, preliminary pass verdicts will be assigned to all events that are directly followed by the last top-level MSC reference in the test case. The preliminary pass verdicts will only be assigned if no event follows the last MSC reference. The event to which a preliminary pass verdict is assigned may appear within the test body as well as within a test step.
During the saving of a test suite, some warnings or error messages may be displayed:
Warning: Test step <TS> resulted in different trees.
The trees are renamed to `<TS>_1', `<TS>_2', etc. in the test suite.Warning: No test suite structure rule defined for test case `<Test Case Name>'.Warning: Test suite parameter/constant `<Name>' is not unique.
It is renamed to `<Name>_1', `<Name>_2', etc. in the test suite.In Modifying Constraints, you have learned how to change constraints. However, assigning sensible names to automatically generated constraints is a tedious task. Especially if you have to refine the SDL specification and then to repeat the test generation process, there is a lot of manual work. Moreover, the number of generated constraints may become very large if you do not use constraint parameterization.
In order to address these problems and some additional issues, you can specify so-called translation rules. These rules control the look of a test suite with regard to the following items:
Translation rules build one integral part of an Autolink configuration (see also Test Suite Structure Rules). Before you start the test generation, you can develop an Autolink configuration file that contains a Define-Autolink-Configuration command. The set of translation rules which tell Autolink how to construct constraints and treat parameters for particular signals, are provided as a kind of long parameter to this command.
For some examples, see Examples of Translation Rules. More information can be found in Defining an Autolink Configuration and Syntax and Semantics of the Autolink Configuration.
A typical translation rule may look like this:
TRANSLATE MDATind
CONSTRAINT NAME "C_" + $0
PARS $1="Type"
END
Example 117 explains how signal MDATind is translated into an suitable TTCN constraint. The rule above states that the name of a constraint for signal MDATind consists of the concatenation of text "C_" and the "nullth" parameter -- which is the name of the signal itself. Therefore, signal MDATind is translated into a constraint called C_MDATind.
Additionally, the first parameter of the signal (referred to by $1) becomes a parameter of the constraint. The name of the formal parameter is Type. It is printed both in the Constraint Name line and the Constraint Value section of the constraint declaration table. The actual parameter of the constraint is printed in the dynamic behavior table of each test case that uses this constraint.
A constraint declaration table for signal MDATind is shown in Figure 259.
Figure 259 : A constraint declaration with a formal parameter
|
|
It is also possible to define a single translation rule for more than one signal. This is especially useful if similar signals exist which can be treated in the same way.
TRANSLATE MDATind | MDATreq
CONSTRAINT NAME "C_" + $0
PARS $1="Type"
END
If either MDATind or MDATreq is identical to the signal for which a constraint is to be created during test generation, the rule in Example 118 is applied. The value of $0 depends on the name of the actual signal investigated at run-time. Since the first signal parameter is always to be replaced by the formal parameter Type, the rule is only valid if each of the alternative signals, i.e. MDATind, and MDATreq,has at least one parameter. When parsing an Autolink configuration, all translation rules are checked automatically for validity.
Constraint names may not only be based on texts and signal names. They can also depend on signal parameters. In a translation rule, a signal parameter is referred to by its number, prefixed with a dollar character ($). (Note that Autolink only supports parameters on the top level -- it is not possible to refer to a component of a nested parameter.)
In some cases, it is not desirable to use the value of a signal parameter directly as part of a constraint name. For example, a protocol engineer might encode complicated signal information with abbreviations or numbers. But for the TTCN output, parameter values should be mapped onto more meaningful expressions.
Therefore, you may define functions which take an arbitrary number of parameters and map them onto text. In Example 119, the value of the first parameter of signal MDATind is passed to function PDUType. Depending on the concrete parameter value, which occurs during test case generation, the function returns a text. This text forms the second part of the constraint name.
TRANSLATE MDATind
CONSTRAINT NAME "Medium_" + PDUType($1)
END
FUNCTION PDUType
$1 == "CR" : "Ind_Connection_Request"
| $1 == "AK" : "Ind_Acknowledge"
| $1 == "DR" : "Ind_Disconnection_Request"
| TRUE : "Indication"
END
|
Note: In a translation rule, |
You may define complex rules whose evaluation is guarded by conditions. This is illustrated in Example 120.
TRANSLATE "MDATind"
IF $1 == "CR" THEN
CONSTRAINT NAME "Medium_Connection_Request"
END
IF $1 == "AK" AND $2 == "zero" THEN
CONSTRAINT NAME "Medium_Acknowledge_Zero"
END
CONSTRAINT NAME "Medium_Indication"
PARS $1="Type"
END
Conditional translations can be defined by IF-statements. Only if the condition(s) following the IF keyword is/are satisfied, the constraint is built according to the subsequent specification. A translation rule can contain several IF-clauses. The first clause which condition is satisfied (or which does not have an IF statement at all) is chosen for translation.
In the example above, signal MDATind is translated into a constraint called Medium_Connection_Request if the first signal parameter equals CR, and to a constraint called Medium_Acknowledge if the first two signal parameters equal AK and zero respectively. If neither condition is satisfied, the unconditioned section is evaluated. In this case, a constraint with name Medium_Indication and formal parameter Type is created. Note that the parameter definition is not taken into account if any of the former IF-conditions is satisfied!
Sometimes, it is useful to indicate that a specific signal parameter is irrelevant. For example, assume that if the first parameter of signal MDATind is CR, the values of the second and third parameter can be ignored. Hence, you can replace them by wildcards in a constraint table. In Example 121, a MATCH statement is added that tells Autolink to replace the values of the signal parameters 2 and 3 by asterisks. The resulting constraint table is displayed in Figure 260.
TRANSLATE "MDATind"
IF $1 == "CR" THEN
CONSTRAINT NAME "Medium_Connection_Request"
MATCH $2="*", $3="*"
END
CONSTRAINT NAME "Medium_Indication"
PARS $1="Type"
END
|
Note: The application of TTCN matching mechanisms is only valid for receive events. Hence, you are not allowed to apply the |
Figure 260 : A constraint table with TTCN matching expressions
|
|
Translation rules also allow to introduce test suite parameters and constants. Test suite constants are useful if a concrete parameter value does not give any clues about its meaning and hence should be replaced globally by a more meaningful name. Test suite parameters should be introduced if signal parameter values are implementation dependent. By defining a test suite constant/parameter, a concrete signal parameter value in a constraint table is replaced by a symbolic constant. The assignment of concrete values to symbolic test suite constants/parameters is made in additional TTCN tables which are created automatically by Autolink.
Example 122 illustrates the use of test suite parameters and constants. If the condition is satisfied, the second signal parameter is replaced globally by SeqNo in the TTCN test suite. The third signal parameter is replaced by a test suite parameter called DataValue. This parameter refers to PICS/PIXIT proforma entry PICS_Data.
If signal MDATreq has not been used for data transfer, the value of the first signal parameter is replaced by a test suite constant which name is based on the concrete signal parameter value. A constraint table and an according constant table for this case is shown in Figure 261 and Figure 262.
TRANSLATE "MDATreq"
IF $1 == "DT" THEN
CONSTRAINT NAME "Medium_Req_Data_Transfer"
TESTSUITE CONSTS $2="SeqNo"
PARS $3="DataValue" / "PICS_Data"
END
CONSTRAINT NAME "Medium_Req_" + PDUType($1)
MATCH $3="*"
TESTSUITE CONSTS $1=PDUType($1)
END
FUNCTION PDUType
$1 == "CC" : "ConConf"
| $1 == "AK" : "Acknowledge"
| $1 == "DR" : "DisconRequest"
| $1 == "DT" : "DataTransfer"
| $1 == "CR" : "ConRequest"
END
Figure 261 : A constraint table with a test suite constant
|
|
Figure 262 : A TTCN table for test suite constant declarations
|
|
An Autolink configuration typically consists of a large number of translation rules which are evaluated from top to bottom. If a constraint cannot be constructed based on the given rules, a generic name will be assigned to the constraint, in the same way as when no translation rules are defined.
In TTCN, test cases can be combined in test groups. Each test group aims at testing the system under test for one particular aspect. Test groups again can be part of other higher level test groups, resulting in a hierarchy of test groups.
Test steps can be put into test groups as well. In the following, test cases and test steps will not be distinguished, as test structure rules apply to both.
When you start designing a test suite, you should have a clear notion of what the structure of the test suite will be. In fact, for successful test suite development, it is important to first determine what should be tested and how the tests can be classified, before individual test cases are specified.
If you use Autolink for test generation, the test cases are described by MSCs. Ideally, the names of the MSCs should give information about the structure of the resulting test suite. Because of this, you may specify rules for the automatic placing of test cases in different test groups, depending on the names of the corresponding MSCs. These test suite structure rules prevent you from repeating a lot of manual work if you regenerate the test suite due to a modification of the underlying SDL specification. Moreover, test suite structure rules (TSS rules) also save you a lot of work if you create a test suite only once, since a single rule can be applied to several test cases. As will be shown in the example below, one rule may be enough to describe the structure of a complete test suite.
Test suite structure rules are part of an Autolink configuration. Before the test generation starts, you can write an Autolink configuration file which contains a Define-Autolink-Configuration command. The TSS rules are provided as a kind of long parameter to this command.
For details on the Autolink configuration commands see Defining an Autolink Configuration. A precise description of the Autolink configuration language is given in Syntax and Semantics of the Autolink Configuration.
In the following, it is assumed that you want to create a test suite in which test cases can be classified according to three different criteria. On the top level, tests can be distinguished by whether they are related to mandatory or optional capabilities. On the next level, tests may focus on particular protocol phases, for example connection establishment, data transfer and disconnection. Finally, valid, invalid or inopportune behavior may be displayed. A resulting test suite should have the following structure:
Mandatory
Connection
Valid
Invalid
Inopportune
DataTransfer
Valid
...
Disconnection
...
Optional
...
It is further assumed that having this structure in mind, you have created MSC test cases with the following names:
V_Con_Man_01 V_Dis_Man_01 IV_Data_Man_01 IO_Data_Opt_01 IO_Data_Opt_02
MSC test cases that belong to the same test group are numbered sequentially.
Now, a simple TSS rule for the scenario above may look like this:
PLACE V_Con_Man_01 IN "Mandatory" / "Connection" / "Valid" END
Example 123 states that test case V_Con_Man_01 is intended to be placed in the test group Valid. Since this test group is placed in another test group (Connection), you have to specify the complete path, composed of all groups in hierarchical order. The names of the test groups are separated by a slash (`/') in analogy to the notation of test group references in the TTCN standard.
If you want to place several test cases in the same test group, you can use the alternative operator (`|') in the header of a TSS rule:
PLACE IO_Data_Opt_01 | IO_Data_Opt_02 IN "Optional" / "DataTransfer" / "Inopportune" END
Example 124 places both IO_Data_Opt_01 and IO_Data_Opt_02 in test group Optional/DataTransfer/Inopportune.
Rules like the one shown in Example 123 and Example 124 can be applied to MSC test cases with arbitrary names. In the best case, you have to write one TSS rule for each test group.
However, there is a direct relation between the MSC names and the test groups. For example, the two characters IO at the beginning of an MSC name indicate that the corresponding test case has to be placed in a test group called Inopportune. Using this information, the number of TSS rules can be further reduced as explained below.
You are allowed to use regular expressions in the header of a test suite structure rule. The following characters have a special meaning when used in the header:
*' matches zero or more arbitrary characters.!', any character not enclosed is matched.
|
Note: Regular expressions can also be used in a similar way in the header of translation rules. This is useful if signals with similar names are to be treated equally. |
Now consider the following complex rule and its auxiliary functions:
PLACE "*" + "_" + "*" + "_" + "???" + "_" + "*" IN OptMan( @5 ) / Phase( @3 ) / Behavior( @1 ) END FUNCTION OptMan $1 == "Opt" : "Optional" | $1 == "Man" : "Manual" END FUNCTION Phase $1 == "Con" : "Connection" | $1 == "Data" : "DataTransfer" | $1 == "Dis" : "Disconnection" END FUNCTION Behavior $1 == "V" : "Valid" | $1 == "IV" : "Invalid" | $1 == "IO" : "Inopportune" END
With the rule in Example 125, all test cases can be placed in their appropriate test groups.
When a test suite structure rule is evaluated, it is first checked whether one of the terms following the keyword PLACE (which are separated by `|') equals the name of the investigated test case. In the rule above, there is only one term consisting of 7 parts, called atoms. These atoms are concatenated by the `+' operator.
While Autolink simply has to compare strings in Example 123 and Example 124, it has to find out whether a concrete test case name matches the regular expression in Example 125. If the test case name matches the regular expression, the atoms in the header of the TSS rule are instantiated.
If, for example, the rule is applied to test case IO_Data_Opt_01 at run-time, the first atom (originally `*') is set to "IO". The value of the third atom becomes "Data", the value of the fifth atom becomes "Opt" and the value of the seventh atom becomes "01". The second, fourth and sixth atom remain unchanged as they do not contain any special characters.
In order to refer to the value of an atom in the rule header, you can use the "at" operator (`@'). For example, "@5" refers to the value of the fifth atom.
Additionally, you may define functions which map parameters onto texts. In Example 125, "@5" is passed to function OptMan. Depending on the concrete parameter value which is passed at run-time, the function returns either the text "Optional" or "Manual" (or an error message if the first function parameter is neither "Opt" nor "Man").
An Autolink configuration typically consists of a number of TSS rules which are evaluated from top to bottom. If a test case or a test step cannot be placed in a test group based on the given rules, Autolink places it on top-level and prints an error message. In this case, you can modify your rules, reload them and apply the Save-Test-Suite command again.
The definition of an Autolink configuration is started with the keyword Define-Autolink-Configuration and is terminated with End. It consists of a sequence of translation rules, test suite structure rules and functions.
<Start> ::= "Define-Autolink-Configuration"
<Configuration>
"End"
<Configuration> ::= { <TransRule> | <TSStructureRule> |
<Function> }*
|
Note: If you want to define both translation rules and test suite structure rules, you have to place them in the same configuration definition. |
Rules and functions can be arbitrarily mixed in a configuration. There is no need to place rules on top of a file, nor do you have to write forward declarations for functions.
Translation rules are evaluated whenever a constraint is created during test case generation.
A translation rule starts with the specification of the names of the signals to which it shall apply (denoted by <AlternativeListOfTerms>).
<TransRule> ::= "TRANSLATE"
[ "SIGNAL" ] <AlternativeListOfTerms>
<TransRuleIf>* [ <TransRuleNoIf> ]
"END"
<TransRuleIf> ::= "IF" <Conditions> "THEN"
<TransRuleNoIf> "END"
<TransRuleNoIf> ::= { "CONSTRAINT" <TransRuleConstraint> |
"TESTSUITE" <TransRuleTestSuite> }*
<TransRuleConstraint> ::= { "NAME" <Term> |
"PARS" <ParameterList1> |
"MATCH" <ParameterList1> }*
<TransRuleTestSuite> ::= { "CONSTS" <ParameterList1> |
"PARS" <ParameterList2> }*
<ParameterList1> ::= <Parameter1> { "," <Parameter1> }*
<Parameter1> ::= "$" <Number> [ "=" <Term> ]
<ParameterList2> ::= <Parameter2> { "," <Parameter2> }*
<Parameter2> ::= "$" <Number> [ "=" <Term> ]
[ "/" <Term> ]
As sketched in the example section, translations can be made dependent on one or more conditions. Hence, the body of a translation rule may consist of one or more statements embedded by IF ... THEN ... END constructs. The first group of statements whose preceding conditions are satisfied (or which do not have an IF statement at all) is evaluated. All subsequent definitions are ignored. If no conditions hold for a given signal, Autolink looks for another translation rule which fits the signal.
There are two groups of directives starting with either the keyword CONSTRAINT or TESTSUITE.
In the CONSTRAINT part you can specify the name (keyword NAME) and the formal parameters of a constraint (keyword PARS) for one or more given signals. Additionally, you can tell Autolink to replace signal parameter values by a TTCN matching mechanism (keyword MATCH). Please note that Autolink does not perform any checks concerning matching mechanisms at run-time. It simply handles it as a textual replacement.
In the TESTSUITE part, you can specify that parameter values of a signal are replaced by test suite parameters and constants. The declaration of constants is preceded by the keyword CONSTS, test suite parameter are introduced with PARS.
It is possible to declare a constraint parameter and a test suite constant/parameter for the same signal parameter. However, Autolink ensures that a signal parameter is not mapped onto a test suite constant and parameter at the same time.
There exist several default values that are used when an optional parameter is not specified:
'c' + $0, i.e. the signal name is prefixed by a 'c'. Par3 for the third parameter).MATCH, but no term is given, its values is replaced by `*' in the constraint table.If there are name clashes, test suite constants and parameters are treated similar to constraints and test steps. That means, if there are two constants with the same name but different values, they are distinguished by a sequence number.
Test suite structure rules are similar to translation rules. They share most of the basic concepts, for example terms, functions and conditions. However, while translation rules are applied during test case generation, TSS rules are evaluated when you save a test suite with the Save-Test-Suite command.
A test suite structure rule starts with the specification of the names of the test cases to which it shall apply (denoted by <AlternativeListOfTerms>).
Conditions can be used in the same way as in translation rules: The first IN statement whose preceding conditions are satisfied (or which is not embedded in an IF ... THEN ... END statement at all), is taken into account. All subsequent statements are ignored. If no conditions hold for a given test case/step, Autolink looks for another TSS rule that fits the test case/step.
<TSStructureRule> ::= "PLACE" <AlternativeListOfTerms>
<TSStructureRuleIf>*
[ <TSStructureRuleNoIf> ]
"END"
<TSStructureRuleIf> ::= "IF" <Conditions> "THEN"
<TSStructureRuleNoIf> "END"
<TSStructureRuleNoIf> ::= "IN" <Term> { "/" <Term> }*
Functions are identified uniquely by their names. If there are two functions with exactly the same name, the one defined first is always evaluated.
Functions are visible globally, that is, they can be called by any constraint or test suite structure rule and other functions. References to functions are resolved at run-time. If there is a call to an unknown function, the text "FunctionXXXNotFound" is returned.
<Function> ::= "FUNCTION" <Identifier> <Mappings> "END"
<Mappings> ::= <Mapping> { "|" <Mapping> }*
<Mapping> ::= <Conditions> ":" <Term>
A function body consists of a number of mapping rules separated by `|'. Mapping rules specify the possible return values of a function. A mapping is performed if its corresponding condition(s) hold. Mappings are evaluated from top to bottom. If the conditions of all mappings fail, a function returns the text "NoConditionHoldsInFunctionXXX".
Function parameters can be accessed in the same way as signal parameters in a translation rule. For example, $2 refers to the second parameter. In the context of functions, the reference $0 denotes the name of the function. Since parameters do not have a name, but are referred to by their position instead, there is no need to declare them in the function header. If you try to access a parameter that has not been passed to the function, the missing parameter is replaced by the text "ParOutOfRange".
|
Note: In conditions, the existence of a particular parameter can be checked. For example, condition |
The only data type defined in the Autolink configuration language is text. Whether you refer to a signal parameter or call a function, the result of the operation is always a text.
<Term> ::= <Atom> { "+" <Atom> }*
<Atom> ::= "$" <Number> | "@" <Number> |
<Text> | <Identifier> |
<FunctionCall>
<FunctionCall> ::= <Identifier>
"(" <SequentialListOfTerms> ")"
<SequentialListOfTerms> ::= <Term> { "," <Term> }*
<AlternativeListOfTerms> ::= <Term> { "|" <Term> }*
<Conditions> ::= <Condition> { "AND" <Condition> }*
<Condition> ::= <Term> { "==" | "!=" } <Term> |
"TRUE"
Texts are constructed by atoms and terms. A single atom can be one of the following expressions, depending on the context in which the atom is used:
"Request").Request)."Sig*").OpName($3)).$2).$2).@2).Since an atom always evaluates to a text and a term is a concatenation of single atoms, you are allowed to use term expressions for the specification of:
|
Note: The text obtained by referring to a signal parameter is identical to the output of the signal parameter value in ASN.1 format. |
A condition checks whether two texts are equal (==) or unequal (!=). There is also a special condition TRUE that always evaluates to true.
Conditions can be combined by AND. Only if all conditions in a conjunction hold, the expression as a whole is true.
There is no OR operator for combination of conditions. However, due to the consecutive evaluation of rules (from top to bottom), this is not a restriction. For example, in a function body, simply place both OR-operands in two subsequent mappings.
One common method to generate a test suite is to select a number of test cases which, taken together, obtain a high coverage of the SDL specification. Ideally, this coverage should be 100%. In that case, every SDL symbol in the specification is executed at least once.
The Validator provides a special state space exploration algorithm, the Power Walk algorithm, which combines exhaustive and random walk state space exploration techniques. The Power Walk algorithm is optimized to generate a set of paths with a high SDL symbol coverage. These paths in turn can be transformed into a test suite by Autolink.
When Power Walk is used, an n-level tree search will be performed, starting from the system root state. During this phase, the state space is built up to the maximum depth n.
If a complete n-level tree search does not produce a transition which increases the coverage, Power Walk switches to random walk exploration.
If the coverage cannot be increased within c transitions, the exploration stops and a report is generated. Another condition for stopping the exploration and creating a report is reaching a maximum depth m.
In both cases above, Power Walk ensures that the system is driven to a stable state before a report is generated.
Figure 263 schematically illustrates the way the power walk algorithm explores a state space.
A new Power Walk exploration is started repetitively from the system root state until one of three abort conditions holds. Abort conditions are the reaching of a coverage v, a maximum running time t or a number u of consecutive unsuccessful explorations, that is, explorations that do not increase the symbol coverage and hence do not produce new reports.
Finally, the reports generated with Power Walk can be transformed into system level MSCs with the Save-Reports-as-MSC-Test-Cases command.
Figure 263 : An example for the power walk algorithm
|
|