Test Coverage Analysis of Embedded System

Features of embedded systems: * Embedded devices have significant resource constraints; typically, less memory, less processing power and limited hardware devices compared to regular desktop computers. * Embedded systems should be more reliable and efficient. If a program in a PC fails, we can restart the program and nothing serious happens. However, for many embedded system programs, if they fail, they can make the device useless.And sometimes they can cause harm also (safety critical conditions). So test coverage should be done with extra care. C is the language of choice for most of the programming done for embedded systems . Different coverage metrics used are statement coverage, decision coverage, Modified Condition / Decision Coverage(MC/DC). To achieve statement coverage, every executable statement in the program is invoked at least once during software testing. To achieve decision coverage every statement must be invoked at last once and every decision must be evaluated as true and as false. The MC/DC criterion requires that every statement must be invoked at least once, every decision must be evaluated as true and as false, and each condition must be shown to independently affect the outcome of the decision. Since MC/DC is stronger than statement/decision coverage, requiring more test cases for full coverage. So we will use MC/DC for most of our cases. We consider the following features of embedded systems for test coverage analysis : 1. Exception Handling 2. Timer 3. Inter task communication : message passing, binary semaphore 4. Concurrency Here we are briefly discussing each of these.

1.

Exception Handling :- Most software testing efforts focus on exercising the correct operation of code, and not determining how robust it is to exceptional conditions. Therefore, exception handling code is the least tested and most susceptible to bugs. An exception is an event that disrupts the normal flow of control of instructions in a program during its execution . Here we are deriving exception handling coverage using a simple example and its control flow diagram.

D0 D1 D2 D3 D4

#include <windows.h> #include <stdlib.h> #include <stdio.h> #include <rtfex.h> #define DIVIDE_BY_ZERO -3 // an error code // some global variables to count // iterations, exceptions, and cleanups

D5 E6 S7 S8 S9 }

int Calls, Exceptions, Finallys; int SomeFunction(int a, int b) // functiom which can raise an exception { if (b == 0) Xraise(DIVIDE_BY_ZERO); return a / b;

E10 int TestFunction(int a, int b) // test function containing an XTRY block { S11 int volatile Result; S12 Calls++; S13 XTRY S14 case XCODE: S15 Result = SomeFunction(a, b); S16 break; S17 case DIVIDE_BY_ZERO: S18 Exceptions++; S19 XHandled(); S20 break; S21 default: S22 printf("unknown exception!\n"); S23 break; S24 case XFINALLY: S25 Finallys++; S26 break; S27 XENDX S28 return Result; } E29 void Bench(int a, int b, int Iterations) // benchmark function to call TestFunction in a loop // and print timing and statistics {

S30 S31 S32 S33 S34 S35 S36 S37 }

DWORD T0, T1; int i; Calls = Exceptions = Finallys = 0; T0 = GetTickCount(); for (i=0; i<Iterations; i++) TestFunction(a, b); T1 = GetTickCount(); printf("%10i %10i %10i %10u\n", Calls, Exceptions, Finallys, T1-T0);

E38 int main(void) { S39 printf("Interation Exceptions Finallys Milliseconds\n" "------------------------------------------------\n"); S40 Bench(1, 1, 1000000); // no exceptions S41 Bench(1, 0, 1000000); // raise one exception per loop S42 return 0; }

We define the exception handling coverage as the number of nodes traversed for each decision case after the try node divided by total number of nodes that are dependent on the execution of that try node.

Here in the above diagram nodes that are dependent on the execution of try node S15 are E6, S7, S8, S9, S16, S17, S18, S19, S20, S21, S22, S23, and S28 i.e 13. There are two cases that are to be considered: case 1: if exception occurs and throw node is executed, nodes traversed in this case are E6, S7, S8, S17, S18, S19, S20 and S28. So coverage = 8/13*100=61.53%

case 2: if exception does not occur,nodes traversed in this case are E6, S7, S9, S16, S28. So coverage = 5/13*100=38.46%

2.

Timer:- An embedded system is a computing environment that reacts to input within a specific time period or that performs works under a given deadline. Therefore a timer is a crucial component of most embedded systems. A time out disrupts the normal flow of control of instructions in a program during its execution . Here we are deriving timer coverage using a simple example and its control flow diagram.

D0 D1 D2 D3 D4 E4 { S5 S6 S7 S8 S9 S10 S11

MSG_Q_ID g_msgq = NULL; Wdog_ID g_wdog = NULL; int g_cruise = ERROR; float g_distance; float g_speed; int main(void) int l_monitor = ERROR; int l_varyspeed = ERROR; int l_status = ERROR; l_monitor = taskspawn(tMonitor,100,0,10000,(FUNCPTR)monitor,0,0,0,0,0,0,0,0,0,0); l_varyspeed = taskspawn(Varyspeed,100,0,10000,(FUNCPTR)vary_speed,0,0,0,0,0,0,0,0,0,0); g_wdog = wdcreate(); if(NULL!=g-wdog) { l_status = taskspawn(tCruise,50,0,10000,(FUNCPTR)cruise,0,0,0,0,0,0,0,0,0,0); } if(ERROR!=l_monitor && ERROR!=l_varyspeed ) g_msgq = msgQCreate(50,4,MSG_Q_FIFO); return 0; } void vary_speed(void) float vel; while (true) { msgQReceive(g_msgq,(char*)&vel,50,WAIT_FOREVER); if(-1==vel) increase_acc(); elseif(1==vel) decrease_acc(); } } void monitor(void) { float speed; int flag=0; while(true) { speed = get_speed(); if(speed>75) { flag = 1; msgQSend(g_msgq,(char*)&flag4,WAIT_FOREVER,MSG_PRI_NORMAL); } elseif(speed<25) { flag = -1; msgQSend(g_msgq,(char*)&flag4,WAIT_FOREVER,MSG_PRI_NORMAL); } else flag = 0; } } void Handle Timeout(int val) { if(1==val) { g_cruise = ERROR; } } void cruise(void) { while(true) { l_status=wdstart(g_wdog,TIMEOUT,(FUNCPTR)HandleTimeout,1); if(OK==l_status) { if((g_distance>30) && (g_speed>40)) { g_cruise = OK;

S12 S13 S14 S15 E16 { S17 S18 S19 S20 S21 S22 S23 E24 S25 S26 S27 S28 S29 S30 S31 S32 S33 S34 S35 E36 S37 S38 S39 S40 S41 S42 S43 S44

} S45 } } } wdcancel(g_wdog);

We define the timer coverage as the number of nodes traversed for each decision case in the timeout handler function of timer divided by total number of nodes in the timeout handler function of timer. Here in the above diagram total nodes in timeout handler function of timer are E36, S37, S38. Two cases arise due to decision node S37: case 1:if 1==val is true, then nodes traversed in this case are E36, S37 and S38. So coverage = 3/3*100=100% case 2: if 1==val is false, then nodes traversed in this case are E36 and S37. So coverage = 2/3*100=66.67%

3.
Inter task communication:- Embedded systems manage sharing data and different resources among multiple tasks. It is usually "unsafe" for two tasks to access the same specific data or resource simultaneously. So proper communication among tasks is required for embedded systems. There are two common approaches to resolve this problem: a) message passing b)semaphores . Semaphores can be binary or counting. Here for our purpose we are considering binary semaphores only.

a)Message Passing :- Embedded programs use messages queues for inter task communication. A variable number of messages, each of
variable length, can be sent to any task. In this paradigm, the resource is managed directly by only one task. Referring to the above program,we are deriving message passing coverage using this control flow diagram.

We define the message passing coverage as the number of nodes traversed for each decision case after the message sending node divided by total number of nodes that depends on the sending node i.e nodes of those receiving blocks which have the same message queue id as that of the sending node. Here in the above diagram total nodes that depends on the sending node are S19, S20, S21, S22 and S23. Three decision cases arises in this case: case 1:if -1==vel, then nodes traversed in this case are S19, S20 and S21. So coverage = 3/5*100=60% case 2:if 1==vel, then nodes traversed in this case are S19, S20, S22 and S23. So coverage = 4/5*100=80% case 3:if vel!=1 or -1, then nodes traversed in this case are S19, S20 and S22. So coverage = 3/5*100=60%

b)Semaphore :- This mechanisms involve system calls, and usually invoke the OS's dispatcher code on exit, so they typically take hundreds of CPU instructions to execute, while masking interrupts may take as few as one instruction on some processors. Here we are deriving semaphore coverage using a simple program and its control flow diagram.
D0 D1 D2 D3 D4 D5 E6 #include "vxWorks.h" #include "semLib.h" #define T_HI_PRIORITY 20 #define T_LO_PRIORITY 200 SEM_ID semMutex; char alphabet[27]; void initialize (void)

{ S7 semMutex = semBCreate (SEM_Q_PRIORITY, SEM_FULL);

S8 S9 E10 E11 S12 S13 S14 S15 S16 E17 S18 S19 S20

taskSpawn ("hi_priority_task", T_HI_PRIORITY, 10000, tHiPriorityTask, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0); taskSpawn ("lo_priority_task", T_LO_PRIORITY, 10000, tLoPriorityTask, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0); void tHiPriorityTask (void) { int i; SemTake (semMutex, WAIT_FOREVER); for (i= 0; i < 26; i++) alphabet[i] = 'A' + i; alphabet[i] = '\0'; semGive (semMutex); } void tLoPriorityTask (void) { semTake (semMutex, WAIT_FOREVER); printf ("alphabet= %s ", alphabet); semGive (semMutex); }

We define the binary semaphore coverage as the number of nodes traversed for each decision case after the semaphore releasing node divided by total number of nodes that depends on a particular semaphore releasing node. Here in the above diagram total nodes that depends on the semaphore releasing node are S18, S19 and S20. Since in this example there is no decision case so the binary semaphore coverage = 3/3*100=100%

4. Concurrency :#include <unistd.h> #include <sys/type.h> #include <errno.h> #include <stdio.h> #include <stdlib.h> #include <pthread.h> #include <string.h> void print_message_function(void *ptr) { thdata *data; int i=0; data = (thdata *) ptr; for(i=0;i<50;i++) printf(thread %d says %s \t, data->thread_no, data->message); pthread_exit(0); } typedef struct str_thdata { int thread_no; char messaga[100]; }thdata; int main() { pthread_t thread1, thread2; thdata data1, data2; data1.thread_no=1; strcpy(data1.message, Hello!); data2.thread_no=2; strcpy(data2.message, Hi!); pthread_create (&thread1, NULL, (void*) &print_message_function, (void *) &data1); pthread_create (&thread2, NULL, (void*) &print_message_function, (void *) &data2); pthread_join(thread2, NULL); pthread_join(thread1, NULL); exit(0); }