Introducing

Microsoft Visual Studio 2005

Team System Web Testing
Acknowledgment
This article is based upon articles written by Mark Michaelis and Josh Christie in MSDN.
Introducing Microsot !isual Studio "##$ Team System %eb Testing 2
Contents
Introduction
&ecording a %eb Test
'sing the %eb Test !iewer to !eriy a %eb Test
(inding Test Data to a Data Source
)enerating %eb Test Code
*+tending !STS %eb Testing
Debugging Common %eb Test ,roblems
Conclusions
Introduction
This article pro-ides an o-er-iew o testing web application unctionality using !STS. It begins by pro-iding a
step.by.step approach on how to set up a %eb test case and customi/e it without writing any code. This article
demonstrates an approachability o !STS %eb Testing by all those participating in the de-elopment process0
including non.de-eloper types. %eb test cases can easily be coded as well0 and we will describe how to use
coded %eb tests or e+tend the built.in %eb testing support.
(eore we begin0 readers should be aware that !STS unctionality is not targeted at testing the user interace. It
doesn1t run Ja-aScript in a webpage or -alidate the appearance o a page within multiple browsers. &ather0 the
testing approach is to e+amine 2TT, data 3owing o-er the wire and pro-ide -arious rules or -alidating this
data.
Process Of Web Testing
4igure 5
Recording a Web Test
!STS %eb Testing supports acti-ity recording0 as long as there e+ists a website to record against. To record
acti-ities o a web application0 we can create a new test pro6ect or use an e+isting one. Adding a web test case
is more speci7c to web testing. %hen it is added0 an acti-ity recorder is started in which we can browse the web
application like browsing through a normal browser.
8n the address bar0 we can enter the '&9 o the personal website0 including the port selected by the AS,.N*T
De-elopment Ser-er. (rowsing to this location will be recorded in the %eb Test &ecorder e+plorer bar: as would
any other '&9s that are entered. 8nce the desired tests ha-e been recorded0 we can close the browser windows
and sa-e the test. The pro6ect will automatically include the %eb test case 7le along with each o the recorded
re;uests.
Recording with Think Time in Mind
The ThinkTime property on a %eb test re;uest reers to the amount o time a user spends <thinking< on the
current page beore issuing the ne+t re;uest. Think time delays are used to appro+imate real user beha-ior
during a load test. (ecause think time can dramatically a=ect the amount o load a %eb test can generate0 it
can be globally disabled in a load test to apply greater load to a target ser-er. Disabling ThinkTime allows you
to issue re;uests to the ser-er as ast as possible without delay between re;uests.
The %eb test recorder automatically records think time at the same time that re;uests to the %eb application
are recorded. During recording0 try to appro+imate the amount o time a user would normally spend on each
page. 8nce the recording is complete0 it is -ery important to check the recorded think time or each re;uest.
Introducing Microsot !isual Studio "##$ Team System %eb Testing 3
Inad-ertently long ThinkTimes can dramatically a=ect the rate at which a %eb test generates re;uests.
ThinkTimes are turned o= by deault in the %eb test -iewer. As a result0 long ThinkTimes might not be
immediately apparent. %hen ThinkTimes are turned on in the %eb test -iewer0 you will see <Thinking>?n@<
displayed in the 2TT, Status column until the ne+t re;uest begins. ThinkTimes are turned on by deault in load
tests. The ThinkTime counter is paused when recording is paused and while entering a comment.
Inserting Comments
Inserting comments during the recording can be a helpul aid or creating an e=ecti-e %eb test0 especially
when the %eb test contains many re;uests. Aou should use comments to make notes about what logical action
is about to take place at di=erent points in the %eb test such as <9ogging in0< <Adding item B to the shopping
cart0< and so on. These comments can be -ery helpul when you later modiy the %eb test in the %eb test
editor.
Aou can also use comments to make notes about -alidation rules you need to add to ensure the %eb test is
successul. It is much easier to decide what needs to be -alidated on each re;uest while recording and looking
at the pages than when looking at a list o 2TT, re;uests in the %eb test editor.
Customization
Selecting any o the nodes within the %ebTest tree will allow you to modiy the data inside the ,roperties
window.
Aou can also group re;uests together using a transaction. (e careul not to conuse the term <transaction< with
a programming concept in which state is committed or not committed as a unit. %ithin %eb test cases0
transactions only encapsulate actions into a group that can later be enumerated using code. Also0 transactions
are used when reporting on loadChow many transactions per second0 or e+ample.
Using the Web Test Viewer to Verify a Web Test
(eore adding a %eb test to a load test0 and running it or a long period o time0 it is important to be sure that
the test works e+actly as intended. This is where the %eb test -iewer comes in to consideration. The %eb test
-iewer allows you to watch a %eb test as it runs0 and to -iew all aspects o a pre-ious test run.
Introducing Microsot !isual Studio "##$ Team System %eb Testing 4
4igure "
!eriying a newly.created %eb test goes beyond looking at the outcome o the test run and seeing whether it
passed. 4or e+ample0 or a %eb test without -alidation rules passed0 means that no e+ceptions were thrown0 no
rules ailed0 and no 2TT, errors occurred. !eri7cation includes making sure the %eb test e+hibits the correct
beha-ior on the target %eb application0 in addition to e+ecuting without any errors. It is important to re-iew the
response or each re;uest to make sure that it is correct.
Running a Web Test Case
Ater recording a test you are ready to begin e+ecuting it. To e+ecute all the tests within a pro6ect0 simply run
the pro6ect. This will open up the Test &esults windows and mark each test as pending while it is in progress0
and ,assedD4ailed once e+ecution completes. Test selection and e+ecution is also a-ailable rom the Test
Manager and Test !iew windows.
Indi-idual %eb test 7les Etest cases or test 7+turesF can also be run by opening them up and clicking the &un
button.
&e;uests can also pro-ide credentials or logging on to the targeted site using standard authentication
methods. The dialogs or credentials allow or loading the login data rom a data source.
*ach result rom a re;uest is sa-ed0 and selecting each re;uest allows you to na-igate its detail0 -iewing the
resulting page1s 2TM9 or raw re;uestDresponse te+t.
As part o a test e+ecution0 the %eb test engine -eri7es i all the '&9s on the page are -alid links. These links
appear as child nodes below the re;uest show the 2TT, status returned by a re;uest to each o the '&9s.
Reuest Ru!es
Although checking or -alid hyperlinks on a response page is a useul eature0 it is not suGcient in -alidating
that the page is unctioning correctly. 4or e+ample0 on entering -alid credentials0 the %eb test needs to -eriy
that the login was successul. Similarly0 when the credentials are in-alid0 you need to check that an appropriate
error message is displayed on the page. To support this0 each %eb re;uest can include e+traction rules and
-alidation rules.
Introducing Microsot !isual Studio "##$ Team System %eb Testing 5
"#traction Ru!es
*+traction rules capture a response -alue so that at a later time the -alue can be used within a re;uest. &ather
than parsing the entire 2TT, response manually0 the e+traction.rules pro-ide a means o ocusing in on a
particular data item within the response. The e+tracted data item can then be -alidated or used again in a
subse;uent post back. 8ne e+traction.rule is automatically added when we record each page. That rule is an
*+tract2idden4ields rule whose data is posted back in the second re;uest o the %eb test case. During the
subse;uent re;uest0 this data is submitted back in a hidden 7eld on the page. 8ther e+traction.rule options are
*+tractAttribute!alue0 *+tract2ttp2eader0 *+tract&egular*+pression0 and *+tractTe+t. &ather than relying on
automatically recorded hidden 7eld data0 e+traction rules can be manually added and customi/ed as needed.
Va!idation Ru!es
!alidation rules allow the test writer to e+amine the entire 2TT, response and -eriy that it is correct. 4or
e+ample0 -alid credentials should cause the 98)8'T hyperlink and <%elcome HusernameI< to appear in the
2TM9 response. A -alidation.rule that checks or these items in the response should be added. The -alidation.
rules -eriy that this te+t appears somewhere within the response body.
I a particular rule ails0 the re;uest will be marked as ailed0 and the Details tab will pro-ide an e+planation or
what ailed.
This type o -alidation.rule is a !alidation&ule4indTe+t. It searches the entire response body0 looking or the
speci7ed te+t. In order to handle comple+ searching0 regular e+pressions are supported as part o
!alidation&ule4indTe+t.
In addition to !alidation&ule4indTe+t0 built.in -alidation.rules include !alidation&ule&e;uestTime0
!alidation&ule&e;uiredAttribute!alue0 and !alidation&ule&e;uiredTag.
Binding Test Data to a Data Source
(oth -alidation and e+traction rules pro-ide or te+t entry within the ,roperties window o -irtually e-ery node.
2owe-er0 the act that the te+t can be pulled rom a database makes !isual Studio %eb Test powerul. This
means we can de7ne a collection o inputs that do or don1t conorm to the speci7ed re;uirements.
To test using a data source0 we can click the Add Data Source button on the toolbar o the %eb test. In the
ensuing dialog bo+0 speciy an 89* D( ,ro-ider0 perhaps using an J.md 7le that can also be added to the test
pro6ect. Ater opening the database in the ser-er e+plorer0 we de7ne a table that will contain the necessary test
data.
8nce a test has been con7gured with a data source0 it is necessary to return to the *dit &un Settings dialog bo+
and change the run count to one run per data source row. In this way0 the test will repeat or each row in the
newly con7gured data source0 and during each run0 the parameters associated with the data source will be
assigned the -alue in the column or the particular row.
$enerating Web Test Code
Taking ad-antage o all the unctionality we ha-e considered so ar has not re;uired any code to be written.
2owe-er0 additional %eb test customi/ation is a-ailable using code. This is necessary to handle constructs like
looping and branching within a test0 or to call out to another %eb test. !STS %eb Testing pro-ides the acility o
generating the code or a particular test case. Included on the toolbar or a web test case is a )enerate Code
button. Clicking this button prompts or a test name0 and then generates a CSD!( 7le corresponding to %eb
case. The generated code includes each -alidation and e+traction rule that may ha-e been added. In addition0
data such as the -iew state is set and passed as part o the %eb re;uest.
4or a CK pro6ect0 the new CK ".# iterator is usedL ater each re;uest0 the code returns the ne+t %eb re;uest
through a yield return statement that separates out one re;uest rom the ne+t.
%e should think o generating the code only i we are going to customi/e a test doing so pro-ides a great
starting point or customi/ation o a particular %eb test case0 or e-en multiple cases0 with a little reactoring.
Introducing Microsot !isual Studio "##$ Team System %eb Testing 6
"#tending V%T% Web Testing
The a-ailable re;uest rules and the ability to write custom code rom generated %eb tests co-ers the most
common %eb testing scenarios. 2owe-er0 it sometimes makes more sense to e+tend !STS %eb Testing by
creating custom -alidation and e+traction rules0 or by coding custom %eb test plug.ins. Such e+tensions must
be de7ned in a separate assembly0 and can be used across multiple %eb testing pro6ects. %hen de7ned0 the
new %eb test e+tension assemblies may be reerenced by the %eb test1s pro6ect so that it appears in a
re;uests selection o Add dialog bo+es.
The a-ailable dialog bo+es are not only or -alidation and e+traction rules0 but or &e;uest and Test plug.ins as
well. &e;uest callbacks can be added separately to each re;uest0 perorming pre.interception and post.
interception on the re;uest. Test callbacks run pre.interception and post.interception code on entire set o
re;uests. They are initially called at the beginning o the test case0 and then again at the end o the same test
case. 4or e+ample0 consider de7ning a callback that checks whether all %eb pages in the test conorm to
B2TM90 or one that sets up a cookie or use within each %eb re;uest. I the plug.in that perormed this
-alidation was a re;uest callback0 then it could be added indi-idually to each re;uest within the test.
Alternati-ely0 the callback could be a test callback that hooked up -alidation or all re;uests during the pre.test
e+ecution stage.
*arlier0 we mentioned that custom -alidation and e+traction rules are also possible. To create such rules0 we
deri-e rom
Microsot.!isualStudio.MualityTools.%ebTest4ramework.!alidation and
Microsot.!isualStudio.MualityTools.%ebTest4ramework.*+traction&ule instead o
Microsot.!isualStudio.MualityTools.%ebTest4ramework.%ebTest,lugin or
Microsot.!isualStudio.MualityTools.%ebTest4ramework.%ebTest&e;uest,lugin.
&rowser User Interface Testing
%eb re;uests through Ja-aScript0 Acti-eB controls0 and applets are not supported within the !STS %eb Testing
unctionality. Similarly0 !STS %eb Testing is not designed to be a user interace E'IF testing tool. It will not
e+ecute client.side Ja-aScript and -eriy the results. *-en a simple menu click.and.e+pand type action cannot
be simulated by the tool. *-en though it simulates particular browser clients to the ser-er0 it does nothing in
the way o -eriying that the response back renders correctly within that client browser0 e-en when it is
Microsot Internet *+plorer. !STS %eb Testing is a wire.based testing protocol. It -eri7es what is sent and
recei-ed across the wire0 and pro-ides no built.in capability or testing how the data is rendered by the browser.
,ro-iding this type o testing is diGcult and cumbersome. 2owe-er0 there are some methods to consider or
certain situations. It is reasonable to assume that i the same response occurs multiple times0 it will render and
unction within the browser in the same manner. Thereore0 i you manually -eriy that a particular response is
correct you can e+pect the same response will beha-e correctly the ne+t time. 4or e+ample0 by checking that
the Ja-aScript is beha-ing appropriately and that the page renders correctly. 'sing this principal0 you can
-isually -eriy the response0 manually checking that a script beha-es appropriately. Now0 a -alidation test can
be created that checks or a similar response0 using wildcards to handle minor data changes such as -ariances
in data time0 user name0 ad-ertising0 and so on. I0 in uture runs o the test0 the page changes0 then the
response should be re.-eri7ed and the test should be updated accordingly0 thereby pro-iding a le-el o change
control or the page. This is not something a team is likely to deploy in mass0 but it does pro-ide a good
baseline testing mechanism and orces controlled -ariation.
'ebugging Common Web Test Prob!ems
Web %er(er Res)onds 'i*erent!y 'uring "#ecution than Recording
In a perect world0 you would record a set o re;uests to a %eb application0 run the %eb test0 and
recei-e the same responses rom the ser-er that you saw during recording. 'nortunately0 %eb
applications sometimes beha-e di=erently during %eb test e+ecution than they do during recording.
This type o problem can occur or a -ariety o reasons and oten results in an error similar to the
ollowingL
Reuest fai!ed+ ,-I''"./011VI"W%T2T" not found in test conte#t0
Introducing Microsot !isual Studio "##$ Team System %eb Testing 7
This error occurs when the %eb test attempts to use a hidden 7eld in the %eb test conte+t that it was
unable to locate and e+tract rom a pre-ious response page it recei-ed.
The ollowing screenshot demonstrates this problem when it is originated rom a ser-er error. In the
second.but.last re;uest0 the ser-er error caused hidden 7elds the ne+t re;uest depends on to not e+ist
on the page.
4igure N
There are many reasons or why a ser-er might respond di=erently during e+ecution than it did during
recording. Some o the more common reasons are summari/ed in the ollowing sections. In all cases0
-alidation rules can be added to re;uests to automatically -eriy that the ser-er responds with the
correct content.
One3Time3Use 'ata
8ne common cause o this problem is one.time.use data0 such as when a %eb application creates a
uni;ue user name. ,laying back this kind o %eb test without adding data binding or a random -alue
can result in the %eb application displaying an error when the test attempts to create a duplicate
username.
4a(a%cri)t Redirects
A %eb application that uses Ja-aScript redirects Esetting window0!ocationF might respond di=erently
during e+ecution than during recording because the %eb test engine does not run script code. This
type o problem can be easily corrected by inserting the '&9 the script redirects to and mo-ing
necessary e+traction rules to the new re;uest rom the page that perorms the redirect. (ecause this
problem e+ists in the %eb test immediately ater recording0 the only e+traction rule likely to be present
is "#tract-idden5ie!ds.
Redirects to an "rror Page
%hen there is a ser-er error0 a %eb application might redirect to an error page0 but not return an 2TT,
O## or $## le-el response code. This indicates that there is either a problem in the %eb application
itsel or a problem in the re;uests being issued by the %eb test.
-and!ing View %tate and Other 'ynamic Parameters
*-en beore AS,.N*T 5.# was introduced0 the PP!I*%STAT* hidden orm 7eld0 %eb applications used
dynamically.generated orm and ;uerystring parameters to pass inormation between pages. These
dynamic parameters re;uire special consideration in a %eb test because they can change e-ery time
the %eb test runs. A %eb test with hard.coded parameter -alues might not work or -ery long ater
Introducing Microsot !isual Studio "##$ Team System %eb Testing 8
recording0 or e-en at all.
%eb tests enable testing with dynamic parameters by using e+traction rules and conte+t binding.
*+traction rules are placed on re;uests or pages that will contain a dynamic -alue. %hen the
e+traction rule runs0 it e+tracts the dynamic -alue into the %eb test conte+t using a con7gurable name
such as <myparam<. A subse;uent re;uest then contains a ;uerystring or orm parameter with a -alue
o 66my)aram77. %hen the %eb test runs0 the -alue in the %eb test conte+t is substituted or
66my)aram77.
The se;uence o e-ents or an e+traction rule is as ollowsL
5. The %eb test engine begins e+ecuting &e;uest5.
". &e;uest5 is sent to the target ser-er.
N. A response is recei-ed rom the target ser-er.
O. The e+traction rule on &e;uest5 runs on the response page.
$. The e+traction rule places an entry in the %eb test conte+t.
Q. The %eb test engine begins e+ecuting &e;uest".
R. Muerystring parameters0 orm parameters0 and any other conte+t.bound -alues on &e;uest" are
substituted rom the %eb test conte+t.
S. &e;uest" is sent to the target ser-er.
2utomatic -idden 5ie!d Tracking
%eb tests contain special support or handling dynamic hidden 7elds0 such as PP!I*%STAT*. %hen a
%eb test is recorded0 hidden 7elds are automatically matched with orm and ;uerystring parameters.
%hat a match is ound0 the "#tract-idden5ie!ds rule is applied to the re;uest generating the source
o the hidden 7eld. At this time0 conte+t bindings are applied to parameters on the re;uest0 making
use o the hidden 7elds.
"#tract-idden5ie!ds is a special e+traction rule because0 unlike rules that e+tract one -alue into the
conte+t0 it e+tracts e-ery hidden 7eld -alue on the page into the %eb test conte+t. Normal e+traction
rules use the Conte#tParameter property to determine the name to use or the conte+t parameter0
but "#tract-idden5ie!ds uses that property only to di=erentiate rom multiple groups o hidden
7elds that might be in the conte+t simultaneously. 4or e+ample0 an "#tract-idden5ie!ds rule with
Conte#tParameter set to / will e+tract PP!I*%STAT* as <T2idden5.PP!I*%STAT*<.
5i#ing 11"V".TT2R$"T and other hidden form 8e!ds modi8ed by 4a(a%cri)t
%hen a hidden 7eld is modi7ed by Ja-ascript in an 8nClick e-ent handler0 it is possible that automatic
hidden 7eld binding will be incorrectly applied. This is a known bug in the release -ersion o !isual
Studio "##$.
<input name="btnNext" type="button" value="Next"
onclick="__doPostBack('btnNext', '');" !
%ith AS,.N*T sites0 this problem most commonly occurs when a %eb control calls the PPdo,ost(ackEF
Ja-aScript method to set the PP*!*NTTA&)*T hidden 7eld as shown abo-e. Automatic hidden 7eld
binding results in the orm parameter ha-ing a -alue such as UUT2IDD*N5.PP*!*NTTA&)*TVV0 instead
o the actual -alue C btn.e#t. To correct this problem0 the parameter -alue must be set to the -alue
being set in Ja-ascript Eor e+ample0 btn.e#tF.
Reuests Missed 'uring Recording
As discussed in the 'nderstanding the %eb Test &ecorder section0 some re;uests might not be
recorded by the %eb Test &ecorder Eor e+ample0 AJAB re;uests and some pop.up windowsF.
4ortunately there is a great tool written by *ric 9awrence called 4iddler. that can help with this. 4iddler
works by acting as a pro+y ser-er and can intercept all 2TT, traGc Eno SS9 support yetF. Two options
are described below or using 4iddler to correct a %eb test that cannot be recorded with the standard
%eb Test &ecorder.
Introducing Microsot !isual Studio "##$ Team System %eb Testing 9
4igure O
Conc!usions
In this article we saw an o-er-iew o !STS %eb Testing unctionality. %e saw how to record and e+ecute as well
as customi/e web test. In so doing0 we saw how !STS %eb Testing is -ery easy to set up0 and how a signi7cant
percentage o testing is supported without e-er ha-ing to write any code. This is a signi7cant eature that
should compel teams to begin testing early and oten within the de-elopment cycle0 not 6ust waiting until MA
engineers obtain access to the product.
!STS %eb Testing doesn1t stop with recording. There are many possibilities or e+tending the recorded tests.
The ability to generate test code makes it easy to mo-e to coded tests when special customi/ation is re;uired.
The code is simple enough that many de-elopers may choose to rely on code rather than a mouse.oriented 'I
or creating %eb test cases. &egardless0 e+tending !STS %eb Testing is simple0 pro-iding an e+cellent platorm
or additional unctionality to be added.