Documente Academic
Documente Profesional
Documente Cultură
com
A
S
utomated .......
T
oftware esting MAGAZINE
September 2010
Automation Is What
We Do!
Ending The Developer/
Automator TUG OF WAR
As a registered user you can submit content directly to the site, providing you with
content control and the ability to network with like minded individuals.
Community
Comments Box >> Community Comments Box - This comments box, available on the home page of
the site, provides an opportunity for users to post micro comments in real time.
>> Announcements & Blog Posts - If you have interesting tool announcements, or
you have a concept that you’d like to blog about, submit a post directly to the ATI
Announcements & Online Reference today. At ATI, you have a community of individuals that would love
Blog Posts to hear what you have to say. Your site profile will include a list of your submitted
articles.
>> Automation Events - Do you know about a cool automated testing meetup,
webinar or conference? Let the rest of us know about it by posting it on the ATI site.
Automation Add the date, time and venue so people will know where to go and when to be there.
Events
Learn more today at http//www.about.automatedtestinginstitute.com
De
A
S
utomated
T
oftware esting
Au
to
m
ve
lo
pe
r
at
September 2010, Volume 2, Issue 3
or
Contents
Ending Developer/Automator Tug of War
If you’re interested in bridging the gap between software developers and software test automators, then you’ve
come to the right place! While both groups may feel that theirs is the only true automation team in Information
Technology, there’s really more that connects the two teams then divides. Delve into this issue of the AST magazine
to learn how automators may participate in software product development, how developers may participate in
software test automation, and how one resource could even function in both capacities at once.
Features
Continuous Integration and Automation 16
Like many of the most popular pairs in history continuous integration and automation have the power to be an unstoppable
force when it comes to ensuring the quality of your software product’s code. By Eric Pugh
Sweat the technique 8 The Automated Software Testing (AST) Magazine is an Automated Testing
Institute (ATI) publication.
Dynamic Data Entry For more information regarding the magazine visit
http://www.astmagazine.automatedtestinginstitute.com
Learn how to create a reusable dynamic data entry function.
September 2010 www.automatedtestinginstitute.com Automated Software Testing Magazine 3
Editorial
In truth, however, there is more that blurred. I’ve been on projects where that software developers may play in the
connects test automators and developers simulators have been developed and used successful creation, implementation and
than divides the two groups. For example, by test teams to aid in testing a software maintenance of software test automation.
I regularly receive software development product, then used by development as a In an article entitled “Development’s
job requisitions from human resource stub for developing related components. Role in Software Test Automation”,
personnel that have located my resume On other projects developers have Hayes addresses crucial developer
from somewhere out on the internet. authored simulators that the test team tasks that are necessary for making test
This is odd given the fact that the bulk was able to repurpose in completing automation both possible and efficient.
of my experience is in quality assurance, their QA tasks. I also know of resources In addition, she relays how assisting
test automation efforts actually results
in developers assisting themselves and
...this issue is dedicated to the celebration of the end user. Next, Mr. Eric Pugh speaks
from the developer side of the fence on
the developer/automator connection. how test automation may aid developer
software builds. His article entitled
“Continuous Integration and Test
Automation Go Together Like Peanut
quality control, requirements analysis that have used skills they acquired as Butter and Jelly” is surprisingly not
and process improvement; a fact that software test automators to springboard really about a sandwich! It reveals the
is clearly reflected in my resume. Why them into a software developer position importance of test automation to ensuring
then would I be receiving software and were able to hit the ground running. code quality through a Continuous
development job requisitions? It is my Inversely, I have a colleague that has Integration cycle. Finally, Mr. Nick
skills and experience in software test staffed her entire automated software test Olivo picks up where our last issue left
automation that piques their interest. team with software developers, that were off with an article dedicated to testing
Maybe they aren’t looking closely able to get going following some QA and with reflection. His aptly titled, “Testing
enough at the resume to notice the automation training. So this relationship with Reflection” article delves much
differences between what I’ve done and should be celebrated as one that is deeper into the reflection pool, however,
what software developers do, but the complimentary as opposed to relegated with instructions for developing a tool
similarities between my test automation as one that is competitive. that automatically produces information
work and the work performed by software and data for learning about and testing
development professionals regularly gets Therefore, this issue is dedicated to .NET assemblies and web services.
their attention. the celebration of the developer/automator This article reveals how a single person
connection. Ms. Linda Hayes provides can simultaneously embody the roles
In many other instances the lines an article that addresses the relationship of software developer and software test
between the two groups are extremely between the groups by expressing the role automator.
ATI Automation
Honors
Celebrating Excellence in the
Discipline of Software Test
Automation
www.atihonors.automatedtestinginstitute.com
Figure 1: Standard Input Code Structure This step opens the data source and
makes it accessible to the function.
the Parameter in the data table is identical
Screen{
to the logical name in the object map, a Step 3. Get Total Number of Input
FirstName{
function can be created to do the bulk of Columns
Class: textbox
the input logic for you. This function may
Id: fname001
resemble Figure 3. This step gets the total number of input
Name: First
columns in the data source. For example,
}
This function is what would be Table 1 has six columns (excluding the
Phone{
Class: listbox
Id: phn001
Name: Phone
}
)
C
requires repeated test/fix/test cycles, and effort to remedy the issues is not permitted
a risky approach at worst because it may
allow errors in new functionality to go
by the schedule.
The key, then, is to engage with
ontribute
undetected. development early to educate and
C
The ideal solution to this challenge indoctrinate them. Explain their role
ontent
is for development to actually supply the and the impact on automation. It’s also
object inventory. For some applications, important to educate management to avoid
this information is already stored in the the perception that testability imposes
metadata or database; for others, the additional overhead without offsetting
T
developers may have to add extra code benefits. Sell them on how automation not
to register or log the objects as they
are defined. But by whatever means,
only helps testers, it accelerates the overall
life cycle for developers and improves oday
if development can produce the object quality for end users. Developers can
inventory for each build, then automation leverage automation for unit, integration
engineers can focus on identifying the and build testing and testers can expand
changes and updating or adding new tests. their test coverage dramatically, both of
Once the object inventory is acquired, which result in a more reliable system.
detecting changes can be fully automated Further, consistency of coding conventions
by simply comparing the previous version makes it easier for developers to transfer
and maintain the software over its useful Community
to the new one and flagging any additions,
deletions or modifications for review. life. Comments Box
It should be noted that not all changes If the basics of automated testability
will be reflected in the object inventory; are communicated and understood from the
functional requirements may introduce start, it is often a straightforward proposition
new business rules or edits that involve to integrate them into the development
existing objects. Thus managing the process. Developers can invest a few
object inventory is not a substitute for a minutes in their coding approach and save Announcements &
robust requirements management process, automation engineers and testers untold Blog Posts
but it can work wonders for improving the hours of work down the line
efficiency of maintaining automated tests.
The next step is to automate impact Summary
analysis, which is relating the object or
requirements changes to all affected tests The way in which an application
so they can be reviewed and updated. That is developed can enable or prevent Automation
is, once changes to the object inventory test automation altogether. If you can
or requirements are identified, then all successfully integrate automated testability Events
tests that reference them are candidates throughout the development process and
for maintenance. Depending on whether application life cycle, you will be able
tests are stored as code or data, or some to redirect enormous effort away from
combination thereof, this process may be struggling with intractable impediments
automated as well. and towards creating better and more As a registered user you
comprehensive test coverage. And that, in
Extending automation into change the end, is what it’s all about. can submit content directly
management assures that all of the effort
invested into test automation is protected to the site, providing you
and enhanced over the life of the
application, instead of compromised or
with content control and
lost by the very changes that make testing the ability to network with
critical.
like minded individuals.
Education and
Indoctrination
As you can see, development’s role Learn more today
in test automation is pervasive, but all too
often testability issues are not uncovered at
until the application has already been
developed and it is too late to resolve
them. In my experience, developers are
not unwilling to fulfill their role, they are http//www.about.automatedtestinginstitute.com
simply uninformed. Most are familiar
with manual test processes in which these
Repurposery
Open Source Tools Built From Other Open Source Tools
Repurposing may very well end up being the vehicle that brings open
source closer to being on par with commercial tools.
tools. Thus, if a user requires specific today. Many of the most popular open support, training and flexibility. While
functionality not currently in the tool, the source tools that are available are built on
repurposing doesn’t directly address
ability exists for the user to create that top of existing, repurposed open source
support and training, it does impact
functionality from an established code automated test tools. Two of these tools
greatly the advantage of flexibility. At
base. In addition, users often integrate are very popular and are finalists in the
times commercial tools are more flexible
various open source tools together into 2nd Annual ATI Automation Honors
with respect to how they may be used
an in-house framework that uses the tool awards: Jsystem and TestMaker.
for a different purpose or in a different and with what technologies they support.
format than it was originally intended. Jsystem – This automated test tool Repurposing takes a good, yet narrowly
This repurposing allows the tool to be functions as a framework for writing and focused tool and expands its reach,
expanded to meet a new and specific managing automated system tests. In order allowing open source tools to be more
challenge. to make it a collaborative environment for competitive in areas beyond just price.
By Eric Pugh
once, it’s not a problem; several times a device that provides you with some Keeping it Running
day, however, can be disruptive. A web information without shouting at you. One
dashboard is problematic as well, given of the most popular ambient information
Getting CI setup is one thing, but
that it requires constant visits to a web devices is to hook up two lava lamps - a
keeping it running is another. I talked
page to find out the status of your build. green one, and a red one - using cheap
to a Project Manager at a conference
The solution? An ambient information X10 protocol power modules. When
once about his CI system, and he talked
the build succeeds, the green lava lamp is
about how rock solid it was. His exact
illuminated, and the red one is turned off:
words were, “I never get an email about
But if the build fails, then the green the build failing”. After the conference
one is turned off and the red one turned I followed up with him about his builds,
on. One of the things that I like about and he sheepishly acknowledged that
using lava lamps is that while at first in fact the builds were succeeding only
glance they are a binary system of green/ because the testing phase of the build had
red, they actually are an analog system of been commented out! The challenges
notification when you add in the time it with having a successful CI system
takes for the lava lamp to melt the wax and revolve around cultural (otherwise known
start it bubbling. In some organizations as people!) issues and project structure
breaking the build, and therefore turning
issues.
on the red lava lamp is a big no-no and is
frowned on. However, I like to think of Often developers feel the CI loop is
the CI build as a buddy looking over my like Big Brother from George Orwell’s
shoulder and helping me work better. So novel 1984, watching over them and trying
if the red light is on, thats okay, as long as to catch them in a mistake. Especially
the red lava doesn’t start bubbling! If I when the CI system is used as leverage
break the build, causing the red lava lamp to perform other tasks not directly related
to light up, I know I have 10 minutes or to verifying the code, like measuring
so to fix the build and turn the light off code quality, looking for inappropriate
before that lava starts bubbling! cut and paste of code, and providing
is to prioritize your testing by looking at such as load tests and compatibility after the successful short CI build loop is
which modules contain complex code and tests. Within the CI world, the tests that completed.
lack decent unit testing code coverage. give you the biggest bang for your buck Another technique is to run just a
The theory being that complex code start from the most focused tests, the unit subset of the full test suite. Either run
with out unit testing is going to be the tests, and work their way outwards to tests related to the specific modules that
source of more bugs. For the Java world, the system level tests. Because of the have been impacted by the changes, or do
there is a great tool called Panopticode overriding requirement that the build loop a rolling sub sample of the tests on each
(http://www.panopticode.com) that you be as fast as possible, with 10 minutes
run so you still get the full test suite run on
can use to build visual treemaps for being the goal, and 30 minutes being too
a frequent basis. No matter what angle
metrics about your code, including code long, you quickly see to the importance of
you take, make sure that the tests that most
coverage and code complexity. This focusing on which tests work feasibly in
directly test the code are run frequently
information you can use to prioritize those time limits. Thousands of true unit
and quickly. A CI loop that takes 2 hours
your automation efforts. For example, tests will have no problem running in just
to run might as well be a daily build loop
by looking at the treemap diagrams for a couple minutes. Functional tests also
for all the value it adds!
the venerable CI tool CruiseControl you shouldn’t take too long, and can easily
can see the method getLabel() on the class have a full suite run in 20 minutes. But
PropertyFileLabelIncrementer is both if you start seeing your tests take longer Successful Test
complex, with a CCN rating of 10, and then that, say because you are doing web Automation is a People
has only 40% code coverage. browser based testing with tools such as Problem
Selenium or Watir, or running massive
You will quickly discover that there regression tests that generate thousands
are many types of automated tests that of test cases, then you need to embark Building up a great suite of automated
you will need to develop for full coverage on different techniques. Some ways of tests is a task that requires input from
of your application; from finely grained reducing the time that these test take is to many people across many areas. In any
unit tests that are tightly coupled to the start looking at writing tests that can be large software project that is embarking
structure of the code and are primarily run in parallel fashion across a farm of on integrating automated tests into their
Automated tests, CI loop, it’s important to seek input from a wide tests from the shoulders of the automated test writer!
variety of folks on what to test. For example, your Successful test automation writers blend the
run frequently
support folks may have the best sense of what areas coding chops of a developer with the analytical
under a CI system, of the application generate the most requests for skills of a business analyst and the rigorous attention
help with bugs. The developers may have a better to detail of a tester, which means this can be a
evolve with the sense of which modules suffer the most technical challenging role to be in! Companies are starting
debt. And your business folks will want to have to develop specific roles for this type of person. At
source code,
the team focus first on testing application areas Microsoft, they are called Software Development
ensuring that prioritized highly by the customer. This means a Engineer in Test (SDET) and are one of the three core
test automation engineer must be able to work with engineering positions, along with program manager
bugs are found, many different types of people, from developers, to and software engineer, and are on an equal footing
business analysts, to business stakeholders, and be as the others! At Google they are called Software
and tests never
comfortable in speaking in their language! Engineer in Test (SET) and looking at a recent job
go broken for Often one of the most challenging relationships posting for a SET show the responsibilities are
is that between the developers and the test automation challenging and exciting:
long!
engineers, because at times the work of one is
impacted by the work of the other. For example, • Work in tandem with other engineers to
if you are developing automated tests using a tool build advanced automated frameworks
like FitNesse, where you need to have custom code to deliver and exercise our world-class
developed to translate your tests into something applications.
that can interact with the Application Under Test • Conduct research on emerging technologies.
(AUT), then you will need to work with a developer
• Train and mentor other team members.
to either implement those hooks, or learn enough
about the code to implement those hooks yourself. • Develop innovative solutions for large
Other times, as you develop the automated tests, you scale automated testing strategies, quality
may find that you need more information about the assurance, data analysis, and monitoring of
interactions of the various modules, especially when our systems.
testing beneath the user interface. This is an area
where collaborating with a developer to come up As with all Google Software Engineers,
with the right test will be critical. responsibilities may also include aspects of project
management, release engineering, task delegation,
Building a solid relationship with the developers team coordination, and production monitoring.
means understanding their concerns about what the
tests are verifying, and how long they take when Test automation is one of the key techniques
the build is being run, but also means that you can embraced by QA teams when the volume of testing
educate them on what it takes to develop successful required to ensure a quality product has grown beyond
automated tests. Get them thinking at the design stage what individuals can perform. Automated tests, run
about how a function would be programmatically frequently under a CI system, evolve with the source
tested, not just at the unit level, but at the system code, ensuring that bugs are found, and tests never go
level as well. And, with a really good relationship broken for long! As the coverage of the automated
and understanding of the goals of the automated tests increases, the quality of the application goes
tests, you can also enlist the developers in updating up, and the value of the CI build goes up. Just like
the automated tests as part of their development PB&J, one without the other doesn’t make much
work when writing new code. This helps lift some sense and is not nearly as satisfying!
of the burden of maintaining the automated suite of I think I’ll have a PB&J for lunch today!
ATI Automation
Honors
Celebrating Excellence in the
Discipline of Software Test
Automation
www.atihonors.automatedtestinginstitute.com
Development’s
Role in Software Test Automation
A
side from their obvious principles described in this article can be
responsibility for developing applied to almost any interface, they are The Software
the software to be tested, specifically directed to testing UIs.
developers also have a
crucial role to play in the automation Within the context of UI test automation, Development Team has
of testing. Simply put, they must design there are certain basics that apply to
and develop the software in a way that almost all tools. First, they rely on an several pieces that help
enables and supports automation. This inventory, map or repository of objects
article describes what is required to make that comprise the application. Next, to ultimately complete
automation possible as well as what is these objects are referred to and located
by names or attributes and accessed via
desired to make it efficient. But lest this the Software Test
all be perceived as work in service of their methods and properties. Without
these fundamentals in place, tools are
automation, it is important to realize that
relegated to using text or images to
Automation Puzzle
most of the factors that make automation
practical also result in an application that locate components, and keystrokes or
is easy for end users to learn and use and mouse events to interact with them;
for developers to support and maintain. these techniques are unreliable and
almost impossible to read or maintain.
Therefore, for purposes of this article,
To set context, it should be noted that
there are many types of applications
we will focus on the preferred object- Accessibility
oriented approach.
and levels of automation. Most of us
think of automating through a user With all that being said, the fundamental
interface (UI), but the fact is there are tenets of automated testability are Consistency
many other types of interfaces: Web accessibility and consistency. That
services, APIs, embedded systems and is, the application components must
databases all represent layers that can be accessible to the test automation
be automated. What is interesting is
Change
tool and consistently implemented.
that these interfaces are usually the Although necessary, accessibility and Management
easiest to automate, simply because they consistency alone may not be sufficient.
are either built using well-recognized If the application is large or complex
standards or because they must already enough, it may defy automation through
support automated interaction with other
Indoctrination
its sheer scope or rate of change. In that
applications or devices. Ironically, it is case, automated testability must shift
the user interface that usually represents from simple automation of execution to
the greatest challenge, and so while the automation of change management.
apply a label that is more useful, such as interact with the application components. For example, there are at least four
“Customer Name”. In a large application Object names should be unique but other different ways to implement a combo
with hundreds or even thousands of properties should not be, and methods box: 1) allows you to type in the selection
objects, this becomes a significant effort. should be exposed and available, not or select it, 2) allows selections only, no
private or hidden. Container objects input, 3) moves the selection list to the
But accessibility also demands that should expose their included objects. For item based on the letters being typed,
the same name be used for the same object example, a frame that contains a grid that moving around the list for each letter, or
across sessions, builds and releases. In contains a cell that contains a combo box 4) selects the items closest to the typed
other words, “TXT_015” cannot suddenly should permit direct access to the methods input. If any one of these can be used for
become “TXT_016” simply because and properties for each layer. Opaque any given object, then the test automation
another object is added to the window or objects that won’t share their contents code will be more complex. And, not
page. This lack of persistence will doom make automation tricky, and in some coincidentally, your end users will
an automation effort, because the test cases impossible. probably be confused.
script cannot reliably locate the same But accessibility alone is not enough. Error handling is another area
object from one run to the next. This risk The application must also be developed in where consistency is important. If each
is especially high when the names are a consistent manner across all components developer handles errors in their own
being generated dynamically. and developers. unique way, then the automation engineer
consist of stored procedures, web services, or dlls; but the thing that these have in common is
that they typically don’t provide an easy way for testers to access them. This prevents testers
from being able to do their job until much later in the development cycle. However, with a
little creativity and access to Visual Studio, you can build your own tool that can learn about
the assemblies and web services that your developers have created, and help you test them
more effectively. This article will show you how to do this via .NET’s reflection capabilities.
P
eople typically use reflection Let’s start out with the basics of learning what
to learn about classes and methods are in a given assembly. To do this, we
4
assemblies. RedGate’s Reflector will:
tool is probably the best known
Build a small utility that will list an assembly’s
example of reflection out in the wild. Reflector
methods, and the parameters those methods
uses reflection to list all the classes, methods and
take.
parameters that live in a given assembly. However,
reflection can go a step further and also be used Write some code that builds input files for
each of those methods.
to automatically generate test cases for use in
white box testing. In this article, you’ll learn how Have our utility call the methods and pass
to use reflection against a .NET assembly to learn them the data stored in the input files.
about its methods, how to autogenerate test data,
and how to automatically generate baseline files
steps
We’ll also have the utility keep a list of all the
methods that the assembly contains, so we
for later comparison, so you can see whether or can keep a tally of all the methods that have
not the methods are behaving properly. been added during the course of the release.
Now, we’ll add the code in Table 3 to the Browse button control.
By testing your
1 private void btnBrowse_Click(object sender, EventArgs e)
backend methods
2 {
3 if (openFileDialog1.ShowDialog() == DialogResult.OK) early, you’ll be
4 txtAssembly.Text = openFileDialog1.FileName;
5 }
able By testing your
backend methods
Table 3: Browse Button Code
early, you’ll be
The next control we’ll add code to is the Get Methods button. Double-click able to get more
the Get Methods button, and add the code from Table 4.
testing done sooner
This code first clears the contents of the list box. Then it tries to load the in the release
Now that we know what methods live inside the assembly, the next step is to
have our code loop through each method and learn about its parameters. We’ll
create an xml file for each method, which will contain a list of the necessary
parameters, and we can enter our desired values for those parameters. In the
case of an overloaded method, there will be one xml file created per overload.
30 Automated Software Testing Magazine www.automatedtestinginstitute.com September 2010
1 private void GetParameters(MethodInfo method) So if you have a method called Hello
2 { which is overloaded three times, there
3 int counter = 0; will be three xml files created for that
4 string filename; method.
5 XmlDocument xmlDoc = new XmlDocument();
6 filename = InputDirectory + "\\" + method.Name + "Input.xml"; We’ll begin this step by checking
7 while (File.Exists(filename)) to see if an input file for our target
8 { method exists. If it does, we’ll append
9 counter++; a 1 to its name, and check if that exists.
10 filename = InputDirectory + "\\" + method.Name + "Input_" + counter.ToString() + ".xml"; (This is how we handle overloaded
11 } methods as mentioned above.) Once
12 XmlTextWriter xmlWriter = new XmlTextWriter(filename, System.Text.Encoding.UTF8); that’s done, we’ll create elements in
13 xmlWriter.Formatting = Formatting.Indented; our xml file for the methods’ name,
14 xmlWriter.WriteProcessingInstruction("xml", "version='1.0' encoding='UTF-8'"); its return type, and its parameters. If a
15 xmlWriter.WriteStartElement("Method"); method has no parameters, then we’ll
16 xmlWriter.Close(); create a text node that states this.
17 xmlDoc.Load(filename);
18 XmlNode root = xmlDoc.DocumentElement;
Once you’ve created the
19 XmlElement nameTag = xmlDoc.CreateElement("Name");
GetParameters method, you can add it
20 XmlText nameVal = xmlDoc.CreateTextNode(method.Name);
to the btnGetMethodList_Click event;
21 XmlElement returnType = xmlDoc.CreateElement("ReturnType");
place it right after line 9. The call will
22 returnType.InnerText = method.ReturnParameter.ToString();
look like this: GetParameters(method);
23 nameTag.AppendChild(nameVal);
24 root.AppendChild(nameTag);
Now, you’ll notice that our current
25 root.AppendChild(returnType);
implementation is just creating a
26
placeholder for our parameter values
27 ParameterInfo[] parameters = method.GetParameters();
(the node called ReplaceMe). While
28 // Point out if there are no parameters
29 if (parameters.GetLength(0) == 0)
it is possible to automatically create
30 {
input data, there’s a good chance
31 XmlElement data = xmlDoc.CreateElement("NoParams");
that the data generated wouldn’t be
32 XmlText noVal = xmlDoc.CreateTextNode("This method takes no parameters"); valid, or they might not be useful. For
33 data.AppendChild(noVal); example, let’s say you had a method
34 root.AppendChild(data); called createCustomer, which took the
35 } customer’s first, middle and last names
36 // Output details for each parameter as parameters. You could automatically
37 foreach (ParameterInfo param in parameters) generate strings for those values, but
38 { they’d be gibberish. I’ve found that
39 XmlElement par = xmlDoc.CreateElement("Parameter"); it’s better to manually edit the xml files
40 par.SetAttribute("Name", param.Name); once they’ve been created to ensure
41 par.SetAttribute("Type", param.ParameterType.ToString()); you’re testing with good data. You can
42 XmlElement val = xmlDoc.CreateElement("Value"); keep multiple copies of a given input
43 XmlText Val = xmlDoc.CreateTextNode("replaceMe"); file, and each copy can have its own
44 val.AppendChild(Val); unique input data.
45 par.AppendChild(val);
46 root.AppendChild(par); Once you have your input data,
47 } the next step is to be able to run a
48 xmlDoc.Save(filename); given method with the desired input.
49 } To do that, we’ll create a method
called executeMethod, which will
Table 5: Get Parameters Method Code
September 2010 www.automatedtestinginstitute.com Automated Software Testing Magazine 31
1 /// <summary>
2 /// This method reads a parameter (as a string) from an input file and converts it to
3 /// the appropriate type. For example, if a parameter is type System.Int32, and its
4 /// string value is "23", then this function converts that string to an Int32. You
5 /// can keep tabs of all the parameter types used by the BLL by calling the listParameters()
6 /// method below.
7 /// </summary>
8 /// <param name="paramType">The type of parameter; e.g. System.Int32</param> with a little
9 /// <param name="paramValue">The parameter's value</param>
10 /// <returns></returns>
creativity and
11
12
public object convertParameter(string paramType, string paramValue)
{
access to Visual
13 string[] sArr; Studio, you can
14 ArrayList paramArr;
15 switch (paramType) build your own
16 {
tool that can learn
17 case "System.String":
18 return paramValue; about the assemblies
19 case "System.Guid":
20 return new Guid(paramValue); and web services
21 case "System.Int32":
22 return Convert.ToInt32(paramValue);
that your developers
23 case "System.Int16": have created, and
24 return Convert.ToInt16(paramValue);
25 case "System.Nullable`1[System.Int32]": help you test them
more effectively.
26 if (paramValue == "null")
27 return null;
28 else
This article will
29 return Convert.ToInt32(paramValue);
30 case "System.Nullable`1[System.Guid]": show you how
31 if (paramValue == "null")
32 return null; to do this via
33
34
else
return new Guid(paramValue);
.NET’s reflection
35 case "System.Nullable`1[System.Int64]": capabilities. People
36 if (paramValue == "null")
37 return null; typically use
38 else
reflection to learn
39 return Convert.ToInt64(paramValue);
40 case "System.Nullable`1[System.Double]": about classes and
41 if (paramValue == "null")
42 return null; assemblies.
43 else
44 return Convert.ToDouble(paramValue);
45 case "System.Nullable`1[System.Decimal]":
46 if (paramValue == "null")
47 return null;
48 else
49 return Convert.ToDecimal(paramValue);
As a registered user you can submit content directly to the site, providing you with
content control and the ability to network with like minded individuals.
Community
Comments Box >> Community Comments Box - This comments box, available on the home page of
the site, provides an opportunity for users to post micro comments in real time.
>> Announcements & Blog Posts - If you have interesting tool announcements, or
you have a concept that you’d like to blog about, submit a post directly to the ATI
Announcements & Online Reference today. At ATI, you have a community of individuals that would love
Blog Posts to hear what you have to say. Your site profile will include a list of your submitted
articles.
>> Automation Events - Do you know about a cool automated testing meetup,
webinar or conference? Let the rest of us know about it by posting it on the ATI site.
Automation Add the date, time and venue so people will know where to go and when to be there.
Events
Learn more today at http//www.about.automatedtestinginstitute.com
open our xml file and call the target 1 case "System.Nullable`1[System.DateTime]":
method with the parameters provided. 2 if (paramValue == "null")
Since all our information is technically 3 return null;
stored as strings inside the xml files,
4 else
we’ll need another method called 5 return Convert.ToDateTime(paramValue);
convertParameter that will change the 6 case "System.DateTime":
stored parameters into their relevant
7 return Convert.ToDateTime(paramValue);
types. The convertParameter method
8 case "System.Boolean":
reads the type of parameter from the
9 return Convert.ToBoolean(paramValue);
xml node and then calls the relevant
Convert function. I’ve tried to cover 10 case "System.Guid[]":
development cycle,
10 xmlDoc.Load(filename);
11 XmlNode root = xmlDoc.DocumentElement;
12 string mName = root.SelectSingleNode("Name").InnerText;
13 Assembly targetAssembly = Assembly.LoadFrom(txtAssembly.Text); developers build
14 Type assb = targetAssembly.GetType();
15 MethodInfo mi = assb.GetMethod(mName); a backend system
16 returnCode = mi.ReturnType;//cast the return object to the appropriate return code type, e.g.
Int32 or System.Data.Dataset that will become
17 if (root.SelectSingleNode("NoParams") != null)//if the function has no parameters, just
invoke it the foundation for
18 returnCode = mi.Invoke(targetAssembly, null);
19 else //otherwise, create an array of parameters an application.
20 {
21 XmlNodeList paramList = root.SelectNodes("Parameter"); This can consist of
22 foreach (XmlNode param in paramList)
23 { stored procedures,
24 string val = param.SelectSingleNode("Value").InnerText;
25 alpha.Add(convertParameter(param.Attributes["Type"].Value, val)); web services, or
26 }
27 methodParams = alpha.ToArray(); dlls ; but the thing
28 try
29 { that these have in
30 returnCode = mi.Invoke(targetAssembly, methodParams); //call the method with its
appropriate params; common is that
31 if (returnCode.ToString() == "System.Data.DataSet")
32 { they typically don’t
33 DataSet d = (DataSet)returnCode;
34 d.WriteXml(ResultsDirectory + "\\" + mi.Name + "Output.xml"); provide an easy way
35 }
36 else for testers to access
37 File.WriteAllText(ResultsDirectory + "\\" + mi.Name + "Output.txt", returnCode.
ToString());
them.
38 }
39 catch (Exception ex)
40 {
41 MessageBox.Show(ex.InnerException.ToString(), mi.Name);
42 }
43 }
44 }
4 executeMethod(openFileDialog1.FileName);
What we’ve created so far is great
5 }
for executing one method at a time, but
chances are you’ll want to run all the
methods, one after another. To do that, Table 9: Browse Button Code
we’ll add the code from Table 10 for
the Test All button. It will loop over all
the xml files in the Input directory, and 1 private void btnTestAll_Click(object sender, EventArgs e)
Methods 2 {
4 lstMethods.Items.Clear();
check and see if there have been any 8 foreach (MethodInfo method in methodInfoArray)
Table 11. 13 }
14 lstMethods.Sorted = true;
This code writes out a list of all 15 string currentTime = DateTime.Now.ToString("yyyy-MM-dd hh-mm-ss");
the methods to a text file, and you can 16 File.WriteAllText(OutputDirectory + "\\methodList" + currentTime + ".txt", mNames.
ToString());
compare that list to a baseline list that
was placed in the Oracle directory, 17 }
which checks all the parameter types that 3 ArrayList types = new ArrayList();
the various methods take. This will allow 4 foreach (string f in Directory.GetFiles(InputDirectory))
you to easily see if the convertParameter 5 {
routine requires any updates. 6 if (f.EndsWith(".xml"))
7 {
Once this method runs, it writes out a
8 XmlDocument xmlDoc = new XmlDocument();
list of the parameters that all your methods
9 xmlDoc.Load(f);
take. Add a call to this method in code for
10 XmlNode root = xmlDoc.DocumentElement;
the btnListParameters button as illustrated
11 XmlNodeList paramList = root.SelectNodes("Parameter");
in Table 13.
12 foreach (XmlNode p in paramList)
Now you have a tool that allows 13 {
you to call methods directly from .NET 14 string val = p.Attributes["Type"].Value;
assemblies and send any input you see fit. 15 if (!types.Contains(val))
The tool can then automatically compare 16 types.Add(val);
the methods’ return values to stored ones, 17 }
and provide you with a report on which
18 }
methods aren’t behaving as expected.
19 }
You can also use reflection against web
20 string parameterList = "";
services. The only catch here is that
21 foreach (string v in types)
the .NET framework doesn’t have a
mechanism to dynamically load web 22 parameterList += v + "\r\n";
services at runtime. So to reflect against a 23 string count = types.Count.ToString() + " Parameter Types\r\n";
web service, you’d need to add a reference 24 string output = count + parameterList;
to your webservice, and then change the 25 MessageBox.Show(output);
code accordingly. In the Table 14 example, 26 File.WriteAllText(OutputDirectory + "\\ParameterList.txt", output);
I’ve modified the btnGetMethodList_ 27 }
Click methods to retrieve methods from a
web service. In my VS project, I’ve added
a reference to a temperature conversion
Table 12: List Parameters Code
web service, which can be found at
http://www.w3schools.com/webservices/ 1 private void btnListParameters_Click(object sender, EventArgs e)
tempconvert.asmx. 2 {
3 listParameters();
Then recompile your app and you’ll 4 }
pick up a reference to any new methods
that have been added. The app will work
without any other changes. Table 13: List Parameters Button Code
By testing your backend methods
1 private void btnGetMethodList_Click(object sender, EventArgs e)
early, you’ll be able to get more testing
2 {
done sooner in the release cycle.
Reflection also provides a relatively easy 3 lstMethods.Items.Clear();
you’re testing, and provides a way to keep 5 tempConvert.TempConvert targetAssembly = new tempConvert.TempConvert();
up with the methods & parameters that 6 Type assemblyType = targetAssembly.GetType();
make up your back-end code. You can 7 MethodInfo[] methodInfoArray = assemblyType.GetMethods();
learn more about reflection on Microsoft’s 8 foreach (MethodInfo method in methodInfoArray)
site, at http://msdn.microsoft.com/en-us/ 9 {
library/f7ykdhsy%28VS.71%29.aspx. 10 lstMethods.Items.Add(method.Name);
11 GetParameters(method);
Wait! There’s More! 12 }
13 lstMethods.Sorted = true;
Want more reflection? Find it at: 14 }
http://www.reflection.
automatedtestinginstitute.com Table 14: Get Method Code (Web Services)
September 2010 www.automatedtestinginstitute.com Automated Software Testing Magazine 37
38 Automated Software Testing Magazine www.automatedtestinginstitute.com September 2010
September 2010 www.automatedtestinginstitute.com Automated Software Testing Magazine 39
I ‘B’Log To U
I have recently been having what seems Bottom line: the reason test automation
like the same discussion with a number of costs so much is that it’s done in a silo
far removed from the development effort.
different people. “Automated tests should
Buffered from the consequences of design
find bugs” or “find more bugs” is a very decisions that decrease testability, the
common misconception. Basically this developers continue to create software
says that finding bugs is a valid objective that’s nigh onto impossible to automate.
for automation. I don’t agree - I think this And isolated from the technical expertise
of how the software was constructed,
is generally a very poor objective for test
the test automation specialists are in a
automation. The reasons are to do with situation where they cannot help but
the nature of testing and of automation. be both inefficient and ineffective.
You see, MS Office apps (Word, None of the steps in the process (with the
Powerpoint Excel, Outlook) are not exception of the running of the checks)
record & playback friendly. I learned can be automated, since each step requires
this the hard way a long time ago, and a thinking person, rather than a machine,
I’m hoping to spare others the same
to seek information, to make decisions,
pain. You will not be able to create a
and to control the overall process.
reliable, robust set of MS Office tests
Parts of unit testing can be assisted by
via a record and playback tool. The only
way I’ve found to effectively work with automation, but the automation isn’t
Office apps is via scripting. The Office doing anything particularly on its own;
object model lets you programmatically it remains an extension of the person’s
access any bit of text, cell or slide. ability to execute and to observe.
will provide them with a positive ROI Several tools, organizations and the winners and nominees will be
in the long run; the discipline of test highlighted in a special November
individuals were nominated in the issue of the Automated Software
automation wins, because it will garner
higher respect and prestige as a distinct following categories: Testing Magazine.