Documente Academic
Documente Profesional
Documente Cultură
About
Careers
Clients
Consulting
Training
Support
Articles
Blog
Having defined our test, we can now design it. Wordplay? Not really. Regardless of how we run our tests, weve got to define
up front what it is we are testing. Once weve got that, we can then work out how to actually create a test that simulates what
weve defined.
Recent Posts
Pretty boring huh? Cant we just, yknow, like, run LoadRunner a bunch of times?
Sure, go ahead. But unless everything is hunky-dory and passes with flying colours, youll only end up retrofitting the
definition to your test in order to properly analyse the problem and create further tests to get to root-cause. Therein lies the
path to madness, and more importantly, wasted time and poor quality tests.
Dont jump the gun and wade into the fun part, otherwise youll spend too much time trying to figure out what to build into
Top Posts
OBIEE 11g Security Week :
Managing Application Roles and
Policies, and Managing Security
Migrations and Deployments
Upgrading OBIEE to 11.1.1.7
OBIEE 11gR1 : Architecture and
Use of WebLogic Server
(NB this is not an exhaustive diagram, for example, it doesnt include any alternative security providers)
pdfcrowd.com
The user accesses OBI through their web browser, which connects to WebLogic. Weblogic is a web and application server.
It connects to OBI which in turn generates queries to run on the database
Examining the stack in a bit more detail, it looks like this:
Random Posts
MDS XML versus MUDE Part 3:
Configuring MDS XML w ith Git
New With OBIEE 11.1.1.7 and
SampleApp v305 - Essbase Cube
Spin-Off
Installing Essbase Analytics Link
for HFM 11.1.2.3
Oracle Data Integrator 12c
release : Part 1
Details on Tw o New Upcoming
Courses in Brighton - Including
ODI12c
Tags
Endeca exalytics
extremebi git
goldengate
hadoop Hive init.d install
linux MDS XML monitoring
new features nqcmd OBIA
oracle data
integrator Oracle
Endeca Oracle Endeca
Information Discovery ow b
pdfcrowd.com
For each Logical SQL request from Presentation Services, there will be one or more Physical SQL queries, but just one
resultset passed back to Presentation Services. For each Dashboard there may be many Logical SQL queries run.
pdfcrowd.com
If our performance test is for a greenfield OBI deployment then wed be pretty foolish not to conduct an end-to-end test.
We need to make sure that all the components are performing together at the required workload.
However, what about if were testing a existing mature deployment where theres a specific problem in the database? We
could do an end to end test, but if all thats changing is the database (for example, new indexes) then we should probably
just focus our test on the database.
What about if the RPD is changing in an existing deployment and nothing else? Is there really any point performance
testing the response time all the way to the user? If the data returned is the same then we can presume that the
performance in Presentation Services and back to the user is going to be the same. Therefore we can build a more
streamlined (and thus efficient) test against just the BI Server (because the RPD has changed) and the database
(because a changed RPD could generate different Physical SQL).
The flipside of not testing what doesnt need testing is to test what does need testing and may get overlooked. For
example, are your users connecting over 3G yet youre conducting your performance tests on a LAN?
End-to-end (almost)
Traditional load testing tools such as JMeter, Oracle Application Testing Suite
(OATS), and HP LoadRunner can be used to simulate a user clicking on a
report in a web browser and recording how long it takes for the report to run.
They can have multiple report definitions, and run many concurrently. This is
usually sufficient for a comprehensive performance test. One should be aware
when using these that there is still an element that has not been tested:
1. Outbound. A user interacting with OBI uses a modern web browser with lots of asynchronous traffic (AJAX). For
example, clicking on a prompt to get a list of values, clicking on the dashboard menu to see a list of available
dashboards. A standard load test will typically only send the HTTP request necessary to trigger the dashboard refresh
itself.
2. Inbound. A real users web browser will receive the data from a dashboard as it is returned from the OBI server and
render it. Once its finished rendering, it is viewable to the user. If it is a large pivot table with hundreds of thousands of
cells (b ecause thats how the Excel report were duplicating does it) then this might take many seconds or even
pdfcrowd.com
minutes or even not work at all. But a load test tool will parse the data that comes back from the server, and once all
the datas received, it will mark that as the completed response time the time to render is not included.
This is not to dismiss this approach of test design generally it is a very good one. These are the subtleties that a standard
load test can overlook. The AJAX dashboard menu call might not take long, but the list of values might be a hidden bottleneck
if it hits an unoptimised data source (for example, SELECT DISTINCT DIM_VALUE FROM 60M_ROW_UNAGGREGATED_DIM_TABLE).
It is important that the above limitations are noted, and the test design adjusted to compensate where these weaknesses
are observed (for example, large pivot tables, list of values, etc).
pdfcrowd.com
BI Server + Database
The BI Server supports ODBC or JDBC for communication with clients (such as Presentation Services). We can use a client
to send Logical SQL requests to it and record the response time. Because Presentation Services also sends Logical SQL to
BI Server this test approach is a good one because from the BI Server + Database point of view, the behaviour is exactly the
same whether the Logical SQL comes from Presentation Services or another client (eg injected by us for performance
testing).
pdfcrowd.com
nqcmd is the ODBC client that is provided with OBIEE. You can wrap this in a series of shell scripts to create a complete test,
or you could build a test using JMeter with the OBIEE JDBC driver for a more flexible and scalable test rig.
pdfcrowd.com
What this method doesnt test is Presentation Services and upstream, so the additional processing of resultsets, passing
the data back to the web browser, and the rendering of it. But, if you are targeting your testing at just the RPD and Database
then this is a very good way to avoid unnecessary complexity.
Database
If nothing in your OBI configuration has changed and you just want to test the
performance of the database, then you can just use standard database
performance test tools. Take the Physical SQL from the report being tested
and run it against the database using your databases standard tools.
This is useful if you are doing tuning work at the database level (partitioning,
indexing, etc), but only use this approach if nothing is changing in your RPD.
As soon as the RPD changes, different physical SQL could potentially result.
Remember also that the response time from a physical SQL executed on the
database is not the same as the response time of a report it still has to go
pdfcrowd.com
back up through the stack back to the users web browser. For a complex
report or a large number of rows this can add significant time.
Which tool?
In an OBI performance test lifecycle I would expect to use several of the different tools described above. Typically a greenfield
test will look at the performance of the stack end-to-end. If there are problems with the end-to-end, or if the testing is in
response to a problem in an existing system, I would use a response time profile (of which more later) to understand where
the bottleneck lay and focus the testing (and tuning) there. For example, if the bottleneck appeared to be the SQL that the
RPD was generating I might use nqcmd to validate changes I made to the RPD and/or the database model
pdfcrowd.com
1. If you use shell scripting, dont forget to KISS a complex mess of shell scripts isnt Simple either.
3. You may see a conflict between the above points. How do you build a test that is granular and well instrumented, yet
isnt overly complex? Well, therein lies the gold ;-) Because all performance tests are different, I can only outline here
good approaches to take, but you have to walk that line between complexity and flexibility in your design.
Resource management
To reduce the risk to a Production system of running ad hoc or untested reports, resource management at the database level
could be used. On Oracle, Database Resource Manager (DBRM) enables queries to be allocated a greater or lower priority
for CPU on the database, or limit the execution time. Fixed reports which have been through a strict performance QA process
pdfcrowd.com
could be given greater resource since the effect has been quantified and analysed in advance, whilst ad hoc or unproven
reports could be more restricted.
To implement this, pass through the report or user name through in the connection pool, and then define a Resource
Manager rule to assign the resource management policy accordingly.
Resources can also be controlled at the BI Server level by setting restrictions on query runtimes and rows returned on a perapplication role basis.
What next?
Now that weve defined the test we want to perform, and have designed how it will be implemented, we move onto actually
building it, here
Introduction
Test Define
Test Design
Test Build
Execute
Analyse
Optimise
Summary and FAQ
Comments?
Id love your feedback. Do you agree with this method, or is it a waste of time? What have I overlooked or overemphasised?
Am I flogging a dead horse?
Because there are several articles in this series, and Id like to retain the discussion in one place, Ive enabled comments on
the summary and FAQ post here, and disabled comments on the others.
Related Posts:
Advanced Presentation Services settings for OBIEE testing & development
Performance and OBIEE part I Introduction
pdfcrowd.com
Tw eet
Like
Services
About Us
> Consulting
> Training
> Support
>
>
>
>
About us
About our team
Contac t us
Our clients
Consulting
Services
>
>
>
>
>
>
Projects
Expert Services
OBIEE 11g
Sustainability
On Discoverer?
Orac le DW
Training
Resources
Blog Authors
> OBIEE
Bootcamp
> OBIEE End-User
> Exalytics
> ODI 11g
Bootcamp
> Oracle BI Apps
> Articles
> Blog
> OBIEE 11g
>
>
>
>
>
>
>
Mark Rittman
Venkat J
Peter Scott
Borkur S
Mike Vickers
Robin Moffatt
Jon Mead
pdfcrowd.com
Regent Prime
Whitefield Main Road
Whitefield
Bangalore
560066
Rittman Mead Belgium
Registered Office : Chausse de Louvain 426
1380 Lasne
Belgium
Privacy Policy
pdfcrowd.com