Oracle E-Business Suite Information Discovery

Performance Guide

ORACLE WHITE PAPER | APRIL 2016

Table of Contents

1. Overview 1

2. Oracle Endeca Server 2

2.1. Performance Issue Indicators 2

2.2. Tuning the Dgraph Process Cache 2

2.3. Multithreaded Mode 5

2.3.1. Monitoring CPU Utilization 5

2.3.2. Tuning the Dgraph Computed Threads 7

3. Oracle Endeca Studio Server 9

3.1. Monitoring Performance of Studio Portlets 9

3.1.1. Measuring Loading Time for Portlet Client Studio Pages 9

3.1.2. Measuring Individual Portlet Performance 10

3.2. Monitoring Query Performance 12

3.2.1. Configuring the Metrics Data to Record 12

3.2.2. Metrics Log File 13

3.2.3. Optimizing EQL 14

3.2.4. Performance Metrics Page 14

ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

1. Overview
Oracle E-Business Suite Information Discovery extends the Oracle E-Business Suite applications by
leveraging Oracle Endeca Information Discovery. Oracle Endeca Information Discovery is an
enterprise discovery platform for the analysis of information from both structured and unstructured
sources, providing the ability to search for and visualize data in many different ways.

The goal of this document is to give users an overview of the performance of Oracle E-Business Suite
Information Discovery and some basic instructions on how to enhance performance for Oracle
E-Business Suite Information Discovery products.

1 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

2. Oracle Endeca Server
Oracle Endeca Server query performance depends on many characteristics of your specific deployment, such as
query workload, query complexity, data domain configuration, and the characteristics of the loaded records, as well
as the size of the data domain's index. In view of these characteristics, a hardware sizing must be performed prior to
deployment, to assess memory consumption and other hardware needs of your deployment. Also, performance
depends on the number of cores; the computed number of threads must be adjusted to utilize the full capacity of a
multithreaded environment.

The Oracle Endeca Server application services requests for one or more Endeca data domains. Each Endeca data
domain has one Dgraph process, or, if you implement clustering, a set of one or more Dgraph processes, that
handle end-user query requests. The Dgraph uses proprietary data structures and algorithms that allow it to provide
real-time responses to client requests. It stores the data files that were created from loading the data into it. After the
data files are stored, the Dgraph:

» Receives client requests through the application tier

» Queries the data files
» Returns the results
The Dgraph is designed to be stateless. This design requires that a complete query be sent to it for each request.
The stateless design facilitates the addition of Dgraph processes for load balancing and redundancy.

2.1. Performance Issue Indicators

Performance issues that users could face while accessing Endeca pages might be related to the utilization of the
cache or threads. Some indicators of performance issues are:

» An Endeca page takes too much time to load.

» Portlets time out.
» Bulk load takes too much time.
» The UI becomes nonresponsive.

2.2. Tuning the Dgraph Process Cache

The Dgraph cache is a storage area in memory that the Dgraph uses to dynamically save potentially useful data
structures, such as partial and complete results of processing queries. Oracle Endeca Server uses caching of
results for the Dgraph processes extensively, for many types of queries.

Allocating more memory to the cache improves performance by increasing the amount of information that can be
stored in it. Thus, this information does not have to be recomputed. Such a caching strategy allows the computing
resources to be utilized better, and therefore improves throughput of the Oracle Endeca Server.

Access the Dgraph statistics page for each Dgraph process running for your data domain, using the following URL:

http://<host>:<port>/endeca-server/admin/<data_domain>?op=stats

Replace < host> and <port> with the host name and port of the Oracle Endeca Server, and <data_domain> with the
name of your data domain.

Note: If your data domain runs on more than one Oracle Endeca Server instance, you should access this page on
each system hosting the Oracle Endeca Server for this data domain.

On the Dgraph statistics page, examine the Dgraph cache, in particular the number of evictions. If this number is
high and the hit ratio is low, it means that the default value is insufficient for your data domain, as shown in Figure 1.

2 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

Figure 1 Eviction Time

On Linux-based systems, you can use operating system tools such as top, ps -efl, or ps -aux to look at the
size of Dgraph processes to determine the process’s resident set size in the physical memory on the machine, and
the actual amount of memory the given process consumes. For example, Figure 2 shows the sample output for the
top command:

Figure 2 Sample Output for Memory Usage Using the Top Command

Use the following recommendations when tuning the performance of your applications running on top of the Oracle
Endeca Server.

» Check the current configuration setting for the Dgraph cache using the following command:

3 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

 ./endeca-cmd get-dd-profile FND_EID_DD_DEFAULT_PROFILE

where FND_EID_DD_DEFAULT_PROFILE is the default name of the profile used by your data domain. This
command returns the setting for the --compute-cache-size for all Dgraph processes serving your data
domain, as shown in Figure 3. This setting represents the amount of RAM, in MB, allocated to the result cache on
each Dgraph node in the data domain.

Figure 3 Computed Cache Size

The default is 0, which is interpreted as follows: when an absolute value is set to 0, the default Dgraph cache size
is computed as 10% of the amount of RAM available on the Oracle Endeca Server node hosting the Dgraph
node.

Alternatively, you can check the cache size for each Dgraph by running the grep command from Linux, as follows:

ps aux | grep dgraph

Figure 4 Cache Size Using grep Command

» Experiment with gradually increasing the Dgraph cache size. For example, you might start by changing the cache
size to 20% of memory instead of leaving it at the default value. You should have enough memory in the system
to accommodate the new value based on the number of data domains that you have on the system. If you later
find that queries are getting canceled because there is not enough available memory to process them, then you
should decrease this amount.
Note: To specify the new value; you would need to calculate the size in MB that corresponds to 20% of the
memory, based on the size of the RAM on the system hosting the Oracle Endeca Server.

4 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

 Be careful when increasing the cache size significantly, such as to 50% of memory, because the disadvantage of
having a large cache size is that memory used by the cache is not available for query evaluation and is not
released back from the cache.

To specify a new Dgraph cache size for a data domain, perform the following steps:

1. Modify the EidConfig.properties file:

» Set the FND_EID_DD_PROFILE_PRODUCTS parameter to a list of the products whose cache you want to
tune.
» Set the FND_EID_DD_CACHE_SIZE parameter to the new cache size value.
2. Run the postSetup.sh script and select Option 2 - Create DD Profile.

3. Run the command endeca-cmd get-dd-profile FND_EID_DD_DEFAULT_PROFILE to return the new setting
for the --compute-cache-size parameter.

4. Monitor the Dgraph cache again on the statistics page, where you should see that the number of evictions is
low and the hit ratio is high.

2.3. Multithreaded Mode

Oracle Endeca Server always runs in multithreaded mode, with the total number of threads set to 4 by default.
Oracle recommends increasing this number to maximize the use of system resources. On processors that are
hyperthreading or multicore, multiple query threads can use a single processor at the same time.

2.3.1. Monitoring CPU Utilization

On Linux, you can examine the CPU utilization using standard tools, such as the top command.

Figure 5 CPU Usage

You can check CPU usage and throughput for the Dgraph by accessing the statistics page:

http://<host>:<port>/endeca-server/admin/<data_domain>?op=stats

5 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

Figure 6 Endeca Server Statistics

You can use command-line tools to review your current configurations.

To check the number of available cores, use the following command:

less /proc/cpuinfo

Figure 7 Available Cores

To check the number of threads for each Dgraph, which is set to 4 by default, you can run the grep command from
Linux as follows:

ps aux | grep dgraph

6 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

Figure 8 Computed Threads

Alternatively, you can run the following endeca-cmd command:

./endeca-cmd get-dd-profile FND_EID_DD_DEFAULT_PROFILE

Figure 9 Number of Computed Threads

2.3.2. Tuning the Dgraph Computed Threads

You can specify the number of computational threads in the Dgraph node's threading pool using the parameter
---num-compute-threads. The value for this parameter must be a positive integer that is equal to or greater
than 2. It is recommended that you tune your Dgraph computed threads in the following cases:
» If you have a high number of concurrent users: Check how many users are currently logged into the
application and engaged in making simultaneous requests to the Oracle Endeca Server. You should check the
peak load for simultaneous requests during a specific time period.
» If you have a high volume full load: Typically the full load process uses a bulk add/replace interface to ingest
data into the data domain. This interface will use up all the threads assigned to the data domain process to handle
the ingest process. Tuning the number of threads assigned to the data domain will affect and improve the
performance of the bulk add/replace API and, consequently, the full load process.
The upper limit for the computed threads is typically the number of cores. However, hyperthreading systems can
allow a maximum value beyond the number of cores. If no value is specified in the EidConfig.properties file, then the
default value is set to 4.

To specify a new number of computed threads for the data domain, perform the following steps:

1. Modify the EidConfig.properties file:

» Set the FND_EID_DD_PROFILE_PRODUCTS parameter to a list of the products for which you want to
increase the number of threads to enhance performance while running the graphs.
» Set the FND_EID_DD_PROFILE_THREADS parameter to the new number of threads.
2. Run the postSetup.sh script and select Option 2 - Create DD Profile.

7 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

3. Run the command endeca-cmd get-dd-profile FND_EID_DD_DEFAULT_PROFILE to return the new
setting for the ---num-compute-threads parameter.

8 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

3. Oracle Endeca Studio Server

3.1. Monitoring Performance of Studio Portlets

You can use browser development tools to measure the performance of Studio pages and of the portlets inside
those pages individually. These measurements reflect client-side performance including the network traffic.

3.1.1. Measuring Loading Time for Portlet Client Studio Pages

The easiest method for measuring loading time for portlet client Studio pages is using the Chrome browser
development tools.
1. Open the Oracle E-Business Suite Information Discovery application in the Google Chrome browser.
2. Log in and navigate to the page you want. Then open the Chrome development tools by pressing the F12 button
from the keyboard, or by pressing Ctrl+Shift+I.
3. The Development Tools window occupies the lower part of the page. Select the Network tab, as shown in Figure
10.

Figure 10 Network Tab in Development Tools Window

4. Press the F5 button to reload the whole page.

5. Wait until the page has loaded completely, and then review the status bar at the bottom of the page, as shown in
Figure 11.
» The Load measurement, which appears in red text, displays the elapsed time taken for the whole application
page to load with data.
» You can also review the number of server requests and the amount of data transferred.

9 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

 Figure 11 Application Loading Time In Development Tools Window

3.1.2. Measuring Individual Portlet Performance

If users report issues with response times on a specific portlet, you can use the browser development tools to
measure individual portlet performance. For this example, suppose you want to review the time required for the
results table portlet data to load.
1. Navigate to the page you want, open the browser development tools, navigate to the Network tab, and reload the
page. Then click the filter icon, as shown in Figure 12, to display the filtering bar.

Figure 12 Development Tools Filter Icon

2. In the filtering bar, select the XHR request type.

10 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

 Figure 13 Development Tools XHR Request Type Filtering

3. Type either the word table or the word results in the input field on the left side of the filtering bar. These
criteria will filter the requests to display only results table data loading requests.
4. In the filtered results, review the size of the data in the Size column and the total amount of time taken to load that
data in the Time column.
» The most important key is the portlet ID. This ID is extracted from the Name (Request URL) column as the
value of the parameter called p_p_id.
» For example, consider the following example name:
<url>?p_p_id=endecaresultstableportlet_WAR_endecaresultstableportlet_INSTANCE_k
0OT&<more_params>

In this example, the portlet ID is:

endecaresultstableportlet_WAR_endecaresultstableportlet_INSTANCE_k0OT

5. You can further analyze the time required for each request by hovering the pointer over the small blue bar in the
Timeline – Start time column for each request, as shown in Figure 14.

Figure 14 Hovering Over the Response Time

The following detailed response timings appear:

» Request Sent / Sending: The time spent issuing the network request. Typically a fraction of a millisecond.
» Waiting (TTFB): The time spent waiting for the initial response, also known as the Time To First Byte. This
time captures the latency of a round trip to the server in addition to the time spent waiting for the server to
deliver the response.
» Content Download / Downloading: The time spent receiving the response data. This time can vary
depending on the number of simultaneous requests being sent.

11 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

 Pay particular attention to the Waiting (TTFB) time. If this time is relatively high, it could indicate higher server
loading time, and most probably a slow server query.

The following section describes how to determine whether this portlet calls a specific slow server query.

3.2. Monitoring Query Performance

You can access the performance metrics data from the metrics log file and from the Performance Metrics page.
First, you must configure the metrics that you want to include.

3.2.1. Configuring the Metrics Data to Record

To configure the metrics to include, use the df.performanceLogging setting in the
portal-ext.properties file. This setting applies to both the eid-studio-metrics.log file and the
Performance Metrics page.

The metrics logging can include the following metrics:

» Oracle Endeca Server queries by the data source.

» Portlet server executions by component. The server-side code is written in Java.
» This code handles configuration updates, configuration persistence, and Oracle Endeca Server queries.
» The server-side code generates results and sends them back to the client-side code.
» Server executions include portlet render, resource, and action requests.
» Portlet client executions by component. The client-side code resides in the browser and is written in JavaScript.
» This code issues requests to the server-side code, and then renders the results as HTML.
» The client-side code also handles any dynamic events within the browser.
By default, only the Oracle Endeca Server queries and portlet server executions are included. You can update the
df.performanceLogging setting in the portal-ext.properties file to specify the metrics you want to
include. Specify this setting as follows:

df.performanceLogging=<metrics to include>

Replace <metrics to include> with a comma-separated list of the values for the metrics you want to include.
The following table shows the available metrics values that you can specify in the list.

TABLE 1 AVAILABLE VALUES FOR USE IN THE METRICS LIST

Value Description

QUERY Specify this value to include information for Oracle Endeca Server queries.

PORTLET Specify this value to include information on portlet server executions.

CLIENT Specify this value to include information on portlet client executions.

In the default configuration, where only the Oracle Endeca Server queries and portlet server executions are
included, the value for this setting is specified as follows:

df.performanceLogging=QUERY,PORTLET

To include all of the available metrics, add the CLIENT option, as follows:

df.performanceLogging=QUERY,PORTLET,CLIENT

12 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

Note: For performance reasons, this configuration is not recommended in production.

If you do not specify a value for this setting, then the metrics log file and the Performance Metrics page will not
display any metrics data.

df.performanceLogging=

3.2.2. Metrics Log File

The eid-studio-metrics.log file contains the metrics logging information. It is located in the same directory as
the eid-studio.log file. The metrics log file contains the following columns:

TABLE 2 COLUMNS IN THE METRICS LOG FILE

Column Name Description

Total duration (msec) The total time for this entry (End time minus Start time).

Start time (msec since epoch) The time when this entry started.
» For Oracle Endeca Server queries and server executions, this column uses the
server's clock.
» For client executions, this column uses the client's clock.

End time (msec since epoch) The time when this entry finished.
» For Oracle Endeca Server queries and server executions, this column uses the
server's clock.
» For client executions, this column uses the client's clock.

Session ID The session ID for the client.

Page ID If client instrumentation is enabled, the number of full page refreshes or actions that the
user has performed. This value is used to help determine how long it takes to load a
complete page.
Some actions that do not affect the overall state of a page, such as displaying attributes
on a Guided Navigation component, do not increment this counter.

Gesture ID The full count of requests to the server.

Portlet ID The ID associated with an individual instance of a component. The ID generally

comprises the following:
» The type of component
» A unique identifier
For example, if a page includes two Chart components, you can use the ID to
differentiate them.

Entry Type The type of entry. For example:

» PORTLET_RENDER - Server execution in response to a full refresh of a component
» DISCOVERY_SERVICE_QUERY - Oracle Endeca Server query
» CONFIG_SERVICE_QUERY - Configuration service query
» SCONFIG_SERVICE_QUERY - Semantic configuration service query
» LQL_PARSER_SERVICE_QUERY - EQL parser service query
» CLIENT - Client-side JavaScript execution
» PORTLET_RESOURCE - Server-side request for resources
» PORTLET_ACTION - Server-side request for an action

Miscellaneous A URL-encoded JSON object containing miscellaneous information about the entry.

13 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

The following example shows a sample record in the log file:

2156 1392134019967 1392134022123

HVQdS6GXfWLR1J5CQ6qFGyWvvll0KF6TvnyhtWGkgdyLwPCsZJJ5%21-
1602730605%211392133783831
endecametricsbarportlet_WAR_endecametricsbarportlet_INSTANCE_o3Si_LAYOUT_15211
DISCOVERY_SERVICE_QUERY {"url":"<encoded_url>","dataSourceId":"qa-quality"}

In this example, the portlet ID is

endecametricsbarportlet_WAR_endecametricsbarportlet_INSTANCE_o3Si.

The portlet ID in the metrics log file should match the portlet ID you obtained in step 4 of Section 3.1.2. Measuring
Individual Portlet Performance.

Check whether the start time and end time of the query reflect a slow server query. If the total duration is relatively
high, you can optimize the query as described in the following section.

3.2.3. Optimizing EQL

You can optimize the EQL performance by following these guidelines:

» Do not include more than 40 attributes in one statement.

» Create a separate EQL for aggregation.
» Create an EQL to join between the aggregated attributes and other attributes using JOIN ... ON.
You can also check the overall page performance using the Performance Metrics page, as described in the following
section.

3.2.4. Performance Metrics Page

The Performance Metrics page on the Control Panel displays information about component and Oracle Endeca
Server query performance. For each type of included metric, the table at the top of the page contains a collapsible
section, as shown in Figure 15.

14 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

Figure 15 Performance Metrics

For each data source or component, the table tracks the following information:

» Total number of queries or executions

» Total execution time
» Average execution time
» Maximum execution time
For each type of included metric, the page also displays a pie chart summarizing the average query or execution
time per data source or component.

Figure 16 Oracle Endeca Server Queries by Avg Time, ms

15 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

Figure 17 Portlet Server Executions by Avg Time, ms

Note: Oracle Endeca Server query performance does not correlate directly to a Studio application page, because a
single Studio page often uses multiple Oracle Endeca Server queries.

16 | ORACLE E-BUSINESS SUITE INFORMATION DISCOVERY PERFORMANCE GUIDE

 Oracle Corporation, World Headquarters Worldwide Inquiries
500 Oracle Parkway Phone: +1.650.506.7000
Redwood Shores, CA 94065, USA Fax: +1.650.506.7200

CONNECT W ITH US

blogs.oracle.com/oracle
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the
contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other
facebook.com/oracle warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or
fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are
formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any
twitter.com/oracle means, electronic or mechanical, for any purpose, without our prior written permission.

oracle.com Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.

Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and
are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are
trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0116

Oracle E-Business Suite Information Discovery Performance Guide

April 2016