How to avoid Informatica - Salesforce

connection timeouts






Sandesh Dhoka








Table of Contents
1. Abstract ................................................................................................................................................ 3
2. Introduction .......................................................................................................................................... 3
3. Resolution ............................................................................................................................................ 4
To resolve the connection timeout, we need to consider the below environment setting and
coding practices...................................................................................................................................... 4
3.1 At Informatica environment ....................................................................................................... 4
3.1.1 Settings at Informatica Session ........................................................................................ 4
3.1.2 Settings at Informatica Integration service ..................................................................... 5
3.1.3 Example ............................................................................................................................... 5
3.2 At Salesforce environment ........................................................................................................ 7
4. Salesforce Informatica Session log ................................................................................................. 7
5. Some additional ways to resolve timeout issue ............................................................................. 7

















1. Abstract
Connection timeout is one of the most important problems that teams face during
Informatica-salesforce data integration projects. There could be many contributing
factors such as huge volume of data in Salesforce, poor SOQL query or issues with
environment setting with Informatica and Salesforce. There are some good practices
that ETL developers should follow to resolve the timeout issue. This paper covers
all the aspects from Informatica and Salesforce which should be considered while
designing and developing applications that integrate Informatica with Salesforce.


2. Introduction
With increased use of cloud based systems for ERP, it is important for businesses to
effectively integrate data on cloud with data in legacy applications. As a result many
data-warehouse projects aim to achieve this goal.
Salesforce is one of the major CRM tools in market used by customers. Salesforce is
integrated with Informatica for data integration projects.
What is Salesforce connection timeout?
In Salesforce framework, any query cursor that is idle for more than specified time will
be timed out. Once Query is submitted to salesforce, Query Cursor read the returned
records, if query cursor doesnt receive the expected data from salesforce within
particular time period, it goes for timeout.
For environment setting, we need to set a time period for which the PowerExchange for
Salesforce adapter would wait for a response from Salesforce before timing out.
Sample connection timeout error in Informatica -


Salesforce connection timeout should be more than 420 seconds, its 600 seconds in
below SFDC Informatica log


3. Resolution
To resolve the connection timeout, we need to consider the below environment setting
and coding practices.
3.1 At Informatica environment

3.1.1 Settings at Informatica Session

We need to increase the timeout interval at session level, by default its 300 seconds.
We can extend it up to 600 seconds.
Ideally, Attribute value of SalesForceConnectionTimeout variable should be in the range
of 420 to 600.










3.1.2 Settings at Informatica Integration service

There are also timeout settings available in Informatica Salesforce.com Connector for
Power Center. If its only set to 300 seconds and SFDC is set at 600 seconds the job
would still have timeout issues because it would be the Informatica connector timing out
first. Settings at integration level can be done by Informatica admin. They can extend it
to 600 seconds.
The SalesForceConnectionTimeout flag should not be set higher than 600 seconds (10
minutes). The ideal range would be between 420-600 seconds. This flag determines the
time period for which the PowerExchange for Salesforce adapter would wait for a
response from Salesforce before timing out. If there is no response after 10 minutes, the
connection is timed out and then a second attempt is made to connect. There can be
three attempts for establishing connection. Note that the response may not have been
received due to network, proxy server issues.
In Salesforce framework, any query cursor that is idle for more than 15 minutes will be
timed out. Therefore, setting connection time out value higher than 10 minutes will be
counterproductive and cause more errors.

3.1.3 Example

If the SalesForceConnectionTimeout is set to 1200 seconds (20 minutes) and the
response has not been received, the adapter will wait for 20 minutes and then time out.
By this duration, the query cursor has already timed out at 15 minutes. When the
second attempt is made after 20 minutes, there will be query_invalid_locator errors
because the query cursor has terminated at the end of 15 minutes.

Thus, there is a higher chance for the flag to be effectively applied for network latency
issues if it is set to 420-600 seconds. If set to 420 seconds, there is a window for two
successful attempts before the query is terminated due to being idle.

To add an undocumented PowerCenter parameter (service Custom Property) does the
following in the Informatica PowerCenter Administration Console:
1. Stop the service (Integration Service, Repository Service or other service).
2. Select the service (Integration Service, Repository Service or other service).
3. Under the Properties tab, click Edit in the Custom Properties section.
4. Under Name enter the name of the custom property.
5. Enter the value for the parameter under Value:
6. Click OK.
7. Start the service.








3.2 At Salesforce environment

Salesforce has a default timeout limit as 2 minute. Salesforce admin can extend the
timeout limit up to 10 minute which is max limit at salesforce. We need to make sure
that salesforce has a maximum timeout limit specified in salesforce environment setting.
There is nothing we can do on our end in Informatica coding to increase the salesforce
timeout period. It has to be done by salesforce admin. Salesforce suggest to use the
latest API 26/27 instead 20. This API connection string is required while setting up a
connection between Informatica and salesforce.

4. Salesforce Informatica Session log

The PowerExchange for Salesforce.com session logs are different from the
PowerCenter session logs. The PowerExchange for Salesforce.com session logs
contain record-level details that are specific to sessions with Salesforce.com targets.
You can also use the PowerCenter session logs to view information about the tasks
performed by the Integration Service, general session errors, and load summary and
transformation statistics for the session.

5. Some additional ways to resolve timeout issue

1. Sometimes rerun the query will help, since the query will already be cached on
Salesforce server it will run faster and return the result soon. Since the caches
are cleared randomly at Salesforce we dont have control over it.
2. Use filters at mapping level rather than at SOQL filter Sometime filters present
at SOQL takes time to execute and cause a timeout issue. Rather we can
process all the records and filter out them at mapping level. This is time
consuming but better than causing failures.
3. Use Query All option If we use Query all option present at session level then it
will not filter out logically deleted records within salesforce. So time required to
filter out logically deleted records will be optimized.
If we dont need logically deleted records to be processed then we can filter out
them at mapping level rather than forcing it to salesforce SOQL.
4. Process salesforce records in batches
If you get a timeout error when processing a batch, split your batch into
smaller batches, and try again. There is a five-minute limit for processing 100
records. Also, if it takes longer than 10 minutes to process a batch, the Bulk
API places the remainder of the batch back in the queue for later processing. If
the Bulk API continues to exceed the 10-minute limit on subsequent attempts,
the batch is placed back in the queue and reprocessed up to 10 times before the
batch is permanently marked as failed.
Even if the batch failed, some records could have completed successfully. If you
get a timeout error when processing a batch, split your batch into smaller
batches, and try again.
5. Make sure that same object is not read concurrently
6. Use Relationship Query while fetching Child Data in reference to Parent instead
of using Informatica Joins
7. Avoid multiple runs from Salesforce Source at the same duration as it uses the
same cache


References
http://www.salesforce.com/
http://www.informatica.com/us/