Documente Academic
Documente Profesional
Documente Cultură
This document describes products and services of Pegasystems Inc. It may contain trade secrets and proprietary information. The document and product are protected by copyright and distributed under licenses restricting their use, copying distribution, or transmittal in any form without prior written authorization of Pegasystems Inc. This document is current as of the date of publication only. Changes in the document may be made from time to time at the discretion of Pegasystems. This document remains the property of Pegasystems and must be returned to it upon request. This document does not imply any commitment to offer or deliver the products or services described. This document may include references to Pegasystems product features that have not been licensed by your company. If you have questions about whether a particular capability is included in your installation, please consult your Pegasystems service consultant. For Pegasystems trademarks and registered trademarks, all rights reserved. Other brand or product names are trademarks of their respective holders. Although Pegasystems Inc. strives for accuracy in its publications, any publication may contain inaccuracies or typographical errors. This document could contain technical inaccuracies or typographical errors. Changes are periodically added to the information herein. Pegasystems Inc. may make improvements and/or changes in the information described herein at any time.
This document is the property of: Pegasystems Inc. 101 Main Street Cambridge, MA 02142-1590 Phone: (617) 374-9600 Fax: (617) 374-9620 www.pega.com PegaRULES Process Commander Document: Declare Indexing Software Version 5.2 Updated: April 5, 2007
Contents
Overview..............................................................................................................1 Implementing a Declare Index............................................................................2 Define Purpose and Structure of Index........................................................2 Inheritance and Rule Resolution........................................................... 3 Create Index Class and Properties..............................................................4 Define Database Structures .........................................................................5 Blob Column.......................................................................................... 6 Create Declare Index Rule...........................................................................6 Index Class Data section ...................................................................... 7 Properties for Indexing And Mapping section ...................................... 9 Indexing Conditions section................................................................ 13 Using a Declare Index...................................................................................... 16 Changing a Declare Index ............................................................................... 21 Changing a Declare Index Instance ................................................... 21 Inconsistent vs. Orphaned index data.............................................. 21 Deleting a Declare Index Instance...................................................... 22 Reindexing........................................................................................... 22
Overview
Overview
Beginning in Version 4.1, declarative indexing was added as an easier and more efficient way to select data stored in the relational database, for a variety of data functions: ordering, reporting, updating/changing, gathering for input into activities, etc. Declare indexes allow access to data that was previously not exposed as database columns. When adding data to tables in the database, PegaRULES stores most of its information in one large column (called the Blob, for Binary Large OBject), with only a few key properties as separate columns in the table. This is a far quicker and more space-efficient way to store the data than having each property have its own column, but the information in the BLOB cannot be accessed directly for reporting or creating lists for inputs. By using Rule-Declare-Index to create a declare index, some or all of the properties that were stored in the BLOB by the standard setup can be exposed in database columns, allowing direct and easy access to the data through report queries. Just as with other declarative processing classes, Declare Indexes are automatically updated as processing runs in the system. The Rule-Declare-Index instances create a relationship between the instance as it is stored in the database (not changed on a page in the clipboard, as with Declarative Processing) and the index; changes will not be recorded unless data is saved/committed. As activities are run, the system keeps track of all the properties that are changed. When the changes are saved to the database, the system uses that list of changed properties to determine what indexes need to be updated to reflect the changes, and then updates those indexes automatically.
IMPORTANT: The overall concept behind a Declare Index is that there are properties in the system which arent exposed, or which are in different classes, and which cannot be easily reported on. One of the main jobs of the Declare Index is to gather together these properties from various places, and to map their values to properties defined in the Index class. The Index properties are kept in sync with the values of the original properties due to the declarative functionality. Then, the properties in the Index class, which are all in one place and in exposed columns in a database table, may be easily displayed in reports. Therefore, the purpose of the Declare Index rule is to specify where the original (Source) properties are, and point them to the properties in the Index class.
CONFIDENTIAL
CONFIDENTIAL
The decision must be made based on what would be most efficient for the companys system.
In the above table, there are a number of Indexes defined against various classes under different levels of YourCo-. For a property on a page on the clipboard, which has a top-level page of class of YourCo-CustService-Loan-Request, all of these Indexes will be tested to check whether the properties being set will cause changes in an Index (as described in the Rule-Declare-Index sections later in this document), except: the Resolved instance defined on the class YourCo-CustService-LoanRequest will supercede the one written on the class YourCo-CustService, as it is more specific; the Vendors instance defined on the class YourCo-CustService-Loan will supercede the one written on the class YourCo-CustService, as it is more specific.
This follows standard rule-resolution practice. NOTE: Depending upon the profiles of the users involved, due to Rule Resolution, different users may end up using different versions of an index, even though they are
CONFIDENTIAL
working on the same class instance. Please reference the Rule Resolution tech note for full details.
IMPORTANT NOTE: Inheritance is followed on a Purpose-by-Purpose basis not based on the individual properties that are defined in that Purpose. For example, from the above RuleDeclare-Index list, there may be several Properties that were indexed in the Vendors Purpose defined on the class YourCo-CustService: CustomerName Address ZipCode There may also be one Property that was indexed in the Vendors Purpose defined on the class YourCo-CustService-Loan: CustomerName
When (by inheritance) the Vendors instance defined on the class YourCoCustService-Loan supercedes the one written on the class YourCo-CustService, only the single Property in that Purpose will be saved as an index entry. The other properties will not be index entries, as they are in the superceded Purpose. Thus, Purposes should be considered atomic units. If specific properties in an Index need to be inherited on their own, they should be defined in separate Purposes.
Creating the class and the properties before saving Rule-Declare-Index instances is PegaRULES Best Practice, and will be supported with validation in the future.
CONFIDENTIAL
Below are some of the properties that are defined on the class Index-, which are exposed in the database table pr_index: Property pzInsKey pxInsName pxObjClass pxIndexCount pxIndexPurpose pxInsIndexedKey pxInsIndexedClass pxCreateDateTime pxUpdateDateTime pxCreateOpName pyLabel pxCreateOperator pxCreateSystemID pxUpdateOpName pxUpdateOperator pxUpdateSystemID Datatype char char char number char char char date date char char char char char char char Width 255 128 96 32 255 64
32 64 32 32 32 32 32
CONFIDENTIAL
Note that there should not necessarily be a one-to-one relationship between the Index- classes and the index tables that are created in the database. Just as with the Rule- classes, several Index- classes may be stored in one database table. However, because all the columns are exposed in the index tables, the grouping of Indexclasses to database tables must be carefully designed to maximize efficiency in the customer implementation. Index classes with common properties can be stored in the same table; indexes with completely different properties may be more efficiently stored in separate tables. For example, if a customer has several declarative indexes defined that all reference properties having to do with client information, they could all be stored in the same database table. Indexes with properties that contain product information should be stored in a separate table. Performance is also an issue if a particular index is going to be heavily used or frequently updated, that data should probably not be in the same table as data that is less heavily used or less frequently updated. When a change is made to an instance of a class that has a Rule-Declare-Index instance defined, the current data for the Source Properties in the Rule-Declare-Index instance will be saved to the appropriate index table. Each instance of a class (upon which an index is defined) will be one index entry in that index table, with the appropriate values saved in each property column.
Blob Column
Despite the Best Practices recommendation above to create a table without the BLOB column (pzPVStream), the base Index- class maps to the pr_index database table, which does includes a BLOB column. This is to prevent data loss, if a customer saves information in properties which are not mapped to an existing column. A customer may create a new Index- class with properties that are not in the original Index- class (listed above). If this customer does not also create a specific index table in the database that includes columns for these new properties - or creates the table, but does not explicitly map this class to the new index table - then the new class by default will be stored in the existing pr_index table, and the data for the new properties (which do not have corresponding columns in the database table) will be stored in the BLOB, until the customer properly maps the Index classes to a new index database table with the appropriate columns exposed.
CONFIDENTIAL
The Applies To field at the top of the form holds the Class on which the Index is defined. This is the class of the top-level page that holds the Source Class Properties listed in this instance. Purpose is the name that shows the reason for this Index. NOTE: The concatenation of the Class and Purpose values become the Index name.
CONFIDENTIAL
The page context path may: refer to a specific embedded page within an instance contain a pattern that matches a number of embedded pages within an instance be blank (in which case it refers to the entire instance page) For a specific class instance and an index definition that applies to it, any page within the instance (including the top-level instance page itself) that matches the index definitions page context path is called a context page. For example, in the form shown above, the class of the top-level page is Work-, and the Source Page Context is .pyWorkParty( ). So any page in the system where the top-level page is of class Work- and the embedded page is one of the values of .pyWorkParty would be considered a context page for this particular index. Note that it is possible to specify embedded pages in the Source Page: Invoices.Orders(2).Items(Manuals) In addition, wildcard subscripts are permitted when specifying a PageList or a PageGroup in the Source Page Context. To use the wildcard designation, the List or Group specified should remain blank: .Orders( ).Customer.Items( ) For Indexes, this means that each value of the group will be stepped through, and a single index record will be created for each embedded page (in a top-level page) that fits the page context pattern. For the above example, a separate index might be created for: .Orders(Cust).Customer.Items(Shirt) .Orders(Cust).Customer.Items(Shoes) .Orders(Cust).Customer.Items(Coat) .Orders(Indiv).Customer.Items(Shirt) .Orders(Indiv).Customer.Items(Shoes) .Orders(Indiv).Customer.Items(Coat) Realize that if a change were made to a Source Class Property in the RuleDeclare-Index instance with one of the above example Page Contexts (on the page .Orders(Cust).Customer.Items(Shirt) for example), then only the change to the actual affected Index would be visible to the user. However, due to the wildcard context path in the index definition, the Index entries for all the above permutations would be regenerated when the change was Committed to the database. Thus, using wildcards in an index Source Page Context can have a definite performance impact developers should keep this in mind when designing their indexes. Source Page Context Class holds the Class name of the page in the Source Page Context. NOTE: The symbolic classes $ANY, $CLASS, $NONE are not allowed as entries in the Declarative forms. Index Class to Write: the class which will index the information (example: IndexZipcode); this is the class where all the Index properties are defined. This field
CONFIDENTIAL
establishes the link between this Index class (Index-WorkPartyURI) and the class being indexed (Work-).
The property references may be relative to the Source Page Context path of the instance being indexed, or they may be fully-qualified properties on other existing pages (see Valid Source Class Property References below). The property reference can include embedded pages, as long as it evaluates to a single property: .Orders(Indiv).Customer.Items(Coat).Price This field must evaluate to a single scalar property. Aggregate properties are not supported; neither are Java Object property types (which dont contain properties to put into columns in the Index table). Thus, wildcard references are not permitted. NOTE: The Source Class Property does not have to be exposed in the database in order to be mapped to an Index Class Property. Valid Source Class Property References 1. 2. 3. 4. Simple Property References References containing Top or Parent References to other Top-Level Pages Constants
1. Simple Property References These are references to properties on the Source Context page - either directly on this page, or in embedded pages or lists. Examples: .pyUserIdentifier .Orders(Indiv).Customer.Items(Coat).Price The property reference using a period is relative, in that it gives the location of the property relative to the Source Page Context page.
2. References Containing Top or Parent If a property reference starts with the word Top, then the rest of the reference is taken to be relative to the top-level page where the property is found - not to the source context page (which may be several levels down). The word Top may only appear
CONFIDENTIAL
once in the reference, at the beginning of a reference. Also, there must not be a period at the beginning of the reference. Example: Top.pyLabel
This property reference is relative, in that it gives the location of the property relative to the existing page. There must not be a period at the beginning of the reference. If a property reference starts with the word Parent, then the rest of the reference is taken to be relative to the page which contains the Source Context page (the parent page of the Source Context page), not the Source Context page itself. The word Parent may only appear once in the reference (or more than once for Version 5.2), at the beginning of the reference; also, there must not be a period at the beginning of the reference. Example: Parent.pyLabel
NOTE: Beginning in Version 5.2, it is possible to use multiple instances of the parent keyword: Parent.Parent.pyLabel (going to the parent page of the parent page of the page where .pyLabel is found)
The number of valid Parent references may not exceed the scope of the Page Context. (If the Page Context is embedded three layers down, having five Parent layers is not valid.) This syntax is only valid in Version 5.2 and later; it is not valid for versions prior to Version 5.2. The Parent property reference is relative, in that it gives the location of the property relative to the Source Page Context page.
3. References to Other Top-Level Pages These are references to properties on other top-level pages on the clipboard. Example: pxRequestor.pyUserIdentifier
IMPORTANT: Only the properties defined within the Source Context Page will trigger an index entry update if they are changed. Property references using Top, Parent, or referencing other top-level pages will not trigger an index update. When the Declare Index processing is triggered by a change in a property value, the system checks to see whether the changed property is on a page which matches both the top-level page and the context path of the defined index. If the changed property is on a different page or context path, even though it is properly referenced in the index definition, this change will not trigger the declarative processing, because it isnt on the page defined in the index. The index will not necessarily be updated unless another property located on the context page where the index is defined triggers the index regeneration. (See Example 1 below.)
10 CONFIDENTIAL
This issue is also present for properties which are on the same top-level page, but in a different context path, or for properties which are defined as Top or Parent. (See Example 2 below.) Because these types of property references do not trigger the index updates, it is recommended that if such properties are to be included as Source Properties in a Declare Index rule, they should be properties with more static values (like Work ID, which should not change once the work item is defined, or a customer Company Name). If the customer must have properties in a different place than the context path triggering index updates, they should design a different definition for their Declare Index.
Example 1: There are two top-level pages on the clipboard: PageA PageB The class of PageA is Work-, which has a Rule-Declare-Index instance definition with purpose PartyURI. One of the Source Properties in this Index definition is PageB.pyStatusWork, which is located on the other top-level page, PageB (of class Data-). As stated earlier, an index is updated only when values of the Source Properties on a context page of that index change. Therefore, if the value of the property .pyStatusWork on PageB changes, then that change will not be tracked for the PartyURI index, because the Index is defined on PageA. It is only when the value of some other Source Property in the PartyURI index changes on PageA (such as .pxPartyRole) that the appropriate PartyURI index entry for this class instance will be regenerated and the value from PageB.pyStatusWork recorded - because that is where the PartyURI index is tracked.
CONFIDENTIAL
11
Example 2: Below is a slightly different definition of properties for the PartyURI index.
In this example, there are two properties with embedded pages defined in the Workclass: .pyWorkParty( ) [embedded pagegroup of class: Data-Party] .pyStatusWork [embedded page of class: Work-Object] The Declare Index in this example is defined as follows: Applies To: WorkPurpose: PartyURI Source Page Context: pyWorkParty( ) Source page Context Class: Data-Party All of the properties in the diagram above in .pyWorkParty are in the Declare Index definition. In addition, the following properties are also referenced: Top.pyStatusWork.pyWorkStatus Top.pyStatusWork.pyWorkID Even though these two embedded pages share the same top-level page, they have different context paths. Therefore, if one of the .pyStatusWork properties changes, that will not necessarily trigger an index entry for the PartyURI index.
12
CONFIDENTIAL
4. Constants A source class property is recognized to be a constant if it starts and ends with a double quote ("). Example: "abc" (The value of the constant itself is between the quotes.)
Mapping The Rule-Declare-Index form has a default mapping of Equals, which is currently the only mapping recognized.
Index Class Property The Index Class Property is the property in the Index- class that the above values will be set into. This could be the actual name of an existing property, or a relative reference that evaluates to a scalar property. The property reference can include embedded pages, as long as it evaluates to a scalar property: .IndexedOrders(Indiv).CustomerIndex.PriceIndex This field must evaluate to a single scalar property. Aggregate properties are not supported; neither are Java Object property types (which dont contain properties to put into columns in the Index table). Thus, wildcard references are not permitted. Important: The Source Class and Index Class properties must be of compatible modes. For example, if the Source Property is String mode, the Index property cannot be Date/Time mode. All of the Index properties should be exposed as columns in the database (see the Index Class and Database Structures section earlier in this document for full details). In addition, the database column types for the Index properties must be of a mode which accepts the data from the Source Class property. IMPORTANT NOTE: Once you have created an Index- class and a Declare Index instance that populates the properties of that class, do not set the values of these Index- properties manually (using Property-Set or custom Java code steps in Activities). The Index- properties specified in a Declare Index instance must be set by the Declarative processing, not set manually, as possible invalid data or errors may result. Never create, update, or delete an Index- property manually.
CONFIDENTIAL
13
Index even if a Source Class Property is Blank It is possible to create a Rule-Declare-Index instance with more properties (mapped to an Index- class) than may end up being present on the page in the Clipboard. Changes to the properties on the page will cause the indexes for those properties to be regenerated in the database. If there are indexed properties that are not present on the Clipboard page, and this box is checked, those properties will still be saved to the database, with blank values. Example ASSUMPTIONS: This example assumes that the Context Path for each Rule-Declare-Index instance is the same as the Context Path for the page in question. If the Context Path does not match, that Rule-Declare-Index instance will not be evaluated. This example also assumes that the Index even if a source class property is blank has been checked for the declarative index instances below.
Acme Company has the following three Rule-Declare-Index instances defined in their system: 1. Rule-Declare-Index instance created on class YourCo-CustService with Purpose Zip. Source Class properties: .CustomerName .CustomerZip (all mapped to Index- Class properties, not relevant to this example)
2. Rule-Declare-Index instance created on class YourCo-CustService with Purpose Address. Source Class properties: .CustomerName .CustomerStreet .CustomerCity .CustomerZip (all mapped to Index- Class properties, not relevant to this example) 3. Rule-Declare-Index instance created on class YourCo-CustService with Purpose Orders. Source Class properties: .Items .Price .Tax (all mapped to Index- Class properties, not relevant to this example)
14
CONFIDENTIAL
If a new page is created as an instance of YourCo-CustService, with the properties .CustomerName and .CustomerZip, then when that page is saved to the database, all the above Rule-Declare-Index instances will be evaluated, as they are all Indexes defined on the YourCo-CustService class. Index Name Zip Data .CustomerName and .CustomerZip will be saved with the appropriate data. .CustomerName and .CustomerZip will be saved with the appropriate data, and CustomerStreet and .CustomerCity will be saved as blank entries into the columns. .Items, .Price, and .Tax will be saved as blank entries into the columns.
Address
Orders
If Index on blank properties is not checked in these instances, then one index will be saved to the database: Index Name Zip Data .CustomerName and .CustomerZip will be saved with the appropriate data.
Since the other two indexes contain properties that were not present on the clipboard, they will not be saved.
CONFIDENTIAL
15
If the user types customer information into any of the fields which have a magnifying glass to the right, and then clicks on the magnifying glass for that field, information on all the other Customer Requests for that customer will be displayed:
These work items contain the pyWorkParty( ) embedded-pagelist property. Each element of pyWorkParty is a page describing a party associated with the work item; for each party that was added to the work item (Customer, Partner, Interested Party, etc.), one page is added to the pyWorkParty pagelist.
16
CONFIDENTIAL
Users may wish to report which work items are associated with a particular party. With the party information stored in embedded pages, it is not possible to easily report on this information, as the Party Role is a Data-Party property stored in different pages (and different database tables) than the work items themselves. Therefore, the Work Party URI Index was created to expose the work item party data so that it could be easily searched and reported using a simple database query. Therefore, the following Declare Index is created:
CONFIDENTIAL
17
The following classes and properties are set up to support this structure (partial list with example values): class: Work.pyWorkParty(Customer) ..pyWorkParty(Partner) class: Data-Party page .Party(Customer) .pxPartyRole .pyFullName .pyHomePostalCode page .Party(Vendor) .pxPartyRole .pyFullName .pyHomePostalCode
These classes and properties would be structured into the following pages at runtime, and create the following two entries (one for each value of the PageGroup .pyWorkParty) into the Index-WorkPartyURI table:
18 CONFIDENTIAL
.pyWorkParty(Partner) INDEX .pxPartyRole .pyFullName (value="Lightning Inc") .pyPartyFullName (value="Lightning Inc") .pyHomePostalCode (value = "02134") .pyPartyPostalCode (value = "02134") (value = "Partner")
.pxPartyRole
(value = "Partner")
The diagram shows an instance of the class Work-, in the page named Invoice. This page has a property at the top level called .pyWorkParty(groupname). This is a PageGroup property, and two of the values of the Group, Customer and Partner, are shown. The page created by the property .pyWorkParty( ) has a number of properties defined; three of these properties are shown in the example: .pxPartyRole .pyFullName .pyHomePostalCode Values are provided as examples of these properties, which then are stored in the database table associated with the Index-WorkPartyURI class. Once the Rule-Declare-Index instance is defined and saved, the system will begin populating the Index database tables as its Source Class properties are changed. For each work item which is defined, one entry will be made into the index table (associated with Index-WorkPartyUri) for each .pyWorkParty( ) role which is defined. Thus, if one work item contains three parties: Customer Vendor Interested
then three entries will be made into the index table (with a corresponding three instances created in the Index-WorkPartyURI class).
CONFIDENTIAL
19
If this work item is updated, and some of its values changed, then depending upon the data which is updated, one, two, or all three of these entries would be changed. For example, suppose that the following properties were involved in the work item: Property .pyID .pyWorkParty(Customer) Value Item-32 Customer .pxPartyFullName = Acme Corp. Vendor .pxPartyFullName = XYZ Corp. Interested .pxPartyFullName = BigBank Co.
.pyWorkParty(Vendor)
.pyWorkParty(Interested)
Three entries would be saved to the appropriate index table, which would include (among the other property values) the following data: 1. .pyID = Item-32 .pyStatusWork = Open .pyWorkParty(Customer) .pxPartyFullName = Acme Corp. 2. .pyID = Item-32 .pyStatusWork = Open .pyWorkParty(Vendor) .pxPartyFullName = XYZ Corp. 3. .pyID = Item-32 .pyStatusWork = Open .pyWorkParty(Interested) .pyPartyFullName = BigBank Co.
Example Change Now the user opens the work item and finds that there is a problem with the bank name for the Interested party it needs to be changed (there was a merger). The new name is AllPowerfulBank. Once this change is made, only one of the three index entries above will be updated, as a targeted change: 3. .pyID = Item-32 .pyStatusWork = Open .pyWorkParty(Interested) .pyPartyFullName = AllPowerfulBank The other two index entries havent changed.
20
CONFIDENTIAL
These changes, if done carelessly, may result in inconsistent data, or even orphaned data.
It is also possible to regenerate the index by setting the pzReindex property of the instance to true (see the Reindexing section below).
old table, that data is no longer accurate (since the updates are now going to the new table). Again, to make sure that index entries are accurate, set the pzReindex property of the instance to true (see the Reindexing section below).
Reindexing
New instances of a class which has an index defined will find all index definitions. However, if there is an existing instance of a class which has indexes defined, and a new index is created or an existing index definition is changed or deleted, the existing class instance may not get that information from the system, even if the instance is resaved. Therefore, if index definitions are changed, it is necessary to reindex all existing instances. Reindexing is done by setting the pzReindex property of the instance to true. This will cause all indexes for that instance to be recalculated the next time the class instance is saved. IMPORTANT: The pzReindex property will only reindex the index entries for the current instance. The work object Item-32 from the example in the prior section had three index entries associated with it one for each party role. Setting pzReindex to true would reindex all three of those entries, but would not reindex Item-40 (for example), or any other work items. Each work item would have to be resaved individually. If all instances/items need to be re-indexed (to implement an index-definition change, for example), that can be a slow process. It is possible, however, to write an activity to accomplish this task. An example in the system for this kind of activity is the RecreateIndexesForClass activity.
22
CONFIDENTIAL
This activity requires a parameter of the class name for which to recalculate the indexes, and then calls RecreateIndexesForInstance. RecreateIndexesForClass opens each instance (item) in the specified class and resaves it, having set .pzReindex to true.
CONFIDENTIAL
23
NOTE: After .pzReindex is set to true for a particular instance, the action of saving that instance automatically removes the .pzReindex property, so reindexing is not always done. Reindexing does have a performance cost. As explained in the Using a Declare Index section above, the system generally tries to make the most targeted change possible. As stated in Example Change 1, it is not always required to save all index entries for a particular item, if only one of the entries has changed. When using the .pzReindex property, all the entries will be changed. And not just updated the entries will be deleted and then readded to the appropriate index table, which has a higher performance cost than just updating an existing entry.
24
CONFIDENTIAL