ESB's rated by analysts

After my post last year [ref: SOA Suite rated by analysts] where I looked into the evaluation of the Oracle SOA Suite by Gartner and Forrester it's time to take a new look on their saying about ESB's (Enterprise Service Bus).
For this I took a look at some newer reports from Forrester and Gartner:

• The Forrester Wave: Enterprise Service Bus, Q2 2011
  April 25, 2011 by Ken Vollmer
• Magic Quadrant for Application Infrastructure for Systematic Application Integration Projects
  June 20, 2012 by Jess Thompson, Yefim Natis, Massimo Pezzini, Daniel Sholler, Ross Altman and Kimihiko Iijima

As shown in the graphs the leaders are clearly IBM, Oracle, Software AG and Tibco.
Software AG - worked with webMethods for many years, good product.
Oracle - worked the last couple of years with SOA Suite, the 11g and 12c are build upon weblogic application server, hence the good score.
IBM - will be working with IBM in the near futurs, the multiple ESB offerings from IBM makes me wonder.

Forrester: [Click to enlarge]

Gartner: [Click to enlarge]

Now let's see what is said about IBM's multiple ESB offering:
Forrester - IBM offers three ESB's: WebSphere Enterprise Service Bus (WESB), WebSphere Enterprise Service Bus Registry Edition (WESBRE) and WebSphere Message Broker (WMB). Funny in the Forrester wave is that the actual ESB from IBM, the WebSphere ESB, scored somewhat lower than the other two.
Gartner - Some caution for IBM as despite plans to rationalize and simplify the product portfolio (e.g., in ESB), the fine-grained differences, functional overlaps and product integration challenges — for example, among WMB, WESB, WebSphere Cast Iron and the WebSphere DataPower integration appliances — make it difficult for potential users to determine the best fit for their requirements.
IBM - the faqs on the IBM website mention three different ESB's: IBM WebSphere ESB, IBM WebSphere Message Broker and IBM WebSphere DataPower Integration Appliance XI50.

Here a selection from IBM [ref: faq] on when to use which ESB:

When to Use WebSphere ESB?

• You use WebSphere Application Server and/or your team has skills with WAS Administration and Java coding
• You are focused on standards based interactions using XML, SOAP, and WS
• Reliability and extensive transactional support are key requirements

When to Use WebSphere Message Broker?

• You are currently using WebSphere Message Broker but not as an ESB
• You are using Industry formats such as SWIFT, EDI, HL7
• You are implementing a wide range of messaging and integration patterns
• You have very complex transformation needs
• Reliability and extensive transactional support are key requirements
• To achieve very high-performance with horizontal and vertical scaling

When To Use WebSphere DataPower?

• Ease of use is a pre-dominant consideration
• You are transforming between XML-and-XML or XML-and-any other format
• Your interaction patterns are relatively simple
• You are using XML-based or WS-Security extensively
• You require use of advanced Web services standards
• You need to minimize message latency when adding an ESB layer
• Your ESB must be in production very quickly

What if you require an ESB from IBM for standards based integration, with complex integration needs, high performance, advanced web service standards and want to move to production quickly?

FOTY0001 and logfiles roulation

A certain person named, or used the alias of, Vivek wrote a few years ago some interesting articles on his blog [ref: OracleD] about Oracle SOA Suite 10g. He experienced many flaws in Oracle OC4J Application Server and recommended the OTN discussion: Oracle BPEL + Oc4j + Jdeveloper = brain damage. Clearly he was not so happy with the features in the 10g version of Oracle SOA Suite...

He referred also to the so-called FOTY0001 errors, these often occur during XRef calls and XSL Transformations. More information about this FOTY0001 can be found in the OPMN log files. For example, a typical error is the following:
subLanguageExecutionFault - XPathExecutionError
  XPath expression failed to execute.
  Error while processing xpath expression, the expression is "ora:processXSLT('myTransformation.xsl',bpws:getVariableData('myInputMessage'))", the reason is FOTY0001: type error.
  Please verify the xpath query.
This error occurs e.g., when in JDeveloper the transform activity is opened and closed immediately. There was no time for the messagePart to be loaded, so it will be missing in the code leading to this FOTY0001 error on runtime. The proper syntax in the code is ora:processXSLT('myTransformation.xsl',bpws:getVariableData('myInputMessage','myMessagePart'))
To prevent this either click cancel in JDeveloper or wait for the message parts to load completely.

To view the FOTY0001 errors in detail the obvious way is to view them using Enterprise Manager. But an easier way is to view them directly by opening the logfiles on filesystem. Depending on the logging level the server logfiles can quickly become pretty huge. In the opmn configuration the roulation can be configured as follows, open $SOA_HOME/opmn/conf/opmn.xml and make the following changes:
<ias-component id="soa_group" status="enabled">
  <process-type id="oc4j_soa" module-id="OC4J" status="enabled">
    <module-data>
      <category id="start-parameters">
        <data id="java-options"
          value="-server
Add here the following parameters:
          -Dstdstream.filesize=10 - File size in MegaBytes
          -Dstdstream.filenumber=50 - Number of files
To separate the output and error messages add the following data element with oc4j-options inside the same category:
<data id="oc4j-options" value="-out $ORACLE_HOME/opmn/logs/oc4j_soa.out -err $ORACLE_HOME/opmn/logs/oc4j_soa.err"/>
To view the FOTY0001 error details simply open (in a good text-editor) the *.err file containing the timestamp of the error.

ORABPEL dehydration store purge scripts

By default all BPEL instances (messages) in the SOA Suite are persisted in the internal database, the so-called dehydration store. Good practice is to purge older messages, to avoid database sizing problems and to increase performance.

Unfortunately the original Oracle scripts were not sufficient to do the task, so many (consulting) companies created their own purge scripts. The good news is that Oracle realized the need and offers improved purge scripts, which are available to download from Oracle. Please take a look at the Oracle 10G FMW purge strategy whitepaper [ref: note ID 1286265.1], this document also contains the BPEL database schema.

For example, the second option from the whitepaper, multi-threaded purge can be found at Oracle Note: New BPEL 10g Purge Scripts From 10.1.3.5 MLR#2 [ref: note ID 1110833.1]. This script can be scheduled with e.g., crontab, or any other scheduling tool. The start parameters can be configured like

• P_OLDER_THAN := sysdate-21; (purge instances older than 21 days)
• P_ROWNUM := 10000000; (purge up to 10 million instances)
• P_DOP := 3; (use three threads in parallel)
• P_CHUNKSIZE := 1000; (commit per 1000 rows)

This last note contains all three scripts mentioned in the whitepaper:

• SINGLE THREADED LOOPED PURGE PROCEDURE
• MULTI THREADED LOOPED PURGE PROCEDURE
• CTAS (Create Table As Select) PROCEDURE

Afterwards if needed you can fine-tune this script a little.
Tips:

• Create a script for all state instances to purge dev and test environments, modify INSERT INTO temp_cube_instance, change WHERE state >= 5 into WHERE state >= 0
• When purging all instances older than the configured days, it might be handy to keep the process history a little longer, like one year, modify DELETE FROM process_log, change WHERE event_date < p_older_than into WHERE event_date < SYSDATE - 365
• When sensor data is used you might want to include this data in the purge script:
  DELETE FROM activity_sensor_values WHERE creation_date < p_older_than;
DELETE FROM fault_sensor_values WHERE creation_date < p_older_than;
DELETE FROM variable_sensor_values WHERE creation_date < p_older_than;
• When using AIA ErrorHandler you might want to prevent the carthesian product by adding a max to the script. The AIA Error Handler doesn't use unique conversation id's, modify INSERT INTO temp_invoke_message, change FROM temp_cube_instance tci into FROM (SELECT MAX (cikey) cikey, conversation_id FROM temp_cube_instance GROUP BY conversation_id) tci

For the ORAESB schema the scripts can simply be found at $SOA_HOME/integration/esb/sql/other.