Add support for streaming Mule attachments using SOAP message with attachments

Details

  • Type: New Feature New Feature
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.1-rc1
  • Fix Version/s: 1.2
  • Component/s: Modules: CXF, Transport: HTTP(S) / Jetty
  • Labels:
    None
  • Environment:

    Suse Professional 9.3, Java 5.0 Update 3

  • Similar Issues:
    None

Description

When utilizing the SOAP transport to send large messages from one Mule endpoint to another, the size of the SOAP message can quickly become an issue since the entire message is read as a byte array before constructing the SOAP message for dispatch. Axis has support for SOAP messages with attachments which allow you to add DataHandler attachments to SOAP messages that are then processed at dispatch time by streaming the attachment using the DataHandler's InputStream. Since Mule recently added the support for adding DataHandler attachments to Mule messages, this support should be extended in the SOAP provider to allow for translation of Mule messages with DataHandler attachments to SOAP messages with DataHandler attachments. To preserve handling of Mule messages with large attachments, any conversions that are done during the processing of the SOAP message with attachments should preserve the DataHandler attachments without fully reading these attachments into memory.

I've updated the SOAP and HTTP providers from CVS to add this support, so i'll add the patch files to this bug with descriptions of the changes i've made to see if those changes are suitable for adding this support.

  1. Text File
    AxisMessageDispatcher-patch.txt
    06/Sep/05 03:34 PM
    2 kB
    Rick Sears
  2. Text File
    AxisServiceComponent-patch.txt
    06/Sep/05 04:05 PM
    0.9 kB
    Rick Sears
  3. Text File
    HttpMessageAdapter-patch.txt
    06/Sep/05 03:55 PM
    1 kB
    Rick Sears
  4. Text File
    HttpMessageReceiver-patch.txt
    06/Sep/05 04:00 PM
    2 kB
    Rick Sears
  5. Text File
    ObjectToHttpClientMethodRequest-patch.txt
    06/Sep/05 03:51 PM
    2 kB
    Rick Sears
  6. Text File
    SoapMethod-patch.txt
    06/Sep/05 03:26 PM
    0.4 kB
    Rick Sears
  7. Text File
    UniversalSender-patch.txt
    06/Sep/05 03:42 PM
    2 kB
    Rick Sears

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Rick Sears added a comment -

First, fixed issue with the SoapMethod class and its parsing of the parameter line. The initParams function was using the name String variable where it should have been using the type String variable.

Show
Rick Sears added a comment - First, fixed issue with the SoapMethod class and its parsing of the parameter line. The initParams function was using the name String variable where it should have been using the type String variable.
Hide
Permalink
Rick Sears added a comment -

For the Axis SOAP message dispatcher, I updated the creation of the Call object to add support for converting Mule message attachments into arguments to the Axis Call object. The payload of the Mule message is still processed in the same way as before, but if there are any attachments on the Mule message, these DataHandler attachments are collected into an DataHandler[] object that is added on as the last parameter to the Axis Call object.

When creating the Call object, we also have to explicitly tell Axis the type of the added DataHandler attachment objects to allow for the proper serialization of the added attachments when sending over the wire. If the soapMethods variable is already defined, then we rely on the passed in variable to define the arguments properly. If it is not, then we construct the parameter descriptions based on the payload object and the added attachment objects.

Show
Rick Sears added a comment - For the Axis SOAP message dispatcher, I updated the creation of the Call object to add support for converting Mule message attachments into arguments to the Axis Call object. The payload of the Mule message is still processed in the same way as before, but if there are any attachments on the Mule message, these DataHandler attachments are collected into an DataHandler[] object that is added on as the last parameter to the Axis Call object. When creating the Call object, we also have to explicitly tell Axis the type of the added DataHandler attachment objects to allow for the proper serialization of the added attachments when sending over the wire. If the soapMethods variable is already defined, then we rely on the passed in variable to define the arguments properly. If it is not, then we construct the parameter descriptions based on the payload object and the added attachment objects.
Hide
Permalink
Rick Sears added a comment -

The UniversalSender class also had to be updated to allow for proper handling of SOAP messages with attachments. The current version of the UniversalSender class reads the entire message into a byte[] object that is used as the payload of a Mule message used to transport the SOAP message over the wire. Since we are attempting to use the streaming capabilities of the attachment support in Axis, I updated the creation of the payload to handle a SOAP message with attachments differently. If the SOAP message has attachments, I use a temporary File object as the content of an InputStream that is passed as the payload of the Mule message. This allows for the actual sender of the SOAP message with attachments to use the InputStream to stream the message across the actual transport used to send the SOAP message. In the case of a synchronous call, the temp file is cleaned up upon completion of the call. For the asynchronous calls, the temp file is marked for deletion upon exit of the VM.

I also added code to explicitly set the Content-Length property for the call to allow the HTTP based sender to properly stream the message. This is due to how the apache commons httpclient library processes HTTP PUT calls.

Show
Rick Sears added a comment - The UniversalSender class also had to be updated to allow for proper handling of SOAP messages with attachments. The current version of the UniversalSender class reads the entire message into a byte[] object that is used as the payload of a Mule message used to transport the SOAP message over the wire. Since we are attempting to use the streaming capabilities of the attachment support in Axis, I updated the creation of the payload to handle a SOAP message with attachments differently. If the SOAP message has attachments, I use a temporary File object as the content of an InputStream that is passed as the payload of the Mule message. This allows for the actual sender of the SOAP message with attachments to use the InputStream to stream the message across the actual transport used to send the SOAP message. In the case of a synchronous call, the temp file is cleaned up upon completion of the call. For the asynchronous calls, the temp file is marked for deletion upon exit of the VM. I also added code to explicitly set the Content-Length property for the call to allow the HTTP based sender to properly stream the message. This is due to how the apache commons httpclient library processes HTTP PUT calls.
Hide
Permalink
Rick Sears added a comment -

Since i'm first focused on SOAP calls over the HTTP transport, i've updated the HTTP provider to add support for the was SOAP message with attachments are passed to this transport. The ObjectToHttpClientMethodRequest transformer was updated to support passed InputStream payloads. In the case of a payload that is type InputStream, this InputStream payload is added as the request body and the body content length value is set to the actual length of the Content-Length variable passed as a property from the UniversalSender. This Content-Length variable has to be set when using the apache commons httpclient library or the library will attempt to fill the value for this variable by reading the entire InputStream into memory (which we want to avoid). The transformer also sets the Content-Type variable in the case where the payload is of type InputStream so that the HTTP receiver can properly parse the PUT request.

Show
Rick Sears added a comment - Since i'm first focused on SOAP calls over the HTTP transport, i've updated the HTTP provider to add support for the was SOAP message with attachments are passed to this transport. The ObjectToHttpClientMethodRequest transformer was updated to support passed InputStream payloads. In the case of a payload that is type InputStream, this InputStream payload is added as the request body and the body content length value is set to the actual length of the Content-Length variable passed as a property from the UniversalSender. This Content-Length variable has to be set when using the apache commons httpclient library or the library will attempt to fill the value for this variable by reading the entire InputStream into memory (which we want to avoid). The transformer also sets the Content-Type variable in the case where the payload is of type InputStream so that the HTTP receiver can properly parse the PUT request.
Hide
Permalink
Rick Sears added a comment -

The HttpMessageAdapter class also had to be updated so that the payload could be something other than a fully populated byte[] object. This allows for the Http transport to support when Axis passes the SOAP message with attachments as an InputStream for sending.

Show
Rick Sears added a comment - The HttpMessageAdapter class also had to be updated so that the payload could be something other than a fully populated byte[] object. This allows for the Http transport to support when Axis passes the SOAP message with attachments as an InputStream for sending.
Hide
Permalink
Rick Sears added a comment -

When receiving the stream SOAP message with attachments in the HttpMessageReceiver class, the code that reads from the InputStream needed updating. First, we recognize the Content-Type of 'multipart/related' as requiring special handling. If the message is of this type, we read the contents into a temporary file instead of fully populating a byte[] object (which is still the case for non-multipart requests). This temporary File object is passed as the payload when returning from the function that reads the request. The expectation is that the receiver of this temporary File object will take care of deleting it once it is processed. If the file is not deleted after processing, it is still marked for deletion when the VM exits.

Show
Rick Sears added a comment - When receiving the stream SOAP message with attachments in the HttpMessageReceiver class, the code that reads from the InputStream needed updating. First, we recognize the Content-Type of 'multipart/related' as requiring special handling. If the message is of this type, we read the contents into a temporary file instead of fully populating a byte[] object (which is still the case for non-multipart requests). This temporary File object is passed as the payload when returning from the function that reads the request. The expectation is that the receiver of this temporary File object will take care of deleting it once it is processed. If the file is not deleted after processing, it is still marked for deletion when the VM exits.
Hide
Permalink
Rick Sears added a comment -

Finally, back on the Axis SOAP provider side, the AxisServiceComponent was updated to allow for processing of File objects passed as the transformed message from the HttpMessageReceiver. This File object is passed to the creation of the SOAP message, and the Axis SOAP message creation routines use this temporary File to recreate the SOAP message with attachments on the receiving side. This SOAP message with attachments is passed to the Axis engine for use in the call that was originally requested on the sending side. After the Axis engine finishes its processing, the temporary file is deleted.

Show
Rick Sears added a comment - Finally, back on the Axis SOAP provider side, the AxisServiceComponent was updated to allow for processing of File objects passed as the transformed message from the HttpMessageReceiver. This File object is passed to the creation of the SOAP message, and the Axis SOAP message creation routines use this temporary File to recreate the SOAP message with attachments on the receiving side. This SOAP message with attachments is passed to the Axis engine for use in the call that was originally requested on the sending side. After the Axis engine finishes its processing, the temporary file is deleted.
Hide
Permalink
Rick Sears added a comment -

Let me know how these changes look. I realize that these changes touch on alot of different parts of the code, but I hope that they make sense and are in the vain of how you'd like to make future changes to the Mule project. Also, I have an example client and processing server object that I have used for testing that may be useful for an example of how to use the streaming SOAP message with attachments support.

Show
Rick Sears added a comment - Let me know how these changes look. I realize that these changes touch on alot of different parts of the code, but I hope that they make sense and are in the vain of how you'd like to make future changes to the Mule project. Also, I have an example client and processing server object that I have used for testing that may be useful for an example of how to use the streaming SOAP message with attachments support.
Hide
Permalink
Ross Mason added a comment -

Hi Rick,

Are all these changes in? Would you mind adding a testcase under the integration tests to run through the changes you've made.
Thanks a lot for your help,

Ross

Show
Ross Mason added a comment - Hi Rick, Are all these changes in? Would you mind adding a testcase under the integration tests to run through the changes you've made. Thanks a lot for your help, Ross
Hide
Permalink
Ross Mason added a comment -

Cheers Rick

Show
Ross Mason added a comment - Cheers Rick

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: