FTP transport doesn't support streaming

Details

  • Type: Patch submission Patch submission
  • Status: Open Open
  • Priority: Major Major
  • Resolution: Unresolved
  • Affects Version/s: 2.0.2
  • Fix Version/s: Product Backlog
  • Component/s: Transport: FTP / SFTP
  • Labels:
    None
  • Environment:

    All plattforms

  • User impact:
    Medium
  • Similar Issues:
    None

Description

The FTP transport currently doesn't support streaming, that causes an OutOfMemory error getting large files from ftp server.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Travis Carlson added a comment -

Danny, can this be closed now?

Show
Travis Carlson added a comment - Danny, can this be closed now?
Hide
Permalink
Cheng Lee added a comment -

In version 2.2.1 of FtpMessageReceiver I see the streaming support. However, the issue is still present on FtpMessageRequester as you can see below:

org.mule.transport.ftp.FtpMessageRequester:119

ByteArrayOutputStream baos = new ByteArrayOutputStream();
if (!client.retrieveFile(file.getName(), baos))

It should take the same approach as FtpMessageReceiver which asks whether the connector is in "streaming" mode and invoke FtpClient.retrieveFileStream() instead. Basically this:

MuleMessage reply;
if (connector.isStreaming())
{
InputStream stream = client.retrieveFileStream(file.getName());
if (stream == null)
{
throw new IOException(MessageFormat.format("Failed to retrieve file {0}. Ftp error: {1}",
file.getName(), client.getReplyCode()));
}
reply = new DefaultMuleMessage(connector.getMessageAdapter(stream));
}
else
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
if (!client.retrieveFile(file.getName(), baos))

{ throw new IOException("Ftp error: " + client.getReplyCode()); }

reply = new DefaultMuleMessage(connector.getMessageAdapter(baos.toByteArray()));
}

I've attached the patched file for version 2.2.1

Show
Cheng Lee added a comment - In version 2.2.1 of FtpMessageReceiver I see the streaming support. However, the issue is still present on FtpMessageRequester as you can see below: org.mule.transport.ftp.FtpMessageRequester:119 ByteArrayOutputStream baos = new ByteArrayOutputStream(); if (!client.retrieveFile(file.getName(), baos)) It should take the same approach as FtpMessageReceiver which asks whether the connector is in "streaming" mode and invoke FtpClient.retrieveFileStream() instead. Basically this: MuleMessage reply; if (connector.isStreaming()) { InputStream stream = client.retrieveFileStream(file.getName()); if (stream == null) { throw new IOException(MessageFormat.format("Failed to retrieve file {0}. Ftp error: {1}", file.getName(), client.getReplyCode())); } reply = new DefaultMuleMessage(connector.getMessageAdapter(stream)); } else { ByteArrayOutputStream baos = new ByteArrayOutputStream(); if (!client.retrieveFile(file.getName(), baos)) { throw new IOException("Ftp error: " + client.getReplyCode()); } reply = new DefaultMuleMessage(connector.getMessageAdapter(baos.toByteArray())); } I've attached the patched file for version 2.2.1
Hide
Permalink
Cheng Lee added a comment -

Patched FtpMessageRequester.java for version 2.2.1

Show
Cheng Lee added a comment - Patched FtpMessageRequester.java for version 2.2.1

People

  • Assignee:
    Daniel Zapata
    Reporter:
    David Borja
Vote (1)
Watch (1)

Dates

  • Created:
    Updated: