Issue Details (XML | Word | Printable)

Key: MULE-3192
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Critical Critical
Assignee: Travis Carlson
Reporter: Ove Gram Nipen
Votes: 21
Watchers: 18
Operations

If you were logged in you would be able to see more operations.
Mule

Ftp connector in mule 2.0 fails with ParserInitializationException

Created: 04/Apr/08 09:51 AM   Updated: 05/Feb/09 03:28 AM
Component/s: Transport: FTP
Affects Version/s: 2.0.0
Fix Version/s: 2.2, Bug Backlog

Time Tracking:
Not Specified

Environment:
Ubuntu 7.10 Linux 2.6.22-14-generic #1 SMP Tue Feb 12 07:42:25 UTC 2008 i686 GNU/Linux
java version "1.5.0_13"
proftpd or vsftpd (both exhibit same behavior)

Labels: 212-candidate
User impact: High
Effort points: 0.5
Configuration:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesource.org/schema/mule/core/2.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:spring="http://www.springframework.org/schema/beans"
        xmlns:ftp="http://www.mulesource.org/schema/mule/ftp/2.0"
        xmlns:file="http://www.mulesource.org/schema/mule/file/2.0"
        xsi:schemaLocation="
               http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
               http://www.mulesource.org/schema/mule/core/2.0
http://www.mulesource.org/schema/mule/core/2.0/mule.xsd
               http://www.mulesource.org/schema/mule/file/2.0
http://www.mulesource.org/schema/mule/file/2.0/mule-file.xsd
               http://www.mulesource.org/schema/mule/ftp/2.0
http://www.mulesource.org/schema/mule/ftp/2.0/mule-ftp.xsd
               ">

        <description>reads from ftp and stores to disk</description>

        <ftp:connector binary="true" name="ftpConnector"
                validateConnections="true" />

        <model name="ftpSample">
                <service name="Fetcher">
                        <inbound>
                                <ftp:inbound-endpoint host="localhost"
                                        path="/tmp/outgoing" port="21"
                                        user="myuser" passive="true"
                                        password="secret"/>
                        </inbound>
                        <echo-component/>
                        <outbound>
                                <outbound-pass-through-router>
                                        <file:outbound-endpoint 
                                        path="/tmp/incoming" />
                                </outbound-pass-through-router>
                        </outbound>
                </service>
        </model>
</mule>
Log Output:
**********************************************************************
* Mule ESB and Integration Platform *
* Version: 2.0.0 Build: 11521 *
* MuleSource, Inc. *
* For more information go to http://mule.mulesource.org *
* *
* Server started: 1/1/70 1:00 AM *
* Server ID: a6212f35-0229-11dd-ac09-17445992a62e *
* JDK: 1.5.0_13 (mixed mode) *
* OS: Linux (2.6.22-14-generic, i386) *
* Host: hotlips (127.0.1.1) *
* *
* Agents Running: None *
**********************************************************************
INFO 2008-04-04 11:29:43,377 [Fetcher.2]
org.mule.component.simple.LogComponent:
********************************************************************************
* Message received in service: Fetcher. Content is: 'hello
' *
********************************************************************************
INFO 2008-04-04 11:29:43,395 [connector.file.0.dispatcher.1]
org.mule.transport.file.FileMessageDispatcher: Connected:
FileMessageDispatcher{this=bb2bc3, endpoint=file:///tmp/incoming,
disposed=false}
INFO 2008-04-04 11:29:43,396 [connector.file.0.dispatcher.1]
org.mule.transport.file.FileConnector: Writing file
to: /tmp/incoming/a8724b2f-0229-11dd-ac09-17445992a62e.dat
INFO 2008-04-04 11:29:43,426 [Fetcher.3]
org.mule.component.simple.LogComponent:
********************************************************************************
* Message received in service: Fetcher. Content is: 'world
' *
********************************************************************************
INFO 2008-04-04 11:29:43,429 [connector.file.0.dispatcher.2]
org.mule.transport.file.FileConnector: Writing file
to: /tmp/incoming/a8772d38-0229-11dd-ac09-17445992a62e.dat
ERROR 2008-04-04 11:29:44,190 [ftpConnector.receiver.4]
org.mule.DefaultExceptionStrategy: Caught exception in Exception
Strategy: Unknown parser type: CWD command successful
org.apache.commons.net.ftp.parser.ParserInitializationException: Unknown
parser type: CWD command successful
        at
org.apache.commons.net.ftp.parser.DefaultFTPFileEntryParserFactory.createFileEntryParser(DefaultFTPFileEntryParserFactory.java:118)
        at
org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:2358)
        at
org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2141)
        at
org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2188)
        at
org.mule.transport.ftp.FtpMessageReceiver.listFiles(FtpMessageReceiver.java:96)
        at
org.mule.transport.ftp.FtpMessageReceiver.poll(FtpMessageReceiver.java:72)
        at
org.mule.transport.PollingReceiverWorker.run(PollingReceiverWorker.java:47)
        at org.mule.work.WorkerContext.run(WorkerContext.java:310)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1061)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:575)
        at java.lang.Thread.run(Thread.java:595)
INFO 2008-04-04 11:29:44,193 [ftpConnector.receiver.4]
org.mule.DefaultExceptionStrategy: There is no current event available,
routing Null message with the exception



This exception is now repeated indefinitely.


 Description  « Hide
I am trying to move a set of files from an ftp server to disk using the ftp connector in mule 2.0, but the ftp connector fails with an exception. It seems that something has gone wrong with the ftp client, so that it gets the output from the CWD command when it expects the output from the SYST command. It works fine in mule 1.4.3. Both versions use commons net 1.4.1.

Steps to reproduce:

Start an ftp server on localhost
mkdir /tmp/incoming
mkdir /tmp/outgoing
cd /tmp/outgoing
echo hello > a
echo world > b



 All   Comments   Work Log   Change History   Transitions   FishEye      Sort Order: Ascending order - Click to sort in descending order
Ove Gram Nipen added a comment - 10/Sep/08 07:06 AM
The behaviour is quite consistent if you follow these steps:

Start an ftp server on localhost
$ mkdir /tmp/incoming
$ mkdir /tmp/outgoing

start mule

$ cd /tmp/outgoing
$ echo hello > a
$ echo world > b
$ echo foo > c

Mule will then get the file 'a', but ignore the file 'b'. No matter how many files you create, Mule will ignore them.
$ ls
b c

stop mule
start mule

Mule will then get 'b' and 'c', but will then start to emit the above error message (Unknown parser type: CWD command successful).

This behavior has been confirmed with mule Version: 2.0.2 Build: 12377

$ java -version
java version "1.5.0_15"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_15-b04)
Java HotSpot(TM) Client VM (build 1.5.0_15-b04, mixed mode, sharing)


Arne Watnelie added a comment - 11/Sep/08 04:18 AM
I'm getting the same problem (but not as consistent) on a different environment:

Windows Vista SR1
Cerberus FTP Server
Mule 2.0.2
jdk-6u10-rc

The error occurs sporadically, and as a result the ftp fetcher will stop to transfer files until a restart of Mule.


Ove Gram Nipen added a comment - 11/Sep/08 12:21 PM
I can make the error go away by modifying FtpConnector.getFtp() such that it always returns a new FTPClient. This is not an acceptable solution, however, since it is pretty heavy on resources. However, this indicates that the problem might have to do with FTPClient objects being returned to the pool in an inconsistent state.

Łukasz Lipka added a comment - 12/Sep/08 07:47 AM
I'm getting the same problem with mule 2.0.2 and 2.0.0 on Mac OS 10.4.11
java -version
java version "1.5.0_13"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05-241)
Java HotSpot(TM) Client VM (build 1.5.0_13-121, mixed mode, sharing)

Chih-Liang Chang added a comment - 04/Oct/08 01:09 AM
I'm also getting the same problem with mule 2.0.2 (Unknown parser type: CWD command successful).

java 1.6.0_06
windows xp sp3

My ftp server is vsftp on fedora core 9


Daniel Feist added a comment - 17/Oct/08 10:10 AM
We need to take a closer look at this one see if there is a specific bug causing this and fix it in 2.1.n maintenance if there is.

Claes Bergsten added a comment - 22/Oct/08 09:45 AM
Step to reproduce this error with 100% certainty:
  • Force the receiver to run singlethreaded with a threading profile.
  • Put 2 or more files in the polled folder on the Ftp and start mule.

It will always fetch the first file successfully and then fail with 250 on all the subsequent files.
( And as someone else mentioned get stuck and not fetch any more files until restart.)


D Henton added a comment - 05/Nov/08 03:50 PM
Same problem here using WSFTP server on XP.

15:44:49,200 | ERROR | org.mule.DefaultExceptionStrategy | 395 | Caught exception in Exception Strategy: Unknown parser type: CWD successful. "/incoming" is current directory.
org.apache.commons.net.ftp.parser.ParserInitializationException: Unknown parser type: CWD successful. "/incoming" is current directory.

Running the latest 2.1.1 community release


Jerome Bernard added a comment - 19/Nov/08 04:52 AM
I think I've located the error.

Actually, by studying the diff between 1.4 and 2.x, I found out that 1.4 provided either a streamed version or a non streamed version.
So, in FtpMessageReceiver, in method processFile(), if you use the code available underneath, it will work.
What I suppose is that there is an issue with the streams. They are probably not closed properly (I don't have enough in-depth knowledge of Mule 2): as said on the Commons-NET JavaDoc for retreiveFileStream (http://commons.apache.org/net/apidocs/org/apache/commons/net/ftp/FTPClient.html#retrieveFileStream(java.lang.String)), we should make sure the stream is closed and call client.completePendingCommands().

Could someone test this with Mule 1.4 when activation the streaming mode?
The same issue should occur if I got that right...

"Patch":
/*
InputStream stream = client.retrieveFileStream(file.getName());
if (stream == null)
{
throw new IOException(MessageFormat.format("Failed to retrieve file {0}. Ftp error: {1}",
new Object[]{file.getName(), new Integer(client.getReplyCode())}));
}
message = new DefaultMuleMessage(connector.getMessageAdapter(stream));
*/

ByteArrayOutputStream baos = new ByteArrayOutputStream();
if (!client.retrieveFile(file.getName(), baos))
{
throw new IOException(MessageFormat.format("Failed to retrieve file {0}. Ftp error: {1}",
new Object[] {file.getName(), new Integer(client.getReplyCode())}));
}
message = new DefaultMuleMessage(connector.getMessageAdapter(baos.toByteArray()));


Daniel Feist added a comment - 19/Nov/08 05:27 AM
Raising to critical based on number of votes.

Ken Yagen added a comment - 13/Dec/08 10:09 PM
There's been some recent discussion on this specific issue over at http://www.manning-sandbox.com/thread.jspa?threadID=27715&tstart=0

Travis Carlson added a comment - 04/Feb/09 07:18 PM
This issue is still present in 2.2 (SNAPSHOT), in both the CE and EE transport.

Travis Carlson added a comment - 04/Feb/09 08:51 PM
http://fisheye.codehaus.org/changelog/mule/?cs=13957

This is more of a workaround than a fix, what I have done is introduce a "streaming" flag to the FTP transport which defaults to false. If streaming = false, the receiver uses the old code from 1.x, as Jerome Bernard suggests, above.

Created MULE-4142 for continuing work on the real issue when streaming = true


Travis Carlson added a comment - 04/Feb/09 09:15 PM

Ove Gram Nipen added a comment - 05/Feb/09 03:28 AM
I just built https://svn.codehaus.org/mule/branches/mule-2.1.x/transports/ftp and copied target/mule-transport-ftp-2.1.3-SNAPSHOT.jar into $MULE_HOME/lib/mule/ replacing mule-transport-ftp-2.1.2.jar.

I used the following config (mule 2.1.2):

<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns="http://www.mulesource.org/schema/mule/core/2.1"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:spring="http://www.springframework.org/schema/beans"
       xmlns:ftp="http://www.mulesource.org/schema/mule/ftp/2.1"
       xmlns:file="http://www.mulesource.org/schema/mule/file/2.1"
    xsi:schemaLocation="
       http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
       http://www.mulesource.org/schema/mule/core/2.1 http://www.mulesource.org/schema/mule/core/2.1/mule.xsd
       http://www.mulesource.org/schema/mule/file/2.1 http://www.mulesource.org/schema/mule/file/2.1/mule-file.xsd
       http://www.mulesource.org/schema/mule/ftp/2.1 http://www.mulesource.org/schema/mule/ftp/2.1/mule-ftp.xsd">

        <description>reads from ftp and stores to disk</description>

        <ftp:connector binary="true" name="ftpConnector"
                validateConnections="true" />

        <model name="ftpSample">
                <service name="Fetcher">
                        <inbound>
                                <ftp:inbound-endpoint host="localhost"
                                        path="/tmp/outgoing" port="21"
                                        user="username" passive="true"
                                        password="secret"/>
                        </inbound>
                        <echo-component/>
                        <outbound>
                                <pass-through-router>
                                        <file:outbound-endpoint 
                                        path="/tmp/incoming" />
                                </pass-through-router>
                        </outbound>
                </service>
        </model>
</mule>

Preliminary tests show that this works now. Thanks, Travis!