Mule
  1. Mule
  2. MULE-4912

HttpRequestBodyToParamMap uses java.net.URLDecoder. But URLDecoder sometimes can't decode query string which encoded by URLCodec.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-M4
    • Fix Version/s: 3.1.0
    • Labels:
      None
    • Environment:

      WindowsXP SP3, Sun JDK 1.6.0_14

    • User impact:
      Medium
    • Similar Issues:
      MULE-4911HttpRequestBodyToParamMap decodes a query string without specifying an encoding.
      MULE-1768Query string is not encoded correctly
      MULE-3236Default transformer doesn't URL-encode parameters and incorrectly URL-decodes endpoint query string
      MULE-4862The HttpRequestBodyToParamMap transformer does not allow for encoded = (equals) character in GET request
      MULE-5849Setting encoding attribute on a transformer has no effect
      MULE-6497inboundProperties should be URL decoded
      MULE-5474Mime decode - Filename/Attachment
      MULE-3690Query string is always encoded as UTF-8 byte stream
      MULE-545Http provider does not correctly handle UTF-8 encoded String payloads
      MULE-6279URI encoded special characters cause some troubles at email transport

      Description

      Hi.

      At first, please check out MULE-4911.

      I found a one more problem in HttpRequestBodyToParamMap class.

      HttpRequestBodyToParamMap uses java.net.URLDecoder.
      But when multibytes string is used, URLDecoder sometimes can't decode query string which encoded by URLCodec.

      For example, Japanese Hiragana characher "\u30a8" is encoded to...
      => "%83%47" (by URLEncoder)
      => "%83G" (by URLCodec)

      ===== sample code =====
      import java.net.URLDecoder;
      import java.net.URLEncoder;

      import org.apache.commons.codec.net.URLCodec;
      import org.junit.Assert;
      import org.junit.Test;

      public class URLDecoderTest {

      private static final String encoding = "Windows-31J";
      private static final String value = "\u30a8"; // a Japanese hiragana character

      @Test
      public void testDecode() throws Exception

      { System.out.println(URLEncoder.encode(value, encoding)); System.out.println(new URLCodec().encode(value, encoding)); Assert.assertEquals(value, URLDecoder.decode(new URLCodec().encode(value, encoding), encoding)); }

      }
      ===== sample code =====

      And according to RFC3986 (Sec 2.3)...
      > For consistency, percent-encoded octets in the ranges of ALPHA
      > (%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E),
      > underscore (%5F), or tilde (%7E) should not be created by URI
      > producers

      In the case of "\u30a8", "%83G"(by URLCodec) is preferred result. (correct?)

      But URLDecoder can't decode "%83G" to "\u30a8". and URLCodec#decode() can.
      So I think that URLCodec#docode() should be used here instead of URLDecoder.

      Here is a patch for this problem and functional tests.
      (this patch includes all the content of the patch for MULE-4911)

      best regards.

      1. NEW-URLDecoder.patch
        15 kB
        Kazuya Uno
      2. URLDecoder.patch
        13 kB
        Kazuya Uno

        Activity

        Kazuya Uno created issue -
        Hide
        Kazuya Uno added a comment -

        Here is a patch for this problem and functional tests.
        (this patch includes all the content of the patch for MULE-4911)

        Show
        Kazuya Uno added a comment - Here is a patch for this problem and functional tests. (this patch includes all the content of the patch for MULE-4911 )
        Kazuya Uno made changes -
        Field Original Value New Value
        Attachment URLDecoder.patch [ 12808 ]
        Hide
        Kazuya Uno added a comment -

        I changed my patch.

        Japanese test message has moved from .java to test-data-messages_ja.src.
        and .properties was generated by "mvn generate-test-resources".
        (this patch includes all the content of the patch for MULE-4911)

        Show
        Kazuya Uno added a comment - I changed my patch. Japanese test message has moved from .java to test-data-messages_ja.src. and .properties was generated by "mvn generate-test-resources". (this patch includes all the content of the patch for MULE-4911 )
        Kazuya Uno made changes -
        Attachment NEW-URLDecoder.patch [ 12810 ]
        Travis Carlson made changes -
        Affects Version/s 3.0.0-M4 [ 10832 ]
        Affects Version/s 3.x [ 10674 ]
        Dirk Olmes made changes -
        Assignee Dirk Olmes [ dirk ]
        Dirk Olmes made changes -
        Link This issue relates to BL-131 [ BL-131 ]
        Dirk Olmes made changes -
        Priority To be reviewed [ 6 ] Major [ 3 ]
        Dirk Olmes made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Show
        Dirk Olmes added a comment - 3.0.x: http://fisheye.codehaus.org/changelog/mule/?cs=20243
        Show
        Dirk Olmes added a comment - 3.x: http://fisheye.codehaus.org/changelog/mule/?cs=20247
        Dirk Olmes made changes -
        Status In Progress [ 3 ] Closed [ 6 ]
        Fix Version/s 3.1.0 [ 10898 ]
        Resolution Fixed [ 1 ]
        Ramiro Rinaudo made changes -
        Workflow Fixed Main Mule Workflow (after JIRA upgrade) [ 77198 ] Main Mule Workflow v1.0 [ 141106 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        114d 21h 47m 1 Dirk Olmes 10/Oct/10 10:58 PM
        In Progress In Progress Closed Closed
        38d 10h 58m 1 Dirk Olmes 18/Nov/10 08:57 AM

          People

          • Assignee:
            Dirk Olmes
            Reporter:
            Kazuya Uno
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development