Skip to Main Content

This is an IBM Automation portal for Integration products. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (

Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:

Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,

Post your ideas
  1. Post an idea.

  2. Get feedback from the IBM team and other customers to refine your idea.

  3. Follow the idea through the IBM Ideas process.

Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal ( - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal ( - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM. - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

Status Submitted
Workspace App Connect
Created by Guest
Created on Jan 25, 2024

Correctly transforms XML into valid JSON arrays

Description of problem submitted to IBM:


We configured our integration servers to send logging data to ElasticSearch using Logstash. We used basic configuration based on following article:

For the Logstash input protocol we used the Beats option.


In our message flow nodes we use a monitoring feature (Monitoring tab on its Properties view) in order to log events to ElasticSearch. Our input messages are in XML format (as attached in this error report) and contain multiple elements with the same name (e.g. Bid_TimeSeries) which is perfectly fine according to XML specification. Actually that is the way XML defines items in an array.


Based on our server configuration, event logs are transported to ElasticSearch (using Logstash). We are able to find it in the index, however the message does not contain the all "repeated" items/elements (e.g. Bid_TimeSeries) as originally defined in the XML message. We assume that XML messages from message flow are transformed to JSON format which is not valid based on JSON specification and thus not correctly logged in ElasticSearch. Please note the message in ElasticSearch does not contain all data elements (Bid_TimeSeries nr.1 and nr.2) as in the original XML message. JSON message is also attached in this error report as discovered in ElasticSearch.

„1. solution IBM“

We had further discussion with development team regarding the process of converting XMLNSC tree to JSON tree. When we perform this, what internally happening is that the XMLNSC tree is being copied to a generic tree and then this generic tree is being serialized as JSON using XPath.

This will not construct a tree which would set the repeating structure/element array types; instead we will get a copy from XMLNSC to JSON where the elements would get presented as standard, name and nameElement pairs.

Copying an XML array to valid JSON array would require scanning through all the children at the tree level and repeatedly call the child element if they are identical and this will slow down tree copy or JSON writer. So the solution opted for was to mark the JSON array as a JSON array and it is up to the user to do that when they create the JSON message.

So you will have to write logic to convert their messages to valid JSON message. Can you try the JSON approach mentioned in earlier update to construct the elements as valid JSON array elements and confirm it helps to get the expected result.

You can refer the documentation link for converting the message.

„Escalation on our part“

While I appreciate the detailed explanation of the technical process and the challenges involved, I must emphasize that our expectation is for IBM to provide a fully functional and efficient solution.

It is concerning to learn that the current conversion process does not effectively handle repeating XML structures or element arrays (which is very common in XML messaging), resulting in an incorrect JSON representation. The suggested workaround, which requires additional logic on our end to correctly handle repeating XML elements, shifts a significant burden on us.

Given the critical nature of this functionality in our operations, I kindly ask to revisit this issue and provide a solution that correctly transforms XML into valid JSON arrays without extra development work on our side. A solution that both maintains performance and ensures data integrity is essential.

„2. solution IBM“

From the update, we understand that you are looking for a solution from IBM that correctly transforms XML into

valid JSON arrays without requiring additional development on your side.

As discussed with our development team and as explained in the earlier update, there are lot of technical challenges involved in it.

Would you please raise an RFE to address / get this functionality added to the product.

Since RFE doesn't have any case open for it, I wonder if we should close this case? Have you opened a RFE yet?

Idea priority High