Integration

Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:

Post your ideas

Start by posting ideas and requests to enhance a product or service. Take a look at ideas others have posted and upvote them if they matter to you,

  1. Post an idea

  2. Upvote ideas that matter most to you

  3. Get feedback from the IBM team to refine your idea

Help IBM prioritize your ideas and requests

The IBM team may need your help to refine the ideas so they may ask for more information or feedback. The offering manager team will then decide if they can begin working on your idea. If they can start during the next development cycle, they will put the idea on the priority list. Each team at IBM works on a different schedule, where some ideas can be implemented right away, others may be placed on a different schedule.

Receive a notification on the decision

Some ideas can be implemented at IBM, while others may not fit within the development plans for the product. In either case, the team will let you know as soon as possible. In some cases, we may be able to find alternatives for ideas which cannot be implemented in a reasonable time.

If you encounter any issues accessing the Ideas portal, please send email describing the issue to ideasibm@us.ibm.com for resolution. For more information about IBM's Ideas program visit ibm.com/ideas.

Status Delivered
Created by Guest
Created on May 20, 2013

Provide topic or subscribtion option to duplicate msg id

When messages are published to a topic, either via a direct publish, or by putting the messages to a qalias with targtype(topic) a message id is assigned. When the message is delivered to any subscriber, a different msg id appears on the message. The message delivered is therefore not identical in all respects to the message sent.

Please provide an option to allow exact duplicates of messages to be delivered.

This could be implemented as a topic option (duplicate msgid) or as a subscriber option (receive duplicate msgid).

Idea priority Medium
Use Case
Imagine an application which puts requests messages and waits for the response to be returned to a replytoq. The organization would like to be able to collect a copy of all messages sent by the application (say for audit purposes). The queue is deleted, and a qalias defined which has targtype(topic). The topic is defined, and 2 subscriptions are created. One subscription puts the message to the original target, and the second one puts it to an audit queue. At the moment, this scenario cannot be implemented without changing the applications, because the server application which receives the request will get a message with a different msgid than that sent. If it follows the default reply mechanism (copy msgid to correlid) and the requesting application also uses this convention, the reply will not be found because the correlid will be wrong. With an option to preserve message ids, this use case becomes feasible and useful.
RFE ID 35062
RFE URL
RFE Product IBM MQ
  • Guest
    Jan 10, 2022
    The streaming queues feature provides a way to duplicate messages while maintaining msgid information
  • Guest
    Dec 8, 2021

    Lack of this also causes problems implementing AMQP request/response pattern with legacy application. The scenario is using an AMQP consumer to a legacy queue based application that uses msgid/correlid to match the request to a response. At the moment the message has to flow over a topic for AMQP/MQLite (although logically it has no need to be a pub/sub). The new message_id means the response's correlid is that of the mub/sub message, not the original.

    Would be great to see this feature implemented, making integrations a lot simpler.

  • Guest
    Sep 25, 2020

    This is something we would like to implement in a future version of MQ

  • Guest
    Jul 27, 2020

    Hello IBM,
    Can you please provide an update on this RFE ?
    Thanks

  • Guest
    Mar 21, 2019

    As was discussed at Think this year, a message produced by pub/sub that was identical to the original message including the MQMD would be fantastic for the IBM Event Streams (Kafka) interface.

    Request/Response messages using standard MQ settings and expectations (copy message id to correl id for example) could still be captured into the event stream without impacting normal MQ application operations.

  • Guest
    Feb 2, 2019

    currently user/customer programs can exactly duplicate messages either using API exits or running alongside the queue manager in bindings mode. obviously it can be done in client mode too but that would be a little inefficient and relies on network connectivity.
    Within MQ on Cloud or MQ Appliance there is no option to run user/customer code or use API exit to duplicate messages.
    Also SYSTEM messages consumed by differenct applications for example from EVENT queues need to be the same for both consuming apps (for example for autit purposes).
    This RFE has been around for a long time, could save MQ Admins a lot of headache and get rid of a lot of maintenance on all kinds of duplication programs.

    if not now... when will there be a better time to deliver this inside the product?

  • Guest
    Jun 20, 2018

    A duplicated message via a topic is losing its original context information and may therefore not used in any scenario where the context is needed like MQ request reply.

  • Guest
    Dec 13, 2016

    Since raising this RFE, I have realised that it is actually too small in scope. The current pub/sub interface does not preserve a number of other MQMD fields, in addition to the MessageID.

    I would actually want an updated version of pub/sub to present to every subscriber exactly the message that was sent by the publisher, whether that was through the publish interface on a topic, or via MQPUT to a QAlias pointing to a topic.

    I realise that this could break existing programs which rely on existing behaviour, so like many other things in MQ, it should be possible to request either new mode (identical MQMD) or legacy mode (whatever happens now).

    Ideally the option of which style message to present should be at the subscriber end, so that subscribers wanting new function, and subscribers wanting old function could both subscribe to the same topic and get what they want. Maybe an open option, with the default value being to preserve current behaviour?

    Regards,

    Neil Casey.

  • Guest
    Oct 26, 2016

    any news to report on this RFE?

  • Guest
    Oct 15, 2015

    Due to processing by IBM, this request was reassigned to have the following updated attributes:
    Brand - WebSphere
    Product family - Integration
    Product - IBM MQ

    For recording keeping, the previous attributes were:
    Brand - WebSphere
    Product family - Connectivity and Integration
    Product - IBM MQ

  • Guest
    Mar 19, 2015

    IBM should delete all event queues, replacing them with topics. A command would be available that would define all the event queues as well as administrative subscriptions for them.

    Arguably, the QMgr event property settings could also be eliminated. The act of subscribing replaces toggling the event on.

    None of this is possible, of course, if event messages received by multiple things monitoring these queues cannot be correlated.

    Auditor: How do you know these two messages represent the same event?
    Admin: Well I dump them as hex files and run a diff. If the only thing that changes is the Message ID they are the same message.
    Auditor: Great. Give me a report on the remaining 3000 event messages from the incident across both the monitoring and audit components by tomorrow.
    Admin: How about I freshen my resume instead?

  • Guest
    Mar 19, 2015

    Back in August 2012 on the MQ Listserv, a discussion took place about an HTTP app doing request/reply over MQ. (Thread: Distribution lists in JMS) Original poster described a need to distribute the MQ reply to multiple interested parties. In the point-to-point interface as was then implemented, the requestor selected the reply by the ID. OP wanted a Dist List due to lack of time to get up to speed on Pub/Sub.

    In a reply Jeff Lowrey pointed out that while Pub/Sub would be the preferred architecture, IBM's implementation breaks the request/reply correlation so it would be infeasable regardless of time available for implementation. Inserting Pub/Sub either inbound or outbound fail equally badly.

  • Guest
    Mar 19, 2015

    At least two large insurance companies currently use exits to duplicate messages to provide audit and operational diagnostics. I know because I've worked engagements with both. Neither of these systems would survive conversion to Pub/Sub since doing so would break the chain and make the message path through the network untraceable. Both companies have implemented alternative architectures in the cases where Pub/Sub would have been the natural solution. Contact me privately if you want names, or look up my ISSW engagement history.

  • Guest
    Feb 27, 2015

    Not just msgid and correlid, but other MQMD fields like userid should be unchanged (or at least an option to set all context from the original message).

    Admin (define) sub must be able to specify any optional parameter added to support this.

  • Guest
    Jul 28, 2014

    Should also include Correlid duplication.

  • Guest
    Jun 8, 2013

    All current comments are valuable reasons for making the product enhancement.

  • Guest
    Jun 7, 2013

    Note that the JMS spec 3.4.3 says:

    "The JMSMessageID header field contains a value that uniquely identifies each message sent by a provider.

    When a message is sent, JMSMessageID is ignored. When the send method returns, the field contains a provider-assigned value."

    Section 3.4.5 says that:

    "A client can use the JMSCorrelationID header field to link one message with
    another. A typical use is to link a response message with its request message.
    ...
    Since each message sent by a JMS provider is assigned a message ID value, it is convenient to link messages via message ID."

    Clearly in 3.4.5 the intent is that the responder puts the message ID into the correlation ID. So if the message consumed contains an ID that differs from the one returned by the send method, it doesn't meet 3.4.3 and isn't possible to use the service as described in 3.4.5.

    Also note that "the provider" as described in the spec includes the pub/sub broker as well as the point-to-point messaging engine. An application that was forwarding messages might move MessageID to CorrelationID but *the* *provider* is not forwarding the messages and the behavior in both PtP and Pub/Sub domains with respect to the MessageID and CorrelationID is supposed to be consistent. Among other things, that's why there are now CF objects rather than QCF and TCF.objects. The app is supposed to be able to resolve these at run time, the implication being that switching between queue and topic delivery should be transparent to the app, not require reengineering to handle differences in how somethign so foundational as message IDs are treated.

  • Guest
    Jun 7, 2013

    Attachment (Description): The JMS Specification requires this functionality. See 3.4.3 and 3.4.11. Note that 3.4.11 says the message ID is set by the send method. If the ID returned to a client is not the same as that returned by the send method, this requirement is not met.

  • Guest
    Jun 7, 2013

    Wtih Pub/Sub, we need the ability to create exact copies of the messages.

  • Guest
    Jun 7, 2013

    It is always beneficial to not lose track of message origination infromation - whenever posssible.

  • Load older comments
0 MERGED

Option to deliver Publishers Useridentifier in MQMD to Subscriber

Merged
Using MQ PublishSubscribe a message delivered to a subscribers queue contains the subscribers useridentifier per default.We have an application where the subscriber needs to know who has published a message. This is not reliable possible when the ...
over 5 years ago in MQ, MQ Advanced & MQ Appliance 1 Delivered
0 MERGED

Built-In Fanning function in MQ

Merged
We have a program that fans multiple copies of messages to multiple queues which are identified in a namelist. We use this extensively across our enterprise for migration purposes as well as just providing a message to several different Program O...
over 5 years ago in MQ, MQ Advanced & MQ Appliance 1 Delivered