Provide topic or subscribtion option to duplicate msg id

Lack of this also causes problems implementing AMQP request/response pattern with legacy application. The scenario is using an AMQP consumer to a legacy queue based application that uses msgid/correlid to match the request to a response. At the moment the message has to flow over a topic for AMQP/MQLite (although logically it has no need to be a pub/sub). The new message_id means the response's correlid is that of the mub/sub message, not the original.

Would be great to see this feature implemented, making integrations a lot simpler.

This is something we would like to implement in a future version of MQ

Hello IBM,
Can you please provide an update on this RFE ?
Thanks

As was discussed at Think this year, a message produced by pub/sub that was identical to the original message including the MQMD would be fantastic for the IBM Event Streams (Kafka) interface.

Request/Response messages using standard MQ settings and expectations (copy message id to correl id for example) could still be captured into the event stream without impacting normal MQ application operations.

currently user/customer programs can exactly duplicate messages either using API exits or running alongside the queue manager in bindings mode. obviously it can be done in client mode too but that would be a little inefficient and relies on network connectivity.
Within MQ on Cloud or MQ Appliance there is no option to run user/customer code or use API exit to duplicate messages.
Also SYSTEM messages consumed by differenct applications for example from EVENT queues need to be the same for both consuming apps (for example for autit purposes).
This RFE has been around for a long time, could save MQ Admins a lot of headache and get rid of a lot of maintenance on all kinds of duplication programs.

if not now... when will there be a better time to deliver this inside the product?

A duplicated message via a topic is losing its original context information and may therefore not used in any scenario where the context is needed like MQ request reply.

Since raising this RFE, I have realised that it is actually too small in scope. The current pub/sub interface does not preserve a number of other MQMD fields, in addition to the MessageID.

I would actually want an updated version of pub/sub to present to every subscriber exactly the message that was sent by the publisher, whether that was through the publish interface on a topic, or via MQPUT to a QAlias pointing to a topic.

I realise that this could break existing programs which rely on existing behaviour, so like many other things in MQ, it should be possible to request either new mode (identical MQMD) or legacy mode (whatever happens now).

Ideally the option of which style message to present should be at the subscriber end, so that subscribers wanting new function, and subscribers wanting old function could both subscribe to the same topic and get what they want. Maybe an open option, with the default value being to preserve current behaviour?

Regards,

Neil Casey.

any news to report on this RFE?

Due to processing by IBM, this request was reassigned to have the following updated attributes:
Brand - WebSphere
Product family - Integration
Product - IBM MQ

For recording keeping, the previous attributes were:
Brand - WebSphere
Product family - Connectivity and Integration
Product - IBM MQ

IBM should delete all event queues, replacing them with topics. A command would be available that would define all the event queues as well as administrative subscriptions for them.

Arguably, the QMgr event property settings could also be eliminated. The act of subscribing replaces toggling the event on.

None of this is possible, of course, if event messages received by multiple things monitoring these queues cannot be correlated.

Auditor: How do you know these two messages represent the same event?
Admin: Well I dump them as hex files and run a diff. If the only thing that changes is the Message ID they are the same message.
Auditor: Great. Give me a report on the remaining 3000 event messages from the incident across both the monitoring and audit components by tomorrow.
Admin: How about I freshen my resume instead?

Back in August 2012 on the MQ Listserv, a discussion took place about an HTTP app doing request/reply over MQ. (Thread: Distribution lists in JMS) Original poster described a need to distribute the MQ reply to multiple interested parties. In the point-to-point interface as was then implemented, the requestor selected the reply by the ID. OP wanted a Dist List due to lack of time to get up to speed on Pub/Sub.

In a reply Jeff Lowrey pointed out that while Pub/Sub would be the preferred architecture, IBM's implementation breaks the request/reply correlation so it would be infeasable regardless of time available for implementation. Inserting Pub/Sub either inbound or outbound fail equally badly.

At least two large insurance companies currently use exits to duplicate messages to provide audit and operational diagnostics. I know because I've worked engagements with both. Neither of these systems would survive conversion to Pub/Sub since doing so would break the chain and make the message path through the network untraceable. Both companies have implemented alternative architectures in the cases where Pub/Sub would have been the natural solution. Contact me privately if you want names, or look up my ISSW engagement history.

Not just msgid and correlid, but other MQMD fields like userid should be unchanged (or at least an option to set all context from the original message).

Admin (define) sub must be able to specify any optional parameter added to support this.

Should also include Correlid duplication.

All current comments are valuable reasons for making the product enhancement.

Note that the JMS spec 3.4.3 says:

"The JMSMessageID header field contains a value that uniquely identifies each message sent by a provider.

When a message is sent, JMSMessageID is ignored. When the send method returns, the field contains a provider-assigned value."

Section 3.4.5 says that:

"A client can use the JMSCorrelationID header field to link one message with
another. A typical use is to link a response message with its request message.
...
Since each message sent by a JMS provider is assigned a message ID value, it is convenient to link messages via message ID."

Clearly in 3.4.5 the intent is that the responder puts the message ID into the correlation ID. So if the message consumed contains an ID that differs from the one returned by the send method, it doesn't meet 3.4.3 and isn't possible to use the service as described in 3.4.5.

Also note that "the provider" as described in the spec includes the pub/sub broker as well as the point-to-point messaging engine. An application that was forwarding messages might move MessageID to CorrelationID but *the* *provider* is not forwarding the messages and the behavior in both PtP and Pub/Sub domains with respect to the MessageID and CorrelationID is supposed to be consistent. Among other things, that's why there are now CF objects rather than QCF and TCF.objects. The app is supposed to be able to resolve these at run time, the implication being that switching between queue and topic delivery should be transparent to the app, not require reengineering to handle differences in how somethign so foundational as message IDs are treated.

Attachment (Description): The JMS Specification requires this functionality. See 3.4.3 and 3.4.11. Note that 3.4.11 says the message ID is set by the send method. If the ID returned to a client is not the same as that returned by the send method, this requirement is not met.

Wtih Pub/Sub, we need the ability to create exact copies of the messages.

It is always beneficial to not lose track of message origination infromation - whenever posssible.