Enhancement to the IBM MQ Source Connector for max poll time

Opening this idea to add new configurable config to IBM MQ Source Connector for max poll time. 

An enhancement to IBM MQ Source Connector under version 2.4.0 was released to expose new configurable config, mq.receive.subsequent.timeout.ms, to adjust the JMS receive timeout on the subsequent receive before returning the batch of messages to Kafka connect. 

This config improved the efficiency of the Connector, allowing bigger batches of messages. 

The draw back, is that you could get into a situation where continuous delay of messages comes in within the mq.receive.subsequent.timeout.ms timeout. This will cause the poll of messages to continually to run until the task hits mq.batch.size or mq.receive.subsequent.timeout.ms. 

For example:

• If we have the following configuration:
  • mq.batch.size = 5000
  • mq.receive.subsequent.timeout.ms = 10
• If an application consistently put 1 message every 5ms to the queue, the connector task will continually build the batch of messages of to 5000 messages. Taking around 25 seconds (5000 messages x 5ms / 1 message) before returning the batch of messages to Kafka Connect. 

To prevent this situation, we would like to have an max poll time to close the batch of messages, even if mq.batch.size or mq.receive.subsequent.timeout.ms was not breached. 

Setting the max poll time of 5000ms, the task would return batch of messages to Kafka connect if poll of messages has been running more than 5 seconds.