I have a GCP subscriber that uses Streaming Pull API to pull messages from a GCP subscription. There is only one GCP subscription and one subscriber attached to this subscription. I am seeing delay of several seconds and I suspect that my subscriber settings are at fault here.

I have the following settings for the subscription:

• Message retention duration = 10minutes
• Expiration Policy = 1 day
• Enable Exactly Once Delivery = true

The following are the settings for my subscriber:

• AckDeadline = 2s
• AckExtensionWindow = 1s
• MaxTotalAckExtension = 1s
• FlowControlSettings - Max Outstanding Element Count = 5000, Max Byte Count = 5000 * 560

Previously, I was having delays in minutes (~10) and seeing lots of stale messages in the logs. I figured this was due to the flow control settings as I had 50K as my outstanding message count limit. I decreased the number to 5K and I saw an amazing improvement in my latency. After the change to the outstanding message count my latency dropped to ~5s. I want to resolve this latency issue and I feel latency of the order of milliseconds is appropriate for my application. What settings do I need to change on my subscriber/subscription? I am running a single subscriber instance on 4vCPU and 16GB Memory.

I have a GCP subscriber that uses Streaming Pull API to pull messages from a GCP subscription. There is only one GCP subscription and one subscriber attached to this subscription. I am seeing delay of several seconds and I suspect that my subscriber settings are at fault here.

I have the following settings for the subscription:

• Message retention duration = 10minutes
• Expiration Policy = 1 day
• Enable Exactly Once Delivery = true

The following are the settings for my subscriber:

• AckDeadline = 2s
• AckExtensionWindow = 1s
• MaxTotalAckExtension = 1s
• FlowControlSettings - Max Outstanding Element Count = 5000, Max Byte Count = 5000 * 560

Previously, I was having delays in minutes (~10) and seeing lots of stale messages in the logs. I figured this was due to the flow control settings as I had 50K as my outstanding message count limit. I decreased the number to 5K and I saw an amazing improvement in my latency. After the change to the outstanding message count my latency dropped to ~5s. I want to resolve this latency issue and I feel latency of the order of milliseconds is appropriate for my application. What settings do I need to change on my subscriber/subscription? I am running a single subscriber instance on 4vCPU and 16GB Memory.

• google-cloud-platform
• google-cloud-pubsub
• publish-subscribe

1 Answer 1

• AckDeadline = 2s

• AckExtensionWindow = 1s

• MaxTotalAckExtension = 1s

These values, especially for MaxTotalAckExtension, are low and can cause frequent redeliveries if your subscriber is not processing and acking the messages in time. Are you seeing ack expirations in the subscription's subscription/expired_ack_deadlines_count metric? Have you tried using the default values for these settings and did you see high delivery latencies then?

And since your subscription is enabled for exactly-once delivery, are you seeing any failures in the subscription/exactly_once_warning_count metric?

You can also reach out to Google Cloud support to take a closer look, providing your project, topic, and subscription names.