Polling Publications
The Poll endpoint allows you to consume messages from the DealCloud Publications service. A Poll request creates a connection that allows you to consume events from the message queue of your specified Publication topic(s). A connection can be defined by the API request body parameters listed below.
Polling Connection Params
| Parameter | Description |
|---|---|
topics | Specify the Publication topic(s) you want to create a polling connection to |
TimeOut | Defines the max connection time (in ms) of the polling request |
Count | Defines the max number of messages the connection stays open for |
A polling connection stays open until either the specified Count of messages has been returned or the TimeOut (in milliseconds) has elapsed, whichever happens first. Set Count and TimeOut together to control responsiveness and throughput: use a low Count with a high TimeOut for long-polling and low latency, or increase Count when you want/expect more events per connection.
Tip: Let's say you create a polling connection, with a timeout of 60000 and a count of 5. Your connection will be initiated and will last for either 60 seconds or until 5 events are returned, whatever happens first. So for example, if the queue receives the 5th event at 15 seconds, it will close out the connection and respond with the 5 events at around 15 seconds.
At that point, you can begin processing those messages while re-initiating the polling connection to continue 'listening' for new messages.
Request
POST {pubUrl}/api/rest/v1/publication/poll
Authorization: Bearer {{token}}
Content-Type: application/json
{
"Topics": [
"my-topic"
],
"TimeOut": 30000,
"Count": 1
}Response
Response format varies between data, schema, and user topics. See event types and responses for details and examples.
Response Errors
400 Bad Request
"One or more topics does not exist in Kafka. Probably no messages has been sent into that topic yet."Sessions
Polling Publications starts a Session that represents a consumer tracking your position in the topic(s). Polling within an active session allows you to cycle through the messages chronographically. The session persists across poll requests and expires after 5 minutes of inactivity. When it expires, the next poll creates a new session and resumes from the last acknowledged offset (or from the start of the queue if nothing has been acknowledged).
Note: Acknowledging a message does not move the current session's position. To start polling from the last acknowledged offset, let the current session expire (sessions expire 5 minutes after the last poll) and then execuse a new poll request — the new session will resume from the latest acknowledged position.
FAQ & Design Recommendations
What does TimeOut do?
Establishes the maximum connection time. When the timeout is reached, a response is returned unless the Count is satisfied first.
What does Count do?
Establishes the maximum number of events returned in a single connection request. If the topic queue has fewer remaining events than the requested Count, the connection will remain open until the TimeOut elapses.
What should my TimeOut be?
We recommend setting a high timeout, so your connection can stay open and "listen" for new messages.
Do Publication connections support long polling?
Yes, long polling is supported and encouraged.
How do we build the most responsive system?
To build the most responsive, low-latency system where you have events as soon as they're published, you should specify a low count and a high timeout. The best design alternates low count high timeout with high count high timeout.
How many topics can I poll against?
While you can poll against multiple topics at once, it is recommended that you only poll against one topic per poll in a single request.