Why is implementing a queue an anti-pattern?

Hi! I’m trying to understand this statement more under Temporal Antipatterns: “Trying to implement a queue in a workflow (because people hear we replace queues)”.

I couldn’t find any prior discussion on this, and would appreciate some elaboration on what “queue” means here and why exactly this is an anti-pattern. Thanks!

1 Like

Hi @evanlimanto I believe the point that statement is trying to make is what Maxim talks about in this video (time included but watch the whole thing :slight_smile: )
Basically that Temporal eliminates many hard problems for you and for some applications can eliminate their use of some queueing technologies.
I don’t think it means you are not allowed to use some programming language queuing structures inside your workflow code (which you absolutely can)

Hi @tihomir, I’m also interested in more explanation around this anti-pattern.

In this presentation, it appears to me that Maxim is, if anything, reinforcing the (mistaken?) idea that Temporal replaces queues. The solution he presents (at 10m5s), with queuing implemented inside the Temporal server, is replacing an implementation that was using Kafka.

To better understand this anti-pattern, I’d like to see an example where “real” queues are needed in addition to Temporal.

Do I understand correctly then that the intent of the anti-pattern is to say “Don’t re-create bad queuing patterns in Temporal”?

My experience tells me that a very narrow set of scenarios fit these limitations. In most cases, tasks are not independent, can execute for a long time, require long retries, require actual error handling, and benefit from transactionality between the database and the queue.

If, rather, the intent is to say “Use queues for queuing when appropriate, not Temporal” (as I interpreted it) then I would like to see one of these hybrid examples.

From Temporal Antipatterns​

Temporal Antipatterns
Please request elaboration on any of these.

  • Trying to implement a queue in a Workflow (because people hear we replace queues)

Is there anything more to added here?

In a workflow that processes an event stream, would the events typically be posted to the workflow using signals?

If so, how to deal with the limit on number of signals?

Would a queue typically be used to buffer the event stream?

Or is there a better way?