Hi friends!
- How much the overall history size affects performance. 50M+ completed processes will be stored for the possibility of searching through them, sorting. Each of these processes contains no more than 100-200 history events. To understand the context: our process is a user action in the bot (step). This history must be stored because various other logic depends on it, for example, send a webhook if there is a previously completed process with such a search parameter and a user ID in the process name. This history is also required to sort the view: show me all users with such and such completed processes (steps) + such and such a search parameter. As an option, store (essentially duplicate) such information in an external database, obliging the process to set the appropriate flags there, etc. But if the overall size of the history does not significantly slow down the speed of the search API, then perhaps an external database is redundant?
- Policies for cleaning up completed processes - can they be configured in such a way that some processes would be stored for a year, and others for a week?
- Support for scylladb never appeared?
Thank you so much for your work! You have a great product!