Cavalcade: WordPress Jobs at Scale — Human Made

At the heart of every web application is a basic process: receive a request, return a response. With the right architecture, this process is able to serve everything from the smallest site all the way up to the very largest. Once sites start getting more complex, there’s quickly a need for two separate-yet-related abilities: scheduled tasks, and asynchronous processing of long-running tasks. WordPress includes the ability to do both of these through a system called wp-cron. It offers scheduled and repeating tasks (just like cron), and can be used for asynchronous processing. However, it has serious issues when running at scale, like unreliability, sequential processing, and compatibility with multisite. Replacements require complex setup processes, don’t integrate well with WordPress, or don’t scale for real production use.
To fix these problems, we built Cavalcade, a horizontally-scalable WordPress jobs processing solution. We’ve been running Cavalcade in production for almost two years, and it’s also in use on WordPress.org, so we’re confident in its stability and capability.
Limitations of wp-cron
Cavalcade originally came out of project requirements
Source: https://managewp.org/articles/15224/cavalcade-wordpress-jobs-at-scale-human-made

source https://williechiu40.wordpress.com/2017/05/29/cavalcade-wordpress-jobs-at-scale-human-made/