Master-Slave : An Architecture For Distributing Work

We switch gears in this episode and look at a pattern for getting work done.  The master-slave pattern can be confused with the client-server.  However, they are very different.  The main difference is how the work flows through the system.

The Master-Slave Pattern Defined

This pattern is a way to get large jobs done.  In contrast, the client-server pattern focuses on multiple users and requests.  The master-slave pattern provides an example of delegating work within a system.

The requests come into the master.  Then, the master splits up the work into pieces that are farmed out to the slaves.  When a slave completes the work, the results are sent back to the master.  The master then puts together the results and provides the result for the request.  The job done may be computational, third-party requests, or cross multiple persistence engines.

Share The Load The strength of this pattern is the ability for a request to be shared across resources.  However, note that this does not come for free.  Additional work is done to split up the task and merge the results.  That alone can be resource-intensive based on the job.  Therefore, we should only use this pattern with large jobs that have well-defined ways to split them up. An example is processing a set of data.  Each item within the collection can be shipped off to a slave process and then results returned.  Thus, the work is broken down per item, and we can spread the work across the slaves available.  Likewise, there are types of work within a request that make for good lines to split it up.  For example, there may be data manipulation required, storage retrieval, and computations to be done.  Again, these chunks of work can be sent to slaves that are best suited for each type of work. Different Patterns and Different Goals You may look at the prior paragraph and think we described a layered pattern.  That, again, is a different focus on how the request flows through the system.  This pattern is a single large request.  The layered pattern handles multiple requests and has a focus on the steps required for any request, not breaking down large pieces of work. Challenges The master-slave pattern is not a good fit for many tasks.  There will be problems breaking the effort into smaller pieces, or the request may be small enough that it does not need to be split.  Remember that we have an overhead in splitting the work and merging results.  That cost can exponentially increase the time and resources for a task if not executed properly.