Parallelize work using parwork
In order to process a lot of work we have to parallelize work across all cores, and especially if it’s CPU bound. Go has goroutines, which can be used to parallelize the work, but there is the cost of context switching for a lot of goroutines. Minimizing this context switching can be achieved by using a fork-join model when processing work. Parwork solves this problem by using goroutines, channels and waitgroups. It creates workers (goroutines) that pull work of a queue (channel), process the work and report the work back to a queue (channel). This is done in a abstracted way so the user has to provide implementation for: ...