Add separate PME rank SimluationWorkload flags

Add separate PME rank SimluationWorkload flags

Added the complete set of flags that indicate whether separate PME ranks
are used and what type of PP-PME communication is used.

These flags allow simplifications in the force schedule as well as reducing
the use of the confusingly defined cr->duty for now in do_force().

The additional benefit is a reduced reliance on passing around commrec
for checks related to parallelization.

Refs #3913