Correction of inter-target blendshape artifacts
Following is a technique for making corrective adjustments to two blendshape targets which must work in conjunction with each other. This technique may be applied in a variety of situations, such as the combination of a left-side smile which crosses the center line with a right-side smile which crosses the center line, a cheek movement that combines poorly with an eye movement, etc. It's most powerful with left/right shape targets that pull the mouth across the center line asymmetrically but also need to look good when dialed in together. Thanks to Hiroki Itokazu for demonstrating this technique!
For starters, let's assume a neutral head. The modeler creates two mouth shapes: a left-side grin (we'll call this "A") which pulls the mouth asymmetrically to the left, and a right-side grin (we'll call this "B") which pulls the mouth asymmetrically to the right. "A" and "B" look great when dialed in separately, but produce artifacts at the middle of the mouth when dialed up together on the neutral. What to do?
First, dial up "A" and "B" together on the neutral, and save the result out as a duplicate (we'll call this "C"). Duplicate this model again (we'll call this "D"). Sculpt the vertices on "C" to look nice.
Create a new blendshape deformer that uses "D" as the base, and applies "A", "B" and "C" as blend targets.
Dial the "C" target to 50%, the "A" target to 100%, and the "B" target to 0%. Duplicate the result (which we'll call "A2").
Dial the "C" target to 50%, the "B" target to 100%, and the "A" target to 0%. Duplicate the result (which we'll call "B2").
What we have done is to propagate half of the corrective vertice adjustment into each of the original grin shapes. The result of this, "A2" and "B2", now become your new left and right grin targets. They should still look good on their own, but when dialed in together will also combine perfectly to form the same shape as the cleaned up combo "C".
While commonly used to correct problems which arise between two shapes, this technique can also be applied to artifacts which are the result of numerous shapes. The key is to adjust the corrective percentage of "C" appropriately. For example: when cleaning up four targets, the "C" shape would be dialed in 25% for each. The correction percentages can actually be varied from target to target, provided they sum to 100%.