Fix crash when activating Passthrough mode for a group with transparency mask

Masks' extent should not be included into the extent of a passthrough
layer, because they are explicitly disabled in this mode.

BUG:351224
Fixes T567