Even more tolerance in layer size sanity check

Appearantly, there are files out there with layers much
larger than the actual image. The layer size is now checked
against larger limits (32K in each dimension, 256M total).

BUG: 308143
FIXED-IN: 4.10