colorimetry: use 4x4 matrices for colorimetry transforms

This is so that offsets can be represented in the matrices and not just
scaled and rotated coordinate systems

(cherry picked from commit b1414033)