InputMethod: Avoid needless complex conversion of key modifiers

As we only need to pass the modifiers around, rather than do anything
internally with them, simply pass them around with minimal conversion.