backends/drm: fix common mode generation

There were overflows in the bandwidth estimation calculation. To simplify things,
also only generate common modes if the display doesn't advertise them on its own
already.

(cherry picked from commit 3f3d41f2)