Panel: Change anchoring to keep ruler size constant

Issue was that when changing the anchoring, two anchors would be
applied at the same time for a brief moment and that would change
the size of the elements.

BUG: 476721