Also remove the action from the menu then delete it. (cherry picked from commit 0f642449)
mentioned in merge request !280 (merged)