followup: Do provide just a "Close" action

We are already confirming deleting entries in the list. Previously we
had to additionally press "Apply" to save these changes. This doesn't
make sense.

Just provide a "Close" button to close the dialog and save the changes
whenever we close the dialog.