We may only want to add gossip keys e.g. for drafts and no main key to add. Make sure it does not fail with an error.