cache: Remove mailboxes when they are, well, removed

Thanks to Caspar for the initial troubleshooting -- I've known about this bug
for several months, but haven't bothered to fix it properly. His analysis
pinpointed the root cause of the problem and made it easy to fix this.

BUG: 320830
REVIEW: 110896