Otherwise we keep the last use of the buffer's which is clearly wrong. (cherry picked from commit a2913d1e)