Removing an album from the playlist currently requires removing each track one by one, or alternatively replacing the whole playlist with another album. It should only take one click to remove an album from the playlist.
Oh yes I have wanted this so often! We could probably add hover buttons for the album header too, just like for individual songs.
I agree that would be a good way to implement this.