Currently there are two avatar caches - one is in the offline database cache, one is in a config file saving the avatar tokens (because they are not available if contact is offline). These should be merged and just the database used.
Thank you for the bug report. As this report hasn't seen any changes in 5 years or more, we ask if you can please confirm that the issue still persists. If this bug is no longer persisting or relevant please change the status to resolved.
Dear user, unfortunately Telepathy is no longer maintained. Please migrate to another solution, e.g. for Jabber a possibility is Kaidan, for Matrix a candidate is NeoChat.