Plasmoids will close if other plasmoids are open in the panel, they should also close if the panel settings is opened.
Confirming.
Git commit e0bc020cf33b6126b2ef425f943012b8473b8312 by Marco Martin. Committed on 14/10/2014 at 16:26. Pushed by mart into branch 'Plasma/5.1'. close all applets when configuring M +3 -0 containments/panel/contents/ui/main.qml http://commits.kde.org/plasma-desktop/e0bc020cf33b6126b2ef425f943012b8473b8312