Sometimes the the dialog has a "save password" with kwallet, sometimes it doesn't. Either way, it prompts every time kmail is opened - I think every time you open kmail. It's a KDE app, shouldn't it just get the password from Kwallet? It breaks sending signed emails in Kmail.
Actually
Actually it's probably that my keyring (KeepassXC) was closed. Still, I though it would just get the values from KWallet. But maybe it would if I had a standard config.