When there are more than one HDMI or Display Port display connected, plasma-pa does not let you switch which HDMI/DP should receive the audio stream. Using other frontends this can be done fine. e.g. on pavucontrol-qt, I can go to Configuration tab and switch the profile over all available ports. In case this is a video card specific bug, mine is a Nvidia 1080 GTX.