The SMS reply button only appears when messaging is the default sms app on the phone. The stock nexus 5 rom does not include messaging, it uses messenger instead. Test by flashing a custom rom that has messagning app and reply button appears and works. Won't work with default messenger app on nexus 5 with stock rom.
If you have an Android that is new enough you will get to reply any notification that exposes the reply feature.