When a file has unusual bytes in its name, Dolphin cannot delete it. I am not sure which bytes are causing the trouble, but after creating 254 files for every byte in the 0x01 to 0xff range, I could neither delete them nor the parent folder. To reproduce it, you can clone the repo at https://github.com/benibela/nasty-files and then you cannot delete it.
If you put random raw bytes 0x80-0xFF in a filename, you are creating illegal UTF-8 encoded filenames. Those are not supported by the QString-based file API.
See also bug 165044.