I think that:
sudo apt purge xfce4*
sudo apt autoremove
should do it.
I’ll point out that the other answers here are also correct. It depends on how you want to clean it from your system.
“apt remove” will only remove the packages, not the config files
“apt purge” will remove the packages and config files
“apt autoremove” will clean up the orphaned dependencies
“xfce4” will only remove the DE
“xfce4*” will remove the DE and most of the other packages that come with xfce
Not every package that comes with xfce has a name that starts with xfce4, just most of them do. You may need to identify and remove other packages too.
In NixOS when you remove the package from the list of packages you don't have the symlinks pointing to it anymore, so it is like it's not there (you can garbage collect it)