Okay... So with the latest update to xorg-server (I think it was 6.5), xdm stopped loading XFCE4 and I couldn't get it to load anything but TWM.
Here was the problem:
Somehow, an important file got removed. The file is this one:
The contents of the file is simply:
With that, (and all the other /etc/rc.conf and /etc/conf.d/xdm settings in place), xdm and slim will now properly load XFCE4!
Just thought I'd share because google was totally confused. Hopefully this will straighten things out.