You try and keep X11 off your servers, but sometimes there is that one application.  If it’s a RHEL/CentOS application there is a good chance you can get away with installing only two fonts.
yum install dejavu-sans-fonts dejavu-serif-fonts
Note: It is possible other packages will be installed as dependencies.