Any time you filter a sample, you run two risks:
- loss of your analytes by adsorption onto the filter
- addition of compounds leached from the filter material.
Paper is the most heterogeneous of the filter media you mentioned, and so presumably has the highest potential for adding impurities.
Paper and glass fiber are both relatively polar (paper because of its water content, and glass fiber because of surface silanols); polar analytes may stick to either (especially if your diluent is an organic solvent).
PTFE and nylon are less polar, so non-polar compounds may stick (especially if the diluent is highly aqueous).
Of course, in many cases, it doesn't matter