The problem with N2 as a carrier gas in GC/MS is that it will provide a strong mass at m/z 28. This gets in the way of instrument optimization steps like a leak check.
With that out of the way, the next problem will be one similar to that seen with hydrogen. Ionization of the carrier gas can result in the carrier gas affecting the ionization processes in the ion source. Hydrogen is commonly used in GC and is occasionally used in GC/MS, and some effects are seen on sensitivity and spectra. I would take some caution moving away from helium to any other carrier gas for this reason.
Remember that a carrier gas is not intended to change the fragmentation process - a collision gas is.
Also, there is the issue that the optimum linear velocity becomes quite low when using N2 in a GC column, so you will have to slow down your chromatography to maintain the quality of a chromatogram. (And this chromatographic issue may be the biggest reason that folks don't go toward nitrogen as a carrier gas.)
I have heard of GC/MS being done with nitrogen carrier. But there are some steps in instrument optimization that would require modification. And then, it depends on the options available for that particular instrument. Also, the instrument design will place some limitations. The efficiency of pumping changes depending on the particular gas used. And the design of the instrument may or may not allow you make this change.
If you have an application that is pushing you toward N2 as a carrier gas, talk with your instrument manufacturer. They may be able to help you out.