What you have seen is the reason for so many stationary phases available. If the retention indicies of two compounds are close on a particular stationary phase, you will have a hard time separating them. So, Is there a partular reason for changinf from a DB-1701 to a DB-5? If not, I would reccomend reinstalling a 1705. If you were attempting to resolve a different coeltuion, then you may have to try again. And, there are some phases designed for specific applications - like pesticides. Check the vendor web sites and see if they have a column that separates all the compunds of interest to you.
If the peaks are almost resolved, sometimes changing to a column that is longer, narrower, or has a thinner film can do the trick. Longer increases the time that the compounds are on the column - and the peaks become broader, adversely affecting LOD's. Narrower columns and thinner films can give a lower height equivalent theoretical plate - but, you can not put as much material on the column.