The situation is a bit more complex. Your question however is very legitimate, since even writers of textbooks show some confusion about this aspect.
It is not the surface metals on the silica that make silanols acidic. The metal impurities inside the matrix of the silica are responsible for acidifying the silanols. Metal ions on the surface of the silica from the stainless steel in the instrument or the brown solvent bottles can be removed again with EDTA or other complexing agents. Metal ions in the matrix of low purity silicas, mostly alumina and iron, are not accessible to EDTA, nor to any acid treatment in the preparation of the silica. Dispite the fact that they are buried in the matrix, or maybe because of that, they acidify the surface silanols.
A high purity modern silica like Symmetry has a pKa around 7. An older silica like Spherisorb has two distinct silanol populations, one of them being rather acidic. I do not recall the pKa's out of the top of my head, but they can be found in the literature.