The reason for the absense of flow through the particles is the large difference between the dimensions of the interparticle channels and the channels inside the particle (yes, the channels inside the particles are not dead end, they go through and through the particle).
For your standard column, the size of a channel between the particles is in the order of 1 micron. The average pore size of a packing is about 10 nm. The ratio in the channel dimensions outside the particle to inside the particle is about 100. The linear velocity is proportional to the square of the channel size, and 100 x 100 is 10000. Therefore the flow inside the particle is about 10 000 times slower than the flow outside the particles, i.e. essentially nonexistent, and MUCH slower than diffusion.
A monolith also has two types of pores, at least when one is talking about the Chromolith. There are the channels through which the liquid is flowing, just like through the interstitial spaces between the particles. And then there are the real pores, where the real surface area is and where the bonded phase is attached. The dimensions are about the same and the ratio is about the same as for particles, and there is no flow in the micropores of a monolith either.