Keep in mind that "sensitivity" is the derivative of the curve of output versus input. When speaking casually, we often mix up the terms sensitivity with detectability. Detection limits are determined by signal to noise. All other factors being equal, the PDA has the same "sensitivity" as a VWD, but it may have greater noise, hence the detection limit is higher for the PDA. 
The noise contributed by the physics of the photon transducer is usually not the dominant source of noise in a real analysis. For the reasons cited, the PDA does have greater transducer noise than the VWD. However, the design of the flow cell, thermal gradients, refractive index effects, lamp quality, bandwidth settings, use (or non-use) of background correction, stray light, pump noise, and sample matrix interferences all contribute noise. Some of these you can influence, some are designed in. Of course, there are well-designed PDAs and poorly-designed VWDs, so no general rule can be made.
Also, we never really see the "sensitivity" of the photon transducer. Internal signal processing converts the signal from the photodiode to absorbance units and that is the value reported to the data system. We don't see the raw photocurrent data. In this context, the "sensitivity" of the PDA is lower than the VWD, but its hidden.
In practice, the instrumental detection limit using a PDA is roughly twice the IDL of a VWD under best-case conditions. 
In the original question, there was a small discrepancy in the calibrated results. There could be many reasons for that, but blaming PDAs in general is not warranted.