The "standard" recovery is probably 80 - 120% because that represents a historical compromise between the cumulative errors of a spiking experiment involving the nausea of extraction from complex media, and the more precise analytical procedure of a pure substance.
80 - 120% usually means that most of the target compound is recovered from the target matrix, and assures that there would be miminal danger of the API dose greatly exceeding the nomimal amount in the final product. Historically, 80 - 120% were also common limits for complex assay tehniques, such as potency testing of antibiotics.
The same applies for the wider acceptance limits for a chemical assay of a formulated product compared to the pure substance. The errors are a mix of the efficiency of the extractions, effects of the media on the compound ( degradation/binding - more common in biological monitoring ), and the effect of media on analysis.
However the acceptance limits are constrained by the need to ensure that close to the nomimal amount of API was ( biological monitoring ) or is ( pharmaceutical manufacturing ) present. Recoveries of less than 80%, or more than 120%, raise serious concerns about the quality of the data and the product.
It's not uncommon for sensitive molecules with very low therapeutic doses to have low spike recoveries, in which case the use of a closely related molecule, or a deuterated version of the molecule, is used as a standard to determine expected recovery from the matrix. Such a procedure involves a lot more validation to confirm that the recovery factors are valid for all expected situations.
For general APIs, if the spiking method can't show 80-120% recovery, you need to identify the cause, and review the reasons. You may need to modify or method, but if you want to pursue methods using low recoveries, you would have to seriously review and consider the conseqences on the monitoring experiment data and/or final product.
Bruce Hamilton