Hypothesis-driven academic research identifies interventions with likely disease-specific effects. Yet, many candidate drugs fail upon further development, emphasizing the need for acquisition of more robust preclinical data. We demonstrate that planning and executing multicentre confirmatory preclinical studies in an academic setting by applying the quality standards of early phase clinical trials is feasible. Randomization, blinding, stratification by sex of and quality control measures were carried out successfully. The primary objective of our specific study – to confirm synergistic effects of BET bromodomain protein 4 (BRD4) and cyclin-dependent kinase 9 (CDK9) inhibitors against PAX3::FOXO1 (P3F)-positive rhabdomyosarcoma (RMS) – was not met. Post-hoc analyses support that single-agent BRD4 inhibition by JQ1 effectively reduced the growth and viability of P3F+ RMS cells ex vivo with adequate on-target activity as evidenced by reduced expression of P3F, MYCN, MYOG, and MYOD. The antiproliferative effects of JQ1 and vincristine were comparable, and there was trend towards reduced and delayed xenograft growth in JQ1-treated mice. Yet, in vivo assays were flawed by lower xenograft penetrance, variable xenograft latency, gastrointestinal toxicity, and inadequate on-target activity of drugs. We conclude that confirmatory preclinical trials allow for robust assessment of the efficacy of candidate interventions and reduce bias in academic research. The study platform established here provides a framework that may be of particular benefit for the development of new drugs for rare cancers.