The authors discuss the challenges of machine- and deep learning–based automatic analysis of obstructive sleep apnea with respect to known issues with the signal interpretation, patient physiology, and the apnea-hypopnea index. Their goal is to provide guidance for sleep and machine learning professionals working in this area of sleep medicine. They suggest that machine learning approaches may well be better targeted at examining and attempting to improve the diagnostic criteria, in order to build a more nuanced understanding of the detailed circumstances surrounding OSA, rather than merely attempting to reproduce human scoring.