Left ventricular diastolic dysfunction exists in patients with heart failure with reduced ejection fraction and causes activity restriction and a poor prognosis, but there have been few reports about exercise tolerance in patients with diastolic dysfunction, regardless of left ventricular ejection fraction (LVEF). In this study, 294 cardiovascular disease patients who performed a cardiopulmonary exercise test (CPX) with an adequate examination by echocardiography at Fukuoka University Hospital from 2011 to 2020 were investigated. Patients were divided into groups with grade I and grade II or III diastolic dysfunction according to diagnostic criteria, regardless of LVEF, by echocardiography. After adjusting for age, gender, body mass index, smoking, and LVEF by propensity score matching, we compared the results of CPX between the grade I and grade II/III groups. There were no significant differences in hemodynamic parameters, or in the respiratory exchange ratio, oxygen uptake per body weight, oxygen uptake per heart rate, or parameters of ventilatory volume. Ventilatory equivalents per oxygen uptake and per carbon dioxide output were significantly worse in the grade II/III group from the rest to peak periods during CPX. In conclusion, left ventricular diastolic dysfunction worsens ventilatory efficacy during CPX. This effect potentially contributes to a poor prognosis in left ventricular diastolic dysfunction.