The association between lifestyle and cardiac structure and function measures, such as global longitudinal strain and diastolic function in a healthy midlife general population, is not well known. A subpopulation of the Northern Finland Birth Cohort 1966 took part in follow-up, including echocardiography (n = 1,155) at the age of 46. All antihypertensive medication users (n = 164), patients with diabetes (n = 70), subjects with any cardiac diseases (n = 24), and subjects with echocardiography abnormalities (n = 21) were excluded. Moderate to vigorous physical activity (MVPA) was recorded with a wrist-worn accelerometer over 14 days and categorized into high, moderate, and low MVPA groups. Similarly, alcohol consumption was categorized as low, moderate, and high-dose users of alcohol and smoking as nonsmokers, former, and current smokers. The total number of healthy subjects included in the study was 715 (44% males). Left ventricular mass index and left atrial end-systolic volume index were significantly higher in the high MVPA group compared with the low MVPA group (adjusted main effect p = 0.002 and p <0.001, respectively). Cardiac function did not differ among the physical activity groups. High alcohol consumption was associated with impaired global longitudinal strain and diastolic function (adjusted main effect p = 0.002 and p = 0.004, respectively) but not with any cardiac structure variables. Smoking was not associated with cardiac structure or function. In healthy middle-aged adults, MVPA was independently associated with structural changes in the heart but not with cardiac function. High alcohol consumption was associated with impaired modern cardiac function measures but not with cardiac structure.