Breast cancer is one of the common type of cancer in females across the world. An early detection and diagnosis of breast cancer may reduce the mortality rate to a great extent. To diagnose breast cancer, different types of imaging modalities are used to collect samples like mammography, Computerized Tomography, Magnetic Resonance Imaging, Ultrasound and Biopsy. Histopathological images obtained from biopsy may influence how and at which stage the cancer is being diagnosed. The Computer Assisted Diagnosis (CAD) system helps the pathologists in early diagnosis of breast cancer. In this survey, the recently reported techniques for breast cancer diagnosis using histopathological images have been summarized. This study could be beneficial for: (i) Clinicians to receive second opinion from the CAD system for early diagnosis, and (ii) Researchers to analyze and enhance the existing state-of-art techniques used in CAD system, which may further reduce the gap of variability between intra and inter observer.