Alzheimer’s Disease (AD) is a neurodegenerative pathology characterized by progressive atrophy of brain and impairment of memory and cognitive functions. Physicians frequently use structural brain imaging to identify abnormal patterns in brain structure that may indicate probable AD. Thus, shape information is central for brain imaging analysis and AD diagnosis. This paper examines how three variants of Pyramid Histograms Of visual Words (PHOW) descriptions, a data-driven approach, handle the complex task of AD classification. 87 pathological cases and 87 controls from OASIS dataset were used to study the impact of shape and surface information. Best performance was 89.3%, a current mark for AD classification, and an increase (27.1%) in contrast to a naive approach. Additionally, controls were better classified than demented subjects (94.5% and 84.0%, respectively), while young, early-onset AD subjects, and elderly controls were the most difficult. Finally, dictionary word analysis revealed discriminative surface features. Also, local patterns induced by global word distribution appear to be more significant for classification than word location.