We present the LUng CAncer Screening (LUCAS) Dataset for evaluating lung cancer diagnosis with both imaging and clinical biomarkers in a realistic screening setting. We extract key information from anonymized clinical records and radiology reports, and we use it as a natural complement to low-dose chest CT scans of patients. We formulate the task as a detection problem and we develop a deep learning baseline to serve as a future reference of algorithmic performance. Our results provide solid empirical evidence for the difficulty of the task in the LUCAS Dataset and for the interest of including multimodal biomarkers in the analysis. All the resources of the LUCAS Dataset are publicly available.