This review highlights the importance of using valid assessments in competency-based medical education (CBME), particularly within urology training. Traditional methods of determining validity, based on outdated theories, have been replaced by a contemporary framework that emphasizes construct validity through five key evidence sources: test content, response process, internal structure, relationship to other variables, and consequences. These components ensure that assessments accurately reflect a trainee's competencies and their implications.

By adopting Messick’s framework, program directors and educators can better evaluate and defend the validity of assessments in both clinical and simulation settings. Understanding the contextual specificity of validity is crucial for making informed, high-stakes decisions regarding trainee competency, ensuring that assessment outcomes are reliable and justifiable.