In December, 2011, Maryland was awarded a $50 million Race to the Top-Early Learning Challenge grant by U.S. Departments of Education and Health and Human Services.  As part of the grant, the state committed to developing a comprehensive early childhood assessment system, supported by statewide technology and professional development infrastructures.  The system is being designed for all children from birth through age six, and includes statewide implementation of screening and formative assessments, as well as a kindergarten entry assessment that is being developed through a unique collaboration with the state of Ohio. 

Maryland is using early childhood assessment theory, research, and best practice to create a system that will conform to the key benchmarks of quality in assessment, including validity, reliability, sensitivity, and cultural and linguistic appropriateness.  The system will also be field-tested and reviewed by a national Technical Advisory Council comprised of developmental psychologists, early childhood experts and psychometricians.

 Based on the Maryland experience, this paper will address a number of questions related to the utility of current early childhood assessment research in building a comprehensive early childhood system.  These questions include:  

•  How was the national research community including Johns Hopkins University, WestEd, and other experts employed in the development of the system?
• In what ways was the early childhood assessment literature most helpful?
• What role did the literature play in the design of the technological and professional development supports of the system?
• What key questions related to the development of the system were not answered by the literature?
• Were there other considerations—like simplicity and accessibility—that took priority over research considerations in the design?

 The paper will discuss these questions and conclude with a critique of the early childhood assessment literature from an implementation perspective.  The paper will also make suggestions for a research agenda that better supports the development of state comprehensive assessment systems.