REU Supplement to EAGER Proposal for Research in Measurement and Modeling: Dynamic STEM Assessment Through Epistemic Network Analysis

In the Dynamic STEM Assessment through Epistemic Network Analysis project, we have been laying the ground work for developing a potentially transformational approach to STEM assessment in the 21st century.

For previous generations, knowledge of basic STEM facts and skills was enough to find good jobs in an industrial economy or, for a select few, to continue to further academic and professional study in math, science, and engineering. Today, however, work that requires only basic skills flows overseas where labor is cheaper, and complex and meaningful STEM thinking means linking skills and knowledge in the context of real world problems and situations.

Problem solving in real STEM practices is characterized by knowledge and skills, to be sure, but also by the way those skills are connected to each other, and to the values and ways of making decisions in STEM fields.

We have been developing a new method of STEM assessment—called epistemic network analysis (ENA)—that focuses not on whether students master specific scientific facts, math skills, or engineering concepts, but on whether and how students link the skills, knowledge, identity, values, and epistemology of a STEM practice into a coherent way of thinking about complex STEM problems.

Central to this work is the problem of quantifying the relative positions of different networks of skills, knowledge, identity, values and epistemologies in a common network space, thus making it possible to use inferential statistics to detect meaningful groups of learners, or the trajectories novices follow in becoming experts. We have developed a technique for doing this in the first phase of the project.

We are working on developing a new network graph representation, the Equiload projection, which links the structure of network’s adjacency matrix to its projection into a common network space, thus making the position of networks in the common network space interpretable. This will make it possible to derive normative information from the trajectories of learning identified in the common network space—that is, to provide feedback about students and course designs to STEM teachers.

This grant is entering in its final year and supplemental funding increases the participation of undergraduate STEM students in our research and development activities in this final year of the project.