You gather data from each school, but nothing seems to match up. Sure, you asked for the same information from everyone, but without standardized definitions for the information, they each interpreted your request differently. Now, you have your hands full with disparate data that requires more work for you to understand and that could compromise your research.
Our recent blog post, Better Education Insights Tomorrow Require Data Standards Today, touched on the importance of data standardization and what it means to standardize. In this blog post, we will elaborate on how to get standardization started.
When do you standardize?
Education is exciting because educators are constantly working to improve. Educators explore new learning strategies, classroom management styles, classroom designs, and school-wide policies as they strive to improve educational outcomes. Each year, high-performing teachers, researchers, and innovative schools earn grants to experiment with new strategies and examine their impact on learning. With all of this exciting work always underway at a national scale, deciding when to standardize data becomes a key challenge.
On occasion, need drives concepts towards standardization. For instance, the need to track and research a natural disaster’s implications on learning leads to standardization. Another example: requirements to report on a topic at the state or federal level can encourage the adoption of standards.
But this is not the case for most education concepts. Most concepts emerge organically and incrementally without the immediate need to standardize. A researcher may devise a new learning strategy and experiment with several classrooms. Simultaneously, another unrelated researcher designs something similar and experiments with a different set of classrooms.
After each researcher writes up their results, they conduct more experiments across schools and classrooms. As they refine the learning strategies further, the strategies evolve into more concrete concepts. Variations may still exist but the concepts have become established enough to be recognized across a growing group of stakeholders. This is when it is time to consider standardization.
Where do you go from here?
Standardization via an established standards body will increase your ability to share and analyze comparable data. To begin, you need to figure out your starting point.
There are three main starting points that will determine where to go and when to involve standards bodies. Leveraging our work on the Christensen Institute’s Canopy Project as an example, we can see how concepts from this field of study fit in each of these three potential starting points.
- Starting Point 1: The need for a definition exists, but definitional work has not yet happened. Typically, this is because the concept is still emerging. There is no real consensus yet towards a concrete concept and, thus, the data is being codified in inconsistent ways.
Example: People are interested in concepts like “learner agency” but common understanding and consistent codification do not yet exist. This concept will continue to develop and will likely acquire a consistent definition, leading to standardization.
- Starting Point 2: Definitional work has already happened, but the concept does not yet exist in data standards. If you are starting here, your concept has already started moving towards consensus. While it still lacks a formal data standard, stakeholders have begun to use the same definition for the concept. This is a great point at which to involve standards bodies and to bring stakeholders to the table to build consensus towards a standard definition.
Example: People are managing data about universal design for learning (UDL), but not necessarily in common ways. Many variations for UDL strategies exist but stakeholders are beginning to understand UDL as a broad concept.
- Starting Point 3: The concept already exists in data standards. Yes, this is still a starting point. Standards evolve and it is up to the stakeholders to maintain applicability of the standards. It is important if you find yourself here to examine the existing element closely. Does the standardized definition meet the use case you had in mind? Does the context fit? If you find any changes you would make, contact the appropriate standards body to inquire of their standards development process and how you can submit a use case.
Example: Several data standards include the concept of blended learning. During the analysis for the Canopy project, we found that the standard for blended learning was only represented at the classroom level and not at the schoolwide implementation level. The Christensen Institute was then able to submit a use case that requested the element be added to the school level for tracking broader implementation.
As you are figuring out where you are starting from, it is important to review existing accepted standards and determine if a concept that suits your needs already exists. We suggest starting with a landscape analysis.
How to conduct a landscape analysis:
1. Determine the concepts you want to consider for standardization. Capture all of the keywords that might relate to the concept. Because it is not yet standardized, consider delineations of the concept. For example, if you were searching for gender you might consider “male,” “female,” “gender,” and “sex” as keywords rather than focusing simply on “gender.”
2. Using the list of keywords, review existing standards. In the case of education data standards, review the Common Education Data Standards (CEDS). CEDS is a national initiative that has been codifying education data across the early learning to K12 to postsecondary to workforce continuum for the past 10 years.
Other education standards that leverage CEDS and support specific education use cases, include Access4Learning, Ed-Fi, IMS Global and the Postsecondary Electronic Standards Council. Review these standards to determine if the keywords appear and prevailing definitions already exist.
As you conduct this analysis, it is important to review the elements and concepts in the standard that are connected to or near the keywords you are examining. These additional elements will help to provide context.
3. Document your findings and conduct comparisons. Make notes for the elements that you find that qualifies them into one of these categories:
-
- Doesn’t exist at all. (Starting Points 1 or 2)
- Is close but would require some tweaks. (Starting Point 3)
- Matches your concept exactly as you would describe it. (Starting Point 3)
The analysis will help you determine where you will be starting from when standardizing. If the element doesn’t exist at all, you will want to work with stakeholders to determine if the concept is still emerging (Starting Point 1) or if the time is right to move forward with standardizing a definition (Starting Point 2). If the element exists but requires tweaks, you will want to connect with the appropriate standards bodies to get involved.
Regardless of your landscape analysis results, by analyzing existing elements and finding out where you are starting from, you are already on your way to promoting universal understanding of your concepts and improving usability of relevant data that can further evolve associated education strategies.
In our final blog for this three-part series, we will delve into engaging with standards bodies and what the standardizing process entails. In the meantime, check out our primer on standardization: Standardizing School Innovation Data. And if you have any questions on conducting your landscape analysis or navigating standards bodies, you can reach out to us any time.
