Many of the data systems at the local level are designed to support the administration of the day-to-day activities of the schools. While some of that data travels upstream to the state and federal government, for the most part, school staff are focused on what data can tell them about their day-to-day operations and about the instruction of children.
Part of the challenge for state and federal education agencies is determining how to ensure that the data collected locally meets data quality standards required for state and federal reporting. Data at this level is used to drive education policies and initiatives, to determine funding for various educational programs and to ensure compliance to state and federal legislation. It’s a tall order for data designed for administrative and operational support.
Improve local data quality by considering these focus areas.
The make-up of school and local districts varies widely from state to state. Some are organized by county, others by township. Some have large centralized staff and lots of resources, while in others, the principal may also fill the role of janitor and bus driver. While no single solution will address the needs of these different local environments, there are some common strategies that can be applied across many different settings.
1. Connect data to its uses.
Data quality and management are not the job of any one person. Everyone who touches or uses the data has a role to play. Creating an awareness of data and the impacts of poor data quality is easier when the data is connected to its use. If the teacher or office clerk who is entering data into the system understands how that data is being used and how it impacts individual students, it naturally encourages attention to quality.
To prompt this awareness, make sure those entering the data understand how the data will be used by expanding the training on data collection to cover data uses. Provide context on why the data is collected and what decisions are based on it. Include local data uses in the discussion as well as how local data is aggregated at the state and federal level to assess policies, programs or funding that eventually funnels back to the school or district.
Data meetings or data retreats can help local staff explore how to use their own data to inform decisions at the local level. Demonstrate how data may be used by teachers to inform instruction, to assess the impact of specific programs and to ensure students are receiving the services they need. This will make the data tangible for end users. Furthermore, identifying the impacts at the school or district level provides a greater incentive to make sure the data is of high quality. The stronger the feedback loop between data entry and data use, the easier it is to identify potential data issues and data quality improvements.
2. Prevent errors before they occur.
Make sure those entering the data understand what data is being collected. Provide clear data definitions and guidance. Clear definitions can help determine if a given data element is appropriate for a specific use. For example, if a date is collected, what does that date represent? How is that date different from other dates that are collected? Providing guidance along with the data definitions can help ensure consistency in how the data entered into the data system.
Don’t assume the definition is clear; document it in a data dictionary. If data definitions don’t currently exist, consider using definitions from the Common Education Data Standards (CEDS). CEDS data definitions were created by a community of education stakeholders in order to provide a common vocabulary for education data.
Identify common scenarios that could cause confusion and provide guidance on which values should be used in different circumstances. To solidify data definitions, add business rule validations or data system edit checks at the point of data entry. Reduce inconsistencies and typographical errors by limiting data entry to a defined set of options. Edit checks can determine whether numeric values are within an acceptable range.
In cases where there are dependencies or relationships between data elements, incorporate business rules to ensure integrity and consistency by preventing combinations of data that are not valid. Many systems generate both errors that prevent entry and warnings that require a second look to ensure the data is correct. Make sure error and warning messages are descriptive, clearly identifying why the error or warning occurred and how to address it.
Creating an immediate feedback loop can prevent invalid or inaccurate data from ever being entered reducing the time and energy needed to correct issues further down the line. Immediate feedback can also provide an opportunity for those entering the data to learn how to avoid similar errors in the future.
3. Build data quality review into the process.
Create ways for schools and districts to review the data they submit to the state or federal government and before the data are finalized. Including data quality reports within the data collection systems allows those entering or reviewing the data to detect data quality issues that may not be detected by business rule validations. Facilitate reports that look at the same data through a variety of different lenses to display common aggregations and depict interactions between related data elements. For example, show comparisons to prior year data to help identify trends or to depict large fluctuations that could be the result of errors in the data.
Consider requiring a sign-off on key reports to ensure that school or district staff have reviewed the data. For this review, provide training about how to detect data quality issues such as:
- Reviewing records for incomplete or missing data
- Comparing aggregate totals to expected counts to determine if data may be missing
- Looking for inconsistencies in the data, especially between related data elements
As an added bonus, data quality reports are a great way for data entry staff to illustrate how the data may be used in the future and are another way of making the purpose of the data more tangible.
4. Develop a data quality feedback loop.
Just as education data is cyclical, data quality improvements are also cyclical. Every reporting cycle is a chance to make things better. At the end of each collection take time to reflect and to identify areas for improvement. Try asking questions of the data quality process.
- What worked well?
- Where were there challenges?
- What surprises were there?
- Were there common data quality issues across schools and districts?
- Were the data definitions clear?
- Was additional guidance needed?
- Were any new potential business rule validations identified?
Taking the time to reflect and answer questions like these can help create an action plan for implementing plans for identified improvements.
Next steps for your data quality approach
These strategies can be applied across many environments and can help build a structure to support data quality for the long term. For more information, try the CEDS guide to building a data dictionary and CEDS resources on data governance. If you have further questions on improving your data quality approach, feel free to contact us.
