Every data-driven company should have a process of checking, cleaning, and ensuring the accuracy, consistency, and relevance of the data collected before these can be used for analysis, reporting, and decision-making. And this is precisely what data validation consists of. However, while it is an essential process to ensure the quality and the integrity of the data being gathered, it hasn’t been until the past few years that companies have come to realize that data quality is critical and that data analytics needs to be tested like any other software development process before it becomes a costly and unmanageable problem.
Indeed, Gartner estimates that “the average financial impact of poor data quality on organizations is $9.7 million per year,” and, only in the US alone, IBM has discovered that businesses lose $3.1 trillion annually due to poor data quality.
That is why, to help you solve data issues before they escalate, the following article aims to shed light on data validation techniques in your digital analytics to help you leverage your data collection efforts by adopting proactive and preventive measures to preserve your data quality.
Table of Contents
Data Validation: The Foundations of Data Quality
To identify and correct errors, inconsistencies, and inaccuracies in the data your apps and websites are sending to third-party integrations (e.g.: Google Analytics, Segment, Mixpanel, etc.) and ensure all this data enters the system correctly with the desired quality standards, it is essential that proper data checks are implemented. This is where data validation techniques come into play.
Cleaning data and validating it after it is already in the system costs more money and time. Moreover, not only does it negatively affect the revenue of the company, but it also becomes a disruption to the regular workflow, as it forces people to waste time on unruly and inaccurate data before it can be used for strategic decision-making.
That is why data validation techniques serve as a proactive mechanism designed to intercept and rectify issues before errors can easily slip through the cracks, compromising the integrity of your results.
Data Validation & Trackingplan
With Trackingplan, you can be automatically notified whenever your properties, events, and user acquisition data don’t conform to the values you have specified.
Data Type Validation
The Data Type Validation check is useful to ensure the data entered into the system has the correct data type.
Trackingplan automatically interprets which data type is being tracked, but you can make as many manual adjustments as necessary. You can choose between boolean, number, string, array, object, regex, enum, or keep it unconstrained with any. If Trackingplan detects that the events sent do not conform to these specifications, a warning will be generated.
Enum Validation
Enums allow you to specify in advance the values you expect for a property to be notified every time it receives a value that does not conform to your list of predefined constants you had already specified.
Moreover, Trackingplan allows you to autocomplete your Enums with the values that have been observed in the last 30 days in that specific property. Doing this will automatically show you all the values detected for this property, but you can also check that all the values are correct and, in case they are not, delete them.
Trackingplan will be checking that all the values in this property match the specified Enum. If it detects a value that is not within this list of predefined constants, Trackingplan will automatically send you a warning.
Moreover, to ensure seamless recognition by the server, Trackingplan also offers the option to ignore cases and diacritics to prevent potential issues where the server might identify them as implementation problems.
RegEx (Regular expressions) Validation
RegEx Validation is useful for implementing real-world data input validation, as it allows you to specify regular expression patterns such as phone numbers or email addresses. Trackingplan will automatically warn you whenever a property does not match with the regular expression pattern to which all the values seen for a property must conform.
In general, this field property is used to perform validation checks (format, length, etc.) on the value that the user enters in a field. If the user enters a value that does not pass these checks, it will throw an error.
The process of setting up Regex for a property is very similar to the one used for Enums. These are completely customizable to your needs and even allow you to insert test values to confirm that your regular expression patterns will work as you expect.
Moreover, for Regexes you can also ignore cases, diacritics, or both to prevent in advance any situation where the server would recognize it as an implementation issue.
From this point on, Trackingplan will be checking that all the values for these properties match the specified regular expression, and if the value does not conform with the pattern you have specified, our dashboard will automatically show you a warning.
Function Validations
Trackingplan also allows you to set up any complex validation at track level and will warn you every time any of your properties don’t conform to the function provided to validate an event. For example, you could add cross-condition validation to ensure that all products logged in a cart carry a valid product_sku given the page section).
Just let our support team know which validation rules you'd like to add, and consider it done!
Campaign Validations
Moreover, Trackingplan also allows you to set up User Acquisition specification warnings to validate your campaigns based on your own requirements.
With it, Trackingplan will automatically inform you any time the attribution data collected in your campaigns doesn’t meet the custom rules you have defined so you can get it fixed before compromising the performance of your marketing investments.
Let’s see some examples:
- If the campaign contains the keyword “black Friday”, the medium attribute cannot be ‘press’. If this combination is detected, please send me a warning.
But not only campaigns are supported. Indeed, you can also set up validations for any of your landings, referrals, pages, mediums, sources, and event attributions. Some examples can be:
- All the landing pages should follow this specific format "/section/abc.
- The event AddToCart should only originate on these 3 pages: a, b, and c. If this event is detected on a page that is not listed above, send me a warning.
Just ask support@trackingplan.com which validation rules you would like to add and we will implement them for you.
What to do with Validation errors?
Trackingplan automatically alerts you about any error happening in your digital analytics, spotting issues for you so you can focus on what you do best.
However, at Trackingplan we know sometimes, after receiving an alert, it can be a bit hard to know what to do next, especially when this has to do with a property or event with validation errors that do not conform to the values you have specified.
Hence, if you usually struggle to find the root cause of those warnings related to properties missing or not conforming to the validation rules or constraints specified, our Warning Debug feature is for you. You can see it in action in this video.
Conclusion
With Trackingplan, validation errors are not just detected, but also categorized and presented in a way that simplifies troubleshooting. Our Warning Debug feature, for instance, provides an in-depth analysis of your warnings related to properties missing or not conforming to your validation rules. By identifying the root cause of these warnings, users can quickly address the issues and ensure data integrity.
Nonetheless, Trackingplan not only takes a step further by providing an early warning system, indicating data inconsistencies before they escalate to larger issues, but it also allows users to take preventive measures to avoid problems before these even reach production, affecting, therefore, the quality of their digital analytics by offering a fully automated data QA and observability solution.
Get started today to experience the benefits of Trackingplan first-hand and automatically validate your Digital Analytics health.
For more information, you can always contact our team.
%20copy%202.webp)