How does Data Reprocessing work?
Data Reprocessing is a paid service that Piano Analytics provides to rewrite and potentially alter historical data. During a Data Reprocessing, all events for the organization within the requested period are reprocessed (even if the change only concerns one or more sites) by Piano teams using the current Data Management (DM) rules and Settings configuration.
Because the process replays data using today’s configuration, it is commonly used to fix incorrect tagging, populate missing values (when derivable from existing event context), or apply updated privacy exclusions.
What Data Reprocessing can be used for
Correct a value sent in a tag parameter
Example: you sent user_id + category concatenated in a single property and want to keep only the user_id. A DM rule can be created to rewrite the parameter during reprocessing.
Change Privacy Exclusions
Example: you currently exclude all hits that are not OPTIN. By updating the DM rule, they can exclude all events that are not OPTIN or EXEMPT, and then reprocess the historical period to apply the new rule.
Fill or correct missing/incorrect values when the information exists in the hit
Data reprocessing is often requested when data is missing or incorrect due to tagging issues. In practice, this requires defining DM processing rules that specify when and how to rewrite fields (for example: “if a field is empty and condition X is true, set it to value Y”).
Exclusions
Example: despite our detection service, a bot has spammed your site, and you managed to identify this traffic by setting an exclusion rule based on diverse criteria. Now you would like to remove permanently past data generated by this bot, data reprocessing can help you.
Key constraints and limitations
Reprocessing applies to a full day period minimum
Reprocessing is applied to a full day period. It is not possible to reprocess data only for a couple of hours.
Site number cannot be changed
Data reprocessing cannot change the site number. As a result, Piano cannot correct historical tagging where the wrong site ID was sent.
Scope of data eligible for reprocessing
Currently, data reprocessing is only meant to work on events collected and stored by Piano teams. If you encountered a technical issue that prevented the tag from sending events, we won’t be able to reprocess them as they haven’t been received.
Preparing a request (required information)
To help Piano assess and execute a reprocessing request efficiently, provide as much information as possible such as:
The property key impacted (if known)
The exact date/time period to be reprocessed
The sites involved (even though the entire organization’s events in the period are reprocessed, this clarifies the business scope)
A clear description of the issue and expected outcome
Confirmation that the needed DM rules / processing rules and any relevant settings changes are already defined (or included in the request)
The support team will then study and verify the technical feasibility of your request, and if approved, will provide you with a quote for this service.