Data Platform/Data Lake/Data Issues

We recommend the following approaches for excluding or annotating data that contains known data quality issues:

• Use date filters to exclude data from analysis for the affected time period
• For time series visualizations:
  • Visually block out the period of the data loss and add annotation with the problem summary and from and to dates. For example:

    Wikistats pageviews time series graph with data loss period visually blocked

  • Use overlays to annotate the data. For users of Superset an annotation layer can be created and reused. For example, for the /2021-06-04 Traffic Data Loss, an annotation layer is available called "Pageview Data Loss June 2021-January 2022":

    Time series graph of pageviews from Superset, with data loss period annotated using Superset annotation layers.

  • For point in time issues, use a data point annotation.
• When it is not feasible to remove data from an existing report or dashboard, add an annotation or footnote describing the impact of the data issue.