WOLFRAM

Enable JavaScript to interact with content and submit forms on Wolfram websites. Learn how
Wolfram Language & System Documentation Center

TransformAnomalies [data,tspec]

transforms anomalies in data using the transformation tspec.

TransformAnomalies [data,dspectspec]

detects the anomalies using the specification dspec.

TransformAnomalies [tab,{col1spec1,…}]

transforms tabular data tab using the specification speci for column coli.

Details
Details and Options Details and Options
Examples  
Basic Examples  
Scope  
Data Types  
Detection Methods  
Transformation Methods  
Applications  
See Also
Related Guides
History
Cite this Page

TransformAnomalies [data,tspec]

transforms anomalies in data using the transformation tspec.

TransformAnomalies [data,dspectspec]

detects the anomalies using the specification dspec.

TransformAnomalies [tab,{col1spec1,…}]

transforms tabular data tab using the specification speci for column coli.

Details

  • TransformAnomalies is also known as anomaly detection or outlier treatment.
  • TransformAnomalies helps maintain data quality by effectively identifying and managing anomalies, ensuring reliable analysis.
  • Possible forms of data include:
  • {el1,el2,…} a list of individual elements
    tab tabular data
  • Possible forms of tabular data tab include:
  • Tabular […] a tabular object
    Dataset […] a dataset object
    TimeSeries […] a time series object
    EventSeries […] an event series object
    {assoc1,assoc2,…} a list of associations
    matrix a matrix
  • Possible transformations tspec for managing the anomalies include:
  • "Clip" limit extreme values to non-anomalies range
    "Replace" replace anomalies with a calculated value
    {"Replace",method} replace anomalies using method
    "Trim" drop the anomalous elements
    "Flag" replace anomalies with Missing ["Anomalous"]
    tfun use the function tfun to transform the anomalies
  • Possible "Replace" methods are the same as in TransformMissing :
  • " Mean" the mean value
    "Median" the median value
    "Mode" the most frequent value
    {"Constant", val} a constant value val
    "RandomElement" randomly selected value
    {"RandomVariate",dist} value sampled from dist
    "NearestElement" use the closest non-missing element
    "NextElement" use the next non-missing element
    "PreviousElement" use the previous non-missing element
    "Interpolation" use interpolation
    {"Interpolation",InterpolationOrder degree} specify the interpolation order degree
  • Excluding the anomalies minimizes their influence on analysis. Clipping or replacing minimizes the impact of anomalies while preserving the dataset's structure. Flagging allows keeping all data points for further analysis while highlighting anomalies for review.
  • Available methods dspec to detect the anomalies include:
  • "IQR" interquartile range thresholds
    "ZScore" standard deviations from the mean
    "ModifiedZScore" robust version using median and MAD
    {spec,t} specifies the threshold t for anomaly detection
    {spec,{t1,t2}} specifies the lower and upper thresholds
  • Additionally, the following specifications are supported:
  • Scaled [f] a fraction f of the smallest and largest elements
    Scaled [{f1,f2}] separate thresholds for smallest and largest elements
    dfun a custom function fun
  • A custom anomaly detector dfun is applied to each element individually and must return a Boolean value.
  • When specifying the desired columns, coli can have the following format:
  • n the n^(th) column
    "name" the column "name"
    TypeSpecifier […] columns matching the specified type
    All all the columns
    Automatic automatically find columns based on spec (default)

Examples

open all close all

Basic Examples  (2)

Handle outliers by removing large and small values:

Wolfram Language code: TransformAnomalies[{-10, 1, 2, 3, 10}, "Trim"]

Clip the data above the 95^(th) percentile:

Wolfram Language code: SeedRandom[1]; data = RandomVariate[GammaDistribution[1, 10], 100];
Wolfram Language code: Histogram[{data, TransformAnomalies[data, Scaled[{0, .05}] -> "Clip"]}, ChartLayout -> "Row"]

Scope  (16)

Data Types  (5)

Detect anomalies in a list:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, "Flag"]

Detect anomalies in a TabularColumn :

Wolfram Language code: tcol = TabularColumn[Association["Data" -> {{-2, 1, 2, 3, 10}, {}, None}, "ElementType" -> "Integer64"]];
Wolfram Language code: tcol//Normal
Wolfram Language code: TransformAnomalies[TabularColumn[Association["Data" -> {{-2, 1, 2, 3, 10}, {}, None}, "ElementType" -> "Integer64"]], "Flag"]
Wolfram Language code: Normal[%]

Detect anomalies along matrix columns:

Wolfram Language code: TransformAnomalies[{{1, 2}, {2, 3}, {4, 12}}, "Flag"]

Detect anomalies in a Tabular object:

Wolfram Language code: tab = Tabular[Association["RawSchema" -> Association["ColumnProperties" -> Association["Species" -> Association["ElementType" -> "String"], "SepalLength" -> Association["ElementType" -> TypeSpecifier["Quantity"]["Real64", "Centimeters"]], "SepalWidth" -> Association["ElementType" -> TypeSpecifier["Quantity"][ "Real64", "Centimeters"]], "PetalLength" -> Association["ElementType" -> TypeSpecifier["Quantity"]["Real64", "Centimeters"]], "PetalWidth" -> Association["ElementType" -> TypeSpecifier["Quantity"]["Real64", "Centimeters"]]], "KeyColumns" -> None, "Backend" -> "WolframKernel"], "Options" -> {}, "BackendData" -> Association["ColumnData" -> DataStructure["ColumnTable", {{TabularColumn[Association["Data" -> {{3, {0, 6, 12, 18, 24, 30, 36}, "setosasetosasetosasetosasetosasetosa"}, {}, None}, "ElementType" -> "String"]], TabularColumn[Association["Data" -> {6, {{{4.7, 4.6, 5.1, 5.7, 5., 4.9}, {}, None}}, None}, "ElementType" -> TypeSpecifier["Quantity"]["Real64", "Centimeters"], "CachedOriginalExpression" -> {Quantity[4.7, "Centimeters"], Quantity[4.6, "Centimeters"], Quantity[5.1, "Centimeters"], Quantity[5.7, "Centimeters"], Quantity[5., "Centimeters"], Quantity[4.9, "Centimeters"]}]], TabularColumn[Association["Data" -> {6, {{{3.2, 3.1, 3.5, 4.4, 3.6, 3.}, {}, None}}, None}, "ElementType" -> TypeSpecifier["Quantity"]["Real64", "Centimeters"], "CachedOriginalExpression" -> {Quantity[3.2, "Centimeters"], Quantity[3.1, "Centimeters"], Quantity[3.5, "Centimeters"], Quantity[4.4, "Centimeters"], Quantity[3.6, "Centimeters"], Quantity[3., "Centimeters"]}]], TabularColumn[Association["Data" -> {6, {{{1.3, 1.5, 1.4, 1.5, 1.4, 1.4}, {}, None}}, None}, "ElementType" -> TypeSpecifier["Quantity"]["Real64", "Centimeters"], "CachedOriginalExpression" -> {Quantity[1.3, "Centimeters"], Quantity[1.5, "Centimeters"], Quantity[1.4, "Centimeters"], Quantity[1.5, "Centimeters"], Quantity[1.4, "Centimeters"], Quantity[1.4, "Centimeters"]}]], TabularColumn[Association["Data" -> {6, {{{0.2, 0.2, 0.2, 0.4, 0.2, 0.2}, {}, None}}, None}, "ElementType" -> TypeSpecifier["Quantity"]["Real64", "Centimeters"], "CachedOriginalExpression" -> {Quantity[0.2, "Centimeters"], Quantity[0.2, "Centimeters"], Quantity[0.2, "Centimeters"], Quantity[0.4, "Centimeters"], Quantity[0.2, "Centimeters"], Quantity[0.2, "Centimeters"]}]]}}]]]];
Wolfram Language code: TransformAnomalies[tab, "Flag"]

Detect anomalies in a list of associations across the same key:

Wolfram Language code: TransformAnomalies[{ <|"A" -> Quantity[0.2, "Centimeters"], "B" -> Quantity[1.3, "Centimeters"]|>, <|"A" -> Quantity[0.2, "Centimeters"], "B" -> Quantity[1.5, "Centimeters"]|>, <|"A" -> Quantity[0.2, "Centimeters"], "B" -> Quantity[1.4, "Centimeters"]|>, <|"A" -> Quantity[0.4, "Centimeters"], "B" -> Quantity[1.5, "Centimeters"]|>, <|"A" -> Quantity[0.2, "Centimeters"], "B" -> Quantity[1.4, "Centimeters"]|>, <|"A" -> Quantity[0.2, "Centimeters"], "B" -> Quantity[1.4, "Centimeters"]|>}, "Flag"]

Detection Methods  (6)

Detect anomalies using the interquartile range:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, "IQR"]

Detect anomalies using the z-score:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, "ZScore"]

Detect anomalies using the modified z-score:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, "ModifiedZScore"]

Specify a custom threshold:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, {"ZScore", 1}]

Specify upper and lower thresholds individually:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, {"ZScore", {2, 1}}]

Flag a fraction of the smallest and largest elements:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, Scaled[.2]]

Specify separate smallest and largest fractions:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, Scaled[{.2, 0}]]

Use a custom function to detect anomalies:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, Negative -> "Flag"]

Transformation Methods  (5)

Remove the anomalies:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, "Trim"]

Clip the anomalies with the closest valid value:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, "Clip"]

Replace the anomalies with missing elements:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, "Flag"]

Apply any TransformMissing specification:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, {"Replace", "Mean"}]

Apply a custom transformation:

Wolfram Language code: TransformAnomalies[{-2, 1, 2, 3, 10}, f]

Applications  (2)

Take a Tabular object of sales data:

Wolfram Language code: data = Tabular[Association["RawSchema" -> Association["ColumnProperties" -> Association["Date" -> Association["ElementType" -> TypeSpecifier["Date"]["Integer32", "Day", "Gregorian", 0]], "City" -> Association["ElementType" -> "String"], "Sales" -> Association["ElementType" -> "Integer64"]], "KeyColumns" -> None, "Backend" -> "WolframKernel"], "Options" -> {}, "BackendData" -> Association["ColumnData" -> DataStructure["ColumnTable", {{TabularColumn[Association["Data" -> {52, {{NumericArray[{16071, 16071, 16071, 16071, 16071, 16071, 16072, 16072, 16072, 16072, 16072, 16072, 16073, 16073, 16073, 16073, 16073, 16073, 16074, 16074, 16074, 16074, 16074, 16074, 16075, 16075, 16075, 16075, 16075, 16075, 16076, 16076, 16076, 16076, 16076, 16076, 16077, 16077, 16077, 16077, 16077, 16077, 16078, 16078, 16078, 16078, 16078, 16078, 16079, 16079, 16079, 16079}, "Integer32"], {}, None}}, None}, "ElementType" -> "Date"["Integer32", "Day", "Gregorian", 0]]], TabularColumn[Association["Data" -> {{3, {0, 6, 14, 19, 25, 33, 38, 44, 52, 57, 63, 71, 76, 82, 90, 95, 101, 109, 114, 120, 128, 133, 139, 147, 152, 158, 166, 171, 177, 185, 190, 196, 204, 209, 215, 223, 228, 234, 242, 247, 253, 261, 266, 272, 280, 285, 291, 299, 304, 310, 318, 323, 329}, "Bos\ tonNew YorkParisLondonShanghaiTokioBostonNew YorkParisLondonShanghaiTokioBostonNew \ YorkParisLondonShanghaiTokioBostonNew YorkParisLondonShanghaiTokioBostonNew \ YorkParisLondonShanghaiTokioBostonNew YorkParisLondonShanghaiTokioBostonNew \ YorkParisLondonShanghaiTokioBostonNew YorkParisLondonShanghaiTokioBostonNew YorkParisLondon"}, {}, None}, "ElementType" -> "String"]], TabularColumn[ Association["Data" -> {{198, 0, 215, 225, 241, 218, 189, 0, 211, 228, 242, 229, 196, 235, 221, 229, 238, 236, 194, 237, 213, 226, 231, 228, 193, 0, 219, 247, 242, 225, 184, 228, 214, 225, 244, 219, 190, 227, 218, 232, 235, 224, 196, 0, 214, 236, 229, 233, 189, 227, 211, 226}, {}, None}, "ElementType" -> "Integer64"]]}}]]]];

The "Sales" column contains outlier zero values, which are standing in for missing values:

Wolfram Language code: Histogram[data -> "Sales"]

Transform the zero anomalies by removing the relevant rows from the original data:

Wolfram Language code: data1 = TransformAnomalies[data, "Trim"]

Transform the zero anomalies by converting them to Missing objects:

Wolfram Language code: data2 = TransformAnomalies[data, "Flag"]

Check the new data:

Wolfram Language code: Histogram[# -> "Sales"]& /@ {data1, data2}

Take a Tabular object of weather observations:

Wolfram Language code: data = Tabular[Association["RawSchema" -> Association["ColumnProperties" -> Association["Time" -> Association["ElementType" -> TypeSpecifier["Date"]["Integer64", "Instant", "Gregorian", -6.]], "Temperature" -> Association["ElementType" -> TypeSpecifier["Quantity"]["NumberExpression", "DegreesCelsius"]], "Pressure" -> Association["ElementType" -> TypeSpecifier["Quantity"]["NumberExpression", "Millibars"]]], "KeyColumns" -> None, "Backend" -> "WolframKernel"], "Options" -> {}, "BackendData" -> Association["ColumnData" -> DataStructure["ColumnTable", {{TabularColumn[Association["Data" -> {23, {{{1701427980000, 1701429480000, 1701431580000, 1701431940000, 1701433080000, 1701435180000, 1701436260000, 1701438780000, 1701442380000, 1701518340000, 1701520680000, 1701521580000, 1701524100000, 1701525180000, 1701604740000, 1701605340000, 1701607980000, 1701611100000, 1701611580000, 1701611700000, 1701612120000, 1701613020000, 1701615180000}, {}, None}}, None}, "ElementType" -> "Date"["Integer64", "Instant", "Gregorian", -6.], "CachedOriginalExpression" -> {DateObject[{2023, 12, 1, 4, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 5, 18, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 5, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 5, 59, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 6, 18, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 6, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 7, 11, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 7, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 1, 8, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 2, 5, 59, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 2, 6, 38, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 2, 6, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 2, 7, 35, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 2, 7, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 5, 59, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 6, 9, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 6, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 7, 45, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 7, 53, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 7, 55, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 8, 2, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 8, 17, 0}, "Instant", "Gregorian", -6.], DateObject[{2023, 12, 3, 8, 53, 0}, "Instant", "Gregorian", -6.]}]], TabularColumn[Association["Data" -> {23, {{{8.9, 8.3, 7.8, 45.68, 7.2, 7.2, 7.2, 7.2, 6.7, 43.52, 6.1, 6.1, 5.6, 5.6, 41.54, 5, 5, 5, 5, 5, 5, 5, 5}, {}, None}}, None}, "ElementType" -> TypeSpecifier["Quantity"]["NumberExpression", "DegreesCelsius"], "CachedOriginalExpression" -> {Quantity[8.9, "DegreesCelsius"], Quantity[8.3, "DegreesCelsius"], Quantity[7.8, "DegreesCelsius"], Quantity[45.68, "DegreesCelsius"], Quantity[7.2, "DegreesCelsius"], Quantity[7.2, "DegreesCelsius"], Quantity[7.2, "DegreesCelsius"], Quantity[7.2, "DegreesCelsius"], Quantity[6.7, "DegreesCelsius"], Quantity[43.52, "DegreesCelsius"], Quantity[6.1, "DegreesCelsius"], Quantity[6.1, "DegreesCelsius"], Quantity[5.6, "DegreesCelsius"], Quantity[5.6, "DegreesCelsius"], Quantity[41.54, "DegreesCelsius"], Quantity[5, "DegreesCelsius"], Quantity[5, "DegreesCelsius"], Quantity[5, "DegreesCelsius"], Quantity[5, "DegreesCelsius"], Quantity[5, "DegreesCelsius"], Quantity[5, "DegreesCelsius"], Quantity[5, "DegreesCelsius"], Quantity[5, "DegreesCelsius"]}]], TabularColumn[Association["Data" -> {23, {{{1011.9, 1012.1, 1012.3, 99, 1012.2, 1012.1, 1012.2, 1012.2, 1011.3, 99, 1011.7, 1012., 99, 1013.4, 99, 1011.5, 1009.9, 1009.4, 1009.1, 1008.8, 1008.7, 1008.4, 1008.3}, {}, None}}, None}, "ElementType" -> TypeSpecifier["Quantity"]["NumberExpression", "Millibars"], "CachedOriginalExpression" -> {Quantity[1011.9, "Millibars"], Quantity[1012.1, "Millibars"], Quantity[1012.3, "Millibars"], Quantity[99, "Millibars"], Quantity[1012.2, "Millibars"], Quantity[1012.1, "Millibars"], Quantity[1012.2, "Millibars"], Quantity[1012.2, "Millibars"], Quantity[1011.3, "Millibars"], Quantity[99, "Millibars"], Quantity[1011.7, "Millibars"], Quantity[1012., "Millibars"], Quantity[99, "Millibars"], Quantity[1013.4, "Millibars"], Quantity[99, "Millibars"], Quantity[1011.5, "Millibars"], Quantity[1009.9, "Millibars"], Quantity[1009.4, "Millibars"], Quantity[1009.1, "Millibars"], Quantity[1008.8, "Millibars"], Quantity[1008.7, "Millibars"], Quantity[1008.4, "Millibars"], Quantity[1008.3, "Millibars"]}]]}}]]]];

Plot the temperature:

Wolfram Language code: DateListPlot[data -> {"Time", "Temperature"}, PlotRange -> All]

It appears the temperature data contains outliers, possibly resulting from entering the data in wrong units. Similarly, the "Pressure" column contains outliers, that is number 99, which is often used in place of missing values. Specify the temperature unit conversion function:

Wolfram Language code: tempF = (UnitConvert[Quantity[QuantityMagnitude[#], "DegreesFahrenheit"], "DegreesCelsius"]&);

Transform anomalies using the above function for the "Temperature" column and in the "Pressure" column replace the anomaly with the nearest valid value:

Wolfram Language code: new = TransformAnomalies[data, {"Temperature" -> tempF, "Pressure" -> {"Replace", "NearestElement"}}]

Plot the observations:

Wolfram Language code: GraphicsRow[DateListPlot[new -> {"Time", #}, FrameLabel -> Automatic, PlotLabel -> #]& /@ {"Temperature", "Pressure"}, ImageSize -> Medium]
Wolfram Research (2025), TransformAnomalies, Wolfram Language function, https://reference.wolfram.com/language/ref/TransformAnomalies.html.

Text

Wolfram Research (2025), TransformAnomalies, Wolfram Language function, https://reference.wolfram.com/language/ref/TransformAnomalies.html.

CMS

Wolfram Language. 2025. "TransformAnomalies." Wolfram Language & System Documentation Center. Wolfram Research. https://reference.wolfram.com/language/ref/TransformAnomalies.html.

APA

Wolfram Language. (2025). TransformAnomalies. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/TransformAnomalies.html

BibTeX

@misc{reference.wolfram_2026_transformanomalies, author="Wolfram Research", title="{TransformAnomalies}", year="2025", howpublished="\url{https://reference.wolfram.com/language/ref/TransformAnomalies.html}", note=[Accessed: 15-August-2026]}

BibLaTeX

@online{reference.wolfram_2026_transformanomalies, organization={Wolfram Research}, title={TransformAnomalies}, year={2025}, url={https://reference.wolfram.com/language/ref/TransformAnomalies.html}, note=[Accessed: 15-August-2026]}
Top [フレーム]

AltStyle によって変換されたページ (->オリジナル) /