Detectors
A Detector is just a collection of hyperparameters. Each detector implements a fit and transform method, where fit refers to learning a model from training data and transform refers to using a learned model to calculate outlier scores of new data. Detectors typically do not classify samples into inliers and outliers; that's a DeterministicDetector wrapper is used to convert the raw scores into binary labels.
Neighbor-based
ABODDetector
#
OutlierDetectionNeighbors.ABODDetector — Type.
ABODDetector(k = 5,
metric = Euclidean(),
algorithm = :kdtree,
static = :auto,
leafsize = 10,
reorder = true,
parallel = false,
enhanced = false)
Determine outliers based on the angles to its nearest neighbors. This implements the FastABOD variant described in the paper, that is, it uses the variance of angles to its nearest neighbors, not to the whole dataset, see [1].
Notice: The scores are inverted, to conform to our notion that higher scores describe higher outlierness.
Parameters
k::Integer
Number of neighbors (must be greater than 0).
metric::Metric
This is one of the Metric types defined in the Distances.jl package. It is possible to define your own metrics by creating new types that are subtypes of Metric.
algorithm::Symbol
One of (:kdtree, :balltree). In a kdtree, points are recursively split into groups using hyper-planes. Therefore a KDTree only works with axis aligned metrics which are: Euclidean, Chebyshev, Minkowski and Cityblock. A brutetree linearly searches all points in a brute force fashion and works with any Metric. A balltree recursively splits points into groups bounded by hyper-spheres and works with any Metric.
static::Union{Bool, Symbol}
One of (true, false, :auto). Whether the input data for fitting and transform should be statically or dynamically allocated. If true, the data is statically allocated. If false, the data is dynamically allocated. If :auto, the data is dynamically allocated if the product of all dimensions except the last is greater than 100.
leafsize::Int
Determines at what number of points to stop splitting the tree further. There is a trade-off between traversing the tree and having to evaluate the metric function for increasing number of points.
reorder::Bool
While building the tree this will put points close in distance close in memory since this helps with cache locality. In this case, a copy of the original data will be made so that the original data is left unmodified. This can have a significant impact on performance and is by default set to true.
parallel::Bool
Parallelize score and predict using all threads available. The number of threads can be set with the JULIA_NUM_THREADS environment variable. Note: fit is not parallel.
enhanced::Bool
When enhanced=true, it uses the enhanced ABOD (EABOD) adaptation proposed by [2].
Examples
using OutlierDetection: ABODDetector, fit, score
detector = ABODDetector()
X = rand(10, 100)
result = fit(detector, X)
test_scores = transform(detector, result.model, X)
References
[1] Kriegel, Hans-Peter; S hubert, Matthias; Zimek, Arthur (2008): Angle-based outlier detection in high-dimensional data.
[2] Li, Xiaojie; Lv, Jian Cheng; Cheng, Dongdong (2015): Angle-Based Outlier Detection Algorithm with More Stable Relationships.
COFDetector
#
OutlierDetectionNeighbors.COFDetector — Type.
COFDetector(k = 5,
metric = Euclidean(),
algorithm = :kdtree,
leafsize = 10,
reorder = true,
parallel = false)
Local outlier density based on chaining distance between graphs of neighbors, as described in [1].
Parameters
k::Integer
Number of neighbors (must be greater than 0).
metric::Metric
This is one of the Metric types defined in the Distances.jl package. It is possible to define your own metrics by creating new types that are subtypes of Metric.
algorithm::Symbol
One of (:kdtree, :balltree). In a kdtree, points are recursively split into groups using hyper-planes. Therefore a KDTree only works with axis aligned metrics which are: Euclidean, Chebyshev, Minkowski and Cityblock. A brutetree linearly searches all points in a brute force fashion and works with any Metric. A balltree recursively splits points into groups bounded by hyper-spheres and works with any Metric.
static::Union{Bool, Symbol}
One of (true, false, :auto). Whether the input data for fitting and transform should be statically or dynamically allocated. If true, the data is statically allocated. If false, the data is dynamically allocated. If :auto, the data is dynamically allocated if the product of all dimensions except the last is greater than 100.
leafsize::Int
Determines at what number of points to stop splitting the tree further. There is a trade-off between traversing the tree and having to evaluate the metric function for increasing number of points.
reorder::Bool
While building the tree this will put points close in distance close in memory since this helps with cache locality. In this case, a copy of the original data will be made so that the original data is left unmodified. This can have a significant impact on performance and is by default set to true.
parallel::Bool
Parallelize score and predict using all threads available. The number of threads can be set with the JULIA_NUM_THREADS environment variable. Note: fit is not parallel.
Examples
using OutlierDetection: COFDetector, fit, score
detector = COFDetector()
X = rand(10, 100)
result = fit(detector, X)
test_scores = transform(detector, result.model, X)
References
[1] Tang, Jian; Chen, Zhixiang; Fu, Ada Wai-Chee; Cheung, David Wai-Lok (2002): Enhancing Effectiveness of Outlier Detections for Low Density Patterns.
DNNDetector
#
OutlierDetectionNeighbors.DNNDetector — Type.
DNNDetector(d = 0,
metric = Euclidean(),
algorithm = :kdtree,
leafsize = 10,
reorder = true,
parallel = false)
Anomaly score based on the number of neighbors in a hypersphere of radius d. Knorr et al. [1] directly converted the resulting outlier scores to labels, thus this implementation does not fully reflect the approach from the paper.
Parameters
d::Real
The hypersphere radius used to calculate the global density of an instance.
metric::Metric
This is one of the Metric types defined in the Distances.jl package. It is possible to define your own metrics by creating new types that are subtypes of Metric.
algorithm::Symbol
One of (:kdtree, :balltree). In a kdtree, points are recursively split into groups using hyper-planes. Therefore a KDTree only works with axis aligned metrics which are: Euclidean, Chebyshev, Minkowski and Cityblock. A brutetree linearly searches all points in a brute force fashion and works with any Metric. A balltree recursively splits points into groups bounded by hyper-spheres and works with any Metric.
static::Union{Bool, Symbol}
One of (true, false, :auto). Whether the input data for fitting and transform should be statically or dynamically allocated. If true, the data is statically allocated. If false, the data is dynamically allocated. If :auto, the data is dynamically allocated if the product of all dimensions except the last is greater than 100.
leafsize::Int
Determines at what number of points to stop splitting the tree further. There is a trade-off between traversing the tree and having to evaluate the metric function for increasing number of points.
reorder::Bool
While building the tree this will put points close in distance close in memory since this helps with cache locality. In this case, a copy of the original data will be made so that the original data is left unmodified. This can have a significant impact on performance and is by default set to true.
parallel::Bool
Parallelize score and predict using all threads available. The number of threads can be set with the JULIA_NUM_THREADS environment variable. Note: fit is not parallel.
Examples
using OutlierDetection: DNNDetector, fit, score
detector = DNNDetector()
X = rand(10, 100)
result = fit(detector, X)
test_scores = transform(detector, result.model, X)
References
[1] Knorr, Edwin M.; Ng, Raymond T. (1998): Algorithms for Mining Distance-Based Outliers in Large Datasets.
KNNDetector
#
OutlierDetectionNeighbors.KNNDetector — Type.
KNNDetector(k=5,
metric=Euclidean,
algorithm=:kdtree,
leafsize=10,
reorder=true,
reduction=:maximum)
Calculate the anomaly score of an instance based on the distance to its k-nearest neighbors.
Parameters
k::Integer
Number of neighbors (must be greater than 0).
metric::Metric
This is one of the Metric types defined in the Distances.jl package. It is possible to define your own metrics by creating new types that are subtypes of Metric.
algorithm::Symbol
One of (:kdtree, :balltree). In a kdtree, points are recursively split into groups using hyper-planes. Therefore a KDTree only works with axis aligned metrics which are: Euclidean, Chebyshev, Minkowski and Cityblock. A brutetree linearly searches all points in a brute force fashion and works with any Metric. A balltree recursively splits points into groups bounded by hyper-spheres and works with any Metric.
static::Union{Bool, Symbol}
One of (true, false, :auto). Whether the input data for fitting and transform should be statically or dynamically allocated. If true, the data is statically allocated. If false, the data is dynamically allocated. If :auto, the data is dynamically allocated if the product of all dimensions except the last is greater than 100.
leafsize::Int
Determines at what number of points to stop splitting the tree further. There is a trade-off between traversing the tree and having to evaluate the metric function for increasing number of points.
reorder::Bool
While building the tree this will put points close in distance close in memory since this helps with cache locality. In this case, a copy of the original data will be made so that the original data is left unmodified. This can have a significant impact on performance and is by default set to true.
parallel::Bool
Parallelize score and predict using all threads available. The number of threads can be set with the JULIA_NUM_THREADS environment variable. Note: fit is not parallel.
reduction::Symbol
One of (:maximum, :median, :mean). (reduction=:maximum) was proposed by [1]. Angiulli et al. [2] proposed sum to reduce the distances, but mean has been implemented for numerical stability.
Examples
using OutlierDetection: KNNDetector, fit, score
detector = KNNDetector()
X = rand(10, 100)
result = fit(detector, X)
test_scores = transform(detector, result.model, X)
References
[1] Ramaswamy, Sridhar; Rastogi, Rajeev; Shim, Kyuseok (2000): Efficient Algorithms for Mining Outliers from Large Data Sets.
[2] Angiulli, Fabrizio; Pizzuti, Clara (2002): Fast Outlier Detection in High Dimensional Spaces.
LOFDetector
#
OutlierDetectionNeighbors.LOFDetector — Type.
LOFDetector(k = 5,
metric = Euclidean(),
algorithm = :kdtree,
leafsize = 10,
reorder = true,
parallel = false)
Calculate an anomaly score based on the density of an instance in comparison to its neighbors. This algorithm introduced the notion of local outliers and was developed by Breunig et al., see [1].
Parameters
k::Integer
Number of neighbors (must be greater than 0).
metric::Metric
This is one of the Metric types defined in the Distances.jl package. It is possible to define your own metrics by creating new types that are subtypes of Metric.
algorithm::Symbol
One of (:kdtree, :balltree). In a kdtree, points are recursively split into groups using hyper-planes. Therefore a KDTree only works with axis aligned metrics which are: Euclidean, Chebyshev, Minkowski and Cityblock. A brutetree linearly searches all points in a brute force fashion and works with any Metric. A balltree recursively splits points into groups bounded by hyper-spheres and works with any Metric.
static::Union{Bool, Symbol}
One of (true, false, :auto). Whether the input data for fitting and transform should be statically or dynamically allocated. If true, the data is statically allocated. If false, the data is dynamically allocated. If :auto, the data is dynamically allocated if the product of all dimensions except the last is greater than 100.
leafsize::Int
Determines at what number of points to stop splitting the tree further. There is a trade-off between traversing the tree and having to evaluate the metric function for increasing number of points.
reorder::Bool
While building the tree this will put points close in distance close in memory since this helps with cache locality. In this case, a copy of the original data will be made so that the original data is left unmodified. This can have a significant impact on performance and is by default set to true.
parallel::Bool
Parallelize score and predict using all threads available. The number of threads can be set with the JULIA_NUM_THREADS environment variable. Note: fit is not parallel.
Examples
using OutlierDetection: LOFDetector, fit, score
detector = LOFDetector()
X = rand(10, 100)
result = fit(detector, X)
test_scores = transform(detector, result.model, X)
References
[1] Breunig, Markus M.; Kriegel, Hans-Peter; Ng, Raymond T.; Sander, Jörg (2000): LOF: Identifying Density-Based Local Outliers.
Network-based
Warning
The neural-network detectors are experimental and subject to change.
AEDetector
#
OutlierDetectionNetworks.AEDetector — Type.
AEDetector(encoder= Chain(),
decoder = Chain(),
batchsize= 32,
epochs = 1,
shuffle = false,
partial = true,
opt = ADAM(),
loss = mse)
Calculate the anomaly score of an instance based on the reconstruction loss of an autoencoder, see [1] for an explanation of auto encoders.
Parameters
encoder::Chain
Transforms the input data into a latent state with a fixed shape.
decoder::Chain
Transforms the latent state back into the shape of the input data.
batchsize::Integer
The number of samples to work through before updating the internal model parameters.
epochs::Integer
The number of passes of the entire training dataset the machine learning algorithm has completed.
shuffle::Bool
If shuffle=true, shuffles the observations each time iterations are re-started, else no shuffling is performed.
partial::Bool
If partial=false, drops the last mini-batch if it is smaller than the batchsize.
opt::Any
Any Flux-compatibale optimizer, typically a struct that holds all the optimiser parameters along with a definition of apply! that defines how to apply the update rule associated with the optimizer.
loss::Function
The loss function used to calculate the reconstruction error, see https://fluxml.ai/Flux.jl/stable/models/losses/ for examples.
Examples
using OutlierDetection: AEDetector, fit, score
detector = AEDetector()
X = rand(10, 100)
result = fit(detector, X)
test_scores = transform(detector, result.model, X)
References
[1] Aggarwal, Charu C. (2017): Outlier Analysis.
DSADDetector
#
OutlierDetectionNetworks.DSADDetector — Type.
DSADDetector(encoder = Chain(),
decoder = Chain(),
batchsize = 32,
epochs = 1,
shuffle = true,
partial = false,
opt = ADAM(),
loss = mse,
eta = 1,
eps = 1e-6,
callback = _ -> () -> ())
Deep Semi-Supervised Anomaly detection technique based on the distance to a hypersphere center as described in [1].
Parameters
encoder::Chain
Transforms the input data into a latent state with a fixed shape.
decoder::Chain
Transforms the latent state back into the shape of the input data.
batchsize::Integer
The number of samples to work through before updating the internal model parameters.
epochs::Integer
The number of passes of the entire training dataset the machine learning algorithm has completed.
shuffle::Bool
If shuffle=true, shuffles the observations each time iterations are re-started, else no shuffling is performed.
partial::Bool
If partial=false, drops the last mini-batch if it is smaller than the batchsize.
opt::Any
Any Flux-compatibale optimizer, typically a struct that holds all the optimiser parameters along with a definition of apply! that defines how to apply the update rule associated with the optimizer.
loss::Function
The loss function used to calculate the reconstruction error, see https://fluxml.ai/Flux.jl/stable/models/losses/ for examples.
eta::Real
Weighting parameter for the labeled data; i.e. higher values of eta assign higher weight to labeled data in the svdd loss function. For a sensitivity analysis of this parameter, see [1].
eps::Real
Because the inverse distance used in the svdd loss can lead to division by zero, the parameters eps is added for numerical stability.
callback::Function
Experimental parameter that might change. A function to be called after the model parameters have been updated that can call Flux's callback helpers, see https://fluxml.ai/Flux.jl/stable/utilities/#Callback-Helpers-1.
Notice: The parameters batchsize, epochs, shuffle, partial, opt and callback can also be tuples of size 2, specifying the corresponding values for (1) pretraining and (2) training; otherwise the same values are used for pretraining and training.
Examples
using OutlierDetection: DSADDetector, fit, score
detector = DSADDetector()
X = rand(10, 100)
y = rand([-1,1], 100)
model = fit(detector, X, y)
train_scores, test_scores = score(detector, model, X)
References
[1] Ruff, Lukas; Vandermeulen, Robert A.; Görnitz, Nico; Binder, Alexander; Müller, Emmanuel; Müller, Klaus-Robert; Kloft, Marius (2019): Deep Semi-Supervised Anomaly Detection.
ESADDetector
#
OutlierDetectionNetworks.ESADDetector — Type.
ESADDetector(encoder = Chain(),
decoder = Chain(),
batchsize = 32,
epochs = 1,
shuffle = false,
partial = true,
opt = ADAM(),
λ1 = 1,
λ2 = 1,
noise = identity)
End-to-End semi-supervised anomaly detection algorithm similar to DeepSAD, but without the pretraining phase. The algorithm was published by Huang et al., see [1].
Parameters
encoder::Chain
Transforms the input data into a latent state with a fixed shape.
decoder::Chain
Transforms the latent state back into the shape of the input data.
batchsize::Integer
The number of samples to work through before updating the internal model parameters.
epochs::Integer
The number of passes of the entire training dataset the machine learning algorithm has completed.
shuffle::Bool
If shuffle=true, shuffles the observations each time iterations are re-started, else no shuffling is performed.
partial::Bool
If partial=false, drops the last mini-batch if it is smaller than the batchsize.
opt::Any
Any Flux-compatibale optimizer, typically a struct that holds all the optimiser parameters along with a definition of apply! that defines how to apply the update rule associated with the optimizer.
λ1::Real
Weighting parameter of the norm loss, which minimizes the empirical variance and thus minimizes entropy.
λ2::Real
Weighting parameter of the assistent loss function to define the consistency between the two encoders.
noise::Function (AbstractArray{T} -> AbstractArray{T})
A function to be applied to a batch of input data to add noise, see [1] for an explanation.
Examples
using OutlierDetection: ESADDetector, fit, score
detector = ESADDetector()
X = rand(10, 100)
y = rand([-1,1], 100)
model = fit(detector, X, y)
train_scores, test_scores = score(detector, model, X)
References
[1] Huang, Chaoqin; Ye, Fei; Zhang, Ya; Wang, Yan-Feng; Tian, Qi (2020): ESAD: End-to-end Deep Semi-supervised Anomaly Detection.
Python-based
Using PyCall, we can easily integrate existing python outlier detection algorithms. Currently, almost every PyOD algorithm is integrated and can thus be easily used directly from Julia.
ABODDetector
#
OutlierDetectionPython.ABODDetector — Type.
ABODDetector(n_neighbors = 5,
method = "fast")
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.abod
CBLOFDetector
#
OutlierDetectionPython.CBLOFDetector — Type.
CBLOFDetector(n_clusters = 8,
alpha = 0.9,
beta = 5,
use_weights = false,
random_state = nothing,
n_jobs = 1)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.cblof
COFDetector
#
OutlierDetectionPython.COFDetector — Type.
COFDetector(n_neighbors = 5)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.cof
COPODDetector
#
OutlierDetectionPython.COPODDetector — Type.
COPODDetector()
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.copod
HBOSDetector
#
OutlierDetectionPython.HBOSDetector — Type.
HBOSDetector(n_bins = 10,
alpha = 0.1,
tol = 0.5)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.hbos
IForestDetector
#
OutlierDetectionPython.IForestDetector — Type.
IForestDetector(n_estimators = 100,
max_samples = "auto",
max_features = 1.0
bootstrap = false,
behaviour = "new",
random_state = nothing,
verbose = 0,
n_jobs = 1)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.iforest
KNNDetector
#
OutlierDetectionPython.KNNDetector — Type.
KNNDetector(n_neighbors = 5,
method = "largest",
radius = 1.0,
algorithm = "auto",
leaf_size = 30,
metric = "minkowski",
p = 2,
metric_params = nothing,
n_jobs = 1)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.knn
LMDDDetector
#
OutlierDetectionPython.LMDDDetector — Type.
LMDDDetector(n_iter = 50,
dis_measure = "aad",
random_state = nothing)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.lmdd
LODADetector
#
OutlierDetectionPython.LODADetector — Type.
LODADetector(n_bins = 10,
n_random_cuts = 100)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.loda
LOFDetector
#
OutlierDetectionPython.LOFDetector — Type.
LOFDetector(n_neighbors = 5,
method = "largest",
algorithm = "auto",
leaf_size = 30,
metric = "minkowski",
p = 2,
metric_params = nothing,
n_jobs = 1)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.lof
LOCIDetector
#
OutlierDetectionPython.LOCIDetector — Type.
LOCIDetector(alpha = 0.5,
k = 3)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.loci
MCDDetector
#
OutlierDetectionPython.MCDDetector — Type.
MCDDetector(store_precision = true,
assume_centered = false,
support_fraction = nothing,
random_state = nothing)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.mcd
OCSVMDetector
#
OutlierDetectionPython.OCSVMDetector — Type.
OCSVMDetector(kernel = "rbf",
degree = 3,
gamma = "auto",
coef0 = 0.0,
tol = 0.001,
nu = 0.5,
shrinking = true,
cache_size = 200,
verbose = false,
max_iter = -1)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.ocsvm
PCADetector
#
OutlierDetectionPython.PCADetector — Type.
PCADetector(n_components = nothing,
n_selected_components = nothing,
copy = true,
whiten = false,
svd_solver = "auto",
tol = 0.0
iterated_power = "auto",
standardization = true,
weighted = true,
random_state = nothing)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.pca
RODDetector
#
OutlierDetectionPython.RODDetector — Type.
RODDetector(parallel_execution = false)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.rod
SODDetector
#
OutlierDetectionPython.SODDetector — Type.
SODDetector(n_neighbors = 5,
ref_set = 10,
alpha = 0.8)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.sod
SOSDetector
#
OutlierDetectionPython.SOSDetector — Type.
SOSDetector(perplexity = 4.5,
metric = "minkowski",
eps = 1e-5)
https://pyod.readthedocs.io/en/latest/pyod.models.html#module-pyod.models.sos