@@ -30,10 +30,11 @@ Sample code for hyper-parameter optimization implementation for machine learning
3030[ HPO_Classification.ipynb] ( https://github.com/LiYangHart/Hyperparameter-Optimization-of-Machine-Learning-Algorithms/blob/master/HPO_Classification.ipynb )
3131** Dataset used:** [ MNIST] ( https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_digits.html#sklearn.datasets.load_digits )
3232
33- ### Machine Learning Algorithms
33+ ### Machine Learning & Deep Learning Algorithms
3434* Random forest (RF)
3535* Support vector machine (SVM)
3636* K-nearest neighbor (KNN)
37+ * Artificial Neural Networks (ANN)
3738
3839### Hyperparameter Configuration Space
3940| ML Model | Hyper-parameter | Type | Search Space |
@@ -42,21 +43,34 @@ Sample code for hyper-parameter optimization implementation for machine learning
4243| | max_depth | Discrete | [ 5,50] |
4344| | min_samples_split | Discrete | [ 2,11] |
4445| | min_samples_leaf | Discrete | [ 1,11] |
45- | | criterion | Categorical | [ 'gini', 'entropy'] |
46+ | | criterion | Categorical | 'gini', 'entropy' |
4647| | max_features | Discrete | [ 1,64] |
4748| SVM Classifier | C | Continuous | [ 0.1,50] |
48- | | kernel | Categorical | [ 'linear', 'poly', 'rbf', 'sigmoid'] |
49+ | | kernel | Categorical | 'linear', 'poly', 'rbf', 'sigmoid' |
4950| KNN Classifier | n_neighbors | Discrete | [ 1,20] |
51+ | ANN Classifier | optimizer | Categorical | 'adam', 'rmsprop', 'sgd' |
52+ | | activation | Categorical | 'relu', 'tanh' |
53+ | | batch_size | Discrete | [ 16,64] |
54+ | | neurons | Discrete | [ 10,100] |
55+ | | epochs | Discrete | [ 20,50] |
56+ | | patience | Discrete | [ 3,20] |
5057| RF Regressor | n_estimators | Discrete | [ 10,100] |
5158| | max_depth | Discrete | [ 5,50] |
5259| | min_samples_split | Discrete | [ 2,11] |
5360| | min_samples_leaf | Discrete | [ 1,11] |
54- | | criterion | Categorical | [ 'mse', 'mae'] |
61+ | | criterion | Categorical | 'mse', 'mae' |
5562| | max_features | Discrete | [ 1,13] |
5663| SVM Regressor | C | Continuous | [ 0.1,50] |
57- | | kernel | Categorical | [ 'linear', 'poly', 'rbf', 'sigmoid'] |
64+ | | kernel | Categorical | 'linear', 'poly', 'rbf', 'sigmoid' |
5865| | epsilon | Continuous | [ 0.001,1] |
5966| KNN Regressor | n_neighbors | Discrete | [ 1,20] |
67+ | ANN Regressor | optimizer | Categorical | 'adam', 'rmsprop' |
68+ | | activation | Categorical | 'relu', 'tanh' |
69+ | | loss | Categorical | 'mse', 'mae' |
70+ | | batch_size | Discrete | [ 16,64] |
71+ | | neurons | Discrete | [ 10,100] |
72+ | | epochs | Discrete | [ 20,50] |
73+ | | patience | Discrete | [ 3,20] |
6074
6175### HPO Algorithms
6276* Grid search
@@ -69,6 +83,7 @@ Sample code for hyper-parameter optimization implementation for machine learning
6983
7084### Requirements
7185* Python 3.5
86+ * [ Keras] ( https://keras.io/ )
7287* [ scikit-learn] ( https://scikit-learn.org/stable/ )
7388* [ hyperband] ( https://github.com/thuijskens/scikit-hyperband )
7489* [ scikit-optimize] ( https://github.com/scikit-optimize/scikit-optimize )
0 commit comments