2020年11月13日 星期五

[ ML 文章收集 ] Keras - Plot training, validation and test set accuracy

 Source From Here

Question
As title.

HowTo
First step, let's prepare some fake training/test data:
  1. from sklearn.datasets import make_classification  
  2. from sklearn.linear_model import LogisticRegression  
  3. from sklearn.model_selection import train_test_split  
  4. from sklearn.metrics import roc_curve  
  5. from sklearn.metrics import roc_auc_score  
  6. from matplotlib import pyplot  
  7.   
  8. # Generate 2 class dataset  
  9. X, y = make_classification(n_samples=1000, n_classes=2, random_state=1)  
  10. # Split into train/test sets  
  11. trainX, testX, trainy, testy = train_test_split(X, y, test_size=0.2, random_state=2)  
Check dataset shape:
  1. print(f"trainX.shape={trainX.shape}")  
Output:
trainX.shape=(800, 20)

Next is to generate a NN model:
  1. import keras  
  2. import tensorflow as tf  
  3. from keras.models import Sequential  
  4. from keras.layers import Dense  
  5. from keras.wrappers.scikit_learn import KerasClassifier  
  6. from keras import regularizers  
  7. from keras.optimizers import SGD  
  8. from keras.layers import Dropout  
  9. from keras.models import model_from_json  
  10. from keras import backend as K  
  11. from keras.callbacks import Callback  
  12.   
  13. def get_nn(input_shape):  
  14.     # Initialising the ANN  
  15.     classifier = Sequential()  
  16.   
  17.     # Adding the dropout layer  
  18.     classifier.add(Dropout(0.01, input_shape=(input_shape,)))  
  19.   
  20.     # Adding the input layer and the first hidden layer  
  21.     classifier.add(Dense(input_shape, kernel_initializer='uniform', activation='sigmoid'))  
  22.   
  23.     # Adding the input layer and the first hidden layer  
  24.     classifier.add(Dense(input_shape*2, kernel_initializer='uniform', activation='sigmoid'))  
  25.   
  26.     # Adding the second hidden layer  
  27.     classifier.add(  
  28.         Dense(  
  29.             units=int(input_shape),  
  30.             kernel_initializer = 'uniform',  
  31.             activation = 'sigmoid',  
  32.             activity_regularizer=regularizers.l1(10e-5)  
  33.         )  
  34.     )  
  35.   
  36.     # Adding the third hidden layer  
  37.     classifier.add(  
  38.         Dense(  
  39.             units=int(input_shape/2),  
  40.             kernel_initializer = 'uniform',  
  41.             activation = 'sigmoid',  
  42.             activity_regularizer=regularizers.l1(10e-5)  
  43.         )  
  44.     )  
  45.       
  46.     # Adding the output layer  
  47.     classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))  
  48.   
  49.     # Compiling the ANN  
  50.     # https://keras.io/api/optimizers/  
  51.     # opt = SGD(lr=0.01, momentum=0.9)  
  52.     m_preccision = keras.metrics.Precision()  
  53.     m_recall = keras.metrics.Recall()  
  54.     # classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = [m_preccision, m_recall])  
  55.     classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy', tf.keras.metrics.Precision()])  
  56.       
  57.     return classifier  
  58.       
  59. nn_model = get_nn(trainX.shape[1])  
  60. nn_model.summary()  
Output:
  1. Model: "sequential_5"  
  2. _________________________________________________________________  
  3. Layer (type)                 Output Shape              Param #     
  4. =================================================================  
  5. dropout_5 (Dropout)          (None, 20)                0           
  6. _________________________________________________________________  
  7. dense_23 (Dense)             (None, 20)                420         
  8. _________________________________________________________________  
  9. dense_24 (Dense)             (None, 40)                840         
  10. _________________________________________________________________  
  11. dense_25 (Dense)             (None, 20)                820         
  12. _________________________________________________________________  
  13. dense_26 (Dense)             (None, 10)                210         
  14. _________________________________________________________________  
  15. dense_27 (Dense)             (None, 1)                 11          
  16. =================================================================  
  17. Total params: 2,301  
  18. Trainable params: 2,301  
  19. Non-trainable params: 0  
It is time for training and collect evaluation result:
  1. from matplotlib import pyplot as plt  
  2.   
  3. history = nn_model.fit(trainX, trainy, validation_split=0.1, epochs=50, batch_size=4)  
  4. print(f"history.history.key()={history.history.keys()}")  
Output:
...
Epoch 50/50
180/180 [==============================] - 0s 3ms/step - loss: 0.1062 - accuracy: 0.9764 - precision_5: 0.9780 - val_loss: 0.4934 - val_accuracy: 0.8625 - val_precision_5: 0.8611
history.history.key()=dict_keys(['loss', 'accuracy', 'precision_5', 'val_loss', 'val_accuracy', 'val_precision_5'])

Now we can check out accuracy trending chart:
  1. plt.rcParams['figure.figsize'] = [105]  
  2. plt.plot(history.history['accuracy'])  
  3. plt.plot(history.history['val_accuracy'])  
  4. plt.title('model accuracy')  
  5. plt.ylabel('accuracy')  
  6. plt.xlabel('epoch')  
  7. plt.legend(['train''val'], loc='upper left')  
  8. plt.show()  


And then the loss trending chart:
  1. plt.plot(history.history['loss'])  
  2. plt.plot(history.history['val_loss'])  
  3. plt.title('model loss')  
  4. plt.ylabel('loss')  
  5. plt.xlabel('epoch')  
  6. plt.legend(['train_loss''val_loss'], loc='upper left')  
  7. plt.show()  


For the complete example, refer to this notebook.

Supplement
How to Use ROC Curves and Precision-Recall Curves for Classification in Python
How to make inline plots in Jupyter Notebook larger?





沒有留言:

張貼留言

[Git 常見問題] error: The following untracked working tree files would be overwritten by merge

  Source From  Here 方案1: // x -----删除忽略文件已经对 git 来说不识别的文件 // d -----删除未被添加到 git 的路径中的文件 // f -----强制运行 #   git clean -d -fx 方案2: 今天在服务器上  gi...