[딥러닝 기초] Convolution Neuron Network for keras

[이 글은 "Do it 딥러닝 입문" 책을 보고 공부한 내용을 복습하고자 정리한 글입니다.]

 

Convolution Neuron Network(CNN)

 

 

필요한 클래스 임포트

 

- tensorflow.keras.layers에 있는 Conv2D, MaxPooling2D, Flatten, Dense를 import해준다.

 

 

합성곱층 쌓기

 

- Conv2D 클래스의 첫 번째 매개변수는 합성곱 커널의 개수.

- 두 번째 매개변수는 합성곱 커널의 크기(높이와 너비)를 "튜플"형태로 전달.

- 세 번째는 활성화 함수.

- 네 번째, 다섯 번째는 각각 패딩, 입력 값 shape이다.

 

풀링층 쌓기

 

- 첫 번째 매개변수는 풀링의 높이오 너비를 나타내는 "튜플"

- 두 번째 매개변수는 스트라이드, 기본 값은 풀링의 크기

- 세 번째 매개변수는 padding, 기본 값은 'valid

 

특성 맵 펼치기

 

- 완전 연결층에 주입할 수 있도록 Flatten()함수로 특성 맵 펼치기

 

드롭아웃의 개념과 특징

 

- 신경망에서 과대적합을 줄이는 방법 중 하나는 드롭아웃인데 드롭아웃은 무작위로 뉴런을 비활성화 시키는 방법이다.

- 무작위로 일부 뉴런을 비활성화 시키면 특정 뉴런에 과도하게 의존하여 훈련하는 것을 막아준다.

- 뉴런이 훈련세트에 있는 패턴을 고르게 감지하므로 전체적인 일반화 성능이 높아진다.

 

** 상대적으로 테스트와 실전의 출력값이 훈련할 때의 출력값보다 높아지므로 테스트나 실전에서는 출력값을 드롭아웃 비율만큼 낮춰야한다. -> but -> 대부분의 딥러닝 프레임워크는 반대로 이 문제를 해결하는데, 방법은 훈련 할 때 드롭아웃의 비율만큼 뉴런의 출력을 높여 훈련시킨다.

 

 

In [1]:

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense

In [2]:

#케라스로 모델 생성하기
conv1 = tf.keras.Sequential()
conv1.add(Conv2D(10, (3,3), activation='relu', padding='same',input_shape=(28,28,1)))
conv1.add(MaxPooling2D((2,2)))
conv1.add(Flatten())
conv1.add(Dense(100, activation='relu'))
conv1.add(Dense(10,activation='softmax'))

In [3]:

conv1.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 28, 28, 10)        100       
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 14, 14, 10)        0         
_________________________________________________________________
flatten (Flatten)            (None, 1960)              0         
_________________________________________________________________
dense (Dense)                (None, 100)               196100    
_________________________________________________________________
dense_1 (Dense)              (None, 10)                1010      
=================================================================
Total params: 197,210
Trainable params: 197,210
Non-trainable params: 0
_________________________________________________________________

In [4]:

conv1.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

In [5]:

#데이터 준비하기
(x_train_all, y_train_all), (x_test,y_test) = tf.keras.datasets.mnist.load_data()

In [6]:

from sklearn.model_selection import train_test_split

x_train, x_val, y_train, y_val = train_test_split(x_train_all,y_train_all, stratify=y_train_all, test_size= 0.2, random_state=42)

x_train = x_train.reshape(-1,28,28,1)
x_val = x_val.reshape(-1,28,28,1)

y_train_encoded = tf.keras.utils.to_categorical(y_train)
y_val_encoded = tf.keras.utils.to_categorical(y_val)

x_train = x_train / 255
x_val = x_val / 255

In [7]:

history = conv1.fit(x_train, y_train_encoded, epochs=20, validation_data=(x_val,y_val_encoded))

WARNING:tensorflow:From C:\Users\USER\Anaconda3\lib\site-packages\tensorflow\python\ops\math_grad.py:1250: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
Train on 48000 samples, validate on 12000 samples
Epoch 1/20
48000/48000 [==============================] - 7s 145us/sample - loss: 0.2161 - accuracy: 0.9366 - val_loss: 0.0948 - val_accuracy: 0.9722
Epoch 2/20
48000/48000 [==============================] - 4s 82us/sample - loss: 0.0744 - accuracy: 0.9773 - val_loss: 0.0693 - val_accuracy: 0.9793
Epoch 3/20
48000/48000 [==============================] - 4s 79us/sample - loss: 0.0523 - accuracy: 0.9838 - val_loss: 0.0632 - val_accuracy: 0.9807
Epoch 4/20
48000/48000 [==============================] - 4s 80us/sample - loss: 0.0396 - accuracy: 0.9873 - val_loss: 0.0602 - val_accuracy: 0.9821
Epoch 5/20
48000/48000 [==============================] - 4s 78us/sample - loss: 0.0308 - accuracy: 0.9900 - val_loss: 0.0716 - val_accuracy: 0.9798
Epoch 6/20
48000/48000 [==============================] - 4s 78us/sample - loss: 0.0234 - accuracy: 0.9924 - val_loss: 0.0707 - val_accuracy: 0.9790
Epoch 7/20
48000/48000 [==============================] - 4s 77us/sample - loss: 0.0188 - accuracy: 0.9937 - val_loss: 0.0587 - val_accuracy: 0.9842
Epoch 8/20
48000/48000 [==============================] - 4s 76us/sample - loss: 0.0143 - accuracy: 0.9955 - val_loss: 0.0593 - val_accuracy: 0.9850
Epoch 9/20
48000/48000 [==============================] - 4s 76us/sample - loss: 0.0118 - accuracy: 0.9964 - val_loss: 0.0669 - val_accuracy: 0.9829
Epoch 10/20
48000/48000 [==============================] - 4s 77us/sample - loss: 0.0095 - accuracy: 0.9969 - val_loss: 0.0683 - val_accuracy: 0.9839
Epoch 11/20
48000/48000 [==============================] - 4s 77us/sample - loss: 0.0080 - accuracy: 0.9974 - val_loss: 0.0611 - val_accuracy: 0.9855
Epoch 12/20
48000/48000 [==============================] - 4s 77us/sample - loss: 0.0072 - accuracy: 0.9976 - val_loss: 0.0635 - val_accuracy: 0.9855
Epoch 13/20
48000/48000 [==============================] - 4s 76us/sample - loss: 0.0058 - accuracy: 0.9982 - val_loss: 0.0717 - val_accuracy: 0.9846
Epoch 14/20
48000/48000 [==============================] - 4s 77us/sample - loss: 0.0052 - accuracy: 0.9982 - val_loss: 0.0759 - val_accuracy: 0.9842
Epoch 15/20
48000/48000 [==============================] - 4s 80us/sample - loss: 0.0049 - accuracy: 0.9984 - val_loss: 0.0855 - val_accuracy: 0.9826
Epoch 16/20
48000/48000 [==============================] - 4s 81us/sample - loss: 0.0043 - accuracy: 0.9987 - val_loss: 0.0829 - val_accuracy: 0.9830
Epoch 17/20
48000/48000 [==============================] - 4s 81us/sample - loss: 0.0036 - accuracy: 0.9989 - val_loss: 0.0842 - val_accuracy: 0.9835
Epoch 18/20
48000/48000 [==============================] - 4s 82us/sample - loss: 0.0041 - accuracy: 0.9986 - val_loss: 0.0787 - val_accuracy: 0.9849
Epoch 19/20
48000/48000 [==============================] - 4s 81us/sample - loss: 0.0033 - accuracy: 0.9990 - val_loss: 0.0780 - val_accuracy: 0.9849
Epoch 20/20
48000/48000 [==============================] - 4s 82us/sample - loss: 0.0016 - accuracy: 0.9996 - val_loss: 0.0898 - val_accuracy: 0.9837

In [8]:

plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylabel('loss')
plt.xlabel('epoch')
plt.show()

plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train_accuray', 'val_accuracy'])
plt.show()

In [9]:

#케라스로 만든 합성곱 신경망에 드롭아웃 적용하기
from tensorflow.keras.layers import Dropout

conv2 = tf.keras.Sequential()
conv2.add(Conv2D(10, (3,3), activation='relu', padding='same', input_shape=(28,28,1)))
conv2.add(MaxPooling2D((2,2)))

In [10]:

conv2.add(Flatten())
conv2.add(Dropout(0.5))
conv2.add(Dense(100,activation='relu'))
conv2.add(Dense(10,activation='softmax'))

In [11]:

conv2.summary()

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 28, 28, 10)        100       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 10)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 1960)              0         
_________________________________________________________________
dropout (Dropout)            (None, 1960)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 100)               196100    
_________________________________________________________________
dense_3 (Dense)              (None, 10)                1010      
=================================================================
Total params: 197,210
Trainable params: 197,210
Non-trainable params: 0
_________________________________________________________________

In [12]:

conv2.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

history= conv2.fit(x_train,y_train_encoded, epochs=20, validation_data=(x_val,y_val_encoded))

Train on 48000 samples, validate on 12000 samples
Epoch 1/20
48000/48000 [==============================] - 6s 116us/sample - loss: 0.2914 - accuracy: 0.9120 - val_loss: 0.1217 - val_accuracy: 0.9647
Epoch 2/20
48000/48000 [==============================] - 5s 112us/sample - loss: 0.1419 - accuracy: 0.9565 - val_loss: 0.0829 - val_accuracy: 0.9765
Epoch 3/20
48000/48000 [==============================] - 5s 112us/sample - loss: 0.1091 - accuracy: 0.9653 - val_loss: 0.0660 - val_accuracy: 0.9805
Epoch 4/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0878 - accuracy: 0.9724 - val_loss: 0.0574 - val_accuracy: 0.9817
Epoch 5/20
48000/48000 [==============================] - 5s 112us/sample - loss: 0.0776 - accuracy: 0.9751 - val_loss: 0.0528 - val_accuracy: 0.9826
Epoch 6/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0654 - accuracy: 0.9786 - val_loss: 0.0490 - val_accuracy: 0.9847
Epoch 7/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0585 - accuracy: 0.9805 - val_loss: 0.0480 - val_accuracy: 0.9849
Epoch 8/20
48000/48000 [==============================] - 5s 112us/sample - loss: 0.0530 - accuracy: 0.9822 - val_loss: 0.0488 - val_accuracy: 0.9847
Epoch 9/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0493 - accuracy: 0.9839 - val_loss: 0.0421 - val_accuracy: 0.9874
Epoch 10/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0456 - accuracy: 0.9849 - val_loss: 0.0450 - val_accuracy: 0.9863
Epoch 11/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0430 - accuracy: 0.9860 - val_loss: 0.0405 - val_accuracy: 0.9883
Epoch 12/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0382 - accuracy: 0.9877 - val_loss: 0.0392 - val_accuracy: 0.9881
Epoch 13/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0390 - accuracy: 0.9874 - val_loss: 0.0401 - val_accuracy: 0.9877
Epoch 14/20
48000/48000 [==============================] - 5s 114us/sample - loss: 0.0337 - accuracy: 0.9887 - val_loss: 0.0427 - val_accuracy: 0.9874
Epoch 15/20
48000/48000 [==============================] - 5s 114us/sample - loss: 0.0331 - accuracy: 0.9891 - val_loss: 0.0406 - val_accuracy: 0.9887
Epoch 16/20
48000/48000 [==============================] - 6s 117us/sample - loss: 0.0325 - accuracy: 0.9894 - val_loss: 0.0413 - val_accuracy: 0.9881
Epoch 17/20
48000/48000 [==============================] - 5s 114us/sample - loss: 0.0311 - accuracy: 0.9896 - val_loss: 0.0385 - val_accuracy: 0.9880
Epoch 18/20
48000/48000 [==============================] - 5s 114us/sample - loss: 0.0303 - accuracy: 0.9897 - val_loss: 0.0428 - val_accuracy: 0.9877
Epoch 19/20
48000/48000 [==============================] - 5s 114us/sample - loss: 0.0279 - accuracy: 0.9908 - val_loss: 0.0378 - val_accuracy: 0.9886
Epoch 20/20
48000/48000 [==============================] - 5s 113us/sample - loss: 0.0272 - accuracy: 0.9903 - val_loss: 0.0417 - val_accuracy: 0.9885

In [13]:

plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.ylabel('loss')
plt.xlabel('epochs')
plt.legend(['train_loss', 'val_loss'])
plt.show()

plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylabel('acc')
plt.xlabel('epoch')
plt.legend(['train_accuracy', 'val_accuracy'])
plt.show()