{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"resent_from_scratch.ipynb","provenance":[],"collapsed_sections":[],"authorship_tag":"ABX9TyOI6Pmo84p8lVvF35hto3Rx"},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"hgnolmRg0O4i"},"source":["# Building Resnet 50 from scratch with Keras"]},{"cell_type":"markdown","metadata":{"id":"MAHPoBSo0udc"},"source":["Resnets are one of the most popular convolutional networks available in deep learning literature. All major libraries (e.g. Keras) have fully baked implementations of Resnets available for engineers to use on daily basis. There are a number of online tutorials available which illuminate the basic principles behind the resnets. Here are a couple of them:\n","\n","\n","*   [Detailed Guide to Understand and Implement ResNets](https://cv-tricks.com/keras/understand-implement-resnets/)\n","*   [Hitchhiker’s Guide to Residual Networks in Keras](https://towardsdatascience.com/hitchhikers-guide-to-residual-networks-resnet-in-keras-385ec01ec8ff)\n","\n","There is also one useful tutorial about building the key modules in popular networks like VGG, Inception and ResNet.\n","\n","*   [How to Develop VGG, Inception and ResNet Modules from Scratch in Keras](https://machinelearningmastery.com/how-to-implement-major-architecture-innovations-for-convolutional-neural-networks/)\n","\n","\n","However, there is a lack of articles walking through the nitty gritties of a complete ResNet implementation. There are several details which need to be properly addressed in building a complete ResNet. In this article, we will focus on building ResNet 50 from scratch. Our presentation in this tutorial is a simplified version of the code available in the [Keras Applications](https://github.com/keras-team/keras-applications) GITHUB repository. \n","\n","One key goal of this tutorial is to give you hands on experience of building large complex CNNs with the help of [Keras Functional API](https://keras.io/guides/functional_api/). \n","\n","For basic introduction to Resnets, we suggest looking at the articles mentioned above or the original paper\n","\n","* [2016, Deep Residual Learning for Image Recognition](https://www.cv-foundation.org/openaccess/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html)\n","\n","Following is table-1 from the paper which describes various ResNet architectures. \n","![Resnet Architectures](https://drive.google.com/uc?id=1CH2tuV2hMFZ7BAyeURm8tLhQUvqvinfm)\n","\n","Please spend some time looking at the column for the architecture of 50 layer ResNet. \n","\n","\n","Without further ado, let's get into implementing a Resnet 50 network with Keras.\n"]},{"cell_type":"markdown","metadata":{"id":"6wRyb2Cm6z2N"},"source":["We start by importing relevant modules from Keras."]},{"cell_type":"code","metadata":{"id":"BnT-xBeuntp6"},"source":["from tensorflow.keras import layers, backend, models, utils\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"CzgTRToy669z"},"source":["In the sequel, we will need to create various batch normalization layers. All of them will have same epsilon value which is a small float added to the variance to avoid dividing by zero. "]},{"cell_type":"code","metadata":{"id":"UXS7lcv1pr5R"},"source":["# epsilon for Batch Normalization layers\n","BN_EPS= 1.001e-5\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"-MwChQ4z7Us9"},"source":["There are five stages of the ResNet which have been labeled as conv1, conv2, conv3, conv4 and conv5 in the paper (first column of the table above). These are followed by a global average pool and a simple fully connected 1000 way classification layer. \n","\n","Input to the ResNet 50 network is typically a batch of images with size ( 224, 224, 3). \n","\n","* Stage conv1 output is (56, 56, 64). i.e. the size has reduced 4 times to (56, 56) and the number of channels has increased to 64.\n","* Stage conv2 output is (56, 56, 256).\n","* Stage conv3 output is (28, 28, 512).\n","* Stage conv4 output is (14, 14, 1024)\n","* Stage conv5 output is (7, 7, 2048)\n","* conv2 has 3 residual blocks, conv3 has 4 residual blocks, conv4 has 6 and conv5 has 3.\n"]},{"cell_type":"markdown","metadata":{"id":"PP404Q-V9jgi"},"source":["## The Residual Blocks\n","Let's start by defining functions for building the residual blocks in the ResNet50 network. We will slowly increase the complexity of residual blocks to cover all the needs of ResNet 50.\n","\n","Every residual block essentially consists of three convolutional layers along the residual path and an identity connection from input to output. There are some details which will come up later. Let's look at the residual blocks in conv2 stage.\n","\n","Output of the conv1 stage is a tensor of size (None, 56, 56, 64). Its implementation will be discussed later.\n","\n","The first convolutional layer in this residual block is a 1x1 layer with 64 filters. Second one is a 3x3 layer with 64 filters. The last is again a 1x1 layer with 4 times number of filters at 256 filters. \n","\n","For now, let's just build the convolutional layers for the residual path."]},{"cell_type":"code","metadata":{"id":"UKf8GsNiohbk"},"source":["def conv_131(input, filters):\n","    # number of channels in output tensor\n","    num_output_channels = 4 * filters\n","    # The 1x1 first convolution layer\n","    net = layers.Conv2D(filters, 1)(input)\n","    net = layers.BatchNormalization(epsilon=BN_EPS,)(net)\n","    net = layers.Activation('relu')(net)\n","    # The 3x3 second convolution layer\n","    net = layers.Conv2D(filters, 3, padding='same')(net)\n","    net = layers.BatchNormalization(epsilon=BN_EPS)(net)\n","    net = layers.Activation('relu')(net)\n","    # The 1x1 third convolution layer\n","    net = layers.Conv2D(num_output_channels, 1)(net)\n","    net = layers.BatchNormalization(epsilon=BN_EPS)(net)\n","    net = layers.Activation('relu')(net)\n","    return net\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"n7hGro_q-isH"},"source":["The 1x1 layers don't require a padding parameter as 1x1 convolution is nothing but an inner product over the channels in the input tensor and it doesn't lead to any changes in the image size (unless a different stride is specified). It can definitely change the number of channels from input to output.\n","\n","Each conv layer is followed by batch normalization and then a relu activation. \n"]},{"cell_type":"markdown","metadata":{"id":"wyDFUsP5_B9o"},"source":["## Building a model\n","\n","We will write a simple function which can build a Keras model from a network building function. A network building function, like `conv_131` above, takes a layer as input, adds some more layers on top of it.\n","\n","The model building function considers the shape of the input tensor to the network and uses it to create an input layer. It then feeds the input layer to the network building function and builds the whole network. Finally, the input and output are combined to form a Keras model.  \n","\n","This function will be quite handy in displaying the architecture of any network in rest of the tutorial."]},{"cell_type":"code","metadata":{"id":"O8AwgJr2qDIT"},"source":["def build_model(input_shape, net_fn):\n","    img_input =  layers.Input(shape=input_shape)\n","    net = net_fn(img_input)\n","    inputs = img_input\n","    outputs = net\n","    model = models.Model(inputs, outputs)\n","    return model"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"qtEs7FKbAGtO"},"source":["Let's use this function to build a small model consisting of a single block of the three convolutional layers in conv2 stage of ResNet 50. As discussed earlier, the input shape is (56, 56, 64). We then print the model architecture summary."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4cZmjDICqsY-","executionInfo":{"status":"ok","timestamp":1607111054703,"user_tz":-330,"elapsed":3092,"user":{"displayName":"Shailesh Kumar","photoUrl":"","userId":"17557417109418259401"}},"outputId":"0539340e-1689-4033-b485-06b6b4b2d854"},"source":["model = build_model((56, 56, 64), lambda input: conv_131(input, 64))\n","model.summary()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Model: \"functional_1\"\n","_________________________________________________________________\n","Layer (type)                 Output Shape              Param #   \n","=================================================================\n","input_1 (InputLayer)         [(None, 56, 56, 64)]      0         \n","_________________________________________________________________\n","conv2d (Conv2D)              (None, 56, 56, 64)        4160      \n","_________________________________________________________________\n","batch_normalization (BatchNo (None, 56, 56, 64)        256       \n","_________________________________________________________________\n","activation (Activation)      (None, 56, 56, 64)        0         \n","_________________________________________________________________\n","conv2d_1 (Conv2D)            (None, 56, 56, 64)        36928     \n","_________________________________________________________________\n","batch_normalization_1 (Batch (None, 56, 56, 64)        256       \n","_________________________________________________________________\n","activation_1 (Activation)    (None, 56, 56, 64)        0         \n","_________________________________________________________________\n","conv2d_2 (Conv2D)            (None, 56, 56, 256)       16640     \n","_________________________________________________________________\n","batch_normalization_2 (Batch (None, 56, 56, 256)       1024      \n","_________________________________________________________________\n","activation_2 (Activation)    (None, 56, 56, 256)       0         \n","=================================================================\n","Total params: 59,264\n","Trainable params: 58,496\n","Non-trainable params: 768\n","_________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"fSE4aY8MAnV1"},"source":["## The identity path\n","It's time to add the identity path in our 3 CNN layer block. However, there is a catch. The input to first residual block is (56, 56, 64). But the output of the third layer in this block is (56, 56, 256). The number of channels changes from 64 to 256. Hence, the input cannot be added to the output of the residual path directly. A solution is to add a 1x1 conv layer in the identity path (whenever the number of input channels is not equal to the number of output channels). \n","\n","The function `residual_131_v1` below incorporates this. Note the following in this:\n","\n","*   We check if the number of input and output channels is different.\n","*   If yes, then a 1x1 conv layer is added with batch normalization but no relu activation.\n","*   The output of the last (1x1) conv layer is bacth normalized.\n","*   Then it is added to the output of the shortcut identity path.\n","*   Finally, the sum undergoes a common relu activation.\n"]},{"cell_type":"code","metadata":{"id":"vdnJn-artHrF"},"source":["def residual_131_v1(input, filters):\n","    # shape of input tensor\n","    input_shape = input.shape\n","    # number of channels in input tensor\n","    num_input_channels = input_shape[3]\n","    # number of channels in output tensor\n","    num_output_channels = 4 * filters\n","    # if input and output channels are same then we can feed\n","    # the input directly as identity shortcut\n","    # otherwise, we need to add a convolutional layer in identity path\n","    conv_in_identity_path = num_output_channels != num_input_channels\n","    if conv_in_identity_path is True:\n","        # add a conv layer to increase the number of channels\n","        shortcut = layers.Conv2D(num_output_channels, 1)(input)\n","        # batch normalize (activation will come later)\n","        shortcut = layers.BatchNormalization(epsilon=BN_EPS)(shortcut)\n","    else:\n","        shortcut = input\n","    # The 1x1 first convolution layer\n","    net = layers.Conv2D(filters, 1)(input)\n","    net = layers.BatchNormalization(epsilon=BN_EPS,)(net)\n","    net = layers.Activation('relu')(net)\n","    # The 3x3 second convolution layer\n","    net = layers.Conv2D(filters, 3, padding='same')(net)\n","    net = layers.BatchNormalization(epsilon=BN_EPS)(net)\n","    net = layers.Activation('relu')(net)\n","    # The 1x1 third convolution layer\n","    net = layers.Conv2D(num_output_channels, 1)(net)\n","    net = layers.BatchNormalization(epsilon=BN_EPS)(net)\n","    # Add identity shortcut to residual output before activation\n","    net = layers.Add()([shortcut, net])\n","    net = layers.Activation('relu')(net)\n","    return net\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"DydzRCanCfRW"},"source":["Let's build a model with a residual block and print its summary."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"W7W8KZ-zt1S2","executionInfo":{"status":"ok","timestamp":1607111054704,"user_tz":-330,"elapsed":3082,"user":{"displayName":"Shailesh Kumar","photoUrl":"","userId":"17557417109418259401"}},"outputId":"54498c22-1d3c-4192-a420-672598e0bd8b"},"source":["model = build_model((56, 56, 64), lambda input: residual_131_v1(input, 64))\n","model.summary()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Model: \"functional_3\"\n","__________________________________________________________________________________________________\n","Layer (type)                    Output Shape         Param #     Connected to                     \n","==================================================================================================\n","input_2 (InputLayer)            [(None, 56, 56, 64)] 0                                            \n","__________________________________________________________________________________________________\n","conv2d_4 (Conv2D)               (None, 56, 56, 64)   4160        input_2[0][0]                    \n","__________________________________________________________________________________________________\n","batch_normalization_4 (BatchNor (None, 56, 56, 64)   256         conv2d_4[0][0]                   \n","__________________________________________________________________________________________________\n","activation_3 (Activation)       (None, 56, 56, 64)   0           batch_normalization_4[0][0]      \n","__________________________________________________________________________________________________\n","conv2d_5 (Conv2D)               (None, 56, 56, 64)   36928       activation_3[0][0]               \n","__________________________________________________________________________________________________\n","batch_normalization_5 (BatchNor (None, 56, 56, 64)   256         conv2d_5[0][0]                   \n","__________________________________________________________________________________________________\n","activation_4 (Activation)       (None, 56, 56, 64)   0           batch_normalization_5[0][0]      \n","__________________________________________________________________________________________________\n","conv2d_3 (Conv2D)               (None, 56, 56, 256)  16640       input_2[0][0]                    \n","__________________________________________________________________________________________________\n","conv2d_6 (Conv2D)               (None, 56, 56, 256)  16640       activation_4[0][0]               \n","__________________________________________________________________________________________________\n","batch_normalization_3 (BatchNor (None, 56, 56, 256)  1024        conv2d_3[0][0]                   \n","__________________________________________________________________________________________________\n","batch_normalization_6 (BatchNor (None, 56, 56, 256)  1024        conv2d_6[0][0]                   \n","__________________________________________________________________________________________________\n","add (Add)                       (None, 56, 56, 256)  0           batch_normalization_3[0][0]      \n","                                                                 batch_normalization_6[0][0]      \n","__________________________________________________________________________________________________\n","activation_5 (Activation)       (None, 56, 56, 256)  0           add[0][0]                        \n","==================================================================================================\n","Total params: 76,928\n","Trainable params: 75,648\n","Non-trainable params: 1,280\n","__________________________________________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"GihWwsEPCrOc"},"source":["## The conv1 stage\n","\n","There are still one problem with the residual block. But before that, let's complete the implementation of the conv1 stage. See the function below.\n","\n","* Input is a (224, 224, 3) image.\n","* The first layer is a large 7x7 convolutional layer that downsamples the resolution to (112, 112) and increases the number of channels to 64.\n","* We achieve this in two steps as follows. \n","* Add a padding of 3 pixels on all sides increasing resolution to (230, 230)\n","* Perform a 7x7 valid convolution with stride 2 with 64 filters and achieve an output of (112, 112, 64).\n","* Next, the output goes through batch normalization and relu activation.\n","* Finally, we add a padding of 1 pixels and then do a max pooling of 3x3 to achieve an output tensor of size (56,56,64). \n","\n","\n"]},{"cell_type":"code","metadata":{"id":"hgNLqiC-ue1k"},"source":["def conv1(img_input):\n","    # pad in advance for valid convolution (output is 230x230)\n","    net = layers.ZeroPadding2D(padding=(3, 3))(img_input)\n","    # perform the big 7x7 convolution with 2x2 stride to (112x112x64)\n","    net = layers.Conv2D(64, (7, 7),\n","                      strides=(2, 2),\n","                      padding='valid',\n","                      kernel_initializer='he_normal')(net)\n","    # batch normalization before activation (output is 112x112x64)\n","    net = layers.BatchNormalization(epsilon=BN_EPS)(net)\n","    # relu activation (output is 112x112x64)\n","    net = layers.Activation('relu')(net)\n","    # pad again for max pooling (output is 114x114x64)\n","    net = layers.ZeroPadding2D(padding=(1, 1))(net)\n","    # 3x3 max pooling with 2x2 stride (output is 56x56x64)\n","    net = layers.MaxPooling2D((3, 3), strides=(2, 2))(net)\n","    return net\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"REdDHMC_EOdK"},"source":["Let's combine the conv1 stage with our first residual block to see if everything is working; build a partial residual network and print it."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"HkyY7WZ_u2n6","executionInfo":{"status":"ok","timestamp":1607111055200,"user_tz":-330,"elapsed":3568,"user":{"displayName":"Shailesh Kumar","photoUrl":"","userId":"17557417109418259401"}},"outputId":"cf4aed47-7a39-4ef1-beb7-ac0dc34faa8a"},"source":["def partial_resnet_v1(img_input):\n","  net = conv1(img_input)\n","  net = residual_131_v1(net, 64)\n","  return net\n","\n","model =  build_model((224, 224, 3), partial_resnet_v1)\n","model.summary()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Model: \"functional_5\"\n","__________________________________________________________________________________________________\n","Layer (type)                    Output Shape         Param #     Connected to                     \n","==================================================================================================\n","input_3 (InputLayer)            [(None, 224, 224, 3) 0                                            \n","__________________________________________________________________________________________________\n","zero_padding2d (ZeroPadding2D)  (None, 230, 230, 3)  0           input_3[0][0]                    \n","__________________________________________________________________________________________________\n","conv2d_7 (Conv2D)               (None, 112, 112, 64) 9472        zero_padding2d[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_7 (BatchNor (None, 112, 112, 64) 256         conv2d_7[0][0]                   \n","__________________________________________________________________________________________________\n","activation_6 (Activation)       (None, 112, 112, 64) 0           batch_normalization_7[0][0]      \n","__________________________________________________________________________________________________\n","zero_padding2d_1 (ZeroPadding2D (None, 114, 114, 64) 0           activation_6[0][0]               \n","__________________________________________________________________________________________________\n","max_pooling2d (MaxPooling2D)    (None, 56, 56, 64)   0           zero_padding2d_1[0][0]           \n","__________________________________________________________________________________________________\n","conv2d_9 (Conv2D)               (None, 56, 56, 64)   4160        max_pooling2d[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_9 (BatchNor (None, 56, 56, 64)   256         conv2d_9[0][0]                   \n","__________________________________________________________________________________________________\n","activation_7 (Activation)       (None, 56, 56, 64)   0           batch_normalization_9[0][0]      \n","__________________________________________________________________________________________________\n","conv2d_10 (Conv2D)              (None, 56, 56, 64)   36928       activation_7[0][0]               \n","__________________________________________________________________________________________________\n","batch_normalization_10 (BatchNo (None, 56, 56, 64)   256         conv2d_10[0][0]                  \n","__________________________________________________________________________________________________\n","activation_8 (Activation)       (None, 56, 56, 64)   0           batch_normalization_10[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_8 (Conv2D)               (None, 56, 56, 256)  16640       max_pooling2d[0][0]              \n","__________________________________________________________________________________________________\n","conv2d_11 (Conv2D)              (None, 56, 56, 256)  16640       activation_8[0][0]               \n","__________________________________________________________________________________________________\n","batch_normalization_8 (BatchNor (None, 56, 56, 256)  1024        conv2d_8[0][0]                   \n","__________________________________________________________________________________________________\n","batch_normalization_11 (BatchNo (None, 56, 56, 256)  1024        conv2d_11[0][0]                  \n","__________________________________________________________________________________________________\n","add_1 (Add)                     (None, 56, 56, 256)  0           batch_normalization_8[0][0]      \n","                                                                 batch_normalization_11[0][0]     \n","__________________________________________________________________________________________________\n","activation_9 (Activation)       (None, 56, 56, 256)  0           add_1[0][0]                      \n","==================================================================================================\n","Total params: 86,656\n","Trainable params: 85,248\n","Non-trainable params: 1,408\n","__________________________________________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"NlrvWvEEEoer"},"source":["## The stack of all residual blocks of conv2 stage\n","\n","Everything is going good so far. Recall from the network table that the conv2 stage for ResNet 50 has 3 residual blocks. Let's write a simple function to build all of them and combine them.\n","\n","This is also known as the stack of residual blocks."]},{"cell_type":"code","metadata":{"id":"aUIBpssiwIWf"},"source":["def residual_stack_v1(input, filters, blocks):\n","    net = input\n","    for i in range(blocks):\n","        net = residual_131_v1(net, filters)\n","    return net\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"v9tEE6LAFJsD"},"source":["If you have been paying attention, you may notice something interesting. \n","\n","\n","\n","* Output of first residual block is (56, 56, 256).\n","* Output of second residual block is also (56, 56, 256).\n","* Thus, for the second (and also the third) residual block, the number of channels in both input and output are same.\n","* Hence, the identity connection doesn't require any 1x1 convolutional layer.\n","* In fact, the first 1x1 conv layer in the second residual block actually reduces the number of channels from 256 back to 64 and the third layer increases it again to 256.\n","\n","Let's now build a partial resnet with both conv1 and conv2 stages complete.\n","\n"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"-Ab7h6GAw0Wk","executionInfo":{"status":"ok","timestamp":1607111055206,"user_tz":-330,"elapsed":3566,"user":{"displayName":"Shailesh Kumar","photoUrl":"","userId":"17557417109418259401"}},"outputId":"7a40b430-bea8-477a-931e-cba7731a660f"},"source":["def partial_resnet_v2(img_input):\n","  net = conv1(img_input)\n","  net = residual_stack_v1(net, 64, 3)\n","  return net\n","\n","model =  build_model((224, 224, 3), partial_resnet_v2)\n","model.summary()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Model: \"functional_7\"\n","__________________________________________________________________________________________________\n","Layer (type)                    Output Shape         Param #     Connected to                     \n","==================================================================================================\n","input_4 (InputLayer)            [(None, 224, 224, 3) 0                                            \n","__________________________________________________________________________________________________\n","zero_padding2d_2 (ZeroPadding2D (None, 230, 230, 3)  0           input_4[0][0]                    \n","__________________________________________________________________________________________________\n","conv2d_12 (Conv2D)              (None, 112, 112, 64) 9472        zero_padding2d_2[0][0]           \n","__________________________________________________________________________________________________\n","batch_normalization_12 (BatchNo (None, 112, 112, 64) 256         conv2d_12[0][0]                  \n","__________________________________________________________________________________________________\n","activation_10 (Activation)      (None, 112, 112, 64) 0           batch_normalization_12[0][0]     \n","__________________________________________________________________________________________________\n","zero_padding2d_3 (ZeroPadding2D (None, 114, 114, 64) 0           activation_10[0][0]              \n","__________________________________________________________________________________________________\n","max_pooling2d_1 (MaxPooling2D)  (None, 56, 56, 64)   0           zero_padding2d_3[0][0]           \n","__________________________________________________________________________________________________\n","conv2d_14 (Conv2D)              (None, 56, 56, 64)   4160        max_pooling2d_1[0][0]            \n","__________________________________________________________________________________________________\n","batch_normalization_14 (BatchNo (None, 56, 56, 64)   256         conv2d_14[0][0]                  \n","__________________________________________________________________________________________________\n","activation_11 (Activation)      (None, 56, 56, 64)   0           batch_normalization_14[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_15 (Conv2D)              (None, 56, 56, 64)   36928       activation_11[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_15 (BatchNo (None, 56, 56, 64)   256         conv2d_15[0][0]                  \n","__________________________________________________________________________________________________\n","activation_12 (Activation)      (None, 56, 56, 64)   0           batch_normalization_15[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_13 (Conv2D)              (None, 56, 56, 256)  16640       max_pooling2d_1[0][0]            \n","__________________________________________________________________________________________________\n","conv2d_16 (Conv2D)              (None, 56, 56, 256)  16640       activation_12[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_13 (BatchNo (None, 56, 56, 256)  1024        conv2d_13[0][0]                  \n","__________________________________________________________________________________________________\n","batch_normalization_16 (BatchNo (None, 56, 56, 256)  1024        conv2d_16[0][0]                  \n","__________________________________________________________________________________________________\n","add_2 (Add)                     (None, 56, 56, 256)  0           batch_normalization_13[0][0]     \n","                                                                 batch_normalization_16[0][0]     \n","__________________________________________________________________________________________________\n","activation_13 (Activation)      (None, 56, 56, 256)  0           add_2[0][0]                      \n","__________________________________________________________________________________________________\n","conv2d_17 (Conv2D)              (None, 56, 56, 64)   16448       activation_13[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_17 (BatchNo (None, 56, 56, 64)   256         conv2d_17[0][0]                  \n","__________________________________________________________________________________________________\n","activation_14 (Activation)      (None, 56, 56, 64)   0           batch_normalization_17[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_18 (Conv2D)              (None, 56, 56, 64)   36928       activation_14[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_18 (BatchNo (None, 56, 56, 64)   256         conv2d_18[0][0]                  \n","__________________________________________________________________________________________________\n","activation_15 (Activation)      (None, 56, 56, 64)   0           batch_normalization_18[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_19 (Conv2D)              (None, 56, 56, 256)  16640       activation_15[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_19 (BatchNo (None, 56, 56, 256)  1024        conv2d_19[0][0]                  \n","__________________________________________________________________________________________________\n","add_3 (Add)                     (None, 56, 56, 256)  0           activation_13[0][0]              \n","                                                                 batch_normalization_19[0][0]     \n","__________________________________________________________________________________________________\n","activation_16 (Activation)      (None, 56, 56, 256)  0           add_3[0][0]                      \n","__________________________________________________________________________________________________\n","conv2d_20 (Conv2D)              (None, 56, 56, 64)   16448       activation_16[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_20 (BatchNo (None, 56, 56, 64)   256         conv2d_20[0][0]                  \n","__________________________________________________________________________________________________\n","activation_17 (Activation)      (None, 56, 56, 64)   0           batch_normalization_20[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_21 (Conv2D)              (None, 56, 56, 64)   36928       activation_17[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_21 (BatchNo (None, 56, 56, 64)   256         conv2d_21[0][0]                  \n","__________________________________________________________________________________________________\n","activation_18 (Activation)      (None, 56, 56, 64)   0           batch_normalization_21[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_22 (Conv2D)              (None, 56, 56, 256)  16640       activation_18[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_22 (BatchNo (None, 56, 56, 256)  1024        conv2d_22[0][0]                  \n","__________________________________________________________________________________________________\n","add_4 (Add)                     (None, 56, 56, 256)  0           activation_16[0][0]              \n","                                                                 batch_normalization_22[0][0]     \n","__________________________________________________________________________________________________\n","activation_19 (Activation)      (None, 56, 56, 256)  0           add_4[0][0]                      \n","==================================================================================================\n","Total params: 229,760\n","Trainable params: 226,816\n","Non-trainable params: 2,944\n","__________________________________________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"kzjEQcn9GJWy"},"source":["## Size reduction from one stage to next stage\n","\n","Look back at the architecture table. conv2 stage has output size of (56, 56) but the conv3 stage works on a size of (28, 28). We need to perform a downsampling here. This is the job of the first convolution layer in the first residual block of a particular stage of the network (conv3, conv4, conv5). \n","\n","A little modification in the residual 131 block generation function achieves this. See the code below:\n","\n","* A new parameter stride1 has been introduced. This only applies to the first 1x1 convolution layer.\n","* If stride1=2, then the first 1x1 conv layer reduces the input size by a factor of 4 (2 in width, 2 in height).\n","* The other two conv layers remain as it is.\n","* This change also applies to the identity path. As the output size is halved, the 1x1 conv layer in the identity path also needs a stride of 2."]},{"cell_type":"code","metadata":{"id":"4YwOyuaNxik2"},"source":["def residual_131_v2(input, filters, stride1=1):\n","    # shape of input tensor\n","    input_shape = input.shape\n","    # number of channels in input tensor\n","    num_input_channels = input_shape[3]\n","    # number of channels in output tensor\n","    num_output_channels = 4 * filters\n","    # if input and output channels are same then we can feed\n","    # the input directly as identity shortcut\n","    # otherwise, we need to add a convolutional layer in identity path\n","    conv_in_identity_path = num_output_channels != num_input_channels\n","    if conv_in_identity_path is True:\n","        # add a conv layer to increase the number of channels\n","        shortcut = layers.Conv2D(num_output_channels, 1, strides=stride1)(input)\n","        # batch normalize (activation will come later)\n","        shortcut = layers.BatchNormalization(epsilon=BN_EPS)(shortcut)\n","    else:\n","        shortcut = input\n","    # The 1x1 first convolution layer\n","    net = layers.Conv2D(filters, 1, strides=stride1)(input)\n","    net = layers.BatchNormalization(epsilon=BN_EPS,)(net)\n","    net = layers.Activation('relu')(net)\n","    # The 3x3 second convolution layer\n","    net = layers.Conv2D(filters, 3, padding='same')(net)\n","    net = layers.BatchNormalization(epsilon=BN_EPS)(net)\n","    net = layers.Activation('relu')(net)\n","    # The 1x1 third convolution layer\n","    net = layers.Conv2D(num_output_channels, 1)(net)\n","    net = layers.BatchNormalization(epsilon=BN_EPS)(net)\n","    # Add identity shortcut to residual output before activation\n","    net = layers.Add()([shortcut, net])\n","    net = layers.Activation('relu')(net)\n","    return net\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"BmjGCKWTIE1M"},"source":["we need to modify our stack building function. The first block in the stack will have a stride of 2 (for conv3, conv4, conv5) layers and stride of 1 in conv2 layer. All other blocks in the stack will have a stride of 1."]},{"cell_type":"code","metadata":{"id":"IaD4re6hx3FV"},"source":["def residual_stack_v2(input, filters, blocks, stride1=2):\n","    net = input\n","    net = residual_131_v2(net, filters, stride1=stride1)\n","    for i in range(blocks-1):\n","        net = residual_131_v1(net, filters)\n","    return net\n"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"9KngLiWmIdPd"},"source":["## The complete ResNet 50 CNN\n","\n","We are now ready to build all the 5 stages of the ResNet 50 CNN. See the code below. The only thing missing is the classification layer on top of the CNN."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"n74ZKietyRZY","executionInfo":{"status":"ok","timestamp":1607111056808,"user_tz":-330,"elapsed":5154,"user":{"displayName":"Shailesh Kumar","photoUrl":"","userId":"17557417109418259401"}},"outputId":"b80c5d8a-9e5e-4e95-cc99-7f354fa2f540"},"source":["def cnn_resnet50(img_input):\n","  net = conv1(img_input)\n","  net = residual_stack_v2(net, 64, 3, stride1=1)\n","  net = residual_stack_v2(net, 128, 4)\n","  net = residual_stack_v2(net, 256, 6)\n","  net = residual_stack_v2(net, 512, 3)\n","  return net\n","\n","model =  build_model((224, 224, 3), cnn_resnet50)\n","model.summary()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Model: \"functional_9\"\n","__________________________________________________________________________________________________\n","Layer (type)                    Output Shape         Param #     Connected to                     \n","==================================================================================================\n","input_5 (InputLayer)            [(None, 224, 224, 3) 0                                            \n","__________________________________________________________________________________________________\n","zero_padding2d_4 (ZeroPadding2D (None, 230, 230, 3)  0           input_5[0][0]                    \n","__________________________________________________________________________________________________\n","conv2d_23 (Conv2D)              (None, 112, 112, 64) 9472        zero_padding2d_4[0][0]           \n","__________________________________________________________________________________________________\n","batch_normalization_23 (BatchNo (None, 112, 112, 64) 256         conv2d_23[0][0]                  \n","__________________________________________________________________________________________________\n","activation_20 (Activation)      (None, 112, 112, 64) 0           batch_normalization_23[0][0]     \n","__________________________________________________________________________________________________\n","zero_padding2d_5 (ZeroPadding2D (None, 114, 114, 64) 0           activation_20[0][0]              \n","__________________________________________________________________________________________________\n","max_pooling2d_2 (MaxPooling2D)  (None, 56, 56, 64)   0           zero_padding2d_5[0][0]           \n","__________________________________________________________________________________________________\n","conv2d_25 (Conv2D)              (None, 56, 56, 64)   4160        max_pooling2d_2[0][0]            \n","__________________________________________________________________________________________________\n","batch_normalization_25 (BatchNo (None, 56, 56, 64)   256         conv2d_25[0][0]                  \n","__________________________________________________________________________________________________\n","activation_21 (Activation)      (None, 56, 56, 64)   0           batch_normalization_25[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_26 (Conv2D)              (None, 56, 56, 64)   36928       activation_21[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_26 (BatchNo (None, 56, 56, 64)   256         conv2d_26[0][0]                  \n","__________________________________________________________________________________________________\n","activation_22 (Activation)      (None, 56, 56, 64)   0           batch_normalization_26[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_24 (Conv2D)              (None, 56, 56, 256)  16640       max_pooling2d_2[0][0]            \n","__________________________________________________________________________________________________\n","conv2d_27 (Conv2D)              (None, 56, 56, 256)  16640       activation_22[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_24 (BatchNo (None, 56, 56, 256)  1024        conv2d_24[0][0]                  \n","__________________________________________________________________________________________________\n","batch_normalization_27 (BatchNo (None, 56, 56, 256)  1024        conv2d_27[0][0]                  \n","__________________________________________________________________________________________________\n","add_5 (Add)                     (None, 56, 56, 256)  0           batch_normalization_24[0][0]     \n","                                                                 batch_normalization_27[0][0]     \n","__________________________________________________________________________________________________\n","activation_23 (Activation)      (None, 56, 56, 256)  0           add_5[0][0]                      \n","__________________________________________________________________________________________________\n","conv2d_28 (Conv2D)              (None, 56, 56, 64)   16448       activation_23[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_28 (BatchNo (None, 56, 56, 64)   256         conv2d_28[0][0]                  \n","__________________________________________________________________________________________________\n","activation_24 (Activation)      (None, 56, 56, 64)   0           batch_normalization_28[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_29 (Conv2D)              (None, 56, 56, 64)   36928       activation_24[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_29 (BatchNo (None, 56, 56, 64)   256         conv2d_29[0][0]                  \n","__________________________________________________________________________________________________\n","activation_25 (Activation)      (None, 56, 56, 64)   0           batch_normalization_29[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_30 (Conv2D)              (None, 56, 56, 256)  16640       activation_25[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_30 (BatchNo (None, 56, 56, 256)  1024        conv2d_30[0][0]                  \n","__________________________________________________________________________________________________\n","add_6 (Add)                     (None, 56, 56, 256)  0           activation_23[0][0]              \n","                                                                 batch_normalization_30[0][0]     \n","__________________________________________________________________________________________________\n","activation_26 (Activation)      (None, 56, 56, 256)  0           add_6[0][0]                      \n","__________________________________________________________________________________________________\n","conv2d_31 (Conv2D)              (None, 56, 56, 64)   16448       activation_26[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_31 (BatchNo (None, 56, 56, 64)   256         conv2d_31[0][0]                  \n","__________________________________________________________________________________________________\n","activation_27 (Activation)      (None, 56, 56, 64)   0           batch_normalization_31[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_32 (Conv2D)              (None, 56, 56, 64)   36928       activation_27[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_32 (BatchNo (None, 56, 56, 64)   256         conv2d_32[0][0]                  \n","__________________________________________________________________________________________________\n","activation_28 (Activation)      (None, 56, 56, 64)   0           batch_normalization_32[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_33 (Conv2D)              (None, 56, 56, 256)  16640       activation_28[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_33 (BatchNo (None, 56, 56, 256)  1024        conv2d_33[0][0]                  \n","__________________________________________________________________________________________________\n","add_7 (Add)                     (None, 56, 56, 256)  0           activation_26[0][0]              \n","                                                                 batch_normalization_33[0][0]     \n","__________________________________________________________________________________________________\n","activation_29 (Activation)      (None, 56, 56, 256)  0           add_7[0][0]                      \n","__________________________________________________________________________________________________\n","conv2d_35 (Conv2D)              (None, 28, 28, 128)  32896       activation_29[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_35 (BatchNo (None, 28, 28, 128)  512         conv2d_35[0][0]                  \n","__________________________________________________________________________________________________\n","activation_30 (Activation)      (None, 28, 28, 128)  0           batch_normalization_35[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_36 (Conv2D)              (None, 28, 28, 128)  147584      activation_30[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_36 (BatchNo (None, 28, 28, 128)  512         conv2d_36[0][0]                  \n","__________________________________________________________________________________________________\n","activation_31 (Activation)      (None, 28, 28, 128)  0           batch_normalization_36[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_34 (Conv2D)              (None, 28, 28, 512)  131584      activation_29[0][0]              \n","__________________________________________________________________________________________________\n","conv2d_37 (Conv2D)              (None, 28, 28, 512)  66048       activation_31[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_34 (BatchNo (None, 28, 28, 512)  2048        conv2d_34[0][0]                  \n","__________________________________________________________________________________________________\n","batch_normalization_37 (BatchNo (None, 28, 28, 512)  2048        conv2d_37[0][0]                  \n","__________________________________________________________________________________________________\n","add_8 (Add)                     (None, 28, 28, 512)  0           batch_normalization_34[0][0]     \n","                                                                 batch_normalization_37[0][0]     \n","__________________________________________________________________________________________________\n","activation_32 (Activation)      (None, 28, 28, 512)  0           add_8[0][0]                      \n","__________________________________________________________________________________________________\n","conv2d_38 (Conv2D)              (None, 28, 28, 128)  65664       activation_32[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_38 (BatchNo (None, 28, 28, 128)  512         conv2d_38[0][0]                  \n","__________________________________________________________________________________________________\n","activation_33 (Activation)      (None, 28, 28, 128)  0           batch_normalization_38[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_39 (Conv2D)              (None, 28, 28, 128)  147584      activation_33[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_39 (BatchNo (None, 28, 28, 128)  512         conv2d_39[0][0]                  \n","__________________________________________________________________________________________________\n","activation_34 (Activation)      (None, 28, 28, 128)  0           batch_normalization_39[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_40 (Conv2D)              (None, 28, 28, 512)  66048       activation_34[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_40 (BatchNo (None, 28, 28, 512)  2048        conv2d_40[0][0]                  \n","__________________________________________________________________________________________________\n","add_9 (Add)                     (None, 28, 28, 512)  0           activation_32[0][0]              \n","                                                                 batch_normalization_40[0][0]     \n","__________________________________________________________________________________________________\n","activation_35 (Activation)      (None, 28, 28, 512)  0           add_9[0][0]                      \n","__________________________________________________________________________________________________\n","conv2d_41 (Conv2D)              (None, 28, 28, 128)  65664       activation_35[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_41 (BatchNo (None, 28, 28, 128)  512         conv2d_41[0][0]                  \n","__________________________________________________________________________________________________\n","activation_36 (Activation)      (None, 28, 28, 128)  0           batch_normalization_41[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_42 (Conv2D)              (None, 28, 28, 128)  147584      activation_36[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_42 (BatchNo (None, 28, 28, 128)  512         conv2d_42[0][0]                  \n","__________________________________________________________________________________________________\n","activation_37 (Activation)      (None, 28, 28, 128)  0           batch_normalization_42[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_43 (Conv2D)              (None, 28, 28, 512)  66048       activation_37[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_43 (BatchNo (None, 28, 28, 512)  2048        conv2d_43[0][0]                  \n","__________________________________________________________________________________________________\n","add_10 (Add)                    (None, 28, 28, 512)  0           activation_35[0][0]              \n","                                                                 batch_normalization_43[0][0]     \n","__________________________________________________________________________________________________\n","activation_38 (Activation)      (None, 28, 28, 512)  0           add_10[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_44 (Conv2D)              (None, 28, 28, 128)  65664       activation_38[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_44 (BatchNo (None, 28, 28, 128)  512         conv2d_44[0][0]                  \n","__________________________________________________________________________________________________\n","activation_39 (Activation)      (None, 28, 28, 128)  0           batch_normalization_44[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_45 (Conv2D)              (None, 28, 28, 128)  147584      activation_39[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_45 (BatchNo (None, 28, 28, 128)  512         conv2d_45[0][0]                  \n","__________________________________________________________________________________________________\n","activation_40 (Activation)      (None, 28, 28, 128)  0           batch_normalization_45[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_46 (Conv2D)              (None, 28, 28, 512)  66048       activation_40[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_46 (BatchNo (None, 28, 28, 512)  2048        conv2d_46[0][0]                  \n","__________________________________________________________________________________________________\n","add_11 (Add)                    (None, 28, 28, 512)  0           activation_38[0][0]              \n","                                                                 batch_normalization_46[0][0]     \n","__________________________________________________________________________________________________\n","activation_41 (Activation)      (None, 28, 28, 512)  0           add_11[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_48 (Conv2D)              (None, 14, 14, 256)  131328      activation_41[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_48 (BatchNo (None, 14, 14, 256)  1024        conv2d_48[0][0]                  \n","__________________________________________________________________________________________________\n","activation_42 (Activation)      (None, 14, 14, 256)  0           batch_normalization_48[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_49 (Conv2D)              (None, 14, 14, 256)  590080      activation_42[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_49 (BatchNo (None, 14, 14, 256)  1024        conv2d_49[0][0]                  \n","__________________________________________________________________________________________________\n","activation_43 (Activation)      (None, 14, 14, 256)  0           batch_normalization_49[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_47 (Conv2D)              (None, 14, 14, 1024) 525312      activation_41[0][0]              \n","__________________________________________________________________________________________________\n","conv2d_50 (Conv2D)              (None, 14, 14, 1024) 263168      activation_43[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_47 (BatchNo (None, 14, 14, 1024) 4096        conv2d_47[0][0]                  \n","__________________________________________________________________________________________________\n","batch_normalization_50 (BatchNo (None, 14, 14, 1024) 4096        conv2d_50[0][0]                  \n","__________________________________________________________________________________________________\n","add_12 (Add)                    (None, 14, 14, 1024) 0           batch_normalization_47[0][0]     \n","                                                                 batch_normalization_50[0][0]     \n","__________________________________________________________________________________________________\n","activation_44 (Activation)      (None, 14, 14, 1024) 0           add_12[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_51 (Conv2D)              (None, 14, 14, 256)  262400      activation_44[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_51 (BatchNo (None, 14, 14, 256)  1024        conv2d_51[0][0]                  \n","__________________________________________________________________________________________________\n","activation_45 (Activation)      (None, 14, 14, 256)  0           batch_normalization_51[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_52 (Conv2D)              (None, 14, 14, 256)  590080      activation_45[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_52 (BatchNo (None, 14, 14, 256)  1024        conv2d_52[0][0]                  \n","__________________________________________________________________________________________________\n","activation_46 (Activation)      (None, 14, 14, 256)  0           batch_normalization_52[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_53 (Conv2D)              (None, 14, 14, 1024) 263168      activation_46[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_53 (BatchNo (None, 14, 14, 1024) 4096        conv2d_53[0][0]                  \n","__________________________________________________________________________________________________\n","add_13 (Add)                    (None, 14, 14, 1024) 0           activation_44[0][0]              \n","                                                                 batch_normalization_53[0][0]     \n","__________________________________________________________________________________________________\n","activation_47 (Activation)      (None, 14, 14, 1024) 0           add_13[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_54 (Conv2D)              (None, 14, 14, 256)  262400      activation_47[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_54 (BatchNo (None, 14, 14, 256)  1024        conv2d_54[0][0]                  \n","__________________________________________________________________________________________________\n","activation_48 (Activation)      (None, 14, 14, 256)  0           batch_normalization_54[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_55 (Conv2D)              (None, 14, 14, 256)  590080      activation_48[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_55 (BatchNo (None, 14, 14, 256)  1024        conv2d_55[0][0]                  \n","__________________________________________________________________________________________________\n","activation_49 (Activation)      (None, 14, 14, 256)  0           batch_normalization_55[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_56 (Conv2D)              (None, 14, 14, 1024) 263168      activation_49[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_56 (BatchNo (None, 14, 14, 1024) 4096        conv2d_56[0][0]                  \n","__________________________________________________________________________________________________\n","add_14 (Add)                    (None, 14, 14, 1024) 0           activation_47[0][0]              \n","                                                                 batch_normalization_56[0][0]     \n","__________________________________________________________________________________________________\n","activation_50 (Activation)      (None, 14, 14, 1024) 0           add_14[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_57 (Conv2D)              (None, 14, 14, 256)  262400      activation_50[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_57 (BatchNo (None, 14, 14, 256)  1024        conv2d_57[0][0]                  \n","__________________________________________________________________________________________________\n","activation_51 (Activation)      (None, 14, 14, 256)  0           batch_normalization_57[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_58 (Conv2D)              (None, 14, 14, 256)  590080      activation_51[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_58 (BatchNo (None, 14, 14, 256)  1024        conv2d_58[0][0]                  \n","__________________________________________________________________________________________________\n","activation_52 (Activation)      (None, 14, 14, 256)  0           batch_normalization_58[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_59 (Conv2D)              (None, 14, 14, 1024) 263168      activation_52[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_59 (BatchNo (None, 14, 14, 1024) 4096        conv2d_59[0][0]                  \n","__________________________________________________________________________________________________\n","add_15 (Add)                    (None, 14, 14, 1024) 0           activation_50[0][0]              \n","                                                                 batch_normalization_59[0][0]     \n","__________________________________________________________________________________________________\n","activation_53 (Activation)      (None, 14, 14, 1024) 0           add_15[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_60 (Conv2D)              (None, 14, 14, 256)  262400      activation_53[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_60 (BatchNo (None, 14, 14, 256)  1024        conv2d_60[0][0]                  \n","__________________________________________________________________________________________________\n","activation_54 (Activation)      (None, 14, 14, 256)  0           batch_normalization_60[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_61 (Conv2D)              (None, 14, 14, 256)  590080      activation_54[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_61 (BatchNo (None, 14, 14, 256)  1024        conv2d_61[0][0]                  \n","__________________________________________________________________________________________________\n","activation_55 (Activation)      (None, 14, 14, 256)  0           batch_normalization_61[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_62 (Conv2D)              (None, 14, 14, 1024) 263168      activation_55[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_62 (BatchNo (None, 14, 14, 1024) 4096        conv2d_62[0][0]                  \n","__________________________________________________________________________________________________\n","add_16 (Add)                    (None, 14, 14, 1024) 0           activation_53[0][0]              \n","                                                                 batch_normalization_62[0][0]     \n","__________________________________________________________________________________________________\n","activation_56 (Activation)      (None, 14, 14, 1024) 0           add_16[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_63 (Conv2D)              (None, 14, 14, 256)  262400      activation_56[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_63 (BatchNo (None, 14, 14, 256)  1024        conv2d_63[0][0]                  \n","__________________________________________________________________________________________________\n","activation_57 (Activation)      (None, 14, 14, 256)  0           batch_normalization_63[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_64 (Conv2D)              (None, 14, 14, 256)  590080      activation_57[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_64 (BatchNo (None, 14, 14, 256)  1024        conv2d_64[0][0]                  \n","__________________________________________________________________________________________________\n","activation_58 (Activation)      (None, 14, 14, 256)  0           batch_normalization_64[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_65 (Conv2D)              (None, 14, 14, 1024) 263168      activation_58[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_65 (BatchNo (None, 14, 14, 1024) 4096        conv2d_65[0][0]                  \n","__________________________________________________________________________________________________\n","add_17 (Add)                    (None, 14, 14, 1024) 0           activation_56[0][0]              \n","                                                                 batch_normalization_65[0][0]     \n","__________________________________________________________________________________________________\n","activation_59 (Activation)      (None, 14, 14, 1024) 0           add_17[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_67 (Conv2D)              (None, 7, 7, 512)    524800      activation_59[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_67 (BatchNo (None, 7, 7, 512)    2048        conv2d_67[0][0]                  \n","__________________________________________________________________________________________________\n","activation_60 (Activation)      (None, 7, 7, 512)    0           batch_normalization_67[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_68 (Conv2D)              (None, 7, 7, 512)    2359808     activation_60[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_68 (BatchNo (None, 7, 7, 512)    2048        conv2d_68[0][0]                  \n","__________________________________________________________________________________________________\n","activation_61 (Activation)      (None, 7, 7, 512)    0           batch_normalization_68[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_66 (Conv2D)              (None, 7, 7, 2048)   2099200     activation_59[0][0]              \n","__________________________________________________________________________________________________\n","conv2d_69 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_61[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_66 (BatchNo (None, 7, 7, 2048)   8192        conv2d_66[0][0]                  \n","__________________________________________________________________________________________________\n","batch_normalization_69 (BatchNo (None, 7, 7, 2048)   8192        conv2d_69[0][0]                  \n","__________________________________________________________________________________________________\n","add_18 (Add)                    (None, 7, 7, 2048)   0           batch_normalization_66[0][0]     \n","                                                                 batch_normalization_69[0][0]     \n","__________________________________________________________________________________________________\n","activation_62 (Activation)      (None, 7, 7, 2048)   0           add_18[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_70 (Conv2D)              (None, 7, 7, 512)    1049088     activation_62[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_70 (BatchNo (None, 7, 7, 512)    2048        conv2d_70[0][0]                  \n","__________________________________________________________________________________________________\n","activation_63 (Activation)      (None, 7, 7, 512)    0           batch_normalization_70[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_71 (Conv2D)              (None, 7, 7, 512)    2359808     activation_63[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_71 (BatchNo (None, 7, 7, 512)    2048        conv2d_71[0][0]                  \n","__________________________________________________________________________________________________\n","activation_64 (Activation)      (None, 7, 7, 512)    0           batch_normalization_71[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_72 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_64[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_72 (BatchNo (None, 7, 7, 2048)   8192        conv2d_72[0][0]                  \n","__________________________________________________________________________________________________\n","add_19 (Add)                    (None, 7, 7, 2048)   0           activation_62[0][0]              \n","                                                                 batch_normalization_72[0][0]     \n","__________________________________________________________________________________________________\n","activation_65 (Activation)      (None, 7, 7, 2048)   0           add_19[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_73 (Conv2D)              (None, 7, 7, 512)    1049088     activation_65[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_73 (BatchNo (None, 7, 7, 512)    2048        conv2d_73[0][0]                  \n","__________________________________________________________________________________________________\n","activation_66 (Activation)      (None, 7, 7, 512)    0           batch_normalization_73[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_74 (Conv2D)              (None, 7, 7, 512)    2359808     activation_66[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_74 (BatchNo (None, 7, 7, 512)    2048        conv2d_74[0][0]                  \n","__________________________________________________________________________________________________\n","activation_67 (Activation)      (None, 7, 7, 512)    0           batch_normalization_74[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_75 (Conv2D)              (None, 7, 7, 2048)   1050624     activation_67[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_75 (BatchNo (None, 7, 7, 2048)   8192        conv2d_75[0][0]                  \n","__________________________________________________________________________________________________\n","add_20 (Add)                    (None, 7, 7, 2048)   0           activation_65[0][0]              \n","                                                                 batch_normalization_75[0][0]     \n","__________________________________________________________________________________________________\n","activation_68 (Activation)      (None, 7, 7, 2048)   0           add_20[0][0]                     \n","==================================================================================================\n","Total params: 23,587,712\n","Trainable params: 23,534,592\n","Non-trainable params: 53,120\n","__________________________________________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"kFZZ3kBfJcjJ"},"source":["## The classification layer.\n","\n","It's important to keep the classification layer separate (called the top in Keras docs). The CNN part can work on images of larger sizes too. A (224,224,3) input image size is necessary only if the network is being used for classification purposes. For transfer learning, people often load the CNN (without the top classification layer) with pre-trained weights and train a different classification layer on top of it.\n","\n","Following is the top classification layer for Res Nets. If the pretrained weights for ImageNet classification are to be loaded, then the number of classes will be 1000.\n","\n","\n","* The output of the CNN is of size (7,7,2048). \n","* There is a simple global average pooling reducing the size to a vector of 2048 length.\n","* This is followed by a dense layer with 1000 outputs. (2048 * 1000 + 1000 weights).\n","* The output of dense layer goes through a softmax activation to convert it to classification probablities. \n","\n"]},{"cell_type":"code","metadata":{"id":"qDoEiY5JzGDi"},"source":["def top_resnet(net, classes=1000):\n","    # add the top classification network\n","    net = layers.GlobalAveragePooling2D()(net)\n","    net = layers.Dense(classes, activation='softmax')(net)\n","    return net"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"noiJEIdPKmLb"},"source":["We now combine the CNN with the classifier to build the overall ResNet 50 classifier.  Build a model around it. \n","\n","Voila! It's done. We have a complete ResNet 50 architecture built for us."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"gnFoTKDQzYCa","executionInfo":{"status":"ok","timestamp":1607111058429,"user_tz":-330,"elapsed":6767,"user":{"displayName":"Shailesh Kumar","photoUrl":"","userId":"17557417109418259401"}},"outputId":"70e53508-806a-4657-fd57-a81f14ec5213"},"source":["def resnet_classifier(img_input):\n","  net = cnn_resnet50 (img_input)\n","  net = top_resnet(net)\n","  return net\n","  \n","model =  build_model((224, 224, 3), resnet_classifier)\n","model.summary()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Model: \"functional_11\"\n","__________________________________________________________________________________________________\n","Layer (type)                    Output Shape         Param #     Connected to                     \n","==================================================================================================\n","input_6 (InputLayer)            [(None, 224, 224, 3) 0                                            \n","__________________________________________________________________________________________________\n","zero_padding2d_6 (ZeroPadding2D (None, 230, 230, 3)  0           input_6[0][0]                    \n","__________________________________________________________________________________________________\n","conv2d_76 (Conv2D)              (None, 112, 112, 64) 9472        zero_padding2d_6[0][0]           \n","__________________________________________________________________________________________________\n","batch_normalization_76 (BatchNo (None, 112, 112, 64) 256         conv2d_76[0][0]                  \n","__________________________________________________________________________________________________\n","activation_69 (Activation)      (None, 112, 112, 64) 0           batch_normalization_76[0][0]     \n","__________________________________________________________________________________________________\n","zero_padding2d_7 (ZeroPadding2D (None, 114, 114, 64) 0           activation_69[0][0]              \n","__________________________________________________________________________________________________\n","max_pooling2d_3 (MaxPooling2D)  (None, 56, 56, 64)   0           zero_padding2d_7[0][0]           \n","__________________________________________________________________________________________________\n","conv2d_78 (Conv2D)              (None, 56, 56, 64)   4160        max_pooling2d_3[0][0]            \n","__________________________________________________________________________________________________\n","batch_normalization_78 (BatchNo (None, 56, 56, 64)   256         conv2d_78[0][0]                  \n","__________________________________________________________________________________________________\n","activation_70 (Activation)      (None, 56, 56, 64)   0           batch_normalization_78[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_79 (Conv2D)              (None, 56, 56, 64)   36928       activation_70[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_79 (BatchNo (None, 56, 56, 64)   256         conv2d_79[0][0]                  \n","__________________________________________________________________________________________________\n","activation_71 (Activation)      (None, 56, 56, 64)   0           batch_normalization_79[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_77 (Conv2D)              (None, 56, 56, 256)  16640       max_pooling2d_3[0][0]            \n","__________________________________________________________________________________________________\n","conv2d_80 (Conv2D)              (None, 56, 56, 256)  16640       activation_71[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_77 (BatchNo (None, 56, 56, 256)  1024        conv2d_77[0][0]                  \n","__________________________________________________________________________________________________\n","batch_normalization_80 (BatchNo (None, 56, 56, 256)  1024        conv2d_80[0][0]                  \n","__________________________________________________________________________________________________\n","add_21 (Add)                    (None, 56, 56, 256)  0           batch_normalization_77[0][0]     \n","                                                                 batch_normalization_80[0][0]     \n","__________________________________________________________________________________________________\n","activation_72 (Activation)      (None, 56, 56, 256)  0           add_21[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_81 (Conv2D)              (None, 56, 56, 64)   16448       activation_72[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_81 (BatchNo (None, 56, 56, 64)   256         conv2d_81[0][0]                  \n","__________________________________________________________________________________________________\n","activation_73 (Activation)      (None, 56, 56, 64)   0           batch_normalization_81[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_82 (Conv2D)              (None, 56, 56, 64)   36928       activation_73[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_82 (BatchNo (None, 56, 56, 64)   256         conv2d_82[0][0]                  \n","__________________________________________________________________________________________________\n","activation_74 (Activation)      (None, 56, 56, 64)   0           batch_normalization_82[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_83 (Conv2D)              (None, 56, 56, 256)  16640       activation_74[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_83 (BatchNo (None, 56, 56, 256)  1024        conv2d_83[0][0]                  \n","__________________________________________________________________________________________________\n","add_22 (Add)                    (None, 56, 56, 256)  0           activation_72[0][0]              \n","                                                                 batch_normalization_83[0][0]     \n","__________________________________________________________________________________________________\n","activation_75 (Activation)      (None, 56, 56, 256)  0           add_22[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_84 (Conv2D)              (None, 56, 56, 64)   16448       activation_75[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_84 (BatchNo (None, 56, 56, 64)   256         conv2d_84[0][0]                  \n","__________________________________________________________________________________________________\n","activation_76 (Activation)      (None, 56, 56, 64)   0           batch_normalization_84[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_85 (Conv2D)              (None, 56, 56, 64)   36928       activation_76[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_85 (BatchNo (None, 56, 56, 64)   256         conv2d_85[0][0]                  \n","__________________________________________________________________________________________________\n","activation_77 (Activation)      (None, 56, 56, 64)   0           batch_normalization_85[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_86 (Conv2D)              (None, 56, 56, 256)  16640       activation_77[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_86 (BatchNo (None, 56, 56, 256)  1024        conv2d_86[0][0]                  \n","__________________________________________________________________________________________________\n","add_23 (Add)                    (None, 56, 56, 256)  0           activation_75[0][0]              \n","                                                                 batch_normalization_86[0][0]     \n","__________________________________________________________________________________________________\n","activation_78 (Activation)      (None, 56, 56, 256)  0           add_23[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_88 (Conv2D)              (None, 28, 28, 128)  32896       activation_78[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_88 (BatchNo (None, 28, 28, 128)  512         conv2d_88[0][0]                  \n","__________________________________________________________________________________________________\n","activation_79 (Activation)      (None, 28, 28, 128)  0           batch_normalization_88[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_89 (Conv2D)              (None, 28, 28, 128)  147584      activation_79[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_89 (BatchNo (None, 28, 28, 128)  512         conv2d_89[0][0]                  \n","__________________________________________________________________________________________________\n","activation_80 (Activation)      (None, 28, 28, 128)  0           batch_normalization_89[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_87 (Conv2D)              (None, 28, 28, 512)  131584      activation_78[0][0]              \n","__________________________________________________________________________________________________\n","conv2d_90 (Conv2D)              (None, 28, 28, 512)  66048       activation_80[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_87 (BatchNo (None, 28, 28, 512)  2048        conv2d_87[0][0]                  \n","__________________________________________________________________________________________________\n","batch_normalization_90 (BatchNo (None, 28, 28, 512)  2048        conv2d_90[0][0]                  \n","__________________________________________________________________________________________________\n","add_24 (Add)                    (None, 28, 28, 512)  0           batch_normalization_87[0][0]     \n","                                                                 batch_normalization_90[0][0]     \n","__________________________________________________________________________________________________\n","activation_81 (Activation)      (None, 28, 28, 512)  0           add_24[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_91 (Conv2D)              (None, 28, 28, 128)  65664       activation_81[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_91 (BatchNo (None, 28, 28, 128)  512         conv2d_91[0][0]                  \n","__________________________________________________________________________________________________\n","activation_82 (Activation)      (None, 28, 28, 128)  0           batch_normalization_91[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_92 (Conv2D)              (None, 28, 28, 128)  147584      activation_82[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_92 (BatchNo (None, 28, 28, 128)  512         conv2d_92[0][0]                  \n","__________________________________________________________________________________________________\n","activation_83 (Activation)      (None, 28, 28, 128)  0           batch_normalization_92[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_93 (Conv2D)              (None, 28, 28, 512)  66048       activation_83[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_93 (BatchNo (None, 28, 28, 512)  2048        conv2d_93[0][0]                  \n","__________________________________________________________________________________________________\n","add_25 (Add)                    (None, 28, 28, 512)  0           activation_81[0][0]              \n","                                                                 batch_normalization_93[0][0]     \n","__________________________________________________________________________________________________\n","activation_84 (Activation)      (None, 28, 28, 512)  0           add_25[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_94 (Conv2D)              (None, 28, 28, 128)  65664       activation_84[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_94 (BatchNo (None, 28, 28, 128)  512         conv2d_94[0][0]                  \n","__________________________________________________________________________________________________\n","activation_85 (Activation)      (None, 28, 28, 128)  0           batch_normalization_94[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_95 (Conv2D)              (None, 28, 28, 128)  147584      activation_85[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_95 (BatchNo (None, 28, 28, 128)  512         conv2d_95[0][0]                  \n","__________________________________________________________________________________________________\n","activation_86 (Activation)      (None, 28, 28, 128)  0           batch_normalization_95[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_96 (Conv2D)              (None, 28, 28, 512)  66048       activation_86[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_96 (BatchNo (None, 28, 28, 512)  2048        conv2d_96[0][0]                  \n","__________________________________________________________________________________________________\n","add_26 (Add)                    (None, 28, 28, 512)  0           activation_84[0][0]              \n","                                                                 batch_normalization_96[0][0]     \n","__________________________________________________________________________________________________\n","activation_87 (Activation)      (None, 28, 28, 512)  0           add_26[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_97 (Conv2D)              (None, 28, 28, 128)  65664       activation_87[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_97 (BatchNo (None, 28, 28, 128)  512         conv2d_97[0][0]                  \n","__________________________________________________________________________________________________\n","activation_88 (Activation)      (None, 28, 28, 128)  0           batch_normalization_97[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_98 (Conv2D)              (None, 28, 28, 128)  147584      activation_88[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_98 (BatchNo (None, 28, 28, 128)  512         conv2d_98[0][0]                  \n","__________________________________________________________________________________________________\n","activation_89 (Activation)      (None, 28, 28, 128)  0           batch_normalization_98[0][0]     \n","__________________________________________________________________________________________________\n","conv2d_99 (Conv2D)              (None, 28, 28, 512)  66048       activation_89[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_99 (BatchNo (None, 28, 28, 512)  2048        conv2d_99[0][0]                  \n","__________________________________________________________________________________________________\n","add_27 (Add)                    (None, 28, 28, 512)  0           activation_87[0][0]              \n","                                                                 batch_normalization_99[0][0]     \n","__________________________________________________________________________________________________\n","activation_90 (Activation)      (None, 28, 28, 512)  0           add_27[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_101 (Conv2D)             (None, 14, 14, 256)  131328      activation_90[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_101 (BatchN (None, 14, 14, 256)  1024        conv2d_101[0][0]                 \n","__________________________________________________________________________________________________\n","activation_91 (Activation)      (None, 14, 14, 256)  0           batch_normalization_101[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_102 (Conv2D)             (None, 14, 14, 256)  590080      activation_91[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_102 (BatchN (None, 14, 14, 256)  1024        conv2d_102[0][0]                 \n","__________________________________________________________________________________________________\n","activation_92 (Activation)      (None, 14, 14, 256)  0           batch_normalization_102[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_100 (Conv2D)             (None, 14, 14, 1024) 525312      activation_90[0][0]              \n","__________________________________________________________________________________________________\n","conv2d_103 (Conv2D)             (None, 14, 14, 1024) 263168      activation_92[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_100 (BatchN (None, 14, 14, 1024) 4096        conv2d_100[0][0]                 \n","__________________________________________________________________________________________________\n","batch_normalization_103 (BatchN (None, 14, 14, 1024) 4096        conv2d_103[0][0]                 \n","__________________________________________________________________________________________________\n","add_28 (Add)                    (None, 14, 14, 1024) 0           batch_normalization_100[0][0]    \n","                                                                 batch_normalization_103[0][0]    \n","__________________________________________________________________________________________________\n","activation_93 (Activation)      (None, 14, 14, 1024) 0           add_28[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_104 (Conv2D)             (None, 14, 14, 256)  262400      activation_93[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_104 (BatchN (None, 14, 14, 256)  1024        conv2d_104[0][0]                 \n","__________________________________________________________________________________________________\n","activation_94 (Activation)      (None, 14, 14, 256)  0           batch_normalization_104[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_105 (Conv2D)             (None, 14, 14, 256)  590080      activation_94[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_105 (BatchN (None, 14, 14, 256)  1024        conv2d_105[0][0]                 \n","__________________________________________________________________________________________________\n","activation_95 (Activation)      (None, 14, 14, 256)  0           batch_normalization_105[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_106 (Conv2D)             (None, 14, 14, 1024) 263168      activation_95[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_106 (BatchN (None, 14, 14, 1024) 4096        conv2d_106[0][0]                 \n","__________________________________________________________________________________________________\n","add_29 (Add)                    (None, 14, 14, 1024) 0           activation_93[0][0]              \n","                                                                 batch_normalization_106[0][0]    \n","__________________________________________________________________________________________________\n","activation_96 (Activation)      (None, 14, 14, 1024) 0           add_29[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_107 (Conv2D)             (None, 14, 14, 256)  262400      activation_96[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_107 (BatchN (None, 14, 14, 256)  1024        conv2d_107[0][0]                 \n","__________________________________________________________________________________________________\n","activation_97 (Activation)      (None, 14, 14, 256)  0           batch_normalization_107[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_108 (Conv2D)             (None, 14, 14, 256)  590080      activation_97[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_108 (BatchN (None, 14, 14, 256)  1024        conv2d_108[0][0]                 \n","__________________________________________________________________________________________________\n","activation_98 (Activation)      (None, 14, 14, 256)  0           batch_normalization_108[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_109 (Conv2D)             (None, 14, 14, 1024) 263168      activation_98[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_109 (BatchN (None, 14, 14, 1024) 4096        conv2d_109[0][0]                 \n","__________________________________________________________________________________________________\n","add_30 (Add)                    (None, 14, 14, 1024) 0           activation_96[0][0]              \n","                                                                 batch_normalization_109[0][0]    \n","__________________________________________________________________________________________________\n","activation_99 (Activation)      (None, 14, 14, 1024) 0           add_30[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_110 (Conv2D)             (None, 14, 14, 256)  262400      activation_99[0][0]              \n","__________________________________________________________________________________________________\n","batch_normalization_110 (BatchN (None, 14, 14, 256)  1024        conv2d_110[0][0]                 \n","__________________________________________________________________________________________________\n","activation_100 (Activation)     (None, 14, 14, 256)  0           batch_normalization_110[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_111 (Conv2D)             (None, 14, 14, 256)  590080      activation_100[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_111 (BatchN (None, 14, 14, 256)  1024        conv2d_111[0][0]                 \n","__________________________________________________________________________________________________\n","activation_101 (Activation)     (None, 14, 14, 256)  0           batch_normalization_111[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_112 (Conv2D)             (None, 14, 14, 1024) 263168      activation_101[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_112 (BatchN (None, 14, 14, 1024) 4096        conv2d_112[0][0]                 \n","__________________________________________________________________________________________________\n","add_31 (Add)                    (None, 14, 14, 1024) 0           activation_99[0][0]              \n","                                                                 batch_normalization_112[0][0]    \n","__________________________________________________________________________________________________\n","activation_102 (Activation)     (None, 14, 14, 1024) 0           add_31[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_113 (Conv2D)             (None, 14, 14, 256)  262400      activation_102[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_113 (BatchN (None, 14, 14, 256)  1024        conv2d_113[0][0]                 \n","__________________________________________________________________________________________________\n","activation_103 (Activation)     (None, 14, 14, 256)  0           batch_normalization_113[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_114 (Conv2D)             (None, 14, 14, 256)  590080      activation_103[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_114 (BatchN (None, 14, 14, 256)  1024        conv2d_114[0][0]                 \n","__________________________________________________________________________________________________\n","activation_104 (Activation)     (None, 14, 14, 256)  0           batch_normalization_114[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_115 (Conv2D)             (None, 14, 14, 1024) 263168      activation_104[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_115 (BatchN (None, 14, 14, 1024) 4096        conv2d_115[0][0]                 \n","__________________________________________________________________________________________________\n","add_32 (Add)                    (None, 14, 14, 1024) 0           activation_102[0][0]             \n","                                                                 batch_normalization_115[0][0]    \n","__________________________________________________________________________________________________\n","activation_105 (Activation)     (None, 14, 14, 1024) 0           add_32[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_116 (Conv2D)             (None, 14, 14, 256)  262400      activation_105[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_116 (BatchN (None, 14, 14, 256)  1024        conv2d_116[0][0]                 \n","__________________________________________________________________________________________________\n","activation_106 (Activation)     (None, 14, 14, 256)  0           batch_normalization_116[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_117 (Conv2D)             (None, 14, 14, 256)  590080      activation_106[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_117 (BatchN (None, 14, 14, 256)  1024        conv2d_117[0][0]                 \n","__________________________________________________________________________________________________\n","activation_107 (Activation)     (None, 14, 14, 256)  0           batch_normalization_117[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_118 (Conv2D)             (None, 14, 14, 1024) 263168      activation_107[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_118 (BatchN (None, 14, 14, 1024) 4096        conv2d_118[0][0]                 \n","__________________________________________________________________________________________________\n","add_33 (Add)                    (None, 14, 14, 1024) 0           activation_105[0][0]             \n","                                                                 batch_normalization_118[0][0]    \n","__________________________________________________________________________________________________\n","activation_108 (Activation)     (None, 14, 14, 1024) 0           add_33[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_120 (Conv2D)             (None, 7, 7, 512)    524800      activation_108[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_120 (BatchN (None, 7, 7, 512)    2048        conv2d_120[0][0]                 \n","__________________________________________________________________________________________________\n","activation_109 (Activation)     (None, 7, 7, 512)    0           batch_normalization_120[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_121 (Conv2D)             (None, 7, 7, 512)    2359808     activation_109[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_121 (BatchN (None, 7, 7, 512)    2048        conv2d_121[0][0]                 \n","__________________________________________________________________________________________________\n","activation_110 (Activation)     (None, 7, 7, 512)    0           batch_normalization_121[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_119 (Conv2D)             (None, 7, 7, 2048)   2099200     activation_108[0][0]             \n","__________________________________________________________________________________________________\n","conv2d_122 (Conv2D)             (None, 7, 7, 2048)   1050624     activation_110[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_119 (BatchN (None, 7, 7, 2048)   8192        conv2d_119[0][0]                 \n","__________________________________________________________________________________________________\n","batch_normalization_122 (BatchN (None, 7, 7, 2048)   8192        conv2d_122[0][0]                 \n","__________________________________________________________________________________________________\n","add_34 (Add)                    (None, 7, 7, 2048)   0           batch_normalization_119[0][0]    \n","                                                                 batch_normalization_122[0][0]    \n","__________________________________________________________________________________________________\n","activation_111 (Activation)     (None, 7, 7, 2048)   0           add_34[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_123 (Conv2D)             (None, 7, 7, 512)    1049088     activation_111[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_123 (BatchN (None, 7, 7, 512)    2048        conv2d_123[0][0]                 \n","__________________________________________________________________________________________________\n","activation_112 (Activation)     (None, 7, 7, 512)    0           batch_normalization_123[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_124 (Conv2D)             (None, 7, 7, 512)    2359808     activation_112[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_124 (BatchN (None, 7, 7, 512)    2048        conv2d_124[0][0]                 \n","__________________________________________________________________________________________________\n","activation_113 (Activation)     (None, 7, 7, 512)    0           batch_normalization_124[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_125 (Conv2D)             (None, 7, 7, 2048)   1050624     activation_113[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_125 (BatchN (None, 7, 7, 2048)   8192        conv2d_125[0][0]                 \n","__________________________________________________________________________________________________\n","add_35 (Add)                    (None, 7, 7, 2048)   0           activation_111[0][0]             \n","                                                                 batch_normalization_125[0][0]    \n","__________________________________________________________________________________________________\n","activation_114 (Activation)     (None, 7, 7, 2048)   0           add_35[0][0]                     \n","__________________________________________________________________________________________________\n","conv2d_126 (Conv2D)             (None, 7, 7, 512)    1049088     activation_114[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_126 (BatchN (None, 7, 7, 512)    2048        conv2d_126[0][0]                 \n","__________________________________________________________________________________________________\n","activation_115 (Activation)     (None, 7, 7, 512)    0           batch_normalization_126[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_127 (Conv2D)             (None, 7, 7, 512)    2359808     activation_115[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_127 (BatchN (None, 7, 7, 512)    2048        conv2d_127[0][0]                 \n","__________________________________________________________________________________________________\n","activation_116 (Activation)     (None, 7, 7, 512)    0           batch_normalization_127[0][0]    \n","__________________________________________________________________________________________________\n","conv2d_128 (Conv2D)             (None, 7, 7, 2048)   1050624     activation_116[0][0]             \n","__________________________________________________________________________________________________\n","batch_normalization_128 (BatchN (None, 7, 7, 2048)   8192        conv2d_128[0][0]                 \n","__________________________________________________________________________________________________\n","add_36 (Add)                    (None, 7, 7, 2048)   0           activation_114[0][0]             \n","                                                                 batch_normalization_128[0][0]    \n","__________________________________________________________________________________________________\n","activation_117 (Activation)     (None, 7, 7, 2048)   0           add_36[0][0]                     \n","__________________________________________________________________________________________________\n","global_average_pooling2d (Globa (None, 2048)         0           activation_117[0][0]             \n","__________________________________________________________________________________________________\n","dense (Dense)                   (None, 1000)         2049000     global_average_pooling2d[0][0]   \n","==================================================================================================\n","Total params: 25,636,712\n","Trainable params: 25,583,592\n","Non-trainable params: 53,120\n","__________________________________________________________________________________________________\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"_Sg-pHgVLEnY"},"source":["## Where to go from here\n","\n","There are some additional details involved in a robust implementation which have been skipped in this tutorial.\n","\n","* Every layer should be named appropriately.\n","* Some image processing systmes may follow channels first approach where the image dimensions are (3,224,224). \n","* Every image needs to go through a basic preprocessing. This involves, convering the image to BGR format (from RGB) if necessary, then zero center each color channel with respect to the ImageNet dataset.\n","* Loading the pretrained weights using the model.load_weights function.\n","* Add a global pooling if the top classification network is ignored.\n","\n","For a more complete implementation addressing these aspects, please look at the source code in Keras Applications repository mentioned above. \n","\n","We hope that we have been able to give you a first hand experience of building large and complex convolutional networks easily with the help of Keras.\n","\n","Enjoy! "]}]}