I am making a project for checking if anime faces is similar or not. I tried training a model by myself, but it doesn't make significant progress. So I decided to take already trained model and tune it to my needs. I found this model on github.
The model structure is:
N_CATEGORIES = 203
IMAGE_SIZE = 224
input_tensor = Input(shape=(IMAGE_SIZE, IMAGE_SIZE, 3))
base_model = VGG16(weights='imagenet', include_top=False,input_tensor=input_tensor)
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
predictions = Dense(N_CATEGORIES, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=predictions)
for layer in base_model.layers[:15]:
layer.trainable = False
Keras model summary:
Model: "model_1"
Layer (type) Output Shape Param
input_1 (InputLayer) [(None, 224, 224, 3)] 0
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
global_average_pooling2d_1 (None, 512) 0
(GlobalAveragePooling2D)
dense_1 (Dense) (None, 1024) 525312
dense_2 (Dense) (None, 203) 208075
=================================================================
I downloaded an .hdf5 file for this model. By I can't convert it to .pt format. I made similar to vgg16 structure in torch:
import torch
import torch.nn as nn
class CustomModel(nn.Module):
def __init__(self):
super(CustomModel, self).__init__()
self.block1 = nn.Sequential(
nn.Conv2d(3, 64, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(64, 64, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2)
)
self.block2 = nn.Sequential(
nn.Conv2d(64, 128, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(128, 128, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2)
)
self.block3 = nn.Sequential(
nn.Conv2d(128, 256, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(256, 256, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(256, 256, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2)
)
self.block4 = nn.Sequential(
nn.Conv2d(256, 512, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(512, 512, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(512, 512, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2)
)
self.block5 = nn.Sequential(
nn.Conv2d(512, 512, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(512, 512, kernel_size=3, padding=1),
nn.ReLU(),
nn.Conv2d(512, 512, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2)
)
self.global_avg_pool = nn.AdaptiveAvgPool2d((1, 1))
self.fc1 = nn.Linear(512, 1024)
self.fc2 = nn.Linear(1024, 203)
def forward(self, x):
x = self.block1(x)
x = self.block2(x)
x = self.block3(x)
x = self.block4(x)
x = self.block5(x)
x = self.global_avg_pool(x)
x = torch.flatten(x, 1)
x = self.fc1(x)
x = self.fc2(x)
return x
pytorch_model = CustomModel()
I imported weight to keras:
from keras.models import load_model
model = load_model('animeface_vgg16.hdf5')
weights=model.get_weights()
Tried to follow method like in this code example. I understand how to assing keras weights to fc layers. But there wasn't any example with convolutional layers.
I found only examples of building model structure in other questions on stack overflow, but nobody showed how to assign weights to it.
Please explain me how to do it.
A easy way to do it is to convert the model directly from keras to pytorch using onnx as the intermediate model. This can be easily done in a couple of steps:
Step 1: Save Keras model as ONNX
from tensorflow.python.keras import backend as K
from tensorflow.python.keras.models import load_model
import onnx
import keras2onnx
onnx_model_name = 'baselineonnxmodel.onnx'
model = load_model('baselinekerasmodel.h5')
onnx_model = keras2onnx.convert_keras(model, model.name)
onnx.save_model(onnx_model, onnx_model_name)
Step 2: Load ONNX model and convert to pytorch. Install onnx2pytorch using pip install onnx2pytorch
import onnx
from onnx2pytorch import ConvertModel
onnx_model = onnx.load('baselineonnxmodel.onnx')
pytorch_model = ConvertModel(onnx_model)
You can now use the pytorch_model for making predictions.