My graduation project is to use transfer learning on a CNN model that can diagnose Covid-19 from Chest X-ray images. After spending days Fine tuning the hyper parameters such as the number of fully connected layers, the number of nodes in the layers, the learning rate, and the drop rate using Keras tuner library with Bayesian Optimizer, I got some very good results, A test accuracy of 98% for multi class classification and a 99% for binary class classification. However, i froze all the layers in the original base model. I only fine tuned the last Fully connected layers after exhaustive hyper parameter optimization. Most articles and papers out there say that they fine the fully connected layers as well as some of the convolutional layers. Am i doing something wrong? I am afraid that this is too good to be true.
My data set is not that big, only 7000 images taken from the Kaggle Covid-19 competition.
I used image enhancement techniques such as N-CLAHE on the images before the training and the classification which improved the accuracy significantly compared to not enhancing the images.
I did the same for multiple State of art models, such as VGG-16 and ResNet50, and they all gave me superb results.
If you mean by "only fine tuned the last Fully connected layers" then NO, you did not.
You can choose to fine-tune any layer of your choice but most importantly the final layers of the model, which is what you did, so you're good to go.