Hey, I recently took the Zero to GANs course here, and I was trying to emulate CNN on this dataset of German Traffic Signs.
I managed to achieve 99% validation accuracy but somehow my model performs very very poorly on the Test dataset, am I missing something? Can someone from the community help me out?
hereās my notebook link.
The dataset is the German Traffic Sign Benchmark from Kaggle.
The only difference between many of them is their size.
So, letās say you split the initial dataset into 80%/20% training/validation.
But there are so many identical images, different only by their size, that the model starts to OVERFIT, because the images in validation set are identical to these from training set (they just have a different size).
While the dataset augmentation is a nice technique, this one gives a false assumption when you create validation set from it, that the model performs very well.
Since itās been overfitting, it didnāt generalize at all what the sign represents, it just memorized the training examples.
Hey, This is the reason I posted it in forum actually, this overfitting issue seems to happen with PyTorch only, I also tried it with tensorflow and achieved decent accuracy of 96%. Somehow same architecture in PyTorch is learning a lot more, and overfitting the data.
I am yet to deduce what extra is happening with PyTorch.
Hello Sir, what you are stating might be correct. But sir, when I tried this same architecture with Tensorflow, it was able to properly learn from the same dataset with same preprocessing.
What extra is happening with PyTorch that is overfitting the model?
Iāve tried to run both models on kaggle, and got the accuracy around 96% as well for keras, but literally no idea why the behavior is so different.
My suspicion is that the fit() method of keras models has some sort of regularization built-in (but Iām not used to keras so canāt be sure about that). Like, you donāt even provide a learning rate or scheduler here, it just happens automagically.
The scary thought is Iāve used pytorch in my master thesis, achieving somewhat poor results. If thereās some problem inside the framework (or Iāve overlooked something important), then my whole thesis could have a completely different conclusion.
If Iāll find some time Iām gonna look at it again.
If anyone wants I can add you as colaborators to these notebooks (gonna need kaggle usernames though).
I published in āCan We Make Machines Understand Human Emotions ?ā in medium : https://link.medium.com/RNGsoLVb9cb
and guess what, I got a opportunity to publish it in @thestartup
Check it out.
I tried to create an image classification model which would segregate images of garbage items into 6 different garbage bins.
Achieved an Accuracy of 80%. Any suggestions to improve accuracy are welcome!
Hi, junta,
Hope all are safe and healthy. I personally thanks to Akaash and his team for this wonderful course in the pandemic. Here, i am sharing my project which could be helpful to many aspirants like me.
My model detects the gender of the person(male/female) from the photo of the person with an accuracy of 95%. Do check it out and all improvements are welcomed.
Hey Rishab, Good work!! Nice implementation,
I would like to suggest that, Do not use any augmentation Transformations on the validation set because it will create new samples of the same data which might show us a different score because of the duplicated images.
Hi everyone,
Iāve attached the link to my submission.
Was still seeing some light fluctuation in the accuracy, so Iām not too sure how good this model is.
But these are my final results. Iāve documented the hyperparameters, metrics and notes. You can have a look at it in the compare section.
Epochs and Learning Rate ->5 epochs with a learning rate of 5e-2, then 5 epochs with a learning rate of 2.5e-2, and then another 10 epochs with a learning rate of 1e-2.