Why do we use RELU for hidden layer activations and SOFTMAX for the last layer?

Why we used relu not softmax.

@haiderratlamwala786 softmax will amplify one out many neurons, this would not work out in the middle of your model.
Softmax will be good at the last layer where you want to predict one class out of many class, that is why CrossEntropyLoss is included of softmax.

Courtesy- @PrajwalPrashanth