Unable to improve accuracy..?

Here is my nn, I tried changing learning rate, and trained it for more no.of epochs, but learning rate is either bouncing(for 0.1, 0.01 …) and constant for some values like 0.0001, …
Is there any way to improve this.

I feel like your model is a bit too complex, and also uses many different activation functions.
I would try lowering the complexity first to a 2 layers (input-activation-output) and using any of “good practice” (no idea how to call them differently) activation functions (ReLU, PReLU, Tanh, Sigmoid).

After that you can try and add more layers.