Android : My layout not getting reback after virtual keyboard hide

Android virtual keybord make my layout push over to the top, after that if i hide keyboard then my layout not back to the same position, i want my layout back to the same position after hiding of keyboard,

Refer my screen shot:

Even

android:windowSoftInputMode="adjustPan"  not working, any suggestion is welcome.