Android : Getting issues with ImageView invalidate in ListView

I have customised the list view in which I have taken an ImageView and some TextViews as roeItems. In the adapter for each rowItem I download an Image from server and In the handler I update the Imageview with the downloaded image and then Invalidate that ImageView. Below is the code for handler -

Handler handler = new Handler() {
    @Override
    public void handleMessage(Message msg) {
        ImageView imgFileicon = (ImageView) msg.obj;
        imgFileicon.setImageDrawable("My Image Path");
        imgFileicon.invalidate();
    }
};

After doing this ImageView Gets updated with each row item, But when I scroll down the list and come back to the top the images of ImageView gets replaced with each other. Means ImageView randomly changes the images. I am not getting the behaviour as this is happening first time. Can anybody tell me about it if someone has faced the same problem with solution.

Thanks in advance.