Android : Android Universal Image Loader and SimpleImageLoadingListener

I am using UniversalImageLoader to display local images (if available) in a CursorAdapter. Since the images correspond to items belonging to different categories, I cannot set a default drawable for loading cancelled/failed/started and I'm trying to use a SimpleImageLoadingListener to load the proper drawable according to the category. Inside the bindView method I'm using:

Drawable d = mCategoryManager.getCategoryDrawableForImage(mImageFilePath);
File theCover = new File(mImageFilePath);
if (theCover.exists()) {
    imageLoader.displayImage(Uri.decode(Uri.fromFile(theCover).toString()),
        mImageView, new SimpleImageLoadingListener(){
            @Override
            public void onLoadingStarted(String imageUri, View view) {
                super.onLoadingStarted(imageUri,view);
                mImageView.setImageDrawable(d);
            }
            @Override
            public void onLoadingFailed(String imageUri, View view, FailReason failReason) {
                super.onLoadingFailed(imageUri, view, failReason);
                mImageView.setImageDrawable(d);
             }
             @Override
             public void onLoadingComplete(String imageUri, View view, Bitmap loadedImage) {
                super.onLoadingComplete(imageUri, view, loadedImage);
                mImageView.setImageBitmap(loadedImage);
             }
             @Override
             public void onLoadingCancelled(String imageUri, View view) {
                mImageView.setImageDrawable(d);
             }
    });
}

With this code I always see the default drawable instead of the loaded image, or sometimes I see the proper image for an instant and then the default drawable, while using a unique default Drawable set in ImageLoader config works, so it should be a listener issue and not related to the image file.