Android : Android, trouble with parallax effect in case of staggered layout manager

I am having trouble implementing parallax effect for staggered layout manager, heres my code

    actionbarBackgroundDrawable = getResources().getDrawable(R.drawable.dark_actionbar_drawable);
    actionbarBackgroundDrawable.setAlpha(0);
    getSupportActionBar().setBackgroundDrawable(actionbarBackgroundDrawable);
    scrolledDistance = 0;

    RecyclerView.OnScrollListener onScrollListener = new RecyclerView.OnScrollListener() {

        @Override
        public void onScrollStateChanged(RecyclerView recyclerView, int newState) {
            super.onScrollStateChanged(recyclerView, newState);
        }

        @Override
        public void onScrolled(RecyclerView recyclerView, int dx, int dy) {

            scrolledDistance += dy;
            if (scrolledDistance < initialHeaderHeight) {
                float ratio = (float) scrolledDistance / initialHeaderHeight;
                int newAlpha = (int)Math.ceil(ratio*255);
                //actionbarBackgroundDrawable.setColorFilter(Color.argb(newAlpha, 37, 43, 53), Mode.SRC_OVER);
                actionbarBackgroundDrawable.setAlpha(newAlpha);
                RelativeLayout.LayoutParams layoutParams = new LayoutParams(LayoutParams.MATCH_PARENT, initialHeaderHeight - scrolledDistance);
                rlHeader.setLayoutParams(layoutParams);

                mImageView.setColorFilter(Color.argb(newAlpha, 37, 43, 53), Mode.SRC_OVER);
            }
            else
            {
                //actionbarBackgroundDrawable.setColorFilter(Color.argb(255, 37, 43, 53), Mode.SRC_OVER);
                actionbarBackgroundDrawable.setAlpha(255);
                mImageView.setColorFilter(Color.argb(255, 37, 43, 53), Mode.SRC_OVER);
                RelativeLayout.LayoutParams layoutParams = new LayoutParams(LayoutParams.MATCH_PARENT, 0);
                rlHeader.setLayoutParams(layoutParams);
            }

            if(scrolledDistance == 0)
            {
                RelativeLayout.LayoutParams layoutParams = new LayoutParams(LayoutParams.MATCH_PARENT, initialHeaderHeight);
                rlHeader.setLayoutParams(layoutParams);
                //actionbarBackgroundDrawable.setColorFilter(Color.argb(0, 37, 43, 53), Mode.SRC_OVER);
                actionbarBackgroundDrawable.setAlpha(0);
                mImageView.setColorFilter(Color.argb(0, 37, 43, 53), Mode.SRC_OVER);
            }
        }

    };

This works great when Grid layout manager is used, but while using staggered grid layout manager, the view distorts on scrolling, can anyone suggest a solution for it