Android : Filterable ListView and ActionMode

I have following widgets in my LinearLayout

<android.support.v7.widget.SearchView
    android:id="@+id/sv_search"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    app:queryHint="@string/hint_search" />

<LinearLayout
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:paddingBottom="@dimen/activity_vertical_margin"
    android:paddingLeft="@dimen/activity_horizontal_margin"
    android:paddingRight="@dimen/activity_horizontal_margin"
    android:paddingTop="@dimen/activity_vertical_margin" >

    <ListView
        android:id="@android:id/list"
        android:layout_width="match_parent"
        android:layout_height="wrap_content" >
    </ListView>
</LinearLayout>

For the ListView adapter I use custom modification of ArrayAdapter, but it does not matter because I tried with standard ArrayAdapter and it works the same. I use simple_list_item_checked in the adapter to display list items.

I use sv_search to filter the items on the list. In code, I have OnQueryTextListener with following code:

@Override
public boolean onQueryTextChange(String newText) {
    if (categoryAdapter != null) {
        categoryAdapter.getFilter().filter(newText);
    }
    return false;
}

It works great until ActionMode is turned on by:

lvList.setChoiceMode(ListView.CHOICE_MODE_MULTIPLE_MODAL);
lvList.setMultiChoiceModeListener(new MultiChoiceModeListener() {
    ...
});

Listener implementation is not important - it only returns true when it should be returned to enable ActionMode. ActionMode also works good.

What does not work is ActionMode and filter. For example: I start ActionMode by long-clicking on first element. It becomes selected. Then, I set filter so only the second element is visible - it is also selected. When I cancel the filter, first element is again the only selected element.

I would like this behaviour to be consistent - filtering should not affect whether item is selected or not only its visibility.