Android : onFling and LinearLayouts

My Linear Layout: main_linear_layout contains four Linear Layouts: ll1 ll2 ll3 ll4. Each LinearLayout contains Buttons. I am trying to implement the onFling method on main_linear_layout. I want to be able to swipe anywhere on the Layout of Buttons and call an action.

Currently I am able to swipe anywhere on the screen except for the Layout of Buttons. I tried using the following code to fix the problem:

swipeLayout = (LinearLayout) findViewById(R.id.main_linear_layout);
//swipeLayout is the main Layout that contains 4 other linear layouts

swipeLayout.setOnTouchListener(new View.OnTouchListener() {

    @Override
    public boolean onTouch(View v, MotionEvent event) {
        // TODO Auto-generated method stub
        gestureScanner.onTouchEvent(event);
        return true;
    }
});

But I am still not able to swipe on the Layout of Buttons. Does anybody know why this is? Is there something else I need to implement? Am I supposed to implement an OnTouchListener to every Linear Layout inside the main Linear Layout? Or to every Button in each Layout?

And in the xml I added a bunch of code to the main Linear Layout:

android:clickable="true"
android:focusable="true"
android:focusableInTouchMode="true"
android:longClickable="true" 

But this did not work either. Can someone please help?