Android : Layout won't line up to create gridview style

I'm trying to create a sort of gridview, it needs to be custom due to some other requirements, so please bear with me.

I have a scrollview, inside the scrollview is a linearlayout I am using a FetchableImageView class (which is an ImageView with remote file support and caching) I know the calculated numbers are correct (from looking at the debugging info) Though somehow the images get pushed down and won't allow the other image to be placed next to each other.

How do I align these, I've tried RelativeLayout as well, but it didnt do anything at all. The for loop would be coming form the String input, I'm still trying to figure out the layout, that's why it's in a plain for loop. (ps: don't mind the image url.. i quickly grabbed a placeholder file, but realize now what the image is about... )

/************************************
 * 
 ************************************/
private void processJsonStringAndUpdateUI(String result)
{

    // Init menu
    DisplayMetrics metrics = new DisplayMetrics();
    getWindowManager().getDefaultDisplay().getMetrics(metrics);

    LinearLayout linearLayout = (LinearLayout) findViewById(R.id.viewslayout);
    linearLayout.removeAllViews();
    linearLayout.setMinimumHeight(metrics.heightPixels);


    //
    // do some calculations about issues
    // we have measured that the image should be 70% of the background image
    // we have also measured that the image should be 10% from the bottom


    // 1. Lets get the background image size
    Drawable d = getResources().getDrawable(R.drawable.background);
    int h = d.getIntrinsicHeight(); 
    int w = d.getIntrinsicWidth();  

    // 2. Calculate 10% of HEIGHT
    int tenPercent = (int) (h*0.1);

    // 3. calculate the height of the image
    int imageHeight = (int) (h*0.7);

    // 4. calculate the image width which is 3:4 ratio
    int imageWidth = (int)(imageHeight/4) * 3;

    // 5. calculate how many we can fit per width (with some padding of course
    int padding = 20;
    int minimumImageArea = (int) imageWidth + (padding*2);
    int columns = (int) Math.floor(metrics.widthPixels / minimumImageArea);



    // 6. Offset and Multipliers
    int heightOffSet = (int) h - tenPercent - imageHeight;
    int widthOffSet = (int) ((w/columns) / 2) - (imageWidth/2);


    int heightMultiplier = h;
    int widthMultiplier = (int)(w/columns);




    int row = 0;
    int counter = 0;
    for(int x = 0; x < 4; x = x+1) {            
        FetchableImageView image = new FetchableImageView(this, null);            
        LinearLayout.LayoutParams params = new LinearLayout.LayoutParams(imageWidth, imageHeight);        
        image.setImageResource(R.drawable.issue);
        image.setId(1);     

        params.setMargins(widthOffSet + (counter*widthMultiplier),heightOffSet + (row*heightMultiplier),0,0);

        linearLayout.addView(image, params);
        image.setImage("http://upload.wikimedia.org/wikipedia/en/2/22/Seventeen_Magazine_cover.jpg");

        counter++;
        if (counter == columns)
        {
            counter = 0;
            row++;
        }
    }


}