Android : Android parse base64Binary pgm to Bitmap

I need to convert a base64Binary-string in the format pgm to a bitmap in android. So I don't have an usual base64-encoded Bitmap. The base64binary-string came from a xml file

<ReferenceImage Type="base64Binary" Format="pgm" WidthPX="309" HeightPX="233" BytesPerPixel="1" > NDY4Ojo9QEFDRUVHRklLTE9OUFFTU1VWV1hZWltZWVlZWlpbW1xdXmBgYmJjZGNlZWRkZGRlZmZnZ2ZnaWpqa21ub29ubm9vb3BwcHBxcHFyc3FzcnJzcnJydH[...]VlaW1xbWltcXFxcXFxd.

Pattern.compile("<ReferenceImage .*>((?s).*)<\\/ReferenceImage>");
...
String sub = r; //base64binary string pattern-matched from xml file
byte[] decodedString = Base64.decode(sub.getBytes(), Base64.NO_WRAP); //probably wrong decoding (needs to be ASCII to binary?)
Bitmap decodedByte = BitmapFactory.decodeByteArray(decodedString, 0, decodedString.length); //always null due to wrong byte-array

I think I understand that pgm images are typically stored as ASCII (like in my xml) or binary (0..255). I also think that Base64.decode needs the binary-variant, not the ASCII that I have. However BitmapFactory.decodeByteArray doesn't understand the decoded byte-array and returns null.

So how can I convert my base64binary-pgm-ASCII-string to a valid byte-array in order to create a valid bitmap?