Hello!

Before running into space problems, i generated an XML file from a 'pipe delimited' file and and then processed that XML file thru a SAXParser 'validator' and the data was correctly validated, using the RELAXNG Schema patterns as the validation criteria!

But as feared, the XML file was huge! (12 billion XML recs. generated from 1 billion 'pipe' recs.) and i am now trying to find a way to process 1 'pipe' record at a time, (ie) read 1 record from the 'pipe delimited' file, convert that rec. to an XML rec. and then send that 1 XML rec. thru the SAXParser 'validator', avoiding the build of a huge temporary XML file!

After testing this approach, its looks like the SAXParser 'validator' (sp.parse) is expecting only (1) StringBufferInputStream as input,and after opening, reading and closing just (1) of the returned StringBufferInputStream objects, the validator wants to close up operations!

Where i have the "<<<<<" you can see where i'm calling the the object.method that creates the 'pipe>XML' records 'sb.createxml' and will be returning many occurances of the StringBufferInputStream object, where (1) StringBufferInputStream object represents (1) 'pipe>XML' record!

So what i'm wondering, is if there is a form of 'inputStream' class that can be loaded and processed at the same time! ie instead of requiring that the 'inputStream' object be loaded in it's entirety, before going to validation?

Or if there is another XML 'validator' that can validate 1 XML record at a time, without requiring that the entire XML file be built first?

1. ---------------------------------class: (SX2) ---------------------------------------------------------------------------------------------------------

import ............

public class SX2
{
 public static void main(String[] args) throws Exception
 {
     MyDefaultHandler dh = new MyDefaultHandler();

     SX1 sx = new SX1();
     SAXParser sp = sx.getParser(args[0]);

     stbuf1 sb = new stbuf1();

     sp.parse(sb.createxml(args[1]),dh); <<<<<< createxml( ) see <<<<<<< below
  }
}

class MyDefaultHandler extends DefaultHandler {

        public int errcnt;

"SX2.java" 87 lines, 2563 characters

2. ----------------------------------class: (stbuf1) method: (createxml) ----------------------------------------------------------------------------

public stbuf1 () { }

        public StringBufferInputStream createxml( String inputFile ) <<<<<< createxml(
        {
                BufferedReader textReader = null;
                if ( (inputFile == null) || (inputFile.length() <= 1) )
                { throw new NullPointerException("Delimiter Input File does not exist");
                }
                String ele = new String();
                try {
                        ele = new String();
                        textReader = new BufferedReader(new FileReader(inputFile));
                        String line = null; String SEPARATOR = "\\|"; String sToken = null;

                        String hdr1=("<?xml version=#1.0# encoding=#UTF-8#?>"); hdr1=hdr1.replace('#','"');
                        String hdr2=("<hlp_data>");
                        String hdr3=("</hlp_data>");
                        String hdr4=("<"+TABLE_NAME+">");
                        String hdr5=("</"+TABLE_NAME+">");

                        while ( (line = textReader.readLine()) != null )
                        {
                              String[] sa = line.split(SEPARATOR);
                              String elel = new String();

                              for (int i = 0; i < NUM_COLS; i++)
                              {
                                 if (i>(sa.length-1)) { sToken = new String(); } else { sToken = sa[i]; }

                                 elel="<"+_columnNames[i]+">"+sToken+"</"+_columnNames[i]+">";

                                 if (i==0) {
                                      ele=ele.concat(hdr1);ele=ele.concat(hdr2);ele=ele. concat(hdr4);ele=ele.concat(elel);
                                 }
                                 else
                                 if (i==NUM_COLS - 1) {
                                      ele=ele.concat(elel);ele=ele.concat(hdr5);ele=ele. concat(hdr3);
                                 }
                                 else {
                                      ele=ele.concat(elel);
                                 }
                              }
                        }
                        textReader.close();
                }
                catch (IOException e) {
                }
                return (new StringBufferInputStream(ele));
          }
public static void main( String args[] ) {
stbuf1 genxml_obj = new stbuf1 ();
String ptxt=new String(args[0]);
genxml_obj.createxml(ptxt); }}