Hi mates!
I am developing a C# Software that reads text files, each text file consists of anywhere between 100 ' s to 10,00,000 ' s of lines. I did the stupid mistake of using string to concat the lines one by one after having parsed through each line and concat it to a string. I have corrected this mistake by employing StringBuilder in my software.
However I am faced with another problem.
Each line has to be tokenized and each token needs to be checked and arranged into an INSERT command in SQL. Now comes the problem, it is costing too much in terms of performance, I end up with an array of type string of the size
number of tokens x number of lines
What would be the best solution for this?
I am enclosing some code here:
Code:
String[] value;
//Keep Reading until no more lines are found
while (line != null)
{
//Get All field values for this line
value = fileReader.tokenizeTopString(line);
//Append Value to The SQL String
sSQL.Append(" INSERT INTO t_");
sSQL.Append(tableNameTextBox.Text);
sSQL.Append(" (");
sSQL.Append(propertyStr);
sSQL.Append(") VALUES (");
for (int i = 0; i < value.Length; i++)
{
if (value[i] == this.replaceTextBox[i].Text)
value[i] = this.replaceByTextBox[i].Text;
if (i <= (value.Length - 2))
{
sSQL.Append(" '");
sSQL.Append(value[i]);
sSQL.Append("' ,");
}
else
{
sSQL.Append(" '");
sSQL.Append(value[i]);
sSQL.Append("' ");
}
}
sSQL.Append(");");
The FileReader class of my application consists of a method called tokenizeTopString which is as follows:
Code:
public String[] tokenizeTopString( String txt)
{
totalFields = 0;
String[] values = txt.Split(seps);
totalFields = values.Length;
return values;
}
Cheers,
Muzz
C# String Performance Issues
