Wrox Programmer Forums
Go Back   Wrox Programmer Forums > Visual Basic > VB 6 Visual Basic 6 > Pro VB 6
Pro VB 6 For advanced Visual Basic coders working in version 6 (not .NET). Beginning-level questions will be redirected to other forums, including Beginning VB 6.
Welcome to the p2p.wrox.com Forums.

You are currently viewing the Pro VB 6 section of the Wrox Programmer to Programmer discussions. This is a community of software programmers and website developers including Wrox book authors and readers. New member registration was closed in 2019. New posts were shut off and the site was archived into this static format as of October 1, 2020. If you require technical support for a Wrox book please contact http://hub.wiley.com
Old March 23rd, 2005, 05:50 PM
Registered User
Join Date: Mar 2005
Posts: 2
Thanks: 0
Thanked 0 Times in 0 Posts
Default Parsing TEXT files - VERY HARD!!

I am trying to get a functional parser to move fields of an original log file (a text file selected through the FileListBox control) to make arrays out of each of the members of data.

Here are some of the parts of the original TXT file:

The data continues like this for 12000 lines. I want to make arrays from these numbers. Each array will be called according to it's topmost label. For example, everything in the set X1_accel are the numbers 2038, 2032 ... etc.

so the array in this category would be:

X1_accel(1) = 2038
X1_accel(2) = 2032

X1_accel(12000) = (the 12000'th number)

likewise another array would be:

Y1_accel(1) = 2095
Y1_accel(2) = 2093 ... all the way to Y1_accel(12000)

Count,X1_accel,X10_accel,Y1_accel,Y10_accel,Z1_acc el,Z10_accel,X_gyro,Y_gyro,Z_gyro,Load1,Load2,Load 3,Load4
1,2038,2095,1983,2023,2816,2103,2037,2018,2025,641 ,1034,978,512
2,2032,2093,1985,2022,2810,2102,2035,2016,2023,643 ,1037,981,514

Here is my code... It runs incredibly slow when I do this. I want to be able to work with the numbers in the arrays so I can take data in and graph it using a graphing toolkit. This is hard. Anyone who can help me at all on this problem is greatly appreciated.


Public Function FilterLength()
'On Error GoTo RealErrorHandler

' ************************************************** ******************************************
' Global Variables
' NumOfSamples long (max less than 15000)
' NumOfSensors integer (1 to 16 is valid range)
' SensorInfo(NumofSensors,16) as string
' RawSensorData(NumOfSensors,NumofSamples) as double

' For sensorcounter = 1 to NumofSensors

' Read the file header
' Do some limit checking on the input header
' REad rest of array data
' Do some checking on array data (# of sensors matches commas, number of samples matches the actual data in file)
' Does it end with the string "end"

' How to indicate VB errors to user via message box, and have clean termination of program (not crash)
' How to indicate logic errors (45r6 instead of 4536)

' ************************************************** ******************************************
Dim iTest As Integer
Dim debugLevel As Integer

Dim objInput As Object ' Object for Input
Dim objFSO As Object ' Object FileSystemObject
Dim objOutput As Object ' Object TOBE the OUTPUT_FILE constant

Dim sampleCount As Long ' modGlobals: { Global numOfSamples as long }

Dim strMsgBox As String ' When the process is complete
Dim numOfSensors As Integer ' Number of sensors in the array

Dim intLineLength As Integer ' To find length of each line
Dim lngWhereIsComma As Long ' To find the commas in the log
Dim bWrite As Boolean ' Boolean because will be T/F
Dim Sensor(15000, 13) As Double
Dim lngValue As String ' The sample count

Dim strCurrentLine As String ' Parse the current line to a string

Dim sensorWriteLine As String ' For debugging and printing LineByLine

Dim numTemp As Integer ' Temporary Integer
Dim strMsgBoxError As String ' Error in RealErrorHandler

Dim myCurrentPath As String ' The path in fileSystemBox modal
Dim prbLCV As Integer ' The Loop Control Variable for progress bar

Dim bCanIDoEvents As Boolean ' T/F for Event handlers

Dim INPUT_FILE As String ' The file being read
Dim OUTPUT_FILE As String ' The file being sent as [original] + "_LOG" + ".txt"

Dim logDTNStamp As String
Dim msgBoxNoFileSelected As String

Dim timeRightNow As DateTime

'************************************************* *******************************************

INPUT_FILE = ddlFile.FileName
OUTPUT_FILE = (INPUT_FILE & "_LOG#" & logDTNStamp & ".TXT")

'TODO: Fix RunTime errors with this problem. Probable causes: The filename already exists.
'VB likes to have constants for the filename, but I want to use a dynamic variable so that
'any TXT file can be selected.

' Set up our FSO object and open the input file
Set objFSO = CreateObject("scripting.FileSystemObject")
Set objInput = objFSO.OpenTextFile(INPUT_FILE, 1)

If chkDebug.Value = vbChecked Then
debugLevel = 1
debugLevel = 0
End If

If INPUT_FILE = "" Then msgBoxNoFileSelected = MsgBox("Please select a file.", vbOKOnly + vbCritical, "No File Selected")

'************************************************* *******************************************
'For Debugging the program, this will create a file for testing the parsed array
If debugLevel = 1 Then
Set objOutput = objFSO.CreateTextFile(OUTPUT_FILE, True)
End If
'************************************************* *******************************************
Let ddlFile.Path = myCurrentPath

'User feedback captions when the loop is started

lblTimEnd.Caption = "": lblTimStart.Caption = "": lblTeller.Caption = ""
lblTimStart.Caption = DateTime.Now 'Record the start of this procedure before loop
lblTeller.Caption = "Please Wait...": lblTeller.ForeColor = &HFF& 'Red

'Create an indicator to tell us if we've hit the part of the file we want to write from
bWrite = False
sampleCount = 0

'Begin parsing the input file
Do While Not objInput.AtEndOfStream


iTest = iTest + 1 'For debugging purposes
strCurrentLine = objInput.ReadLine 'Read the current line

If strCurrentLine = "end" Then GoTo ProcessDone

If bWrite = True Then 'Write the current line of data to the logfile
For numOfSensors = 1 To 13 'Assign inputs to array variables
lngWhereIsComma = InStr(strCurrentLine, ",")
Sensor(sampleCount, numOfSensors) = Left(strCurrentLine, (lngWhereIsComma - 1))
intLineLength = Len(strCurrentLine)
strCurrentLine = Right(strCurrentLine, (intLineLength - lngWhereIsComma))

If debugLevel = 1 Then 'Write the output file here
For numOfSensors = 1 To 12
'TODO: Do not write CR-LF
sensorWriteLine = Sensor(sampleCount, numOfSensors) & ","
objOutput.Write sensorWriteLine 'Use WRITE not Writeline
objOutput.Write Sensor(sampleCount, 13)
sampleCount = sampleCount + 1 'Increment the Counter
lblSampleCount.Caption = sampleCount
prbProg.Value = sampleCount 'Increment the ProgressBar
End If
End If

'Check to see if we've hit either bound of the data we want
If Left(strCurrentLine, 5) = "Count" Then
bWrite = True
ElseIf strCurrentLine = "end" Then
bWrite = False
End If


ProcessDone: 'For debugging purposes
lblTimEnd.Caption = DateTime.Now 'Record the start of this procedure after loop
lblTeller.Caption = "Process Complete. The variables have been created.": lblTeller.ForeColor = &HFF0000 'Blue
cmdRun.Enabled = True

prbProg.Value = 0 'Reset to 0 so when progress runs again it has a starting point

GoTo SkipOverErrorHandler
'TODO: Close files.

'Displays whatever runtime error occurs without halting the program
strMsgBoxError = MsgBox(Err.Description + " in " + Err.Source, vbOKOnly + vbCritical, "Syntax or critical error")


End Function

Old March 23rd, 2005, 10:40 PM
Friend of Wrox
Join Date: Jun 2003
Posts: 627
Thanks: 0
Thanked 0 Times in 0 Posts

In the "For numOfSensors = 1 To 13" loop you are doing a lot of string manipulation, that is very slow in VB. You can consider using the Split function to separate the input string in an array, and copy that in your big matrix, something like:

dim st() as string
st = split(strCurrentLine,",")
For numOfSensors = 1 To 13
Sensor(sampleCount, numOfSensors) = st(numOfSensors)
Old March 24th, 2005, 05:55 AM
Friend of Wrox
Join Date: Jun 2003
Posts: 1,212
Thanks: 0
Thanked 1 Time in 1 Post

Have you considered using a text driver instead? With ADO's GetRows method you can get a 2D array of your data for free:
    Dim oConn As ADODB.Connection
    Dim oRs As ADODB.Recordset
    Dim Sensor

    Set oConn = New ADODB.Connection

    ' the directory goes in the Data Source
    oConn.Open "Provider=Microsoft.Jet.OLEDB.4.0;" & _
       "Data Source=c:\Temp\;" & _
       "Extended Properties=""text;HDR=Yes;FMT=Delimited"""

    Set oRs = New ADODB.Recordset

    ' the file name goes in the SELECT statement
    oRs.Open "SELECT * FROM pgtemp.txt", oConn, _
             adOpenForwardOnly, adLockReadOnly, adCmdText

    If Not oRs.EOF Then

        Sensor = oRs.GetRows

    End If


    Set oRs = Nothing
    Set oConn = Nothing

Similar Threads
Thread Thread Starter Forum Replies Last Post
Parsing from a text string jroxit Classic ASP Basics 5 November 18th, 2008 05:08 PM
Parsing a text string medix_911 Access 8 November 21st, 2006 01:37 PM
parsing a changing text file on the fly Kannkor VB How-To 0 November 14th, 2005 01:47 AM
Parsing a block of text SiliconFuRy Pro PHP 6 December 15th, 2004 06:43 AM
Parsing Large Delimited Text String Coop64 SQL Language 1 February 16th, 2004 03:19 PM

Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright (c) 2020 John Wiley & Sons, Inc.