Wrox Programmer Forums

Need to download code?

View our list of code downloads.

Go Back   Wrox Programmer Forums > Visual Basic > VB 6 Visual Basic 6 > Pro VB 6
Password Reminder
Register
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read
Pro VB 6 For advanced Visual Basic coders working in version 6 (not .NET). Beginning-level questions will be redirected to other forums, including Beginning VB 6.
Welcome to the p2p.wrox.com Forums.

You are currently viewing the Pro VB 6 section of the Wrox Programmer to Programmer discussions. This is a community of tens of thousands of software programmers and website developers including Wrox book authors and readers. As a guest, you can read any forum posting. By joining today you can post your own programming questions, respond to other developers’ questions, and eliminate the ads that are displayed to guests. Registration is fast, simple and absolutely free .
DRM-free e-books 300x50
Reply
 
Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old March 23rd, 2005, 05:50 PM
Registered User
 
Join Date: Mar 2005
Location: , , .
Posts: 2
Thanks: 0
Thanked 0 Times in 0 Posts
Default Parsing TEXT files - VERY HARD!!

I am trying to get a functional parser to move fields of an original log file (a text file selected through the FileListBox control) to make arrays out of each of the members of data.

Here are some of the parts of the original TXT file:

The data continues like this for 12000 lines. I want to make arrays from these numbers. Each array will be called according to it's topmost label. For example, everything in the set X1_accel are the numbers 2038, 2032 ... etc.

so the array in this category would be:

X1_accel(1) = 2038
X1_accel(2) = 2032

...
X1_accel(12000) = (the 12000'th number)

likewise another array would be:

Y1_accel(1) = 2095
Y1_accel(2) = 2093 ... all the way to Y1_accel(12000)


Count,X1_accel,X10_accel,Y1_accel,Y10_accel,Z1_acc el,Z10_accel,X_gyro,Y_gyro,Z_gyro,Load1,Load2,Load 3,Load4
1,2038,2095,1983,2023,2816,2103,2037,2018,2025,641 ,1034,978,512
2,2032,2093,1985,2022,2810,2102,2035,2016,2023,643 ,1037,981,514

Here is my code... It runs incredibly slow when I do this. I want to be able to work with the numbers in the arrays so I can take data in and graph it using a graphing toolkit. This is hard. Anyone who can help me at all on this problem is greatly appreciated.

------------------------------------------------------------------

Public Function FilterLength()
'On Error GoTo RealErrorHandler

' ************************************************** ******************************************
' Global Variables
'
' NumOfSamples long (max less than 15000)
' NumOfSensors integer (1 to 16 is valid range)
' SensorInfo(NumofSensors,16) as string
' RawSensorData(NumOfSensors,NumofSamples) as double

' For sensorcounter = 1 to NumofSensors

' Read the file header
' Do some limit checking on the input header
' REad rest of array data
' Do some checking on array data (# of sensors matches commas, number of samples matches the actual data in file)
' Does it end with the string "end"

' How to indicate VB errors to user via message box, and have clean termination of program (not crash)
' How to indicate logic errors (45r6 instead of 4536)

' ************************************************** ******************************************
Dim iTest As Integer
Dim debugLevel As Integer

Dim objInput As Object ' Object for Input
Dim objFSO As Object ' Object FileSystemObject
Dim objOutput As Object ' Object TOBE the OUTPUT_FILE constant

Dim sampleCount As Long ' modGlobals: { Global numOfSamples as long }

Dim strMsgBox As String ' When the process is complete
Dim numOfSensors As Integer ' Number of sensors in the array

Dim intLineLength As Integer ' To find length of each line
Dim lngWhereIsComma As Long ' To find the commas in the log
Dim bWrite As Boolean ' Boolean because will be T/F
Dim Sensor(15000, 13) As Double
Dim lngValue As String ' The sample count

Dim strCurrentLine As String ' Parse the current line to a string

Dim sensorWriteLine As String ' For debugging and printing LineByLine

Dim numTemp As Integer ' Temporary Integer
Dim strMsgBoxError As String ' Error in RealErrorHandler

Dim myCurrentPath As String ' The path in fileSystemBox modal
Dim prbLCV As Integer ' The Loop Control Variable for progress bar

Dim bCanIDoEvents As Boolean ' T/F for Event handlers

Dim INPUT_FILE As String ' The file being read
Dim OUTPUT_FILE As String ' The file being sent as [original] + "_LOG" + ".txt"

Dim logDTNStamp As String
Dim msgBoxNoFileSelected As String

Dim timeRightNow As DateTime


'************************************************* *******************************************



INPUT_FILE = ddlFile.FileName
OUTPUT_FILE = (INPUT_FILE & "_LOG#" & logDTNStamp & ".TXT")

'TODO: Fix RunTime errors with this problem. Probable causes: The filename already exists.
'VB likes to have constants for the filename, but I want to use a dynamic variable so that
'any TXT file can be selected.


' Set up our FSO object and open the input file
Set objFSO = CreateObject("scripting.FileSystemObject")
Set objInput = objFSO.OpenTextFile(INPUT_FILE, 1)

If chkDebug.Value = vbChecked Then
debugLevel = 1
Else
debugLevel = 0
End If

If INPUT_FILE = "" Then msgBoxNoFileSelected = MsgBox("Please select a file.", vbOKOnly + vbCritical, "No File Selected")


'************************************************* *******************************************
'For Debugging the program, this will create a file for testing the parsed array
If debugLevel = 1 Then
Set objOutput = objFSO.CreateTextFile(OUTPUT_FILE, True)
Else
End If
'************************************************* *******************************************
Let ddlFile.Path = myCurrentPath

'User feedback captions when the loop is started

lblTimEnd.Caption = "": lblTimStart.Caption = "": lblTeller.Caption = ""
lblTimStart.Caption = DateTime.Now 'Record the start of this procedure before loop
lblTeller.Caption = "Please Wait...": lblTeller.ForeColor = &HFF& 'Red

'Create an indicator to tell us if we've hit the part of the file we want to write from
bWrite = False
sampleCount = 0

'Begin parsing the input file
Do While Not objInput.AtEndOfStream

DoEvents

iTest = iTest + 1 'For debugging purposes
strCurrentLine = objInput.ReadLine 'Read the current line

If strCurrentLine = "end" Then GoTo ProcessDone

If bWrite = True Then 'Write the current line of data to the logfile
For numOfSensors = 1 To 13 'Assign inputs to array variables
lngWhereIsComma = InStr(strCurrentLine, ",")
Sensor(sampleCount, numOfSensors) = Left(strCurrentLine, (lngWhereIsComma - 1))
intLineLength = Len(strCurrentLine)
strCurrentLine = Right(strCurrentLine, (intLineLength - lngWhereIsComma))
Next

If debugLevel = 1 Then 'Write the output file here
For numOfSensors = 1 To 12
'TODO: Do not write CR-LF
sensorWriteLine = Sensor(sampleCount, numOfSensors) & ","
objOutput.Write sensorWriteLine 'Use WRITE not Writeline
Next
objOutput.Write Sensor(sampleCount, 13)
sampleCount = sampleCount + 1 'Increment the Counter
lblSampleCount.Caption = sampleCount
prbProg.Value = sampleCount 'Increment the ProgressBar
End If
End If

'Check to see if we've hit either bound of the data we want
If Left(strCurrentLine, 5) = "Count" Then
bWrite = True
ElseIf strCurrentLine = "end" Then
bWrite = False
End If

Loop

ProcessDone: 'For debugging purposes
lblTimEnd.Caption = DateTime.Now 'Record the start of this procedure after loop
lblTeller.Caption = "Process Complete. The variables have been created.": lblTeller.ForeColor = &HFF0000 'Blue
cmdRun.Enabled = True

prbProg.Value = 0 'Reset to 0 so when progress runs again it has a starting point


GoTo SkipOverErrorHandler
'TODO: Close files.

RealErrorHandler:
'Displays whatever runtime error occurs without halting the program
strMsgBoxError = MsgBox(Err.Description + " in " + Err.Source, vbOKOnly + vbCritical, "Syntax or critical error")

SkipOverErrorHandler:

End Function





Reply With Quote
  #2 (permalink)  
Old March 23rd, 2005, 10:40 PM
Friend of Wrox
 
Join Date: Jun 2003
Location: Alameda, ca, USA.
Posts: 627
Thanks: 0
Thanked 0 Times in 0 Posts
Default

In the "For numOfSensors = 1 To 13" loop you are doing a lot of string manipulation, that is very slow in VB. You can consider using the Split function to separate the input string in an array, and copy that in your big matrix, something like:


dim st() as string
st = split(strCurrentLine,",")
For numOfSensors = 1 To 13
Sensor(sampleCount, numOfSensors) = st(numOfSensors)
Next
Reply With Quote
  #3 (permalink)  
Old March 24th, 2005, 05:55 AM
Friend of Wrox
 
Join Date: Jun 2003
Location: , , United Kingdom.
Posts: 1,212
Thanks: 0
Thanked 1 Time in 1 Post
Default

Have you considered using a text driver instead? With ADO's GetRows method you can get a 2D array of your data for free:
Code:
    Dim oConn As ADODB.Connection
    Dim oRs As ADODB.Recordset
    Dim Sensor

    Set oConn = New ADODB.Connection

    ' the directory goes in the Data Source
    oConn.Open "Provider=Microsoft.Jet.OLEDB.4.0;" & _
       "Data Source=c:\Temp\;" & _
       "Extended Properties=""text;HDR=Yes;FMT=Delimited"""

    Set oRs = New ADODB.Recordset

    ' the file name goes in the SELECT statement
    oRs.Open "SELECT * FROM pgtemp.txt", oConn, _
             adOpenForwardOnly, adLockReadOnly, adCmdText

    If Not oRs.EOF Then

        Sensor = oRs.GetRows

    End If

    oRs.Close
    oConn.Close

    Set oRs = Nothing
    Set oConn = Nothing
hth
Phil
Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Parsing from a text string jroxit Classic ASP Basics 5 November 18th, 2008 05:08 PM
Parsing a text string medix_911 Access 8 November 21st, 2006 01:37 PM
parsing a changing text file on the fly Kannkor VB How-To 0 November 14th, 2005 01:47 AM
Parsing a block of text SiliconFuRy Pro PHP 6 December 15th, 2004 06:43 AM
Parsing Large Delimited Text String Coop64 SQL Language 1 February 16th, 2004 03:19 PM



All times are GMT -4. The time now is 04:51 PM.


Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
© 2013 John Wiley & Sons, Inc.