Lesson37TryIt_LINQtoObjects_AddProgressBar

Wendel · November 26th, 2013, 11:00 PM

Hi
Having completed lesson37, I tested all my new found creative skills by adding a âProgressBarâ to Lesson37TryIt with limited (0) success.
Can you elaborate on how to / where to:
1) add a counter (loop) on the number of files found â is it in //Search for files
2) sample code for âBackgroundWorkerâ and âProgressBarâ .
This is slightly out of scope, but any assistance would be appreciated.
My creative code (which produces a belated progress bar) is attached.
Regards
Wendel

Code:

namespace SearchForFiles
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        // Start at the startup directory.
        private void Form1_Load(object sender, EventArgs e)
        {
            directoryTextBox.Text = Application.StartupPath;
        }

        // Search for files matching the pattern
        // and containing the target string.
        public void searchButton_Click(object sender, EventArgs e)
        {
            //Start BackgroundWorker
            backgroundWorker1.RunWorkerAsync();
            
            //initialise the pickDaysComboBox
            int noDays = int.Parse(pickDaysComboBox.Text);

            // Get the file pattern and target string.
            string pattern = patternComboBox.Text;
            string target = targetTextBox.Text.ToLower();
            string writeDate = writeDateTimePicker.Text;

            // Search for files.
            DirectoryInfo dirinfo = 
                new DirectoryInfo(directoryTextBox.Text);
            var fileQuery =
                from FileInfo fileinfo
                in dirinfo.GetFiles(pattern, SearchOption.AllDirectories)
                where
                 (File.ReadAllText(fileinfo.FullName).ToLower().Contains(target) &&
                 (fileinfo.LastWriteTime.Subtract(DateTime.Now).TotalDays >= -noDays))
select (fileinfo.Length.ToString() + "\t" + fileinfo.FullName + "\t" +     fileinfo.LastWriteTime);

            //Display result
            fileListBox.DataSource = fileQuery.ToArray();
            int countLBItems = fileListBox.Items.Count;
            this.Text = ("Number of files found: " + countLBItems);
            MessageBox.Show("Done");
        }

        private void openDirButton_Click(object sender, EventArgs e)
        {
            if (folderBrowserDialog1.ShowDialog() == DialogResult.OK)
                directoryTextBox.Text = folderBrowserDialog1.SelectedPath;
        }

        public void backgroundWorker1_DoWork(object sender, DoWorkEventArgs e)
        {
            //Loop counter
            int countLBItems = fileListBox.Items.Count;
            for (int j = 0; j < countLBItems -1; j++)
            {
                j++;
                backgroundWorker1.ReportProgress(j);
            }
        }

        private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
        {
            //Change the value of the progressBar to the BackgroundWorker progress
            progressBar1.Value = e.ProgressPercentage;
        }
    }
}

Rod Stephens · November 27th, 2013, 12:34 AM

1) add a counter (loop) on the number of files found â is it in //Search for files

Do you mean make a counter show you the number of files found as the program searches for them? This could be hard because normally you would use Directory.GetFiles or something similar to do the search and that doesn't return until it is finished. It's also pretty fast so it wouldn't be worth displaying a counter.

Now if you're performing some slower search, that's different. For example, if you need to look through every file in a big hierarchy and then search through the files for some sort of target text, that would be slow enough. Then what you would do is update the display whenever you felt like it. For example, you could update a Label whenever you found a file. Or perhaps after every 100 files you check. (Or every 10 files if the search is slow enough.)

To use a progress bar, you would need to know how far through the task you are. You set its Minimum and Maximum properties to indicate how big the task is. For example, if you are searching 1000 files, you could set Minimum = 0 and Maximum = 1000. Then every now and then, you set Value = # files you have checked. You may also have to call the control's Refresh method to make it refresh before you continue searching.

2) sample code for âBackgroundWorkerâ and âProgressBarâ

BackgroundWorker does some work in its DoWork method. Whenever it wants to tell the main program about its progress, it should call its ReportProgress method. It should pass into that method a measure of how much progress it has made.

The main program should then catch its ProgressChanged event. That event handler can then display the progress to the user, for example, with a progress bar.

It seems a bit awkward to do it this way but it's what you have to do. The reason is only the thread that built the user interface can directly interact with the interface controls. That means the background worker's working thread cannot directly update labels, progress bars, etc. Instead it calls ReportProgress which raises the ProgressChanged event. The main UI thread catches that event and it IS allowed to modify the controls.

For a BackgroundWorker examples, see:

http://blog.csharphelper.com/2010/06...ound-in-c.aspx

I hope that helps. Let me know if you get stuck or if I've been unclear.

Wendel · November 28th, 2013, 02:20 AM

Hi
Lesson37TryIt_LINQtoObjects_AddProgressBar
Thanks for your prompt response. I added you sample code for âBackgroundWorker and âProgressBarâ to my code and it works as expected (i.e. it produces a progessBar for a duration of 1 * 10 * 10 *1000 milliseconds).

You were saying that using âDirectory.GetFilesâ does not return until it is finished. So I presume that is not an option for counting say 100 files, it will only produce a total count of files found..
1) How can I find and display every say 100 files as they are found and then all files.
2) How can I update the display in a label or listBox every time I find say 100 files.
3) How can I refresh the label or list box say every 100 files
4) Is the refresh done in the main program ââ¦.Button_Click(object sender,EventArgs e)â or in the â..Background worker_DoWorkâ¦â
5) Help shows refresh as âpublic override void Refresh () â. Not sure where this code should go.

In attached code I have displayed a count of total files in:

Code:

this.Text = ("Number of files found: " + countLBItems);

Regards
Wendel

Code:

            // Get the file pattern and target string.
            string pattern = patternComboBox.Text;
            string target = targetTextBox.Text.ToLower();
            string writeDate = writeDateTimePicker.Text;

            // Search for files.
            DirectoryInfo dirinfo = 
                new DirectoryInfo(directoryTextBox.Text);
            var fileQuery =
                from FileInfo fileinfo
                in dirinfo.GetFiles(pattern, SearchOption.AllDirectories)
                where
                 (File.ReadAllText(fileinfo.FullName).ToLower().Contains(target) &&
                 (fileinfo.LastWriteTime.Subtract(DateTime.Now).TotalDays >= -noDays))
select (fileinfo.Length.ToString() + "\t" + fileinfo.FullName + "\t" +     fileinfo.LastWriteTime);

            //Display result
            fileListBox.DataSource = fileQuery.ToArray();
            int countLBItems = fileListBox.Items.Count;
            this.Text = ("Number of files found: " + countLBItems);
            MessageBox.Show("Done");

Rod Stephens · November 28th, 2013, 01:59 PM

Quote:

You were saying that using âDirectory.GetFilesâ does not return until it is finished. So I presume that is not an option for counting say 100 files, it will only produce a total count of files found..

Right. You can only display progress if your code is doing something so it can interrupt itself every now and then to make a report. Something like GetFiles runs off and doesn't give you control back until it's finished.

Quote:

1) How can I find and display every say 100 files as they are found and then all files.
2) How can I update the display in a label or listBox every time I find say 100 files.
3) How can I refresh the label or list box say every 100 files

If you use GetFiles, you can't really do these. It's quite fast so most of the time it shouldn't be an issue for only 100 files.

If you somehow search the file system yourself to look for files, then you might be able to do this. It would probably be a lot slower than GetFiles, however, so you're probably better off just letting GetFiles rip.

What you can do is show progress if you're processing files. For example, suppose you need to open 1,000 image files to make thumbnails for them. Your code would enter a loop over the file names. Every file (or 100 files or whatever) you could update the ProgressBar/Label (be sure to Refresh it so the user sees the result).

That's the sort of file operation for which you can report progress.

Quote:

4) Is the refresh done in the main program ââ¦.Button_Click(object sender,EventArgs e)â or in the â..Background worker_DoWorkâ¦â
5) Help shows refresh as âpublic override void Refresh () â. Not sure where this code should go.

The problem with showing progress is that the program is pounding away on the CPU in its loop so it doesn't give the user interface any time to update the controls. If you just update a Label, then it sits there on the screen unchanged until the loop ends and then after the loop finishes it quickly updates to show the final message.

A control's Refresh method makes it immediately repaint itself so you can see the new value when you change it.

Because Refresh works directly with the controls, it must be called from the UI thread. That means a BackgroundWorker's code cannot do it. The ProgressChanged event runs in the UI thread so that's where it would go. That event handler would do something like this:

Code:

    progressLabel.Text = pct.ToString() "%";
    progressLabel.Refresh();

In your example code, you're using LINQ to do the search. That will work but it does everything in one call so there's nowhere for you to pause the search once in a while to report progress.

Something like the following code would give you more control. It's a bit more work but actually might not be any slower because LINQ isn't all that fast. (This is typed off the cuff so it may need modification to run.)

Code:

    fileListBox.Items.Clear();
    DirectoryInfo dirinfo = 
        new DirectoryInfo(directoryTextBox.Text);
    int filesProcessed = 0;
    foreach (FileInfo fileinfo in dirinfo.GetFiles(pattern,
        SearchOption.AllDirectories)
    {
        // Report progress every 100 files.
        if (++filesProcessed % 100 == 0)
        {
            progressLabel.Text = "Processed " + filesProcessed.ToString() +
                " files";
            progressLabel.Refresh();
        }

        // Search the file.
        if (File.ReadAllText(fileinfo.FullName).ToLower().Contains(target) &&
            (fileinfo.LastWriteTime.Subtract(DateTime.Now).TotalDays
            >= -noDays))
        {
            // Do something with the file.
            fileListBox.Items.Add(fileinfo.Length.ToString() + "\t" +
                fileinfo.FullName + "\t" + fileinfo.LastWriteTime);
        }
    }

In this example the loop is under our control not LINQ's so we can report progress. (Note that this code would run directly in the UI thread not in a BackgroundWorker. If you want to use a BackgroundWorker, you need to use ReportProgress and ProgressChanged.)

I don't think I have time this week but next week I may be able to write some examples with and without a BackgroundWorker.

Wendel · December 2nd, 2013, 09:31 PM

Hi Rod
Thanks. Now I get it.
Your code works great and it is pretty fast too. For a period of 730 days & Search for:
16,000 *.* files took less than 2 seconds,
4,000 files with a specific pattern took 10 minutes,
16,000 files with a specific patter threw a âSystem Out Of Memory Exception. (I might have been a bit ambitious there).
Thanks again.
Regards
Wendel

Rod Stephens · December 2nd, 2013, 10:20 PM

I'm glad it helped!

One of the "practices of good programmers" is pushing a program until it breaks to see when and why it happens so you did the right thing!

The next step is to figure out where it died and see if there's an easy way to fix it. For example, if it dies in GetFiles while searching a big directory hierarchy, you might be able to search the hierarchy one level at a time. (If it dies in GetFiles in a single enormous directory, you may be stuck.)

Or you can leave well enough alone!