Wrox Programmer Forums
Go Back   Wrox Programmer Forums > .NET > .NET 3.5 and Visual Studio. 2008 > Visual Studio 2008
Visual Studio 2008 For discussing Visual Studio 2008. Please post code questions about a specific language (C#, VB, ASP.NET, etc) in the correct language forum instead.
Welcome to the p2p.wrox.com Forums.

You are currently viewing the Visual Studio 2008 section of the Wrox Programmer to Programmer discussions. This is a community of software programmers and website developers including Wrox book authors and readers. New member registration was closed in 2019. New posts were shut off and the site was archived into this static format as of October 1, 2020. If you require technical support for a Wrox book please contact http://hub.wiley.com
Old February 15th, 2012, 01:25 PM
Authorized User
Join Date: Dec 2004
Posts: 30
Thanks: 0
Thanked 0 Times in 0 Posts
Default Processing large UTF-8 non-text files

Hi, I have to filter out strings of UTF-8 strings in a large (100 MB) non-text file, i.e. the file cannot be read StreamReader.ReadLine or ReadAll. Since its UTF-8, it has 2 bytes/character, I've never worked with byte arrays that encode UTF-8, and I would prefer not to.

Is there a way to read fixed chunks of UTF-8 (say 1000 chars / 500 chars) ?

If NOT, can anyone give me a working example of what to do correctly with the byte arrays ?

I guess I'd have to read an even number of bytes, and then insert a CR/LF after those bytes, and save this garbage in a new file. Would this work ?

Old February 16th, 2012, 06:12 AM
Authorized User
Join Date: Dec 2004
Posts: 30
Thanks: 0
Thanked 0 Times in 0 Posts

Problem solved with Notepad++, where I managed to automatically insert CR/LF at the "right" places. The rest was simple.


Last edited by mike_abc; February 16th, 2012 at 06:14 AM..

Similar Threads
Thread Thread Starter Forum Replies Last Post
Unicode UTf-8? System.Text.UTF8Encoding from VBA? forum1 VBScript 11 June 2nd, 2017 05:13 AM
Copying large files across network RobCarter Visual Studio 2005 0 July 17th, 2009 06:24 AM
Reading large files ravichandrae Pro Java 1 January 11th, 2008 04:42 AM
Strategies for large XML files asearle XSLT 7 September 28th, 2006 02:38 AM
Uploading Large Files to a Doc Lib viccoleman SharePoint Admin 1 May 15th, 2006 01:13 PM

Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
Copyright (c) 2020 John Wiley & Sons, Inc.