How to do get the Total lines of the file when we are within in a StreamWriter scope.
Based on the total number of lines count I am writing some more lines at the end of the file.
I have tried the below code : But it throws an error message
The process cannot access the file ‘C:\a.txt ' because it is being used by another process.
var lineCount = File.ReadLines(outputFilePath).Count()
This is my Code
private string CreateAndPushFile(string fileName)
{
string outputFilePath = string.Format(#"{0}\{1}", “C:\\a.txt”, fileName);
using (StreamWriter output = new StreamWriter(outputFilePath))
{
// Creates the file header
string fileHeader =”kjhakljdhkjhkj”;
output.Write(fileHeader);
string batchControl = “1515151”; // This value comes from database
output.Write(batchControl);
// Here there is some other logic which will writes many lines to the File using foreach loop
string fileControl = “3123123”; // This value comes from database
output.WriteLine(fileControl);
// After this I need write a few more lines only if total number of lines in a File Total records multiple of 10
var lineCount = File.ReadLines(outputFilePath).Count(); // I am getting error here
int remainder;
Math.DivRem(lineCount, 10, out remainder);
for (int i = 1; i <= 10 - remainder; i++)
{
output.WriteLine(“9999999999999”);
}
}
}
private static void CreateAndPushFile(string outputFilePath) {
using (var output = new StreamWriter(outputFilePath)) {
// Creates the file header
var fileHeader = "kjhakljdhkjhkj";
output.Write(fileHeader);
var batchControl = "1515151"; // This value comes from database
output.Write(batchControl);
// Here there is some other logic which will writes many lines to the File using foreach loop
var fileControl = "3123123"; // This value comes from database
output.WriteLine(fileControl);
// After this I need write a few more lines only if total number of lines in a File Total records multiple of 10
}
var lineCount = TotalLines(outputFilePath); // I am getting error here
var remainder = lineCount % 10;
using (var output2 = new StreamWriter(outputFilePath, true)) { // second parameter is for append
for (var i = 0; i < 10 - remainder; i++) {
output2.WriteLine("9999999999999");
}
}
}
private static int TotalLines(string filePath) {
using (var reader = new StreamReader(filePath)) {
char[] buffer = new char[1024];
var lineCount = 0;
while (!reader.EndOfStream) {
var charsRead = reader.Read(buffer, 0, 1024);
lineCount += buffer.Take(charsRead).Count(character => character == '\n');
}
return lineCount;
}
}
I want to remember where, that my program had processed.
I call StreamReader ReadLine()
Then next time I can jump right back to the location keep processing.
I don't want to do the line count and skip, I want to store a physicial location or something like that. So next time I can jump right back.
Thanks
you can count your text length, and then seek it before reading like below code:
hope it can help you.
internal class Program
{
static void Main(string[] args)
{
var fileName = "your file path";
long count = 0;
var firstLine = GetContent(fileName, count);
count += firstLine.position;
var secondLine = GetContent(fileName, count);
count += secondLine.position;
var thirdLine = GetContent(fileName, count);
}
public static (string content, int position) GetContent(string fileName, long position)
{
using var x = File.OpenRead(fileName);
using var ww = new StreamReader(x);
x.Seek(position, SeekOrigin.Begin);
var k = ww.ReadLine();
return (k, ww.CurrentEncoding.GetByteCount(k) + 2);
}
}
I'm using this, slightly modified, to copy large files from a file share with the ability to continue copying, if the download was disrupted. It runs in a BackroudWorker and reports progress. This works fine, but I'd like to have the ability to write the current MD5 hash to disk (the current total, not once for each block) each time a block of file data is written to disk WITH MINIMAL ADDITIONAL OVERHEAD. If a partial file is discovered, I'd like to read the MD5 hash from file, and if it is identical to that of the partial file, continue copying. When the file has been copied completely, the MD5 hash in the file should be that of the completly copied file. I'd like to use that later to determine that the files in source and destination are identical. Thanks for any help!
This is my current copy method:
public static bool CopyFile(List<CopyObjects> FileList, FSObjToCopy job, BackgroundWorker BW)
{
Stopwatch sw = new Stopwatch();
long RestartPosition = 0;
bool Retry = false;
int BYTES_TO_READ = (0x200000)
foreach (CopyObjects co in FileList)
{
FileInfo fi = co.file;
FileInfo fo = null;
if (fi.Directory.FullName.StartsWith($#"{Test_Updater_Core.ServerName}\{Test_Updater_Core.ServerTemplateRoot}"))
{
if (File.Exists(fi.FullName.Replace($#"{Test_Updater_Core.ServerName}\{Test_Updater_Core.ServerTemplateRoot}", $#"{ Test_Updater_Core.USBStore_Drive.driveInfo.Name.Replace("\\", "")}\{Test_Updater_Core.UsbTemplateRoot}")))
{
fi = new FileInfo(fi.FullName.Replace($#"{Test_Updater_Core.ServerName}\{Test_Updater_Core.ServerTemplateRoot}", $#"{Test_Updater_Core.USBStore_Drive.driveInfo.Name.Replace("\\", "")}\{Test_Updater_Core.UsbTemplateRoot}"));
co.destination = co.destination.Replace($#"{Test_Updater_Core.USBStore_Drive.driveInfo.Name.Replace("\\", "")}\{Test_Updater_Core.UsbTemplateRoot}", $#"{Test_Updater_Core.LocalInstallDrive}\{Test_Updater_Core.LocalTemplateRoot}");
fo = new FileInfo($"{fi.FullName.Replace($#"{Test_Updater_Core.USBStore_Drive.driveInfo.Name.Replace("\\", "")}\{Test_Updater_Core.UsbTemplateRoot}", $#"{Test_Updater_Core.LocalInstallDrive}\{Test_Updater_Core.LocalTemplateRoot}")}{Test_Updater_Core.TempFileExtension}");
}
}
//If a clean cancellation was requested, we do it here, otherwise the BackgroundWorker will be killed
if (BW.CancellationPending)
{
job.Status = FSObjToCopy._Status.Complete;
return false;
}
//If a pause is requested, we loop here until resume or termination has been signaled
while (job.PauseBackgroundWorker == true)
{
Thread.Sleep(100);
if (BW.CancellationPending)
{
job.Status = FSObjToCopy._Status.Complete;
return false;
}
Application.DoEvents();
}
if (fo == null)
fo = new FileInfo($"{fi.FullName.Replace(job.Source, co.destination)}{Test_Updater_Core.TempFileExtension}");
if (fo.Exists)
{
Retry = true;
RestartPosition = fo.Length - BYTES_TO_READ;
}
else
{
RestartPosition = 0;
Retry = false;
}
if (RestartPosition <= 0)
{
Retry = false;
}
sw.Start();
try
{
// Read source files into file streams
FileStream source = new FileStream(fi.FullName, FileMode.Open, FileAccess.Read);
// Additional way to write to file stream
FileStream dest = new FileStream(fo.FullName, FileMode.OpenOrCreate, FileAccess.Write);
// Actual read file length
int destLength = 0;
// If the length of each read is less than the length of the source file, read in chunks
if (BYTES_TO_READ < source.Length)
{
byte[] buffer = new byte[BYTES_TO_READ];
long copied = 0;
if (Retry)
{
source.Seek(RestartPosition, SeekOrigin.Begin);
dest.Seek(RestartPosition, SeekOrigin.Begin);
Retry = false;
}
while (copied <= source.Length - BYTES_TO_READ)
{
destLength = source.Read(buffer, 0, BYTES_TO_READ);
source.Flush();
dest.Write(buffer, 0, BYTES_TO_READ);
dest.Flush();
// Current position of flow
dest.Position = source.Position;
copied += BYTES_TO_READ;
job.CopiedSoFar += BYTES_TO_READ;
if (sw.ElapsedMilliseconds > 250)
{
job.PercComplete = (int)(float)((float)job.CopiedSoFar / (float)job.TotalFileSize * 100);
sw.Restart();
sw.Start();
job.ProgressCell.Value = job.PercComplete;
BW.ReportProgress(job.PercComplete < 100 ? job.PercComplete : 99);
}
if (BW.CancellationPending)
{
job.Status = FSObjToCopy._Status.Complete;
return false;
}
while (job.PauseBackgroundWorker == true)
{
Thread.Sleep(100);
if (BW.CancellationPending)
{
job.Status = FSObjToCopy._Status.Complete;
return false;
}
Application.DoEvents();
}
}
int left = (int)(source.Length - copied);
destLength = source.Read(buffer, 0, left);
source.Flush();
dest.Write(buffer, 0, left);
dest.Flush();
job.CopiedSoFar += left;
}
else
{
// If the file length of each copy is longer than that of the source file, the actual file length is copied directly.
byte[] buffer = new byte[source.Length];
source.Read(buffer, 0, buffer.Length);
source.Flush();
dest.Write(buffer, 0, buffer.Length);
dest.Flush();
job.CopiedSoFar += source.Length;
job.PercComplete = (int)(float)((float)job.CopiedSoFar / (float)job.TotalFileSize * 100);
job.ProgressCell.Value = job.PercComplete;
BW.ReportProgress(job.PercComplete < 100 ? job.PercComplete : 99);
}
source.Close();
dest.Close();
fo.LastWriteTimeUtc = fi.LastWriteTimeUtc;
if (File.Exists(fo.FullName))
{
if (File.Exists(fo.FullName.Replace($"{Test_Updater_Core.TempFileExtension}", "")))
{
File.Delete(fo.FullName.Replace($"{Test_Updater_Core.TempFileExtension}", ""));
}
File.Move(fo.FullName, fo.FullName.Replace($"{Test_Updater_Core.TempFileExtension}", ""));
}
job.ProgressCell.Value = job.PercComplete;
BW.ReportProgress(job.PercComplete);
}
catch (Exception ex)
{
MessageBox.Show($"There was an error copying:{Environment.NewLine}{fi}{Environment.NewLine}to:" +
$"{Environment.NewLine}{fo}{Environment.NewLine}The error is: {Environment.NewLine}{ex.Message}",
"Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
job.Status = FSObjToCopy._Status.Error;
return false;
}
finally
{
sw.Stop();
}
}
return true;
}
I decided to create Checksum files on the server that contain a series of checksums in each. As I copy the file, I add the checksums to an internal list, and compare them to the server list. If at some point, they do not match, I go back to the point where they were identical and start back up there. At the end of the copy job, I write the checksums from the internal list to disk, with the same name as of the server. If I'd like to check the integrity of a file, I can compare the server file to the local file and verify the checksums.
I'm writing a simple c# console app that uploads files to sftp server. However, the amount of files are large. I would like to display either percentage of files uploaded or just the number of files upload already from the total number of files to be upload.
First, I get all the files and the total number of files.
string[] filePath = Directory.GetFiles(path, "*");
totalCount = filePath.Length;
Then I loop through the file and upload them one by one in foreach loop.
foreach(string file in filePath)
{
string FileName = Path.GetFileName(file);
//copy the files
oSftp.Put(LocalDirectory + "/" + FileName, _ftpDirectory + "/" + FileName);
//Console.WriteLine("Uploading file..." + FileName);
drawTextProgressBar(0, totalCount);
}
In the foreach loop I have a progress bar which I have issues with. It doesn't display properly.
private static void drawTextProgressBar(int progress, int total)
{
//draw empty progress bar
Console.CursorLeft = 0;
Console.Write("["); //start
Console.CursorLeft = 32;
Console.Write("]"); //end
Console.CursorLeft = 1;
float onechunk = 30.0f / total;
//draw filled part
int position = 1;
for (int i = 0; i < onechunk * progress; i++)
{
Console.BackgroundColor = ConsoleColor.Gray;
Console.CursorLeft = position++;
Console.Write(" ");
}
//draw unfilled part
for (int i = position; i <= 31 ; i++)
{
Console.BackgroundColor = ConsoleColor.Green;
Console.CursorLeft = position++;
Console.Write(" ");
}
//draw totals
Console.CursorLeft = 35;
Console.BackgroundColor = ConsoleColor.Black;
Console.Write(progress.ToString() + " of " + total.ToString() + " "); //blanks at the end remove any excess
}
The output is just [ ] 0 out of 1943
What am I doing wrong here?
EDIT:
I'm trying to display the progress bar while I'm loading and exporting XML files. However, it's going through a loop. After it finishes the first round it goes to the second and so on.
string[] xmlFilePath = Directory.GetFiles(xmlFullpath, "*.xml");
Console.WriteLine("Loading XML files...");
foreach (string file in xmlFilePath)
{
for (int i = 0; i < xmlFilePath.Length; i++)
{
//ExportXml(file, styleSheet);
drawTextProgressBar(i, xmlCount);
count++;
}
}
It never leaves the for loop...Any suggestions?
I was also looking for a console progress bar. I didn't find one that did what I needed, so I decided to roll my own. Click here for the source code (MIT License).
Features:
Works with redirected output
If you redirect the output of a console application (e.g., Program.exe > myfile.txt), most implementations will crash with an exception. That's because Console.CursorLeft and Console.SetCursorPosition() don't support redirected output.
Implements IProgress<double>
This allows you to use the progress bar with async operations that report a progress in the range of [0..1].
Thread-safe
Fast
The Console class is notorious for its abysmal performance. Too many calls to it, and your application slows down. This class performs only 8 calls per second, no matter how often you report a progress update.
Use it like this:
Console.Write("Performing some task... ");
using (var progress = new ProgressBar()) {
for (int i = 0; i <= 100; i++) {
progress.Report((double) i / 100);
Thread.Sleep(20);
}
}
Console.WriteLine("Done.");
I know this is an old thread, and apologies for the self promotion, however I've recently written an open source console library available on nuget Goblinfactory.Konsole with threadsafe multiple progress bar support, that might help anyone new to this page needing one that doesnt block the main thread.
It's somewhat different to the answers above as it allows you to kick off the downloads and tasks in parallel and continue with other tasks;
cheers, hope this is helpful
A
var t1 = Task.Run(()=> {
var p = new ProgressBar("downloading music",10);
... do stuff
});
var t2 = Task.Run(()=> {
var p = new ProgressBar("downloading video",10);
... do stuff
});
var t3 = Task.Run(()=> {
var p = new ProgressBar("starting server",10);
... do stuff .. calling p.Refresh(n);
});
Task.WaitAll(new [] { t1,t2,t3 }, 20000);
Console.WriteLine("all done.");
gives you this type of output
The nuget package also includes utilities for writing to a windowed section of the console with full clipping and wrapping support, plus PrintAt and various other helpful classes.
I wrote the nuget package because I constantly ended up writing lots of common console routines whenever I wrote build and ops console scripts and utilities.
If I was downloading several files, I used to slowly Console.Write to the screen on each thread, and used to try various tricks to make reading the interleaved output on the screen easier to read, e.g. different colors or numbers. I eventually wrote the windowing library so that output from different threads could simply be printed to different windows, and it cut down a ton of boilerplate code in my utility scripts.
For example, this code,
var con = new Window(200,50);
con.WriteLine("starting client server demo");
var client = new Window(1, 4, 20, 20, ConsoleColor.Gray, ConsoleColor.DarkBlue, con);
var server = new Window(25, 4, 20, 20, con);
client.WriteLine("CLIENT");
client.WriteLine("------");
server.WriteLine("SERVER");
server.WriteLine("------");
client.WriteLine("<-- PUT some long text to show wrapping");
server.WriteLine(ConsoleColor.DarkYellow, "--> PUT some long text to show wrapping");
server.WriteLine(ConsoleColor.Red, "<-- 404|Not Found|some long text to show wrapping|");
client.WriteLine(ConsoleColor.Red, "--> 404|Not Found|some long text to show wrapping|");
con.WriteLine("starting names demo");
// let's open a window with a box around it by using Window.Open
var names = Window.Open(50, 4, 40, 10, "names");
TestData.MakeNames(40).OrderByDescending(n => n).ToList()
.ForEach(n => names.WriteLine(n));
con.WriteLine("starting numbers demo");
var numbers = Window.Open(50, 15, 40, 10, "numbers",
LineThickNess.Double,ConsoleColor.White,ConsoleColor.Blue);
Enumerable.Range(1,200).ToList()
.ForEach(i => numbers.WriteLine(i.ToString())); // shows scrolling
produces this
You can also create progress bars inside a window just as easily as writing to the windows. (mix and match).
You might want to try https://www.nuget.org/packages/ShellProgressBar/
I just stumbled upon this progress bar implementation - its cross platform, really easy to use, quite configurable and does what it should right out of the box.
Just sharing because i liked it a lot.
This line is your problem:
drawTextProgressBar(0, totalCount);
You're saying the progress is zero in every iteration, this should be incremented. Maybe use a for loop instead.
for (int i = 0; i < filePath.length; i++)
{
string FileName = Path.GetFileName(filePath[i]);
//copy the files
oSftp.Put(LocalDirectory + "/" + FileName, _ftpDirectory + "/" + FileName);
//Console.WriteLine("Uploading file..." + FileName);
drawTextProgressBar(i, totalCount);
}
I quite liked the original poster's progress bar, but found that it did not display progress correctly with certain progress/total item combinations. The following, for example, does not draw correctly, leaving an extra grey block at the end of the progress bar:
drawTextProgressBar(4114, 4114)
I re-did some of the drawing code to remove the unnecessary looping which fixed the above issue and also sped things up quite a bit:
public static void drawTextProgressBar(string stepDescription, int progress, int total)
{
int totalChunks = 30;
//draw empty progress bar
Console.CursorLeft = 0;
Console.Write("["); //start
Console.CursorLeft = totalChunks + 1;
Console.Write("]"); //end
Console.CursorLeft = 1;
double pctComplete = Convert.ToDouble(progress) / total;
int numChunksComplete = Convert.ToInt16(totalChunks * pctComplete);
//draw completed chunks
Console.BackgroundColor = ConsoleColor.Green;
Console.Write("".PadRight(numChunksComplete));
//draw incomplete chunks
Console.BackgroundColor = ConsoleColor.Gray;
Console.Write("".PadRight(totalChunks - numChunksComplete));
//draw totals
Console.CursorLeft = totalChunks + 5;
Console.BackgroundColor = ConsoleColor.Black;
string output = progress.ToString() + " of " + total.ToString();
Console.Write(output.PadRight(15) + stepDescription); //pad the output so when changing from 3 to 4 digits we avoid text shifting
}
I have copy pasted your ProgressBar method. Because your error was in the loop as the accepted answer mentioned. But the ProgressBar method has some syntax errors too. Here is the working version. Slightly modified.
private static void ProgressBar(int progress, int tot)
{
//draw empty progress bar
Console.CursorLeft = 0;
Console.Write("["); //start
Console.CursorLeft = 32;
Console.Write("]"); //end
Console.CursorLeft = 1;
float onechunk = 30.0f / tot;
//draw filled part
int position = 1;
for (int i = 0; i < onechunk * progress; i++)
{
Console.BackgroundColor = ConsoleColor.Green;
Console.CursorLeft = position++;
Console.Write(" ");
}
//draw unfilled part
for (int i = position; i <= 31; i++)
{
Console.BackgroundColor = ConsoleColor.Gray;
Console.CursorLeft = position++;
Console.Write(" ");
}
//draw totals
Console.CursorLeft = 35;
Console.BackgroundColor = ConsoleColor.Black;
Console.Write(progress.ToString() + " of " + tot.ToString() + " "); //blanks at the end remove any excess
}
Please note that #Daniel-wolf has a better approach: https://stackoverflow.com/a/31193455/169714
I've created this handy class that works with System.Reactive. I hope you find it lovely enough.
public class ConsoleDisplayUpdater : IDisposable
{
private readonly IDisposable progressUpdater;
public ConsoleDisplayUpdater(IObservable<double> progress)
{
progressUpdater = progress.Subscribe(DisplayProgress);
}
public int Width { get; set; } = 50;
private void DisplayProgress(double progress)
{
if (double.IsNaN(progress))
{
return;
}
var progressBarLenght = progress * Width;
System.Console.CursorLeft = 0;
System.Console.Write("[");
var bar = new string(Enumerable.Range(1, (int) progressBarLenght).Select(_ => '=').ToArray());
System.Console.Write(bar);
var label = $#"{progress:P0}";
System.Console.CursorLeft = (Width -label.Length) / 2;
System.Console.Write(label);
System.Console.CursorLeft = Width;
System.Console.Write("]");
}
public void Dispose()
{
progressUpdater?.Dispose();
}
}
Console Progress Bar in C# (Complete Code - Just copy and paste on your IDE)
class ConsoleUtility
{
const char _block = '■';
const string _back = "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b";
const string _twirl = "-\\|/";
public static void WriteProgressBar(int percent, bool update = false)
{
if (update)
Console.Write(_back);
Console.Write("[");
var p = (int)((percent / 10f) + .5f);
for (var i = 0; i < 10; ++i)
{
if (i >= p)
Console.Write(' ');
else
Console.Write(_block);
}
Console.Write("] {0,3:##0}%", percent);
}
public static void WriteProgress(int progress, bool update = false)
{
if (update)
Console.Write("\b");
Console.Write(_twirl[progress % _twirl.Length]);
}
}
//This is how to call in your main method
static void Main(string[] args)
{
ConsoleUtility.WriteProgressBar(0);
for (var i = 0; i <= 100; ++i)
{
ConsoleUtility.WriteProgressBar(i, true);
Thread.Sleep(50);
}
Console.WriteLine();
ConsoleUtility.WriteProgress(0);
for (var i = 0; i <= 100; ++i)
{
ConsoleUtility.WriteProgress(i, true);
Thread.Sleep(50);
}
}
I just stumbled upon this thread looking for something else, and I thought I'd drop off my code that I put together that downloads a List of files using the DownloadProgressChanged. I find this super helpful so I not only see the progress, but the actual size as the file is coming through. Hope it helps someone!
public static bool DownloadFile(List<string> files, string host, string username, string password, string savePath)
{
try
{
//setup FTP client
foreach (string f in files)
{
FILENAME = f.Split('\\').Last();
wc.DownloadFileCompleted += new AsyncCompletedEventHandler(Completed);
wc.DownloadProgressChanged += new DownloadProgressChangedEventHandler(ProgressChanged);
wc.DownloadFileAsync(new Uri(host + f), savePath + f);
while (wc.IsBusy)
System.Threading.Thread.Sleep(1000);
Console.Write(" COMPLETED!");
Console.WriteLine();
}
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
return false;
}
return true;
}
private static void ProgressChanged(object obj, System.Net.DownloadProgressChangedEventArgs e)
{
Console.Write("\r --> Downloading " + FILENAME +": " + string.Format("{0:n0}", e.BytesReceived / 1000) + " kb");
}
private static void Completed(object obj, AsyncCompletedEventArgs e)
{
}
Here's an example of the output:
Hope it helps someone!
I am still a little new to C# but I believe the below might help.
string[] xmlFilePath = Directory.GetFiles(xmlFullpath, "*.xml");
Console.WriteLine("Loading XML files...");
int count = 0;
foreach (string file in xmlFilePath)
{
//ExportXml(file, styleSheet);
drawTextProgressBar(count, xmlCount);
count++;
}
Based on all posts above, I did an improved version.
No jumping cursors. It's invisible now.
Improved performance (costs 1/5~1/10 time of the origin one).
Interface based. Easy to move to something else.
public class ConsoleProgressBar : IProgressBar
{
private const ConsoleColor ForeColor = ConsoleColor.Green;
private const ConsoleColor BkColor = ConsoleColor.Gray;
private const int DefaultWidthOfBar = 32;
private const int TextMarginLeft = 3;
private readonly int _total;
private readonly int _widthOfBar;
public ConsoleProgressBar(int total, int widthOfBar = DefaultWidthOfBar)
{
_total = total;
_widthOfBar = widthOfBar;
}
private bool _intited;
public void Init()
{
_lastPosition = 0;
//Draw empty progress bar
Console.CursorVisible = false;
Console.CursorLeft = 0;
Console.Write("["); //start
Console.CursorLeft = _widthOfBar;
Console.Write("]"); //end
Console.CursorLeft = 1;
//Draw background bar
for (var position = 1; position < _widthOfBar; position++) //Skip the first position which is "[".
{
Console.BackgroundColor = BkColor;
Console.CursorLeft = position;
Console.Write(" ");
}
}
public void ShowProgress(int currentCount)
{
if (!_intited)
{
Init();
_intited = true;
}
DrawTextProgressBar(currentCount);
}
private int _lastPosition;
public void DrawTextProgressBar(int currentCount)
{
//Draw current chunk.
var position = currentCount * _widthOfBar / _total;
if (position != _lastPosition)
{
_lastPosition = position;
Console.BackgroundColor = ForeColor;
Console.CursorLeft = position >= _widthOfBar ? _widthOfBar - 1 : position;
Console.Write(" ");
}
//Draw totals
Console.CursorLeft = _widthOfBar + TextMarginLeft;
Console.BackgroundColor = ConsoleColor.Black;
Console.Write(currentCount + " of " + _total + " "); //blanks at the end remove any excess
}
}
public interface IProgressBar
{
public void ShowProgress(int currentCount);
}
And some test code:
var total = 100;
IProgressBar progressBar = new ConsoleProgressBar(total);
for (var i = 0; i <= total; i++)
{
progressBar.ShowProgress(i);
Thread.Sleep(50);
}
Thread.Sleep(500);
Console.Clear();
total = 9999;
progressBar = new ConsoleProgressBar(total);
for (var i = 0; i <= total; i++)
{
progressBar.ShowProgress(i);
}
We are trying with below code.
public static int SplitFile(string fileName, string tmpFolder, List<string> queue, int splitSize = 100000)
{
int chunk = 0;
if (!Directory.Exists(tmpFolder))
Directory.CreateDirectory(tmpFolder);
using (var lineIterator = File.ReadLines(fileName).GetEnumerator())
{
bool stillGoing = true;
for (chunk = 0; stillGoing; chunk++)
{
stillGoing = WriteChunk(lineIterator, splitSize, chunk, tmpFolder, queue);
}
}
return chunk;
}
private static bool WriteChunk(IEnumerator<string> lineIterator,
int splitSize, int chunk, string tmpFolder, List<string> queue)
{
try
{
//int tmpChunkSize = 1000;
//int tmpChunkInc = 0;
string splitFile = Path.Combine(tmpFolder, "file" + chunk + ".txt");
using (var writer = File.CreateText(splitFile))
{
queue.Add(splitFile);
for (int i = 0; i < splitSize; i++)
{
if (!lineIterator.MoveNext())
{
return false;
}
writer.WriteLine(lineIterator.Current);
}
}
return true;
}
catch (Exception)
{
throw;
}
}
It creates around 36 text files (around 800 MB), but starting throwing "Out of memory exception" in creation of 37th File at lineIterator.MoveNext().
While lineIterator.Current shows the value in debugger.
As It s a huge file you should read it Seek and ReadBytes methods of BinaryReader.
You can see a simple example here. After you use the ReadBytes check for the last new lines and write the process file in certain amount of lines you read. Don t write every line you read and also don t keep all the data in the memory.
The rest is in your hands.
Maybe it is realted to that one When does File.ReadLines free resources
IEnumerable doesn't inherit from IDisposable because typically, the class that implements it only gives you the promise of being enumerable, it hasn't actually done anything yet that warrants disposal.