We Keep Coding

Blazor c# - how to XmlDocument.load() a local file

New to Blazor (web-assembly), so go easy :)
I would like to be able to select an xml file from local disk (in the region of 100mb in size), via InputFile and have it loaded in to XmlDocument so I can interrogate it.
Upon trying to load a file of this size, it crashes out on XmlDocument.load(). Unsure why.
I can get it to load smaller file sizes via a OpenReadStream and setting the maxAllowedSize to like 105000000 but they take a complete age in comparison to say loading it from a WPF c# app.
I'm unsure whether the stream is causing the slowness, or whether I'm missing something fundamental in accessing local disk files during the XmlDocument load process of this size?
Any help is greatly appreciated.
So basically, all i want to be able to do is something like this...
<InputFile OnChange="LoadFile">Select file...</InputFile>
#{
private void LoadFile()
{
XmlDocument newXml = new XmlDocument();
newXml.load(ChosenFilePath); //ChosenFilePath or hardcoded path for testing purposes still fails i.e. #"C:\temp\TestFile.xml"
}
}

You need to accept the InputFileChangeEventArgs and get your file through it, that's relatively simple, your event handler should be this:
private void LoadFile(InputFileChangeEventArgs args)
{
// The argument is to increase the maximum size, by default it only allows
// you to open files less than or equal to 500KiB, the number below is 200MiB
// Alternatively you could pass 'args.File.Size' to allow opening an arbitraily large file
var fileStream = args.File.OpenReadStream(200 * 1024 * 1024);
var newXml = new XmlDocument();
newXml.Load(fileStream);
}

is it possible to get the size of a zip file while it is being created? CreateEntryFromFile

I would like to know if it is possible to check the size of the zip file that is being created dynamically, because I am reading a directory and generate a 19 MB zip and I would like two zips instead to be created, one 10MB and the other 9MB. However, when I give a .Length in the zip file inside my loop it says the size is 0. When I finish adding the files it says that is 19MB. Would anyone know how to do this?
I am using only System.IO.Compression to this task.
here is some example to show how I am trying
String FilePath = "D:\Invoices";
string[] oFiles = Directory.GetFiles(FilePath,"*.pdf");
string name = DateTime.Now.Ticks.ToString()+".zip";
using(FileStream archive1 = File.Open(name,FileMode.Create))
{
using(var arch = new ZipArchive(archive1,ZipArchiveMode.Update))
{
for(int i =0; i<oFiles.Length;i++)
{
var fileinf = new FileInfo(oFiles[i]);
arch.CreateEntryFromFile(fileinf.FullName,fileinf.Name);
//here the zip file is always 0
Console.WriteLine(archive1.Length);
}
}
}
//here the zip file is updated

From the documentation:
When you set the mode to Update … The content of the entire archive is held in memory, and no data is written to the underlying file or stream until the archive is disposed.
If you want to be able to read the size of the file as you're adding things, you need to use ZipArchiveMode.Create.
Otherwise, you should use the ZipArchiveEntry.CompressedSize property to monitor how much you've added to the archive.

Prune simple text log file using C# .NET 4.0

An external Windows service I work with maintains a single text-based log file that it continuously appends to. This log file grows unbounded over time. I'd like to prune this log file periodically to maintain, say the most recent 5mb of log entries. How can I efficiently implement the file I/O code in C# .NET 4.0 to prune the file to say 5mb?
Updated:
The way service dependencies are set up, my service always starts before the external service. This means I get exclusive access to the log file to truncate it, if required. Once the external service starts up, I will not access the log file. I can gain exclusive access to the file on desktop startup. The problem is - the log file may a few gigabytes in size and I'm looking for an efficient way to truncate it.

It's going to take the amount of memory that you want to store to process the "new" log file but if you only want 5Mb then it should be fine. If you are talking about Gb+ then you probably have other problems; however, it could still be accomplished using a temp file and some locking.
As noted before, you may experience a race condition but that's not the case if this is the only thread writing to this file. This would replace your current writing to the file.
const int MAX_FILE_SIZE_IN_BYTES = 5 * 1024 * 1024; //5Mb;
const string LOG_FILE_PATH = #"ThisFolder\log.txt";
string newLogMessage = "Hey this happened";
#region Use one or the other, I mean you could use both below if you really want to.
//Use this one to save an extra character
if (!newLogMessage.StartsWith(Environment.NewLine))
newLogMessage = Environment.NewLine + newLogMessage;
//Use this one to imitate a write line
if (!newLogMessage.EndsWith(Environment.NewLine))
newLogMessage = newLogMessage + Environment.NewLine;
#endregion
int newMessageSize = newLogMessage.Length*sizeof (char);
byte[] logMessage = new byte[MAX_FILE_SIZE_IN_BYTES];
//Append new log to end of "file"
System.Buffer.BlockCopy(newLogMessage.ToCharArray(), 0, logMessage, MAX_FILE_SIZE_IN_BYTES - newMessageSize, logMessage.Length);
FileStream logFile = File.Open(LOG_FILE_PATH, FileMode.Open, FileAccess.ReadWrite);
int sizeOfRetainedLog = (int)Math.Min(MAX_FILE_SIZE_IN_BYTES - newMessageSize, logFile.Length);
//Set start position/offset of the file
logFile.Position = logFile.Length - sizeOfRetainedLog;
//Read remaining portion of file to beginning of buffer
logFile.Read(logMessage, logMessage.Length, sizeOfRetainedLog);
//Clear the file
logFile.SetLength(0);
logFile.Flush();
//Write the file
logFile.Write(logMessage, 0, logMessage.Length);
I wrote this really quick, I apologize if I'm off by 1 somewhere.

depending on how often it is written to I'd say you might be facing a race condition to modify the file without damaging the log. You could always try writing a service to monitor the file size, and once it reaches a certain point lock the file, dupe and clear the whole thing and close it. Then store the data in another file that the service controls the size of easily. Alternatively you could see if the external service has an option for logging to a database, which would make it pretty simple to roll out the oldest data.

You could use a file observer to monitor the file:
FileSystemWatcher logWatcher = new FileSystemWatcher();
logWatcher.Path = #"c:\example.log"
logWatcher.Changed += logWatcher_Changed;
Then when the event is raised you can use a StreamReader to read the file
private void logWatcher_Changed(object sender, FileSystemEventArgs e)
{
using (StreamReader readFile = new StreamReader(path))
{
string line;
string[] row;
while ((line = readFile.ReadLine()) != null)
{
// Here you delete the lines you want or move it to another file, so that your log keeps small. Then save the file.
}
}
}
It´s an option.

When modifying a file, is it always necessary to rewrite the entire file?

When modifying a file, is it always necessary to rewrite the entire file or is it possible to find and change some small part of a file without having to rewrite the whole thing?

If you're not inserting or deleting data, then you don't need to rewrite the file. You will be replacing existing bytes with new values, or appending to the file.
If you need to insert or delete, you only need to rewrite the file from that point onwards. The only time you would need to rewrite the entire file is if you are inserting bytes at the beginning.
It's okay to open a file with both read and write permissions. That way you can search the file for whatever you're looking for, and once you have the position you can seek to it (from memory, the write pointer is separate from the read pointer) and overwrite data to your heart's content =)

If you're not changing the length of the data, you can always just seek to the appropriate position in the file, and write a new set of bytes. This replaces whatever bytes where originally there.

There are two possible ways:
when you use StreamWriter sw = new StreamWriter it will delete the data from file.
To change only, you can store the file data in a List<string>. This works nice for me:
List<string> lines_list = new List<string>();
int file_l = 0
StreamReader sr_temp = new StreamReader(_path);
string line;
while ((line = sr_temp.ReadLine()) != null)
{
lines_list.Add(line);
file_l++;
}
sr_temp.Close();
StreamWriter sw = new StreamWriter(_path);
for (int i = 0; i < file_l; i++)
{
sw.WriteLine(lines_list[i]);
}
//here you add some data
sw.Close();

Insert data into text file

I want to insert the data at some positions in the text file without actually overwriting on the existing data. I have two text file. "one.txt" file have 1000 lines, "two.txt" file have 10000 lines. I want to read "one.txt" file content and insert into first 1000 lines of "two.txt" file content(Append the content of "one.txt" to the beginning of "two.txt").
Criteria:
Minimum code .
Less Memory consumption(irrespective of programming language )
Performance (will be considered based on size of the file).

just open up a streamreader for the first file, and a stream writer (in append mode) for the second file. As your reading the first 1000 lines from the first file, insert them into the second.
Something like this:
StreamReader sr = new StreamReader("one.txt");
StreamWriter sw = new StreamWriter("two.txt", true); //true for append
index i = 0;
while (i < 1000) {
sw.WriteLine(sr.ReadLine());
i++;
}
You may want to check for end of file on the StreamReader, but this will give you the general idea....
Based on the new information in OP:
You can use this same type of method, but just create a brand new file, reading the data from the first file, followed by the data from the second file. Once it's inside the new file, replace the original "two.txt".

If you're not limited to c# you can just do the following from a windows command line:
copy one.txt + two.txt three.txt
This would create the file you want, but it would be called three.txt. If you must have it in two.txt, you could simply rename two.txt to something else first and then do the copy append with two.txt as the third parm.

If you only have to do this once, here is some code that will do what you want. I did not compile this, but I believe there are no issues.
string[] linesOne = File.ReadAllLines(pathToFile1);
string[] linesTwo = File.ReadAllLines(pathToFile2);
List<string> result = new List<string>();
for(int i=0;i<1000;i++)
{
result.Add(linesOne[i]);
}
result.AddRange(linesTwo);
File.WriteAllLines(pathToFile2, result);
Hope this gets you started.
Bob