I'm designing an API. Currently I'm trying to safely handle a condition where we run out of disk space. Basically, we have a series of files holding some data. When the disk is full, when we go to write another data file, it will of course throw an error. At this point, we delete a single file(loop through file list from oldest to newest and retry after we successfully delete a file). Then, we retry writing the file. Repeat that process until the file is written without error.
Now the fun part. All of this happens concurrently. Like, at some point there are 8 threads doing this at once. This makes things extra interesting, and has lead to an odd error.
Here is the code
public void Save(string text, string id)
{
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
var existing = store.GetFileNames(string.Format(Prefix + "/*-{0}.dat", id));
if (existing.Any()) return; //it already is saved
string name = string.Format(Prefix + "/{0}-{1}.dat", DateTime.UtcNow.ToString("yyyyMMddHHmmssfffffff"), id);
tryagain:
bool doover=false;
try
{
AttemptFileWrite(store, name, text);
}
catch (IOException)
{
doover = true;
}
catch (IsolatedStorageException) //THIS LINE
{
doover = true;
}
if (doover)
{
Attempt(() => store.DeleteFile(name)); //because apparently this can also fail.
var files = store.GetFileNames(Path.Combine(Prefix, "*.dat"));
foreach (var file in files.OrderBy(x=>x))
{
try
{
store.DeleteFile(Path.Combine(Prefix, file));
}
catch
{
continue;
}
break;
}
goto tryagain; //prepare the velociraptor shield!
}
}
}
void AttemptFileWrite(IsolatedStorageFile store, string name, string text)
{
using (var file = store.OpenFile(
name,
FileMode.Create,
FileAccess.ReadWrite,
FileShare.None | FileShare.Delete
))
{
using (var writer = new StreamWriter(file))
{
writer.Write(text);
writer.Flush();
writer.Close();
}
file.Close();
}
}
static void Attempt(Action func)
{
try
{
func();
}
catch
{
}
}
static T Attempt<T>(Func<T> func)
{
try
{
return func();
}
catch
{
}
return default(T);
}
public string GetSaved()
{
string content=null;
using (var store = IsolatedStorageFile.GetUserStoreForApplication())
{
var files = store.GetFileNames(Path.Combine(Prefix,"*.dat")).OrderBy(x => x);
if (!files.Any()) return new MessageBatch();
foreach (var filename in files)
{
IsolatedStorageFileStream file=null;
try
{
file = Attempt(() =>
store.OpenFile(Path.Combine(Prefix, filename), FileMode.Open, FileAccess.ReadWrite, FileShare.None | FileShare.Delete));
if (file == null)
{
continue; //couldn't open. assume locked or some such
}
file.Seek(0L, SeekOrigin.Begin);
using (var reader = new StreamReader(file))
{
content = reader.ReadToEnd();
}
//take note here. We delete the file, while we still have it open!
//This is done because having the file open prevents other readers, but if we close it first,
//then there is a race condition that right after closing the stream, another reader could pick it up and
//open exclusively. It looks weird, but it's right. Trust me.
store.DeleteFile(Path.Combine(Prefix, filename));
if (!string.IsNullOrEmpty(content))
{
break;
}
}
finally
{
if (file != null) file.Close();
}
}
}
return content;
}
At the line marked THIS LINE, is what I'm talking about. When doing AttemptFileWrite, I can look over at store.AvailableSpace and see that there is enough room to fit the data into it, but upon trying to open the file, it throws this IsolatedStorageException with the description of Operation Not Permitted. Aside from this weird case, in all other cases it's just an IOException thrown with a message about the disk being full
I'm trying to figure out if I have some odd race condition, or if this is an error I just have to deal with or what?
Why does this error occur?
Related
I saw other threads about this problem, and non of them seems to solve my exact problem.
static void RecordUpdater(string username,int points,string term) //Updates Record File with New Records.
{
int minPoints = 0;
StreamWriter streamWriter = new StreamWriter($#"Record\{term}");
Player playersRecord = new Player(points, username);
List<Player> allRecords = new List<Player>();
StreamReader reader = new StreamReader($#"Record\{term}");
while (!reader.EndOfStream)
{
string[] splitText = reader.ReadLine().Split(',');
Player record = new Player(Convert.ToInt32(splitText[0]), splitText[1]);
allRecords.Add(record);
}
reader.Close();
foreach (var playerpoint in allRecords )
{
if(minPoints > playerpoint.points)
minPoints = playerpoint.points;
}
if (points > minPoints)
{
allRecords.Add(playersRecord);
allRecords.Remove(allRecords.Min());
}
allRecords.Sort();
allRecords.Reverse();
streamWriter.Flush();
foreach (var player in allRecords)
{
streamWriter.WriteLine(player.points + "," + player.username);
}
}
So after I run the program and get to that point in code I get an error message:
"The process cannot access the file 'fileName/textFile.txt' because it is being used by another process."
You should use the using statement around disposable objects like streams. This will ensure that the objects release every unmanaged resources they hold. And don't open the writer until you need it. Makes no sense to open the writer when first you need to read the records
static void RecordUpdater(string username,int points,string term)
{
Player playersRecord = new Player(points, username);
List<Player> allRecords = new List<Player>();
int minPoints = 0;
try
{
using(StreamReader reader = new StreamReader($#"Record\{term}"))
{
while (!reader.EndOfStream)
{
.... load you data line by line
}
}
..... process your data .....
using(StreamWriter streamWriter = new StreamWriter($#"Record\{term}"))
{
... write your data...
}
}
catch(Exception ex)
{
... show a message about the ex.Message or just log everything
in a file for later analysis
}
}
Also you should consider that working with files is one of the most probable context in which you could receive an exception due to external events in which your program has no control.
It is better to enclose everything in a try/catch block with proper handling of the exception
I'm working on an ASP.NET Core 5 project. I have this action method:
public async Task<IActionResult> CreateV3EnterCheckFile(IFormFile MarksFile)
{
var filesCount = Directory.GetFiles("Uploads").Length;
string path = Path.Combine("Uploads", filesCount + 1 + ".xlsx");
await MarksFile.SaveToAsync(path);
var xlImporter = new XLImporter();
var importedData = await xlImporter.ImportSheetAsync(path, 0);
var r = (from x in importedData select new { ID = x[0], StudentId = x[1] }).ToList();
System.IO.File.Delete(path);
return View();
}
I tried to get IFormFile uploaded file by the user to save it on the server and querying it using one of my projects (that uses LinqToExcel library).
I am querying the data and everything is perfect I still have just one problem it is this line of code:
System.IO.File.Delete(path);
It throws an exception and the message is I can't delete that file because it is still being used by another process.
I'm very sure that the process is related to the LinqToExcel library.
More details :
SaveToAsync is an extension method created by me that is its definition
public static Task SaveToAsync(this IFormFile file, string pathToSaveTo)
{
return Task.Factory.StartNew(() =>
{
using (Stream fileStream = File.Open(pathToSaveTo, FileMode.Create, FileAccess.ReadWrite, FileShare.None))
{
file.CopyTo(fileStream);
}
});
}
Please - is there any way or method or solution to delete this file even if it is being used by another process?
Massive thanks in advance.
Based on the source code of ExcelQueryFactory (https://github.com/paulyoder/LinqToExcel/blob/master/src/LinqToExcel/ExcelQueryFactory.cs) I would try the following:
ExcelQueryFactory has a ReadOnly Property. For read only access (if applicable) I would set it to true when creating the instance.
More important: IExcelQueryFactory implements IDisposable, so you can (should) use a using block:
using (var excelFile = new ExcelQueryFactory(pathToExcelFile) {ReadOnly = true})
{
// Do your work.
}
Of course you can use using var ..., but if you need a more reduced scope, the "old" using syntax allows more control.
I assumed that your Uploads folder is under webroot.
You can try this:-
public YourControllerName(IHostingEnvironment he) //input parameter
{
_he = he;
}
public async Task<IActionResult> CreateV3EnterCheckFile(IFormFile MarksFile)
{
try
{
var filesCount = Directory.GetFiles("Uploads").Length;
string contentRootPath = _he.ContentRootPath;
string path = Path.Combine(contentRootPath +"\\Uploads", filesCount + 1 + ".xlsx");
await MarksFile.SaveToAsync(path);
var xlImporter = new XLImporter();
var importedData = await xlImporter.ImportSheetAsync(path, 0);
var r = (from x in importedData select new { ID = x[0], StudentId = x[1] }).ToList();
//System.IO.File.Delete(path);
if (File.Exists(path))
{
File.Delete(path);
}
else
{
Debug.WriteLine("File does not exist.");
}
return View();
}
catch(Exception e)
{
Console.WriteLine(e);
}
Or you can try another process:-
try
{
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
System.IO.File.Delete(path);
}
catch(Exception e){
}
}
Or this:-
if (System.IO.File.Exists(path))
{
try
{
System.GC.Collect();
System.GC.WaitForPendingFinalizers();
System.IO.File.Delete(path);
}
catch (Exception e) { }
}
it should resolve your issue I hope. by the way, if your Upload folder is not under the webroot path. you can find your path using your process.
I'm reading a file to my C# application and decompressing a tile_data BLOB using a gzip stream. I'm currently accessing the BLOB data through this method:
SQLiteCommand command = new SQLiteCommand(query, DbConn);
SQLiteDataReader reader = command.ExecuteReader();
bool isMet = false;
while (reader.Read())
{
using (var file = reader.GetStream(0))
using (var unzip = new GZipStream(file, CompressionMode.Decompress))
using (var fileReader = new StreamReader(unzip))
{
var line = fileReader.ReadLine();
while (!fileReader.EndOfStream)
{
}
Console.WriteLine("End of tile_data");
}
}
reader.Close();
Console.WriteLine("Reader closed");
Console.ReadKey();
}
catch (Exception e)
{
Console.Write(e.StackTrace);
Console.ReadKey();
}
I'm looking to wait until the fileReader detects "tertiary" (string) and then prints all data afterwards. I attempted to use a bool and a nested while loop but it came back as an infinite loop, hence the question.
The code that I used (and failed with):
if (line.Contains("tertiary"))
{
isMet = true;
}
while (!fileReader.EndOfStream && isMet)
{
Console.WriteLine(line);
}
How can I perform an operation only with my fileReader once a condition has been met?
Your fileReader.EndOfStream loop will only work if your stream has only a single line. The problem is that you're only reading from the stream once - so unless you've already read the whole thing, you're in an endless loop.
Instead, do something like this:
string line;
while ((line = fileReader.ReadLine()) != null)
{
if (line.Contains("...")) break; // Or whatever else you want to do
}
You can use this code. It find any character in the string. It is working for me.
string matchStr = "tertiary";
if (line.Any(matchStr.Contains)
{
isMet = true;
}
while (!fileReader.EndOfStream && isMet)
{
Console.WriteLine(line);
}
The line var line = fileReader.ReadLine(); be inside the while loop?
while (!fileReader.EndOfStream)
{
var line = fileReader.ReadLine();
// do some other stuff
}
Try something like this
var yourval;
var secval = fileReader.ReadLine()
while ((yourval = secval ) != null)
{
if (line.Contains("your string here"))
{
break;
}
}
I'm writing a plugin for AutoCAD and want to import all the blocks it will use at the beginning to make sure that they are available when needed. To do that, I use this method
public static void ImportBlocks(string[] filesToTryToImport, string filter = "")
{
foreach (string blockToImport in filesToTryToImport)
{
if (blockToImport.Contains(filter))
{
Database sourceDb = new Database(false, true); //Temporary database to hold data for block we want to import
try
{
sourceDb.ReadDwgFile(blockToImport, System.IO.FileShare.Read, true, ""); //Read the DWG into a side database
ObjectIdCollection blockIds = new ObjectIdCollection(); // Create a variable to store the list of block identifiers
Autodesk.AutoCAD.DatabaseServices.TransactionManager tm = sourceDb.TransactionManager;
using (Transaction myT = tm.StartTransaction())
{
// Open the block table
BlockTable bt = (BlockTable)tm.GetObject(sourceDb.BlockTableId, OpenMode.ForRead, false);
// Check each block in the block table
foreach (ObjectId btrId in bt)
{
BlockTableRecord btr = (BlockTableRecord)tm.GetObject(btrId, OpenMode.ForRead, false);
// Only add named & non-layout blocks to the copy list
if (!btr.IsAnonymous && !btr.IsLayout)
{
blockIds.Add(btrId);
}
btr.Dispose();
}
}
// Copy blocks from source to destination database
IdMapping mapping = new IdMapping();
sourceDb.WblockCloneObjects(blockIds, _database.BlockTableId, mapping, DuplicateRecordCloning.Replace, false);
_editor.WriteMessage("\nCopied " + blockIds.Count.ToString() + " block definitions from " + blockToImport + " to the current drawing.");
}
catch (Autodesk.AutoCAD.Runtime.Exception ex)
{
_editor.WriteMessage("\nError during copy: " + ex.Message);
}
finally
{
sourceDb.Dispose();
}
}
}
}
That method appears to work because it successfully executes. However when I go to insert a block in the drawing via AutoCAD's interface it doesn't show up as an option and when I try to insert it programmatically it throws a FileNotFound exception meaning it didn't work. What's wrong with this method? Thanks in advance!
EDIT: Here is a less complicated method with a test method
public static void ImportSingleBlock(string fileToTryToImport)
{
using (Transaction tr = _database.TransactionManager.StartTransaction())
{
Database sourceDb = new Database(false, true); //Temporary database to hold data for block we want to import
try
{
sourceDb.ReadDwgFile(fileToTryToImport, System.IO.FileShare.Read, true, ""); //Read the DWG into a side database
_database.Insert(fileToTryToImport, sourceDb, false);
_editor.WriteMessage("\nSUCCESS: " + fileToTryToImport);
}
catch (Autodesk.AutoCAD.Runtime.Exception ex)
{
_editor.WriteMessage("\nERROR: " + fileToTryToImport);
}
finally
{
sourceDb.Dispose();
}
tr.Commit();
}
}
[CommandMethod("TESTSINGLEBLOCKIMPORTING")]
public void TestSingleBlockImporting()
{
OpenFileDialog ofd = new OpenFileDialog();
DialogResult result = ofd.ShowDialog();
if (result == DialogResult.Cancel) //Ending method on cancel
{
return;
}
string fileToTryToImport = ofd.FileName;
using (Transaction tr = _database.TransactionManager.StartTransaction())
{
EntityMethods.ImportSingleBlock(fileToTryToImport);
tr.Commit();
}
}
This file is the block I'm trying to import. Hope this inspires someone cause I am desperately lost right now.
Your code is correct and should work. In fact I did tried it and works fine. You're probably missing to Commit() a outer transaction (where you call this ImportBlocs() method). Check:
using (Transaction trans = _database.TransactionManager.StartTransaction())
{
ImportBlocks(... parameters here ...);
trans.Commit(); // remember to call this commit, if omitted, Abort() is assumed
}
I had the same issue, very similar code. Issue was that
_database.Insert(fileToTryToImport, sourceDb, false);
should be
_database.Insert(blockName, sourceDb, false);
You can see that the first parameter has to be "blockName", and not the file path.
I am using the elency solutions CSV library for C# to save and load some data from a file.
My code saves and loads correctly, but when I load and then try to save an error occurs, saying that another process is using the file.
The load method is this:
private void loadfile(string name)
{
int key = 696969;
CsvReader read = new CsvReader("data.csv");
try
{
do
{
read.ReadNextRecord();
} while (name != read.Fields[0]);
int decAgain = int.Parse(read.Fields[1], System.Globalization.NumberStyles.HexNumber); //convert to int
int dec = decAgain ^ key;
MessageBox.Show(dec.ToString());
}
catch (Exception)
{
MessageBox.Show("Not Found");
}
read = null;
}
As you can see, I am sort of disposing the "read" object.
Here is the save method:
private void savefile(string encrypted, string name)
{
CsvFile file = new CsvFile();
CsvRecord rec = new CsvRecord();
CsvWriter write = new CsvWriter();
rec.Fields.Add(name);
rec.Fields.Add(encrypted);
file.Records.Add(rec);
write.AppendCsv(file, "data.csv");
file = null;
rec = null;
write = null;
}
It always gets stuck on append csv.
I do understand the problem. The reader is not being closed successfully. How can I correctly close the file?
NB: I have tried read.Dispose() but it is not working.
Can you please help me out?
Regards
Use using to automatically dispose object. It may solve your issue.
private void savefile(string encrypted, string name)
{
using(CsvFile file = new CsvFile())
{
using(CsvRecord rec = new CsvRecord())
{
using(CsvWriter write = new CsvWriter())
{
rec.Fields.Add(name);
rec.Fields.Add(encrypted);
file.Records.Add(rec);
write.AppendCsv(file, "data.csv");
}
}
}
}