In a windows store app, how can I delete text from a file ? For example
If I have
StorageFile file = await roamingfolder.CreateFileAsync(filename,
CreationCollisionOption.OpenIfExists);
await FileIO.AppendTextAsync(file, temp);
How can i remove some text from this file ?
You generally read the text into a string, remove the text, and rewrite the file.
Here I get a file then I put the content to a stringbuilder then do some string operations, finally put the string back to the file using DataWriter
public static async Task UpdateTextContent(string contentItemId)
{
var storageFolder = await ApplicationData.Current.LocalFolder.GetFolderAsync(TARGET_FOLDER);
StorageFile sf = null;
try
{
//get content of the file make sure that it exist
sf = await storageFolder.GetFileAsync(TARGET_FILE);
}
catch (Exception ex)
{
}
if (sf != null)
{
var targettxtfile = await Windows.Storage.FileIO.ReadTextAsync(sf);
var sbProcessedTextToWrite = new StringBuilder(targettxtfile);
if (targettxtfile.IndexOf(contentItemId) >= 0)
{
string startmarker = new StringBuilder("[").Append(contentItemId).Append("#start]").ToString();
string endmarker = new StringBuilder("[").Append(contentItemId).Append("#end]").ToString();
int start = sbProcessedTextToWrite.ToString().IndexOf(startmarker);
int end = sbProcessedTextToWrite.ToString().IndexOf(endmarker);
int slen = end + endmarker.Length - start;
//compute position to remove
sbProcessedTextToWrite.Remove(start, slen);
}
using (IRandomAccessStream fileStream = await sf.OpenAsync(FileAccessMode.ReadWrite))
{
using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
{
using (DataWriter dataWriter = new DataWriter(outputStream))
{
dataWriter.WriteString(sbProcessedTextToWrite.ToString());
await dataWriter.StoreAsync();
// For the in-memory stream implementation we are using, the flushAsync call
// is superfluous,but other types of streams may require it.
await dataWriter.FlushAsync();
// In order to prolong the lifetime of the stream, detach it from the
// DataWriter so that it will not be closed when Dispose() is called on
// dataWriter. Were we to fail to detach the stream, the call to
// dataWriter.Dispose() would close the underlying stream, preventing
// its subsequent use by the DataReader below.
dataWriter.DetachStream();
}
//same here flush the outputStream as well
await outputStream.FlushAsync();
}
}
}
}
Some references for this code
Related
I want to send the content of file as memory stream to S3 bucket via Amazon Firehose. below is my attempt which works fine for small files, but I have a file of 1 GB and I am getting {"Exception of type 'System.OutOfMemoryException' was thrown."}.
My code snippet:
[HttpPost]
public async Task<bool> Upload()
{
try
{
var filesReadToProvider = await Request.Content.ReadAsMultipartAsync();
foreach (var stream in filesReadToProvider.Contents)
{
var fileBytes = await stream.ReadAsByteArrayAsync(); // THIS IS WHERE EXCEPTION COMES
using (MemoryStream memoryStream = new MemoryStream(fileBytes))
{
PutRecordRequest putRecord = new PutRecordRequest();
putRecord.DeliveryStreamName = myStreamName;
Record record = new Record();
record.Data = memoryStream;
putRecord.Record = record;
await kinesisClient.PutRecordAsync(putRecord);
}
}
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
return true;
}
I looked into this link OutOfMemoryExceptoin but I could not comprehend it. Please help me.
Attempt 1:
var filesReadToProvider = await Request.Content.ReadAsMultipartAsync();
foreach (var stream in filesReadToProvider.Contents)
{
var fileByte = await stream.ReadAsStreamAsync();
MemoryStream _ms = new MemoryStream();
fileByte.CopyTo(_ms); // EXCEPTION HERE
try
{
PutRecordRequest putRecord = new PutRecordRequest();
putRecord.DeliveryStreamName = myStreamName;
Record record = new Record();
record.Data = _ms;
putRecord.Record = record;
await kinesisClient.PutRecordAsync(putRecord);
}
catch (Exception ex)
{
Console.WriteLine("Failed to send record to Kinesis. Exception: {0}", ex.Message);
}
}
[HttpPost]
public async Task<bool> Upload()
{
try
{
using(var requestStream = await Request.Content.ReadAsStreamAsync())
{
PutRecordRequest putRecord = new PutRecordRequest();
putRecord.DeliveryStreamName = myStreamName;
Record record = new Record();
record.Data = requestStream ;
putRecord.Record = record;
await kinesisClient.PutRecordAsync(putRecord);
}
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
return true;
}
This will read the data in chunks. Keep everything in the Stream so you don't keep all the bytes around in a huge array.
When reading large files, I use StreamReader's Readline() method. It works on large files as it manages file system caching internally. Can you use this method, instead? Is there a reason why you are implementing the MemoryStream class? You have a comment asking how to inject the data? Did you try using one of MemoryStream's methods???
https://learn.microsoft.com/en-us/dotnet/api/system.io.memorystream?view=netframework-4.7.2
Update:
Not sure if this is helpful since the code is substantially different from what you are using. But, yours isn't working either, so just a suggestion.
http://www.tugberkugurlu.com/archive/efficiently-streaming-large-http-responses-with-httpclient
I am attempting to download a list of files from urls stored in my database, and then upload them to my Azure FileStorage account. I am successfully downloading the files and can turn them into files on my local storage or convert them to text and upload them. However I lose data when converting something like a pdf to a text and I do not want to have to store the files on the Azure app that this endpoint is hosted on as I do not need to manipulate the files in any way.
I have attempted to upload the files from the Stream I get from the HttpContent object using the UploadFromStream method on the CloudFile. Whenever this command is run I get an InvalidOperationException with the message "Operation is not valid due to the current state of the object."
I've tried converting the original Stream to a MemoryStream as well but this just writes a blank file to the FileStorage account, even if I set the position to the beginning of the MemoryStream. My code is below and if anyone could point out what information I am missing to make this work I would appreciate it.
public DownloadFileResponse DownloadFile(FileLink fileLink)
{
string fileName = string.Format("{0}{1}{2}", fileLink.ExpectedFileName, ".", fileLink.ExpectedFileType);
HttpStatusCode status;
string hash = "";
using (var client = new HttpClient())
{
client.Timeout = TimeSpan.FromSeconds(10); // candidate for .config setting
client.DefaultRequestHeaders.Add("User-Agent", USER_AGENT);
var request = new HttpRequestMessage(HttpMethod.Get, fileLink.ExpectedURL);
var sendTask = client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var response = sendTask.Result; // not ensuring success here, going to handle error codes without exceptions
status = response.StatusCode;
if (status == HttpStatusCode.OK)
{
var httpStream = response.Content.ReadAsStreamAsync().Result;
fileStorage.WriteFile(fileLink.ExpectedFileType, fileName, httpStream);
hash = HashGenerator.GetMD5HashFromStream(httpStream);
}
}
return new DownloadFileResponse(status, fileName, hash);
}
public void WriteFile(string targetDirectory, string targetFilePath, Stream fileStream)
{
var options = SetOptions();
var newFile = GetTargetCloudFile(targetDirectory, targetFilePath);
newFile.UploadFromStream(fileStream, options: options);
}
public FileRequestOptions SetOptions()
{
FileRequestOptions options = new FileRequestOptions();
options.ServerTimeout = TimeSpan.FromSeconds(10);
options.RetryPolicy = new NoRetry();
return options;
}
public CloudFile GetTargetCloudFile(string targetDirectory, string targetFilePath)
{
if (!shareConnector.share.Exists())
{
throw new Exception("Cannot access Azure File Storage share");
}
CloudFileDirectory rootDirectory = shareConnector.share.GetRootDirectoryReference();
CloudFileDirectory directory = rootDirectory.GetDirectoryReference(targetDirectory);
if (!directory.Exists())
{
throw new Exception("Target Directory does not exist");
}
CloudFile newFile = directory.GetFileReference(targetFilePath);
return newFile;
}
Had the same problem, the only way it worked is by reading the coming stream (in your case it is httpStream in DownloadFile(FileLink fileLink) method) to a byte array and using UploadFromByteArray (byte[] buffer, int index, int count) instead of UploadFromStream
So your WriteFile(FileLink fileLink) method will look like:
public void WriteFile(string targetDirectory, string targetFilePath, Stream fileStream)
{
var options = SetOptions();
var newFile = GetTargetCloudFile(targetDirectory, targetFilePath);
const int bufferLength= 600;
byte[] buffer = new byte[bufferLength];
// Buffer to read from stram This size is just an example
List<byte> byteArrayFile = new List<byte>(); // all your file will be here
int count = 0;
try
{
while ((count = fileStream.Read(buffer, 0, bufferLength)) > 0)
{
byteArrayFile.AddRange(buffer);
}
fileStream.Close();
}
catch (Exception ex)
{
throw; // you need to change this
}
file.UploadFromByteArray(allFile.ToArray(), 0, byteArrayFile.Count);
// Not sure about byteArrayFile.Count.. it should work
}
According to your description and codes, I suggest you could use Steam.CopyTo to copy the stream to the local memoryStream firstly, then upload the MemoryStream to azure file storage.
More details, you could refer to below codes:
I just change the DownloadFile method to test it.
HttpStatusCode status;
using (var client = new HttpClient())
{
client.Timeout = TimeSpan.FromSeconds(10); // candidate for .config setting
// client.DefaultRequestHeaders.Add("User-Agent", USER_AGENT);
//here I use my blob file to test it
var request = new HttpRequestMessage(HttpMethod.Get, "https://xxxxxxxxxx.blob.core.windows.net/media/secondblobtest-eypt.txt");
var sendTask = client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var response = sendTask.Result; // not ensuring success here, going to handle error codes without exceptions
status = response.StatusCode;
if (status == HttpStatusCode.OK)
{
MemoryStream ms = new MemoryStream();
var httpStream = response.Content.ReadAsStreamAsync().Result;
httpStream.CopyTo(ms);
ms.Position = 0;
WriteFile("aaa", "testaa", ms);
// hash = HashGenerator.GetMD5HashFromStream(httpStream);
}
}
I had a similar problem and got to find out that the UploadFromStream method only works with buffered streams. Nevertheless I was able to successfully upload files to azure storage by using a MemoryStream. I don't think this to be a very good solution as you are using up your memory resources by copying the content of the file stream to memory before handing it to the azure stream. What I have come up with is a way of writing directly to an azure stream by using instead the OpenWriteAsync method to create the stream and then a simple CopyToAsync from the source stream.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse( "YourAzureStorageConnectionString" );
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
CloudFileShare share = fileClient.GetShareReference( "YourShareName" );
CloudFileDirectory root = share.GetRootDirectoryReference();
CloudFile file = root.GetFileReference( "TheFileName" );
using (CloudFileStream fileWriteStream = await file.OpenWriteAsync( fileMetadata.FileSize, new AccessCondition(),
new FileRequestOptions { StoreFileContentMD5 = true },
new OperationContext() ))
{
await fileContent.CopyToAsync( fileWriteStream, 128 * 1024 );
}
I'm working in VS2012, WinRT and C#.
I'm trying to delete some files after decompressing them. I'm getting an "Access is denied" error. If I stop the app and re-start it the same code works fine so it appears there is a handle still attached.
If I don't call the unZipFile method, I can delete the files.
Is there a definitive way to release a file? I've set it to null (file = null;) before the call to delete.
Here's the block of code that calls the unzip method:
StorageFile file = await CreateOutputFile(fileName, path);
MemoryStream theMemStream = new MemoryStream();
theMemStream.Write(bytes, 0, bytes.Length);
await FileIO.WriteBytesAsync(file, bytes);
await theMemStream.FlushAsync();
theMemStream.Dispose();
var result = await unZipFile(file, path);
file = null;
Here's the unZipFile method:
private async Task<string> unZipFile(StorageFile file, string path)
{
StorageFolder sf = await GetOutputFolder(path);
using (var zipStream = await file.OpenStreamForReadAsync())
{
using (MemoryStream zipMemoryStream = new MemoryStream((int)zipStream.Length))
{
await zipStream.CopyToAsync(zipMemoryStream);
try
{
var archive = SharpCompress.Archive.ArchiveFactory.Open(file.Path);
foreach (var entry in archive.Entries)
{
entry.WriteTo(zipMemoryStream);
Stream fileData = entry.OpenEntryStream();
StorageFile outputFile = await sf.CreateFileAsync(entry.FilePath, CreationCollisionOption.ReplaceExisting);
using (Stream outputFileStream = await outputFile.OpenStreamForWriteAsync())
{
await fileData.CopyToAsync(outputFileStream);
await outputFileStream.FlushAsync();
outputFileStream.Dispose();
}
}
archive = null;
}
catch (Exception ex)
{
throw new IOException("Error writing decompressed output file: " + ex.Message);
}
await zipStream.FlushAsync();
zipStream.Dispose();
await zipMemoryStream.FlushAsync();
zipMemoryStream.Dispose();
}
}
return "success";
}
Here's the delete method. This is called for each file after decompression:
private async Task<string> deleteFile(string path, string filename)
{
StorageFolder folder = await GetOutputFolder(path);
var files = await folder.GetFilesAsync();
foreach (StorageFile file in files)
{
try
{
if (file != null)
{
if (file.Name == filename)
await file.DeleteAsync();
}
}
catch (Exception ex)
{
return ex.Message;
}
}
return "success";
}
On what file do you get the exception, the extracted files or the zip archive itself?
If the latter is the case, ArchiveFactory.Open() returns an IArchive which inherits IDisposable, so you should wrap var archive = SharpCompress.Archive.ArchiveFactory.Open(file.Path); in a using block so it gets disposed after use.
I'm trying to convert an old game of mine to windows 8 and I'm having a lot of trouble with my file loading. I'm trying a simple test with DataReader but I don't get the correct values.
First I write like this:
StorageFolder folder = ApplicationData.Current.LocalFolder;
StorageFile file = await folder.CreateFileAsync("file.dat",CreationCollisionOption.ReplaceExisting);
using (IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
{
using (DataWriter writer = new DataWriter(outputStream))
{
try
{
writer.UnicodeEncoding = UnicodeEncoding.Utf8;
writer.ByteOrder = ByteOrder.LittleEndian;
writer.WriteInt32(1);
writer.WriteInt32(2);
await writer.StoreAsync();
writer.DetachStream();
}
catch (IOException)
{
}
}
}
}
Then I read
StorageFolder folder = ApplicationData.Current.LocalFolder;
StorageFile file = await folder.GetFileAsync("file.dat");
using (var fileStream = await file.OpenReadAsync())
{
using (IInputStream inputStream = fileStream.GetInputStreamAt(0))
{
using (DataReader reader = new DataReader(inputStream))
{
reader.UnicodeEncoding = UnicodeEncoding.Utf8;
reader.ByteOrder = ByteOrder.LittleEndian;
await reader.LoadAsync((uint)fileStream.Size);
var number = reader.ReadInt32();
var number2 = reader.ReadInt32();
reader.DetachStream();
}
}
}
But I don't get 1 and 2 when I read, just two really big numbers. So, something I missed this is now how you do it? I'm also trying to figure out the best way to work with strings, am I suppose to also write the byte length now as it ask for a "codeUnitCount" when I read?
It just seems like everything is a step back from the old binary reader.
I'm having trouble when creating a StreamWriter object in windows-8, usually I just create an instance just passing a string as a parameter, but in Windows 8 I get an error that indicates that it should recieve a Stream, but I noticed that Stream is an abstract class, Does anybody knows how will be the code to write an xml file?, BTW I'm using .xml because I want to save the serialized object, or does anyone knows how to save to a file a serialized object in Windows 8?.
Any ideas?
Currently using Windows 8 Consumer Preview
Code:
StreamWriter sw = new StreamWriter("person.xml");
Error:
The best overloaded method match for 'System.IO.StreamWriter.StreamWriter(System.IO.Stream)' has some invalid arguments
Instead of StreamWriter you would use something like this:
StorageFolder folder = ApplicationData.Current.LocalFolder;
StorageFile file = await folder.CreateFileAsync();
using (IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
{
using (DataWriter dataWriter = new DataWriter(outputStream))
{
//TODO: Replace "Bytes" with the type you want to write.
dataWriter.WriteBytes(bytes);
await dataWriter.StoreAsync();
dataWriter.DetachStream();
}
await outputStream.FlushAsync();
}
}
You can look at the StringIOExtensions class in the WinRTXamlToolkit library for sample use.
EDIT*
While all the above should work - they were written before the FileIO class became available in WinRT, which simplifies most of the common scenarios that the above solution solves since you can now just call await FileIO.WriteTextAsync(file, contents) to write text into file and there are also similar methods to read, write or append strings, bytes, lists of strings or IBuffers.
You can create a common static method which you can use through out application like this
private async Task<T> ReadXml<T>(StorageFile xmldata)
{
XmlSerializer xmlser = new XmlSerializer(typeof(List<myclass>));
T data;
using (var strm = await xmldata.OpenStreamForReadAsync())
{
TextReader Reader = new StreamReader(strm);
data = (T)xmlser.Deserialize(Reader);
}
return data;
}
private async Task writeXml<T>(T Data, StorageFile file)
{
try
{
StringWriter sw = new StringWriter();
XmlSerializer xmlser = new XmlSerializer(typeof(T));
xmlser.Serialize(sw, Data);
using (IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
{
using (DataWriter dataWriter = new DataWriter(outputStream))
{
dataWriter.WriteString(sw.ToString());
await dataWriter.StoreAsync();
dataWriter.DetachStream();
}
await outputStream.FlushAsync();
}
}
}
catch (Exception e)
{
throw new NotImplementedException(e.Message.ToString());
}
}
to write xml simply use
StorageFile file = await ApplicationData.Current.LocalFolder.CreateFileAsync("data.xml",CreationCollisionOption.ReplaceExisting);
await writeXml(Data,file);
and to read xml use
StorageFile file = await ApplicationData.Current.LocalFolder.GetFileAsync("data.xml");
Data = await ReadXml<List<myclass>>(file);