We Keep Coding

Writing to a blob stream - how to fail and clean up?

Using Azure storage, I'm writing to a blob using a stream. I have a method something like this:
public async Task<BlobSteamContainer> GetBlobStreamAsync(string filename, string contentType = "text/csv")
{
var blob = container.GetBlockBlobReference($"{filename}--{Guid.NewGuid().ToString()}.csv");
blob.Properties.ContentType = contentType;
return new BlobSteamContainer(blob.Uri.ToString(), await blob.OpenWriteAsync());
}
Where BlobStreamContainer is just a simple object so I can keep track of the filename and the stream together:
public class BlobSteamContainer : IDisposable
{
public CloudBlobStream Stream { get; private set; }
public string Filename { get; private set; }
public BlobSteamContainer(string filename, CloudBlobStream stream)
{
Stream = stream;
Filename = filename;
}
public void Dispose()
{
Stream.Close();
Stream?.Dispose();
}
}
And then I use it something like this:
using (var blobStream = await GetBlobStreamAsync(filename))
using (var outputStream = new StreamWriter(blobStream.Stream))
using (var someInputStream = ...)
{
try
{
outputStream.WriteLine("write some stuff...");
//....processing
if (someCondition) {
throw new MyException("can't write the file");
}
//....more processing
outputStream.Flush();
}
catch(MyException e)
{
// what to do here? I want to stop writing
// and remove any trace of the file in azure
throw; // let the higher ups handle this
}
}
Where somecondition is something that I know before hand (obviously there's more going on involving processing an input stream and writing out as I go). If everything is fine, then this works great. My problem is on figuring out the best way to handle the case where an exception is thrown during writing.
I tried just deleting the file in the catch like this:
DeleteBlob(blobStream.Filename);
where:
public void DeleteBlob(string filename)
{
var blob = container.GetBlobReference(filename);
blob.Delete();
}
But the problem is that the file might not have been created yet and so this will throw a Microsoft.WindowsAzure.Storage.StorageException telling me the file wasn't found (and then the file will end up getting created anyway!).
So what would be the cleanest way to handle this?

Extract an archive with progress bar?

How i can use an progress bar in this case?
void Client_DownloadFileCompleted(object sender, AsyncCompletedEventArgs e)
{
//System.Windows.MessageBox.Show("Update Complete!", "Message", MessageBoxButton.OK, MessageBoxImage.Information);
Uri uri = new Uri(url);
string filename = System.IO.Path.GetFileName(uri.AbsolutePath);
ZipFile.ExtractToDirectory(filePathDir + "/" + filename, filePathDir);
}
EDIT:
#Alessandro D'Andria , But in this case?:
WebClient wc = new WebClient();
Stream zipReadingStream = wc.OpenRead(url);
ZipArchive zip = new ZipArchive(zipReadingStream);
ZipFileExtensions.ExtractToDirectory(zip, filePathDir);

You can see the source of ExtractToDirectory on GitHub, the only thing you need to do is pass in a Progress<ZipProgress> and call it inside the foreach loop.
//This is a new class that represents a progress object.
public class ZipProgress
{
public ZipProgress(int total, int processed, string currentItem)
{
Total = total;
Processed = processed;
CurrentItem = currentItem;
}
public int Total { get; }
public int Processed { get; }
public string CurrentItem { get; }
}
public static class MyZipFileExtensions
{
public static void ExtractToDirectory(this ZipArchive source, string destinationDirectoryName, IProgress<ZipProgress> progress)
{
ExtractToDirectory(source, destinationDirectoryName, progress, overwrite: false);
}
public static void ExtractToDirectory(this ZipArchive source, string destinationDirectoryName, IProgress<ZipProgress> progress, bool overwrite)
{
if (source == null)
throw new ArgumentNullException(nameof(source));
if (destinationDirectoryName == null)
throw new ArgumentNullException(nameof(destinationDirectoryName));
// Rely on Directory.CreateDirectory for validation of destinationDirectoryName.
// Note that this will give us a good DirectoryInfo even if destinationDirectoryName exists:
DirectoryInfo di = Directory.CreateDirectory(destinationDirectoryName);
string destinationDirectoryFullPath = di.FullName;
int count = 0;
foreach (ZipArchiveEntry entry in source.Entries)
{
count++;
string fileDestinationPath = Path.GetFullPath(Path.Combine(destinationDirectoryFullPath, entry.FullName));
if (!fileDestinationPath.StartsWith(destinationDirectoryFullPath, StringComparison.OrdinalIgnoreCase))
throw new IOException("File is extracting to outside of the folder specified.");
var zipProgress = new ZipProgress(source.Entries.Count, count, entry.FullName);
progress.Report(zipProgress);
if (Path.GetFileName(fileDestinationPath).Length == 0)
{
// If it is a directory:
if (entry.Length != 0)
throw new IOException("Directory entry with data.");
Directory.CreateDirectory(fileDestinationPath);
}
else
{
// If it is a file:
// Create containing directory:
Directory.CreateDirectory(Path.GetDirectoryName(fileDestinationPath));
entry.ExtractToFile(fileDestinationPath, overwrite: overwrite);
}
}
}
}
This is used like
public class YourClass
{
public Progress<ZipProgress> _progress;
public YourClass()
{
// Create the progress object in the constructor, it will call it's ReportProgress using the sync context it was constructed on.
// If your program is a UI program that means you want to new it up on the UI thread.
_progress = new Progress<ZipProgress>();
_progress.ProgressChanged += Report
}
private void Report(object sender, ZipProgress zipProgress)
{
//Use zipProgress here to update the UI on the progress.
}
//I assume you have a `Task.Run(() => Download(url, filePathDir);` calling this so it is on a background thread.
public void Download(string url, string filePathDir)
{
WebClient wc = new WebClient();
Stream zipReadingStream = wc.OpenRead(url);
ZipArchive zip = new ZipArchive(zipReadingStream);
zip.ExtractToDirectory(filePathDir, _progress);
}
//...

Maybe something like this can work for you:
using (var archive = new ZipArchive(zipReadingStream))
{
var totalProgress = archive.Entries.Count;
foreach (var entry in archive.Entries)
{
entry.ExtractToFile(destinationFileName); // specify the output path of thi entry
// update progess there
}
}
It's simple a workaround to keep track of the progress.

Is there a construct or pattern similar to C# `using` which will return an object?

I have a WCF message inspector which inspects requests and responses: Message. The inspector works fine. A Message object can only be read once so once you read it, you cannot simply propagate as WCF will complain that the message has been read. Therefore, I am creating a brand new copy of the message and propagating that.
I have designed a class that allows message reading and after the caller has read whatever they want, they need to call Close which will return a copy of the message. Here is the skeleton of my class:
using System.ServiceModel.Channels;
internal abstract class MessageReader
{
internal string ReadSomething(string id)
{
// Return string
}
internal string ReadSomethingElse(string id)
{
// Return string
}
internal Message Close()
{
// Create copy and return it.
}
}
Users of my class may forget to call Close() which is fine because WCF will yell at them. Right now I have documentation to let users know they need to call Close().
Here is the question
Is there a pattern, or something similar, to C#'s using construct but one which returns an object at the end? This will be really convenient because then users of my class can just use a construct like that and at the end it will return the copy of the message. Something like this:
UsingSomeConstruct(var reader = new MessageReader(ref originalMessage))
{
var a = reader.ReadSomething("something");
var b = reader.ReadSomethingElse("something");
// Do something with what was read
}
// At this point originalMessage will be the copy of the message and no longer the original message.
EDIT
I thought about hacking IDisposable to achieve this but I am NOT going to do it that way so looking for other ideas.

There is no such language construct of course.
What I could suggest is to use IDisposable for cleaning up, and add ref Message message argument to each ReadXXX method. I know it will not be so convenient for your users, but from the other side they cannot forget passing the parameter.
So the implementation would be something like this:
internal class MessageReader : IDisposable
{
private MessageBuffer buffer;
private Message message;
private void Release()
{
if (buffer == null) return;
buffer.Close();
buffer = null;
message = null;
}
protected void OnReadRequest(ref Message message)
{
if (message == null) throw new ArgumentNullException("message");
if (this.message == message) return;
Release();
this.buffer = message.CreateBufferedCopy(int.MaxValue);
message = this.message = buffer.CreateMessage();
}
public void Dispose()
{
Release();
}
internal string ReadSomething(ref Message message, string id)
{
OnReadRequest(ref message);
// Return string
}
internal string ReadSomethingElse(ref Message message, string id)
{
OnReadRequest(ref message);
// Return string
}
}
and the sample usage:
using (var reader = new MessageReader())
{
var a = reader.ReadSomething(ref originalMessage, "something");
var b = reader.ReadSomethingElse(ref originalMessage, "something");
// Do something with what was read
}
// At this point originalMessage will be the copy of the message and no longer the original message.

The way I'd do this is as follows:
public MessageReader: IDisposable
{
public static MessageReader Create(ref Message message)
{
var buffer = message.CreateBufferedCopy(/*whatever is fit*/);
try
{
var reader = new MessageReader(buffer);
message = buffer.CreateMessage();
return reader;
}
catch
{
buffer.Close();
throw;
}
}
private readonly MessageBuffer buffer;
private bool disposed;
private MessageReader(MessageBuffer buffer) { this.buffer = buffer; }
public void Dispose()
{
if (disposed)
return;
buffer.Close();
disposed = true;
}
public string Read(string id)
{
var newCopy = buffer.CreateMessage();
//work with new copy...
}
}
And you'd simply use it like this:
using (var reader = MessageReader.Create(ref message))
//message here is already an untouched copy with no need of user active
//intervention and is never touched again by the reader.
{
var a = reader.Read("something"); //reads copy
...
}
IMHO, this is as clean as it can be. Note that MessageReader implements IDisposable exclusively because it holds a reference to the disposable private MessageBuffer.

Thanks to all the help from #InBetween, #quetzalcoatl, and #Ivan Stoev. Upvoted your answers because it helped me arrive at the following.
In the constructor, I create a copy of the message and set the original message to the copy. Since the status of this message is Created WCF will be happy propogating it. I create another copy and use that for reading multiple times.
#Ivan said but what if the user does not ask for anything to be read then the copying was wasted work. That is a good point but in my case, this is an interceptor and all messages are intercepted to be read.
Here is the code I ended up with suggestions from all of you:
public class MessageReader : IDisposable {
private readonly Message message;
public MessageReader(ref Message originalMessage) {
using( var buffer = originalMessage.CreateBufferedCopy( int.MaxValue ) ) {
// Keep original message for reading
this.message = buffer.CreateMessage();
// Set original message to a copy of the original
originalMessage = buffer.CreateMessage();
}
}
public int ReadSomething(string id) {
// Read from this.message;
}
public int ReadSomethingElse(string id) {
// Read from this.message;
}
public void Close() {
this.Dispose();
}
public void Dispose() {
this.message.Close();
}
}
The caller can either use it in a using block or without it. The using block is used for good reasons and not as a hack.
public object AfterReceiveRequest(ref Message request, IClientChannel channel,
InstanceContext instanceContext) {
try {
using( var rdr = new MessageReader(ref request) ) {
var value= rdr.ReadSomething( someIdentifier );
return value;
}
}
catch( System.Exception ex ) {
throw CreateFault( ex, request );
}
}

Nope, there is no such construct. It is simply too specific to exist there out of the box. There are extension methods which often are very helpful, but you won't be able to use them on this ref Message parameter..
However, if you are willing to use ref at all, then why dont simply include all that logic it in Reader's constructor?
Here's an example, somewhat contrived, but it should show what I mean. Like others mentioned in comments, I also suggest implementing IDisposable on the Reader object instead of Close, so I included that already.
TL;DR: In example below, the most important thing is in Reader(ref msg) constructor which clones the message, copies the data, and replaces the original message with a safe-message class which can be read many times..
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;
namespace Rextester
{
public class Program
{
public static void Main(string[] args)
{
// real-world variables, keep them typed as base Message
// to be able to silently replace them with different objects
Message original1;
Message original2;
// let's construct some one-time readable messages
{
var tmp1 = new OneTimeMessage();
tmp1.data["mom"] = "dad";
tmp1.data["cat"] = "dog";
original1 = tmp1;
var tmp2 = new OneTimeMessage();
tmp2.data["mom"] = "dad";
tmp2.data["cat"] = "dog";
original2 = tmp2;
}
// test1 - can't read twice
Console.WriteLine("test0A:" + original1.GetData("mom"));
//Console.WriteLine("test0B:" + original1.GetData("mom")); // fail
// test2 - can read twice with Reader's help
var backup1 = original2;
using(var rd1 = new Reader(ref original2))
{
Console.WriteLine("test1A:" + rd1.ReadSomething("mom"));
}
var backup2 = original2;
using(var rd2 = new Reader(ref original2))
{
Console.WriteLine("test1A:" + rd2.ReadSomething("mom"));
//^ ok - becase Reader replaced 'original2' with SafeMessage
}
// test3: Reader's ctor is intelligent
// so no more SafeMessages created during future usage
var backup3 = original2;
using(var rd3 = new Reader(ref original2))
{
}
var backup4 = original2;
using(var rd4 = new Reader(ref original2))
{
}
Console.WriteLine("checking for copies:" + (original2 == backup1));
Console.WriteLine("checking for copies:" + (original2 == backup2));
Console.WriteLine("checking for copies:" + (original2 == backup3));
Console.WriteLine("checking for copies:" + (original2 == backup4));
}
}
}
public abstract class Message
{
public abstract string GetData(string id);
}
public class OneTimeMessage : Message // this models your current one-time-readable message
{
public IDictionary<string, string> data = new Dictionary<string, string>();
public override string GetData(string id)
{
var tmp = data[id];
data.Remove(id);
// that's nonsense, but I want to show that you can't
// read the same thing twice from this object
return tmp;
}
}
public class SafeMessage : Message
{
public IDictionary<string, string> data;
public override String GetData(string id)
{
return data[id];
}
public SafeMessage(Message msg)
{
// read out the full msg's data and store it
// since this is example, we can do it in a pretty simple way
// in your code that will probably be more complex
this.data = new Dictionary<string,string>(((OneTimeMessage)msg).data);
}
}
public class Reader : IDisposable
{
private Message message;
public Reader(ref Message src)
{
src = src is SafeMessage ? src : new SafeMessage(src);
this.message = src;
}
public string ReadSomething(string id){ return message.GetData(id); }
public void Dispose(){ Close(); }
public void Close(){ message=null; Console.WriteLine("reader closed"); }
}
EDIT: improved example
using System;
using System.Collections.Generic;
using System.Linq;
using System.ServiceModel.Channels;
using System.Text.RegularExpressions;
using System.Xml;
namespace MyProgram
{
public class Program
{
public static void Main(string[] args)
{
// real-world variables, keep them typed as base Message
// to be able to silently replace them with different objects
Message original1;
Message original2;
// let's construct some one-time readable messages
{
original1 = new TheMessage("dad", "dog");
original2 = new TheMessage("dad", "dog");
}
// test1 - can't read twice
Console.WriteLine("test0A:" + original1.GetReaderAtBodyContents().ReadOuterXml());
// Console.WriteLine("test0B:" + original1.GetReaderAtBodyContents().ReadOuterXml()); // fail: InvalidOperationException - it was already read
// test2 - can read ONCE with Reader's help, but the message is replaced and is usable again
var backup1 = original2;
using (var rd1 = new ReaderOnce(ref original2))
{
Console.WriteLine("is message replaced after opening Reader:" + (original2 != backup1));
Console.WriteLine("test1A:" + rd1.ReadBodyXml());
// Console.WriteLine("test1B:" + rd1.ReadBodyXml()); // fail: InvalidOperationException - it was already read
}
// test3 - can read MANY TIMES with ReaderMany's help
// also note we use 'original2' again, which was already used above, so in fact ReaderOnce really works as well
var backup2 = original2;
using (var rd1 = new ReaderMany(ref original2))
{
Console.WriteLine("is message replaced after opening Reader:" + (original2 != backup2));
Console.WriteLine("test2A:" + rd1.ReadBodyXml());
Console.WriteLine("test2B:" + rd1.ReadBodyXml()); // ok
}
Console.WriteLine("Press enter to exit");
Console.ReadLine();
}
}
}
// solution1
public class ReaderOnce : IDisposable
{
private Message localCopy;
public ReaderOnce(ref Message src)
{
// create a WCF MessageBuffer to assist in copying messages
// btw. I suppose you should set some sane limit instead of that below
using (var tempBuffer = src.CreateBufferedCopy(int.MaxValue))
{
src = tempBuffer.CreateMessage(); // FIRST copy for outer use
localCopy = tempBuffer.CreateMessage(); // SECOND copy for internal use in the Reader
}
}
public void Dispose() { Close(); }
public void Close()
{
localCopy.Close(); // but that does NOT affect FIRST copy sent to outer scope outside reader
Console.WriteLine("reader closed");
}
public string ReadBodyXml() // careful: that's again ONE TIME readable
{
return localCopy.GetReaderAtBodyContents().ReadOuterXml();
}
}
// solution2
public class ReaderMany : IDisposable
{
private MessageBuffer localBuffer;
public ReaderMany(ref Message src)
{
localBuffer = src.CreateBufferedCopy(int.MaxValue);
src = localBuffer.CreateMessage(); // FIRST copy for outer use
}
public void Dispose() { Close(); }
public void Close()
{
localBuffer.Close();
Console.WriteLine("reader closed");
}
public string ReadBodyXml() // this is readable multiple times
{
using (var tmp = localBuffer.CreateMessage())
return tmp.GetReaderAtBodyContents().ReadOuterXml();
}
}
// let's fake some Message type to have something to test the Reader on
public class TheMessage : Message
{
public override MessageHeaders Headers => _mh;
public override MessageProperties Properties => _mp;
public override MessageVersion Version => _mv;
private MessageHeaders _mh;
private MessageProperties _mp;
private MessageVersion _mv;
private string data1;
private string data2;
// btw. below: surprise! XmlDictionaryWriter is in "System.Runtime.Serialization", not in "System.Xml"
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
writer.WriteStartElement("foo");
writer.WriteAttributeString("data1", data1);
writer.WriteAttributeString("data2", data2);
writer.WriteEndElement();
}
public TheMessage(string data1, string data2)
{
// remember, this class is just an example, you will work on your own messages you already have
_mv = MessageVersion.Soap12;
_mh = new MessageHeaders(_mv);
_mp = new MessageProperties();
// below: yeah, that's super-naive and wrong, but that's an example
this.data1 = data1;
this.data2 = data2;
}
}

There is no language construct in c# that does what you are asking. As stated in comments, you could abuse IDisposable and the language and use a using block to achieve what you want.
But, I fail see what you are gaining, you are just punting the problem; now users will need to remember to use usinginstead of Close. The latter is simple and clean, the former uses a very known language construct to do something different to what it was thought for, something that will potentially be very confusing.

How to get the text sent to console output in an event or method

I need to capture the event when my application is trying to write something on console.
Console.WriteLine("Any text");
Is it possible to get the text sent to console output in an event or method?

One approach would be to create a new stream override, as shown in this article:
http://mel-green.com/2010/01/progressstream/
Then, you would need to set this as the stream that Console writes to, e.g.
MemoryStream ms = new MemoryStream();
ProgressStream progressStream = new ProgressStream(ms);
Console.SetOut(new StreamWriter(progressStream));
Then use the events of the progress stream to see when it is written to.

This may be helpful to you:
using System;
using System.IO;
namespace nsStreams
{
public class Redirect
{
static public void Main ()
{
FileStream ostrm;
StreamWriter writer;
TextWriter oldOut = Console.Out;
try
{
ostrm = new FileStream ("./Target.txt", FileMode.OpenOrCreate, FileAccess.Write);
writer = new StreamWriter (ostrm);
}
catch (Exception e)
{
Console.WriteLine (e.Message);
return;
}
Console.SetOut (writer);
Console.SetOut (oldOut);
writer.Close();
ostrm.Close();
Console.WriteLine ("Done");
}
}
}

Here is an example I used to make a thread-safe listener for Console content that we're "printing" into an observable collection that a WPF Element is bound to.
Create a custom TextWriter class so you can hook into the process:
public class CustomTextWriter : TextWriter
{
private readonly StringBuilder lineBuffer = new();
public event EventHandler<StringEventArgs>? LogModelReadyToAdd;
public override Encoding Encoding => Encoding.UTF8;
public override void Write(char value)
{
// This is the inner-most write, as it is down to a single character
if (value is '\r' or '\n')
{
if (lineBuffer.Length > 0)
{
LogModelReadyToAdd?.Invoke(
this,
new(lineBuffer.ToString()));
lineBuffer.Clear();
}
return;
}
lineBuffer.Append(value);
}
}
With a quick string event args object:
public class StringEventArgs : EventArgs
{
public StringEventArgs()
{
}
public StringEventArgs(string #string)
{
String = #string;
}
public string? String { get; init; }
}
And in my main window (where the context is for me to bind, I have this in my OnInitialized):
CustomTextWriter writer = new();
writer.LogModelReadyToAdd += async (_, args) =>
{
if (Application.Current.CheckAccess())
{
TheContext.StepLog?.Add(new() { Log = args.String, });
return;
}
await Application.Current.Dispatcher.BeginInvoke(
DispatcherPriority.Background,
new Action(() => TheContext.StepLog?.Add(new() { Log = args.String, })));
};
Console.SetOut(writer);

How can I manage which items are in the ThreadPool?

I have a windows service that runs a method when the services main Timer elapses (OnElapse).
The OnElapse method gets a list of .xml files to process.
Each xml file is inserted into a ThreadPool.
I want to make sure I don't insert 2 XML's with the same name into the ThreadPool.
How can I manage which items are in the ThreadPool? I basically want to do this:
if xmlfilename not in threadpool
insert in threadpool

This is pretty trick because you need to closely monitor the ThreadPool and it will require a form of synchronization. Here's a quick and dirty example of a way to do this.
class XmlManager {
private object m_lock = new object();
private HashSet<string> m_inPool = new HashSet<string>();
private void Run(object state) {
string name = (string)state;
try {
FunctionThatActuallyProcessesFiles(name);
} finally {
lock ( m_lock ) { m_inPool.Remove(name); }
}
}
public void MaybeRun(string xmlName) {
lock ( m_lock ) {
if (!m_pool.Add(xmlName)) {
return;
}
}
ThreadPool.QueueUserWorkItem(Run, xmlName);
}
}
This is not a foolproof solution. There is at least one race condition in the code. Namely that an item could be being removed from the pool while you're trying to add it back in and it won't actually get added. But if your are only concerned with them being processed a single time, this doesn't matter.

Something like this should do it (use a HashSet instead of a Dictionary if you are using .Net 3.5 or higher):
using System;
using System.Collections.Generic;
using System.Threading;
namespace Something
{
class ProcessFilesClass
{
private object m_Lock = new object();
private Dictionary<string, object> m_WorkingItems =
new Dictionary<string, object>();
private Timer m_Timer;
public ProcessFilesClass()
{
m_Timer = new Timer(OnElapsed, null, 0, 10000);
}
public void OnElapsed(object context)
{
List<string> xmlList = new List<string>();
//Process xml files into xmlList
foreach (string xmlFile in xmlList)
{
lock (m_Lock)
{
if (!m_WorkingItems.ContainsKey(xmlFile))
{
m_WorkingItems.Add(xmlFile, null);
ThreadPool.QueueUserWorkItem(DoWork, xmlFile);
}
}
}
}
public void DoWork(object xmlFile)
{
//process xmlFile
lock (m_Lock)
{
m_WorkingItems.Remove(xmlFile.ToString());
}
}
}
}

OnElaspe can't you rename the xml file ? so it has a unqiue name on the threadpool.

Couldn't you make a dictionary> and check that before you do the insertion? Something like this...
Dictionary<ThreadPool, List<String>> poolXmlFiles = new Dictionary<ThreadPool, List<String>>();
if(poolXmlFiles.ContainsKey(ThreadPool) && !poolXmlFiles[ThreadPool].Contains(xmlFileName))
{
poolXmlFiles[ThreadPool].Add(xmlFileName);
//Add the xmlFile to the ThreadPool
}
Sorry if there are syntax errors, I'm coding in VB these days.