We Keep Coding

How to save entire MailKit mime message as a byte array

I'm building a simple .net MailKit IMAP client. Rather then pulling emails again and again from the IMAP server, is it possible to store the entire MailKit mime message (in full, including attachments) as a byte array? If so, how?
Then I could write it to MySql or a file and reuse it for testing code changes.

As Lucas points out, you can use the MimeMessage.WriteTo() method to write the message to either a file name or to a stream (such as a MemoryStream).
If you want the message as a byte array in order to save it to an SQL database, you could do this:
using (var memory = new MemoryStream ()) {
message.WriteTo (memory);
var blob = memory.ToArray ();
// now save the blob to the database
}
To read it back from the database, you'd first read the blob as a byte[] and then do this:
using (var memory = new MemoryStream (blob, false)) {
message = MimeMessage.Load (memory);
}

Pass the Length of uncertain Stream to WCF Service

Is there any way to pass the Length of uncertain Stream to WCF Service?
Unsertain Stream means the stream of
The stream provides its length only after process and writing the data.
e.g. GZipStream
Background
I'm making a WCF Service receiving multiple Streams from client.
As WCF Streaming only allows one stream in the message, I decided to concatenate all streams into one stream and divide it in server code.
The streams client provides will contains variable kinds of stream like FileStream, MemoryStreamwith data from DataTable serialization and
using (var fileStream = new FileStream(filePath, FileMode.Open))
using (var memoryStream = new MemoryStream())
using (var concatStream = new ConcatenatedStream(fileStream, memoryStream))
{
client.UploadStreams(concatStream);
}
ConcatenatedStream is a Stream implementation suggested in c# - How do I concatenate two System.Io.Stream instances into one? - Stack Overflow.
In server side, Length of each Streams will be needed to divide single stream to multiple streams.
As I want to save memory in client side, I decided to use PullStream.
PullStream will Write buffer on demand of Read.
But this causes a big problem. I cannot get Length of PullStream before starting streaming.
Any helps will be appreciated.
Thanks

Let's make it simple:
If you have the length of a part of the stream on client before you start pushing it to server you can append a structure before the payload and read that structure on server. That is a standard data transfer template. Doing so i.e. appending a header before each payload you give your server a hint on how long the next part is going to be.
If you do not have the length of a part of the stream on client before you start pushing it to server, you are going to have to 'insert' the header inside the payload. That's not very intuitive and not that useful but it does work. I used such a thing when I had my data prepared asynchronously on client and the first buffers were ready before the length was known. In this scenario you are going to need a so called marker i.e. a set of bytes that could not be found anywhere in the stream but before the header.
This scenario is the toughest of the 3 to implement when done for the first time. Buckle up. In order to do it right you should create an artificial structure of your stream. Such a structure is used for streaming video over network and called Network Abstraction Layer or NAL, read about it. It is also called stream format AnnexB from the h264 standard. You should abstract from the field in which the standard is described, the idea is very versatile.
In short the payload is divided into parts, so called NAL Units or NALUs, each part has a byte sequence which marks it's start, then goes the type indicator and length of the current NALU, then follows the payload of the NALU. For your purposes you would need to implement NALUs of two types:
Main data payload
Metadata
After you imagine how your stream should look like, you have to grip on the idea of "stream encoding". Those are fearsome words but do not worry. You just have to ensure that the byte sequence that is used to mark the start of the NALU is never met inside the payload of the NALU. In order to achieve that you are going to implement some replacement tactic. Browse for samples.
When you are done thinking this through and before you dive into that, think twice about it. Might be the scenario 3 would fit you easier.
In the case you are sure you will never have to process a part of the streamed data you can greatly simplify the scenario i.e. totally skip the stream encoding and implement something like this:
Client Stream principal code:
private byte[] mabytPayload;
private int mintCurrentPayloadPosition;
private int? mintTotalPayloadLength;
private bool mblnTotalPayloadLengthSent;
public int Read(byte[] iBuffer, int iStart, int iLength)
{
if (mintTotalPayloadLength.HasValue && !mblnTotalPayloadLengthSent)
{
//1. Write the packet type (0)
//3. Write the total stream length (4 bytes).
...
mblnTotalPayloadLengthSent = true;
}
else
{
//1. Write the packet type (1)
//2. Write the packet length (iLength - 1 for example, 1 byte is for
//the type specification)
//3. Write the payload packet.
...
}
}
public void TotalStreamLengthSet(int iTotalStreamLength)
{
mintTotalPayloadLength = iTotalStreamLength;
}
Server stream reader:
Public void WCFUploadCallback(Stream iUploadStream)
{
while(!endOfStream)
{
//1. Read the packet type.
if (normalPayload)
{
//2.a Read the payload packet length.
//2.b Read the payload.
}
else
{
//2.c Read the total stream length.
}
}
}
In the scenario where your upload is no-stop and the metadata about the stream is ready on client long after the payload, that happens as well, you are going to need two channels i.e. one channel for payload stream and another channel with metadata where you server will answer to the client with another question like 'what did you just started sending me' or 'what have you sent me' and the client will explain itself in the next message.
If you are ready to stick to one of the scenarios, one could give you some further details and/or recommendations.

Sending some constraints to client from server in C#

I have created a simple server using socket programming in C# which will receive a file from the client side. My sample code segment is given below.
I want to add some restrictions. I want to make a limit on the file size (such as 4 KB or 2 KB) and allowable file formats (such as .doc, .txt, .cpp, etc.) which will be sent to the client as soon as the client connects to the server so that the client can send files accordingly. How will I do that?
Sample code segment:
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Sockets;
using System.IO;
using System.Text;
namespace FileTransfer
{
class Program
{
static void Main(string[] args)
{
// Listen on port 1234
TcpListener tcpListener = new TcpListener(IPAddress.Any, 1234);
tcpListener.Start();
Console.WriteLine("Server started");
//Infinite loop to connect to new clients
while (true)
{
// Accept a TcpClient
TcpClient tcpClient = tcpListener.AcceptTcpClient();
Console.WriteLine("Connected to client");
byte[] data = new byte[1024];
NetworkStream ns = tcpClient.GetStream();
int recv = ns.Read(data, 0, data.Length);
StreamReader reader = new StreamReader(tcpClient.GetStream());
//Will add some lines to add restrictions...
}
}
}
}
Which additional lines will I have to add to the code to send the restrictions to client?

Basically I think mainly you need two things:
define application protocol as suggested in other answer
and handle partial read/writes
For handling partial reads (not sure how much such function is needed for write) you may use function like below:
public static void ReadWholeArray (Stream stream, byte[] data)
{
int offset=0;
int remaining = data.Length;
while (remaining > 0)
{
int read = stream.Read(data, offset, remaining);
if (read <= 0)
throw new EndOfStreamException
(String.Format("End of stream reached with {0} bytes left to read", remaining));
remaining -= read;
offset += read;
}
}
Thing is traditional Stream.Read() doesn't guarantee to read as many bytes as you told it, this method on the other hand, will ensure to have read as many bytes as specified in data.Length parameter. So you can use such function to implement the desired application protocol instead.
Some relevant information about such application protocols you will find here too
Ok this is for example how the server could send file length limit and the file extension:
// Send string
string ext = ".txt";
byte [] textBytes = Encoding.ASCII.GetBytes(ext);
ns.Write(textBytes, 0, textBytes.Length);
// Now, send integer - the file length limit parameter
int limit = 333;
byte[] intBytes = BitConverter.GetBytes(limit);
ns.Write(intBytes, 0, intBytes.Length); // send integer - mind the endianness
But you will still need some kind of protocol otherwise you should let client read the "full" stream and parse these data later somehow, which isn't trivial if the data doesn't have fixed length etc - otherwise how will the client distinguish which part of the message is text, which integer?

You seem to be making the classical socket mistake. The given code and explanation seem to assume sockets handle in messages. They don't. When used this way, you're using streaming internet sockets, which provide a stream, not messages.
You don't show any code that does the actual sending, so I'm guessing that you just pump a file's data to the other side and close the connection. How else will you know you've successfully transferred an entire file?
This set of rules that client and server have to follow in order to usefully exchange data through sockets is called an application protocol. You will have to have one, otherwise you'll just be sending data to $deity knows where, and you'll have no control over it at all. This means server nor client will know what's going on, they'll just be sending and receiving data and hoping all goes well. So there's not "a few lines" you have to add to your code, you'll have to restructure it entirely.
There are many ways to define an application protocol and many options to choose from, so I'm going to show you an arbitrary one: a textual explanation of messages that are prefixed with an ID and a payload length (if applicable), both in unspecified numeric variables. You could choose little-endian four-byte unsigned integers, for example.
Messages in this format are known as "Type/Length/Value" or TLV.
So we define these messages:
ID Name Direction Description Payload
1 ServerHello Server -> Client The server sends this message None.
to every connecting client. Or maybe server or
protocol version.
2 MaxUpload Server -> Client Sent after the ServerHello. Maximum upload size
in bytes.
3 AllowedExts Server -> Client Allowed upload extensions, The allowed extensions.
comma-separated. Sent after
MaxUpload message.
10 IncomingFile Client -> Server There's a file coming. The file name.
11 FileUpload Client -> Server The file to upload. The file data.
Sent after IncomingFile.
Now all that's required is to implement this application protocol in server and client and you're done.
You also have to decide what to do if a client or server doesn't adhere to the prototol. It can for example send a message that you can't parse, an unknown message ID, a message length that you don't want to support, an out-of-order message (FileUpload before IncomingFile) or a message that isn't conform the messages sent earlier, like a client uploading a larger file than the server said it would accept or an invalid extension. You also have to think about "acknowledgement" or response messages, like the server telling the client "OK, go ahead, send the next message".
All in all, this is a very broad question and not answered easily. I tried to address that in my comment to your question, which got removed. So here you have your answer.
You can learn more about this on the web, for example Beej's Guide to Network Programming as linked to by Giorgi (be sure to read the entire guide) and Stephen Cleary's blog.

What is the exact method to send an Image over network [An attempt using Sockets]

This is my Server side code:
public void ReceivingData(object sender, EventArgs e)
{
while (mysocket.Connected)
{
buffer = new byte[accepted.SendBufferSize];
int bytesRead = accepted.Receive(buffer);
MemoryStream Data = new MemoryStream(buffer);
if ( picbox.InvokeRequired)
{
picbox.Invoke(new MethodInvoker(delegate { picbox.Image = Image.FromStream(Data); }));
}
}
}
The connection gets established and the file is being received without any issue. However the image gets distorted on Transfer. I do not understand why this is happening. Here is the screenshot:
I remember i had to format the strings which i used to send over sockets using Encoding.ASCII.GetString(StringToFormat). What do i need to do in case of Images?

In your ReceivingData callback you may not receive all the data back in one pop. Some data can be partially received and the rest of it in a subsequent (or multiple) callbacks and it will be your task to re-assemble the original message.
You will need to define a protocol to ensure that you have read all necessary data.
You could for example use base64 to encode the image on the server and decode it on the client. You would need to know how many bytes you should anticipate. This can be done, either by prefixing your response with the total bytes that the client should anticipate or by having a special marker (such as byte value 0x00) to distinguish message boundaries.
Using Base64 will also have the effect of increasing file sizes by 33% since base64 basically encodes every 6bits of the incoming stream to an 8bit readable character. So for every 3 'real' bytes you would like to transfer you will need 4 encoded bytes.

C# Video Streaming

I'm trying to do a application with video stream, and by now I can send only one image from the server to the client. When I try to send more than only one image at the client I receive the following error: "Parameter is not valid." at pictureBox1.Image = new Bitmap(ms);
Client side code:
while((data = cliente.receiveImage()) != null)
{
ms = new MemoryStream(data);
pictureBox1.Image = new Bitmap(ms);
ms.Close();
}
Server side code (this code is repeated continuously):
servidor.sendImage(ms.GetBuffer());

ms.GetBuffer() returns the entire buffer of the memory stream, including any extra unused portion.
You should call ToArray(), which only returns actual contents.
(Or, your data might be invalid for some other reason, such as an issue in sendImage or receiveImage)

Images are nit-picky things, and you have to have the entire set of bytes that comprise the image in order to reconstruct an image.
I would bet my left shoe that the issue is that when the client object is receiving data, it's getting it in chunks comprised of partial images, not the whole image at once. This would cause the line that says
pictureBox1.Image = new Bitmap(ms);
to fail because it simply doesn't have a whole image's bytes.
Alternatives
Rather than having the server push images out to the client, perhaps another approach would be to have the client pull images from the server.
Use an existing streaming mechanism. I personally think that streaming video manually from C# may be more complex than you're bargaining for, and I'd humbly recommend using an existing component or application to stream the video rather than writing your own. There are already so many different options out there (wmv, Flash, and a hundred more) that you're reinventing a wheel that really doesn't need to be re-invented.