We Keep Coding

c# Send/Recieve Image from Stream

I have currently wrote the following codes. Client and Server side. I want to send on demand an image from server to client. The image will be a screenshot from the server, so I will always be different size.
Sending the first image is a task well accomplished. But when I am sending the next the picturebox does not refresh.
From debugging I can see that the bytes from server to client get through successfully. But the code seems to just receive the bytes and not continue with the rest of the code
The "commented" line for the "while" loop ( client ) seem to work in debugging, but I cannot see the result 'cause the program gets stuck in the while loop and I can't get the application's window on the foreground
I am currently testing in windows but the client is going to get adjusted to work on android
SERVER picture
Client picture
SERVER
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
Socket client;
private void button2_Click(object sender, EventArgs e)
{
Bitmap bmp = TakingScreenshotEx1();
bmp.Save("1.jpeg", ImageFormat.Jpeg);
byte[] buffer = ReadImageFile("1.jpeg");
int v = client.Send(buffer, buffer.Length, SocketFlags.None);
Console.WriteLine("Image SENT!");
}
private Bitmap TakingScreenshotEx1()
{
//Create a new bitmap.
var bmpScreenshot = new Bitmap(Screen.PrimaryScreen.Bounds.Width,
Screen.PrimaryScreen.Bounds.Height,
PixelFormat.Format32bppArgb);
// Create a graphics object from the bitmap.
var g = Graphics.FromImage(bmpScreenshot);
// Take the screenshot from the upper left corner to the right bottom corner.
g.CopyFromScreen(Screen.PrimaryScreen.Bounds.X,
Screen.PrimaryScreen.Bounds.Y,
0,
0,
Screen.PrimaryScreen.Bounds.Size,
CopyPixelOperation.SourceCopy);
return bmpScreenshot;
}
private void button1_Click(object sender, EventArgs e)
{
IPEndPoint iep = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9999);
Socket server = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
server.Bind(iep);
server.Listen(100);
Console.WriteLine("Waiting for client....");
client = server.Accept();
}
private static byte[] ReadImageFile(String img)
{
FileInfo fileinfo = new FileInfo(img);
byte[] buf = new byte[fileinfo.Length];
FileStream fs = new FileStream(img, FileMode.Open, FileAccess.Read);
fs.Read(buf, 0, buf.Length);
fs.Close();
GC.ReRegisterForFinalize(fileinfo);
GC.ReRegisterForFinalize(fs);
return buf;
}
}
CLIENT
public Form1()
{
InitializeComponent();
pictureBox1.SizeMode = PictureBoxSizeMode.StretchImage;
}
IPEndPoint iep = new IPEndPoint(IPAddress.Parse("127.0.0.1"), 9999);
Socket client = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
byte[] buffer = new byte[1000000];
private void button1_Click(object sender, EventArgs e)
{
if(client.Connected != true)
client.Connect(iep);
// while (true)
// {
int v = client.Receive(buffer, buffer.Length, SocketFlags.None);
Console.WriteLine("Data Received!");
Stream stream = new MemoryStream(buffer);
var img = Bitmap.FromStream(stream);
pictureBox1.Image = img;
// }
}

As Peter Duniho pointed out the while loop is blocking the UI thread
I found a fitting solution here
How to wait for thread to complete without blocking UI
Also you can check Invoking the Main Thread inside Timer Method

How to get better performance in Desktop Streaming over TCP?

I am working on a software solution to stream a Windows Desktop to a Xamarin application, using TCP.
It works, but I cannot manage to get better framerate.
Here's how I do :
The client, a Xamarin app, listens on a Tcp socket. The server (C# console app) connects to it, captures the Desktop and sends the image. Then, the app show it as an Image.
When sending a bitmap, the server starts sending its size, on a 4 bytes array. The client first listens 4 bytes, then the requested size.
DesktopRecorder.cs (Server side)
private void CaptureScreen()
{
bitmap = new Bitmap(SIZE.Width, SIZE.Height, PixelFormat.Format32bppRgb);
graphics = Graphics.FromImage(bitmap);
graphics.CopyFromScreen(0, 0, 0, 0, SIZE, CopyPixelOperation.SourceCopy);
CaptureReady(BitmapToArray(bitmap));
}
private byte[] BitmapToArray(Bitmap bitmap)
{
using (var ms = new MemoryStream())
{
bitmap.Save(ms, ImageFormat.Png);
return ms.ToArray();
}
}
TcpNetworker.cs (Server side, connecting in TCP):
public override void SendData(byte[] data)
{
if (Tcp.Connected)
{
int size = data.Length;
var sizeArray = BitConverter.GetBytes(size);
if (BitConverter.IsLittleEndian)
sizeArray.Reverse();
Console.WriteLine($"Sending {size} o.");
Write(sizeArray);
Stream.Flush();
Write(data);
}
void Write(byte[] b)
{
Stream.Write(b, 0, b.Length);
}
}
ViewModel.cs (Client side, Xamarin):
private void Accept()
{
try
{
var client = Tcp.AcceptTcpClient();
Console.WriteLine("---------------- Client Connected ----------------");
Stream = client.GetStream();
byte[] sizeArray;
byte[] dataArray;
while (client.Connected)
{
sizeArray = new byte[4];
Stream.Read(sizeArray, 0, 4);
if (BitConverter.IsLittleEndian)
sizeArray.Reverse();
int size = BitConverter.ToInt32(sizeArray, 0);
Console.WriteLine($"Size: {size}");
dataArray = new byte[size];
Stream.Read(dataArray, 0, size);
Dispatcher.Invoke(() => ImageReceived?.Invoke(this, new MemoryStream(dataArray)));
}
}
catch(Exception e)
{
Console.WriteLine($"Error: {e.Message} {Environment.NewLine} {e.StackTrace}");
}
Accept();
}
MainWindow.cs (Page's code behind):
public MainWindow()
{
InitializeComponent();
var model = new MainWindowViewModel();
model.ImageReceived += Model_ImageReceived;
}
private void Model_ImageReceived(object sender, System.IO.MemoryStream memoryStream)
{
memoryStream.Position = 0;
//var image = System.Drawing.Image.FromStream(memoryStream);
var imageSource = new BitmapImage();
imageSource.BeginInit();
imageSource.CacheOption = BitmapCacheOption.OnLoad;
imageSource.StreamSource = memoryStream;
imageSource.EndInit();
MainImage.Source = imageSource;
memoryStream.Dispose();
}
The CaptureScreen() method of DesktopRecorder is called on a given framerate. It works fine at 10 FPS, but when increasing this framerate, I get trouble in the client, the app cannot manage to find the correct size. After receiving some images, the 4 bytes read doesn't contain the size. It is like an offset has been added, and the server/client unsychronize.
So, I am asking 2 things :
How can I keep a synchronization between the server and client after sending many TCP packets, since I need to keep 4 bytes for the size ? I don't feel using a fixed size of bitmap or something like that.
Currently, I am encoding the bitmap into a PNG image, which can be handled natively by Xamarin. Is there a better way to compress my image and sends it faster, even there is a loss in quality ? I already did that kind of things between a C server and an Angular app, and I had to encode it in PNG and Base64. Can I do the same ?
P.S : I know I can manage to get better performance using UDP, but for some reasons, I have to keep with TCP :/.
Thanks for your help !

out of memory exception in image.fromFile(Stream)

i searched in all similar problem but still couldn't solve the problem
this is a server code it is work successfully and the image File created successfully BUT if i cannot access the image file
image i = Image.FromStream(StreamObject);
NOTES:
1- the image not too large
2- the image have valid image format
I know the problem related to the stream ... how can i control this problem
i want to retrive the saved image in the image object for some reason..
How can i keep the stream open for the lifetime of the image.??
static void Main(string[] args)
{
IPAddress ipAdress = IPAddress.Parse("192.160.1.8");
// Initializes the Listener
TcpListener tcpListener = new TcpListener(ipAdress, 8001);
tcpListener.Start();
int no;
for (;;)
{
Socket socket = tcpListener.AcceptSocket();
if (socket.Connected)
{
Stream os = File.OpenWrite("Target.jpg",);
byte[] buffer = new byte[8000000];
NetworkStream networkStream = new NetworkStream(socket);
no = networkStream.Read(buffer, 0, 8000000);
os.Write(buffer, 0, no);
///here the problem in the following line
///
Image i = Image.FromFile("Target.jpg");
///
networkStream.Close();
socket.Close();
break;
}
}
}

While bytes have been written to the buffer, the bytes may not have been flushed to the disk. Additionally, the code example provided keeps the file open for writing while it is getting read into an image. For things like streams, you should wrap the usage in using statements in order to avoid these types of memory errors.
static void Main(string[] args)
{
IPAddress ipAdress = IPAddress.Parse("192.160.1.8");
// Initializes the Listener
TcpListener tcpListener = new TcpListener(ipAdress, 8001);
tcpListener.Start();
int no;
for (;;)
{
Socket socket = tcpListener.AcceptSocket();
if (socket.Connected)
{
byte[] buffer = new byte[8000000];
using (Stream os = File.OpenWrite("Target.jpg"))
{
using (NetworkStream networkStream = new NetworkStream(socket))
{
no = networkStream.Read(buffer, 0, 8000000);
os.Write(buffer, 0, no);
}
}
///here the problem in the following line
///
Image i = Image.FromFile("Target.jpg");
///
socket.Close();
break;
}
}
}
Alternatively, and probably more appropriately, you should consider creating your GDI+ image directly from the stream using Image.FromStream. The Image.FromStream method is documented here: https://msdn.microsoft.com/en-us/library/system.drawing.image.fromstream%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396

Sending jpg using tcp socket

I have a few problem to send jpg data using a socket.
Sender side:
Socket client_s = new Socket(AddressFamily.InterNetwork, SocketType.Stream,ProtocolType.Tcp);
IPEndPoint serverEP = new IPEndPoint(IPAddress.Loopback, 4567);
client_s.Connect(serverEP);
NetworkStream stream = new NetworkStream(client_s);
bmp.Save(stream, System.Drawing.Imaging.ImageFormat.Jpeg);
Receiver side:
clientSocket = listenSocket.Accept();
NetworkStream s = new NetworkStream(clientSocket);
Bitmap slice_bmp = new Bitmap(s);
Graphics g = this.CreateGraphics();
g.DrawImage(slice_bmp,0,0);
When I execute the above, Bitmap slice_bmp = new Bitmap(s); is blocked until closing sender's socket. After sender's socket is closed, an image is drawn normally. I want to know why that operation is blocked?
Also, is it right using instance of Networkstream as argument of constructor of Bitmap? (My code is based on inheritance)

First of all, I would suggest you to use Flush() method of the NetworkStream to be sure that all the data has been sent.
Socket client_s = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
IPEndPoint serverEP = new IPEndPoint(IPAddress.Loopback, 4567);
client_s.Connect(serverEP);
NetworkStream stream = new NetworkStream(client_s);
bmp.Save(stream, System.Drawing.Imaging.ImageFormat.Jpeg);
stream.Flush(); //flush everything
On the receiver's side, you should first read all the bytes of the image into a byte array. From that byte array you could construct the MemoryStream and than Bitmap from that stream.
clientSocket = listenSocket.Accept();
NetworkStream s = new NetworkStream(clientSocket);
int bytesRead = 0;
int howMany = 0;
byte [] byteBuffer = new byte[100000];
do
{
howMany = s.Read(byteBuffer, bytesRead, 10000);
bytesRead += howMany;
if(bytesRead>=byteBuffer.Length)
byteBuffer = ResizeByteArray(byteBuffer);
}
while (howMany > 0 && s.DataAvailable);
MemoryStream ms = new MemoryStream(byteBuffer);
Bitmap slice_bmp = new Bitmap(ms);
Graphics g = this.CreateGraphics();
g.DrawImage(slice_bmp, 0, 0);
g.Dispose();
If the image is larger from the capacity of the instantiated byte array, then you need to resize it. You could use this helper method:
private byte[] ResizeByteArray(byte[] arr)
{
byte[] newArr = new byte[arr.Length * 2];
Array.Copy(arr, newArr, arr.Length);
return newArr;
}
And as a side note, always call Dispose on Graphics object that you create manually (like in this case). I checked the code and it works.

It appears you are working on a single threaded-application, or at least on the UI-thread of a multi-threaded-application. The blocking appears because your bitmap object is waiting for its construction to be complete. And as it takes a while to read from a stream over a network, your program will not continue until your bitmap-object is fully created.
If you wanted your program to go on, you would need to create a thread and have it read from the stream and create the bitmap. When the bitmap is ready you could fire an event and act appropriately.
As NetworkStream is a subclass of System.IO.Stream it is perfectly fine! You can use any System.IO.Stream to create an object that requires a stream. The only result is, as you experienced, it might take a little longer than creating the image, e.g. from a System.IO.FileStream.
Have a look at MSDN most of the stuff is well described.

This is because bitmap class waits for transmission to end:
First receive byte data buffer separately,
then create memory stream from it and then create bitmap.

Deserializing data sent via TCP

I have problem with sending objects via TCPClient. At first, I serialize them into byte array and then I send them. TCPListener recieves some data, but deserializer is throwing exception "Unexpected end of stream".
Here is reciever code:
public void start()
{
TcpListener listener = new TcpListener(IPAddress.Any, 8090);
listener.Start();
while (true)
{
TcpClient client = listener.AcceptTcpClient();
processClient(client);
}
}
public void processClient(TcpClient client)
{
NetworkStream net = client.GetStream();
ReadData(net);
byte[] response = Encoding.UTF8.GetBytes("Hello from the server.");
net.Write(response, 0, response.Length);
net.Close();
client.Close();
}
void ReadData(NetworkStream netstream)
{
byte[] buffer = new byte[2048];
System.IO.MemoryStream memStream = new System.IO.MemoryStream();
netstream.ReadTimeout = 5000;
int bytes = -1;
while ((bytes = netstream.ReadByte()) != -1)
{
memStream.WriteByte((byte)bytes);
}
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bform = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
Packet packet = (Packet)bform.Deserialize(memStream);
OnMessageArrived(this, new MessageEventArgs(packet.From.ToString(), packet.Data.ToString()));
memStream.Close();
netstream.Close();
}
And here is sender code:
public void sendData(string to, Packet data)
{
TcpClient client = new TcpClient();
MemoryStream mstream = new MemoryStream();
client.Connect(new IPEndPoint(IPAddress.Parse("127.0.0.1"), 8090));
if (client.Connected)
{
NetworkStream stream = client.GetStream();
System.Runtime.Serialization.Formatters.Binary.BinaryFormatter bform = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
bform.Serialize(mstream, data);
byte[] buffer = new byte[2048];
mstream.Read(buffer, 0, buffer.Length);
stream.Write(buffer, 0, buffer.Length);
stream.Flush();
stream.Close();
client.Close();
}
}
Sender main method:
static void Main(string[] args)
{
SimplestTCPIP.Client client = new SimplestTCPIP.Client();
Packet packet = new Packet("client", "server", IPAddress.Parse("127.0.0.1"));
client.sendData("server", packet);
Console.WriteLine("IP: " + GetIP().ToString());
Console.Read();
}
Reciever main method:
static void Main(string[] args)
{
SimplestTCPIP.Server server = new SimplestTCPIP.Server();
server.OnMessageArrived += new SimplestTCPIP.Server.MessageArrived(server_OnMessageArrived);
Thread thread = new Thread(server.start);
thread.Start();
}
static void server_OnMessageArrived(object sender, SimplestTCPIP.Server.MessageEventArgs m)
{
Console.WriteLine(m.From + " : " + m.Text);
}

In your sendData method you serialize the object to a memory stream and then read it back into a buffer of 2048 bytes before writing it to the network stream. If the serialzed object is > 2048 bytes you would have a problem. I would try just serializing directly to the network stream or at least using the same kind of code as in your ReadData method where you write byte by byte.
Edit:
The size of the data is probably not the problem. You should still avoid the hard coded buffer size and stick with the code change you mention in the comment. Given your comment the problem lies elsewhere.
In both your sender and your receiver you write to a memory stream and then read from it. You can't do that unless you set the current position in the stream back to zero between the write and read.
So in your sendData method on the client add the line
mstream.Seek(0, SeekOrigin.Begin);
right after
bform.Serialize(mstream, data);
And in the ReadData method of your server add the line
memStream.Seek(0, SeekOrigin.Begin);
right before
Packet packet = (Packet)bform.Deserialize(memStream);
That way the memory stream is set to the beginning before you try and read from it.
I think you could just skip the memory stream all together and just read and write to the network stream, but you might have other reasons for it.