I'm developing my own simple webserver in C#, and I have normal html files and subfolders etc in place and working fine... however my issue lies with sending PNG files.
Once the browser has made the HTTP request and my other code has done its magic, I need to send a response header. Here is my code to do this:
else if (URL.EndsWith(".ico") || URL.EndsWith(".png"))
{
if (File.Exists(Program.Start_Directory + URL))
{
byte[] ImgFile = File.ReadAllBytes(Program.Start_Directory + URL);
sw.Write("HTTP/1.0 200 OK\r\n");
sw.Write("Content-Type: image/png\r\n");
sw.Write("Content-Length: " + ImgFile.Length + "\r\n");
sw.Write("\r\n");
sw.Write(ImgFile);
}
else
{
FileNotFoundPage(sw);
}
}
sw in this instance being the StreamWriter for the socket connection to the browser.
When I run this, the browser screen goes black like it usually does when its about to load an image, but no image loads and the spinning loading wheel remains spinny indefinitely.
How can I get this to work? Thanks.
Things get messy when you use a StreamWriter (which is designed to write strings with a specific Encoding to a stream) with binary data.
It looks like you are invoking this overload of StreamWriter.Write with the belief that it writes the bytes verbatim to the output stream. The docs state that this overload actually...
Writes the text representation of an object to the text string or stream by calling the ToString method on that object
You have two options. Carry on using StreamWriter and Flush, then write the binary data directly to the underlying stream:
byte[] ImgFile = File.ReadAllBytes(Program.Start_Directory + URL);
sw.Write("HTTP/1.0 200 OK\r\n");
//...etc
sw.Flush();
sw.BaseStream.Write(ImgFile,0,ImgFile.Length);
or just do everything in bytes:
var sb=new StringBuilder();
sb.AppendLine("HTTP/1.0 200 OK");
//...etc
var headerBytes = Encoding.ASCII.GetBytes(sb.ToString());
Stream str = iGotTheStreamFromSomewhere;
str.Write(headerBytes,0,headerBytes.Length);
str.Write(ImgFile,0,ImgFile.Length);
Related
I am working on a project where I am populating some pdfs on the back-end, then I convert those pdfs into a List of byte[] which gets merged into one very large array and finally, send back via the response body as a Memory Stream.
My issue is that this is a large amount of data and during the process of getting the list of byte arrays to merge I am using a lot of memory.
I am wondering if instead of converting the final merged byte[] into a Memory Stream and adding that to the response body; could I create several Memory Stream objects that I an append to the Response.Body as they are created? Alternatively, I wondered if there was a way to use the one Memory Stream and just keep adding to it as a create each new byte[] for each pdf document?
Edit: This is probably a little long winded but I was too vague with my original post. At the core of what I am trying to do I have several pdf documents, they are each several pages long. Each of them is represented in the code below as one of the byte[] items in the filesToMerge List. Ideally, I would like to go through these one by one and convert them into a memory stream and send them to the client one right after the other in a loop. However, when I try to do this I get errors that the Response body has already been sent. Is there a way to append something to the response body so it is updated each time through the loop?
[HttpGet("template/{formId}/fillforms")]
public void FillForms(string formId/*, [FromBody] IList<IDictionary<string, string>> fieldDictionaries*/)
{
List<byte[]> filesToMerge = new List<byte[]>();
// For testing
var mockData = new MockData();
IList<IDictionary<string, string>> fieldDictionaries = mockData.GetMock1095Dictionaries();
foreach(IDictionary<string, string> dictionary in fieldDictionaries)
{
var populatedForm = this.dataRepo.PopulateForm(formId, dictionary);
// write to rb
filesToMerge.Add(populatedForm);
}
byte[] mergedFilesAsByteArray = this.dataRepo.GetMergedByteArray(filesToMerge);
this.SendResponse(formId + "_filled.pdf", new MemoryStream(mergedFilesAsByteArray));
}
private void SendResponse(string formName, MemoryStream ms, IDictionary<string, string> fieldData = null)
{
Response.Clear();
Response.ContentType = "application/pdf";
Response.Headers.Add("content-disposition", $"attachment;filename={formName}.pdf");
ms.WriteTo(Response.Body);
}
Memory streams are really just byte arrays with a bunch of nice methods on top. So switching to byte arrays won't help that much. A problem that a log of people run into when dealing with byte arrays and memory streams is not releasing the memory when you are done with the data since they occupy the memory of the machine you are running on so you can easily run out of memory. So you should be disposing of data as soon as you don't need it anymore with "using statements" as an example. Memory streams has a method called Dispose that will release all resources used by the stream
If you wanted to transfer the data from your application as quickly as possible the best approach would be to cut the stream into smaller parts and re-assemble them in the correct order at the destination. You could cut them to 1mb or 126kb really whatever you want. When you send the data to the destination you need to also pass what the order number of this part is because this method allows you to POSt the data in parallel and there is no guarantee of order.
To split a stream into multiple streams
private static List<MemoryStream> CreateChunks(Stream stream)
{
byte[] buffer = new byte[4000000]; //set the size of your buffer (chunk)
var returnStreams = new List<MemoryStream>();
using (MemoryStream ms = new MemoryStream())
{
while (true) //loop to the end of the file
{
var returnStream = new MemoryStream();
int read = stream.Read(buffer, 0, buffer.Length); //read each chunk
returnStream.Write(buffer, 0, read); //write chunk to [wherever];
if (read <= 0)
{ //check for end of file
return returnStreams;
}
else
{
returnStream.Position = 0;
returnStreams.Add(returnStream);
}
}
}
}
I then looped through the streams that were created to create tasks to post to the service and each task would post to the server. I would await all of the tasks to finish then call my server again to tell it I had finished uploading and it could combine all of the data into one in the correct order. My service has the concept of an upload session to keep track of all of the parts and which order they would go in. It would also save each part to the database as they came in; in my case Azure Blob storage.
It's not clear why you would be getting errors copying the contents of multiple MemoryStreams to the Response.Body. You should certainly be able to do this, although you'll need to be sure not to try and change response headers or the status code after you begin writing data (also don't try to call Response.Clear() after you begin writing data).
Here is a simple example of starting a response and then writing data:
[ApiController]
[Route("[controller]")]
public class RandomDataController : ControllerBase {
private readonly ILogger<RandomDataController> logger;
private const String CharacterData = "abcdefghijklmnopqrstuvwxyz0123456789 ";
public RandomDataController(ILogger<RandomDataController> logger) {
this.logger = logger;
}
[HttpGet]
public async Task Get(CancellationToken cancellationToken) {
this.Response.ContentType = "text/plain";
this.Response.ContentLength = 1000;
await this.Response.StartAsync(cancellationToken);
logger.LogInformation("Response Started");
var rand = new Random();
for (var i = 0; i < 1000; i++) {
// You should be able to copy the contents of a MemoryStream or other buffer here instead of sending random data like this does.
await this.Response.Body.WriteAsync(Encoding.UTF8.GetBytes(CharacterData[rand.Next(0, CharacterData.Length)].ToString()), cancellationToken);
Thread.Sleep(50); // This is just to demonstrate that data is being sent to the client as it is written
cancellationToken.ThrowIfCancellationRequested();
if (i % 100 == 0 && i > 0) {
logger.LogInformation("Response In Flight {PercentComplete}", (Double)i / 1000);
}
}
logger.LogInformation("Response Complete");
}
}
You can verify that this streams data back to the client using netcat:
% nc -nc 127.0.0.1 5000
GET /randomdata HTTP/1.1
Host: localhost:5000
Connection: Close
(Enter an extra blank line after Connection: Close to begin the request). You should see data appear in netcat as it is written to Response.Body on the server.
One thing to note is that this approach involves calculating the length of the data to be sent up front. If you are unable to calculate the size of the response up front, or prefer not to, you can look into Chunked Transfer Encoding, which ASP.Net should automatically use if you start writing data to the Response.Body without specifying the Content-Length.
I am generating some JSON content, and then GZipping that content, before returning the gzipped content to the user, from an MVC Controller Action.
The generation of the content, and gzipping, is working correctly as I can output the generated file to disk, and then I can open that file using GZip. However, when the content is returned to the browser, the content has been corrupted.
I have tried several different approaches to returning the content to the browser, such as
return File(byte[], "application/gzip");
return new FileStreamResult(stream, "application/gzip")
And also writing directly to the Response using BinaryWrite() and WriteFile() methods
No matter what I do, the file I receive in the browser is corrupt.
This code shows the manner in which I am currently trying to return the file content.
// This line writes my content byte[] array to disk. This file when opened with gzip works fine.
System.IO.File.WriteAllBytes(#"C:\temp\test.vcp", result.FileBytes);
// Writing out the byte array to the Response results in a corrupt file. I have also attempted to Response.WriteFile(#"C:\temp\test.vcp") which also results in a corrupt file.
Response.Clear();
Response.ContentType = "application/gzip";
Response.AppendHeader("Content-Disposition", cd.ToString());
Response.AddHeader("Content-Length", result.FileBytes.Length.ToString());
Response.BinaryWrite(result.FileBytes);
Response.Flush();
Response.Close();
Response.End();
As the file I am creating can be written to disk, and can be read using Gzip, but the file received by the browser is corrupt, I am confident that my file creation is OK. But somehow after writing the file to the Response, it is being corrupted.
I did wonder if maybe some sort of HTTPHandler is manipulating the result, but I haven't added any Handlers (that I can see).
I am running the application locally currently through IISExpress. How can I check what HttpHandlers/HttpModules are being applied to the pipeline?
Ultimately I expect to receive the exact same file in my browser as is written to disk.
For reference, my generated content is 132 bytes in length, but the browser receives 216 bytes. I have noticed when looking at the byte structure of the received data, there is a repeating pattern of 3 bytes in the content, with the values 239, 191, 189. It almost looks like the resultant byte array has been stuffed or padded with these 3 bytes.
EDIT
Here is a standalone Action method which demonstrated the issue.
[HttpGet]
public void GetFile()
{
byte[] text = Encoding.ASCII.GetBytes(#"{""PetName"":""Doggy McDocFace"",""OwnerName"":""Kurt""}");
byte[] compressed = Compress(text);
var cd = new System.Net.Mime.ContentDisposition
{
// for example foo.bak
FileName = "ExampleFile.vcp",
// always prompt the user for downloading, set to true if you want
// the browser to try to show the file inline
Inline = true,
};
System.IO.File.WriteAllBytes(#"C:\temp\ExampleFile.vcp", compressed);
Response.Clear();
Response.ContentType = "application/gzip";
Response.AppendHeader("Content-Disposition", cd.ToString());
Response.AddHeader("Content-Length", compressed.Length.ToString());
Response.BinaryWrite(compressed);
Response.Flush();
Response.Close();
Response.End();
}
public byte[] Compress(byte[] raw)
{
using (var memory = new MemoryStream())
{
using (var gzip = new GZipStream(memory, CompressionMode.Compress, true))
{
gzip.Write(raw, 0, raw.Length);
}
return memory.ToArray();
}
}
Here I am spoofing my JSON content, and then compressing it. The file written to disk works fine, and can be opened with my GZip application (I use 7-zip). However, the file received by the browser is corrupt. 7-zip cannot recognise it as a gzip file.
EDIT 2
So it looks like (Thanks to #Will) that the content when written to Response is falling foul of UTF-8 encoding. I cannot work out how though, as in my example above I am using Encoding.ASCII.GetBytes() to convert my string to a byte[] array.
I've tried setting the
Response.Charset = Encoding.ASCII.EncodingName;
Response.ContentEncoding = Encoding.ASCII;
But this still doesn't result in a valid file downloaded.
Edit 3
I've narrowed down the issue to the GZip encryption of the data. If I do not encrypt the data, then the plain text file downloads fine. However, encrypting the byte[] array and then writing that byte[] array to the Repsonse is resulting in what seems like UTF-8 encoding issues. Any bytes with a value over 127 are corrupted with the 3 bytes I mention further up. I cannot work out why the Response is treating this encrypted data in this way. My assumption is that when the Byte[] array is just plain text as a byte[] array, then this is handled fine. As soon as it is a proper byte[] array, i.e not just a string as a byte[] array, then some other conversion of encoding is going on in the Response.
You can try ActionFilterAttribute
Basically response filters look at the Response output stream as it's written and convert the data flowing through it.
GZip/Deflate Compression in ASP.NET MVC
public class CompressAttribute : ActionFilterAttribute
{
public override void OnActionExecuting(ActionExecutingContext filterContext)
{
var encodingsAccepted = filterContext.HttpContext.Request.Headers["Accept-Encoding"];
if (string.IsNullOrEmpty(encodingsAccepted)) return;
encodingsAccepted = encodingsAccepted.ToLowerInvariant();
var response = filterContext.HttpContext.Response;
if (encodingsAccepted.Contains("deflate"))
{
response.AppendHeader("Content-encoding", "deflate");
response.Filter = new DeflateStream(response.Filter, CompressionMode.Compress);
}
else if (encodingsAccepted.Contains("gzip"))
{
response.AppendHeader("Content-encoding", "gzip");
response.Filter = new GZipStream(response.Filter, CompressionMode.Compress);
}
}
}
[Compress]
[HttpGet]
public ActionResult GetFile()
{...}
I have function name uploadLayerIcons which is as follows:
private void uploadLayerIcon(string LayerName)
{
Bitmap icon= new Bitmap(#"C:\Users\HP\Desktop\911\Prism\Prism_Resources\m.png");
System.IO.MemoryStream stream = new System.IO.MemoryStream();
icon.Save(stream, System.Drawing.Imaging.ImageFormat.Bmp);
byte[] imageBytes = stream.ToArray();
// Convert byte[] to Base64 String
string base64String = Convert.ToBase64String(imageBytes);
HttpWebRequest m_ObjRequest; //Request which needed to be sent to server
HttpWebResponse m_ObjResponse; // Response which is sent back from the server to the client
StreamReader reader = null; // making a stream reader to read the web pageand initialize it to null
string m_Url = "http://192.168.1.30/muneem/erp/uploadIcon.php" + "?bitmap=" + base64String + "&layerName=" + LayerName; // the url of that web page
string m_Response = "";
m_ObjRequest = (HttpWebRequest)WebRequest.Create(m_Url); // creating the url and setting the values
m_ObjRequest.Method = "GET";
m_ObjRequest.ContentType = "application/json; charset=utf-8";
//m_ObjRequest.ContentLength = 500;
m_ObjRequest.KeepAlive = false;
m_ObjResponse = (HttpWebResponse)m_ObjRequest.GetResponse(); // getting response from the server
using (reader = new StreamReader(m_ObjResponse.GetResponseStream())) // using stream reader to read the web page
{
m_Response = reader.ReadToEnd();
reader.Close(); // Close the StreamReader
}
m_ObjResponse.Close();
m_ObjRequest = null;
m_ObjResponse = null;
}
UploadIcon.php file is as follows:
<?php
$bitmap=$_GET['bitmap'];
$name=$_GET['layerName'];
$data = base64_decode($bitmap);
$filepath="app/uams/uploadedImages/".$name.".jpg";
file_put_contents($filepath,$data);
?>
Its not converting correctly the same image which i have sent to server.
I have search on internet many thing but all in vain. I have also tried this thing
Bitmap icon= new Bitmap(#"C:\Users\HP\Desktop\911\Prism\Prism_Resources\m.png");
icon.save("Path of srrver")
But its not working.
So, you are doing it pretty much wrong. First of all, if you change the extension of the file to .jpg it does not automagically become jpg image.
So, what I suggest you to do is to send the raw png data instead of bitmap, and then using something like this in php:
<?
$imagedata = $_POST["data"];
$im = imagecreatefromstring($imagedata);
$filepath="app/uams/uploadedImages/image.jpg";
imagejpeg($im,$filepath);
?>
Also, as pointed out in previous answer by #DoXicK, do not send file by GET method, you should post it instead, and that is what this example is based on.
PHP's function imagecreatefromstring identifies the image type, and creates the gdlib object accordingly (but it does not work very well with bitmaps). That is why I suggested that you use raw png data instead of converting it to bitmap. Also, bitmap data is unneccesary large for transfer.
For imagecreatefromstring to work you need GD Library installed and enabled. To see if it is enabled create an empty file (named for example info.php) and inside it put only
<?
phpinfo();
?>
If you see GD Support set to Enable on the page, when you open the file, you have gdlib enabled. If you do not see it, do the following:
On windows find ;extension=php_gd2.dll in php.ini file of your php installation, and uncomment it (remove ; from the beginning) so it now is extension=php_gd2.dll and then restart Apache.
On linux you need to do sudo apt-get install php5-gd and then restart Apache.
you are loading a PNG to BMP file format
You are sending a file by GET
you are saving the BMP file as JPG te minute you receive the BMP
So:
Don't open as PNG as BMP. open a PNG as PNG as it is either smaller or the same size. There is no need for the BMP here...
POST it
Just because you call it a JPG, doesn't make it a JPG. It currently is a BMP, saved to a JPG.
IF it even saves a .jpg file, it is a .bmp file with the wrong extension.
So due to content being on a different drive than my server application's virtual directory, I've had to implement a file getting service.
I am able to the get the file size. Then I want to set the content headers so that the browser knows the total size and thus knows how to proportion the seek bar.
Here I set the total size header:
string bytes = Convert.ToString( fInfo.Length );
Response.AddHeader("Content-Length", bytes );
Response.AddHeader("Content-Range", "bytes 0-" + bytes + "/" + bytes);
Resulting in:
Content-Length: 1389363
Content-Type: video/ogg
Content-Range: bytes 0-1389363/1389364
I downloaded the resulting file and confirmed that the bytes match. Works fine in Firefox, all wacky in chrome. In chrome it plays to the end, but does not move the seek bar and then shows negative infinity in the current time when it reaches the end. ALSO I cannot scrub the video at all, presumably because it has an invalid duration.
playing the same file directly in chrome works properly, so maybe it's some content header that chrome wants that firefox don't give a s#%t about?
Any ideas? Wrong length unit? built-in server side g-zip encoding interfering?
I'm using standard video object:
<video class="videoplayer" controls="" autoplay="autoplay" tabindex="0">
<source type="video/ogg" src="http://vb_html.dev/GetFile.aspx?filename=c:/_assets/4c2c09c2-f2ff-e011-a992-009056af18ff/softsignage/softsignage-00000000600.ogv"></source>
Your browser does not support the video tag.
</video>
For more reference this is how I'm passing the file data:
FileStream mystream = new FileStream(anyFilePath, FileMode.Open, FileAccess.Read);
byte[] buffer = new byte[4096];
using (BinaryReader reader = new BinaryReader(mystream))
{
while (true)
{
int bytesRead = mystream.Read(buffer, 0, buffer.Length);
if (bytesRead == 0) break;
Response.OutputStream.Write(buffer, 0, bytesRead);
}
}
Try setting status code to 206:
response.StatusCode = 206;
Basically, you want to get the request range in bytes, and return that portion of the file. You can do this in MVC with
RangeFileStreamResult(Stream fileStream, string contentType, string fileName, DateTime modificationDate);
Using the Codeplex example found in this library http://mvcresumingactions.codeplex.com/
--
I'm trying to obtain an image to encode to a WordML document. The original version of this function used files, but I needed to change it to get images created on the fly with an aspx page. I've adapted the code to use HttpWebRequest instead of a WebClient. The problem is that I don't think the page request is getting resolved and so the image stream is invalid, generating the error "parameter is not valid" when I invoke Image.FromStream.
public string RenderCitationTableImage(string citation_table_id)
{
string image_content = "";
string _strBaseURL = String.Format("http://{0}",
HttpContext.Current.Request.Url.GetComponents(UriComponents.HostAndPort, UriFormat.Unescaped));
string _strPageURL = String.Format("{0}{1}", _strBaseURL,
ResolveUrl("~/Publication/render_citation_chart.aspx"));
string _staticURL = String.Format("{0}{1}", _strBaseURL,
ResolveUrl("~/Images/table.gif"));
string _fullURL = String.Format("{0}?publication_id={1}&citation_table_layout_id={2}",
_strPageURL, publication_id, citation_table_id);
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_fullURL);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream image_stream = response.GetResponseStream();
// Read the image data
MemoryStream ms = new MemoryStream();
int num_read;
byte[] crlf = System.Text.Encoding.Default.GetBytes("\r\n");
byte[] buffer = new byte[1024];
for (num_read = image_stream.Read(buffer, 0, 1024); num_read > 0; num_read = image_stream.Read(buffer, 0, 1024))
{
ms.Write(buffer, 0, num_read);
}
// Base 64 Encode the image data
byte[] image_bytes = ms.ToArray();
string encodedImage = Convert.ToBase64String(image_bytes);
ms.Position = 0;
System.Drawing.Image image_original = System.Drawing.Image.FromStream(ms); // <---error here: parameter is not valid
image_stream.Close();
image_content = string.Format("<w:p>{4}<w:r><w:pict><w:binData w:name=\"wordml://{0}\">{1}</w:binData>" +
"<v:shape style=\"width:{2}px;height:{3}px\">" +
"<v:imagedata src=\"wordml://{0}\"/>" +
"</v:shape>" +
"</w:pict></w:r></w:p>", _word_image_id, encodedImage, 800, 400, alignment.center);
image_content = "<w:br w:type=\"text-wrapping\"/>" + image_content + "<w:br w:type=\"text-wrapping\"/>";
}
catch (Exception ex)
{
return ex.ToString();
}
return image_content;
Using a static URI it works fine. If I replace "staticURL" with "fullURL" in the WebRequest.Create method I get the error. Any ideas as to why the page request doesn't fully resolve?
And yes, the full URL resolves fine and shows an image if I post it in the address bar.
UPDATE:
Just read your updated question. Since you're running into login issues, try doing this before you execute the request:
request.Credentials = CredentialCache.DefaultCredentials
If this doesn't work, then perhaps the problem is that authentication is not being enforced on static files, but is being enforced on dynamic files. In this case, you'll need to log in first (using your client code) and retain the login cookie (using HttpWebRequest.CookieContainer on the login request as well as on the second request) or turn off authentication on the page you're trying to access.
ORIGINAL:
Since it works with one HTTP URL and doesn't work with another, the place to start diagnosing this is figuring out what's different between the two requests, at the HTTP level, which accounts for the difference in behavior in your code.
To figure out the difference, I'd use Fiddler (http://fiddlertool.com) to compare the two requests. Compare the HTTP headers. Are they the same? In particular, are they the same HTTP content type? If not, that's likely the source of your problem.
If headers are the same, make sure both the static and dynamic image are exactly the same content and file type on the server. (e.g. use File...Save As to save the image in a browser to your disk). Then use Fiddler's Hex View to compare the image content. Can you see any obvious differences?
Finally, I'm sure you've already checked this, but just making sure: /Publication/render_citation_chart.aspx refers to an actual image file, not an HTML wrapper around an IMG element, right? This would account for the behavior you're seeing, where a browser renders the image OK but your code doesn't.