I'm trying to upload a file to VirusTotal using .Net Core.But the uploaded file size is Zero Bytes.Why does this happen?
[Route("api/[controller]")]
public class ScannerController : Controller
{ [HttpGet]
public async Task<VirusTotalNet.Results.FileReport> ScanAsync(string file_id)
{
file_id = "./wwwroot/Upload/node-v12.14.1-x64.msi";
VirusTotal virusTotal = new VirusTotal("");
// virusTotal.UseTLS = true;
FileStream stream = System.IO.File.OpenRead(file_id);
byte[] fileBytes = new byte[stream.Length];
stream.Read(fileBytes, 0, fileBytes.Length);
VirusTotalNet.Results.FileReport report = await virusTotal.GetFileReportAsync(stream);
return report;
}
}
You've read the entire file into a byte[] and there's an overload of GetFileReportAsync that will take that, so change the parameter from stream to fileBytes:
VirusTotalNet.Results.FileReport report = await virusTotal.GetFileReportAsync(fileBytes);
Derviş Kayımbaşıoğlu suggested resetting the stream's position but it turns out that the location mentioned was incorrect. Either of these:
stream.Seek(0L, SeekOrigin.Begin);
// or
stream.Position = 0L;
Needed to be done immediately before calling GetFileReportAsync, after the file had been read, not before. That would've worked.
But wait, there's more!
There's no need to read the file into fileBytes, which means there's no need to reset the position. The stream can be opened and passed directly to GetFileReportAsync. Including proper resource disposal, the entire method becomes this:
[HttpGet]
public async Task<VirusTotalNet.Results.FileReport> ScanAsync(string file_id)
{
file_id = "./wwwroot/Upload/node-v12.14.1-x64.msi";
VirusTotal virusTotal = new VirusTotal("");
// virusTotal.UseTLS = true;
using (FileStream stream = System.IO.File.OpenRead(file_id))
{
VirusTotalNet.Results.FileReport report = await virusTotal.GetFileReportAsync(stream);
return report;
}
}
This allows both the file to be read and the socket to be written asynchronously, and the data can be buffered in small amounts so that large files don't have to be loaded entirely into memory.
Related
I have small sig files that are exactly 256 bytes. When uploading to a file upload controller on asp.net core web app, the buffer is occupied correctly for the 256 positions but they aren't written to the output stream and the file is empty. CopyToAsync works fine. This will only happen to certain files. The problem is reproducible on a console application:
string SoureFile = #"C:\Users\me\source\repos\files\mySigFile.sig";
byte[] buffer = new byte[1024 * 64];
string tmp = #"C:\Users\me\Downloads\tempsigfile.tmp";
string tmp2 = #"C:\Users\me\Downloads\tempsigfile2.tmp";
var inputStream = File.Open(SoureFile, FileMode.OpenOrCreate);
//doesn't work
using FileStream writer = new(tmp, FileMode.Create);
int read;
while ((read = await inputStream.ReadAsync(buffer)) != 0)
{
await writer.WriteAsync(buffer.AsMemory(0, read));
}
inputStream.Position = 0;
//works
using (var stream = new FileStream(tmp2, FileMode.Create))
{
await inputStream.CopyToAsync(stream);
}
FileInfo info = new FileInfo(tmp);
Console.WriteLine(info.Length); //0
FileInfo info2 = new FileInfo(tmp2);
Console.WriteLine(info2.Length);//256
Doing this (using declaration, no bracers):
using FileStream writer = new(tmp, FileMode.Create);
means writer will only be disposed at the end of the scope, so at the end of the method. Doing WriteAsync does not necessary mean that information will be written to file right away, it might be written to the internal in-memory buffer and only written to the file when this buffer fills or when you close the file, or when you explicitly call Flush on a stream. You don't do anything of that, the file is only closed at the end of the method, but your check:
FileInfo info = new FileInfo(tmp);
Console.WriteLine(info.Length); //0
is performed before the actual write to file happens, so you see 0. If you actually check the contents of the file after this method (program) completes - you'll see it contains the correct data.
In second case you use using statement:
using (var stream = new FileStream(tmp2, FileMode.Create))
{
await inputStream.CopyToAsync(stream);
}
so you write to a file, close it, and only after that check the contents. Then it works as you expect.
My code is pretty simple; it uses a StringBuilder and a FileStream to write data to a text file and then [optionally] opens the text file for viewing:
public async Task ExportData(data)
{
var sb = new StringBuilder();
sb.AppendLine(BuildStringFromData());
var dir = $#"C:\Ortund\xExports\{DateTime.Now.Date.ToString
("yyyy-MM-dd", CultureInfo.InvariantCulture)}";
var fullPath = $#"{dir}\{filename}.txt";
var stream = new FileStream(fullPath, FileMode.CreateNew, FileAccess.ReadWrite);
var bytes = Encoding.UTF8.GetBytes(sb.ToString());
await stream.WriteAsync(bytes, 0, bytes.Length);
if (MessageBox.Show("Open the file?", "Open?", MessageBoxButton.YesNo)
== MessageBoxResult.Yes)
{
System.Diagnostics.Process.Start(fullPath);
}
}
I awaited the write operation in the file stream thinking this would pause execution of this particular method until all the bytes had been written but that isn't what's happening.
What's happening is that the prompt to open the file is coming up immediately and when I open it, its blank. Notepad++ only informs me a few seconds later that changes have been made to the file and asks me if I want to reload it whereupon I see the data that was exported.
How can I make execution wait until the file data is completely written before asking the user to open the file?
I overlooked the obvious on this one...
As per comment on the question by #Evk, I put the FileStream into a using block, but I also moved it to a new method which takes the data and file path as parameters:
private async Task WriteDataToFile(List<ViewModel> data, string path)
{
using (var fs = new FileStream(path, FileMode.CreateNew, FileAccess.ReadWrite))
{
var sb = new StringBuilder();
// Loop through the data and build the string.
var bytes = Encoding.UTF8.GetBytes(sb.ToString());
await fs.WriteAsync(bytes, 0, bytes.Length);
}
}
The using closes the FileStream which is what I overlooked.
Using the AWSSDK.S3 nuget package, I'm trying to return a file that has been retrieved from an S3 bucket. My starting point is based on the example given in this SO answer.
Example controller code:
public FileResult GetFile(Guid id)
{
// no using block as it will be disposed of by the File method
var amazonResponse = _foo.GetAmazonResponseWrapper(_user, id);
// set Response content-length header
// set Response content-type header
var bufferSize = 1024;
var buffer = new byte[bufferSize];
int bytesRead;
while ((bytesRead = amazonResponse.ResponseStream.Read(buffer, 0, buffer.Length)) > 0 && Response.IsClientConnected)
{
Response.OutputStream.Write(buffer, 0, bytesRead);
Response.OutputStream.Flush();
buffer = new byte[bufferSize];
}
// this will not work (can't read from this stream)
return File(Response.OutputStream, "text/plain", "bar.txt");
}
If I write to a MemoryStream I create and use in the while loop, I will get a file, but there won't be any content.
The only way I've found to get content in the file is to call .ToArray() on the stream like so:
return File(memStream.ToArray(), "text/plain", "foo.txt");
Is there a way to actually stream a file to the browser without loading it into the memory of the web server?
Just pass the stream forward
public FileResult GetFile(Guid id) {
// no using block as it will be disposed of by the File method
var amazonResponse = _foo.GetAmazonResponseWrapper(_user, id);
return File(amazonResponse.ResponseStream, "text/plain", "bar.txt");
}
You have already shown that you can read from the response stream. Then just pass the response stream on and the File result will read it and return the response.
I need to upload a file using Stream (Azure Blobstorage), and just cannot find out how to get the stream from the object itself. See code below.
I'm new to the WebAPI and have used some examples. I'm getting the files and filedata, but it's not correct type for my methods to upload it. Therefore, I need to get or convert it into a normal Stream, which seems a bit hard at the moment :)
I know I need to use ReadAsStreamAsync().Result in some way, but it crashes in the foreach loop since I'm getting two provider.Contents (first one seems right, second one does not).
[System.Web.Http.HttpPost]
public async Task<HttpResponseMessage> Upload()
{
if (!Request.Content.IsMimeMultipartContent())
{
this.Request.CreateResponse(HttpStatusCode.UnsupportedMediaType);
}
var provider = GetMultipartProvider();
var result = await Request.Content.ReadAsMultipartAsync(provider);
// On upload, files are given a generic name like "BodyPart_26d6abe1-3ae1-416a-9429-b35f15e6e5d5"
// so this is how you can get the original file name
var originalFileName = GetDeserializedFileName(result.FileData.First());
// uploadedFileInfo object will give you some additional stuff like file length,
// creation time, directory name, a few filesystem methods etc..
var uploadedFileInfo = new FileInfo(result.FileData.First().LocalFileName);
// Remove this line as well as GetFormData method if you're not
// sending any form data with your upload request
var fileUploadObj = GetFormData<UploadDataModel>(result);
Stream filestream = null;
using (Stream stream = new MemoryStream())
{
foreach (HttpContent content in provider.Contents)
{
BinaryFormatter bFormatter = new BinaryFormatter();
bFormatter.Serialize(stream, content.ReadAsStreamAsync().Result);
stream.Position = 0;
filestream = stream;
}
}
var storage = new StorageServices();
storage.UploadBlob(filestream, originalFileName);**strong text**
private MultipartFormDataStreamProvider GetMultipartProvider()
{
var uploadFolder = "~/App_Data/Tmp/FileUploads"; // you could put this to web.config
var root = HttpContext.Current.Server.MapPath(uploadFolder);
Directory.CreateDirectory(root);
return new MultipartFormDataStreamProvider(root);
}
This is identical to a dilemma I had a few months ago (capturing the upload stream before the MultipartStreamProvider took over and auto-magically saved the stream to a file). The recommendation was to inherit that class and override the methods ... but that didn't work in my case. :( (I wanted the functionality of both the MultipartFileStreamProvider and MultipartFormDataStreamProvider rolled into one MultipartStreamProvider, without the autosave part).
This might help; here's one written by one of the Web API developers, and this from the same developer.
Hi just wanted to post my answer so if anybody encounters the same issue they can find a solution here itself.
here
MultipartMemoryStreamProvider stream = await this.Request.Content.ReadAsMultipartAsync();
foreach (var st in stream.Contents)
{
var fileBytes = await st.ReadAsByteArrayAsync();
string base64 = Convert.ToBase64String(fileBytes);
var contentHeader = st.Headers;
string filename = contentHeader.ContentDisposition.FileName.Replace("\"", "");
string filetype = contentHeader.ContentType.MediaType;
}
I used MultipartMemoryStreamProvider and got all the details like filename and filetype from the header of content.
Hope this helps someone.
Im trying to download and save a file from a HttpWebResponse but im having problems saving the file (other than Text Files) properly.
I think its something to do with this part:
byte[] byteArray = Encoding.UTF8.GetBytes(http.Response.Content);
MemoryStream stream = new MemoryStream(byteArray);
Text Files work fine with the above code but when I try to save the Content to an Image file it gets corrupted.
How do i write this 'string' data to an image file (and other binary files)
Forgot to mention, This is .NET CP 3.5 and I have a wrapper class around the HttpWebResponse class to add OAuth etc.
The problem is you're interpreting the binary data as text, even if it isn't - as soon as you start treating the content as a string instead of bytes, you're in trouble. You haven't given the details of your wrapper class, but I'm assuming your Content property is returning a string - you won't be able to use that. If your wrapper class doesn't let you get at the raw data from the web response, you'll need to modify it.
If you're using .NET 4, you can use the new CopyTo method:
using (Stream output = File.OpenWrite("file.dat"))
using (Stream input = http.Response.GetResponseStream())
{
input.CopyTo(output);
}
If you're not using .NET 4, you have to do the copying manually:
using (Stream output = File.OpenWrite("file.dat"))
using (Stream input = http.Response.GetResponseStream())
{
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = input.Read(buffer, 0, buffer.Length)) > 0)
{
output.Write(buffer, 0, bytesRead);
}
}
Use WebClient.DownloadFile. You can do it manually (something like this), but WebClient is the best bet for simple downloads.
Simplest way I found is:
Use ReadAsByteArrayAsync to get the data.
List item
File.WriteAllBytes to save the file.
No need to loop through anything. Example in context below:
[TestClass]
public class HttpTests
{
[TestMethod]
public void TestMethod1()
{
var jsObject = (dynamic)new JObject();
jsObject.html = "samplehtml.html";
jsObject.format = "jpeg";
const string url = "https://mywebservice.com/api";
var result = GetResponse(url, jsObject).GetAwaiter().GetResult();
File.WriteAllBytes($#"c:\temp\httpresult.{jsObject.format}", result);
}
static async Task<byte[]> GetResponse(string uri, dynamic jsObj)
{
var httpClient = new HttpClient();
var load = jsObj.ToString();
var content = new StringContent(load, Encoding.UTF8, "application/json");
var response = await httpClient.PostAsync(uri, content);
return await response.Content.ReadAsByteArrayAsync();
}
}