I'm interested in pulling a file from online a .txt file.
The txt file stores:
filename
md5 hash
filename
md5 hash
I am interested in getting the data from online then comparing the data to local files.
byte[] buffer = new byte[512];
WebRequest test = WebRequest.Create("http://www.domain.com/file.txt");
Stream something = test.GetRequestStream();
something.Read(buffer,0,20);
I don't quite understand streams and how to go about reading just one line from the file. I do not want to download the file first then retrieve the data. I'm interested in just pulling it from online. How different are "streams" vs normal IO, with StreamWriter and StreamReader?
EDIT--
WebRequest myWebRequest = WebRequest.Create("http://www.domain.com/file.txt");
WebResponse myReponse = myWebRequest.GetResponse();
Stream recStream = myReponse.GetResponseStream();
StreamReader reader = new StreamReader(recStream);
txt_status.Text = reader.ReadLine();
GetRequestStream provides a stream for writing to. If you want the returned data to walk through make use of GetResponseStream
...
Stream ReceiveStream = myWebResponse.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
// Pipe the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader( ReceiveStream, encode );
Console.WriteLine("\nResponse stream received");
Char[] read = new Char[256];
// Read 256 charcters at a time.
int count = readStream.Read( read, 0, 256 );
Console.WriteLine("HTML...\r\n");
while (count > 0)
{
// Dump the 256 characters on a string and display the string onto the console.
String str = new String(read, 0, count);
Console.Write(str);
count = readStream.Read(read, 0, 256);
}
...
If you're reading text, try using a TextReader
WebRequest test = WebRequest.Create("http://www.domain.com/file.txt");
Stream something = test.GetRequestStream();
TextReader reader = (TextReader)new StreamReader(something);
string textfile = reader.ReadToEnd();
All a stream is, is a sequence of bytes. MemoryStreams, FileStream, etc. all inherit from System.IO.Stream
If you are simply attempting to compare the MD5 has against a local file, you could do something such as the following (not tested):
// Download File
WebClient wc = new WebClient();
byte[] bytes = wc.DownloadData();
MD5 md5 = System.Security.Cryptography.MD5.Create();
byte[] hash = md5.ComputeHash(bytes);
StringBuilder onlineFile = new StringBuilder();
for (int i = 0; i < hash.Length; i++)
{
onlineFile.Append(hash[i].ToString("X2"));
}
// Load Local File
FileStream fs = new FileStream(#"c:\yourfile.txt",FileMode.Open);
byte[] fileBytes = new byte[fs.Length];
fs.Read(fileBytes, 0, fileBytes.Length);
byte[] hash = md5.ComputeHash(fileBytes);
StringBuilder localFile = new StringBuilder();
for (int i = 0; i < hash.Length; i++)
{
onlineFile.Append(hash[i].ToString("X2"));
}
if(localFile.ToString() == onlineFile.ToString())
{
// Match
}
Related
I am trying to create a web service that returns a pdf file as a byte[] and then the app that consumes it grabs the byte[] and saves it as a pdf file and then opens it. The file fails to open at the end.
Here is the Web Service that returns a byte[]
[WebMethod]
public byte[] XXXX(int fileID)
{
try
{
using (EntitiesModel dbContext = new EntitiesModel())
{
string fileFullPath = .....
.......
if (fileFullNamePath != null)
{
FileStream fileStream = new FileStream(fileFullNamePath, FileMode.Open, System.IO.FileAccess.Read);
int len = fileStream.Length.ToInt();
Byte[] documentContents = new byte[len];
fileStream.Read(documentContents, 0, len);
fileStream.Close();
return documentContents;
Then it is called from an app with the following code
string soap = "<?xml version=\"1.0\" encoding=\"utf - 8\"?>" +
"<soap:Envelope xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\">" +
"<soap:Body>" +
"<XXXX xmlns=\"http://tempuri.org/\">" +
"<fileID>XXXXX</fileID>" +
"</XXXX>" +
"</soap:Body>" +
"</soap:Envelope>";
string localhostContext = #"http://localhost:3381/";
string webserviceAddress = #"XXXX/XXXX/XXXXX.asmx";
string url = localhostContext + webserviceAddress ;
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "text/xml";
request.ContentLength = soap.Length;
request.Timeout = 20000;
request.Method = "POST";
using (Stream stream = request.GetRequestStream())
{
using (StreamWriter streamWriter = new StreamWriter(stream))
{
streamWriter.Write(soap); }
}
}
byte[] bytes;
try
{
WebResponse response = request.GetResponse();
bytes = ReadFully(response.GetResponseStream());
}
catch (Exception exception)
{
throw;
}
private byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16*1024];
using (MemoryStream memoryStream = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
memoryStream.Position = 0;
memoryStream.Write(buffer, 0, read);
}
return memoryStream.ToArray();
}
}
FileStream objfilestream =
new FileStream(fileName, FileMode.Create,FileAccess.ReadWrite);
objfilestream.Write(bytes, 0, bytes.Length);
objfilestream.Close();
var process = Process.Start(fileName);
The code runs fine and creates a pdf and then tries to open that pdf. But the file can not be opened. Adobe Acrobat gives the error
Adobe Acrobat Reader could not open XXX.pdf because it is either not a
supported file type or because the file has been damaged (for example, it
was sent as an email attachment and wasn't correctly decoded).
Because I am not getting an error in the code I am at a loss to know where the error is that is not creating the proper file.
There was an issue with the Stream variable called input was not giving length so I used Jon Skeet's suggestion here Stackoverflow:Creating a byte array from a stream
new byte[16*1024];
rather than
new byte[input.length]
There were three things wrong.
memoryStream.Position = 0;
in the while loop was problematic so I removed it.
Secondly when reading the stream. What it returned was the SOAP XMl message with the encoded base64 string in the the XXXXResult XML tag. So I had to extract that.
Finally I had to use
byte[] fileResultBytes = Convert.FromBase64String(resultString);
to get the byte[] from the resultString extracted from the SOAP message. In the test SOAP message, that can be generated locally, it tells you the type of this result string. I missed that initially.
Thanks to VC.One and CodeCaster for their correct suggestions.
Problem still there while i tried below three methods.
Using Window API "URLDownloadToFile"
WebClient Method
webclient.DownloadFile(url,dest) ''With/Without credientials
HTTP WebRequest Method:
public static void Download(String strURLFileandPath, String strFileSaveFileandPath)
{
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(strURLFileandPath);
HttpWebResponse ws = (HttpWebResponse)wr.GetResponse();
Stream str = ws.GetResponseStream();
byte[] inBuf = new byte[100000];
int bytesToRead = (int) inBuf.Length;
int bytesRead = 0;
while (bytesToRead > 0)
{
int n = str.Read(inBuf, bytesRead,bytesToRead);
if (n==0)
break;
bytesRead += n;
bytesToRead -= n;
}
FileStream fstr = new FileStream(strFileSaveFileandPath, FileMode.OpenOrCreate, FileAccess.Write);
fstr.Write(inBuf, 0, bytesRead);
str.Close();
fstr.Close();
}
Still i m facing the problem, file i am able to download at my local system, but when i open that it show Corrupt pdf.
!!!!I just want to download the pdf from URL and thats my query in VB.net/C# not using response method of ASP.net.
Please help if someone face this real problem.
Thanks in Advance!!!
Your code only writes 100000 bytes of the downloaded PDF and hence every PDF that is bigger than 100000 bytes gets corrupted.
To read more bytes you have to write the contents of every buffer to the FileStream.
The following should do it:
HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(strURLFileandPath);
using (HttpWebResponse ws = (HttpWebResponse)wr.GetResponse())
using (Stream str = ws.GetResponseStream())
using (FileStream fstr = new FileStream(strFileSaveFileandPath, FileMode.OpenOrCreate, FileAccess.Write))
{
byte[] inBuf = new byte[100000];
int bytesRead = 0;
while ((bytesRead = str.Read(inBuf, 0, inBuf.Length)) > 0)
fstr.Write(inBuf, 0, bytesRead);
}
(It's good coding practice to use a using on every IDisposable instead of manually closing the streams.)
I have an application that reads string data in from a stream. The string data is typically in English but on occasion it encounters something like 'Jalapeño' and the 'ñ' comes out as '?'. In my implementation I'd prefer to read the stream contents into a byte array but I could get by reading the contents into a string. Any idea what I can do to make this work right?
Current code is as follows:
byte[] data = new byte[len]; // len is known a priori
byte[] temp = new byte[2];
StreamReader sr = new StreamReader(input_stream);
int position = 0;
while (!sr.EndOfStream)
{
int c = sr.Read();
temp = System.BitConverter.GetBytes(c);
data[position] = temp[0];
position++;
}
input_stream.Close();
sr.Close();
You can pass the encoding to the StreamReader as in:
StreamReader sr = new StreamReader(input_stream, Encoding.UTF8);
However, I understand that Encoding.UTF8 is used by default according to the documentation.
Update
The following reads 'Jalapeño' fine:
byte[] bytes;
using (var stream = new FileStream("input.txt", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var index = 0;
var count = (int) stream.Length;
bytes = new byte[count];
while (count > 0)
{
int n = stream.Read(bytes, index, count);
if (n == 0)
throw new EndOfStreamException();
index += n;
count -= n;
}
}
// test
string s = Encoding.UTF8.GetString(bytes);
Console.WriteLine(s);
As does this:
byte[] bytes;
using (var stream = new FileStream("input.txt", FileMode.Open, FileAccess.Read, FileShare.Read))
{
var reader = new StreamReader(stream);
string text = reader.ReadToEnd();
bytes = Encoding.UTF8.GetBytes(text);
}
// test
string s = Encoding.UTF8.GetString(bytes);
Console.WriteLine(s);
From what I understand the 'ñ' character is represented as 0xc391 in the text when the text is stored with UTF encoding. When you only read a byte, you'll loose data.
I'd suggest reading the whole stream as a byte array (the first example) and then do the encoding. Or use StreamReader to do the work for you.
Since you're trying to fill the contents into a byte-array, don't bother with the reader - it isn't helping you. Use just the stream:
byte[] data = new byte[len];
int read, offset = 0;
while(len > 0 &&
(read = input_stream.Read(data, offset, len)) > 0)
{
len -= read;
offset += read;
}
if(len != 0) throw new EndOfStreamException();
I have an application in ASP.NET where user can upload ZIP file. I'm trying to extract file using ICSharpZipLib (I also tried DotNetZip, but had same issue).
This zip file contains single xml document (9KB before compress).
When I open this file with other applications on my desktop (7zip, windows explorer) it seems to be ok.
My unzip method throws System.OutOfMemoryException and I have no idea why is that. When I debugged my unziping method I noticed that zipInputStreams' Length property throws Exception and is not available:
Stream UnZipSingleFile(Stream memoryStream)
{
var zipInputStream = new ZipInputStream(memoryStream);
memoryStream.Position = 0;
zipInputStream.GetNextEntry();
MemoryStream unzippedStream = new MemoryStream();
int len;
byte[] buf = new byte[4096];
while ((len = zipInputStream.Read(buf, 0, buf.Length)) > 0)
{
unzippedStream.Write(buf, 0, len);
}
unzippedStream.Position = 0;
memoryStream.Position = 0;
return unzippedStream;
}
and here's how I get string of unzippedStream:
string GetString()
{
var reader = new StreamReader(unzippedStream);
var result = reader.ReadToEnd();
unzippedStream.Position = 0;
return result;
}
From their wiki:
"Sharpzip supports Zip files using both stored and deflate compression methods and also supports old (PKZIP 2.0) style and AES encryption"
Are you sure the format of the uploaded zip file is acceptable for SharpZipLib?
While this post is quite old, I think it could be beneficial to illustrate how I did this for compression and decompression using ICSharpZipLib (C# package version 1.1.0). I put this together by looking into the examples shown here (see ie. these compression and decompression examples).
Assumption: The input to the compression and decompression below should be in bytes. If you have ie. an xml file you could load it to an XDocument, and convert it into an XmlDocument with .ToXmlDocument(). From there, you could access the string contents by calling .OuterXml, and converting the string to a byte array.
// Compression (inputBytes = ie. string-to-compress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipOutputStream(outputStream))
{
zipStream.SetLevel(3);
var newEntry = new ZipEntry("someFilename.someExtension");
newEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(newEntry);
StreamUtils.Copy(dataStream, zipStream, new byte[4096]);
zipStream.CloseEntry();
zipStream.IsStreamOwner = false;
}
outputStream.Position = 0;
var outputBytes = outputStream.ToArray();
// Decompression (inputBytes = ie. string-to-decompress, as bytes)
using var dataStream = new MemoryStream(inputBytes);
var outputStream = new MemoryStream();
using (var zipStream = new ZipInputStream(dataStream))
{
while (zipStream.GetNextEntry() is ZipEntry zipEntry)
{
var buffer = new byte[4096];
StreamUtils.Copy(zipStream, outputStream, buffer);
}
}
var outputBytes = outputStream.ToArray();
I'm using the following code to grab a wmv file through a WebResponse. I'm using a thread to call this function:
static void GetPage(object data)
{
// Cast the object to a ThreadInfo
ThreadInfo ti = (ThreadInfo)data;
// Request the URL
WebResponse wr = WebRequest.Create(ti.url).GetResponse();
// Display the value for the Content-Length header
Console.WriteLine(ti.url + ": " + wr.Headers["Content-Length"]);
string toBeSaved = #"C:\Users\Kevin\Downloads\TempFiles" + wr.ResponseUri.PathAndQuery;
StreamWriter streamWriter = new StreamWriter(toBeSaved);
MemoryStream m = new MemoryStream();
Stream receiveStream = wr.GetResponseStream();
using (StreamReader sr = new StreamReader(receiveStream))
{
while (sr.Peek() >= 0)
{
m.WriteByte((byte)sr.Read());
}
streamWriter.Write(sr.ReadToEnd());
sr.Close();
wr.Close();
}
streamWriter.Flush();
streamWriter.Close();
// streamReader.Close();
// Let the parent thread know the process is done
ti.are.Set();
wr.Close();
}
The file seems to download just fine, but Windows Media Viewer cannot open the file properly. Some silly error about not being able to support the file type.
What incredibly easy thing am I missing?
You just need to download it as binary instead of text. Here's a method that should do the trick for you.
public void DownloadFile(string url, string toLocalPath)
{
byte[] result = null;
byte[] buffer = new byte[4097];
WebRequest wr = WebRequest.Create(url);
WebResponse response = wr.GetResponse();
Stream responseStream = response.GetResponseStream;
MemoryStream memoryStream = new MemoryStream();
int count = 0;
do {
count = responseStream.Read(buffer, 0, buffer.Length);
memoryStream.Write(buffer, 0, count);
if (count == 0) {
break;
}
}
while (true);
result = memoryStream.ToArray;
FileStream fs = new FileStream(toLocalPath, FileMode.OpenOrCreate, FileAccess.ReadWrite);
fs.Write(result, 0, result.Length);
fs.Close();
memoryStream.Close();
responseStream.Close();
}
I do not understand why you are filling MemoryStream m one byte at a time, but then writing the sr to the file. At that point, I believe the sr is empty, and MemoryStream m is never used.
Below is some code I wrote to do a similar task. It gets a WebResponse in 32K chunks at a time, and dumps it directly to a file.
public void GetStream()
{
// ASSUME: String URL is set to a valid URL.
// ASSUME: String Storage is set to valid filename.
Stream response = WebRequest.Create(URL).GetResponse().GetResponseStream();
using (FileStream fs = File.Create(Storage))
{
Byte[] buffer = new Byte[32*1024];
int read = response.Read(buffer,0,buffer.Length);
while (read > 0)
{
fs.Write(buffer,0,read);
read = response.Read(buffer,0,buffer.Length);
}
}
// NOTE: Various Flush and Close of streams and storage not shown here.
}
You are using a StreamReader and a StreamWriter to transfer your stream, but those classes are for handling text. Your file is binary and chances are that sequences of CR, LF and CR LF may get clobbered when you transfer the data. How NUL characters are handled I have no idea.