We Keep Coding

downloading an unknown type of image to temp and sending to browser with finding content type using magic numbers

Subject:
I'm coming from nodejs so using some examples from nodejs to get the concept across.
Temp Directory: I'm using dotnet core so the app can run on either mac, windows or linux and the confusion lies for temp directory across operating systems where this image will be downloaded. (find temp dir in nodejs on any os -> os.tmpDir())
File format unknown: file format is unknown, not necessary to know in order to download the image and save but is necessary when sending it to the browsers in headers and it can be done using magic numbers. Reference
Download Image from Web
using (WebClient webClient = new WebClient())
{
byte [] data = webClient.DownloadData("https://fbcdn-sphotos-h-a.akamaihd.net/hphotos-ak-xpf1/v/t34.0-12/10555140_10201501435212873_1318258071_n.jpg?oh=97ebc03895b7acee9aebbde7d6b002bf&oe=53C9ABB0&__gda__=1405685729_110e04e71d9");
using (MemoryStream mem = new MemoryStream(data))
{
using (var yourImage = Image.FromStream(mem))
{
// how to save in the temp directory for all operating systems
}
}
}
Send the image to browser
byte[] ar;
using(FileStream fstream = new FileStream(tempPathForImage, FileMode.Open, FileAccess.Read);)
{
ar = new byte[(long)fstream.Length];
fstream.read(ar, 0, fstream.Length);
}
sw.WriteLine("Content-Type: "); // image/jpeg unknown, check first 4 bytes
sw.WriteLine("Content-Length: {0}", ar.Length); //Let's
sw.WriteLine();
sw.BaseStream.Write(ar, 0, ar.Length);
Magic Numbers:
magic numbers are basically first 4 bytes of a file that can help to find out the file type/extension. I've done something similar in nodejs where as image right before it gets streamed, i get a chance to read the first four bytes and then i set the header (content type) and continue streaming there. It's important to set the header before an image starts streaming, that's why you see in the following code checks for writeStream == null
response.on('data', function(chunk){
if(writeStream == null) {
url += '.' + getExtension(chunk.toString('hex', 0, 4));
writeStream = fs.createWriteStream(url);
writeStream.on('error', reject);
writeStream.on('finish', function(){
data.file = url;
resolve(data);
});
}
writeStream.write(chunk);
});
Some file formats and their magic numbers.
"ffd8ffDB": "jpg",
"ffd8ffe0": "jpg",
"ffd8ffe1": "jpg",
"ffd8ffe2": "jpg",
"ffd8ffe3": "jpg",
"ffd8ffe8": "jpg",
"ffd8ffdb": "jpg",
"89504e47": "png",
"47494638": "gif",
Question
Solve the problem of finding the temp directory
How to read first four bytes during the phase of streaming but just before sending, set the headers for content type (for once) and continue streaming.

You can create a specialized stream that wraps another stream, then override Read() to handle that detection. Here's a starting point. Most of this is boilerplate that defers to the wrapped stream.
class ImageDetectionStream : Stream
{
public string Mime { get; private set; }
private readonly Stream _stream;
private readonly byte[] _consideredBytes = new byte[MaxMagicNumberSize];
private int _consideredPosition;
private static readonly IDictionary<byte[], string> Magics = new Dictionary<byte[], string>
{
[new byte[] { 0xff, 0xdb, 0xff, 0xdb }] = "image/jpeg",
[new byte[] { 0xff, 0xd8, 0xff, 0xe0 }] = "image/jpeg",
[new byte[] { 0xff, 0xd8, 0xff, 0xe1 }] = "image/jpeg",
// and so on...
};
private static readonly int MaxMagicNumberSize = Magics.Keys.Max(x => x.Length);
public ImageDetectionStream(Stream stream)
{
_stream = stream ?? throw new ArgumentNullException(nameof(stream));
}
public override int Read(byte[] buffer, int offset, int count)
{
var value = _stream.Read(buffer, offset, count);
if (Mime != null) return value;
Array.Copy(buffer, 0, _consideredBytes, 0, _consideredBytes.Length);
_consideredPosition += value;
if (_consideredPosition < MaxMagicNumberSize) return value;
foreach (var magic in Magics)
{
var possibleMagic = buffer.Take(magic.Key.Length).ToArray();
if (possibleMagic.SequenceEqual(magic.Key))
{
Mime = magic.Value;
break;
}
}
return value;
}
// boilerplate
public override void Flush()
{
_stream.Flush();
}
public override long Seek(long offset, SeekOrigin origin)
{
return _stream.Seek(offset, origin);
}
public override void SetLength(long value)
{
_stream.SetLength(value);
}
public override void Write(byte[] buffer, int offset, int count)
{
_stream.Write(buffer, offset, count);
}
public override bool CanRead => _stream.CanRead;
public override bool CanSeek => _stream.CanSeek;
public override bool CanWrite => _stream.CanWrite;
public override long Length => _stream.Length;
public override long Position
{
get => _stream.Position;
set => _stream.Position = value;
}
}
Example use -
using (var fs = File.OpenRead("\\path\\to\\image\\file"))
using (var imageStream = new ImageDetectionStream(fs))
{
var bytes = new byte[128];
var bytesRead = imageStream.Read(bytes, 0, bytes.Length);
Console.WriteLine($"Image has {imageStream.Mime} type.");
}
Outputs:
Image has image/jpeg type.

How can I play an audio stream and load into it at the same time?

I'm building an application that should play audio from a server. The app should play the audio at the same time that it receives the bytes from the server (similarily to how YouTube loads your video while playing it).
Problem is, I can't read from a stream (in order to play it) while at the same time writing into it. Streams don't allow that.
I've been thinking how I can acheive this but I'm not sure. I searched online but found nothing that solves this problem. Would appreciate advice. What's a preferably simple and good way to approach this problem?

You need a buffer-file where you can read and write the data (you wont get your data in that speed which you want to play it).
Then you have to lock the Stream when you read data (buffering for example 200kb) so your streamwriter have to wait.
after that you have to lock the stream to write data from server to file.
edit:
Here is an example what I mean:
class Program
{
static FileStream stream;
static void Main(string[] args)
{
stream = File.Open(#"C:\test", FileMode.OpenOrCreate);
StreamWriterThread();
StreamReaderThread();
Console.ReadKey();
}
static void StreamReaderThread()
{
ThreadPool.QueueUserWorkItem(delegate
{
int position = 0; //Hold the position of reading from the stream
while (true)
{
lock (stream)
{
byte[] buffer = new byte[1024];
stream.Position = position;
position += stream.Read(buffer, 0, buffer.Length); //Add the read bytes to the position
string s = Encoding.UTF8.GetString(buffer);
}
Thread.Sleep(150);
}
});
}
static void StreamWriterThread()
{
ThreadPool.QueueUserWorkItem(delegate
{
int i = 33; //Only for example
int position = 0; //Holds the position of writing into the stream
while (true)
{
lock (stream)
{
byte[] buffer = Encoding.UTF8.GetBytes(new String((char)(i++),1024));
stream.Position = position;
stream.Write(buffer, 0, buffer.Length);
position += buffer.Length;
}
i%=125;//Only for example
if (i == 0) i = 33;//Only for example
Thread.Sleep(100);
}
});
}
}

Load File from SDCard

I integrated (this)EPUB Reader reader to my project. It is working fine. & I want to load the file from SDCard instead of Isolated storage of device
To open file from Isolated storage we have IsolatedStorageFileStream like this
IsolatedStorageFileStream isfs;
using (IsolatedStorageFile isf = IsolatedStorageFile.GetUserStoreForApplication())
{
try
{
isfs = isf.OpenFile([Path to file], FileMode.Open);
}
catch
{
return;
}
}
ePubView.Source = isfs;
For file in SDcard I tried like this
ExternalStorageDevice sdCard = (await ExternalStorage.GetExternalStorageDevicesAsync()).FirstOrDefault();
// If the SD card is present, get the route from the SD card.
if (sdCard != null)
{
ExternalStorageFile file = await sdCard.GetFileAsync(_sdFilePath);
// _sdFilePath is string that having file path of file in SDCard
// Create a stream for the route.
Stream file = await file.OpenForReadAsync();
// Read the route data.
ePubView.Source = file;
}
Here I am getting exception System.IO.EndOfStreamException
If You want try.. Here is my project sample link
Question : How can I give my file as source to epubView control
Is this is proper way, please give a suggestion regarding this..
Thanks

Although I've not tried your approach, and I cannot say exactly where is an error (maybe file from SD is read async and thus you get EndOfStream, and please keep in mind that as it is said at EPUB Reader Site - it's under heavy developement). Check if after copying the file to ISolatedStorage, you will be able to use it. I would try in this case first copying from SD to Memory stream like this:
ExternalStorageDevice sdCard = (await ExternalStorage.GetExternalStorageDevicesAsync()).FirstOrDefault();
if (sdCard != null)
{
MemoryStream newStream = new MemoryStream();
using (ExternalStorageFile file = await sdCard.GetFileAsync(_sdFilePath))
using (Stream SDfile = await file.OpenForReadAsync())
newStream = await ReadToMemory(SDfile);
ePubView.Source = newStream;
}
And ReadToMemory:
private async Task<MemoryStream> ReadToMemory(Stream streamToRead)
{
MemoryStream targetStream = new MemoryStream();
const int BUFFER_SIZE = 1024;
byte[] buf = new byte[BUFFER_SIZE];
int bytesread = 0;
while ((bytesread = await streamToRead.ReadAsync(buf, 0, BUFFER_SIZE)) > 0)
{
targetStream.Write(buf, 0, bytesread);
}
return targetStream;
}
Maybe it will help.

There's a bug with the stream returned from ExternalStorageFile. There's two options to get around it...
If the file is small then you can simply copy the stream to a MemoryStream:
Stream s = await file.OpenForReadAsync();
MemoryStream ms = new MemoryStream();
s.CopyTo(ms);
However, if the file is too large you'll run in to memory issues so the following stream wrapper class can be used to correct Microsoft's bug (though in future versions of Windows Phone you'll need to disable this fix once the bug has been fixed):
using System;
using System.IO;
namespace WindowsPhoneBugFix
{
/// <summary>
/// Stream wrapper to circumnavigate buggy Stream reading of stream returned by ExternalStorageFile.OpenForReadAsync()
/// </summary>
public sealed class ExternalStorageFileWrapper : Stream
{
private Stream _stream; // Underlying stream
public ExternalStorageFileWrapper(Stream stream)
{
if (stream == null)
throw new ArgumentNullException("stream");
_stream = stream;
}
// Workaround described here - http://stackoverflow.com/a/21538189/250254
public override long Seek(long offset, SeekOrigin origin)
{
ulong uoffset = (ulong)offset;
ulong fix = ((uoffset & 0xffffffffL) << 32) | ((uoffset & 0xffffffff00000000L) >> 32);
return _stream.Seek((long)fix, origin);
}
public override bool CanRead
{
get { return _stream.CanRead; }
}
public override bool CanSeek
{
get { return _stream.CanSeek; }
}
public override bool CanWrite
{
get { return _stream.CanWrite; }
}
public override void Flush()
{
_stream.Flush();
}
public override long Length
{
get { return _stream.Length; }
}
public override long Position
{
get
{
return _stream.Position;
}
set
{
_stream.Position = value;
}
}
public override int Read(byte[] buffer, int offset, int count)
{
return _stream.Read(buffer, offset, count);
}
public override void SetLength(long value)
{
_stream.SetLength(value);
}
public override void Write(byte[] buffer, int offset, int count)
{
_stream.Write(buffer, offset, count);
}
}
}
Code is available here to drop in to your project:
https://github.com/gavinharriss/ExternalStorageFileWrapper-wp8
Example of use:
ExternalStorageFile file = await device.GetFileAsync(filename); // device is an instance of ExternalStorageDevice
Stream streamOriginal = await file.OpenForReadAsync();
ExternalStorageFileWrapper streamToUse = new ExternalStorageFileWrapper(streamOriginal);

Streaming MP3 to Android from C# server

I have set up a C# server which at present serves up one test mp3 file over TCP. The code to send the file is as follows
public void RunStreamer()
{
log.MakeLog("Starting streamer");
Run = true;
//Need to look into how to thread this to not block the main window
TcpListener listen = new TcpListener(localAddr, _port);
listen.Start(); //startlistening to client requests
//blocks until a client request comes in
for (; ; )
{
Socket socket = listen.AcceptSocket();
if (socket.Connected)
{
SendFileToClient(socket);
socket.Disconnect(false);
}
}
}
void SendFileToClient(Socket socket)
{
log.MakeLog("Connection made");
NetworkStream netStream = new NetworkStream(socket);
StreamWriter writer = new StreamWriter(netStream);
//Todo - set specfified file - this file just for test
FileStream filestream = File.Open(#"C:\MusicTest\Test.mp3", FileMode.Open, FileAccess.Read, FileShare.Read);
filestream.CopyTo(netStream);
netStream.Flush();
netStream.Close();
}
In my test android set up I am making a call to the server on a button click:
public void btngo_click(View v)
{
final TcpClient client = new TcpClient();
new Thread(new Runnable(){
#Override
public void run() {
final MediaPlayer mediaPlayer = new MediaPlayer();
client.GetStream();
runOnUiThread(new Runnable(){
public void run()
{
int length = client.GetLength();
if(length > 0)
{
byte[] result = client.GetResult();
try {
// create temp file that will hold byte array
File tempMp3 = File.createTempFile("test", "mp3", getCacheDir());
tempMp3.deleteOnExit();
FileOutputStream fos = new FileOutputStream(tempMp3);
fos.write(result);
fos.close();
mediaPlayer.reset();
FileInputStream fis = new FileInputStream(tempMp3);
mediaPlayer.setDataSource(fis.getFD());
mediaPlayer.prepare();
mediaPlayer.start();
} catch (IOException ex) {
String s = ex.toString();
ex.printStackTrace();
}
}
}
});
}
}).start();
}
the stream is received in the TcpClient class which is as follows:
public class TcpClient {
public final static String SERVER_ADDRESS = "127.0.0.1";
public final static int SERVER_PORT = 65000;
public String TotalResult;
public int Length;
byte[] result = new byte[21000000];
public TcpClient()
{
}
public int GetLength()
{
return Length;
}
public byte[] GetResult()
{
return result;
}
public void GetStream()
{
try
{
final Socket socket = new Socket("192.0.0.5", 85000);
final InputStream input = new BufferedInputStream(socket.getInputStream());
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nread;
while((nread = input.read(result, 0, result.length)) != -1)
{
buffer.write(result, 0, nread);
}
buffer.flush();
//input.read(result);
Length = result.length;
input.close();
socket.close();
} catch (UnknownHostException e) {
String exc = e.getMessage();
e.printStackTrace();
} catch (IOException e) {
String exc2 = e.getMessage();
e.printStackTrace();
}
}
}
With apologies for all the code here is my problem.
I am receiving the stream. The temp MP3 file is created and the media player starts. I then only get a short snippet of the test MP3 file (which is a full song). It also jumps about a bit. The length is not the same and the section of the song played is different each time.
How do I receive the full file in a ordered way such that it will provide full play back of the song.
I have tried to route around for this and have an idea that I need to tell my client what file size it should suspect and then perform some loop until all data is received although I have no idea how to successfully implement this if that is the correct solution.
Any pointers on where I am going wrong or what I can do to rectify would be greatly appreciated!!

Having received no answers on this I dug around a bit more. Two things were wrong:
Firstly I had not included the size of the stream as a int sized header in my stream. I understand that for smaller files this will not be a problem but as file sizes grow it is necessary to make sure that the whole stream has been received.
This in turn raised another issue. The int I was sending as byte[] form c# was not returning the correct value in Java. Turns out Java uses sbytes -128 to 127 range as opposed to byte. This then involved a bit of code to convert to an int. then I could instruct the reader to readfully passing in the byte[] buffer with the actual size of the expected stream = voila it worked. MP3 files is received and plays just fine.

How to join 2 or more .WAV files together programmatically?

I need the ability to join 2 or more .wav files together in to one .wav file. I must do this programmatically, using C# (3rd-party products are not an option). I know of the System.Media.SoundPlayer class, but I am not looking to play the the .wav, but only to create it.

Here's a basic WAV concatenation function built using NAudio. This will ensure that only the data chunks are concatenated (unlike the code example in this CodeProject article linked in another answer). It will also protect you against concatenating WAV files that do not share the same format.
public static void Concatenate(string outputFile, IEnumerable<string> sourceFiles)
{
byte[] buffer = new byte[1024];
WaveFileWriter waveFileWriter = null;
try
{
foreach (string sourceFile in sourceFiles)
{
using (WaveFileReader reader = new WaveFileReader(sourceFile))
{
if (waveFileWriter == null)
{
// first time in create new Writer
waveFileWriter = new WaveFileWriter(outputFile, reader.WaveFormat);
}
else
{
if (!reader.WaveFormat.Equals(waveFileWriter.WaveFormat))
{
throw new InvalidOperationException("Can't concatenate WAV Files that don't share the same format");
}
}
int read;
while ((read = reader.Read(buffer, 0, buffer.Length)) > 0)
{
waveFileWriter.WriteData(buffer, 0, read);
}
}
}
}
finally
{
if (waveFileWriter != null)
{
waveFileWriter.Dispose();
}
}
}

One comment on Mark's answer:
The == operator does not seem to work for me when comparing wave formats. It's safer to do this:
if (!reader.WaveFormat.Equals(waveFileWriter.WaveFormat))
Alternatively, you could wrap the reader in a WaveFormatConversionStream and get rid of the format check altogether (not sure if it will work on all scenarios but I was able to succesfully test it).

Check out this codeproject example, seems to be exactly what you need with a good explanation of how to do it too:
Concatenating Wave Files Using C# 2005
It seems to comprise essentially of extracting and merging the sound data from all the wav files into one chunk of data with a new file header on top
EDIT: I have no experience of using this, nor am I an expert. I just came across this article and thought it may be useful. See Mark Heath's answer for a better solution

Use from How to join .Wav files together
private void JoinWav()
{
string[] files = new string[] { "1990764-ENG-CONSEC-RESPONSE7.WAV","1990764-ND_A.WAV", "1990764-SIGHT-SP.WAV",
"1990764-SP-CONSEC-RESPONSE6.WAV","1990764-VOCABWORD-004-12-SP.WAV","bi-consec-1-successful.wav",
"bi-transition-instruct.wav","nd_B.wav","sightreceived_B.wav","teststamp_A.wav" };
AudioCompressionManager.Join("res.wav", files);
}

If you need get only byte array, to insert in database or somebody else. You may use memory stream:
public static byte[] Concatenate(IEnumerable<byte[]> sourceData)
{
var buffer = new byte[1024 * 4];
WaveFileWriter waveFileWriter = null;
using (var output = new MemoryStream())
{
try
{
foreach (var binaryData in sourceData)
{
using (var audioStream = new MemoryStream(binaryData))
{
using (WaveFileReader reader = new WaveFileReader(audioStream))
{
if (waveFileWriter == null)
waveFileWriter = new WaveFileWriter(output, reader.WaveFormat);
else
AssertWaveFormat(reader, waveFileWriter);
WaveStreamWrite(reader, waveFileWriter, buffer);
}
}
}
waveFileWriter.Flush();
return output.ToArray();
}
finally
{
waveFileWriter?.Dispose();
}
}
}
private static void AssertWaveFormat(WaveFileReader reader, WaveFileWriter writer)
{
if (!reader.WaveFormat.Equals(writer.WaveFormat))
{
throw new InvalidOperationException("Can't concatenate WAV Files that don't share the same format");
}
}
private static void WaveStreamWrite(WaveFileReader reader, WaveFileWriter writer, byte[] buffer)
{
int read;
while ((read = reader.Read(buffer, 0, buffer.Length)) > 0)
{
writer.Write(buffer, 0, read);
}
}