Background:
I have a PHP server hosting a RESTful API that returns an image when a function /api/GenerateImage is called. I need the fastest and lowest memory using method to achieve this.
Ideas:
I can think of a few ways of achieving this:
Write it entirely in PHP - Image requires a lot of byte manipulation and it feels bad to do this in PHP.
Write a continually running C# program and use an API to generate the image and copy the response stream to the client via PHP - Need to avoid overhead of unnecessary socket connections.
Write a C/C++ command line program that, when called, generates and stores the image in a file, which PHP reads and sends - Need to avoid overhead due to starting a program, writing to a file and then reading to it.
Write it in C/C++ and use a PHP library wrapper to call the function directly from PHP - Seems to be the fastest, but most difficult for a single function.
The two most obvious solutions to me seem absent from your set of options:
Use PHP's GD extension.
Use imagemagick.
Whether these are practical options depends on what exactly you have to do with the image. If they are just static images you don't need anything, just serve them up as files from your web server. The fact you need some code to generate the image implies that the image is dynamically generated.
If neither of these suit, then I would say: your last (4th) solution would be the fastest, but a modified version of your 3rd solution would be simplest: write a command-line program that returns the image data to stdout, and call that from PHP.
Related
In my laravel application, I want to provide the users with the opportunity to download a copy of their stored data in the form of a Word document. I found that certain parts of this can only be accomplished using C#/.NET.
For this, I wrote a C# application alongside a method called GetWordProfile(User user) which returns FileInfo set to the actual path of the output file (this is always within the storage folder of laravel, so laravel has access to it). I only need the path and everything's done and dusted since from this point on, I can manage my laravel application to download this for the user.
However, the question is how do I get there? I must not forget about potential errors which may occur and thus display them (the errors are (inside my C# application) handled by log4net in a file as well as on the console; same goes for all output).
I tried to run my application using shell_exec respectively exec, however, both only returned zero results (null) (despite having set $output for exec) and thus seem not to be suitable. Also, I usually don't want loops (inside PHP/laravel) too much since you're then using a lot of computing power which is unnecessary for this sort of task, also you don't want to let your users wait more than, say, 5 secs, seeing nothing in your browser but the script being executed within a blank page (during the execution there's no content, obviously).
EDIT: I also approached the use of COM which ultimately did not work properly out either.
What is an appropriate approach towards this?
I did something similar with Python + C# a while back using IPC (Inter-process Communication) using named pipes.
EDIT: URL is broken. Here's the question someone asked previously on this topic.
Interprocess Communication using Named Pipes in C# + PHP
I've ran into a bit of a stupid problem today:
In my project I have to use a library (that I can't replace), he problem is that I'm using MemoryStream instead of frequently saving to the HDD (because there are many files, and they are small in size, so it's perfect for MemoryStream). The problem is that the library API is built around filesystem access - and one of the functions accepts only direct path to file.
How can I still send a string (path) to the method, which makes a new FileStream without actually touch the hard-drive?
For example "\MEMORY\myfile.bin"?
Well - that's thought.
Basically, you have three possible solutions:
You can use a reflector to modify the library given.
You can inspect the appropriate method, and then, by using some reflection magic you might be able to modify the object at runtime (very un-recommended)
You can play around with system calls and API - and by going into low-level ring0 assembly modify kernal.dll to referrer I/O queries from your path to the memory. (maybe that's possible without ring0 access - I am not sure).
Obviously, the most recommended is to use a reflector to modify the library given. otherwise, I can't see a solution for you.
In respond to the first comment, you can:
use RAMDrive (a program which allocates small chunks of the system memory and show it as partition)
If the file must exist on the disk (and only disk paths are accepted), then the main option is a virtual filesystem which lets you expose custom data as a filesystem. There exist several options, such as now-dead Dokan, our Solid File System OS Edition and Callback File System (see description of our Virtual Storage product line) and maybe Pismo File Mount would work (never looked at it closely).
It all depends on how the library is constructed.
If it's a 100% managed library that uses a FileStream, you are probably stuck.
If it takes the provided filename and call a native WIN32 CreateFile function, it's possible to give it something else than a file such as a named pipe.
To test quickly if it's possible, pass #"\\.\pipe\random_name" to the method: if it responds by saying explicitely that it can't open pipes and filenames begining with \\.\, well, sorry. ON the other hand, if it says it can't find the file, you have a chance to make it work.
You can then create a NamedPipeServerStream and use the same name for your library method call prepended with \\.\pipe\.
You can't "represent" it as a file, but you could "convert" it to a file using a StreamWriter class.
we have two applications, a native C++ application and a managed C#/WPF UI that creates input for, executes & reads output generated by the native application. Currently, communication between the two is done with XML & plain txt files.
However, the amount of output data is quite large & we are looking for a better approach to this. A solution that uses something like Memorystream would be ideal because it would easy to switch the output generation from a filestream to a memorystream.
However, how does one bridge the gap between managed & unmanaged? What is the most efficient way to do this?
Note: Many of the questions related to this are about a function call from a managed to an unmaged dll. These are two separate applications running independently. The UI spawns the native application, this is the only link between the two.
Thanks
You could try a named pipe
http://www.switchonthecode.com/tutorials/interprocess-communication-using-named-pipes-in-csharp
it depends on the way you produce/consume data
named pipes or socket - constant stream of data
shared memory - continuous updated data
Why not use standard input/output? Your C++ program can write to stdout using normal "printf" or "cout" commands.
Your .NET app can then capture this data using http://msdn.microsoft.com/en-us/library/system.diagnostics.process.standardoutput.aspx, for example. Same goes for standard input: use stdin to send commands to the C++ program.
Using named pipes is nice and might be the answer if you need more than two streams. But it would probably be more work on the C++ end of things.
This is likely a long shot, but I thought I'd ask anyway. I'm using a document management system's API. They provide a "WriteFile" method to save a given document to disk. However, the library does not have a way to simply read a document into memory. My only option, it seems, is to write to disk, then read it back in again. I'm wondering if there is a better way to work around this obvious limitation.
The method takes a string for the resulting file path. Method signature:
void ImageInfo.WriteFile(string Filename);
Theoretically, it is possible to intercept the WriteFile win32 API calls of any process, be it .NET, C++, etc using something called as Import Address Table Hooking which actually is a valuable tool in software testing on windows.
Basically you could overwrite the WriteFile,kernel32.dll entry in the Import Address Table to point to your method and then intercept the bytes which are attempted to be written.
There are probably other ways in layers above, like in .NET where you could possibly change the ILASM code of the 3rd party app dll. Or have your own version of some of the .NET dlls which replace some of the standard .NET classes.
Practically, it might not really be worth it, for e.g. If the API does not explicitly flush the file to disk, your subsequent reads might end up coming from the OS file cache and won't be that big a perf problem. You could probably achieve this by creating the file and keeping it open before calling WriteFile (just a guess).
Of course, I suppose you have profiled and measured it already.
You'd need a Windows API hooking library that can call a managed code callback. Easyhook is one such library. Beware that you might out that you haven't gained anything after you're done, the file system cache already provides direct memory access to file data.
It sounds like the API does not provide the reading part because they can't provide a better (more performant) manner than what is already available in the .NET framework.
I have a "simple" task. I have an existing project with a web service written in C# which has a method that will send a huge XML file to the client. (This is a backup file of data stored on the server that needs to be sent somewhere else.) This service also had some additional authentication/authorization set up.
And I have an existing Delphi 2007 application for WIN32 which calls the web service to extract the XML data for further processing. It's a legacy system that runs without a .NET installation.
Only problem: the XML file is huge (at least 5 MB) and needs to be sent as a whole. Due to system requirements I cannot just split this up into multiple parts. And I'm not allowed to make major changes to either the C# or the Delphi code. (I can only change the method call on both client and server.) And I'm not allowed to spend more than 8 (work) hours to come up with a better solution or else things will just stay unchanged.
The modification I want to add is to compress the XML data (which reduces it to about 100 KB) and then send it to the client as a binary stream. The Delphi code should then accept this incoming stream and de compress the XML data again. Now, with a minimum of changes to the existing code, how should this be done?
(And yes, I wrote the original client and server in the past and it was never meant to send that much data at once. Unfortunately, the developer who took it over from me had other ideas, made several dumb changes, did more damage and left the company before my steel-tipped boot could connect to his behind so now I need to fix a few things. Fixing this web service has a very low priority compared to the other damage that needs to be restored.)
The server code is based on legacy ASMX stuff, the client code is the result of the Delphi SOAP import with some additional modifications.
The XML is a daily update for the 3000+ users which happens to be huge in it's current design. We're working on this but that takes time. There are more important items that need to be fixed first, but as I said, there's a small amount of time available to fix this problem quickly.
This sounds like a good candidate for an HttpHandler
My good links are on my work computer (I'll add them when I get to work), but you can look to see if it will be a good fit.
-- edit --
Here are the links...
http://www.ddj.com/windows/184416694
http://visualstudiomagazine.com/articles/2006/08/01/create-dedicated-service-handlers.aspx?sc_lang=en&sc_mode=edit
What is the problem with a 5MB file in a soap message? I have written a document server that runs over soap and this server has no problem with large files.
If the size is a problem for you I would just compress and decompress the xml data. This can easily be done with one of the many (free) available components for compression of a TStream descendant.
If you get that kind of compression, merely convert each byte to its hex equivalent, which will only double the size, and send this. Then do the opposite on the other end. Or am I missing something?
I would agree with Brad Bruce, HttpHandler would be fast, and using GZIP or Deflate Compression with I might be wrong... browsers support natively. you can get easy great compression on text based data for cheap cpu time.
System.IO.Compression.GZipStream GZipStream = new System.IO.Compression.GZipStream("Your XML Doc Stream",System.IO.Compression.CompressionMode.Compress)