We Keep Coding

How to read text from 'simple' screenshot fast and effectively?

I'm working on a small personal application that should read some text (2 sentences at most) from a really simple Android screenshot. The text is always the same size, same font, and in approx. the same location. The background is very plain, usually a few shades of 1 color (think like bright orange fading into a little darker orange). I'm trying to figure out what would be the best way (and most importantly, the fastest way) to do this.
My first attempt involved the IronOcr C# library, and to be fair, it worked quite well! But I've noticed a few issues with it:
It's not 100% accurate
Despite having a community/trial version, it sometimes throws exceptions telling you to get a license
It takes ~400ms to read a ~600x300 pixel image, which in the case of my simple image, I consider to be rather long
As strange as it sounds, I have a feeling that libraries like IronOcr and Tesseract may just be too advanced for my needs. To improve speeds I have even written a piece of code to "treshold" my image first, making it completely black and white.
My current IronOcr settings look like this:
ImageReader = new AdvancedOcr()
{
CleanBackgroundNoise = false,
EnhanceContrast = false,
EnhanceResolution = false,
Strategy = AdvancedOcr.OcrStrategy.Fast,
ColorSpace = AdvancedOcr.OcrColorSpace.GrayScale,
DetectWhiteTextOnDarkBackgrounds = true,
InputImageType = AdvancedOcr.InputTypes.Snippet,
RotateAndStraighten = false,
ReadBarCodes = false,
ColorDepth = 1
};
And I could totally live with the results I've been getting using IronOcr, but the licensing exceptions ruin it. I also don't have $399 USD to spend on a private hobby project that won't even leave my own PC :(
But my main goal with this question is to find a better, faster or more efficient way to do this. It doesn't necessarily have to be an existing library, I'd be more than willing to make my own kind of letter-detection code that would work (only?) for screenshots like mine if someone can point me in the right direction.

I have researched about this topic and the best solution which I could find is Azure cognitive services. You can use Computer vision API to read text from an image. Here is the complete document.

How fast does it have to be?
If you are using C# I recommend the Google Cloud Vision API. You pay per request but the first 1000 per month are free (check pricing here). However, it does require a web request but I find it to be very quick
using Google.Cloud.Vision.V1;
using System;
namespace GoogleCloudSamples
{
public class QuickStart
{
public static void Main(string[] args)
{
// Instantiates a client
var client = ImageAnnotatorClient.Create();
// Load the image file into memory
var image = Image.FromFile("wakeupcat.jpg");
// Performs label detection on the image file
var response = client.DetectText(image);
foreach (var annotation in response)
{
if (annotation.Description != null)
Console.WriteLine(annotation.Description);
}
}
}
}
I find it works well for pictures and scanned documents so it should work perfectly for your situation. The SDK is also available in other languages too like Java, Python, and Node

How do I change PCap.Net packet data?

I have a (production) WireShark capture file that I need to 'replay' over my (debug) network. I can read and interpret the packets, but I need to tweak a few details before the packets can be sent, like source IP addresses and ports.
The problem, however, is that all data in the PcapDotNet.Packets.Packet is read-only. It just has setters.
So like setting for instance the Ethernet.IpV4.Source will not work.
IpV4Address.TryParse("192.168.1.10", out var newAddress); // for demo sake.
packet.Ethernet.IpV4.Source = newAddress; // Won't work
Is there a simple way to avoid building a new packet from scratch, or is that the only way to create a slightly different packet?

Instead of changing the packet inplace, you should just create a new one based on the old one.
You can use ExtractLayer() on each layer you want to keep from the old packet, and then change the layer properties if necessary.
In this case, you can do:
IpV4Layer ipV4Layer = packet.Ethernet.IpV4.ExtractLayer();
ipV4Layer.Source = newAddress;
Packet newPacket = PacketBuild.Build(DateTime.Now, packet.Ethernet.ExtractLayer(), ipV4Layer, packet.Ethernet.IpV4.Payload.ExtractLayer());
You probably also want to reset the IPv4 checksum, so you should do:
ipV4Layer.HeaderChecksum = null;
And you might need to something similar to the UDP or TCP layers on top of the IPv4 layer, in case you have them.

Windows named pipe in node js (preferred shared memory)

I am using named pipe to share some data between 2 processes in windows. One is a node process and other is a C# process. Here is a sample of code I use in my node process:
var net = require('net');
var PIPE_NAME = "mypipe";
var PIPE_PATH = "\\\\.\\pipe\\" + PIPE_NAME;
var L = console.log;
var server = net.createServer(function(stream) {
L('Server: on connection')
stream.on('data', function(c) {
L('Server: on data:', c.toString());
});
stream.on('end', function() {
L('Server: on end')
server.close();
});
stream.write('Take it easy!');
});
server.on('close',function(){
L('Server: on close');
})
server.listen(PIPE_PATH,function(){
L('Server: on listening');
})
I use a NamedPipeClientStream in c# to read the data. I do this in a loop on both the sides, such as my node process is a producer and C# process is a consumer.
This works fine.
But sometimes the C# loop hangs and at that point in my node process I want to overwrite the new data over the old data. I was wondering if I can specify some max size in my pipe (the one I create in nodejs) or a timeout for the data but couldn't find such things in standard documentation.
If it cannot be solved this way, there is a shared memory route to solve the problem but I couldn't find any stable shared memory library for nodejs which works nicely on windows (and I don't have much time to write one right now). I need some pointers to move in the right direction.
Any advice is appreciated. Thanks.
EDIT: I would really want to implement the above stuff using shared memory since I need to share large amount of data at a fast rate and I need to tweak for performance. Any pointers on how to implement it?

I figured out a way to use the drain event in writable stream of nodejs as per my requirement.

PcapDotNet and Datalink type

I'm trying to manipulate some network captures (pcap format) using Pcap.net.
I'm opening the pcap file and creating the dumper with:
OfflinePacketDevice selectedDevice = new OfflinePacketDevice(pcapInFile);
using (PacketCommunicator PcapReader = selectedDevice.Open(655360, PacketDeviceOpenAttributes.Promiscuous, 1000))
{
PacketDumpFile PcapWriter = PcapReader.OpenDump(pcapOutFile);
PcapReader.ReceivePackets(count, PacketDispatcher);
}
And the PacketDispatcher would be something like:
private void PacketDispatcher(Packet packet)
{
// Manipulate the packet
PcapWriter.Dump(packet);
}
Everything is ok as far as the pcapInFile Datalink is ethernet type. But i have several captures without ethernet layer (rawip) and i have to build a new ethernet layer. In this kind of caps the datalink type is the same as the pcapInFile (raw ip) and i want to change it to ethernet...
If i store all the manipulated packets in a ienumerable and dump them with:
PacketDumpFile.Dump(pcapOutFile, new PcapDataLink(1), Packets.Count(), Packets);
It works fine... But, this is not very useful if you are dealing with files of several gigas...
Any idea?
Thanks!

Assuming the concern is about having all packets in RAM, you can create an IEnumerable that doesn't contain everything in RAM using yield.
This way as Dumper dumps packets it will call your method to populate the next item using yield.

Well, despite this is not an answer but a workaround, and since nobody wrote a better solution this is how a fixed it:
Use SharPcap instead of PcapDotNet, then you can declare a reader and a writer this way:
public CaptureFileReaderDevice PcapReader;
public CaptureFileWriterDevice PcapWriter;
PcapReader = new CaptureFileReaderDevice(fileIn);
PcapReader.OnPacketArrival += packetDispatcher;
PcapReader.Capture();
In the packetDispatcher function:
RawCapture raw = new RawCapture(LinkLayers.Ethernet, e.Packet.Timeval,RebuildNullLinkLayer(e, offset));
CaptureHandler.PcapWriter.Write(raw);
And in the RebuildNullLinkLayer function you can add the ethernet layer, modify whatever you want, etc...
Note that when you call the RawCapture Constructor you can choose the Link Layer (mi original issue with Pcap.Net...), so if you are parsing a RawIp capture, you can convert the packets to Ethernet.

Playing (not recording/exporting) multiple MIDI tracks

I would like to have multiple MIDI instruments playing at once with NAudio. I've found instructions for how to play a single MIDI instrument, and I've found instructions for how to export multiple tracks in a single MidiEventCollection to a file. However, I can't seem to put these ideas together.
Here is some dumb example code I have that cycles through all my MIDI instruments, playing a major 3rd harmony for each one:
var midiOut = new MidiOut(0);
for (var i = 0; i <= 127; i++)
{
midiOut.Send(MidiMessage.ChangePatch(i, 1).RawData);
midiOut.Send(MidiMessage.StartNote(60, 127, 1).RawData);
midiOut.Send(MidiMessage.StartNote(64, 127, 1).RawData);
Thread.Sleep(500);
}
This works fine of course, but if I wanted that C andE to be played by different instruments, I'd be out of luck. I only have the one MIDI device and I can only have one connection to that open at a time, and MidiOut does not appear to support adding multiple tracks.
On the other hand, the MidiEventController code looks like it would be more or less what I want, but I only see examples of exporting that to a file, rather than actually playing the events. I put together something like this:
var events = new MidiEventCollection(1, 120);
var track = events.AddTrack();
var setInstrument = new PatchChangeEvent(0, 1, 66);
var play = new NoteOnEvent(0, 1, 60, 127, 1000);
track.Add(setInstrument);
track.Add(play);
But at this point I cannot figure out how to actually play the track, rather than export it.

If you want to play two different patches at the same time, this is what MIDI channels are for.
At your disposal are 16 channels, of which channel 10 is reserved for percussion if you're using a GM scheme.
In your first code snippet, you appear to be using only MIDI channel 1.
How about using more than one channel and loading different patches for each channel?