How to read text from 'simple' screenshot fast and effectively? - c#

I'm working on a small personal application that should read some text (2 sentences at most) from a really simple Android screenshot. The text is always the same size, same font, and in approx. the same location. The background is very plain, usually a few shades of 1 color (think like bright orange fading into a little darker orange). I'm trying to figure out what would be the best way (and most importantly, the fastest way) to do this.
My first attempt involved the IronOcr C# library, and to be fair, it worked quite well! But I've noticed a few issues with it:
It's not 100% accurate
Despite having a community/trial version, it sometimes throws exceptions telling you to get a license
It takes ~400ms to read a ~600x300 pixel image, which in the case of my simple image, I consider to be rather long
As strange as it sounds, I have a feeling that libraries like IronOcr and Tesseract may just be too advanced for my needs. To improve speeds I have even written a piece of code to "treshold" my image first, making it completely black and white.
My current IronOcr settings look like this:
ImageReader = new AdvancedOcr()
{
CleanBackgroundNoise = false,
EnhanceContrast = false,
EnhanceResolution = false,
Strategy = AdvancedOcr.OcrStrategy.Fast,
ColorSpace = AdvancedOcr.OcrColorSpace.GrayScale,
DetectWhiteTextOnDarkBackgrounds = true,
InputImageType = AdvancedOcr.InputTypes.Snippet,
RotateAndStraighten = false,
ReadBarCodes = false,
ColorDepth = 1
};
And I could totally live with the results I've been getting using IronOcr, but the licensing exceptions ruin it. I also don't have $399 USD to spend on a private hobby project that won't even leave my own PC :(
But my main goal with this question is to find a better, faster or more efficient way to do this. It doesn't necessarily have to be an existing library, I'd be more than willing to make my own kind of letter-detection code that would work (only?) for screenshots like mine if someone can point me in the right direction.

I have researched about this topic and the best solution which I could find is Azure cognitive services. You can use Computer vision API to read text from an image. Here is the complete document.

How fast does it have to be?
If you are using C# I recommend the Google Cloud Vision API. You pay per request but the first 1000 per month are free (check pricing here). However, it does require a web request but I find it to be very quick
using Google.Cloud.Vision.V1;
using System;
namespace GoogleCloudSamples
{
public class QuickStart
{
public static void Main(string[] args)
{
// Instantiates a client
var client = ImageAnnotatorClient.Create();
// Load the image file into memory
var image = Image.FromFile("wakeupcat.jpg");
// Performs label detection on the image file
var response = client.DetectText(image);
foreach (var annotation in response)
{
if (annotation.Description != null)
Console.WriteLine(annotation.Description);
}
}
}
}
I find it works well for pictures and scanned documents so it should work perfectly for your situation. The SDK is also available in other languages too like Java, Python, and Node

Related

Windows named pipe in node js (preferred shared memory)

I am using named pipe to share some data between 2 processes in windows. One is a node process and other is a C# process. Here is a sample of code I use in my node process:
var net = require('net');
var PIPE_NAME = "mypipe";
var PIPE_PATH = "\\\\.\\pipe\\" + PIPE_NAME;
var L = console.log;
var server = net.createServer(function(stream) {
L('Server: on connection')
stream.on('data', function(c) {
L('Server: on data:', c.toString());
});
stream.on('end', function() {
L('Server: on end')
server.close();
});
stream.write('Take it easy!');
});
server.on('close',function(){
L('Server: on close');
})
server.listen(PIPE_PATH,function(){
L('Server: on listening');
})
I use a NamedPipeClientStream in c# to read the data. I do this in a loop on both the sides, such as my node process is a producer and C# process is a consumer.
This works fine.
But sometimes the C# loop hangs and at that point in my node process I want to overwrite the new data over the old data. I was wondering if I can specify some max size in my pipe (the one I create in nodejs) or a timeout for the data but couldn't find such things in standard documentation.
If it cannot be solved this way, there is a shared memory route to solve the problem but I couldn't find any stable shared memory library for nodejs which works nicely on windows (and I don't have much time to write one right now). I need some pointers to move in the right direction.
Any advice is appreciated. Thanks.
EDIT: I would really want to implement the above stuff using shared memory since I need to share large amount of data at a fast rate and I need to tweak for performance. Any pointers on how to implement it?
I figured out a way to use the drain event in writable stream of nodejs as per my requirement.

Fellow Oak DICOM - changing image window level

I am not an experienced programmer, just need to add a DICOM viewer to my VS2010 project. I can display the image in Windows Forms, however can't figure out how to change the window center and width. Here is my script:
DicomImage image = new DicomImage(_filename);
int maxV = image.NumberOfFrames;
sbSlice.Maximum = maxV - 1;
image.WindowCenter = 7.0;
double wc = image.WindowCenter;
double ww = image.WindowWidth;
Image result = image.RenderImage(0);
DisplayImage(result);
It did not work. I don't know if this is the right approach.
The DicomImage class was not created with the intention of it being used to implement an image viewer. It was created to render preview images in the DICOM Dump utility and to test the image compression/decompression codecs. Maybe it was a mistake to include it in the library at all?
It is difficult for me to find fault in the code as being buggy when it is being used for something far beyond its intended functionality.
That said, I have taken some time to modify the code so that the WindowCenter/WindowWidth properties apply to the rendered image. You can find these modifications in the Git repo.
var img = new DicomImage(fileName);
img.WindowCenter = 2048.0;
img.WindowWidth = 4096.0;
DisplayImage(img.RenderImage(0));
I looked at the code and it looked extremely buggy. https://github.com/rcd/fo-dicom/blob/master/DICOM/Imaging/DicomImage.cs
In the current buggy implementation setting the WindowCenter or WindowWidth properties has no effect unless Dataset.Get(DicomTag.PhotometricInterpretation) is either Monochrome1 or Monochrome2 during Load(). This is already ridiculous, but it still cannot be used because the _renderOptions variable is only set in a single place and is immediately used for the _pipeline creation (not giving you chance to change it using the WindowCenter property). Your only chance is the grayscale _renderOptions initialization: _renderOptions = GrayscaleRenderOptions.FromDataset(Dataset);.
The current solution: Your dataset should have
DicomTag.WindowCenter set appropriately
DicomTag.WindowWidth != 0.0
DicomTag.PhotometricInterpretation == Monochrome1 or Monochrome2
The following code accomplishes that:
DicomDataset dataset = DicomFile.Open(fileName).Dataset;
//dataset.Set(DicomTag.WindowWidth, 200.0); //the WindowWidth must be non-zero
dataset.Add(DicomTag.WindowCenter, "100.0");
//dataset.Add(DicomTag.PhotometricInterpretation, "MONOCHROME1"); //ValueRepresentations tag is broken
dataset.Add(new DicomCodeString(DicomTag.PhotometricInterpretation, "MONOCHROME1"));
DicomImage image = new DicomImage(dataset);
image.RenderImage();
The best solution: Wait while this buggy library is fixed.

Webcams, C#, and a reliable wrapper

I'm in dire need of a replacement for a wrapper being used in a C# application. Basically, what we need to do is attach a webcam feed to one of two picture boxes. This will be used to take still images at the push of a button, which may detach the camera feed and attach the still image to that picture box, then reattach the camera feed later. We had previously found some free code to utilize with a CaptureDevice.cs file and Pinvoke.dll to tie it into avicap32.dll. Unfortunately, this seems to have random, intermittent errors that cannot be reliably reproduced. It's just too flaky. At some random point, one of those picture boxes may go black and won't show the feed until the picture is taken, at which point the proper picture is attached to the picture box. Then, even if there's only one webcam attached, it'll keep prompting for the webcam to be selected, something it wouldn't do otherwise.
Quite frankly, I'm surprised and dismayed that Microsoft hasn't included anything in .NET to cover webcam video feeds. I'm looking for something reliable and relatively simple to implement to replace this buggy webcam system.
It is time to use MediaCapture object. Use this sample for MS Windows 10 https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CameraStarterKit/
And read this article as well http://www.codepool.biz/csharp-camera-api-video-frame.html
May I suggest
http://www.emgu.com/wiki/index.php/Main_Page
I have used OpenCV in many C++ libraries and it seems to work very well for webcams from other things I've tried. Emgu is just a C# wrapper for OpenCV.
Here is a sample project to try it out on. It's very basic and simple, but should work right away.
http://dl.dropbox.com/u/18919663/vs%20samples/OpenCVCSharpTest.zip (just uploaded)
Sample:
using Emgu.CV;
using Emgu.CV.Structure;
...
public partial class Form1 : Form
{
public Capture cvWebCam;
public Form1()
{
InitializeComponent();
try
{
cvWebCam = new Capture();
timer1.Start();
}
catch
{
Console.WriteLine("Default camera not found or could not start");
}
}
private void timer1_Tick(object sender, EventArgs e)
{
if (cvWebCam != null)
using (Emgu.CV.Image<Bgr, byte> frame = cvWebCam.QueryFrame())
{
pictureBox1.BackgroundImage = frame.ToBitmap();
}
}
}
Try DirectShow.net- it's a free wrapper library to access DirectShow functionality from .NET:
http://directshownet.sourceforge.net
Its code samples contain a sample app for capturing video from webcams, too:
http://sourceforge.net/projects/directshownet/files/DirectShowSamples/

Alternatives to System.Drawing for use with ASP.NET?

After several days of tracking down bizarre GDI+ errors, I've stumbled across this little gem on MSDN:
Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions.
I don't know whether "ASP.NET service" means "web application" in this context, but "diminished service performance" certainly seems to cover the random assortment of "A generic error occurred in GDI+" and "Out of memory" errors that my app is throwing - intermittent, non-reproducible errors reading and writing JPEG images that - in many cases - were actually created by System.Drawing.Imaging in the first place.
So - if GDI+ can't read and write JPEG files reliably in a Web app, what should I be using instead?
I want users to be able to upload images (JPEG required, other formats nice-to-have), resample them reliably, and display useful error messages if anything goes wrong. Any ideas? Are the System.Media namespaces from WPF worth considering?
EDIT: Yeah, I know GDI+ works "most of the time". That's not good enough, because when it fails, it does so in a way that's impossible to isolate or recover from gracefully. I am not interested in examples of GDI+ code that works for you: I am looking for alternative libraries to use for image processing.
There is an excellent blog post including C# code about using the ImageMagick graphics library through Interop over at TopTen Software Blog. This post deals specifically with running ASP.net on linux under mono; however, the C# code should be perfectly copy-paste-able, the only thing you'll need to change is the Interop attributes if you are running under windows referencing a window binary (DLL).
ImageMagick® is a software suite to create, edit, compose, or convert
bitmap images. It can read and write images in a variety of formats
(over 100) including DPX, EXR, GIF, JPEG, JPEG-2000, PDF, PhotoCD,
PNG, Postscript, SVG, and TIFF. Use ImageMagick to resize, flip,
mirror, rotate, distort, shear and transform images, adjust image
colors, apply various special effects, or draw text, lines, polygons,
ellipses and Bézier curves.
There is also an ImageMagick .Net development project on codeplex that wraps up everything for you. But it doesn't show active development since 2009, so it may be lagging behind the current ImageMagick library version. For a small trivial resizing routine, I'd probably stick with the interop. You just need to watch your implementation carefully for your own memory leak or unreleased resources (the library itself is well tested and vetted by the community).
The library is free and open source. The Apache 2 license appears to be compatible with both personal and commercial purposes. See ImageMagick License Page.
The library is totally cross platform and implements many powerful image handling and transformation routines that are not found in GDI+ (or not implemented under mono) and has a good reputation as an alternative for ASP.net image processing.
Update: Looks like there is an updated version of a .NET wrapper here: http://magick.codeplex.com/
Yes, use the WPF System.Windows.Media classes. Being fully managed they don't suffer the same problems as the GDI stuff.
Here's an excerpt from some MVC code I use to render gradients, to give you an idea how to get from a WPF Visual to a PNG:
using System;
using System.IO;
using System.Web.Mvc;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
namespace MyMvcWebApp.Controllers
{
public class ImageGenController : Controller
{
// GET: ~/ImageGen/Gradient?color1=red&color2=pink
[OutputCache(CacheProfile = "Image")]
public ActionResult Gradient(Color color1, Color color2, int width = 1, int height = 30, double angle = 90)
{
var visual = new DrawingVisual();
using (DrawingContext dc = visual.RenderOpen())
{
Brush brush = new LinearGradientBrush(color1, color2, angle);
dc.DrawRectangle(brush, null, new Rect(0, 0, width, height));
}
return new FileStreamResult(renderPng(visual, width, height), "image/png");
}
static Stream renderPng(Visual visual, int width, int height)
{
var rtb = new RenderTargetBitmap(width, height, 96, 96, PixelFormats.Default);
rtb.Render(visual);
var frame = BitmapFrame.Create(rtb);
var encoder = new PngBitmapEncoder();
encoder.Frames.Add(frame);
var stream = new MemoryStream();
encoder.Save(stream);
stream.Position = 0;
return stream;
}
}
}
You can find a very good article from a Microsoft Employee here: Resizing images from the server using WPF/WIC instead of GDI+ that proposes to use WPF instead of GDI+. It's more about thumbnailing but it's overall the same issues.
Anyway, at the end it states this:
I contacted the WPF team to have the final word on whether this is
supported. Unfortunately, it's not, and the documentation is being
updated accordingly. I apologize about any confusion this may have
caused. We're looking at ways to make that story more acceptable in
the future.
So WPF is also unsupported in web apps and still is I believe :-S
ImageSharp
ImageSharp is an open-source cross-platform 2D graphics library. It's written in C# on top of the new .NET Standard, with no dependency on any OS-specific API.
It's currently still in pre-release on MyGet (you'll have to add the package source in the VS options or a NuGet.config file), but we are already using it with some very positive results.
Most of the issues I have read about pertain to resources not being disposed properly.
I have used variants of this code time and time again with no issues from web applications:
public void GenerateThumbNail(HttpPostedFile fil, string sPhysicalPath,
string sOrgFileName,string sThumbNailFileName,
System.Drawing.Imaging.ImageFormat oFormat, int rez)
{
try
{
System.Drawing.Image oImg = System.Drawing.Image.FromStream(fil.InputStream);
decimal pixtosubstract = 0;
decimal percentage;
//default
Size ThumbNailSizeToUse = new Size();
if (ThumbNailSize.Width < oImg.Size.Width || ThumbNailSize.Height < oImg.Size.Height)
{
if (oImg.Size.Width > oImg.Size.Height)
{
percentage = (((decimal)oImg.Size.Width - (decimal)ThumbNailSize.Width) / (decimal)oImg.Size.Width);
pixtosubstract = percentage * oImg.Size.Height;
ThumbNailSizeToUse.Width = ThumbNailSize.Width;
ThumbNailSizeToUse.Height = oImg.Size.Height - (int)pixtosubstract;
}
else
{
percentage = (((decimal)oImg.Size.Height - (decimal)ThumbNailSize.Height) / (decimal)oImg.Size.Height);
pixtosubstract = percentage * (decimal)oImg.Size.Width;
ThumbNailSizeToUse.Height = ThumbNailSize.Height;
ThumbNailSizeToUse.Width = oImg.Size.Width - (int)pixtosubstract;
}
}
else
{
ThumbNailSizeToUse.Width = oImg.Size.Width;
ThumbNailSizeToUse.Height = oImg.Size.Height;
}
Bitmap bmp = new Bitmap(ThumbNailSizeToUse.Width, ThumbNailSizeToUse.Height);
bmp.SetResolution(rez, rez);
System.Drawing.Image oThumbNail = bmp;
bmp = null;
Graphics oGraphic = Graphics.FromImage(oThumbNail);
oGraphic.CompositingQuality = CompositingQuality.HighQuality;
oGraphic.SmoothingMode = SmoothingMode.HighQuality;
oGraphic.InterpolationMode = InterpolationMode.HighQualityBicubic;
Rectangle oRectangle = new Rectangle(0, 0, ThumbNailSizeToUse.Width, ThumbNailSizeToUse.Height);
oGraphic.DrawImage(oImg, oRectangle);
oThumbNail.Save(sPhysicalPath + sThumbNailFileName, oFormat);
oImg.Dispose();
}
catch (Exception ex)
{
Response.Write(ex.Message);
}
}
You may have a look at http://gd-sharp.sourceforge.net/ which is a wrapper for the GD library. I haven't tested it but it seems promising.
I've had good behavior from the Cairo library (http://www.cairographics.org) in an ASP.Net webserver environment. I actually moved to cairo from WPF due to WPF's poor memory usage model for web-based stuff.
WPF actually tends to run your worker process out of memory. None of the WPF objects implement IDisposable, and many of them reference unmanaged memory that's only freed via a finalizer. Heavy use of WPF (especially if your server is significantly CPU-taxed) will eventually run you out of memory because your finalizer queue gets saturated. When I was profiling my app, for instance, the finalization queue had upwards of 50,000 objects on it, many of them holding references to unmanaged memory. Cairo has behaved much better for me, and its memory usage pattern has been much more predictable than WPF's.
If you're interested in using cairo, grab the libs from GTK+'s website. They have an x86 as well as an x64 set of binaries.
The only downside is that cairo can't read/write JPG natively; however, you could easily adapt WPF's stuff for reading/writing JPG and do the resampling/scaling/drawing/whatever else using Cairo.
Aspose.Drawing is a drop-in replacement for System.Drawing that is fully managed and can be safely used in web applications. (I'm one of the developers.)

Extracting keyboard layouts from windows

OK, this is a slightly weird question.
We have a touch-screen application (i.e., no keyboard). When users need to enter text, the application shows virtual keyboard - hand-built in WinForms.
Making these things by hand for each new language is monkey work. I figure that windows must have this keyboard layout information hiding somewhere in some dll. Would there be anyway to get this information out of windows?
Other ideas welcome (I figure at least generating the thing from a xml file has got to be better than doing it by hand in VS).
(Note: having said all which, I note that there is a Japanese keyboard, state machine and all..., so XML might not be sufficient)
UPDATE: pretty good series on this subject (I believe) here
Microsoft Keyboard Layout Creator can load system keyboards and export them as .klc files. Since it’s written in .NET you can use Reflector to see how it does that, and use reflection to drive it. Here's a zip file of .klc files for the 187 keyboards in Windows 8 created using the below C# code. Note that I originally wrote this for Windows XP, and now with Windows 8 and the on-screen keyboard, it is really slow and seems to crash the taskbar :/ However, it does work :)
using System;
using System.Collections;
using System.IO;
using System.Reflection;
class KeyboardExtractor {
static Object InvokeNonPublicStaticMethod(Type t, String name,
Object[] args)
{
return t.GetMethod(name, BindingFlags.Static | BindingFlags.NonPublic)
.Invoke(null, args);
}
static void InvokeNonPublicInstanceMethod(Object o, String name,
Object[] args)
{
o.GetType().GetMethod(name, BindingFlags.Instance |
BindingFlags.NonPublic) .Invoke(o, args);
}
static Object GetNonPublicProperty(Object o, String propertyName) {
return o.GetType().GetField(propertyName,
BindingFlags.Instance | BindingFlags.NonPublic)
.GetValue(o);
}
static void SetNonPublicField(Object o, String propertyName, Object v) {
o.GetType().GetField(propertyName,
BindingFlags.Instance | BindingFlags.NonPublic)
.SetValue(o, v);
}
[STAThread] public static void Main() {
System.Console.WriteLine("Keyboard Extractor...");
KeyboardExtractor ke = new KeyboardExtractor();
ke.extractAll();
System.Console.WriteLine("Done.");
}
Assembly msklcAssembly;
Type utilitiesType;
Type keyboardType;
String baseDirectory;
public KeyboardExtractor() {
msklcAssembly = Assembly.LoadFile("C:\\Program Files\\Microsoft Keyboard Layout Creator 1.4\\MSKLC.exe");
utilitiesType = msklcAssembly.GetType("Microsoft.Globalization.Tools.KeyboardLayoutCreator.Utilities");
keyboardType = msklcAssembly.GetType("Microsoft.Globalization.Tools.KeyboardLayoutCreator.Keyboard");
baseDirectory = Directory.GetCurrentDirectory();
}
public void extractAll() {
DateTime startTime = DateTime.UtcNow;
SortedList keyboards = (SortedList)InvokeNonPublicStaticMethod(
utilitiesType, "KeyboardsOnMachine", new Object[] {false});
DateTime loopStartTime = DateTime.UtcNow;
int i = 0;
foreach (DictionaryEntry e in keyboards) {
i += 1;
Object k = e.Value;
String name = (String)GetNonPublicProperty(k, "m_stLayoutName");
String layoutHexString = ((UInt32)GetNonPublicProperty(k, "m_hkl"))
.ToString("X");
TimeSpan elapsed = DateTime.UtcNow - loopStartTime;
Double ticksRemaining = ((Double)elapsed.Ticks * keyboards.Count)
/ i - elapsed.Ticks;
TimeSpan remaining = new TimeSpan((Int64)ticksRemaining);
String msgTimeRemaining = "";
if (i > 1) {
// Trim milliseconds
remaining = new TimeSpan(remaining.Hours, remaining.Minutes,
remaining.Seconds);
msgTimeRemaining = String.Format(", about {0} remaining",
remaining);
}
System.Console.WriteLine(
"Saving {0} {1}, keyboard {2} of {3}{4}",
layoutHexString, name, i, keyboards.Count,
msgTimeRemaining);
SaveKeyboard(name, layoutHexString);
}
System.Console.WriteLine("{0} elapsed", DateTime.UtcNow - startTime);
}
private void SaveKeyboard(String name, String layoutHexString) {
Object k = keyboardType.GetConstructors(
BindingFlags.Instance | BindingFlags.NonPublic)[0]
.Invoke(new Object[] {
new String[] {"", layoutHexString},
false});
SetNonPublicField(k, "m_fSeenOrHeardAboutPropertiesDialog", true);
SetNonPublicField(k, "m_stKeyboardTextFileName",
String.Format("{0}\\{1} {2}.klc",
baseDirectory, layoutHexString, name));
InvokeNonPublicInstanceMethod(k, "mnuFileSave_Click",
new Object[] {new Object(), new EventArgs()});
((IDisposable)k).Dispose();
}
}
Basically, it gets a list of all the keyboards on the system, then for each one, loads it in MSKLC, sets the "Save As" filename, lies about whether it's already configured the custom keyboard properties, and then simulates a click on the File -> Save menu item.
It is a fairly well-known fact that MSKLC is unable to faithfully import & reproduce keyboard layouts for all of the .DLL files supplied by Windows–especially those in Windows 8 & above. And it doesn't do any good to know where those files are if you can't extract any meaningful or helpful information from them.
This is documented by Michael Kaplan on his blog (he was a developer of MSKLC) which I see you have linked to above.
When MSKLC encounters anything it does not understand, that portion is removed.
Extracting the layout using MSKLC will work for most keyboards, but there are a few–namely the Cherokee keyboard, and the Japanese & Korean keyboards (to name a few, I'm not sure how many more there are)–for which the extracted layout will NOT accurately or completely reflect the actual usage & features of the keyboard.
The Cherokee keyboard has chained dead keys which MSKLC doesn't support. And the far Eastern keyboards have modifier keys which MSKLC isn't aware of–that means entire layers/shift states which are missing!
Michael Kaplan supplies some code and unlocks some of the secrets of MSLKC and the accompanying software that can be used to get around some of these limitations but it requires a fair amount of doing things by hand–exactly what you're trying to avoid! Plus, Michael's objectives are aimed at creating keyboards with features that MSKLC can not create or understand, but which DO work in Windows (which is the opposite of what the OP is trying to accomplish).
I am sure that my solution comes too late to be of use to the OP, but perhaps it will be helpful in the future to someone in a similar situation. That is my hope and reason for posting this.
So far all I've done is explain that the other answers are insufficient. Even the best one will not and can not fully & accurately reproduce all of Windows' native keyboards and render them into KLC source files. This is really unfortunate and it is certainly not the fault of its author because that is a very clever piece of code/script! Thankfully the script & the source files (whose link may or may not still work) is useful & effective for the majority of Windows' keyboards, as well as any custom keyboards created by MSKLC.
The keyboards that have the advanced features that MSKLC doesn't support were created by the Windows DDK, but those features are not officially documented. Although one can learn quite a bit about their potential by studying the source files provided with MSKLC.
Sadly the only solution I can offer is 3rd party, paid software called KbdEdit. I believe it is the only currently available solution which is actually able to faithfully decode & recreate any of the Windows supplied keyboards–although there are a few advanced features which even it can not reproduce (such as keys combinations/hotkeys which perform special native language functions; for example: Ctrl+CapsLock to activate KanaLock (a Japanese modifier layer). KbdEdit DOES faithfully reproduce that modifier layer which MSKLC with strip away, it just doesn't support this alternate method of activating that shift state if you don't have a Japanese keyboard with a Kana lock key. Although, it will allow you to convert a key on your keyboard to a Kana key (perhaps Scroll Lock?).
Fortunately, none of those unsupported features are even applicable to an on-screen keyboard.
KbdEdit is a really powerful & amazing tool, and it has been worth every penny I paid for it! (And that's NOT something I would say about virtually any other paid software…)
Even though KbdEdit is 3rd party software, it is only needed to create the keyboards, not to use them. All of the keyboards it creates work natively on any Windows system without KbdEdit being installed.
It supports up to 15 modifier states and three addition modifier keys, one which is togglable–like CapsLock. It also supports chained dead keys, and remapping any of the keys on most any keyboard.
Why don't you use the on-screen keyboard (osk.exe)? Looks like you re-inventing the wheel. And not the easiest one!
I know where are these DLL files' path:
In your registry, you see:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layouts
where each branch has some value like "Layout File"="KBDSP.dll". The root directory is
C:\Windows\System32
and
C:\Windows\SystemWOW64
Those are all the keyboard layout files are located. For example, KBDUS.dll means "keyboard for US".
I tried to substitute the DLL file with my custom DLL made by MSKLC, and I found it loads the layout mapping images automatically in the "Language" - "input method" - "preview":
So we know that the mapping is there in the DLL.
Please check following Windows API
[DllImport("user32.dll")]
private static extern long LoadKeyboardLayout(string pwszKLID, uint Flags);
Check MSDN here

Categories