Why does my Blink based browser play hide and seek? - c#

We have a C# tool (that I wrote) that records online broadcasts taking place a custom written (that we wrote) flash app. (There are no DRM or copyright issues here.)
We've coded up a system whereby this tool is installed on a Windows Server 2012 R2 Amazon AWS instance. After we boot the instance, the tool loads, waits for the right time to start recording, launches a browser and passes the command line argument of the URL to access the broadcast. The browser will then load the flash app and the interview audio and video will start arriving at the browser instance on AWS
By way of a virtual audio cable driver, screen / audio capture directshow filters and ffmpeg a screen recording is taken. The C# tool calls ffmpeg and ffmpeg will record the screen reliably for the entire interview, then the tool shuts the whole thing down
The problem I'm having is that both Chrome and Electron browser sometimes simply don't draw themselves on the screen so all ffmpeg ends up recording is a blank desktop and the audio of the broadcast (hence, the browser IS running)
We found this out when recordings started turning up with X hours of merely recording the windows desktop and the tool's main window with a countdown timer.
A screenshotting facility was built into the tool and added to its web control interface, and this way we can test whether the browser is visible - a human looks at the screenshot of every broadcast, just after recording has started (the browser is supposed to be on show by this time)
We notice that 50% of the time, the browser isn't drawing itself on screen. By 50% I mean that every other recording that the AWS instance carries out, will be blank: AWS starts, records ok, shuts down. AWS starts again an hour later for a different broadcast, recording is blank, shuts down.. Starts/ok/shutdown. Starts/blank/shutdown. Repeat ad infinitum
What's even more strange is that if I run VNCviewer on my dev machine and connect up to an instance that is having a problem, the instant that the VNC connection is up and the remote desktop is showing on my screen, the browser suddenly appears as if nothing was ever wrong. A screenshot from before the VNC connect shows blank desktop, connect VNC, take another screenshot and the browser is there. All through it the audio is fine - the browser connected to the boadcast is fine, for sure
It's as though Chrome/Electron thinks "you know what, noone is looking at me so I'm not going to bother drawing myself". No screen saver is set, though the power plan has the setting "turn off the display after 15 minutes".
Perhaps Chrome/Electron have a test amounts to "if the display is off, don't draw". I can't explain the inconsistency though - the recorder launches at least 1 hour before it's needed, and sits there idle until it's time to start the browser. You'd hence imagine that the "power off the monitor after 15 mins" setting would reliably have ensured the "monitor" is "off" by the time every recording start comes around
This behaviour doesn't happen with any of the other browsers (but unfortunately the app doesn't and cannot work in them because it uses some weird chrome-only technology/API).
Can anyone suggest anything to look at to help debug this, or anything I can build into the C# tool to overcome the problem? Coding it up to connect to itself via VNC for a few seconds after it has launched the browser.. Well that just tastes nasty.
Naturally, as soon as I connect to the machine via VNC (rather than RDP - RDP isn't usable because the recording context is in a logged on session for a particular user) the problem goes away, which makes it frustratingly hard to debug.

I am not sure what exactly causes your problem, but it sounds like interacting wit the system prevents it. One way to interact with a system is to use the keyboard and this can be automated.
You could try sending a keystroke (like "F15") evey so many seconds in C# using
Windows Input Simulator or maybe SendKeys.Send and
Combine the above with some kind of Timer
Maybe take a quick peek at this app called Caffeine...it presses the "F15" key for you every so many seconds. They claim "F15" generally doesnt trigger anything in windows (since a release they made back in 2010).
Caffeine App

Related

Trouble Opening Finder Window and Accessing Hardware on MacOS

I have a fairly large project on Unity 2018.4.2 that is failing in two places when I build: connecting to the user's microphone and opening the Finder window. Opening Finder takes up to 30 seconds to actually open it and microphone hangs whenever I try to start it.
We use the code below to open the Finder Window
System.Diagnostics.Process p = System.Diagnostics.Process.Start("open", $"\"{macPath}\"");
It eventually opens the Finder window, but only after 10-30 seconds. Meanwhile, a MacOS process called "tccd" kicks in causing the CPU to spike.
For Microphone, we use Unity's built-in Microphone class and we get hangs that last for 15-50 seconds. Same thing happens where the "tccd" process kicks in and spikes our CPU.
When we try to connect to the microphone the first time the OS asks us once if we want to allow the app to access the microphone, but it always hangs even after confirming. It never asks us about accessing Finder.
It seems like the MacOS is actively trying to slow us down from accessing these, but not completely stopping us, as Finder eventually opens and the microphone eventually connects (sometimes). We do not have these problems on Windows, it's only on MacOS. Any thoughts as to what is happening?
Note: There are no problems in Editor, only in Mac builds
We found the problem. Our code had to be signed in order to use it on a Mac. Now that the app is signed, we have access to everything we need as listed in our entitlements file

How can you start up RDP and run Selenium Tests?

I've received an interesting requirement for my Selenium tests where I need my code to RDP into a server, open up a browser and then run my tests on that.
Initially I thought Browserstack or even Selenium Grid, but the requirement is to actually open the RDP session and run tests through that.
Is there a way to achieve this? I wanted to try using something like Microsoft UI Automation to open the RDP session but then my Selenium tests would just run locally after starting up RDP, right? I'm having some trouble getting starting on this and can't seem to find a good place to start.
The RDP window is basically an image of a desktop and Selenium cannot control the web browser through it. You would need to deploy your Selenium tests on the remote machine and run them from there.
This is theoretically possible, but not exactly in the way you are describing it. You seem to be saying that you want to interact with things inside an RDP windows as if the RDP window were a standard browser or native app, and things inside it were elements in a DOM. By running selenium on a host machine, and then opening up an RDP, and clicking within the RDP window from the original host machine.
It is impossible to connect like this (in a manner that this approach would require) directly over an RDP. If for no other reason than security. For example; if you have ever watched people on youtube infect VM's with crazy viruses to show off the destructive effect of the virus on a VM without hurting the computer they are RDP'ed into from. But there seem to be ways to communicate over TCP connections via RDP, which would facilitate other ways to interact with the remote machine.
Additionally, you could just SSH/Enter-PSSession into the machine, and set up a communication channel between the machines outside of the RDP session. If you had a custom built application on the machine being remoted into, you could use System32 libraries to tell the primary machine x y coordinates of things to click on. Then, from the primary machine, you can maximize the RDP window, and click on the provided X Y coordinates.
I suspect that this would successfully send the click all the way through down to the VM, in the manner you are suggesting. If Appium cannot send the click down in this manner, you may need to develop your own abstraction of the user32.dll to perform clicks using the core Windows logic for keyboards and mouse clicks.
All in all, this is an INSANELY in depth project that accomplishes (seemingly) nothing for the effort. I would push back on whomever is giving you this requirement, by investigating some of the things I mentioned above (along with any other advice), and explaining the cost versus the return on investment.
Hopefully, they will just let you use Selenium Grid to communicate with VM's or attached devices to push out applications and test them given industry standard approaches.
Here is an interesting and relevant read: https://superuser.com/questions/130552/tunneling-a-tcp-ip-connection-through-remote-desktop-connection

Is there a way to keep internet connections / sockets open after iOS forces app into sleep mode?

I'm developing a Xamarin app that uses location background modes to keep itself awake in the background, scan and report on iBeacons in its range over AMQP, and also make HTTPS requests to a log server.
This all works great, until a sustained period of time where the GPS location does not change and there are no changes to the radio frequency environment (i.e. no new iBeacons introduced, none taken away).
It appears that iOS puts the app into sleep mode (it's a slightly different sort of sleep mode to normal, as background modes are involved, so I don't see an OS callback for this). Moving the phone, or introducing new iBeacons wakes the phone up, but sometimes the app is no longer able to access the internet, using objects that were created before the app was sent to sleep. It appears as if the kernel is closing sockets.
Any ideas as to how I could diagnose this issue more effectively, or suggestions to how I can keep these sockets open, would be most appreciated.
Thank you for reading!

Autoconnect to MS Wireless display on Windows 10

I want to write a Windows service (in c#) or a powershell script that connects my laptop automatically (at boot or key combination) to my MS wireless display adapter for screen mirroring. In Windows 10 I can only do it manually by going to the notifications and click Connect>MS Wireless adapter>connect.
What I found is that there is a Miracast API but there is not much documentation on how to use it.
I also found this documentation on MiraDisp.dll and there are two functions OpenMiracastSession and CloseMiracastSession.
The problem is I don't know how to use these functions in c#. I know I will probably have to use pInvoke. Can anyone point me in the right direction?
First of all, thanks to #CodingGorilla for the suggestion on AutoHotkey. I've been playing around with that the past couple of days.
I went the AutoHotkey route as I couldn't find an easy place to start with any Windows 10 API. All kinds of documentation out there to push toast notifications but I couldn't find anything to control the action center. If anyone has suggestions on that front, please post them.
Here is what I came up with using AutoHotkey. Pretty simple but not an ideal solution as there are a few variables with this. Below is AutoHotkey script code I used to open the action center, click connect, then click the top-most listed wireless display:
Send #a ;Sends Windows button + A to open the action center
Sleep, 750 ; Give it some time to slide open
ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, C:\Connect.png ;Try to find the Connect button tile
if ErrorLevel = 2
MsgBox Could not conduct the search for the connect button in action center. Make sure your search image is in the correct location.
else if ErrorLevel = 1
MsgBox Connect button cannot be found on the screen.
else
MoveMouseAndClick(FoundX, FoundY)
Sleep, 1250 ;Delay so the wireless displays have a chance to load into the Action Center window
ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, C:\WirelessDisplay.png
if ErrorLevel = 2
MsgBox Could not conduct the search for the wireless display.
else if ErrorLevel = 1
{
;Search image cannot be found. Try 1 more time in case it took a long time for the wireless displays to appear
Sleep, 750
ImageSearch, FoundX, FoundY, 0, 0, A_ScreenWidth, A_ScreenHeight, C:\WirelessDisplay.png ;try to find the first Wireless Display device listed
if ErrorLevel = 1
MsgBox Wireless display image cannot be found on the screen. Make sure the wireless device is turned on.
else
MoveMouseAndClick(FoundX, FoundY)
}
else
MoveMouseAndClick(FoundX, FoundY)
Send {Esc} ;Send Esc to get rid of the Action Center window
Return
MoveMouseAndClick(x, y) {
MouseMove, x + 25, y + 25 ;Move it down the right a bit to make sure we click the button
Sleep, 250
MouseClick, left
}
I've also attached the images as an example of what I made. You will need to make your own search images. Before making those images, you must also turn off the transparency of the Action Center, start and taskbar in Windows 10 - Settings->Personalization->Colors->Make Start, taskbar, and action center transparent->Off. It is especially important to redo the 2nd image as mine image lists "Roku Stick" within the image. I had to redo my search image between my desktop development machine and the MS Surface 3 I'm running this script on. Resolutions and such will change between devices. Follow the instructions on how to create your own search image here:
https://autohotkey.com/docs/commands/ImageSearch.htm
Lastly, this likely won't work if the wireless display is already connected. In my environment connecting the wireless display causes the resolution on the tablet to change and therefore it can't find the images on screen.
First I want to say that #jaredbaszler has provided a really good solution. It worked like a charm thank you :)
I was playing around with AutoHotkey too since I wanted to find out if there is another way to do this. After a while I came up with the following script:
Send #k ; Sends Windows button + K to open the Action Center Connect window
Sleep, 3000 ; Wait some time so the wireless display dongle can be found
Send {Enter} ; Send ENTER key to connect to wireless display dongle (works when only 1 is found)
Send {Esc} ; Send ESC key to close the Action Center Connect window
Ok. Now let me explain how this script works:
First it will press WIN+K which will open the Action Center Connect window
Then it will wait for 3 seconds so that the wireless display dongle can be found (You can adjust this value as you please but I needed to wait more than 2 seconds for my wireless display dongle to show up)
After the wait it will press ENTER which will automatically choose the first wireless display dongle in the list and trigger the connect function (If no wireless display dongles can be found your Default Browser will open the "Help" link)
The last thing the script does is to press the ESC key to close the Action Center
Well, that´s it. It´s nothing special but it works. I have tested this script a few times with my tablet and my wireless display dongle (I have this one here) and it seems to work just fine.
Unfortunately my script will not work as expected if you have multiple wireless display dongles up and running at the same time because my script will simply choose the first one that shows up. (This is not a problem for me since I only have one wireless display dongle)
The question is a bit old, but currently I've faced the same problem.
I suggest writing a UWP App utilizing the ProjectionManager class according to sample 5 + 6 in the Advanced casting sample.
The essentially steps are:
getting a DeviceID (for example by picking with DevicePicker class)
calling ProjectionManager.StartProjectingAsync(newViewId, currentViewId, selectedDeviceInformation) to start the projection
The DeviceInformation object can be obtained by calling DeviceInformation.CreateFromIdAsync() using the acquired (and saved) deviceId.
The viewIds may be set to 0 (or other invalid values) the projection is starting anyway.
If the projection has started, the app can be closed, the projection is still keeping alive.
If anyone is interested in the solution, I will upload the code to GitHub.
This was the AutoHotKey script that I wrote;
Run, explorer.exe ms-settings-connectabledevices:devicediscovery
Sleep, 800
Send, {Tab}
Send,+{Tab}
Send,{Enter}
Sleep, 200
; then next Send command types the first few letters of the name of the WiDi adaptor
Send, Microsoft
Sleep, 200
Send, {Tab}
Sleep, 200
Send, {Enter}
To automate and control Miracast, you want to write a UWP app using the Windows.Media.Casting namespace and Windows Universal Samples.
They can be written in many common languages including C#, C++, Javascript, and Visual Basic.
Windows provided two sets of functional sample programs to demonstrate what you can do with Miracast that can be opened and launched as Visual Studio solutions (.sln).
Basic Casting
Advanced Casting
Because UWP apps are designed to be made available on the Windows app store, they have specific requirements and regulations for functioning. (I recommend researching these.)
Also avoid the pitfall I made when I started writing a generic program and then wondered where the Universal Windows library was:
Starting a UWP project
I ended up using the vbs-based solution from here:
https://superuser.com/questions/1062789/how-to-connect-to-a-wireless-display-adapter-automatically-through-scripts-or-ta
While it basically does the same steps, I found this much more robust than the other solutions. What is more, vbs is available (I think) on most Win machines.

Automating cursor movements, mouse clicks and keyboard inputs in not helping to show that user is present on system

I have a software installed on my system which basically peeks my activity during working hours. It send a report saying how much time i was away from my system etc.
Since i m .net c# developer hence trying to break this system through my programming skills.
I have basically written up a windows form application which opens up an notepad for me and start typing some random characters into it. simultaneously it also performs some random clicks and moves the mouse cursor to give a feel that some one is on system and working even when i am not :)
I also use "SetThreadExecutionState" Pinvoke calls to keep system awake.I have also observed the User idle time and last interactivity time through "GetLastInputInfo" PInvoke system call while my mouse/keyboard are in automation process and it is also fine.
But this is not working for me. It works perfectly till 5 mins and then somehow it detects that user is not present on system.
any suggestions for the same.
Get a decompiler, crack-open the spying application and see how it is working inside. Only knowing the internal workings, you can create a "workable" any-spying program. But I would not bother with emulation of the activity, but rather spoof the way it reports to the server.
However, depending on where you live, it might be illegal to do these things and I would not recommend cheating your employer. And if spying gets on your nerves I'd also suggest to brush up your CV and start searching for a new job.

Categories