I've been working on a small Microsoft Teams extension that takes an incoming Audio/Video feed from each participant and records the data into an mp4 file for each participant. In all the MS Teams documentation they focus on getting access to the raw data itself and gloss over any kind of persistence of it into a usable video container. The video comes in as an NV12 encoded byte array and the audio is wav format. I've been trying to use Gstreamer to take the raw video data, push it to a videoconvert then on to a filesink to try and save the data as an h264 encoded mp4. I've pulled together a mix and match of code examples and think I'm close but missing something in the process. My pipeline creates successfully but when pushing buffers into the appsrc I get the following in my console:
pausing after gst_pad_push() = not-linked
error: Internal data stream error.
error: streaming stopped, reason not-linked (-1)
The code I use to create my pipeline, pads and sinks etc is as follows:
private Gst.App.AppSrc CreatePipeline(Guid identifier, string identity, int width, int height, string directory)
{
Directory.CreateDirectory($"{directory}\\temporary\\{identifier}");
var pipeline = new Gst.Pipeline($"pipeline_{identity}");
var VideoQueue = Gst.ElementFactory.Make("queue", $"video_queue_{identity}");
var VideoConvert = Gst.ElementFactory.Make("videoconvert", $"video_convert_{identity}");
var VideoSink = Gst.ElementFactory.Make("autovideosink", $"video_sink_{identity}");
var AppQueue = Gst.ElementFactory.Make("queue", $"app_queue_{identity}");
var AppSink = new Gst.App.AppSink($"app_sink_{identity}");
var AppSource = new Gst.App.AppSrc($"app_src_{identity}");
var Tee = Gst.ElementFactory.Make("tee", $"tee_{identity}");
var FileSink = Gst.ElementFactory.Make("filesink", $"file_sink_{identity}");
AppSource.Caps = Gst.Caps.FromString($"video/x-raw,format=NV12,width={width},height={height},framerate={this.fixedFps}");
AppSource.IsLive = true;
AppSink.EmitSignals = true;
AppSink.Caps = Gst.Caps.FromString($"video/x-raw,format=NV12,width={width},height={height},framerate={this.fixedFps}");
AppSink.NewSample += NewSample;
Console.WriteLine("Setting Filesink location");
FileSink["location"] = $"{directory}\\temporary\\{identifier}\\{identity}.mp4";
pipeline.Add(AppSource, Tee, VideoQueue, VideoConvert, VideoSink, AppQueue, AppSink, FileSink);
var teeVideoPad = Tee.GetRequestPad("src_%u");
var queueVideoPad = VideoQueue.GetStaticPad("sink");
var teeAppPad = Tee.GetRequestPad("src_%u");
var queueAppPad = AppQueue.GetStaticPad("sink");
if ((teeVideoPad.Link(queueVideoPad) != Gst.PadLinkReturn.Ok) ||
(teeAppPad.Link(queueAppPad) != Gst.PadLinkReturn.Ok))
{
Console.WriteLine("Tee could not be linked.");
throw new Exception("Tee could not be linked.");
}
AppSource.PadAdded += new Gst.PadAddedHandler(this.OnVideoPadAdded);
var bus = pipeline.Bus;
bus.AddSignalWatch();
bus.Connect("message::error", HandleError);
pipeline.SetState(Gst.State.Playing);
return AppSource;
}
And this gets called every time a new participant joins the call. Then each time a new video frame is sent to the call from a participant the following code is executed:
private bool NV12toMP4(byte[] array, string identity, int width, int height, long timestamp, int length)
{
var buffer = new Gst.Buffer(null, (ulong)length, Gst.AllocationParams.Zero)
{
Pts = (ulong)timestamp,
Dts = (ulong)timestamp
};
buffer.Fill(0, array);
var ret = this.participantSources[identity].PushBuffer(buffer);
buffer.Dispose();
if (ret != Gst.FlowReturn.Ok)
{
return false;
}
return true;
}
I was expecting my PadAdded method to get called on the AppSrc when it first detects the type of input but this never gets triggered so I'm not sure where to look next on this.
Related
New Info:
I thought I would paste this in full as I can not seem to find any samples on the web of a c# solution for StarLink so hopefully anyone else looking for something may find this helpful and may contribute.
My New Proto File - (partial) - I took the advise of Yuri below. Thanks for the direction here. I was able to I have been using this tool and it has brought a lot of insight but I am still stuck on the c# side of the solution. I am an old VB.Net developer though I have done a bunch in c# I am by no means savvy in it and am probably missing something so simple. Again, any insight would be awesome. I can not post the full proto here as stack has char limit on posts. this is the first bit with messages etc. I can post more if it helps but trying to keep it to the important part.
syntax = "proto3";
option csharp_namespace = "SpaceX.API.Device";
package SpaceX.API.Device;
service Device {
//rpc Handle (.SpaceX.API.Device.Request) returns (.SpaceX.API.Device.Response) {}
//rpc Stream (stream .SpaceX.API.Device.ToDevice) returns (stream .SpaceX.API.Device.FromDevice) {}
rpc Handle (Request) returns (Response);
rpc Stream (Request) returns (Response);
}
message ToDevice {
string message = 1;
}
message Request {
uint64 id = 1;
string target_id = 13;
uint64 epoch_id = 14;
oneof request {
SignedData signed_request = 15;
RebootRequest reboot = 1001;
SpeedTestRequest speed_test = 1003;
GetStatusRequest get_status = 1004;
AuthenticateRequest authenticate = 1005;
GetNextIdRequest get_next_id = 1006;
GetHistoryRequest get_history = 1007;
GetDeviceInfoRequest get_device_info = 1008;
GetPingRequest get_ping = 1009;
SetTrustedKeysRequest set_trusted_keys = 1010;
FactoryResetRequest factory_reset = 1011;
GetLogRequest get_log = 1012;
SetSkuRequest set_sku = 1013;
UpdateRequest update = 1014;
GetNetworkInterfacesRequest get_network_interfaces = 1015;
PingHostRequest ping_host = 1016;
GetLocationRequest get_location = 1017;
EnableFlowRequest enable_flow = 1018;
GetHeapDumpRequest get_heap_dump = 1019;
RestartControlRequest restart_control = 1020;
FuseRequest fuse = 1021;
GetPersistentStatsRequest get_persistent_stats = 1022;
GetConnectionsRequest get_connections = 1023;
FlushTelemRequest flush_telem = 1026;
StartSpeedtestRequest start_speedtest = 1027;
GetSpeedtestStatusRequest get_speedtest_status = 1028;
ReportClientSpeedtestRequest report_client_speedtest = 1029;
InitiateRemoteSshRequest initiate_remote_ssh = 1030;
SelfTestRequest self_test = 1031;
SetTestModeRequest set_test_mode = 1032;
DishStowRequest dish_stow = 2002;
DishGetContextRequest dish_get_context = 2003;
DishSetEmcRequest dish_set_emc = 2007;
DishGetObstructionMapRequest dish_get_obstruction_map = 2008;
DishGetEmcRequest dish_get_emc = 2009;
DishSetConfigRequest dish_set_config = 2010;
DishGetConfigRequest dish_get_config = 2011;
StartDishSelfTestRequest start_dish_self_test = 2012;
WifiSetConfigRequest wifi_set_config = 3001;
WifiGetClientsRequest wifi_get_clients = 3002;
WifiSetupRequest wifi_setup = 3003;
WifiGetPingMetricsRequest wifi_get_ping_metrics = 3007;
WifiGetDiagnosticsRequest wifi_get_diagnostics = 3008;
WifiGetConfigRequest wifi_get_config = 3009;
WifiSetMeshDeviceTrustRequest wifi_set_mesh_device_trust = 3012;
WifiSetMeshConfigRequest wifi_set_mesh_config = 3013;
WifiGetClientHistoryRequest wifi_get_client_history = 3015;
TransceiverIFLoopbackTestRequest transceiver_if_loopback_test = 4001;
TransceiverGetStatusRequest transceiver_get_status = 4003;
TransceiverGetTelemetryRequest transceiver_get_telemetry = 4004;
}
reserved 1025, 3011, 3014;
}
message SignedData {
bytes data = 1;
bytes signature = 2;
}
My New .cs
I have tried many things from Microsoft's examples to thing I can gather from other samples. I simply can not get it to work and am lost. Again, any insight would be amazing and hopefully helpful to others looking for a solution in c#. You will see my commented code of this I have been playing with. Basically I am attempting to achieve three things and have made some movement in one of them.
Goals:
1 - Use Server Reflection to discover services.
I think I got this one resolved with dot-net grpc.
2 - Simply want to check available methods under a service and potentially either check or generate a new .proto file in case things change. StaLink does not publish its proto schema so I assume it could change anytime without warning.
3 - Just run any one of the available methods. I have tried the GetDeviceInfoRequest but can not seem to construct the request message properly. I have not been able to get this accomplishe in the gRPCurl tool either. I can do it on the basic service shown by Microsoft of course but these methods seem to be more complex and I simply get all kinds of errors.
Again, any insight or assistance would be amazing. Thanks to any and all in advance.
New .cs File
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Grpc.Core;
using Grpc.Net.Client;
using Grpc.Reflection.V1Alpha;
using ServerReflectionClient = Grpc.Reflection.V1Alpha.ServerReflection.ServerReflectionClient;
using SpaceX.API.Device;
public class Program
{
static async Task Main(string[] args)
{
//SETUP CHANNEL AND CLIENT
using var channel = GrpcChannel.ForAddress("http://192.168.100.1:9200");
var client = new ServerReflectionClient(channel);
var StarLinkClient = new Device.DeviceClient(channel);
//using var call = StarLinkClient.StreamAsync(new ToDevice { Request = GetDeviceInfoRequest });
//await foreach (var response in call.ResponseStream.ReadAllAsync())
//var request = Device.GetDeviceInfoRequest;
//var reply = await StarLinkClient.HandleAsync(
// new Request {'"getDeviceInfo" : {} '});
//Console.WriteLine(reply.Message);
//=============================================SERVER REFLECTION=============================================================
Console.WriteLine("Calling reflection service:");
var response = await SingleRequestAsync(client, new ServerReflectionRequest
{
ListServices = "" // Get all services
});
Console.WriteLine("Services:");
foreach (var item in response.ListServicesResponse.Service)
{
Console.WriteLine("- " + item.Name);
Console.WriteLine();
var StarLink = item.Name;
//Console.WriteLine(StarLink.getStatus());
}
//=============================================SERVER REFLECTION=============================================================
Console.WriteLine("Press any key to exit...");
Console.ReadKey();
}
void setupchannel()
{
}
private static Task SingleRequestAsync(ServerReflectionClient client, Metadata metadata)
{
throw new NotImplementedException();
}
private static async Task<ServerReflectionResponse> SingleRequestAsync(ServerReflectionClient client, ServerReflectionRequest request)
{
using var call = client.ServerReflectionInfo();
await call.RequestStream.WriteAsync(request);
Debug.Assert(await call.ResponseStream.MoveNext());
var response = call.ResponseStream.Current;
await call.RequestStream.CompleteAsync();
return response;
}
}
Again, thanks in advance to anyone willing to assist here. Hopefully this helps others as well.
I basically need to bring information from users using biometrics, for this I am using a biometric reader of the digital person (U.are.U 4500).
The problem is that sometimes it happens to come information from another user, in case I put my finger on the reader, it is read and instead of returning my data, it is returned from another user.
I've noticed one thing, I'm saving biometrics in byte format ([byte]). When I am going to register biometrics, my byte array usually consists of 1000 items. When I make the comparison, the bytes are usually returned from the database, but the byte array I will use to compare with the database is 300. It may sound like bullshit, but does it have any interference? Anyway, I will make the source available !! Thank you for your help.
About the small details of the bytes that I noticed:
Bytes in register:
bytes in check:
I tried to post the images here but I was blocked by stack overflow :)
Register Biometric:
memoryTemplate = new MemoryStream();
Template.Serialize(memoryTemplate);
memoryTemplate.Position = 0;
brReader = new BinaryReader(memoryTemplate);
serializedTemplate = brReader.ReadBytes((Int32)memoryTemplate.Length);
DALUsuario usuario = new DALUsuario();
//I execute the method to register biometrics
usuario.CadastrarBiometria(Apoio.stCodUsuario, Finger, serializedTemplate);
//usuario.CadastrarBiometria:
con = new SqlConnection();
apoio = new Apoio();
con = apoio.conexaoBD(con);
command = new StringBuilder();
fkMao = VerificarMao(fkDedo);
command.Append("INSERT INTO TBL_BIOMETRIA (FK_USUARIO, FK_DEDO, FK_MAO, BIOMETRIA) ");
command.Append("VALUES (#idUser, #fkDedo, #fkMao, #Biometria)");
cmd = new SqlCommand(command.ToString(), con);
cmd.Parameters.AddWithValue("#IdUser", idUsuario);
cmd.Parameters.AddWithValue("#fkDedo", fkDedo);
cmd.Parameters.AddWithValue("#fkMao", fkMao);
cmd.Parameters.AddWithValue("#Biometria", Template);
cmd.ExecuteNonQuery();
//Biometrics Check
private void VerificationControl1_OnComplete(object Control, DPFP.FeatureSet FeatureSet, ref DPFP.Gui.EventHandlerStatus EventHandlerStatus) {
DPFP.Verification.Verification ver = new DPFP.Verification.Verification();
DPFP.Verification.Verification.Result res = new DPFP.Verification.Verification.Result();
Usuario user = null;
DALUsuario dalUser = null;
// Compare feature set with all stored templates.
user = new Usuario();
dalUser = new DALUsuario();
// I execute the method that takes the user's biometry and compares it with the one of the database
user = dalUser.RetornaBiometria(FeatureSet);
}
//dalUser.RetornaBiometria:
while (reader.Read()) {
user.iCodUsuario = Convert.ToInt32(reader["FK_USUARIO"]);
user.Biometria = (byte[]) reader["BIOMETRIA"];
memoryStream = new MemoryStream(user.Biometria);
DPFP.Template tmpObj = new DPFP.Template();
tmpObj.DeSerialize(memoryStream);
DPFP.Verification.Verification.Result result = new DPFP.Verification.Verification.Result();
// I MAKE COMPARISON OF THE USER'S BIOMETRY WITH THE BIOMETRIES REGISTERED IN THE DATABASE
ver.Verify(FeatureSet, tmpObj, ref res);
Data.IsFeatureSetMatched = res.Verified;
Data.FalseAcceptRate = res.FARAchieved;
if (res.Verified) {
break; // success
}
fpTemp = null;
}
I am sending this from my BLE device:
BTLEserial.print ("one");
BTLEserial.print (",");
BTLEserial.print ("two");
So what I am sending is: "one, two" and i am now trying to get this value but with my current code I get 54 and 44 instead and I do not quite know why.
I use this plugin: https://github.com/xabre/xamarin-bluetooth-le (Plugin.BLE)
This is how I read the data:
var adapter = CrossBluetoothLE.Current.Adapter;
await adapter.ConnectToDeviceAsync(mydevice);
var service = await mydevice.GetServiceAsync(Guid.Parse("6e400001-b5a3-f393-e0a9-e50e24dcca9e"));
var services = await mydevice.GetServicesAsync();
var RXcharacteristics = await service.GetCharacteristicAsync(Guid.Parse("6e400003-b5a3-f393-e0a9-e50e24dcca9e"));
var characteristics = await service.GetCharacteristicsAsync();
int whatResult = 0;
string valueone;
string valuetwo;
RXcharacteristics.ValueUpdated += (sender, e) =>
{
var result = e.Characteristic.Value;
foreach (var items in result)
{
if (whatResult == 0)
{
valueone = Convert.ToString(items);
System.Diagnostics.Debug.Writeline(valueone);
whatResult++;
}
else {
valuetwo = Convert.ToString(items);
System.Diagnostics.Debug.Writeline(valuetwo);
whatResult = 0;
}
}
};
await RXcharacteristics.StartUpdatesAsync();
How come I cannot get the correct data from the result I am receiving? I also tried with var result = e.Characteristic.StringValue; but with the same result.
I googled and came across a person with the same issue:
https://github.com/xabre/xamarin-bluetooth-le/issues/88
He for example said this, which showcase that my device has CanUpdate as true. And as I said above I succesfully get data from my code but I do not get the correct values.
The RXCharacteristic has CanWrite = false, CanRead = false, CanUpdate = true
If I use this app:
https://github.com/adafruit/Bluefruit_LE_Connect_v2
That is open source (coded with swift) I can succesfully get the correct value
The github documentation for this library indicates that the Value is a byte array
public override byte[] Value => _nativeCharacteristic.Value?.ToArray();
Which would mean that instead of using a foreach loop over each byte you should attempt to use a text encoding instead 1
var result = e.Characteristic.Value;
var str = System.Text.Encoding.UTF8.GetString(result,0,result.Length);
1. Andy0708, Fri Jan 06 2017, Oksana, "How convert byte array to string [duplicate]", Jul 25 '12 at 16:39, https://stackoverflow.com/a/11654825/1026459
In ServiceNow, I am able to get only a maximum of 250 records in a SOAP request. How to get all the records?
Web Reference Url = https://*****.service-now.com/rm_story.do?WSDL
Code:
var url = "https://*****.service-now.com/rm_story.do?SOAP";
var userName = *****;
var password = *****;
var proxy = new ServiceNow_rm_story
{
Url = url,
Credentials = new NetworkCredential(userName, password)
};
try
{
var objRecord = new Namespace.WebReference.getRecords
{
// filters..
};
var recordResults = proxy.getRecords(objRecord);
}
catch (Exception ex)
{
}
In recordResults, I am getting only 250 records. How to get all the records ?
Also see this stack overflow answer which provides info.
Get ServiceNow Records Powershell - More than 250
Note that returning a large number of records can affect performance of the response and it may be more efficient to process your query in batches using offsets (i.e., get 1-100, then 101-200, ...). This can be achieved by using a sort order and offset. The ServiceNow REST Table API actually returns link headers from Get requests providing you links for the first, next and last set of records making it easy to know the url to query the next batch of records.
See: http://wiki.servicenow.com/index.php?title=Table_API#Methods
and look under 'Response Header'.
Have u tried to pass/override __limit parameter?
Google / wiki / Users manual / Release notes are always helpful
In your code snippet in line where it says //filter you should define __limit (and potentially __first_row and __last_row as explained in the example bellow)
int Skip = 0;
int Take = 250;
while (true)
{
using (var soapClient = new ServiceNowLocations.ServiceNow_cmn_location())
{
var cred = new System.Net.NetworkCredential(_user, _pass);
soapClient.Credentials = cred;
soapClient.Url = _apiUrl + "cmn_location.do?SOAP";
var getParams = new ServiceNowLocations.getRecords()
{
__first_row = Skip.ToString(),
__last_row = (Skip + Take).ToString(),
__limit = Take.ToString()
};
var records = soapClient.getRecords(getParams);
if (records != null)
{
if (records.Count() == 0)
{
break;
}
Skip += records.Count();
if (records.Count() != Take)
{
// last batch or everything in first batch
break;
}
}
else
{
// service now web service endpoint not configured correctly
break;
}
}
}
I made an library that handles interacting with ServiceNow Rest API much easier
https://emersonbottero.github.io/ServiceNow.Core/
I have a SL4 project that is successfully streaming a great sounding WMA audio stream from a remote location. All of the MediaElement actions are straight forward.
What I want to do is read the attributes that are passed as text along with the Audio stream. For instance the encoder of the stream embeds the title of the stream, the title of the song playing and the name of the artist for the current song.
How would I pick this out using Silverlight 4 and then display it in a Label to the user?
It sure would be easier than writing a bunch of web services to do the same thing. Windows Media Player and WinAmp all get the information I am just not seeing it in the MediaElement object collection.
I found the answer after searchting the web as well as fiddling with Expression 3 a little as well.
It turns out that a live audio stream has markers that are sent across as well as the audio. Markers can contain almost anything but one is called a "Caption". The caption is basically a free-form string field that you can read. With my stream the encoder sends a lot of information across as a caption that can then be broken down. So here is the code I am using:
Starts with registering a few events, the last one is the important one.
public MainPage()
{
InitializeComponent();
this.mediaElement1.BufferingProgressChanged += new RoutedEventHandler(mediaElement1_BufferingProgressChanged);
this.mediaElement1.MarkerReached += new TimelineMarkerRoutedEventHandler(mediaElement1_MarkerReached);
}
Then the actual marker handler does the following:
private void mediaElement1_MarkerReached(object sender, TimelineMarkerRoutedEventArgs e)
{
Dictionary<string, string> songAttribs = new Dictionary<string, string>();
string playerFeed = HttpUtility.UrlDecode(e.Marker.Text);
char[] delims = { '&' };
string[] Attribs = playerFeed.Split(delims);
foreach (String attrib in Attribs)
{
string[] keypair = attrib.Split('=');
string key = "";
string value = "";
try
{
key = keypair[0];
}
catch
{
key = null;
}
if (key != null)
{
try
{
value = keypair[1];
}
catch
{
value = "";
}
songAttribs.Add(keypair[0], keypair[1]);
}
}
nowplaying.Title = songAttribs["title"];
nowplaying.Artist = songAttribs["artist"];
nowplaying.Duration = 0;
this.label2.Content = "Artist: " + nowplaying.Artist;
this.label3.Content = "Title: " + nowplaying.Title;
this.label1.Content = playerFeed;
}
Still working on some of the code but so far things seem to be working.
Mayba WMP and WinAmp get the Informations from a Website or something like this, and dont read it out of the stream...
Werewolve