We Keep Coding

Updating BigQuery dataset access from C#

I am working with BigQuery. I create a DataSet and I want to define access rights with C# language.
It's not clear to me how to do it.
In GOOGLE web page https://cloud.google.com/bigquery/docs/dataset-access-controls is explained how to do it with some example for Java and Pyton (see below), but no example is provided for c#.
example in pyton:
dataset = client.get_dataset(dataset_id) # Make an API request.
entry = bigquery.AccessEntry(
role="READER",
entity_type="userByEmail",
entity_id="sample.bigquery.dev#gmail.com",
)
entries = list(dataset.access_entries)
entries.append(entry)
dataset.access_entries = entries
dataset = client.update_dataset(dataset, ["access_entries"]) # Make an API request.
full_dataset_id = "{}.{}".format(dataset.project, dataset.dataset_id)
Can anybody help please?

It's probably best to use the BigQueryDataset.Patch method, from the Google.Cloud.BigQuery.V2 package:
// Fetch the existing dataset
var client = BigQueryClient.Create(projectId);
var dataset = client.GetDataset(datasetId);
var accessList = dataset.Resource.Access ?? new List<AccessData>();
accessList.Add(new AccessData
{
Role = "Reader",
UserByEmail = "sample.bigquery.dev#gmail.com"
});
var patchedResource = new Dataset { Access = accessList };
// Push the changes back up to BigQuery
dataset.Patch(patchedResource, matchETag: true);
As an alternative, you can use Update to update the replace the dataset resource completely:
// Fetch the existing dataset
var client = BigQueryClient.Create(projectId);
var dataset = client.GetDataset(datasetId);
// Modify it in memory
var resource = dataset.Resource;
if (resource.Access is null)
{
// If there's no access list yet, create one.
resource.Access = new List<AccessData>();
}
var newAccess = new AccessData
{
Role = "Reader",
UserByEmail = "sample.bigquery.dev#gmail.com"
};
resource.Access.Add(newAccess);
// Push the changes back up to BigQuery
dataset.Update();

At the and I managed to make it work.
I used the same solution suggested by Jon Skeet, using the Patch method.
I attach my code here by.
public static bool GrantDatasetAccess(string dataSetId, bool online, string role, string email,
string bigQueryJsonPath, string bigQueryScope, string projectId, clsLog log)
{
try
{
BigQueryClient client = GetBigQueryClient(online, bigQueryJsonPath, bigQueryScope, projectId);
BigQueryDataset dataset = GetDataSet(online, dataSetId, bigQueryJsonPath, bigQueryScope, projectId, log);
List<AccessData> accessList = new List<AccessData>();
var accessData = new AccessData()
{
Role = role,
GroupByEmail = null,
UserByEmail = email,
SpecialGroup = null,
IamMember = null,
Domain = null,
View = null
};
accessList.Add(accessData);
dataset.Resource.Access = accessList;
dataset.Patch(dataset.Resource, true);
}
catch (Exception e)
{
log.ManageError("Error GetDataSet: {0}\nError: {1}", dataSetId, string.Concat(e.Message, "\n", e.StackTrace));
}
return true;
}

Get "initial Key" value from .mp3 file

I can't find a way to read the "initial key" property from an mp3 file to use the song information in my application.
I've already tried to find libraries which do the job for me. I found TagLib# which is a very cool solution for getting tags/properties of different file formats. (including mp3).
I can use this library to get the title, the artist, the beats per minute and so on.. only the initial key value is missing for my use which isn't featured, unfortunately.
I also tried to find other solutions which support the initial key property but I haven't found one.
I already found a source which seems to address the same issue and solves it with using TagLib#, but I can't figure out how he solved that problem.
Use Ctrl + F and search for "Initial" to find the code block.
You can find the link here
I'll post a short part of my code which can be used to determine different info about one song in a pattern like this: (["bpm"]"title" - "artist")
var file = TagLib.File.Create(filePath);
return $"[{file.Tag.BeatsPerMinute}]{file.Tag.Title} - {file.Tag.FirstPerformer}";
Thanks for any help or recommendations in advance! :)

Try this:
public static void Main(string[] args)
{
var path = …
var file = TagLib.File.Create (path);
var id3tag = (TagLib.Id3v2.Tag)file.GetTag (TagTypes.Id3v2);
var key = ReadInitialKey (id3tag);
Console.WriteLine ("Key = " + key);
}
static string ReadInitialKey(TagLib.Id3v2.Tag id3tag)
{
var frame = id3tag.GetFrames<TextInformationFrame>().Where (f => f.FrameId == "TKEY").FirstOrDefault();
return frame.Text.FirstOrDefault() ;
}
On Windows 10 you can also use:
async Task<string> ReadInitialKey(string path)
{
StorageFile file = await StorageFile.GetFileFromPathAsync(path);
Windows.Storage.FileProperties.MusicProperties musicProperties = await file.Properties.GetMusicPropertiesAsync();
var props = await musicProperties.RetrievePropertiesAsync(null);
var inkp = props["System.Music.InitialKey"];
return (string)inkp;
}
See here for documentation on MusicProperties object and here for the valid music properties.

You can use the Shell to read all MP3 properties.
Test on Windows 10, VS 2015 =>
// Add Reference Shell32.DLL
string sFolder = "e:\\";
string sFile= "01. IMANY - Don't Be so Shy (Filatov & Karas Remix).mp3";
List<string> arrProperties = new List<string>();
Shell objShell = new Shell();
Folder objFolder;
objFolder = objShell.NameSpace(sFolder);
int nMaxProperties = 332;
for (int i = 0; i < nMaxProperties; i++)
{
string sHeader = objFolder.GetDetailsOf(null, i);
arrProperties.Add(sHeader);
}
FolderItem objFolderItem = objFolder.ParseName(sFile);
if (objFolderItem != null)
{
for (int i = 0; i < arrProperties.Count; i++)
{
Console.WriteLine((i + ('\t' + (arrProperties[i] + (": " + objFolder.GetDetailsOf(objFolderItem, i))))));
}
}

Just borrowing code from nuget: mono TaglibSharp:
var tfile = TagLib.File.Create(#"..");
string initialKey = null;
if (tfile.GetTag(TagTypes.Id3v2) is TagLib.Id3v2.Tag id3v2)
{
/*
// test: add custom Initial Key tag
var frame = TextInformationFrame.Get(id3v2, "TKEY", true);
frame.Text = new[] {"qMMM"};
frame.TextEncoding = StringType.UTF8;
tfile.Save();
*/
var frame = TextInformationFrame.Get(id3v2, "TKEY", false);
initialKey = frame?.ToString();
}

Get image download url from firebase storage Xamarin

Im trying to get the image url from a photo I uploaded to firebase storage so I can store the reference to the image in another table which will allow me to be able to display the image elsewhere throughout the app.
I currently have
private void UploadPhoto()
{
if (filePAth != null)
{
progressDialog = new ProgressDialog(this);
progressDialog.SetTitle("Uploading...");
progressDialog.Window.SetType(Android.Views.WindowManagerTypes.SystemAlert);
progressDialog.Show();
var images = storageRef.Child("images/" + Guid.NewGuid().ToString());
images.PutFile(filePAth)
.AddOnProgressListener(this)
.AddOnSuccessListener(this)
.AddOnFailureListener(this);
}
}
public async void OnSuccess(Java.Lang.Object result)
{
try
{
var newImageDetails = storageRef.Child("images" + "/" + filePAth);
Photo photos = new Photo();
photos.categoryName = spinner.SelectedItem.ToString();
photos.photoId = newImageDetails.Name;
photos.ImageUrl = storageRef.DownloadUrl.ToString();
photos.tagName = addTag.Text;
if (user != null)
{
var uid = user.Uid;
//set the users id to the category
photos.uid = uid;
}
var firebase = new FirebaseClient(FirebaseURL);
var item = await firebase.Child("photos").PostAsync(photos);
}
}
The storageref.DownloadUrl does not give me the correct url
This is what I am looking for

I think that the trouble is Guid.NewGuid().
It generates a new code each time you insert a new image and your storageref points to it (image/GUID code).
In your OnSuccess you get image info from "images/" + filePath, that's different from upload path.
Why you use a new GUID? You can't determinate it, it will create a different path each time.
Consider then that your download url contains the media token too, it's not simply the clean path as you expect

I got a solution in debugging mode
i saw the downloadurl's properties and found the Scheme and SchemeSpecificPart
Scheme = "https"
SchemeSpecificPart = "//firebasestorage.googleapis.com/v0/b/maplog-e4ba5.appspot.com/o/-L0AMbihF23YKxsL1uss?alt=media&token=5c7ccef1-c857-4982-a288-fded2f0ff1aa"
so here is my code:
void IOnSuccessListener.OnSuccess(Java.Lang.Object result)
{
var snapShot = (UploadTask.TaskSnapshot)result;
string imgUrl = snapShot.DownloadUrl.Scheme
+ ":"
+ snapShot.DownloadUrl.SchemeSpecificPart;
}
and it works! i was looking for the solution :(( but i finally found it myself XD

EWS Search for Fields like Company or Street in Global Address List

I try to write a Webservice that can access to my exchange-server and search for names, companys and cities. At the moment i get the names like this:
ExchangeServiceBinding esb = new ExchangeServiceBinding();
esb.UseDefaultCredentials = true;
// Create the ResolveNamesType and set
// the unresolved entry.
ResolveNamesType rnType = new ResolveNamesType();
rnType.ReturnFullContactData = true;
rnType.UnresolvedEntry = "searchname";
// Resolve names.
ResolveNamesResponseType resolveNamesResponse
= esb.ResolveNames(rnType);
ArrayOfResponseMessagesType responses
= resolveNamesResponse.ResponseMessages;
// Check the result.
if (responses.Items.Length > 0 && responses.Items[0].ResponseClass != ResponseClassType.Error)
{
ResolveNamesResponseMessageType responseMessage = responses.Items[0] as
ResolveNamesResponseMessageType;
// Display the resolution information.
ResolutionType[] resolutions = responseMessage.ResolutionSet.Resolution;
foreach (ResolutionType resolution in resolutions)
{
Console.WriteLine(
"Name: " +
resolution.Contact.DisplayName
);
Console.WriteLine(
"EmailAddress: " +
resolution.Mailbox.EmailAddress
);
if (resolution.Contact.PhoneNumbers != null)
{
foreach (
PhoneNumberDictionaryEntryType phone
in resolution.Contact.PhoneNumbers)
{
Console.WriteLine(
phone.Key.ToString() +
" : " +
phone.Value
);
}
}
Console.WriteLine(
"Office location:" +
resolution.Contact.OfficeLocation
);
Console.WriteLine("\n");
}
}
But anybody know how i can serach for Propertys like Company and Street?

EWS only has limited Directory operations if your using OnPrem Exchange then the easiest way to do this is just use LDAP and lookup Active Directory directly. The resolveName operation is meant to be used to resolve a partial number and doesn't work with any other properties. If you have Exchange 2013 then there is the FindPeople operation http://msdn.microsoft.com/en-us/library/office/jj191039(v=exchg.150).aspx which supports using a QueryString which should work if those properties are indexed. eg
EWSProxy.FindPeopleType fpType = new EWSProxy.FindPeopleType();
EWSProxy.IndexedPageViewType indexPageView = new EWSProxy.IndexedPageViewType();
indexPageView.BasePoint = EWSProxy.IndexBasePointType.Beginning;
indexPageView.Offset = 0;
indexPageView.MaxEntriesReturned = 100;
indexPageView.MaxEntriesReturnedSpecified = true;
fpType.IndexedPageItemView = indexPageView;
fpType.ParentFolderId = new EWSProxy.TargetFolderIdType();
EWSProxy.DistinguishedFolderIdType Gal = new EWSProxy.DistinguishedFolderIdType();
Gal.Id = EWSProxy.DistinguishedFolderIdNameType.directory;
fpType.QueryString = "Office";
fpType.ParentFolderId.Item = Gal;
EWSProxy.FindPeopleResponseMessageType fpm = null;
do
{
fpm = esb.FindPeople(fpType);
if (fpm.ResponseClass == EWSProxy.ResponseClassType.Success)
{
foreach (EWSProxy.PersonaType PsCnt in fpm.People)
{
Console.WriteLine(PsCnt.EmailAddress.EmailAddress);
}
indexPageView.Offset += fpm.People.Length;
}
else
{
throw new Exception("Error");
}
} while (fpm.TotalNumberOfPeopleInView > indexPageView.Offset);
Cheers
Glen

Encoder SDK 4 - Push to Publishing Point

I'm coding an application in c# using EC4 SP2 SDK.
I want to publish my file to a media server publishing point. I've searched and found 2 examples regarding seting up and auth on publishing points, but either are from older sdk's or do not work (and are for console). basicly my application doesn't encode nothing, as if it had nothing to encode.
When in degub mode checkpont i can see the correct properties for the source file and for the server.
The encoding process takes 0secs to process. I checked the logs on the server events and i get a warning "the security system has received and auth request that could not be decoded". I just havo no knowledge to break up further than this. Any help would be appreciated.
this is the piece of code:
private void broadcastSourceFileToMediaServer2()
{
using (LiveJob job = new LiveJob())
{
String filetoencode = #"c:\temp\niceday.wmv";
LiveFileSource filesource = job.AddFileSource(filetoencode);
filesource.PlaybackMode = FileSourcePlaybackMode.Loop;
job.ActivateSource(filesource);
job.ApplyPreset(LivePresets.VC1Broadband4x3);
//don't know which one is good to use
job.AcquireCredentials += new EventHandler<AcquireCredentialsEventArgs>(job_AcquireCredentials);
_myUserName = "indes";
_pw = PullPW("indes");
Uri url = new Uri("http://192.168.1.74:8080/live");
PushBroadcastPublishFormat pubpoint = new PushBroadcastPublishFormat();
pubpoint.PublishingPoint = url;
pubpoint.UserName = _myUserName;
pubpoint.Password = _pw;
job.PublishFormats.Add(pubpoint);
job.PreConnectPublishingPoint();
job.StartEncoding();
statusBox.Text = job.NumberOfEncodedSamples.ToString();
job.StopEncoding();
job.Dispose();
}
}
public static string _myUserName { get; set; }
public static SecureString _pw { get; set; }
//codificação de Password a enviar
private static SecureString PullPW(string pw)
{
SecureString s = new SecureString();
foreach (char c in pw) s.AppendChar(c);
return s;
}
static void job_AcquireCredentials(object sender, AcquireCredentialsEventArgs e)
{
e.UserName = _myUserName;
e.Password = _pw;
e.Modes = AcquireCredentialModes.None;
}

Progresses:
I managed to authenticate (at least get a positive audit event) on the server.
I changed from this:
//don't know which one is good to use
job.AcquireCredentials += new EventHandler<AcquireCredentialsEventArgs>(job_AcquireCredentials);
_myUserName = "indes";
_pw = PullPW("indes");
Uri url = new Uri("http://192.168.1.74:8080/live");
PushBroadcastPublishFormat pubpoint = new PushBroadcastPublishFormat();
pubpoint.PublishingPoint = url;
pubpoint.UserName = _myUserName;
pubpoint.Password = _pw;
To this:
job.AcquireCredentials += new EventHandler<AcquireCredentialsEventArgs>(job_AcquireCredentials);
_myUserName = #"mediaservername\user";
_pw = PullPW("user_password");
Uri url = new Uri("http://192.168.1.74:8080/live");
PushBroadcastPublishFormat pubpoint = new PushBroadcastPublishFormat();
pubpoint.PublishingPoint = url;
If you see on one side if had to include the domain (either domain or computername) before username. this changed the failed audit events on the server, so i could eliminate the manual credentials pubpoint.username and pubpoint.Password.
Now I'm just dealing with a lack of output format exception. On to it.

How about using SMOOTH Streaming, I managed to get my project going but I didn't get much more beyond Look below, to the part that has the PUBLISH switch type. ignore the file portion
internal bool StartStream()
{
Busy = true;
// Instantiates a new job for encoding
//
//***************************************Live Stream Archive******************************
if (blnRecordFromFile)
{
// Sets up publishing format for file archival type
FileArchivePublishFormat fileOut = new FileArchivePublishFormat();
// job.ApplyPreset(LivePresets.VC1512kDSL16x9);
// Gets timestamp and edits it for filename
string timeStamp = DateTime.Now.ToString();
timeStamp = timeStamp.Replace("/", "-");
timeStamp = timeStamp.Replace(":", ".");
// Sets file path and name
string path = "C:\\output\\";
string filename = "Capture" + timeStamp + ".ismv";
if (!Directory.Exists(path))
Directory.CreateDirectory(path);
fileOut.OutputFileName = Path.Combine(path, filename);
// Adds the format to the job. You can add additional formats as well such as
// Publishing streams or broadcasting from a port
job.PublishFormats.Add(fileOut);
}
//******************************END OF Stream PORTION****************************************
////////////////////////////////////////////////////////////////////////////////////////////////////
//*************************************** Process Files or Live Stream******************************
if (blnRecordFromFile)
{
job.ApplyPreset(LivePresets.VC1IISSmoothStreaming720pWidescreen);
job = new LiveJob();
// Verifies all information is entered
if (string.IsNullOrWhiteSpace(sourcePath) || string.IsNullOrWhiteSpace(destinationPath))
return false;
job.Status += new EventHandler<EncodeStatusEventArgs>(StreamStatus);
LiveFileSource fileSource;
try
{
// Sets file to active source and checks if it is valid
fileSource = job.AddFileSource(sourcePath);
}
catch (InvalidMediaFileException)
{
return false;
}
// Sets to loop media for streaming
// fileSource.PlaybackMode = FileSourcePlaybackMode.Loop;
// Makes this file the active source. Multiple files can be added
// and cued to move to each other at their ends
job.ActivateSource(fileSource);
}
//******************************END OF FILE PORTION****************************************
// Sets up variable for fomat data
switch (publishType)
{
case Output.Archive:
// Verifies destination path exists and if not creates it
try
{
if (!Directory.Exists(destinationPath))
Directory.CreateDirectory(destinationPath);
}
catch (IOException)
{
return false;
}
FileArchivePublishFormat archiveFormat = new FileArchivePublishFormat();
// Gets the location of the old extention and removes it
string filename = Path.GetFileNameWithoutExtension(sourcePath);
// Sets the archive path and file name
archiveFormat.OutputFileName = Path.Combine(destinationPath, filename + ".ismv");
job.PublishFormats.Add(archiveFormat);
break;
case Output.Publish:
// Setups streaming of media to publishing point
job = new LiveJob();
// Aquires audio and video devices
Collection<EncoderDevice> devices = EncoderDevices.FindDevices(EncoderDeviceType.Video);
EncoderDevice video = devices.Count > 0 ? devices[0] : null;
for (int i = 0; i < devices.Count; ++i)
// devices[i].Dispose();
devices.Clear();
devices = EncoderDevices.FindDevices(EncoderDeviceType.Audio);
EncoderDevice audio = devices.Count > 0 ? devices[0] : null;
for (int i = 1; i < devices.Count; ++i)
devices[i].Dispose();
devices.Clear();
// Checks for a/v devices
if (video != null && audio != null)
{
//job.ApplyPreset(Preset.FromFile(#"C:\Tempura\LivePreset3.xml"));
job.ApplyPreset(LivePresets.H264IISSmoothStreamingLowBandwidthStandard);
job.OutputFormat.VideoProfile.SmoothStreaming = true;
deviceSource = job.AddDeviceSource(video, audio);
// Make this source the active one
job.ActivateSource(deviceSource);
}
else
{
error = true;
}
PushBroadcastPublishFormat publishFormat = new PushBroadcastPublishFormat();
try
{
// checks the path for a valid publishing point
publishFormat.PublishingPoint = new Uri(destinationPath);
}
catch (UriFormatException)
{
return false;
}
// Adds the publishing format to the job
try
{
// job.ApplyPreset(LivePresets.VC1IISSmoothStreaming480pWidescreen);
job.PublishFormats.Add(publishFormat);
job.PreConnectPublishingPoint();
}
catch (Exception e)
{
MessageBox.Show(e.StackTrace.ToString());
}
break;
default:
return false;
}
job.StartEncoding();
return true;
}

Sadly I dont have enough rep to comment, so I have to write it as an answer.
Due to you are starting a live job, in order to stream you should not call job.StopEncoding() right after StartEncoding. I think usually you would use an event to stop the encoding. If you start encoding and immediately stop it, it is only logical you have no, or only a very small output.

I changed your code to the following and it seems work well. I guess your problem is that you disposed the instance of LiveJob class. You have to keep the instance alive before it finished encoding the whole stream. So change the using part and remove the StopEncoding and Dispose will be OK.
private void broadcastSourceFileToMediaServer2()
{
LiveJob job = new LiveJob();
String filetoencode = #"c:\temp\niceday.wmv";
LiveFileSource filesource = job.AddFileSource(filetoencode);
filesource.PlaybackMode = FileSourcePlaybackMode.Loop;
job.ActivateSource(filesource);
job.ApplyPreset(LivePresets.VC1Broadband4x3);
//don't know which one is good to use
job.AcquireCredentials += new EventHandler<AcquireCredentialsEventArgs>(job_AcquireCredentials);
_myUserName = "indes";
_pw = PullPW("indes");
Uri url = new Uri("http://192.168.1.74:8080/live");
PushBroadcastPublishFormat pubpoint = new PushBroadcastPublishFormat();
pubpoint.PublishingPoint = url;
pubpoint.UserName = _myUserName;
pubpoint.Password = _pw;
job.PublishFormats.Add(pubpoint);
job.PreConnectPublishingPoint();
job.StartEncoding();
statusBox.Text = job.NumberOfEncodedSamples.ToString();
}
public static string _myUserName { get; set; }
public static SecureString _pw { get; set; }
//codificação de Password a enviar
private static SecureString PullPW(string pw)
{
SecureString s = new SecureString();
foreach (char c in pw) s.AppendChar(c);
return s;
}
static void job_AcquireCredentials(object sender, AcquireCredentialsEventArgs e)
{
e.UserName = _myUserName;
e.Password = _pw;
e.Modes = AcquireCredentialModes.None;
}