We Keep Coding

How to use Google Cloud Speech (V1 API) for speech to text - need to be able to process over 3 hours audio files properly and efficiently

I am looking for documentation and stuff but could not find a solution yet
Installed NuGet package
Also generated API key
However can't find proper documentation how to use API key
Moreover, I want to be able to upload very long audio files
So what would be the proper way to upload up to 3 hours audio files and get their results?
I have 300$ budget so should be enough
Here my so far code
This code currently fails since I have not set the credentials correctly at the moment which I don't know how to
I also have service account file ready to use
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
}
private void Button_Click(object sender, RoutedEventArgs e)
{
var speech = SpeechClient.Create();
var config = new RecognitionConfig
{
Encoding = RecognitionConfig.Types.AudioEncoding.Flac,
SampleRateHertz = 48000,
LanguageCode = LanguageCodes.English.UnitedStates
};
var audio = RecognitionAudio.FromStorageUri("1m.flac");
var response = speech.Recognize(config, audio);
foreach (var result in response.Results)
{
foreach (var alternative in result.Alternatives)
{
Debug.WriteLine(alternative.Transcript);
}
}
}
}
I don't want to set environment variable. I have both API key and Service Account json file. How can I manually set?

You need to use the SpeechClientBuilder to create a SpeechClient with custom credentials, if you don't want to use the environment variable. Assuming you've got a service account file somewhere, change this:
var speech = SpeechClient.Create();
to this:
var speech = new SpeechClientBuilder
{
CredentialsPath = "/path/to/your/file"
}.Build();
Note that to perform a long-running recognition operation, you should also use the LongRunningRecognize method - I strongly suspect your current RPC will fail, either explicitly because it's trying to run on a file that's too large, or it'll just time out.

You need to set the environment variable before create the instance of Speech:
Environment.SetEnvironmentVariable("GOOGLE_APPLICATION_CREDENTIALS", "text-tospeech.json");
Where the second param (text-tospeech.json) is your file with credentials generated by Google Api.

How to provide configuration for Library (dll) which can be modified in Runtime application

I am working on a library that may be used in one or more runtime applications. This library has a function which searches a string for a set of "keys". If one is found, the return should return that key's value. If not found, return the original string.
protected string CorrectFloatingIndex(string indexDescription)
{
if (indexDescription== null)
{
return "";
}
foreach (var mapper in FloatingIndex.Mapping)
{
if (Regex.IsMatch(indexDescription, mapper.Item1,
RegexOptions.IgnoreCase))
{
return mapper.Item2;
}
}
return indexDescription;
}
I have currently been hard-coding the FloatingIndex.Mapping with a list of Tuples. When any .Item1 is found, then return the associated .Item2.
public static class FloatingIndex
{
public static List<Tuple<string, string>> Mapping =
new List<Tuple<string, string>>()
{
new Tuple<string, string>("TIIE", "TIIE"),
new Tuple<string, string>("CAMARA PROMEDIO INDEX", "CAMAR"),
new Tuple<string, string>("LIBOR", "LIBOR")
};
}
This works fine; however when a new example of key-value pair is found and should be added to the list, this requires a change to the DLL, which then requires a new release of the runtime application using this DLL. This list will build over time as new key-value pairs are found are determined.
Keep in mind, the CorrectFloatingIndex method is used in the DLL. The runtime application does not use this method or the mapping table directly.
Is there a way to have a reference file or application configuration file which is located in the DLL (for testing and responsibility purposes), which cascades to the runtime application? This would allow testing and full functionality of the DLL while providing the runtime application the ability to provide updated key-value pairs to that file, without requiring releases.

AzureDirectory Library for Lucene.NET: Performance Issues when indexing

I am trying to use the AzureDirectory library to store a Lucene.NET index on a Azure Cloud Storage account.
I am using the following versions:
Microsoft.Windows.Azure.Storage 4.3.0.0
Lucene.Net 3.0.3.0
Lucene.Net.Store.Azure 3.0.5553.21100
And calling the following method:
public void UpdateDocument(Term keyTerm, Document document, string indexName)
{
using (var analyser = new StandardAnalyzer(LuceneVersion))
{
using (var directory = new AzureDirectory(cloudStorage.GetStorageAccount(), indexName, new RAMDirectory()))
{
using (var indexWriter = new IndexWriter(directory, analyser, true, IndexWriter.MaxFieldLength.UNLIMITED))
{
indexWriter.UpdateDocument(keyTerm, document);
}
}
}
}
When I call the method even as little as 10 times (from a unit test) the overall time is around 30 seconds.
I have tried various changes with the index writer to see if any performance gains can be made but so far nothing. I have tried changing the code to reuse the index writer and directory classes but I end up with file locks. I also wanted to keep the index code abstracted away from the caller to keep Lucene isolated. If I comment out indexWriter.UpdateDocument(keyTerm, document); then its responsive which tells me this is where the slowness is.
I would like to know if I am doing something wrong or missing something here?

The method above just needed to be adjusted to work better with the resources as opening the directory and index writer for each document was too costly. My adjusted method works fine:
public void UpdateDocumentBatch(Term keyTerm, IEnumerable<Document> documents, string indexName)
{
using (var analyser = new StandardAnalyzer(LuceneVersion))
{
using (var directory = new AzureDirectory(cloudStorage.GetStorageAccount(), indexName, new RAMDirectory()))
{
var createIndex = !IndexReader.IndexExists(directory);
using (var indexWriter = new IndexWriter(directory, analyser, createIndex, IndexWriter.MaxFieldLength.UNLIMITED))
{
indexWriter.SetRAMBufferSizeMB(100);
foreach (var document in documents)
{
keyTerm.Text = document.GetField(keyTerm.Field).StringValue;
indexWriter.UpdateDocument(keyTerm, document);
}
indexWriter.Commit();
}
}
}
}

asp.net caching limit? [duplicate]

This question already has an answer here:
Closed 10 years ago.
Possible Duplicate:
ASP.NET cache maximum size
I'm caching quite a lot of datatables using asp.net caching (the floowing code):
HttpContext.Current.Cache.Insert(GlobalVars.Current.applicationID + "_" + cacheName, itemToCache, null, System.Web.Caching.Cache.NoAbsoluteExpiration, TimeSpan.FromMinutes(240));
However I think that the cache on the server is getting full and having to re-obtain the datatable data from the database. Is there any limit to the amount of data that can be cached on the server or any IIS settings that can be tweaked?

There is a way to upgrade the limit but I would strongly recommend that you use other kind of Caching System (more about this below).
.NET Cache
To know more about the .NET Caching limitation, please read this great answer from a Microsoft .NET Team member.
If you want to see the current limits of .NET Cache, you can try:
var r = new Dictionary<string, string>();
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache % Machine Memory Limit Used", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_MachineMemoryUsed", String.Concat(pc.NextValue().ToString("N1"), "%"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache % Process Memory Limit Used", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_ProcessMemoryUsed", String.Concat(pc.NextValue().ToString("N1"), "%"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Entries", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_Entries", pc.NextValue().ToString("N0"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Misses", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_Misses", pc.NextValue().ToString("N0"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Hit Ratio", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_HitRatio", String.Concat(pc.NextValue().ToString("N1"), "%"));
}
using (var pc = new PerformanceCounter("ASP.NET Applications", "Cache API Trims", true))
{
pc.InstanceName = "__Total__";
r.Add("Total_Trims", pc.NextValue().ToString());
}
MemCached
I'm currently using Memcached, and if you're hosting your site somewhere, you can use a paid service like:
http://www.memcachier.com/
Or, if you're using your own server, you can download Couchbase Community Edition and hosted our own.
You will find more questions here about the use of MemCache, such as:
Which .NET Memcached client do you use, EnyimMemcached vs. BeITMemcached?
how to start with memcached
Make room for any Cache system
To use other cache system without changing your code, you could adopt to create an interface, like
public interface ICacheService
{
T Get<T>(string cacheID, Func<T> getItemCallback) where T : class;
void Clear();
}
then is you're using .NET Cache, your implementation would be something like
public class InMemoryCache : ICacheService
{
private int minutes = 15;
public T Get<T>(string cacheID, Func<T> getItemCallback) where T : class
{
T item = HttpRuntime.Cache.Get(cacheID) as T;
if (item == null)
{
item = getItemCallback();
HttpRuntime.Cache.Insert(
cacheID,
item,
null,
DateTime.Now.AddMinutes(minutes),
System.Web.Caching.Cache.NoSlidingExpiration);
}
return item;
}
public void Clear()
{
IDictionaryEnumerator enumerator = HttpRuntime.Cache.GetEnumerator();
while (enumerator.MoveNext())
HttpRuntime.Cache.Remove(enumerator.Key.ToString());
}
}
and you would use it as:
string cacheId = string.Concat("myinfo-", customer_id);
MyInfo model = cacheProvider.Get<MyInfo>(cacheId, () =>
{
MyInfo info = db.GetMyStuff(customer_id);
return info;
});
if you're using Memcached, all you need to do is create a new class that implement ICacheService and select the class you want, either by using IoC or direct call as:
private ICacheService cacheProvider;
protected override void Initialize(System.Web.Routing.RequestContext requestContext)
{
if (cacheProvider == null) cacheProvider = new InMemoryCache();
base.Initialize(requestContext);
}

The cache uses the memory allocation for the worker process. By default the worker process is allowed to get 60 percent of the machine memory in order to do its work .
As per the link, this can be changed to allow more of the machine memory to be used by the worker process by editing the machine.config file. Presumably you have the cache built to already update when it detects that data is out of date, so this should allow you to put more objects into cache.

When inserting an item into the cache add an CacheItemRemovedCallback method.
In the callback log the reason why the item was removed. This way you see if it memory pressure or something else.
public static void OnRemove(string key,
object cacheItem,
System.Web.Caching.CacheItemRemovedReason reason)
{
AppendLog("The cached value with key '" + key +
"' was removed from the cache. Reason: " +
reason.ToString());
}
http://msdn.microsoft.com/en-us/library/aa478965.aspx

Saving simple user preferences on Windows Forms with C#

I'm writing my first Windows Forms application using VS 2010 and C#. It does not use a database but I would like to save user settings like directory path and what check boxes are checked. What is the easiest way to save these preferences?

I suggest you to use the builtin application Settings to do it. Here is an article talking about it.
Sample usage:
MyProject.Properties.Settings.Default.MyProperty = "Something";

You can use the serializable attribute in conjunction with a 'settings' class. For small amount of information this is really your best bet as it is simple to implement. For example:
[Serializable]
public class MySettings
{
public const string Extension = ".testInfo";
[XmlElement]
public string GUID { get; set; }
[XmlElement]
public bool TurnedOn { get; set; }
[XmlElement]
public DateTime StartTime { get; set; }
public void Save(string filePath)
{
XmlSerializer serializer = new XmlSerializer(typeof(MySettings));
TextWriter textWriter = new StreamWriter(filePath);
serializer.Serialize(textWriter, this);
textWriter.Close();
}
public static MySettings Load(string filePath)
{
XmlSerializer serializer = new XmlSerializer(typeof(MySettings));
TextReader reader = new StreamReader(filePath);
MySettings data = (MySettings)serializer.Deserialize(reader);
reader.Close();
return data;
}
}
There you go. You can prety much cut and paste this directly into your code. Just add properties as needed, and don't forget the [XMLElement] attribute on your interesting properties.
Another benefit to this design is that you don't have to fiddle with the cumbersome Application.Settings approaches, and you can modify your files by hand if you need to.

I'd save the settings in an XML file. That way it's easy for the user to share their settings across machines etc.
You'll also be able to deserialize the XML as a class in your application, giving you easy access to the settings you require.

The easiest way would be in the app.config settings which you can set in the designer under project properties settings (make sure you set them as user setting not application settings or you wont be able to save them) you can then read and write them with C#
to read write just access properties on
Properties.Settings.Default.<your property>
there are also methods to save the properties to the users profile or to reset to defaults
Properties.Settings.Default.Reset();
Properties.Settings.Default.Save();
http://msdn.microsoft.com/en-us/library/a65txexh.aspx

What about adding a local dataset to your project (then create a setting table) and export the data finally to an xml file, its easy to use and more fixable
1- add columns (e.g.; settingName and settingValue) to your local table (DataTable1) using the designer,
then
//data set
DataSet1 ds = new DataSet1();
DataSet1.DataTable1DataTable settingsTable = (DataSet1.DataTable1DataTable)ds.Tables[0];
//add new setting
settingsTable.Rows.Add(new string[] { "setting1", "settingvalue1" });
//export to xml file
settingsTable.WriteXml("settings.xml");
//if you want to read ur settings back... read from xml
settingsTable.ReadXml("settings.xml");