Getting many system out of memory exceptions - c#

My program reads data off of a datatable using linq queries and writes it to a local file. I have the stream and the streamwriter wrapped in using statements but I'm getting many system out of memory exceptions. Am I supposed to call the dispose method on the datatables as well in my code? That is the only solution I can think of
My code:
public static async void getTrainingData(string symbol, string market)
{
decimal returnPct = 0, shortRating = 0, rsi = 0, mfi = 0, williamsR = 0, macd = 0, sma20 = 0, sma50 = 0, sma200 = 0;
string path = "training.csv";
List<Task> taskList = new List<Task>();
int count = 0;
try
{
using (Calculations calc = createCalcClass(symbol, market))
{
using (FileStream fs = new FileStream(path, FileMode.Append, FileAccess.Write, FileShare.Write, 4096, FileOptions.Asynchronous))
using (TextWriter writer = new StreamWriter(fs))
{
DateTime date;
var amexQuery = from c in calc.dailyAmexTable.AsEnumerable()
orderby c.Date descending
select c;
count = Convert.ToInt32(calc.dailyAmexAdapter.ScalarQuerySymbol(symbol));
for (int i = 0; i < count - 14; i++)
{
if (i != count - 15)
{
returnPct = calc.calculateReturnPercentage(amexQuery.ElementAtOrDefault(i + 1).AdjustedClose, amexQuery.ElementAtOrDefault(i).AdjustedClose);
date = amexQuery.ElementAtOrDefault(i + 1).Date;
if (returnPct >= 10)
{
calc.calculatePctGainsLosses(14, date);
shortRating = calculateRating(calc, ratingType.short_rating, date);
rsi = calc.calculateRSI(14, date);
mfi = calc.calculateMFI(14, date);
williamsR = calc.calculateWilliamsR(14, date);
macd = calc.calculateMACDDivergence(date);
sma20 = calc.calculateReturnPercentage(calc.calculateSMA(20, date),
calc.getSpecificValues(Calculations.valueType.Most_Recent_Close, 0, date));
sma50 = calc.calculateReturnPercentage(calc.calculateSMA(50, date),
calc.getSpecificValues(Calculations.valueType.Most_Recent_Close, 0, date));
sma200 = calc.calculateReturnPercentage(calc.calculateSMA(200, date),
calc.getSpecificValues(Calculations.valueType.Most_Recent_Close, 0, date));
Task write = TextWriter.Synchronized(writer).WriteLineAsync(returnPct + "," + symbol + "," + shortRating + "," +
rsi + "," + mfi + "," + williamsR + "," + macd + "," + sma20 + "," + sma50 + "," + sma200);
taskList.Add(write);
// add the necessary info to the training data file
// stock rating, pct change, rsi, mfi, williams r, macd, 20 day sma, 50 day sma, 200 day sma
}
}
}
await writer.FlushAsync();
}
// wait for all tasks to complete
Task.WaitAll(taskList.ToArray());
Console.WriteLine("Return percentages for " + symbol + " saved to training file.");
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
Example output:

Related

Passing the length and the value on a string format

I have the below code that i m reading from datagrid and i export to ascii file and work fine
FileInfo info = new FileInfo(#"C:\SqlExports\");
info.Directory.Create();
string fileName = condition + " " + date + ".txt";
FileStream stream = new FileStream(#"C:\SqlExports\" + fileName, FileMode.OpenOrCreate);
StreamWriter sw = new StreamWriter(stream, Encoding.Default);
for (int i = 0; i < Grid.Rows.Count; i++)
{
List<string> rowprint = new List<string>();
for (int f = 0; f < Grid.Columns.Count; f++)
{
if (Grid.Rows[i].Cells[f].Value != DBNull.Value)
{ rowprint.Add(Grid.Rows[i].Cells[f].Value.ToString()); }
else
{
rowprint.Add(" ");
}
}
sw.Write(string.Format("{0,-19}", rowprint[0]) + String.Format("{0,10}", rowprint[1]) + sw.NewLine, Encoding.Default);
}
sw.Close();
I need to make my code reusable so if the grid populated with other datas to read each time what i need.
i tried
sw.Write(string.Format("0, -(rowprint[0].length + 2)", rowprint[0]) + string.Format("{"0,rowprint[1].Length + 2"} + sw.Newlinn, Encoding.Default);
but of course not working so how can i have the length plus 2 blanks and the value?
of course i know that i need a var with the columns because its time maybe the columns is 2 or 3 or etc.

Accessing Variables by multiple threads in web application [duplicate]

This question already has an answer here:
Asp.net losing values of variables
(1 answer)
Closed 4 years ago.
I am running a web Form where I call a new thread on a button press to execute an Rdotnet snippet of code ( I needed the thread because the current thread kept sending stack overload errors ).
In the Rdotnet function , I have it access a global class variable and put data into it. Once the function completes and the button submit function is over I try to access that global class on another button click ( separate button ), however it is empty. I assume that the thread ends so all data in it ends to, so I tried returning the class and putting it into the global variable in the buttonclick function itself,same result.
I need help.
public partial class BTForm : Page
{
List<DirectorDetail> Details = new List<DirectorDetail>();
BindingList<string> Test = new BindingList<string>();
List<string> Line = new List<string>();
List<string> output = new List<string>();
WebDetails BTlibrary;
OpenFileDialog ofd = new OpenFileDialog();
List<string> Months = new List<string>();
List<string> Year = new List<string>();
BTBill Billfile = new BTBill();
protected void Page_Load(object sender, EventArgs e)
{
if(!IsPostBack)
{
addMonth();
addyear();
foreach (var x in Months)
{
DropDownList1.Items.Add(x);
}
foreach (var y in Year)
{
DropDownList2.Items.Add(y);
}
}
try
{
ListBox4.DataSource = Test;
ListBox4.DataBind();
}
catch(Exception x)
{
Console.Write(x);
}
}
protected void Submit_Click(object sender, EventArgs e)
{
Thread current = Thread.CurrentThread;
WebDetails BTlibrary2 =BTlibrary;
Thread t = new Thread(() => { BTlibrary2 = processes(BTlibrary2); }, 2500000);
t.Start();
t.Join();
BTlibrary = BTlibrary2;
// Thread.Sleep(10000);
}
public WebDetails processes(WebDetails library)
{
if (FileUpLoad1.HasFile)
{
string location = #"C:/BTBill/" + FileUpLoad1.FileName;
Billfile.Tablename = FileUpLoad1.FileName;
try
{
Billfile.Month = DropDownList1.SelectedItem.ToString();
Billfile.Year = DropDownList2.SelectedItem.ToString();
Billfile.Filename = Cleaning.PathCleaning(location);
Billfile.Limit = TextBox3.Text.ToString();
string fpath = Billfile.Month + " " + Billfile.Year + " Query " + "Limit -£ " + Billfile.Limit + "\\";
string filename = Billfile.Month + " " + Billfile.Year + " Query " + "Limit -£ " + Billfile.Limit;
Billfile.Savelocation = "//lonvmfs01/commonwealth_Data/BT BILL Alert/" + filename;
Billfile.Sfilename = "//lonvmfs01/commonwealth_Data/BT BILL Alert/" + filename;
}
catch (Exception x)
{
Error1.Text = "Please Select Month and year\nERROR : " + x;
}
FileUpLoad1.SaveAs(#"C:\BTBill\" + FileUpLoad1.FileName);
library = Executable.executable(Billfile);
// FileUpLoad1.
// = "File Uploaded: " + FileUpLoad1.FileName;
}
else
{
Label1.Text = "No File Uploaded.";
}
DataFrame Director = library.Director;
DataFrame bill = library.Query;
DataFrame limit = library.Btlim;
int colcount = Director.RowCount;
for (int x = 0; x < colcount; x++)
{
string cost_centre = Director[x, 0].ToString();
string director = Director[x, 1].ToString();
string email_address = Director[x, 2].ToString();
double total = SendMail.calculatetotal(bill, limit, cost_centre);
if (total > 0)
{
Line.Add(x + " )\t" + cost_centre + "\t" + director + "\t\t" + email_address + "\t" + total);
Test.Add(x + " )\t" + cost_centre + "\t" + director + "\t\t" + email_address + "\t" + total);
}
}
ListBox4.DataSource = Test;
ListBox4.DataBind();
//foreach (var x in getline().ToArray())
// {
// ListBox4.Items.Add(x);
// }
// ListBox4.DataBind();
return library;
}
protected void Sendmail(object sender, EventArgs e)
{
DataFrame Director = BTlibrary.Director;
DataFrame bill = BTlibrary.Query;
DataFrame limit = BTlibrary.Btlim;
string test;
foreach (Object selecteditem in ListBox4.GetSelectedIndices() )
{
test = ListBox4.Items[(int)selecteditem].Text;
System.Diagnostics.Debug.WriteLine(test);
test = test.Substring(0, 1);
output.Add(test);
}
List<int> index = new List<int>();
for (int y = 0; y < output.Count; y++)
{
index.Add(Convert.ToInt32(output[y]));
}
for (int y = 0; y < index.Count; y++)
{
DirectorDetail temp = new DirectorDetail();
temp.cost_centre = Director[index[y], 0].ToString();
temp.director = Director[index[y], 1].ToString();
temp.email_address = Director[index[y], 2].ToString();
for (int count = 0; count < limit.RowCount; count++)
{
if (limit[count, 0].ToString() == temp.cost_centre)
{
temp.limit = limit[count, 1].ToString();
}
}
Details.Add(temp);
}
SendMail.Mailing(BTlibrary.Query, BTlibrary.Deplist, BTlibrary.Btlim, BTlibrary.Bill, Details);
//Session["Details"] = Details;
// this.Close();
}
private void addyear()
{
DateTime time = System.DateTime.UtcNow;
int baseyear = time.Year - 3;
for (int x = baseyear; x < (baseyear + 10); x++)
{
Year.Add(x.ToString());
}
}
// returns the list of years
public List<string> getYear()
{
return Year;
}
// returns the list of months
public List<String> getMonth()
{
return Months;
}
//adds months to a list
private void addMonth()
{
Months.Add("January");
Months.Add("February");
Months.Add("March");
Months.Add("April");
Months.Add("May");
Months.Add("June");
Months.Add("July");
Months.Add("August");
Months.Add("September");
Months.Add("October");
Months.Add("November");
Months.Add("December");
}
public List<string> getline()
{
return Line;
}
}
public static WebDetails executable(BTBill bill)
{
StartupParameter rinit = new StartupParameter();
rinit.Quiet = true;
rinit.RHome = #"C:\Program Files\R\R-3.4.4\";
rinit.Home = #"C:\R";
REngine.SetEnvironmentVariables();
REngine engine = REngine.GetInstance(null,true,rinit);
// engine.Initialize();
//install and make connection to Postgres
// engine.Evaluate("install.packages('RPostgreSQL') \n install.packages('gridExtra')");
engine.Evaluate("require('RPostgreSQL')" + "\n" +
"pw <- {'admin'}" + "\n" +
"drv <- RPostgreSQL::PostgreSQL()" + "\n" +
"drv <- dbDriver('PostgreSQL')" +"\n"+
"con <- dbConnect(drv, dbname = 'postgres'," +"\n"+
"host = 'localhost', port = 5432," +"\n"+
"user = 'postgres', password = pw)");
engine.Evaluate("postgresmfile<- dbGetQuery(con,'select * from masterfile')" + "\n" +
"Contact <- dbGetQuery(con, 'select * from contact')"+"\n"+
"btlim<- dbGetQuery(con, ' select * from bt_departmentlimit')"+"\n"+
"dbDisconnect(con)");
engine.Evaluate("BTBill = read.csv(file<-'"+bill.Filename+"', header=TRUE, sep=',',skip=1)");
// building dataframes and queries
DataFrame BTBill = engine.Evaluate("BTBill").AsDataFrame();
DataFrame MasterFile = engine.Evaluate("postgresmfile").AsDataFrame();
DataFrame BTLim = engine.Evaluate("btlim").AsDataFrame();
DataFrame Contact= engine.Evaluate("Contact ").AsDataFrame();
DataFrame Query = engine.Evaluate("Merged <- merge(BTBill,postgresmfile,by.x='SERVICE.NO',by.y = 'service_number')" + "\n"+ "Merged_2 <- merge(Merged,Contact,by.x='cost_centre',by.y='cost_centre') " + "\n"+
"query <- Merged_2[c('SERVICE.NO','username','cost_centre','job_post','USAGE.CHARGES','TOTAL.COST','USAGE.START.DATE','USAGE.END.DATE','director','email_address')]").AsDataFrame();
DataFrame Merge2 = engine.Evaluate("Merged_2").AsDataFrame();
DataFrame maillist = engine.Evaluate("data.frame(query)" +"\n"+
"test <-subset(query, TOTAL.COST >= "+bill.Limit+ ", select = c(SERVICE.NO,username,cost_centre,job_post, TOTAL.COST, USAGE.START.DATE, USAGE.END.DATE,director,email_address,USAGE.CHARGES))").AsDataFrame();
DataFrame DepList = engine.Evaluate("x<-test[c('cost_centre','director','email_address')]" + "\n" +
"ux<-unique(x) ").AsDataFrame();
DataFrame DepList2=engine.Evaluate("y <-query[c('cost_centre', 'director', 'email_address')]" + "\n" +
"uy<-unique(y) ").AsDataFrame();
engine.Evaluate("dir.create(file.path('" + bill.Savelocation + "'))");
//creating pdf files for each department head
engine.Evaluate("write.csv(Merged_2, file = '" + bill.Savelocation + "/MasterFile.csv');");
for (int count = 0; count < DepList.RowCount; count++)
{
DataFrame temp = engine.Evaluate("data.frame(query);" +
"test2 <-subset(query, USAGE.CHARGES >= " + bill.Limit + " & cost_centre=='"+DepList[count,0]+"', select = c(SERVICE.NO,username,cost_centre,job_post, USAGE.CHARGES, USAGE.START.DATE, USAGE.END.DATE,director,email_address))").AsDataFrame();
engine.Evaluate("library(gridExtra);");
engine.Evaluate("pdf('" + bill.Sfilename +"/"+ DepList[count, 0] + ".pdf', height=20, width=20);" );
try
{
engine.Evaluate("grid.table(test2); dev.off() ;");
}
catch (Exception e)
{
}
}
SendMailForm form = new SendMailForm();
WebDetails web = new WebDetails(DepList2, Query, BTLim);
web.Deplist = DepList;
web.Bill = bill;
engine.Evaluate("write.csv(test, file = '" + bill.Savelocation + "/Users over threshold.csv')");
engine.Dispose();
return web;
// form.Director=DepList2;
//form.bill = Query;
//form.limit = BTLim;
List<DirectorDetail> output = form.Details;
SendMail.Mailing(Query, DepList,BTLim, bill,output);
// to filter by department
// DataFrame maillist = engine.Evaluate("data.frame(query)" + "\n" +
// "test <-subset(query, TOTAL.COST >= " + bill.Limit + "& Cost.Centre=='"+bill.Dep+"', select = c(SERVICE.NO, User.Name, Cost.Centre, ROLE.JOB.POST, TOTAL.COST, USAGE.START.DATE, USAGE.END.DATE,DIRECTOR,EMAIL.ADDRESS))").AsDataFrame();
//engine.Evaluate("install.package('dplyr')");
}
}
}
It seems my problem turned out to be unrelated to variable sharing through threads, but instead to deal with the way button click instances worked.
long story short, in order to share a variable between two event instances, the easiest way to do it is through Sessions.
I needed to put the BTlibrary in a session and access it in the second button click event for it to get the value saved from the previous.
protected void Submit_Click(object sender, EventArgs e)
{
Thread current = Thread.CurrentThread;
WebDetails BTlibrary2 =BTlibrary;
Thread t = new Thread(() => { BTlibrary2 = processes(BTlibrary2); }, 2500000);
t.Start();
t.Join();
Session["BTLib"] = BTlibrary2;
// Thread.Sleep(10000);
}
protected void Sendmail(object sender, EventArgs e)
{
List<DirectorDetail> Details = new List<DirectorDetail>();
BTlibrary = (WebDetails) Session["BTLib"];
this worked

Why when giving files names numbers to the names using a counter it's giving it strange numbers?

The first saved file name on the hard disk is: 201701311645---0 then 201701311645---1 then 201701311645---20 then 201701311645---21 then 201701311645---40 and 201701311645---41
But i want it to be saved as: 201701311645---0 then 201701311645---1 then 201701311645---2 then 201701311645---3 then 201701311645---4 and 201701311645---5
In the top i added a counter variable
private int countFilesNames = 0;
Then in a dowork event i also reset the counter to 0 once so if i start the backgroundworker over again it will start from 0.
private void bgwDownloader_DoWork(object sender, DoWorkEventArgs e)
{
Int32 fileNr = 0;
countFilesNames = 0;
if (this.SupportsProgress) { calculateFilesSize(); }
if (!Directory.Exists(this.LocalDirectory)) { Directory.CreateDirectory(this.LocalDirectory); }
while (fileNr < this.Files.Count && !bgwDownloader.CancellationPending)
{
m_fileNr = fileNr;
downloadFile(fileNr);
if (bgwDownloader.CancellationPending)
{
fireEventFromBgw(Event.DeletingFilesAfterCancel);
cleanUpFiles(this.DeleteCompletedFilesAfterCancel ? 0 : m_fileNr, this.DeleteCompletedFilesAfterCancel ? m_fileNr + 1 : 1);
}
else
{
fileNr += 1;
}
}
}
Then in the downloadFile method
private void downloadFile(Int32 fileNr)
{
FileStream writer = null;
m_currentFileSize = 0;
fireEventFromBgw(Event.FileDownloadAttempting);
FileInfo file = this.Files[fileNr];
Int64 size = 0;
Byte[] readBytes = new Byte[this.PackageSize];
Int32 currentPackageSize;
System.Diagnostics.Stopwatch speedTimer = new System.Diagnostics.Stopwatch();
Int32 readings = 0;
Exception exc = null;
try
{
writer = new FileStream(this.LocalDirectory + "\\" + file.Name +
"---" + countFilesNames + ".png", System.IO.FileMode.Create);
}
catch(Exception err)
{
string ggg = err.ToString();
}
HttpWebRequest webReq;
HttpWebResponse webResp = null;
try
{
webReq = (HttpWebRequest)System.Net.WebRequest.Create(this.Files[fileNr].Path);
webResp = (HttpWebResponse)webReq.GetResponse();
size = webResp.ContentLength;
}
catch (Exception ex) { exc = ex; }
m_currentFileSize = size;
fireEventFromBgw(Event.FileDownloadStarted);
if (exc != null)
{
bgwDownloader.ReportProgress((Int32)InvokeType.FileDownloadFailedRaiser, exc);
}
else
{
m_currentFileProgress = 0;
while (m_currentFileProgress < size && !bgwDownloader.CancellationPending)
{
while (this.IsPaused) { System.Threading.Thread.Sleep(100); }
speedTimer.Start();
currentPackageSize = webResp.GetResponseStream().Read(readBytes, 0, this.PackageSize);
m_currentFileProgress += currentPackageSize;
m_totalProgress += currentPackageSize;
fireEventFromBgw(Event.ProgressChanged);
writer.Write(readBytes, 0, currentPackageSize);
readings += 1;
if (readings >= this.StopWatchCyclesAmount)
{
m_currentSpeed = (Int32)(this.PackageSize * StopWatchCyclesAmount * 1000 / (speedTimer.ElapsedMilliseconds + 1));
speedTimer.Reset();
readings = 0;
}
}
speedTimer.Stop();
writer.Close();
webResp.Close();
if (!bgwDownloader.CancellationPending) { fireEventFromBgw(Event.FileDownloadSucceeded); }
}
fireEventFromBgw(Event.FileDownloadStopped);
countFilesNames += 1;
}
I build the file name:
writer = new FileStream(this.LocalDirectory + "\\" + file.Name +
"---" + countFilesNames + ".png", System.IO.FileMode.Create);
The move the counter forward by 1:
countFilesNames += 1;
But i'm getting other files names then i wanted.
Maybe there is a better way to give the files names some identity ? The problem is that if i will not give the files names some identity it will overwrite the files all the time. The files names are the same the content is not so i need to give each file another name.
Why don't you just increment the counter only when a file is written (since the variable doesn't look like it is accessed elsewhere) and not below:
writer = new FileStream(this.LocalDirectory + "\\" + file.Name +
"---" + countFilesNames++ + ".png", System.IO.FileMode.Create);
This way the counter won't be incremented on errors.

Reading txt files using task concept working slow

my requirement was reading some 300 text files each of around 40-70 MB line by line to perform some kind of checks . As the files are huge so I thought of using TPL concept i.e. tasks. if I am not using task concept, it is taking around 7 minutes but if I am doing it by using task , it is taking long time don't know why. I will be so thankful if someone please review my code and tell me where I am doing wrong in using task. Any help would be appreciated. Below is my code :
private void browse_Click(object sender, EventArgs e)
{
try
{
string newFileName1 = "";
string newFileName2 = "";
week = textBox2.Text;
if (week == null || week == "")
{
MessageBox.Show("Week cannot be null.");
return;
}
DialogResult result = folderBrowserDialog1.ShowDialog();
if (result == DialogResult.OK)
{
DateTime starttime = DateTime.Now;
string folderPath = Path.GetDirectoryName(folderBrowserDialog1.SelectedPath);
string folderName = Path.GetFileName(folderPath);
DirectoryInfo dInfo = new DirectoryInfo(folderPath);
foreach (DirectoryInfo folder in dInfo.GetDirectories())
{
newFileName1 = "Files_with_dates_mismatching_the_respective_week_" + folder.Name + ".txt";
newFileName2 = "Files_with_wrong_date_format_" + folder.Name + ".txt";
if (File.Exists(folderPath + "/" + newFileName1))
{
File.Delete(folderPath + "/" + newFileName1);
}
if (File.Exists(folderPath + "/" + newFileName2))
{
File.Delete(folderPath + "/" + newFileName2);
}
FileInfo[] folderFiles = folder.GetFiles();
if (folderFiles.Length != 0)
{
List<Task> tasks = new List<Task>();
foreach (var file in folderFiles)
{
var task = Task.Factory.StartNew(() =>
{
bool taskResult = ReadFile(file.FullName, folderPath, folderName, week);
return taskResult;
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
DateTime stoptime = DateTime.Now;
TimeSpan totaltime = stoptime.Subtract(starttime);
label6.Text = Convert.ToString(totaltime);
textBox1.Text = folderPath;
DialogResult result2 = MessageBox.Show("Read the files successfully.", "Important message", MessageBoxButtons.OK, MessageBoxIcon.Information);
}
}
}
}
catch (Exception)
{
throw;
}
}
public bool ReadFile(string file, string folderPath, string folderName, string week)
{
int LineCount = 0;
string fileName = Path.GetFileNameWithoutExtension(file);
using (FileStream fs = File.Open(file, FileMode.Open))
using (BufferedStream bs = new BufferedStream(fs))
using (StreamReader sr = new StreamReader(bs))
{
for (int i = 0; i < 2; i++)
{
sr.ReadLine();
}
string oline;
while ((oline = sr.ReadLine()) != null)
{
LineCount = ++LineCount;
string[] eachLine = oline.Split(';');
string date = eachLine[30].Substring(1).Substring(0, 10);
DateTime dt;
bool valid = DateTime.TryParseExact(date, "dd/MM/yyyy", CultureInfo.InvariantCulture, DateTimeStyles.None, out dt);
if (!valid)
{
Filecount = ++Filecount;
StreamWriter sw = new StreamWriter(folderPath + "/" + "Files_with_wrong_date_format_" + folderName + ".txt", true);
sw.WriteLine(fileName + " " + "--" + " " + "Line number :" + " " + LineCount);
sw.Close();
}
else
{
DateTime Date = DateTime.ParseExact(date, "d/M/yyyy", CultureInfo.InvariantCulture);
int calculatedWeek = new GregorianCalendar(GregorianCalendarTypes.Localized).GetWeekOfYear(Date, CalendarWeekRule.FirstFourDayWeek, DayOfWeek.Saturday);
if (calculatedWeek == Convert.ToInt32(week))
{
}
else
{
Filecount = ++Filecount;
StreamWriter sw = new StreamWriter(folderPath + "/" + "Files_with_dates_mismatching_the_respective_week_" + folderName + ".txt", true);
sw.WriteLine(fileName + " " + "--" + " " + "Line number :" + " " + LineCount);
sw.Close();
}
}
}
}
return true;
}
Since you are using Task.WaitAll(tasks.ToArray()). Wait will synchronously block until the task completes. But can you try with await (Not in TPL), await will asynchronously wait until the task completes

Verify BouncyCastle ECDsa signature with .NET libraries ECDsaCng

An existing system generated signatures using Bouncy Castle (.NET) and I need to verify these existing signatures using the Microsoft ECDsaCng class.
Consider the following code that attempts to do this:
public static void InterchangeTest()
{
//AsymmetricCipherKeyPair bKeyPair_0 = Crypto.GenerateEcdsaKey();
String sPassPhrase = "bob is your uncle";
byte[] bPassPhrase = new UTF8Encoding(false).GetBytes(sPassPhrase);
int SaltBitSize = 128;
int EcdsaBitLength = 521;
byte[] bSalt = new byte[SaltBitSize / 8];
new SecureRandom().NextBytes(bSalt);
if (EcdsaBitLength != 192 && EcdsaBitLength != 256 && EcdsaBitLength != 521)
{
throw new ArgumentException("Invalid EcdsaBitLength length () " + EcdsaBitLength + " ECDSA supports 192, 256, and 521 bit lengths.");
}
string curveName = "P-" + EcdsaBitLength.ToString();
X9ECParameters ecP = NistNamedCurves.GetByName(curveName);
ECDomainParameters ecSpec = new ECDomainParameters(ecP.Curve, ecP.G, ecP.N, ecP.H, ecP.GetSeed());
IAsymmetricCipherKeyPairGenerator g = GeneratorUtilities.GetKeyPairGenerator("ECDH");
g.Init(new ECKeyGenerationParameters(ecSpec, new SecureRandom()));
AsymmetricCipherKeyPair aKeyPair = g.GenerateKeyPair();
ECPrivateKeyParameters mPrivateKey = (ECPrivateKeyParameters)aKeyPair.Private;
ECPublicKeyParameters mPublicKey = (ECPublicKeyParameters)aKeyPair.Public;
byte[] bPrivateKey = ((ECPrivateKeyParameters)aKeyPair.Private).D.ToByteArray();
String SignerName = "SHA-256withECDSA";
ISigner bSigner = SignerUtilities.GetSigner(SignerName);
bSigner.Init(true, aKeyPair.Private);
bSigner.BlockUpdate(bPassPhrase, 0, bPassPhrase.Length);
byte[] bouncySignature = bSigner.GenerateSignature();
ISigner bVerifier = SignerUtilities.GetSigner(SignerName);
bVerifier.Init(false, aKeyPair.Public);
bVerifier.BlockUpdate(bPassPhrase, 0, bPassPhrase.Length);
bool passed = bVerifier.VerifySignature(bouncySignature);
Console.WriteLine("Verified with Bouncy: " + passed);
var xmlImport = "<ECDSAKeyValue xmlns='http://www.w3.org/2001/04/xmldsig-more#'>\n"
+ " <DomainParameters>\n"
+ " <NamedCurve URN='urn:oid:1.3.132.0.35' />\n"
+ " </DomainParameters >\n"
+ " <PublicKey >\n"
+ " <X Value='" + ((ECPublicKeyParameters)aKeyPair.Public).Q.X.ToBigInteger().ToString() + "' xsi:type='PrimeFieldElemType' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' />\n"
+ " <Y Value='" + ((ECPublicKeyParameters)aKeyPair.Public).Q.Y.ToBigInteger().ToString() + "' xsi:type='PrimeFieldElemType' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' />\n"
+ " </PublicKey >\n"
+ " </ECDSAKeyValue>";
//using (StreamWriter outputFile = new StreamWriter(#"C:\Dev\x2.txt"))
//{
// outputFile.WriteLine(xmlImport);
//}
ECDsaCng eccImporter = new ECDsaCng();
eccImporter.FromXmlString(xmlImport, ECKeyXmlFormat.Rfc4050);
Console.WriteLine("hash algorithm = " + eccImporter.HashAlgorithm + " Probably " + CngAlgorithm.Sha256);
Console.WriteLine("Signature algorithm = " + eccImporter.SignatureAlgorithm + " Probably ECDsa");
Console.WriteLine("After import, key size = " + eccImporter.KeySize + " probably 521");
try
{
if (eccImporter.VerifyData(bPassPhrase, bouncySignature))
{
Console.WriteLine("Verified the signature from bouncy castle using .NET");
}
}
catch (CryptographicException e)
{
// "The parameter is incorrect"
Console.WriteLine("Did not verify bouncy signature with .NET because: " + e.Message);
}
CngKey msKey = CngKey.Create(CngAlgorithm.ECDsaP521, null, new CngKeyCreationParameters { ExportPolicy = CngExportPolicies.AllowPlaintextArchiving });
ECDsaCng ms_ecdsaCNG = new ECDsaCng(msKey);
String xmlExport = ms_ecdsaCNG.ToXmlString(ECKeyXmlFormat.Rfc4050);
eccImporter = new ECDsaCng();
eccImporter.FromXmlString(xmlExport, ECKeyXmlFormat.Rfc4050);
byte[] ms_SignedData = ms_ecdsaCNG.SignData(bPassPhrase);
Console.WriteLine("Verify .NET signature with .NET: " + ms_ecdsaCNG.VerifyData(bPassPhrase, ms_SignedData));
Console.WriteLine("Verify .NET signature with imported .NET: " + eccImporter.VerifyData(bPassPhrase, ms_SignedData));
//Console.WriteLine();
//Console.WriteLine(xmlExport);
//Console.WriteLine();
}
Everything works fine until I attempt to verify the signature in with the Microsoft classes, at which point it generates an exception stating that the Parameter is incorrect.
any thoughts?
Maarten Bodewes is correct. My problem is that the signature is encoded using BouncyCastly using ASN.1/DER format. MS uses an smaller format (I think that it is IEEE P-1393). So, I wrote this little routine in C# to transform the signatures.
public static byte[] ConvertDerToP1393(byte[] data)
{
byte[] b = new byte[132];
int totalLength = data[1];
int n = 0;
int offset = 4;
int thisLength = data[offset++];
if (data[offset] == 0) {
// Negative number!
++offset;
--thisLength;
}
for (int i= thisLength; i < 66; ++i) {
b[n++] = 0;
}
if (thisLength > 66) {
System.Console.WriteLine("BAD, first number is too big! " + thisLength);
} else {
for (int i = 0; i < thisLength; ++i) {
b[n++] = data[offset++];
}
}
++offset;
thisLength = data[offset++];
for (int i = thisLength; i < 66; ++i){
b[n++] = 0;
}
if (thisLength > 66) {
System.Console.WriteLine("BAD, second number is too big! " + thisLength);
} else {
for (int i = 0; i < thisLength; ++i) {
b[n++] = data[offset++];
}
}
return b;
}
Although I have done only limited testing, the code has worked with my tests.
I convert private Key to public, with VB:
Dim Point As Org.BouncyCastle.Math.EC.ECPoint = PrivateKey.Parameters.G.Multiply(PrivateKey.D)
Dim ToPublic = New Org.BouncyCastle.Crypto.Parameters.ECPublicKeyParameters(PrivateKey.AlgorithmName, Point, PrivateKey.PublicKeyParamSet)

Categories