Am doing a windows application in c#, where i read web.config files inside a folder and load the appsettings where users can edit them and apply changes.
I store the settings 'key' and 'value' in a dictionary and the effected values in a separate dictionary . It works well, but it takes lot of time to apply the changes.
How can i speed it up?
here is my code
public List<AppSettings> OldAppSetting;
public List<AppSettings> NewAppSetting;
foreach (var oldSetList in OldAppSetting)
{
Document = Document = XDocument.Load(#oldSetList.FilePathProp);
var appSetting = Document.Descendants("add").Select(add => new
{
Key = add.Attribute("key"),
Value = add.Attribute("value")
}).ToArray();
foreach (var oldSet in appSetting)
{
foreach (var newSet in NewAppSetting)
{
if (oldSet.Key != null)
{
if (oldSet.Key.Value == newSet.AppKey)
{
oldSet.Value.Value = newSet.AppValue;
}
}
Document.Save(#oldSetList.FilePathProp);
}
}
}
here is the Appsettings class
public class AppSettings
{
public string AppKey { get; set; }
public string AppValue { get; set; }
public string FilePathProp{ get; set; }
}
I think your primary speed concern is that you're saving the document after checking every item. Seems like you could change your code to reduce the number of times you call save. For example:
foreach (var oldSetList in OldAppSetting)
{
Document = Document = XDocument.Load(#oldSetList.FilePathProp);
var appSetting = Document.Descendants("add").Select(add => new
{
Key = add.Attribute("key"),
Value = add.Attribute("value")
}).ToArray();
foreach (var oldSet in appSetting)
{
foreach (var newSet in NewAppSetting)
{
if (oldSet.Key != null)
{
if (oldSet.Key.Value == newSet.AppKey)
{
oldSet.Value.Value = newSet.AppValue;
}
}
}
}
Document.Save(#oldSetList.FilePathProp);
}
Also, you could use a Dictionary<string, AppSetting> rather than an array for your appSetting. That would speed things up quite a bit if the number of items is large. It would take some restructuring of your code. I don't know what all of your types are, so I can't give you the exact code, but it would look something like this:
var appSetting = Document.Descendants("add")
.ToDictionary(add => add.Attribute("key"));
foreach (var newSet in NewAppSetting)
{
if (appSetting.ContainsKey(newSet.AppKey))
{
var oldSet = appSetting[newSet.AppKey];
oldSet.Value.Value = newSet.AppValue;
}
}
Your code is a little bit confusing, but I think that's right. The idea here is to build a dictionary of the old values so that we can look them up directly when scanning the new values. It turns your O(n^2) algorithm into an O(n) algorithm, which will make a difference if there are a lot of settings. Plus, the code is smaller and easier to follow.
Put the
Document.Save(#oldSetList.FilePathProp);
Outside the loop!
Related
The following code sample writes a simple object to a couchbase lite (version 2) database and reads all objects afterwards. This is what you can find in the official documentation here
This is quite a lot of manual typing since every property of every object must be transferred to the MutableObject.
class Program
{
static void Main(string[] args)
{
Couchbase.Lite.Support.NetDesktop.Activate();
const string DbName = "MyDb";
var db = new Database(DbName);
var item = new Item { Name = "test", Value = 5 };
// Serialization HERE
var doc = new MutableDocument();
doc.SetString("Name", item.Name);
doc.SetInt("Value", item.Value);
db.Save(doc);
using (var qry = QueryBuilder.Select(SelectResult.All())
.From(DataSource.Database(db)))
{
foreach (var result in qry.Execute())
{
var resultItem = new Item
{
// Deserialization HERE
Name = result[DbName].Dictionary.GetString("Name"),
Value = result[DbName].Dictionary.GetInt("Value")
};
Console.WriteLine(resultItem.Name);
}
}
Console.ReadKey();
}
class Item
{
public string Name { get; set; }
public int Value { get; set; }
}
}
From my research Couchbase lite uses JsonConvert internally, so there might be a way to simplify all that with the help of JsonConvert.
Anything like:
var json = JsonConvert.SerializeObject(item);
var doc = new MutableDocument(json); // No overload to provide raw JSON
or maybe
var data = JsonConvert.SerializeToDict(item); // JsonConvert does not provide this
var doc = new MutableDocument(data);
Is there or is this some kind of optimization and the preferred approach is by intend?
People ask about this quite often, but Couchbase Lite does not actually store JSON strings in the database. They are stored in a different format so this would not give the benefit that you think (the JSON would need to be reparsed and then broken down into the other format). I'd been pushing for a way to serialize classes directly instead of going through dictionary objects (which seems like the ultimate goal here) but our priority is on things that enterprise clients want and this doesn't seem to be one of them. Note that for it to make it in, it needs to be implemented in C# Java and Objective-C / Swift.
I don't know about JsonConvert but there seems to be a constructor that takes IDictionary<string, object> as argument. So I would try something like this (brain-compiled):
MutableDocument CreateDocument(object data)
{
if (data == null) return null;
var propertyValues = new Dictionary<string, object>();
foreach (var property in data.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance))
{
propertyValues[property.Name] = property.GetValue(data);
}
return new MutableDocument(propertyValues);
}
See if this works.
i'm trying to load some data into gridview, sorry if my phrasing is bad or weird, but what i mean exactly is, lets say my text file has this data
Name1,123
Name1,133
Name1,143
Name2,533
Name2,233
Name2,973
Name2,313
Name3,533
Name3,233
Name3,973
Name3,213
...
and i only want to take the last line of Name1, Name2, Name3 and inserting to gridview
Name1,143
Name2,313
Name3,213
subsequently, i'll be trying to update the gridview cells, if further down in the text file there is the same Name again. but yeah. i'm trying to even get this part first.
so i have been trying to use foreach to try to readline and like add them all in and then removing the ones that have the same Name. but it's not removing any of it.
my codes currently is
public class LiveData
{
public string Name { get; set; }
public double Price { get; set; }
public static List<LiveData> LoadDataFromFile(string path)
{
var data = new List<LiveData>();
foreach (var line in File.ReadAllLines(path))
{
var dataLine = line.Split(',');
data.Add(new LiveData
{
Name = dataLine[0],
Price = Convert.ToDouble(dataLine[1])
});
}
return data;
}
}
and
private void btnLoadLDgridView_Click(object sender, EventArgs e)
{
StreamReader file = new StreamReader(tbTSSelectFile.Text);
List<LiveData> list = LiveData.LoadDataFromFile(tbTSSelectFile.Text);
List<LiveData> lst2 = LiveData.LoadDataFromFile(tbTSSelectFile.Text);
while (file.ReadLine() != null)
{
string name = lst2[0].Name;
foreach (LiveData item in list)
{
if (item.Name == name)
{
lst2.Add(item);
}
}
foreach (LiveData item in list)
{
if (item.Name == name)
{
lst2.Remove(item);
}
}
gvLiveData.DataSource = lst2;
}
i'm thinking maybe .Last can be used but i am not sure how to implement it.
the text file data is supposedly Live Stocks data, hence the need to update later on.
thanks in advance :)
After you've parsed your file into the data variable, you could try using Linq:
var lastItems = data.GroupBy(d => d.Name).Select(g => g.Last());
This utilizes the GroupBy and Last operators.
So i have a main directory with sub folders and around 500k images. I know alot of theese images does not exist in my database and i want to know which ones so that i can delete them.
This is the code i have so far:
var listOfAdPictureNames = ImageDB.GetAllAdPictureNames();
var listWithFilesFromImageFolder = ImageDirSearch(adPicturesPath);
var result = listWithFilesFromImageFolder.Where(p => !listOfAdPictureNames.Any(q => p.FileName == q));
var differenceList = result.ToList();
listOfAdPictureNames is of type List<string>
here is my model that im returing from the ImageDirSearch:
public class CheckNotUsedAdImagesModel
{
public List<ImageDirModel> ListWithUnusedAdImages { get; set; }
}
public class ImageDirModel
{
public string FileName { get; set; }
public string Path { get; set; }
}
and here is the recursive method to get all images from my folder.
private List<ImageDirModel> ImageDirSearch(string path)
{
string adPicturesPath = ConfigurationManager.AppSettings["AdPicturesPath"];
List<ImageDirModel> files = new List<ImageDirModel>();
try
{
foreach (string f in Directory.GetFiles(path))
{
var model = new ImageDirModel();
model.Path = f.ToLower();
model.FileName = Path.GetFileName(f.ToLower());
files.Add(model);
}
foreach (string d in Directory.GetDirectories(path))
{
files.AddRange(ImageDirSearch(d));
}
}
catch (System.Exception excpt)
{
throw new Exception(excpt.Message);
}
return files;
}
The problem I have is that this row:
var result = listWithFilesFromImageFolder.Where(p => !listOfAdPictureNames.Any(q => p.FileName == q));
takes over an hour to complete. I want to know if there is a better way to check in my images folder if there are images there that doesn't exist in my database.
Here is the method that get all the image names from my database layer:
public static List<string> GetAllAdPictureNames()
{
List<string> ListWithAllAdFileNames = new List<string>();
using (var db = new DatabaseLayer.DBEntities())
{
ListWithAllAdFileNames = db.ad_pictures.Select(b => b.filename.ToLower()).ToList();
}
if (ListWithAllAdFileNames.Count < 1)
return new List<string>();
return ListWithAllAdFileNames;
}
Perhaps Except is what you're looking for. Something like this:
var filesInFolderNotInDb = listWithFilesFromImageFolder.Select(p => p.FileName).Except(listOfAdPictureNames).ToList();
Should give you the files that exist in the folder but not in the database.
Instead of the search being repeated on each of these lists its optimal to sort second list "listOfAdPictureNames" (Use any of n*log(n) sorts). Then checking for existence by binary search will be the most efficient all other techniques including the current one are exponential in order.
As I said in my comment, you seem to have recreated the FileInfo class, you don't need to do this, so your ImageDirSearch can become the following
private IEnumerable<string> ImageDirSearch(string path)
{
return Directory.EnumerateFiles(path, "*.jpg", SearchOption.TopDirectoryOnly);
}
There doesn't seem to be much gained by returning the whole file info where you only need the file name, and also this only finds jpgs, but this can be changed..
The ToLower calls are quite expensive and a bit pointless, so is the to list when you are planning on querying again so you can get rid of that and return an IEnumerable again, (this is in the GetAllAdPictureNames method)
Then your comparison can use equals and ignore case.
!listOfAdPictureNames.Any(q => p.Equals(q, StringComparison.InvariantCultureIgnoreCase));
One more thing that will probably help is removing items from the list of file names as they are found, this should make the searching of the list quicker every time one is removed since there is less to iterate through.
I have the following piece of code that I use to try to see if copying data from one table to an other missed some records.
There are reasons why this can happen but I won't go into the details here.
Now fortunately, this code runs against a few hundred records at a time, so I can allow myself lo load them into memory and use LINQ to Objects.
As I expected, my code is very slow and I'm wondering if anyone could suggest any way to improve the speed.
void Main()
{
var crossed_data = from kv in key_and_value_table
from ckv in copy_of_key_and_value_table
where kv.key != ckv.key
select new { KeyTable = kv, copyKeyTable = ckv };
List<Key_and_value> difference = new List<Key_and_value>();
foreach (var v in crossed_data)
{
if (crossed_data.Select(s => s.Kv.key).ToList().
Contains(v.ckv.Key) == false)
{
difference.Add(v.ckv);
}
}
}
public class Key_and_value
{
public string Key { get; set; }
public decimal Value { get; set; }
}
many thanks in advance
B
You are doing your Select every iteration when you do not need to. You can move it to the external scope like so.
var keys = crossed_data.Select(s=>s.ckv.key).ToList();
foreach(var v in crossed_data )
{
if (keys.Contains(v.kv.Key) == false)
{
difference.Add(v.Kv);
}
}
This should improve the speed a fair bit.
I have this bit of code in a class:
public class TicketSummary
{
//get all the development tickets
public List<IncidentSummary> AllDevelopmentTickets { get; set; }
public List<string> TicketNames()
{
List<string> v = new List<string>();
foreach (var developmentTicket in AllDevelopmentTickets)
{
var ticketIds = developmentTicket.id.ToString(CultureInfo.InvariantCulture);
v.Add(ticketIds);
}
return v;
}
}
}
And I am trying to see if my API connection (plus all the code) did it's job and pulls back the tickets and their info, more specifically the ids.
In my main program I have no clue how to check if it did the job. I tried something but it isn't quite right and doesn't return anything ( I know I need a Console.WriteLine)
static void Main(string[] args)
{
Console.ReadLine();
var tickets = new TicketSummary();
tickets.TicketNames();
while ( tickets != null )
{
Console.WriteLine(tickets);
}
}
Any suggestions, please?
Thank you!
You've dropped the returned result: tickets.TicketNames(); returns List<String> that you have to assign and then itterate:
var tickets = new TicketSummary();
var names = tickets.TicketNames(); // <- names, List<String> according to the code
// printing out all the names
foreach(var name in names)
Console.WriteLine(name);
Do you mean you just want to print all the tickets out?
foreach (var ticket in tickets.TicketNames())
{
Console.WriteLine(ticket);
}
You have several problems in your code, that should keep it from even compiling, but aside from that, It seem's what you're really after is rather transforming the data in AllDevelopmentTickets, rather than moving it somewhere. So you could probably do it with a Select call (from LINQ). So, in your main method:
var tickets = new TicketSummary();
// add some tickets to tickets.AllDevelopmentTickets here...
var ticketNames = tickets.AllDevelopmentTickets.Select(ticket => ticket.id.ToString();
// Yes, you should probably use an UI culture in the ToString call.
// I'm just trying to limit my line width =)
Now, ticketNames should be an IEnumerable<string> holding all the ticket ids. To, for example, print them out, you can iterate over them and write to console output:
foreach (var name in ticketNames) {
Console.WriteLine(name);
}
You need to assign / use the return value of the TicketNames() method. This seems a lot of work just to return the string version of the TicketId. This can be reduced
public List<string> TicketNames()
{
return AllDevelopmentTickets
.Select(t => t.id.ToString(CultureInfo.InvariantCulture))
.ToList();
}
var ticketSummary = new TicketSummary();
var ticketNames = ticketSummary.TicketNames();
foreach(var ticketName in ticketNames)
{
Console.WriteLine(ticketName);
}
or Even just:
foreach(var ticketName in AllDevelopmentTickets
.Select(t => t.id.ToString(CultureInfo.InvariantCulture)))
{
Console.WriteLine(ticketName);
}
You're ignoring the returned value.
static void Main(string[] args)
{
Console.ReadLine();
var tickets = new TicketSummary();
var res = tickets.TicketNames();
while ( for r in res )
{
Console.WriteLine(r);
}
}