We Keep Coding

How can I create multiple XML files in C# when a stored procedure that is called returns a large number of rows?

Currently, I am developing a console application which does the following:
Executes a SQL Server stored procedure to get data like a mobile phone number, etc..
Creates one or multiple XML files with the following catch - there can only be a maximum of 100 elements per file.
So imagine the below:
I have a Car class, and the Car class as the following properties:
Make, Year, Model, Color, Number Of Axles
Then I have a
List<Car> myCars = new List<Car>();
The Car class is now populated by the stored procedure, and in the console application code I use the default constructor, populate the list, etc...
I find that myCars.Count = 200. I want to have two XML files that would look like this:
<!--First xml file-->
<myCars>
<!-- first set of 100 cars-->
</myCars>
Then the second file
<!-- Second xml file-->
<myCars>
<!-- second set of 100 cars from the myCars list -->
</myCars>
I am using the XDocument and the Linq to XML technology. Please help!

Just use XmlSerializer to create xml of your cars. Select in batch of 100 to create new xml file for that set.
Something like this:
XmlSerializer ser = new XmlSerializer(typeof(List<Car>));
int carsPerFile = 100;
int fileIndex = 1;
for(int i=0; i<myCars.Count; )
{
var cars = myCars.Skip(i).Take(carsPerFile);
using (var text = XmlWriter.Create(string.Contact("myCars_",fileIndex++,".xml"))
{
ser.Serialize(text, cars.ToList());
}
i += carsPerFile;
}

You can create partitions from a collection with
int partitionSize = 100;
var partitions = collection
.Select( (e,i) => new { Partition = i / partitionSize, Element = e } )
.ToLookup( e => e.Partition, e => e.Element );
and then serialize each partition into a file
foreach ( var partition in partitions )
{
var filename = string.Format( "file_{0}.xml", partition.Key ));
// write the partition to the file
}

List<Car> firstHundred = myCars.Take(100); // first 100
List<Car> secondHundred = myCars.Skip(100).Take(100) // 100-200
Then serialize these two lists to a different xml file.

DocumentDB filter an array by an array

I have a document that looks essentially like this:
{
"Name": "John Smith",
"Value": "SomethingIneed",
"Tags: ["Tag1" ,"Tag2", "Tag3"]
}
My goal is to write a query where I find all documents in my database whose Tag property contains all of the tags in a filter.
For example, in the case above, my query might be ["Tag1", "Tag3"]. I want all documents whose tags collection contains Tag1 AND Tag3.
I have done the following:
tried an All Contains type linq query
var tags = new List<string>() {"Test", "TestAccount"};
var req =
Client.CreateDocumentQuery<Contact>(UriFactory.CreateDocumentCollectionUri("db", "collection"))
.Where(x => x.Tags.All(y => tags.Contains(y)))
.ToList();
Created a user defined function (I couldn't get this to work at all)
var tagString = "'Test', 'TestAccount'";
var req =
Client.CreateDocumentQuery<Contact>(UriFactory.CreateDocumentCollectionUri("db", "collection"),
$"Select c.Name, c.Email, c.id from c WHERE udf.containsAll([${tagString}] , c.Tags)").ToList();
with containsAll defined as:
function arrayContainsAnotherArray(needle, haystack){
for(var i = 0; i < needle.length; i++){
if(haystack.indexOf(needle[i]) === -1)
return false;
}
return true;
}
Use System.Linq.Dynamic to create a predicate from a string
var query = new StringBuilder("ItemType = \"MyType\"");
if (search.CollectionValues.Any())
{
foreach (var searchCollectionValue in search.CollectionValues)
{
query.Append($" and Collection.Contains(\"{searchCollectionValue}\")");
}
}
3 actually worked for me, but the query was very expensive (more than 2000 RUs on a collection of 10K documents) and I am getting throttled like crazy. My result set for the first iteration of my application must be able to support 10K results in the result set. How can I best query for a large number of results with an array of filters?
Thanks.

The UDF could be made to work but it would be a full table scan and so not recommended unless combined with other highly-selective criteria.
I believe the most performant (index-using) approach would be to split it into a series of AND statements. You could do this programmatically building up your query string (being careful to fully escape and user-provided data for security reasons). So, the resulting query would look like:
SELECT *
FROM c
WHERE
ARRAY_CONTAINS(c.Tags, "Tag1") AND
ARRAY_CONTAINS(c.Tags, "Tag3")

Sitefinity: Dynamic Content query optimization with field values

I will attempt to be as specific as possible. So we are using Sitefinity 8.1.5800, I have a couple dynamic content modules named ReleaseNotes and ReleaseNoteItems. ReleaseNotes has some fields but no reference to ReleaseNoteItems.
Release Note Items has fields and a related data field to ReleaseNotes.
So I can query all ReleaseNoteItems as dynamic content pretty quickly less than a second.
I then use these objects provided by sitefinity and map them to a C# object so I can use strong type. This mapping process is taking almost a minute and using over 600 queries for only 322 items (N+1).
In Short: I need to get all sitefinity objects and Map them to a usable c# object quicker than I currently am.
The method for fetching the dynamic content items (takes milliseconds):
private IList<DynamicContent> GetAllLiveReleaseNoteItemsByReleaseNoteParentId(Guid releaseNoteParentId)
{
DynamicModuleManager dynamicModuleManager = DynamicModuleManager.GetManager(String.Empty);
Type releasenoteitemType = TypeResolutionService.ResolveType("Telerik.Sitefinity.DynamicTypes.Model.ReleaseNoteItems.Releasenoteitem");
string releaseNoteParentTypeString = "Telerik.Sitefinity.DynamicTypes.Model.ReleaseNotes.Releasenote";
var provider = dynamicModuleManager.Provider as OpenAccessDynamicModuleProvider;
int? totalCount = 0;
var cultureName = "en";
Thread.CurrentThread.CurrentUICulture = new CultureInfo(cultureName);
Type releasenoteType = TypeResolutionService.ResolveType("Telerik.Sitefinity.DynamicTypes.Model.ReleaseNotes.Releasenote");
// This is how we get the releasenote items through filtering
DynamicContent myCurrentItem = dynamicModuleManager.GetDataItem(releasenoteType, releaseNoteParentId);
var myMasterParent =
dynamicModuleManager.Lifecycle.GetMaster(myCurrentItem) as DynamicContent;
var relatingItems = provider.GetRelatedItems(
releaseNoteParentTypeString,
"OpenAccessProvider",
myMasterParent.Id,
string.Empty,
releasenoteitemType,
ContentLifecycleStatus.Live,
string.Empty,
string.Empty,
null,
null,
ref totalCount,
RelationDirection.Parent).OfType<DynamicContent>();
IList<DynamicContent> allReleaseNoteItems = relatingItems.ToList();
return allReleaseNoteItems;
}
This is the method that takes almost a minute that is mapping sitefinity object to C# object:
public IList<ReleaseNoteItemModel> GetReleaseNoteItemsByReleaseNoteParent(ReleaseNoteModel releaseNoteItemParent)
{
return GetAllLiveReleaseNoteItemsByReleaseNoteParentId(releaseNoteItemParent.Id).Select(rn => new ReleaseNoteItemModel
{
Id = rn.Id,
Added = rn.GetValue("Added") is bool ? (bool)rn.GetValue("Added") : false,
BugId = rn.GetValue<string>("bug_id"),
BugStatus = rn.GetValue<Lstring>("bugStatus"),
Category = rn.GetValue<Lstring>("category"),
Component = rn.GetValue<Lstring>("component"),
#Content = rn.GetValue<Lstring>("content"),
Criticality = rn.GetValue<Lstring>("criticality"),
Customer = rn.GetValue<string>("customer"),
Date = rn.GetValue<DateTime?>("date"),
Grouped = rn.GetValue<string>("grouped"),
Override = rn.GetValue<string>("override"),
Patch_Num = rn.GetValue<string>("patch_num"),
PublishedDate = rn.PublicationDate,
Risk = rn.GetValue<Lstring>("risk"),
Title = rn.GetValue<string>("Title"),
Summary = rn.GetValue<Lstring>("summary"),
Prod_Name = rn.GetValue<Lstring>("prod_name"),
ReleaseNoteParent = releaseNoteItemParent,
McProductId = GetMcProductId(rn.GetRelatedItems("McProducts").Cast<DynamicContent>()),
}).ToList();
}
Is there any way to optimize this all into one query or a better way of doing this? Taking almost a minute to map this objects is too long for what we need to do with them.
If there is no way we will have to cache the items or make a SQL query. I would rather not do caching or SQL query if I do not have to.
Thank you in advance for any and all help you can provide, I am new to posting questions on stackoverflow so if you need any additional data please let me know.

Is there a reason why you are doing a .ToList() for the items? Is it possible for you to avoid that. In my opinion, most of the time(of the 1 minute) is taken to convert all your items into a list. Conversion from Sitefinity object to C# object is not the culprit here.

Look Arno's answer here: https://plus.google.com/u/0/112295105425490148444/posts/QrsVtxj1sCB?cfem=1
You can use the "Content links manager" to query dynamic modules relationships (both by parent -ParentItemId- or by child -ChildItemId-) much faster:
var providerName = String.Empty;
var parentTitle = "Parent";
var relatedTitle = "RelatedItem3";
DynamicModuleManager dynamicModuleManager = DynamicModuleManager.GetManager(providerName);
Type parentType = TypeResolutionService.ResolveType("Telerik.Sitefinity.DynamicTypes.Model.ParentModules.ParentModule");
Type relatedType = TypeResolutionService.ResolveType("Telerik.Sitefinity.DynamicTypes.Model.RelatedModules.RelatedModule");
ContentLinksManager contentLinksManager = ContentLinksManager.GetManager();
// get the live version of all parent items
var parentItems = dynamicModuleManager.GetDataItems(parentType).Where(i => i.GetValue<string>("Title").Contains(parentTitle) && i.Status == ContentLifecycleStatus.Live && i.Visible);
// get the ids of the related items.
// We use the OriginalContentId property since we work with the live vesrions of the dynamic modules
var parentItemIds = parentItems.Select(i => i.OriginalContentId).ToList();
// get the live versions of all the schedules items
var relatedItems = dynamicModuleManager.GetDataItems(relatedType).Where(i => i.Status == ContentLifecycleStatus.Live && i.Visible && i.GetValue<string>("Title").Contains(relatedTitle));
// get the content links
var contentLinks = contentLinksManager.GetContentLinks().Where(cl => cl.ParentItemType == parentType.FullName && cl.ComponentPropertyName == "RelatedField" && parentItemIds.Contains(cl.ParentItemId) && cl.AvailableForLive);
// get the IDs of the desired parent items
var filteredParentItemIds = contentLinks.Join<ContentLink, DynamicContent, Guid, Guid>(relatedItems, (cl) => cl.ChildItemId, (i) => i.OriginalContentId, (cl, i) => cl.ParentItemId).Distinct();
// get the desired parent items by the filtered IDs
var filteredParentItems = parentItems.Where(i => filteredParentItemIds.Contains(i.OriginalContentId)).ToList();

I would imagine that every release note item under a single release note would be related to the same product wouldn't it?
If so, do you need to do the GetMcProductId method for every item?

How to write XPath expression to select node name from its value

I'm trying to write an XPath expression to select the name of a node from its value in "qualities" and then select in "qualityNames" the value inside node whose name has previously captured.
E.g. In "qualities" - got value "4", take name "rarity3" then in "qualityNames" I got node named "rarity3" and take value "amazingrarity"
<result>
<status>1</status>
<qualities>
<Normal>0</Normal>
<rarity1>1</rarity1>
<rarity2>2</rarity2>
<vintage>3</vintage>
<rarity3>4</rarity3>
<rarity4>5</rarity4>
</qualities>
<qualityNames>
<Normal>Normal</Normal>
<rarity1>Genuine</rarity1>
<rarity2>rarity2</rarity2>
<vintage>Vintage</vintage>
<rarity3>amazingrarity</rarity3>
<rarity4>Unusual</rarity4>
</qualityNames>
</result>
I'm doing this in C# (It's a MVC App) and I'd prefer to use XPath because I'm indexing the XML and I haven't found a fastest way to query in-memory technique (this XML file has ~3MB and I'm using IndexingXPathNavigator).

Use the local-name() and text() functions + predicates. For value "4" it will be
//qualityNames/*[local-name()=local-name(//qualities/*[text() = '4'])]
Tested with http://www.xpathtester.com

Sounds like you want to create a dictionary of key/value pairs (assuming the node names are only needed to find matches and aren't important to your code).
If so, you can use the following:
var doc = XElement.Parse(#"<result>
<status>1</status>
<qualities>
<Normal>0</Normal>
<rarity1>1</rarity1>
<rarity2>2</rarity2>
<vintage>3</vintage>
<rarity3>4</rarity3>
<rarity4>5</rarity4>
</qualities>
<qualityNames>
<Normal>Normal</Normal>
<rarity1>Genuine</rarity1>
<rarity2>rarity2</rarity2>
<vintage>Vintage</vintage>
<rarity3>amazingrarity</rarity3>
<rarity4>Unusual</rarity4>
</qualityNames>
</result>");
var query = from quality in doc.XPathSelectElements("qualities/*")
join qualityName in doc.XPathSelectElements("qualityNames/*")
on quality.Name equals qualityName.Name
select new { Key = quality.Value, Value = qualityName.Value };
var qualities = query.ToDictionary(a => a.Key, a => a.Value);
var quality3 = qualities["3"];
// quality3 == "Vintage"
var quality4 = qualities["4"];
// quality4 == "amazingrarity"
EDIT: example of how to cache this dictionary
// add reference to System.Web dll
public Dictionary<string, string> GetQualities()
{
// assuming this code is in a controller
var qualities = this.HttpContext.Cache["qualities"] as Dictionary<string, string>;
if (qualities == null)
{
// LoadQualitiesFromXml() is the code above
qualities = LoadQualitiesFromXml();
this.HttpContext.Cache["qualities"] = qualities;
}
return qualities;
}

I think this is what you asked
var rarity3ValueInQualities = xml.SelectSingleNode("/result/qualities/rarity3").InnerText;
var rarity3ValueInqualityNames = xml.SelectSingleNode("/result/qualityNames/rarity3").InnerText;

How do I order a sql datasource of uniqueidentifiers in Linq by an array of uniqueindentifiers

I have a string list(A) of individualProfileId's (GUID) that can be in any order(used for displaying personal profiles in a specific order based on user input) which is stored as a string due to it being part of the cms functionality.
I also have an asp c# Repeater that uses a LinqDataSource to query against the individual table. This repeater needs to use the ordered list(A) to display the results in the order specified.
Which is what i am having problems with. Does anyone have any ideas?
list(A)
'CD44D9F9-DE88-4BBD-B7A2-41F7A9904DAC',
'7FF2D867-DE88-4549-B5C1-D3C321F8DB9B',
'3FC3DE3F-7ADE-44F1-B17D-23E037130907'
Datasource example
IndividualProfileId Name JobTitle EmailAddress IsEmployee
3FC3DE3F-7ADE-44F1-B17D-23E037130907 Joe Blo Director dsd#ad.com 1
CD44D9F9-DE88-4BBD-B7A2-41F7A9904DAC Maxy Dosh The Boss 1
98AB3AFD-4D4E-4BAF-91CE-A778EB29D959 some one a job 322#wewd.ocm 1
7FF2D867-DE88-4549-B5C1-D3C321F8DB9B Max Walsh CEO 1

There is a very simple (single-line) way of doing this, given that you get the employee results from the database first (so resultSetFromDatabase is just example data, you should have some LINQ query here that gets your results).
var a = new[] { "GUID1", "GUID2", "GUID3"};
var resultSetFromDatabase = new[]
{
new { IndividualProfileId = "GUID3", Name = "Joe Blo" },
new { IndividualProfileId = "GUID1", Name = "Maxy Dosh" },
new { IndividualProfileId = "GUID4", Name = "some one" },
new { IndividualProfileId = "GUID2", Name = "Max Walsh" }
};
var sortedResults = a.Join(res, s => s, e => e.IndividualProfileId, (s, e) => e);
It's impossible to have the datasource get the results directly in the right order, unless you're willing to write some dedicated SQL stored procedure. The problem is that you'd have to tell the database the contents of a. Using LINQ this can only be done via Contains. And that doesn't guarantee any order in the result set.

Turn the list(A), which you stated is a string, into an actual list. For example, you could use listAsString.Split(",") and then remove the 's from each element. I’ll assume the finished list is called list.
Query the database to retrieve the rows that you need, for example:
var data = db.Table.Where(row => list.Contains(row.IndividualProfileId));
From the data returned, create a dictionary keyed by the IndividualProfileId, for example:
var dic = data.ToDictionary(e => e.IndividualProfileId);
Iterate through the list and retrieve the dictionary entry for each item:
var results = list.Select(item => dic[item]).ToList();
Now results will have the records in the same order that the IDs were in list.