How to cache XML url? - c#

I am retrieving an XML string through a URL. My code works great, but I do not know how to add caching to it. I am not sure if I am able to cache XML streams or if that is even the right approach. What is the best way to add caching here?
XmlTextReader xmlTextReader = new XmlTextReader(this.RssUrl);
XmlDataDocument xdoc1 = new XmlDataDocument();
xdoc1.DataSet.ReadXml(xmlTextReader, XmlReadMode.InferSchema);
return xdoc1.DataSet.Tables["item"];

You could save the XML together with a timestamp, and if you need to reread the XML, you read from the stored XML until the timestamp gets older than a preconfigured value. Most RSS-readers do this to avoid flooding the RSS-service with requests.
Or, if you control the RSSUrl, you could implement the caching there. That would utilize HTTP caching and the fact that the web server can return 304 Not modified if no new items are added to the feed.

Cache the entire XmlDataDocument. If you only cache the XML, you'll have to parse it all the time.

Related

How to read only a small part of a .XML

I built an application in order to read a file, but even with the fact that my connection is fast, the page takes several seconds to load, I would like to know how to read only the first records of this .xml
string rssURL = "http://www.cnt.org.br/Paginas/feed.aspx?t=n";
System.Net.WebRequest myRequest = System.Net.WebRequest.Create(rssURL);
System.Net.WebResponse myResponse = myRequest.GetResponse();
System.IO.Stream rssStream = myResponse.GetResponseStream();
System.Xml.XmlDocument rssDoc = new System.Xml.XmlDocument();
rssDoc.Load(rssStream);
System.Xml.XmlNodeList rssItems = rssDoc.SelectNodes("rss/channel/item");
Tks..
As the fore posters mention you can’t download part of a web request. But you can start parsing Xml before the request finished. Using XmlDocument is the wrong approach for your use case, because it needs the complete request to create the object. Try using XmlTextReader.
There is no easy way to download part of a web request and ensure it is what you want. One workaround would be to use the Google Feed API.
You'd have to use the JSON interface since they don't provide a library for C#, but since it's going through Google's servers it will be much faster. You'd have to modify your code a little bit, since it returns JSON by default instead of XML, but that is a trivial change to make. You can also change the parameter output=xml to retrieve the XML representation of the data.
Try going to this page, that is your same feed, with fewer elements and loads much faster. That only returns a few elements, but if you want 10 elements, all you have to do is add num=10 to the URL. For example, this url has 10 elements. Read the API documentation a little more to see what variables you can add to cater the request to what you want to do.

How to create a component by providing the XML source file as input

I want to create a component by giving XML source input directly using core service 2011, in SDL Tridion 2011.
I want to write code to create a component by uploading source XML. Using the core service 2011.
Say name of the xml file is helloworld.xml and location is D:\abcd\cdef\all\helloworld.xml.
I have written the code like this, but its not working.
XmlDocument contentxml = new XmlDocument();
contentxml.LoadXml(#"D:\abcd\cdef\all\helloworld.xml");
Response.Write("<BR>" + contentxml.ToString());
component.Content = contentxml.ToString();
ComponentData comp = (ComponentData)client.Create(component, new ReadOptions());
The Response.write is displaying nothing. Correct me if I missed any thing.
It's not creating any component and error is coming.
When i tried this:
XmlDocument contentxml = new XmlDocument();
try
{
contentxml.LoadXml(#"D:\abcd\cdef\all\helloworld.xml");
}
catch (XmlException exp)
{
Console.WriteLine(exp.Message);
}
StringWriter sw = new StringWriter();
XmlTextWriter xw = new XmlTextWriter(sw);
contentxml.WriteTo(xw);
Response.Write("<BR>" + sw.ToString());
component.Content = sw.ToString();
ComponentData comp = (ComponentData)client.Create(component, new ReadOptions());
This time it's showing unable to find UUId: some thing like that.
My helloworld.xml looks like this.
<Content xmlns="uuid:1111eb85-0X11-11f9-1e2X-1X82X78fX920">
<first>Hello World.This is Fisrt field</first>
<second>Hello World.This is second field</second>
</Content>
It would be great if some one share some sample code to do it.
Loading any XML from a a file and trying to create a component won't work unless the XML uses the format the CMS is expecting.
The XML structure of a component in SDL Tridion has some fixed parts (Example Nodes Content, Metadata) plus some flexible parts
(The way you define the fields).
First you need to have the XML with the same structure that the CMS is expecting.
Typically the nodes that should be in your xml are the CONTENT and METADATA, load those in an XML Document and use the Core Service API to
create a component using the content contained in those nodes.
The best way to know how which is the structure of a component based on an schema is to create a sample component using the Tridion UI and
see how the XML is constructed. After that you need to create your XML Sources using that structure.
I posted recently an example of how to create a component using the Core Service, please have a look at that.
Faulted State error while creating component with Core Service
Following this code, you can access the nodes Content and Metadata
componentData.Content = xmlUtil.GetNewXmlNode("Content", schemaData.NamespaceUri);
componentData.Metadata = xmlUtil.GetNewXmlNode("Metadata", schemaData.NamespaceUri);
And replace those with your content
The general outline:
Load the XML from the file into an XDocument / XmlDocument.
Create a new Component by calling GetDefaultData on the client.
Set the Content property of the Component to the XML.
Save the Component by calling Save on the client.
If you haven't already, please have a look at the Core Service API documentation available on SDL Tridion World.
If you have trouble implementing this, please post the code that you have and what you have tried in order to make it work.
Using XmlDocument.LoadXML() expects an XML string as input, as commented by Peter you should use XMLDocument.Load() instead, see here for more details http://msdn.microsoft.com/en-us/library/a8ta6tz4.aspx
When you have passed that hurdle you will need the information Miguel gave in his answer to continue.

Read entire elements from an XML network stream

I am writing a network server in C# .NET 4.0. There is a network TCP/IP connection over which I can receive complete XML elements. They arrive regularly and I need to process them immediately. Each XML element is a complete XML document in itself, so it has an opening element, several sub-nodes and a closing element. There is no single root element for the entire stream. So when I open the connection, what I get is like this:
<status>
<x>123</x>
<y>456</y>
</status>
Then some time later it continues:
<status>
<x>234</x>
<y>567</y>
</status>
And so on. I need a way to read the complete XML string until a status element is complete. I don't want to do that with plain text reading methods because I don't know in what formatting the data arrives. I can in no way wait until the entire stream is finished, as is often described elsewhere. I have tried using the XmlReader class but its documentation is weird, the methods don't work out, the first element is lost and after sending the second element, an XmlException occurs because there are two root elements.
Try this:
var settings = new XmlReaderSettings
{
ConformanceLevel = ConformanceLevel.Fragment
};
using (var reader = XmlReader.Create(stream, settings))
{
while (!reader.EOF)
{
reader.MoveToContent();
var doc = XDocument.Load(reader.ReadSubtree());
Console.WriteLine("X={0}, Y={1}",
(int)doc.Root.Element("x"),
(int)doc.Root.Element("y"));
reader.ReadEndElement();
}
}
If you change the "conformance level" to "fragment", it might work with the XmlReader.
This is a (slightly modified) example from MSDN:
XmlReaderSettings settings = new XmlReaderSettings();
settings.ConformanceLevel = ConformanceLevel.Fragment;
XmlReader reader = XmlReader.Create(streamOfXmlFragments, settings);
You could use XElement.Load which is meant more for streaming of Xml Element fragments that is new in .net 3.5 and also supports reading directly from a stream.
Have a look at System.Xml.Linq
I think that you may well still have to add some control logic so as to partition the messages you are receiving, but you may as well give it a go.
I'm not sure there's anything built-in that does that.
I'd open a string builder, fill it until I see a </status> tag, and then parse it using the ordinary XmlDocument.
Not substantially different from dtb's solution, but linqier
static IEnumerable<XDocument> GetDocs(Stream xmlStream)
{
var xmlSettings = new XmlReaderSettings() { ConformanceLevel = ConformanceLevel.Fragment };
using (var xmlReader = XmlReader.Create(xmlStream, xmlSettings))
{
var xmlPathNav = new XPathDocument(xmlReader).CreateNavigator();
foreach (var selectee in xmlPathNav.Select("/*").OfType<XPathNavigator>())
yield return XDocument.Load(selectee.ReadSubtree());
}
}
I ran into a similar problem in PowerShell, but the asker's question was in C#, so I've attempted to translate it (and verified that it works). Here is where I found the clue that got me over the last little bumps (". . .The way the XPathDocument does its magic is by creating a “transparent” root node, and holding the fragments from it. I say it’s transparent because your XPath queries can use the root node axis and still get properly resolved to the fragments. . .")
The fragments of XML I'm working with happen to be smallish. If you had bigger chunks, you'd probably want to look into XStreamingElement - it can add a lot of complexity but also greatly decrease memory usage when dealing with large volumes of XML.

Load Xdocument with RSS feed in MVVM app

I am learning MVVM & Linq to xml by converting a simple wpf RSS app. Originally, I used XmlDataProvider to load local xml files or RSS urls. I am now using the code below which works fine loading local xml files, but throws the "FileNotFoundException" when its a url. During debugging I see the correct url address in (string RSS_URL), yet the file is not found. My initial searching led me to Webclient & HttpWebRequest, but I haven't had any success with them. Am I on the right track? Any code or tutorial links available?
public static List<RSSItem> Load(string RSS_URL)
{
if (!File.Exists(RSS_URL))
{
throw new FileNotFoundException("Datasource file could not be found", RSS_URL);
}
var rssfiles = XDocument.Load(RSS_URL).Descendants("item").Select(
x => new RSSItem(
(string)x.Element("title"),
(string)x.Element("link"),
(string)x.Element("description")));
return rssfiles.ToList();
}
Thank You
XDocument.Load() will accept URLs without any problem. The issue in your code is that you're using File.Exists() to determine whether or not he URL is valid. File.Exists() only accepts a filesystem path, not a uri.
Quick piece of additional info: the Load() method relies on an underlying XmlReader and a call to Create(). If the resource (the URL in this case) doesn't exist, a WebException will be thrown indicating that the resource doesn't exist.
XDocument.Load info:
http://msdn.microsoft.com/en-us/library/bb343181.aspx
XmlReader.Create info:
http://msdn.microsoft.com/en-us/library/w8k674bf.aspx
The XDocument.Load overload you are using is specifically for loading from a file. You would have to download from the RSS feed to get the XML data locally. Look at the MSDN document here for alternatives such as reading from a stream, which might be better suited to what you need.

How to Parse XML in C#

I have managed to read in a saved .csv file on my hard disc and get the data to poputlate a data table. I now want to take that data and put it into XML format, then send the XML to a web service. How can I do this?
Thanks for your time.
One option would be to connect to the Web Service using WCF (or ASP.NET Web Services depending on which .NET version you're using). You can then easily fill the proxy classes the framework creates for you and call the service.
The second option would be to use an XmlTextWriter and use that to build the XML document in a MemoryStream. Once you have the XML document built in memory, you can flush the document and send it out to the Web Service.
A third option would be to use LINQ to XML to build the XML document on the fly. Depending on the structure you parse your CSV file into this may be easier/harder for you than using the XmlTextWriter.
XLinq is excellent for putting the data into XML. If you can use an up to date .NET framework I highly recommend it.
There's some info to get started with here (for example)
http://www.c-sharpcorner.com/UploadFile/mahesh/xLinkDoc06202007130827PM/xLinkDoc.aspx
As for sending that XML to a web service, perhaps you should be calling the web service via a client automatically generated using the Visual Studio ServiceReference tool.
You may or may not need to send XML to that -- in many cases services are object-based (i.e. you'd not need XLinq, provided you can parse the CSV into the correct objects.)
/EDIT:
Rough example for calling webservices using WCF:
using(var client = new ServiceReference1.ThirdPartyServiceClient())
{
client.SendSomething("123", "hello");
string output = client.GetSomething();
Console.WriteLine(output);
}
Following code can also be used.
public static List<Student> convertXMLtoList(string filePath)
{
XDocument doc = XDocument.Load(filePath);
List<Student> studentsMarks = doc.Descendants("Student").Select(x => new Student()
{
RollNo = int.Parse(x.Element("roll_no").Value),
Name = x.Element("name").Value,
}).ToList();
return studentsMarks;
}
Where XML looks like
<?xml version="1.0" encoding="utf-8"?>
<Students>
<Student>
<roll_no>1</roll_no>
<name>XYZ</name>
</Student>
...
</Students>
You can find more details at http://bit.ly/1eveLz3. you can find CSV to XML also on this post.
I haven't done this with a datatable, but it work great for other objects.
Try something like this:
public void writeToXML(DataTable inputData, string fileName) {
XmlSerializer xml = new XmlSerializer(typeof(DataTable));
StreamWriter sw = new StreamWriter(fileName);
xml.Serialize(sw, inputData);
sw.Close();
}
Edit: Just noticed you needed to pass it along to a web service. So instead of StreamWriter, use a memory stream...but same idea.
There are different options depending on how robust of a solution you want. Assuming you want to follow the quickest route you should look at the XmlWriter. Using an XmlWriter you can quickly generate an XML document, save to string and then pass along to your web service.

Categories