After converting multiple MapInfo files into a unique Shapefile, and then converting that file again to .KML, I got the following .XML file. My idea is to extract each set of 'coordinates' sections, and build polygons using them.
Other attempted solution:
Given the excessive time facing this blockage, I tried obtaining each pair of 'coordinates' tags and using Substring to get the coordinates. Unfortunately given the size of the file (>400 MB) this dirty approach is not practical.
Xml file
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<Document id="root_doc">
<Schema id="PruebaKML4g.schema">
<SimpleField name="FID" type="float"/>
<SimpleField name="REGION" type="float"/>
<SimpleField name="NOMBRE" type="string"/>
<SimpleField name="layer" type="string"/>
<SimpleField name="path" type="string"/>
</Schema>
<Document id="PruebaKML4g">
<name>PruebaKML4g</name>
<Placemark id="PruebaKML4g.1">
<ExtendedData>
<SchemaData schemaUrl="#PruebaKML4g.schema">
<SimpleData name="FID">5</SimpleData>
<SimpleData name="REGION">1</SimpleData>
<SimpleData name="NOMBRE">BAJA CALIFORNIA</SimpleData>
<SimpleData name="layer">LBS_REGION_1_region</SimpleData>
<SimpleData name="path">C:/Files/LBS_REGION_1_region.shp</SimpleData>
</SchemaData>
</ExtendedData>
<MultiGeometry>
<Polygon>
<outerBoundaryIs>
<LinearRing>
<coordinates>
-105.258751,21.782028,0
-105.247174,21.81173,0
-105.241826,21.809401,0
-105.236994,21.806241,0
-105.232822,21.802344,0
-105.229439,21.79783,0
-105.228552,21.796052,0
-105.228974,21.795899,0
-105.230294,21.79522,0
-105.231872,21.79511,0
-105.234048,21.79431,0
-105.235131,21.794083,0
-105.236824,21.793857,0
-105.238518,21.793295,0
-105.239365,21.792389,0
-105.240327,21.790914,0
-105.242379,21.79046,0
-105.243829,21.790459,0
-105.245644,21.788766,0
-105.247331,21.785709,0
-105.24817,21.783115,0
-105.248701,21.780372,0
-105.258751,21.782028,0
</coordinates>
</LinearRing>
</outerBoundaryIs>
</Polygon>
...
<Polygon>
<outerBoundaryIs>
<LinearRing>
<coordinates>
-103.704559,20.767933,0
-103.702714,20.773608,0
-103.701694,20.77322,0
-103.700762,20.772672,0
-103.699944,20.77198,0
-103.699267,20.771165,0
-103.698751,20.770252,0
-103.698411,20.769268,0
-103.698258,20.768243,0
-103.698297,20.76721,0
-103.704559,20.767933,0
</coordinates>
</LinearRing>
</outerBoundaryIs>
</Polygon>
<Polygon>
<outerBoundaryIs>
<LinearRing>
<coordinates>
-105.160778,20.766278,0
-105.162411,20.77201,0
-105.161328,20.77219,0
-105.160228,20.77219,0
-105.159145,20.77201,0
-105.158111,20.771656,0
-105.157159,20.771139,0
-105.156317,20.770474,0
-105.15561,20.769682,0
-105.15506,20.768786,0
-105.160778,20.766278,0
</coordinates>
</LinearRing>
</outerBoundaryIs>
</Polygon>
<Polygon>
<outerBoundaryIs>
<LinearRing>
<coordinates>
-117.125814,32.524285,0
-117.125516,32.524512,0
-117.125142,32.524428,0
-117.124876,32.524169,0
-117.124754,32.524513,0
-117.124784,32.525361,0
</coordinates>
</LinearRing>
</outerBoundaryIs>
</Polygon>
</MultiGeometry>
</Placemark>
</Document>
</Document>
</kml>
I tried to use the following code:
Main
public T DeserializeToObject<T>(string filepath) where T : class
{
System.Xml.Serialization.XmlSerializer xmlSerializer = new System.Xml.Serialization.XmlSerializer(typeof(T));
using (StreamReader streamReader = new StreamReader(filepath))
{
return (T)xmlSerializer.Deserialize(streamReader);
}
}
private void Form1_Load(object sender, EventArgs e)
{
String pathKml = #"C:\PruebaKML4g.kml";
List<Kml> elementsList = DeserializeToObject<List<Kml>>(pathKml);
}
Kml.cs
/*[XmlRoot(ElementName = "kml")] changed by Mike Clark suggestion*/
[XmlRoot(ElementName = "kml", Namespace = "http://www.opengis.net/kml/2.2")]
public class Kml
{
public List<Polygon> polygons = new List<Polygon>();
}
public class Polygon
{
[XmlAttribute("outerBoundaryIs")]
public String outerBoundaryIs { get; set; }
[XmlAttribute("linearRing")]
public String linearRing { get; set; }
[XmlAttribute("coordinates")]
public String coordinates { get; set; }
}
However, the SimpleData elements in the XML file appear to be interfering with my parsing, generating the following error
InvalidOperationException: xmlns='http://www.opengis.net/kml/2.2'> was not expected.
Any clue about where my mistake is will be appreciated.
With all those conversion steps, perhaps the XML file is malformed? Or maybe a memory error with so much data? Try parsing the file with a low-memory requirement SAX parser to let it find any syntactical errors that might be buried deep in the file. Do you have Python installed?
python -c "import xml.sax;p=xml.sax.make_parser();p.parse(r'yourfile.xml')"
Change yourfile.xml to the correct path and filename of your XML file.
If it prints nothing, the file is syntactically valid. If it prints an error, try to use the line:column info in the error to spot the error in your XML.
Part 2:
List<Kml> elementsList = DeserializeToObject<List<Kml>>(pathKml);
might be wrong. XML documents can have one and only one root <element> (in this case, <kml>) so I think having a list of Kml instances will not make make sense for the parser. Try this instead:
Kml root = DeserializeToObject<Kml>(pathKml);
But that is a simple problem compared to the next problem, which is that I think your C# class structure needs to mirror exactly the hierarchical structure of the XML. The polygons are under this hierarchy:
kml > Document > Document > Placemark > MultiGeometry
which means you would need something like
class Kml {
Document Document;
}
class Document {
Document Document;
Placemark Placemark;
}
class Placemark {
Polygon[] MultiGeometry;
}
class Polygon {
OuterBoundaryIs outerBoundaryIs;
}
class OuterBoundaryIs {
LinearRing LinearRing;
}
class LinearRing {
string coordinates;
}
Then you would need something like
var polygons = kml.Document.Document.Placemark.MultiGeometry;
for(int i = 0; i < polygons.Length; i++) {
var polygon = polygons[i];
string coordinates = polygon.outerBoundaryIs.LinearRing.coordinates;
// do something with coordinates
}
By the way, a better kind of parser for this type of thing would be an XPath parser which can avoid the need to model the XML structure with C# classes. It takes a little practice and research to craft an XPath query, but the resulting code is cleaner, and it's a good skill to have some experience with. More XPath see:
https://stackoverflow.com/a/16012736/11611195
https://learn.microsoft.com/en-us/dotnet/standard/data/xml/select-nodes-using-xpath-navigation
My friend wrote this piece of XML code for simple testing in our game when it comes to loading and writing save data. The problem is that his code does not work at all and I have never wrote any piece of XML code before so this is new for me and I decided to learn XML.
All it wants is the x and y values to create a new Tile object.
When running the code it gives the error:
System.NullReferenceException was unhandled
HResult=-2147467261
Message=Object reference not set to an instance of an object.
TL;DR It loads simple Tiles and all it's looking for is the x and y position to later add to a list that is accessed in the Level class.
The code he wrote:
public class XmlHandler
{
private List<Base.Tile> tiles;
public XmlHandler()
{
}
public void Load()
{
XmlDocument documentFile = new XmlDocument();
documentFile.Load(#"C:\Tiles\0.xml");
var listOfTiles = documentFile.GetElementById("tiles").GetElementsByTagName("tile");
foreach(XmlElement tile in listOfTiles)
{
var x = ((XmlElement)tile.GetElementsByTagName("position")[0]).GetAttribute("x");
var y = ((XmlElement)tile.GetElementsByTagName("position")[0]).GetAttribute("y");
Classes.Base.Tile t = new Base.Tile(new Vector2(float.Parse(x), float.Parse(y)));
this.tiles.Add(t);
}
}
public List<Base.Tile> GetTiles()
{
return this.tiles;
}
}
The current XML file, modified from the original as shown below
<tiles>
<tile>
<position x="10" y="20" />
</tile>
<tile>
<position x="50" y="20" />
</tile>
<tile>
<position x="30" y="40" />
</tile>
</tiles>
And this is the original XML that I modified because the first line caused errors
<?xml encoding="utf-8"?>
<tiles>
<tile>
<id>1</id>
<position x="10" y="20" />
</tile>
<tile>
<id>2</id>
<position x="50" y="20" />
</tile>
<tile>
<id>3</id>
<position x="30" y="40" />
</tile>
</tiles>
Thanks for viewing/reading. Any help is appreciated!
The bug is here:
var listOfTiles = documentFile.GetElementById("tiles").GetElementsByTagName("tile");
Change it this way:
var listOfTiles = documentFile.GetElementsByTagName("tiles").GetElementsByTagName("tile");
<tiles> is a tag and it hasn't got any id.
I'm not sure whether this is the complete class or not, but firstly it looks like you haven't created a list for the tiles.
Somewhere before you start adding tiles to the list 'tiles', you need to write:
tiles = new List();
I suggest the constructor.
It would also be helpful to know where the null reference exception is happening. Make sure your list is set, and then if the problem still occurs, add a comment and I'll run your code on my machine.
Per the docs, GetElementById will return elements with an id attribute as defined in a DTD (or just id by default). You don't have any id attributes, so this returns null - hence the exception.
If you change the offending line to this:
var listOfTiles = documentFile.GetElementsByTagName("tile");
Then your current code will work fine. However... LINQ to XML is a far cleaner API, you could write your entire method as below:
var doc = XDocument.Load(#"C:\Tiles\0.xml");
var tiles =
from tile in doc.Descendants("tile")
from position in tile.Elements("position")
let x = (float)position.Attribute("x")
let y = (float)position.Attribute("y")
select new Base.Tile(new Vector2(x, y));
this.tiles = tiles.ToList();
I'm having some trouble using XmlReader to read an XML file. I can open and close the file okay (I think), but when it comes to parsing out the information I need, I'm a bit lost. Here is the bit of the file I need to parse:
<?xml version="1.0" encoding="UTF-8"?>
<database name="Dictionary">
<data>
<Translations>
<Translation UniversalAbbv="Enu" lang="en" localization="US" unicode="0">
<Set>
...
</Set>
<Set>
...
</Set>
<Set>
<CaseSensitive value="0" />
<Enums translate="1">
<Enum_Entry ENUM_H="STOPRUN_STOP" EnumID="0" EnumString="Stop" SetID="160" />
<Enum_Entry ENUM_H="STOPRUN_RUN" EnumID="1" EnumString="Run" SetID="160" />
<Enum_Entry ENUM_H="STOPRUN_HOLD " EnumID="2" EnumString="Hold" SetID="160" />
</Enums>
<IncludeFiles_cs name="CSFile" value="StopRun.cs" />
<IncludeFiles_h name="Header" value="NULL" />
<IncludeFiles_java name="Java" value="NULL" />
<SetID value="160" />
<SetName value="Stop Run" />
<TwoSet ENUM_H="STOPRUN_ENUM_SET" />
</Set>
<Set>
...
</Set>
</Translation>
</Translations>
</data>
</database>
I need to find where EnumID="0" or EnumID="1" (or "STOPRUN_STOP" or "STOPRUN_RUN") and respectively pull out the "Stop" or "Run" strings. Here's what I have for code so far:
static class Dictionary
{
static private XmlReader Reader = null;
static public void Open()
{
XML_Generator.Dictionary.Reader = XmlReader.Create(XML_Generator.Program.DictionaryFilename);
}
static public void Close()
{
XML_Generator.Dictionary.Reader.Close();
}
static public void Read()
{
while (Reader.Read())
{
Trace.TraceInformation(XML_Generator.Dictionary.Reader.ReadElementContentAsString()); // <-- This throw an error. :(
}
}
}
I know it's not much, but I'm a bit lost on where to go with this. Any help would be appreciated. Thanks.
Here is example to read xml file using XML Reader
int intCount = 0;
XmlReaderSettings objSettings = new XmlReaderSettings();
objSettings.IgnoreWhitespace = true;
objSettings.IgnoreComments = true;
string booksFile = Server.MapPath("books.xml");
using (XmlReader objReader = XmlReader.Create(booksFile, objSettings))
{
while (objReader.Read())
{
if (objReader.NodeType == XmlNodeType.Element && "Book" == objReader.LocalName)
{
intCount++;
}
if (objReader.NodeType ==XmlNodeType.Text )
{
Response.Write("<BR />" + objReader.Value);
}
}
}
Response.Write(String.Format("<BR /><BR /><BR /><b> Total {0} books.</b>", intCount));
You probably want to take a look at the XpathNavigator. The syntax of it is really easy to use, much easier than doing it with the XMLReader
All you'd need to do to get the EnumID="1" item is //Enums/Enum_Entry[#EnumID=1]
How do I save a large collection with NHibernate which has elements that surpass the amount of memory allowed for the process?
I am trying to save a Video object with nhibernate which has a large number of Screenshots (see below for code). Each Screenshot contains a byte[], so after nhibernate tries to save 10,000 or so records at once, an OutOfMemoryException is thrown. Normally I would try to break up the save and flush the session after every 500 or so records, but in this case, I need to save the collection because it automatically saves the SortOrder and VideoId for me (without the Screenshot having to know that it was a part of a Video). What is the best approach given my situation? Is there a way to break up this save without forcing the Screenshot to have knowledge of its parent Video?
For your reference, here is the code from the simple sample I created:
public class Video
{
public long Id { get; set; }
public string Name { get; set; }
public Video()
{
Screenshots = new ArrayList();
}
public IList Screenshots { get; set; }
}
public class Screenshot
{
public long Id { get; set; }
public byte[] Data { get; set; }
}
And mappings:
<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2"
assembly="SavingScreenshotsTrial"
namespace="SavingScreenshotsTrial"
default-access="property">
<class name="Screenshot"
lazy="false">
<id name="Id"
type="Int64">
<generator class="hilo"/>
</id>
<property name="Data" column="Data" type="BinaryBlob" length="2147483647" not-null="true" />
</class>
</hibernate-mapping>
<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2"
assembly="SavingScreenshotsTrial"
namespace="SavingScreenshotsTrial" >
<class name="Video"
lazy="false"
table="Video"
discriminator-value="0"
abstract="true">
<id name="Id"
type="Int64"
access="property">
<generator class="hilo"/>
</id>
<property name="Name" />
<list name="Screenshots"
cascade="all-delete-orphan"
lazy="false">
<key column="VideoId" />
<index column="SortOrder" />
<one-to-many class="Screenshot" />
</list>
</class>
</hibernate-mapping>
When I try to save a Video with 10000 screenshots, it throws an OutOfMemoryException. Here is the code I'm using:
using (var session = CreateSession())
{
Video video = new Video();
for (int i = 0; i < 10000; i++)
{
video.Screenshots.Add(new Screenshot() {Data = camera.TakeScreenshot(resolution)});
}
session.SaveOrUpdate(video);
}
For this reason, we have typically had the child entity reference the parent rather than vice versa.
Create a custom type by using AbstractType and IParameterizedType these are in NHibernate.Type namespace. Use stateless session and provide batch size.
The chapter 13 of nhibernate documentation handle this issue.
"A naive approach to inserting 100 000 rows in the database using NHibernate might look like this:
using (ISession session = sessionFactory.OpenSession())
using (ITransaction tx = session.BeginTransaction())
{
for (int i = 0; i < 100000; i++)
{
Customer customer = new Customer(.....);
session.Save(customer);
}
tx.Commit();
}
This would fall over with an OutOfMemoryException somewhere around the 50 000th row[...]"
To resume... the solution is work with batch size and no second level cache by setting this properties:
Batch size:
adonet.batch_size 20
Second level cache:
cache.use_second_level_cache false
It should be enougth to solve OutOfMemoryException.
More datails at documentation reference: http://nhibernate.info/previous-doc/v5.0/single/nhibernate_reference.pdf
I am writing a class library which abstracts data contained in XML files on a web site. Each XML file uses the same root element: page. The descendant(s) of page depend on the specific file that I am downloading. For example:
<!-- http://.../groups.xml -->
<page url="/groups.xml">
<groups>
<group id="1" >
<members>
<member name="Adrian" />
<member name="Sophie" />
<member name="Roger" />
</members>
</group>
</groups>
</page>
<!-- http://.../project.xml?n=World%20Domination -->
<page url="/project.xml">
<projectInfo>
<summary classified="true" deadline="soon" />
<team>
<member name="Pat" />
<member name="George" />
</team>
</projectInfo>
</page>
There are also several additional XML files that I would like to download and process, eventually. For that reason, I have been trying to come up with a nice, clean way to deserialize the data. I've tried a few approaches, but each approach leaves me feeling a little dirty when I look back over my code. My latest incarnation utilizes the following method:
internal class Provider
{
/// <summary>Download information from the host.</summary>
/// <typeparam name="T">The type of data being downloaded.</typeparam>
internal T Download<T>(string url) where T : IXmlSerializable, new()
{
try
{
var request = (HttpWebRequest)WebRequest.Create(url);
var response = (HttpWebResponse)request.GetResponse();
using (var reader = XmlReader.Create(response.GetResponseStream()))
{
// Skip the XML prolog (declaration and stylesheet reference).
reader.MoveToContent();
// Skip the `page` element.
if (reader.LocalName == "page") reader.ReadStartElement();
var serializer = new XmlSerializer(typeof(T));
return (T)serializer.Deserialize(reader);
}
}
catch (WebException ex) { /* The tubes are clogged. */ }
}
}
[XmlRoot(TypeName = "groups")]
public class GroupList : List<Group>, IXmlSerializable
{
private List<Group> _list;
public void ReadXml(XmlReader reader)
{
if (_list == null) _list = new List<Group>();
reader.ReadToDescendant("group");
do
{
var id = (int)reader["id"];
var group = new Group(id);
if (reader.ReadToDescendant("member"))
{
do
{
var member = new Member(reader["name"], group);
group.Add(member);
} while (reader.ReadToNextSibling("member"));
}
_list.Add(group);
} while (reader.ReadToNextSibling("group"));
reader.Read();
}
}
This works, but I feel like there is a better way that I'm not seeing. I tried using the xsd.exe utility when I started this project. While it would minimize the amount of code for me to write, it did not feel like the ideal solution. It would be the same approach that I'm using now -- I would just get there faster. I'm looking for a better solution. All pages have the page element in common -- isn't there a way to take advantage of that? Would it be possible to have a serializable container class Page that could contain a combination of other objects depending on the file downloaded? Are there any simpler ways to accomplish this?
.NET provides a "xsd.exe" utility on the command line.
Run xsd.exe (xmlfilename) on your original xml file and it'll derive a XML schema (xsd) from your XML data file.
Run xsd.exe (xsd file name) /C and it'll create a C# class which can be used to deserialize such an XML file into a C# class.
Of course, since it only has a single XML file to go on, xsd.exe isn't perfect in its XML schema it derives - but that could be quick'n'easy starting point for you to get started.