I am currently working on new programs that converts Excel Files to XBRL using the C# language (or any programming language).
As I read over the net, XBRL is following XML schema and it's where financial system are heading to, where all this institution wanted to have their own standard language.
There are ready to use software for this in the market but the client we have today requires us to write a program just for them.
Has anyone here in the forum had written a program similar to my need?
XBRL has their own (lengthy) format specification, which you can find here. Excel isn't innately compatible with that, unless you have a specific template or usage pattern in mind. There have been a number of "Excel to XBRL" scripts, but it appears as if they're all just a custom Excel document with a script that can generate the compliant XBRL or iXBRL document.
It's likely that you're going to need to map the requirements of the customer to the XBRL output they need, and I'm not aware of a .NET library that does that as it stands. I hate to deliver the bad news, but you'll need to get familiar with the nitty gritty requirements of the XBRL format specification, understand the XBRL Taxonomy the client wants to use, and build the code from that point forward.
That being said, there are a number of libraries to de/serialize XML in .NET. Start with the System.XML and System.XML.Schema namespaces to define your XBRL schema and generate/read your XML files.
You could also look into AutoMapper to convert tab delimited/csv/excel data into your objects (and vice versa). Specifically, look into the "Flattening" functionality there.
Related
so I feel like this is a thing that I've seen before, but I can't think what it's called.
I have a java program which loads up an object from a file (I'm going to call this object the catalogue). Defining the file format, and writing the parser for it has taken a fair amount of time. Recently I've had developers that are want to load the file so they can use the catalogue for their own tools, but are working in a different language (current just C#, but potentially others in future).
Most of our tools run out on a server, so to prevent other people having to write their own parser for the file format, I plan on having a web service running which uses my program to load the catalogue, then returns it's contents as a json string. (Encoding the original file as json is not an option). In future I may just return bits of the json string, since the whole contents of the file can be pretty massive.
What I'm wondering is, do I have to write my catalogue object in each language, and write a json parser for it in that language, or is there a tool that will allow me to do something similar? I'm hoping for something that just has a simple format for declaring storage classes, generate corresponding code for various languages, and have a default serialize/deserialize for something like json. Currently I'd need it to support java and C#.
Does such a thing exist?
As long as the target language can make a web call, and parse JSON strings, you should be OK. This covers just about all mainstream languages I think. C# is certainly OK.
We need to implement a WCF Webservice using the ACORD Standard.
However, I don't know where to start with this since this standard is HUMONGOUS and very convoluted. A total chaos to my eyes.
I am trying to use WSCF.Blue to extract the classes from the multiple XSD I have but so far all I get is a bunch of crap: A .cs file with 50,000+ lines of code that freezes my VS2010 all the time.
Has anybody walked already thru the Valley of Death (ACORD Standard) and made it? I really would appreciate some help.
I wrote a ACORD to c# class library converter which was then used in several large commercial insurance products. It featured a very nice mapping of all of the ACORD XML into nice concise, extendable C# classes. So I know from whence you come!
Once you dig into it its not so bad, but I maintain the average coder will not 'get it' for about 3-4 months if they work at it full time (assuming anything but inquiry style messages). The real problem comes when trying to do mapping from a backend database and to/from another ACORD WS. All of the carriers, vendors, and agencies have custom rules.
My best suggestion is to find working code examples (I have tons if you need them) and maybe even a vendor or carrier who will let you hook up to a ACORD ws in a test environment.
It sounds like you are heading down the right path but are lost in the forest.
The ACORD Standard is huge and intentionally so, as it provides support for hundreds of different messages. Just as you do not download all of Wikipedia to get just a few articles, you do not need all of the classes in the ACORD Standard to support an implementation of a few messages. If you know what messages you need to support then you can generate a subset of the full XSD that will be quite manageable.
As mentioned in Hugh’s response, for any one message only a fraction of the full XSD is used. How you go about doing that will depend on the specifics of your project. If you are looking for ideas on how generate a subset of the full XSD try reaching out to the ACORD staff for help at PCS#acord.org. They should be able to offer you some help in getting started.
I have worked with the Accord PCS exposure reporting standards and yes it was a nightmare. I have also worked with other large standards like FPML and SportsML.
You need to work out exactly which types from the schema that are needed. How you do this is up to you, but VS schema viewer should be able to handle it. If not try XmlSpy or just go through it by hand if you have to. Make sure you have a good BA to hand...
Chances are you will find that you can meet your requirements by using around 1% of the types available in the standard.
What you'll probably find is that you can express the core objects with a very minimal set of values, as most nodes will be minOccurs=0 or nillable.
Then you can use the /element switch on xsd.exe to generate the code for just the types you need.
As one commenter says there is no easy pill to swallow here. The irony is that standards are supposed to make everyone's lives easier.
If you are looking to read/write ACORD documents using .NET, I just stumbled across the "IVC Software Factory for ACORD Standards" on CodePlex at http://ivc.codeplex.com.
From the limited documentation it appears as if this library can convert objects to ACORD XML documents, and vice-versa. The source code comes with different "providers" i.e. different ACORD transaction types, like 103 or 121.
Hope this helps.
I would recommend not creating a model for the entire standard. One could just pass XML and not serialize into a model but instead load it into XDocument/XElement and use Linq to query it and update the DOM using Linq to Xml. So, one is not loading the XML to a strongly typed model, but just loading the XML. There is no model, just an XML document.
From there, one can pick the data off of the XML as needed.
Using this approach, the code will be ugly and have little context since XElements will be passed everywhere, and there will be tons of magic strings of XPaths to query and define elements, but it can work. Also, everything is a string so there will be utility conversion methods to convert to numbers, date times, etc.
From my prospective, I have modeled part of the Acord into an object model using the XmlSerializer but it's well over 500 classes. The model was not tooled from XSD or other, but crafted manually and took some time. Tooling will produce monster unusable classes (as you have mentioned) and/or flat out crash. As an example, I tried to load the XSD into Stylus Studio and it crashed several times.
So, your best bet if your strapped for time is loading into an XDocument as opposed to trying to map out everything in a model. I know that sucks but Acord in general is basically a huge data hot mess.
I am looking for an LDIF parser for C#. I am trying to parse an LDIF file so that I can check objects don't exist before adding them. Adding them when the already exist using ntdsSchemaAdd) causes an entry in the error logs.
A quick websearch revealed: http://wiki.github.com/skradel/Zetetic.Ldap/. They have provided a .net API.
From the page:
Zetetic.Ldap is a .NET library for
.NET 2 and above, which makes it
easier to work with directory servers
(like Active Directory, ADAM, Red Hat
Directory Server, and others). Some of
the key features of Zetetic.Ldap are:
1.LDIF file parsing and generation – Read and write the file format used
for moving data around between
directory systems
2.LDAP Entry-oriented API with change tracking – Create and modify directory
objects in a more natural way
3.LDAP Schema interrogation – Quick programmatic access to the kinds of
objects and fields your directory
server understands. Learn if an
attribute is a string, a number, a
date, etc., without lots of manual
research and re-parsing
4.LDIF Pivoter – Turn an LDIF file into a (comma or tab-delimited) flat
file for analysis or loading into
systems that don’t speak LDIF We built
the Zetetic.Ldap library to make
directory projects and programming
faster and easier, and release it here
in the hopes that others will find it
useful too. As far as we know, this is
the only .NET library that really
understands the LDIF specification.
Download link: http://github.com/downloads/skradel/Zetetic.Ldap/Zetetic.Ldap_20090831.zip
I would parse it myself.
If you look at the LDIF RFC for the EBNF, you'll see that it's not a very complex grammar.
I've parsed a large amount of LDIF before using Regexes reliably. Though your mileage may vary.
I have two application that need to talk to each other. App1 needs to be able to serialize an object that App2 can then deserialize. Easily done, right? Here's the problem; App1 is C# based, App2 is Java based. So App1 needs to write out the file in the Java binary file format. How can this be done?
The way I see it, I have two options. The first is figure out some way to serialize a Java object in C#, so that App1 just creates the appropriate file. My other option would be to write a converter in Java that reads in a file and populates the object accordingly and serializes the newly populated object. That way the C# app would only have to write out some sort of formatted text file that the converter then interprets.
I can't make any changes to the Java application.
How should this be done?
Update:
The Java application is already in the hands of customers so changing the serialization scheme would cause the customers existing data to be incompatible. The Java App uses the native java serialization when dealing with this object. Modifications to the Java app can't happen.
The C# app uses protocol buffers to serialize its own data.
The best answer is option 3:
use a language-neutral serialization scheme.
I use JavaScript. Thrift is another option, protocol buffers I believe are more focused on RPC, but should be usable for serialization as well. XML or a custom binary format would be other options.
Edit:
Sorry, didn't notice that you can't make changes to the Java application. That said, the best way to do it would probably be to create your own well defined format, write a java app that can read that format, then output a serialized java object for the legacy app.
"IKVM" might be something you could use. This product allows you to convert compiled java bytecode (.jar, etc.) into a .NET DLL. It's super easy to use, and might give you the interop you need.
Other than this, the easiest way to accomplish this without a binary-level interop is to just use a plain text format, such as a CSV or XML.
Just use XML serialization. Both frameworks have good support, and the simplicity will make it easier to debug / maintain. Write a small program in Java that just imports the XML and writes the binary file.
Your best bet would be to write something that uses Java Native interface. Not fun, but it'll work.
You can do this directly using JNI (not fun but doable) or there may be some tools out there that will generate code for you -- take a look at SWIG: http://www.swig.org/
You would call Java from C# to do the persistence for you.
I'm going to start the development of my own document format(like PDF, XPS, DOC, RTF...), but I want to know where I can read some tutorials, how-to's...? I don't want code, this is a project that I want to learn how to build it, not use the experience of someone other.
PS: I want to make it like a XML file:
[Command Argument="Define it" Argument2="Something"]
It's like PDF, but this syntax will be interpreted by a program that I will build it using C#, just like HTML and your browser ;)
Remember that my question is about the program that will interpret this code, but it's good to start with a tutorial of interpreting XML code ;)
I assume you're doing this for the sake of learning how to do it. If that's the case, it is a worthwhile venture and I understand.
You'll want to start out by learning LL parsers and grammars. That will help you interpret the document that has been read from a file into a document object model (DOM). From there you can create routines to manipulate or render that document tree.
Good luck!
I'm confused as to what you're asking, but if you need your own format like an XML file, why not just use XML to describe the format?
Edit: Okay, I think I understand now. If you're doing this for fun and for learning (which is great), then there are lots of approaches to take. In fact, it may even be better to not do any research, try to come up with a solution on your own and see if it works, what you need to do to make it better, etc.
Sounds like a good learning project and you've got some good pointers here already. I would just add that you should remember that there is a difference between a document file language and a document format.
Consider OOXML, it is a document format that is built on top of XML (what I'd describe as the file language). If your purpose is to learn about building your own document format then I'd highly recommend starting with XML so that you don't have to reinvent a language parser. This will let you focus on the concerns around building the format.
That said, good on you if you want to play around with creating your own language; just wanted to make sure you realized that they are different beasts.
Here are some links that will help you get started using XML in C#:
Xml Tutorial (video)
XML Document overview
Reading Xml data with an XmlReader
Writing Xml data with an XmlWriter
Far be it from me to forbid you from re-inventing the wheel for the sake of learning something new. Good for you for trying this out. However, if you are going to ask questions about how to do it you are going to need to specify your questions a little more.
Are you looking for help on:
Designing your framework / format
Planning your time / Estimating deadlines
Working with XML
Working with C#
Building a web-based C# application
Building a PC-based C# application
Other aspects of development entirely
There are many people here who want to help -- but the best answers are given to focused questions (not necessarily specific, but always focused.)
There are a couple of ways to approach this. One way would be to define the format of the file first, then use a parser-generator to crate C# code that can read that format. doing a Google search on "c# parser generator" will get you links to a number of different libraries you can use.
Alternatively, you could code your own parser, from scratch. This will be more work than using a parser generation tool, but might be more educational in the end.
The define-a-grammar approach may be total overkill for a simple format. Another way to approach the problem is to design the object tree that you'll use in-app first, then write serialization and de-serialization routines to save and load the contents from a file. The serialization interface in C# is pretty flexible, and you can serialize to binary or XML files easily.
I think it should be relatively straightforward to create your own serializer to create a file formatted however you like, but MSDN is not being my friend today, so I can't find the relevant documentation.