Morning all,
I have been tasked with developing a client tool for a cloud web service API (A simple WSDL). I am not a seasoned or even qualified developer, I have an intermediate knowledge of C# and enough I believe to make this work, but I don't want a solution that just works, I want to build something clean and well coded which another dev can read and understand and which is intuitive.
You may want to stop me there and say "That is something you can only learn through experience." if that is the case then I can accept that and move on, but if you do have some advice the rest of the details are below.
The solution will be a C# Console application. I have produced a spec for this, it is below:
1.) Create a console application in .NET which has the following
capabilities:
2.) Consume CSV file containing Processed Data OR ODBC
Connection to staging SQL database and read records directly out of
load table
3.) Make the following calls to Zuora Webservice (Asynchronous) ·
SubscribeWithExisitingAccount() · Create() ·
Login() · Subscribe() · Update() · Delete()
(*) Calls marked with this are possibly avoidable,
*it is possible to create a subscription, account and contact with a
single call (Subscribe())
*Create() may be the exception as a scenario may occur where we need
to create an instance of an object with no corresponding subscription.
4.) Report back the success and errors of every record into a CSV
file.
Mappings will be done on a 1 to 1 basis, where the input file
will have the same column names as the target
Where I lack knowledge is following a design which will make this app make sense and work efficiently. I am not looking for someone to do this for me, what I am looking for is tips on how I can improve on what I am already doing
Currently I am just organically building the solution due to a lack of foresight on jobs like this, so I am also interested in things I can do post development.
ALL Advice and criticism is welcomed.
Thanks in advance,
Matt
Design principles are a big subject, and how to apply them correctly is only something that comes with experience. There's a lot more of them out there then you'd ever use in a given project, and in some cases using them correctly means not using them at all (or only choosing specific ones that suit the project). The first step is wanting to write good code though, so you're starting in the right place. :) A couple of things did stand out to me:
2.) Consume CSV file containing Processed Data OR ODBC Connection to
staging SQL database and read records directly out of load table
What you're going to want to do here is only build the logic that does something with this data once. The most direct way to achieve that is to have your logic expect the data in a certain format (probably business classes that hold the parsed data and that your logic an use).
So what you'll do is take the input data (CSV/SQL Table/Whatever) and first parse it into your internal business classes. Then you feed the parsed data to your logic that does whatever your app does with it. The advantage here is that you can change the logic once and it will work with both data types, AND if someone comes along later and says "now we need it to read this Excel file" all you'll have to do is add another parser to get the Excel data into your internal format. No changes to the logic will be required.
4.) Report back the success and errors of every record into a CSV
file.
Mappings will be done on a 1 to 1 basis, where the input file
will have the same column names as the target
Same as above. Don't assume that you'll be exporting to CSV forever, make a simple "ReportError" class or some such that holds error details and stick it into a List while doing your processing. At the end when it's time to output your errors, you can convert that into a CSV. So if this requirement changes and you instead report errors to a web service, you only have to change a small part of the code (and none of it is your processing logic).
There's a theme here. :) Try to encapsulate logical bits so that if something changes it's easy to find where that something is in code. If you can learn to do that, you'll wind up with maintainable code even if you don't follow any other process or pattern (particularly since as one person you won't be making huge projects).
3.) Make the following calls to Zuora Webservice (Asynchronous) ·
SubscribeWithExisitingAccount() · Create() · Login() · Subscribe() ·
Update() · Delete()
As a console app, I'm going to question if you actually need these to be asynchronous or not. What do you hope to gain from an async call to Login()? Can your program do anything while waiting for Login() to return?
It's not that async is terribly difficult, but it IS more to manage then synchronous calls. For a console app from someone whose not that experienced in the technology yet, I'm not sure what benefit you're gaining to weigh against the extra effort it requires of you.
I would recomend you read a book on webservices (this is a good one) They arent really something you can just pick up from playing about and can be quite frustraiting if you dont know what your doing.
As for development, I recomend you prototype it first. Hammer something out thats messy but lets you get an idea of how to do things. You can then use that as a reference for when your actually building your app.
Related
In my laravel application, I want to provide the users with the opportunity to download a copy of their stored data in the form of a Word document. I found that certain parts of this can only be accomplished using C#/.NET.
For this, I wrote a C# application alongside a method called GetWordProfile(User user) which returns FileInfo set to the actual path of the output file (this is always within the storage folder of laravel, so laravel has access to it). I only need the path and everything's done and dusted since from this point on, I can manage my laravel application to download this for the user.
However, the question is how do I get there? I must not forget about potential errors which may occur and thus display them (the errors are (inside my C# application) handled by log4net in a file as well as on the console; same goes for all output).
I tried to run my application using shell_exec respectively exec, however, both only returned zero results (null) (despite having set $output for exec) and thus seem not to be suitable. Also, I usually don't want loops (inside PHP/laravel) too much since you're then using a lot of computing power which is unnecessary for this sort of task, also you don't want to let your users wait more than, say, 5 secs, seeing nothing in your browser but the script being executed within a blank page (during the execution there's no content, obviously).
EDIT: I also approached the use of COM which ultimately did not work properly out either.
What is an appropriate approach towards this?
I did something similar with Python + C# a while back using IPC (Inter-process Communication) using named pipes.
EDIT: URL is broken. Here's the question someone asked previously on this topic.
Interprocess Communication using Named Pipes in C# + PHP
I'm developing a PC app in Visual Studio where I'm showing the status of hundreds of sensors that are connected via WiFi. The thing is that I need to hold on to the sensor data even after I close the app, so I'm considering some form of permanent storage. These are the options I've considered:
1) My Sensor object is relatively compact with only a few properties. I could serialize all the objects before closing the app and load them every time the app starts anew.
2) I could throw all the properties (which are mostly strings and doubles) into a simple text file and create a custom protocol for storage and retrieval.
3) I could integrate a database with my app. Someone told me this is the best way to go about it, but I'm a bit hesitant seeing as I'm not familiar with DBs.
Which method would yield the best results in terms of resource usage and speed? Or is there some other, better way to go about this?
First thing you need is to understand is your problem. For example, when the program is running do you need to have everything in memory at the same time or do you work with your sensors one at a time?
What is a "large amount of data"? For example, to me that will never be less than million (or billion in some cases).
Once you know that you shouldn't be scared of using something just because you are not familiar to it. Otherwise you are not looking for the best solution for your problem, you are just hacking around it in a way that you feel comfortable.
This being said, you have several ways of doing this. Like you said you can serialize data, using json to store and a few other alternatives but if we are talking about a "large amount of data that we want to persist" I would always call for the use of Databases (the name says a lot). If you don't need to have everything in memory at the same time then I believe that this is you best option.
I personally don't like them (again, personal choice) but one way of not learning SQL (a lot) while you still use your objects is to use an ORM like NHibernate (you will also need to learn how to use it so you don't get things a slower).
If you need to have everything loaded at the same time (most often that is not the case so be sure of this) you need to know what you want to keep and serialize it. If you want that data to be readable by another tool or organize in a given way consider a data format like XML or JSON.
Also, you can use mmap-file.
File is permanent, and keep data between program run.
So, you just keep your data structs in the mmap-ed area, and no more.
MSDN manual here:
https://msdn.microsoft.com/en-us/library/windows/desktop/aa366556%28v=vs.85%29.aspx
Since you need to load all the data once at the start of the program, the database case seems doubtful. The DB necessary when you need to load a bit of data many times.
So first two cases seem more preferred. I would advice to hide a specific solution behind an interface, then you'll can change it later.
Standard .NET serialization of sensors' array is more simple probably, and it will be easier to expand.
Fairly new to coding and i want a project to work on that could help me advance my skills. I'm not sure what language would be best for this sort of undertaking but i would definitely prefer to use C++ or C#.
For the first part of the program i basically would like to try and take all my pandora likes and put them on a spreadsheet with song name is one column and artist in the other. I don't see the formatting being too hard once i actually get the data i need, but i'm not really sure how to communicate with a server at all in this point in time. I'm guessing i probably won't be able to grab a raw list of likes so the i'm thinking my best course of action will be to first expand the likes list all the way, and then i need to read the text on the screen ro in the source code.
For the first step, expanding my like i found the HTML source code that actually does this:
<div class="show_more tracklike" data-nextLikeStartIndex="0" data-nextThumbStartIndex="5">Show more</div>"
Not sure if this is something i can work with but i was thinking if i could set data-nextThumbStartIndex="5" to be equal to the # of likes - 5 (the amount it shows by default) it would be fairly easy to expand the list. If not i would probably have to click the "show more" link repeatedly until i have all the likes on the page.
For the next step, getting the data i want, i think my best option would be to basically just grab the text that i physically see on the screen and worry about filtering and manipulating the data afterwards. The other option is looking at the source code, which i actually found the pieces of code where the info i want is stored. If i could retrieve the page's source code i think it would be relatively easy to pick out the data i actually want from that.
So yea that's about it, i know i'm pretty noob atm and what i'm saying is probably wrong and/or much more complicated than i think but i'm a pretty quick learner and at the very least if someone could point me in the right direction to communicate with a server that would be much appreciated.
This question is quite "wide" (and I have absolutely no knowledge of Pandora itself - can't access it from where I live).
In general, there are several different ways to solve this type of problem:
Screen Scraping - basically access the website as if you were a web-server, and from the HTML string that comes back, dig out the information you need. The problem here is that the data is not very suitable for "machine reading", as it often has no distinct points for the "reader" to find the relevant information, and it's difficult to sort the data from the "chaff".
AJAX api - "Asynchronous Java Script and XML" where the provider of the website has an interface to fetch certain data within to the web-browser - of course, if you "pretend" to be the web-browser, requesting the same type of information. You are relying on the website to have such an interface, but if it exists, the data is generally in a "more suitable form to be machine read" (typically XML, but not always).
JSON api - "Java Script Object Notation" is a similar solution to AJAX - like XML, JSON is a "human and machine readable format".
The latter two are definitely preferable, as the data coming back is meant for machine reading. The drawback is that you need to have "server side cooperation". The good thing here is that Pandora does have a JSON API. The bad thing is that it seems to be hard to use... Here's one discussion on the subject:
Making JSON calls to Unoffical Pandora API
The main principle here is that you send some stuff to the webserver, and receive a reply with the requested information. Exactly how this is done depends on the language/programming environment. A popular C++ solution is libcurl.
There is a Ruby Client here, using the JSON interface
https://github.com/nixme/pandora_client
A C# implementation to interface with Pandora is here:
http://pandoraunleashed.googlecode.com/svn/trunk/PandoraUnleashed/Pandora.cs
Unfortunately, I can't find any direct reference to "listing likes".
I'm virtually a complete novice, I've tried Googling for answers and become totally confused.
Using Visual Studio 2010, I have a C# application which is an email notifier for a friend. The external (Arduino) hardware works, the main code (from a website) works but I'm sending it to her on the other side of the world to use and she is very 'non-technical' - hence the need for a 'setup form'.
I have created a form where she can enter comm port (selected from a list), username and password (all to be used by the main code), but that form should run only when the application is first installed on the PC.
At the moment it runs in VS-2010 (though I need to iron out a couple of snags), validates and hides - but I don't know how to a) store the data and make it available to the main code, b) ensure that the form only runs at setup, or c) exactly what I need to do or include to create an installable application.
Could somebody either help or direct me to some tutorials that don't assume I understand all the terminology?
I just want to create something that she can instal from a memory stick. I know it can be done and it's proababy quite simple for those who understand - I'm trying to learn but I'm no longer young and it's a struggle.
Thanks
a) store the data and make it available to the main code,
write the data on a file!
you have millions of possibilities, for isntance reading and writing a plain text file can be done with few lines of code, but if you want to encrypt your file (it may be the case if you want to store the password) you can use System.Security.Cryptography as shown in this guide
b) ensure that the form only runs at setup,
once you have written the file, then it means that the program has run already at least once, so you don't need to ask the user again (just read the data from the file)
c) exactly what I need to do or include to create an installable application.
Visual Studio already comes with the Setup project for this task. See this good guide.
From your comment and link to the code project for the Arduino, I gather that this is your first venture into writing code in C#, or very close to it. And ideally you'd like to make this as easy for your friend as possible. The best advice I can give you is not to try to run before you learn to walk. If you try to create a custom setup project and use a configuration file, which is what you are talking about doing, you may hit so many barriers that you never get to a successful end of the project. That kind of experience is discouraging and I'd hate for you to lose the drive to ever want to try another software project.
Make this initial project easy on yourself. This is not good programming practice for most situations, but if you only have one user, hard-code her configruation information for this first version. In other words, put her username, password, com port, etc directly into the main program. This eliminates the need for both the configuration, and any custom setup form. If you still want to make the whole thing configurable and versatile, do that in your next version. Custom setup is not a beginner task. It will be a lot easier to take on with the encouragement of your friend's excitement and compliments over a first version that works.
Imagine there's a mission-critical process that'll be used in a business which handles sensitive information (think of Credit Card, social security, patient records...etc). I would think this unit ideally should do whatever it has to do on-the-fly, meaning it won't intentionally write files to disk containing sensitive information. The idea here is that if the computer that runs this process is compromised, no sensitive information can be leaked, at least not by means of files.
What approaches could be taken to, say, come up with a unit test that will fail if the unit under test tries to write any file to disk?
There is the FileSystemWatcher (http://www.c-sharpcorner.com/uploadfile/puranindia/filesystemwatcher-in-C-Sharp/) however this requires you to know a specific directory. In your case this probably isn't very helpful since the program could write anything to disk any where. This introduces a unique problem. However, I have also found something called Detours from Microsoft. This appears to intercept all native win32 api calls. http://research.microsoft.com/en-us/projects/detours/ The issue with this is that its kind of hard to test, and integrating it into unit testing will be a challenge.
When you have to treat your software as "untrusted" in the sense that you need to prove it doesn't do something, testing becomes a complex task that requires you to run them on very controlled environments. When hooking in to the Win32 API, you will be deluged with API calls that need to be processed quickly. This can result in unintentional side effects because the application is not running in a truly native environment.
My suggestion to you (having worked several years doing software testing for Pharma automation to the exacting standards of the FDA) is to create a controlled environment, eg a virtual machine, that has a known starting state. This can be accomplished by never actually saving vmdk changes to disk. You have to take a snapshot of the file system. You can do this by writing a C# app to enumerate all files on the virtual drive, getting their size, some timestamps and maybe even a hash of the file. This can be time consuming so you may want (or be able) to skip the hashing. Create some sort of report, easiest would be by dropping them in a CSV or XML export. You then run your software under normal circumstances for a set period of time. Once this is complete, you run a file system analysis again and compare the results. There are some good apps out there for comparing file contents (like WinMerge). When taking these snap shots, the best way to do it would be to mount the vmdk as a drive in the host OS. This will bypass any file locks the guest OS might have.
This method is time intensive but quite thorough. If you don't need something of this depth, you can use something like Process Monitor and write the output to a file and run a report against that. However in my work I would have to prove that Process Monitor shows all IO before I could use it which can be just as hard as the method I spoke of above.
Just my 2 cents.
UPDATE:
I've been thinking about it, and you might be able to achieve fairly reliable results if you remove all references to System.IO from your code. Write a library to wrap around System.IO that either does not implement a write method, or only implements one that also writes to a log file. In this case, you simply have to validate that every time a write occurs using your library, it gets logged. Then validate using reflection that you don't reference System.IO outside of this new wrapper library. Your tests can then simply look at this log file to make sure only approved writes are occurring. You could make use of a SQL Database instead of a flat log file to help avoid cases of tampering or contaminated results. This should be much easier to validate than trying to script a virtual machine setup like I described above. This, of course, all requires you to access to the source code of the "untrusted" application, although since you are unit testing it, I assume you do.
1st option:
Maybe you could use Code Access Security, but the "Deny" is obsolete in .NET 4 (but should works in previous version):
[FileIOPermission(SecurityAction.Deny)]
public class MyClass
{
...
}
You may reactivate this behavior in .NET 4 using NetFx40_LegacySecurityPolicy
2nd option:
reducing the level of privilege may also works, as I know that downloaded app can't write on the disk and must use a special storage area.
3rd option:
Remove any reference to System.IO and replace by an interface that your code must use to write data to disk.
Then write an implementation that use System.IO (in a separate project)
In the nunit test, mock this interface and throw an exception when a method id called.
Problem is to ensure any developers will not call System.IO anymore. You can try to do this by enforcing coding rules using FxCop (or other similar tools)