I need to write a WCF services that bring data to reporting tool.
Reporting tool presents data in lazy way, till user not clicked it's not showing the data.
I cannot send everything in once because there can be several megabits of data and because of that I need to send it in portions.
The problem is that I don't want to create a lot of web functions for each report , because this way part of the BL will be in the reporting tool.
Is it possible somehow to make each report run in it's own web session and each time it's asks a next portion of data I will be able to send it back and everything made in the same session?
May be you have better solution to my problem .
There are a number of technologies that could help. I would take a look at WCF Data Services which allow you to do flexible querying (IQuerable) and association traversal which should take care of your lazy loading concerns without having to create a whole load of seperate WCF calls.
Also take a look at SQl Server Reporting Services which is a more general reporting solution that may appeal to you.
Either of these technologies should help you avoid your BL leaking into your reporting tool. There are probably a whole host of similar non-MS solutions that do similar things, but I have listed the two above as your are using WCF, so you are probably more familiar with the MS stack (but maybe that was a silly assumption by me...if thats not the case they will get you started on what to look for!)
Related
Basically, I have a new desktop application my team and I are working on that will run on Windows 7 desktops on our manufacturing floor. This program will be used fairly heavily as it gets introduced and will need to interact with our manufacturing database. I would estimate there will (eventually) be around 100 - 200 machines running this application at the same time.
We're lucky here, we get to do everything from scratch, so we define the database, any web services, the program design, and any interaction between the aforementioned.
As it is right now, our legacy applications just have direct access to a database, which is icky. We want to not do that with the new application.
So my question is, how do I do this? Vague, I know, but basically I have a lot at my disposal here, and I'm not entirely sure what the right direction to go is.
My initial thought, based on what I've perceived others doing, is to basically wall off the database by using webservices. i.e. all database interactions from the floor MUST occur through the webservices, providing a layer of security by doing much of the database logic behind closed doors. Webservice calls are then secured to individual users via Active Directory.
As I've found though, that has some implications of its own... We have to abstract the data before it reaches the application. There's still potential for malicious abuse by using webservice calls repeatedly to ruin or spam data. We've looked at Entity Framework and really like what it provides, but as best I can tell, that's going to be unavailable by the time we're at the application level in this instance.
It just seems like I can't come to a conclusion on what is "right". So, what is right?
WebServices sounds like a right approach. Implementing a SOA-oriented layer on the webservices layer gives you a lot of control over what happens to the data at the database server.
I don't quite share your doubts about repeated calls doing any damage - first you can have an audit log of every single call so that detecting possible misuses is obvious. But you also could implement a role based security so that web service methods are exposed to users in roles, which means that not everyone will be able to call just any method.
You could even secure your webservices with forms authentication so that authentication is done against any datasource, not only the active directory.
And last thing, the application itself could be published as a ClickOnce application so that it is downloaded and executed from the web page and it automatically updates itself just as you publish new versions.
If you need some technical guidance, I've blogged on that years ago:
http://netpl.blogspot.com/2008/02/clickonce-webservice-and-shared-forms.html
My suggestion since you are greenfield is to use an API wrapper approach with Servicestack.
Check out: http://www.servicestack.net/ServiceStack.Northwind/
Doing that you can use servicestack authentication, abstract away your db layer (because you could move to a different DB provider, change its location, provide queues for work items etc...) and in time perhaps move your whole infrastructure to an internal intranet app.
Plus Servicestack is incredibly fast, interoperable with almost any protocol you through at it, and provides for running it through MONO, so you are not stuck with a MS backend that could be very expensive.
My two cents. :)
First of all this question is not appropiate for StackOverflow, you might get close-votes really quickly.
Second, You may want to have a look at WCF RIA Services for this.
These will allow you to create basic CRUD operations for all your entities, and stuff like that.
I never used this myself, no I'm not sure what the potential issues might be.
Otherwise, Just do what we did:
Create generic (<T>) interfaces and services and contracts and everything. This will allow you to adapt your CRUD functionality in your Services, DAOs, ViewModels and such to any entity type.
I've a big database which contains a lot of data from a big enterprise.
We would like to be able to dispatch this data to different external applications (external, meaning that are not developed by us, but only accessible in our local network).
Consumers can be of very different kinds: accounting, reporting, tech(business), website, ...
With a big variety of formats: CSV, webservice, RSS, Excel, ...
The execution of these exports can be of two different types: scheduled (like every hour), or on demand.
There is mostly two kind of exports: almost-real-time-data(meaning we want to have current data), or statistical data(meaning we are taking in account a period of time).
I've yet to find a good approach to allows those access.
I thought about Biztalk, but I don't know this product very well, and I'm not sure it can make scheduled calls and have business logic. Does anyone have enough knowledge of Biztalk to indicate to me if it can fit my needs?
If Biztalk isn't a good way, is there any libraries which can ease the development of a custom service?
Biztalk can be made to do what you want to do i.e. Extract data from your database, transform it into various formats and send it to various systems on a scheduled basis or as and when required by exposing this as a webservice/WCF Service (Not entirely out of the box, but you might need to purchase additional adapters, pipelines, etc).
But, the question here is, how database intensive is this task? If its large volumes of data, clearly Biztalk is not a favorite candidate, as Biztalk struggles with large data. Its good for routing (without transforming/inspecting) though, even if its large data files.
SSIS, on the other hand is good for data intensive tasks. If your existing databases are on SQL Server, then it fits even better for your data intensive exports/imports and transformations. But it falls short when it comes to the variety of ways you need to connect to external systems (protocols).
So, you are looking at a combination of a good ETL tool, like SSIS, as well as something good at routing like Biztalk. Neither of them clearly fit your needs on their own, in terms of scalability, volumes, connectivity, data formats, etc.
Your question can result in quite a broad implementation. You could consider using a service bus (pub/sub) along with some form of CQRS (if applicable).
My FOSS Shuttle ESB project is here: http://shuttle.codeplex.com/
It has a generic scheduler built in. You could, of course, go with any other service bus such as MassTransit, or NServiceBus.
I think you could use ASP.NET MVC API. http://www.asp.net/web-api
I find it the easiest way to export different kind of info and file formats.
It won't generate scheduled reports or files, you will need the client app or a windows service to call the app. Similar to webservices, but it can return different formats and also files.
And creating excel files, etc. you have to create them manually. Thats a bit of a turndown, but i like this approach because it can be easily hosted on IIS and all the functions your clients are going to call can be on the same place and even called from javascript, so as i see it is a bit more work for you, but it creates really easy to consume services.
By dispatch, I'm assuming you're looking for a pub/sub model. Take a hard look at NServiceBus's (NSB) pub/sub capabilities, http://nservicebus.com/docs/Samples/PublishSubscribe.aspx. Underneath the covers NSB makes heavy use of MSMQ, which has become a lot more stable over time.
If you want to venture outside of your .NET comfort zone, check out Apache Camel or Fuse's Enterprise Service Bus. Either of these tools will support what you need as well. I've used Camel in some extremely high throughput areas without any major issues.
I am on a quest to find the best method to implement this data structure in my WPF c# .NET 4.0 web application.
App Details: This application will have many users at one time. I'm working with a database that has 2 tables linked together, both with <10k rows, but they will grow with time. One table will need to be checked for updates periodically.
Question:
I created a dataset to hold all information. My unit test shows it takes ~0.2 seconds to populate the dataset from the database. This is no time if the dataset will persist across the application. Can it persist across the application for many users?
The other method I can think of is using observable collections for the data and storing them in a singleton. Is there any method that I am missing here or something that will work better for me?
I think you're misunderstanding some really important concepts here. First of all, WPF is not ASP.Net, WPF applications do not "run at server". WPF applications run at the computer that executes them (I.E the client) regardless of being an XBAP that runs inside Internet Explorer.
Therefore, you cannot have such thing as a "singleton" shared between many instances of your WPF application because each instance will be run on a separate computer on its own process. There's no practical way to share memory between these instances.
Then, if you need a "Web Application", WPF might not be suitable for your project. WPF is not a web technology, it is a rich desktop application technology that does not depend on the crappy implementations of HTML or anything like that.
Now, if you need (or desire) the richness and awesomeness and ease of development WPF provides, you will have to implement a Client/Server application, possibly using WCF or any other communications technology to enable your rich Client to communicate with the Server.
The "Server" will be an application responsible for querying the data from the database, then possibly caching it in memory, and then sending it to the client. Usually a WCF Service application is enough for this, although depending on your requirements you may need to implement more layers on top of it.
Beside this, I don't think there is a good reason to send a WPF client 10k rows of information, so you will have to come up with some paging mechanism in order to send the data in "chunks" instead.
Please provide more details about what you need and I can give you more insight.
We have a service-based system. The system is tiered or layered so that a single service call from an outside entity might hit one, two or three other services, depending on the type of call and system state.
What we want to be able to do is to track the progress of a given call across these different services.
Ideally, as the external call comes in, a tracking number is generated and this follows all subsequent calls throughout our system.
Are there any specific design patterns or WCF features (implementations of the pattern) that we can use to track this progress?
This page gives an example of using session IDs, but it's not clear what the right thing to do is once there are several services involved.
This page may also have some relevance.
We are specifically interested in C# / WCF implementation, but references to any resources that are relevant are interesting (Java / PHP / whatever).
I would use aspect orientated programming to handle this. I believe the right piece is a Service Behavior.
This will let you create an attribute that you stick on your service methods (or service). That has BeforeAction/AfterAction/OnError.
For the entry point service I would have the action create your personal sessionid, stuff it into the WCF context, and then use the Before, After, and Error methods to post data to your data store or however you plan identify progress.
Chris's response is a good one, but we found this information from Microsoft very, very useful --- and it's built-in to IIS / WCF support.
Enabling the log indicated in the screen capture below really helped.
More of a design/conceptual question.
At work the decision was made to have our data access layer be called through webservices. So our website would call the webservices for any/all data to and from the database. Both the website & the webservices will be on the same machine(so no trip across the wire), but the database is on a separate machine(so that would require a trip across the wire regardless). This is all in-house, the website, webservice, and database are all within the same company(AFAIK, the webservices won't be reused by another other party).
To the best of my knowledge: the website will open a port to the webservices, and the webservices will in turn open another port and go across the wire to the database server to get/submit the data. The trip across the wire can't be avoided, but I'm concerned about the webservices standing in the middle.
I do agree there needs to be distinct layers between the functionality(such as business layer, data access layer, etc...), but this seems overly complex to me. I'm also sensing there will be some performance problems down the line.
Seems to me it would be better to have the (DAL)assemblies referenced directly within the solution, thus negating the first port to port connection.
Any thoughts(or links) both for and against this idea would be appreciated
P.S. We're a .NET shop(migrating from vb to C# 3.5)
Edit/Update
Marked Dathan as answer, I'm still not completely sold(I'm still kind of on the fence, though leaning it may not be as bad as I feared), he provided a well thought out answer. I appreciated all the feedback.
Both designs (app to web service to db; app to db via DAL) are pretty standard. Web services are often used when interfacing with clients to standardize the semantics of data access. The web service is usually able to more accurately represent the semantics of your data model than the underlying persistence store, and thus helps the maintainability of the system by abstracting and encapsulating IO-specific concerns. Web services also serve the additional purpose of providing a public interface (though "public" may still mean internal to your company) to your data via a protocol that's commonly accessible across firewalls. When using a DAL to connect directly to the DB, it's possible to encapsulate the data IO concerns in a similar way, but ultimately your client has to have direct access to the database. By restricting IO to well-defined semantics (usually CRUD+Query), you add an additional layer of security. This isn't such a big deal for you, since you're running a web app, though - all DB access is already done from trusted code. The web service does provide an increase in robustness against SQL injection, though.
All web service justifications aside, the real questions are:
How much will it be used? The website/web service/database format does impose slightly higher overhead on the web server - if the website gets hammered, you want to consider long and hard before putting another service on the same machine. Otherwise, the added small inefficiency is probably not a big deal. On the other hand, if the site is getting hammered, you probably want to scale horizontally anyway, and you should be able to scale the web service at the same time.
How much to you gain? One of the big reasons for having a web service is to provide data accessibility to client code - particularly when multiple possible application versions need to be supported. Since your web app is the only client to use the web service, this isn't a concern - it's probably actually less effort to version the app by itself.
Are you looking to expand? You say it probably won't ever be used by any client other than the single web app, but these things have a way of gaining in size. If there's any chance your web app might grow in scope or popularity, consider the web service. By designing around a web service, you're already targeting a modular, multi-host solution, so your app will probably scale with fewer growing pains.
In case you couldn't guess, I'm a web service fan. But the above are also my honest (if somewhat biased) opinions on the subject. If you do go the web service route, be sure to make it simple - keep application logic in the app and service logic in the service, and try to draw a bright line between them when extending the two. And do design your service for efficiency and configure the hosting to keep it running as smoothly as possible.
This is a questionable design, but your shop isn't the only one using it.
Since you're using .NET 3.5 and running on the same machine, you should use WCF with the netNamedPipesBinding, which uses binary data transfer over named pipes, only on the same machine. That should mitigate the performance issue somewhat.
I like the idea because it gives you flexibility. We use a very similar approach because we can have more than 1 type of database storing our data (MSSQL or Oracle) depending on our customer install choices.
It also gives customers the ability to hook into our database if they choose not to use our front end web site. As a result we get an open API for little to no extra effort.
If speed is your most critical issue than you have to lessen your layers. However in most cases the time it takes for your web Service to process the request from the database does not add allot of time. (This is assuming you do your Web Service Layer Correctly, you can easily make it slow if you don't watch it.)