save Website Content & access it - c#

I'm starting with C# again after 3 years (have average experience with object orientated languages; here I'm mainly missing function names). I'm not too sure it's possible in c#, so if you can recommend another language I will try to look there.
My Question(s):
On program start (or button) I want to extract a part of a Website and save it (temporary of file don't matter). That way I wont need to buffer/load (loadtime) anything again and can access the content if I go offline afterward.
I want to extract some numbers out of the content and do simple math with them.
Would be great to know if its possible and how. I'm happy if you can tell me the main functions I should look into. Some basic code would be great too if its not too much to ask.

If you want to have access to the information even if your program closes/restarts then you will need to export the source code to a file as follows:
using (WebClient wb = new WebClient())
{
string source = wb.DownloadString("http://example.com");
File.WriteAllText("c:\\exampleFile.txt", source);
}
Otherwise you can remove the File.WriteAllText("c:\\exampleFile.txt", source); and simply parse the parts you want from the source and do your calculations.
Keep in mind this will download the source code of the url as 'it is' that means you will need to do some parsing of the text in order to get the information you want out of it.

May be you are looking for this:
var contents = new System.Net.WebClient().DownloadString(url);

Related

Read/Write array to a file

I need guidance, someone to point me in the right direction. As the tittle says, I need to save information to a file: Date, string, integer and an array of integers. And I also need to be able to access that information later, when an user wants to review it.
Optional: File is plain text and I can directly check it and it is understandable.
Bonus points if chosen method can be "easily" converted to working with a database in the future instead of individual files.
I'm pretty new to C# and what I've found so far is that I should turn the array into a string with separators.
So, what'd you guys suggest?
// JSON.Net
string json = JsonConvert.SerializeObject(objOrArray);
File.WriteAllText(path, json);
// (note: can also use File.Create etc if don't need the string in memory)
or...
using(var file = File.Create(path)) { // protobuf-net
Serializer.Serialize(file, objOrArray);
}
The first is readable; the second will be smaller. Both will cope fine with "Date, string, integer and an array of integers", or an array of such objects. Protobuf-net would require adding some attributes to help it, but really simple.
As for working with a database as columns... the array of integers is the glitch there, because most databases don't support "array of integers" as a column type. I'd say "separation of concerns" - have a separate model for DB persistence. If you are using the database purely to store documents, then: pretty much every DB will support CLOB and BLOB data, so either is usable. Many databases now have inbuilt JSON support (helper methods, etc), which might make JSON as a CLOB more tempting.
I would probably serialize this to json and save it somewhere. Json.Net is a very popular way.
The advantage of this is also creating a class that can be later used to work with an Object-Relational Mapper.
var userInfo = new UserInfoModel();
// write the data (overwrites)
using (var stream = new StreamWriter(#"path/to/your/file.json", append: false))
{
stream.Write(JsonConvert.SerializeObject(userInfo));
}
//read the data
using (var stream = new StreamReader(#"path/to/your/file.json"))
{
userInfo = JsonConvert.DeserializeObject<UserInfoModel>(stream.ReadToEnd());
}
public class UserInfoModel
{
public DateTime Date { get; set; }
// etc.
}
for the Plaintext File you're right.
Use 1 Line for each Entry:
Date
string
Integer
Array of Integer
If you read the File in your code you can easily seperate them by reading line to line.
Make a string with a specific Seperator out of the Array:
[1,2,3] -> "1,2,3"
When you read the line you can Split the String by "," and gets a Array of Strings. Parse each Entry to int into an Array of Int with the same length.
How to read and write the File get a look at Easiest way to read from and write to files
If you really wants the switch to a database at a point, try a JSON Format for your File. It is easy to handle and there are some good Plugins to work with.
Mfg
Henne
The way I got started with C# is via the game Space Engineers from the Steam Platform, the Mods need to save a file Locally (%AppData%\Local\Temp\SpaceEngineers\ or %AppData%\Roaming\SpaceEngineers\Storage\) for various settings, and their logging is similar to what #H. Sandberg mentioned (line by line, perhaps a separator to parse with later), the upside to this is that it's easy to retrieve, easy to append, easy to overwrite, and I'm pretty sure it's even possible to retrieve File Size, which when combined with File Deletion and File Creation can prevent runaway File Sizes as this allows you to set an Upper Limit to check against, allowing you to run it on a Server with minimal impact (probably best to include a minimum Date filter {make sure X is at least Y days old before deleting it for being over Z Bytes} to prevent Debugging Data Loss {"Why was it over that limit?"})
As far as the actual Code behind the idea, I'm approximately at the same Skill Level as the OP, which is to say; Rookie, but I would advise looking at the Coding in the Space Engineers Mods for some Samples (plus it's not half bad for a Beta Game), as they are almost all written in C#. Also, the Programmable Blocks compile in C# as well, so you'll be able to use that to both assist in learning C# and reinforce and utilize what you already know (although certain C# commands aren't allowed for security reasons, utilizing the Mod API you'll have more flexibility to do things such as Creating/Maintaining Log Files, Retrieving/Modifying Object Properties, etc.), You are even capable of printing Text to various in Game Text Monitors.
I apologise if my Syntax needs some work, and I'm sorry I am not currently capable of just whipping up some Code to solve your issue, but I do know
using System;
Console.WriteLine("Hello World");
so at least it's not a total loss, but my example Code likely won't compile, since it's likely missing things like: an Output Location, perhaps an API reference or two, and probably a few other settings. Like I said, I'm New, but that is a valid C# Command, I know I got that part correct.
Edit: here's a better attempt:
using System;
class Test
{
static void Main()
{
string a = "Hello Hal, ";
string b = "Please open the Airlock Doors.";
string c = "I'm sorry Dave, "
string d = "I'm afraid I can't do that."
Console.WriteLine(a + b);
Console.WriteLine(c + d);
Console.Read();
}
}
This:
"Hello Hal, Please open the Airlock Doors."
"I'm sorry Dave, I'm afraid I can't do that."
Should be the result. (the "Quotation Marks" shouldn't appear in the readout {the last Code Block}, that's simply to improve readability)

Copy and paste in different formats

I have a datagrid that I want to be able to copy and paste to/from excel. Pretty common scenario. I have the copy and paste functions implemented. However, this application has several datagrids, and I'd like to prevent the user from trying to copy data from one grid to another since the data is different.
I can serialize the objects in these grids to any format I want, so adding some kind of metadata that says "This data only goes in that grid" is trivial. But I can't add the metadata because then it would show up in excel. Is there some solution to this problem that allows me to paste data in one format in my application, but that excel will still handle correctly?
If you look at the clipboard class you can set the text but there is also quite a bit more you can do with it. Most of the advanced things you will want to do with the clipboard revolve around a pair of routines "SetDataObject" and "GetDataObject". To use this with multiple formats you can specify:
var serializableObject = new MyObject();
var clipData = new DataObject();
clipData.SetData(DataFormats.Text, "abcdefg");
clipData.SetData("CustomFormat", serializableObject);
Clipboard.SetDataObject(data);
Once you have done this you can get the data back from the clipboard by reversing this and requesting the data from the custom format. Briefly the reverse call looks like:
var clipData = (DataObject)Clipboard.GetDataObject();
var myObject = clipData.GetData("CustomFormat") as MyObject;
For a more complete example from Microsoft, see this page: http://msdn.microsoft.com/en-us/library/637ys738(v=vs.110).aspx. Just look at the bottom where it explains the use of multiple formats.
Hope this helps. Best of luck!

Trying to work with Request.Files in WebMatrix

everyone,
I am trying to work with uploading images to my site, and I have successfully gotten it to work, however, I need to extend the functionality beyond that of one simple image. The examples I have seen use the WebMatrix File Helper (File Helper? Is that right? Oh well, it's a helper of some kind that auto plots the html necessary for the input=type"file" field). The line of code I have in the form:
#FileUpload.GetHtml(initialNumberOfFiles:1, allowMoreFilesToBeAdded:false, includeFormTag:false)
The line of code I have in (IsPost):
var UploadedPicture = Request.Files[0];
if(Path.GetFileName(UploadedPicture.FileName) != String.Empty)
{
var ContentType = UploadedPicture.ContentType;
var ContentLength = UploadedPicture.ContentLength;
var InputStream = UploadedPicture.InputStream;
Session["gMugshot"] = new byte[ContentLength];
InputStream.Read((byte[])Session["gMugshot"], 0, ContentLength);
}
else
{
Session["gMugshot"] = new byte[0];
}
More code in the (IsPost) after this stores it in the database as binary data, and I can get the image back on the page from that (I have no desire to save the actual image files in a folder on the server and use GUID, etc. etc. Binary data is fine, and I imagine takes up a lot less space).
I have it set up to click-ably scroll through pictures by using jQuery to read the clicks of manually created buttons and subsequently hide and unhide the divs that contain the images rendered by C# (which gets them from reading the database). Sorry if that's a little TMI, just trying to be thorough, but to refine the question: I don't know enough about file uploading to know how to work with the uploaded data that well yet. I tried researching this information, but I didn't find any information that seemed pertinent to me (actually, I didn't find much useful information on input type="file", or the FileUpload method, at all, really).
Would it be better to use input type="file" id="pic1id"? Is there something that I can use such as Request.Files["pic1id"] that could get the file from the id of the input element? Or does the program simply take all uploaded files, stick them in a logistical group somewhere waiting to be called by index like this: "Request.Files[0]". If so, what order does the index get put in? Do I need to use Request.Files.Count to test how many have been uploaded before I begin working with the data?
Please note that I want separate input type="file" fields (whether plotted by the helper or not). I do not want to accept multiple files in one input (mainly because of a lack of knowledge, e.g., I am afraid I won't know how to work with the data). So far, the plan is that the separate input type="file" fields will be within the divs that get hidden/unhidden upon scrolling through pictures, so each picture (space) will have its own input type="file" field. The hiding and unhiding of divs, (the one) picture being displayed, storing and receiving binary data from the database, and clicking through the picture placeholders all function great. Pretty much I just need to know how to work with more than one uploaded picture at a time for storage in their individual database "image" fields.
Any examples of the syntax I need to use will be much appreciated.
Sorry so many questions, but I just couldn't find much useful information on this at all.
Thanks to any who try to help!
Okay, in order to solve this, I had to test and test and test, until something finally worked for me. Here's what I did:
First, I abandoned my use of the part of the helper that plotted the html, that is I took out:
#FileUpload.GetHtml(initialNumberOfFiles:1, allowMoreFilesToBeAdded:false, includeFormTag:false)
And added a regular input type="file" with a certain id, such as id="pic1".
Next I was able to get the individual file post based on id, which was really the main thing I needed to know how to do, and it really was as simple as this:
Request.Files["pic1"];

Having trouble saving multiple items to Isolated Storage

I have a noteapp, two pages:
MainPage.xaml — the creation of notes;
NoteList.xaml — a list of notes.
Notes are saved by means of IsolatedStorage, and appear in NoteList.xaml (listbox), but notes with the same name is not stored, how to fix it?
I need to be able to add notes with the same name (but with different content).
Thanks!
Are you using the note name as the file name? If so... don't do that. Save each file with a unique name. There are myriad ways of doing this. You could use a GUID or a timestamp, or you could append a timestamp to the end of the file name. If you were so inclined you could store all of the notes in a single formatted file-- perhaps XML.
What you need is a way to uniquely identify each note without using:
a. The note's name
b. The note's contents
While using a timestamp might make sense for your application right now (since a user probably cannot create two disparate notes simultaneously), using a timestamp to identify each note could lead to problems down the line if you wanted to implement say... a server side component to your application. What happens if in version 23 of your application (which obviously sells millions in the first months), you decide to allow users to collaborate on notes, and a Note is shared between two instances of your app where they happened to be created at the EXACT same time? You'd have problems.
A reasonable solution to finding a unique identifier for each Note in your application is through the use of the Guid.NewGuid method. You should do this when the user decides to "save" the note (or if your app saves the note the moment it's created, or at some set interval to allow for instant "drafts".
Now that we've sufficiently determined a method of uniquely identifying each Note that your application will allow a user to create, we need to think about how that data should be stored.
A great way to do this is through the use of XmlSerializer, or better yet using the third party library Json.Net. But for the sake of simplicity, I recommend doing something a bit easier.
A simpler method (using good ole' plain text) would be the following:
1: {Note.Name}
2: {Guid.ToString()}
3: {Note.Contents}
4: {Some delimiter}
When you are reading the file from IsolatedStorage, you would read through the file line by line, considering each "chunk" of lines between the start of the file and each {Some delimiter} and the end of the file to be the data for one "Note".
Keep in mind there are some restrictions with this format. Mainly, you have to keep the user from having the last part of their note's contents be equal to the {Some delimiter} (which you are free to arbitrarily define btw). To this end, it may be helpful to use a string of characters the user is not likely to enter, such as "##&&ENDOFNOTE&&##" Regardless of how unlikely it is the user will type that in, you need to check to make sure before you save to IsolatedStorage that the end of the Note does not contain this string, because it will break your file format.
If you want a simple solution that works, use the above method. If you want a good solution that's scalable, use JSON or XML and figure out a file format that makes sense to you. I highly encourage you to look into JSON, it's value reaches so much further than this isolated scenario.
I've had a need to write notes to IsolatedStorage. What I did was to them them to a file.IsolatedStorageFile I write date on which the note was written and then note. From the list box i store them to two arrays. Then before exiting the app, write them to a file.
try
{
using (IsolatedStorageFile storagefile = IsolatedStorageFile.GetUserStoreForApplication())
{
if (storagefile.FileExists("NotesFile"))
{
using (IsolatedStorageFileStream fileStream = storagefile.OpenFile("NotesFile", FileMode.Open, FileAccess.ReadWrite))
{
StreamWriter writer = new StreamWriter(fileStream);
for (int i = 0; i < m_noteCount; i++)
{
//writer.Write(m_arrNoteDate[i].ToShortDateString());
writer.Write(m_arrNoteDate[i].ToString("d", CultureInfo.InvariantCulture));
writer.Write(" ");
writer.Write(m_arrNoteString[i]);
writer.WriteLine("~`");
}
writer.Close();
}
}

Implementing a save function in a C# image manipulation app

I started thinking about how to handle the save functionality of my app, and thought about 2 options:
The application has nodes like:
Blur
Contrast
Sharpen
Invert
...
1. Interpreting the saved file, like:
Blur name:"Blur01" Amount:5
...
2. Having the saved file in a self executable format, like:
Blur blur = new Blur ();
blur.Name = "Blur01"
blur.Amount = 5
...
Which one should I go for? Is there a better way to handle this?
I want the saved file to be backwards and forwards compatible.
EDIT: Thanks for all the replies. Anyone can please explain why #2 would not be future proof? Is it because one can change the load/open code for #1, but not for #2?
You could probably use XML Serialization, since it's widely accepted and human readable.
Here's a tutorial on that: XML Serialization
I would go with something more like the first option.
Although, in general, I think XML would be a better approach to this than making your own syntax. This is much better from a compatibility/future-proofing standpoint than trying to make your own syntax parsers for your file.
What about something like:
<Filters>
<Blur Name="Blur01" />
<Sharpen Name="Sharpen01" Amount=5 />
</Filters>
I too would go with an XML file as this will allow you to ensure compatibility both forwards and backwards.
This is because you look for properties rather than parsing the file line by line.
For example, if blur changes from:
<Blur>
<name>Blur01</name>
<amount>5</amount>
</Blur>
to:
<Blur>
<name>Blur01</name>
<amount>5</amount>
<feather>true</feather>
</Blur>
Older versions of the app will still be able to read the file as they won't look for the feather property. All you need to do is ensure that you set default values when you create your objects so that the older files can be read without leaving unset data.
In response to the update - there's no reason why you couldn't make #2 future proof. You'd just have to do the versioning yourself.
The reason having a self-executing "save format" is generally bad is that today your "Blur" function might look like:
public class Blur
{
int Amount = 5;
}
but in the future, you might improve your blur "system" to instead have something like:
public class Blur
{
int HorizontalAmount = 5;
int VerticalAmount = 10;
}
and now when you execute that saved file, it will no longer compile because there is no longer an 'Amount' property. Then to get backwards compatibility you will need to 'interpret' the Amount value to now mean HorizontalAmount = 5 AND VerticalAmount = 5 (or whatever).
So really, in the long run, you will be better off by having an interpreted file format from the start.
Are you saving it in a text file?
If that is so wouldn't it be better to save it as XML?
<Blur>
<name>Blur01</name>
<amount>5</amount>
</Blur>
Otherwise I am not sure I understand the question :)

Categories