Imagine there is a game with a lot of content, like car models.
I don't want to have them in the RAM all the time to save memory and want to load them only when needed. This would be easily when having one file per car - but I would have so many files at the end and the project would become hard to share etc.
But when I store everything into one file, I don't know where what is.
Is there a way storing everything into one file and navigate to the content as easily as if I do one file per content entry? Only thing I can imagine is saving the byte positions where something begins in a 2nd file (or as header in the content file) but I'm very unsure about that solution.
A simple way to do it is when you write the file you keep track (in memory) where stuff is. Then, at the end of the file you write an index. So your file looks like this:
offset 0: data for object 1
offset 0x0473: data for object 2
offset 0x1034: data for object 3
etc.
You can write this file using BinaryWriter. Before you write each object, you query writer.BaseStream.Position to get the current position of the file. You save that information in memory (just the object name and its position).
When you're done writing all of the objects, you save the current position:
indexPosition = writer.BaseStream.Position;
Then write the index at the end of the file:
name: "object 1", position: 0
name: "object 2", position: 0x0473
etc.
Write an empty index entry to signify the end of objects:
name: "", position: 0xFFFFFFFF
And the last thing you do is write the index position at the end of the file:
writer.Write(indexPosition);
Now, you can open the file with a BinaryReader. Seek to end-of-file minus 8 bytes, read the long integer there. That gives you the position of the index. Seek to the index and start reading index entries forward until you get to one that has a position of 0xFFFFFFFF.
You now have the index in memory. You can create a Dictionary<string, long> so that, given an object name, you can find its position in the file. Get the position, seek there, and read the object.
This kind of thing was pretty common when we were writing games in the late '90s. You can still do it, although you're probably better off going with a simple database.
There are quite a few different compression and archiving methods you could use to hold your files if you're looking to store them temporarily in a larger file for transport. Almost any compression method could work, which you choose is entirely up to you.
Example .zip compression can be found here:
http://msdn.microsoft.com/en-us/library/ms404280.aspx
There's also the .cab file format that you can easily pack multiple files into and unpack later when you have need for them. There's an interesting article on creating .cab files in C# found here:
http://mnarinsky.blogspot.com/2009/11/creating-cab-file-in-c.html
It does require that you add references to Microsoft.Deployment.Compression.Cab.dll and Microsoft.Deployment.Compression.dll but the code itself is fairly simple after that. If you find none of those to be a suitable answer to your question, my only other suggestion is that you better organize the files rather than cramming them all into a single folder as that can make it quite difficult to navigate.
Also try using a collection to keep track of the file names if that helps too. You could define files in the XML and load everything into a dictionary or hash table if it needs to be more dynamic or define it in the code itself if you prefer that.
EDIT:
You can also try using a third party installer for transport. They offer many functions outside of compressing and packing files and will handle the data compression for you. I prefer NSIS personally as I find it to be highly user friendly but any installer can work. A few example installers are:
Installshield: http://www.installshield.com/ (integrates with Visual Studio)
WIX: http://wix.sourceforge.net/ (also integrates with Visual Studio, good if you're looking for something more XML based)
NSIS: http://nsis.sourceforge.net/Main_Page (scriptable, doesn't integrate with Visual Studio, easy guided design with Nsisqssg if you'd prefer not to do the bulk of the scripting on your own)
They all function differently but essentially achieve the same end result. It all depends on exactly what you're looking for.
Related
I am trying to make a custom launcher for Minecraft in C# but I have come across a bump.
I want to add something into it, Minecraft Forge, but the only way I could think of is to change the extension of minecraft.jar to minecraft.zip, extract the contents of the Minecraft Forge.zip and the minecraft.zip into the same folder and then zip that entire folder up into minecraft.jar.
However minecraft.jar has a file named aux.class so whenever my extract script (Made in java) tries to extract it, it simply says:
Unable to find file G:\Programming\C#\Console\Forge Installer\Forge Installer\bin\Debug\Merge\aux.class.
The only other way I can think of is to merge minecraft_forge.zip into minecraft.zip, I have spent around 2 hours looking on Google (watch as someone sees it within a couple of minutes) but it always shows me results for "How to zip multiple files", "How to make a zip file in C#" etc.
So I have come here looking for my answer, sorry if this is a lot to read but I always see comments on here saying "You didn't give enough information for us to help you with".
EDIT: The question in case it wasn't clear is: How am I able to put the contents of minecraft_forge.zip into minecraft.zip?
In your case, if you cannot unzip the files due to OS limitations, you need to "skip" unzipping temporary files to zip them. Instead, only handle input & output streams, as suggested in the answers found here: How can I add entries to an existing zip file in Java?
As you pointed out, "aux" is a protected keyword within windows and it does not matter what the file suffix may be; windows won't let you use it. Here are a couple of threads that discusses this in general.
Ref 1: Windows reserved words.
Ref 2: Windows reserved words.
If you are typing in commands to perform the copy or unzip, there is a chance you can get this to work by using a path prefix of the following \\.\ or \\?\. When I tested this, it worked with either a single or double back-slash following the period or question mark. Such that the following work:
\\.\c:\paths\etc
\\.\\c:\paths\etc
\\?\c:\path\etc
\\?\\c:\path\etc
I used the following command to test this. When trying to rename through windows explorer it gave a "The specified device name is invalid." error message. From the command line it worked just fine. I should point out, that once you create these files, you will have to manually delete them using the same technique. Windows Explorer reports that these text files which have a size of 0 bytes "is too large for the destination file system", ie... the recycle bin.
rename "\.\c:\temp\New Text Document.txt" aux.txt
del "\.\c:\temp\aux.txt"
As far as copying directly from zip or jar files, I tried this myself and it appeared to work. I used 7-zip and opened the jars directly using the "open archive..." windows explorer context menu. I then dragged-and-dropped the contents from forge.jar to the minecraft jar file. Since it is the minecraft jar file with the offending file name the chance of needing to create a temporary file on the filesystem is reduced. I did see someone mention that 7-zip may extract to a temporary file when copying between jars and zips.
7-zip reference on copying between archives
I should point out that my copy of minecraft jar (minecraft_server.1.8.7.jar) did not contain a file named aux.class. I also did not try to use the jar after the copy/merge. Nor did I spend too much time trying to figure out how well it merged the two contents since it appears like there may be a conflict with com\google\common\base\ since there are similar class name but with different $ variable suffixes on them.
I hope these two possible suggestions could give you some room to work with to find a solution for your needs... if you're still looking.
I am working on an application that keeps track and uses files stored on the file system. Users are allowed to open, create, delete and move files in the file system. Meanwhile my application is not constantly running so I can't keep track of all changes real-time. Afterwards my application has to find out what file is whom (i.c. as identified in my application).
The most preferred solution for the users is that the application solves every change itself. Each user-interaction is less preferred.
One of my ideas was to use an attribute of a file and assign a key-value to it so when it has been identified once, it can always been recognized afterwards. But I don't know if there is such an attribute. This article didn't give much hope: There is in Windows file systems a pre computed hash for each file?.
Does somebody know if there is such an attribute I can use? And how can I used it in C#?
Is there anyone who is running up against this problem? And how did you solve it?
I'd like to hear good suggestions.
regards, Jaap
If your files don't leave NTFS, this is easily achievable by alternative data streams, where you can store your data along with files. This is more-or-less good article about ADS: http://www.flexhex.com/docs/articles/alternate-streams.phtml
There is another suitable method - it's very efficient, but also very complicated to use, it requires quite good knowledge about NTFS internals - USN Change Journal; see http://msdn.microsoft.com/en-us/library/windows/desktop/aa363798.aspx. With USN Change Journal, you can "get" very efficiently all files that were changed (even all change events) within specified time period.
Nevertheless, if your files leave NTFS realm, e.g. if it's copied to FAT32, contents of ADS is lost.
Relying on a File attribute is "dangerous" in that some user could alter the attribute while your program isn't running. This could lead you to believe that a certain file is (or isn't) tracked by the program while it really isn't.
I would suggest to keep track of the files in a database, XML, or some other file. When your application starts you read the file/db and check for new/deleted/editted files.
You could store a Hash of the files to find out if a file has been moved/editted. Keeping track of files that are moved AND editted is going to be pretty difficult. (I have no clue how you could achieve it)
PS: Have you considered making your application a Windows service? Having the file-management running in the background no matter if the GUI part of your application is running or not?
I have a wpf application and inside using Image resizer as a third party tool in that app. I am trying to achieve the following scenario:
I have text box (Employee Name) for example "Madona"
I have a image text box linked with third party tool "Image Resizer".
If i choose the image (eg: Madona123.jpg), the Image resizer will generate 17 different size images and drop it in my source folder (e.g. Madona123_80x60.jpg, Madona123_150x150.jpg, etc..)
But am trying to change the file name like e.g: Madona_80x60.jpg, Madona_150x150.jpg
I have two options but not sure which one is effective
Create Temp Dir from the source folder where the image is there and change the name based on the "Employee Name" and pass that image to "Image Resizer" tool
Once the Image is generated in the destination folder,then change the file name.
Am not sure which one is best and what other concerns i have to look ,thanks for your input.
Option 1 has an extra copy command - you can rename when you copy. But with option 2 you have multiple renames. I would do 1 for simplicity.
From your suggested scenarios I would go for option 2.
As you stated, creating an extra temp folder might cause rights issues so if you're not entirely certain about the setup of the machines that will be using the app then option 1 might be causing your app to fail.
As a secondary (and very minor) reason, option 2 will, even if it doesn't fail on rights still create an extra folder. If you would choose to make that temp folder part of your application, people have the tendency of removing temp folders and their contents.. Or you might create it every time on startup of the app, but that also requires, a little bit, coding. (altough point taken, this is just a minor reason and not really a reason that would tip the scales..)
I do agree with Blam's statement that option one would make the base coding a bit more simple but I think aldo the bit of extra work, option 2 will give you less possibility for failure.
So if you're gonna be using it in a manageable environment where you know the setup of your targetmachines, go with option one, it requires less and more straightforward coding, if you're gonna be using it in any other sort of environment, go with option 2. It 'll give you the highest chance of success.
Aspiring programmer here!
You can skip this part if you find it irrelevant.
Im currently working on a project where i give every program a username and i let people edit their own part of of each file. This will work more or less like onenote (great program), but i want to be able to restrict access to certian parts of it. I also need the file to be stored locally (on a shared location that is). In time i also want the files to be encrypted, but ill stick with the editing part of my question for now.
So my real problem:
I need people to be able to edit their part of a .txt file. The setup will probably look like this:
Document name:
Task 1: (Task name, like "Write a paragraph about Einstein"
User 1: (User 1s answer)
User 2: (User 2s answer and etc.)
Then ill follow up with more tasks / users. What i need to make my program understand is to read-only document name and other users answer, and read-write on their own part of the file.
What code can i use to accomplish this?
You won't be able to do that in a text file. You'll have to spend a little bit more effort in solving that task. Use a database and manage those tasks in a table and the user responses in another table. That will shurely work much better, trust me.
If you really want to use a text based solution for this instead of a database I would look into XML files.
They give you the ability to structure a document and have capabilities of being queried by C# trough XPath or Linq to XML queries.
This documentation should get you started.
Yea a database (use an MSAccess MDB file if you don't have a SQL server) would be best for this.
However, if you really want to use a text file, what you do is
1. Use System.IO to read the file and parse the file to get the piece of data you want them to edit (only read it out of the file, don't modify the file at this point).
2. Display the piece of data to them in a textbox or whatever.
3. After they've mdofied it, then read in the whole file into a variable
4. replace the old piece of data with the new data. in the variable.
5. output the entire variable back into the file and save and close it.
If this is REALLY what you want to do, i can post some code for you.
I'm creating an WP7 app that shows an inspirational text for every day and allows you to mark some of this texts as favorites. You can see the text for today, jump to an day in the calendar oder browse your favorites.
All texts are known prior roll out / installation, I don't want to lazy load them via cloud/web, I want to "install" them together with the app.
How should I store them? Should I use one of the open source databases for WP7 and create all rows on installation? Should I just hardcode them and save the favorites in an IsolatedStorage file?
EDIT: Is it possible to have the read only data in a XML file in the Visual Studio Project and mark it as a ressource? Will this later roll out the file automatically? Does this make sense?
If your concern is speed of loading / efficiency of reading the files then you'll have to test to see what works best. I'd start with what's simplest to implement and then change if necessary.
What is right for your app will depend on the total size of data and the size of individual pieces of text. As well as considering where you store the data, be sure to also consider the format you store it in as deserialization/parsing is also an overhaed you should consider.
Remember to test this on an actual device as the performance you see on the emulator is not likely to be realistic of what your users will see.
Update
If it's readonly data you probably want to add it as multiple content files (set the build Action) within the XAP.
The format of the files and how you divide the data between them will depend on the data and the app.
Having multiple files means you don't have to load all the data at once. (I assume you don't need to do that.) Just open the file you need.
Update 2
For an example of loading a resource file from the XAP see: http://blogs.msdn.com/b/silverlight_sdk/archive/2010/08/27/loading-a-static-xml-file-to-your-windows-phone-silverlight-app.aspx