We Keep Coding

XML Serialization leaves file blank after restart

We have a problem where our industrial equipments software's .XML settings files become blank, yet they still have the correct number of bytes.
I have a feeling it might be caused by the way the customers are shutting down the PC as it tends to happen after they've down a shutdown, isolate, and boot. The way I save the files is,
Serialize to %temp% file
Validate that the newly created file starts with <?xml
If the /backup folders version of the file is older than a day, copy the existing file to the /backup folder
Copy new file to overwrite existing file.
I thought maybe it's related to encoding, disk caching, Windows Update, or Windows Recovery.
Looking for ideas as I've spent two years chasing down why this is happening.
As per request, here is the code.
public static bool SerializeObjXml(object Object2Serialize, string FilePath, Type type, bool gzip = false)
{
if (!Path.IsPathRooted(FilePath))
FilePath = Path.Combine(ApplicationDir, FilePath);
bool isSuccess = false;
var tmpFile = Path.GetTempFileName();
try
{
for (int i = 0; i < 3; i++)
{
try
{
Directory.CreateDirectory(Path.GetDirectoryName(FilePath));
if (gzip)
{
using (var ms = new MemoryStream())
{
XmlSerializer bf = new XmlSerializer(type);
bf.Serialize(ms, Object2Serialize);
ms.Position = 0;
using (var fileStream = new BinaryWriter(File.Open(tmpFile, FileMode.Create)))
{
using (GZipStream gzipStream = new GZipStream(fileStream.BaseStream, CompressionMode.Compress))
{
byte[] buffer = new byte[4096];
int numRead;
while ((numRead = ms.Read(buffer, 0, buffer.Length)) != 0)
{
gzipStream.Write(buffer, 0, numRead);
}
}
}
}
if (!FileChecker.isGZip(tmpFile))
throw new XmlException("Failed to write valid XML file " + FilePath);
}
else
{
using (var fs = new StreamWriter(File.Open(tmpFile, FileMode.Create), Encoding.UTF8))
{
XmlSerializer bf = new XmlSerializer(type);
bf.Serialize(fs, Object2Serialize);
}
if (!FileChecker.isXML(tmpFile))
throw new XmlException("Failed to write valid XML file " + FilePath);
}
isSuccess = true;
return true;
}
catch (XmlException)
{
return false;
}
catch (System.IO.DriveNotFoundException) { continue; }
catch (System.IO.DirectoryNotFoundException) { continue; }
catch (System.IO.FileNotFoundException) { continue; }
catch (System.IO.IOException) { continue; }
}
}
finally
{
if (isSuccess)
{
lock (FilePath)
{
try
{
//Delete existing .bak file
if (File.Exists(FilePath + ".bak"))
{
File.SetAttributes(FilePath + ".bak", FileAttributes.Normal);
File.Delete(FilePath + ".bak");
}
}
catch { }
try
{
//Make copy of file as .bak
if (File.Exists(FilePath))
{
File.SetAttributes(FilePath, FileAttributes.Normal);
File.Copy(FilePath, FilePath + ".bak", true);
}
}
catch { }
try
{
//Copy the temp file to the target
File.Copy(tmpFile, FilePath, true);
//Delete .bak file if no error
if (File.Exists(FilePath + ".bak"))
File.Delete(FilePath + ".bak");
}
catch { }
}
}
try
{
//Delete the %temp% file
if (File.Exists(tmpFile))
File.Delete(tmpFile);
}
catch { }
}
return false;
}
public static class FileChecker
{
const string gzipSig = "1F-8B-08";
static string xmlSig = "EF-BB-BF";// <?x";
public static bool isGZip(string filepath)
{
return FileChecker.CheckSignature(filepath, (3, gzipSig)) != null;
}
public static bool isXML(string filepath)
{
return FileChecker.CheckSignature(filepath, (3, xmlSig)) != null;
}
public static bool isGZipOrXML(string filepath, out bool isGZip, out bool isXML)
{
var sig = FileChecker.CheckSignature(filepath, (3, gzipSig), (3, xmlSig));
isXML = (sig == xmlSig);
isGZip = (sig == gzipSig);
return isXML || isGZip;
}
public static string CheckSignature(string filepath, params (int signatureSize, string expectedSignature)[] pairs)
{
if (String.IsNullOrEmpty(filepath))
throw new ArgumentException("Must specify a filepath");
if (String.IsNullOrEmpty(pairs[0].expectedSignature))
throw new ArgumentException("Must specify a value for the expected file signature");
int signatureSize = 0;
foreach (var pair in pairs)
if (pair.signatureSize > signatureSize)
signatureSize = pair.signatureSize;
using (FileStream fs = new FileStream(filepath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
if (fs.Length < signatureSize)
return null;
byte[] signature = new byte[signatureSize];
int bytesRequired = signatureSize;
int index = 0;
while (bytesRequired > 0)
{
int bytesRead = fs.Read(signature, index, bytesRequired);
bytesRequired -= bytesRead;
index += bytesRead;
}
foreach (var pair in pairs)
{
string actualSignature = BitConverter.ToString(signature, 0, pair.signatureSize);
if (actualSignature == pair.expectedSignature)
return actualSignature;
}
}
return null;
}
}

Using the operating system's move or copy file to overwrite an existing file is an atomic operation meaning the it wholly succeeds or doesn't and doesn't overlap other file operations.
Therefore what you have should work if that is how you are achieving step 4.
Copy new file to overwrite existing file.
If instead you are blanking out the existing file and re-writing the data I suspect that could be the the point of failure..

The issues while file space is being allocated the write is not occurring during shutdown, which leaves you when a file with bytes allocated without the data being flushed to disk.
During the OS shutdown, likely a ThreadAbortException is raised which triggers your finally block.
You can attempt to reproduce by calling Process.Start("shutdown", "-a") before your return statement but after you have set success = true.
I would suggest simplifying your code and have everything run inside of your try {} statement. This removes the possibility of having a state where success = true before your attempted your write to disk, which is then triggered in a finally statement trigged by a windows shutdown.
public static bool SerializeObjXml(
object Object2Serialize,
string FilePath,
Type type,
bool gzip = false)
{
if (!Path.IsPathRooted(FilePath))
FilePath = Path.Combine(ApplicationDir, FilePath);
Directory.CreateDirectory(FilePath);
for (int i = 0; i < 3; i++)
{
try
{
var tempFi = SerializeToXmlFile(Object2Serialize, type, gzip);
var fi = new FileInfo(FilePath);
if (fi.Exists)
fi.CopyTo(fi.FullName + ".bak", true);
tempFi.CopyTo(fi.FullName, true);
tempFi.Delete();
return true;
}
catch (Exception ex)
{
string message = $"[{DateTime.Now}] Error serializing file {FilePath}. {ex}";
File.WriteAllText(FilePath + ".log", message);
}
}
return false;
}
As a side note, you can simply use [Stream.CopyTo][1] and write directly to your temp file, without the need for intermediary streams or for manual buffer/byte read/write operations:
private static FileInfo SerializeToXmlFile(
object Object2Serialize,
Type type,
bool gzip)
{
var tmpFile = Path.GetTempFileName();
var tempFi = new FileInfo(tmpFile);
if (!gzip)
{
using (var fs = File.Open(tmpFile, FileMode.Create))
(new XmlSerializer(type)).Serialize(fs, Object2Serialize);
if (!FileChecker.isXML(tmpFile))
throw new Exception($"Failed to write valid XML file: {tmpFile}");
}
else
{
using (var fs = File.Open(tmpFile, FileMode.CreateNew))
using (var gz = new GZipStream(fs, CompressionMode.Compress))
(new XmlSerializer(type)).Serialize(fs, Object2Serialize);
if (!FileChecker.isGZip(tmpFile))
throw new Exception($"Failed to write valid XML gz file: {tmpFile}");
}
return tempFi;
}

c# block length does not match with its complement

I am stuck with this error and I don't understand why. Can you help me out? I am trying to edit a PowerPoint file on a Sharepoint through the Client Object Model and OpenXML. The editing is working fine but for some reason I get thrown the block length does not match with its complementError. I have not yet found anything on google that helped.
Here is the Code in question:
class Program
{
// Credentials to log into the Sharepoint.
private static NetworkCredential credential = new NetworkCredential("login", "pwd", "domain");
static private void CopyStream(Stream source, Stream destination)
{
byte[] buffer = new byte[32768];
int bytesRead;
do
{
bytesRead = source.Read(buffer, 0, buffer.Length);
destination.Write(buffer, 0, bytesRead);
} while (bytesRead != 0);
}
static void Main(string[] args)
{
try
{
// Creating the Clientcontext
ClientContext clientContext = new ClientContext("https://sharepoint.adress.com/folder/");
clientContext.Credentials = credential;
Web oWebsite = clientContext.Web;
// Collect all lists from Sharepoint.
ListCollection collList = oWebsite.Lists;
clientContext.Load(collList);
clientContext.ExecuteQuery();
Console.WriteLine("Connected to Sharepoint.");
// Getting the Presentation.
Console.WriteLine("Working on " + filename + "..");
Guid id = new Guid("2BBF9030-66C6-4F71-86E6-DFE41F57B9F3"); // List ID for the Slide Library.
List sharedDocumentsList = clientContext.Web.Lists.GetById(id);
CamlQuery camlQuery = new CamlQuery();
String queryString = "<View><Query><Where><Eq><FieldRef Name='FileLeafRef'/><Value Type='Text'>" + filename +
"</Value></Eq></Where><RowLimit>1</RowLimit></Query></View>"; // FileLeafRef is the column "name" on the Sharepoint.
camlQuery.ViewXml = #queryString;
ListItemCollection listItems = sharedDocumentsList.GetItems(camlQuery);
clientContext.Credentials = credential;
clientContext.Load(sharedDocumentsList);
clientContext.ExecuteQuery();
clientContext.Credentials = credential;
clientContext.Load(listItems);
clientContext.ExecuteQuery();
Console.WriteLine("Opened Document.");
if (listItems.Count == 1)
{
ListItem item = listItems[0];
clientContext.Credentials = credential;
FileInformation fileInformation = ClientOM.File.OpenBinaryDirect(clientContext, (string)item["FileRef"]);
using (MemoryStream memoryStream = new MemoryStream())
{
CopyStream(fileInformation.Stream, memoryStream);
using (PresentationDocument ppt = PresentationDocument.Open(memoryStream, true))
{
string rId = GetSlidePart(ppt, slideNumber - 1);
PresentationPart pptPart = ppt.PresentationPart;
SlidePart slide = (SlidePart)pptPart.GetPartById(rId);
Boolean exists = false;
//groupExists(slide.Slide, "IDGroup", exists);
if (!groupExists(slide.Slide, "IDGroup", exists))
{
Console.WriteLine(exists);
Console.WriteLine("IDGroup not found in " + filename + ", adding it..");
EditPresentation(memoryStream, clientContext, item);
}
else
{
Console.WriteLine(exists);
Console.WriteLine("IDGroup already exists, skipping Presentation..");
}
} // Error gets thrown here.
}
}
else
{
Console.WriteLine("Document not found.");
}
Console.WriteLine("Done.");
Console.Read();
}
catch (Exception ex) when (ex is ServerException || ex is InvalidCastException || ex is OpenXmlPackageException || ex is WebException)
{
Console.WriteLine(ex.ToString());
Console.Read();
}
}
And this is the Method that edits the File. Trimmed to the parts where I use the memoryStream, the full code can be viewed on pastebin:
public static void EditPresentation(MemoryStream memoryStream, ClientContext clientContext, ListItem item)
{
using (PresentationDocument ppt = PresentationDocument.Open(memoryStream, true))
{
string rId = GetSlidePart(ppt, slideNumber - 1);
PresentationPart pptPart = ppt.PresentationPart;
SlidePart slide = (SlidePart)pptPart.GetPartById(rId);
ShapeTree tree = new ShapeTree();
CommonSlideData commonSlideData1 = new CommonSlideData();
// 250 Lines of OpenXML go here.
slide.AddHyperlinkRelationship(new System.Uri(adress, System.UriKind.Absolute), true, "rId99");
slide.Slide.CommonSlideData.ShapeTree.Append(groupShape1);
slide.Slide.Save();
pptPart.Presentation.Save();
}
memoryStream.Seek(0, SeekOrigin.Begin);
ClientOM.File.SaveBinaryDirect(clientContext, (string)item["FileRef"], memoryStream, true);
}
Is there something fundamental I am doing wrong? I've never done anything like this and as always all suggestions for improvements would be appreciated.

Using Renci ssh to upload to SFTP directory

I am not getting an error but my files are not getting uploaded. Am trying to upload to a targetDirectory on SFTP.
public string TryUploads(string targetDirectory)
{
string _localDirectory = LocalDirectory; //The directory in SFTP server where the files are present
if (oSftp == null)
{
oSftp = Instance;
}
lock (thisLock)
{
try
{
oSftp.Connect();
List<string> fileList = Directory.GetFiles(_localDirectory, "*.*").ToList<string>();
oSftp.ChangeDirectory(targetDirectory);
if (fileList != null && fileList.Count() > 1)
{
for (int i = 0; i < fileList.Count(); i++)
{
string ftpFileName = Path.GetFileName(fileList[i]);
if (!String.IsNullOrEmpty(targetDirectory))
ftpFileName = String.Format("{0}/{1}", targetDirectory, ftpFileName);
using (var stream = new FileStream(Path.GetFileName(fileList[i]), FileMode.Create))
{
oSftp.BufferSize = 4 * 1024;
oSftp.UploadFile(stream, ftpFileName);
// stream.Close();
}
}
}
oSftp.Disconnect();
}
catch (Exception e)
{
throw new ApplicationException(e.Message);
}
}
return Strings.StatusOk;
}

I solved the issue. I needed to put the location of where am getting the file from in the stream
using (var stream = new FileStream(LocalDirectory + "\\" + Path.GetFileName(fileList[i]), FileMode.Open))
Solved. Files uploaded ;)

Attempted to seek before beginning of stream inside a using() statement

So I got the IOException: Attempted to Seek before the beginning of the stream. But when I looked into it the seek statement was inside of a using statement. I might be missunderstanding the using() because as far as I knew this initializes the in this case filestream before running the encased code.
private string saveLocation = string.Empty;
// This gets called inside the UI to visualize the save location
public string SaveLocation
{
get
{
if (string.IsNullOrEmpty(saveLocation))
{
saveLocation = Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory) + #"\Pastes";
Initializer();
}
return saveLocation;
}
set { saveLocation = value; }
}
And this is the function it calls
private void Initializer()
{
// Check if the set save location exists
if (!Directory.Exists(saveLocation))
{
Debug.Log("Save location did not exist");
try
{
Directory.CreateDirectory(saveLocation);
}
catch (Exception e)
{
Debug.Log("Failed to create Directory: " + e);
return;
}
}
// Get executing assembly
if (string.IsNullOrEmpty(executingAssembly))
{
string codeBase = Assembly.GetExecutingAssembly().CodeBase;
UriBuilder uri = new UriBuilder(codeBase);
executingAssembly = Uri.UnescapeDataString(uri.Path);
}
// Get the last received list
if (!string.IsNullOrEmpty(executingAssembly))
{
var parent = Directory.GetParent(executingAssembly);
if (!File.Exists(parent + #"\ReceivedPastes.txt"))
{
// empty using to create file, so we don't have to clean up behind ourselfs.
using (FileStream fs = new FileStream(parent + #"\ReceivedPastes.txt", FileMode.CreateNew)) { }
}
else
{
using (FileStream fs = new FileStream(parent + #"\ReceivedPastes.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
if (fs.Seek(-20000, SeekOrigin.End) >= 0)
{
fs.Position = fs.Seek(-20000, SeekOrigin.End);
}
using (StreamReader sr = new StreamReader(fs))
{
while (sr.ReadLine() != null)
{
storedPastes.Add(sr.ReadLine());
}
}
}
}
}
isInitialized = true;
}

Are the commentors have posted: the file is less than 20000 bytes. It seems like you assume that Seek will stay at position 0 if the file is not large enough. It doesn't. It throws ArgumentException in that case.
Another thing. Seek will move the position for you. No need to do both. Either use:
fs.Seek(-20000, SeekOrigin.End);
or set the position:
fs.Position = fs.Length - 20000;
So what you really wanted to write is:
if (fs.Length > 20000)
fs.Seek(-20000, SeekOrigin.End);

WPF: Join Images in a Single File

My name's Lucas and I'm learning about WPF/C#.
I would like to join several images in a single file, as well as a game that use multiple textures that are all in one file, but I have no idea how to do. Could anyone help me at least to think how to do?
//Convert Image to Byte[]
public byte[] getByteFromImage()
{
byte[] imageArray = File.ReadAllBytes(op.FileName);
return imageArray;
}
//Convert Byte[] to Image
public void getImageFromByte()
{
FileStream f = new FileStream("escudos.bcf", FileMode.Open);
BinaryReader b = new BinaryReader(f);
Int64 c = f.Length+1;
MemoryStream ms = new MemoryStream(b.ReadBytes(int.Parse(c.ToString())));
Image image = new Image();
image.Source = BitmapFrame.Create(ms, BitmapCreateOptions.None,
BitmapCacheOption.OnLoad);
imgPatch2.Source = image.Source;
f.Dispose();
}
//Create Binary File
public void save(byte[] img)
{
FileStream f;
if (!File.Exists("escudos.bcf"))
{
f = new FileStream("escudos.bcf", FileMode.Create);
}
else
{
f = new FileStream("escudos.bcf", FileMode.Append);
}
BinaryWriter b = new BinaryWriter(f);
b.Write(img);
b.Close();
f.Dispose();
}
I thought in doing so, create a file and store it in binary images.
Until I got that part, but as this file will have multiple images in binary, I do not know how to pick just one binary image.
public void xmlCreate(string name, Int64 ini, Int64 fin)
{
if (!File.Exists("Escudos.xml"))
{
XmlTextWriter w = new XmlTextWriter("Escudos.xml", System.Text.Encoding.UTF8);
w.Formatting = Formatting.Indented;
w.WriteStartDocument();
w.WriteStartElement("Time");
w.WriteStartElement(name);
w.WriteElementString("Inicio", ini.ToString());
w.WriteElementString("Fim", fin.ToString());
w.WriteEndElement();
w.WriteEndDocument();
w.Close();
}
else
{
XDocument doc = XDocument.Load("Escudos.xml");
doc.Root.Add(new XElement(name));
doc.Root.Element(name).Add(new XElement("Inicio", ini.ToString()));
doc.Root.Element(name).Add(new XElement("Fim", fin.ToString()));
doc.Save("Escudos.xml");
}
}
Now I have created an xml file to store the start and end of the bytes. I can add only when I create a new xml file, I can not get a xml created and add new bytes. When I go to load the xml file gives an error message.
" An unhandled exception of type 'System.Xml.XmlException' occurred in System.Xml.dll
Additional information: '>' is an unexpected token. The expected token is '='. Line 3, position 15. "
UPDATE
when I'm reading the bytes to form an image, always the same way, even I adding different images. I'll add code below
//Add Image
private void btAddImage_Click(object sender, RoutedEventArgs e)
{
OpenFileDialog op = new OpenFileDialog();
op.Title = "Selecione a Imagem";
op.Filter = "All supported graphics|*.jpg;*.jpeg;*.png|" +
"JPEG (*.jpg;*.jpeg)|*.jpg;*.jpeg|" +
"Portable Network Graphic (*.png)|*.png";
if (op.ShowDialog() == true)
{
imgPatch.Source = new BitmapImage(new Uri(op.FileName));
txtName.Focus();
}
}
//Convert Image
private void btConvertImage_Click(object sender, RoutedEventArgs e)
{
if (String.IsNullOrEmpty(txtName.Text))
{
txtName.Focus();
MessageBox.Show("Preencha o Nome", "Error");
}
else
{
save(ConvertFileToByteArray(op.FileName), txtName.Text);
}
}
//Image to Byte Array
private static byte[] ConvertFileToByteArray(String FilePath)
{
return File.ReadAllBytes(FilePath);
}
//Save Binary File and XML File
public void save(byte[] img, string nome)
{
FileStream f;
long ini, fin = img.Length;
if (!File.Exists("Escudos.bcf"))
{
f = new FileStream("Escudos.bcf", FileMode.Create);
ini = 0;
}
else
{
f = new FileStream("Escudos.bcf", FileMode.Append);
ini = f.Length + 1;
bin = new TestBinarySegment();
}
bin.LoadAddSave("Escudos.xml", "Brasileiro", nome, ini, fin);
BinaryWriter b = new BinaryWriter(f);
b.Write(img);
b.Close();
f.Dispose();
}
//Load Image from Byte
private void btLoad_Click(object sender, RoutedEventArgs e)
{
getImageFromByte();
}
//Byte to Image
public void getImageFromByte(int start, int length)
{
using (FileStream fs = new FileStream("Escudos.bcf", FileMode.Open))
{
byte[] iba = new byte[fs.Length+1];
fs.Read(iba, start, length);
Image image = new Image();
image.Source = BitmapFrame.Create(fs, BitmapCreateOptions.None,
BitmapCacheOption.OnLoad);
imgPatch2.Source = image.Source;
}
}
Thanks

You have a binary file where its contents are segments and each segment contains the binary information needed to create an image. You need to store the starting index and length of each segment so that you can retrieve it. One way to do this is with an xml file.
To begin, create a container class for the segments. It looks like this...
public class BinarySegment
{
private const string FileName = "SegmentData.xml";
private static XmlSerializer serializer = new XmlSerializer(typeof(List<BinarySegment>));
public string SegmentName { get; set; }
public long SegmentStartIndex { get; set; }
public long SegmentLength { get; set; }
public static List<BinarySegment> LoadFromFile()
{
if (!File.Exists(FileName))
{
throw new Exception("File must be created first");
}
try
{
using (StreamReader sr = new StreamReader(FileName))
{
return serializer.Deserialize(sr) as List<BinarySegment>;
}
}
catch
{
throw new Exception("File as become corrupted");
}
}
public static void Save(List<BinarySegment> list)
{
try
{
using (StreamWriter sw = new StreamWriter(FileName))
{
serializer.Serialize(sw, list);
}
}
catch
{
throw;
}
}
}
There is one instance of this class for each image in your binary file. It will read/write a List of BinarySegments.
To test the class, create a test class like this...
public class TestBinarySegment
{
public TestBinarySegment()
{
List<BinarySegment> myBinarySegments = new List<BinarySegment>();
myBinarySegments.Add(new BinarySegment{SegmentName = "Segment1", SegmentStartIndex = 0, SegmentLength = 1111});
myBinarySegments.Add(new BinarySegment { SegmentName = "Segment2", SegmentStartIndex = 1111, SegmentLength = 1111 });
myBinarySegments.Add(new BinarySegment { SegmentName = "Segment3", SegmentStartIndex = 2222, SegmentLength = 1111 });
BinarySegment.Save(myBinarySegments);
}
public void LoadAddSave()
{
List<BinarySegment> myBinarySegments = BinarySegment.LoadFromFile();
myBinarySegments.Add(new BinarySegment { SegmentName = "Segment4", SegmentStartIndex = 333330, SegmentLength = 1111 });
BinarySegment.Save(myBinarySegments);
}
}
This class shows how to create the list and to save it. It also shows how to add new segments and resave the xml file.
When the test is run, you get a file like this...
<?xml version="1.0" encoding="utf-8"?>
<ArrayOfBinarySegment xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<BinarySegment>
<SegmentName>Segment1</SegmentName>
<SegmentStartIndex>0</SegmentStartIndex>
<SegmentLength>1111</SegmentLength>
</BinarySegment>
<BinarySegment>
<SegmentName>Segment2</SegmentName>
<SegmentStartIndex>1111</SegmentStartIndex>
<SegmentLength>1111</SegmentLength>
</BinarySegment>
<BinarySegment>
<SegmentName>Segment3</SegmentName>
<SegmentStartIndex>2222</SegmentStartIndex>
<SegmentLength>1111</SegmentLength>
</BinarySegment>
<BinarySegment>
<SegmentName>Segment4</SegmentName>
<SegmentStartIndex>333330</SegmentStartIndex>
<SegmentLength>1111</SegmentLength>
</BinarySegment>
</ArrayOfBinarySegment>
To run the test, use code like this...
TestBinarySegment test = new TestBinarySegment();
test.LoadAddSave();
This shows how to use xml serialization to create a file and add new segments to it. You will need to test this concept and then integrate it into your project.