We Keep Coding

How to iterate through LiteDB.LiteCollection for specific key values from a large litedb database file

I am creating a standalone database using LiteDB with C# code for storing every files metadata like filesize, absolute path ,createtime, extension etc. I am having problem in querying and iterating for the exact key value from the db file, please help out Thankyou.
This is my Insert query which is taking the absolute paths of every files from directories and passing the path as variable to my class for setting the file metadata values and returning the object with values to store in the LiteDB.
/INSERT QUERY/
public void Search(/* Drive paths */)
{
foreach(string file in Directory.GetFiles(path, "."))
{
try
{
using (var db = new LiteDatabase(/path of my DB file/))
{
var FileDBptr = db.GetCollection("FileDB");
var data = new FileInfoDB(file);
FileDBptr.Insert(data);
Console.WriteLine("SUCCESS INSERT");
}
}
catch (Exception e)
{
Console.WriteLine("COuld Not Create DB!!!!" + e);
}
}
foreach (string subDir in Directory.GetDirectories(path))
{
try
{
Search(subDir);
}
catch
{
throw;
}
}
}
/* CLASS MODEL */
[Serializable]
public class FileInfoDB
{
[BsonId]
public ObjectId FileId { get; set; }
public string FileName { get; set; }
public string FilePath { get; set; }
public string FileExtension { get; set; }
public long FileSize { get; set; }
public DateTime FileCreateTime { get; set; }
public DateTime FileLastAccess { get; set; }
public DateTime FileLastWrite { get; set; }
public FileInfoDB(string path)
{
if (File.Exists(path))
{
try
{
FileInfo f = new FileInfo(path);
FileId= ObjectId.NewObjectId();
FileName = f.Name;
FilePath = f.FullName;
FileCreateTime = f.CreationTime;
FileSize = f.Length; //Size in bytes
FileExtension = f.Extension;
FileLastAccess = f.LastAccessTime;
FileLastWrite = f.LastWriteTime;
}
catch (Exception)
{
throw;
}
}
}
}
//QUERY CODE FOR RETRIEVAL
string fileName = "$ICN03Z0.txt";
try
{
using (var db = new LiteDatabase(_jsonpath))
{
var FileDBptr = db.GetCollection<FileInfoDB>("FileDB");
FileDBptr.EnsureIndex(x=>x.FileName);
var data2 = FileDBptr.Find(Query.EQ("FileName", fileName));
int c = FileDBptr.Count();
Console.WriteLine(c); //Correct output
if (data2 !=null)
{
foreach (var a in data2) //Throwing an Exception
{
Console.WriteLine(a.FileName);
}
}
}
}
This is the data format which is stored in lite database file
{
"_id": {"$oid":"5c4ebee0f2e2d05814dcf865"},
"FileName": "$ICN03Z0.txt",
"FilePath": "C:\$Recycle.Bin\S-1-5-21-3439349906-2439027251-2956315770-1001\$ICN03Z0.txt",
"FileExtension":".txt",
"FileSize": {"$numberLong":"114"},
"FileCreateTime": {"$date":"2019-01-16T09:04:16.0810000Z"},
"FileLastAccess": {"$date":"2019-01-16T09:04:16.0810000Z"},
"FileLastWrite": {"$date":"2019-01-16T09:04:16.0810000Z"}
}
I expect to search the value according to filename first and then extract all the other key value pairs data of same file.
I tried with different query using LINQ as mentioned in GitHub but always throwing exception. Also have verified the data stored inside Lite database using LiteDB shell its inserting as the format above, but retrieval is giving problem.
Thanks in Advance!

Read file/folder Availability Status

C# - I'm building a console app to read the OneDrive folder status.
I can read so many attributes of file/folder but don't know how to get the Status column value here.
UPDATED: This approach is different that the one that describe here (using Shell32, registry...)
How can I check local OneDrive folder is in sync?
So it is not duplicated question

Add reference to Shell32 e.g C:\Windows\SysWOW64\shell32.dll
( I did try the WindowsAPICodePack ShellPropertyCollection but that did not work )
public static class OneDriveExtensions
{
private static int GetAvailabilityStatusIndex(Folder folder)
{
var index = 0;
while (true)
{
var details = folder.GetDetailsOf(folder, index);
if (details == "Availability status")
{
return index;
}
index++;
}
}
public static string OneDriveAvailability(this FileInfo file)
{
int availabilityStatusIndex;
return OneDriveAvailability(file, out availabilityStatusIndex);
}
public static string OneDriveAvailability(this FileInfo file,out int availabilityStatusIndex)
{
Shell shell = new Shell();
Folder objFolder = shell.NameSpace(file.DirectoryName);
availabilityStatusIndex = GetAvailabilityStatusIndex(objFolder);
return objFolder.GetDetailsOf(objFolder.ParseName(file.Name), availabilityStatusIndex);
}
public static string OneDriveAvailability(this FileInfo file, int availabilityStatusIndex)
{
Shell shell = new Shell();
Folder objFolder = shell.NameSpace(file.DirectoryName);
FolderItem objFolderItem = objFolder.ParseName(file.Name);
return objFolder.GetDetailsOf(objFolderItem, availabilityStatusIndex);
}
public static IEnumerable<OneDriveFileInfo> OneDriveAvailability(this DirectoryInfo directory,Func<DirectoryInfo,IEnumerable<FileInfo>> files)
{
var requireIndex = true;
int availabilityStatusIndex = 0;
return files(directory).Select(f =>
{
string oneDriveAvailability;
if (requireIndex)
{
requireIndex = false;
oneDriveAvailability= f.OneDriveAvailability(out availabilityStatusIndex);
}
else
{
oneDriveAvailability= f.OneDriveAvailability(availabilityStatusIndex);
}
return new OneDriveFileInfo(oneDriveAvailability, f);
});
}
public static IEnumerable<OneDriveFileInfo> OneDriveAvailability(this IEnumerable<FileInfo> files,int availabilityIndex)
{
return files.Select(f => new OneDriveFileInfo(f.OneDriveAvailability(availabilityIndex), f));
}
}
public class OneDriveFileInfo
{
public OneDriveFileInfo(string availabilityStatus, FileInfo file)
{
this.AvailabilityStatus = availabilityStatus;
this.File = file;
}
public string AvailabilityStatus { get; private set; }
public FileInfo File { get; private set; }
}

Skip reading the first line of the csv file

I am a beginner in programming,It's really difficult for me to analyze and debug how to skip reading the first line of the csv file. I need some help.
I need my id to fill my combobox in my form that contains all
Id's.In order to not include the header in browsing and
displaying.I need to skip the first line.
public bool ReadEntrie(int id, ref string name, ref string lastname, ref
string phone, ref string mail, ref string website)
{
int count = 0;
CreateConfigFile();
try
{
fs = new FileStream(data_path, FileMode.Open);
sr = new StreamReader(fs);
string temp = "";
bool cond = true;
while (cond == true)
{
if ((temp = sr.ReadLine()) == null)
{
sr.Close();
fs.Close();
cond = false;
if (count == 0)
return false;
}
if (count == id)
{
string[] stringSplit = temp.Split(',');
int _maxIndex = stringSplit.Length;
name = stringSplit[0].Trim('"');
lastname = stringSplit[1].Trim('"');
phone = stringSplit[2].Trim('"');
mail = stringSplit[3].Trim('"');
website = stringSplit[4].Trim('"');
}
count++;
}
sr.Close();
fs.Close();
return true;
}
catch
{
return false;
}
}

#Somadina's answer is correct, but I would suggest a better alternative. You could use a CSV file parser library such as CSV Helpers.
You can get the library from Nuget or Git. Nuget command would be:
Install-Package CsvHelper
Declare the following namespaces:
using CsvHelper;
using CsvHelper.Configuration;
Here's how simple your code looks when you use such a library:
class Program
{
static void Main(string[] args)
{
var csv = new CsvReader(File.OpenText("Path_to_your_csv_file"));
csv.Configuration.IgnoreHeaderWhiteSpace = true;
csv.Configuration.RegisterClassMap<MyCustomObjectMap>();
var myCustomObjects = csv.GetRecords<MyCustomObject>();
foreach (var item in myCustomObjects.ToList())
{
// Apply your application logic here.
Console.WriteLine(item.Name);
}
}
}
public class MyCustomObject
{
// Note: You may want to use a type converter to convert the ID to an integer.
public string ID { get; set; }
public string Name { get; set; }
public string Lastname { get; set; }
public string Phone { get; set; }
public string Mail { get; set; }
public string Website { get; set; }
public override string ToString()
{
return Name.ToString();
}
}
public sealed class MyCustomObjectMap : CsvClassMap<MyCustomObject>
{
public MyCustomObjectMap()
{
// In the name method, you provide the header text - i.e. the header value set in the first line of the CSV file.
Map(m => m.ID).Name("id");
Map(m => m.Name).Name("name");
Map(m => m.Lastname).Name("lastname");
Map(m => m.Phone).Name("phone");
Map(m => m.Mail).Name("mail");
Map(m => m.Website).Name("website");
}
}
Some more details in an answer here.

To skip the first line, just replace the line:
if (count == id)
with
if (count > 0 && count == id)
MORE THOUGHTS ON YOUR APPROACH
Because you used the ref keyword, each line you read will override the previous values you stored in the parameters. A better way to do this is to create a class to hold all the properties of interest. Then, for each line you read, package an instance of the class and add it to a list. You method signature (even the return type) will change eventually.
From your code, the class will look like this:
public class DataModel
{
public string Name { get; set; }
public string LastName { get; set; }
public string Phone{ get; set; }
public string Mail { get; set; }
public string Website{ get; set; }
}
Then your method will be like this:
public IList<DataModel> ReadEntrie(int id, string data_path)
{
int count = 0;
CreateConfigFile();
var fs = new FileStream(data_path, FileMode.Open);
var sr = new StreamReader(fs);
try
{
var list = new List<DataModel>();
string temp = "";
bool cond = true;
while (cond == true)
{
if ((temp = sr.ReadLine()) == null)
{
cond = false;
if (count == 0)
throw new Exception("Failed");
}
if (count > 0 && count == id)
{
string[] stringSplit = temp.Split(',');
var item = new DataModel();
item.Name = stringSplit[0].Trim('"');
item.LastName = stringSplit[1].Trim('"');
item.Phone = stringSplit[2].Trim('"');
item.Mail = stringSplit[3].Trim('"');
item.Website = stringSplit[4].Trim('"');
// add item to list
list.Add(item);
}
count++;
}
return list;
}
catch
{
throw; // or do whatever you wish
}
finally
{
sr.Close();
fs.Close();
}
}

Dump a list of files on a JSON file in c#

I am learning to write JSON and I have done this:
try
{
string path = #"C:\";
string[] dirs = Directory.GetFiles(path, "*.exe");
using (StreamWriter file = File.CreateText(#"filelist.json"))
foreach (string dir in dirs)
{
FolderFiles aFile = new FolderFiles
{
FileName = dir.ToString(),
FileType = Path.GetExtension(path + #"\" + dir.ToString())
};
JsonSerializer serializer = new JsonSerializer();
serializer.Serialize(file, aFile);
}
MessageBox.Show("Process Done");
}
I am trying to dump a list of filenames in a JSON files, but when I validate it says that it is invalid, and also it is only on a one line when I opened it.
Can JSON write to a file like a sequential order.
Any help will be great, thanks
Current output:
{"FileName":"C:\\Users\\Joko\\Documents\\Visual Studio 2015\\Projects\\Research\\GetFilesFromFolder\\GetFilesFromFolder\\bin\\Debug\\GetFilesFromFolder.exe","FileType":".exe"}{"FileName":"C:\\Users\\Joko\\Documents\\Visual Studio 2015\\Projects\\Research\\GetFilesFromFolder\\GetFilesFromFolder\\bin\\Debug\\GetFilesFromFolder.vshost.exe","FileType":".exe"}
This is the FolderFiles:
internal class FolderFiles {
public string FileName { get; set; }
public string FileType { get; set; }
}

It is not possible to write to a JSON file sequentially using out of box tools in .NET. You would have to implement that by yourself, manually writing to a file piece by piece.
Instead, just build the object containing an array of your folder paths, serialize it to JSON and just write it all at once to a file.
Examplary code to implement that, with hardcoded data you've provided as example output:
public class FolderFilesAggregate
{
public IEnumerable<FolderFiles> FolderFiles { get; set; }
}
public class FolderFiles
{
public string FileName { get; set; }
public string FileType { get; set; }
}
class Program
{
static void Main(string[] args)
{
// get your folder files in any way you want to, I hardcoded it to simplify the example
var folderFiles = new List<FolderFiles>
{
new FolderFiles
{
FileName = #"C:\\Users\\Joko\\Documents\\Visual Studio 2015\\Projects\\Research\\GetFilesFromFolder\\GetFilesFromFolder\\bin\\Debug\\GetFilesFromFolder.exe",
FileType = ".exe"
},
new FolderFiles
{
FileName = #"C:\\Users\\Joko\\Documents\\Visual Studio 2015\\Projects\\Research\\GetFilesFromFolder\\GetFilesFromFolder\\bin\\Debug\\GetFilesFromFolder.vshost.exe",
FileType = ".exe"
}
};
var folderFilesAggregate = new FolderFilesAggregate
{
FolderFiles = folderFiles
};
// serialize your aggregate object
var serializedFolderFilesAggregate = Newtonsoft.Json.JsonConvert.SerializeObject(folderFilesAggregate, Formatting.Indented);
// write it to a file
System.IO.File.WriteAllText(#"C:\output.json", serializedFolderFilesAggregate);
}
}

Upload files to share point using C#

I'm trying to upload files to a folder using SharePoint and C#.
I managed to create folders with my code and it works fine.
This is my Document class:
[DataContractAttribute]
public class Document
{
[DataMemberAttribute]
public string Name { get; set; }
[DataMemberAttribute]
public byte[] Content { get; set; }
[DataMemberAttribute]
public bool ReplaceExisting { get; set; }
[DataMemberAttribute]
public string Folder { get; set; }
[DataMemberAttribute]
public Dictionary<string, object> Properties { get; set; }
public Document()
{
Properties = new Dictionary<string, object>();
}
public Document(string name, byte[] content)
{
Name = name;
ReplaceExisting = false;
Content = content;
Properties = new Dictionary<string, object>();
}
public Document(string name, byte[] content, bool replace)
{
Name = name;
Content = content;
ReplaceExisting = replace;
Properties = new Dictionary<string, object>();
}
}
And this is the class where I use it to upload files (Document) to an existing share point folder:
public class SharePointHandler : IDisposable
{
private static string sharePointSite = "My Site";
private static string documentLibraryName = "Root folder";
public SharePointHandler() { }
public void Upload(List<Document> documents, string company, int year, string quarter)
{
string Year = year.ToString();
try
{
using (ClientContext context = new ClientContext(sharePointSite))
{
var list = context.Web.Lists.GetByTitle(documentLibraryName);
context.Load(list);
var root = list.RootFolder;
context.Load(root);
context.ExecuteQuery();
.
.
.
foreach (var document in documents)
{
var fileCreationInformation = new FileCreationInformation();
fileCreationInformation.Content = document.Content;
fileCreationInformation.Overwrite = true;
fileCreationInformation.Url = list.RootFolder.ServerRelativeUrl + "/" + company + "/" + Year + "/" + quarter + "/" + document.Name;
Microsoft.SharePoint.Client.File uploadFile = quarterFolder.Files.Add(fileCreationInformation);
foreach (KeyValuePair<string, object> property in document.Properties)
uploadFile.ListItemAllFields[property.Key] = property.Value;
try
{
uploadFile.CheckOut();
context.ExecuteQuery();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
uploadFile.ListItemAllFields.Update();
context.ExecuteQuery();
uploadFile.CheckIn("", CheckinType.MajorCheckIn);
context.ExecuteQuery();
}
}
}
catch (Exception ex)
{
EventLog.WriteEntry(ex.Message, EventLogEntryType.Error);
return;
}
}
public void Dispose() { }
}
When I run the code I have one document with:
Content: {byte[11430]}
Folder: null
Name: Testing.docx
Properties: Count = 0
ReplaceExisting: false
Everything works fine and I get the URL needed.
But when I get to these commands:
try
{
uploadFile.CheckOut();
context.ExecuteQuery();
}
The program falls and I get error that says: File not found.
What am I doing wrong?
Thank you for your help!

Here is a working example of an upload to SharePoint via CSOM:
using (ClientContext conext = new ClientContext(site.url))
{
List projectFiles = projects.Web.Lists.GetByTitle("Project Files");
context.Load(projectFiles.RootFolder, w => w.ServerRelativeUrl);
context.ExecuteQuery();
FileStream documentStream = System.IO.File.OpenRead(filePath);
byte[] info = new byte[documentStream.Length];
documentStream.Read(info, 0, (int)documentStream.Length);
string fileURL = projectFiles.RootFolder.ServerRelativeUrl + "/Folder/FileName.ext";
FileCreationInformation fileCreationInformation = new FileCreationInformation();
fileCreationInformation.Overwrite = true;
fileCreationInformation.Content = info;
fileCreationInformation.Url = fileURL;
Microsoft.SharePoint.Client.File uploadFile = projectFiles.RootFolder.Files.Add(fileCreationInformation);
context.Load(uploadFile, w => w.MajorVersion, w => w.MinorVersion);
context.ExecuteQuery();
}
In your case, I would upload the file and ExecuteQuery() and then add your metadata and execute a second query, make sure you add a context.Load after your files.Add(). You are trying to do too many things at once, so just use this code to get the file uploaded and work your way through your other needs after. Also, the file add will not create folders for you so make sure you are specifying a valid path.