I can't seem to figure out why this bit of code is failing, it seems simple enough.
Code:
string[] ignore = File.ReadAllLines(#"logicfiles\[flag]-[ignore-these-links].txt");
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(rawHtml);
foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[#href]"))
{
string linkUrl = link.GetAttributeValue("href", string.Empty);
if (!ignore.Any(linkUrl.Contains) && linkUrl.Length < 10 && !linkUrl.StartsWith("/"))
{
DataGridViewLinks.Rows.Add(linkUrl, keywordUsed, "", "", engineUsed);
}
}
The above code does not work as in it just adds every URL to the DataGrid this part !ignore.Any(linkUrl.Contains) is the part that is failing to work right, the ignore array contains strings like facebook, youtube etc if the url linkUrl does NOT contain one of these strings in it, then add it to the DataGrid (is how it should work)
But if i do this:
string[] ignore = File.ReadAllLines(#"logicfiles\[flag]-[ignore-these-links].txt");
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(rawHtml);
foreach (HtmlNode link in doc.DocumentNode.SelectNodes("//a[#href]"))
{
string linkUrl = link.GetAttributeValue("href", string.Empty);
if (linkUrl.Length < 10 && !linkUrl.StartsWith("/"))
{
DataGridViewLinks.Rows.Add(linkUrl, keywordUsed, "", "", engineUsed);
}
}
And take that part of the code away, the other 2 conditions work perfectly, so I know the part of the logic not working is !ignore.Any(linkUrl.Contains)
I cannot see why, if someone could point out the issue it would be appreciated.
Your Contains logic is fine. There may be something wrong with the values that are being passed in from the text file. An upper / lower case issue or similar.
I recommend printing out or otherwise inspecting both the parsed url values, and the filter strings coming in from the text file and ensuring you're comparing what you think you are.
Here is just the logic for the Contains with a set of values showing it working:
using System;
using System.Collections.Generic;
using System.Linq;
public class Program
{
public static void Main()
{
var linkUrls = new List<string>{
"https://youtube.com/2134",
"https://google.com/2134",
"https://microsoft.com/2134"
};
var ignores = new List<string>{
"youtube",
"somethingElse"
};
foreach (var linkUrl in linkUrls)
{
if (!ignores.Any(linkUrl.Contains))
{
Console.WriteLine($"passed filter: {linkUrl}");
}
}
}
}
Output:
passed filter: https://google.com/2134
passed filter: https://microsoft.com/2134
See:
https://dotnetfiddle.net/8wud9O
Whenever I run this code, it returns one character. I've tried various things, and at most it returns 5 or so lines, each containing one character. I'm trying to find each folder in the "Users" folder, and make my code list them, any help would be appreciated.
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
namespace ConsoleApp3
{
class Program
{
static void Main(string[] args)
{
//Search for directories and foreach, try to start discord
try
{
var Bruhm = new List<string>();
Bruhm.Add(Directory.GetDirectories(#"C:\Users\").ToString());
int y = 0;
foreach (string x in Bruhm) {
Console.WriteLine(x[y]);
y = y+1;
}
}
catch(Exception e)
{
Console.WriteLine(e);
System.Threading.Thread.Sleep(-1);
}
System.Threading.Thread.Sleep(-1);
}
}
}
The first mistake you are making is;
Bruhm.Add(Directory.GetDirectories(#"C:\Users\").ToString());
will add only one string, which is System.String[] to your list.
The second mistake is, x[y] will print only one character, which is the character at y index inside the string x.
Change the Add() to AddRange(), and do not call ToString() on your list.
Try this:
var Bruhm = new List<string>();
Bruhm.AddRange(Directory.GetDirectories(#"C:\Users\"));
foreach (string x in Bruhm)
{
Console.WriteLine(x);
}
It's quite simple. You're doing error with this: Directory.GetDirectories(#"C:\Users\").ToString()
Instead try this:
var Bruhm = Directory.GetDirectories(#"C:\Users\");
foreach(var subdir in Bruhm)
Console.WriteLine(subDir);
You are converting a list into a string and printing an index of it and it makes wrong result. You can extract the folder name using this code. "C:\Users\" has removed from the folder's path to provide folders' name not path.
var path = #"C:\Users\";
var Bruhm = Directory.GetDirectories(path).Select(x => x.Substring(path.Length));
foreach (string x in Bruhm)
{
Console.WriteLine(x);
}
In C#,
I want directory names list.
Type is List and I have full pah list(List fullPathList).
fullpathList is like this..
[0] C:\temp\image\a.jpg;
[1] C:\temp\image\b.bmp;
[2] c:\temp\bin\my.exe;
[3] c:\temp\document\resume.doc;
[4] c:\temp\document\timetable.xlsx;
In this case,
I want it.
List<string> dirs;
[0] iamge
[1] bin
[2] document
I guess possible if i using Regx.
But, i dont know detailed method.
how can I do for it?
(dont use loop statement)
I believe this prints what you want...
using System.Data;
using System.Diagnostics;
using System.IO;
using System.Linq;
namespace Test {
class Program {
static void Main(string[] args) {
var files = new string[] {
#"C:\temp\image\a.jpg",
#"C:\temp\image\b.bmp",
#"c:\temp\bin\my.exe",
#"c:\temp\document\resume.doc",
#"c:\temp\document\timetable.xlsx",
};
var dirNames = files.Select(x => new DirectoryInfo(Path.GetDirectoryName(x)).Name);
Debug.WriteLine($"dirNames={string.Join(",", dirNames)}");
}
}
}
The Path.GetDirectoryName() returns the full path and new DirectoryInfo().Name returns just the name of the last part of the path.
If you want the result in a list, use...
var dirNames = files.Select(x => new DirectoryInfo(Path.GetDirectoryName(x)).Name).ToList();
Check if this solution Works.
it gets all the directory in the specified drive.
Please go through the reference link ref link
List<string> dir = new List<string>();
System.IO.DriveInfo di = new System.IO.DriveInfo(#"D:\");
System.IO.DirectoryInfo dirInfo = di.RootDirectory;
System.IO.DirectoryInfo[] dirInfos = dirInfo.GetDirectories("*.*");
foreach (System.IO.DirectoryInfo d in dirInfos)
{
dir.Add(d.Name);
}
For a real filesystem DirectoryInfo is the way to go.
If you do not have a real filesystem to query, but have a list of strings that are like paths you can use System.IO.Path - Static methods to split them up like this:
using System;
using System.Linq;
using System.IO;
public class Program
{
public static void Main()
{
// some strings that are like paths
var dirs = new[]
{
#"C:\temp\image\a.jpg", #"C:\temp\image\b.bmp", #"c:\temp\bin\my.exe",
#"c:\temp\document\resume.doc", #"c:\temp\document\timetable.xlsx"
};
// this will use a static method to extract only the path-part
var fullDirs = dirs.Select(d => Path.GetDirectoryName(d)).ToList();
// this will use the same method but split the result at the default systems
// path-seperator char and use only the last part, only uses distinct values
var lastDirsDistinct = dirs.Select(d => Path.GetDirectoryName(d)
.Split(Path.DirectorySeparatorChar).Last()
).Distinct().ToList();
// joins the list with linebreaks and outputs it
Console.WriteLine(string.Join("\n", fullDirs));
// joins the list with linebreaks and outputs it
Console.WriteLine(string.Join("\n", lastDirsDistinct ));
}
}
Output:
C:\temp\image
C:\temp\image
c:\temp\bin
c:\temp\document
c:\temp\document
image
bin
document
So I've been reading that I shouldn't write my own CSV reader/writer, so I've been trying to use the CsvHelper library installed via nuget. The CSV file is a grey scale image, with the number of rows being the image height and the number columns the width. I would like to read the values row-wise into a single List<string> or List<byte>.
The code I have so far is:
using CsvHelper;
public static List<string> ReadInCSV(string absolutePath)
{
IEnumerable<string> allValues;
using (TextReader fileReader = File.OpenText(absolutePath))
{
var csv = new CsvReader(fileReader);
csv.Configuration.HasHeaderRecord = false;
allValues = csv.GetRecords<string>
}
return allValues.ToList<string>();
}
But allValues.ToList<string>() is throwing a:
CsvConfigurationException was unhandled by user code
An exception of type 'CsvHelper.Configuration.CsvConfigurationException' occurred in CsvHelper.dll but was not handled in user code
Additional information: Types that inherit IEnumerable cannot be auto mapped. Did you accidentally call GetRecord or WriteRecord which acts on a single record instead of calling GetRecords or WriteRecords which acts on a list of records?
GetRecords is probably expecting my own custom class, but I'm just wanting the values as some primitive type or string. Also, I suspect the entire row is being converted to a single string, instead of each value being a separate string.
According to #Marc L's post you can try this:
public static List<string> ReadInCSV(string absolutePath) {
List<string> result = new List<string>();
string value;
using (TextReader fileReader = File.OpenText(absolutePath)) {
var csv = new CsvReader(fileReader);
csv.Configuration.HasHeaderRecord = false;
while (csv.Read()) {
for(int i=0; csv.TryGetField<string>(i, out value); i++) {
result.Add(value);
}
}
}
return result;
}
If all you need is the string values for each row in an array, you could use the parser directly.
var parser = new CsvParser( textReader );
while( true )
{
string[] row = parser.Read();
if( row == null )
{
break;
}
}
http://joshclose.github.io/CsvHelper/#reading-parsing
Update
Version 3 has support for reading and writing IEnumerable properties.
The whole point here is to read all lines of CSV and deserialize it to a collection of objects. I'm not sure why do you want to read it as a collection of strings. Generic ReadAll() would probably work the best for you in that case as stated before. This library shines when you use it for that purpose:
using System.Linq;
...
using (var reader = new StreamReader(path))
using (var csv = new CsvReader(reader))
{
var yourList = csv.GetRecords<YourClass>().ToList();
}
If you don't use ToList() - it will return a single record at a time (for better performance), please read https://joshclose.github.io/CsvHelper/examples/reading/enumerate-class-records
Please try this. This had worked for me.
TextReader reader = File.OpenText(filePath);
CsvReader csvFile = new CsvReader(reader);
csvFile.Configuration.HasHeaderRecord = true;
csvFile.Read();
var records = csvFile.GetRecords<Server>().ToList();
Server is an entity class. This is how I created.
public class Server
{
private string details_Table0_ProductName;
public string Details_Table0_ProductName
{
get
{
return details_Table0_ProductName;
}
set
{
this.details_Table0_ProductName = value;
}
}
private string details_Table0_Version;
public string Details_Table0_Version
{
get
{
return details_Table0_Version;
}
set
{
this.details_Table0_Version = value;
}
}
}
You are close. It isn't that it's trying to convert the row to a string. CsvHelper tries to map each field in the row to the properties on the type you give it, using names given in a header row. Further, it doesn't understand how to do this with IEnumerable types (which string implements) so it just throws when it's auto-mapping gets to that point in testing the type.
That is a whole lot of complication for what you're doing. If your file format is sufficiently simple, which yours appear to be--well known field format, neither escaped nor quoted delimiters--I see no reason why you need to take on the overhead of importing a library. You should be able to enumerate the values as needed with System.IO.File.ReadLines() and String.Split().
//pseudo-code...you don't need CsvHelper for this
IEnumerable<string> GetFields(string filepath)
{
foreach(string row in File.ReadLines(filepath))
{
foreach(string field in row.Split(',')) yield return field;
}
}
static void WriteCsvFile(string filename, IEnumerable<Person> people)
{
StreamWriter textWriter = File.CreateText(filename);
var csvWriter = new CsvWriter(textWriter, System.Globalization.CultureInfo.CurrentCulture);
csvWriter.WriteRecords(people);
textWriter.Close();
}
I have a class as follows :
public class Test
{
public int Id {get;set;}
public string Name { get; set; }
public string CreatedDate {get;set;}
public string DueDate { get; set; }
public string ReferenceNo { get; set; }
public string Parent { get; set; }
}
and I have a list of Test objects
List<Test>testobjs=new List();
Now I would like to convert it into csv in following format:
"1,John Grisham,9/5/2014,9/5/2014,1356,0\n2,Stephen King,9/3/2014,9/9/2014,1367,0\n3,The Rainmaker,4/9/2014,18/9/2014,1";
I searched for "Converting list to csv c#" and I got solutions as follows:
string.Join(",", list.Select(n => n.ToString()).ToArray())
But this will not put the \n as needed i.e for each object
Is there any fastest way other than string building to do this? Please help...
Use servicestack.text
Install-Package ServiceStack.Text
and then use the string extension methods ToCsv(T)/FromCsv()
Examples:
https://github.com/ServiceStack/ServiceStack.Text
Update:
Servicestack.Text is now free also in v4 which used to be commercial. No need to specify the version anymore! Happy serializing!
Because speed was mentioned in the question, my interest was piqued on just what the relative performances might be, and just how fast I could get it.
I know that StringBuilder was excluded, but it still felt like probably the fastest, and StreamWriter has of course the advantage of writing to either a MemoryStream or directly to a file, which makes it versatile.
So I knocked up a quick test.
I built a list half a million objects identical to yours.
Then I serialized with CsvSerializer, and with two hand-rolled tight versions, one using a StreamWriter to a MemoryStream and the other using a StringBuilder.
The hand rolled code was coded to cope with quotes but nothing more sophisticated. This code was pretty tight with the minimum I could manage of intermediate strings, no concatenation... but not production and certainly no points for style or flexibility.
But the output was identical in all three methods.
The timings were interesting:
Serializing half a million objects, five runs with each method, all times to the nearest whole mS:
StringBuilder 703 734 828 671 718 Avge= 730.8
MemoryStream 812 937 874 890 906 Avge= 883.8
CsvSerializer 1,734 1,469 1,719 1,593 1,578 Avge= 1,618.6
This was on a high end i7 with plenty of RAM.
Other things being equal, I would always use the library.
But if a 2:1 performance difference became critical, or if RAM or other issues turned out to exaggerate the difference on a larger dataset, or if the data were arriving in chunks and was to be sent straight to disk, I might just be tempted...
Just in case anyone's interested, the core of the code (for the StringBuilder version) was
private void writeProperty(StringBuilder sb, string value, bool first, bool last)
{
if (! value.Contains('\"'))
{
if (!first)
sb.Append(',');
sb.Append(value);
if (last)
sb.AppendLine();
}
else
{
if (!first)
sb.Append(",\"");
else
sb.Append('\"');
sb.Append(value.Replace("\"", "\"\""));
if (last)
sb.AppendLine("\"");
else
sb.Append('\"');
}
}
private void writeItem(StringBuilder sb, Test item)
{
writeProperty(sb, item.Id.ToString(), true, false);
writeProperty(sb, item.Name, false, false);
writeProperty(sb, item.CreatedDate, false, false);
writeProperty(sb, item.DueDate, false, false);
writeProperty(sb, item.ReferenceNo, false, false);
writeProperty(sb, item.Parent, false, true);
}
If you don't want to load library's than you can create the following method:
private void SaveToCsv<T>(List<T> reportData, string path)
{
var lines = new List<string>();
IEnumerable<PropertyDescriptor> props = TypeDescriptor.GetProperties(typeof(T)).OfType<PropertyDescriptor>();
var header = string.Join(",", props.ToList().Select(x => x.Name));
lines.Add(header);
var valueLines = reportData.Select(row => string.Join(",", header.Split(',').Select(a => row.GetType().GetProperty(a).GetValue(row, null))));
lines.AddRange(valueLines);
File.WriteAllLines(path, lines.ToArray());
}
and than call the method:
SaveToCsv(testobjs, "C:/PathYouLike/FileYouLike.csv")
Your best option would be to use an existing library. It saves you the hassle of figuring it out yourself and it will probably deal with escaping special characters, adding header lines etc.
You could use the CSVSerializer from ServiceStack. But there are several other in nuget.
Creating the CSV will then be as easy as string csv = CsvSerializer.SerializeToCsv(testobjs);
You could use the FileHelpers library to convert a List of objects to CSV.
Consider the given object, add the DelimitedRecord Attribute to it.
[DelimitedRecord(",")]
public class Test
{
public int Id {get;set;}
public string Name { get; set; }
public string CreatedDate {get;set;}
public string DueDate { get; set; }
public string ReferenceNo { get; set; }
public string Parent { get; set; }
}
Once the List is populated, (as per question it is testobjs)
var engine = new FileHelperEngine<Test>();
engine.HeaderText = engine.GetFileHeader();
string dirPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + "\\" + ConfigurationManager.AppSettings["MyPath"];
if (!Directory.Exists(dirPath))
{
Directory.CreateDirectory(dirPath);
}
//File location, where the .csv goes and gets stored.
string filePath = Path.Combine(dirPath, "MyTestFile_" + ".csv");
engine.WriteFile(filePath, testobjs);
This will just do the job for you. I'd been using this to generate data reports for a while until I switched to Python.
PS: Too late to answer but hope this helps somebody.
Use Cinchoo ETL
Install-Package ChoETL
or
Install-Package ChoETL.NETStandard
Sample shows how to use it
List<Test> list = new List<Test>();
list.Add(new Test { Id = 1, Name = "Tom" });
list.Add(new Test { Id = 2, Name = "Mark" });
using (var w = new ChoCSVWriter<Test>(Console.Out)
.WithFirstLineHeader()
)
{
w.Write(list);
}
Output CSV:
Id,Name,CreatedDate,DueDate,ReferenceNo,Parent
1,Tom,,,,
2,Mark,,,,
For more information, go to github
https://github.com/Cinchoo/ChoETL
Sample fiddle: https://dotnetfiddle.net/M7v7Hi
LINQtoCSV is the fastest and lightest I've found and is available on GitHub. Lets you specify options via property attributes.
Necromancing this one a bit; ran into the exact same scenario as above, went down the road of using FastMember so we didn't have to adjust the code every time we added a property to the class:
[HttpGet]
public FileResult GetCSVOfList()
{
// Get your list
IEnumerable<MyObject> myObjects =_service.GetMyObject();
//Get the type properties
var myObjectType = TypeAccessor.Create(typeof(MyObject));
var myObjectProperties = myObjectType.GetMembers().Select(x => x.Name);
//Set the first row as your property names
var csvFile = string.Join(',', myObjectProperties);
foreach(var myObject in myObjects)
{
// Use ObjectAccessor in order to maintain column parity
var currentMyObject = ObjectAccessor.Create(myObject);
var csvRow = Environment.NewLine;
foreach (var myObjectProperty in myObjectProperties)
{
csvRow += $"{currentMyObject[myObjectProperty]},";
}
csvRow.TrimEnd(',');
csvFile += csvRow;
}
return File(Encoding.ASCII.GetBytes(csvFile), "text/csv", "MyObjects.csv");
}
Should yield a CSV with the first row being the names of the fields, and rows following. Now... to read in a csv and create it back into a list of objects...
Note: example is in ASP.NET Core MVC, but should be very similar to .NET framework. Also had considered ServiceStack.Text but the license was not easy to follow.
For the best solution, you can read this article: Convert List of Object to CSV File C# - Codingvila
using Codingvila.Models;
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Linq;
using System.Text;
using System.Web;
using System.Web.Mvc;
namespace Codingvila.Controllers
{
public class HomeController : Controller
{
public ActionResult Index()
{
CodingvilaEntities entities = new CodingvilaEntities();
var lstStudents = (from Student in entities.Students
select Student);
return View(lstStudents);
}
[HttpPost]
public FileResult ExportToCSV()
{
#region Get list of Students from Database
CodingvilaEntities entities = new CodingvilaEntities();
List<object> lstStudents = (from Student in entities.Students.ToList()
select new[] { Student.RollNo.ToString(),
Student.EnrollmentNo,
Student.Name,
Student.Branch,
Student.University
}).ToList<object>();
#endregion
#region Create Name of Columns
var names = typeof(Student).GetProperties()
.Select(property => property.Name)
.ToArray();
lstStudents.Insert(0, names.Where(x => x != names[0]).ToArray());
#endregion
#region Generate CSV
StringBuilder sb = new StringBuilder();
foreach (var item in lstStudents)
{
string[] arrStudents = (string[])item;
foreach (var data in arrStudents)
{
//Append data with comma(,) separator.
sb.Append(data + ',');
}
//Append new line character.
sb.Append("\r\n");
}
#endregion
#region Download CSV
return File(Encoding.ASCII.GetBytes(sb.ToString()), "text/csv", "Students.csv");
#endregion
}
}
}