I am trying to load a large excel file (~30MB) with 11 columns and 485k rows using the following code:
OpenFileDialog ofd = new OpenFileDialog();NameOfFile = ofd.SafeFileName;
if (NameOfFile.Contains("xlsx"))
{
NameOfFile = NameOfFile.Substring(0, NameOfFile.Length - 5);
}
else
{
NameOfFile = NameOfFile.Substring(0, NameOfFile.Length - 4);
}
string dbConn = "Provider=Microsoft.ACE.OLEDB.12.0;" + "Data Source=" + ofd.FileName + ";Extended Properties=" + "Excel 12.0;";
OleDbConnection con = new OleDbConnection(dbConn);
OleDbDataAdapter da = new OleDbDataAdapter("select * from [Contacts$]", con); //Contacts$ -> onoma filou sto excel
da.Fill(dataBase1);
dataGridView1.DataSource = dataBase1.Tables[0];
dataGridView1.Columns[0].Width = 35;
dataGridView1.Columns[1].Width = 35;
dataGridView1.Columns[2].Width = 35;
dataGridView1.Columns[3].Width = 35;
dataGridView1.Columns[4].Width = 35;
dataGridView1.Columns[5].Width = 40;
dataGridView1.Columns[9].Width = 55;
dataGridView1.Columns[10].Width = 60;
The problem is whenever I run my program and try to load the file the program freezes and has to be shut down. The program has been tested with smaller excel file (11 columns and 100 rows) and runs perfectlly.
Any ideas?
With 485K rows, hard to say if the problem is OLEDB or the DataGridView. I suggest you dump OLEDB and use a native excel library and then use Virtual Mode with the DataGridView.
The EPPlus library open source Excel library that can read the xlsx format (Office 2007 and later) and the ExcelLibrary project that can read the older xls binary format (prior to Office 2007).
Check this tutorial for implementing virtual mode in the DataGridView so that the Grid doesn't have to manage all 485K rows on the screen and in memory at once.
Using OLEDB to load large Excel files can be a problem. Take a look on this code sample that loads the Excel file and fill the DataGridView:
http://www.easyxls.com/manual/FAQ/import-excel-to-datagridview.html
It uses EasyXLS Excel library. If your DataGridView cannot display so big volume of data, you can adjust the code to load ranges from the Excel file and display data page by page or only the data that shows interest.
I have faced the same problem with big excel files and solved it by using Excel Data Reader library:
https://exceldatareader.codeplex.com/
It is lightweight, fast and free library written in C#.
Usage:
FileStream stream = File.Open( file, FileMode.Open, FileAccess.Read );
IExcelDataReader excelReader = null;
if (file.ToLower().EndsWith("xls"))
//1. Reading from a binary Excel file ('97-2003 format; *.xls)
excelReader = ExcelReaderFactory.CreateBinaryReader( stream );
else
//2. Reading from a OpenXml Excel file (2007 format; *.xlsx)
excelReader = ExcelReaderFactory.CreateOpenXmlReader( stream );
DataSet result = excelReader.AsDataSet();
and then you can do anything you want with the DataSet.
Related
I´m facing this issue a long time, after months i´m still not able to find any solution. Here´s the scenario:
VS 2019, Framework 4.6 and Crystal reports 13_0_27.
The following code takes hours to export a pdf (about 400 pages and 30.000 rows) . If i open the report
with crystal reports and export the document, same query by code, only takes seconds.
I tried a couple things, like ExportToStream and save the stream to file, or exporting direcly to disk
and other post did read that "pdfFormatOptions.UsePageRange = True" should help, but same result.
The code works fine with smalls pdfs with, for example, 100 rows.
Informe.Load(Application.StartupPath + #"\informes\report.rpt");
for (i = 0; i < Informe.Database.Tables.Count; ++i)
{
logOnInfo.ConnectionInfo.ServerName = "Server";
logOnInfo.ConnectionInfo.DatabaseName = "BBDD";
logOnInfo.ConnectionInfo.UserID = "user";
logOnInfo.ConnectionInfo.Password = "user";
Informe.Database.Tables[i].ApplyLogOnInfo(logOnInfo);
}
diskOpts.DiskFileName = PDFPath + _cabe.Guid + "_minutos.pdf";
ExportOptions exportOpts2 = Informe.ExportOptions;
exportOpts2.DestinationOptions = diskOpts;
exportOpts2.ExportFormatType = ExportFormatType.PortableDocFormat;
exportOpts2.ExportDestinationType = ExportDestinationType.DiskFile;
try
{
Informe.RecordSelectionFormula = #" {CabeceraFacturas.Guid}='{" + _cabe.Guid.ToString() + "}'";
//Informe.Export();
Stream oStream;
oStream = (Stream)Informe.ExportToStream(ExportFormatType.PortableDocFormat);
using (FileStream fileStream = File.Create(RutaGeneracionPDF + _cabe.Guid + "_minutos.pdf", (int)oStream.Length))
{
byte[] bytesInStream = new byte[oStream.Length];
oStream.Read(bytesInStream, 0, bytesInStream.Length);
fileStream.Write(bytesInStream, 0, bytesInStream.Length);
fileStream.Close();
}
}
Thanks!
After expending days and hours and headaches i finally did the trick.
in each detail (About 30.000 rows) i had a formula wich calculated some value with two fields of the detail and two from a joined view. the view was the problem when i was exporting by code (Exporting within Crystal Reports worked ok with no delay). I had to create a new table in SQL, inserting all the rows in view in this new table and add this table to report and ..voilá, it worked, exported report in seconds.
I am trying to convert a byte[] I got from an XLS file I have elsewhere, into a new XLSX file and the save it. I'm using Free Spire.XML, but can't figure out how to do it.
public byte[] ConvierteAXLSX(string cuerpo)
{
Workbook wb = new Workbook();
Worksheet sheet = wb.Worksheets[0];
byte[] array = Convert.FromBase64String(cuerpo);
sheet.InsertArray(array, 1, 1, true);
wb.SaveToFile(AppDomain.CurrentDomain.BaseDirectory + "sample.xlsx", ExcelVersion.Version2013);
byte[] fileContent = File.ReadAllBytes(AppDomain.CurrentDomain.BaseDirectory + "sample.xlsx");
//File.Delete(AppDomain.CurrentDomain.BaseDirectory + "sample.xlsx");
return fileContent;
}
This code creates the XLSX file, but just inserts the byte[] into the excel file like an array, instead of converting the data.
Edit:
My problem is slightly different from that other question. I can't just read the original file and then save it again, since the file is in another server and can't access it. The best thing I can do is send the document body and parse it into byte[].
It also works if I can convert my byte[] into a XLS file and save it, then I could use the answer to the other similar question.
Workbook workbook = new Workbook();
workbook.LoadFromFile("Input.xls");
workbook.SaveToFile("Output.xlsx", ExcelVersion.Version2013);
This is not going to war as the two file types store data completely different. The data in the xls file is stored in a proprietary binary format and the xmls file data is stored in Open XML.
I did it, saved the byte[] into a XLS file, read it and saved it again into a XLSX file.
public byte[] ConvierteAXLSX(string cuerpo)
{
File.WriteAllBytes(AppDomain.CurrentDomain.BaseDirectory + "viejo.xls", Convert.FromBase64String(cuerpo));
Workbook workbook = new Workbook();
workbook.LoadFromFile(AppDomain.CurrentDomain.BaseDirectory + "viejo.xls");
workbook.SaveToFile(AppDomain.CurrentDomain.BaseDirectory + "nuevo.xlsx", ExcelVersion.Version2013);
byte[] fileContent = File.ReadAllBytes(AppDomain.CurrentDomain.BaseDirectory + "nuevo.xlsx");
File.Delete(AppDomain.CurrentDomain.BaseDirectory + "viejo.xls");
File.Delete(AppDomain.CurrentDomain.BaseDirectory + "nuevo.xlsx");
return fileContent;
}
Thanks for your help!
I have a DataGrid which I filled with data from a DataSet. The DataSet gets the data from an Excel file, with a OpenFileDialog. I want to Insert this data into my SQL Server Express table. How can I do that?
Code with which I fill the DataGrid:
private void btnOpen_Click_1(object sender, RoutedEventArgs e)
{
OpenFileDialog openfile = new OpenFileDialog();
openfile.DefaultExt = ".xlsx";
openfile.Filter = "(.xlsx)|*.xlsx";
//openfile.ShowDialog();
var browsefile = openfile.ShowDialog();
if (browsefile == true)
{
txtFilePath.Text = openfile.FileName;
FileStream stream = File.Open(txtFilePath.Text, FileMode.Open, FileAccess.Read);
Excel.IExcelDataReader excelReader = Excel.ExcelReaderFactory.CreateOpenXmlReader(stream);
excelReader.IsFirstRowAsColumnNames = true;
DataSet resultexc = excelReader.AsDataSet();
dtGrid.ItemsSource = resultexc.Tables[0].DefaultView;
excelReader.Close();
}
}
I have seen some tutorials, but I don't understand them. Thanks for the help!
The DataSet object contains a Tables property. Following the example they give within the documentation, you should be able to create SqlCommands to update your SQL server tables appropriately.
This option gives a little more control over which data you insert, and provides flexibility if your source's data doesn't align with the destination as nicely as a BULK INSERT or SqlBulkCopy requires.
I'm trying to insert multiple records with excel file to database. Scenerio is, the user selects the excel file from his hard drive then the code reads first row and shows it as the title columns. These columns are not ordered. So user matches right columns with drop down.
Everything is okay until now. But after matching I have to read the file again from another action. But i am not saving file to server. Is there any way to read or hold the file in variable or session?
My code similiar like this
public JsonResult ExcelUpload(FormCollection formCollection)
{
if (Request != null)
{
HttpPostedFileBase file = Request.Files["ExcelFile"];
}
}
If you mean by "another action", you mean a different HTTP request, then you can't do that without it persisting somewhere.
Better read file in data table by using library given here
FileStream stream = File.Open(filePath, FileMode.Open, FileAccess.Read);
//1. Reading from a binary Excel file ('97-2003 format; *.xls)
IExcelDataReader excelReader = ExcelReaderFactory.CreateBinaryReader(stream);
//...
//2. Reading from a OpenXml Excel file (2007 format; *.xlsx)
IExcelDataReader excelReader = ExcelReaderFactory.CreateOpenXmlReader(stream);
//...
//3. DataSet - The result of each spreadsheet will be created in the result.Tables
DataSet result = excelReader.AsDataSet();
//...
//4. DataSet - Create column names from first row
excelReader.IsFirstRowAsColumnNames = true;
DataSet result = excelReader.AsDataSet();
//5. Data Reader methods
while (excelReader.Read())
{
//excelReader.GetInt32(0);
}
//6. Free resources (IExcelDataReader is IDisposable)
excelReader.Close();
and store DataSet into session for further processing.
I have a WebApi wich will receive an excel file uploaded by the user as multipart/form-data.
I need to read the content of that file in order to update the database. I was thinking on using EPPlus but I can't access the file.
Here is the code
public class MyController : APIController
{
[Route("import")]
[HttpPost]
public async Task<HttpResponseMessage> importFile()
{
if (!Request.Content.IsMimeMultipartContent())
return Request.CreateErrorResponse(HttpStatusCode.BadRequest, "ERROR");
Stream stream = await Request.Content.ReadAsStreamAsync();
var excel = new ExcelPackage(stream);
var workbook = excel.Workbook;
var sheet = excel.Workbook.Worksheets.First();
...
}
The error is on the var sheet = excel.Workbook.Worksheets.First() because the Workbook doesn't have any Worksheet (but the physical file has 2).
What am I doing wrong? Is it the Stream?
I'm trying to have separate library for each type of Excel file (.xls or .xlsx) but I'm not able to make this work with the .xls files.
I'm using ExcelDataReader and the code is now like this:
public async Task<HttpResponseMessage> importFile()
{
if (!Request.Content.IsMimeMultipartContent())
return Request.CreateErrorResponse(HttpStatusCode.BadRequest, "NOT MULTIPART");
Stream stream = await Request.Content.ReadAsStreamAsync();
//open xlsx file
var excel = new ExcelPackage(stream);
var workbook = excel.Workbook;
try
{
var sheet = excel.Workbook.Worksheets.First();
//working fine with EPPlus for .xlsx files
return Request.CreateResponse(HttpStatusCode.OK, errors);
}catch(Exception)//open xls file
{
//if its a .xls file it will throw an Exception
}
//using ExcelDataReader to open .xls file
IExcelDataReader excelReader = ExcelReaderFactory.CreateBinaryReader(stream);
DataSet result = excelReader.AsDataSet();
// the DataSet is null, stream is setted as ReadOnlyStream and stream.length is throwing an ObjectDisposedException
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, "erro");
}
You can use Microsoft.Interop.Excel to read xls files, but even Microsoft disaproved this technique, because it is slow and it is not designed to run on servers. Furthermore, their support just ended.
As alternative you can use EasyXLS library. You can use it to read XLS files.
Take a look on this code sample that explains how to import an Excel file into a SQL table:
http://www.easyxls.com/manual/FAQ/import-excel-to-sql.html