I am importing a large set of Excel data (about 150k rows and 115 columns) into SQL Server.
using IExcelDataReader
Dim stream As FileStream = File.Open(filePath, FileMode.Open, FileAccess.Read)
' Reading from a binary Excel file ('97-2003 format; *.xls)
Dim excelReader As IExcelDataReader = ExcelReaderFactory.CreateBinaryReader(stream)
' Reading from a OpenXml Excel file (2007 format; *.xlsx)
Dim excelReader As IExcelDataReader = ExcelReaderFactory.CreateOpenXmlReader(stream)
result.Tables
Dim result As DataSet = excelReader.AsDataSet()
excelReader.IsFirstRowAsColumnNames = True
Dim result As DataSet = excelReader.AsDataSet()
' Do a Bulk copy Here
excelReader.Close()
After waiting 2-3 minutes, I got a System.OutOfMemoryException
Is there a way to get rid of this exception and import the data faster?
I tried the ACE.OLEDB.12.0 provider but it is not working on iis .
You should use the excelReader.Read() method to iterate through each row, instead of returning the entire spreadsheet in a single call, which is what excelReader.AsDataSet() is doing (which is then causing you to run out of memory).
See here for details on how to use a reader (although in that example it's a SqlDataReader, the logic should be the same).
The documentation for the ExcelDataReader here (scroll down) has a vague example which matches the SqlDataReader docs.
Related
I want to open Excel from byte[] because my file is encrypted and I want to open after decrypt but without write in a file.
The office has "restricted access" and I want to open my file with this protection but without saving the decrypted content in a file.
myApp.Workbooks.Open only supports a path.
Is it possible?
As an alternative to OpenXml there's also ExcelDataReader which from my experience is a lot faster in processing data compared to Interop.Excel(around 3 times+).
It can also open encrypted Excel files directly(stackoverflow)
The github page for ExcelDataReader has some great examples on how to use it. The only thing you'd have to do is:
This:
using (var stream = File.Open(filePath, FileMode.Open, FileAccess.Read))
Becomes this:
using (var stream = new MemoryStream(yourByte[])
And if you just want to open the password protected excel file you'd do this:
var conf = new ExcelReaderConfiguration { Password = "yourPassword" }; //Add this
excelReader = ExcelReaderFactory.CreateReader(stream, conf); //change the excel Reader to this
Make sure to check the Github page for more info!
It is not possible because the interop is actually an interface for programs to run and operate existing excel on the computer.
I think you need to use openxml created by Microsoft to work with excel word and PowerPoint.
DocumentFormat.OpenXml
Then you can use:
ExcelPackage excelPackage = new ExcelPackage(stream)
or
var pck = new OfficeOpenXml.ExcelPackage();
pck.Load(File.OpenRead(path));
pck.Load(Stream) can use any stream as input not only from a file.
It depends on your needs.
I have a very large excel file which I opened with notepad++ count to 1,261,286 rows. How can I have the row's value from 1,048,577 to 1,261,286 of this sheet by C#? I have tried to read it but... please take a look at 2 pictures.
Read by c#
Open with notepad++
Any help would be appreciated!
Given that you can open this in Notepad++ and it looks like a well-formatted text file rather than binary data, it is very likely that you are actually dealing with tab-delimited data.
If that is the case, you can use the fast, efficient and open-source CSV reader
http://www.codeproject.com/Articles/9258/A-Fast-CSV-Reader
I use that in numerous projects and it works like a charm.
You would read all rows, ignoring rows that are not in the range you are interested in.
Code would be along the lines of
int lineNr = 1;
using (FileStream fs = File.Open(pathToTheFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
using (BufferedStream bs = new BufferedStream(fs))
using (CsvReader csv = new CsvReader(new StreamReader(bs), true))
{
while (csv.ReadNextRecord())
{
if (YouCareAboutLine(lineNr))
{
DoSomethingWithThisLine();
}
lineNr++;
}
}
I have an Excel Template which i want to write into with data from a database. Whenever I edit and Save the file in c#, when I open the template, Microsoft Office Excel says the file is corrupt. Apparently, I think I'm going about editing it wrongly. this is how i went about it below. I am using NPOI 2.0 beta 2. if it matters, the template contains macros and formulas
FIleStream fs = new FileStream(pathString, FIleMode.Open, FileAccess.Read);
IWorkbook wkb = WorkbookFactory.Create(fs);
ISheet sheet = wkb.GetSheet("sheet1");
ICell cell = sheet.GetRow(row).GetCell(column);
if(cell != null)
{
cell.SetCellValue(value);
}
FileStream fs1 = new FileStream(pathString, FileMode.OpenOrCreate);
wkb.Write(fs1);
fs.CLose();
fs1.Close();
But If I try to read the corrupted excel file, i can still retrieve values from the sheet using NPOI. Any pointers as to my errors. Thanks in anticipoation
I have an excel file contains Attachment column, which has another excel object embedded into it.
I need to read that embedded excel file and process its data.
So far i used Excel Reader to read the normal excel file and retrieve as DataSet.
FileStream stream = File.Open(filePath, FileMode.Open, FileAccess.Read);
//Reading from a binary Excel file ('97-2003 format; *.xls)
IExcelDataReader excelReader = ExcelReaderFactory.CreateBinaryReader(stream);
// Reading from a OpenXml Excel file (2007 format; *.xlsx)
IExcelDataReader excelReader = ExcelReaderFactory.CreateOpenXmlReader(stream);
DataSet result = excelReader.AsDataSet();
excelReader.IsFirstRowAsColumnNames = true;
DataSet result = excelReader.AsDataSet();
while (excelReader.Read())
{
//excelReader.GetInt32(0);
}
excelReader.Close();
Now i want to read the embedded object(excel file)
Is there any dll/component/code which will help us to read embedded object inside excel file?
Note: I dont have a Microsoft Office Package installed in server. So i dont want to use Microsoft.Interoperability package
I am wondering how do I extract data out of a 2007 excel file? I am using asp.net mvc 3. My plan is to have a upload section that you choose a file and hit upload. I have no clue after that what kind of format it will be or what I need to do to extract the values out.
Thanks
Once you have the spreadsheet uploaded and you save it to a file on the web server it is quite easy to use LINQ to select the rows from the spreadsheet. Check this out for more info.
http://code.google.com/p/linqtoexcel/
The easiest way to read excel spread sheets IMO is to use a DataAdapter and an OleDB connection as shown in this code project sample. The good thing about this is it does not have any dependencies on COM or the MS office libraries.
For reading Excel files, I learned to love Koogra. It's an open source library that reads both xls and xlsx files, and is very easy to use.
http://sourceforge.net/projects/koogra/
I've used NPOI and it's quite simple to use:
Using Xlfile As FileStream = New FileStream(FileName, FileMode.Open, FileAccess.Read)
Using XLBook As HSSFWorkbook = New HSSFWorkbook(Xlfile)
Using XLSheet As NPOI.SS.UserModel.Sheet = XLBook.GetSheetAt(0)
Dim CurrentRow As NPOI.HSSF.UserModel.HSSFRow
Dim CurrentCell As NPOI.SS.UserModel.Cell
Dim RowEnum As IEnumerator = XLSheet.GetRowEnumerator()
While RowEnum.MoveNext
If (RowEnum.Current IsNot Nothing) Then
CurrentRow = TryCast(RowEnum.Current, NPOI.HSSF.UserModel.HSSFRow)
Select Case CurrentCell.CellType
Case NPOI.SS.UserModel.CellType.STRING
' CurrentCell.StringCellValue
Case NPOI.SS.UserModel.CellType.NUMERIC
' CurrentCell.NumericCellValue.ToString()
End Select
End While
End Using
End Using
Xlfile.Close()
End Using