We Keep Coding

Export large data to xls format: System out of memory Exception

I am exporting a large DataTable to an Excel sheet (.xls) format. I call stored procedure here, which returns the DataTable. The stored procedure used to return 7,50,000 rows with 93 columns.
In back end, if I pass the parameters for the stored procedure, it takes 8 mins to complete the process, since it is a long process.
But when I call the procedure from the front end, while assigning the records to the DataTable, "System Out Of Memory" exception occurring.
So When I google it, I came to know it is because the size exceeds the server RAM size, so it throws this exception.
When I export small DataTable it works fine.
What is the alternate way for this? Anyone suggest me. Below is my code,
C#:
DataTable dt = BindDatatable();
Response.ClearContent();
Response.Buffer = true;
Response.AddHeader("content-disposition", string.Format("attachment; filename={0}", "MBQ_Auto.xls"));
Response.ContentType = "application/ms-excel";
//Response.ContentType = "application/octet-stream";
//Response.ContentType = "text/tab-separated-values";
string str = string.Empty;
foreach (DataColumn dtcol in dt.Columns)
{
Response.Write(str + dtcol.ColumnName);
str = "\t";
}
Response.Write("\n");
int rowCount = 0;
foreach (DataRow dr in dt.Rows)
{
str = "";
for (int j = 0; j < dt.Columns.Count; j++)
{
Response.Write(str + Convert.ToString(dr[j]));
str = "\t";
}
Response.Write("\n");
if (unchecked(++rowCount % 1024 == 0))
Response.Flush();
}
Response.End();

You should not use DataTable for this kind of problem.
DataTable stores the entire data in memory.
You should use the DataReader to fetch rows from database.

Best way to download excel sheet using ASP.Net

I am using the below code to download the excel file:
Response.ClearContent();
Response.Buffer = true;
Response.AddHeader("content-disposition", string.Format("attachment; filename={0}", "MBQ.xls"));
Response.ContentType = "application/ms-excel";
DataTable dt = BindDatatable();
string str = string.Empty;
foreach (DataColumn dtcol in dt.Columns)
{
Response.Write(str + dtcol.ColumnName);
str = "\t";
}
Response.Write("\n");
foreach (DataRow dr in dt.Rows)
{
str = "";
for (int j = 0; j < dt.Columns.Count; j++)
{
Response.Write(str + Convert.ToString(dr[j]));
str = "\t";
}
Response.Write("\n");
}
Response.End();
In my case, the table has more than 800k records. I am applying more filters before downloading the excel sheet. What is the best way to download a excel sheet from the datatable which has hundreds of thousands of rows?

Firstly since that isn't an excel spreadsheet but a tab-separated-values file (which can be opened in excel and does what you want), then don't lie in the header as that can only bring confusion:
Response.AddHeader("content-disposition", string.Format("attachment; filename={0}", "MBQ.csv"));
Response.ContentType = "text/tab-separated-values";
(Strictly ".csv" is an extension normally defined as being for comma-separated-values rather than tab, but it's more commonly set up to open in Excel, LibreOffice Calc, or whatever spreadsheet program people have installed, and they'll generally consider CSV and TSV as variants of each other).
Secondly, don't buffer. You have Response.Buffer = true; set to explicitly make sure you keep masses of data in memory before sending it, you want to do the opposite, so Response.Buffer = false; is the way to go.
You might find it better still to explicitly call Response.Flush() every couple of hundred or thousand rows:
int rowCount = 0;
foreach (DataRow dr in dt.Rows)
{
str = "";
for (int j = 0; j < dt.Columns.Count; j++)
{
Response.Write(str + Convert.ToString(dr[j]));
str = "\t";
}
Response.Write("\n");
if (unchecked(++rowCount % 1024 == 0))
Response.Flush();
}
Experiment with higher or lower values than 1024. Higher sends larger chunks which is inefficient in needing more memory to hold it, but efficient in the actual sending of the chunk is snappier, so the optimal value will be a matter of finding the best balance between those two factors.

NewLine on CSV export of datatable

I have a datatable in a .Net web app that my customers have the option to export to CSV. A few customers have complained that the CSV is not able to re-import to our application. after further review, I noticed that on some large exports, the file has line breaks in it. is there a way i can prevent this?
here is my export method
DataTable dt = (DataTable)Session["Findings"];
string fileName = "test.csv";
string delimiter = ",";
//prepare the output stream
Response.Clear();
Response.ContentEncoding = System.Text.Encoding.UTF8;
Response.ContentType = "application/octet-stream";
Response.AppendHeader("Content-Disposition",
string.Format("attachment; filename={0}", fileName));
//write the csv column headers
for (int i = 0; i < dt.Columns.Count; i++)
{
Response.Write(dt.Columns[i].ColumnName);
Response.Write((i < dt.Columns.Count - 1) ? delimiter : Environment.NewLine);
}
//write the data
foreach (DataRow row in dt.Rows)
{
for (int i = 0; i < dt.Columns.Count; i++)
{
Response.Write("\"" + row[i].ToString() + "\"");
Response.Write((i < dt.Columns.Count - 1) ? delimiter : Environment.NewLine);
}
}
Response.End();
any help would be appreciated.
thanks

How to export from DataTable to Excel file in text format (even if there are numbers)

I am using the following procedure to create an Excel spreadsheet from a DataTable:
protected void UploadDataTableToExcel(DataTable dtEmp, string filename)
{
string attachment = "attachment; filename=" + filename;
Response.ClearContent();
Response.AddHeader("content-disposition", attachment);
Response.ContentType = "application/vnd.ms-excel";
string tab = string.Empty;
foreach (DataColumn dtcol in dtEmp.Columns)
{
Response.Write(tab + dtcol.ColumnName);
tab = "\t";
}
Response.Write("\n");
foreach (DataRow dr in dtEmp.Rows)
{
tab = "";
for (int j = 0; j < dtEmp.Columns.Count; j++)
{
Response.Write(tab + Convert.ToString(dr[j]));
tab = "\t";
}
Response.Write("\n");
}
Response.End();
}
It works fine except when the cell only has numbers. For example:
This "6969062201223401504" becomes "6.96902E+18"
This "076854957" becomes "76854957"
I would like the Excel spreadsheet to preserve every column as text, instead of converting them into integers

Export Datatable to Excel without taking into consideration HTML tags

There are lots of methods in order to export datatable to Excel file.
But when a column includes HTML tags, the structure of Excel is corrupted, it seems like HTML document.
How can I prevent exporting datatable to excel without taking into consideration HTML tags?
Code I use is below:
public static void ExportToExcel(DataTable table, string name)
{
HttpContext context = HttpContext.Current;
context.Response.Clear();
foreach (DataColumn column in table.Columns)
{
context.Response.Write(column.ColumnName + ",");
}
context.Response.Write(Environment.NewLine);
foreach (DataRow row in table.Rows)
{
for (int i = 0; i < table.Columns.Count; i++)
{
context.Response.Write(row[i].ToString().Replace(",", string.Empty) + ",");
}
context.Response.Write(Environment.NewLine);
}
context.Response.ContentType = "application/vnd.ms-excel";
context.Response.AppendHeader("Content-Disposition", "attachment; filename=" + name + ".xls");
context.Response.End();
}

Use epplus
private void CreateExcel(DataTable dataTable)
{
using (ExcelPackage pck = new ExcelPackage())
{
ExcelWorksheet ws = pck.Workbook.Worksheets.Add("Demo");
ws.Cells["A1"].LoadFromDataTable(dataTable, true);
Response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
Response.AddHeader("content-disposition", "attachment; filename=ExcelDemo.xlsx");
Response.BinaryWrite(pck.GetAsByteArray());
}
}