I am getting this issue while trying to read data from an excel file using OleDb.
This is working fine on my test server (Window Server 2008) but not working in UAT server with the same configuration.
It used to work on the UAT server, but suddenly stopped working.
I tried solutions from these sources, but nothing worked:
Intermittent "System resource exceeded" exception for OleDB connection to Microsoft Access data file
OleDbException System Resources Exceeded
System resource exceeded
using (OleDbConnection conn = new OleDbConnection(connectionString))
{
using (OleDbCommand cmd = new OleDbCommand())
{
using (OleDbDataAdapter dataAdapter = new OleDbDataAdapter())
{
cmd.Connection = conn;
//Fetch 1st Sheet Name
//conn.Open();
DataTable dtSchema;
dtSchema = conn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string ExcelSheetName = dtSchema.Rows[0]["TABLE_NAME"].ToString();
//conn.Close();
////Read all data of fetched Sheet to a Data Table
//conn.Open();
cmd.CommandText = "SELECT * From [" + ExcelSheetName + "] Where (F3 = 'Reconstructive' OR F3 = 'Neurovascular' OR F3 = 'Orthobiologics') AND F2 = '"+ SterileProduct.CatalogNumber +"'";
dataAdapter.SelectCommand = cmd;
dataAdapter.Fill(dt);
}
}
}
Any help would be appreciated.
Thanks
Hope this will be helpful for others.
I went through numerous resource regarding this issue and found many solutions -programmatically and configuration based answers, unfortunately, none of the solutions worked for me and finally, i decided to use CSV file instead excel.
everything is working fine now.
Let me know if anybody found any answer to this question.
Thanks
Related
I have a large excel file(530K Rows with a lot of columns). Ends up being 247MB in .xlsb format. I am attempting to import to SQL Server using BulkCopy in C#, however I am having issues where the datareader ends up running out of memory before it even starts reading the file once I run the ExecuteReader() command.
string exlConnString = $"Provider=Microsoft.ACE.OLEDB.12.0;Data Source={_filepath};Extended Properties=\"Excel 12.0;HDR=YES;\"";
string sqlQuery = $"SELECT * FROM [{SheetName}]";
using OleDbConnection conn = new OleDbConnection(_connstring)) {
OleDbCommand exlCmd = new OleDbCommand(sqlQuery, conn)
conn.Open();
OleDbDataReader dr = exlcmd.ExecuteReader(); <---NEVER GETS PAST THIS LINE BEFORE RUNNING OUT OF MEMORY.
SqlBulkCopy bulkCopy = new SqlBulkCopy(sqlConnString);
bulkCopy.DestinationTable = TableName;
while(dr.Read()) {
bulkcopy.WriteToServer(dr);
}
dr.Close();
}
I am running in x86 mode because I was geting an error that the ACE Database was not installed on my local machine and corporate policy restrictions prevent me from downloading and installing the needed file to run it in x64 mode.
The code works perfectly fine when I test it on smaller files, but not when I test it on this bigger file, so it definitely is the filesize causing the issue. Any suggestions or help would be appreciated. Doesn't make much sense that a bulk copy runs out of memory when it is meant for handling large sets of data, which also means that the filesize is going to be large as well...
And yes, I know I SHOULD be able to import this using OPENROWSET or OPENDATASOURCE in SQL Server but THAT is ALSO Turned off and they will not enable it, so this is not an option.
So your problem is next.
When you try to ExecuteReader DataReader attempt to read all data from your excel file to memory. You could think about this, like a speciality working with excel through a OleDbProvider.
So my suggestion is to work with csv files instead of excel, because with csv file you have an ability to read and parse them line by line. For this aim i would recommend you to use CSV helper
Refer this code..
Here dtExcelData is datatable variable & da is OleDbDataAdapter variable.
string excelConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source={_filepath};Extended Properties='Excel 12.0;HDR=YES';";
// Create Connection to Excel Workbook
using (OleDbConnection connection = new OleDbConnection(excelConnectionString))
{
connection.Open();
da = new OleDbDataAdapter("Select * FROM [Sheet1$]", connection);
da.Fill(dtExcelData);
//store data in sql server database table
// below connection string "conString" is I mention in app.config file.(sql server connection string to store data in sql server database)
string str = ConfigurationManager.ConnectionStrings["conString"].ConnectionString;
using (SqlConnection con = new SqlConnection(str))
{
// Bulk Copy to SQL Server
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(con))
{
bulkCopy.DestinationTableName = "TableName";
con.Open();
bulkCopy.WriteToServer(dtExcelData);
con.Close();
}
}
connection.Close();
}
Mark it as a answer if it is useful to you. :)
I am trying to read excel file in c# however it gives me 'External table is not in the expected format.' error.
Below is my sample code
string con =#"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filePath + reportname + #".xls;" +
#"Extended Properties='Excel 8.0;HDR=Yes;'";
using (OleDbConnection connection = new OleDbConnection(con))
{
connection.Open();
OleDbCommand command = new OleDbCommand("select * from ["+reportname+"$]", connection);
using (OleDbDataReader dr = command.ExecuteReader())
{
while (dr.Read())
{...
...
}}}
The strange this is when I try to open the excel manually and save it back, it gives me a pop-up saying 'The report.xls may contain features that are not compatible with web page.Do you want to keep the workbook in this format?' If i select yes the file size reduce from 2.5MB to 10KB.
Now if I run my code, it also works fine.
What should be done in this scenario, as I cannot keep saving the file manually to make it work?
i am creating connection string for oracle in c#
below is my code
OracleConnection cn = new OracleConnection("Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.5 )(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=ORCL)));User Id=nTireoffice ;Password=nTireoffice;");
DataTable dt = new DataTable();
string strSql = " select to_number (nvl(max(nvl(Region_ID ,0)),0)+1 ) as No from BO_REGION_MASTER ;";
cn.Open();
OracleDataAdapter objSda = new OracleDataAdapter(strSql, cn);
objSda.Fill(dt);
string s = dt.Rows[0][0].ToString();
its throwing exception ORA-06413: Connection not open.
i am using windows 8 enterprises 64 bit os
thank u...
Have a look here. (This is a very useful site for clues about most ORA errors.)
It sounds like it could be the folder your executable is in, or the actual name of your executable.
PS It would be useful if you could tell us the actual line the error occurs on.
I've created the connection like this.
ora_cmd = new OracleCommand();
ora_con = new OracleConnection("Data Source={YOUR_HOST}:{YOUR_PORT}/{YOUR_DB};Persist Security Info=True;User ID={YOUR_ID};Password={YOUR_PASSWORD};Pooling=True;Max Pool Size=200;");
ora_cmd.Connection = ora_con;
ora_connect.Open();
Then use the ora_cmd to executes statements
I've been doing research on this problem for about a day and a half and haven't found a solution to my problem yet. This is my scenario:
I have a AsyncFileUpload and a button which is configured to accept only xls and xlsx files (Excel) file. This file that is uploaded is used to import information into a gridview. The old format xls files works fine. The problem comes when I want to upload the xlsx file. I've tested multiple scenarios and I have found that it works perfectly when the xlsx file is currectly/actively open. When it is closed it gives me errors. I have fiddled and explored solutions to this problem and the range of errors I have received is quite large. But the most prominent errors where the following:
1. External table is not in the expected format.
2. could-not-find-installable-isam
I have found an article relating to the second error (Could not find installable ISAM) but I haven't tried all the suggested solutions yet because I would rather avoid making changes to the windows registry because I do not have exstensive knowledge regarding this matter.
I'm attaching the code that I use to make the connection between the OleDb and the excel file:
protected void AsyncUpload_UploadedComplete(object sender, AjaxControlToolkit.AsyncFileUploadEventArgs e)
{
string connString = "";
string strFileType = Path.GetExtension(e.FileName).ToLower();
string path = e.FileName;
////Connection String to Excel Workbook
if (strFileType.Trim() == ".xls")
{
connString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + path + ";Extended Properties=\"Excel 8.0;HDR=Yes;IMEX=2\"";
}
else if (strFileType.Trim() == ".xlsx")
{
connString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + path + ";Extended Properties=Excel 12.0;HDR=Yes;IMEX=1";
}
string query = "SELECT * FROM [sheet1$]";
OleDbConnection conn = new OleDbConnection(connString);
if (conn.State == ConnectionState.Closed)
conn.Open();
OleDbCommand cmd = new OleDbCommand(query, conn);
OleDbDataAdapter da = new OleDbDataAdapter(cmd);
DataSet ds = new DataSet();
da.Fill(ds);
using (OleDbDataReader dr = cmd.ExecuteReader())
{
_InsertWrapper.clearLists();
while (dr.Read())
{
var row1Col0 = dr[0];
Console.WriteLine(row1Col0);
_InsertWrapper.GenerateList(dr);
}
}
da.Dispose();
conn.Close();
conn.Dispose();
InventoryGrid.DataBind();
ErrorsGrid.DataBind();
}
If someone has additional knowledge or an explanation that can contribute please add it so that more people that are struggling with the same type of problem can gain as much information from this question as possible.We are all still learning.
So, main idea: Need to find a solution to successfully upload the xlsx file
Try using third party tools. They work efficiently:
http://code.google.com/p/excellibrary/
I've been tasked with a one time migration from a 4D database to our MSSQL structure. I've got a datasource set up in the ODBC administrator and I was able to connect to it without issues.
I 'reverse engineered' the schema in Visio so I can get a good visual of the relationships between the tables and plan out how I'm going to re-structure the 4D data to fit into our schema. I created a simple console application as this will be a one time run, and I'm able to make the connection to the data source, but as soon as I execute anything, the connection drops or is disabled.
System.Data.Odbc.OdbcConnection conn =
new System.Data.Odbc.OdbcConnection("Dsn=4D v12 datasource");
try
{
conn.Open();
Console.WriteLine("Status of Connection:" + conn.State.ToString());
//Here it says the connection to the DB is open and ready for action
Console.ReadLine();
//pause to visually confirm the connection is open
OdbcCommand com = new OdbcCommand("SELECT * FROM ATable", conn);
com.CommandType = CommandType.Text;
Console.Write(com.ExecuteNonQuery());
//Right here it blows up and closes the connection
}
I also tried to do something with data set and data adapter, but to no avail.
System.Data.Odbc.OdbcConnection conn = new System.Data.Odbc.OdbcConnection("Dsn=4D v12 datasource");
try
{
conn.Open();
Console.WriteLine("Status of Connection:" + conn.State.ToString());
Console.ReadLine();
OdbcCommand com = new OdbcCommand("SELECT * FROM ATable", conn);
com.CommandType = CommandType.Text;
//Also tried using data sets and data adapters
DataSet dsTest = new DataSet();
OdbcDataAdapter dataAdapter = new OdbcDataAdapter(com);
//but right at this line the connection suddenly disconnects
dataAdapter.Fill(dsTest);
}
catch (Exception e)
{
Console.WriteLine("Failure:" + e.Message.ToString());
// the exception message reads simply connection has been disabled.
Console.WriteLine("Status of Connection: " + conn.State.ToString());
Console.ReadLine();
}
finally
{
Console.Write("Closing connection.");
conn.Close();
Console.Write(".");
conn.Dispose();
Console.WriteLine(".");
Console.WriteLine("Connection Closed and Disposed");
Console.ReadLine();
}
I've tried to look for anyone experiencing this same difficulty, but the documentation I've found has been scarce and not very helpful in this specific regard. There is a good amount of information about executing queries on the 4D product, but not from across the digital divide. Anyone with experience please advise. Thanks for your time and for any help you may be able to give, avid reader.
Apparently the root cause of the issue was in the driver. Upon replacing the driver I got from the website with one sent from the proprietor, the connections ceased being disabled and all is working as intended.