I am trying to read the same SQL Server file stream in parallel threads, but am having no success.
Mostly I get the following exception (although from time to time I get a other errors):
System.InvalidOperationException: "The process cannot access the file specified because it has been opened in another transaction."
I have searched the internet, and found just a few posts, but as I understand this is supposed to work. I'm using SQL Server 2008 R2.
I've simplified the code to the following: the main code opens a main transaction, and then runs 2 threads in parallel, each thread using a DependentTransaction and copies the SQL Server file stream to a temporary file on the disk.
If I change threadCount to 1, then the code works.
Any idea why this fails?
The code:
class Program
{
private static void Main(string[] args)
{
string path = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
Directory.CreateDirectory(path);
try
{
using (var transactionScope = new TransactionScope(TransactionScopeOption.Required))
{
TransactionInterop.GetTransmitterPropagationToken(Transaction.Current);
const int threadCount = 2;
var transaction = Transaction.Current;
// Create dependent transactions, one for each thread
var dependentTransactions = Enumerable
.Repeat(transaction.DependentClone(DependentCloneOption.BlockCommitUntilComplete), threadCount)
.ToList();
// Copy the file from the DB to a temporary files, in parallel (each thread will use a different temporary file).
Parallel.For(0, threadCount, i =>
{
using (dependentTransactions[i])
{
CopyFile(path, dependentTransactions[i]);
dependentTransactions[i].Complete();
}
});
transactionScope.Complete();
}
}
finally
{
if (Directory.Exists(path))
Directory.Delete(path, true);
}
}
private static void CopyFile(string path, DependentTransaction dependentTransaction)
{
string tempFilePath = Path.Combine(path, Path.GetRandomFileName());
// Open a transaction scope for the dependent transaction
using (var transactionScope = new TransactionScope(dependentTransaction, TransactionScopeAsyncFlowOption.Enabled))
{
using (Stream stream = GetStream())
{
// Copy the SQL stream to a temporary file
using (var tempFileStream = File.OpenWrite(tempFilePath))
stream.CopyTo(tempFileStream);
}
transactionScope.Complete();
}
}
// Gets a SQL file stream from the DB
private static Stream GetStream()
{
var sqlConnection = new SqlConnection("Integrated Security=true;server=(local);initial catalog=DBName");
var sqlCommand = new SqlCommand {Connection = sqlConnection};
sqlConnection.Open();
sqlCommand.CommandText = "SELECT GET_FILESTREAM_TRANSACTION_CONTEXT()";
Object obj = sqlCommand.ExecuteScalar();
byte[] txContext = (byte[])obj;
const string path = "\\\\MyMachineName\\MSSQLSERVER\\v1\\DBName\\dbo\\TableName\\TableName\\FF1444E6-6CD3-4AFF-82BE-9B5FCEB5FC96";
var sqlFileStream = new SqlFileStream(path, txContext, FileAccess.Read, FileOptions.SequentialScan, 0);
return sqlFileStream;
}
}
kk
Related
I saw other threads about this problem, and non of them seems to solve my exact problem.
static void RecordUpdater(string username,int points,string term) //Updates Record File with New Records.
{
int minPoints = 0;
StreamWriter streamWriter = new StreamWriter($#"Record\{term}");
Player playersRecord = new Player(points, username);
List<Player> allRecords = new List<Player>();
StreamReader reader = new StreamReader($#"Record\{term}");
while (!reader.EndOfStream)
{
string[] splitText = reader.ReadLine().Split(',');
Player record = new Player(Convert.ToInt32(splitText[0]), splitText[1]);
allRecords.Add(record);
}
reader.Close();
foreach (var playerpoint in allRecords )
{
if(minPoints > playerpoint.points)
minPoints = playerpoint.points;
}
if (points > minPoints)
{
allRecords.Add(playersRecord);
allRecords.Remove(allRecords.Min());
}
allRecords.Sort();
allRecords.Reverse();
streamWriter.Flush();
foreach (var player in allRecords)
{
streamWriter.WriteLine(player.points + "," + player.username);
}
}
So after I run the program and get to that point in code I get an error message:
"The process cannot access the file 'fileName/textFile.txt' because it is being used by another process."
You should use the using statement around disposable objects like streams. This will ensure that the objects release every unmanaged resources they hold. And don't open the writer until you need it. Makes no sense to open the writer when first you need to read the records
static void RecordUpdater(string username,int points,string term)
{
Player playersRecord = new Player(points, username);
List<Player> allRecords = new List<Player>();
int minPoints = 0;
try
{
using(StreamReader reader = new StreamReader($#"Record\{term}"))
{
while (!reader.EndOfStream)
{
.... load you data line by line
}
}
..... process your data .....
using(StreamWriter streamWriter = new StreamWriter($#"Record\{term}"))
{
... write your data...
}
}
catch(Exception ex)
{
... show a message about the ex.Message or just log everything
in a file for later analysis
}
}
Also you should consider that working with files is one of the most probable context in which you could receive an exception due to external events in which your program has no control.
It is better to enclose everything in a try/catch block with proper handling of the exception
I wanted to write a Program that reads an online Calendar, compares it with Names in a Database and uses this Data in some way. But if I use the WebClient, it reads the Source Code of the Website, not the Content. This is my Code:
using System;
using System.Diagnostics;
using System.Net;
using MySql.Data.MySqlClient;
namespace CalendarCrawler
{
class Program
{
static void KillTask(string Task)
{
Process[] Process = new Process[] { };
Process = Process.GetProcessesByName(Task);
foreach (Process Instance in Process)
{
Instance.Kill();
}
}
static String ReadContent(String Website)
{
WebClient web = new WebClient();
System.IO.Stream stream = web.OpenRead(Website);
using (System.IO.StreamReader reader = new System.IO.StreamReader(stream))
{
String text = reader.ReadToEnd();
return text;
}
}
static void Main(string[] args)
{
Console.WriteLine("Getting Connection ...");
var datasource = "localhost";//your server
var database = "database"; //your database name
var username = "username"; //username of server to connect
var password = "password"; //password
//your connection string
string connStr = $"Server={datasource};Database={database};Uid={username};Pwd={password}";
//create instanace of database connection
using (var conn = new MySqlConnection(connStr))
{
try
{
Console.WriteLine("Openning Connection ...");
//open connection
conn.Open();
Console.WriteLine("Connection successful!");
}
catch (Exception e)
{
Console.WriteLine("Error: " + e.Message);
}
String Websitetext = ReadContent("http://www.esel.at/termine");
var stm = $"INSERT INTO content(Content) VALUES (#1);";
var cmd = new MySqlCommand(stm, conn);
cmd.Parameters.AddWithValue("#1", Websitetext);
cmd.ExecuteNonQuery();
Console.WriteLine(Websitetext);
KillTask("CalendarCrawler");
}
}
}
}
The Killtask Method is only to Clear it from the Background Processes, so there are no Problems with building a new Version.
I hope someone can help me.
There are effectively 2 different users of the web. People and computers. People like shiney things like buttons and tables (UI), computers prefer things like XML or JSON (API).
Your calendar web site has a UI, this is what you are currently seeing (both in a browser and when you 'download the code'). It probably has an API too and that is what you should be using in your program.
I've just had a quick look at esel.at and it doesn't appear to have a (public) API (but maybe that is because Google can't translate the page properly).
I am attempting to create a Web API that can convert a styled HTML file into a PDF.
I am using TuesPechkin and have installed my application into IIS (as a 32-bit app: I have modified the application pool to run in 32bit mode).
IIS 8.5 is running on Windows Server 2012 R2.
PDFConversion class in C#:
using System.Drawing.Printing;
using System.IO;
using TuesPechkin;
namespace PDFApi
{
public class PDFcreator
{
public void convert(string path, string uri)
{
IConverter converter = new StandardConverter(
new RemotingToolset<PdfToolset>(
new Win64EmbeddedDeployment(
new TempFolderDelpoyment())));
var document = new HtmlToPdfDocument
{
GlobalSettings =
{
ProduceOutline = true,
DocumentTitle = "Converted Form",
PaperSize = PaperKind.A4,
Margins =
{
All = 1.375,
Unit = Unit.Centimeters
}
},
Objects =
{
new ObjectSettings { RawData = File.ReadAllBytes(uri) }
}
};
byte[] pdfBuf = converter.Convert(document);
FileStream fs = new FileStream(path, FileMode.Create);
fs.Write(pdfBuf, 0, pdfBuf.Length);
fs.Close();
}
}
}
The Controller is as follows:
[Route("converthtml")]
[HttpPost]
[MIMEContentFilter]
public async Task<HttpResponseMessage> ConvertHtml()
{
string temppath = System.IO.Path.GetTempPath();
var streamProvider = new MultipartFormDataStreamProvider(temppath);
await Request.Content.ReadAsMultipartAsync(streamProvider);
string filepath = streamProvider.FileData.Select(entry => entry.LocalFileName.Replace(temppath + "\\", "")).First<string>();
string pdfpath = System.IO.Path.GetTempFileName();
pdfpath = pdfpath.Substring(0, pdfpath.LastIndexOf('.')) + ".pdf";
new PDFcreator().convert(pdfpath, filepath);
var stream = new FileStream(pdfpath, FileMode.Open);
var result = new HttpResponseMessage(HttpStatusCode.OK);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
return result;
}
Here's where it gets a little odd: Experimenting in Fiddler, sending a file once will return the PDF immediately. However, all subsequent POSTs will leave Fiddler hanging.
Examining Task Manager shows the CPU and Memory for this task to jump up to 13.5% and ~96MB respectively.
The Temp folder (where the files are stored), on a successful run, will have three files in it: the original uploaded file (stored with a GUID-like name), the file generated via wkHtmlToPdf (in the form "wktemp-"), and the generated pdf (as tempXXXX.pdf).
In the case of it hanging, only the first file can be found, indicating that the problem is somewhere in wkHtmlToPdf itself.
However, the real kicker is when the process is manually killed in Task Manager, the API completes successfully, returns the pdf, fully created!
If IIS is reset, the process returns to the original state; a new attempt will work without issue.
Obviously, resetting IIS after every call is hardly viable, nor is manually killing the process each time...
Is this a bug / are there any solutions to this issue?
EDIT: Very important - this answer had a lot of votes, but is not complete. Check the marked solution
var tempFolderDeployment = new TempFolderDeployment();
var win32EmbeddedDeployment = new Win32EmbeddedDeployment(tempFolderDeployment);
var remotingToolset = new RemotingToolset<PdfToolset>(win32EmbeddedDeployment);
var converter =
new ThreadSafeConverter(remotingToolset);
byte[] pdfBuf = converter.Convert(document);
remotingToolset.Unload();
Unload remotingToolset will prevent hanging
RemotingToolset.Unload();
Very important-
#toshkata-tonev answer helped me, but when a lot of sessions used this function our server crushed because over CPU!
The point is that the process should be shared by all sessions as static shared function.
So this was the solution for me:
Implementation:
Create a static class in your application:
public static class TuesPechkinInitializerService {
private static string staticDeploymentPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "wkhtmltopdf");
public static void CreateWkhtmltopdfPath()
{
if (Directory.Exists(staticDeploymentPath) == false)
{
Directory.CreateDirectory(staticDeploymentPath);
}
}
public static IConverter converter =
new ThreadSafeConverter(
new RemotingToolset<PdfToolset>(
new Win64EmbeddedDeployment(
new StaticDeployment(staticDeploymentPath)
)
)
);
}
In GLOBAL.asax, I initialize that class on project start:
TuesPechkinInitializerService.CreateWkhtmltopdfPath();
And to use it:
HtmlToPdfDocument pdfDocument = new HtmlToPdfDocument
{
GlobalSettings = new GlobalSettings(),
Objects =
{
new ObjectSettings
{
ProduceLocalLinks = true,
ProduceForms = true,
HtmlText = htmlContent
}
}
};
byte[] pdfDocumentData = TuesPechkinInitializerService.converter.Convert(pdfDocument);
Thanks to:
https://github.com/tuespetre/TuesPechkin/issues/152
It seems you are using WkHtmlToPdf 64 bits and you've installed the 32 bits one.
You should change :
IConverter converter = new StandardConverter(
new RemotingToolset<PdfToolset>(
new Win64EmbeddedDeployment(
new TempFolderDelpoyment())));
to
IConverter converter = new StandardConverter(
new RemotingToolset<PdfToolset>(
new Win32EmbeddedDeployment(
new TempFolderDelpoyment())));
I am using a Streamwriter method for a button click to run a test, but after the test is complete I want to run it again and record the results in the same text file.
I always receive an error that the file is still in another process even though the previous test has been completed. I have tried using .Close() but then I get a "Can't write to closed textwriter" error message.
StreamWriter ResultsFile = new StreamWriter(FileName, true);
public void ToFile()
{
ResultsFile.AutoFlush = true;
Console.SetOut(ResultsFile);
Console.WriteLine("Regression Test Performed at {0}", thisDate);
Console.WriteLine("-----");
}
[Test]
public void NewTextFile()
{
ToFile();
const int requiredNumber = 5;
for (var i = 0; i < requiredNumber; i++)
{
Console.WriteLine("New ID assigned to ABC");
Console.WriteLine("-----");
}
}
You should not reuse the StreamWriter like that, try the following instead.
public void ToFile(StreamWriter sw)
{
sw.AutoFlush = true;
Console.SetOut(sw);
Console.WriteLine("Regression Test Performed at {0}", thisDate);
Console.WriteLine("-----");
}
[Test]
public void NewTextFile()
{
using (var sw = new StreamWriter(FileName, true))
{
ToFile(sw);
const int requiredNumber = 5;
for (var i = 0; i < requiredNumber; i++)
{
Console.WriteLine("New ID assigned to ABC");
Console.WriteLine("-----");
}
}
}
Explanation of using can be found here. I would also advise you to read about IDisposable interface.
Beware that after exiting the using statement, calling Console.WriteLine would result into ObjectDisposedException as you would be trying to write into disposed stream. To prevent this situation you can cache the return value of Console.Out into local variable before entering the using statement and after exiting the using statement you can restore the state by again calling the SetOut with cached value of original stream, i.e.:
var original = Console.Out;
using (var sw = new StreamWriter(FileName, true))
{
// ...
}
Console.SetOut(original);
I'm trying to run a command on a remote server via SSH.
I need the output of the command that is run to be saved in a file on that remote server.
I've been attempting to this the following way
// ssh is the SshClient which is already set up
ssh.Connect();
ssh.RunCommand("echo 1 > C:\test.csv"); //Doesn't create a file
ssh.Disconnect();
Why doesn't this work with SSH.NET? If I run this via putty using the same credentials it works perfectly fine.
EDIT (Working Code):
I did some more playing around and have found the following to work:
// ssh is the SshClient which is already set up
ssh.Connect();
var shell = ssh.CreateShellStream("cmd.exe", 80, 24, 800, 600, 1024);
var reader = new StreamReader(shell);
var writer = new StreamWriter(shell);
writer.AutoFlush = true;
while (!shell.DataAvailable)
System.Threading.Thread.Sleep(1000); //This wait period seems required
writer.WriteLine("echo 1 > C:\test.csv");
while (!shell.DataAvailable)
System.Threading.Thread.Sleep(1000); //This wait period seems required
ssh.Disconnect();
While that works I still don't understand what's really happening here. Could someone explain?
Try this function:
Just save the result to a variable or write the result using StreamWriter
private void writeMe()
{
using (StreamWriter sw = new StreamWriter(filename)
{
string result = eSshCom(command);
sw.WriteLine(result);
}
}
private string eSshCom(string getCommand)
{
this.res = "";
var connectionInfo = new KeyboardInteractiveConnectionInfo(ipaddress, 22, username);
connectionInfo.AuthenticationPrompt += delegate(object asender, AuthenticationPromptEventArgs xe)
{
foreach (var prompt in xe.Prompts)
{
if (prompt.Request.Equals("Password: ", StringComparison.InvariantCultureIgnoreCase))
{
prompt.Response = password;
}
}
};
using (var ssh = new SshClient(connectionInfo))
{
ssh.Connect();
var cmd = ssh.RunCommand(getCommand);
this.res = cmd.Result;
ssh.Disconnect();
}
return this.res;
}