Recently, I found that Windows Phone Background Transfer Service seems to have memory leak issue.
Every background transfer you add will take a memory space, which cannot be removed by GC forever.
I already read through http://msdn.microsoft.com/en-us/library/windowsphone/develop/hh202959(v=vs.105).aspx , and still have no idea where the memory leak comes from.
What I test is very simple, add a background transfer request to the BackgroundTransferService and when that request is complete, remove it from BackgroundTransferService and add another one. If I keep doing it, I'll see the memory grows even when GC.collect being called every sec. Please download the testing code in http://hdtp.synology.me/BTS.zip and you'll know what I'm saying. The following is the testing code summary.
private int _transferCount = 1000;
private void CreateTask()
{
if (--_transferCount < 0)
{
MessageBox.Show("End");
return;
}
// Get the URI of the file to be transferred from the Tag property
// of the button that was clicked.
//string transferFileName = ((Button)sender).Tag as string;
string transferFileName = "http://hdtp.synology.me/a.jpg";
Uri transferUri = new Uri(Uri.EscapeUriString(transferFileName + "?ranNum=" + _transferCount), UriKind.RelativeOrAbsolute);
// Create the new transfer request, passing in the URI of the file to
// be transferred.
BackgroundTransferRequest transferRequest = new BackgroundTransferRequest(transferUri);
// Set the transfer method. GET and POST are supported.
transferRequest.Method = "GET";
// Get the file name from the end of the transfer Uri and create a local Uri
// in the "transfers" directory in isolated storage.
string downloadFile = transferFileName.Substring(transferFileName.LastIndexOf("/") + 1);
Uri downloadUri = new Uri("shared/transfers/" + downloadFile, UriKind.RelativeOrAbsolute);
transferRequest.DownloadLocation = downloadUri;
// Pass custom data with the Tag property. This value cannot be more than 4000 characters.
// In this example, the friendly name for the file is passed.
transferRequest.Tag = downloadFile;
// Add the transfer request using the BackgroundTransferService. Do this in
// a try block in case an exception is thrown.
try
{
BackgroundTransferService.Add(transferRequest);
}
catch (InvalidOperationException ex)
{
// TBD - update when exceptions are finalized
MessageBox.Show("Unable to add background transfer request. " + ex.Message);
}
catch (Exception)
{
MessageBox.Show("Unable to add background transfer request.");
}
InitialTansferStatusCheck();
}
private void InitialTansferStatusCheck()
{
UpdateRequestsList();
foreach (var transfer in transferRequests)
{
transfer.TransferStatusChanged += new EventHandler<BackgroundTransferEventArgs>(transfer_TransferStatusChanged);
ProcessTransfer(transfer);
}
}
private void transfer_TransferStatusChanged(object sender, BackgroundTransferEventArgs e)
{
ProcessTransfer(e.Request);
}
private void UpdateRequestsList()
{
// The Requests property returns new references, so make sure that
// you dispose of the old references to avoid memory leaks.
if (transferRequests != null)
{
foreach (var request in transferRequests)
{
request.Dispose();
}
}
transferRequests = BackgroundTransferService.Requests;
}
private void ProcessTransfer(BackgroundTransferRequest transfer)
{
switch (transfer.TransferStatus)
{
case TransferStatus.Completed:
// If the status code of a completed transfer is 200 or 206, the
// transfer was successful
if (transfer.StatusCode == 200 || transfer.StatusCode == 206)
{
// Remove the transfer request in order to make room in the
// queue for more transfers. Transfers are not automatically
// removed by the system.
RemoveTransferRequest(transfer.RequestId);
// In this example, the downloaded file is moved into the root
// Isolated Storage directory
using (IsolatedStorageFile isoStore = IsolatedStorageFile.GetUserStoreForApplication())
{
string filename = transfer.Tag;
if (isoStore.FileExists(filename))
{
isoStore.DeleteFile(filename);
}
isoStore.MoveFile(transfer.DownloadLocation.OriginalString, filename);
}
CreateTask();
}
else
{
// This is where you can handle whatever error is indicated by the
// StatusCode and then remove the transfer from the queue.
RemoveTransferRequest(transfer.RequestId);
if (transfer.TransferError != null)
{
// Handle TransferError, if there is one.
}
}
break;
}
}
private void RemoveTransferRequest(string transferID)
{
// Use Find to retrieve the transfer request with the specified ID.
BackgroundTransferRequest transferToRemove = BackgroundTransferService.Find(transferID);
// try to remove the transfer from the background transfer service.
try
{
BackgroundTransferService.Remove(transferToRemove);
}
catch (Exception ex)
{
}
}
Another few questions, according to the documentation above, we'll get the new instance from BackgroundTransferService.Requests every time, but if I called GetHashCode(), I get the same hash code every time and the hash code is even the same with the one I newed and added into BackgroundTransferService. So is it because MS override the GetHashCode method of BackgroundTransferRequest? or I misunderstand something. But in the sample code above, I did not use BackgroundTransferService.Requests to get any instance, the memory still keep growing.
Please tell me what I do wrong or any workaround, thanks...
Related
On VS2019, when using this OneDrive sample with UWP from Microsoft, I am getting the following error. An online search shows some relevant links (such as this or this or this) but their context are different (as they are using web apps or Python etc.):
AADSTS50011: The reply URL specified in the request does not match the reply URLs configured for the application: '55dbdbc9-xxxxxxxxxxxxx-a24'
I have followed the sample's instructions for Registering and Configuring the app where Redirect URI I have selected is Public client (mobile & desktop), and have set it's value to https://login.microsoftonline.com/common/oauth2/nativeclient
Question: What I may be doing wrong, and how can we resolve the issue?
UPDATE:
Error occurs at line FolderLoaded?.Invoke(this, EventArgs.Empty); of the method shown below. This is line 180 of file OneDriveList.xaml.cs in the sample. And it is not the error OperationCanceledException since error goes to the second catch statement.
private async Task LoadFolderAsync(string id = null)
{
// Cancel any previous operation
_cancellationTokenSource?.Cancel();
_cancellationTokenSource = new CancellationTokenSource();
// Check if session is set
if (AuthenticationService == null) throw new InvalidOperationException($"No {nameof(AuthenticationService)} has been specified");
// Keep a local copy of the token because the source can change while executing this function
var token = _cancellationTokenSource.Token;
// Add an option to the REST API in order to get thumbnails for each file
// https://learn.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_list_thumbnails
var options = new[]
{
new QueryOption("$expand", "thumbnails"),
};
// Create the graph request builder for the drive
IDriveRequestBuilder driveRequest = AuthenticationService.GraphClient.Me.Drive;
// If folder id is null, the request refers to the root folder
IDriveItemRequestBuilder driveItemsRequest;
if (id == null)
{
driveItemsRequest = driveRequest.Root;
}
else
{
driveItemsRequest = driveRequest.Items[id];
}
// Raise the loading event
FolderLoading?.Invoke(this, EventArgs.Empty);
try
{
try
{
// Make a API request loading 50 items per time
var page = await driveItemsRequest.Children.Request(options).Top(50).GetAsync(token);
token.ThrowIfCancellationRequested();
// Load each page
await LoadGridItemsAsync(page, token);
token.ThrowIfCancellationRequested();
}
finally
{
// Raise the loaded event
FolderLoaded?.Invoke(this, EventArgs.Empty);
}
}
catch (OperationCanceledException)
{ }
catch (Exception ex)
{
// Raise the error event
LoadingError?.Invoke(this, ex);
}
}
I have a C# WinForms application running on Raspbian with Mono. It has a timer. When the OnTimedEvent fires, I check if I have exclusive access to a file that I want to upload (to make sure it is finished being written to disk), then attempt to upload. If the upload is successful, I move the file to an archive folder, otherwise I leave it there and wait for the next timer event. I have no problems when connected to the Internet, but when I test without and my upload fails, the second OnTimedEvent gets an exception when checking if the same file is ready (again). I am getting :
Error message: ***Sharing violation on path 'path'
***HResult: ***-2147024864
Method to check if file is ready:
public static bool IsFileReady(string filename)
{
// If the file can be opened for exclusive access it means that the file
// is no longer locked by another process.
try
{
var inputStream = File.Open(filename, FileMode.Open, FileAccess.Read, FileShare.None);
bool test = inputStream.Length > 0;
inputStream.Close();
inputStream.Dispose();
return test;
}
catch (Exception e)
{
//log
throw e;
}
}
This is what executes on the OntimedEvent:
var csvFiles = from f in di.GetFiles()
where f.Extension == ".csv"
select f; //get csv files in upload folder
foreach (var file in csvFiles)
{
if (IsFileReady(file.FullName)) //check that file is done writing before trying to move.
{
bool IsUploadSuccess = await WritingCSVFileToS3Async(file);//.Wait(); //upload file to S3
if (IsUploadSuccess)
{
File.Move(file.FullName, archivePath + file.Name); //move to completed folder if upload successful. else, leave there for next upload attempt
}
}
}
From what I can understand, it looks like my first FileStream (File.Open) still has the file locked when the 2nd event fires. However, I've added .Close() and .Dispose() to the IsFileReady method but that doesn't seem to be working.
Any help would be appreciated!
EDIT: Below is the WritingCSVFileToS3Async method.
static async Task<bool> WritingCSVFileToS3Async(FileInfo file)
{
try
{
client = new AmazonS3Client(bucketRegion);
// Put the object-set ContentType and add metadata.
var putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = file.Name,
FilePath = file.FullName ,
ContentType = "text/csv"
};
//putRequest.Metadata.Add("x-amz-meta-title", "someTitle"); //don't need meta data at this time
PutObjectResponse response = await client.PutObjectAsync(putRequest);
if (response.HttpStatusCode == System.Net.HttpStatusCode.OK)
return true;
else
return false;
}
catch (AmazonS3Exception e)
{
ErrorLogging.LogErrorToFile(e);
return false;
}
catch (Exception e)
{
ErrorLogging.LogErrorToFile(e);
return false;
}
Also, I ran the same application on Windows, and am getting a similar exception:
The process cannot access the file 'path' because it is being used by another process.
I believe I've found the problem. I noticed that I was not catching the client timeout exception for the PUT request(not connected to internet). My timer interval was 20 seconds, which is shorter than the S3 client timeout (30 seconds). So the client still had the file tied up by the time the second timer event fired, hence the access violation. I increased the timer interval to 60 seconds, and I now catch the client timeout exception and can handle it before the next timer event.
Thanks for your help.
I have been trying to create a metro application but there is a problem: StreamSocket doesn't really do what I want to do (I think)
Here is an excerpt my code from .Net that works:
try
{
TCP = new TcpClient(server, port);
Stream = TCP.GetStream();
Read = new StreamReader(Stream);
Write = new StreamWriter(Stream);
}
catch (Exception e)
{
Console.WriteLine("Error connecting to " + server + ": " + e);
return;
}
// Identify
Write.WriteLine("LOGIN " + Username);
Write.Flush();
while (Connected)
{
try
{
if ((line = Read.ReadLine()) != null && Connected)
I can't get StreamSocket to work... it requires you to know the length of the string that's coming in and I don't know what it will be - it varies. Is there any way to do this that will work?
This is what I have but it doesn't work:
try
{
// Connect to the server (in our case the listener we created in previous step).
await Socket.ConnectAsync(new HostName("example.com"), "1111");
}
catch (Exception exception)
{
// If this is an unknown status it means that the error is fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
}
// Create a DataWriter if we did not create one yet. Otherwise use one that is already cached.
Writer = new DataWriter(Socket.OutputStream);
Listener = new DataReader(Socket.InputStream);
Debug.WriteLine(Socket.Information.RemoteAddress.CanonicalName); //Check if IP is correct
SendRaw("LOGIN " + Nickname);
string line = "";
Connected = true;
while (Connected)
{
if (Listener.UnconsumedBufferLength != 0)
{
line = Listener.ReadString(Listener.UnconsumedBufferLength);
Debug.WriteLine(line);
}
}
}
async public void SendRaw(string str)
{
Writer.WriteString(str);
// Write the locally buffered data to the network.
try
{
await Writer.StoreAsync();
}
catch (Exception exception)
{
// If this is an unknown status it means that the error if fatal and retry will likely fail.
if (SocketError.GetStatus(exception.HResult) == SocketErrorStatus.Unknown)
{
throw;
}
}
}
Any help would be appreciated!
First things first: your original code is a DOS attack waiting to happen. If possible, I would recommend changing the protocol to include a length prefix before every string so you can tell how big it will be before allocating memory for it.
Second things second: the DataReader class must read a number of bytes into its internal buffer before it can interpret them. You read into this buffer by calling LoadAsync.
However, if you want to read a string of arbitrary length, you'll have to read into a buffer and scan for your newline yourself, resizing the buffer (or adding new buffers) as necessary if the newline isn't found, up to some maximum size.
Update:
Set InputStreamOptions to Partial; the you can call LoadAsync with an arbitrary large buffer size (e.g. 1024). After getting data, call ReadString(UnconsumedBufferLength). Each time you do this, you may get part of a line, a line, or more than a line. So you'll have to build up a string and then Split by \n, keeping any partial line at the end for the next time through the loop.
I have a page which is fetching data from a webservice using async call.
If i get the response from webservice control goes to catch where a message box is pooped.
The code is given below:
string uri = "http://free.worldweatheronline.com/feed/weather.ashx?key=b7d3b5ed25080109113008&q=Mumbai&num_of_days=5";
UriBuilder fullUri = new UriBuilder("http://free.worldweatheronline.com/feed/weather.ashx");
fullUri.Query = "key=b7d3b5ed25080109113008&q=Mumbai&num_of_days=5";
HttpWebRequest forecastRequest = (HttpWebRequest)WebRequest.Create(fullUri.Uri);
// set up the state object for the async request
ForecastUpdateState forecastState = new ForecastUpdateState();
forecastState.AsyncRequest = forecastRequest;
// start the asynchronous request
forecastRequest.BeginGetResponse(new AsyncCallback(HandleForecastResponse), forecastState);
This part is response
private void HandleForecastResponse(IAsyncResult asyncResult)
{
try
{
// get the state information
ForecastUpdateState forecastState = (ForecastUpdateState)asyncResult.AsyncState;
HttpWebRequest forecastRequest = (HttpWebRequest)forecastState.AsyncRequest;
// end the async request
forecastState.AsyncResponse = (HttpWebResponse)forecastRequest.EndGetResponse(asyncResult);
Stream streamResult;
string newCityName = "";
//int newHeight = 0;
// get the stream containing the response from the async call
streamResult = forecastState.AsyncResponse.GetResponseStream();
// load the XML
XElement xmlWeather = XElement.Load(streamResult);
}
catch (Exception ex)
{
MessageBox.Show("Connection Error");
}
}
Problem :
when the page is loaded it starts fetching data from webservice(consider the case when the web service is not responding and control goes to catch part).
In the mean time if we press the back button or navigate the page the message box popps on the new page.
How could i stop that.
Thanks and Regards
Haven't tested it, but it may work:
1/ Store the value of the NavigationService.CurrentSource property somewhere it can be retrieved (the best would be in the asyncState parameter, but a property may work as well
2/ In the HandleForecastResponse, compare the old and new value of the NavigationService.CurrentSource. This way, you should be able to deduce if the active page has changed or not.
ifixed that problem by add
System.Windows.Deployment.Current.Dispatcher.BeginInvoke(() =>
{
});
try this
private void HandleForecastResponse(IAsyncResult asyncResult)
{
try
{
// get the state information
ForecastUpdateState forecastState = (ForecastUpdateState)asyncResult.AsyncState;
HttpWebRequest forecastRequest = (HttpWebRequest)forecastState.AsyncRequest;
// end the async request
forecastState.AsyncResponse = (HttpWebResponse)forecastRequest.EndGetResponse(asyncResult);
Stream streamResult;
string newCityName = "";
//int newHeight = 0;
// get the stream containing the response from the async call
streamResult = forecastState.AsyncResponse.GetResponseStream();
// load the XML
XElement xmlWeather = XElement.Load(streamResult);
}
catch (Exception ex)
{
System.Windows.Deployment.Current.Dispatcher.BeginInvoke(() =>
{
MessageBox.Show("Connection Error");
});
}
}
Finally solved it.
catch (Exception x)
{
Deployment.Current.Dispatcher.BeginInvoke(() =>
{
var currentPage = ((App)Application.Current).RootFrame.Content as PhoneApplicationPage;
if ((currentPage.ToString()).Equals("MumbaiMarathon.Info.News"))
{
MessageBox.Show("Connection Error");
}
});
}
I just checked at the time of popping the message box the name of the Current UI application page. If its same as the page from which message box is initiated than it pops otherwise not.
I am trying to build a c# console app that will monitor about 3000 urls (Just need to know that HEAD request returned 200, not necessarily content, etc.)
My attempt here was to build a routine the checks the web URLS, looping and creating threads each executing the routine. What's happening is if i run with <20 threads, it executes ok most of the time, but if i use >20 threads, some of the url's time out. I tried increasing the Timeout to 30 seconds, same occurs. The network I am running this on is more than capable of executing 50 HTTP HEAD requests (10MBIT connection at ISP), and both the CPU and network run very low when executing the routine.
When a timeout occurs, i test the same IP on a browser and it works fine, I tested this repeatedly and there was never a case during testing that a "timed out" url was actually timing out.
The reason i want to run >20 threads is that i want to perform this test every 5 minutes, with some of the URL's taking a full 10sec (or higher if the timeout is set higher), i want to make sure that its able to run through all URLs within 2-3 minutes.
Is there a better way to go about checking if a URL is available, or, should I be looking at the system/network for an issue.
MAIN
while (rdr.Read())
{
Thread t = new Thread(new ParameterizedThreadStart(check_web));
t.Start(rdr[0]);
}
static void check_web(object weburl)
{
bool isok;
isok = ConnectionAvailable(weburl.ToString());
}
public static bool ConnectionAvailable(string strServer)
{
try
{
strServer = "http://" + strServer;
HttpWebRequest reqFP = (HttpWebRequest)HttpWebRequest.Create(strServer);
reqFP.Timeout = 10000;
reqFP.Method = "HEAD";
HttpWebResponse rspFP = (HttpWebResponse)reqFP.GetResponse();
if (HttpStatusCode.OK == rspFP.StatusCode)
{
Console.WriteLine(strServer + " - OK");
rspFP.Close();
return true;
}
else
{
Console.WriteLine(strServer + " Server returned error..");
rspFP.Close();
return false;
}
}
catch (WebException x)
{
if (x.ToString().Contains("timed out"))
{
Console.WriteLine(strServer + " - Timed out");
}
else
{
Console.WriteLine(x.Message.ToString());
}
return false;
}
}
Just remember, you asked.
Very bad implementation.
Do not go creating threads like that. It does very little good to have more threads than processor cores. The extra threads will pretty much just compete with each other, especially since they're all running the same code.
You need to implement using blocks. If you throw an exception (and chances are you will), then you will be leaking resources.
What is the purpose in returning a bool? Do you check it somewhere? In any case, your error and exception processing are a mess.
When you get a non-200 response, you don't display the error code.
You're comparing against the Message property to decide if it's a timeout. Microsoft should put a space between the "time" and "out" just to spite you.
When it's not a timeout, you display only the Message property, not the entire exception, and the Message property is already a string and doesn't need you to call ToString() on it.
Next Batch of Changes
This isn't finished, I don't think, but try this one:
public static void Main()
{
// Don't mind the interpretation. I needed an excuse to define "rdr"
using (var conn = new SqlConnection())
{
conn.Open();
using (var cmd = new SqlCommand("SELECT Url FROM UrlsToCheck", conn))
{
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
// Use the thread pool. Please.
ThreadPool.QueueUserWorkItem(
delegate(object weburl)
{
// I invented a reason for you to return bool
if (!ConnectionAvailable(weburl.ToString()))
{
// Console would be getting pretty busy with all
// those threads
Debug.WriteLine(
String.Format(
"{0} was not available",
weburl));
}
},
rdr[0]);
}
}
}
}
}
public static bool ConnectionAvailable(string strServer)
{
try
{
strServer = "http://" + strServer;
var reqFp = (HttpWebRequest)WebRequest.Create(strServer);
reqFp.Timeout = 10000;
reqFp.Method = "HEAD";
// BTW, what's an "FP"?
using (var rspFp = (HttpWebResponse) reqFp.GetResponse()) // IDisposable
{
if (HttpStatusCode.OK == rspFp.StatusCode)
{
Debug.WriteLine(string.Format("{0} - OK", strServer));
return true; // Dispose called when using is exited
}
// Include the error because it's nice to know these things
Debug.WriteLine(String.Format(
"{0} Server returned error: {1}",
strServer, rspFp.StatusCode));
return false;
}
}
catch (WebException x)
{
// Don't tempt fate and don't let programs read human-readable messages
if (x.Status == WebExceptionStatus.Timeout)
{
Debug.WriteLine(string.Format("{0} - Timed out", strServer));
}
else
{
// The FULL exception, please
Debug.WriteLine(x.ToString());
}
return false;
}
}
Almost Done - Not Tested Late Night Code
public static void Main()
{
using (var conn = new SqlConnection())
{
conn.Open();
using (var cmd = new SqlCommand("", conn))
{
using (var rdr = cmd.ExecuteReader())
{
if (rdr == null)
{
return;
}
while (rdr.Read())
{
ThreadPool.QueueUserWorkItem(
CheckConnectionAvailable, rdr[0]);
}
}
}
}
}
private static void CheckConnectionAvailable(object weburl)
{
try
{
// If this works, it's a lot simpler
var strServer = new Uri("http://" + weburl);
using (var client = new WebClient())
{
client.UploadDataCompleted += ClientOnUploadDataCompleted;
client.UploadDataAsync(
strServer, "HEAD", new byte[] {}, strServer);
}
}
catch (WebException x)
{
Debug.WriteLine(x);
}
}
private static void ClientOnUploadDataCompleted(
object sender, UploadDataCompletedEventArgs args)
{
if (args.Error == null)
{
Debug.WriteLine(string.Format("{0} - OK", args.UserState));
}
else
{
Debug.WriteLine(string.Format("{0} - Error", args.Error));
}
}
Use ThreadPool class. Don't spawn hundreds of threads like this. Threads have such a huge overhead and what happens in your case is that your CPU will spend 99% time on context switching and 1% doing real work.
Don't use threads.
Asynch Call backs and queues. Why create a thread when the resource that they are all wanting is access to the outside world. Limit your threads to about 5, and then implement a class that uses a queue. split the code into two parts, the fetch and the process. One controls the flow of data while the other controls access to the outside world.
Use whatever language you like but you won't got wrong if you think that threads are for processing and number crunching and async call backs are for resource management.