I'm using restsharp to make a call out to an api and it is not encoding &s in parameter values (that's all I've tried so far with characters that need to be url encoded). I've used it before and looked at the source to double check that it does url encode both the key and value of parameters. Maybe I'm doing something wrong.
...
private static readonly RestClient _client = new RestClient();
public Guid Create(Dto myDto)
{
var request = new RestRequest(Method.GET)
{
Resource = "GetGuid"
};
request.AddParameter("name", myDto.Name);
var response = _client.Execute();
if (response.StatusCode != HttpStatusCode.OK)
{
Log.Error(string.Format("Could not register user with email {0} in crm", user.Email), this);
throw new Exception("Response from crm was not OK");
}
return Guid.Parse(response.Content);
}
...
The version of I was using was 105.0.0 which seems to have some encoding issues: https://github.com/restsharp/RestSharp/blob/master/releasenotes.markdown
I haven't looked at the source for that but bumping my version to 105.0.1 seemed to fix the issue.
Commit with the revert that fixed the encoding issue I encountered.
Related
I have implemented the following method format request body
private async Task<string> FormatRequest(HttpRequest request)
{
request.EnableBuffering();
//Create a new byte[] with the same length as the request stream
var buffer = new byte[Convert.ToInt32(request.ContentLength)];
//Copy the entire request stream into the new buffer
await request.Body.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);
//Convert the byte[] into a string using UTF8 encoding
var bodyAsText = Encoding.UTF8.GetString(buffer);
request.Body.Position = 0;
return bodyAsText;
}
I got the following result
------WebKitFormBoundaryY8OPXY2MlrKMjBRe
Content-Disposition: form-data; name="RoleId"
2
------WebKitFormBoundaryY8OPXY2MlrKMjBRe
Content-Disposition: form-data; name="AuthenticationSettingsId"
3
.....
Expected result
"{\"fields\":[\"RoleId\",\"2\",\"AuthenticationSettingsId\",\"1\",\"recommendation\",\"reviewerId\"],\"size\":100,\"filter\":[{\"id\":\"ApplicationId\",\"operator\":\"and\",\"parent\":\"\",\"nested\":false,\"type\":\"integer\",\"value\":[360]}],\"aggregate\":[],\"sort\":[]}"
Note: Previously we used request.EnableRewind() it was returning the above result and later upgraded to .net core 3.0
Here is a high level of how I handle JSON queries. If you really want to get fancy you can implement all this into an abstract class and inherit direct to your data model.
There are plenty of different ways to get where you want to be, hopefully this helps you get there.
I've put comments in the code, but feel free to ask away if something doesn't make sense.
class SomeHttpJsonUtility
{
//If you want to parse your return data
//directly into a data model
class DataModel{
class ReturnData
{
[JsonPropertyName("fields")]
public Field[] Fields { get; set; }
}
class Field
{
[JsonPropertyName("RoleId")]
public int RoleId { get; set; }
//...you get the idea
}
}
//Some data if your sending a post request
private Dictionary<string, string> postParameters = new Dictionary<string, string>();
//Creates a HTTP Client With Specified Parameters
//You can do this any number of ways depending on the
//source you are querying
private HttpClient GetClient()
{
HttpClient _client = new HttpClient();
_client.DefaultRequestHeaders.Clear();
_client.DefaultRequestHeaders.Add(
"UserAgent",
new string[] { "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:103.0) Gecko/20100101 Firefox/103.0" });
_client.DefaultRequestHeaders.Add(
"AcceptLanguage",
new string[] { "en-US" });
_client.DefaultRequestHeaders.Add(
"AcceptEncoding",
new string[] { "gzip", "deflate", "br" });
_client.DefaultRequestHeaders.Add(
"Accept",
new string[] { "*/*" });
_client.DefaultRequestHeaders.Add(
"Connection",
new string[] { "keep-alive" });
return _client;
}
private void GetJson(Uri from_uri)
{
//Get the HttpClient With Proper Request Headers
HttpClient _client =
GetClient();
Task.Run(async () =>
{
//If your data comes from a get request
HttpResponseMessage _httpResponse =
await _client.GetAsync(
requestUri:from_uri);
//Or if your response comes from a post
_httpResponse =
await _client.PostAsync(
requestUri: from_uri,
content: new FormUrlEncodedContent(postParameters)
);
//since your initial post used a stream, we can
//keep going in that direction
//Initilize a memory stream to process the data
using(MemoryStream _ms = new MemoryStream())
{
//Send the http response content
////into the memory stream
await _httpResponse.Content.CopyToAsync(
stream: _ms);
//Goto the start of the memory stream
_ms.Seek(
offset: 0,
loc: SeekOrigin.Begin);
//Option 1:
//Send direct to data model
// This is utilizing the Microsoft Library:
// System.Text.Json.Serialization;
DataModel dataModel =
JsonSerializer.Deserialize<DataModel>(
utf8Json: _ms);
//Option 2:
//Send to a string
using(StreamReader _sr = new StreamReader(_ms))
{
string dataAsSting = _sr.ReadToEnd();
}
}
}).Wait();
}
}
If your query is only a Get request, then it's pretty easy get get the exact headers you need.
Using Firefox hit F12 and goto the web address.
Click the Network Tab, then Headers and view the request data.
You really only need a few of these:
Accept
Accept-Encoding
Accept-Language
Connection
User-Agent
Mozilla has some nice resources regarding the different header objects.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept
Host should be taken care of by the HttpClient.
Cookies should be handled by the HttpClient (if you need them)
If you are actually getting the data back as Gzip you'll have to implement a reader, unless the HttpClient you are using will automatically decode it.
And at the end, victory :-)
I think you need to set the content-type on the request when you send it to application/json
Could you try to read this way?
var reader = new System.IO.StreamReader(request.Body);
var body = reader.ReadToEndAsync().Result;
Then you can use Newtonsoft or a similar library on body.
You need to to tokenize / encode your string with some JSON encoder.
Here you have two choices:
the internal (Microsoft) JsonConverter
the Newtonsoft.Json JsonConverter
Karthik, it appears you are sending a multipart request from a webkit browser.
If you would be able to just change it on the client side from multipart to application/json your problem would be fixed.
If this is not possible, you can just use:
private async Task<string> FormatRequest(HttpRequest request)
{
var form = request.Form.ToDictionary(x => x.Key, x => x.Value);
return JsonSerializer.Serialize(form);
}
This could parses your form into a dictionary and returns it as a Json.
(This code is written in dotnet 6, which has System.Text.Json. If you need to stick in .net 3.1, you would need to use a JsonSerializer like Newtonsoft.)
I am trying to create a TeamCity project from .NET Core. I have written the code below but when I am trying to run the code I get an error.
Code:
public void CreateProject(string name, string newProjectId, string parentProjectId)
{
InvokeTeamCityApi(() => _apiClient.CreateProject(new NewProjectDescription
{
Name = name,
Id = newProjectId,
SourceProject = new GetProjectResponse
{
Id = _toolContext.HostingContext.TeamCityProjectTemplate
},
ParentProject = new GetProjectResponse
{
Id = parentProjectId
},
CopyAllAssociatedSettings = true,
}));
}
Error:
403 Forbidden: CSRF Header X-TC-CSRF-Token does not match CSRF session value
Also I did a Google search and I tried adding the header origin but I dont have access to disable the internal teamcity properties to disable CSRF check.
So I am passing the token X-tc-CSRF-Token in the request but it says the value doesn't match. How can I solve this issue?
Note: I am getting this issue only when I am using bearer token and with basic auth it works fine.
I recently had the same issue when upgrading from TeamCity 10 to TeamCity 2022 and eventually managed to find out how to fix it. Importantly, I didn't have to have access the internal TeamCity properties to disable the CSRF check as I figured that it's not a bad thing to have the extra layer of security.
Things that were probably important that I changed:
I generated an access token for the account that I was using to access the REST API so I wouldn't need to put my password in the code. The access token can be associated with more restrictive permissions than your own account.
Using HttpClient, authorisation is achieved by adding a default header (via the DefaultHeaders property) with the name "Authorization" and value "Bearer {apiToken}".
I read the /authenticationTest.html?csrf endpoint to get a CSRF token then add that value as a default header ("X-TC-CSRF-Token"). It's also possible to use the "tc-csrf-token" header in the same way.
Note that because the "Authorization" and "X-TC-CSRF-Token"/"tc-csrf-token" headers are added as default headers, there's no need to do this explicitly on every HTTP (GET, POST ,DELETE, etc.) method.
Here's some sample code which covers all of the above.
public class Sample
{
private readonly string _baseUrl;
private readonly HttpClient _httpClient = new HttpClient();
public Sample(string baseUrl, string accessToken)
{
_baseUrl = baseUrl;
_httpClient.DefaultRequestHeaders.Add("Authorization", $"Bearer {accessToken}");
var csrfToken = get("/authenticationTest.html?csrf");
_httpClient.DefaultRequestHeaders.Add("X-TC-CSRF-Token", csrfToken);
}
public void CreateProject(string id, string name)
{
post("/app/rest/projects", $"<newProjectDescription id=\"{id}\" name=\"{name}\" />");
}
private string get(string relativeUrl)
{
var getTask = _httpClient.GetAsync(_baseUrl + relativeUrl);
getTask.Wait();
var response = getTask.Result;
response.EnsureSuccessStatusCode();
var readTask = response.Content.ReadAsStreamAsync();
readTask.Wait();
using ( var stream = readTask.Result )
using ( var reader = new StreamReader(stream) )
return reader.ReadToEnd();
}
private void post(string relativeUrl, string data)
{
using ( var content = new StringContent(data, Encoding.UTF8, "application/xml") )
{
var postTask = _httpClient.PostAsync(_baseUrl + relativeUrl, content);
postTask.Wait();
var response = postTask.Result;
response.EnsureSuccessStatusCode();
var readTask = response.Content.ReadAsStreamAsync();
readTask.Wait();
}
}
}
Call it using, e.g.
// Change these according to your situation
var baseUrl = "http://localhost:8111";
var accessToken = "1234567890";
var sample = new Sample(baseUrl, accessToken);
sample.CreateProject("ProjectID", "ProjectName");
Note that this is just a sample to get you on your feet: I've kept the code short by taking various shortcuts in the code (e.g. the HttpClient should really be one static HttpClient for the application: see Guidelines for using HttpClient on learn.microsoft.com).
I am struggling to send some protobuf binary data to get back some other binary data.
The code is the following:
Client:
HttpClient client = new HttpClient
{
BaseAddress = new Uri("https://localhost:44302/")
};
// "Credentials" is a proto message
Credentials cred = new Credentials()
{
Email = "my#email.com",
Password = "mypassword"
};
var content = new ByteArrayContent(cred.ToByteArray());
var response = await client.PostAsync("api/database", content);
Server:
[HttpPost]
public async Task<IActionResult> Post(ByteArrayContent data)
{
Credentials c = Credentials.Parser.ParseFrom(await data.ReadAsByteArrayAsync());
// Get user from database, etc, etc...
// "user" being another proto defined message
return File(user.ToByteArray(), "application/octet-stream");
}
The thing is it doesn't even get to the server Post method. It fails directly to the client.PostAsync one. I get the Unsupported Media Type error:
I even tried the:
content.Headers.ContentType = MediaTypeHeaderValue.Parse("application/octet-stream");
Doesn't work...
All the answers I find to this issue are either old (This is the method I'm failing to apply) or have some weird Base64 string Json encoded serialization I absolutely want to avoid...
There also are some protobuf-net related answer but I want to avoid any thirdparty package.
Ok, finally got to find the solution. Issue was in the controller. Way to do is:
[HttpPost]
public async Task<IActionResult> LogIn()
{
Credentials cred = Credentials.Parser.ParseFrom(Utils.ReadRequestBody(Request));
// Use cred...
}
with method:
public static byte[] ReadStream(in Stream input)
{
using (MemoryStream ms = new MemoryStream())
{
input.CopyTo(ms);
return ms.ToArray();
}
}
public static byte[] ReadRequestBody(in HttpRequest request)
{
using (Stream stream = request.BodyReader.AsStream()) // Package "System.IO.Pipelines" needed here
{
return ReadStream(stream);
}
}
Due to the fact that I need to convert this C# dll into a tlp file to be called from Visual Basic 6, I need avoid using external dependencies. I have used RestSharp to consume a WebAPI by doing the following (working):
using RestSharp;
using Newtonsoft.Json;
..
public string GetToken (string Key, string Password) {
var client = new RestClient (BaseUrl + "auth/GetToken");
var request = new RestRequest (Method.POST);
request.AddHeader ("cache-control", "no-cache");
request.AddHeader ("Content-Type", "application/json");
Dictionary<string, string> data = new Dictionary<string, string> {
{ "APIKey", Key },
{ "APIPassword", Password }
};
var dataJSON = JsonConvert.SerializeObject (data);
request.AddParameter ("undefined", dataJSON, ParameterType.RequestBody);
IRestResponse response = client.Execute (request);
GetTokenResults g = JsonConvert.DeserializeObject<GetTokenResults> (response.Content);
return g.Token;
}
where GetTokenResults was a struct that contained a declaration for the string Token. I want to achieve this same functionality without using RestSharp. Here is my unsuccessful attempt:
using System.Net.Http;
using System.Net.Http.Headers;
..
public async void GetToken (string Key, string Password) {
var client = new HttpClient ( );
client.DefaultRequestHeaders.Accept.Clear ( );
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue ("application/json"));
client.BaseAddress = new Uri (BaseUrl + "auth/GetToken");
Dictionary<string, string> data = new Dictionary<string, string> {
{ "APIKey", Key },
{ "APIPassword", Password }
};
var dataJSON = JsonConvert.SerializeObject (data);
var content = new StringContent (dataJSON, Encoding.UTF8, "application/json");
var response = await client.PostAsync ("", content);
}
I am unclear on how to achieve the same results (send API key and password, return token as string) using HttpClient as I did using RestSharp earlier. Anything that can point me in the right direction is greatly appreciated!
I think you got stung by this issue. In short, the URI in client.BaseAddress needs a slash at the end of it.
However, I wouldn't simply add it, I'd consider doing it a little different. Presumably your BaseUrl already has a trailing slash, given you're appending "auth/GetToken" to it. I'd do it this way:
client.BaseAddress = new Uri(BaseUrl);
...
var response = await client.PostAsync("auth/GetToken", content);
As you can see, HttpClient fits very cleanly with how your code is already set up, i.e. you have a "base" address with a trailing slash and you want to append to it for a specific call.
That should get you un-stuck to this point. The next thing you'll need to tackle is deserializing the JSON response so you can get the token out of it. It's similar to how you did it in RestSharp, except that response.Content is not a string in the HttpClient world, so you need one more step to get that:
var json = await response.Content.ReadAsStringAsync();
GetTokenResults g = JsonConvert.DeserializeObject<GetTokenResults>(json);
return g.Token;
Last thing you'll need to do to get this to compile is change the method signature to:
public async Task<string> GetTokenAsync
One final note: you are now in the async world, and that's a good thing, but you need to know how to use it correctly or you could end up with deadlocks and other mysterious bugs. In short, don't block on async code by calling .Result or .Wait() anywhere up the call stack. That's by far most common mistake people make. Use async/await all the way down.
I think you are missing first parameter in the method PostAsync i.e. requestUri=Client.BaseAddress (see my implementation below).
Try with this first, if did not work, read below. I have a little different implementation where I passed client.BaseAddress as first parameter and I am passing my content as ByteArrayContent. In my case I have to pass my content as "application/x-www-form-urlencoded" excerpt of my code:
var buffer = Encoding.UTF8.GetBytes(content);
var byteContent = new ByteArrayContent(buffer);
//as I can't send JSON, probably, you can skip as it's already JSON
byteContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/x-www-form-urlencoded");
//requestUri=client.BaseAddress
await client.PostAsync(requestUri, byteContent).ConfigureAwait(false);
We have somewhat different need but I think you are pretty close. If it does not help, write me I will share my code. After reading the comment, I would like to share how I have made my HttpClient. The code is as it is:
using (var client = CreateMailClientForPOST($"{BaseUrl}/"))
{
//removed code, you can call above code as method like
var response= await client.DoThingAsAsync($"{client.BaseAddress}, content").ConfigureAwait(false);
}
protected HttpClient CreateMailClientForPOST(string resource)
{
var handler = new HttpClientHandler();
if (handler.SupportsAutomaticDecompression)
{
handler.AutomaticDecompression = System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate;
}
var client = new HttpClient(handler)
{
BaseAddress = new Uri($"https://api.address.com/rest/{resource}")
};
return client;
}
I am writing a small application that sends data to a server through REST API as simple URL calls. I use the RestSharp library to do this. My problem is, that some data strings I am sending include the / character.
I can't leave the character as it is, since the called URL would then be invalid. But when I replace it with %2F (which is then translated back on the server side), the RestSharp replaces the % character again, giving %252F. The Rest call than fails since server is missing the backslash.
I have searched the web but found no working solution to this problem. Do you have any idea how to solve it, without using different library and rewriting it myself? Also, not using the backslash is NOT an option.
My code is here:
using RestSharp; //Version 104
private string RestRequest(string URL, RestSharp.Method Method)
{
var Client = new RestClient();
Client.Authenticator = new HttpBasicAuthenticator(ID, Password);
var Request = new RestRequest(URL, Method);
IRestResponse Response = Client.Execute(Request);
return Response.Content;
}
Sample URL that is passed to the function:
htp://localhost:8080/api/jsonws/knowledge-base-portlet.kbarticle/add-kb-article/portlet-id/1_WAR_knowledgebaseportlet/parent-resource-class-name-id/20704/parent-resource-prim-key/20200/title/SomeTitle/url-title/%2FTitle/content/SomeContent
After snooping around Resharper's Github issues, like this one, it seems you have to use RestRequest.AddURLSegment(). Tested with RestSharp v. 104.0.0
var url = "http://localhost:4422/api/jsonws/knowledge-base-portlet.kbarticle/add-kb-article/portlet-id/1_WAR_knowledgebaseportlet/parent-resource-class-name-id/20704/parent-resource-prim-key/20200/title/SomeTitle/url-title/{segment}/content/SomeContent";
var Client = new RestClient();
var Request = new RestRequest(url,Method.GET);
Request.AddUrlSegment("segment", "%2Ftitle");
I don't know if it's possible for you to pass multiple arguments. If you cannot, the simplest scenario would be splitting by %2F and concatenate multiple arguemnts. Something like this:
private string RestRequest(string URL, RestSharp.Method Method)
{
var Client = new RestClient();
string requestUrl;
bool hasBackslashArgument = ParseEncodedBackSlash(URL, out requestUrl);
RestRequest request;
if (hasBackslashArgument)
{
request = new RestRequest(requestUrl, Method);
request.AddUrlSegment("segment", "%2F");
}
else
{
request = new RestRequest(URL, Method);
}
IRestResponse response = Client.Execute(request);
return response.Content;
}
private bool ParseEncodedBackSlash(string url, out string preformattedString)
{
preformattedString = null;
var urlSegments = url.Split(new string[] { "%2F" }, StringSplitOptions.None);
if (urlSegments.Length == 0) return false;
preformattedString = string.Join("{segment}", urlSegments);
return true;
}