We Keep Coding

Self hosted windows service (Console App is running fine) on Windows Server 2016 RSTful api is not accessible

My main question is:
What is the difference between self hosted service inside a console app and a windows service?
I wrote a Self-Host Web API Console application which provides a simple api. For that I used basically the MSDN exemple:
public class Startup
{
public void Configuration(IAppBuilder appBuilder)
{
var config = new HttpConfiguration();
config.Routes.MapHttpRoute(
name: "DefaultApi",
routeTemplate: "api/{controller}",
defaults: new { id = RouteParameter.Optional }
);
appBuilder.UseWebApi(config);
}
}
public class HeartbeatController : ApiController
{
public HttpResponseMessage Get()
{
var jsonToSend = GetJsonHeartbeatStatus();
if (!string.IsNullOrEmpty(jsonToSend))
{
var response = this.Request.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(jsonToSend, Encoding.UTF8, "application/json");
return response;
}
throw new HttpResponseException(HttpStatusCode.NotFound);
}
private string GetJsonHeartbeatStatus()
{
var hs = new HeartbeatStatus();
hs.Toolname = "html Scraper";
hs.Heartbeat = DateTime.Now.ToString();
hs.Plant = "plant1";
hs.Department = "departmentA";
hs.Servername = Environment.MachineName;
return JsonConvert.SerializeObject(hs);
}
}
protected override void OnStart(string[] args)
{
base.OnStart(args);
logger.Info("In OnStart");
var UserDomain = System.Net.NetworkInformation.IPGlobalProperties.GetIPGlobalProperties().DomainName;
var Port = "8377";
string baseAddress = string.Format("http://{0}.{1}:{2}", Environment.MachineName, UserDomain, Port);
var completeAddress = baseAddress + "/api/heartbeat";
try
{
host = WebApp.Start<Startup>(baseAddress);
HttpClient client = new HttpClient();
var response = client.GetAsync(completeAddress).Result;
logger.Info("Api address: {0}", completeAddress);
logger.Info("Service Host started #{0}", DateTime.Now.ToString());
}
catch (Exception ex)
{
logger.Info("{0}", ex.ToString());
}
finally
{
if (host != null)
{
host.Dispose();
}
}
}
Next Step was to write a Firewall inbound role to get access to the port.
Then I tested it on our Windows Server 2016 Standard.
-> Test passed in Postman on another machine!
Now as you already saw (in the code) I took the code to a Windows Service Application (user= Local System). I am logging in Windows Event and I have no error in starting the host (that's why I used the try catch).
-> Test with the browser on the server failed and also with Postman on a different machine.
I figured out that I have to reserve the path with netsh
I tried:
netsh http add urlacl url=http://+:8377/api/heartbeat user=everyone
netsh http add urlacl url=http://machineName.corp:8377/api/heartbeat/ user=everyone
Installed the Service application with my user (admin) and set up the urls
netsh http add urlacl url=http://+:8377/api/heartbeat user=myUser
netsh http add urlacl url=http://machineName.corp:8377/api/heartbeat/ user=myUser
-> Test with the browser on the server failed and also with Postman on a different machine.
At this point I could not find any solution why my api is not working.
Does anybody have any ideas?
-edit-
Console appöication:
C:\Windows\system32>netstat -na | find "8378"
TCP 0.0.0.0:8378 0.0.0.0:0 LISTENING
TCP [::]:8378 [::]:0 LISTENING
TCP [fe80::5505:852c:952e:74f%6]:8378 [fe80::5505:852c:952e:74f%6]:62555 ESTABLISHED
TCP [fe80::5505:852c:952e:74f%6]:62555 [fe80::5505:852c:952e:74f%6]:8378 ESTABLISHED
Windows Service application:
C:\Windows\system32>netstat -na | find "8377"
no response!

FiddlerCore with Selenium proxy settings

I use FiddlerCore to capture HTTP traffic when performing web UI tests with Selenium (C#) ChromeDriver. I have finally managed to get it working by trial and error, but need a hint on why my solution works and the logical way (for me) doesn't.
I configure the fiddler proxy as:
CONFIG.IgnoreServerCertErrors = false;
FiddlerApplication.Prefs.SetBoolPref("fiddler.network.streaming.abortifclientaborts", true);
var startupSettings =
new FiddlerCoreStartupSettingsBuilder()
.ListenOnPort(0)
.DecryptSSL()
.OptimizeThreadPool()
.Build();
FiddlerApplication.Startup(startupSettings);
HttpPort = FiddlerApplication.oProxy.ListenPort;
FiddlerApplication.Log.LogFormat("Created HTTP endpoint listening on port {0}", HttpPort);
secureEndpoint = FiddlerApplication.CreateProxyEndpoint(HttpsPort, true, Host);
if (secureEndpoint != null)
{
HttpsPort = secureEndpoint.ListenPort;
FiddlerApplication.Log.LogFormat("Created secure endpoint listening on port {0}, using a HTTPS certificate for '{1}'",HttpsPort, Host);
}
CertMaker.trustRootCert();
This basically follows the demo project provided for FiddlerCore. Note, that the proxy is not registered as system proxy.
Then, ChromeDriver is configured as:
var chromeDriverService = ChromeDriverService.CreateDefaultService();
chromeDriverService.HideCommandPromptWindow = true;
var options = new ChromeOptions();
options.Proxy = new Proxy()
{
HttpProxy = $"{SkicoProxy.Host}:{SkicoProxy.HttpPort}",
SslProxy = $"{SkicoProxy.Host}:{SkicoProxy.HttpPort}"
};
_driver = new ChromeDriver(chromeDriverService, options);
What doesn't work, although I would expect it to is:
options.Proxy = new Proxy()
{
HttpProxy = $"{SkicoProxy.Host}:{SkicoProxy.HttpPort}",
SslProxy = $"{SkicoProxy.Host}:{SkicoProxy.HttpsPort}" // HTTPS PORT INSTEAD HTTP PORT
};
Still, if I do not configure the endpoint in Fiddler, SSL websites are shown as unsafe in Chrome.
Why doesn't SSL traffic needs to be routed through the according endpoint and why does it even work if it is not proxied through the secure endpoint?

How to one proxy server [duplicate]

I have downloaded Privoxy few weeks ago and for the fun I was curious to know how a simple version of it can be done.
I understand that I need to configure the browser (client) to send request to the proxy. The proxy send the request to the web (let say it's a http proxy). The proxy will receive the answer... but how can the proxy send back the request to the browser (client)?
I have search on the web for C# and http proxy but haven't found something that let me understand how it works behind the scene correctly. (I believe I do not want a reverse proxy but I am not sure).
Does any of you have some explication or some information that will let me continue this small project?
Update
This is what I understand (see graphic below).
Step 1 I configure the client (browser) for all request to be send to 127.0.0.1 at the port the Proxy listen. This way, request will be not sent to the Internet directly but will be processed by the proxy.
Step2 The proxy see a new connection, read the HTTP header and see the request he must executes. He executes the request.
Step3 The proxy receive an answer from the request. Now he must send the answer from the web to the client but how???
Useful link
Mentalis Proxy : I have found this project that is a proxy (but more that I would like). I might check the source but I really wanted something basic to understand more the concept.
ASP Proxy : I might be able to get some information over here too.
Request reflector : This is a simple example.
Here is a Git Hub Repository with a Simple Http Proxy.

I wouldn't use HttpListener or something like that, in that way you'll come across so many issues.
Most importantly it'll be a huge pain to support:
Proxy Keep-Alives
SSL won't work (in a correct way, you'll get popups)
.NET libraries strictly follows RFCs which causes some requests to fail (even though IE, FF and any other browser in the world will work.)
What you need to do is:
Listen a TCP port
Parse the browser request
Extract Host connect to that host in TCP level
Forward everything back and forth unless you want to add custom headers etc.
I wrote 2 different HTTP proxies in .NET with different requirements and I can tell you that this is the best way to do it.
Mentalis doing this, but their code is "delegate spaghetti", worse than GoTo :)

I have recently written a light weight proxy in c# .net using TcpListener and TcpClient.
https://github.com/titanium007/Titanium-Web-Proxy
It supports secure HTTP the correct way, client machine needs to trust root certificate used by the proxy. Also supports WebSockets relay. All features of HTTP 1.1 are supported except pipelining. Pipelining is not used by most modern browsers anyway. Also supports windows authentication (plain, digest).
You can hook up your application by referencing the project and then see and modify all traffic. (Request and response).
As far as performance, I have tested it on my machine and works without any noticeable delay.

You can build one with the HttpListener class to listen for incoming requests and the HttpWebRequest class to relay the requests.

Proxy can work in the following way.
Step1, configure client to use proxyHost:proxyPort.
Proxy is a TCP server that is listening on proxyHost:proxyPort.
Browser opens connection with Proxy and sends Http request.
Proxy parses this request and tries to detect "Host" header. This header will tell Proxy where to open connection.
Step 2: Proxy opens connection to the address specified in the "Host" header. Then it sends HTTP request to that remote server. Reads response.
Step 3: After response is read from remote HTTP server, Proxy sends the response through an earlier opened TCP connection with browser.
Schematically it will look like this:
Browser Proxy HTTP server
Open TCP connection
Send HTTP request ----------->
Read HTTP header
detect Host header
Send request to HTTP ----------->
Server
<-----------
Read response and send
<----------- it back to the browser
Render content

If you are just looking to intercept the traffic, you could use the fiddler core to create a proxy...
http://fiddler.wikidot.com/fiddlercore
run fiddler first with the UI to see what it does, it is a proxy that allows you to debug the http/https traffic. It is written in c# and has a core which you can build into your own applications.
Keep in mind FiddlerCore is not free for commercial applications.

Agree to dr evil
if you use HTTPListener you will have many problems, you have to parse requests and will be engaged to headers and ...
Use tcp listener to listen to browser requests
parse only the first line of the request and get the host domain and port to connect
send the exact raw request to the found host on the first line of browser request
receive the data from the target site(I have problem in this section)
send the exact data received from the host to the browser
you see you dont need to even know what is in the browser request and parse it, only get the target site address from the first line
first line usually likes this
GET http://google.com HTTP1.1
or
CONNECT facebook.com:443 (this is for ssl requests)

Things have become really easy with OWIN and WebAPI. In my search for a C# Proxy server, I also came across this post http://blog.kloud.com.au/2013/11/24/do-it-yourself-web-api-proxy/ . This will be the road I'm taking.

Socks4 is a very simple protocol to implement. You listen for the initial connection, connect to the host/port that was requested by the client, send the success code to the client then forward the outgoing and incoming streams across sockets.
If you go with HTTP you'll have to read and possibly set/remove some HTTP headers so that's a little more work.
If I remember correctly, SSL will work across HTTP and Socks proxies. For a HTTP proxy you implement the CONNECT verb, which works much like the socks4 as described above, then the client opens the SSL connection across the proxied tcp stream.

For what it's worth, here is a C# sample async implementation based on HttpListener and HttpClient (I use it to be able to connect Chrome in Android devices to IIS Express, that's the only way I found...).
And If you need HTTPS support, it shouldn't require more code, just certificate configuration: Httplistener with HTTPS support
// define http://localhost:5000 and http://127.0.0.1:5000/ to be proxies for http://localhost:53068
using (var server = new ProxyServer("http://localhost:53068", "http://localhost:5000/", "http://127.0.0.1:5000/"))
{
server.Start();
Console.WriteLine("Press ESC to stop server.");
while (true)
{
var key = Console.ReadKey(true);
if (key.Key == ConsoleKey.Escape)
break;
}
server.Stop();
}
....
public class ProxyServer : IDisposable
{
private readonly HttpListener _listener;
private readonly int _targetPort;
private readonly string _targetHost;
private static readonly HttpClient _client = new HttpClient();
public ProxyServer(string targetUrl, params string[] prefixes)
: this(new Uri(targetUrl), prefixes)
{
}
public ProxyServer(Uri targetUrl, params string[] prefixes)
{
if (targetUrl == null)
throw new ArgumentNullException(nameof(targetUrl));
if (prefixes == null)
throw new ArgumentNullException(nameof(prefixes));
if (prefixes.Length == 0)
throw new ArgumentException(null, nameof(prefixes));
RewriteTargetInText = true;
RewriteHost = true;
RewriteReferer = true;
TargetUrl = targetUrl;
_targetHost = targetUrl.Host;
_targetPort = targetUrl.Port;
Prefixes = prefixes;
_listener = new HttpListener();
foreach (var prefix in prefixes)
{
_listener.Prefixes.Add(prefix);
}
}
public Uri TargetUrl { get; }
public string[] Prefixes { get; }
public bool RewriteTargetInText { get; set; }
public bool RewriteHost { get; set; }
public bool RewriteReferer { get; set; } // this can have performance impact...
public void Start()
{
_listener.Start();
_listener.BeginGetContext(ProcessRequest, null);
}
private async void ProcessRequest(IAsyncResult result)
{
if (!_listener.IsListening)
return;
var ctx = _listener.EndGetContext(result);
_listener.BeginGetContext(ProcessRequest, null);
await ProcessRequest(ctx).ConfigureAwait(false);
}
protected virtual async Task ProcessRequest(HttpListenerContext context)
{
if (context == null)
throw new ArgumentNullException(nameof(context));
var url = TargetUrl.GetComponents(UriComponents.SchemeAndServer, UriFormat.Unescaped);
using (var msg = new HttpRequestMessage(new HttpMethod(context.Request.HttpMethod), url + context.Request.RawUrl))
{
msg.Version = context.Request.ProtocolVersion;
if (context.Request.HasEntityBody)
{
msg.Content = new StreamContent(context.Request.InputStream); // disposed with msg
}
string host = null;
foreach (string headerName in context.Request.Headers)
{
var headerValue = context.Request.Headers[headerName];
if (headerName == "Content-Length" && headerValue == "0") // useless plus don't send if we have no entity body
continue;
bool contentHeader = false;
switch (headerName)
{
// some headers go to content...
case "Allow":
case "Content-Disposition":
case "Content-Encoding":
case "Content-Language":
case "Content-Length":
case "Content-Location":
case "Content-MD5":
case "Content-Range":
case "Content-Type":
case "Expires":
case "Last-Modified":
contentHeader = true;
break;
case "Referer":
if (RewriteReferer && Uri.TryCreate(headerValue, UriKind.Absolute, out var referer)) // if relative, don't handle
{
var builder = new UriBuilder(referer);
builder.Host = TargetUrl.Host;
builder.Port = TargetUrl.Port;
headerValue = builder.ToString();
}
break;
case "Host":
host = headerValue;
if (RewriteHost)
{
headerValue = TargetUrl.Host + ":" + TargetUrl.Port;
}
break;
}
if (contentHeader)
{
msg.Content.Headers.Add(headerName, headerValue);
}
else
{
msg.Headers.Add(headerName, headerValue);
}
}
using (var response = await _client.SendAsync(msg).ConfigureAwait(false))
{
using (var os = context.Response.OutputStream)
{
context.Response.ProtocolVersion = response.Version;
context.Response.StatusCode = (int)response.StatusCode;
context.Response.StatusDescription = response.ReasonPhrase;
foreach (var header in response.Headers)
{
context.Response.Headers.Add(header.Key, string.Join(", ", header.Value));
}
foreach (var header in response.Content.Headers)
{
if (header.Key == "Content-Length") // this will be set automatically at dispose time
continue;
context.Response.Headers.Add(header.Key, string.Join(", ", header.Value));
}
var ct = context.Response.ContentType;
if (RewriteTargetInText && host != null && ct != null &&
(ct.IndexOf("text/html", StringComparison.OrdinalIgnoreCase) >= 0 ||
ct.IndexOf("application/json", StringComparison.OrdinalIgnoreCase) >= 0))
{
using (var ms = new MemoryStream())
{
using (var stream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
{
await stream.CopyToAsync(ms).ConfigureAwait(false);
var enc = context.Response.ContentEncoding ?? Encoding.UTF8;
var html = enc.GetString(ms.ToArray());
if (TryReplace(html, "//" + _targetHost + ":" + _targetPort + "/", "//" + host + "/", out var replaced))
{
var bytes = enc.GetBytes(replaced);
using (var ms2 = new MemoryStream(bytes))
{
ms2.Position = 0;
await ms2.CopyToAsync(context.Response.OutputStream).ConfigureAwait(false);
}
}
else
{
ms.Position = 0;
await ms.CopyToAsync(context.Response.OutputStream).ConfigureAwait(false);
}
}
}
}
else
{
using (var stream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
{
await stream.CopyToAsync(context.Response.OutputStream).ConfigureAwait(false);
}
}
}
}
}
}
public void Stop() => _listener.Stop();
public override string ToString() => string.Join(", ", Prefixes) + " => " + TargetUrl;
public void Dispose() => ((IDisposable)_listener)?.Dispose();
// out-of-the-box replace doesn't tell if something *was* replaced or not
private static bool TryReplace(string input, string oldValue, string newValue, out string result)
{
if (string.IsNullOrEmpty(input) || string.IsNullOrEmpty(oldValue))
{
result = input;
return false;
}
var oldLen = oldValue.Length;
var sb = new StringBuilder(input.Length);
bool changed = false;
var offset = 0;
for (int i = 0; i < input.Length; i++)
{
var c = input[i];
if (offset > 0)
{
if (c == oldValue[offset])
{
offset++;
if (oldLen == offset)
{
changed = true;
sb.Append(newValue);
offset = 0;
}
continue;
}
for (int j = 0; j < offset; j++)
{
sb.Append(input[i - offset + j]);
}
sb.Append(c);
offset = 0;
}
else
{
if (c == oldValue[0])
{
if (oldLen == 1)
{
changed = true;
sb.Append(newValue);
}
else
{
offset = 1;
}
continue;
}
sb.Append(c);
}
}
if (changed)
{
result = sb.ToString();
return true;
}
result = input;
return false;
}
}

The browser is connected to the proxy so the data that the proxy gets from the web server is just sent via the same connection that the browser initiated to the proxy.

How to work with WCF wsHttpBinding and SSL?

I need to develop a WCF Hosted in a console app WebService.
I made it work using the Mutual Certificate (service and client) method using SecurityMode.Message.
But now i need to change the Security Mode to SecurityMode.Transport and use the wsHttpBinding with SSL. I made this code to host the service but i cannot get the wsdl with the browser, or execute some webmethod in the console app client.
static void Main()
{
var httpsUri = new Uri("https://localhost:8089/HelloServer");
var binding = new WSHttpBinding();
binding.Security.Mode = SecurityMode.Transport;
binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Certificate;
var host = new ServiceHost(typeof(WcfFederationServer.HelloWorld), httpsUri);
host.AddServiceEndpoint(typeof(WcfFederationServer.IHelloWorld), binding, "", httpsUri);
var mex = new ServiceMetadataBehavior();
mex.HttpsGetEnabled = true;
host.Description.Behaviors.Add(mex);
// Open the service.
host.Open();
Console.WriteLine("Listening on {0}...", httpsUri);
Console.ReadLine();
// Close the service.
host.Close();
}
The service is up, but i cannot get nothing on the https://localhost:8089/HelloServer.
On fiddler the get request via browser shows me this message:
fiddler.network.https> HTTPS handshake to localhost failed. System.IO.IOException
What im missing here?
Thanks
EDIT:
The Console Application Client Code
static void Main()
{
try
{
var client = new HelloWorldHttps.HelloWorldClient();
client.ClientCredentials.ClientCertificate.SetCertificate(
StoreLocation.LocalMachine,
StoreName.TrustedPeople,
X509FindType.FindBySubjectName,
"www.client.com");
Console.WriteLine(client.GetData());
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
Console.ReadKey();
}
Getting this error:
Could not establish trust relationship for the SSL/TLS secure channel

When it comes to the service, you need to map the certificate to the specific port as described here
http://msdn.microsoft.com/en-us/library/ms733791(v=vs.110).aspx
As for the client, you need to skip the verification of certificate properties like valid date, the domain by relaxing the certificate acceptance policy. An easiest way would be to accept any certiticate
ServicePointManager.ServerCertificateValidationCallback = (a,b,c,d) => true
You can finetune the acceptance callback according to the docs to best fit your needs.

How to create a simple proxy in C#?

I have downloaded Privoxy few weeks ago and for the fun I was curious to know how a simple version of it can be done.
I understand that I need to configure the browser (client) to send request to the proxy. The proxy send the request to the web (let say it's a http proxy). The proxy will receive the answer... but how can the proxy send back the request to the browser (client)?
I have search on the web for C# and http proxy but haven't found something that let me understand how it works behind the scene correctly. (I believe I do not want a reverse proxy but I am not sure).
Does any of you have some explication or some information that will let me continue this small project?
Update
This is what I understand (see graphic below).
Step 1 I configure the client (browser) for all request to be send to 127.0.0.1 at the port the Proxy listen. This way, request will be not sent to the Internet directly but will be processed by the proxy.
Step2 The proxy see a new connection, read the HTTP header and see the request he must executes. He executes the request.
Step3 The proxy receive an answer from the request. Now he must send the answer from the web to the client but how???
Useful link
Mentalis Proxy : I have found this project that is a proxy (but more that I would like). I might check the source but I really wanted something basic to understand more the concept.
ASP Proxy : I might be able to get some information over here too.
Request reflector : This is a simple example.
Here is a Git Hub Repository with a Simple Http Proxy.

I wouldn't use HttpListener or something like that, in that way you'll come across so many issues.
Most importantly it'll be a huge pain to support:
Proxy Keep-Alives
SSL won't work (in a correct way, you'll get popups)
.NET libraries strictly follows RFCs which causes some requests to fail (even though IE, FF and any other browser in the world will work.)
What you need to do is:
Listen a TCP port
Parse the browser request
Extract Host connect to that host in TCP level
Forward everything back and forth unless you want to add custom headers etc.
I wrote 2 different HTTP proxies in .NET with different requirements and I can tell you that this is the best way to do it.
Mentalis doing this, but their code is "delegate spaghetti", worse than GoTo :)

I have recently written a light weight proxy in c# .net using TcpListener and TcpClient.
https://github.com/titanium007/Titanium-Web-Proxy
It supports secure HTTP the correct way, client machine needs to trust root certificate used by the proxy. Also supports WebSockets relay. All features of HTTP 1.1 are supported except pipelining. Pipelining is not used by most modern browsers anyway. Also supports windows authentication (plain, digest).
You can hook up your application by referencing the project and then see and modify all traffic. (Request and response).
As far as performance, I have tested it on my machine and works without any noticeable delay.

You can build one with the HttpListener class to listen for incoming requests and the HttpWebRequest class to relay the requests.

Proxy can work in the following way.
Step1, configure client to use proxyHost:proxyPort.
Proxy is a TCP server that is listening on proxyHost:proxyPort.
Browser opens connection with Proxy and sends Http request.
Proxy parses this request and tries to detect "Host" header. This header will tell Proxy where to open connection.
Step 2: Proxy opens connection to the address specified in the "Host" header. Then it sends HTTP request to that remote server. Reads response.
Step 3: After response is read from remote HTTP server, Proxy sends the response through an earlier opened TCP connection with browser.
Schematically it will look like this:
Browser Proxy HTTP server
Open TCP connection
Send HTTP request ----------->
Read HTTP header
detect Host header
Send request to HTTP ----------->
Server
<-----------
Read response and send
<----------- it back to the browser
Render content

If you are just looking to intercept the traffic, you could use the fiddler core to create a proxy...
http://fiddler.wikidot.com/fiddlercore
run fiddler first with the UI to see what it does, it is a proxy that allows you to debug the http/https traffic. It is written in c# and has a core which you can build into your own applications.
Keep in mind FiddlerCore is not free for commercial applications.

Agree to dr evil
if you use HTTPListener you will have many problems, you have to parse requests and will be engaged to headers and ...
Use tcp listener to listen to browser requests
parse only the first line of the request and get the host domain and port to connect
send the exact raw request to the found host on the first line of browser request
receive the data from the target site(I have problem in this section)
send the exact data received from the host to the browser
you see you dont need to even know what is in the browser request and parse it, only get the target site address from the first line
first line usually likes this
GET http://google.com HTTP1.1
or
CONNECT facebook.com:443 (this is for ssl requests)

Things have become really easy with OWIN and WebAPI. In my search for a C# Proxy server, I also came across this post http://blog.kloud.com.au/2013/11/24/do-it-yourself-web-api-proxy/ . This will be the road I'm taking.

Socks4 is a very simple protocol to implement. You listen for the initial connection, connect to the host/port that was requested by the client, send the success code to the client then forward the outgoing and incoming streams across sockets.
If you go with HTTP you'll have to read and possibly set/remove some HTTP headers so that's a little more work.
If I remember correctly, SSL will work across HTTP and Socks proxies. For a HTTP proxy you implement the CONNECT verb, which works much like the socks4 as described above, then the client opens the SSL connection across the proxied tcp stream.

For what it's worth, here is a C# sample async implementation based on HttpListener and HttpClient (I use it to be able to connect Chrome in Android devices to IIS Express, that's the only way I found...).
And If you need HTTPS support, it shouldn't require more code, just certificate configuration: Httplistener with HTTPS support
// define http://localhost:5000 and http://127.0.0.1:5000/ to be proxies for http://localhost:53068
using (var server = new ProxyServer("http://localhost:53068", "http://localhost:5000/", "http://127.0.0.1:5000/"))
{
server.Start();
Console.WriteLine("Press ESC to stop server.");
while (true)
{
var key = Console.ReadKey(true);
if (key.Key == ConsoleKey.Escape)
break;
}
server.Stop();
}
....
public class ProxyServer : IDisposable
{
private readonly HttpListener _listener;
private readonly int _targetPort;
private readonly string _targetHost;
private static readonly HttpClient _client = new HttpClient();
public ProxyServer(string targetUrl, params string[] prefixes)
: this(new Uri(targetUrl), prefixes)
{
}
public ProxyServer(Uri targetUrl, params string[] prefixes)
{
if (targetUrl == null)
throw new ArgumentNullException(nameof(targetUrl));
if (prefixes == null)
throw new ArgumentNullException(nameof(prefixes));
if (prefixes.Length == 0)
throw new ArgumentException(null, nameof(prefixes));
RewriteTargetInText = true;
RewriteHost = true;
RewriteReferer = true;
TargetUrl = targetUrl;
_targetHost = targetUrl.Host;
_targetPort = targetUrl.Port;
Prefixes = prefixes;
_listener = new HttpListener();
foreach (var prefix in prefixes)
{
_listener.Prefixes.Add(prefix);
}
}
public Uri TargetUrl { get; }
public string[] Prefixes { get; }
public bool RewriteTargetInText { get; set; }
public bool RewriteHost { get; set; }
public bool RewriteReferer { get; set; } // this can have performance impact...
public void Start()
{
_listener.Start();
_listener.BeginGetContext(ProcessRequest, null);
}
private async void ProcessRequest(IAsyncResult result)
{
if (!_listener.IsListening)
return;
var ctx = _listener.EndGetContext(result);
_listener.BeginGetContext(ProcessRequest, null);
await ProcessRequest(ctx).ConfigureAwait(false);
}
protected virtual async Task ProcessRequest(HttpListenerContext context)
{
if (context == null)
throw new ArgumentNullException(nameof(context));
var url = TargetUrl.GetComponents(UriComponents.SchemeAndServer, UriFormat.Unescaped);
using (var msg = new HttpRequestMessage(new HttpMethod(context.Request.HttpMethod), url + context.Request.RawUrl))
{
msg.Version = context.Request.ProtocolVersion;
if (context.Request.HasEntityBody)
{
msg.Content = new StreamContent(context.Request.InputStream); // disposed with msg
}
string host = null;
foreach (string headerName in context.Request.Headers)
{
var headerValue = context.Request.Headers[headerName];
if (headerName == "Content-Length" && headerValue == "0") // useless plus don't send if we have no entity body
continue;
bool contentHeader = false;
switch (headerName)
{
// some headers go to content...
case "Allow":
case "Content-Disposition":
case "Content-Encoding":
case "Content-Language":
case "Content-Length":
case "Content-Location":
case "Content-MD5":
case "Content-Range":
case "Content-Type":
case "Expires":
case "Last-Modified":
contentHeader = true;
break;
case "Referer":
if (RewriteReferer && Uri.TryCreate(headerValue, UriKind.Absolute, out var referer)) // if relative, don't handle
{
var builder = new UriBuilder(referer);
builder.Host = TargetUrl.Host;
builder.Port = TargetUrl.Port;
headerValue = builder.ToString();
}
break;
case "Host":
host = headerValue;
if (RewriteHost)
{
headerValue = TargetUrl.Host + ":" + TargetUrl.Port;
}
break;
}
if (contentHeader)
{
msg.Content.Headers.Add(headerName, headerValue);
}
else
{
msg.Headers.Add(headerName, headerValue);
}
}
using (var response = await _client.SendAsync(msg).ConfigureAwait(false))
{
using (var os = context.Response.OutputStream)
{
context.Response.ProtocolVersion = response.Version;
context.Response.StatusCode = (int)response.StatusCode;
context.Response.StatusDescription = response.ReasonPhrase;
foreach (var header in response.Headers)
{
context.Response.Headers.Add(header.Key, string.Join(", ", header.Value));
}
foreach (var header in response.Content.Headers)
{
if (header.Key == "Content-Length") // this will be set automatically at dispose time
continue;
context.Response.Headers.Add(header.Key, string.Join(", ", header.Value));
}
var ct = context.Response.ContentType;
if (RewriteTargetInText && host != null && ct != null &&
(ct.IndexOf("text/html", StringComparison.OrdinalIgnoreCase) >= 0 ||
ct.IndexOf("application/json", StringComparison.OrdinalIgnoreCase) >= 0))
{
using (var ms = new MemoryStream())
{
using (var stream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
{
await stream.CopyToAsync(ms).ConfigureAwait(false);
var enc = context.Response.ContentEncoding ?? Encoding.UTF8;
var html = enc.GetString(ms.ToArray());
if (TryReplace(html, "//" + _targetHost + ":" + _targetPort + "/", "//" + host + "/", out var replaced))
{
var bytes = enc.GetBytes(replaced);
using (var ms2 = new MemoryStream(bytes))
{
ms2.Position = 0;
await ms2.CopyToAsync(context.Response.OutputStream).ConfigureAwait(false);
}
}
else
{
ms.Position = 0;
await ms.CopyToAsync(context.Response.OutputStream).ConfigureAwait(false);
}
}
}
}
else
{
using (var stream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
{
await stream.CopyToAsync(context.Response.OutputStream).ConfigureAwait(false);
}
}
}
}
}
}
public void Stop() => _listener.Stop();
public override string ToString() => string.Join(", ", Prefixes) + " => " + TargetUrl;
public void Dispose() => ((IDisposable)_listener)?.Dispose();
// out-of-the-box replace doesn't tell if something *was* replaced or not
private static bool TryReplace(string input, string oldValue, string newValue, out string result)
{
if (string.IsNullOrEmpty(input) || string.IsNullOrEmpty(oldValue))
{
result = input;
return false;
}
var oldLen = oldValue.Length;
var sb = new StringBuilder(input.Length);
bool changed = false;
var offset = 0;
for (int i = 0; i < input.Length; i++)
{
var c = input[i];
if (offset > 0)
{
if (c == oldValue[offset])
{
offset++;
if (oldLen == offset)
{
changed = true;
sb.Append(newValue);
offset = 0;
}
continue;
}
for (int j = 0; j < offset; j++)
{
sb.Append(input[i - offset + j]);
}
sb.Append(c);
offset = 0;
}
else
{
if (c == oldValue[0])
{
if (oldLen == 1)
{
changed = true;
sb.Append(newValue);
}
else
{
offset = 1;
}
continue;
}
sb.Append(c);
}
}
if (changed)
{
result = sb.ToString();
return true;
}
result = input;
return false;
}
}

The browser is connected to the proxy so the data that the proxy gets from the web server is just sent via the same connection that the browser initiated to the proxy.