Hello folks! This is the last part of our 3 part series. If you haven't read the previous parts make sure you do so first before diving in.

Last time we generated unique URLs for different clients using our CLI. This part focuses only on the handleProxyRequest function mentioned in the previous part. The part where a user HTTP requests the generated URL. How will we forward the request? The diagram below explains what's going to be happening.

The user's request goes to our Go HTTP server, handleProxyRequest gets executed which briefly passes the request over a raw TCP connection. The request then reaches the CLI running on the user that wants to expose his local website. The CLI takes this request forwards it and requests from the local server running on his machine. Then the response gets passed back the same way.

This is how handleProxyRequest works:

func handleProxyRequest(w http.ResponseWriter, r *http.Request, mapping *mapping) {
    // STEP 1 PARSING THE INCOMING REQUEST
************************************************************
    host := r.Host
    // split the domain and get the first part
    // this is the subdomain
    uuid := strings.Split(host, ".")[0]
    // get the connection from the mapping
    conn := mapping.get(uuid)
    if conn == nil {
        // if the connection is nil, return a 404
        w.WriteHeader(404)
        return
    }
    // read the body into byte array
    buf := make([]byte, 4096)
    r.Body.Read(buf)
************************************************************
    // STEP 2 PASSING THE REQUEST OVER A RAW TCP CONNECTION
    req := []byte(r.Method + " " + r.URL.String() + " " + r.Proto + "\n\n")
    for k, v := range r.Header {
        req = append(req, []byte(k+": "+strings.Join(v, ",")+"\n")...)
    }
    req = append(req, []byte("\n")...)
    _ = append(req, buf...)
    // write the request to the server
    conn.Write(response)


************************************************************
    // STEP 3 READING THE RESPONSE FROM THE TCP CONNECTION

    // read the response from the server
    buf = make([]byte, 4096)
    n, err := conn.Read(buf)
    if err != nil {
        w.WriteHeader(500)
        return
    }
    // parse the response as its an http request
    response = buf[:n]
    // check if this response is a http response or a raw response
    if strings.Split(string(response), "\n\n")[0] == "HTTP/1.1 200 OK" {
        // split the response into headers and body
        splitResponse := strings.Split(string(response), "\n\n")
        // // split the headers into an array
        headers := strings.Split(splitResponse[1], "\n")
        // // get the status line
        statusLine := splitResponse[0]
        // // get the status code
        statusCode, _ := strconv.Atoi(strings.Split(statusLine, " ")[1])
************************************************************
     // STEP 4 BUILD AND WRITE THE RESPONSE BACK TO THE CLIENT

        // write the status code
        w.WriteHeader(statusCode)
        // write the headers
        for _, header := range headers {
            if header == "" {
                continue
            }
            splitHeader := strings.Split(header, ": ")
            w.Header().Add(splitHeader[0], splitHeader[1])
        }
        w.Write([]byte(strings.Join(splitResponse[2:], "\n\n")))
    } else {
        // this is a raw response
        w.Write(response)
    }
}

I split the method into 4 parts. Of course, this is by no means production code the function is too big and does several things. But I'll explain each step (steps are commented on in the code above)

1. Parsing the incoming request; we parse the client's request and get the headers, body and method. This is because we're going to be writing these over the raw TCP connection we created with the CLI.

2. Passing the request over to the CLI; Since it's a raw TCP connection we're going to bare-bone send this request. We first send the method, URL and scheme over and we separate them with a \n\n (you can use any separator it's your code really) as long as it's mutual between both the client and server it should be ok. We separate the method and URL part from the headers and we separate the headers from the body.

3. Reading the response from the TCP connection; We receive a response from our CLI and using the separators we defined \n\n we start parsing that response.

4. We then build from the previous step an HTTP response to send back to the client.

This is what handleProxyRequest does. It simply forwards the incoming request to our CLI and returns the response. Now let's take a look at what happens when our CLI gets the client's request from the raw TCP connection. (step 2 above but from the CLI's POV)

// PART 1 BUILDING THE REQUEST AND PASSING IT TO THE LOCAL SERVER

let request_path = "http://".to_string() + LOCAL_ADDRESS.deref() + ":" + local_port.deref() + request.path.unwrap();
// The port was defined earlier when we executed the cli command http -p 3000 for example.
let request_headers = request.headers;
let request_method = Method::from_bytes(request.method.unwrap().as_bytes()).unwrap();

let mut request_builder = client.request(request_method, request_path);
for header in request_headers {
let header_name = header.name;
let header_value = header.value;
let header_value = HeaderValue::from_bytes(header_value).unwrap();
request_builder = request_builder.header(header_name, header_value);
}

let response = request_builder.send().await;

We use the package reqwest. The reqwest crate provides a convenient, higher-level HTTP Client. We're going to use it to send a request to our local server.

What we do here is simply just parsing the incoming request, building it and forwarding it to our local server. request_builder.send().await; takes care of firing the request.

Now next part takes the response from our local server and returns it over the raw TCP connection

    // STEP 2 PARSING THE LOCAL RESPONSE AND RETURNING IT BACK OVER TCP CONN
    match response {
        Ok(r) => {
            let response_headers = r.headers();
            let response_status = r.status();

            let mut response = String::new();
            response.push_str("HTTP/1.1 ");
            response.push_str(&response_status.to_string());

            response.push_str("\n\n");

            for header in response_headers {
                let header_name = header.0;
                let header_value = header.1;
                response.push_str(header_name.as_str());
                response.push_str(": ");
                response.push_str(header_value.to_str().unwrap());
                response.push_str("\n");
                println!("{}: {}", header_name.as_str(), header_value.to_str().unwrap());
            }
            response.push_str("\n");
            let body = r.text().await.unwrap();
            response.push_str(&body);


            return Ok(response);
        }
        Err(e) => {
            println!("Error: {}", e);
            return Err(e);
        }
    }
}

We do the same thing we did when sending the request in handleProxyRequest we send an array of bytes with a separator and send over the wire.

NOTE that this code doesn't cover every single case and it's very basic it just aims to show the basic idea.

Summary

All in all after 3 parts and me playing around with this. It's a very very deep and interesting concept. Not needing to configure any Router or Network settings and allowing remote users to access a local application has lots of layers to it and the layers are even more when it comes to deploying this to be production ready. I hope my simple approach gave a high level of a % of what's happening. That's it for this series of articles till the next one!