HttpTunnel工作原理及源程序分析

2012年01月13日點擊數(shù): 23703 字體: 大中小一鍵關(guān)注匯訊

這份報告對開源工程HttpTunnel的工作原理及源程序運行流程進行分析，重點分析HttpTunnel對HTTP協(xié)議的封裝與實現(xiàn)。

1 工作原理概述

HttpTunnel通過HTTP請求方式，提供了一個雙向的、虛擬的數(shù)據(jù)通路，可以通過http proxy來使用（不是必需）。

HttpTunnel編譯連接之后得到兩個可執(zhí)行的文件，hts和htc，hts是服務(wù)器端程序，用于要連接的外部主機之上，htc是客戶端程序，用于本地主機之上。這兩個部分連接用于產(chǎn)生一個虛擬的數(shù)據(jù)通道（tunnel）。

Http Tunnel利用HTPP的POST與GET兩個命令建立了兩個連接，分別用于客戶端向服務(wù)器發(fā)送和接收數(shù)據(jù)，而且考慮到了HTTP數(shù)據(jù)的合法性，會隨時檢查所接收或發(fā)出的數(shù)據(jù)是否超過content-length規(guī)定的長度，如果是則填充完后重新開連接進行處理新的數(shù)據(jù)，保證了數(shù)據(jù)始終是合法的HTTP數(shù)據(jù)，完全等價于客戶端在同時用HTTP協(xié)議在上傳和下載一個大文件，文件數(shù)據(jù)中不需要任何的HTTP命令或HTML語言標(biāo)記。

2 HttpTunnel程序流程分析

2.1 服務(wù)器端程序流程分析

hts是服務(wù)器端,安裝在外部網(wǎng)絡(luò)一側(cè)，也就是沒有防火墻的一端。

hts源文件包括hts.c, common.c, tunnel.c, http.c這幾個文件和port目錄下的庫。

流程如下：

main() (hts.c)

parse_arguments (argc, argv, &arg),解析命令行參數(shù);

調(diào)用tunnel = tunnel_new_server (arg.port, arg.content_length)創(chuàng)建新的tunnel的服務(wù)端;

初始化tunnel, in_fd, out_fd都為-1

在tunnel->server_socket監(jiān)聽server_socket (tunnel->dest.host_port, 1), backlog=1, 注意用了SO_REUSEADDR選項; àcommon.c

通道的一些選項:strict_content_length, keep_alive, max_connection_age, 這些在HTTP頭說明中會用到;

寫PID文件

進入無限循環(huán)

如果定義輸入輸出的終端設(shè)備(arg.device),打開設(shè)備fd = open_device (arg.device),common.c;

tunnel_accept接受外部連接 àtunnel.c

accept()連接

解析HTTP協(xié)議請求數(shù)據(jù): http_parse_request (s, &request), http.c

POST或PUT命令, 將此socket設(shè)置為tunnel的in_fd, 用于接收客戶端的數(shù)據(jù);

GET命令, 將此socket設(shè)置為tunnel的out_fd, 用于發(fā)送去往客戶端的數(shù)據(jù);

如果設(shè)置了轉(zhuǎn)發(fā)端口(arg.forward_port != -1), 調(diào)用do_connect連接到這個端口,描述符為fd, 注意這個連接是本機的

自己連自己的內(nèi)部連接, hts起到代理轉(zhuǎn)發(fā)作用, 目標(biāo)端口也就是真正的被HTTP包裹的協(xié)議端口;

進入下一個循環(huán)

poll: fd 和tunnel->server_socket, 也就是網(wǎng)絡(luò)通道數(shù)據(jù)和內(nèi)部連接數(shù)據(jù)對倒;

handle_input()

handle_device_input()處理fd輸入信息; common.c

handle_input()

handle_tunnel_input()處理來自通道的信息; common.c

服務(wù)器端接收了客戶端的兩個連接, POST命令對應(yīng)的連接服務(wù)器接收數(shù)據(jù), GET命令對應(yīng)的連接服務(wù)器發(fā)送數(shù)據(jù)。

2.2 客戶端程序流程分析

htc是客戶端,安裝在防火墻內(nèi)部網(wǎng)絡(luò)一側(cè)，也即客戶瀏覽器一端。

htc源文件包括htc.c, common.c, tunnel.c, http.c, base64.c這幾個文件和port目錄下的庫程序

流程如下：

main.c (htc.c)
parse_arguments (argc, argv, &arg); 解析命令行參數(shù), àhtc.c

如果轉(zhuǎn)發(fā)端口(arg.forward_port != -1),在此端口打開socket監(jiān)聽s = server_socket

(arg.forward_port, 0),

backlog=0, 其實不收連接, 注意用了SO_REUSEADDR選項, àcommon.c
進入無限循環(huán)for (;;);

定義輸入輸出的終端設(shè)備(arg.device),打開設(shè)備fd = open_device (arg.device),common.c;

否則如果定義了轉(zhuǎn)發(fā)端口(arg.forward_port != -1),輸入輸出通過連接的socket來進行fd = wait_for_connection_on_socket (s),就是accept() , à htc.c

打開一個新的通道 tunnel = tunnel_new_client (arg.host_name, arg.host_port,

arg.proxy_name, arg.proxy_port,

arg.content_length); tunnel.c

注意tunnel結(jié)構(gòu)中的in_fd, out_fd都初始化為-1,表示沒有連接, 缺省的content_length是100K字節(jié).

設(shè)置通道的一些選項:strict_content_length, keep_alive, max_connection_age, user_agent這些在HTTP頭說明中會用到, 支持代理;

如果要進行代理認(rèn)證(arg.proxy_authorization != NULL),將認(rèn)證參數(shù)進行base64編碼,作為通道的proxy_authorization參數(shù);

通道連接對方tunnel_connect (tunnel), tunnel.c, 建立http tunnel,主要是調(diào)用函數(shù)tunnel_write_request (tunnel, TUNNEL_OPEN, auth_data, sizeof auth_data)

如果要寫入和已經(jīng)寫入的數(shù)據(jù)長度超過content_length, 對tuenel進行填充;

對于客戶端已經(jīng)連接好的tunnel,超時時進行斷開;

對于斷開(或第一次連接)的客戶端, 調(diào)用tunnel_out_connect (tunnel)發(fā)起連接

調(diào)用do_connect()連接服務(wù)器(第一個連接), socket描述符為tunnel->out_fd.

設(shè)置該socket的一些選項;

調(diào)用shutdown(out_fd, 0), 不接收數(shù)據(jù);

調(diào)用http_post()函數(shù)向服務(wù)器發(fā)送HTTP的POST命令; http.c

調(diào)用tunnel_write_data (tunnel, &request, sizeof request)函數(shù)向服務(wù)器寫要執(zhí)行的請求(此時為TUNNEL_OPEN)

繼續(xù)寫data部分,先寫長度,然后是數(shù)據(jù)(此時為dummy的auth_data=42, length=1),(即HTTP的POST命令向服務(wù)器寫TUNNEL_OPEN命令和一個dummy數(shù)據(jù)),然后進行數(shù)據(jù)長度判斷是否在此連接中數(shù)據(jù)寫多了.

進入函數(shù)tunnel_in_connect (tunnel) , 數(shù)據(jù)進入的連接

調(diào)用do_connect()連接服務(wù)器(第2個連接),socket描述符為tunnel->in_fd.

設(shè)置該socket的一些選項;

調(diào)用http_get()函數(shù)向服務(wù)器發(fā)送HTTP的GET命令; à http.c

調(diào)用shutdown(in_fd, 1), 不再寫數(shù)據(jù);
調(diào)用http_parse_response (tunnel->in_fd, &response)解析HTTP服務(wù)器返回數(shù)據(jù);

處理統(tǒng)計信息

此時tunnel_connect完成

進入下一個循環(huán)

poll選擇用戶的輸入設(shè)備fd和網(wǎng)絡(luò)tunnel的in_fd(第2條連接)數(shù)據(jù),也就是用戶的輸入信息和服務(wù)器返回的信息進行對倒:

fd輸入->tunnel->out_fd輸出; tunnel->in_fd輸入->fd輸出.

handle_input()

handle_device_input()處理用戶輸入信息, àcommon.c

handle_input()

handle_tunnel_input()處理來自通道的信息, àcommon.c

客戶端一共向服務(wù)器發(fā)起了兩個連接, 第一個連接用于發(fā)送數(shù)據(jù),第2個連接用于接收數(shù)據(jù)

3 HttpTunnel源碼相關(guān)分析

3.1 HTTP頭的封裝與實現(xiàn)

3.1.1 HTTP頭結(jié)構(gòu)

在HttpTunnel中，將HTTP頭實現(xiàn)為名值對。是一個遞歸的結(jié)構(gòu)。

typedef struct http_header Http_header;

struct http_header

{

const char *name;

const char *value;

Http_header *next; /* FIXME: this is ugly; need cons cell. */

};

3.1.2 創(chuàng)建HTTP頭

static inline Http_header *

http_alloc_header (const char *name, const char *value)

{

Http_header *header;

header = malloc (sizeof (Http_header));

if (header == NULL)

return NULL;

header->name = header->value = NULL;

header->name = strdup (name);

header->value = strdup (value);

if (name == NULL || value == NULL)

{

if (name == NULL)

free ((char *)name);

if (value == NULL)

free ((char *)value);

free (header);

return NULL;

}

return header;

}

3.1.3 添加HTTP頭

Http_header *

http_add_header (Http_header **header, const char *name, const char *value)

{

Http_header *new_header;

new_header = http_alloc_header (name, value);

if (new_header == NULL)

return NULL;

new_header->next = NULL;

while (*header)

header = &(*header)->next;

*header = new_header;

return new_header;

}

3.1.4 解析HTTP頭

static ssize_t

parse_header (int fd, Http_header **header)

{

unsigned char buf[2];

unsigned char *data;

Http_header *h;

size_t len;

ssize_t n;

*header = NULL;

n = read_all (fd, buf, 2);

if (n <= 0)

return n;

if (buf[0] == ' ' && buf[1] == ' ')

return n;

h = malloc (sizeof (Http_header));

if (h == NULL)

{

log_error ("parse_header: malloc failed");

return -1;

}

*header = h;

h->name = NULL;

h->value = NULL;

n = read_until (fd, ':', &data);

if (n <= 0)

return n;

data = realloc (data, n + 2);

if (data == NULL)

{

log_error ("parse_header: realloc failed");

return -1;

}

memmove (data + 2, data, n);

memcpy (data, buf, 2);

n += 2;

data[n - 1] = 0;

h->name = data;

len = n;

n = read_until (fd, ' ', &data);

if (n <= 0)

return n;

data[n - 1] = 0;

h->value = data;

len += n;

n = read_until (fd, ' ', &data);

if (n <= 0)

return n;

free (data);

if (n != 1)

{

log_error ("parse_header: invalid line ending");

return -1;

}

len += n;

log_verbose ("parse_header: %s:%s", h->name, h->value);

n = parse_header (fd, &h->next);

if (n <= 0)

return n;

len += n;

return len;

}

3.1.5 寫HTTP頭

static ssize_t

http_write_header (int fd, Http_header *header)

{

ssize_t n = 0, m;

if (header == NULL)

return write_all (fd, " ", 2);

m = write_all (fd, (void *)header->name, strlen (header->name));

if (m == -1)

{

return -1;

}

n += m;

m = write_all (fd, ": ", 2);

if (m == -1)

{

return -1;

}

n += m;

m = write_all (fd, (void *)header->value, strlen (header->value));

if (m == -1)

{

return -1;

}

n += m;

m = write_all (fd, " ", 2);

if (m == -1)

{

return -1;

}

n += m;

m = http_write_header (fd, header->next);

if (m == -1)

{

return -1;

}

n += m;

return n;

}

3.1.6 查找HTTP頭

static Http_header *

http_header_find (Http_header *header, const char *name)

{

if (header == NULL)

return NULL;

if (strcmp (header->name, name) == 0)

return header;

return http_header_find (header->next, name);

}

3.1.7 獲取HTTP頭

const char *

http_header_get (Http_header *header, const char *name)

{

Http_header *h;

h = http_header_find (header, name);

if (h == NULL)

return NULL;

return h->value;

}

3.1.8 銷毀HTTP頭

static void

http_destroy_header (Http_header *header)

{

if (header == NULL)

return;

http_destroy_header (header->next);

if (header->name)

free ((char *)header->name);

if (header->value)

free ((char *)header->value);

free (header);

}

3.2 HTTP方法的封裝與實現(xiàn)

3.2.1 HTTP方法簡介

OPTIONS :

GET :

HEAD :

POST :

PUT :

DELETE:

TRACE :

CONNECT:

3.2.2 HTTP方法枚舉

在程序中，定義以下的枚舉類型。

typedef enum

{

HTTP_GET,

HTTP_PUT,

HTTP_POST,

HTTP_OPTIONS,

HTTP_HEAD,

HTTP_DELETE,

HTTP_TRACE

} Http_method;

3.2.3 HTTP方法的通用實現(xiàn)

在HttpTunnel里面，首先定義了一個通用的實現(xiàn)HTTP方法的函數(shù)，即。

static inline ssize_t

http_method (int fd, Http_destination *dest,

Http_method method, ssize_t length)

{

char str[1024]; /* FIXME: possible buffer overflow */

Http_request *request;

ssize_t n;

if (fd == -1)

{

log_error ("http_method: fd == -1");

return -1;

}

n = 0;

if (dest->proxy_name != NULL)

n = sprintf (str, "http://%s:%d", dest->host_name, dest->host_port);

sprintf (str + n, "/index.html?crap=%ld", time (NULL));

request = http_create_request (method, str, 1, 1);

if (request == NULL)

return -1;

sprintf (str, "%s:%d", dest->host_name, dest->host_port);

http_add_header (&request->header, "Host", str);

if (length >= 0)

{

sprintf (str, "%d", length);

http_add_header (&request->header, "Content-Length", str);

}

http_add_header (&request->header, "Connection", "close");

if (dest->proxy_authorization)

{

http_add_header (&request->header,

"Proxy-Authorization",

dest->proxy_authorization);

}

if (dest->user_agent)

{

http_add_header (&request->header,

"User-Agent",

dest->user_agent);

}

n = http_write_request (fd, request);

http_destroy_request (request);

return n;

}

該函數(shù)工作流程分析如下：

3.2.4 GET方法的實現(xiàn)

ssize_t

http_get (int fd, Http_destination *dest)

{

return http_method (fd, dest, HTTP_GET, -1);

}

3.2.5 POST方法的實現(xiàn)

http_post (int fd, Http_destination *dest, size_t length)

{

return http_method (fd, dest, HTTP_POST, (ssize_t)length);

}

3.3 HTTP請求的封裝與實現(xiàn)

3.3.1 HTTP請求結(jié)構(gòu)

typedef struct

{

Http_method method;

const char *uri;

int major_version;

int minor_version;

Http_header *header;

} Http_request;

3.3.2 為HTTP請求分配內(nèi)存空間

static inline Http_request *

http_allocate_request (const char *uri)

{

Http_request *request;

request = malloc (sizeof (Http_request));

if (request == NULL)

return NULL;

request->uri = strdup (uri);

if (request->uri == NULL)

{

free (request);

return NULL;

}

return request;

}

3.3.3 創(chuàng)建HTTP請求

Http_request *

http_create_request (Http_method method,

const char *uri,

int major_version,

int minor_version)

{

Http_request *request;

request = http_allocate_request (uri);

if (request == NULL)

return NULL;

request->method = method;

request->major_version = major_version;

request->minor_version = minor_version;

request->header = NULL;

return request;

}

3.3.4 解析HTTP請求

ssize_t

http_parse_request (int fd, Http_request **request_)

{

Http_request *request;

unsigned char *data;

size_t len;

ssize_t n;

*request_ = NULL;

request = malloc (sizeof (Http_request));

if (request == NULL)

{

log_error ("http_parse_request: out of memory");

return -1;

}

request->method = -1;

request->uri = NULL;

request->major_version = -1;

request->minor_version = -1;

request->header = NULL;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

free (request);

return n;

}

request->method = http_string_to_method (data, n - 1);

if (request->method == -1)

{

log_error ("http_parse_request: expected an HTTP method");

free (data);

free (request);

return -1;

}

data[n - 1] = 0;

log_verbose ("http_parse_request: method = "%s"", data);

free (data);

len = n;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

free (request);

return n;

}

data[n - 1] = 0;

request->uri = data;

len += n;

log_verbose ("http_parse_request: uri = "%s"", request->uri);

n = read_until (fd, '/', &data);

if (n <= 0)

{

http_destroy_request (request);

return n;

}

else if (n != 5 || memcmp (data, "HTTP", 4) != 0)

{

log_error ("http_parse_request: expected "HTTP"");

free (data);

http_destroy_request (request);

return -1;

}

free (data);

len = n;

n = read_until (fd, '.', &data);

if (n <= 0)

{

http_destroy_request (request);

return n;

}

data[n - 1] = 0;

request->major_version = atoi (data);

log_verbose ("http_parse_request: major version = %d",

request->major_version);

free (data);

len += n;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

http_destroy_request (request);

return n;

}

data[n - 1] = 0;

request->minor_version = atoi (data);

log_verbose ("http_parse_request: minor version = %d",

request->minor_version);

free (data);

len += n;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

http_destroy_request (request);

return n;

}

free (data);

if (n != 1)

{

log_error ("http_parse_request: invalid line ending");

http_destroy_request (request);

return -1;

}

len += n;

n = parse_header (fd, &request->header);

if (n <= 0)

{

http_destroy_request (request);

return n;

}

len += n;

*request_ = request;

return len;

}

3.3.5 銷毀HTTP請求

void

http_destroy_request (Http_request *request)

{

if (request->uri)

free ((char *)request->uri);

http_destroy_header (request->header);

free (request);

}

3.4 HTTP響應(yīng)的封裝與實現(xiàn)

3.4.1 HTTP響應(yīng)結(jié)構(gòu)

typedef struct

{

int major_version;

int minor_version;

int status_code;

const char *status_message;

Http_header *header;

} Http_response;

3.4.2 為HTTP響應(yīng)分配內(nèi)存空間

static inline Http_response *

http_allocate_response (const char *status_message)

{

Http_response *response;

response = malloc (sizeof (Http_response));

if (response == NULL)

return NULL;

response->status_message = strdup (status_message);

if (response->status_message == NULL)

{

free (response);

return NULL;

}

return response;

}

3.4.3 創(chuàng)建HTTP響應(yīng)

Http_response *

http_create_response (int major_version,

int minor_version,

int status_code,

const char *status_message)

{

Http_response *response;

response = http_allocate_response (status_message);

if (response == NULL)

return NULL;

response->major_version = major_version;

response->minor_version = minor_version;

response->status_code = status_code;

response->header = NULL;

return response;

}

3.4.4 解析HTTP響應(yīng)

ssize_t

http_parse_response (int fd, Http_response **response_)

{

Http_response *response;

unsigned char *data;

size_t len;

ssize_t n;

*response_ = NULL;

response = malloc (sizeof (Http_response));

if (response == NULL)

{

log_error ("http_parse_response: out of memory");

return -1;

}

response->major_version = -1;

response->minor_version = -1;

response->status_code = -1;

response->status_message = NULL;

response->header = NULL;

n = read_until (fd, '/', &data);

if (n <= 0)

{

free (response);

return n;

}

else if (n != 5 || memcmp (data, "HTTP", 4) != 0)

{

log_error ("http_parse_response: expected "HTTP"");

free (data);

free (response);

return -1;

}

free (data);

len = n;

n = read_until (fd, '.', &data);

if (n <= 0)

{

free (response);

return n;

}

data[n - 1] = 0;

response->major_version = atoi (data);

log_verbose ("http_parse_response: major version = %d",

response->major_version);

free (data);

len += n;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

free (response);

return n;

}

data[n - 1] = 0;

response->minor_version = atoi (data);

log_verbose ("http_parse_response: minor version = %d",

response->minor_version);

free (data);

len += n;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

free (response);

return n;

}

data[n - 1] = 0;

response->status_code = atoi (data);

log_verbose ("http_parse_response: status code = %d",

response->status_code);

free (data);

len += n;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

free (response);

return n;

}

data[n - 1] = 0;

response->status_message = data;

log_verbose ("http_parse_response: status message = "%s"",

response->status_message);

len += n;

n = read_until (fd, ' ', &data);

if (n <= 0)

{

http_destroy_response (response);

return n;

}

free (data);

if (n != 1)

{

log_error ("http_parse_request: invalid line ending");

http_destroy_response (response);

return -1;

}

len += n;

n = parse_header (fd, &response->header);

if (n <= 0)

{

http_destroy_response (response);

return n;

}

len += n;

*response_ = response;

return len;

}

3.5 HTTP目標(biāo)的封裝

typedef struct

{

const char *host_name;

int host_port;

const char *proxy_name;

int proxy_port;

const char *proxy_authorization;

const char *user_agent;

} Http_destination;

還有一些其他相關(guān)函數(shù)可以參考源程序。

上一篇:http tunnel 原理及穿透防火墻方法

下一篇:Kad的出現(xiàn) 結(jié)束了edonkey時代

汇讯WiseUC--连接人与业务，以即时通讯为基础的应用集成平台

HttpTunnel工作原理及源程序分析

相關(guān)文章：

匯訊WiseUC

產(chǎn)品應(yīng)用

合作

友情導(dǎo)航

匯訊在線客服