HttpTunnel工作原理及源程序分析
2012年01月13日 點(diǎn)擊數(shù): 22692 字體: 大 中 小
這份報(bào)告對開源工程HttpTunnel的工作原理及源程序運(yùn)行流程進(jìn)行分析，重點(diǎn)分析HttpTunnel對HTTP協(xié)議的封裝與實(shí)現(xiàn)。
1 工作原理概述
HttpTunnel通過HTTP請求方式，提供了一個(gè)雙向的、虛擬的數(shù)據(jù)通路，可以通過http proxy來使用（不是必需）。
HttpTunnel編譯連接之后得到兩個(gè)可執(zhí)行的文件，hts和htc，hts是服務(wù)器端程序，用于要連接的外部主機(jī)之上，htc是客戶端程序，用于本地主機(jī)之上。這兩個(gè)部分連接用于產(chǎn)生一個(gè)虛擬的數(shù)據(jù)通道（tunnel）。
Http Tunnel利用HTPP的POST與GET兩個(gè)命令建立了兩個(gè)連接，分別用于客戶端向服務(wù)器發(fā)送和接收數(shù)據(jù)，而且考慮到了HTTP數(shù)據(jù)的合法性，會(huì)隨時(shí)檢查所接收或發(fā)出的數(shù)據(jù)是否超過content-length規(guī)定的長度，如果是則填充完后重新開連接進(jìn)行處理新的數(shù)據(jù)，保證了數(shù)據(jù)始終是合法的HTTP數(shù)據(jù)，完全等價(jià)于客戶端在同時(shí)用HTTP協(xié)議在上傳和下載一個(gè)大文件，文件數(shù)據(jù)中不需要任何的HTTP命令或HTML語言標(biāo)記。
2 HttpTunnel程序流程分析
2.1 服務(wù)器端程序流程分析
hts是服務(wù)器端,安裝在外部網(wǎng)絡(luò)一側(cè)，也就是沒有防火墻的一端。
hts源文件包括hts.c, common.c, tunnel.c, http.c這幾個(gè)文件和port目錄下的庫。
流程如下：
main() (hts.c)
parse_arguments (argc, argv, &arg),解析命令行參數(shù);
調(diào)用tunnel = tunnel_new_server (arg.port, arg.content_length)創(chuàng)建新的tunnel的服務(wù)端;
初始化tunnel, in_fd, out_fd都為-1
在tunnel->server_socket監(jiān)聽server_socket (tunnel->dest.host_port, 1), backlog=1, 注意用了SO_REUSEADDR選項(xiàng); àcommon.c
通道的一些選項(xiàng):strict_content_length, keep_alive, max_connection_age, 這些在HTTP頭說明中會(huì)用到;
寫PID文件
進(jìn)入無限循環(huán)
如果定義輸入輸出的終端設(shè)備(arg.device),打開設(shè)備fd = open_device (arg.device),common.c;
tunnel_accept接受外部連接 àtunnel.c
accept()連接
解析HTTP協(xié)議請求數(shù)據(jù): http_parse_request (s, &request), http.c
POST或PUT命令, 將此socket設(shè)置為tunnel的in_fd, 用于接收客戶端的數(shù)據(jù);
GET命令, 將此socket設(shè)置為tunnel的out_fd, 用于發(fā)送去往客戶端的數(shù)據(jù);
如果設(shè)置了轉(zhuǎn)發(fā)端口(arg.forward_port != -1), 調(diào)用do_connect連接到這個(gè)端口,描述符為fd, 注意這個(gè)連接是本機(jī)的
自己連自己的內(nèi)部連接, hts起到代理轉(zhuǎn)發(fā)作用, 目標(biāo)端口也就是真正的被HTTP包裹的協(xié)議端口;
進(jìn)入下一個(gè)循環(huán)
poll: fd 和tunnel->server_socket, 也就是網(wǎng)絡(luò)通道數(shù)據(jù)和內(nèi)部連接數(shù)據(jù)對倒;
handle_input()
handle_device_input()處理fd輸入信息; common.c
handle_input()
handle_tunnel_input()處理來自通道的信息; common.c
服務(wù)器端接收了客戶端的兩個(gè)連接, POST命令對應(yīng)的連接服務(wù)器接收數(shù)據(jù), GET命令對應(yīng)的連接服務(wù)器發(fā)送數(shù)據(jù)。
2.2 客戶端程序流程分析
htc是客戶端,安裝在防火墻內(nèi)部網(wǎng)絡(luò)一側(cè)，也即客戶瀏覽器一端。
htc源文件包括htc.c, common.c, tunnel.c, http.c, base64.c這幾個(gè)文件和port目錄下的庫程序
流程如下：
main.c (htc.c)
parse_arguments (argc, argv, &arg); 解析命令行參數(shù), àhtc.c
如果轉(zhuǎn)發(fā)端口(arg.forward_port != -1),在此端口打開socket監(jiān)聽s = server_socket
(arg.forward_port, 0),
backlog=0, 其實(shí)不收連接, 注意用了SO_REUSEADDR選項(xiàng), àcommon.c
進(jìn)入無限循環(huán)for (;;);
定義輸入輸出的終端設(shè)備(arg.device),打開設(shè)備fd = open_device (arg.device),common.c;
否則如果定義了轉(zhuǎn)發(fā)端口(arg.forward_port != -1),輸入輸出通過連接的socket來進(jìn)行fd = wait_for_connection_on_socket (s),就是accept() , à htc.c
打開一個(gè)新的通道 tunnel = tunnel_new_client (arg.host_name, arg.host_port,
arg.proxy_name, arg.proxy_port,
arg.content_length); tunnel.c
注意tunnel結(jié)構(gòu)中的in_fd, out_fd都初始化為-1,表示沒有連接, 缺省的content_length是100K字節(jié).
設(shè)置通道的一些選項(xiàng):strict_content_length, keep_alive, max_connection_age, user_agent這些在HTTP頭說明中會(huì)用到, 支持代理;
如果要進(jìn)行代理認(rèn)證(arg.proxy_authorization != NULL),將認(rèn)證參數(shù)進(jìn)行base64編碼,作為通道的proxy_authorization參數(shù);
通道連接對方tunnel_connect (tunnel), tunnel.c, 建立http tunnel,主要是調(diào)用函數(shù)tunnel_write_request (tunnel, TUNNEL_OPEN, auth_data, sizeof auth_data)
如果要寫入和已經(jīng)寫入的數(shù)據(jù)長度超過content_length, 對tuenel進(jìn)行填充;
對于客戶端已經(jīng)連接好的tunnel,超時(shí)時(shí)進(jìn)行斷開;
對于斷開(或第一次連接)的客戶端, 調(diào)用tunnel_out_connect (tunnel)發(fā)起連接
調(diào)用do_connect()連接服務(wù)器(第一個(gè)連接), socket描述符為tunnel->out_fd.
設(shè)置該socket的一些選項(xiàng);
調(diào)用shutdown(out_fd, 0), 不接收數(shù)據(jù);
調(diào)用http_post()函數(shù)向服務(wù)器發(fā)送HTTP的POST命令; http.c
調(diào)用tunnel_write_data (tunnel, &request, sizeof request)函數(shù)向服務(wù)器寫要執(zhí)行的請求(此時(shí)為TUNNEL_OPEN)
繼續(xù)寫data部分,先寫長度,然后是數(shù)據(jù)(此時(shí)為dummy的auth_data=42, length=1),(即HTTP的POST命令向服務(wù)器寫TUNNEL_OPEN命令和一個(gè)dummy數(shù)據(jù)),然后進(jìn)行數(shù)據(jù)長度判斷是否在此連接中數(shù)據(jù)寫多了.
進(jìn)入函數(shù)tunnel_in_connect (tunnel) , 數(shù)據(jù)進(jìn)入的連接
調(diào)用do_connect()連接服務(wù)器(第2個(gè)連接),socket描述符為tunnel->in_fd.
設(shè)置該socket的一些選項(xiàng);
調(diào)用http_get()函數(shù)向服務(wù)器發(fā)送HTTP的GET命令; à http.c
調(diào)用shutdown(in_fd, 1), 不再寫數(shù)據(jù);
調(diào)用http_parse_response (tunnel->in_fd, &response)解析HTTP服務(wù)器返回?cái)?shù)據(jù);
處理統(tǒng)計(jì)信息
此時(shí)tunnel_connect完成
進(jìn)入下一個(gè)循環(huán)
poll選擇用戶的輸入設(shè)備fd和網(wǎng)絡(luò)tunnel的in_fd(第2條連接)數(shù)據(jù),也就是用戶的輸入信息和服務(wù)器返回的信息進(jìn)行對倒:
fd輸入->tunnel->out_fd輸出; tunnel->in_fd輸入->fd輸出.
handle_input()
handle_device_input()處理用戶輸入信息, àcommon.c
handle_input()
handle_tunnel_input()處理來自通道的信息, àcommon.c
客戶端一共向服務(wù)器發(fā)起了兩個(gè)連接, 第一個(gè)連接用于發(fā)送數(shù)據(jù),第2個(gè)連接用于接收數(shù)據(jù)
3 HttpTunnel源碼相關(guān)分析
3.1 HTTP頭的封裝與實(shí)現(xiàn)
3.1.1 HTTP頭結(jié)構(gòu)
在HttpTunnel中，將HTTP頭實(shí)現(xiàn)為名值對。是一個(gè)遞歸的結(jié)構(gòu)。
typedef struct http_header Http_header;
struct http_header
{
const char *name;
const char *value;
Http_header *next; /* FIXME: this is ugly; need cons cell. */
};
3.1.2 創(chuàng)建HTTP頭
static inline Http_header *
http_alloc_header (const char *name, const char *value)
{
Http_header *header;
header = malloc (sizeof (Http_header));
if (header == NULL)
return NULL;
header->name = header->value = NULL;
header->name = strdup (name);
header->value = strdup (value);
if (name == NULL || value == NULL)
{
if (name == NULL)
free ((char *)name);
if (value == NULL)
free ((char *)value);
free (header);
return NULL;
}
return header;
}
3.1.3 添加HTTP頭
Http_header *
http_add_header (Http_header **header, const char *name, const char *value)
{
Http_header *new_header;
new_header = http_alloc_header (name, value);
if (new_header == NULL)
return NULL;
new_header->next = NULL;
while (*header)
header = &(*header)->next;
*header = new_header;
return new_header;
}
3.1.4 解析HTTP頭
static ssize_t
parse_header (int fd, Http_header **header)
{
unsigned char buf[2];
unsigned char *data;
Http_header *h;
size_t len;
ssize_t n;
*header = NULL;
n = read_all (fd, buf, 2);
if (n <= 0)
return n;
if (buf[0] == ' ' && buf[1] == ' ')
return n;
h = malloc (sizeof (Http_header));
if (h == NULL)
{
log_error ("parse_header: malloc failed");
return -1;
}
*header = h;
h->name = NULL;
h->value = NULL;
n = read_until (fd, ':', &data);
if (n <= 0)
return n;
data = realloc (data, n + 2);
if (data == NULL)
{
log_error ("parse_header: realloc failed");
return -1;
}
memmove (data + 2, data, n);
memcpy (data, buf, 2);
n += 2;
data[n - 1] = 0;
h->name = data;
len = n;
n = read_until (fd, ' ', &data);
if (n <= 0)
return n;
data[n - 1] = 0;
h->value = data;
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
return n;
free (data);
if (n != 1)
{
log_error ("parse_header: invalid line ending");
return -1;
}
len += n;
log_verbose ("parse_header: %s:%s", h->name, h->value);
n = parse_header (fd, &h->next);
if (n <= 0)
return n;
len += n;
return len;
}
3.1.5 寫HTTP頭
static ssize_t
http_write_header (int fd, Http_header *header)
{
ssize_t n = 0, m;
if (header == NULL)
return write_all (fd, " ", 2);
m = write_all (fd, (void *)header->name, strlen (header->name));
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, ": ", 2);
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, (void *)header->value, strlen (header->value));
if (m == -1)
{
return -1;
}
n += m;
m = write_all (fd, " ", 2);
if (m == -1)
{
return -1;
}
n += m;
m = http_write_header (fd, header->next);
if (m == -1)
{
return -1;
}
n += m;
return n;
}
3.1.6 查找HTTP頭
static Http_header *
http_header_find (Http_header *header, const char *name)
{
if (header == NULL)
return NULL;
if (strcmp (header->name, name) == 0)
return header;
return http_header_find (header->next, name);
}
3.1.7 獲取HTTP頭
const char *
http_header_get (Http_header *header, const char *name)
{
Http_header *h;
h = http_header_find (header, name);
if (h == NULL)
return NULL;
return h->value;
}
3.1.8 銷毀HTTP頭
static void
http_destroy_header (Http_header *header)
{
if (header == NULL)
return;
http_destroy_header (header->next);
if (header->name)
free ((char *)header->name);
if (header->value)
free ((char *)header->value);
free (header);
}
3.2 HTTP方法的封裝與實(shí)現(xiàn)
3.2.1 HTTP方法簡介
OPTIONS :
GET :
HEAD :
POST :
PUT :
DELETE:
TRACE :
CONNECT:
3.2.2 HTTP方法枚舉
在程序中，定義以下的枚舉類型。
typedef enum
{
HTTP_GET,
HTTP_PUT,
HTTP_POST,
HTTP_OPTIONS,
HTTP_HEAD,
HTTP_DELETE,
HTTP_TRACE
} Http_method;
3.2.3 HTTP方法的通用實(shí)現(xiàn)
在HttpTunnel里面，首先定義了一個(gè)通用的實(shí)現(xiàn)HTTP方法的函數(shù)，即。
static inline ssize_t
http_method (int fd, Http_destination *dest,
Http_method method, ssize_t length)
{
char str[1024]; /* FIXME: possible buffer overflow */
Http_request *request;
ssize_t n;
if (fd == -1)
{
log_error ("http_method: fd == -1");
return -1;
}
n = 0;
if (dest->proxy_name != NULL)
n = sprintf (str, "http://%s:%d", dest->host_name, dest->host_port);
sprintf (str + n, "/index.html?crap=%ld", time (NULL));
request = http_create_request (method, str, 1, 1);
if (request == NULL)
return -1;
sprintf (str, "%s:%d", dest->host_name, dest->host_port);
http_add_header (&request->header, "Host", str);
if (length >= 0)
{
sprintf (str, "%d", length);
http_add_header (&request->header, "Content-Length", str);
}
http_add_header (&request->header, "Connection", "close");
if (dest->proxy_authorization)
{
http_add_header (&request->header,
"Proxy-Authorization",
dest->proxy_authorization);
}
if (dest->user_agent)
{
http_add_header (&request->header,
"User-Agent",
dest->user_agent);
}
n = http_write_request (fd, request);
http_destroy_request (request);
return n;
}
該函數(shù)工作流程分析如下：
3.2.4 GET方法的實(shí)現(xiàn)
ssize_t
http_get (int fd, Http_destination *dest)
{
return http_method (fd, dest, HTTP_GET, -1);
}
3.2.5 POST方法的實(shí)現(xiàn)
http_post (int fd, Http_destination *dest, size_t length)
{
return http_method (fd, dest, HTTP_POST, (ssize_t)length);
}
3.3 HTTP請求的封裝與實(shí)現(xiàn)
3.3.1 HTTP請求結(jié)構(gòu)
typedef struct
{
Http_method method;
const char *uri;
int major_version;
int minor_version;
Http_header *header;
} Http_request;
3.3.2 為HTTP請求分配內(nèi)存空間
static inline Http_request *
http_allocate_request (const char *uri)
{
Http_request *request;
request = malloc (sizeof (Http_request));
if (request == NULL)
return NULL;
request->uri = strdup (uri);
if (request->uri == NULL)
{
free (request);
return NULL;
}
return request;
}
3.3.3 創(chuàng)建HTTP請求
Http_request *
http_create_request (Http_method method,
const char *uri,
int major_version,
int minor_version)
{
Http_request *request;
request = http_allocate_request (uri);
if (request == NULL)
return NULL;
request->method = method;
request->major_version = major_version;
request->minor_version = minor_version;
request->header = NULL;
return request;
}
3.3.4 解析HTTP請求
ssize_t
http_parse_request (int fd, Http_request **request_)
{
Http_request *request;
unsigned char *data;
size_t len;
ssize_t n;
*request_ = NULL;
request = malloc (sizeof (Http_request));
if (request == NULL)
{
log_error ("http_parse_request: out of memory");
return -1;
}
request->method = -1;
request->uri = NULL;
request->major_version = -1;
request->minor_version = -1;
request->header = NULL;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (request);
return n;
}
request->method = http_string_to_method (data, n - 1);
if (request->method == -1)
{
log_error ("http_parse_request: expected an HTTP method");
free (data);
free (request);
return -1;
}
data[n - 1] = 0;
log_verbose ("http_parse_request: method = "%s"", data);
free (data);
len = n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (request);
return n;
}
data[n - 1] = 0;
request->uri = data;
len += n;
log_verbose ("http_parse_request: uri = "%s"", request->uri);
n = read_until (fd, '/', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
else if (n != 5 || memcmp (data, "HTTP", 4) != 0)
{
log_error ("http_parse_request: expected "HTTP"");
free (data);
http_destroy_request (request);
return -1;
}
free (data);
len = n;
n = read_until (fd, '.', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
data[n - 1] = 0;
request->major_version = atoi (data);
log_verbose ("http_parse_request: major version = %d",
request->major_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
data[n - 1] = 0;
request->minor_version = atoi (data);
log_verbose ("http_parse_request: minor version = %d",
request->minor_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
free (data);
if (n != 1)
{
log_error ("http_parse_request: invalid line ending");
http_destroy_request (request);
return -1;
}
len += n;
n = parse_header (fd, &request->header);
if (n <= 0)
{
http_destroy_request (request);
return n;
}
len += n;
*request_ = request;
return len;
}
3.3.5 銷毀HTTP請求
void
http_destroy_request (Http_request *request)
{
if (request->uri)
free ((char *)request->uri);
http_destroy_header (request->header);
free (request);
}
3.4 HTTP響應(yīng)的封裝與實(shí)現(xiàn)
3.4.1 HTTP響應(yīng)結(jié)構(gòu)
typedef struct
{
int major_version;
int minor_version;
int status_code;
const char *status_message;
Http_header *header;
} Http_response;
3.4.2 為HTTP響應(yīng)分配內(nèi)存空間
static inline Http_response *
http_allocate_response (const char *status_message)
{
Http_response *response;
response = malloc (sizeof (Http_response));
if (response == NULL)
return NULL;
response->status_message = strdup (status_message);
if (response->status_message == NULL)
{
free (response);
return NULL;
}
return response;
}
3.4.3 創(chuàng)建HTTP響應(yīng)
Http_response *
http_create_response (int major_version,
int minor_version,
int status_code,
const char *status_message)
{
Http_response *response;
response = http_allocate_response (status_message);
if (response == NULL)
return NULL;
response->major_version = major_version;
response->minor_version = minor_version;
response->status_code = status_code;
response->header = NULL;
return response;
}
3.4.4 解析HTTP響應(yīng)
ssize_t
http_parse_response (int fd, Http_response **response_)
{
Http_response *response;
unsigned char *data;
size_t len;
ssize_t n;
*response_ = NULL;
response = malloc (sizeof (Http_response));
if (response == NULL)
{
log_error ("http_parse_response: out of memory");
return -1;
}
response->major_version = -1;
response->minor_version = -1;
response->status_code = -1;
response->status_message = NULL;
response->header = NULL;
n = read_until (fd, '/', &data);
if (n <= 0)
{
free (response);
return n;
}
else if (n != 5 || memcmp (data, "HTTP", 4) != 0)
{
log_error ("http_parse_response: expected "HTTP"");
free (data);
free (response);
return -1;
}
free (data);
len = n;
n = read_until (fd, '.', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->major_version = atoi (data);
log_verbose ("http_parse_response: major version = %d",
response->major_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->minor_version = atoi (data);
log_verbose ("http_parse_response: minor version = %d",
response->minor_version);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->status_code = atoi (data);
log_verbose ("http_parse_response: status code = %d",
response->status_code);
free (data);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
free (response);
return n;
}
data[n - 1] = 0;
response->status_message = data;
log_verbose ("http_parse_response: status message = "%s"",
response->status_message);
len += n;
n = read_until (fd, ' ', &data);
if (n <= 0)
{
http_destroy_response (response);
return n;
}
free (data);
if (n != 1)
{
log_error ("http_parse_request: invalid line ending");
http_destroy_response (response);
return -1;
}
len += n;
n = parse_header (fd, &response->header);
if (n <= 0)
{
http_destroy_response (response);
return n;
}
len += n;
*response_ = response;
return len;
}
3.5 HTTP目標(biāo)的封裝
typedef struct
{
const char *host_name;
int host_port;
const char *proxy_name;
int proxy_port;
const char *proxy_authorization;
const char *user_agent;
} Http_destination;
還有一些其他相關(guān)函數(shù)可以參考源程序。
