Go常用库-网络框架Gin(1) - 原生http包

一. 写在前面:

• Go提供了原生的Http实现-net/http包,封装的挺好,作为服务器使用把底层的TCP连接,请求报文解析,Header解析,Cookie解析,返回报文组装等功能都实现好了.使用起来只需要写真正逻辑相关的Handler即可.作为客户端使用也提供了默认的Client,只需要构造出请求,即可发送出去,并获得返回.
• 但是,由于定义和实现是绑定在一起的,如果不想用原生的东西,扩展起来可就麻烦了.不花时间啃几千行的server.go和transport.go肯定改不动的.相比之下,感觉javax.servlet就漂亮的多,只是定义好接口,具体怎么实现容器或客户端自行决定;也很容易升级Http版本的支持.

二. net/http包源码简单阅读分析

注: 由于目标只是了解处理流程,对于文件传输(chunk)和HTTP/2的部分,都直接略过了.

1. http子包与关联包

2. http公用源码

Request结构体--(去了注释)

type Request struct {
 Method string
 URL *url.URL
 Proto string // "HTTP/1.0"
 ProtoMajor int // 1
 ProtoMinor int // 0
 Header Header
 Body io.ReadCloser
 GetBody func() (io.ReadCloser, error)
 ContentLength int64
 TransferEncoding []string
 Close bool
 Host string
 Form url.Values
 PostForm url.Values
 MultipartForm *multipart.Form
 Trailer Header
 RemoteAddr string
 TLS *tls.ConnectionState
 Response *Response
 ctx context.Context
}

包含了解析好的URL,eader,Form值;其中Repsonse应该是空的,只有重定向交易才有值.
ctx是后面加入了,为了能优雅进行超时关闭.

Response结构体--(去了注释)

type Response struct {
 Status string // e.g. "200 OK"
 StatusCode int // e.g. 200
 Proto string // e.g. "HTTP/1.0"
 ProtoMajor int // e.g. 1
 ProtoMinor int // e.g. 0
 Header Header
 Body io.ReadCloser
 ContentLength int64
 TransferEncoding []string
 Close bool
 Uncompressed bool
 Trailer Header
 // Request is the request that was sent to obtain this Response.
 // Request's Body is nil (having already been consumed).
 // This is only populated for Client requests.
 Request *Request
 TLS *tls.ConnectionState
}

代表了准备返回给Client的Response,其中Body是个ReadCloser

3. http客户端相关源码

transport的核心方法roundTrip(): -- 太长了,略..

client结构为:

type Client struct {
 Transport RoundTripper
 Jar CookieJar
 Timeout time.Duration
}

client东西都藏在Transport里面了,额外增加一个cookie控制和超时.
client的核心方法do(): (这点起名风格很spring啊)

func (c *Client) do(req *Request) (retres *Response, reterr error) {
 if testHookClientDoResult != nil {
 defer func() { testHookClientDoResult(retres, reterr) }()
 }
 if req.URL == nil {
 req.closeBody()
 return nil, &url.Error{
 Op: urlErrorOp(req.Method),
 Err: errors.New("http: nil Request.URL"),
 }
 }
 // 做了基础的校验后,创建函数变量
 var (
 deadline = c.deadline()
 reqs []*Request
 resp *Response
 copyHeaders = c.makeHeadersCopier(req)
 reqBodyClosed = false // have we closed the current req.Body?
 // Redirect behavior:
 redirectMethod string
 includeBody bool
 )
 // 定义错误处理函数
 uerr := func(err error) error {
 // the body may have been closed already by c.send()
 if !reqBodyClosed {
 req.closeBody()
 }
 var urlStr string
 if resp != nil && resp.Request != nil {
 urlStr = stripPassword(resp.Request.URL)
 } else {
 urlStr = stripPassword(req.URL)
 }
 return &url.Error{
 Op: urlErrorOp(reqs[0].Method),
 URL: urlStr,
 Err: err,
 }
 }
 // 这里直接死循环...
 for {
 // For all but the first request, create the next
 // request hop and replace req.
 // 第一次进来这边是空的,不会进去,知道下面加入reqs才第一次外呼
 if len(reqs) > 0 {
 loc := resp.Header.Get("Location")
 if loc == "" {
 resp.closeBody()
 return nil, uerr(fmt.Errorf("%d response missing Location header", resp.StatusCode))
 }
 u, err := req.URL.Parse(loc)
 if err != nil {
 resp.closeBody()
 return nil, uerr(fmt.Errorf("failed to parse Location header %q: %v", loc, err))
 }
 host := ""
 if req.Host != "" && req.Host != req.URL.Host {
 // If the caller specified a custom Host header and the
 // redirect location is relative, preserve the Host header
 // through the redirect. See issue #22233.
 if u, _ := url.Parse(loc); u != nil && !u.IsAbs() {
 host = req.Host
 }
 }
 ireq := reqs[0]
 // 重新构造本次轮询请求的Request
 req = &Request{
 Method: redirectMethod,
 Response: resp,
 URL: u,
 Header: make(Header),
 Host: host,
 Cancel: ireq.Cancel,
 ctx: ireq.ctx,
 }
 if includeBody && ireq.GetBody != nil {
 req.Body, err = ireq.GetBody()
 if err != nil {
 resp.closeBody()
 return nil, uerr(err)
 }
 req.ContentLength = ireq.ContentLength
 }
 // Copy original headers before setting the Referer,
 // in case the user set Referer on their first request.
 // If they really want to override, they can do it in
 // their CheckRedirect func.
 copyHeaders(req)
 // Add the Referer header from the most recent
 // request URL to the new one, if it's not https->http:
 if ref := refererForURL(reqs[len(reqs)-1].URL, req.URL); ref != "" {
 req.Header.Set("Referer", ref)
 }
 err = c.checkRedirect(req, reqs)
 // Sentinel error to let users select the
 // previous response, without closing its
 // body. See Issue 10069.
 if err == ErrUseLastResponse {
 return resp, nil
 }
 // Close the previous response's body. But
 // read at least some of the body so if it's
 // small the underlying TCP connection will be
 // re-used. No need to check for errors: if it
 // fails, the Transport won't reuse it anyway.
 const maxBodySlurpSize = 2 << 10
 if resp.ContentLength == -1 || resp.ContentLength <= maxBodySlurpSize {
 io.CopyN(ioutil.Discard, resp.Body, maxBodySlurpSize)
 }
 resp.Body.Close()
 if err != nil {
 // Special case for Go 1 compatibility: return both the response
 // and an error if the CheckRedirect function failed.
 // See https://golang.org/issue/3795
 // The resp.Body has already been closed.
 ue := uerr(err)
 ue.(*url.Error).URL = loc
 return resp, ue
 }
 }
 // 先把调用存到列表
 reqs = append(reqs, req)
 var err error
 var didTimeout func() bool
 // 这里调用send--用到transport发送报文
 if resp, didTimeout, err = c.send(req, deadline); err != nil {
 // c.send() always closes req.Body
 reqBodyClosed = true
 if !deadline.IsZero() && didTimeout() {
 err = &httpError{
 // TODO: early in cycle: s/Client.Timeout exceeded/timeout or context cancellation/
 err: err.Error() + " (Client.Timeout exceeded while awaiting headers)",
 timeout: true,
 }
 }
 return nil, uerr(err)
 }
 var shouldRedirect bool
 redirectMethod, shouldRedirect, includeBody = redirectBehavior(req.Method, resp, reqs[0])
 // 这里判断本次发送结果,如果不是重定向,就可以返回了,退出for循环.
 if !shouldRedirect {
 return resp, nil
 }
 // 本次body已经消费掉了,这里要关闭才行.
 req.closeBody()
 }
}

4. http服务端相关源码

Handler的声明

// Handler的接口,只需要实现ServeHTTP即可
type Handler interface {
 ServeHTTP(ResponseWriter, *Request)
}
// 定义了HandlerFunc类型,它本身是方法,同时实现了ServeHTTP接口-->实际是调用自身
type HandlerFunc func(ResponseWriter, *Request)
// ServeHTTP calls f(w, r).
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
 f(w, r)
}

Server的定义

// A Server defines parameters for running an HTTP server.
// The zero value for Server is a valid configuration.
type Server struct {
 Addr string // TCP address to listen on, ":http" if empty
 // Server也只有一个默认的Handler,http.DefaultServeMux
 Handler Handler // handler to invoke, http.DefaultServeMux if nil
 TLSConfig *tls.Config
 ReadTimeout time.Duration
 ReadHeaderTimeout time.Duration
 WriteTimeout time.Duration
 
 IdleTimeout time.Duration
 MaxHeaderBytes int
 TLSNextProto map[string]func(*Server, *tls.Conn, Handler)
 ConnState func(net.Conn, ConnState)
 ErrorLog *log.Logger
 BaseContext func(net.Listener) context.Context
 ConnContext func(ctx context.Context, c net.Conn) context.Context
 
 disableKeepAlives int32 // accessed atomically.
 inShutdown int32 // accessed atomically (non-zero means we're in Shutdown)
 nextProtoOnce sync.Once // guards setupHTTP2_* init
 nextProtoErr error // result of http2.ConfigureServer if used
 mu sync.Mutex
 listeners map[*net.Listener]struct{}
 activeConn map[*conn]struct{}
 doneChan chan struct{}
 onShutdown []func()
}

从代码层面跟踪一下如何进行服务的处理:

1. 实际上Server并不直接处理Request,从其Serve()方法最后可以看出,它只负责连接的建立

 .......
 tempDelay = 0 
 c := srv.newConn(rw)
 c.setState(c.rwc, StateNew) // before Serve can return
 // *** 注意,这里使用goroutine,也就是说对于每个连接,都是单独的协程独立工作 ***
 go c.serve(ctx)

2. 通过newConn构造出新的conn连接,调用连接的serve方法处理交互,一个conn是可以有多次交互的.
   首先,从连接读取客户端request

for {
 // 在连接还存活的状态下,会一直读取客户端的请求.
 w, err := c.readRequest(ctx)
 ...
 // 解析好Request后,创建serverHandler对象,调用他的ServeHTTP处理请求
 serverHandler{c.server}.ServeHTTP(w, w.req)
 }

然后,调用server的ServeHTTP()方法,找处理的ServeMux

// serverHandler delegates to either the server's Handler or
// DefaultServeMux and also handles "OPTIONS *" requests.
type serverHandler struct {
 srv *Server
}
func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
 handler := sh.srv.Handler
 // 默认用Default
 if handler == nil {
 handler = DefaultServeMux
 }
 // 对"*"和"OPTIONS请求特殊处理
 if req.RequestURI == "*" && req.Method == "OPTIONS" {
 handler = globalOptionsHandler{}
 }
 // 调用ServeMux的ServeHTTP方法
 handler.ServeHTTP(rw, req)
}

3. 调用ServeMux的ServeHTTP,找到程序员注册到ServeMux结构中的处理函数

type ServeMux struct {
 mu sync.RWMutex
 m map[string]muxEntry
 es []muxEntry // slice of entries sorted from longest to shortest.
 hosts bool // whether any patterns contain hostnames
}
type muxEntry struct {
 h Handler // 这个handler就是我们自定义的处理函数
 pattern string // 这个pattern就是我们自定义的处理路径
}

可以看出,ServeMux是一个map存储器,把具体的muxEntry保存住

// ServeHTTP dispatches the request to the handler whose
// pattern most closely matches the request URL.
func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) {
 if r.RequestURI == "*" {
 if r.ProtoAtLeast(1, 1) {
 w.Header().Set("Connection", "close")
 }
 w.WriteHeader(StatusBadRequest)
 return
 }
 // 这个方法其实是默认的路由,根据request找具体处理的handler,内部调用了match()方法,实际是一致循环所有muxEntry,然后用strings.HasPrefix判断
 // httprouter说这种方式太慢,用基数树存储结构进行加速.--原生的方法确实不高效
 h, _ := mux.Handler(r)
 // 这里h就是我们调用http.HandleFunc("/hello", hello)时候注册的hello了.
 h.ServeHTTP(w, r)
}

从server代码可以看出,MuxEntry相当于Servlet,负责具体处理,ServerMux是所有Entry的集合,通过match进行router匹配,指派任务.系统提供了默认的Mux统一管理,调用http.handle()实际是想ServerMux进行注册...
ServeHTTP()才是真正具体工作的地方,调用流程为serverHandle-> DefaultServeMux -> 自定义HandlerFunc或者Handler (都叫ServeHTTP,就是为了让人看不懂吗....)

三. httpServer运行分析

好像已经包含在上面的源码分析了...略

四. 文档参考

https://juejin.im/post/5dd11baff265da0c0c1fe813

https://colobu.com/2018/07/25/exposing-go-on-the-internet/

https://cizixs.com/2016/08/17/golang-http-server-side/

本文来自:简书

感谢作者:沉寂之舟

查看原文:Go常用库-网络框架Gin(1) - 原生http包

• 编辑
• 预览