×

OKHttp源码浅析与最佳实践

96
砌墙的民工
2016.07.01 14:14* 字数 3228

1、概述

OKHttp 是 Square 开源的一款高效的处理网络请求的工具。不仅限于处理 Http 请求。

功能特点:

  • 链接复用
  • Response 缓存和 Cookie
  • 默认 GZIP
  • 请求失败自动重连
  • DNS 扩展
  • Http2/SPDY/WebSocket 协议支持

OKHttp 类似于 HttpUrlConnection, 是基于传输层实现应用层协议的网络框架。 而不止是一个 Http 请求应用的库。

2、请求处理流程


  1. execute()/enqueue() 提交请求开始执行
  2. RealCall.getResponse()
  3. HttpEngine 处理 sendRequest, readResponse
  4. sendRequest 之前,会先从 Cache 中判断当前请求是否可以从缓存中返回
  5. connect 发起连接, 先从 ConnectPool 中找到缓存的连接,如果没有会建立一个新的 RealConnection
  6. RealConnection 本身是基于 Socket 的, 在 Socket 之上建立各种协议. buildConnection(), establishProtocol()
  7. Platform 处理真实的 socket 连接。 通过反射适配 Android 与 Java, 以及 ALPN。

3、特性分析

缓存处理

public void sendRequest() throws RequestException, RouteException, IOException { 
  if (cacheStrategy != null) return; // Already sent. 
  if (httpStream != null) throw new IllegalStateException();  
  Request request = networkRequest(userRequest);  
  InternalCache responseCache =
       Internal.instance.internalCache(client); 
  Response cacheCandidate = responseCache != null  ?  responseCache.get(request) : null; 
  long now = System.currentTimeMillis();  
  cacheStrategy = new CacheStrategy.Factory(now, request, cacheCandidate).get();  
  networkRequest = cacheStrategy.networkRequest; 
  cacheResponse = cacheStrategy.cacheResponse;
...
}

CacheStrategy 中会根据 Http Headers 从 Cache 中加载缓存。

// If we don't need the network, we're done.
if (networkRequest == null) { 
  userResponse = cacheResponse.newBuilder()     
   .request(userRequest)      
   .priorResponse(stripBody(priorResponse))      
   .cacheResponse(stripBody(cacheResponse))      
   .build();  
  userResponse = unzip(userResponse);  
  return;
}

缓存判断通过, 直接返回 Response。 不会再发起请求。

链路复用

StreamAllocation 负责创建 Stream 和 Connection.

Stream 有各种协议版本的实现, 负责网络读写数据

private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout,
    boolean connectionRetryEnabled) throws IOException, RouteException {
  Route selectedRoute;
  synchronized (connectionPool) {
    if (released) throw new IllegalStateException("released");
    if (stream != null) throw new IllegalStateException("stream != null");
    if (canceled) throw new IOException("Canceled");

    RealConnection allocatedConnection = this.connection;
    if (allocatedConnection != null && !allocatedConnection.noNewStreams) {
      return allocatedConnection;
    }

    // Attempt to get a connection from the pool. 
    //先从 ConnectPool 中取出可用的 Connection
    RealConnection pooledConnection = Internal.instance.get(connectionPool, address, this);
    if (pooledConnection != null) {
      this.connection = pooledConnection;
      return pooledConnection;
    }

    selectedRoute = route;
  }

  if (selectedRoute == null) {
    selectedRoute = routeSelector.next();
    synchronized (connectionPool) {
      route = selectedRoute;
      refusedStreamCount = 0;
    }
  }
  RealConnection newConnection = new RealConnection(selectedRoute);
  acquire(newConnection);

  // 创建完后, 将 Connection 存入 ConnectionPool
  synchronized (connectionPool) {
    Internal.instance.put(connectionPool, newConnection);
    this.connection = newConnection;
    if (canceled) throw new IOException("Canceled");
  }

// 发起连接
  newConnection.connect(connectTimeout, readTimeout, writeTimeout, address.connectionSpecs(),
      connectionRetryEnabled);
  routeDatabase().connected(newConnection.route());

  return newConnection;
}

链接何时释放

ConnectionPool 中有一个 Runnable, 负责清除不再使用的链接

  private final Runnable cleanupRunnable = new Runnable() {
    @Override public void run() {
      while (true) {
        long waitNanos = cleanup(System.nanoTime());
        if (waitNanos == -1) return;
        if (waitNanos > 0) {
          long waitMillis = waitNanos / 1000000L;
          waitNanos -= (waitMillis * 1000000L);
          synchronized (ConnectionPool.this) {
            try {
              ConnectionPool.this.wait(waitMillis, (int) waitNanos);
            } catch (InterruptedException ignored) {
            }
          }
        }
      }
    }
  };

能够被清除,前提是这个链接已经被标记为 Idle 了。处理代码在 RealCall getResponse() 请求结束之后。

失败重连

RealCall getResponse 时, 如果中间出现异常或者需要重定向请求, 会再 new HttpEngine 继续发起请求。

 */
  Response getResponse(Request request, boolean forWebSocket) throws IOException {
    // Copy body metadata to the appropriate request headers.

    …   

    int followUpCount = 0;
    while (true) {
      if (canceled) {
        engine.releaseStreamAllocation();
        throw new IOException("Canceled");
      }

      boolean releaseConnection = true;
      try {
        engine.sendRequest();
        engine.readResponse();
        releaseConnection = false;
      } catch (RequestException e) {
        // The attempt to interpret the request failed. Give up.
        throw e.getCause();
      } catch (RouteException e) {
        // The attempt to connect via a route failed. The request will not have been sent.
        HttpEngine retryEngine = engine.recover(e.getLastConnectException(), true, null);
        if (retryEngine != null) {
          releaseConnection = false;
          engine = retryEngine;
          continue;
        }
        // Give up; recovery is not possible.
        throw e.getLastConnectException();
      } catch (IOException e) {
        // An attempt to communicate with a server failed. The request may have been sent.
        HttpEngine retryEngine = engine.recover(e, false, null);
        if (retryEngine != null) {
          releaseConnection = false;
          engine = retryEngine;
          continue;
        }

        // Give up; recovery is not possible.
        throw e;
      } finally {
        // We're throwing an unchecked exception. Release any resources.
        if (releaseConnection) {
          StreamAllocation streamAllocation = engine.close();
          streamAllocation.release();
        }
      }

      Response response = engine.getResponse();
      Request followUp = engine.followUpRequest();

      if (followUp == null) {
        if (!forWebSocket) {
          engine.releaseStreamAllocation();
        }
        return response;
      }

      StreamAllocation streamAllocation = engine.close();

      if (++followUpCount > MAX_FOLLOW_UPS) {
        streamAllocation.release();
        throw new ProtocolException("Too many follow-up requests: " + followUpCount);
      }

      if (!engine.sameConnection(followUp.url())) {
        streamAllocation.release();
        streamAllocation = null;
      } else if (streamAllocation.stream() != null) {
        throw new IllegalStateException("Closing the body of " + response
            + " didn't close its backing stream. Bad interceptor?");
      }

      request = followUp;
      engine = new HttpEngine(client, request, false, false, forWebSocket, streamAllocation, null,
          response);
    }
  }
}

DNS 扩展

在 findConnection() 的过程中,会通过 RouterSelector 组件找到需要链接的地址。 在如果未找到, 会通过 DNS 接口根据 hostname 查询对应的 IP 地址列表,并且在内存缓存这个列表。

 /** Prepares the socket addresses to attempt for the current proxy or host. */
  private void resetNextInetSocketAddress(Proxy proxy) throws IOException {
    // Clear the addresses. Necessary if getAllByName() below throws!
    inetSocketAddresses = new ArrayList<>();

    String socketHost;
    int socketPort;
    if (proxy.type() == Proxy.Type.DIRECT || proxy.type() == Proxy.Type.SOCKS) {
      socketHost = address.url().host();
      socketPort = address.url().port();
    } else {
      SocketAddress proxyAddress = proxy.address();
      if (!(proxyAddress instanceof InetSocketAddress)) {
        throw new IllegalArgumentException(
            "Proxy.address() is not an " + "InetSocketAddress: " + proxyAddress.getClass());
      }
      InetSocketAddress proxySocketAddress = (InetSocketAddress) proxyAddress;
      socketHost = getHostString(proxySocketAddress);
      socketPort = proxySocketAddress.getPort();
    }

    if (socketPort < 1 || socketPort > 65535) {
      throw new SocketException("No route to " + socketHost + ":" + socketPort
          + "; port is out of range");
    }

    if (proxy.type() == Proxy.Type.SOCKS) {
      inetSocketAddresses.add(InetSocketAddress.createUnresolved(socketHost, socketPort));
    } else {
      // Try each address for best behavior in mixed IPv4/IPv6 environments.
      List<InetAddress> addresses = address.dns().lookup(socketHost);
      for (int i = 0, size = addresses.size(); i < size; i++) {
        InetAddress inetAddress = addresses.get(i);
        inetSocketAddresses.add(new InetSocketAddress(inetAddress, socketPort));
      }
    }

    nextInetSocketAddressIndex = 0;
  }

在 Dns 类中, 有一个默认的 SYSTEM 实现,如果没单独配置 Dns, 会使用默认的实现。

public interface Dns {
  /**
   * A DNS that uses {@link InetAddress#getAllByName} to ask the underlying operating system to
   * lookup IP addresses. Most custom {@link Dns} implementations should delegate to this instance.
   */
  Dns SYSTEM = new Dns() {
    @Override public List<InetAddress> lookup(String hostname) throws UnknownHostException {
      if (hostname == null) throw new UnknownHostException("hostname == null");
      return Arrays.asList(InetAddress.getAllByName(hostname));
    }
  };

  /**
   * Returns the IP addresses of {@code hostname}, in the order they will be attempted by OkHttp. If
   * a connection to an address fails, OkHttp will retry the connection with the next address until
   * either a connection is made, the set of IP addresses is exhausted, or a limit is exceeded.
   */
  List<InetAddress> lookup(String hostname) throws UnknownHostException;
}

Http2/SPDY 协议支持

使用 Http2/SPDY 协议需要用到 ALPN 组件,并且服务器支持。 在 Android 5.0 以后系统包含了 ALPN。
协议兼容代码在 AndroidPlatform 类中

 public static Platform buildIfSupported() {
    // Attempt to find Android 2.3+ APIs.
    try {
      Class<?> sslParametersClass;
      try {
        sslParametersClass = Class.forName("com.android.org.conscrypt.SSLParametersImpl");
      } catch (ClassNotFoundException e) {
        // Older platform before being unbundled.
        sslParametersClass = Class.forName(
            "org.apache.harmony.xnet.provider.jsse.SSLParametersImpl");
      }

      OptionalMethod<Socket> setUseSessionTickets = new OptionalMethod<>(
          null, "setUseSessionTickets", boolean.class);
      OptionalMethod<Socket> setHostname = new OptionalMethod<>(
          null, "setHostname", String.class);
      OptionalMethod<Socket> getAlpnSelectedProtocol = null;
      OptionalMethod<Socket> setAlpnProtocols = null;

      // Attempt to find Android 5.0+ APIs.
      try {
        Class.forName("android.net.Network"); // Arbitrary class added in Android 5.0.
        getAlpnSelectedProtocol = new OptionalMethod<>(byte[].class, "getAlpnSelectedProtocol");
        setAlpnProtocols = new OptionalMethod<>(null, "setAlpnProtocols", byte[].class);
      } catch (ClassNotFoundException ignored) {
      }

      return new AndroidPlatform(sslParametersClass, setUseSessionTickets, setHostname,
          getAlpnSelectedProtocol, setAlpnProtocols);
    } catch (ClassNotFoundException ignored) {
      // This isn't an Android runtime.
    }

    return null;
  }

WebSocket 协议支持

加入 okhttp-ws 扩展包,可以使用 WebSocket 协议连接服务器。

compile 'com.squareup.okhttp3:okhttp-ws:3.3.1'

最佳实践

OKHttp 是一个更倾向于协议的网络框架。 通常在使用上,我们会再包装一层,以简化调用。

封装的 BJNetwork 库主要实现了:

  • 缓存的自动配置(cache + cookie)
  • 请求日志开关
  • 基于腾讯的 DnsPod 实现的 DNS 扩展(可选)
  • Get & Post 请求参数的简化
  • 上传&下载进度
  • 根据 tag 关闭请求; 或者 tag 被销毁后,自动关闭请求
  • Http2/SPDY 配置
  • 支持 RxJava 调用
  • WebSocket 的调用封装以及自动重连处理

示例代码

集成

compile 'io.github.yangxlei:bjnetwork:1.5.1'

创建 BJNetworkClient :

 BJNetworkClient client  = new BJNetworkClient.Builder()
                // 设置缓存文件存储路径. 会自动将请求的缓存和 cookie 存储在该路径下
                .setCacheDir(context.getCacheDir())
                // 开启开关后, 会在支持协议列表内加入 Http2 & SPDY. 如果服务器不支持该协议, 建议关闭
                .setEnableHttp2x(false)
                // 开启日志. 建议 Debug 环境开启, Release 环境关闭.
                .setEnableLog(true)
                .setConnectTimeoutAtSeconds(30)
                .setReadTimeoutAtSeconds(30)
                .setWriteTimeoutAtSeconds(30)
                // DnsPod 的实现. 不设置默认使用 SYSTEM
                .setDns(new DnsPodImpl(context.getCacheDir()))
                .build();

请求管理类

 // 建议由调用者维护一份实例
  mNetRequestManager = new BJNetRequestManager(client);

发起请求

       //创建请求
        BJNetCall call = mNetRequestManager.newGetCall("http://xxx.com");
//        call = mNetRequestManager.newPostCall("http://xxx.com/xx", requestBody);

        Object tag = new Object();

        //同步请求
        try {
            BJResponse response = call.executeSync(tag);
        } catch (IOException e) {
            e.printStackTrace();
        }

        //异步请求
        call.executeAsync(tag, new BJNetCallback() {
            @Override
            public void onFailure(Exception e) {
            }

            @Override
            public void onResponse(BJResponse bjResponse) {
            }
        });

        // tag 会被 JVM 回收. 响应的请求也会被自动关闭
        tag = null;

执行请求后, RequestManager 内会建立一个 tag 和对应的 calls 的虚引用 list。 当 tag 被回收后, 自动关闭 list 中的 call。
需要注意的是, 如果这个 tag 传入的是 this, 因为 callback 会一直被持有直到请求结束, callback 中会有一个 this 的引用。 所以 this 不会销毁也无法自动关闭请求。可以调用 cancelCalls 主动关闭

    mNetRequestManager.cancelCalls(tag);

下载进度

       BJNetCall call = requestManager.newDownloadCall("http://d.gsxservice.com/app/genshuixue.apk", getCacheDir());
        call.executeAsync(tag, new BJDownloadCallback() {
            @Override
            public void onProgress(long progress, long total) {
                System.out.println("download  onProgress " + progress +"," + total +" " + Thread.currentThread());
            }

            @Override
            public void onDownloadFinish(BJResponse response, File file) {
                System.out.println("download  onDownloadFinish " + Thread.currentThread());
            }

            @Override
            public void onFailure(Exception e) {
                e.printStackTrace();
                System.out.println("download  onFailure " + Thread.currentThread());
            }
        });

上传进度

 BJRequestBody requestBody = BJRequestBody.createWithMultiForm(params, "Filedata",
                new File("aaaa.jpg"),BJRequestBody.MEDIA_TYPE_IMAGE);

        requestManager.newPostCall("http://xxx.com/doc/upload",requestBody)
                .executeAsync(this, new BJProgressCallback(){
                    @Override
                    public void onFailure(Exception e) {
                        System.out.println("1upload  onFailure " + Thread.currentThread());
                    }

                    @Override
                    public void onResponse(BJResponse response) {
                        System.out.println("1upload  onResponse "+ " "+ Thread.currentThread());
                    }

                    @Override
                    public void onProgress(long progress, long total) {
                        System.out.println("1upload  onProgress " + progress*100/total+" " + Thread.currentThread());
                    }
                });

提供了 BJProgressCallback 和 BJDownloadCallback 两种回调实现。 在异步执行时传入 ProgressCallback 会自动提供进度的回调。

RxJava 扩展

compile 'io.github.yangxlei:rx-bjnetwork:1.5.1'

示例:

        mRxNetRequestManager = new BJRxNetRequestManager(client);
        mRxNetRequestManager.rx_newGetCall("http://xxx.com")
                .subscribeOn(Schedulers.io())
                .observeOn(AndroidSchedulers.mainThread())
                .subscribe(new Action1<BJResponse>() {
                    @Override
                    public void call(BJResponse response) {
                        try {
                            String result = response.getResponseString();
                        } catch (IOException e) {
                            e.printStackTrace();
                            throw new HttpException(e);
                        }
                    }
                }, new Action1<Throwable>() {
                    @Override
                    public void call(Throwable throwable) {
                       HttpException exception = (HttpException) throwable;
                    }
                });

WebSocket 扩展

compile 'io.github.yangxlei:bjnetwork-ws:1.5.1'

示例:

        mWebsocketClient = new BJWebsocketClient("TestWS");
        mWebsocketClient.setAddress("ws://xxx.com");
        // LogLevel.Body 会打印出收发的数据以及行为. Info 只打印行为不打印详细数据
        mWebsocketClient.setLogLevel(BJWebsocketClient.LogLevel.Body);
        mWebsocketClient.setListener(new BJWebsocketListener() {

            @Override
            public void onReconnect(BJWebsocketClient client) {

            }

            @Override
            public void onClose(BJWebsocketClient client) {

            }

            @Override
            public void onSentMessageFailure(BJWebsocketClient client, BJMessageBody messageBody) {

            }

            @Override
            public void onMessage(BJWebsocketClient client, String message) {
                System.out.println("WS receive:" + message);
            }

            @Override
            public void onMessage(BJWebsocketClient client, InputStream inputStream) {

            }

            @Override
            public void onStateChanged(BJWebsocketClient client, BJWebsocketClient.State state) {

            }
        });

        // 启动链接
        mWebsocketClient.connect();
        // 消息会进入发送队列. 链接建立成功后, 逐个发送
        mWebsocketClient.sendMessage("message");
        // 关闭链接
        mWebsocketClient.disconnect();

1.WebSocketClient 内维护一个消息发送线程和消息队列。 链接成功后自动发送消息。 不排除消息可能会在本地发送失败, 内部处理默认情况会自动重试五次, 如果都失败会将 message 通过 onSentMessageFailure 返回上层。
2.WebSocketClient 维护三种状态: Offline, Connecting, Connected.
3.自动重连,无法避免某些情况下链接会被断开。 内部处理如果非主动断开链接, 都会自动重新建立链接。

踩过的几个坑

1、gzip 问题

OKHttp 响应会自动处理 gzip 解压缩. 在 HttpEngine 类中收到响应后会调用:

  private Response unzip(final Response response) throws IOException {
    if (!transparentGzip || !"gzip".equalsIgnoreCase(userResponse.header("Content-Encoding"))) {
      return response;
    }

    if (response.body() == null) {
      return response;
    }

    GzipSource responseBody = new GzipSource(response.body().source());
    Headers strippedHeaders = response.headers().newBuilder()
        .removeAll("Content-Encoding")
        .removeAll("Content-Length")
        .build();
    return response.newBuilder()
        .headers(strippedHeaders)
        .body(new RealResponseBody(strippedHeaders, Okio.buffer(responseBody)))
        .build();
  }

发现里面有个 transparentGzip 的判断,再看来这个值是怎么回事:

  private Request networkRequest(Request request) throws IOException {
    Request.Builder result = request.newBuilder();

    if (request.header("Host") == null) {
      result.header("Host", hostHeader(request.url(), false));
    }

    if (request.header("Connection") == null) {
      result.header("Connection", "Keep-Alive");
    }

    if (request.header("Accept-Encoding") == null) {
      transparentGzip = true;
      result.header("Accept-Encoding", "gzip");
    }

    List<Cookie> cookies = client.cookieJar().loadForRequest(request.url());
    if (!cookies.isEmpty()) {
      result.header("Cookie", cookieHeader(cookies));
    }

    if (request.header("User-Agent") == null) {
      result.header("User-Agent", Version.userAgent());
    }

    return result.build();
  }

sendRequest 的时候会先补全常用的 Http Headers。 如果不存在 “Accept-Encoding” 头会 transparentGzip=true。
也就是说,OKHttp 的处理规则是:如果你的请求自己主动加了 gzip 支持, 那么响应也自己处理 gzip 解压缩; 否则我给你做。

2、WebSocket SocketTimeOutException

WebSocket 在链接成功之后,可能过一会就会抛个 timeout 异常然后断开链接。出问题的地方是:

 private void createWebSocket(Response response, WebSocketListener listener) throws IOException {
    if (response.code() != 101) {
      throw new ProtocolException("Expected HTTP 101 response but was '"
          + response.code()
          + " "
          + response.message()
          + "'");
    }

    String headerConnection = response.header("Connection");
    if (!"Upgrade".equalsIgnoreCase(headerConnection)) {
      throw new ProtocolException(
          "Expected 'Connection' header value 'Upgrade' but was '" + headerConnection + "'");
    }
    String headerUpgrade = response.header("Upgrade");
    if (!"websocket".equalsIgnoreCase(headerUpgrade)) {
      throw new ProtocolException(
          "Expected 'Upgrade' header value 'websocket' but was '" + headerUpgrade + "'");
    }
    String headerAccept = response.header("Sec-WebSocket-Accept");
    String acceptExpected = Util.shaBase64(key + WebSocketProtocol.ACCEPT_MAGIC);
    if (!acceptExpected.equals(headerAccept)) {
      throw new ProtocolException("Expected 'Sec-WebSocket-Accept' header value '"
          + acceptExpected
          + "' but was '"
          + headerAccept
          + "'");
    }

    StreamAllocation streamAllocation = Internal.instance.callEngineGetStreamAllocation(call);
    RealWebSocket webSocket = StreamWebSocket.create(
        streamAllocation, response, random, listener);

    listener.onOpen(webSocket, response);

    while (webSocket.readMessage()) {
    }
  }

最后一行会进入循环不断的读取数据。 默认 OkHttpClient 的 readTimeOut 时间是 10s。所以如果服务器 10s 内没有数据返回,客户端就会自动断开。 所以配置的时候可以把 readTimeout 值设置长一点。 BJWebsocketClient 默认是 10 分钟。

Refference

HTTP DNS的实现
OkHttp3之Cookies管理及持久化
Https 详解
Http2
WebSocket

Android实战
Web note ad 1