文章目录

前言

HttpClient 成立于2001年，是 Apache Jakarta Commons 项目下的子项目，2004 年离开 Commons，提升成为一个单独的 Jakarta 项目。2005 年，Jakarta 创建了 HttpComponents 项目，目标是开发 HttpClient 3.x 的继任者。2007 年，Commons 项目，也就是 HttpClient 项目的发源地，离开了 Jakarta， 成为了1个新的顶级项目。不久之后，HttpComponents 也离开了 Jakarta， 成为一个独立的顶级项目，负责维护 HttpClient 的工作。

  • HttpClient 提供了高效、最新、功能丰富的支持 HTTP 协议的客户端编程工具包，支持最新版本的 HTTP 协议。

  • HttpComponents 项目，包含 HttpClient 和 HttpCore, AsyncClient 三大模块，提供了更好的性能和更大的灵活性。

  • HttpClient 是依赖于 HttpCore 的，最新的 HttpClient 版本为 5.2

  • HttpClient 是以 3.1 版本为分隔，大版本之间用法有很多不同

  • 最新文档地址：https://hc.apache.org/httpcomponents-client-5.2.x/index.html

  • 旧版文档地址：https://hc.apache.org/httpclient-legacy/userguide.html

  • github 地址：https://github.com/apache/httpcomponents-client

  • pom 依赖

    <!-- 最新版本5 -->
<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5</artifactId>
    <version>5.2.1</version>
</dependency>

<!-- 版本4 -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.13</version>
</dependency>

<!-- 旧版本3，07年后没更新 -->
<dependency>
    <groupId>commons-httpclient</groupId>
    <artifactId>commons-httpclient</artifactId>
    <version>3.1</version>
</dependency>

一、简单使用

1.1 get 请求

String url = "http://httpbin.org/get";
try (final CloseableHttpClient httpclient = HttpClients.createDefault()) {
    final HttpGet httpget = new HttpGet(url);

    // Create a custom response handler
    final HttpClientResponseHandler<String> responseHandler = response -> {
        final int status = response.getCode();
        if (status >= HttpStatus.SC_SUCCESS && status < HttpStatus.SC_REDIRECTION) {
            final HttpEntity entity = response.getEntity();
            try {
                return entity != null ? EntityUtils.toString(entity) : null;
            } catch (final ParseException ex) {
                throw new ClientProtocolException(ex);
            }
        } else {
            throw new ClientProtocolException("Unexpected response status: " + status);
        }
    };
    final String responseBody = httpclient.execute(httpget, responseHandler);
    System.out.println(responseBody);
}

1.2 post 简单表单请求

String url = "http://httpbin.org/post";
String username = "root";
String loginPw = "";
try (final CloseableHttpClient httpclient = HttpClients.createDefault()) {
    final HttpPost httppost = new HttpPost(url);
    final List<NameValuePair> params = new ArrayList<>();
    params.add(new BasicNameValuePair("username", username));
    params.add(new BasicNameValuePair("password", loginPw));
    httppost.setEntity(new UrlEncodedFormEntity(params));

    // Create a custom response handler
    final HttpClientResponseHandler<String> responseHandler = response -> {
        final int status = response.getCode();
        if (status >= HttpStatus.SC_SUCCESS && status < HttpStatus.SC_REDIRECTION) {
            final HttpEntity entity = response.getEntity();
            try {
                return entity != null ? EntityUtils.toString(entity) : null;
            } catch (final ParseException ex) {
                throw new ClientProtocolException(ex);
            }
        } else {
            throw new ClientProtocolException("Unexpected response status: " + status);
        }
    };
    final String responseBody = httpclient.execute(httppost, responseHandler);
    System.out.println(responseBody);
}

1.3 表单上传文件

final HttpPost httppost = new HttpPost(url);
            
final MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addTextBody("username", username);
builder.addTextBody("password", password);
builder.addBinaryBody("file", new File("src/test/resources/test.txt"), ContentType.APPLICATION_OCTET_STREAM, "test.txt");

final HttpEntity multipart = builder.build();

httppost.setEntity(multipart);

1.4 上传 json 数据

final HttpPost httppost = new HttpPost(url);

httppost.setHeader("Accept", "application/json");
httppost.setHeader("Content-type", "application/json");

final String json = "{\"id\":1,\"name\":\"John\"}";
final StringEntity stringEntity = new StringEntity(json);
httppost.setEntity(stringEntity);

二、高级用法

2.1 超时和重试

超时控制可以通过 RequestConfig 这个类控制

String url = "http://httpbin.org/get";

RequestConfig requestConfig = RequestConfig.custom()
    .setConnectionRequestTimeout(Timeout.ofSeconds(100L))//连接请求超时, 0为无限。默认值：3分钟。
    .setResponseTimeout(Timeout.ofSeconds(600L)) // 响应超时时间，0为无限。带有消息复用的HTTP传输可能不支持响应超时
    .build();

try (final CloseableHttpClient httpclient = HttpClients.createDefault()) {
    final HttpGet httpGet = new HttpGet(url);
    httpGet.setConfig(requestConfig);
    final HttpClientResponseHandler<String> responseHandler = response -> {
        final int status = response.getCode();

        if (status >= HttpStatus.SC_SUCCESS && status < HttpStatus.SC_REDIRECTION) {
            final HttpEntity entity = response.getEntity();
            try {
                return entity != null ? EntityUtils.toString(entity) : null;
            } catch (final ParseException ex) {
                throw new ClientProtocolException(ex);
            }
        } else {
            throw new ClientProtocolException("Unexpected response status: " + status);
        }
    };
    httpclient.execute(httpGet, responseHandler);
}

重试，默认重试策略为最大次数1次，重试间隔为1秒。

String url = "http://httpbin.org/get";

try (final CloseableHttpClient httpclient = HttpClients.custom()
     .setRetryStrategy(new DefaultHttpRequestRetryStrategy(3, TimeValue.ofSeconds(20L)))
     .build()) {
    final HttpGet httpGet = new HttpGet(url);

    final HttpClientResponseHandler<String> responseHandler = response -> {
        final int status = response.getCode();

        if (status >= HttpStatus.SC_SUCCESS && status < HttpStatus.SC_REDIRECTION) {
            final HttpEntity entity = response.getEntity();
            try {
                return entity != null ? EntityUtils.toString(entity) : null;
            } catch (final ParseException ex) {
                throw new ClientProtocolException(ex);
            }
        } else {
            throw new ClientProtocolException("Unexpected response status: " + status);
        }
    };
    httpclient.execute(httpGet, responseHandler);
}

2.2 Cookie

HttpClients.createDefault 已经内置默认 Cookie 管理器可以用来携带 Cookie 访问

String url = "http://httpbin.org/cookies/set/foo/bar";
String url2 = "http://httpbin.org/cookies";

try (final CloseableHttpClient httpclient = HttpClients.createDefault()) {
    final HttpGet httpGet = new HttpGet(url);

    final HttpClientResponseHandler<String> responseHandler = response -> {
        final int status = response.getCode();

        if (status >= HttpStatus.SC_SUCCESS && status < HttpStatus.SC_REDIRECTION) {
            final HttpEntity entity = response.getEntity();
            try {
                return entity != null ? EntityUtils.toString(entity) : null;
            } catch (final ParseException ex) {
                throw new ClientProtocolException(ex);
            }
        } else {
            throw new ClientProtocolException("Unexpected response status: " + status);
        }
    };
    final HttpGet httpGet2 = new HttpGet(url2);

    String responseBody2 = httpclient.execute(httpGet2, responseHandler);
    System.out.println(responseBody2);

    final String responseBody = httpclient.execute(httpGet, responseHandler);
    System.out.println(responseBody);

    responseBody2 = httpclient.execute(httpGet2, responseHandler);
    System.out.println(responseBody2);
}

还可以访问通过本地上下文绑定 cookie，从而获取cookie 信息

String url = "http://httpbin.org/cookies/set/foo/bar";

try (final CloseableHttpClient httpclient = HttpClients.createDefault()) {
    // 创建一个本地的 Cookie 存储
    final CookieStore cookieStore = new BasicCookieStore();

    final HttpClientContext localContext = HttpClientContext.create();
    // 绑定 cookieStore 到 localContext
    localContext.setCookieStore(cookieStore);

    final HttpGet httpget = new HttpGet(url);

    final HttpClientResponseHandler<String> responseHandler = response -> {
        final int status = response.getCode();

        if (status >= HttpStatus.SC_SUCCESS && status < HttpStatus.SC_REDIRECTION) {
            final HttpEntity entity = response.getEntity();
            try {
                final List<Cookie> cookies = cookieStore.getCookies();
                for (Cookie cookie : cookies) {
                    System.out.println("Local cookie: " + cookie);
                }
                return entity != null ? EntityUtils.toString(entity) : null;
            } catch (final ParseException ex) {
                throw new ClientProtocolException(ex);
            }
        } else {
            throw new ClientProtocolException("Unexpected response status: " + status);
        }
    };

    String response = httpclient.execute(httpget, localContext, responseHandler);
    System.out.println(response);
}

2.3 拦截器

httpclient 支持通过拦截器对请求进行一定的处理，有如下几个方法添加拦截器

  • addRequestInterceptorFirst
  • addRequestInterceptorLast
  • addResponseInterceptorFirst
  • addResponseInterceptorLast
  • addExecInterceptorFirst
  • addExecInterceptorLast
  • addExecInterceptorBefore
  • addExecInterceptorAfter

添加的拦截器可分为3种类型: request, response和 exec，对应请求，响应和执行。其中Exec执行的名字在枚举ChainElement 中，在 HttpClientBuilder 类源码中，可以发现除了 CACHING 其它都可以通过配置使用，并且枚举中的顺序也是Exec执行的顺序，其中 MAIN_TRANSPORT 执行包含 request 和 response 拦截器执行

ChainElement 定义了一组可用于构建HTTP请求处理管道的元素，每个元素都可以实现特定的功能，如添加自定义HTTP头、添加身份验证信息等。

public enum ChainElement {
    REDIRECT, COMPRESS, BACK_OFF, RETRY, CACHING, PROTOCOL, CONNECT, MAIN_TRANSPORT
}

下面是一个对官方拦截器例子修改的代码

AtomicLong count = new AtomicLong();

try (final CloseableHttpClient httpclient = HttpClients.custom()
     .addExecInterceptorAfter(ChainElement.PROTOCOL.name(), "custom", (request, scope, chain) -> {
         request.setHeader("request-id", Long.toString(count.incrementAndGet()));
         return chain.proceed(request, scope);
     })
     .addExecInterceptorAfter("custom", "quit3rd", ((request, scope, chain) -> {
         final Header idHeader = request.getFirstHeader("request-id");
         if (idHeader != null && "3".equalsIgnoreCase(idHeader.getValue())) {
             final ClassicHttpResponse response = new BasicClassicHttpResponse(HttpStatus.SC_NOT_FOUND, "Oppsie");
             response.setEntity(new StringEntity("bad luck", ContentType.TEXT_PLAIN));
             return response;
         } else {
             return chain.proceed(request, scope);
         }
     }))

     .addExecInterceptorBefore(ChainElement.CONNECT.name(), "AAA", (request, scope, chain) -> {
         System.out.println("AAA");
         return chain.proceed(request, scope);
     })
     .addExecInterceptorBefore("AAA", "BBB", (request, scope, chain) -> {
         System.out.println("BBB");
         return chain.proceed(request, scope);
     })
     .addExecInterceptorAfter("AAA", "CCC", (request, scope, chain) -> {
         System.out.println("CCC");
         return chain.proceed(request, scope);
     })

     .addRequestInterceptorFirst((request, entity, context) -> {
         System.out.println("第一个request first现在获取：" + context.getAttribute("foo"));
     })
     .addRequestInterceptorFirst((request, entity, context) -> {
         System.out.println("第二个request first, 现在设置name");
         context.setAttribute("foo", "bar");
     })
     .addRequestInterceptorLast((request, entity, context) -> {
         System.out.println("第一个request last现在获取：" + context.getAttribute("foo"));
     })

     .build()) {


    for (int i = 0; i < 5; i++) {
        final HttpGet httpget = new HttpGet("http://httpbin.org/get");

        System.out.println("Executing request " + httpget.getMethod() + " " + httpget.getUri());

        httpclient.execute(httpget, response -> {
            System.out.println("----------------------------------------");
            System.out.println(httpget + "->" + new StatusLine(response));
            EntityUtils.consume(response.getEntity());
            return null;
        });
    }
}

下面动图显示的是调试过程中 execChain 执行链的顺序

内置拦截器

下面是 调试过程中的request和response 拦截器，从名字就可以看出除了Main类是自定义的拦截器，其余都是自带的，其中cookie处理也是通过拦截器实现的。

请求响应拦截器

2.4 fluent API

HttpClienet 4.5 版本以上支持fluent API, 优点是代码更简洁，同时为线程安全的。

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>fluent-hc</artifactId>
    <version>4.5.13</version>
</dependency>

String urlGet = "http://httpbin.org/get";
String urlPost = "http://httpbin.org/post";

String response = Request.Get(urlGet)
    .addHeader("Authorization", "Bear:dw")
    .execute()
    .handleResponse(httpResponse -> {
        int code = httpResponse.getStatusLine().getStatusCode();
        if (code == HttpStatus.SC_SUCCESS) {
            return org.apache.http.util.EntityUtils.toString(httpResponse.getEntity());
        }
        return null;
    });

System.out.println(response);

String result = Request.Post(urlPost)
    .bodyForm(Form.form().add("foo", "bar").build())
    .execute()
    .returnContent()
    .asString();

System.out.println(result);

三、3.1旧版本使用

3.1 Get 请求

String url = "http://httpbin.com";
HttpClient client = new HttpClient();
GetMethod method = new GetMethod(url);
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
                                new DefaultHttpMethodRetryHandler(3, false));
try {
    // Execute the method.
    int statusCode = client.executeMethod(method);

    if (statusCode != HttpStatus.SC_OK) {
        System.err.println("Method failed: " + method.getStatusLine());
    }

    // Read the response body.
    byte[] responseBody = method.getResponseBody();

    // Deal with the response.
    // Use caution: ensure correct character encoding and is not binary data
    System.out.println(new String(responseBody));

} catch (HttpException e) {
    System.err.println("Fatal protocol violation: " + e.getMessage());
    e.printStackTrace();
} catch (IOException e) {
    System.err.println("Fatal transport error: " + e.getMessage());
    e.printStackTrace();
} finally {
    // Release the connection.
    method.releaseConnection();
}

3.2 Post 请求

String url = "http://httpbin.org/post";
HttpClient client = new HttpClient();
PostMethod method = new PostMethod(url);
NameValuePair[] data = {
    new NameValuePair("user", "joe"),
    new NameValuePair("password", "bloggs")
};
method.setRequestBody(data);

try {
    int statusCode = client.executeMethod(method);
    if (statusCode != HttpStatus.SC_OK) {
        System.err.println("Method failed: " + method.getStatusLine());
    }
    byte[] responseBody = method.getResponseBody();
    System.out.println(new String(responseBody));
} catch (HttpException e) {
    System.err.println("Fatal protocol violation: " + e.getMessage());
    e.printStackTrace();
} catch (IOException e) {
    System.err.println("Fatal transport error: " + e.getMessage());
    e.printStackTrace();
} finally {
    method.releaseConnection();
}

四、异步版本使用

4.1 基本请求

final IOReactorConfig ioReactorConfig = IOReactorConfig.custom()
                .setSoTimeout(Timeout.ofSeconds(5))
                .build();

final CloseableHttpAsyncClient client = HttpAsyncClients.custom()
    .setIOReactorConfig(ioReactorConfig)
    .build();

client.start();

final HttpHost target = new HttpHost("httpbin.org");
final String[] requestUris = new String[] {"/", "/ip", "/user-agent", "/headers"};

for (final String requestUri: requestUris) {
    final SimpleHttpRequest request = SimpleRequestBuilder.get()
        .setHttpHost(target)
        .setPath(requestUri)
        .build();

    System.out.println("请求url:" + requestUri);
    final Future<SimpleHttpResponse> future = client.execute(
        SimpleRequestProducer.create(request),
        SimpleResponseConsumer.create(),
        new FutureCallback<SimpleHttpResponse>() {

            @Override
            public void completed(final SimpleHttpResponse response) {
                System.out.println(requestUri + " 返回状态码：" + response.getCode() + ",返回内容:" + response.getBodyText());
            }

            @Override
            public void failed(final Exception ex) {
                System.out.println(request + "->" + ex);
            }

            @Override
            public void cancelled() {
                System.out.println(request + " cancelled");
            }

        });
    future.get();
}

System.out.println("Shutting down");
client.close(CloseMode.GRACEFUL);

4.2 请求流水线执行

final MinimalHttpAsyncClient client = HttpAsyncClients.createMinimal(
    H2Config.DEFAULT,
    Http1Config.DEFAULT,
    IOReactorConfig.DEFAULT,
    PoolingAsyncClientConnectionManagerBuilder.create()
    .setDefaultTlsConfig(TlsConfig.custom()
                         .setVersionPolicy(HttpVersionPolicy.FORCE_HTTP_1)
                         .build())
    .build());

client.start();

final HttpHost target = new HttpHost("httpbin.org");
final Future<AsyncClientEndpoint> leaseFuture = client.lease(target, null);
final AsyncClientEndpoint endpoint = leaseFuture.get(30, TimeUnit.SECONDS);
try {
    final String[] requestUris = new String[] {"/", "/ip", "/user-agent", "/headers"};

    final CountDownLatch latch = new CountDownLatch(requestUris.length);
    for (final String requestUri: requestUris) {
        final SimpleHttpRequest request = SimpleRequestBuilder.get()
            .setHttpHost(target)
            .setPath(requestUri)
            .build();

        System.out.println("Executing request " + request);
        endpoint.execute(
            SimpleRequestProducer.create(request),
            SimpleResponseConsumer.create(),
            new FutureCallback<SimpleHttpResponse>() {

                @Override
                public void completed(final SimpleHttpResponse response) {
                    latch.countDown();
                    System.out.println(request + "->" + new StatusLine(response));
                    System.out.println(response.getBody());
                }

                @Override
                public void failed(final Exception ex) {
                    latch.countDown();
                    System.out.println(request + "->" + ex);
                }

                @Override
                public void cancelled() {
                    latch.countDown();
                    System.out.println(request + " cancelled");
                }

            });
    }
    latch.await();
} finally {
    endpoint.releaseAndReuse();
}

System.out.println("Shutting down");
client.close(CloseMode.GRACEFUL);

参考

  1. https://hc.apache.org/httpclient-legacy/index.html
# java # 开源 # apache # http
Logo
AtomGit 开源协作平台测评赛

瓜分20万奖金 获得内推名额 丰厚实物奖励 易参与易上手

更多推荐

白盒测试体系的探索 [ 光影人像 东海陈光剑 的博客 ]

什么是白盒测试？很多人都听说过白盒测试。通常的说法是，白盒测试是能看到全部产品源代码的测试。常常，白盒测试都是和牛人绑在一起的。个人认为这是一种比较狭隘的说法。然而究竟什么是白盒测试呢？可能有很多的人在做了很长时间的白盒测试以后发现，自己其实不是在做白盒测试，而是在做灰盒测试，原因是不够“白”，因为他没有看到全部的产品代码。其实，我个人认为，这是做白盒测试的误区。从广义来说，个人认为，白...

avatar 开放原子开发者工作坊

现代程序设计 homework-01

搞了6个小时individual project...看看博客做一做第一次现代程序设计作业1) 建立 GitHub 账户, 把课上做的 “最大子数组之和” 程序签入我的github地址是https://github.com/oldoldb,以前没有用过各种不熟练啊....代码我放到<现代程序设计课后作业>这个repository里了,是昨天完成的在hdu和poj上找的5道关于一...

avatar 开放原子开发者工作坊

精心收集的 48 个 JavaScript 代码片段，仅需 30 秒就可理解

原文：Chalarangelo 译文：IT168https://github.com/Chalarangelo/30-seconds-of-code#anagrams-of-string-with-duplicates该项目来自于 Github 用户 Chalarangelo，目前已在 Github 上获得了 5000 多Star，精心收集了多达 48 个有用的 JavaS...

avatar 开放原子开发者工作坊