直接使用 HttpClient 进行 https 请求，会由于证书问题导致请求失败，既然我们想利用程序访问某个网站(比如做爬虫)，其实我们对证书并不关注，可以采用忽略证书校验的方式来实现对 https 请求的访问。
具体实现流程
-
依赖的 HttpClient jar 包版本
<dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.3</version> </dependency> 封装获取 HttpClient 的方法
public static CloseableHttpClient getHttpClient() {
try {
SSLContextBuilder builder = new SSLContextBuilder();
builder.loadTrustMaterial(null, new TrustSelfSignedStrategy());
//不进行主机名验证
SSLConnectionSocketFactory sslConnectionSocketFactory = new SSLConnectionSocketFactory(builder.build(),
NoopHostnameVerifier.INSTANCE);
Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory> create()
.register("http", new PlainConnectionSocketFactory())
.register("https", sslConnectionSocketFactory)
.build();
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(registry);
cm.setMaxTotal(100);
CloseableHttpClient httpclient = HttpClients.custom()
.setSSLSocketFactory(sslConnectionSocketFactory)
.setDefaultCookieStore(new BasicCookieStore())
.setConnectionManager(cm).build();
return httpclient;
} catch (KeyManagementException e) {
e.printStackTrace();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (KeyStoreException e) {
e.printStackTrace();
}
return HttpClients.createDefault();
}
- 创建一个利用 HttpClient 发送 get 请求的工具方法
public static String get(String uri, Map<String, Object> params, Map<String, String> headers) throws IOException {
HttpClient client = getHttpClient();
return get(client, uri, params, headers);
}
public static String get(HttpClient client, String uri, Map<String, Object> params, Map<String, String> headers) throws IOException {
String result = StringUtils.EMPTY;
String fullUrl = buildUrlWithParams(uri, params);
HttpGet httpGet = new HttpGet(fullUrl);
if (headers != null) {
for (Entry<String, String> entry : headers.entrySet()) {
httpGet.addHeader(entry.getKey(), entry.getValue());
}
}
HttpResponse httpResponse = client.execute(httpGet);
InputStream input = httpResponse.getEntity().getContent();
if (null != input) {
try {
result = IOUtils.toString(input, "UTF-8");
} catch (IOException e) {
throw e;
} finally {
IOUtils.closeQuietly(input);
}
}
return result;
}
private static String buildUrlWithParams(String uri, Map<String, Object> params) throws UnsupportedEncodingException {
StringBuilder urlBuilder = new StringBuilder(uri);
if (null != params && !params.isEmpty()) {
if (!uri.contains("?")) {
urlBuilder.append("?");
}
for (Map.Entry<String, Object> entry : params.entrySet()) {
String key = entry.getKey();
Object value = entry.getValue();
String valueStr = null == value ? "" : value.toString();
if (!urlBuilder.toString().endsWith("?")) {
urlBuilder.append("&");
}
urlBuilder.append(key).append("=").append(URLEncoder.encode(valueStr, "utf-8"));
}
}
String fullUrl = urlBuilder.toString();
return fullUrl;
}
- 测试代码
public class HttpsReqTest extends TestCase {
public void reqHttpsUrlTest() throws IOException {
String url = "https://www.xxx.com/";
String result = get(url, null, null);
System.out.println(result);
}
}
问题总结
上面这段代码其实已经可以满足大部分 https 地址的请求，本来我也以为高枕无忧了，但是昨天我请求一个 https 地址的时候(地址就不暴露了)，又发现了新的问题，刚好补充下，后续有其他问题也会继续补充。
问题一
- 问题描述
Exception in thread "main" javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1446)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:209)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:901)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:837)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1023)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1343)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
Caused by: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints
at sun.security.ssl.AbstractTrustManagerWrapper.checkAlgorithmConstraints(SSLContextImpl.java:1018)
at sun.security.ssl.AbstractTrustManagerWrapper.checkAdditionalTrust(SSLContextImpl.java:944)
at sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:886)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1428)
... 26 more
-
问题原因
原因是 Java 7/8 版本增加了安全性
解决方案
把 %JAVA_HOME%/jre/lib/security/java.security 文件里的
jdk.certpath.disabledAlgorithms=MD2, DSA, RSA keySize < 1024
改成
jdk.certpath.disabledAlgorithms=
就是去掉 "MD2, DSA, RSA keySize < 1024"。
用 notepad++ 修改的时候会提示文件被占用，可以用管理员运行 notepad++ 在进行修改即可。
