程式扎記

來源自這裡
Preface:
在一次服務器異常的排查過程當中，我們決定使用HttpClient4.X替代HttpClient3.X或者HttpConnection。為什麼使用HttpClient4？主要是HttpConnection沒有連接池的概念，多少次請求就會建立多少個IO，在訪問量巨大的情況下服務器的IO可能會耗盡。HttpClient3 也有連接池的東西在裡頭，使用 MultiThreadedHttpConnectionManager，大致過程如下：

view plaincopy to clipboardprint?
MultiThreadedHttpConnectionManager connectionManager =  new  MultiThreadedHttpConnectionManager();    
HttpClient client =  new  HttpClient(connectionManager);... //在某個線程中。    
GetMethod get =  new  GetMethod( "http://jakarta.apache.org/" );    
try  {    
    client.executeMethod(get); // print response to stdout    
    System.out.println(get.getResponseBodyAsStream());    
}  finally  {    
    // be sure the connection is released back to the connection     
    managerget.releaseConnection();    
}   

可以看出來，它的方式與jdbc連接池的使用方式相近，這裡比較麻煩的就是需要手動調用 releaseConnection 去釋放連接。對每一個 HttpClient.executeMethod 須有一個 method.releaseConnection() 與之匹配!

HttpClient4.x 連接池:
這裡使用4.2.x的版本實現了一個簡單的 HttpConnectionManager，代碼如下:

view plaincopy to clipboardprint?
package nlg.utils;  
  
import org.apache.http.client.HttpClient;  
import org.apache.http.impl.client.DefaultHttpClient;  
import org.apache.http.impl.conn.PoolingClientConnectionManager;  
import org.apache.http.params.BasicHttpParams;  
import org.apache.http.params.HttpConnectionParams;  
import org.apache.http.params.HttpParams;  
  
public class HttpConnectionManager {  
    private static HttpParams httpParams;  
    private static PoolingClientConnectionManager connectionManager;  
  
    /** 
     * 最大连接数 
     */  
    public final static int MAX_TOTAL_CONNECTIONS = 2000;  
  
    /** 
     * 每个路由最大连接数 
     */  
    public final static int MAX_ROUTE_CONNECTIONS = 400;  
    /** 
     * 连接超时时间 
     */  
    public final static int CONNECT_TIMEOUT = 10000;  
    /** 
     * 读取超时时间 
     */  
    public final static int READ_TIMEOUT = 10000;  
  
    static {          
        httpParams = new BasicHttpParams();   
        HttpConnectionParams.setConnectionTimeout(httpParams, CONNECT_TIMEOUT);  
        HttpConnectionParams.setSoTimeout(httpParams, READ_TIMEOUT);          
          
        connectionManager = new PoolingClientConnectionManager();  
        connectionManager.setMaxTotal(MAX_TOTAL_CONNECTIONS);  
        connectionManager.setDefaultMaxPerRoute(MAX_ROUTE_CONNECTIONS);  
    }  
  
    public static HttpClient GetHttpClient() {  
        return new DefaultHttpClient(connectionManager, httpParams);  
    }  
  
    public static synchronized void Shutdown()  
    {  
        if(DHC!=null)  
        {  
            DHC.getConnectionManager().shutdown();  
            DHC = null;  
        }  
    }  
}  

接著便可以如下使用上面建立的類別. 另外 HttpClient4 使用我們常用的 InputStream.close() 來確認連接關閉（4.1版本之前使用entity.consumeContent()來確認內容已經被消耗關閉連接）:

view plaincopy to clipboardprint?
package nlg.test;  
  
import nlg.utils.HttpConnectionManager;  
  
import org.apache.http.HttpEntity;  
import org.apache.http.HttpResponse;  
import org.apache.http.client.HttpClient;  
import org.apache.http.client.methods.HttpGet;  
import org.apache.http.protocol.BasicHttpContext;  
import org.apache.http.protocol.HttpContext;  
import org.apache.http.util.EntityUtils;  
  
public class ClientMultiThreadedEx {  
    public static void main(String args[]) throws Exception  
    {  
        try {  
            // create an array of URIs to perform GETs on  
            String[] urisToGet = {  
                "http://hc.apache.org/",  
                "http://hc.apache.org/httpcomponents-core-ga/",  
                "http://hc.apache.org/httpcomponents-client-ga/",  
                "http://svn.apache.org/viewvc/httpcomponents/"  
            };  
  
            // create a thread for each URI  
            GetThread[] threads = new GetThread[urisToGet.length];  
            for (int i = 0; i < threads.length; i++) {  
                HttpGet httpget = new HttpGet(urisToGet[i]);  
                threads[i] = new GetThread(HttpConnectionManager.getHttpClient(), httpget, i + 1);  
            }  
  
            // start the threads  
            for (int j = 0; j < threads.length; j++) {  
                threads[j].start();  
            }  
  
            // join the threads  
            for (int j = 0; j < threads.length; j++) {  
                threads[j].join();  
            }  
  
        } finally {  
            // When HttpClient instance is no longer needed,  
            // shut down the connection manager to ensure  
            // immediate deallocation of all system resources  
            HttpConnectionManager.Shutdown();  
        }  
    }  
      
      
    /** 
     * A thread that performs a GET. 
     */  
    static class GetThread extends Thread {  
  
        private final HttpClient httpClient;  
        private final HttpContext context;  
        private final HttpGet httpget;  
        private final int id;  
  
        public GetThread(HttpClient httpClient, HttpGet httpget, int id) {  
            this.httpClient = httpClient;  
            this.context = new BasicHttpContext();  
            this.httpget = httpget;  
            this.id = id;  
        }  
  
        /** 
         * Executes the GetMethod and prints some status information. 
         */  
        @Override  
        public void run() {  
  
            System.out.println(id + " - about to get something from " + httpget.getURI());  
  
            try {  
  
                // execute the method  
                HttpResponse response = httpClient.execute(httpget, context);  
  
                System.out.println(id + " - get executed");  
                // get the response body as an array of bytes  
                HttpEntity entity = response.getEntity();  
                if (entity != null) {  
                    byte[] bytes = EntityUtils.toByteArray(entity);  
                    System.out.println(id + " - " + bytes.length + " bytes read");  
                }  
  
            } catch (Exception e) {  
                httpget.abort();  
                System.out.println(id + " - error: " + e);  
            }  
        }  
  
    }  
}  

Supplement:
* HttpClient 3.1 Threading issue
* Class PoolingClientConnectionManager

Manages a pool of client connections and is able to service connection requests from multiple execution threads. Connections are pooled on a per route basis. A request for a route which already the manager has persistent connections for available in the pool will be services by leasing a connection from the pool rather than creating a brand new connection.

This message was edited 9 times. Last update was at 14/07/2013 16:08:34

程式扎記

標籤

2013年7月14日星期日

[ Java 文章收集 ] HttpClient4.2.X 升級入門+ http連接池使用

沒有留言:

張貼留言

[Git 常見問題] error: The following untracked working tree files would be overwritten by merge

檢舉濫用情形

學習筆記

標籤

2013年7月14日 星期日