[ Java 常見問題 ] 只處理 "\r\n" 斷行的類別 JBReader

前言 :
最近工作中遇到的鳥問題, 一般我們在處理文字檔案時, 會慣用 BufferedReader 類別上的 readLine() 來幫我們一次讀取一行來進行處理. 而當我們在處理檔案同時具有 "\r", "\n", "\r\n" 時, 到底哪個會被 BufferedReader 視為斷行的字元呢? 看看 JDK 上 readLine() 的說明 :

Read a line of text. A line is considered to be terminated by any one of a line feed ('\n'), a carriage return ('\r'), or a carriage return followed immediately by a linefeed.

說白了就是都會! 但其實我只希望使用 "\r\n" 當作斷行符號怎麼辦呢? 底下我們將一步步說明並實作一個 Wrapper 將 InputStream 進行封裝, 只對 "\r\n" 做斷行處理.

前置工作 :
在一開始, 我們要來寫一些簡單的代碼幫我們產生測試文件. 這邊的測試文件使用的編碼為 "Big5" :

view plaincopy to clipboardprint?

1. package test;  
2.   
3. import java.io.File;  
4. import java.io.FileOutputStream;  
5.   
6. public class GenTestFile {  
7.     public static void main(String[] args) throws Exception{  
8.         String encoding = "big5";  
9.         File outf = new File("test.txt");  
10.         FileOutputStream fos = new FileOutputStream(outf);  
11.         fos.write("Line1 with \\r!\r".getBytes(encoding));  
12.         fos.write("Line2 with \\r\\n!\r\n".getBytes(encoding));  
13.         fos.write("Line3 with \\n!\n".getBytes(encoding));  
14.         fos.write("Line4".getBytes(encoding));  
15.         fos.close();  
16.     }  
17. }  

執行後應該會在執行目錄下產生 "test.txt" 檔案, 用 UltraEdit 打開後使用 Hex mode 觀察 :


可以觀察到我們產生了一個使用不同 "\r" (0x0d) 與 "\n" (0x0a) 組合當作斷行的範例. 底下使用我們慣用的方法來讀取, 看看 Java 的 BufferedReader 如何斷行 :

view plaincopy to clipboardprint?

1. public static void read() throws Exception  
2. {  
3.     String encoding = "big5";  
4.     File outf = new File("test.txt");  
5.     BufferedReader br = new BufferedReader(new InputStreamReader(new FileInputStream(outf), encoding));  
6.     String line = null;  
7.     while((line=br.readLine())!=null)  
8.     {  
9.         System.out.printf("\t[Info] Read: %s\n", line);  
10.     }  
11.     br.close();  
12. }  

執行該方法後, 可以發現每一種斷行組合都可以成功斷行 :

[Info] Read: Line1 with \r!
[Info] Read: Line2 with \r\n!
[Info] Read: Line3 with \n!
[Info] Read: Line4

只處理 "\r\n" 斷行的類別 JBReader :
但我們的需求是只對 "\r\n" 斷行, 實際上 Windows 使用的斷行符號是 "\r\n" ; Linux 使用的斷行符號是 "\n" ; Mac 使用的斷行符號是 "\r", 這是為什麼 BufferedReader 會將這些組合都視為斷行的原因, 為了跨平台摟. 因此我們要嗎繼承 BufferedReader 並改寫它上面的方法(s), 或是根據自己需求撰寫一個全新的類別. 這邊採用後面的方法. 首先使用了設計模式中的 Decorator Pattern (裝飾模式 - 動態增加類別功能), 我們設計一個類別 JBReader 並於建構子中傳入 InputStream 物件, 並撰寫方法 readLine() 將檔案的 binary 使用定義好的 buffer (固定 buffer size, 避免一次一個 byte 讀出造成過多的 IO read overhead)讀出來, 當遇到 "\r\n" (0x0d0a) 便將之前讀出來的 binary 使用 String 上的建構子 String(byte[] bytes, String charsetName) 建立字串物件並返回.

完整代碼如下 :
- JBReader.java :

view plaincopy to clipboardprint?

1. package flib.util.io;  
2.   
3. import java.io.File;  
4. import java.io.FileInputStream;  
5. import java.io.IOException;  
6. import java.io.InputStream;  
7. import java.util.ArrayList;  
8. import java.util.LinkedList;  
9. import java.util.List;  
10.   
11. public class JBReader {  
12.     public static String    ENCODING = "big5";  
13.     public static String    OUT_ENCODING = null;  
14.     public static int       BUFFER_SIZE = 1024;   
15.     public static boolean   REMOVE_END_SPECIAL_CHAR = true; // Check '\r' and '\n' only.  
16.       
17.     private InputStream     is = null;  
18.     private byte            buffer[] = null;  
19.     private byte            prevByte = -1;  
20.     private int             prevLastFrom=-1, prevLastEnd=-1;  
21.     private List        strBuf = new ArrayList();     
22.       
23.     public JBReader(){}  
24.     public JBReader(InputStream is) {  
25.         this.is = is;  
26.         buffer = new byte[BUFFER_SIZE];  
27.     }  
28.       
29.     public JBReader(InputStream is, String encoding)  
30.     {  
31.         this(is);  
32.         this.ENCODING = encoding;  
33.     }  
34.   
35.     public void assignIS(InputStream is) throws IOException  
36.     {  
37.         assignIS(is, false);  
38.     }  
39.       
40.     public void assignIS(InputStream is, boolean reallocBuffer) throws IOException  
41.     {  
42.         close();  
43.         this.is = is;  
44.         if(reallocBuffer || buffer == null) buffer = new byte[BUFFER_SIZE];  
45.     }  
46.       
47.     public void close() throws IOException{  
48.         if(is!=null) {  
49.             is.close();   
50.             buffer = null;  
51.         }         
52.         strBuf.clear();  
53.         prevByte = -1;  
54.         prevLastFrom = prevLastEnd = -1;  
55.     }  
56.       
57.     public byte[] readLineInBinary() throws IOException  
58.     {  
59.         if(is!=null)  
60.         {  
61.             strBuf.clear();  
62.             // Restore last binary from previous loop  
63.             if(prevLastFrom>0)  
64.             {  
65.                 for(int i=prevLastFrom; i
66.                 prevLastFrom=prevLastEnd=-1;  
67.             }  
68.               
69.             int rs = -1;  
70.             boolean isNewLine = false;  
71.               
72.             // Loop to read binary data  
73.             while((rs = is.read(buffer)) > 0)  
74.             {  
75.                 for(int i=0; i
76.                 {  
77.                     strBuf.add(buffer[i]);  
78.                     if(prevByte == (byte)0x0d && buffer[i] == (byte)0x0a)  
79.                     {  
80.                         isNewLine = true;  
81.                         prevByte = -1;  
82.                         if(i+1< rs) {  
83.                             prevLastFrom=i+1;  
84.                             prevLastEnd=rs;  
85.                         }  
86.                         break;  
87.                     }  
88.                     else  
89.                     {                         
90.                         prevByte = buffer[i];  
91.                     }  
92.                 }  
93.                 if(isNewLine) break;  
94.             }  
95.             if(rs<0) return null;  
96.               
97.             // Process returned string  
98.             byte binarys[] = new byte[strBuf.size()];  
99.             for(int i=0; i
100.             return binarys;  
101.         }  
102.         return null;  
103.     }  
104.       
105.     protected String processRtnStr() throws IOException  
106.     {  
107.         byte binarys[] = null;        
108.         if(REMOVE_END_SPECIAL_CHAR)  
109.         {  
110.             List scIdxList = new LinkedList();  
111.             for(int i=0; i
112.             {  
113.                 byte cb = strBuf.get(i);   
114.                 if(cb == (byte)0x0d || cb == (byte)0x0a) scIdxList.add(i);                
115.             }  
116.             binarys = new byte[strBuf.size()-scIdxList.size()];  
117.             int j=0;  
118.             for(int i=0; i
119.                 if(!scIdxList.contains(i))  
120.                 {  
121.                     binarys[j++] = strBuf.get(i);  
122.                 }  
123.                 else  
124.                 {  
125.                     scIdxList.remove(new Integer(i));  
126.                 }  
127.         }  
128.         else  
129.         {  
130.             binarys = new byte[strBuf.size()];  
131.             for(int i=0; i
132.         }  
133.           
134.         if(OUT_ENCODING==null)  
135.         {  
136.             return new String(binarys, ENCODING);  
137.         }  
138.         else  
139.         {  
140.             return new String(new String(binarys, ENCODING).getBytes(OUT_ENCODING));  
141.         }  
142.     }  
143.           
144.     public String readLine() throws IOException  
145.     {  
146.         if(is!=null)  
147.         {  
148.             strBuf.clear();  
149.             // Restore last binary from previous loop  
150.             if(prevLastFrom>0)  
151.             {                 
152.                 boolean isNewLine = false;  
153.                 for(int i=prevLastFrom; i
154.                 {  
155.                     strBuf.add(buffer[i]);  
156.                     if(prevByte == (byte)0x0d && buffer[i] == (byte)0x0a)  
157.                     {                         
158.                         isNewLine = true;  
159.                         prevByte = -1;  
160.                         if(i+1< prevLastEnd) {  
161.                             prevLastFrom=i+1;                             
162.                         } else {  
163.                             prevLastFrom=prevLastEnd=-1;  
164.                         }  
165.                         break;  
166.                     }  
167.                     else  
168.                     {                         
169.                         prevByte = buffer[i];  
170.                     }  
171.                 }  
172.                 if(isNewLine)  
173.                 {  
174.                     return processRtnStr();  
175.                 }  
176.                 prevLastFrom=prevLastEnd=-1;  
177.             }  
178.               
179.             int rs = -1;  
180.             boolean isNewLine = false;  
181.               
182.             // Loop to read binary data  
183.             while((rs = is.read(buffer)) > 0)  
184.             {  
185.                 for(int i=0; i
186.                 {  
187.                     strBuf.add(buffer[i]);  
188.                     if(prevByte == (byte)0x0d && buffer[i] == (byte)0x0a)  
189.                     {  
190.                         isNewLine = true;  
191.                         prevByte = -1;  
192.                         if(i+1< rs)   
193.                         {  
194.                             prevLastFrom=i+1;  
195.                             prevLastEnd=rs;  
196.                         }  
197.                         break;  
198.                     }  
199.                     else  
200.                     {                         
201.                         prevByte = buffer[i];  
202.                     }  
203.                 }  
204.                 if(isNewLine) break;  
205.             }  
206.               
207.             if(rs<0 && strBuf.size()==0)   
208.             {  
209.                 //System.out.printf("\t[Test] EOF!");  
210.                 return null;  
211.             }  
212.               
213.             // Process returned string  
214.             return processRtnStr();  
215.         }  
216.         return null;  
217.     }  
218. }  

接著你便可以如下使用該類別讀出剛剛我們產生的測試檔案 :

view plaincopy to clipboardprint?

1. public static void main(String[] args) throws Exception{  
2.     File testf = new File("test.txt");  
3.     FileInputStream fis = new FileInputStream(testf);  
4.     //BufferedReader br = new BufferedReader(new InputStreamReader(fis, ENCODING));  
5.     JBReader br = new JBReader(fis);  
6.     String line = null; int lc = 0;  
7.     while((line=br.readLine())!=null)  
8.     {  
9.         lc++;  
10.         System.out.printf("\t[Info] Read : %s\n", line.trim());  
11.     }  
12.     System.out.printf("\t[Info] Total %d line(s).\n", lc);  
13.     br.close();  
14. }  

讀出結果可以發現, 只有 "\r\n" 會被視為斷行 :

[Info] Read : Line1 with \r!Line2 with \r\n!
[Info] Read : Line3 with \n!Line4
[Info] Total 2 line(s).

This message was edited 19 times. Last update was at 21/08/2012 10:15:45