程式扎記

來源自這裡
Preface:
因為我的專案有用到 CRF (Conditional random fields), 因此就去拜 Google 大神找找看有沒有好用的 Toolkit. 如果你是 C 的專家, 那你可以考慮 Wapiti , 或者是 CRF++. 這邊要介紹的是使用 Java 實作 CRF 的另一個選擇. 它是由 Prof. Sunita Sarawagi 開發並且可以在這裡得到更多該工具的使用說明與介紹.

有關 CRF 的介紹這邊我也不多說, 有興趣的就去看看 wiki, 而它一個很常見的運用便是在句子 POS (Part of speech) 的偵測: 當給你一連串的 word 的 sequence, 你能否根據 training 的 model 來預測每一個 word 的詞性 (名詞, 動詞或是形容詞 etc). 因此在使用這個工具時首先便需要知道如何將 sequence 的資料餵到工具裡, 並進行 training 然後 testing.

底下將一步步從 1) 資料的格式與餵入 2) Training 3) Testing 從新手的角度告訴你如何使用該工具.

Data Preparation and feeding:
因為我們輸入的資料需要是 sequence (序列) 的, 因此在該工具中提供了介面 DataSequence 讓我們來定義我們的資料格式並由它負責來解析輸入的資料. 這邊我們使用下面的訓練資料:

# R->Rail; S->Sun; C->Cloud
# 1->Play; 0->Stay Home;
R:0-S:1-S:1-S:0-C:0-R:0
S:0-R:0-R:0-S:1-S:0-C:0-C:0-C:1
C:0-C:1-C:0-S:0-S:1-C:1-R:0-R:0-S:1
R:0-S:1-S:0-C:1-C:0-R:0-R:0-R:1
R:1-C:0-C:0-S:1-S:0-S:0-C:1-R:0-C:0-R:0-R:0-C:1
R:0-R:0-C:0-S:1-S:0-S:0-S:1-C:0-C:0-C:1
S:1-R:0-R:0-S:1-S:0-C:0-R:0-R:0
R:0-R:0-R:0-C:1-S:0-C:1-S:1-C:0-C:0-R:0
C:0-S:1-S:0-S:0-C:1-R:0-R:0-R:0-C:1-S:1
R:0-R:0-C:0-S:1-S:0-S:0-S:1-C:0-C:0-C:1
S:0-S:1-C:0-C:0-R:0-R:0-C:0-C:1-C:0-S:1

每一行就是一個 sequence, 而 "S:0" 說明欲檢視的輸入資料為 "S" (Suny) 而對應該資料的 Label 為 "0" (Stay Home); 而每一個序列中的輸入資料都是以 "-" 當作分隔符. 而這邊我們建立類別DDataSequence 來實作介面 DataSequence 以讀入訓練或測試資料:

view plaincopy to clipboardprint?
package demo;  
  
import iitb.CRF.DataSequence;  
  
import java.util.ArrayList;  
import java.util.List;  
  
public class DDataSequence implements DataSequence{  
    private static final long serialVersionUID = 1L;  
    public int                  miss = 0;  
    private List         labelList; // List to keep label  
    private List      tokenList; // List to keep token  
  
    public DDataSequence(){}  
    public DDataSequence(String line){loadData(line);}  
      
    @Override  
    public int length() {  
        return labelList.size();  
    }  
  
    @Override  
    public void set_y(int idx, int value) {  
        labelList.set(idx, value);        
    }  
  
    @Override  
    public Object x(int idx) {  
        return tokenList.get(idx);  
    }  
  
    @Override  
    public int y(int idx) {  
        return labelList.get(idx);  
    }  
  
    public int size(){return length();}  
      
    /** 
     * BD: We define how to load in data from file. 
     * @param line: File path 
     * @return 
     */  
    public boolean loadData(String line)  
    {         
        labelList = new ArrayList();  
        tokenList = new ArrayList();  
        String pairs[] = line.split("-");  
        try  
        {  
            for(String pair:pairs)  
            {  
                String ps[] = pair.split(":");  
                if(ps.length==2)  
                {  
                    tokenList.add(ps[0]);  
                    labelList.add(Integer.valueOf(ps[1]));  
                }  
                else {  
                    tokenList.add(ps[0]);  
                    labelList.add(-1);  
                }                 
            }  
        }  
        catch(Exception e){e.printStackTrace(); return false;}  
        return true;  
    }  
}  

接著多個 Sequence 可以包裝在介面 DataIter 裡面 (可以把一個 sequence 看做一個 training 或 testing 的 instance), 這邊我定義類別 DDataIter 來實作該介面:

view plaincopy to clipboardprint?
package demo;  
  
import java.util.List;  
import iitb.CRF.DataIter;  
import iitb.CRF.DataSequence;  
  
public class DDataIter implements DataIter{  
    public List datas = null;  
    public int pos = 0;  
  
    public DDataIter(List datas){this.datas = datas;}  
      
    @Override  
    public boolean hasNext() {  
        return pos < datas.size();  
    }  
  
    @Override  
    public DataSequence next() {  
        return datas.get(pos++);  
    }  
  
    @Override  
    public void startScan() {  
        pos = 0;          
    }  
  
    public int size(){return datas.size();}  
}  

到目前為止我們已經能讀入測試與訓練資料, 接著要處理的便是如何訓練與測試模型了

Handling Model:
同樣的資料, 你可能有多種模型, 但是模型的訓練與測試流程卻是一樣的, 因此下面我定義一個抽象類別 AbstModel 並使用它來實作我們要的模型:

view plaincopy to clipboardprint?
package demo.proto;  
  
import iitb.CRF.DataSequence;  
  
import java.io.BufferedReader;  
import java.io.File;  
import java.io.FileReader;  
import java.util.HashMap;  
  
public abstract class AbstModel {  
    public int          nlabels=5;  /*Number of label*/  
      
    /** 
     * BD : Allocate default feature. 
     */  
    public abstract void allocFeat();  
    /** 
     * BD : Allocate feature as param 'feat' with weight as param 'wt'. 
     * @param feat 
     * @param wt 
     */  
    public abstract void allocFeat(IFeature feat, double wt);  
    /** 
     * BD : Save trained statistic data into file with path as param 'fn'. 
     * @param fn 
     * @throws Exception 
     */  
    public abstract void saveStatisticData(String fn) throws Exception;  
    /** 
     * BD : Load trained statistic data from binary file with path as param 'filename'. 
     * @param filename 
     * @throws Exception 
     */  
    public abstract void loadStatisticData(String filename) throws Exception;  
    /** 
     * BD : Predict label for input sequence as param 'seq'. 
     * @param seq 
     * @return 
     */  
    public abstract int[] predictSeq(DataSequence seq);  
      
    /** 
     * BD : Predict label for input sequence as param 'seq'. 
     * @param seq 
     * @return 
     */  
    public abstract int[] predictSeq(DataSequence seq, PI pi);  
      
    /** 
     * BD : Train the model. Before doing this, please call API parseRawdata(). 
     * @throws Exception 
     */  
    public abstract void train() throws Exception;  
    /** 
     * BD : Load in the sequence data from file with path as param 'fn'. 
     * @param fn 
     */  
    public abstract void parseRawdata(String fn);  
      
    /** 
     * BD : Tagged file indicated by "testDataPath" and output to file indicated by "outputPath". 
     * @param testDataPath 
     * @param outputPath 
     * @throws Exception 
     */  
    public abstract PI validate(String testDataPath, String outputPath) throws Exception;  
      
    /** 
     * BD : Performance Index 
     * @author John 
     */  
    public class PI /*Performance Index*/  
    {  
        public HashMap> missTagStat = new HashMap>();  
        public int hit = 0;  
        public int miss = 0;  
        public int s_hit = 0;  
        public int s_miss = 0;  
        public int size(){return hit+miss;}  
        public int ssize(){return s_hit+s_miss;}  
        public void hit(){hit++;}  
        public void shit(){s_hit++;}  
        public void miss(){miss++;}  
        public void smiss(){s_miss++;}  
        public float hitRate() {return ((float)hit)/(hit+miss);}  
        public float shitRate() {return ((float)s_hit)/(s_hit+s_miss);}  
        public float missRate() {return 1-hitRate();}  
        public float smissRate() {return 1-shitRate();}  
        public void miss(int answer, int tagged){  
            miss++;  
            if(missTagStat.containsKey(answer)) {  
                HashMap mtagged = missTagStat.get(answer);  
                if(mtagged.containsKey(tagged)) mtagged.put(tagged, mtagged.get(tagged)+1);  
                else mtagged.put(tagged, 1);  
                missTagStat.put(answer, mtagged);  
            }  
            else  
            {  
                HashMap mtagged = new HashMap();  
                mtagged.put(tagged, 1);  
                missTagStat.put(answer, mtagged);  
            }  
        }  
    }         
}  

而一個實作的範例類別 DModel 如下:

view plaincopy to clipboardprint?
package demo;  
  
import iitb.CRF.CRF;  
import iitb.CRF.DataSequence;  
import iitb.Model.FeatureGenImpl;  
import iitb.Utils.Options;  
  
import java.io.BufferedReader;  
import java.io.BufferedWriter;  
import java.io.File;  
import java.io.FileReader;  
import java.io.FileWriter;  
import java.util.HashMap;  
import java.util.LinkedList;  
import java.util.List;  
import demo.proto.AbstModel;  
import demo.proto.IFeature;  
import flib.util.TimeStr;  
  
public class DModel extends AbstModel{  
    String              modelGraphType = "naive";  // "semi-markov" or "naive"  
    FeatureGenImpl      featureGen;  
    CRF                 crfModel;     
    Options             options;  
    DDataIter               dataIter = null;  
    public String       baseDir = "IRProj";  
    public String       outDir = "Test";  
  
    public DModel(Options o){this.options = o;};  
    public DModel()  
    {  
        this(new Options());  
    }  
      
    public void  allocModel() throws Exception {  
        featureGen = new FeatureGenImpl(modelGraphType, nlabels);  
        crfModel=new CRF(featureGen.numStates(),featureGen,options);  
    }  
      
    public void saveModel(String savePath) throws Exception  
    {  
        File bfdr = new File(savePath);  
        if(!bfdr.exists()) bfdr.mkdirs();  
        crfModel.write(new File(bfdr, "crf.txt").getAbsolutePath());  
        featureGen.write(new File(bfdr, "features.txt").getAbsolutePath());  
    }  
      
    public void loadModel(String path) throws Exception  
    {  
        allocModel();  
        File bfdr = new File(path);         
        crfModel.read(new File(bfdr, "crf.txt").getAbsolutePath());  
        featureGen.read(new File(bfdr, "features.txt").getAbsolutePath());  
    }  
  
    public void test(String testDataPath, String outputPath) throws Exception{        
        doTest(testDataPath, outputPath);  
    }  
      
    protected void outputTaggedData(BufferedWriter bw, DataSequence seq) throws Exception{  
        if(seq.length()>0)  
        {  
            bw.append(String.format("%s:%d", seq.x(0), seq.y(0)));  
            for(int i=1; i" %s:%d", seq.x(i), seq.y(i)));  
            bw.append("\r\n");  
        }  
    }  
      
    public void doTest(String testDataPath, String outputPath) throws Exception  
    {  
        BufferedWriter bw = new BufferedWriter(new FileWriter(new File(outputPath)));  
        parseTestData(testDataPath);  
        DataSequence seq = null;  
        int labels[] = null;  
        while(dataIter.hasNext())  
        {  
            seq = dataIter.next();  
            labels = predictSeq(seq);  
            outputTaggedData(bw, seq);  
        }  
        bw.close();  
    }  
  
      
    public void parseTestData(String fn)  
    {  
        List trainReds = new LinkedList();  
        try  
        {  
            File trainData = new File(fn);  
            BufferedReader br = new BufferedReader(new FileReader(trainData));  
            String line = null;  
            DDataSequence seq = null;  
            while((line=br.readLine())!=null)  
            {  
                if(line.trim().isEmpty()||line.trim().startsWith("#")) continue;  
                System.out.printf("\t[Test] Read line=%s\n", line);               
                seq = new DDataSequence();                
                seq.loadData(line);  
                if(seq.length()>0) trainReds.add(seq);  
            }  
              
            br.close();  
            dataIter = new DDataIter(trainReds);  
            System.out.printf("\t[Test] Total %d sequence data...\n", dataIter.size());  
        }  
        catch(Exception e)  
        {  
            e.printStackTrace();  
        }  
    }  
  
    @Override  
    public void allocFeat() {  
        // TODO Auto-generated method stub  
          
    }  
  
    @Override  
    public void allocFeat(IFeature feat, double wt) {  
        // TODO Auto-generated method stub  
          
    }  
  
    @Override  
    public void saveStatisticData(String fn) throws Exception {  
        // TODO Auto-generated method stub  
          
    }  
  
    @Override  
    public void loadStatisticData(String filename) throws Exception {  
        // TODO Auto-generated method stub  
          
    }  
  
    @Override  
    public int[] predictSeq(DataSequence seq) {  
        int labels[] = new int[seq.length()];  
        crfModel.apply(seq);  
        featureGen.mapStatesToLabels(seq);  
        for(int i=0; i
        return labels;  
    }  
      
    @Override  
    public int[] predictSeq(DataSequence seq, PI pi)  
    {  
        int labels[] = new int[seq.length()];  
        for(int i=0; i
        crfModel.apply(seq);  
        featureGen.mapStatesToLabels(seq);  
        for(int i=0; i
        {  
            if(labels[i] != seq.y(i))  
            {  
                pi.miss(); labels[i] = seq.y(i);  
                if(i+1==seq.length()) pi.smiss();  
            }  
            else  
            {  
                pi.hit();  
                if(i+1==seq.length()) pi.shit();  
            }  
        }  
        return labels;  
    }  
  
    @Override  
    public void train() throws Exception {  
        long st = System.currentTimeMillis();  
        allocModel();  
        featureGen.train(dataIter);  
        double featureWts[] = crfModel.train(dataIter);  
        System.out.printf("\t[Test] Training done...\n");          
        System.out.printf("\t[Test] Total spending time : %s...\n", TimeStr.toStringFrom(st));  
          
    }  
  
    @Override  
    public void parseRawdata(String fn) {  
        parseTestData(fn);        
    }  
  
    @Override  
    public PI validate(String testDataPath, String outputPath) throws Exception {  
        BufferedReader br = new BufferedReader(new FileReader(new File(testDataPath)));  
        BufferedWriter bw = new BufferedWriter(new FileWriter(new File(outputPath)));  
          
        PI pi = new PI();  
        String line = null;  
        DDataSequence seq = null;  
        int seqCnt = 0;  
        while((line=br.readLine())!=null)  
        {  
            seq = new DDataSequence(line);  
            if(seq.size()>0)  
            {                 
                predictSeq(seq, pi);  
                outputTaggedData(bw, seq);  
                seqCnt++;  
            }  
        }  
        bw.close();  
        br.close();  
        System.out.printf("\t[CRF-Mode] Total processing seq=%d...\n", seqCnt);  
        return pi;  
    }  
}  

你可以使用屬性 modelGraphType = "naive" 來決定你要使用的模型. 更多支援的模型可以參考這裡

Training process:
接著下面的範例代碼使用模型類別 DModel 來對輸入訓練資料 "data/wtraining_data.txt" 進行模型的訓練:

view plaincopy to clipboardprint?
String trainData = "data/wtraining_data.txt";         
String outPath = "data/model";  
/*Training model*/  
DModel model = new DModel();  
model.nlabels = 100;  
model.parseTestData(trainData);  
model.train();  
model.saveModel(outPath);  

訓練完後會在 "data/model" 目錄下面產生檔案:

* features.txt : 訓練產生的 features.
* crf.txt : 對應 features 的 weighting.

Testing/Prediction Process:
當你使用上面的步驟產生模型的檔案後, 你便可以使用下面範例代碼載入模型檔案並對測試資料 "data/wtest_data.txt" 進行 Label 的預測並導出結果到 "data/wtag_data.txt":
- "data/wtest_data.txt"

R:0-R:0-C:0-S:1-S:0-C:1-R:0
S:1-C:1-C:0-R:0-R:0-R:0-C:1-C:0-S:1
C:1-C:0-C:0-R:0-S:1-S:1-R:0-C:0-C:1
C:1-R:0-R:0-C:0-R:0-S:1-S:0-C:1-R:0-R:0
S:0-S:1-C:1-C:0-R:0-R:1-R:0-R:0-S:1-C:1
S:1-C:1-C:0-R:0-C:0-S:1-S:0-R:0-R:0-C:1

view plaincopy to clipboardprint?
String testData= "data/wtest_data.txt";  
String tagData = "data/wtag_data.txt";  
String outPath = "data/model";  
/*Training model*/  
DModel model = new DModel();  
model.nlabels = 100;  
  
model.loadModel(outPath);  
PI pi = model.validate(testData, tagData);  
System.out.printf("\t[Test] Hit rate=%.02f; Miss rate=%.02f\n", pi.hitRate(), pi.missRate());  
  
System.out.printf("\t[Info] Done!\n");  

輸出結果如下:

[CRF-Mode] Total processing seq=6...
[Test] Hit rate=0.71; Miss rate=0.29
[Info] Done!

This message was edited 18 times. Last update was at 24/05/2013 14:39:35

程式扎記

標籤

2013年5月23日星期四

[ Tools ] CRF - Condition Random Field Toolkit in Java 使用入門

14 則留言:

[Git 常見問題] error: The following untracked working tree files would be overwritten by merge

檢舉濫用情形

學習筆記

標籤

2013年5月23日 星期四