[ 範例代碼 ] Array's collect with index?

Source From Here 
Preface 
We know Array provides a useful API:collect/API:collect! to help use collect element from Array. For example, if I want to collect even number from an array: 

>> a = [1,2,3,4,5,6,7,8,9,10]
=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>> a.collect{|i| i%2==0}
=> [false, true, false, true, false, true, false, true, false, true] # This is not what I want...Orz
>> a.collect{|i| i if i%2==0}.compact
=> [2, 4, 6, 8, 10] # API:compact will help us to remove nil element inside array

But what if I need to use index of element as judge condition? For example, I have another array b with true/false to tell me if this element will be kept in collected array: 

>> b = Array.new(a.size, false)
=> [false, false, false, false, false, false, false, false, false, false]
>> b.map{|a| true if rand(2)>0}
=> [false, true, false, false, false, true, true, false, false, false]
>> b.each_with_index do |v, i|; puts i if v; end
1
5
6 # So here means only a[1], a[5] and a[6] will be collected!

How-To 
Fortunately, we have a Enumerator module to help on this case. Let's try it as below: 

>> require "enumerator"
=> true
>> a.to_enum(:each_with_index)
=> #
>> a.to_enum(:each_with_index).collect{|v,a| v if b[a]}.compact
=> [2, 6, 7]

AP:obj.to_enum(method = :each, *args) will create a new Enumerator which will enumerate by on calling method on obj. 

Supplement 
* Stackoverflow - How to get a random number in Ruby?

[文章收集] Snort : Customized AppId Lua Script as Detector

Overview 
In Snort version 2.9.7, Cisco will release a new dynamic preprocessor OpenAppID, which will add application identification to Snort capabilities. Application identification can be used to view how applications are using network resources and to enforce application aware rules to control and manage applications running on network. 

Cisco will release open source code for hundreds of application detectors that can be used to identify frequently used applications. Users are free to copy and modify Cisco-provided detectors and create new detectors. The detectors will be small Lua programs that use a C-Lua API to interact with OpenAppID preprocessor. So far, OpenAppId support Up to 4 apps identified in each session. 

* Server App
* Client App
* Payload
* MISC

Detector Code Structure 
A detector has the following main components. The Detector Info Header, the included required libraries, the DetectorPackageInfo, the DetectorInit function, the DetectorValidator function, and the DetectorClean. We will describe each of them in more details below. 
 

Rule Example 
So you can Writing Snorts Rules to detect App: 

view plaincopy to clipboardprint?

1. alert tcp any any -> any any(msg: "Get Back to Work";  
2.                                     appid: facebook reddit;  
3.                                     sid:1000000;  
4.                                     rev:1)  

Custom Detector 
Here we will go through a simple example on how to write a lua script to detect "CircleCityCon.com"! 

Port 
Besides the lua script, the simplest way to identify app is using port information (List of TCP/UDP port number). Below is one of the example: 
- /odp/port/port_finger.yaml 

view plaincopy to clipboardprint?

1. name: finger  
2. service_name: finger  
3. protocol: tcp/udp  
4. ports: 79  
5. appId: 637  

More can be found under /odp/port/: 
 

Lua Script 
Another approach is to write lua script to capture the symptom of traffic. First thing is to defined the header: 

view plaincopy to clipboardprint?

1. require "DetectorCommon"  
2. local DC = DetectorCommon  
3. DetectorPackageInfo = {  
4.     name =  "circleCity",  
5.     proto =  DC.ipproto.tcp,  
6.     client = {  
7.         init =  'DetectorInit',  
8.         clean =  nil,  
9.         validate =  nil,  
10.         minimum_matches =  0  
11.     },  
12.     server = {  
13.         init = nil,  
14.         validate = nil,  
15.         clean = nil  
16.     },  
17. }  

In order to keep the detectors short, some commonly used code can placed into a library. Here the Cisco detector is including a library DetectorCommon.lua (Under odp/lib folder) and creating a shortcut ‘DC’ to it. Cisco libraries must be placed into the odp/libs subdirectory under the directory where Cisco Open Detector Package (ODP) was installed. This path is automatically included in the Lua path. Users can create more libraries and place them in/custom/libs, which is also automatically included in the Lua path. 

The DetectorPackageInfo structure is required in each detector. It identifies client and server functions that will be called to initialize, validate (process packets) and cleanup the detector. OpenAppID preprocessor reads this structure after loading Lua code and calls initialization functions. 

The structure has the following elements: 
* DetectorPackageInfo.name: This is a name for the detector that is used for logging purpose. 
* DetectorPackageInfo.proto: Protocol value. It can be DC.ipproto.tcp or DC.ipproto.udp. 
* DetectorPackageInfo.client: If the detector identifies client side application (for example Firefox) then this structure is populated. Detectors for payload application (example Facebook) will provide client section only. The following functions can be provided: 

- init: Name of callback function that initialize a detector. See “Detector Initialize” section for details.
- validate: Name of callback function that process packets in the detector. The function typically inspects packet contents and may use stored results from previous packets to detect an application. Before finishing, the functions call an appropriate API function to indicate results of detection. These functions are not required for some specific applications. See “Detector Validate” for more details.
- clean: Name of callback function that perform cleanup when Snort is exiting. The function is optional and may be omitted in the DetectorPackageInfo structure.

* DetectorPackageInfo.server: If the detector identifies a server side application (for example Apache web server) then this structure is populated. The structure provides init, validate and clean function names that have same purpose as in client side. 

Each detector must have an initializer function that is present in the DetectorPackageInfo structure. OpenAppID preprocessor will call this function directly upon loading the detector. The function is given detectorInstance, an instance of Detector class, which should be stored globally and used for calling all Lua-C API functions. The function may perform one or more of the following: 

1. Create a new application name by calling open_createApp().
2. Setup fast patterns and the port for an application. These are used for selecting a detector for a flow. See service_registerPattern() and service_addPorts().
3. Add patterns for specific headers for HTTP. See open_addUrlPattern() etc.

So for our customized detector for "circlecitycon.com", we prepare below three AppId inside DetectorInit(): 

view plaincopy to clipboardprint?

1. function DetectorInit(detectorInstance)  
2.     gDetector = detectorInstance  
3.     gAppId = gDetector:open_createApp('CrcCtyCon');  
4.     if gDetector.open_addUrlPattern then  
5.         gDetector:open_addUrlPattern(0,0,gAppId,"circlecitycon.com", "/", "http:");  
6.     end  
7.     gAppId = gDetector:open_createApp('CrcCtyConSched');  
8.     if gDetector.open_addUrlPattern then  
9.         gDetector:open_addUrlPattern(0,0,gAppId,"circlecitycon.com", "/talks/schedule", "http:");  
10.     end  
11.     gAppId = gDetector:open_createApp('CrcCtyConTix');  
12.     if gDetector.open_addUrlPattern then  
13.         gDetector:open_addUrlPattern(0,0,gAppId,"brownpapertickets.com", "/event/505248", "http:");  
14.     end  
15. end  

Ps. App names should be less than 16 characters. 

- void open_addUrlPattern(serviceAppId, clientAppId, payloadAppId, hostpattern, pathPattern, schemePattern) : 

API to add patterns for user agent and other special purpose patterns. Not to be used by new detectors.

- int open_createApp(appName) : 

Converts appName (a string) to an AppId (unique number). If appName does not match any existing application name then a new application is created and a unique AppId is assigned dynamically. For existing applications, the matching AppId is returned. This AppId should be used subsequently with C-Lua API wherever an AppId is required. Dynamic AppId values can change between different Snort runs.

So our entire lua script to detect "circlecitycon.com" is named circlecity.lua : 
- circlecity.lua 

view plaincopy to clipboardprint?

1. require "DetectorCommon"  
2. local DC = DetectorCommon  
3. DetectorPackageInfo = {  
4.     name =  "circleCity",  
5.     proto =  DC.ipproto.tcp,  
6.     client = {  
7.         init =  'DetectorInit',  
8.         clean =  nil,  
9.         validate =  nil,  
10.         minimum_matches =  0  
11.     },  
12.     server = {  
13.         init = nil,  
14.         validate = nil,  
15.         clean = nil  
16.     },  
17. }  
18. function DetectorInit(detectorInstance)  
19.     gDetector = detectorInstance  
20.     gAppId = gDetector:open_createApp('CrcCtyCon');  
21.     if gDetector.open_addUrlPattern then  
22.         gDetector:open_addUrlPattern(0,0,gAppId,"circlecitycon.com", "/", "http:");  
23.     end  
24.     gAppId = gDetector:open_createApp('CrcCtyConSched');  
25.     if gDetector.open_addUrlPattern then  
26.         gDetector:open_addUrlPattern(0,0,gAppId,"circlecitycon.com", "/talks/schedule", "http:");  
27.     end  
28.     gAppId = gDetector:open_createApp('CrcCtyConTix');  
29.     if gDetector.open_addUrlPattern then  
30.         gDetector:open_addUrlPattern(0,0,gAppId,"brownpapertickets.com", "/event/505248", "http:");  
31.     end  
32. end  
33. function DetectorValidate()  
34. end  

Demo 
So we have our lua script circlecity.lua to detect "circlecitycon.com". First thing first, let's put this lua script under path /custom/lua (In my case, it is/usr/local/lib/openappid/custom/lua/circlecity.lua). Then open a terminal to start snort: 

# snort -c /etc/snort/snort.conf --daq afpacket -i eth0 -k none -A fast
...
Commencing packet processing (pid=18635) 
Decoding Ethernet 

Then try to open browser and key-in "circlecitycon.com": 
 

And click on the top menu: Talks->Schedule: 
 

Finally, click on the link "Buy Tickets Here" in the right side: 
 

And wait for while until you see the log message look like below appear in the terminal running snort: 

...
Opening /var/log/snort/appstats-unified.log.1414568940 for output

We can use command u2openappid to read the log file /var/log/snort/appstats-unified.log.1414568940 or use another simple useful shell chkAppidLog.sh  to do it for us: 

# ./chkAppidLog.sh 
...
statTime="1414568880",appName="dns",txBytes="1689",rxBytes="3183" 
statTime="1414569060",appName="chrome",txBytes="9679",rxBytes="80191" 
statTime="1414569060",appName="http",txBytes="9679",rxBytes="80191" 
statTime="1414569060",appName="CrcCtyCon",txBytes="5153",rxBytes="38103" 
statTime="1414569060",appName="CrcCtyConSched",txBytes="3474",rxBytes="41634" 
statTime="1414569060",appName="CrcCtyConTix",txBytes="1052",rxBytes="454" 
statTime="1414568820",appName="https",txBytes="24550",rxBytes="127069" 
... 

Or 

# u2openappid /var/log/snort/appstats-unified.log.1414568940 # Using sudo if necessary

Reference 
* OpenDetectorDeveloperGuide.pdf 
* Youtube - circlecitycon 114 openappid open source ...irewall with snort adam hogan 
* Snort User Manual 2.9.6