2016年11月30日 星期三

[Linux 常見問題] Iptables Open VNC Port To Allow Incoming VNC Connections

Source From Here
Question
How do I configure Linux system firewall to allow incoming VNC connections?

How-To
VNC server listens on the following TCP ports:
=> VNC server on display 0 will listen on TCP ports 5800, 5900 and 6000
=> VNC server on display 1 will listen on TCP ports 5801, 5901 and 6001
=> VNC server on display N will listen on TCP ports 580N, 590N and 600N

In other words a VNC server listens for a VNC client on TCP ports 5800+N, 5900+N, and 6000+N where N is the display which starts at zero. So,
5800+N – Java-based vncviewer;
5900+N – VNC Client Port;
6000+N – X Server port.

Find Out VNC Port
Type the following command:
# netstat -tulp | grep vnc
tcp 0 0 0.0.0.0:5901 0.0.0.0:* LISTEN 3240/Xvnc

Update /etc/sysconfig/iptables
Edit /etc/sysconfig/iptables file:
  1. ...  
  2. -A INPUT -p tcp -m state --state NEW -m tcp --dport 5801 -j ACCEPT  
  3. -A INPUT -p tcp -m state --state NEW -m tcp --dport 5901 -j ACCEPT  
  4. -A INPUT -p tcp -m state --state NEW -m tcp --dport 6001 -j ACCEPT  
Save and close the file. Restart iptables:
# service iptables restart

A Note About Other Linux Distributions
/etc/sysconfig/iptables works only on RHEL / CentOS / Fedora Linux. For other distros update your iptables shell script as follows:
# iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 5801 -j ACCEPT
# iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 5901 -j ACCEPT
# iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 6001 -j ACCEPT
# service iptables save


[ Java 文章收集 ] HashMap Vs. ConcurrentHashMap Vs. SynchronizedMap – How a HashMap can be Synchronized

Source From Here
Preface
HashMap is a very powerful data structure in Java. We use it everyday and almost in all applications. There are quite a few examples which I have written before on How to Implement Threadsafe cache, How to convert HashMap to Arraylist? We used HashMap in both below examples but those are pretty simple use cases of HashMapHashMap is a non-synchronized collection class.


Do you have any of below questions?
* What’s the difference between ConcurrentHashMap and Collections.synchronizedMap(Map)?
* What’s the difference between ConcurrentHashMap and Collections.synchronizedMap(Map) in term of performance?
ConcurrentHashMap vs Collections.synchronizedMap(Map)
* Popular HashMap and ConcurrentHashMap interview questions

In this tutorial we will go over all above queries and reason why and how we could Synchronize Hashmap?

Why?
The Map object is an associative containers that store elements, formed by a combination of a uniquely identify key and a mapped value. If you have very highly concurrent application in which you may want to modify or read key value in different threads then it’s ideal to use Concurrent Hashmap. Best example is Producer Consumer which handles concurrent read/write.

So what does the thread-safe Map means? If multiple threads access a hash map concurrently, and at least one of the threads modifies the map structurally, it must be synchronized externally to avoid an inconsistent view of the contents.

How?
There are two ways we could synchronized HashMap:
1. Java Collections synchronizedMap() method
2. Use ConcurrentHashMap

- HashMap Vs. synchronizedMap Vs. ConcurrentHashMap
  1. //Hashtable  
  2. Map normalMap = new Hashtable();  
  3.   
  4. //synchronizedMap  
  5. synchronizedHashMap = Collections.synchronizedMap(new HashMap());  
  6.   
  7. //ConcurrentHashMap  
  8. concurrentHashMap = new ConcurrentHashMap();  
ConcurrentHashMap
* You should use ConcurrentHashMap when you need very high concurrency in your project.
* It is thread safe without synchronizing the whole map.
* Reads can happen very fast while write is done with a lock.
* There is no locking at the object level.
* The locking is at a much finer granularity at a hashmap bucket level.
ConcurrentHashMap doesn’t throw a ConcurrentModificationException if one thread tries to modify it while another is iterating over it.
ConcurrentHashMap uses multitude of locks.

SynchronizedHashMap
* Synchronization at Object level.
Every read/write operation needs to acquire lock.
Locking the entire collection is a performance overhead.
* This essentially gives access to only one thread to the entire map & blocks all the other threads.
* It may cause contention.
* SynchronizedHashMap returns Iterator, which fails-fast on concurrent modification.

Now let’s take a look at code
1. Create class CrunchifyConcurrentHashMapVsSynchronizedHashMap.java
2. Create object for each HashMapSynchronizedMap and ConcurrentHashMap
3. Add and retrieve 500k entries from Map
4. Measure start and end time and display time in milliseconds
5. We will use ExecutorService to run 5 threads in parallel
- CrunchifyConcurrentHashMapVsSynchronizedMap.java
  1. package crunchify.com.tutorials;  
  2.   
  3. import java.util.Collections;  
  4. import java.util.HashMap;  
  5. import java.util.Hashtable;  
  6. import java.util.Map;  
  7. import java.util.concurrent.ConcurrentHashMap;  
  8. import java.util.concurrent.ExecutorService;  
  9. import java.util.concurrent.Executors;  
  10. import java.util.concurrent.TimeUnit;  
  11.   
  12. /** 
  13. * @author Crunchify.com 
  14. * 
  15. */  
  16.   
  17. public class CrunchifyConcurrentHashMapVsSynchronizedMap {  
  18.   
  19.     public final static int THREAD_POOL_SIZE = 5;  
  20.   
  21.     public static Map crunchifyHashTableObject = null;  
  22.     public static Map crunchifySynchronizedMapObject = null;  
  23.     public static Map crunchifyConcurrentHashMapObject = null;  
  24.   
  25.     public static void main(String[] args) throws InterruptedException {  
  26.   
  27.         // Test with Hashtable Object  
  28.         crunchifyHashTableObject = new Hashtable();  
  29.         crunchifyPerformTest(crunchifyHashTableObject);  
  30.   
  31.         // Test with synchronizedMap Object  
  32.         crunchifySynchronizedMapObject = Collections.synchronizedMap(new HashMap());  
  33.         crunchifyPerformTest(crunchifySynchronizedMapObject);  
  34.   
  35.         // Test with ConcurrentHashMap Object  
  36.         crunchifyConcurrentHashMapObject = new ConcurrentHashMap();  
  37.         crunchifyPerformTest(crunchifyConcurrentHashMapObject);  
  38.   
  39.     }  
  40.   
  41.     public static void crunchifyPerformTest(final Map crunchifyThreads) throws InterruptedException {  
  42.   
  43.         System.out.println("Test started for: " + crunchifyThreads.getClass());  
  44.         long averageTime = 0;  
  45.         for (int i = 0; i < 5; i++) {  
  46.   
  47.             long startTime = System.nanoTime();  
  48.             ExecutorService crunchifyExServer = Executors.newFixedThreadPool(THREAD_POOL_SIZE);  
  49.   
  50.             for (int j = 0; j < THREAD_POOL_SIZE; j++) {  
  51.                 crunchifyExServer.execute(new Runnable() {  
  52.                     @SuppressWarnings("unused")  
  53.                     @Override  
  54.                     public void run() {  
  55.   
  56.                         for (int i = 0; i < 500000; i++) {  
  57.                             Integer crunchifyRandomNumber = (int) Math.ceil(Math.random() * 550000);  
  58.   
  59.                             // Retrieve value. We are not using it anywhere  
  60.                             Integer crunchifyValue = crunchifyThreads.get(String.valueOf(crunchifyRandomNumber));  
  61.   
  62.                             // Put value  
  63.                             crunchifyThreads.put(String.valueOf(crunchifyRandomNumber), crunchifyRandomNumber);  
  64.                         }  
  65.                     }  
  66.                 });  
  67.             }  
  68.   
  69.             // Make sure executor stops  
  70.             crunchifyExServer.shutdown();  
  71.   
  72.             // Blocks until all tasks have completed execution after a shutdown request  
  73.             crunchifyExServer.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);  
  74.   
  75.             long entTime = System.nanoTime();  
  76.             long totalTime = (entTime - startTime) / 1000000L;  
  77.             averageTime += totalTime;  
  78.             System.out.println("2500K entried added/retrieved in " + totalTime + " ms");  
  79.         }  
  80.         System.out.println("For " + crunchifyThreads.getClass() + " the average time is " + averageTime / 5 + " ms\n");  
  81.     }  
  82. }  
One possible execution result:
Test started for: class java.util.Hashtable
500K entried added/retrieved in 1432 ms
500K entried added/retrieved in 1425 ms
500K entried added/retrieved in 1373 ms
500K entried added/retrieved in 1369 ms
500K entried added/retrieved in 1438 ms
For class java.util.Hashtable the average time 1407 ms

Test started for: class java.util.Collections$SynchronizedMap
500K entried added/retrieved in 1431 ms
500K entried added/retrieved in 1460 ms
500K entried added/retrieved in 1387 ms
500K entried added/retrieved in 1456 ms
500K entried added/retrieved in 1406 ms
For class java.util.Collections$SynchronizedMap the average time 1428 ms

Test started for: class java.util.concurrent.ConcurrentHashMap
500K entried added/retrieved in 413 ms
500K entried added/retrieved in 351 ms
500K entried added/retrieved in 427 ms
500K entried added/retrieved in 337 ms
500K entried added/retrieved in 339 ms
For class java.util.concurrent.ConcurrentHashMap the average time 373 ms <== Much faster

This message was edited 10 times. Last update was at 30/11/2016 20:59:38

2016年11月29日 星期二

[Windows 文章] Windows Batch Scripting: Getting Started

Source From Here 
Getting Started with Windows Batch Scripting 
Windows batch scripting is incredibly accessible – it works on just about any modern Windows machine. You can create and modify batch scripts on just about any modern Windows machine. The tools come out of the box: the Windows command prompt and a text editor like Notepad.exe. It’s definitely far from the best shell scripting langauge, but, it gets the job done. It’s my “duct tape” for Windows. 

Launching the Command Prompt 
Windows gurus launch the command prompt using the keyboard shortcut Windows Logo Key+R (i.e., “Run”) > Type cmd.exe then Enter. This is way faster than navigating the Windows Start Menu to find the Command Prompt. 


Editing Batch Files 
The universal text editor for batch files is Notepad (Windows Logo Key + R > Type notepad then Enter). Since batch files are just ASCII text, you can probably use just about any text editor or word processor. Very few editors do anything special for Batch files like syntax highlighting or keyword support, so notepad is good enough fine and will likely be installed on just about every Windows system you encounter. 

Viewing Batch Files 
I would stick with Notepad for viewing the contents of a batch file. In Windows Explorer (aka, “My Computer”), you should be able to view a batch file in Notepad by right clicking the file and seleting Edit from the context menu. If you need to view the contents within a command prompt window itself, you can use a DOS command like TYPE myscript.cmd or MORE myscript.cmd or notepad myscript.cmd 

Batch File Names and File Extensions 
Assuming you are using Windows XP or newer, I recommend saving your batch files with the file extension .cmd. Some seriously outdated Windows versions used .bat, though I recommend sticking with the more modern .cmd to avoid some rare side effects with .bat files. 

With the .cmd file extension, you can use just about filename you like. I recommend avoiding spaces in filenames, as spaces only create headaches in shell scripting. Pascal casing your filenames is an easy way to avoid spaces (e.g., HelloWorld.cmd instead of Hello World.cmd). You can also use punctuation characters like . or - or _ (e.g. Hello.World.cmdHello-World.cmd, Hello_World.cmd). 

Another thing with names to consider is avoiding names that use the same name of any built-in commands, system binaries, or popular programs. For example, I would avoid naming a script ping.cmd since there is a widely used system binary named ping.exe. Things might get very confusing if you try to run ping and inadvertently call ping.cmd when you really wanted ping.cmd. (Stay tuned for how this could happen.) I might called the script RemoteHeartbeat.cmd or something similar to add some context to the script’s name and also avoid any naming collisions with any other executable files. Of course, there could be a very unique circumstance in which you want to modify the default behavior of ping in which this naming suggestion would not apply. 

Saving Batch Files in Windows 
Notepad by default tries to save all files as plain jane text files. To get Notepad to save a file with a .cmd extension, you will need to change the “Save as type” to “All Files (.)”. See the screenshot below for an example of saving a script named “HelloWorld.cmd” in Notepad. 
 
Notes. 
I’ve used a shortcut in this screenshot that you will learn more about later. I’ve saved the file to my “user profile folder” by naming the file %USERPROFILE%\HelloWorld.cmd. The %USERPROFILE% keyword is the Windows environmental variable for the full path to your user profile folder. On newer Windows systems, your user profile folder will typically be C:\Users\. This shortcut saves a little bit of time because a new command prompt will generally default the “working directory” to your user profile folder. This lets you run HelloWorld.cmd in a new command prompt without changing directories beforehand or needing to specify the path to the script.

Running your Batch File 
The easy way to run your batch file in Windows is to just double click the batch file in Windows Explorer (aka “My Computer”). Unfortunately, the command prompt will not give you much of a chance to see the output and any errors. The command prompt window for the script will disappear as soon as the script exits. (We will learn how to handle this problem in Part 10 – Advanced Tricks ). 

When editing a new script, you will likely need to run the batch file in an existing command window. For newbies, I think the easiest foolproof way to run your script is to drag and drop the script into a command prompt window. The command prompt will enter the full path to your script on the command line, and will quote any paths containing spaces. 

Some other tips to running batch files: 
* You can recall previous commands using the up arrow and down arrow keys to navigate the command line history.
* I usually run the script as %COMPSPEC% /C /D "C:\Users\User\SomeScriptPath.cmd" Arg1 Arg2 Arg3 This runs your script in a new command prompt child process. The /C option instructs the child process to quit when your script quits. The /Ddisables any auto-run scripts (this is optional, but, I use auto-run scripts). The reason I do this is to keep the command prompt window from automatically closing should my script, or a called script, call the EXIT command. The EXIT command automatically closes the command prompt window unless the EXIT is called from a child command prompt process. This is annoying because you lose any messages printed by your script.

Comments 
The official way to add a comment to a batch file is with the REM (Remark) keyword: 
  1. REM This is a comment!  
The power user method is to use ::, which is a hack to uses the the label operator : twice, which is almost always ignored. Most power authors find the :: to be less distracting than REM. Be warned though there are a few places where ::will cause errors. 
  1. :: This is a comment too!! (usually!)  
For example, a FOR loop will error out with :: style comments. Simply fall back to using REM if you think you have a situation like this. 

Silencing Display of Commands in Batch Files 
The first non-comment line of a batch file is usually a command to turn off printing (ECHO’ing) of each batch file line. 
The @ is a special operator to suppress printing of the command line. Once we set ECHO’ing to off, we won’t need the @ operator again in our script commands. You restore printing of commands in your script with: 
Upon exit of your script, the command prompt will automatically restore ECHO to it’s previous state. 

Debugging Your Scripts 
Batch files invole a lot of trial and error coding. Sadly, I don’t know of any true debugger for Windows batch scripts. Worse yet, I don’t know of a way to put the command processor into a verbose state to help troubleshoot the script (this is the common technique for Unix/Linux scripts.) Printing custom ad-hoc debugging messages is about your only option using the ECHO command. Advanced script writers can do some trickery to selectively print debugging messages, though, I prefer to remove the debugging/instrumentation code once my script is functioning as desired.

[ Py DS ] Ch3 - Data Manipulation with Pandas (Part5)

Source From  Here   Pivot Tables   We have seen how the  GroupBy  abstraction lets us explore relationships within a dataset. A pivot ta...