2015年6月28日 星期日

[Linux 文章收集] "xargs" All-IN-One Tutorial Guide

Source From Here
Preface
xargs is a just like "awk" ,"find" & "grep" commands processes the standard input on all unix flavoured operating sysems. Basically "xargs" is used to remove or do some operation on long list of file names which were produced by "find" & "grep" commands.

Usually many UNIX operating system doesn't accept such a long list of argument.UNIX xargs command divide that list into sub-list with acceptable length and made it work. For example I'd like to find out all *.sh file located in 100s of sub-directories and move them to another directory called ~/back.scripts. How do I use command line args with xargs to achieve the same?

as per man page "xargs" is used to execute a command, passing constructed argument list(s). The arguments are typically a long list of filenames (generated byls or find etc) that are passed to xargs via a pipe.

Some features:
* xargs can execute the command supplying some initial arguments directly, and reading the remaining arguments from standard input (or piped input).
* xargs passes arguments to command in several bundles, this allows command to process more arguments than it could normally handle at once.
* Arguments in the standard input must be separated by unquoted blank characters, or unescaped blank characters or newline characters.
* Characters can be quoted by enclosing them in "double-quotes" (non-double-quote and non-newline chars only).
* Characters can be quoted by enclosing them in 'apostrophes' (non-apostrophe and non-newline chars only).
* Any unquoted character can be escaped by preceding it with a backslash.

Exit Status
This command returns the following exit values:


Examples
Find all the .conf files under /etc/ and pass to the ls command, -print0 is required if any filenames contain whitespace.:
# find /etc/ -name "*.conf" -print0 | xargs -0 ls -l

Find all files in the work folder, pass to grep and search for 'profit':
# find ./work -print | xargs grep "profit"


{} as the argument list marker
{} is the default argument list marker. You need to use {} this with various command which take more than two arguments at a time. For example mv command need to know the file name. The following will find all .sh files in or below the current directory and move them to ~/.old.files directory:
# find . -name "*.sh" -print0 | xargs -0 -I {} mv {} ~/back.scripts

You can rename {} to something else. In the following example {} is renamed as file. This is more readable as compare to previous example:
# find . -name "*.sh" -print0 | xargs -0 -I file mv file ~/back.scripts

Where,
-0 If there are blank spaces or characters (including newlines) many commands will not work. This option take cares of file names with blank space.
-I Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separator is the newline character.

10 Popular "XARGS" Command Examples:
1) With& Without "xargs" observation:
In this example of xargs command we will see how output changes with use of xargs command in unix or Linux. Here is the output of find command withoutxargs first and than with xargs, you can clearly see that multiline output is converted into single line:
# find . -name "*sh*"
./.bash_history
./.bash_profile
./.bash_profile.cf-before-edit
./.cshrc
...

# find . -name "*bash*" | xargs
./.bash_history ./.bash_profile ./.bash_profile.cf-before-edit ...

2) Xargs with grep:
When you use "xargs" in conjusction with find and grep , the grep will look for the specific word in each file in the from the standard input.
# find . -name "*.sh" | xargs grep "ksh"

In the above example first find all .sh files from current directory or below and than on each .sh file look for word "ksh".

3) Covert muti-line output into single line
Best example of xargs is converting output of one command into one line. For example you can run any command and then combine xargs to convert output into single line. here is an example xargs in unix which does that.
# ls -1 *.sh
linux_sysinfo.sh
aix_sysinfo.sh
audit_script.sh
chperm_messages.sh

# ls -1 *.sh | xargs
linux_sysinfo.sh aix_sysinfo.sh audit_script.sh chperm_messages.sh

4) To Delete temporary files using xargs & find:
Another common example of xargs command in unix is removing temporary files from system.
# find /tmp -name "*.tmp" | xargs rm

This will remove all .tmp file from /tmp or below directory. xargs in unix is very fast as compared to deleting single file at a time which can also be done by usingfind command alone.

5) xargs -0 to handle space in file name
Above example of xargs command in unix will not work as expected if any of file name contains space or new line on it. To avoid this problem we use find -print0to produce null separated file name and xargs -0 to handle null separated items. Here is an example of xargs command in unix which can handle file name with spaces and newline:
# find /tmp -name "*.tmp" -print0 | xargs -0 rm

6) Counting number of lines/words/characters in each file using xargs & find:
"find" in conjuction with "xargs" and "wc" we can count number of lines/words/characters in each file under a particular directory.
# ls -1 *.sh | xargs wc -l 
112 linux_sysinfo.sh
85 aix_sysinfo.sh
35 audit_script.sh
18 chperm_messages.sh
250 total

Note: you can use '-c' & '-w' with wc to obtain number of characters and words respectively.

7) xargs and cut command in Unix:
Though most of xargs examples in unix will be along with find and grep command but xargs is not just limited to this two it can also be used with any command which generated long list of input for example we can use xargs with cut command in unix. In below example of unix xargs we will xargs example with cutcommand. for using cut command let's first create a test file with some data e.g.
# cat fruits.txt
Orange,Greenorange
Mango,Redmango
Banana,Pinkbanana


// Now we will display name of actual fruit from first column using xargs command in one line:
# cut -d, -f1 fruits.txt | sort | xargs
Banana Mango Orange

8) To insert file names into the middle of command lines, type:
This command sequence renames all files in the current directory by adding .old to the end of each name. The -I flag tells the xargs command to insert each line of the ls directory listing where {} (braces) appear. If the current directory contains the files chap1, chap2, and chap3, this constructs the following commands:
// -t, --verbose: Print the command line on the standard error output before executing it.
# ls | xargs -t -I {} mv {} {}.old
# mv chap1 chap1.old
# mv chap2 chap2.old
# mv chap3 chap3.old

9) To run a command on files that you select individually, type:
This command sequence allows you to select files to add to the lib.a library. The -p flag tells the xargs command to display each ar command it constructs and to ask if you want to run it. Type y to run the command. Press the any other key if you do not want to run the command.
// -p, --interactive: Prompt the user about whether to run each command line and read a line from the terminal.
// -n max-args, --max-args=max-args: Use at most max-args arguments per command line.

# ls | xargs -p -n 1 ar r lib.a
ar r lib.a chap1 ?...
ar r lib.a chap2 ?...
ar r lib.a chap3 ?...

10) To construct a command that contains a specific number of arguments and to insert those arguments into the middle of a command line, type:
// If the current directory contains files chap1 through chap10, the output constructed will be the following
# ls | xargs -n6 | xargs -I {} echo {} - some files in the directory
chap1 chap2 chap3 chap4 chap5 chap6 - some files in the directory
chap7 chap8 chap9 chap10 - some file in the directory


Supplement
10 Xargs Command Examples in Linux / UNIX
This tutorials explains the usage of xargs command using few simple examples...


沒有留言:

張貼留言

[Git 常見問題] error: The following untracked working tree files would be overwritten by merge

  Source From  Here 方案1: // x -----删除忽略文件已经对 git 来说不识别的文件 // d -----删除未被添加到 git 的路径中的文件 // f -----强制运行 #   git clean -d -fx 方案2: 今天在服务器上  gi...