Friday, November 26, 2010

Oracle DBA's useful linux commands


Basic LINUX commands that a DBA should know

groupadd  : This is the command used to create new group. At OS level group is used to give and take  pivillages.
Syntax : groupadd <group name>
# groupadd group1
View :
# cat /etc/group  -
This command used to view which user belongs to which group.
Output: group1:x:607:

Useradd :This is the command used to create a new user in a group.
Syntax : useradd -g <group name> <user name>
[root@rac5 ~]# useradd -g group1 user1

passwd : This is the command used to give password for create use or to update the password.
Syntax : passwd <user name>
Ex: [root@rac5 ~]# passwd user1
Output :
# Changing password for user sukhi.
New UNIX password:
BAD PASSWORD: it is based on a dictionary word
Retype new UNIX password:
passwd: all authentication tokens updated successfully.

date : This is the command used to view the current system date.
# date
Output : Wed Oct 27 21:55:36 IST 2010

In order to update the date we can give :
Syntax :
# date -s "2 OCT 2010 14:00:00"  
OR
# date --set="27 OCT 2010 21:56:00"
Output : Sat Oct  2 14:00:00 IST 2010

cal : This command shows the calender of current year or any.
#  Cal
Output : [root@rac5 ~]#    October 2010
                        Su Mo Tu We Th Fr Sa
                            1  2
             3  4  5  6  7  8  9
            10 11 12 13 14 15 16
            17 18 19 20 21 22 23
            24 25 26 27 28 29 30
            31

pwd : This command is to view the present working directory.
# pwd
Output : [root@rac5 ~]# /root.

ls : This command is used to list all contents of directories
$ ls

ls –lt :This command is used to list lot of information about contents of directories
$ ls -lt

The permissions are the first 10 characters of the line (-rwxrwx---) and can be broken down as follows.



-
rwx
r--
r--
1
root
root
765
Apr 23
file.txt
File type
Owner
Group
All
Links
Owner
Group
Size
Mod date
Filename

cd : This is the command used to change a directory
$ ls
authorized_keys  file   file2  oraInventory  stand.ora
authorized-keys  file1  file3  sukhi
$ cd sukhi
[oracle@rac5 sukhi]$

This is used to go back to parent directory
$ cd ..

mkdir : This command is used for make a new directory.
$ mkdir dir1

rmdir : This commad is used for remove a directory.
$ rmdir dir1

rm -rf : This command is used to forcefully remove a directory.
$ rm -fr dir1

man : This command is used to show the online manual pages of related commands
$ man ls

touch : This command is used create an empty file
$ touch file1

find : This command is used find a file

For a case-sensitive search, use the -name option:
$ find . -name "file*"

For a case-insensitive search, use the -iname option:
$ find . -iname "file*"
You can limit your search to a specific type of files only. For instance, the above command will get the files of all types: regular files, directories, symbolic links, and so on. To search for only regular files, you can use the -type f parameter.

$ find . -name "orapw*" -type f
./orapw+ASM
./orapwDBA102
./orapwRMANTEST
./orapwRMANDUP
./orapwTESTAUX
The -type can take the modifiers f (for regular files), l (for symbolic links), d (directories), b (block devices), p (named pipes), c (character devices), s (sockets).
For the files with extension "trc" and remove them if they are more than three days old. A simple command does the trick:
find . -name "*.trc" -ctime +3 -exec rm {} \;

To forcibly remove them prior to the three-day limit, use the -f option.
find . -name "*.trc" -ctime +3 -exec rm -f {} \;

If you just want to list the files:
find . -name "*.trc" -ctime +3 -exec ls -l {} \;

cp : This command is used to copy a file from one to another 
$ cp file1 filenew 

mv  : This command is used to rename the name of a file to other
$ mv file1 filenew
su : This command gives you root permissions but it does not change the PATH and current working directory. So you could not execute file in /usr/sbin directory. This command is used to switch one user to other. it doesnot change the current working directory. so you cant access the /usr/sbin  directories.
$ su sukhi
su -  : This command changes the path too and root home becomes your current wokring directory. This command is used to switch one user with changing current working directory.
$ su – sukhi

How to use chown and chgrp commands to change ownership and group of the files.
# ls -l
total 8
-rw-r--r--    1 user1     users          70 Aug  4 04:02 file1
-rwxr-xr-x    1 oracle   dba           132 Aug  4 04:02 file2
-rwxr-xr-x    1 oracle   dba           132 Aug  4 04:02 file3
-rwxr-xr-x    1 oracle   dba           132 Aug  4 04:02 file4
-rwxr-xr-x    1 oracle   dba           132 Aug  4 04:02 file5
-rwxr-xr-x    1 oracle   dba           132 Aug  4 04:02 file6

and you need to change the permissions of all the files to match those of file1. Sure, you could issue chmod 644 * to make that change—but what if you are writing a script to do that, and you don’t know the permissions beforehand? Or, perhaps you are making several permission changes and based on many different files and you find it infeasible to go though the permissions of each of those and modify accordingly.

A better approach is to make the permissions similar to those of another file. This command makes the permissions of file2 the same as file1:

chmod --reference file1 file2
Now if you check:
# ls -l file[12]
total 8
-rw-r--r--    1 user1   users          70 Aug  4 04:02 file1
-rw-r--r--    1 oracle   dba           132 Aug  4 04:02 file2

The file2 permissions were changed exactly as in file1. You didn’t need to get the permissions of file1 first.
You can also use the same trick in group membership in files. To make the group of file2 the same as file1, you would issue:
# chgrp --reference file1 file2
# ls -l file[12]
-rw-r--r--    1 user1   users          70 Aug  4 04:02 file1
-rw-r--r--    1 oracle   users         132 Aug  4 04:02 file2

Of course, what works for changing groups will work for owner as well. Here is how you can use the same trick for an ownership change. If permissions are like this:

# ls -l file[12]
-rw-r--r--    1 user1   users          70 Aug  4 04:02 file1
-rw-r--r--    1 oracle   dba           132 Aug  4 04:02 file2

You can change the ownership like this:

# chown --reference file1 file2
# ls -l file[12]
-rw-r--r--    1 user1   users          70 Aug  4 04:02 file1
-rw-r--r--    1 user1   users         132 Aug  4 04:02 file2

Note that the group as well as the owner have changed.

This is a trick you can use to change ownership and permissions of Oracle executables in a directory based on some reference executable. This proves

especially useful in migrations where you can (and probably should) install as a different user and later move them to your regular Oracle software owner.

cmp. : The command cmp is similar to diff
# cmp file1 file2
file1 file2 differ: byte 10, line 1

The output comes back as the first sign of difference. You can use this to identify where the files might be different. Like diff, cmp has a lot of options, the

most important being the -s option, that merely returns a code:
0, if the files are identical
1, if they differ
Some other non-zero number, if the comparison couldn’t be made

Here is an example:
# cmp -s file3 file4
# echo $?
0

The special variable $? indicates the return code from the last executed command. In this case it’s 0, meaning the files file1 and file2 are identical.
# cmp -s file1 file2
# echo $?
1
means file1 and file2 are not the same.

Recall from a previous tip that when you relink Oracle executables, the older version is kept prior to being overwritten. So, when you relink, the executable sqlplus is renamed to “sqlplusO” and the newly compiled sqlplus is placed in the $ORACLE_HOME/bin. So how do you ensure that the sqlplus that was just created is any different? Just use:
# cmp sqlplus sqlplusO
sqlplus sqlplusO differ: byte 657, line 7

If you check the size:
# ls -l sqlplus*
-rwxr-x--x    1 oracle   dba          8851 Aug  4 05:15 sqlplus
-rwxr-x--x    1 oracle   dba          8851 Nov  2  2005 sqlplusO

Even though the size is the same in both cases, cmp proved that the two programs differ


md5sum.
This command generates a 32-bit MD5 hash value of the files:
# md5sum file1
ef929460b3731851259137194fe5ac47  file1

Two files with the same checksum can be considered identical. However, the usefulness of this command goes beyond just comparing files. It can also provide a mechanism to guarantee the integrity of the files.

Suppose you have two important files—file1 and file2—that you need to protect. You can use the --check option check to confirm the files haven't changed. First, create a checksum file for both these important files and keep it safe:

# md5sum file1 file2 > f1f2
Later, when you want to verify that the files are still untouched:

# md5sum --check f1f2    
file1: OK
file2: OK

This shows clearly that the files have not been modified. Now change one file and check the MD5:

# cp file2 file1
# md5sum --check f1f2
file1: FAILED
file2: OK
md5sum: WARNING: 1 of 2 computed checksums did NOT match

The output clearly shows that file1 has been modified.

md5sum is an extremely powerful command for security implementations. Some of the configuration files you manage, such as listener.ora, tnsnames.ora, and init.ora, are extremely critical in a successful Oracle infrastructure and any modification may result in downtime. These are typically a part of your change control process. Instead of just relying on someone’s word that these files have not changed, enforce it using MD5 checksum. Create a checksum file and whenever you make a planned change, recreate this file. As a part of your compliance, check this file using the md5sum command. If someone inadvertently updated one of these key files, you would immediately catch the change.

In the same line, you can also create MD5 checksums for all executables in $ORACLE_HOME/bin and compare them from time to time for unauthorized modifications.

alias and unalias

Suppose you want to check the ORACLE_SID environment variable set in your shell. You will have to type:
echo $ORACLE_HOME

As a DBA or a developer, you frequently use this command and will quickly become tired of typing the entire 16 characters. Is there is a simpler way?

There is: the alias command. With this approach you can create a short alias, such as "os", to represent the entire command:

alias os='echo $ORACLE_HOME'
Now whenever you want to check the ORACLE_SID, you just type "os" (without the quotes) and Linux executes the aliased command.

However, if you log out and log back in, the alias is gone and you have to enter the alias command again. To eliminate this step, all you have to do is to put the command in your shell's profile file. For bash, the file is .bash_profile (note the period before the file name, that's part of the file's name) in your home

directory. For bourne and korn shells, it's .profile, and for c-shell, .chsrc.

You can create an alias in any name. For instance, I always create an alias for the command sqlplus "/as sysdba",
alias sql=’sqlplus "/as sysdba"

Here is a list of some very useful aliases I like to define:

alias bdump='cd $ORACLE_BASE/admin/$ORACLE_SID/bdump'
alias l='ls -d .* --color=tty'
alias ll='ls -l --color=tty'
alias mv='mv -i'
alias oh='cd $ORACLE_HOME'
alias os='echo $ORACLE_SID'
alias tns='cd $ORACLE_HOME/network/admin'

To see what aliases have been defined in your shell, use alias without any parameters
$alias

To remove an alias previously defined, just use the unalias command:

$ unalias rm

xargs

Most Linux commands are about getting an output: a list of files, a list of strings, and so on. But what if you want to use some other command with the output of the previous one as a parameter? For example, the file command shows the type of the file (executable, ascii text, and so on); you can manipulate the output to show only the filenames and now you want to pass these names to the ls -l command to see the timestamp. The command xargs

does exactly that. It allows you to execute some other commands on the output.

file -Lz * | grep ASCII | cut -d":" -f1 | xargs ls -ltr

Let's dissect this command string. The first, file -Lz *, finds files that are symbolic links or compressed. It passes the output to the next command, grep

ASCII, which searches for the string "ASCII" in them and produces the output similar to this:
alert_DBA102.log:         ASCII English text
alert_DBA102.log.Z:       ASCII text (compress'd data 16 bits)
dba102_asmb_12307.trc.Z:  ASCII English text (compress'd data 16 bits)
dba102_asmb_20653.trc.Z:  ASCII English text (compress'd data 16 bits)

Since we are interested in the file names only, we applied the next command, cut -d":" -f1, to show the first field only:
alert_DBA102.log
alert_DBA102.log.Z
dba102_asmb_12307.trc.Z
dba102_asmb_20653.trc.Z

Now, we want to use the ls -l command and pass the above list as parameters, one at a time. The xargs command allowed you to to that. The last part,

xargs ls -ltr, takes the output and executes the command ls -ltr against them, as if executing:

ls -ltr alert_DBA102.log
ls -ltr alert_DBA102.log.Z
ls -ltr dba102_asmb_12307.trc.Z
ls -ltr dba102_asmb_20653.trc.Z

Thus xargs is not useful by itself, but is quite powerful when combined with other commands.

Here is another example, where we want to count the number of lines in those files:

$ file * | grep ASCII | cut -d":" -f1  | xargs wc -l
  47853  alert_DBA102.log
     19  dba102_cjq0_14493.trc
  29053  dba102_mmnl_14497.trc
    154  dba102_reco_14491.trc
     43  dba102_rvwr_14518.trc
  77122  total

(Note: the above task can also be accomplished with the following command:)

$ wc -l ‘file * | grep ASCII | cut -d":" -f1 | grep ASCII | cut -d":" -f1‘

The xargs version is given to illustrate the concept. Linux has several ways to achieve the same task; use the one that suits your situation best.

Using this approach you can quickly rename files in a directory.

$ ls | xargs -t -i mv {} {}.bak

The -i option tells xargs to replace {} with the name of each item. The -t option instructs xargs to print the command before executing it.

Another very useful operation is when you want to open the files for editing using vi:

$ file * | grep ASCII | cut -d":" -f1 | xargs vi

This command opens the files one by one using vi. When you want to search for many files and open them for editing, this comes in very handy.

It also has several options. Perhaps the most useful is the -p option, which makes the operation interactive:

$ file * | grep ASCII | cut -d":" -f1 | xargs -p vi
vi alert_DBA102.log dba102_cjq0_14493.trc dba102_mmnl_14497.trc   dba102_reco_14491.trc dba102_rvwr_14518.trc ?...

Here xarg asks you to confirm before running each command. If you press "y", it executes the command. You will find it immensely useful when you take some potentially damaging and irreversible operations on the file—such as removing or overwriting it.

The -t option uses a verbose mode; it displays the command it is about to run, which is a very helpful option during debugging.

What if the output passed to the xargs is blank? Consider:

$ file * | grep SSSSSS | cut -d":" -f1 | xargs -t wc -l
wc -l
            0
$

Here searching for "SSSSSS" produces no match; so the input to xargs is all blanks, as shown in the second line (produced since we used the -t, or the

verbose option). Although this may be useful, In some cases you may want to stop xargs if there is nothing to process; if so, you can use the -r option:
$ file * | grep SSSSSS | cut -d":" -f1 | xargs -t -r wc -l
$

The command exits if there is nothing to run.

Suppose you want to remove the files using the rm command, which should be the argument to the xargs command. However, rm can accept a limited

number of arguments. What if your argument list exceeds that limit? The -n option to xargs limits the number of arguments in a single command line.

Here is how you can limit only two arguments per command line: Even if five files are passed to xargs ls -ltr, only two files are passed to ls -ltr at a time.

$ file * | grep ASCII | cut -d":" -f1 | xargs -t -n2 ls -ltr
ls -ltr alert_DBA102.log dba102_cjq0_14493.trc
-rw-r-----    1 oracle   dba           738 Aug 10 19:18 dba102_cjq0_14493.trc
-rw-r--r--    1 oracle   dba       2410225 Aug 13 05:31 alert_DBA102.log
ls -ltr dba102_mmnl_14497.trc dba102_reco_14491.trc
-rw-r-----    1 oracle   dba       5386163 Aug 10 17:55 dba102_mmnl_14497.trc
-rw-r-----    1 oracle   dba          6808 Aug 13 05:21 dba102_reco_14491.trc
ls -ltr dba102_rvwr_14518.trc
-rw-r-----    1 oracle   dba          2087 Aug 10 04:30 dba102_rvwr_14518.trc

Using this approach you can quickly rename files in a directory.

$ ls | xargs -t -i mv {} {}.bak

The -i option tells xargs to replace {} with the name of each item.

 *********************************************************


Check Ram Size From Redhat Linux Desktop System


Cat : This command is used to create and view files of directories
$ cat file1
$ cat file1 > newfile   // owerwrite newfile with file1
$ cat file1 >> newfile  // append newfile the contents with file1
$ cat /proc/meminfo

free
To display amount of free and used memory (including total in the system), enter:
$ free -m
$ free -g
$ free -k


System copying Command in linux

scp
This command is used for copying the files from one system to another.
$ scp /home/oracle/sukhi.txt oracle@rac4:/home/oracle/sukhi.txt 

Here the target machine name , location , filename shows in red color    

Linux Compressing Utilites







Compression Tool
File Extension
Decompression Tool
bzip2
.bz2
bunzip2
gzip
.gz
gunzip
zip
.zip
unzip

bzip2
This command is used to compress files.
$ bzip2 mydb2
The file is compressed and saved as mydb2.bz2
$ bunzip2 mydb2.bz2

gzip
This command is used to compress files.
$ gzip2 mydb2
The file is compressed and saved as mydb2.gz
$ bunzip2 mydb2.gz
zip
This command is used to compress a directory.
$ zip -r mydb2.zip filesdir  // directory

The file is compressed and saved as
mydb2.zip
$ bunzip2 mydb2.bz2
 

Connect to other system

ssh
This is the command used to connect the one system to another.
$ ssh oracle@rac4
Last login: Sun Nov 28 13:41:50 2010 from 10.17.57.57

Find the space utilization

du -k
This command is used for checking disc space.
$ du -k /home/oracle

8       /home/oracle/sukhi
24      /home/oracle/.ssh
16      /home/oracle/.kde/Autostart
20      /home/oracle/.kde
28      /home/oracle/oraInventory/logs
440     /home/oracle/oraInventory/Contents
16      /home/oracle/oraInventory/ContentsXML
500     /home/oracle/oraInventory
644     /home/oracle


df -k
This command is used for getting information of filesystem (/dev/sda1), mounted poin, used space ,available space, use % etc. size will dipaled in KB.
$ df -k /home/oracle
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1             28898080  10812328  16617816  40% /


df -h
This command is used for getting information of filesystem (/dev/sda1), mounted poin, used space ,available space, use % etc. in humanly readable format that is size will give in GB etc
[oracle@rac5 ~]$ df -h /home/oracle
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              28G   11G   16G  40% /

# du -ch|grep total        -- Total Size of a folder

Command for read and print in shell scripts

Read : This command is used to read something from the user. It read and strored in a variable.
read variable

echo : This commnad used to print soemthing to the screen. We can display the vlaues of varibles.
echo "sowfeer" OR echo $varibale

 

How to list the contents of a directory to a text file

Ls : By using the ls command we can do it.
ls /home/oracle/* > /tmp/sowfeer.txt

 

Change ownership Command

chown
This command used to change the ownership of file.
Syntax : chown [-R] newowner filenames
Give permissions as owner to user hope for the file file.txt.
chown chope file.txt
Give chown permissions to hope for all files in the work directory.
chown -R hope work

Changing file permissions

chmod
This command is used for changing the file permissions. .
# chmod o+r remove3.txt // for others
# chmod u+r remove3.txt // for owner or user[root@rac5 oracle]
# chmod g+r remove3.txt // for groups .

The permissions are encoded as octal number (green in color as shown below)
chmod 755 file  # Owner=rwx Group=r-x Other=r-x
chmod 500 file2 # Owner=r-x Group=--- Other=---
chmod 644 file3 # Owner=rw- Group=r-- Other=r--
chmod +x  file  # Add execute permission to file for all
chmod o-r file  # Remove read permission for others
chmod a+w file  # Add write permission for everyone


********************************************************************


OS Users Management


useradd : command is used to add OS users.
root> useradd -G oinstall -g dba -d /usr/users/my_user -m -s /bin/ksh my_user
  • The "-G" flag specifies the primary group.

  • The "-g" flag specifies the secondary group.

  • The "-d" flag specifies the default directory.

  • The "-m" flag creates the default directory.

  • The "-s" flag specifies the default shell.


usermod : command is used to modify the user settings after a user has been created.
root> usermod -s /bin/csh my_user

userde : command is used to delete existing users.
root> userdel -r my_user

The "-r" flag removes the default directory.
passwd : command is used to set, or reset, the users login password.
root> passwd my_user

who : command can be used to list all users who have OS connections.
root> who
root> who | head -5
root> who | tail -5
root> who | grep -i ora
root> who | wc -l
  • The "head -5" command restricts the output to the first 5 lines of the who command.

  • The "tail -5" command restricts the output to the last 5 lines of the who command.

  • The "grep -i ora" command restricts the output to lines containing "ora".

  • The "wc -l" command returns the number of lines from "who", and hence the number of connected users.


 

Process Management

Ps : command lists current process information.
root> ps
root> ps -ef | grep -i ora

Specific processes can be killed by specifying the process id in the kill command.
root> kill -9 12345

 

uname and hostname : commands can be used to get information about the host.
root> uname -a
OSF1 oradb01.lynx.co.uk V5.1 2650 alpha
 
root> uname -a | awk '{ print $2 }'
oradb01.lynx.co.uk
 
root> hostname
oradb01.lynx.co.uk

 

Error Lines in Files

You can return the error lines in a file using.
root> cat alert_LIN1.log | grep -i ORA-

The "grep -i ORA-" command limits the output to lines containing "ORA-". The "-i" flag makes the comparison case insensitive. A count of the error lines can be returned using the "wc" command. This normally give a word count, but the "-l" flag alteres it to give a line count.
root> cat alert_LIN1.log | grep -i ORA- | wc -l

 

File Exists Check

The Korn shell allows you to check for the presence of a file using the "test -s" command. In the following script a backup log is renamed and moved if it is present.
#!/bin/ksh
if test -s /backup/daily_backup.log
then
  DATE_SUFFIX=`date +"%y""%m""%d""%H""%M"`
  mv /backup/daily_backup.log /backup/archive/daily_backup$DATE_SUFFIX.log
fi

 

Remove Old Files

The find command can be used to supply a list of files to the rm command.
find /backup/logs/ -name daily_backup* -mtime +21 -exec rm -f {} ;

 

Remove DOS CR/LFs (^M)

Remove DOS style CR/LF characters (^M) from UNIX files using.
sed -e 's/^M$//' filename > tempfile

The newly created tempfile should have the ^M character removed.

 

Run Commands As Oracle User From Root

The following scripts shows how a number of commands can be run as the "oracle" user the "root" user.
#!/bin/ksh
su - oracle <<EOF
ORACLE_SID=LIN1; export ORACLE_SID
rman catalog=rman/rman@w2k1 target=/ cmdfile=my_cmdfile log=my_logfile append 
EOF

This is often necessary where CRON jobs are run from the root user rather than the oracle user.

 

Compress Files

In order to save space on the filesystem you may wish to compress files such as archived redo logs. This can be using either the gzip or the compress commands. The gzip command results in a compressed copy of the original file with a ".gz" extension.
The gunzip command reverses this process.
gzip myfile
gunzip myfile.gz

The compress command results in a compressed copy of the original file with a ".Z" extension. The uncompress command reverses this process.
compress myfile
uncompress myfile

 

General Performance, System Activity, Hardware and System Information

 

Vmstat

# vmstat 3

Display Memory Utilization Slabinfo

# vmstat -m

 

Get Information About Active / Inactive Memory Pages

# vmstat -a

$ vmstat 5 3
Displays system statistics (5 seconds apart; 3 times).






procs
memory
page
disk
faults
cpu
r
b
w
swap
free
re
mf
pi
po
fr
de
sr
s0
s1
s2
s3
in
sy
cs
us
sy
id
0
0
0
28872
8792
8
5
172
142
210
0
24
3
11
17
2
289
1081
201
14
6
80
0
0
0
102920
1936
1
95
193
6
302
1264
235
12
1
0
3
240
459
211
0
2
97
0
0
0
102800
1960
0
0
0
0
0
464
0
0
0
0
0
107
146
29
0
0
100
Having any processes in the b or w columns is a sign of a problem system. Having an id of 0 is a sign that the cpu is over-burdoned. Having high values in pi and po show excessive paging.
  • procs (Reports the number of processes in each of the following states)

    • r : in run queue

    • b : blocked for resources (I/O, paging etc.)

    • w : runnable but swapped

  • memory (Reports on usage of virtual and real memory)

    • swap : swap space currently available (Kbytes)

    • free : size of free list (Kbytes)

  • page (Reports information about page faults and paging activity (units per second)

    • re : page reclaims

    • mf : minor faults

    • pi : Kbytes paged in

    • po : Kbytes paged out

    • fr : Kbytes freed

    • de : anticipated short-term memory shortfall (Kbytes)

    • sr : pages scanned by clock algorith

  • disk (Reports the number of disk operations per second for up to 4 disks

  • faults (Reports the trap/interupt rates (per second)

    • in : (non clock) device interupts

    • si : system calls

    • cs : CPU context switches

  • cpu (Reports the breakdown of percentage usage of CPU time (averaged across all CPUs)

    • us : user time

    • si : system time

    • cs : idle time

 

 

Find Out Who Is Logged on And What They Are Doing

w command displays information about the users currently on the machine, and their processes.
# w username
eg : # w sukhi

Tell How Long The System Has Been Running

The uptime command can be used to see how long the server has been running. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.
# uptime

 

Top command to find out Linux cpu usage

$ top



CPU Usage

sar

$ sar -u 10 8
Reports CPU Utilization (10 seconds apart; 8 times).











Time
%usr
%sys
%wio
%idle
11:57:31
72
28
0
0
11:57:41
70
30
0
0
11:57:51
70
30
0
0
11:58:01
68
32
0
0
11:58:11
67
33
0
0
11:58:21
65
28
0
7
11:58:31
73
27
0
0
11:58:41
69
31
0
0
Average
69
30
0
1
%usr: Percent of CPU in user mode
%sys: Percent of CPU in system mode
%wio: Percent of CPU running idle with a process waiting for block I/O
%idle: Percent of CPU that is idle


Memory Usage

The command free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
# free

 

Average CPU Load, Disk Activity

The command iostat report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).
# iostat

 

Linux Track NFS Directory / Disk I/O Stats

# iostat -x –n
# iostat -n

 

Linux Find Out Virtual Memory PAGESIZE

To display size of a page in bytes, enter:
$ getconf PAGESIZE
OR
$ getconf PAGE_SIZE

Collect and Report System Activity

The sar command is used to collect, report, and save system activity information. To see network counter, enter:
# sar -n DEV | more

To display the network counters from the 24th:
# sar -n DEV -f /var/log/sa/sa24 | more

You can also display real time usage using sar:
# sar 4 5

 

Howto collect Linux system utilization data into a file

The sa1 command is designed to be started automatically by the cron command. Type the following command to list files:
# ls /var/log/sa

How do I copy log files?

You can copy all these logs files using ssh/scp or ftp to another computer. You can run use sar command to read binary raw data files, enter
# sar -f sa13


Comparison of CPU utilization

display comparison of CPU utilization; 2 seconds apart; 5 times, use:
# sar -u 2 5                   

Output (for each 2 seconds. 5 lines are displayed):
Linux 2.6.9-42.0.3.ELsmp (www1lab2.xyz.ac.in)         01/13/2007
05:33:24 AM       CPU     %user     %nice   %system   %iowait     %idle
05:33:26 AM       all      9.50      0.00     49.00      0.00     41.50
05:33:28 AM       all     16.79      0.00     74.69      0.00      8.52
05:33:30 AM       all     17.21      0.00     80.30      0.00      2.49
05:33:32 AM       all     16.75      0.00     81.00      0.00      2.25
05:33:34 AM       all     14.29      0.00     72.43      0.00     13.28
Average:          all     14.91      0.00     71.49      0.00     13.61

Where,
  • -u 12 5 : Report CPU utilization. The following values are displayed:

    • %user: Percentage of CPU utilization that occurred while executing at the user level (application).

    • %nice: Percentage of CPU utilization that occurred while executing at the user level with nice priority.

    • %system: Percentage of CPU utilization that occurred while executing at the system level (kernel).

    • %iowait: Percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request.

    • %idle: Percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request.

To get multiple samples and multiple reports set an output file for the sar command. Run the sar command as a background process using.
# sar -o output.file 12 8 >/dev/null 2>&1 &

Better use nohup command so that you can logout and check back report later on:
# nohup sar -o output.file 12 8 >/dev/null 2>&1 &

All data is captured in binary form and saved to a file (data.file). The data can then be selectively displayed ith the sar command using the -f option.
# sar -f data.file

 

 













Multiprocessor Usage

Mpstat : The mpstat command displays activities for each available processor, processor 0 being the first one. mpstat -P ALL to display average CPU utilization per processor:
# mpstat -P ALL


Display the utilization of each CPU individually using mpstat

# mpstat

 

Display five reports of global statistics among all processors at two second intervals, enter:

# mpstat 2 5

 

Display five reports of statistics for all processors at two second intervals, enter:

# mpstat -P ALL 2 5

$ mpstat 10 2
Reports per-processor statistics on Sun Solaris (10 seconds apart; 8 times).




CPU
minf
mjf
xcal
intr
ithr
csw
icsw
migr
smtx
srw
syscl
usr
sys
wt
idl
0
6
8
0
438
237
246
85
0
0
21
8542
23
9
9
59
0
0
29
0
744
544
494
206
0
0
95
110911
65
29
6
0


Process Memory Usage

The command pmap report memory map of a process. Use this command to find out causes of memory bottlenecks.
# pmap -d PID

To display process memory information for pid # 47394, enter:
# pmap -d 47394

To display process mappings, type
$ pmap pid
$ pmap 3724

The -x option can be used to provide information about the memory allocation and mapping types per mapping. The amount of resident, non-shared anonymous, and locked memory is shown for each mapping:

pmap -x 3526



 

Displays The Processes

ps command will report a snapshot of the current processes. ps is just like top but provides more information.
To select all processes use the -A or -e option:
# ps -A

 

Show Long Format Output

# ps -Al
To turn on extra full mode (it will show command line arguments passed to process):
# ps -AlF

 

To See Threads ( LWP and NLWP)

# ps -AlFH

 

To See Threads After Processes

# ps -AlLm

 

Print All Process On The Server

# ps ax
# ps axu

 

Print A Process Tree

# ps -ejH
# ps axjf
# pstree

 

Print Security Information

# ps -eo euser,ruser,suser,fuser,f,comm,label
# ps axZ
# ps -eM

 

See Every Process Running As User Vivek

# ps -U vivek -u vivek u

 

Set Output In a User-Defined Format

# ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
# ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
# ps -eopid,tt,user,fname,tmout,f,wchan

 

Display Only The Process IDs of Lighttpd

# ps -C lighttpd -o pid=
OR
# pgrep lighttpd
OR
# pgrep -u vivek php-cgi

 

Display The Name of PID 55977

# ps -p 55977 -o comm=

 

Find Out The Top 10 Memory Consuming Process

# ps -auxf | sort -nr -k 4 | head -10

 

Find Out top 10 CPU Consuming Process

# ps -auxf | sort -nr -k 3 | head -10

Displays the top 20 CPU users on the system.
$ ps -e -o pcpu -o pid -o user -o args | sort -k 1 | tail -21r








%CPU
PID
  USER
COMMAND
78.1
4789
  oracle
ora_dbwr_DDDS2
8.5
4793
  oracle
ora_lgwr_DDDS2
2.4
6206
  oracle
oracleDDDS2 (LOCAL=NO)
0.1
4797
  oracle
ora_smon_DDDS2
0.1
6207
  oracle
oracleDDDS2 (LOCAL=NO)
etc.
etc.
  etc.
etc.

The PID column can then be matched with the SPID column on the V$PROCESS view to provide more information on the process.
SELECT a.username, 
       a.osuser, 
       a.program, 
       spid, 
       sid, 
       a.serial#
FROM   v$session a,
       v$process b
WHERE  a.paddr = b.addr
AND    spid = '&pid';





Find out who is monopolizing or eating the CPUs

Finally, you need to determine which process is monopolizing or eating the CPUs. Following command will displays the top 10 CPU users on the Linux system.
# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
OR
# ps -eo pcpu,pid,user,args | sort -r -k1 | less

Output:
%CPU   PID USER     COMMAND
  96  2148 vivek    /usr/lib/vmware/bin/vmware-vmx -C /var/lib/vmware/Virtual Machines/Ubuntu 64-bit/Ubuntu 64-bit.vmx -@ ""
 0.7  3358 mysql    /usr/libexec/mysqld --defaults-file=/etc/my.cnf --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --socket=/var/lib/mysql/mysql.sock
 0.4 29129 lighttpd /usr/bin/php
 0.4 29128 lighttpd /usr/bin/php
 0.4 29127 lighttpd /usr/bin/php
 0.4 29126 lighttpd /usr/bin/php
 0.2  2177 vivek    [vmware-rtc]
 0.0     9 root     [kacpid]
 0.0     8 root     [khelper]
Now you know vmware-vmx process is eating up lots of CPU power. ps command displays every process (-e) with a user-defined format (-o pcpu). First field is pcpu (cpu utilization). It is sorted in reverse order to display top 10 CPU eating process.

 

iostat : You can also use iostat command which report Central Processing Unit (CPU) statistics and input/output statistics for devices and partitions. It can be used to find out your system's average CPU utilization since the last reboot.
# iostat


output:
Linux 2.6.15.4 (debian)         Thursday 06 April 2006
 
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
     16.36    0.00    2.99    1.06    0.00   79.59
 
Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
hda               0.00         0.00         0.00         16          0
hdb               6.43        85.57       166.74     875340    1705664
hdc               0.03         0.16         0.00       1644          0
sda               0.00         0.00         0.00         24          0 
You may want to use following command, which gives you three outputs every 5 seconds (as previous command gives information since the last reboot):$ iostat -xtc 5 3

 

 

How to count a word, line, character

wc
This command is used for word count.
cat sukhi.txt | wc -l    // for line count
cat sukhi.txt | wc -m   //for charecter count
cat sukhi.txt | wc -w   // for word count

 

How to find the count of files which starts with 'r' in a directory

cat /home/oracle/* | ls r* | wc

This is the command for finding the count of files that strats with character
'r' from a directory. Here r* represents list the file starts with 'r'. 'wc' is the count of the listed files.

 

How to search a pattern and print the contents

cat description.txt | grep 'india'

This is the command to search a pattern and print that. Here Grep command is used for patern seacrhing and cat command is used to print and | pipe symbol is used to concatenate .

 

grep - globally search for regular expression and printout

grep
This commands represent 'globally search fro regular expression and printout '. It searches for perticular pattern of characters and displays all lines that contain that pattern. grep expext a standard input , if we give a line as input , it searches the pattern in that line.

How do I forcefully unmount a Linux disk partition?

If your device name is /dev/sdb1, enter the following command as root user:
# lsof | grep '/dev/sda1'
Output:
vi 4453       vivek    3u      BLK        8,1                 8167 /dev/sda1

Above output tells that user vivek has a vi process running that is using /dev/sda1. All you have to do is stop vi process and run umount again. As soon as that program terminates its task, the device will no longer be busy and you can unmount it with the following command:
# umount /dev/sda1

 

Linux fuser command to forcefully unmount a disk partition

Suppose you have /dev/sda1 mounted on /mnt directory then you can use fuser command as follows:
Type the command to unmount /mnt forcefully:
# fuser -km /mnt

Where,
  • -k : Kill processes accessing the file.

  • -m : Name specifies a file on a mounted file system or a block device that is mounted. In above example you are using /mnt


Linux umount command to unmount a disk partition
You can also try umount command with –l option:
# umount -l /mnt

Where,
  • -l : Also known as Lazy unmount. Detach the filesystem from the filesystem hierarchy now, and cleanup all references to the filesystem as soon as it is not busy anymore. This option works with kernel version 2.4.11+ and above only.


If you would like to unmount a NFS mount point then try following command:
# umount -f /mnt

Where,
  • -f: Force unmount in case of an unreachable NFS system

Caution: Using these commands or option can cause data loss for open files; programs which access files after the file system has been unmounted will get an error.

 

GUI tools for your laptops/desktops

Above tools/commands are quite useful on remote server. For local system with X GUI installed you can try out gnome-system-monitor. It allows you to view and control the processes running on your system. You can access detailed memory maps, send signals, and terminate the processes.
$ gnome-system-monitor


Various Kernel Statistics

/proc file system provides detailed information about various hardware devices and other Linux kernel information. Common /proc examples:
# cat /proc/cpuinfo
# cat /proc/meminfo
# cat /proc/zoneinfo
# cat /proc/mounts

Automatic Startup Scripts on Linux

Create a file in the "/etc/init.d/" directory, in this case the file is called "myservice", containing the commands you wish to run at startup and/or shutdown.

Use the chmod command to set the privileges to 750.
chmod 750 /etc/init.d/myservice

Link the file into the appropriate run-level script directories.
ln -s /etc/init.d/myservice /etc/rc0.d/K10myservice
ln -s /etc/init.d/myservice /etc/rc3.d/S99myservice

Associate the "myservice" service with the appropriate run levels.
chkconfig --level 345 dbora on

The script should now be automatically run at startup and shutdown (with "start" or "stop" as a commandline parameter) like other service initialization scripts.

 

NFS Mount (Sun)

The following deamons must be running for the share to be seen by a PC.
  • /usr/lib/nfs/nfsd -a

  • /usr/lib/nfs/mountd

  • /opt/SUNWpcnfs/sbin/rpc.pcnfsd

To see a list of the nfs mounted drives already present type.
exportfs

First the mount point must be shared so it can be seen by remote machines.
share -F nfs -o ro /cdrom

Next the share can be mounted on a remote machine by root using.
mkdir /cdrom#1



mount -o ro myhost:/cdrom /cdrom#1

 

Useful Files

Here are some files that may be of use.







Path
Contents
/etc/passwd
User settings
/etc/group
Group settings for users.
/etc/hosts
Hostname lookup information.
/etc/system
Kernel parameters for Solaris.
/etc/sysconfigtab
Kernel parameters for Tru64.




Network Statistics

ss
The ss command is used to dump socket statistics

 

Display Sockets Summary

List currently established, closed, orphaned and waiting TCP sockets, enter:
# ss -s

 

Display All Open Network Ports

# ss -l

Type the following to see process named using open socket:
# ss –pl

Find out who is responsible for opening socket / port # 4949:
# ss -lp | grep 4949

 

Display All TCP Sockets

# ss -t -a

 

Display All UDP Sockets

# ss -u -a

 

Display All Established SMTP Connections

# ss -o state established '( dport = :smtp or sport = :smtp )'

 

Display All Established HTTP Connections

# ss -o state established '( dport = :http or sport = :http )'

 

Find All Local Processes Connected To X Server

# ss -x src /tmp/.X11-unix/*

List All The Tcp Sockets in State FIN-WAIT-1

List all the TCP sockets in state -FIN-WAIT-1 for our httpd to network 202.54.1/24 and look at their timers:
# ss -o state fin-wait-1 '( sport = :http or sport = :https )' dst 202.54.1/24

 

Get Detailed Information about Particular IP address Connections Using netstat Command

You can also list abusive IP address using this method.
# netstat -nat | awk '{print $6}' | sort | uniq -c | sort –n

Dig out more information about a specific ip address:
# netstat -nat |grep {IP-address} | awk '{print $6}' | sort | uniq -c | sort –n

Busy server can give out more information:
# netstat -nat |grep 202.54.1.10 | awk '{print $6}' | sort | uniq -c | sort –n

Get List Of All Unique IP Address

To print list of all unique IP address connected to server, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq

To print total of all unique IP address, enter:
# netstat -nat | awk '{ print $5}' | cut -d: -f1 | sed -e '/^$/d' | uniq | wc -l

 

Find Out If Box is Under DoS Attack or Not

If you think your Linux box is under attack, print out a list of open connections on your box and sorts them by according to IP address, enter:
# netstat -atun | awk '{print $5}' | cut -d: -f1 | sed -e '/^$/d' |sort | uniq -c | sort -n

 

Display Summary Statistics for Each Protocol

Simply use netstat -s:
# netstat -s | less
# netstat -t -s | less
# netstat -u -s | less
# netstat -w -s | less
# netstat -s

 

netstat command to display established connections

Type the command as follows:
$ netstat -nat
To display client / server ESTABLISHED connections only:
$ netstat -nat | grep 'ESTABLISHED'
      

How do I use tcptract to monitor and track TCP connections ?

tcptrack requires only one parameter to run i.e. the name of an interface such as eth0, eth1 etc. Use the -i flag followed by an interface name that you want tcptrack to monitor.
# tcptrack -i eth0
# tcptrack -i eth1

You can just monitor TCP port 25 (SMTP)
# tcptrack -i eth0 port 25

The next example will only show web traffic monitoring on port 80:
# tcptrack -i eth1 port 80

tcptrack can also take a pcap filter expression as an argument. The format of this filter expression is the same as that of tcpdump and other libpcap-based sniffers. The following example will only show connections from host 76.11.22.12:
# tcptrack -i eth0 src or dst 76.11.22.12

 

Display Interface Table

You can easily display dropped and total transmitted packets with netstat for eth0:
# netstat --interfaces eth0

Other netstat related articles / tips:

$ man netstat
$ man cut
$ man awk
$ man sed
$ man grep

 

Get Information about All Running Services Remotely

All you have to do is open /etc/inetd.conf under UNIX / Linux:
# vi /etc/inetd.conf

Append following line:
netstat stream tcp nowait root /bin/netstat netstat –a

Restart inetd:
# /etc/init.d/openbsd-inetd restart

Next, use telnet to connect to the netstat service (port 15) and get network connection information:
$ telnet server-name netstat
$ telnet 192.168.1.5 15

Linux / UNIX Find Out What Program / Service is Listening on a Specific TCP Port

Under Linux and UNIX you can use any one of the following command to get listing on a specific TCP port:
=> lsof : list open files including ports.
=> netstat : The netstat command symbolically displays the contents of various network-related data and information.

 

lsof

Type the following command to see IPv4 port(s), enter:
# lsof -Pnl +M -i4

Type the following command to see IPv6 listing port(s), enter:
# lsof -Pnl +M -i6

First column COMMAND - gives out information about program name. Please see output header for details. For example, gweather* command gets the weather report weather information from the U.S National Weather Service (NWS) servers (140.90.128.70), including the Interactive Weather Information Network (IWIN) and other weather services.

Where,
  1. -P : This option inhibits the conversion of port numbers to port names for network files. Inhibiting the conver-
    sion may make lsof run a little faster. It is also useful when port name lookup is not working properly.

  2. -n : This option inhibits the conversion of network numbers to host names for network files. Inhibiting conversion may make lsof run faster. It is also useful when host name lookup is not working properly.

  3. -l : This option inhibits the conversion of user ID numbers to login names. It is also useful when login name lookup is working improperly or slowly.

  4. +M : Enables the reporting of portmapper registrations for local TCP and UDP ports.

  5. -i4 : IPv4 listing only

  6. -i6 : IPv6 listing only

 


netstat

Type the command as follows:
# netstat -tulpn
OR
# netstat -npl

Last column PID/Program name gives out information regarding program name and port.
Where,
  • -t : TCP port

  • -u : UDP port

  • -l : Show only listening sockets.

  • -p : Show the PID and name of the program to which each socket / port belongs

  • -n : No DNS lookup (speed up operation)

 

/etc/services file

/etc/services is a plain ASCII file providing a mapping between friendly textual names for internet services, and their underlying assigned port numbers and protocol types. Every networking program should look into this file to get the port number (and protocol) for its service. You can view this file with the help of cat or less command:
$ cat /etc/services
$ grep 110 /etc/services
$ less /etc/services

Detailed Network Traffic Analysis

The tcpdump is simple command that dump traffic on a network. However, you need good understanding of TCP/IP protocol to utilize this tool. For.e.g to display traffic info about DNS, enter:
# tcpdump -i eth1 'udp port 53'

To display all IPv4 HTTP packets to and from port 80, i.e. print only packets that contain data, not, for example, SYN and FIN packets and ACK-only packets, enter:
# tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

To display all FTP session to 202.54.1.5, enter:
# tcpdump -i eth1 'dst 202.54.1.5 and (port 21 or 20'

To display all HTTP session to 192.168.1.5:
# tcpdump -ni eth0 'dst 192.168.1.5 and tcp and port http'

Use wireshark to view detailed information about files, enter:
# tcpdump -n -i eth1 -s 0 -w output.txt src or dst port 80

 

Monitor HTTP Packets ( packet sniffing )

Login as a root and type the following command at console:
# tcpdump -n -i {INTERFACE} -s 0 -w {OUTPUT.FILE.NAME} src or dst port 80
# tcpdump -n -i eth1 -s 0 -w output.txt src or dst port 80


System Calls

Run strace against /bin/foo and capture its output to a text file in output.txt:
$ strace -o output.txt /bin/foo

You can strace the webserver process and see what it's doing. For example, strace php5 fastcgi process, enter:
$ strace -p 22254 -s 80 -o /tmp/debug.lighttpd.txt

To see only a trace of the open, read system calls, enter :
$ strace -e trace=open,read -p 22254 -s 80 -o debug.webserver.txt

Where,
  • -o filename : Write the trace output to the file filename rather than to screen (stderr).

  • -p PID : Attach to the process with the process ID pid and begin tracing. The trace may be terminated at any time by a keyboard interrupt signal (hit CTRL-C). strace will respond by detaching itself from the traced process(es) leaving it (them) to continue running. Multiple -p options can be used to attach to up to 32 processes in addition to command (which is optional if at least one -p option is given).

  • -s SIZE : Specify the maximum string size to print (the default is 32).


Refer to strace man page for more information:
$ man strace

Linux / UNIX: Scanning network for open ports with nmap command

nmap port scanning

TCP Connect scanning for localhost and network 192.168.0.0/24
# nmap -v -sT localhost
# nmap -v -sT 192.168.0.0/24

 

nmap TCP SYN (half-open) scanning

# nmap -v -sS localhost
# nmap -v -sS 192.168.0.0/24

 

nmap TCP FIN scanning

# nmap -v -sF localhost
# nmap -v -sF 192.168.0.0/24

 

nmap TCP Xmas tree scanning

Useful to see if firewall protecting against this kind of attack or not:
# nmap -v -sX localhost
# nmap -v -sX 192.168.0.0/24

 

nmap TCP Null scanning

Useful to see if firewall protecting against this kind attack or not:
# nmap -v -sN localhost
# nmap -v -sN 192.168.0.0/24

nmap TCP Windows scanning

# nmap -v -sW localhost
# nmap -v -sW 192.168.0.0/24

 

nmap TCP RPC scanning

Useful to find out RPC (such as portmap) services
# nmap -v -sR localhost
# nmap -v -sR 192.168.0.0/24

 

nmap UDP scanning

Useful to find out UDP ports
# nmap -v -O localhost
# nmap -v -O 192.168.0.0/24

 

nmap remote software version scanning

You can also find out what software version opening the port.
# nmap -v -sV localhost
# nmap -v -sV 192.168.0.0/24





Regards,
Sukhwinder Singh

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.