Linux basics

linux bash programming basics

Shell Script Basics

A shell script is a plain-text file that contains shell commands. It can be executed by typing its name into a shell (Linux Terminal), or by placing its name in another shell script.
To be executable, a shell script file must meet some conditions:
The file must have a special first line that names an appropriate command processor. The following will work in most cases:

#!/bin/bash

Note: If this command doesn't work, you will need to find out where your Bash shell executable is located and substitute that location in the above example.
Here is one way to find out:

$ whereis bash

The file must be made executable by changing its permission bits. An example:

$ chmod +x (shell script filename)
            or
$ chmod 755 (shell script filename)

A shell script file may optionally have an identifying suffix, like ".sh". This only helps the user remember which files are which. The command processor responsible for executing the file uses the executable bit, plus the file's first line, to decide how to handle a shell script file.
One normally executes a shell script this way: '$ ./scriptname.sh' 'This special entry is a way to tell the command processor that the desired script is located in the current directory. Always remember: if you cannot get your shell script to run, remember this trick to provide its location as well as its name.

First Shell Script

This will get you past the details of writing and launching a simple script.  Choose a text editor you want to use. It can be a command-line editor like emacs, pico or vi, vim, or an X Windows editor if you have this option. Run your choice of editor and type the following lines:

#!/bin/bash
echo "Hello, world."

NOTE: Be sure to place a linefeed at the end of your script. Forgetting a terminating linefeed is a common beginner's error.

1. Save the file in the current working directory as "myscript.sh".
2. Move from the text editor to a command shell.
3. From the command shell, type this: $ chmod +x myscript.sh
4. To execute the script, type this: $ ./myscript.sh

Output: Hello, world.

Linux Commands

Introduction

The purpose of this document is to provide the reader with a fast and simple introduction to using
the Linux command shell and some of its basic utilities.

command shell

1- A program that interprets commands
2- Allows a user to execute commands by typing them manually at a terminal, or automatically
in programs called shell scripts .
3- A shell is not an operating system. It is a way to interface with the operating system and run
commands.

BASH

1- BASH = Bourne Again SHell
2- Bash is a shell written as a free replacement to the standard Bourne Shell (/bin/sh)
originally written by Steve Bourne for UNIX systems
3- It has all of the features of the original Bourne Shell, plus additions that make it easier to
program with and use from the command line.

Features of BASH

Case Sensitivity:
commands and filenames are case sensitive, meaning
that typing “EXIT” instead of the proper “exit” is a mistake.

“\” vs. “/”:
Linux/UNIX, the“/” is the directory separator, and the “\” is an escape character. More
about these special characters in a minute!

Filenames:
Periods can be placed at any part of the filename, and “extensions” may be interpreted differently by all
programs, or not at all.

Special Characters

Character	Description
\	Escape character. If you want to reference a special character, you must “escape” it with a backslash first.             Example:touch /tmp/filename\*
/	Directory separator, used to separate a string of directory names.        Example: /usr/src/linux
.	Current directory. Can also “hide” files when it is the first character in a filename.
..	Parent directory
~	User's home directory
*	Represents 0 or more characters in a filename, or by itself, all files in a directory.    Example:pic*2002 can represent the files pic2002, picJanuary2002,                 picFeb292002, etc.
?	Represents a single character in a filename.         Example:hello?.txt can represent hello1.txt, helloz.txt, but not                 hello22.txt
[]	Can be used to represent a range of values, e.g. [0-9], [A-Z], etc.         Example:hello[0-2].txt represents the names hello0.txt,                 hello1.txt, and hello2.txt
|	“Pipe”. Redirect the output of one command into another command.       Example: ls | more
>	Redirect output of a command into a new file. If the file already exists, over-write it.      Example:   ls > myfiles.txt
>>	Redirect the output of a command onto the end of an existing file.         Example:echo “Mary 555-1234” >> phonenumbers.txt
<	Redirect a file as input to a program.         Example:more < phonenumbers.txt
;	Command separator. Allows you to execute multiple commands on a single line.        Example: cd /var/log ; less messages
&&	Command separator as above, but only runs the second command if the first one finished without errors.         Example:cd /var/logs && less messages
&	Execute a command in the background, and immediately get your shell back.        Example:find / -name core > /tmp/corefiles.txt &

Executing Commands

The Command PATH:

1-Most common commands are located in your shell's “PATH”, meaning that you can just
type the name of the program to execute it.
Example: Typing “ ls” will execute the “ ls” command.

2-Your shell's “PATH” variable includes the most common program locations, such as
/bin, /usr/bin, /usr/X11R6/bin, and others.

3-To execute commands that are not in your current PATH, you have to give the complete
location of the command.

Examples:    /home/bob/myprogram
                    ./program (Execute a program in the current directory)
                     ~/bin/program (Execute program from a personal bin directory)

Command Syntax

1-  Commands can be run by themselves, or you can pass in additional arguments to make them do
different things. Typical command syntax can look something like this:
command [-argument] [-argument] [--argument] [file]
Examples:

ls	List files in current directory
ls -l	Lists files in “long” format
ls -l --color	As above, with colourized output
cat filename	Show contents of a file
cat -n filename	Show contents of a file, with line numbers

Getting Help

When you're stuck and need help with a Linux command, help is usually only a few keystrokes
away! Help on most Linux commands is typically built right into the commands themselves,
available through online help programs (“man pages” and “info pages”), and of course online.

Using a Command's Built-In Help

Many commands have simple “help” screens that can be invoked with special command flags.
These flags usually look like “-h” or “--help”.      
Example:  grep --help

Online Manuals: “Man Pages”

The best source of information for most commands can be found in the online manual pages,
known as “man pages” for short. To read a command's man page, type “man command”.

Examples: man ls Get help on the “ls” command.
                man man A manual about how to use the manual!

To search for a particular word within a man page, type “/word”. To quit from a man page, just
type the “Q” key.

Sometimes, you might not remember the name of Linux command and you need to search for it.
For example, if you want to know how to change a file's permissions, you can search the man page
descriptions for the word “permission” like this:
man -k permission

If you look at the output of this command, you will find a line that looks something like:
             chmod                       (1)    - change file access permissions
Now you know that “chmod” is the command you were looking for. Typing “man chmod” will
show you the chmod command's manual page!

Info Pages

Info pages are similar to man page, but instead of being displayed on one long scrolling screen, they
are presented in shorter segments with links to other pieces of information. Info pages are accessed
with the “info” command, or on some Linux distributions, “pinfo” (a nicer info browser).

For example: info df         Loads the “df” info page.

Navigating the Linux Filesystem

The Linux filesystem is a tree-like hierarchy hierarchy of directories and files. At the base of the
filesystem is the “/” directory, otherwise known as the “root” (not to be confused with the root
user). Unlike DOS or Windows filesystems that have multiple “roots”, one for each disk drive, the
Linux filesystem mounts all disks somewhere underneath the / filesystem. The following table
describes many of the most common Linux directories.

The Linux Directory Layout

Directory	Description
	The nameless base of the filesystem. All other directories, files, drives, and devices are attached to this root. Commonly (but incorrectly) referred to as the “slash” or “/” directory. The “/” is just a directory separator, not a directory itself.
/bin	Essential command binaries (programs) are stored here (bash, ls, mount, tar, etc.)
/boot	Static files of the boot loader.
/dev	Device files. In Linux, hardware devices are acceessd just like other files, and they are kept under this directory.
/etc	Host-specific system configuration files.
/home	Location of users' personal home directories (e.g. /home/susan).
/lib	Essential shared libraries and kernel modules.
/proc	Process information pseudo-filesystem. An interface to kernel data structures.
/root	The root (superuser) home directory.
/sbin	Essential system binaries (fdisk, fsck, init, etc).
/tmp	Temporary files. All users have permission to place temporary files here.
/usr	The base directory for most shareable, read-only data (programs, libraries,documentation, and much more).
/usr/bin	Most user programs are kept here (cc, find, du, etc.).
/usr/include	Header files for compiling C programs.
/usr/lib	Libraries for most binary programs.
/usr/local	“Locally” installed files. This directory only really matters in environments where files are stored on the network. Locally-installed files go in /usr/local/bin, /usr/local/lib, etc.). Also often used for software packages installed from source, or software not officially shipped with the distribution.
/usr/sbin	Non-vital system binaries (lpd, useradd, etc.)
/usr/share	Architecture-independent data (icons, backgrounds, documentation, terminfo,man pages, etc.).
/usr/src	Program source code. E.g. The Linux Kernel, source RPMs, etc.
/usr/X11R6	The X Window System.
/var	Variable data: mail and printer spools, log files, lock files, etc.

Commands for Navigating the Linux Filesystems

The first thing you usually want to do when learning about the Linux filesystem is take some time
to look around and see what's there! These next few commands will:
a) Tell you where you are,
b) take you somewhere else, and
c) show you what's there. The following table describes the basic operation of the pwd, cd, and
ls commands, and compares them to certain DOS commands that you might already be familiar with.

Linux Command	Description
pwd	“Print Working Directory”.  Shows the current location in the directory tree.
cd	“Change Directory”. When typed all by itself, it returns you to your home directory.
cd directory	Change into the specified directory Example: cd /usr/src/linux
cd ~	“~” is an alias for your home directory. It can be used as a shortcut to your “home”, or other directories relative to your home.
cd ..	Move up one directory. For example, if you are in /home/vic and you type “cd ..”, you will end up in /home.
cd -	Return to previous directory. An easy way to get back to your previous location!
ls	List all files in the current directory, in column format.
ls directory	List the files in the specified directory. ls directory Example: ls /var/log
ls -l	List files in “long” format, one file per line. This also shows you additional info about the file, such as ownership, permissions, date, and size.
ls -a	List all files, including “hidden” files. Hidden files are those files that begin with a “.”, e.g. The .bash_history file in your home directory.
ls -ld directory	A “long” list of “directory”, but instead of showing the directory contents, show the directory's detailed information. For example, compare the output of the following two commands: ls -l /usr/bin ls -ld /usr/bin
ls /usr/bin/d*	List all files whose names begin with the letter “d” in the /usr/bin directory.

Piping and Re-Direction

It is the responsibility of the programmer or user to combine these utilities to make more useful
command sequences.

Piping Commands Together

The pipe character, “|”, is used to chain two or more commands together. The output of the first
command is “piped” into the next program, and if there is a second pipe, the output is sent to the
third program, etc. For example:
               ls -la /usr/bin | less
In this example, we run the command “ls -la /usr/bin”, which gives us a long listing of all
of the files in /usr/bin. Because the output of this command is typically very long, we pipe the
output to a program called “less”, which displays the output for us one screen at a time.

Redirecting Program Output to Files

There are times when it is useful to save the output of a command to a file, instead of displaying it
to the screen. For example, if we want to create a file that lists all of the MP3 files in a directory,
we can do something like this, using the “>” redirection character:
             ls -l /home/vic/MP3/*.mp3 > mp3files.txt
A similar command can be written so that instead of creating a new file called mp3files.txt,
we can append to the end of the original file:
             ls -l /home/vic/extraMP3s/*.mp3 >> mp3files.txt

Other Linux Commands

Working With Files and Directories

These commands can be used to: find out information about files, display files, and manipulate
them in other ways (copy, move, delete).

Linux Command	Description
file	Find out what kind of file it is. For example, “file /bin/ls” tells us that it is a Linux executable file.
cat	Display the contents of a text file on the screen. For example: cat mp3files.txt would display the file we created in the previous section.
head	Display the first few lines of a text file. Example: head /etc/services
tail	Display the last few lines of a text file. Example: tail /etc/services
tail -f	Display the last few lines of a text file, and then output appended data as the file grows (very useful for following log files!). Example: tail -f /var/log/messages
cp	Copies a file from one location to another. Example: cp mp3files.txt /tmp (copies the mp3files.txt file to the /tmp directory)
mv	Moves a file to a new location, or renames it. For example: mv mp3files.txt /tmp (copy the file to /tmp, and delete it from the original location)
rm	Delete a file. Example: rm /tmp/mp3files.txt
mkdir	Make Directory. Example: mkdir /tmp/myfiles/
rmdir	Remove Directory. Example: rmdir /tmp/myfiles/

Finding Things

The following commands are used to find files. “ls” is good for finding files if you already know
approximately where they are, but sometimes you need more powerful tools such as these:
          

Linux Command	Description
which	Shows the full path of shell commands found in your path. For example, if you want to know exactly where the “grep” command is located on the filesystem, you can type “which grep”. The output should be something like: /bin/grep
whereis	Locates the program, source code, and manual page for a command (if all information is available). For example, to find out where “ls” and its man page are, type: “whereis ls” The output will look something like: ls: /bin/ls /usr/share/man/man1/ls.1.gz
locate	A quick way to search for files anywhere on the filesystem. For example, you can find all files and directories that contain the name “mozilla” by typing: locate mozilla
find	A very powerful command, but sometimes tricky to use. It can be used to search for files matching certain patterns, as well as many other types of searches. A simple example is: find . -name \*mp3 This example starts searching in the current directory “.” and all sub- directories, looking for files with “mp3” at the end of their names.

Informational Commands

The following commands are used to find out some information about the user or the system.

Linux Command	Explanation
ps	Lists currently running process (programs).
w	Show who is logged on and what they are doing.
id	Print your user-id and group id's
df	Report filesystem disk space usage (“Disk Free” is how I remember it)
du	Disk Usage in a particular directory. “du -s” provides a summary for the current directory.
top	Displays CPU processes in a full-screen GUI. A great way to see the activity on your computer in real-time. Type “Q” to quit.
free	Displays amount of free and used memory in the system.
cat /proc/cpuinfo	Displays information about your CPU.
cat /proc/meminfo	Display lots of information about current memory usage.
uname -a	Prints system information to the screen (kernel version, machine type,etc.)

Other Utilities

Here are some other commands that are useful to know.

Linux Command	Description
clear	Clear the screen
echo	Display text on the screen. Mostly useful when writing shell scripts. For example: echo “Hello World”
more	Display a file, or program output one page at a time. Examples: more mp3files.txt ls -la | more
less	An improved replacement for the “more” command. Allows you to scroll backwards as well as forwards.
grep	Search for a pattern in a file or program output. For example, to find out which TCP network port is used by the “nfs” service, you can do this: grep “nfs” /etc/services This looks for any line that contains the string “nfs” in the file “/etc/services” and displays only those lines.
lpr	Print a file or program output. Examples: lpr mp3files.txt - Print the mp3files.txt file ls -la | lpr  - Print the output of the “ls -la” command.
sort	Sort a file or program output. Example: sort mp3files.txt
su	“Switch User”. Allows you to switch to another user's account temporarily. The default account to switch to is the root/superuser account. Examples: su           - Switch the root account su -         - Switch to root, and log in with root's environment su larry  - Switch to Larry's account

Shortcuts to Make it all Easier!

When you start using the Bash shell more often, you will appreciate these shortcuts that can save
you very much typing time.

Shortcut	Description
Up/Down Arrow Keys	Scroll through your most recent commands. You can scroll back to an old command, hit ENTER, and execute the command without having to re-type it.
“history” command	Show your complete command history.
TAB Completion	If you type a partial command or filename that the shell recognizes, you can have it automatically completed for you if you press the TAB key. Try typing the first few characters of your favourite Linux command, then hit TAB a couple of times to see what happens.
Complete recent commands with “!”	Try this: Type “!” followed by the first couple of letters of a recent command and press ENTER! For example, type: find /usr/bin -type f -name m\* ...and now type: !fi
Search your command history with CTRL-R	Press CTRL-R and then type any portion of a recent Search your command history with CTRL-R , command. It will search the commands for you, and once  you find the command you want, just press ENTER.
Scrolling the screen with Shift-PageUp and Page Down	Scroll back and forward through your terminal.

linux format and mount hard drives

Following command will list all detected hard disks

# fdisk -l
# fdisk -l | grep '^Disk'

To format Linux partitions using ext2fs on the new disk

# mkfs.ext3 /dev/hdd

First create a mount point /disk1 and use mount command to mount /dev/hdd, enter

# mkdir /mnt/hdd
# mount /dev/hdd /mnt/hdd
# df -H

Hard Disk Clone

# dd if=/dev/sda of=/dev/sdb

Linux Security with Examples

BIOS Security

Always set a password on BIOS to disallow booting from CD by changing the BIOS settings. This will block undesired people from trying to boot your Linux system with a special boot disk and will protect you from people trying to change BIOS feature like allowing boot from floppy drive or booting the server without password prompt.

LILO Security

Add the three parameters in "/etc/lilo.conf" file i.e. time-out, restricted and password. These options will ask for password if boot time options (such as "linux single") are passed to the boot loader.
Step 1
Edit the lilo.conf file (vi /etc/lilo.conf) and add or change the three options :
boot=/dev/hda
map=/boot/map
install=/boot/boot.b
time-out=00 #change this line to 00
prompt
Default=linux
restricted #add this line
password=<password> #add this line and put your password
image=/boot/vmlinuz-2.2.14-12
label=linux
initrd=/boot/initrd-2.2.14-12.img
root=/dev/hda6
read-only

Step 2
The "/etc/lilo.conf" file should be readable by only root because it contains unencrypted passwords.
[root@kapil /]# chmod 600 /etc/lilo.conf (will be no longer world readable).

Step 3
Update your configuration file "/etc/lilo.conf" for the change to take effect.
[Root@kapil /]# /sbin/lilo -v (to update the lilo.conf file).

Step 4
One more security measure you can take to secure the "/etc/lilo.conf" file is to set it immutable, using the chattr command.
* To set the file immutable simply, use the command:
[root@kapil /]# chattr +i /etc/lilo.conf
This will prevent any changes (accidental or otherwise) to the "lilo.conf" file.Remote login Security

GRUP loader Security

To begin securing the boot menu, first generate an MD5 password. Use following command to genrate a password.

[root@localhost ~]# grub-md5-crypt
Password:
Retype password:
$1$y8Rdu/$aThXVSLRKC89BrmwhGWer.

Edit the GRUB configuration file /boot/grub/grub.conf. Open the file and below the timeout line in the main section of the document, add the following line:

password --md5 <password-hash>

Replace <password-hash> with the value returned by /sbin/grub-md5-crypt
The next time the system boots, the GRUB menu prevents access to the editor or command interface without first pressing p followed by the GRUB password.

Remote Login

We must disable remote login for root user. Open sshd_config file

[root@localhost ~]# vim /etc/ssh/sshd_config

Uncomment following line and replace 'yes' by 'no'.

Old	New
#PermitRootLogin yes	PermitRootLogin no

Restart ssh service or wait for the reboot to test the changes

/etc/rc.d/init.d/sshd restart

Linux services

List of LINUX ServicesLinux is a UNIX-like operating system. It is one of the most prominent open source software. The below service commands in alphabetical order will help you to know about the services which is needed in Home PC and Server.

autofs

Automount service. One of the several automounting of file systems on demand services i.e., it auto mount CDs and other file system-like devices and media.

cpuspeed

Power management based CPU Speed control. It adjusts the CPU speed dynamically based on the demand for processing power.

crond

Periodic Command Scheduler. It executes scheduled commands according to the /etc/crontab file. It is one of an essential part of Linux systems.

gpm

Console mode mouse driver. Provides mouse support to Linux. It is used for cut/paste on the non-graphical console.

haldaemon

HAL is a Hardware Abstraction Layer. It is a Hardware Monitoring System. Auto-recognizes various kinds of hardware and mountable media.

messagebus

Event monitoring service. Provides a communication bus for dbus.

microcode_ctl

Microcode utility for Intel IA32 processors.

netfs

Mounts and unmounts Network Fils System (NFS), Windows (SMB), and Netware (NCP) file systems.

network

Provides network services to system.

sshd

SSH daemon. It is required for SSH access to the system.

syslog

System Logging. System logging daemon which records system events to log files.

xfs

X Font Server. Used by X windows to support a graphical desktop.

xinetd

Super daemon, launches network related daemons on demand.

Linux useful commands

Linux swap memory

Linux divides its physical RAM (random access memory) into chucks of memory called pages. Swapping is the process whereby a page of memory is copied to the pre-configured space on the hard disk, called swap space, to free up that page of memory. The combined sizes of the physical memory and the swap space is the amount of virtual memory available.
Swapping is necessary for two important reasons. First, when the system requires more memory than is physically available, the kernel swaps out less used pages and gives memory to the current application (process) that needs the memory immediately. Second, a significant number of the pages used by an application during its startup phase may only be used for initialization and then never used again. The system can swap out those pages and free the memory for other applications or even for the disk cache.
Linux has two forms of swap space: the swap partition and the swap file. The swap partition is an independent section of the hard disk used solely for swapping; no other files can reside there. The swap file is a special file in the filesystem that resides amongst your system and data files.
To see what swap space you have, use the command:

$ swapon -s

swap file

Linux supports swap files that you can create, prepare, and mount in a fashion similar to that of a swap partition. The advantage of swap files is that you don't need to find an empty partition or repartition a disk to add additional swap space.

To create a swap file, use the dd command to create an empty file. To create a 1GB file, An example:

$ dd if=/dev/zero of=/swapfile bs=1024 count=1048576

/swapfile is the name of the swap file, and the count of 1048576 is the size in kilobytes (i.e. 1GB).

Prepare the swap file using mkswap just as you would a partition, but this time use the name of the swap file, An example

$ mkswap /swapfile

And similarly, mount it using the swapon command:

$ swapon /swapfile

Open file /etc/fstab and add a new entry for swap file:

/swapfile    none    swap    sw    0    0

After adding this entry, swap is used by system after booting automatically.

Linux X-windows settings

How to use graphical environoment while connecting a user using Terminal? If a user login performed from Linux Terminal we can enable x-windows by two methods. You can use eigther of the methods to use x-windows for a specific user.

1. Using ssh login
2. Using configuration files

Create a User

$ useradd (user name)

User Passwd Add

$ userpasswd (insert passwd)

linux useful files

Setting yum repository

There 2 methods to configure repository in linux

First method

Open /etc/yum.conf and enter a new entry with yum repository location.

[main]
cachedir=/var/cache/yum
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
pkgpolicy=newest
distroverpkg=redhat-release
tolerant=1
exactarch=1
obsoletes=1
gpgcheck=1
plugins=1
metadata_expire=1800

main section is provided by linux itself. Add a new entry at the end of file. Example:-

[myrepo]
name=RHEL 5 $releasever - $basearch
baseurl=http://local/path/to/yum/repository/
enabled=1

Second method

We just need to add new entry in CentOS-Base.repo. Example:-

[packagetypename]
name=CentOS-$releasever - Base
mirrorlist=http://provide/the/mirror/list/repo/location
gpgcheck=1
gpgkey=http://if/gpg/key/please/provide/the/location/this/is/optional

After adding new location to yum repository use "yum clean all" command to apply the changes in running service:

Creating Sudo User

sudo is a package which will allow priveleged users to run commands as other users. This is sort of like assigning users to different groups to give them special permissions to files. However, this can allow users acccess to specific commands on specific machines, making it a more effective and more organized way of giving special priveleges to users.

First, you'll have to get the sudo package. (This is already installed in your system)
you should find a file called /etc/sudoers

using root login run the following command:

sudoedit /etc/sudoers

Add new entry followed by:
root ALL=(ALL) ALL
oracle ALL=(ALL) ALL

Sudo Login

use the following command

sudo su - sudousername

sudo su - oracle

Changing linux runlevel

Open /etc/inittab file
Go to line id:<runlevel>:initdefault:
Replace old <runlevel> to new value (1-5)

Setting a BIOS Password

Your computers BIOS is the first program that is run when your computer starts. You can tell the BIOS to ask for a password when it starts, thus restricting access to your computer.Turn on or reboot your computer. The screen will display a series of diagnostics and a memory check.
A message like "Hit the <DEL> key to enter the BIOS setup program" will appear.

Network settings

tags: linux service files, linux configuration files, linux etc files

OSI Model

Physical Layer

covers the physical interface between devices and the rules by which bits are passed from one to another.

Data Link Layer

attempts o make the physical link reliable and provides the means to activate, maintain, and deactivate the link.

Network Layer

provides for the transfer of information between end systems across some sort communications network.

Transport Layer

provides a mechanism for the exchange of data between end system.

Session Layer

provides the mechanism for controlling the dialogue between applications in end systems.

Presentation Layer

defines the format of the data to be exchanged between applications and offers application programs a set of data transformation services.

Application Layer

provides a means for application programs to access the OSI environment.

Start Linux System on Windows

Sun Virtual box setup

step1

Step2

Step2

Step3

Step4

Step5 

Step6  

Step7

Step8

Step9

Step10

Step11

Step12

Step13

New Hard-Disk Setup

Step14

Step15

Step16 

 Attach Vdi or Vmdk File

Step17 

Step18

Step19

Step20

Step21

Step22

Unix IPC basics

Interprocess Communications techniques in unix

In the Unix operating system, the first interprocess communications (IPC) techniques provided were message queues, semaphores, and shared memory. While Unix has added many mechanisms, these three are still popular and heavily used in many applications.

semaphores

A semaphore is a special variable that takes only whole positive numbers and upon which only two operations are allowed: wait and signal. They are used to ensure that a single executing process has exclusive access to a resource. It simply means if two or more processes accessing a resource and want to do some operations on that resource, semaphore technique doesn't allow both processes to access the resource asynchronously (same time). One thread should wait while another one using that resource.
A semaphore generally takes one of two forms:

1. Binary semaphore
2. Counting semaphore

Binary semaphore

A binary semaphore is a simple "true/false" (locked/unlocked) flag that controls access to a single resource.

Counting semaphore

A counting semaphore is a counter for a set of available resources.

semaphore example

semaphore declaration - pseudo-code

// Assign memory to semaphore variable
global semaphore_variable;

// To lock the resource access to another processes
Lock(semaphore_variable pointer);

// To unlock the resource for another processes
Unlock(semaphore_variable pointer);

Supposed we have two processes process1 and process2, both of which need exclusive access to a database at some point in their execution.
We define a single binary semaphore, semaphore_variable, that starts with the value 1.

semaphore usage by processes - pseudo-code

semaphore_variable = 1;

code block {

    // some operation

    Lock(semaphore_variable);
    // exclusive access to database resource
    Unlock(semaphore_variable);

    // some operation
}

UNIX Semaphore Facilities

All the UNIX semaphore functions operate on arrays of general semaphores, rahter than a single binary semaphore.
The semaphore functions are:

1. semctl - The semget function creates a new semaphore or obtains the semaphore key of an existing semaphore.
2. semget - The function semop is used for changing the value of the semaphore.
3. semop - The semctl function allows direct control of semaphore information.

Mutex

A Mutex is a binary semaphore. Mutex is meant to be used for mutual exclusion (post/release operation is restricted to thread that called pend/acquire) only and binary semaphores are meant to be used for event notification (post-ability from any thread) and mutual exclusion.

shared memory

Shared memory allows two unrelated processes to access the same logical memory. Shared memory is a special range of addresses that is created by IPC for one process and appears in the address space of that process. Other processes can then 'attach' the same shared memory segment into their own address space.
Shared memory doesn't provide and synchronization facilities. There are no automatic facilities to prevent a second process starting to read the shared memory before the first process has finished writing to it. It's the responsibility of progammer to synchronize access.

Shared Memory Functions

1. shmget - We create shared memory using the shmget function
2. shmat - when you first crete a shared memory segment, it's not accessible by any process.
3. shmdt - The shmdt function detaches the shared memory from the current process.
4. shmctl - The control functions for shared memory are (thankfully) rather simpler than the more complex ones for semaphores:

message queues

Message queues provide a way of sending a block of data from one process to another. Message queues provide a reasonably easy and efficient way of passing data between two unrelated processes.

Message Queue Functions

msgget - We create and access a message queue using the msgget funciton.
msgsnd - The msgsnd function allows us to add a message to a message queue.
msgrcv - The msgrcv function retrieves messages from a message queue.
msgctl - The msgctl is similar to the control function for shared memory.

IPC Unix Commands

Most UNIX systems with semaphores provide the ipcs and ipcrm commands that allow command line access to IPC information.

Semaphores commands

To examine the state of semaphores on the system, use the ipcs -s command. If any semaphores are present, the output will have the form:

$ ipcs -s

You can use the ipcrm command to remove any semaphores left by programs. To delete the above semaphores, the command (on Linux) would be:

$ ipcrm sem <sem-id>
or
$ ipcrm -s <sem-no>

Shared Memory commands

The ipcs -m and ipcrm shm <:id> (or ipcrm -q <id>) commands provide command line programs for accessing the details of shared memory). See the following commmands:

$ ipcs -m

$ ipcrm shm <shm-id>

Message Queues commands

For messge queues the commands are ipcs -q and ipcrm msg <id> (or ipcrm -q <id>). Run the following command in your Unix terminal:

$ ipcs -q

vim editor commands

Vim editor

Vim is an editor to create or edit a text file using linux terminal. One can choose another GUI editors available in linux but in case of runlevel limitations and to use as development perpose vi and vim is used most. We can take the benefits of cTags using vim. we will discuss sTags later (please understand cTags after vim, otherwise you are only cramming the things).

NOTE: Microsoft PC Notepad users who do not wish to use "vi" should use "gedit" (GNOME edit) or "gnp" (GNOME Note Pad) on Linux. This is very similar in operation to the Microsoft Windows editor, "Notepad". (Other Unix systems GUI editors: "dtpad", which can be found in /usr/dt/bin/dtpad for AIX, vuepad on HP/UX, or xedit on all Unix systems.)

Vim modes

There are two modes in vim. One is the command mode and another is the insert mode.

1. In the command mode, user can move around the file, delete text, etc.
2. In the insert mode, user can insert text.

Changing mode from one to another

1. From command mode to insert mode type a/A/i/I/o/O. Any of the key can be pressed after opening the document in vim to enable insert mode.
2. From insert mode to command mode type Esc (escape key). Now one can use vim commands applicable with vim within or out of scope of the opened file.
3. From command mode to visual mode type v and use any arrow keys to select text lines. You can use Ctrl + v to take control over vertical text selection.

Vim insert mode commands

        a    Append text following current cursor position

        A    Append text to the end of current line

        i     Insert text before the current cursor position

        I     Insert text at the beginning of the cursor line

        o    Open up a new line following the current line and add text there

        O   Open up a new line in front of the current line and add text there

Note: For a beginner this is not required to learn all the options but use the important one. Even me using 'i' option always.

Vim command mode commands

Cursor Movement Commands

Cursor keys - to move the cursor anywhere in file.
Ctrl + e    - 1 line up  
Ctrl + d    - 1/2 page up
Ctrl + f    - 1 page up
Ctrl + y    - 1 line down  
Ctrl + u    - 1/2 page down
Ctrl + b    - 1 page down

%    - use with '{','}','(',')' to jump with the matching one.
0      - first column of the line$      - jump to the last character of the line

Editing Commands

d …delete the characters from the cursor position up the position given by the next command (for example d$ deletes all character from the current cursor position up to the last column of the line).
c …change the character from the cursor position up to the position indicated by the next command.
y …copy the characters from the current cursor position up to the position indicated by the next command.
p …paste previous deleted or yanked (copied) text after the current cursor position.
Note: Doubling d, c or y operates on the whole line, for example yy copies the whole line.

Undo and Redo

u         - you can undo almost anything using u in the command mode.Ctrl+r - undo is undoable using Ctrl-r.

External commands

In Vim it’s easy to include the output of external commands or to filter the whole line or just a part through an external filter.
To issue an external command type ':'

Searching and Replacing

:s/old/new/gc
:s/old/new/g

Save the file

:wq

Exit file without saving the changes

:q!

cTags

ctags allow fast jumping to function call even the function definition source code are from other directories. In order to use ctags, first run the command at destination directory where the source codes are located.

$ ctags -R *

-R is to recursively goes across all the directories, and a ‘tags’ file will be created.

ctrl + ]     - Jump from function call to function definition.
ctrl + t     - Go back to where you came from.

ctrl + i     - To travel forward.
ctrl + o - Backward of the check point.

Note: When you add a function at your class or module, you need to run ctags again too.


tags: linux editor, edit files from terminal, vi editor, vim editor, vim text editor, vim c language editor, ctags

vim search replace

Insert to begin of line

:%s/^[9]/insert into 'data' values('9/g

Replace Tab

:%s/^\t/,/g

Remove lines begin from ','

:%s/^[\,\t]*\n//g

Replace all

:%s/,/','/g

insert to end of line

:%s/$/');/gc

Removing spaces from begining of line

:%s/^\s\+//
" Same thing (:le = :left = left-align given range):
:%le

Removing spaces from end of line

:%s/\s\+$//

Replace line with comma

:%s/\n/,/g