ROAR User Guide   »   Roar User Guide   »   Useful Unix/Linux Commands
Feedback [ + ]

Useful Unix/Linux Commands

Shells

Unix/Linux shells are command line interpreters that allow for a user to interact with their operating system through utility commands and programs. The Default Unix/Linux shell is BASH (the Bourne-Again SHell) which has extensive online documentation, and common or necessary commands are shown in the table below.

CommandDescription (For full documentation, use the command 'man command' to bring up the manual or find online documentation)
lsThe 'list' (ls) command is used to display all the files in your current directory. Using the '-a' flag will also show any hidden files (typically files beginning with a '.' like .bashrc)
cdThis is the 'change directory' command. Use this to traverse directories (like 'cd work'). To move back a directory level, use 'cd..'.
mvThe 'move' command takes two arguments, the first being the file to move and the second being the directory said file should be moved to ('mv file.txt /work/newdirectory/'). Note: 'mv' can also be used to rename a file if the second argument is simply a new file name instead of a directory.
mkdirThis command is used to make directories.
rmdirThis command is used to delete directories ('rm -rf directory' would also work).
touchThis command is used to create files in a similar way to mkdir. ('touch test.txt' will create an empty text file named test).
rmThis is the 'remove' command. As mentioned above, it can be used recursively to delete entire directory trees, or it can be used with no flags and a file as the argument to delete a single file.
locateThis command is used to locate files on a system. The flag '-i' will make the query case-
insensitive, and asterisks ('*') will indicate wildcard characters.
clearClears the terminal of all previous outputs leaving you with a clean prompt.
historyShows the previous commands entered.
findUsed for finding files, typically with the -name flag and the name of the file.
grepUsed for searching within files.
awkA programming language typically used for data extraction.
idShow all of the groups a user is in.
duShow the disk usage. Typically used with -h (for human readable) and –max-depth=1 to limit to only the directories in that level rather than all files.
envPrint out all of the current environment variables.
lessView a file.
cpCopy a file. Note the -r (recursive) flag can be used to copy directories.
aliasCreate an alias (something short) that is interpreted as something else (a complicated command).
pwdPrint the current working directory.
chmodChange file permissions.
chgrpChange group for a file or directory.
lddShow the shared libraries required for an executable or library.
topSee the node usage. Often used with command U .
/usr/bin/timeShow time and memory statistics for a command being run. Often used with the -v (verbose) flag.
bgContinue running a paused task in the background
fgBring a background task into the foreground
Ctrl + cKill a process.
Ctrl + zSuspend a process
Ctrl + r Search through your history for a command that includes the text typed next.

Special Characters

There are also some special characters to be aware of that will be helpful.

~ is your home directory

. means here

.. means up one directory

* is the wildcard: * for all files or *.png for all png files

|is pipe (send the output to another command)

> means write command output to a file (Example: ls > log.ls)

Most commands have a manual that show all of the different ways the command can be used. For example,

man ls

shows all of the info for the ls command. You can use the arrows to scroll through the manual and the letter q for quit. Some commands will also provide a shortened version of the manual showing the available flags if an incorrect flag is used. For example,

mam-list-funds -banana

brings up a list of all of the flags available for mam-list-funds. Any non-working flag will allow for this. Note that this doesn’t give information about what the flags do, just what the flags are. This may be enough to remind you of something you had done previously.

All shells utilize configuration files. For BASH, this is split between 2 files: ~/.bash_profile and ~/.bashrc.

NOTE: ~/ in Linux is a way to specify your own home directory. The .bash profile file is always parsed when a terminal is open, including with an SSH session. To connect the two in such a way that .bashrc will always be sourced for a session, make sure this code is included in your ~/.bash_profile:

if [ -f ~/.bashrc ]; then . ~/.bashrc fi

Alternative Shells

BASH is only the default shell, and it doesn’t come with quite a few features that many Linux power-users would like to have on the command-line. Other common shells include KSH (KornSHell), ZSH (Z SHell), and FISH (Friendly-Interface SHell). These shells all have documentation available online regarding their installation and customization.

Environmental Variables

Environment variables are values that pertain to certain aspects of an operating system’s configurations. These variables are typically used by utilities and programs for things like finding out where the user’s home directory is ($HOME) or where to look for executable files ($PATH). The prompt for BASH is held as the variable PS1.

You can print the environment variable to the screen using the echo command:

echo $HOME

A good way to view environment variables that are set is by using the env command:

env

which outputs all of the variables currently in use.

To change the value of an existing variable or to create and set a new variable, we use export. For example, to set a variable called workDir to a directory called here within your home directory, the command would be:

export workDir=$HOME/here

Once this environment variable is set, you are able to use this. For example, to change to this directory, the command would be:

cd $workDir

For something like PATH where you really do not want to overwrite what values are already stored, you can append values with

export PATH=$PATH:/new/dir/path/

In lists of values, the colon (:) is used as the delimiter. The dollar sign ($) is used to reference variables, so that export command essentially appends the new directory to the list of existing directories searched for executables. It is possible to prepend as well, which may come in handy if you compile a different version of an existing command.

For more general reading on environment variables in Linux, see these pages on variables and environmental variables.

The environment variables allow for script portability between different systems. By referencing variables like the home directory ($HOME) you can generalize a script’s functionality to work across systems and accounts.

Variable NameDescription
USERYour user ID
HOSTNAMEThe name of the server that the script is being run on
HOMEYour home directory
WORKYour work directory
SCRATCHYour scratch directory
TMPDIRThe directory in which a job's temporary files are placed (created and deleted automatically)

References

The Linux terminal and submitting jobs are not unique to Roar. You can find many different training resources online for these. The Linux foundation offers free training. Lots of great information and tutorials for everyone from beginner Linux user to advanced users can be found here. Linux has been around for a long time. Therefore, any problem you might be having, someone has probably already had. It is always worthwhile to look around stack exchange to see if your question has already been answered.