Abstract
A complex part of the job of every Oracle DBA job involves dealing with files and directories. Therefore, DBAs must be experts in file manipulation. Your job requires skills such as implementing database security, performing backups and recovery, monitoring, and troubleshooting performance issues. These critical tasks are all dependent on a command-line knowledge of managing files. Expert DBAs know how to administer files and navigate within the filesystem.
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
A complex part of the job of every Oracle DBA job involves dealing with files and directories. Therefore, DBAs must be experts in file manipulation. Your job requires skills such as implementing database security, performing backups and recovery, monitoring, and troubleshooting performance issues. These critical tasks are all dependent on a command-line knowledge of managing files. Expert DBAs know how to administer files and navigate within the filesystem.
A file, which is the basic building block of a Linux/Solaris system, is a container for information stored on disk. You access a file by its file name. We use the terms file and file name synonymously in this book. File names can be up to 256 characters long and can contain regular letters; numbers; and the . (period), _ (underscore), and – (hyphen) characters.
A directory is like a folder; its purpose is to provide a logical container that facilitates working with groups of files. Every server has a root directory indicated by a forward slash (/); think of the forward slash as a tree falling forward from left to right. The / directory, which is the topmost directory on every server, is like an upside-down tree in which the trunk is the root directory, and the branches of the tree are subdirectories.
Figure 5-1 shows a partial hierarchy directory structure on an Oracle database server. Be aware that Figure 5-1 shows only a fraction of the directories typically created. The main point of the diagram is to give you an idea of the treelike directory structure that is used for a typical Oracle system. Because of the complexity of the directory structures, DBAs must be fluent with command–line directory navigation and file manipulation.
Common directories used on an Oracle database server
This chapter discusses common problems and solutions that you’ll encounter when working with files and directories. It starts with the basics, such as viewing directory structures, and then progresses to more complicated topics such as finding certain types of files.
5-1. Showing the Current Working Directory
Problem
You’re logged on to a database server and you want to view the current directory path.
Solution
Use the pwd (print working directory) command to display the full pathname of your current working directory:
$ pwd
/home/oracle
From the previous output shows that /home/oracle is the current working directory.
Note
If you’re a Windows user, the Linux/Solaris pwd command is similar to the DOS cd command when issued with no options. The DOS cd command without any options simply prints the current working directory.
How It Works
In Linux/Solaris, the directory you are working in is defined to be your current working directory. The pwd command isn’t very complicated; it simply prints the current working directory. As simple as it is, you’ll be using it all the time. DBAs constantly use this command to verify that they are in the correct directory. Before you manipulate directories or files, it’s wise to verify that you are where you think you should be.
The pwd command has two interesting options: -L and -P. The -L option prints the logical path and is the default. It always prints the value of the OS PWD variable. For example, the following two commands always display the same directory:
$ echo $PWD
/home/oracle
$ pwd
/home/oracle
The -P option prints the actual physical path. These options are useful if you’re working on systems that have directories that have been navigated to via a symbolic link (see recipe 5-33 for a discussion on soft links). The -L option prints the directory name as defined by the symbolic link. The -P option displays the directory as defined by the actual physical path.
An example can help illustrate the value of knowing when to use the -P option. On a database server, there is a symbolic link defined to be oradev that exists under the root directory. Here is an example of the long listing of the symbolic link:
$ ls -altr /oradev
lrwxrwxrwx 1 root root 9 Apr 15 19:49 oradev -> /oradisk2
First, navigate to the directory via the symbolic link and issue a pwd command with the -L option:
$ cd /oradev
$ pwd -L
/oradev
Now without changing directories, use the pwd command with the -P option:
$ pwd -P
/oradisk2
If you work in environments that use symbolic links, it is important to understand the difference between the -L and -P options of the pwd command.
5-2. Changing Directories
Problem
You want to change your current working directory to a different location.
Solution
Use the cd (change directory) command to navigate within the filesystem. The basic syntax for this command is as follows:
cd <directory>
This example changes the current working directory to /orahome/app:
$ cd /orahome/app
It is usually a good idea to use the pwd command to verify that the cd command worked as expected:
$ pwd
/orahome/app
You can also navigate to a directory path that is stored in an OS variable. The next set of commands displays the contents of the TNS_ADMIN variable and then navigates to that directory:
$ echo $TNS_ADMIN
/orahome/app/oracle/product/12.1.0.2/db_1/network/admin
$ cd $TNS_ADMIN
$ pwd
/orahome/app/oracle/product/12.1.0.2/db_1/network/admin
If you attempt to navigate to a directory that doesn’t exist, you’ll receive an error similar to this one:
No such file or directory
The owner of a directory must have the execute permission set at the owner level of the directory before the owner can navigate to the directory. An example illustrates this concept; listed here are the permissions for a scripts directory that oracle owns:
$ ls -ld scripts
d---rwxrwx 2 oracle oinstall 4096 Jul 30 19:26 scripts
As the oracle user, you receive an error when attempting to navigate to the scripts directory:
$ cd scripts
-bash: cd: scripts: Permission denied
If you modify the directory to include the owner execute permission, you can now navigate to it successfully:
$ chmod 100 scripts
$ cd scripts
How It Works
The cd command is a powerful utility that you’ll use often in your DBA life. The following sections contain techniques to make you more effective when using this command.
Navigating HOME
If you don’t supply a directory to the cd command, the directory will be changed to the value in the variable of HOME by default. This example demonstrates the concept by viewing the current directory, displaying the value of HOME, and using cd to navigate to that directory:
$ pwd
/orahome/app/oracle/product/12.1.0.2/db_1
Next, display the contents of the HOME variable:
$ echo $HOME
/orahome/oracle
Change directories to the value contained in HOME by not supplying a directory name to the cd command:
$ cd
$ pwd
/orahome/oracle
In the Bash and Korn shells, the ∼ (tilde) character is a synonym for the value contained in the HOME OS variable. The following two lines of code also change your directory to the HOME directory:
$ cd ∼
$ cd $HOME
Navigating to the Parent Directory
The .. (two dots) directory entry contains the value of the parent directory of the current working directory. If you want to change your directory to the parent directory, use the following syntax:
$ cd ..
You can navigate up as many parent directories as there are in a given path by separating the .. strings with a forward slash character. For example, to navigate up three directories, use this command syntax:
$ cd ../../..
You can also use the .. directory entry to navigate up a directory tree and then down to a different subdirectory. In the following example, the current working directory is /orahome/oracle/scripts, and the cd command is used to navigate to /orahome/oracle/bin:
$ pwd
/orahome/oracle/scripts
$ cd ../bin
$ pwd
/orahome/oracle/bin
Navigating to a Subdirectory
To navigate to a subdirectory, specify the directory name without a forward slash in front of it. This example first prints the current working directory, navigates to the admin subdirectory, and finally verifies success with the pwd command:
$ pwd
/home/oracle
$ cd admin
$ pwd
/home/oracle/admin
Using Wildcards
You can also use the wildcard asterisk (*) character with the cd command to navigate to other directories. In this next example, the current working directory is /oracle, and the product subdirectory is the target directory:
$ cd p*
$ pwd
/oracle/product
When navigating to a subdirectory, you must specify enough of the directory to make its name unique to any other subdirectories beneath the current working directory. If multiple directories match a wildcard string, you might not get the desired directory navigation, depending on your OS version. Always verify your current working directory with the pwd command.
When using the Bash shell, you can also use the Tab key to complete keystroke sequences. For example, if you have only one subdirectory that starts with the letter p, you can cd to it as follows:
$ cd p<Tab>
In this example, there is only one subdirectory beneath the current working directory that starts with a p, so you now see the following on the terminal:
$ cd product/
Now you can press Enter or Return to complete the command. This feature of the Bash shell is known as tab completion (see recipe 2-2 for more details).
Navigating to the Previous Directory
The hyphen (-) character is commonly used to navigate to the previous working directory. In the next example, the current working directory is /oracle01, and the previous working directory is /oracle02. To navigate to /oracle02, provide - to the cd command, as shown here:
$ cd -
Another way to navigate to the previous working directory is via the OLDPWD variable, which contains the location of the previous directory. To navigate to the most recently visited directory, you can change directories, as shown here:
$ cd $OLDPWD
5-3. Creating a Directory
Problem
You want to store your SQL scripts in a special directory. To do this, you first need to create a directory.
Solution
Use the mkdir (make directory) command to create a new directory. This example creates the directory named scripts underneath the /home/oracle directory:
$ cd /home/oracle
$ mkdir scripts
Now use the cd and pwd commands to verify that the directory exists:
$ cd scripts
$ pwd
/home/oracle/scripts
When navigating to another directory, if the directory doesn’t exist, you’ll receive an error message similar to this:
No such file or directory
How It Works
Before you create a directory, you must have write permission on the parent directory to create a subdirectory. If you attempt to create a directory and don’t have write permission on either the user or group level, you’ll receive an error. This example attempts to create a directory named oradump under the / directory:
$ mkdir /oradump
mkdir: cannot create directory `/oradump’: Permission denied
The permissions on the / directory show that only the root user has write permissions (and is therefore the only user who can create a directory under /):
$ ls -altrd /
drwxr-xr-x 29 root root 4096 Apr 15 19:49 /
If you don’t have root access, you’ll need to work with your SA to create any desired directories under the / directory. See recipe 3-11 for examples of obtaining access to root privileges.
Sometimes you’ll find it convenient to create several directories in a path with one command. This example uses the -p (parent) option to create the directory backups and any parent directories that don’t already exist in the path:
$ mkdir -p /oradump/db/dev/backups
The previous directory creation technique is extremely handy when you need to create long complex directory structures and you don’t want to create them one directory at a time.
5-4. Viewing a List of Directories
Problem
You want to just list the directories, not the other regular files in your current working location.
Solution
Use the ls -l command in combination with grep to list only directories. Here is some sample ls -l output without using grep to filter anything:
$ ls -l
drwxr-x--- 3 oracle dba 4096 Apr 25 19:44 orahome
drwxr-xr-x 9 oracle dba 4096 Dec 29 07:43 orainst
-rw-r----- 1 oracle dba 124506 Apr 25 20:10 ora.zip
-rw-r----- 1 oracle dba 112640 Apr 25 18:14 o.tar
-rw-r--r-- 1 oracle dba 82 Apr 4 14:30 output.txt
Next add in the grep filter:
$ ls -l | grep ’^d’
drwxr-x--- 3 oracle dba 4096 Apr 25 19:44 orahome
drwxr-xr-x 9 oracle dba 4096 Dec 29 07:43 orainst
In the preceding line of code, the ls -l output is piped to grep, which looks for files that begin with the d character. The caret ^ (caret) character is a regular expression that tells the grep command to match the d character at the beginning of the string. In this manner, you can list out just the directories.
How It Works
DBAs typically create an alias or function to facilitate typing the command shown in the “Solution” section of this recipe. This command creates an alias named lsd that can be used to list directories:
$ alias lsd="ls -l | grep ’^d’"
After the alias is created, type lsd. It will run the ls and grep commands. See recipe 2-7 for details on creating aliases and functions.
Another way to view directories is to use ls -p and grep for the forward slash character. The next example uses ls -p, which instructs the ls command to append a / on the end of every directory. The output of ls -p is piped to grep, which searches for the / character:
$ ls -p | grep /
orahome/
orainst/
When trying to list out a directory, it can sometimes be convenient to use a wildcard character. For example, say you want to determine all directories and files that are in the ORACLE_HOME directory that begin with the b character. To determine this, you attempt to issue this command:
$ ls $ORACLE_HOME/b*
The output of this command may not be what you expect. If a wildcard matches a directory name, the files contained in the directory (not the directory name) will be listed. In this example, the output contains all the files listed in the ORACLE_HOME/bin directory; here’s a short snippet of the output:
acfsroot exp lsnrctl0 plshprofO
adapters expdp lxchknlb proc
adrci expdpO lxegen procob
To avoid this behavior, use ls -d to list directories, not their contents. The following command lists all directories that begin with the letter b that are beneath ORACLE_HOME:
$ ls -d $ORACLE_HOME/b*
/orahome/app/oracle/product/12.1.0.2/db_1/bin
5-5. Removing a Directory
Problem
You want to remove a directory and all files that exist beneath that directory.
Solution
Use the rmdir command to remove a directory. This command can be used only to remove directories that don’t contain other files. In this example, the rmdir command is used to remove a directory named scripts that exists beneath the current working directory:
$ rmdir scripts
If the directory isn’t empty, you’ll see an error similar to this:
rmdir: scripts: Directory not empty
If you want to remove directories that contain files, use the rm -r (remove recursively) command. This example removes the directory scripts plus any files and subdirectories that exist beneath the scripts directory:
$ rm -r scripts
How It Works
If the rm -r command encounters any files that don’t have write permission enabled, a message like this will be displayed:
rm: remove write-protected regular file ’<file name>’?
If you want to remove the file, type y (for yes). If many files are write-protected (such as in oracle-owned directories), typing y over and over again can get tedious.
You can instruct rm to remove write-protected files without being prompted with the -f (force) option. This example removes all files beneath the subdirectory scripts without prompting for protected files:
$ rm -rf scripts
Sometimes when you’re removing old database installations, it is convenient to use the rm -rf command. This command will wipe out entire directory trees without asking for confirmation when deleting write-protected files. Make sure you know exactly what you’re removing before running this command.
Caution
Use the rm -rf command judiciously. This command will recursively remove every file and directory beneath the specified directory without prompting you for confirmation.
5-6. Listing Files
Problem
You want to see what files exist in a directory.
Solution
Use the ls (list) command to list the files (and directories) in a specified directory. This line of code uses the ls command without any options to list the files in the current working directory:
$ ls
Here is a partial listing of the output:
dlock.sql dropem.sql login.sql proc.sql rmfile.bsh
How It Works
The ls command without any options is not very useful; it displays only a limited amount of file information. One of the more useful ways to use ls is to list all the files, protections, ownership, sizes, and modification times—all sorted from most recently created to last. This is achieved with the -altr options:
$ ls -altr
Here is a partial listing of the output:
-rwxr-x--- 1 oracle dba 1543 Mar 29 16:09 proc.sql
-rw-r----- 1 oracle dba 1082 May 7 19:53 dlock.sql
-rw-r----- 1 oracle dba 442 May 7 19:53 login.sql
drwxr-x--- 4 oracle dba 4096 May 7 19:53 ..
-rw-r----- 1 oracle dba 0 May 7 19:54 dropem.sql
-rw-r----- 1 oracle dba 0 May 7 19:54 rmfile.bsh
The -a (all) option specifies that all files should be listed, including hidden files. The -l (long listing) option displays permissions, ownership, size, and modification time. The -t (time) option causes the output to be sorted by time (newest first). To have the latest file modified listed at the bottom, use the -r (reverse) option. Table 5-1 shows how to interpret the long listing of the first line of the previous output.
The first column of Table 5-1 has 10 characters. The first character displays the file type. Characters 2 through 10 display the file permissions. The characters r, w, and x indicate read, write, and execute privileges, respectively. A hyphen (-) indicates the absence of a privilege. The following output summarizes the first-column character positions and meanings of the long listing of a file:
File Type User Perms Group Perms Other Perms
Column 1 2 3 4 5 6 7 8 9 10
-, d, l, s, c, b r w x r w x r w x
In the first character of the first column of output, the hyphen indicates that it is a regular file. Similarly, if the first character is a d, it’s a directory. If the first character is an l, it’s a symbolic link. Table 5-2 lists the different file types.
The ls command may vary slightly between versions of Linux/Solaris. This command typically has more than 50 different options. Use the man ls command to view all features available on your system.
One last note: when listing a file with ls -l, you may notice an extra + at the end of the permissions; for example:
$ ls -l
drwxr-xr-x+ 2 oracle dba 3 May 2 15:02 scripts
-rw-r--r--+ 1 oracle dba 1627 May 2 15:12 act.sql
This means that your file has extended security permissions. Run the getfacl (get file access control lists) command to see the full permissions for the file; for example:
$ getfacl scripts
# file: scripts
# owner: oracle
# group: dba
user::rwx
group::r-x
mask::rwx
other::r-x
USING ECHO TO DISPLAY FILES
Interestingly, you can also use the echo command to list files. For example, you can use this command to list files in the current working directory:
$ echo *
The echo command is a built-in command (see recipe 2-15 for details on built-in commands). This means that if the filesystem that contains the ls executable is unavailable for some reason (perhaps because of corruption), you can still use the echo command to list files.
5-7. Creating a File Quickly
Problem
You’re setting up Oracle RMAN backups. You want to quickly create a file so that you can test whether the oracle user has the correct permissions to write to a newly created directory.
Solution
In the directory in which you want to determine whether you can create a file, use the touch command to quickly determine whether a file can be created. This example uses touch to create a file named test.txt in the current working directory:
$ touch test.txt
Now use the ls command to verify that the file exists:
$ ls -al
-rw-r----- 1 oracle dba 0 May 7 20:00 test.txt
From the output, the file is created and has nothing in it (indicated by a 0-byte size).
Note
See Chapter 4 for details on how to edit a text file.
How It Works
Sometimes you’ll just need to create a file to test being able to write to a backup location or check the functionality of some aspect of a shell program. You can use the touch command for these purposes. If the file you are touching already exists, the touch command will update the file’s last-modified date.
If you touch a file that already exists, its access time and modification time will be set to the current system time (this includes a date component). If you want to modify only the access time, use the -a option of touch. Similarly, the -m option will update only the modification time. Use the --help option to display all options available with touch on your system.
Note that you can also quickly create a file using the following cat command:
$ cat /dev/null> test.txt
Be careful when running the previous command; if the file already exists, concatenating /dev/null to a file will erase anything contained within the file.
5-8. Changing File Permissions
Problem
You want to change the permission on a file so that there is no public-level access.
Solution
Use the chmod command to alter a file’s permissions. This example changes the permission of the scrub.bsh file to 750:
$ chmod 750 scrub.bsh
A quick check with the ls command shows that the permissions are set correctly:
$ ls -altr scrub.bsh
-rwxr-x--- 1 oracle dba 0 May 7 20:07 scrub.bsh
The previous output indicates that the owner has read, write, and execute permissions; the group has read and execute; and the rest of the world has no permissions (see recipe 5-6 for a discussion on file permissions listed by the ls command).
Note
You must have root access or be the owner of the file or directory before you can change its permissions.
How It Works
DBAs often use the chmod command to change the permissions of files and directories. It is important that you know how to use this command. Correct file access is critical for database security. In many circumstances, you will not want to grant any public access to files that contain sensitive information.
You can change a file’s permissions by either using the numerical format (such as 750) or by using letters. When using the numerical format, the first number maps to the owner, the second number to the group, and the third number to all other users on the system. The permissions of 750 are translated to indicate read, write, and execute for the owner; read and execute for the group; and no permissions for other users. Inspect Table 5-3 for the translations of the numeric permissions.
You can also change a file’s permissions by using letters, which is sometimes more intuitive to new Linux/Solaris users. When using letters, keep in mind that the o permission doesn’t designate “owner”; it specifies “other.” Table 5-4 lists the meanings of to whom the permissions are applied.
This next example makes the file executable by the user (owner), group, and other:
$ chmod ugo+x mvcheck.bsh
This line of code takes away write and execute permissions from the group (g) and all others (o) for all files that end with the extension of .bsh:
$ chmod go-wx *.bsh
You can use three operands to apply permissions: +, -, and =. The plus (+) character adds permissions, and the minus (-) character takes away privileges. The equals (=) sign operand assigns the specified permissions and removes any not listed. For example, the following two lines are equivalent:
$ chmod 760 mvcheck.bsh
$ chmod u=rwx,g=rw,o= mvcheck.bsh
A quick listing of the file verifies that the permissions are set as expected:
$ ls -altr mvcheck.bsh
-rwxrw---- 1 oracle dba 0 May 7 20:10 mvcheck.bsh
You can also recursively change file permissions in a directory and its subdirectories. Sometimes this is useful when installing software. The following bit of code recursively changes the permissions for all files in the current directory and any files in subdirectories to 711 (owner read, write, execute; group execute; other execute):
$ chmod -R 711 *.*
You can also use the chmod utility to change the permissions of files to match the settings on an existing file. This example changes all files ending with the extension of .bsh in the current directory to have the same permissions as the master.bsh file:
$ chmod --reference=master.bsh *.bsh
Default File Permissions
Default permissions are assigned to a file upon creation based on the umask setting. The file creation mask determines which permissions are excluded from a file. To view the current setting of your file creation mask, issue umask with no options:
$ umask
0022
You can also view the character version of the umask settings by using the -S option:
$ umask -S
u=rwx,g=rx,o=rx
When you create a regular text file, the permissions are set to the value of 0666 minus the umask setting. If the umask setting is 0022, the permissions of the file are set to 0644, or -rw-r--r--.
Set User ID on Execution
Another concept related to the chmod command is the setuid permission (sometimes referred to as suid). Inspect the permissions of the oracle binary file:
$ cd $ORACLE_HOME/bin
$ ls -l oracle
-rwsr-s--x 1 oracle dba 126812248 Jun 12 15:24 oracle
Notice that the owner and group executable setting is an s (not an x), which indicates that the setuid permission bit has been set. This means that when somebody runs the program, it is run with the permissions of the owner of the file, not the permissions of the process running the file. This allows a user to run the oracle binary file as if it had the permissions of the oracle user. Therefore, server processes can execute the oracle binary file as if they were the owner (usually the oracle OS user) to read and write to database files.
To set the setuid permission, you must specify a preceding fourth digit to the numeric permissions when changing file permissions with chmod. If you want to enable the setuid permission on both the user and group level, use a preceding 6, as shown here:
$ chmod 6751 $ORACLE_HOME/bin/oracle
$ ls -l oracle
-rwsr-sr-x 1 oracle dba 118965728 Jun 16 2014 oracle
If you want to enable the setuid permission only at the owner level, use a preceding 4, as shown here:
$ chmod 4751 $ORACLE_HOME/bin/oracle
$ ls -l oracle
-rwsr-x--x 1 oracle dba 118965728 Jun 16 2014 oracle
As a DBA, it is important to be aware of the setuid permission because you may have to troubleshoot file permission issues, depending on the release of Oracle. For example, see MOS note 271598.1 for issues related to Enterprise Manager Grid Control and setuid dependencies. Additionally, you can run into Oracle accessibility issues when there are non-oracle users on the same server as the database software. In these situations, it’s important to understand how the setuid permission affects file access.
THE STICKY BIT
Do a long listing of the /tmp directory and inspect the permissions:
$ ls -altrd /tmp
drwxrwxrwt 4 root root 4096 May 10 17:24 /tmp
At first glance, it looks like all users have all permissions on files in the /tmp directory. However, notice that the “other” permissions are set to rwt. The last permission character is a t, which indicates that the sticky bit has been enabled on that directory. When the sticky bit is enabled, only the file owner can delete a file within that directory. The sticky bit is set with the following syntax:
chmod +t <shared directory>
or
chmod 3775 <shared directory>
Setting the sticky bit enables file sharing in a directory among many different users, but prevents users from deleting a file that they don’t own (within the directory that has the sticky bit enabled).
5-9. Changing File Ownership and Group Membership
Problem
You need to change a file’s file ownership and group membership so that it is owned by the oracle OS user and its group is dba.
Solution
You need root privileges to change the owner of a file. Use the chown (change owner) command to change a file’s owner and its group. This example changes the owner on the /var/opt/oracle directory to oracle and its group to dba:
# chown oracle:dba /var/opt/oracle
The file listing now shows that the directory owner is the oracle user, and the group it belongs to is dba:
$ ls -altrd /var/opt/oracle
drwxr-xr-x 2 oracle dba 4096 Dec 28 10:31 /var/opt/oracle
If you want to change only the group permissions of a file, use the chgrp command. You must be the file owner or have root privileges to change the group of a file. This example recursively changes the group to dba for all files with the extension of .sql in the current directory and all subdirectories:
$ chgrp -R dba *.sql
How It Works
When setting up or maintaining database servers, it is sometimes required to change the ownership on a file or directory. The following lines show the chown syntax for changing various combinations of the owner and/or group:
chown user file
chown user:group file
chown :group file
If you have root access, you can directly change file ownership. If you don’t have root privileges, sometimes SAs will grant you access to commands that require the root privilege through the sudo utility (see recipe 3-11 for details).
5-10. Viewing the Contents of a Text File
Problem
You want to view the contents of a text file, but you don’t want to open the file with an editor (such as vi) because you are afraid you might accidentally modify the file.
Solution
Use either the view, less, or more command to view only (not modify) a file’s contents. The view command will open a file using either the vi or vim editor in read-only mode. When you open a file in read-only mode, you are prevented from saving the file with the vi editor :wq (write and then quit) command. The following example views the initBRDSTN.ora file:
$ view initBRDSTN.ora
Using the view command is the same as running the vi -R command or the vim -R command (see Chapter 4 for more details about vi). To exit the view utility, enter the command :q.
Note
When viewing a file, you can force a write when exiting with the :wq! command.
If you want to display the contents of a file one page at a time, use a paging utility such as more or less. This example uses less to view the initBRDSTN.ora file:
$ less initBRDSTN.ora
The less utility will display a : (colon) prompt at the bottom-left corner of the screen. You can use the spacebar to go to the next page and use the up and down arrows to scroll through the documentation line by line. Enter q to exit less.
This next example uses the more command to page through the file:
$ more initBRDSTN.ora
Like the less utility, use the spacebar to display the next page and q to exit more.
How It Works
The more and less utilities are referred to as pagers because they display information onscreen one page at a time. These utilities have similar features, and one could argue that they are more or less the same. For the way that DBAs use these utilities, that’s mostly true. For hardcore geeks, the less utility is a bit more robust than more. Use the man less and man more commands to view all options available with these utilities.
When using either more or less, you can use vi commands to navigate within the displayed output. For example, if you want to search for a string, you can enter a forward slash and a string to search for text within the more or less output. This example searches for the string "sga_max_size" within the output of less: $ less initBRDSTN.ora
/sga_max_size
You can also use the cat command to quickly display the contents of a file to your standard output (usually your screen). This example dumps the output of the initBRDSTN.ora file to the screen:
$ cat initBRDSTN.ora
Using cat to display the contents of files works fine when you have small files. However, if the file is large, you’ll see a large amount of text streaming by too fast to make any sense. It’s almost always better to use view, less, or more (rather than cat) to view a file’s contents. These commands allow you to quickly inspect a file’s contents without risking accidental modifications.
5-11. Viewing Nonprinting Characters in a File
Problem
You’re trying to load text strings from a file into the database with a utility such as SQL*Loader, but the data appears to be corrupted after it is inserted into the target table. You want to view any control characters that may be embedded into the file.
Solution
Use the cat -v command to view nonprinting and control characters. This example displays nonprinting and control characters in the data.ctl file:
$ cat -v data.ctl
Note
The cat -v command does not display linefeed or Tab characters.
How It Works
When dealing with data being loaded into the database from text files, you sometimes might find that your SQL queries don’t behave as expected. For example, you might search for a string, yet the SQL query doesn’t return the expected data. This might occur because nonprinting characters are being inserted into the database. Use the cat -v command as described in this recipe to troubleshoot these kinds of data issues.
To illustrate viewing nonprinting characters, you can spool the following output from a SQL*Plus session:
SQL> spool out.txt
SQL> select chr(7) || ’ring the bell’ from dual;
SQL> exit;
Here you use cat to display the contents of the file out.txt:
$ cat out.txt
SQL> select chr(7) || ’ring the bell’ from dual;
CHR(7)||’RINGT
--------------
ring the bell
SQL> exit;
Notice the ^G ASCII ring bell or beep control character in the last line of the output when you use the -v option:
$ cat -v out.txt
SQL> select chr(7) || ’ring the bell’ from dual;
CHR(7)||’RINGT
--------------
^Gring the bell
SQL> exit;
5-12. Viewing Hidden Files
Problem
You’re trying to clean up your home directory and want to view the names of hidden configuration files and/or hidden directories.
Solution
Use the ls command with the -a (all) option. This bit of code lists all files using a long listing format and sorted in the reverse order in which they were modified:
$ ls -altr $HOME
Here is a sample of part of the output:
drwxr-xr-x 3 root root 4096 Sep 29 13:30 ..
-rw-r--r-- 1 oracle oinstall 124 Sep 29 13:30 .bashrc
-rw-r--r-- 1 oracle oinstall 24 Sep 29 13:30 .bash_logout
-rw-r--r-- 1 oracle oinstall 223 Sep 29 13:53 .bash_profile
drwxr-xr-x 2 oracle oinstall 4096 Oct 2 17:55 db
drwxr-xr-x 2 oracle oinstall 4096 Oct 15 08:33 scripts
drwx------ 2 oracle oinstall 4096 Oct 15 08:34 .ssh
-rw------- 1 oracle oinstall 6076 Oct 15 13:19 .bash_history
-rw------- 1 oracle oinstall 5662 Oct 15 13:41 .viminfo
drwx------ 5 oracle oinstall 4096 Oct 15 13:55 .
Any of the files in the previous listing that begin with a . (dot or period) are classified as hidden files. When using the Bash shell, common hidden files in your home directory are .bash_profile, .bashrc, .bash_logout, and .bash_history (see recipe 2-5 for uses of these files).
If you want to list only hidden files, you can do so as follows:
$ ls -d .*
Here is the corresponding output:
.a .bash_profile .lesshst .sh_history .viminfo .Xauthority
.. .bash_history .history .ocm .ssh .vnc
You may want to create an alias for the preceding command, such as this:
$ alias ls.=’ls -d .*’
How It Works
The only difference between a hidden file and a nonhidden file is that the hidden file begins with a . (dot or period) character. There isn’t anything secretive or secure about hidden files. Hidden files are usually well-known files with distinct purposes (such as storing environment configuration commands).
You may not want to muddle the output of an ls command with every file in a directory. The default behavior of the ls command is not to list hidden files. The -a option specifically tells the ls command to list all files, including hidden files. If you want to list all files except the . and .. files, use the -A option:
$ ls -A
Note
The . file is a special file that refers to the current working directory. The .. file refers to the parent directory of the current working directory.
5-13. Determining File Type
Problem
You want to display whether a file is a directory or a regular file.
Solution
Use the ls command with the -F option to display the file name and file type. This example lists file names and file types within the current working directory:
$ ls -F
Here is a partial listing of some sample output:
alert.log gcc-3.4.6-3.1.x86_64.rpm ora01/ ss.bsh*
anaconda-ks.cfg install.log ora02/ test/
The ls -F command appends a special character to the file name to indicate the file type. In the previous output, the file names appended with / are directories, and the file name appended with * is an executable file.
Tip
Another method of determining file type is to use the ls --color command, which colorizes the file depending on its type.
You can also use the file command to display characteristics of a file. This command will display whether the file is an ASCII file, a tar file, or a details executable file. For example, one way that DBAs use the type command is to tell whether the oracle binary file is 32-bit or 64-bit. The following shows the oracle binary file on a 64-bit server:
$ file $ORACLE_HOME/bin/oracle
Here is the corresponding output:
/orahome/app/oracle/product/12.1.0.2/db_1/bin/oracle: setuid setgid ELF 64-bit LSB executable,
AMD x86-64, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs),
not stripped
When using the Bash shell, if the file of interest is located in a directory in your PATH variable, you can use command substitution to provide input to the file command. Command substitution takes the output of the command enclosed in $() and provides that as input to a given command; for example:
$ file $(which oracle)
In the preceding line of code, the output of the which command is used as input to the file command.
How It Works
You can display an indicator for a file by using the -F option of the ls command. Table 5-5 describes the different file name type indicators. File type indicators allow you to filter the output and look for a certain file type. For example, to list all directories, search the output of the ls -F command for the / character:
$ ls -F | grep /
DBAs often encapsulate strings of commands like this within aliases or functions, which allows them to create shortcuts to long commands (see recipe 2-7 for details).
Tip
Use the type command to determine the characteristics of a command file. It will show whether the command is a utility, a built-in command, an alias, or a function.
The stat command is another useful command for displaying file characteristics. This command prints in human-readable format the contents of an inode. An inode (pronounced “eye-node”) is a Linux/Solaris data structure that stores information about the file. The next example displays the inode information for the oracle binary file:
$ stat $ORACLE_HOME/bin/oracle
Here is the corresponding output:
File: `/orahome/app/oracle/product/12.1.0.2/db_1/bin/oracle’
Size: 323762476 Blocks: 632992 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 34838017 Links: 1
Access: (6751/-rwsr-s--x) Uid: ( 2000/ oracle) Gid: ( 500/ dba)
Access: 2015-04-21 11:32:14.000000000 -0600
Modify: 2014-12-29 09:17:28.000000000 -0700
Change: 2014-12-29 09:17:28.000000000 -0700
You can obtain some of the preceding output from the ls command. However, notice that the stat output also contains information such as the number of blocks allocated; the inode device type; and the last time a file was accessed, the last time a file was modified, or when its status was changed.
5-14. Finding Differences Between Files
Problem
You have two databases that were supposed to be set up identically. You want to see any differences in the initialization files.
Solution
Use the diff (difference) command to identify differences in files. The general syntax for this command is as follows:
$ diff <file1> <file2>
This example uses diff to view the differences between two files named initDEV1.ora and initDEV2.ora:
$ diff initDEV1.ora initDEV2.ora
Here is some sample output showing the differences in the files:
6,7c6,7
< sga_max_size=400M
< sga_target=400M
---
> sga_max_size=600M
> sga_target=600M
20a21
> # star_transformation_enabled=true
How It Works
The key to understanding the output from diff is that it provides you with instructions on how to make file1 look like file2. The output tells you how to append, change, and delete lines. These instructions are signified by a, c, or d in the output.
Lines prepended by < are from file1. Lines prepended by > are from file2. The line numbers to the left of a, c, or d apply to file1. The line numbers to the right of a, c, or d apply to file2.
From the previous output in this recipe’s solution, the first line, 6,7c6,7, is translated to mean “change lines 6 and 7 in file1 to lines 6 and 7 in file2.” The second-to-last line of the output is 20a21, which means “after line 20 in file1, append line 21 from file2.”
The output of diff is known as a difference report, which can be used in conjunction with the patch command to make file1 look like file2. Before you can use the patch command, you first have to save the difference report output in a file. The following example stores the difference output in a file named init.diff:
$ diff initDEV1.ora initDEV2.ora > init.diff
To convert initDEV1.ora to initDEV2.ora, use the patch command with the difference report output:
$ patch initDEV1.ora init.diff
You can also use the sdiff (side-by-side) utility to display file differences. The sdiff output is usually easier to interpret than the diff command because differences are juxtaposed visually in the output. The following example uses sdiff to display the differences between two files:
$ sdiff initDEV1.ora initDEV2.ora
Here is a snippet of the side-by-side differences:
sga_max_size=400M | sga_max_size=600M
sga_target=400M | sga_target=600M
...
>#star_transformation_enabled=true
Tip
Use the diff3 utility to compare the differences between three files.
5-15. Comparing Contents of Directories
Problem
You want to ensure that two different directories have identical contents in terms of the number of files, file names, and file contents.
Solution
You can use the diff command to display any differences in two directories in terms of file names and file contents. This example compares the files in the /ora01/upgrade directory with the files in the /cvsroot/prod_scripts directory:
$ diff /cvsroot/prod_scripts /ora01/upgrade
If there are no differences, you won’t see any output. If there’s a file that exists in one directory but not in the other directory, you’ll see a message similar to this:
Only in /ora01/upgrade: tab.sql
If there are differences in any files in each directory, you’ll see a message similar to the following:
22c22
< # cd to udump
---
> # cd to udump directory.
See recipe 5-14 for details on interpreting the output of the diff utility. If you want to see only the names of the files that are different (and not how the files differ), use the --brief option:
$ diff --brief /cvsroot/prod_scripts /ora01/upgrade
How It Works
Occasionally you may need to compare the contents of directories when maintaining database environments. In these situations, use the diff command to compare the contents of one directory with another.
If you want to recursively look in subdirectories and compare files with the same name, you can use the -r option. This example recursively searches through subdirectories and reports any differences in files that have the same name:
diff -r /cvsroot/prod_scripts /ora01/upgrade
You can also use the long listing of the recursive option to achieve the same result:
diff --recursive /cvsroot/prod_scripts /ora01/upgrade
5-16. Copying Files
Problem
You want to make a copy of a file before you modify it.
Solution
This first example shows how to use the cp (copy) command to create a copy of a file. For example, cp is used here to make a backup of the listener.ora file:
$ cp listener.ora listener.old.ora
You can verify that the copy worked with the ls command:
$ ls listener*.ora
listener.old.ora listener.ora
If you need to copy a file over the network, you can use a command-line utility such as scp, ftp, rsync, wget, or curl. We’ll show an example of using scp in this recipe. There are examples of using scp and rsync to copy directories and files in recipe 5-17. We generally don’t use ftp because it’s not considered to be a secure way to copy files over the network. There are examples of using wget and curl to download files in recipe 5-36.
Having said that, let’s look at scp; the basic syntax for scp is as follows:
scp [options] sourcefile destinationfile
The source/destination can be directories and/or files. The source/destination directory/file in the preceding syntax line can take one of the following general forms:
-
directory/file
-
host:directory/file
-
user@host:directory/file
In the next line of code, a file is being copied from the remote host to the local host. The remote user is oracle, the remote host is srv2, the remote file is initTRG.ora in the /u01 directory, and the file is being copied to the local current working directory (signified by a dot):
$ scp oracle@srv2:/u01/initTRG.ora .
The scp command prompts you for the password of the remote user. If you’ve never copied from the remote server, you’ll also be prompted to ensure that you want to copy from the specified remote server.
This example copies a file from the local server to the remote host:
$ scp startup.sql oracle@srv2 :.
Inspect the preceding syntax carefully. The local file being copied is startup.sql and exists in the current working directory. The file is copied using the remote user oracle, to the remote server srv2, and to the oracle user’s HOME directory (specified by a dot immediately following the colon). Assuming that the remote HOME directory is /home/oracle, the following example copies the file to the same directory as the preceding example:
$ scp startup.sql oracle@srv2:/home/oracle
How It Works
DBAs often need to create copies of files. For example, the cp utility provides a method to create backups of files or quickly replicate directories. The cp command has this basic syntax:
cp [options] source_file target_file
Be careful when copying files. If the target file exists prior to issuing the copy command, it will be overwritten with the contents of the source file. If you want to be warned before overwriting an existing file, use the -i (interactive) option. In this example, there already exists a file named init.old.ora:
$ cp -i init.ora init.old.ora
cp: overwrite `init.old.ora’?
Now you can answer y or n (for yes or no, respectively) depending on whether you want the target file overwritten with the source. Many DBAs create a shortcut command for cp that maps to cp -i (see recipe 2-7 for details on how to create shortcuts). For example, this code helps prevent you from accidentally overwriting previously existing files:
$ alias cp=’cp -i’
You can also copy files directly into an existing directory structure using this syntax:
cp [options] source_file(s) directory
If the destination is a directory, the cp command will copy the file (or files) into the directory. The directory will not be overwritten. This example copies all files in the current working directory with the extension .sql to the scripts directory:
$ cp *.sql scripts
When you copy a file, the original timestamp and file permissions may differ between the original file and the file newly created by the copy command. Sometimes it’s desirable to preserve the original attributes of the source file. For example, you may want to make a copy of a file, but for troubleshooting purposes want to still be able to view the original timestamp and ownership. If you want to preserve the original timestamp, ownership, and file permissions, use the -p (preserve) option:
$ cp -p listener.ora listener.old.ora
You can also use the cp utility to create the directory structure associated with the source file by using the --parents option. For this command to work, the destination must be a directory. This example creates a network/admin/log directory and copies any files ending with the extension of .ora to a directory beneath the destination ∼/backup directory:
$ cp --parents network/admin/*.ora ∼/backup
Any files with the extension of .ora in the source directory should now exist in the ∼/backup/network/admin destination directory.
5-17. Copying Directories
Problem
You want to copy all files and subdirectories beneath a directory to a new location.
Solution
Use the cp command with the -r option to recursively copy all files in a directory and subdirectories. This example copies all files in the /orahome/scripts directory tree to the /orahome/backups directory:
$ cp -r /orahome/scripts /orahome/backups
The /orahome/backups directory now should have an identical copy of the files and subdirectories in the /orahome/scripts source directory. Be aware that existing files in the destination directory will be overwritten if they have the same name as files being copied from the source directory. If you want to be prompted before a file is overwritten, also use the -i (interactive) option:
$ cp -ri /orahome/scripts /orahome/backups
If you need to copy directories (and files) securely over the network, use the scp (secure copy) command. The basic syntax for scp is as follows:
scp [options] sourcefile destinationfile
The source/destination can be directories and/or files. The source/destination directory/file in the preceding syntax line can take one of the following general forms:
-
directory/file
-
host:directory/file
-
user@host:directory/file
To recursively copy and preserve directories and files use the -r and -p options of the scp command. This example recursively copies the scripts directory (and any subdirectories and files) from the local box to a remote box named rmougdev2 as the oracle user:
$ scp -rp scripts oracle@rmougdev2:/home/oracle/scripts
In the preceding line of code, if the destination directory does not exist, it will be created. If the directory already exists, a subdirectory named scripts will be created underneath the existing scripts directory.
The scp command will prompt you for the password of the remote user. If you’ve never copied from the remote server, you’ll also be prompted to ensure that you want to copy from the specified remote server.
How It Works
As part of the daily routine, DBAs and developers often copy directories and files from one location to another. The location could be local or remote. You might require this if you are installing software or if you just want to ensure that you have a backup of files copied to a different location.
Another powerful utility used to synchronize directories is the rsync command. The basic syntax for rsync is as follows:
rsync [options] sourcefiles destinationfile
By default, the rsync tool will transfer only the differences that it finds between the source and destination. This makes it an extremely flexible and efficient method to synchronize one directory tree with another.
If the source and destination are on the same server, ordinary file and directory names can be used. Use the -r and -a options to recursively copy a directory tree and preserve permissions and ownership; the --delete option also specifies removing any files that exist in the destination that do not exist in the source. This example ensures that two local directories have the exact same directory structure and files; in other words, it ensures that test2 is identical to test1:
$ rsync -ra --delete /home/oracle/test1/ /home/oracle/test2/
You can copy locally to a remote server or from a remote server to your local server. If the directory or file is remote, it takes the following general form:
user@host:port/filename
For example, you can use rsync to synchronize a remote directory structure with a local directory structure. This line of code recursively copies the contents of the local scripts directory to the remote rmougdev2 server as the oracle user:
$ rsync -ra --delete --progress scripts/ oracle@rmougdev2:/home/oracle/scripts
You’ll be prompted for the oracle user’s password. The / at the end of the source folder ensures that if the destination folder exists, rsync will synchronize the two directories. If the destination folder doesn’t exist, it will be created. Without the / at the end of the source directory, if the destination directory already exists, a subdirectory will be created underneath it.
The rsync command is very flexible and powerful. If you’ve never used it, you should become familiar with it and incorporate it into your bag of file transfer tricks.
5-18. Moving Files and Directories
Problem
You want to rename or relocate a file.
Solution
Use the mv (move) command to relocate a file or rename it. This example renames a file from initdw.ora to a new name of initDWDB.ora:
$ mv initdw.ora initDWDB.ora
You can also use the mv command to relocate a file to a different directory. This bit of code moves a file from the current working directory to its parent directory:
$ mv scrub.sql ..
Quite often, you’ll need to move a file from the current working directory to a subdirectory. This example moves a file from the current working directory to a subdirectory named scripts:
$ mv scrub.sql scripts
In the previous line of code, if the scripts subdirectory didn’t exist, you would end up renaming the scrub.sql file to a file named scripts. In other words, the destination subdirectory must exist before you issue the mv command (otherwise you’ll end up renaming the file).
It is also possible to relocate directories. The following example moves the scripts directory to the sqlscripts directory:
$ mv scripts sqlscripts
In the preceding line of code, if the sqlscripts directory already exists, the scripts directory is created as a subdirectory beneath the sqlscripts directory. This might seem a little confusing if you’re not expecting this behavior. One way to think of this is that the mv command does not overwrite directories if they already exist.
How It Works
The mv command is used to relocate or rename a file or a directory. The mv utility uses the following syntax:
mv [options] source(s) target
Be aware that the mv command will unceremoniously overwrite a file if it already exists. For example, say you have the following two files in a directory:
$ ls
initdw.ora init.ora
If you move initdw.ora to the name of init.ora, it will overwrite the contents of the init.ora file without prompting you. To protect yourself against accidentally overwriting files, use the -i (interactive) option:
$ mv -i initdw.ora init.ora mv: overwrite `init.ora’?
You can now enter a y or an n to indicate a yes or no answer, respectively. You can easily implement the mv command as mv -i via a function or an alias to protect yourself against erroneously overwriting files (see recipe 2-7 for details on command shortcuts).
Table 5-6 describes the various results of the mv operation, depending on the status of the source and target.
5-19. Renaming a File or Directory
Problem
You want to change the name of a file or directory.
Solution
Use the mv (move) command to rename a file. For example, the following line of code renames a file from credb1.sql to credatabase.sql:
$ mv credb1.sql credatabase.sql
You can also rename a directory. The following renames a directory from dev to test:
$ mv dev test
Be aware that when renaming directories, if you attempt to rename a directory to the name of an existing directory, a new directory will be created as a subdirectory beneath the already existing directory. See Table 5-6 for details on the behavior of the mv command.
How It Works
You can also use the rename command to change file names. The rename utility has the following syntax:
rename oldname newname files
This command has a big advantage over the mv command because it allows you to rename several files at once. For example, here is a way to rename all files in a directory that ends with the extension of .trc to the new extension of .trace:
$ rename .trc .trace *.trc
You can also use rename to change the name of just one file. Here the file initDEV.ora is renamed to initTEST.ora:
$ rename initDEV.ora initTEST.ora initDEV.ora
5-20. Removing a File
Problem
You want to remove a file from disk.
Solution
First, use the ls command to identify the files you want to remove. In this example, any files with the extension of .trc are displayed:
$ ls -altr *.trc
After visually verifying the files you want to remove, use the rm command to permanently delete files:
$ rm *.trc
How It Works
Be very careful when using the rm command. Once the files have been removed, the only way to get them back is from a backup (if there is one). DBAs can get in a lot of trouble by accidentally removing files.
DBAs typically are logged on to a server as the oracle OS user. This special user is usually the owner of all the critical database files, so this user can remove database files, even if they are currently in use.
Because the rm command doesn’t prompt you for confirmation, we recommend that you always use the ls command to verify which files will be removed.
If you want confirmation before removing a file, use the -i option:
$ rm -i *.trc
You will now be prompted for confirmation before each file is deleted:
rm: remove regular file `rmdb1_j001_11186.trc’?
Type y to have the file removed or n if you want to keep the file. This method takes longer, but gives you some reassurance that you’re deleting the correct files.
Another technique for preventing the accidental deletion of the wrong files is to use the !$ variable. The !$ character contains the last string entered on the command line. For example, to use !$ to remove files, first use the ls command to list the files targeted for deletion:
$ ls *.trc
Here is some sample output:
ora.trc
Now the value *.trc is stored in the !$ parameter. You can use rm to remove the files listed by the previous ls command:
$ rm !$
If you’re ever unsure of the contents of the !$ variable, use the echo command to display its contents:
$ echo !$
echo *.trc
ora.trc
5-21. Removing Protected Files Without Being Prompted
Problem
You want to remove all the files associated with an old installation of the database. However, when you issue the rm (remove) command, you are presented with this prompt:
rm: remove write-protected regular empty file
You wonder whether you can run the rm command without being prompted.
Solution
There are two techniques for removing write-protected files: rm -f and yes. This example uses rm -rf (remove, recursive, force) to recursively remove all files beneath a directory without being prompted:
$ rm -rf /oracle/product/11.0
This example uses the yes command to recursively remove all files beneath a directory without being prompted:
$ yes | rm -r /oracle/product/11.0
If you type the yes command without any options, the subsequent output will be a repeating y on your screen until you press Ctrl+C. You can pipe the output of the yes command to another command that is expecting a y or n as input for it to proceed.
How It Works
Be very careful when using the removal methods described in the “Solution” section of this recipe. These techniques allow you to easily remove entire directories and subdirectories with one command. Use these techniques only when you’re absolutely sure you don’t need a directory’s contents. Consider using tar or cpio to recursively back up a directory tree before you delete it (see Chapter 6 for details)
5-22. Removing Oddly Named Files
Problem
Somehow a file was created with the odd name of -f, and apparently it cannot be removed with the rm (remove) command. You wonder how you can remove it using the rm command.
Solution
First use the ls command to view the oddly named file:
$ ls
-f
You can attempt to remove the file with the rm command:
$ rm -f
However, the rm command thinks -f is the force argument to the command and does nothing with the -f file. To remove the file, specify the current path with the file name, as shown here:
$ rm ./-f
How It Works
Files with odd names are occasionally created by accident. Sometimes you can type a command with the wrong syntax and end up with a file with an undesirable name. For example, the following will create a file name of -f:
$ ls > "-f"
Now when you list the contents of the directory, you’ll see a file named -f:
$ ls
-f
Worse yet, you might have a malicious user on your system who creates a file like this:
$ ls > "-r home"
Be extremely careful in this situation. If you attempt to remove the file without specifying a path, the command will look like this:
$ rm -r home
If you happen to have a directory named home in the current directory, this command will remove the home directory. To remove the file, use the current path ./, as shown here:
$ rm "./-r home"
In the previous command, you need to enclose the pathname and file name in quotes because there is a space in the file name. Without quotes, the rm command will attempt to remove a file named ./-r and another file named home.
5-23. Finding Files
Problem
You want to locate a certain file on the database server.
Solution
Use the find command to search for a file. The most basic way to search for a file is to instruct find to look for a file recursively in the current working directory and any of its subdirectories. The following command looks in the current directory and any subdirectories for any file that begins with the string "alert" and ends with the extension of .log:
$ find . -name "alert*.log"
Here’s some sample output that indicates the location of the found file relative to the current working directory:
./RMDB1/admin/bdump/alert_RMDB1.log
How It Works
It’s well worth the effort to spend some time getting to know the find command. This command will allow you to easily search for files from the command line. Because this utility is used in so many different ways, we decided to include individual recipes to document these tasks. The next several recipes of this chapter show examples of how DBAs use the find command.
If your OS account doesn’t have correct access permissions on a directory or file, find will display an error message. This example changes directories to the / directory and issues a find command:
$ cd /
$ find . -name "alert*.log"
Here is a partial listing of output, indicating that there is no access to certain directories:
find: ./proc/11686/task/11686/fd: Permission denied
find: ./proc/11688/task/11688/fd: Permission denied
find: ./proc/15638/task/15638/fd: Permission denied
To eliminate those error messages, send the error output to the null device:
$ find . -name "alert*.log" 2>/dev/null
5-24. Finding Strings in Files
Problem
You want to search for a string in a text file that could be located somewhere beneath a given directory path.
Solution
Use a combination of the find and grep commands to search for a string that exists in a file in a directory tree. The first example uses find to locate all SQL files beneath a directory and pipes the output to xargs, which executes the grep command to search for a create database string:
$ find . -name "*.sql" | xargs grep -i "create database"
If your system supports it, consider displaying the string being searched for in color:
$ find . -name "*.sql" | xargs grep -i --color "create database"
You can also use the find command with exec, grep, and print to search for strings within files. The following command is equivalent to the prior command that uses xargs:
$ find . -name "*.sql" -exec grep -i "create database" ’{}’ \; -print
In the previous line of code, the find command finds all files in a directory tree with the extension of *.sql. The output is passed to the -exec ’{}’ command, which feeds each file found to the grep -i command. The \; marks the end of the -exec command, and -print displays any files found.
You can also use command substitution to achieve the same functionality; for example:
$ grep -i "create database" $(find . -name "*.sql")
Depending on your version of the OS, the grep command may support the -r (recursive search) option. The following command recursively searches all subdirectories and files beneath the current working directory for the create database string:
$ grep -ir "create database" .
The preceding command can take a long time, depending on the number of files it searches through. Prior examples in this solution section are more efficient because they search for a particular type of file and then search within the file for a string.
How It Works
Searching through files for a particular string is a very common task. Although the “Solution” section demonstrated several techniques for accomplishing this task, there are a few other examples that you may find relevant. For example, suppose that you want to display only the file name, not instances of the search string. To achieve this, use the -q option of grep. This example searches trace files for the word error and displays only the file name containing the search string:
$ find . -name "*.trc" -exec grep -qi "error" ’{}’ \; -print
When you troubleshoot issues, it is also helpful to see the file names and at what time the file was last modified:
$ find . -name "*.trc" -exec grep -qi "error" ’{}’ \; \
-printf "%p %TY-%Tm-%Td %TH:%TM:%TS %Tz\n"
Note
On some systems, the -q option may not be available. For example, similar functionality on Solaris would be implemented with the -l option. Use man grep to display all options available on your server.
Sometimes you want to search for the incidence of two or more strings in a file. Use grep with the -e option to accomplish this. This command searches for the "error" or "ora-" strings:
$ find . -name "*.trc" -exec grep -ie "error" -e "ora-" ’{}’ \; -print
You can also use egrep to search for multiple strings in a file:
$ find . -name "*.trc" -exec egrep "error|ora-" ’{}’ \; -print
Occasionally, you might have the need to inspect a binary file. For example, suppose that when using a spfile (server parameter file), you set a parameter erroneously as follows:
SQL> alter system set processes=10000000 scope=spfile;
System altered.
You then subsequently discover the bad setting when you attempt to stop and start the database:
ORA-00821: Specified value of sga_target 512M is too small
In this situation, you can’t even start your database in nomount mode, so you can’t use the ALTER SYSTEM command to modify the spfile. However, you can use the strings command to extract text strings from the binary spfile to quickly create a text-based init.ora file:
$ cd $ORACLE_HOME/dbs
$ strings spfileORA12CR1.ora >initORA12CR1.ora
Now modify the newly created init.ora file so that the value causing the problem is eliminated and then rename the spfile so that Oracle automatically uses the init.ora file when starting the instance.
This situation is just one example of how a DBA might have to use the strings command; the important thing to keep in mind is that this utility provides you with a way to look for text strings in binary files.
DOES DATABASE WRITER WRITE TO DATAFILES IN BACKUP MODE?
Back in the days before RMAN, a misconception existed with some DBAs that the database writer stops writing to datafiles while a datafile’s tablespace is in hot backup mode. The following example uses the strings command to verify that the database writer does indeed continue to write to datafiles, even while in backup mode.
First verify that a string does not exist in a datafile:
$ strings users01.dbf | grep -i denver
Verify that nothing is returned by the previous command. Next create a table and place it in the USERS tablespace:
SQL> create table city(name varchar2(50)) tablespace users;
Next alter a tablespace into backup mode:
SQL> alter tablespace users begin backup;
Now insert a string into the CITY table:
SQL> insert into city values(’Denver’);
Connect as SYS and run the following command to flush modified blocks from memory to disk:
SQL> alter system checkpoint;
From the OS command line, search for the "denver" string in the USERS database file:
$ strings users01.dbf | grep -i denver
You should see the following output:
Denver
This verifies that the database writer continues to write to datafiles, even while the corresponding tablespace is in backup mode. Don’t forget to take the USERS tablespace out of backup mode.
5-25. Finding a Recently Modified File
Problem
You recently created a file, but can’t remember where it is located on the server. You want to find any files with a recent creation date.
Solution
Use the find command with the -mmin (modified minutes) option to find very recently modified files. This example finds any files that have changed in the last 30 minutes beneath the current working directory:
$ find . -mmin -30
To find all files that were modified more than 30 minutes ago, use the + sign instead of the - sign:
$ find . -mmin +30
Sometimes when you troubleshoot issues, it is helpful to additionally pinpoint the exact time the file was modified; you can use the stat command to accomplish this:
$ find . -mmin -30 -exec stat -c "%n %y" {} \;
Additionally, the -printf option will show the time of file modification:
$ find . -mmin -30 -printf "%p %TY-%Tm-%Td %TH:%TM:%TS %Tz\n"
Here’s some sample output:
./dbcreate.sql 2015-05-09 10:21:46 -0700
./.mozilla/firefox/q5xf2w9k.default 2015-05-09 12:07:05 -0700
How It Works
The find command with a time-related option is useful for locating files that have recently been updated or changed. This command can be useful when you can’t remember where you placed recently modified or downloaded files.
If you’re using a version of find that does not support the -mmin option, try the -ctime option instead. The following command locates any files that have changed on the server in the last day beneath the ORACLE_HOME directory:
$ find $ORACLE_HOME -ctime -1
Many options are available when you are trying to find a file. For example, use the -amin (access minutes) option to find a file based on when it was last accessed. This line of code finds all files that were accessed beneath the current working directory exactly 60 minutes ago:
$ find . -amin -60
Table 5-7 describes a subset of time-related options commonly used with the find command.
A wide variety of options are available with the find command. Use the man find command to display the options available on your system.
5-26. Finding and Removing Old Files
Problem
You noticed that there are thousands of trace files being created in a diagnostic directory that consume disk space. You want to find old trace files and remove them.
Solution
Use the find command to locate files older than a certain age. Once the old files are identified, use the rm command to remove them. The following example identifies files greater than 14 days old and removes them all with one line of code:
$ find $ORACLE_BASE/diag/rdbms/dwrep/DWREP/trace/*.trc -type f -mtime +14 -exec rm -f {} \;
The preceding command finds all files (the option -type f indicates a regular file) in the specified directory and its subdirectories that are older than 14 days. The rm command is executed (-exec) once for each file name located by the find command. The function of {} is to insert each file returned (by find) into the rm -f command line. When using the -f (force) option, you will not be prompted if you really want to remove write-protected files (files without write permission enabled); \; denotes the end of the exec command line.
You can also use the find command in conjunction with xargs to find and remove old files:
$ find $ORACLE_BASE/diag/rdbms/dwrep/DWREP/trace/*.trc -mtime +14 | xargs rm
In the preceding line of code, the xargs command provides as input to the rm command any file names returned by the find command.
Another variation of this is to use command substitution $(<command>); for example:
$ rm $(find $ORACLE_BASE/diag/rdbms/dwrep/DWREP/trace/*.trm -mtime +14)
In the preceding line of code, any file names returned by the find command enclosed by $() will be removed by the rm command.
You might be wondering why you cannot directly pipe the standard output of the find command to be used as standard input to the rm command. For example, this does not work:
$ find $ORACLE_BASE/diag/rdbms/trg1/TRG/trace/*.trm -mtime +14 | rm
rm: missing operand
Some commands (e.g., rm and kill) don’t directly accept another command’s standard output as standard input. To use standard output of the find command as standard input to the rm command, you have to use one of the techniques described in this “Solution section” (e.g., exec, xargs, or command substitution).
How It Works
An active database will regularly produce trace files as part of its normal operations. These files often contain detailed information about potential problems or issues with your database. You usually don’t need to keep trace and audit files lying around on disk forever. As these files grow older, the information in them becomes less valuable.
DBAs will typically write a small shell script to clean up old files. This shell script can be run automatically on a periodic basis from a utility such as cron. See Chapter 7 for details on shell scripting and Chapter 11 for techniques for details on automating tasks through cron.
5-27. Finding the Largest Files
Problem
Your database is experiencing availability issues because a disk is 100 percent full. You want to locate the largest files in a directory tree.
Solution
Use the find command to locate files recursively in a directory tree. The following command sends the output of the find operation to the sort and head commands to restrict the output to just the five largest files located in any directory beneath the current working directory:
$ find . -ls | sort -nrk7 | head -5
Here is a sample of the output:
6602760 820012 -rw-r----- 1 oracle oinstall 838868992 Jan 21 14:55
./RMDB1/undotbs01.dbf
6602759 512512 -rw-r----- 1 oracle oinstall 524296192 Jan 21 14:55
./RMDB1/system01.dbf
6602758 51260 -rw-r----- 1 oracle oinstall 52429312 Jan 20 22:00
./RMDB1/redo03a.log
6602757 51260 -rw-r----- 1 oracle oinstall 52429312 Jan 19 06:00
./RMDB1/redo02a.log
6602756 51260 -rw-r----- 1 oracle oinstall 52429312 Jan 21 14:55
./RMDcB1/redo01a.log
The -nrk7 option of the preceding sort command orders the output numerically, in reverse order, based on the seventh column position. As shown in the output, the output is sorted largest to smallest. The top listing shows that the largest file is about 800MB in size.
How It Works
You can also use the find command to look for certain types of files. To look for a file of a particular extension, use the -name option. For example, the following command looks for the largest files beneath the current working directory and subdirectories that have an extension of .log:
$ find . -name "*.log" -ls | sort -nrk7 | head
DBAs often create shortcuts (via shell functions or aliases) that encapsulate long strings of commands. This line of code shows how to create an alias command shortcut:
$ alias flog=’find . -name "*.log" -ls | sort -nrk7 | head’
Command shortcuts can save time and prevent typing errors. See recipe 2-7 for details on creating functions and aliases.
5-28. Finding a File of a Certain Size
Problem
You’re running out of disk space, and you want to recursively locate all files beneath a directory that exceed a certain size.
Solution
Use a combination of the find command with the -size option to accomplish this task. This example uses the -size option to find any files more than 1GB in the current working directory and any subdirectories:
$ find . -size +1000000k
Here’s a small snippet of the output:
./ORA1212/sysaux01.dbf
./ORA12CR1/users01.dbf
./ORA12CR1/undotbs01.dbf
If you want to see the size of the file, use the stat command to do so:
$ find . -size +1000000k -exec stat -c "%n %s" {} \;
Here’s the corresponding output:
./ORA1212/sysaux01.dbf 1073750016
./ORA12CR1/users01.dbf 5368717312
./ORA12CR1/undotbs01.dbf 4294975488
How It Works
You can use the -size option of the find command in a number of useful ways. For example, if you want to find a file smaller than a certain size, use the – (minus) sign. This line of code finds any files smaller than 20MB beneath the directory named /home/oracle:
$ find . -size -20000k
If you want to find a file of an exact size, leave off the plus or minus sign before the size of the file designator. This example finds all files with the size of 16,384 bytes:
$ find . -size 16384c
5-29. Sorting Files by Size
Problem
You want to list files from largest to smallest.
Solution
The ls -alS command displays the long listing of all files sorted from largest to smallest; for example:
$ ls - alS
Here is a sample of the output:
total 4001584
-rwxr----- 1 oracle oinstall 2039488512 Jan 21 16:39 o1_mf_undotbs1_3gpysv9n_.dbf
-rwxr----- 1 oracle oinstall 983834624 Jan 21 16:37 o1_mf_sysaux_3gpystwj_.dbf
-rwxr----- 1 oracle oinstall 775954432 Jan 21 16:39 o1_mf_system_3gpysttv_.dbf
-rwxrwxr-x 1 oracle oinstall 176168960 Jan 21 02:31 o1_mf_temp_3gpz8s70_.tmp
To eliminate directories from the output, use the following technique:
$ ls -lS | grep ’^-’
If you want to reverse the order of the sort (smallest to largest), include the -r (reverse) switch:
$ ls -arlS
How It Works
If there are many files in a directory, you can combine ls and head to just list the “top n” files in a directory. The following example restricts the output of ls to the first five lines:
$ ls -alS | head -5
If you’re using Solaris, it might not have the -S option for the ls command. On Solaris systems, use a command such as the following to display files sorted by size:
$ ls -l | sort -nrk5 | head
Also be aware that the sort column (5 in the preceding line of code) may differ, depending on the long listing of the output.
5-30. Finding the Largest Space-Consuming Directories
Problem
You have a mount point that is out of space and you need to identify which directories are consuming the most space.
Solution
Use the du command to report on disk usage. The following example reports the top five directories consuming the most disk space beneath the current working directory:
$ du -S . | sort -nr | head -5
The -S (do not include size of subdirectories) option instructs du to report the amount of space used in each individual directory. By default, the output of space used is reported in kilobytes. Here’s a sample of the output:
1068448 ./lib
680104 ./assistants/dbca/templates
550140 ./bin
260136 ./rdbms/audit
227868 ./inventory/Scripts/ext/lib
If you want to report the cumulative space consumed by a directory, including its subdirectories, leave off the -S option:
$ du . | sort -nr | head -5
Here is the corresponding output:
6197828 .
1074732 ./lib
695236 ./assistants
684212 ./assistants/dbca
680104 ./assistants/dbca/templates
When not using the -S option, the top directory will always report the most consumed space because it is an aggregate of its disk space plus any spaced used by its subdirectories.
On some systems, there may not be an -S option. For example, on Solaris the -o option performs the same feature as the Linux -S option:
$ du -o . | sort -nr | head -10
Use man du to list all options available on your database server.
How It Works
The du command recursively lists the amount of disk space used by a directory and every subdirectory beneath it. If you don’t supply a directory name as an argument, du starts with the current working directory by default. The du command reports the amount of space consumed and the name of the directory on one line.
The du command has a variety of useful options. For example, the -s (summary) option is used to report a grand total of all space used beneath a directory and its subdirectories. This command reports on the total disk space used beneath the /orahome directory:
$ du -s /orahome
3324160 /orahome
You can also use the -h option to make the output more readable:
$ du -sh /orahome
3.2G /orahome
5-31. Truncating an Operating System File
Problem
You have a large trace file that is being written to by a database process. You know that the trace file doesn’t contain anything that needs to be retained. The trace file has filled up a disk, and you want to make the size of the file 0 bytes without removing the file because you know that a database process is actively writing to the file.
Solution
Copy the contents of /dev/null to the file. You can use either the cat command or the echo command to accomplish this. This example uses the cat command to make an existing log file 0 bytes in size:
$ cat /dev/null > listener.log
The other way to zero out the file is with the cp command. This example copies the contents of /dev/null to the trace file:
$ cp /dev/null listener.log
How It Works
One of us recently had a database that hung because one of the mount points was full, which prevented Oracle from writing to disk and subsequently hung the database. Upon further inspection, it was discovered that an Oracle Net trace file had grown to 4GB in size. The file had grown large because a fellow DBA had enabled verbose tracing in this environment and had forgotten to monitor the file or inform the other DBAs about this new level of tracing.
In this case, there was an Oracle Net process actively writing to the file, so we didn’t want to simply move or remove the file because we weren’t sure how the background process would react. In this case, it is safer to make the file 0 bytes. The /dev/null device is colloquially called the bit bucket. It is often used for a location to send output when you don’t need to save the output. It can also be used to make a file 0 bytes without removing the file.
Caution
Zeroing out a file permanently deletes its contents. Use the techniques in this recipe only if you’re certain you don’t need the information contained within the file.
5-32. Counting Lines and Words in a File
Problem
You want to count the number of lines and words in a shell script.
Solution
Use the wc (word count) command to count the number of lines and words in a file. This example counts the number of words in the rmanback.bsh shell script:
$ wc rmanback.bsh
35 204 1361 rmanback.bsh
The preceding output indicates that there are 35 lines, 204 words, and 1,361 characters in the file.
How It Works
If you want to see only the number of lines in a file, use wc with the -l option:
$ wc -l rmanback.bsh
35 rmanback.bsh
Similarly, if you want to display only the number of words, use the -w option:
$ wc -w rmanback.bsh
204 rmanback.bsh
If you want to see the line count of all files in a directory from smallest to largest, use the following:
$ wc -l *.* | sort -nk1
The preceding command pipes the output of wc to the sort command (sorting on the first column of the output).
5-33. Creating a Second Name for a File
Problem
When performing a new install of the Oracle binaries, your initialization parameter file is located in an Oracle Flexible Architecture (OFA) directory such as /ora01/admin/DBS/pfile. When starting a database, Oracle looks for the initialization file in the ORACLE_HOME/dbs directory by default.
You don’t want to maintain the initialization file in two separate directories. Instead, you want to create a link from the OFA directory to the default directory.
Solution
Use the ln -s command to create a soft link to another file name. The following creates a soft link for the physical file in /ora01/admin/DEV/pfile/initDEV.ora to the link of /ora01/product/12.1.0/dbs/initDEV.ora:
$ ln -s /ora01/admin/DEV/pfile/initDEV.ora /ora01/product/12.1.0/dbs/initDEV.ora
A long listing of the soft link shows it pointing to the physical file:
$ ls -altr /ora01/product/12.1.0/dbs/initDEV.ora
lrwxrwxrwx 1 oracle dba 39 Apr 15 15:58 initDEV.ora ->
/ora01/admin/DEV/pfile/initDEV.ora
If you need to remove a soft link, you can use the rm or unlink commands. As a precaution, you may want to create a copy of the file before you remove the soft link. Be careful that you remove the soft link, not the physical file. For this example, the soft link is removed (and not the physical file):
$ unlink /ora01/product/12.1.0/dbs/initDEV.ora
The physical file located in the /ora01/admin/DEV directory should still exist.
How It Works
A soft link (also referred to as a symbolic link) creates a file that acts as a pointer to another physical file. Soft links are used by DBAs when they need a file to appear as if it were in two separate directories, but physically resides in only one location.
The technique described in the solution of this recipe is commonly used by Oracle DBAs to manage the initialization file. This technique allows DBAs to view and edit the file from either the soft link name or the actual physical file name.
5-34. Creating a Second Name for a Directory
Problem
You want to physically move a datafile to a different disk location without having to change any of the Oracle metadata.
Solution
Use soft links to make a directory look like it exists, when it is really just a pointer to a physical location. This example shows how to move a tablespace datafile from one mount point to another, without having to change the datafile’s name as it appears in the data dictionary. In this example, the datafile will be moved from /oradisk1/DBS to /oradisk2/DBS.
On this server, the following physical mount points exist:
/oradisk1/DBS
/oradisk2/DBS
A long listing shows the ownership of the mount points as follows:
$ ls -altrd /oradisk*
drwxr-xr-x 3 oracle oinstall 4096 Apr 15 19:17 /oradisk2
drwxr-xr-x 3 oracle oinstall 4096 Apr 15 19:19 /oradisk1
Create the following soft link as the root user:
# ln -s /oradisk1 /oradev
Here’s a simple test to help you understand what is happening under the hood. Change directories to the soft link directory name:
$ cd /oradev/DBS
Notice that if you use the built-in Bash pwd command, the soft link directory is reported:
$ pwd
/oradev/DBS
Compare that with the use of the pwd utility located in the /bin directory, which reports the actual physical location:
$ /bin/pwd
/oradisk1/DBS
Note
You can also make the Bash built-in pwd command display the physical location by using the -P (physical) option (see recipe 5-1 for more details).
Next, create a tablespace that references the soft link directory. Here’s an example:
SQL> CREATE TABLESPACE td01
DATAFILE ’/oradev/DBS/td01.dbf’ SIZE 50M;
A query from V$DATAFILE shows the soft link location of the datafile:
SQL> select name from v$datafile;
Here’s the output pertinent to this example:
/oradev/DBS/td01.dbf
Next, shut down your database:
SQL> shutdown immediate;
Now move the datafile to the new location:
$ mv /oradisk1/DBS/td01.dbf /oradisk2/DBS/td01.dbf
Next (as root) remove the previously defined soft link:
# rm /oradev
Now (as root) redefine the soft link to point to the new location:
# ln -s /oradisk2 /oradev
Now (as oracle) restart the database:
SQL> startup
If everything goes correctly, your database should start. You have physically moved a datafile without having to change any data dictionary metadata information.
How It Works
Using soft links on directories gives you some powerful options when you relocate datafiles. This technique allows you to make Oracle think that a required directory exists when it is really a soft link to a different physical location.
The techniques in the “Solution” section of this recipe are useful when duplicating databases to a remote server using RMAN. In this situation, you can use symbolic links to make the auxiliary database server look similar to the source database server filesystem. It provides a method for relocating databases to servers with different mount points from the original server in which you can make a mount point or directory look like it exists to Oracle when it is really a soft link.
5-35. Viewing a Large File
Problem
The database has dumped a large trace file, and you’re troubleshooting the issue and are attempting to view the file with a text editor (e.g., vi). You receive the following error:
Tmp file too large
:
You need to somehow read this file to diagnose the problem.
Solution
In this scenario, if you want to scroll through the file, you can use a tool such as more or less that allow you to view portions of the file; for example:
$ more TRG_m000_1489.trc
If you know the information you’re interested in is near the end of the file, you can use tail to create a separate file containing the content you’re interested in:
$ tail -100000 TRG_m000_1489.trc > out.txt
You can also use a utility such as split with the -l (lines) option to break the file into pieces:
$ split -l 100000 TRG_m000_1489.trc
The original file is still intact, but you should now see several files that begin with an x character. Each x file contains a portion of the original trace file based on the number of lines that you specified; for example:
$ ls x*
xaa xab xac xad xae xaf
You should now be able to view these smaller files individually with the text editor. You have some control over the names of the split files; for example, the following names the split files with the string "new":
$ split -l 100000 TRG_m000_1489.trc new
A quick listing verifies this:
$ ls new*
newaa newab newac ...
How It Works
Sometimes databases dump large trace files when encountering problems. If a file is too big to fit in the memory area being used by the text editor, you won’t be able to directly view it; you’ll have to use one of the techniques discussed in the “Solution” section to view the file.
The more, less, and tail commands operate on smaller portions of the large file, so they can present the large file in a piecemeal fashion. The split command is very useful for taking a large file and breaking it into smaller pieces.
Depending on your version of the OS, the split command may be equipped with the -n parameter, which allows you to specify the number of chunks a file is divided into. For example, to create four split files that have roughly the same size, use the following:
$ split -n4 TRG_m000_1489.trc
If the -n option isn’t available, you can use the expr command and command substitution to calculate the sizes. The following example splits the trace file into four equal pieces based on line count:
$ split -l $(expr $(wc -l TRG_m000_1489.trc | awk ’{print $1}’) / 4) TRG_m000_1489.trc
5-36. Downloading Files
Problem
You want to use a command-line tool to download a file from a remote web site.
Solution
There are multiple tools for downloading files from the Internet; this recipe focuses on two feature-rich utilities: curl and wget. First up is curl.
curl
The curl (transfer a URL) command is an extremely robust tool for downloading files from remote web sites using common network protocols (e.g., HTTP, HTTPS, FTP, FTPS, and so on). For example, suppose you want to download a useful DBA shell script from the github.com web site. You can do so as follows:
$ curl -kL github.com/ardentperf/racattack/raw/master/makeDVD/auto.sh -o auto.sh
The -k option is for an unsecured download, and -L signifies the location. The -o option allows you to specify the name of the file created locally. If successful, you should now have a copy of the auto.sh file.
Here’s an example of downloading a file from an FTP site:
$ curl http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz -o wget-1.5.3.tar.gz
In this manner, you can use the command line to download files from the Internet. This section contains only a few examples of how to use curl. There are many options and features available with the robust downloading tool. Use the curl --help command to display all options.
wget
The wget (network downloader) utility can also be used to download files from remote web sites. For example, here we download a file from the github.com web site:
$ wget https://github.com/ardentperf/racattack/raw/master/makeDVD/auto.sh
Here’s another example of using wget, in which a tar file is downloaded from an FTP site:
$ wget http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz
If you want to rename a file, use the -O option; for example:
$ wget http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz -O my.tar.gz
You can also create a text file with the names of files you want to download and then use the -i option to instruct wget to download the file names within the text file. For example, suppose the file names are placed in a file named download.txt:
$ wget -i download.txt
In this way, you can efficiently automate file download from the Internet from the command line. We’ve only scratched the surface of the features available with wget. Use wget --help for a quick reference of all parameters available.
How It Works
It’s occasionally useful to be able to download files from remote servers. These files could be scripts that a DBA has posted or Oracle installation software. The curl and wget commands allow you to download remote files, provided that you have the download URL address. The basic syntax for these commands is as follows:
$ curl "download_url" -o file_name
$ wget "download_url" -O file_name
Files can be downloaded in this manner from any web site that allows downloads. For example, it is possible to download files from the My Oracle Support (MOS) web site. If you have an authenticated account, you can download files directly by using the following syntax (you must have a valid username and password to do this):
$ wget --http-user=user@domain.com --ask-password "file_url" -O file_name
Using this method to download files allows you to script and automate tasks that otherwise would require using a web browser to initiate a download. There’s nothing wrong with using a web browser (indeed, this is how most files are downloaded), but if you have the need to automate a download task via the command line, curl and wget are extremely flexible and powerful utilities for this task.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2015 Darl Kuhn, Charles Kim, and Bernard Lopuz
About this chapter
Cite this chapter
Kuhn, D., Kim, C., Lopuz, B. (2015). Chapter 5: Managing Files and Directories. In: Linux and Solaris Recipes for Oracle DBAs. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-1254-7_5
Download citation
DOI: https://doi.org/10.1007/978-1-4842-1254-7_5
Publisher Name: Apress, Berkeley, CA
Print ISBN: 978-1-4842-1255-4
Online ISBN: 978-1-4842-1254-7
eBook Packages: Professional and Applied ComputingProfessional and Applied Computing (R0)Apress Access Books