Python is an excellent language to learn, for DBA’s who want to automate all the repetitive tasks, they need to perform. Once you start using python it is likely that you want to setup multiple environments with different versions of Python and libraries, based on the project you are working on. Virtualenv is a tool to create isolated python environments.
Below are the steps that i followed, to install a brand new working Python 2.7.6 environment with the following packages.
SQLAlchemy - Object Relational Mapper and SQL Toolkit for Python
numpy - Fundamental package for scientific computing
matplotlib - Python 2D plotting library
ipython - Interactive Python Shell
pandas - Python Data Analysis Library
Flask - An easy to use Python Lightweight Micro Framework
This installation is performed on Ubuntu Linux, and i have already installed the libsqlite3-dev package and the oracle instant client.
Install Python 2.7.6
Download Python-2.7.6.tgz from http://www.python.org/getit
Install python 2.7.6 to your directory of choice.
tar -xvf Python-2.7.6.tgz
Now you have python 2.7.6 installed into the /u01/Rk/Apps/Python/Python276 directory. (You will have a python binary in /u01/Rk/Apps/Python/Python276/bin)
OraChk (Previously known as Raccheck) is a utility from oracle to perform configuration checks on Oracle database platforms, and report on configurations that do not match oracle’s best practices. OraChk has the ability to upload the results of its checks into an oracle database. Details about this utility can be found in Mos Note 1268927.1
Oracle has now released OraChk Collection Manager which is a companion application to OraChk, which has an Oracle Application Express, Front End which can be used as a dashboard, in which customers can track their ORAchk, RACcheck and Exachk collection data in one easy to use interface.
Details about downloading and using “OraChk Collection Manager” can be found in Mos Note 1602329.1
On Dec 11 2013 Oracle corporation announced the general availability of, the 5th generation of the Oracle Exadata Database Machine X4. This new version of the database machine introduces new hardware and software to accelerate performance, increase capacity, and improve efficiency and quality of service for database deployments.
In this blog post i’ll review in detail the enhancements in the Oracle Exadata x4-2 database machine.
The 2 socket version of the Database machine, X4-2 gets new database servers and new storage servers. The 8 socket version of the database machine, still called the x3-8, gets the new storage servers but keeps the same database serves from the previous release. Hence the Lack of change in Name to the X4-8. The name stays as X3-8.
Improvements in the database servers (X4-2)
The database servers in the x4-2 are the Sun X4-2 servers. They come with the 2 Twelve-Core Intel® Xeon® E5-2697 v2 Processors (Ivy Bridge) (2.7GHz), with turbo boost enabled by default. This cpu at times can clock upto 3.5Ghz.
Comparing this with the previous version x3-2, this is a 50% increase in number of cores per database server. The x3-2 had 16 cores per node and the x4-2 has 24 cores per node. In a x3-2 full rack there were 128 cores and in the x4-2 full rack there are 192 cores. That gives 64 more cores in the x4-2 full rack compared to the x3-2 full rack.
Each database server has 256Gb of Dram. This is optionally expandable to 512Gb per node.So you can have either 2Tb or 4Tb of Ram in a full rack of x4-2.
The database servers now have 4, 600Gb hard disks in them (Internal storage, used for O/S, Oracle Binaries, Logs etc). The x3-2 used to have 300Gb disks. The disk controller batteries are online replaceable
The database servers have 2 X Infiniband 4 X QDR (40Gb/S) Ports (PCIe 3.0). Both ports are Active. The improvements in the x4-2 are that these are now PCIe 3.0, and that the ports are used active active. (The ports were used Active/Passive in the previous release)
Improvements in the storage servers (Exadata cells x4-2)
Each x4-2 exadata cells come with 2, Six-Core Intel® Xeon® E5-2630 v2 Processors (2.6 GHz).
Each x4-2 cell, now has 96Gb of Ram. The x3-2 used to have 64 Gb of Ram.
Flash Cache Improvements
The Exadata x4-2 cells, now have the F80 PCIe Flash cards. There are 4, 800Gb F80 flash cards in each cell. So each cell has 3.2Tb of flash cache. In an Oracle Exadata database machine x4-2 full rack there is 44TB of flash cache (Used to be 22Tb in the x3-2). The Oracle Exadata database machine x4-2 full rack now provides, 2.66Million read iops , 1.96Million write iops from the flash cache. This is 70% more flash iops than the previous generation.
The Exadata cell x4-2 High Performance version, now uses 1.2Tb 10k Rpm High Performance SAS disks.
A full rack of High Performance disks gives 200TB of raw space, 90TB of usable space with normal mirroring and 60TB of usable space with High Mirroring
The Exadata cell x4-2 High Capacity version, now uses 4Tb 7.2k Rpm High Capacity SAS disks.
A full rack of High Capacity disks gives 672TB of raw space, 300TB of usable space with normal mirroring and 200TB of usable space with High Mirroring
The Disk controller batteries are online replacable.
The Exadata storage servers have 2 X Infiniband 4 X QDR (40Gb/S) Ports (PCIe 3.0). Both ports are Active. The improvements in the x4-2 are that these are now PCIe 3.0, and that the ports are used active active. (The ports were used Active/Passive in the previous release)
Software Improvement Highlights
The new x4-2’s get a new version of the exadata storage server software version 188.8.131.52.0 (This version of the cell software can be installed on the v2,x2,x3 and x4 platforms).
Exadata Flash Cache Improvements
Data written to the flash cache is now compressed. This compression is done by the controller (Hence near Zero overhead). This data is automatically decompressed when it is read from the flash cache. Up to 2X more data fits in smart flash cache (If you get 2x compression of the data, you could fit upto 80TB of data on the flash cache), so flash hit rates will improve and performance will improve for large data sets. This feature can be turned on in both x3-2 and x4-2 storage servers. Customers need to license the Advanced Compression option to use this new feature.
Enhancements to the smart flash cache software, enables exadata software to understand database table and partition scans and automatically caches them when it makes sense (This will help eliminate the need to specify CELL FLASH CACHE KEEP).
Exadata network resource management
With a new version of the Infiniband switch firmware 2.1.3-4, Exadata network resource management now prioritizes messages through the entire infiniband fabric. Latency sensitive messages like redo log writes are prioritized over batch, reporting and backup messages.
Infiniband Active Active Ports
Double-ported infiniband PCIe-3.0 Cards used in the database servers and storage servers, implement active-active mode usage of the infiniband ports, providing a 80GigaBits Per Second network bandwidth (Used to be 40Gigbits ber second on the x3-2, since it was Active Passive bonding).
The Rdbms software and the clusterware software already had the ability to send packets via multipe interfaces. Enhancements have been done to the RDS kernel drivers, which now have the ability to sense if one of the ports is down, and route the network traffic through the surviving port. On the x4-2’s when active active Infiniband networking is setup you will not see the bondib0 interface, instead you will see a ib0 and ib1.
The power,cooling and airflow requirements remain similar to that of the x3-2.
The storage expansion racks have also been refreshed and provides the increased flash cache and increased disk space.
A single Database Machine configuration can have servers and storage from different generations V2, X2, X3, X4.
Databases and Clusters can span across multiple hardware generations.
The half and full x4-2 racks, do not ship the spine switch anymore. The storage expansion racks still ship with the spine switch.
A new One Command available through patch 17784784 has the new Exadata Deployment assistant that supports the x4-2’s.
This post is equally applicable to Rac databases deployed, both on oracle database machines as well as traditional server/storage architectures, running RHEL5 or OEL5 (OEL6 has a new feature called transparent hugepages).
High amounts of memory and cpu consumption can result in some real bad things happening (Like for eg: Node evictions). So it is important to configure the operating system and the databases you deploy on the machine optimally, to use the available CPU and memory resources.
Let us first review what those available memory and cpu resources are on an Oracle Exadata Database Machine, on each database node.
[table “1” not found /]
Anytime a workload is deployed on a database server, that exceeds the, operating systems ability to efficiently use the above mentioned, available cpu and memory resources, bad things can happen. (I know i am over simplifying in the prior comment, but you get the idea).
Managing Memory Resources.
Setting up HugePages
First we need to make sure that HugePages on each node, is setup correctly, for the databases you have deployed on the node.
Tanel Poder has explained the reasons for setting up HugePages on Exadata systems.
When the database machines are installed, the HugePages is configured to accomodate the one database that is pre-installed. So if you change the SGA setting for that database or deploy more databases you should now adjust the HugePages setting accordingly. It goes without saying that if you have already configured the hugepages to accomodate all the databases you have, and then you removed some databases from the node, or resized the SGA’s to be smaller, you should again resize your hugepages to free up the excess hugepages.
Once all the database instances are configured and started up you can run the script, hugepages_settings.sh from Mos Note 401749.1, which calculates the number of hugepages you need to setup. I usually recommend adding 10% to the value that hugepages_settings.sh suggests.
With 184.108.40.206 and 220.127.116.11 you should also be setting USE_LARGE_PAGES, preferably to ONLY, following the instructions in Mos Note 1392497.1.
You can then follow the instructions in MOS note 361323.1, under the section “Kernel version 2.6”, to set the value. You need to restart the databases so the SGA gets allocated with the hugepages.
You should also follow instructions from MOS note 1546861.1, and set the value of vm.min_free_kbytes = 524288, in /etc/sysctl.conf, to avoid page allocation failure messages when kernel memory is depleted.
Managing Memory Usage
Ideally for critical implementations, your SGA+PGA+Individual server processes, memory allocations should not exceed 75% of the physical memory on the database node.
The Exadata consolidation white paper above suggests that
For critical Hardware Pools, we recommend an even more conservative approach by not exceeding 75% physical memory per database node. OLTP applications:
SUM of databases (SGA_TARGET +PGA_AGGREGATE_TARGET) + 4 MB * (Maximum PROCESSES) < Physical Memory per Database Node DW/BI applications:
SUM of databases (SGA_TARGET + 3 * PGA_AGGREGATE_TARGET) < Physical Memory per Database Node
You can monitor the pga usage a few different ways.
1) SELECT s.inst_id, s.sid, s.serial#, p.spid, s.username, s.program, p.pga_used_mem FROM gv$session s JOIN gv$process p ON p.addr = s.paddr AND p.inst_id = s.inst_id WHERE s.type != ‘BACKGROUND’ and s.program not like ‘%(P%’ and p.pga_used_mem > <APPLICATION_MEMORY_THRESHOLD>
order by s.inst_id, s.sid, s.serial#;
The value you use for APPLICATION_MEMORY_THRESHOLD is dependent on your application and howmuch pga it needs to efficiently run your queries (Performing the sorts in memory using the space for PGA is usually faster than using the TEMP tablespace). However the white paper suggests using 1Gb for OLTP applications and 10Gb for DSS applications.
2) You can monitor the sysmetric “Total PGA Allocated” from dba_hist_sysmetric_summary, to see how much PGA is being allocated.
3) You can use the “PGA Memory Advisory” and “SGA Target Advisory” sections of the awr report, to guage, the impacts of increasing or decreasing the SGA and PGA Allocations.
4) You can monitor the memory usage at the o/s level using the top (Mem: free),free or vmstat (bo (page-out),bi (page-in)) commands.
Managing O/S Processes (And as a consequence CPU usage).
It is important to control the number of processes that are actively executing on each node. Each of these processes consume cpu and memory resources.
The following guidelines are important to follow.
– Use Application connection pooling to appropriately limit the number of sessions connected to the oracle database.
This is important for two reasons.
– Right-Size the resource consumption on each database node.
– Minimize the impact of connection storms.
– Configure the parallel query servers appropriately.
– For all the database’s deployed on a node the total setting for PARALLEL_MAX_SERVERS should be less than or equal to the following
X2-2 or X3-2, <= 240
X2-8 or X3-8, <= 1280
I often see implimentations where parallel_max_servers = 240 (or a high value) in each instance on the machine, which results in not so good consequences. I have occassionaly seen systems where there is sustained 100% cpu usage with load averages of > 300, resulting in node reboots.
– Use Instance Caging and Database Resource Manager to manage cpu allocation on the server.
It would be a good idea to review the performance management screens from Enterprise Manager 12c every day to have a good understanding of the database and server workloads, and/or analyze the oswatcher data to understand the server workload profile.
Oracle 12c has introduced the ability to define a column in a database table as a “Identity” Column. This results in the database automatically using a sequence to generate the values that get inserted into this column. The Oracle-Base blog has an article on how to use this functionality and some performance comparisons between this and trigger based methods.
Below is a bit more detail regarding this functionality.
As stated in the Oracle-Base article, oracle auto-creates a sequence and uses it to generate the values. There seems to be a new data dictionary view, USER_TAB_IDENTITY_COLS, which shows you which columns in the table are defined as the Datatype IDENTITY.
The query below can be used to, identify the sequence name, that oracle generated, for a given column, defined as an IDENTITY datatype.
SQL> set long 999999999
SQL> select data_default from user_tab_columns where table_name = 'T1'
2 and column_name = 'COL1';
SEQUENCE_NAME INCREMENT_BY CACHE_SIZE
-------------------- ------------ ----------
ISEQ$$_92434 1 20
The sequence it uses, gets created with a default of 20 for the cache size. This will likely be unacceptable in Real Application Clusters environments. If you try to modify the cache size of the sequence after the creation you will get a ORA-32793 error “cannot alter a system-generated sequence”.
In such cases it would be wise to use the syntax below during table creation, and specify the CACHE_SIZE and other sequence parameters you would want to change.
SQL> create table t1 (col1 number generated as identity (start with 100 increment by 2 cache 1000), col2 varchar2(50));
Once the table is re-created, you can check the cache size again to verify.
Now that we have a 12cr1 database Up and running, it is time to connect to the database and start working. I prefer using the oracle instant client and sqlplus from my Ubuntu Linux desktop to connect to the 12cr1 database. Below are the steps to install, configure and use the 12cr1 instant client on a Ubuntu Linux 64 bit desktop and connect to the oracle 12cr1 database.
Download and Install Instant Client 12cr1
From the “Instant Client Downloads for Linux x86-64” page, download the highlighted zip files. This gets you the basic files needed for Sql*Net connection to the 12cr1 database and also the sqlplus executable.
Unzip the files to a directory on your desktop. (In my case i unzipped the files to the directory /u01/Rk/Apps/oracle/instantclient_12_1).
In the following steps, i’ll be using dbca (Database creation assistant) to create a container database (CDB) with one pluggable database (PDB).
The following steps assume that
You have successfully installed Oracle Linux 6
You have Installed the 12cR1 rdbms binaries
You are still setup to redirect your DISPLAY to your desktop.
Create a Listener
Login as oracle to your virtual host (Redirect your display to your desktop)
Invoke the executable “netca” (Network configuration assistant) from the linux command line. You will see the screen shown above. Click Next
Choose “Add” in the screen above, Click Next.
Choose the default name “LISTENER” for the listener Name, Click Next.
Choose the default, TCP protocol, Click Next.
Choose the default standard port 1521, Click Next.
Choose “No” for configuring another listener, Click Next.
The netca assistant will complete the Listener Configuration and will prompt you to Exit.
Create a New Database
Prior to Oracle 12c all databases were Non-CDB databases (ie a database that is Not a Multi Tenant Container Database). Starting with 12c you can create databases as Non-CDB databases or CDB (MultiTenant Container) databases.
In the following steps i’ll be using dbca (Database Configuration assistant) to create a CDB with one PDB in it (Pluggable database).
Login as oracle to the Oracle 6 Linux, virtual box image, and invoke dbca from the o/s prompt, you will see the banner first and then the screen below.
Choose “Create Database” and Click Next.
Choose “Advanced Mode” and Click Next.
Choose “General Purpose Or Transaction Processing” and Click Next.
Provide your chosen “Global Database Name”, SID Name, Check the box against “Create As Container Database”, Choose the Radio Button “Create a Container Database with one or More PDB’s” , Provide a “PDB Name” and Click Next.
Check the Box to Configure “Em Express”, Click Next.
Enter the Credentials for the Administrative Users, Click Next.
Choose the default Listener “LISTENER” and Click Next.
Choose “Use Common Location for all database files”, Provide the location of the datafiles (I provided “/u02/oradata”) and Uncheck “Specify Fast Recovery Area” (This being a test database, i am not setting it up to be in archivelog mode). Click Next.
Choose to install “Sample Schema’s”, Click Next.
Choose to use “Automatic Memory Management”, Click on “Character Sets” and choose “AL32UTF8”, Click Next.
Check the boxes for “Create Database”, “Save as Database Template” (For future reuse), “Generate Database Creation Scripts”, Click Next.
In the screen above, Click “Finish” and the database installation will start.
If all goes well, the installation will complete, and dbca will prompt you to exit.
Congratulations !!! Now you have a brand new 12cr1 database. Now it is time to connect and start exploring the Ground Breaking Innovations in the Oracle 12c database.
In this previous post, i showed how to build a Virtual Box image, that runs the Oracle Linux 6 Update 4 x86-64 bit operating system. I set the image up with all the pre-requisites for installing the Oracle Rdbms 12c Release1. In this blog post i’ll describe the steps i executed to install a single instance, oracle 12c Oracle Home on Oracle Linux 6.
Download the Oracle 12c Software
Download the above two files to your desktop.
Scp them to the Oracle Linux 6 server.
I copied the zip files into a directory named /u02/sw (Owned by oracle) and unzipped both of them in that directory.
Since this is a test install, i am not really following the oracle best practices on separating the Oracle Base, Oracle Inventory location and Oracle Home. In my installation all these are going into /u01/12cr1
Setting up the X Display
Since the installer is a graphical user interface, we need to redirect the display to the client (Unless you are installing using a Vnc client, or logged into the Oracle Linux 6 virtual machine using a Gui).
I am running this on ubuntu linux, so my ubuntu linux desktop is the X client.
I login to the Oracle Linux 6 virtual machine using the following command, which redirects the display to my desktop.
ssh -X -l oracle 192.168.56.101
Install Oracle 12c Binaries
Login as the oracle user to the OEL6 virtual machine
Invoke runInstaller as shown in the screen above
The 12c Logo displays
Click Next on the above 2 screens.
In the screen shown above, choose “Install database software only”. This way we install just the rdbms binaries and then we’ll create the database afterwards.
In the screen above choose “Single Instance database installation”.
In the Screen above choose “English” as the language.
In the Screen above, chooose “Enterprise Edition”
In the Screen above, choose the Oracle Base as “/u01/12cr1” and the Oracle Home as “/u01/12cr1/dbh1”
In the Screen above Choose the location of the Oracle Inventory as “/u01/12cr1/oraInventory”
In the above Screen, choose the defaults
In the screen above, the installer complains about the physical memory, shmall and shmmax.
In my case shmall and shmmax are set much higher than the values that oracle recommends.
If you have set the memory for the virtual machine to 2Gb as i mentioned in my previous post, the warning about the physical memory should not be present. So it is ok to ignore these warnings and proceed with the installation, as shown in the picture above.
The installer displays the installation locations. Click on “Install” to start the installation.
The installation proceeds as shown in the figure above.
After the binary installation, the installer prompts you to run two scripts.
Login as root, to the Oracle 6 virtual machine, from another screen.
Then click Ok on the screen above.
Click “Close” in the screen above to complete the rdbms binary installation.
In the next blog post, i will document the steps to create and start a Container database and Pluggable database.
Oracle Database 12c was recently announced (On June 1 2013) to be Generally Available for customers to use. I created a new Oracle Linux 6 virtual box, virtual machine and installed Oracle 12c Release 1 on it. So i’ll blog about the steps i used, with screenshots, in a 3 part blog series. This first one is this post, where i’ll document the steps i followed to create the Oracle Linux 6 – 64 bit virtual machine (In the next 2 posts i’ll document the process used to install the rdbms binaries and create the database).
Start the Oracle VM VirtualBox Manager from your program Menu
Click on “New”, you get the screen Below
Choose the Name as “OEL6” (Any name you want) and Type “Linux” and Version “Oracle (64 Bit)” as in the picture above, Click Next.
In the above screen, choose the file type to be “VDI”, Click Next.
Choose the file location and size in the screen above, Click Next.
Add the Oracle Linux ISO as a bootable Drive.
In the Oracle VM VirtualBox Manager, click and Highlight the new virtual Machine “OEL6” that we created.
In the right hand side of the window you will see the details regarding this new image.
Now would be a good time to click on “System” and increase the memory allocated to the Virtual Machine to 2Gb and the Number of Processors to 2.
Then click on “Storage”, which brings you to the screen below.
Click on the icon pointed to by the Red Arrow in the Figure Above, to create a new Dvd Drive.
Choose the Oracle Linux 6 iso image that you just downloaded from edelivery.oracle.com.
This action will result in a new DVD drive being displayed in the storage screen, as in the Figure above. Click OK and return to the “Oracle VM VirtualBox Manager” main screen.
Install Oracle Linux 6 Update 4
In the Left panel of the Main Menu, Highlight your new Virtual Box image “OEL6”, and click on “Start”. It brings up the screen below
Hit Enter, it will Bring you to the screen below
In this screen i chose to skip the test of the media, tab to skip and hit enter.
In this screen i chose “English”, hit Enter
In this screen i chose “U.S English”, hit Enter
Choose “Basic Storage Devices” and in the next screen Choose “Yes Discard Any Data”
In the above screen, i left the hostname as “localhost.localdomain” and did not configure networking. (Configured these manually later).
It autodetects your timezone, click Next.
Enter a password for the root user and click Next
Choose “Use All Space” and click Next, Confirm “Write Changes to Disk”, Click Next.
I chose “Database Server” in the screen above. Keep in mind that once you do this, you do not get a GUI to login to the machine. All access has to be via the command line. If you want a GUI, you can choose “Basic Server”.
The above two screens appear when the installation is in progress, and as soon as the installation is complete.
Congratulations, you now have a OEL6 Update 4, 64 bit Virtual Box image !!!
Configure Networking and Hostname
At this point it would be best if you configure the virtualbox image to use a “Bridged Network”, so that it can communicate with a ULN server, to get the updates and required packages.
This sets up a static IP address and plumbs up eth0 after reboots.
Install Pre-requisite RPM’s (For a 12cR1 oracle installation)
Login as root
Run “yum update” (This will bring any new versions of installed packages that are available on public-yum)
Run “yum install oracle-rdbms-server-12cR1-preinstall” (This will install all the pre-requisite rpm’s, setup the oracle user, setup the oinstall and dba groups, update kernel parameter settings and security settings required for an oracle installation).
Most of the work is done, you are almost there.
Create Additional Hard Disks
I like to separate my binary and database installations into separate hard disks, so now i go ahead and create two additional hard disks.
Create two new hard disks in Virtual Box
From the Oracle VM VirtualBox Manager, click on your Virtual Box Image “OEL6”
Click on Storage in the right hand side panel.
Click on the Hard Disk Icon next to “Controller SATA” pointed to by the Red Arrow
Click on “Create New Disk”
Choose “VDI” for hard disk file type
Choose “Dynamically allocated” for allocation type
Choose the proper File Location and File size as 10GB
Follow the above procedure and add a second hard disk (This one 20Gb) as shown below.
Now you should have two additional hard disks in this Virtual Box Image.
I will be using the disk OEL6-u01 as the device for the /u01 (10Gb) file system and OEL6-u02 as the device for the /u02 (20Gb) file system.
Now when you reboot the Linux virtual machine, OEL6-u01 will show up as /dev/sdb and OEL6-u02 will show up as /dev/sdc.
Partition, Format and Mount the hard disks
Once the devices are created and the VirtualBox Virtual machine has been rebooted, login as root.
Use the fdisk -l commands as shown above, to check that the disks /dev/sdb and /dev/sdc exist and are the correct sizes.
Partition the disks as shown above.
Format the disk /dev/sdb1 as shown above. Repeat the process for /dev/sdc1
Add the entries shown above to /etc/fstab so that the filesystems are mounted everytime you reboot the machine.
At this point, i shut down the virtual machine and switched it from using a “Bridged Network” to using a “Host Only” network.
Now we have a Oracle Linux 6 Virtual Machine that is ready for a Oracle 12c database Release 1 installation.
My next post will walk through the steps to install a single instance Oracle 12c Release 1 Rdbms Environment.
On the oracle database machine, oswatcher is installed during setup time, both on the database nodes and the exadata cells. This utility collects linux operating system level statistics, which comes in very handy when troubleshooting operating system level issues. The data is collected in text files. There is a Java based utility (OSWG) provided by oracle support to graph the contents of these files, however that utility does not work on the oswatcher files generated on exadata.
Here is a python script that can graph the cpu used from the mpstat information that oswatcher captures. It has been tested on new oswatcher files on an x3-2. You need to first install a python environment that has the “numpy” and “matplotlib” modules installed.
Install a Python Virtualenv.
If you create multiple applications using Python and end up using different versions, it is easier to maintain different virtualenv’s. You can create a python virtualenv as shown below (On ubuntu linux).
Now that you have a python environment, with your required libraries, you can go ahead and execute the script as shown below.
The oswatcher files in /opt/oracle/oswatcher are .bz2 files and there will be one file per hour per day. Copy the mpstat .bz2 files into a directory and use bunzip2 to unzip them. In this example let us say that the directory name is /u01/oswatcher/mpstat/tmp