What you should know about Docker Containers for Oracle Data Integrator

Not long ago, Oracle adopted the Docker engine as one of the accepted platforms for its products and published a set of scripts and examples in the Oracle GitHub repository. This includes sets for rapid deployment of Oracle databases, Weblogic and for a number of other products. I tried some of the published docker implementations including Oracle database using my sandbox and it worked pretty well for me. Among the published containers, I didn’t find an example for Oracle Data Integrator (ODI) and decided to make one just for myself. I created it and found it to be useful when you need just to test one or another ODI scenario and wipe everything out after the tests.

I’ve decided to share my scripts with you and describe how it had been made. Just before publishing the post, I checked the Oracle git repository for docker and found a new “OracleDataIntegrator” template there. So, now we have the proper version from Oracle but even having the “official” deployment, you may find my version useful and I hope it may help you to understand the Docker engine.

To build my version, I used the Oracle developed images for Oracle Linux, Oracle Java, and Oracle Database as a basis for ODI making only minor modifications to have the final product better suited to my needs. You may find that other solutions are better and perhaps more adapted for your needs, but please keep in mind that my example is purely educational and not designed for any production use.

First we need the Docker engine on your machine. Depending on your platform, you need to download an appropriate engine from the Docker site and install it. The installation is pretty straightforward and easy. If you are on Linux you may need to add your user to the “docker” group. For example, I am adding user oracle to the group to be able to run docker containers and have access to the docker registry from the Oracle account:

[[email protected] ~]# gpasswd -a oracle docker
Adding user oracle to group docker
[[email protected] ~]# 

And make sure the docker service is up and running:

[[email protected] ~]# systemctl status  docker.service
? docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/docker.service.d
           ??docker-sysconfig.conf
   Active: active (running) since Tue 2017-06-06 12:29:35 EDT; 1 months 24 days ago

Then you need to clone or download the Oracle git repository for docker containers to be able to build necessary base images before starting with the ODI. On your system with docker, you either run “git clone https://github.com/oracle/docker-images.git” or go to the https://github.com/oracle/docker-images in your browser, download and unzip the full archive. There you will be able to find scripts to build docker images with all the different sets of software. Here is my listing after cloning it from the Oracle git repository:

[[email protected] docker-images]$ ls -l
total 68
drwxr-xr-x. 4 oracle oinstall  4096 May 11 16:42 ContainerCloud
drwxr-xr-x. 6 oracle oinstall  4096 May 11 16:42 GlassFish
drwx------. 2 root   root     16384 May 11 16:54 lost+found
drwxr-xr-x. 2 oracle oinstall  4096 May 11 16:42 MySQL
drwxr-xr-x. 7 oracle oinstall  4096 May 11 16:42 NoSQL
drwxr-xr-x. 5 oracle oinstall  4096 May 11 16:42 OpenJDK
drwxr-xr-x. 4 oracle oinstall  4096 May 11 16:42 OracleCoherence
drwxr-xr-x. 5 oracle oinstall  4096 Jul 18 14:40 OracleDatabase
drwxr-xr-x. 4 oracle oinstall  4096 May 11 16:42 OracleHTTPServer
drwxr-xr-x. 6 oracle oinstall  4096 May 11 16:42 OracleJava
drwxr-xr-x. 4 oracle oinstall  4096 May 11 16:42 OracleTSAM
drwxr-xr-x. 4 oracle oinstall  4096 May 11 16:42 OracleTuxedo
drwxr-xr-x. 5 oracle oinstall  4096 May 11 16:42 OracleWebLogic
-rw-r--r--. 1 oracle oinstall  1588 Jul 17 09:10 README.md

(The listing represents the state as of May 2017 and may look different now)

Let’s continue and go step by step preparing our images. To understand the process, let me recall what we need for ODI in standalone mode. We need a Linux box with installed JDK or JRE and Oracle database as a repository. Also, if we plan to use the ODI Studio, it makes sense to have either X window or a VNC server installed on the box. We start with building a Linux image with Java JRE or JDK for our ODI. Oracle provides “OracleJava” docker configuration where we can build an image with Java 7 or 8:

[[email protected] docker-images]$ ll OracleJava
total 20
drwxr-xr-x. 2 oracle oinstall 4096 May 11 16:42 java-7
drwxr-xr-x. 2 oracle oinstall 4096 May 12 10:56 java-8
-rw-r--r--. 1 oracle oinstall 1886 May 11 16:42 README.md
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 windows-java-7
drwxr-xr-x. 4 oracle oinstall 4096 May 11 16:42 windows-java-8

I used Oracle JDK 8 instead of a server JRE distribution. To make that happen, I’ve slightly modified the Dockerfile in the “OracleJava/java-8” directory replacing the server JRE distribution by JDK. It is optional and you may choose to keep JRE instead. In my case, the original string in the file was replaced from:

ENV JAVA_PKG=server-jre-8u*-linux-x64.tar.gz

to:

ENV JAVA_PKG=jdk-8u*-linux-x64.tar.gz

After that, I downloaded the JDK from the Oracle OTN site, put it in the folder and ran the build.sh script. The script prepares an image with Oracle Linux 7 in minimal configuration with the Oracle JDK 8 installed.

[[email protected] java-8]$ ll
total 181204
-rwxr-xr-x. 1 oracle oinstall        47 May 11 16:42 build.sh
-rw-r--r--. 1 oracle oinstall       644 May 11 16:42 Dockerfile
-rw-r--r--. 1 oracle oinstall 185540433 May 12 10:43 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 oracle oinstall       263 May 11 16:42 server-jre-8u101-linux-x64.tar.gz.download
[[email protected] java-8]$ cp Dockerfile Dockerfile.orig
[[email protected]129-132 java-8]$ vi Dockerfile
[[email protected] java-8]$ ./build.sh 
Sending build context to Docker daemon 185.5 MB
Step 1 : FROM oraclelinux:7-slim
7-slim: Pulling from library/oraclelinux
...............
Successfully built 381e0684cea2
[[email protected] java-8]$ 
[[email protected] java-8]$ docker images
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
oracle/serverjre      8                   381e0684cea2        10 weeks ago        490.6 MB
oraclelinux           7-slim              442ebf722584        3 months ago        114.4 MB

The next step is optional, but it makes life a bit easier. We need a number of tools to be installed on top of our minimal installation. In Docker, you can create a container, modify it and save as another image using command “commit”. The beauty of this that you are not really doubling your space consumption by those incremental images. Docker will just add those extra changes you’ve made as a separate volume behind the scenes and will use original image plus your changes for the newly created image. You may think about it as a set of “snapshots”. So, I created a container with name “orajrevnc” from the “oracle/serverjre:8” image, installed additional packages including some diagnostic, configuration and other useful packages like a VNC server, vi editor, and others.

[[email protected] java-8]$ docker run --name serverjrevnc -p 5901:5901 -ti oracle/serverjre:8
bash-4.2# 
bash-4.2# yum -y install vim
bash-4.2# yum -y install net-tools
bash-4.2# yum -y install telnet
bash-4.2# yum -y install strace
bash-4.2# yum -y install gcc
bash-4.2# yum -y install xterm
.....

After that, I used the container to commit it as a new image and saved it as “oracle/serverjrevnc:8”.

[[email protected] java-8]$ docker commit serverjrevnc oracle/serverjrevnc:8
sha256:ac5b4d85fccc5427c92e65a6c3b1c06e3e8d04ffbe7725bcca1759a2165353d7
[[email protected] java-8]$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/serverjrevnc 8 ac5b4d85fccc 3 minutes ago 1.661 GB
oracle/serverjre 8 381e0684cea2 34 minutes ago 490.6 MB
oraclelinux 7-slim 442ebf722584 3 weeks ago 114.4 MB
[[email protected] java-8]$

The container “orajrevnc” can be deleted now :

[[email protected] java-8]$ docker stop orajrevnc
[[email protected] java-8]$ docker rm orajrevnc

Now we have a basic Oracle Linux image with the java installed and all necessary tools and utilities we need. The image can be used for the next step as a basis for our Oracle Database image. The database will serve as a repository for our ODI. We go to the folder “docker-images/OracleDatabase/dockerfiles/ ” where we replace a line “FROM oraclelinux:7-slim” by “FROM oracle/serverjrevnc:8”, download the 12.2.0.1 EE database software from “Oracle site” and build the image for Oracle Database 12.2.0.1.

[[email protected] java-8]$ cd ../../OracleDatabase/dockerfiles/
[[email protected] dockerfiles]$ ll
total 16
drwxr-xr-x. 2 oracle oinstall 4096 May 11 16:42 11.2.0.2
drwxr-xr-x. 2 oracle oinstall 4096 Jul 18 14:18 12.1.0.2
drwxr-xr-x. 2 oracle oinstall 4096 Jul 25 14:37 12.2.0.1
-rwxr-xr-x. 1 oracle oinstall 3975 May 11 16:42 buildDockerImage.sh
[[email protected] dockerfiles]$ vi 12.2.0.1/Dockerfile.ee
[[email protected] dockerfiles]$ ll 12.2.0.1/*.zip
total 3372836
-rw-r--r--. 1 oracle oinstall 3453696911 May 12 09:26 linuxx64_12201_database.zip

[[email protected] dockerfiles]$ ./buildDockerImage.sh -v 12.2.0.1 -e 
Checking if required packages are present and valid...
......

  Build completed in 906 seconds.
  
[[email protected] dockerfiles]$

Here is the new list of images we have after building the Oracle Database image.

[[email protected] dockerfiles]$ docker images
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
oracle/database       12.2.0.1-ee         91aaea30a651        28 minutes ago      16.18 GB
oracle/serverjrevnc   8                   2d7be7d163dc        50 minutes ago      1.543 GB
oracle/serverjre      8                   c2c247029798        About an hour ago   490.5 MB
oraclelinux           7-slim              08a01cc7be97        5 weeks ago         114.4 MB

Having all necessary images prepared we can start building our ODI. I have prepared the scripts to build the Docker image and published them on “https://github.com/gotochkin/docker-images”. You can either download or clone them using the git software. Let’s have a look at the scripts and discuss what they do:

[[email protected] ~]$ ls -l docker-images/ODI/dockerfiles/
total 8
drwxr-xr-x. 2 oracle oinstall 4096 Jul 27 14:18 12.2.1.2.6
-rwxr-xr-x. 1 oracle oinstall 3195 Jul 27 11:54 buildDockerImage.sh
[[email protected] ~]$ ls -l docker-images/ODI/dockerfiles/12.2.1.2.6/
total 2390372
-rw-r--r--. 1 oracle oinstall        277 May  9 14:42 Checksum.standalone
-rw-r--r--. 1 oracle oinstall       8572 Jul 17 12:43 createOdiDomainForStandaloneAgent.py
-rw-r--r--. 1 oracle oinstall       4270 Jul 26 10:41 Dockerfile.standalone
-rw-r--r--. 1 oracle oinstall       1289 May  9 14:49 install.file
-rw-r--r--. 1 oracle oinstall       6477 Jul 14 13:03 oracledi.gz
-rw-r--r--. 1 oracle oinstall         55 May  8 13:57 oraInst.loc
-rw-rw-r--. 1 oracle oinstall       2695 May 15 15:08 rcuResponseFile.properties
-rw-r--r--. 1 oracle oinstall       7920 Jul 27 14:18 README.md
-rwxr-xr-x. 1 oracle oinstall       5332 Jul 14 15:51 runOracle.sh
-rwxr-xr-x. 1 oracle oinstall       4406 Jul 27 11:08 startAgent.sh
[[email protected] ~]$ 

— buildDockerImage.sh – A script to build the ODI image. It takes parameter -v as the version for the ODI (so far only 12.2.1.2.6) and -t to tell that we are going to configure ODI Agent in standalone mode.
— Checksum.standalone – to verify checksum for the downloaded installation files which you will need to put into the docker-images/ODI/dockerfiles/12.2.1.2.6 directory
— createOdiDomainForStandaloneAgent.py – A python script to create a domain for an ODI standalone agent. The original script had been taken from the “Oracle tutorial” and slightly modified for our needs.
— Dockerfile.standalone – The instructions/script for Docker how to build the image for the ODI standalone agent.
— install.file – A response file for ODI silent installation.
— oracledi.gz – Gzipped files with ODI connection properties, it will be uncompressed into the $HOME/.odi directory.
— oraInst.loc – The configuration for Oracle Inventory.
— rcuResponseFile.properties – A parameter file for Repository Creation Utility.
— README.md – An instruction how to build an image.
— runOracle.sh – A startup script for the Oracle database which is going to be used as a repository.
— startAgent.sh – A script to configure and start the ODI agent.

We need to download ODI installation files fmw_12.2.1.2.6_odi_Disk1_1of2.zip and fmw_12.2.1.2.6_odi_Disk1_2of2.zip files from “Oracle OTN site” and put them to the docker-images/ODI/dockerfiles/12.2.1.2.6 folder.

At last, everything is ready and we can build the image using the buildDockerImage.sh script. Of course, you can build it without the script since it is only a wrapper for “docker build” command. The script just makes it bit easier.

[[email protected] dockerfiles]$ ./buildDockerImage.sh -v 12.2.1.2.6 -t

..............

Successfully built 239bdf178fbe

  ODI Docker Image for 'standalone' version 12.2.1.2.6 is ready to be extended: 
    
    --> oracle/odi:12.2.1.2.6-standalone

  Build completed in 1032 seconds.

We can see the built image on the list:

[[email protected] dockerfiles]$ docker images
REPOSITORY            TAG                     IMAGE ID            CREATED             SIZE
oracle/odi            12.2.1.2.6-standalone   239bdf178fbe        24 seconds ago      23.73 GB
oracle/database       12.2.0.1-ee             91aaea30a651        About an hour ago   16.18 GB
oracle/serverjrevnc   8                       2d7be7d163dc        About an hour ago   1.543 GB
oracle/serverjre      8                       c2c247029798        About an hour ago   490.5 MB
oraclelinux           7-slim                  08a01cc7be97        5 weeks ago         114.4 MB

We are ready to create our container with ODI. When we create the container it will do several steps which can be in general listed as:
— Create a container with Oracle Linux 7 with JDK and other supplemental packages.
— Create or start an Oracle database.
— Create an ODI repository, if it is not created already.
— Configure an ODI agent in the repository or adjust hostname for the agent if it has been already configured in the repository.
— Create a Weblogic domain for standalone ODI agent.
— Start the agent.

We have an option to create a fresh repository database every time when we deploy a new container using the command:

[[email protected] dockerfiles]$ docker run --name oditest -p 1521:1521 -p 5500:5500 -p 5901:5901 -p 5902:5902 --env ORACLE_BASE=/opt/oracle --env ORACLE_HOME=/opt/oracle/product/12.2.0.1/dbhome_1 oracle/odi:12.2.1.2.6-standalone

And the database will be created inside docker file system (docker FS). It is convenient when you want to have a fresh repository every time but it takes a time to create a new database and, as result, your deployment will be delayed.

Or we can define a volume for database files out of the docker file system in which case you may reuse database for your containers again and again which can save some time during deployment.

[[email protected] dockerfiles]$ docker run --name oditest -p 1521:1521 -p 5500:5500 -p 5901:5901 -p 5902:5902 -v /home/oracle/docker-images/OracleDatabase/oradata:/opt/oracle/oradata --env ORACLE_BASE=/opt/oracle --env ORACLE_HOME=/opt/oracle/product/12.2.0.1/dbhome_1 oracle/odi:12.2.1.2.6-standalone

Just be aware that if you want to use more than one container with the same Oracle database the scripts should be adapted. In the current implementation a new deployment will try to use the same repository.
After executing the command you will see the log of creation and in the end, you get the container with a running standalone ODI agent.

[[email protected] ~]$ docker ps
CONTAINER ID        IMAGE                              COMMAND                  CREATED             STATUS              PORTS                                                                                        NAMES
71969452f5b3        oracle/odi:12.2.1.2.6-standalone   "/u01/app/oracle/star"   4 days ago          Up 2 days           0.0.0.0:1521->1521/tcp, 0.0.0.0:5500->5500/tcp, 0.0.0.0:5901-5902->5901-5902/tcp, 5903/tcp   oditest
[[email protected] ~]$ 

Inside the container you can start a vnc server and run an ODI studio.

[[email protected] ~]$ docker exec -ti oditest bash
[[email protected] Middleware]$ vncserver

You will require a password to access your desktops.

Password:

After starting the ODI studio you get the usual questions like whether you want to import any settings or to allow send data to Oracle about ODI studio usage. You don’t have anything to import from the previous installation since this is the first one, so, you can ignore and say “no”. The studio is eventually up and you need to connect to the repository. When you push the button to connect to the repository, you will be asked if you want to store the credentials in a wallet.

You have an option to refuse and use the pre-created connection and saved credentials.

Ideally this will make the usage bit easier and convenient. Of course, for any kind of production development it is strongly recommended to use a wallet and a proper password.
If you’re someone making your first steps in Docker, I hope this article has been helpful. In my opinion, Docker can be extremely useful for test deployments. The persistent database files makes the deployment easy and quick. I have some reservations about using Docker for any production deployments of Oracle databases, but that discussion deserves a dedicated post. Stay tuned.