Installing Oracle Database on ARM small single-board computer

In this article I will describe how to install an Oracle Database in a small single-board computer, making it the smallest Oracle DB server in the world! =]

How is it possible? What hardware did I use? What is the performance of it?

As there many questions and steps involved, I've split this article in the sections below:

1. Why TinkerBoard and not RaspiberryPi3 ?
2. Preparing TinkerBoard
3. Setting up qemu.
4. Installing Oracle Linux i686 over Debian ARM
5. Installing Oracle Database 11.2 on Oracle Linux i686
6. Some performance testing
7. Conclusion

So let's start!!

 1. Why TinkerBoard and not RaspiberryPi3 ?

First of all, for this appliance I didn't use a RaspiberryPi, but a TinkerBoard made by ASUS. Why?

Well, I have all current RaspiberryPi models (1,2,3) and recently Asus released TinkerBoard which is a big RbPi competitor. It is little bit more expensive ( US$ 35 vs US$ 58 in Amazon) but 30% faster accordingly some benchmarks. It is also a ARM based CPU and both have the exactly same shape, making it possible to use the same cases. I did some research in some other appliances (Banana Pi / Orange Pi / ODROID) and in my opinion TinkerBoard offers today (June-2018) the best benefit-cost ratio.

(Source: http://www.geeks3d.com/)

Some benchmarks:

• http://www.geeks3d.com/20180112/tested-asus-tinker-board-vs-raspberry-pi-3-cpu-and-gpu-benchmarks/
• https://mybroadband.co.za/news/hardware/195442-asus-tinker-board-vs-raspberry-pi-3-specifications-and-pricing.html
• https://www.jeffgeerling.com/blog/2018/asus-tinker-board-compelling-upgrade-raspberry-pi-3-b

The best advantages are the 2GB RAM and faster CPU. With the 1GB of RaspiberryPi 3 would be hard to run an Oracle Database. The 2.0 USB ports limited a lot the I/O throughput, but it's still hard to find some board with 3.0.

Most of those small-board computers use ARM due to costs. Oracle 11.2.0.4 binaries are provided only for the platforms below, and ARM is not one in the list:

You may be asking, how you are going to install a Oracle Database in an ARM based CPU if Oracle don't provide binaries for it? My solution was to emulate x86 over ARM using qemu:

Of course that adding all those layers between CPU and Oracle Database will impact performance, but that's what we have for today.

Note: there are some other small single-boards like 86Duino and SolidPC Q4 that have x86 based CPUs and would make life easier. However, I didn't have one to try, they are expensive and have also low RAM memory to run an Oracle DB.
 

2. Preparing TinkerBoard

First step is to put the peaces together. For this Database Appliance (I will call it miniODA), I used:

• TinkerBoard
• Black Case
• 16 GB Micro SD HC1
• External 500 GB SSD (But I'm using only 20GB of it)

• 2.5A Power Supply

Technically I could make everything fit in a 32GB Micro SD, however, to improve a bit the I/O performance, I preferred to store my database in the SSD drive. Using an external HDD instead of a SSD will give you probably the same performance is the I/O bottleneck is the 2.0 USB port. I will talk more about that in the performance section.

After putting everything together, I downloaded the official Debian OS image for TinkerBoard provided by ASUS, and burned the image on the SD using dd:

$ sudo dd if=~/Desktop/Tinker\ Board/20180222-tinker-board-linaro-stretch-alip-v2.0.5.img of=/dev/rdisk2 bs=1m

After plugin the SD and turning it on for the first time, I didn't need a monitor or a keyboard. I just plugged the ethernet cable, retrieved the IP acquired in my router page and connected via ssh using default linaro/linaro user and password.

My further steps were:

• Configure wireless connection in /etc/wpa_supplicant/wpa_supplicant.conf and disconnect ethernet cable.
• Add ssh key and disable password authentication /etc/ssh/sshd_config: "PasswordAuthentication no".
• Update all the packages via apt-get update / upgrade / autoremove.
• Configure and start vncserver.

And that's it. In 5 minutes my miniODA was ready for Database installation!
 

3. Setting up qemu

As I said in section 1, as oracle does not provide yet a Oracle DB installation for ARM based CPU, my solution was to emulate a minimal Oracle Linux with qemu.

To install qemu in my Debian OS, I needed some extra packages:

linaro@tinkerboard:~$ sudo su -
root@tinkerboard:~# apt-get install -y qemu virt-manager virt-viewer 
root@tinkerboard:~# apt-get install -y libvirt-daemon-system
root@tinkerboard:~# apt-get install -y ebtables

Basically I created a VM machine named linux-i686 with:

• i686 architecture.
• 1.5 GB RAM.
• 1 socket with 4 CPUs.
• 20 GB SDD created on my external SDD.
• NAT network connection.

Why I've choosed x86 over x64? Basically because the arm processor of TinkerBoard is 32 bits and I also want to reduce complexity as my VM RAM will only be 1.5 GB.

Below is the dumpxml of the qemu VM:

root@tinkerboard:~# virsh dumpxml linux-i686
<domain type='qemu' id='2'>
  <name>linux-i686</name>
  <uuid>f645cd60-33ae-4eaa-839c-c0fc810668aa</uuid>
  <memory unit='KiB'>1572864</memory>
  <currentMemory unit='KiB'>1572864</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='i686' machine='pc-i440fx-2.8'>hvm</type>
    <boot dev='cdrom'/>
    <boot dev='hd'/>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-i386</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/media/linaro/Samsung_T3/TinkerBoard/images/linux-i686.qcow2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <backingStore/>
      <target dev='hdb' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x2'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:13:4e:f1'/>
      <source network='default' bridge='virbr0'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-2-linux-i686/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='vga' vram='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <stats period='5'/>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='none' model='none'/>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+64055:+64055</label>
    <imagelabel>+64055:+64055</imagelabel>
  </seclabel>
</domain>

And the dumpxml for NAT network:

root@tinkerboard:~# virsh net-dumpxml default
<network connections='1'>
  <name>default</name>
  <uuid>c14d97fd-208b-4c23-9d07-8d8ba970d56c</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:23:43:ab'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
      <host mac='52:54:00:13:4e:f1' name='linux-i686' ip='192.168.122.10'/>
    </dhcp>
  </ip>
</network>

After creating my VM named linux-i686, I've also followed this article to redirect any connection on my Tinkerboard port 1521 to my VM nat ip.

$ iptables -I FORWARD -o virbr0 -d  192.168.122.10 -j ACCEPT
$ iptables -t nat -I PREROUTING -p tcp --dport 1521 -j DNAT --to 192.168.122.10:1521
$ iptables -I FORWARD -o virbr0 -d  192.168.1.140 -j ACCEPT
$ iptables -t nat -A POSTROUTING -s 192.168.122.0/24 -j MASQUERADE
$ iptables -A FORWARD -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT
$ iptables -A FORWARD -i virbr0 -o eth0 -j ACCEPT
$ iptables -A FORWARD -i virbr0 -o lo -j ACCEPT

4. Installing Oracle Linux i686 over Debian ARM

Now it's time to install Linux x86 in the newly created qemu VM. The OS I've chosen was OracleLinux 6.9-i686. Oracle does not provide Oracle Linux 7.x in 32 bits version, so this is the option was the way to go. I tried also to use Debian on x86 for performance, but Oracle Database is not supported on Debian and I had to do so many workarounds that database become extremely unstable. So Oracle Linux was the best option.

During the installation process, I've disabled LVM and swap partition, splitting the 20 GB disk in:

• /dev/vda1  /boot    480 MBs
• /dev/vda2  /      19660 MBs

The OS needed to be the lightest possible. So I did a minimal Oracle Linux installation (something around 300 rpms, no X Server) and added just the packages below after installation was completed:

[root@olinuxtb ~]# yum install -y yum-utils
[root@olinuxtb ~]# yum update -y
[root@olinuxtb ~]# yum install -y openssh-clients # For scp
[root@olinuxtb ~]# yum install -y wget
[root@olinuxtb ~]# yum install -y unzip
[root@olinuxtb ~]# yum install -y man
[root@olinuxtb ~]# yum install -y oracle-rdbms-server-11gR2-preinstall

After all RPMs installed, the OS was occupying around 2.5 GB of my 20 GB VM. I've also disabled some unnecessary services:

[root@olinuxtb ~]# chkconfig iptables off
[root@olinuxtb ~]# chkconfig ip6tables off
[root@olinuxtb ~]# chkconfig nfslock off
[root@olinuxtb ~]# chkconfig rpcbind off
[root@olinuxtb ~]# chkconfig rpcgssd off
[root@olinuxtb ~]# chkconfig rpcidmapd off
[root@olinuxtb ~]# chkconfig kdump off

5. Installing Oracle Database 11.2 on Oracle Linux i686

Next and final step was to install and create a new Oracle Database. I've chosen 11g as Oracle does not provide 12c in 32 bits format. I used /u01 for database installation files and /u02 for oracle datafiles.

After tranfering the 11.2.0.4 database installation zip files (1/7 and 2/7) and unzipping them, I called runInstaller in cli mode.

[root@olinuxtb ~]# mkdir /u01 /u02
[root@olinuxtb ~]# chown oracle: /u01/ /u02/
[oracle@olinuxtb ~]$ ./runInstaller -responseFile /home/oracle/database/response/db_install.rsp -showProgress -silent -ignoreSysPreReqs

When installation was completed, I also applied both DB and OJVM latest PSUs (April-2018):

• p27338049_112040_LINUX.zip
• p27475598_112040_LINUX.zip

Everything was ready for database creation. Now l had to create this database the smallest and lesser complex possible. So I did via "CREATE DATABASE" statement:

$ orapwd file=$ORACLE_HOME/dbs/orapworcl

$ cat <<EOF >> /etc/oratab
orcl:$ORACLE_HOME:Y
EOF

$ cat <<EOF > $ORACLE_HOME/dbs/initorcl.ora
db_name='orcl'
sga_target=500M
pga_aggregate_target=300M
processes=150
audit_file_dest='/u02/adump/'
audit_trail='none'
db_block_size=8192
db_domain=''
diagnostic_dest='/u02/'
dispatchers='(PROTOCOL=TCP)(SERVICE=orclXDB)'
open_cursors=300
remote_login_passwordfile='EXCLUSIVE'
undo_tablespace='UNDOTBS1'
control_files = '/u02/oradata/control1.ctl'
compatible ='11.2.0.4'
EOF

$ mkdir /u02/oradata/
$ mkdir /u02/adump/

$ sqlplus / as sysdba <<EOF
create spfile from pfile;
startup nomount;
EOF

$ sqlplus / as sysdba <<EOF
CREATE DATABASE orcl
USER SYS IDENTIFIED BY "Oracle.123"
USER SYSTEM IDENTIFIED BY "Oracle.123"
LOGFILE GROUP 1 ('/u02/oradata/redo01.log') SIZE 50M,
GROUP 2 ('/u02/oradata/redo02.log') SIZE 50M,
GROUP 3 ('/u02/oradata/redo03.log') SIZE 50M
MAXLOGFILES 5
MAXLOGMEMBERS 5
MAXLOGHISTORY 1
MAXDATAFILES 100
CHARACTER SET US7ASCII
NATIONAL CHARACTER SET AL16UTF16
EXTENT MANAGEMENT LOCAL
DATAFILE '/u02/oradata/system01.dbf' SIZE 10M AUTOEXTEND ON NEXT 1M MAXSIZE 1G
SYSAUX
DATAFILE '/u02/oradata/sysaux01.dbf' SIZE 10M AUTOEXTEND ON NEXT 1M MAXSIZE 1G
DEFAULT TABLESPACE USERS
DATAFILE '/u02/oradata/users01.dbf' SIZE 10M AUTOEXTEND ON NEXT 1M MAXSIZE 1G SEGMENT SPACE MANAGEMENT AUTO
DEFAULT TEMPORARY TABLESPACE TEMPTS1
TEMPFILE '/u02/oradata/temp01.dbf' SIZE 10M AUTOEXTEND ON NEXT 1M MAXSIZE 100M
UNDO TABLESPACE UNDOTBS1
DATAFILE '/u02/oradata/undotbs01.dbf' SIZE 10M AUTOEXTEND ON NEXT 1M MAXSIZE 100M;
EOF

$ sqlplus / as sysdba <<EOF
@?/rdbms/admin/catalog.sql
@?/rdbms/admin/catproc.sql
EOF

$ sqlplus system/Oracle.123 <<EOF
@?/sqlplus/admin/pupbld.sql
EOF

$ mkdir /u02/backup

$ rman target / <<EOF
shutdown immediate;
startup mount;
CONFIGURE COMPRESSION ALGORITHM 'HIGH' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD FALSE;
backup as compressed backupset database format '/u02/backup/%U';
alter database open;
EOF

After the database get created, all datafiles and redologs combined had less than 600 MBs in size while the backup had 40 MBs:

[oracle@olinuxtb ~]$ du -sh /u02/*
416K	/u02/adump
40M	/u02/backup
14M	/u02/diag
570M	/u02/oradata

I've also created some DEMO schemas (HR,OE,PM,IX,SH,BI) to play later with performance:

$ sqlplus / as sysdba <<EOF
CREATE TABLESPACE "EXAMPLE" DATAFILE
'/u02/oradata/example01.dbf' SIZE 10M
AUTOEXTEND ON NEXT 1M MAXSIZE 500M
LOGGING ONLINE PERMANENT BLOCKSIZE 8K
EXTENT MANAGEMENT LOCAL AUTOALLOCATE DEFAULT
NOCOMPRESS SEGMENT SPACE MANAGEMENT MANUAL;
EOF

$ wget https://github.com/oracle/db-sample-schemas/archive/master.zip
$ unzip master.zip
$ rm -f master.zip
$ cd db-sample-schemas-master
$ perl -p -i.bak -e 's#__SUB__CWD__#'$(pwd)'#g' *.sql */*.sql */*.dat

$ lsnrctl start

$ sqlplus system/Oracle.123 <<EOF
@mksample Oracle.123 Oracle.123 HR OE PM IX SH BI example tempts1 /home/oracle/db-sample-schemas-master localhost:1521/orcl
EOF

6. Some performance testing

Finally it was time to test the performance of my US$ 60 miniODA. I had to believe and have faith
that it could beat a US$ 1M Exadata appliance!

My first problem was the I/O of the USB 2.0 ports:

In DebianOS layer:

linaro@tinkerboard:~$ dd if=/dev/zero of=/media/linaro/Samsung_T3/testfile bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 33.7197 s, 31.8 MB/s

This ~ 30 MB/s is the limit for USB 2.0. So this can't be improved unless I change the storage model to emmc.

In Oracle Linux layer:

[oracle@olinuxtb ~]$ dd if=/dev/zero of=/tmp/testfile bs=100M count=1 oflag=direct
1+0 records in
1+0 records out
104857600 bytes (105 MB) copied, 14.898 s, 7.0 MB/s

So my I/O speed limit for Database was around 7 MB/s!

As I'm not a qemu expert, I tried to improve the disk performance by trial and error, playing with the cache and IO modes, but I couldn't get better results.

So let me give the timings I computed:

• Start Tinkerboard until Debian get fully ready ~ 30 seconds
• Start qemu Oracle Linux 6.9 ~ 10 minutes
• Start Oracle DB 11gR2 ~ 40 minutes
• Login on Database ~ 2 minutes
• Select 1 from dual ~ 5 seconds

Impressive...

Just an example of Oracle Startup:

linaro@tinkerboard:~$ ssh oracle@192.168.122.10
oracle@192.168.122.10's password: 
Last login: Mon Jun 18 12:12:04 2018
[oracle@olinuxtb ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Mon Jun 18 14:13:13 2018

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup;
ORACLE instance started.

Total System Global Area  523108352 bytes
Fixed Size                  1365740 bytes
Variable Size             213911828 bytes
Database Buffers          301989888 bytes
Redo Buffers                5840896 bytes
Database mounted.
Database opened.
SQL> select systimestamp from dual;

SYSTIMESTAMP
---------------------------------------------------------------------------
18-JUN-18 02.53.47.320532 PM -04:00

SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
[oracle@olinuxtb ~]$

The database login time was so badly slow that I had to add a "sqlnet.inbound_connect_timout=600" in sqlnet.ora to avoid timeouts.

Some queries, after first execution, got also better result times thanks to shared pool and buffer cache:

$ sqlplus system/Oracle.123@\"192.168.1.140/orcl\"

SQL*Plus: Release 12.1.0.2.0 Production on Mon Jun 18 17:01:08 2018

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> set timing on
SQL> select 1 from dual;

	 1
----------
	 1

Elapsed: 00:00:09.96
SQL> /

	 1
----------
	 1

Elapsed: 00:00:01.93
SQL> /

	 1
----------
	 1

Elapsed: 00:00:04.38
SQL> select count(*) from dba_objects;

  COUNT(*)
----------
     14023

Elapsed: 00:06:21.90
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:01:01.99
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:02:15.16
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:00:35.57
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:00:20.05
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:00:03.73
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:00:03.54
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:00:01.84
SQL> /

  COUNT(*)
----------
     14023

Elapsed: 00:00:04.79
SQL> select count(*) from dba_objects where object_type='TABLE';

  COUNT(*)
----------
      1219

Elapsed: 00:00:59.80
SQL> /

  COUNT(*)
----------
      1219

Elapsed: 00:00:05.31
SQL> exit

I've also downloaded swingbench but due to login timings I couldn't make it even connect.
 

7. Conclusion

It was funny to play with my US$ 60 miniODA. However, this is unusable as an Oracle Database Server. Maybe if I had an emmc storage + ARM database binaries (or a x86 small single-board), I could make it run with an acceptable performance, but not slow as hell.

Now it's time to take a backup and make a better use of this hardware. =]

+14