Thursday, December 9, 2010

Propel and Oracle - Installing from scratch on Ubuntu 10.04 / 10.10

Propel (http://www.propelorm.org/) is a great ORM package for PHP which is compatible with most popular database systems, including MySQL, PostgreSQL, SQLite, MSSQL, and Oracle.
Nevertheless putting everything up and running can be quite tricky: it depends on phing, php cli, oracle drivers, etc, which are not ready out of the box.

After many hours reading tutorials and source code, and many failed attempts, I was finally able to put it running, which I will explain below.

Rule #1: Don't reinvent the wheel!
There is a crucial element in this story: the database driver. Installing Oracle Call Interface (OCI) and insert all oci modules into php can be tricky. Therefore, if you are installing from scratch I'd HIGHLY recommend to use Zend Server CE, the successor of ZendCore. This is the procedure described in this guide.
If you're willing to do it by hand, I'd suggest you to visit the following resources:
https://help.ubuntu.com/community/PHPOracle


Step #1: Install ZendServer (apache+php+DB drivers)

Go to http://www.zend.com/en/products/server-ce/downloads and download the latest ZendServer CE. (it's free). You could also try setting up a deb source for apt-get, but in the case of Ubuntu 10.10 this didn't work.
Locate and extract the package:
tar -xzf ZendServer-CE-php-xxxx
Move in the newly created directory and begin installation:
cd ZendServerxxxx
sudo ./install.sh

Great! your Apache server should be up and running. Refer to the printed INSTALLATION SUMMARY message to know your Zend administration page address. By default: https://localhost:10082/ZendServer

In the Administration page go to Server Setup -> extensions and check whether oci8 and pdo_oci extensions are "ON". Otherwise your system is probably missing libaiol. In such case install it:
sudo apt-get install libaio1
Back to the administration page hit "Restart PHP".

Finally we need to add php binaries and libraries to the path. In order to do that, append the following lines to the end of your /etc/profile file (check the path according to your installation).

PATH=$PATH:/usr/local/zend/bin
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/zend/lib


and load the new settings:
source /etc/profile


Step 2: Installing and fixing propel

The easiest way to install propel is, without doubt, using php-pear
ZendServer will also provide this utility. If you are installing the system by hand make sure you have php-pear installed before proceeding.

In order to install the Propel packages, you must add new channels from where pear will fetch it and its dependencies:

pear channel-discover pear.propelorm.org
pear channel-discover pear.phing.info

Install propel
pear install -a propel/propel_generator
pear install -a propel/propel_runtime

After the installation is finished, there are two details which must not be forgotten.
In first place, add propel_generator library folder to php include path. The easiest way is to through the zend administration page, under Server Setup->Directives->Paths and directories. Add to the box the path of the libraries, separating with ":", typically:
:/usr/local/zend/share/pear/data/propel_generator/lib
Restart PHP

Further, the current version of phing doesnt support multiple classpaths, which makes Propel to raise the following error every-time it's executed:
Swallowing exception: Could not create task of type: propel-sql [wrapped: Could not instantiate class PropelSQLTask, even though a class was specified. (Make sure that the specified class file contains a class with the correct name.)]

To correct this, edit propel_gen config file:
vi /usr/local/zend/share/pear/data/propel_generator/build-propel.xml

Locate the block:

path id="propelclasses">
pathelement dir="${propel.home}/lib/"/
pathelement dir="${propel.project.dir}/"/ --
/path


and transform it into

path id="propelclasses"
pathelement dir="${propel.home}/lib/"/
!-- pathelement dir="${propel.project.dir}/"/ --
/path

(note the commented second pathelement entry)

After this you should be able to run propel_gen


Step 3: Connecting to the database

The last step is to define the connection to the database. In this case I'll show an example considering a connection to a remote server. Just be sure all php oci modules are loaded and working.

Each propel project has its configurations in a build.properties file, including the connection to the database.
In case of oracle drivers, one must refer to the system OCI drivers, although we must generate Oracle like queries.

Please provide the following details, replacing {} according to your case:

propel.project = {proj name}

# The Propel driver to use for generating SQL, etc.
propel.database = oracle

propel.database.driver = oci

#tnsnames entry
#propel.database.url = oci:dbname={user}@{DBNAME}/{SID}

#Full qualified conn string (no need for TNSNAMES)
propel.database.url = oci:dbname=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST={db_server})(PORT={db_port})))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME={SID})))

propel.database.creole.url = ${propel.database.url}
propel.database.user = {user}
propel.database.password = {passwd}


Thursday, October 28, 2010

Unix File-Systems: why EXT4?

Ext4 is here since the Linux 2.6.28 kernel, and henceforth some of the most popular Linux distributions have been adopting it. In some cases, e.g. Ubuntu, it's actually the default file system.
It it really worth to abandon the very well known EXT3 or ReiserFS, which have proven its value for several years now?

Just for minimal information, EXT4 is mostly like an update to EXT3, not really a revolution, which has several advantages over it and is backwards compatible. In the mainline of the improvements is the use of extents instead of block mapping like XFS or Apple HFS+ already do, which is designed to improve performance with the creation of larger files, and will reduce fragmentation on the hard disk. Furthermore, EXT4 features delayed allocation, persistent pre-allocation, and journal check summing.

For a clearer idea, here you find some benchmark results from Phoronix
Ok, one can see that the large file creation really benefits from extents with a significant 50% margin.

For sequential read the performance gain is also very significant, although XFS performs a little bit better. EXT3 and ReiserFS are outperformed by more than 55%

And if you had doubts between EXT4 and XFS, this last test shows how the filesystems perform against a server disk activity simulation. EXT4 performs 3x better than XFS and almost 100% better than the usual competitors.

So, if you are or ever been in doubt, I assume you have good reasons to not think again in the same issue. EXT4 is there and is actually faster for most disk usage patterns.

Tuesday, September 28, 2010

The device mess!

Phones, Smartphones, PDAs, MIDs, UMPCs, Tablets/Pads, Netbooks, Laptops, PCs... the real device mess has come and there is options for every taste, for every pocket and eye.

Notably the whole new terms for devices between Phones and PCs reach the maximum confusion, and there is no clear border in the definitions, so they're likely to overlap. So let's see...

Displays:
Featuring anything from from 4 to 12+ inches, they can be non-touch, or touch. Then the touch ones can be Capacitive, Resistive, Surface Acoustic Wave and a bunch of other strange technologies. From these the Resistive is pretty popular since it supports for multi-touch screens. Check Wikipedia

CPU
powered by a x86, ARM or imitation chip, you can also find of everything that will influence, of course, the performance of your device but also the battery life, the Operating System and all the applications to be run. Nevertheless here a new story begins, which we extend next.

Operating System
Let's present the traditional scenario first:
-Small device -> ARM processor -> proprietary OS / Chrome / Windows CE
-Larger/Work device -> x86 processor -> Windows / Linux
.
If you still believe it, then forget it. Here the things get really complicated!
The latest days I've seen of everything, with a thousand chinese companies creating of everything with every kind of processor, and of course every kind of O.S. running on it.

Some curiosities:
A cheap laptor with an ARM processor running Android.
.
It's not Intel, it's not AMD, neither VIA... It's Loongson! A x86 compatible CPU being developed in China!

And... 5 inch tablet (therefore Ultra Mobile PC - UMPC), which throttles a 1,1GHz Atom

In the end, can can be loaded with all kind of stuff... This is my personal favorite part!
Gadgets
Connectivity: USBs, RS232, WiFi 802.11 a/b/g/n/XXX, ITU GPRS/EDGE/3G/HSDPA/UMTS/HSPA+, Bluetooth, even WiMAX, IRDA, Ethernet...
Extras: GPS, 1D/2D/Barcode reader, RFID and whatever you can imagine to be integrated...

And of course, some resistive versions intended for the professional use, that simply would cost as many times as the amount of plastic around..

Tuesday, June 22, 2010

Resume your remote SSH sessions

How many of us already suffered from a network failure while working remotely over SSH? I'd bet most of us, and isn't that frustating that you just lose all the unsaved changes? It's gone and there's no way you can get back to your session.



Although this is the very common situation, fortunately there is a very simple way you can work over a persistent SSH session.
All you have to do is to create a session holder on the server, which you can do with an utility like "screen" [http://www.gnu.org/software/screen/]

Screnn works server side so it's completely independent on the client software you use for the remote connection.

The steps are just:
  1. Remote login (normally) to your server, from windows, linux, whatever client...
  2. run "screen" (make sure it's installed on the server)
You are now working on a server session. To test it, put something to run, e.g. "top" and close the window.
"Oh God, is my session dead?" - not really. Login again and type "screen -ls". You will hopefully see your sessions, like,

There are screens on:

2477.pts-0.server1 (Detached)
2522.pts-0.server1 (Detached)
2 Sockets in /var/run/screen/S-root.


To reconnect to one of them use:

screen -r 2477.pts-0.server1

There are several options to contol the server session,. Below is a short list of the most important ones:
  • Ctrl a c - Creates a new screen session so that you can use more than one screen session at once.
  • Ctrl a n - Switches to the next screen session (if you use more than one).
  • Ctrl a p - Switches to the previous screen session (if you use more than one).
  • Ctrl a d - Detaches a screen session (without killing the processes in it - they continue).

To close a screen session where all tasks are finished you can type Ctrl-D

Wednesday, March 24, 2010

VirtualBox Raw partition - Windows Host Linux Guest

Some virtualization systems have recently been able to handle real hard disk partitions. This is specially useful if you have a dual boot system (lets say Windows and Linux) but you frequently have to access a system while running the other. Then the solution is to use a virtual machine with your real OS installation.

In my case I'm accessing Linux from my Windows XP installation using VirtualBox. For the other way round you can easily find several how-to's around.


The most important and not obvious step is how to create the mapping virtual disk.

First you'll need to have the boot sector of your Linux Machine accessible. To create it use dd to extract the first 512 bytes from your bootable Linux partition. (In this case I have grub installed on it, NOT in the MBR).

After, please find the VirtualBox name of your partition.
  • VBoxManage.exe internalcommands listpartitions -rawdisk \\.\PhysicalDrive0
Switch PhysicalDrive0 to PhysicalDrive1, PhysicalDrive2... as needed.

You should see something like this:

Number Type StartCHS EndCHS Size (MiB) Start (Sect)
1    0x07  0   /32 /33  13  /163/19           100         2048
2    0x07  13  /163/20  1023/239/63         20480       206848
3    0x07  1023/239/63  1023/239/63         26410     42149888
4    0x83  1023/239/63  1023/239/63         10239     96238800


To create the VirtualBox disk issue the command:

  • VBoxManage.exe internalcommands createrawvmdk -filename e:\vm\ubunturaw.vmdk -rawdisk \\.\PhysicalDrive0 -register -partitions 4 -mbr c:\linux.bin

-filename is the VirtualDisk name (may include path)
-register tells VirtualBox to register the virtual disk in the Virtual Media Manager
-partitions specifies which partition is to be mapped. Make sure it's your linux partition. If you don't specify VirtualBox will try to map the whole Disk, which can corrupt your data, besides not accepting your mbr file.
-mbr Specifies the master boot record for your system (required)

Now just create a common virtual machine with the recently created disk!

Tuesday, March 2, 2010

NIC Bonding/Teaming

NIC binding/teaming is about creating a virtual network device that can make use of several real NICs to achieve higher throughput and/or fault-tolerance.

This means both will have the same MAC address. Depending on the protocol, you might not even have to change router configuration, and several Linux distributions have good support on that.

Here I provide links to some how-to’s on enabing NIC bonding for some of the most popular Linux distributions. Windows users don’t have the same luck, unless the NIC manufacturer had provided some utility for this specific purpose.

In all the cases an additional kernel module has to be loaded (bonding). Debian systems make use of the ifenslave utility.

RedHat Linux Enterprise 4
http://www.cyberciti.biz/tips/linux-bond-or-team-multiple-network-interfaces-nic-into-single-interface.html

Debian
http://www.5dollarwhitebox.org/wiki/index.php/Howtos_NIC_Bonding_Debian
http://www.howtoforge.com/nic-bonding-on-debian-lenny

Ubuntu
https://help.ubuntu.com/community/UbuntuBonding

Thursday, January 14, 2010

HP zt3000 / Compaq nx7000 and Mobility Radeon 9200

It seems like many people are having a lot of problems getting their 9200 video drivers to work on their zt3000 and nx7000 laptops.

Starting around Catalyst 5.6, something changed in the original ATI driver code that stopped working with the laptop's LCD screen.
The newer drivers actually work but the laptop LCD either has a strange color behavior, or it just shows nothing (like in my case).

Solution? Just use a modded Catalyst 5.6 or earlier.

This should be sufficient for most zt3000/nx7000 users... if they're running on XP. If you're trying to run Vista, you can still install them by downloaded the modded drivers install package then enabling Windows XP SP2 compatibility mode.

You can also try Omega Catalyst drivers.

Read more: HP zt3000 / Compaq nx7000 and Mobility 9200