Boosting your Kernel on Gentoo

Everybody knows that Gentoo is a performance-architeture-oriented Linux. You can optimize your entire system changing the compiler (gcc) flags to attend the hardware specificities.

Example: I have a Sony VAIO with Intel Core 2 Duo processor, then I can use this flags on my make.conf (I unmasked sys-devel/gcc to use gcc-4.3.3 that have native support to core2 technology)

CFLAGS="-O3 -march=core2 -pipe -fomit-frame-pointer -mfpmath=sse -mmmx -msse -msse2 -msse3 -mssse3"

Then, all binaries in your system will be optimized…but and the kernel ???

Well, when you compile the kernel it uses itself compiler flags. And how to optimize it ?

Good news to you…If you have a Intel processor and use Gentoo Linux in your machine you are a lucky man. You must know the LinuxDNA project. The idea of the project is optimize the kernel to Intel architecture using the icc (Intel Compiler).

Currently, I’m working on Gentoo ebuild to linuxdna package. You’ll have more news about it soon…

Memory Debugging on (Open)Solaris

We know there are many commercial tools to debug memory on Solaris systems as Dynamic Suite and Purify. But I prefer to use open source tools always it is possible.

Well, Linux has many open source tools and the best in my opinion is Valgrind, but Valgrind will not be ported to sparc architecture as well to (open)solaris. Then, what tool I should use ?

(Open)Solaris has useful tools to debug memory, the most known is DTrace, but I’ll talk of two important and easiest tools to use: libumem and watchmalloc (equivalent to MALLOC_CHECK from linux).

Using libumem

It’s not need to recompile your application. The library is loaded dynamically by LD_PRELOAD environment variable.

• Prepare your environment

$ setenv UMEM_DEBUG default

$ setenv UMEM_LOGGING transaction

$ setenv LD_PRELOAD libumem.so.1

• Run the application normally

 $ ./myapp

• After the program has passed by all fluxes, generate a core to analyze with mdb

$ gcore `pgrep myapp`

• mdb means Solaris Modular Debugger

$ mdb myapp core.pid

• With mdb opened, just run a simple command (findleaks):

> ::findleaks -dvf
CACHE     LEAKED   BUFCTL CALLER
0014f12f       1 00067000 libumem.so.1`malloc+0x0
----------------------------------------------------------------------
 Total       1 buffer, 48 bytes

Using watchmalloc

Watchmalloc is the linux equivalent to MALLOC_CHECK. The use is most simple than libumem. See it:

• Prepare your environment

$ setenv MALLOC_DEBUG=WATCH,RW

$ setenv LD_PRELOAD watchmalloc.so.1

• Run the application normally

$ ./myapp

• When a leakage happens the program will generate a core dumped and will be possibly to analyze with gdb.

Brazilian Planet OpenSolaris

Born Planet OpenSolaris in PT_BR !!!

Planet OpenSolaris BR

Setting keyboard on XWindow

It’s a little tip showing how you can change your keyboard layout on X11 when you are not root and you cannot use the console (when you are able to use the console, you can use yourself X11 config file through the startx command).

Well, forget KDE, GNOME and modern X11 clients. Think you are using a simple client as Fluxbox and OpenBox…How do you can change your keyboard layout ?

It’s simple…use setxkbmap. In my case (I have a Sony Vaio with Spanish layout):

setxkbmap es

My two cents

Listing Shared Memory Segments in C

In this week I had to write a little code to remove all IPC Shared Memory segments that were created by one process.

I had already done it on Linux and *BSD systems, but never on (Open)Solaris. Then, I tried to search (googling) some method to do it but without successful.

Linux and *BSD’s have a syscall named sysctl() to interface it, but sunos kernel has not.

Well, after some minutes breaking my brain, I discover a syscall easiest to use than sysctl…look the code below:

#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
void show_shm (void)
{
  int i;
  int     *buf;
  uint_t  nids;
  uint_t  pnids;
  for (;;)
  {
    if (shmids(buf, nids, &pnids) != 0)
      exit(1);
    if (pnids <= nids)
      break;
    buf = realloc(buf, (nids = pnids) * sizeof (int));
  }
  for (i = 0; i < pnids; i++)
    printf ("%dn", buf[i]);
}

Calling C functions inside Python

In my current work, I’m developing new ways to test a Telecom framework. Firstly, I was building automation tests with cgreen to blackbox/unit test model. But the development stages were going a little more slow than I thought because write and debug C code is not so easy.

Suddenly, I though…”Hey, why do I not build these tests with python scripts ???”. And my response was: “Ctypes is your friend!”

Ctypes permits you call C functions inside python code. It’s wonderful. Let’s see how to do it in OpenSolaris environment:

First, download ctypes:

$ wget -c \
http://ufpr.dl.sourceforge.net/sourceforge/ctypes/ctypes-1.0.2.tar.gz

Install ctypes in a local path (as normal user):

$ tar -zxvf ctypes-1.0.2.tar.gz

$ cd ctypes-1.0.2

$ python setup.py install --prefix=$HOME/python

Export your PYTHONPATH environment variable:

$ export PYTHONPATH=$HOME/python/lib/python2.4/site-packages

Testing ctypes:

$ python

>>> import ctypes

OK ! Ctypes is there, but how do I use it ???

See a little example that I made to explain how to use ctypes:

• A file named test.c with a function (sum) that returns a sum of two values:

int sum (int a, int b)
{
    return (a+b);
}

• Compile it as a shared library:

$ gcc -fPIC -c test.c

$ gcc -shared -o test.o libtest.so

• Make sure that our library will be found

$ cp test.so $HOME/example

$ export LD_LIBRARY_PATH=$HOME/example

• Let’s call the function sum through a Python script

$ python

>>> from ctypes import *
>>> cdll.LoadLibrary('libtest.so')
<CDLL 'libtest.so', handle fe7806e0 at 80dfcec>
>>> libc = CDLL('libtest.so')
>>> libc.sum()
112710980
>>> libc.sum(1,2)
3

I can see a smile in your face. Yeah! Ctypes is a great trick to C programmers

Enjoy it!

Useful links:

1. http://docs.python.org/library/ctypes.html

Enabling a Marvel Yukon NIC on OpenSolaris

When I installed OpenSolaris 2008.05 in my laptop (Sony Vaio), the operating system does not recognized my ethernet card (marvel yukon), but identified my wireless NIC.

I don’t use ethernet card frequently because I have a Wireless Access Point. But, some days ago I had to enable my Marvel Yukon.

Let’s see how did I do this:

By default, snv_86 build does not come with the Marvel Yukon driver enabled in the kernel. But you can download it at:

http://homepage2.nifty.com/mrym3/taiyodo/myk-2.6.5.tar.gz

Done that, I take these steps:

1. tar -zxvf myk-2.6.5.tar.gz

2. cd myk-2.6.5

3. rm -Rf obj Makefile

4. ln -s Makefile.i386_gcc Makefile

5. ln -s i386 obj

6. export PATH=/usr/ccs/bin:$PATH

7. which gcc (/usr/ccs/bin/gcc expected)

8. which make (/usr/ccs/bin/make expected)

9. pfexec make install

10. pfexec ./adddrv.sh (should says ok!)

11. ifconfig myk0 plumb

12. ifconfig -a (wow ! I see my ethernet card!!!   )

PS: Before try my tips, please, read the installation document in tarball ’cause you can have to do different steps.

Links:

http://homepage2.nifty.com/mrym3/taiyodo/eng/

Jailing OpenBSD

Jail system is well known by FreeBSD users. But the super-hyper-ultra security OpenBSD has not a jail-like system ???

A few weeks ago, I and my friend were talking about security and how to expose a OpenBSD host in the wild Internet…One of his ideas was to use the ChrootDirectory option in OpenSSH…but I wanted a plus…

As I knew the FreeBSD and Jail system very well (I started with this stuff in 5 years past)…I tried to search something equal to OpenBSD…and then I found the incredible sysjail.

Well, let’s see how to install sysjail in OpenBSD 4.4:

• Download the tarball port:

wget http://sysjail.bsd.lv/dist/sysjail-obsd-1.2.34.tar.gz

• Decompress into ports directory:

tar -zxvf sysjail-obsd-1.2.34.tar.gz -C /usr/ports/security

• Try your best (just make)

(cd /usr/ports/security/sysjail && make)

• How you could see, it needs a patch, because currently version works only in OpenBSD 4.3 and earlier

patch.txt:

--- intercepts.h    Fri Jul 18 11:01:50 2008
+++ intercepts.h.new    Tue Dec 30 21:54:10 2008
@@ -1714,7 +1714,7 @@

 #if defined(__OpenBSD__)
 # if OpenBSD == 200711 || OpenBSD == 200705 || \
-         OpenBSD == 200805
+         OpenBSD == 200805 || OpenBSD == 200811
 #  define sj_inttbl    { { sj_obsd_4_0_native, \
                NULL, NULL, \
                sj_obsd_4_1_freebsd, \

cd w-sysjail-1.2.34/sysjail-1.2.34/
patch -p0 < patch.txt

• Try again and be happy

make && make install

Now, it’s time to action!!!

• Preparing the environment:

mkdir -p /usr/local/jail/cerberus

• Getting the binaries:

wget -c \
    ftp://ftp.openbsd.org/pub/OpenBSD/4.4/`uname -m`/\*.tgz

for i in *.tgz;
do
    tar -zxvf $i -C /usr/local/jail/cerberus/;
done

• Essential devices:

cd /usr/local/jail/cerberus/dev

sh ./MAKEDEV std random

• Preparing network (NIC alias)

ifconfig ne3 inet 172.16.0.2 netmask 255.255.255.0 alias

• Creating the jail

/usr/local/sbin/sysjail --quiet --allow-sysvipc \

    --cmd-enable /usr/local/jail/cerberus cerberus \
    172.16.0.2 /bin/sh /etc/rc

• Testing

  • jls

  • ssh 172.16.0.2

  • jail /usr/local/jail/cerberus cerberus 172.16.0.2 /bin/sh

PS.: take care using Sysjail + Systrace

History in sqlplus

I think the sqlplus command is the better app to Oracle DBA’s daily work. Since I do not working with complex pl/sql scripts, I prefer sqlplus too. However, sqlplus has not a history feature as psql (PostgreSQL) and mysql (MySQL) – when these tools are compiled with readline library.

It exits a trick to enable history in sqlplus. The “workaround” is known as rlwrap. So, how do I do it ?

<———–

rlwrap sqlplus

or

alias sqlplus='rlwrap sqlplus'

———–>

Try it also with rman command.

You will save your time and your patience.

rlwrap: http://utopia.knoware.nl/~hlub/uck/rlwrap/

sqlplus documentation: http://download-west.oracle.com/docs/cd/B12037_01/server.101/b12170/toc.htm

Decompiling Python Bytecodes

Oh no ! I lost my python source code !!! And now ?

You can use an online Python Bytecode Decompiler (DePython) made by Team509, a chinese group (I guess).

Recently, I received this tip/trick site by Python-Brasil mailing list.

Online Decompiler: http://www.depython.net/

Author(s): http://www.team509.com/