There are various kinds of techniques that can be used to create thread-safe algorithms and data structures. The are two main classes called blocking algorithms and non-blocking algorithms. On a blocking algorithm, locks are used to encapsulate fragments of code where you must not do concurrent writes to avoid racing conditions, where that fragment is called critical section. On non-blocking algorithms, you do not use locks, instead you use atomic operations to switch between the used resources, for example using a technique called compare and swap or CAS.

A racing condition, is an event that occurs when two or more threads are accessing the same resource at the same time, leading to data corruption or system faults. The most classic lock techniques in Java and C, are the synchronized keyword on Java and pthread_mutex_lock() call in C. On Java we protect our critical section using synchronized as follows.

class SingletonSampleImpl extends Object {  

    private static Object singletonMutex = new Object();
    private static SingletonSampleImpl instance = null;  

    public static SingletonSampleImpl getInstance() {
        synchronized (singletonMutex) {
            /* when we instantiate the singleton
               we have a critical section to protect
               from racing conditions */
            if (instance == null) {
                instance = new SingletonSampleImpl();
            }
        }
        return instance;
    }
}

The singletonMutex object acts as mutex, where a mutex — term that comes from mutual exclusion — has a flag that indicates to the other threads if it is blocked or not to jump into the critical section and execute the code. So, a mutex or semaphore is a mere flag that indicates to the program if the resource is busy or not. Using locks or blocking-algorithms is expensive on most systems, depending on how are they used. For example is very expensive to use a blocking algorithm to protect a socket, where it must wait its chance to be written by other threads, with inherent network time wait.

On the non-blocking side, we have three types of algorithms: wait-free, lock-free and obstruction-free. Where wait free is the strongest technique, but the hardest to handle, because once you implement it you have an algorithm that do not waits for writing or reading. Lock-free usually uses CAS, so it using a compare and swap operation to ensure that the critical section is protected, without locking. Obstruction-free uses locking, but a different one called live-locking.

The atomic operations used on the non-blocking techniques usually are operating on a very low level of the processor, because they must executed without the interference of any other threads. For example to work with atomic operations using the GCC compiler we have some special extensions to work with them.

Copyright © 2011 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2011. | Permalink | No comment
Post tags:

The sophistication of the analysis performed by tools varies from those that only consider the behaviour of individual statements and declarations, to those that include the complete source code of a program in their analysis. Uses of the information obtained from the analysis vary from highlighting possible coding errors (e.g., the lint tool) to formal methods that mathematically prove properties about a given program (e.g., its behaviour matches that of its specification) [Static program analysis, Wikipedia].

I’ve created a very simple Python script that wraps the execution of various of those tools to analyze C source code, grouping the execution of splint static analyzer, TenDRA static checker and cppcheck static checker. Those tools will allow you to verify your C source code using static analysis, and will allow you to find bugs in your C source code finding patterns of well known programming errors. Along the time, with practice, you will not write the same errors twice, and if you use those tools, you will enhance your programming habits by doing better code, with the assistance of those tools, for example using uninitiated variables or unchecked variables will not be a problem for you.

The tool cppcheck do not requires configuration, you just need to run it using a small set of flags, usually cppcheck --enable style --enable information --enable unusedFunction, so you do not require to create a configuration file and those tests made by cppcheck are very lightweight. The TenDRA compile just checks for standard conformance, and common flags in TenDRA are like tchk -Xs -Yxpg4 -Yposix. Among those three tools, the most complex one is splint, since it is a guided static analyzer, which uses comments to guide the static analyzer on program analysis. It requires a configuration file, that is called .splintrc and should be placed in your $HOME directory. My .splintrc configuration file is as follows:

##
## .splintrc example file
##

+strict
+matchanyintegral
+trytorecover
+unixlib
-sysdirerrors
-syntax

-bugslimit 50
-sysdirs /usr:/usr/include:/usr/local/include:/usr/include/bits
-DHAVE_CONFIG=1
-DHAVE_CONFIG_H=1
-DNDEBUG=1
-I/usr/include
-I/usr/local/include
-I/opt/include
-I./include
-I../include
-I.
-I..

That is a very basic configuration, but is almost complete to allow you to check your C source code. You will find splint reporting errors that your compiler will not display, for example if you are using GCC and your compiler flags are strict, like -Wall -Wextra -Wshadow -pedantic -std=c99 or -Wall -Wextra -Wshadow -pedantic -std=c90, will not report the same errors than splint does. You will find splint as a very strict tool to check your source code, displaying well known bugs, but not all reports should be handled as real, you must think on those errors that really have a real significance.

You can find the bugbuster.py script that I’ve wrote to use all those tools in one command from Emacs, by running the Emacs compile command on its web page here, and the source code repository to contribute or download the script is located on github.com here. Enjoy enhancing your source code.

Copyright © 2011 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2011. | Permalink | No comment
Post tags:

Some notes on performance. Windows has a very poor performance compared to Linux and FreeBSD platforms. Please check the following plot that compares the performance on Windows, Linux and FreeBSD:

Compare pyxser-1.5.2 on Windows, Linux and FreeBSD

The performance on Windows is very poor. The effort required to migrate the source from POSIX platforms to Windows platforms is minimum. The difference the major effort was made on porting the setup script. My code is almost standard and it requires a C99 compiler, and usually I use the flags -Wall -Wextra -Wshadow -pedantic -std=c99 to compile my sources, so I’m using very strict dialect. Seems that the Windows C compiler is C90 by default. No way to deal with the Windows compiler for some C keywords. I’m expecting to bring constant support for Win32 platform on future releases. I will be waiting your feedback :)

Some notes on the FreeBSD install, is the fact that it is a standard distribution, without kernel optimization and it is using distribution binaries — that means that everything is made for 80×386 machines — and also it does not have any compiler optimization. Also, is well known that FreeBSD default binaries are slow, mainly because it lacks kernel optimization and standard C library optimizations. The libxml2 port on FreeBSD seems to be the problem, since it is not using the jemalloc library, and it is using the internal libxml2 memory allocator.

Copyright © 2011 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2011. | Permalink | One comment
Post tags:

The ChangeLog for this release is as follows:

1.5.1r (2010.10.11):

        Daniel Molina Wegener <dmw@coder.cl>

        * On all files: algorithms were optimized, the code
        was flattened applying "The Zen of Python" and the
        performance was enhanced in 10%.

        * Was added the cinit argument to deserialization
        functions, which control whether or not, the default
        constructor is called, instead of creating a raw
        instance of deserialized objects.

        Thanks to pyxser users for their feedback.

The cinit = False argument makes pyxser to skip the default constructor, calling PyInstance_NewRaw() instead of the default constructor. This skips the initialization code on the __init__() method. The impact on the performance — if your objects do not require of the default constructor — is as follows:

This enhancement is minimal, but saves time when you work with very simple objects. To take a look on the documentation, the pyxser extension is self documented, so you just need to call pydoc -p8181, connect your browser to http://localhost:8181/ and search for the pyxser extension documentation.

I hope that you will enjoy this release :D

Copyright © 2010 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2010. | Permalink | No comment
Post tags:

This release is quite special, I’ve added lazy initialization of some resources, so it runs 15% faster, and I’ve removed all memory leaks, so it just keeps using about 65MB of RAM when is doing the 1,500,000.00 serialization and deserialization test. I’ve spent more time coding pyxser for this release. Just comparing the performance between the older release 1.4.6r — a buggy release — with the current release 1.5r we can see the performance enhancement as follows:

The memory usage was reduced, on what object creation refers, but not on certain tasks — remember that now I’m using lazy initialization for some resources — and we can see a very similar to previous releases plot on how the memory is handled:

The memoization has suffered some important changes. Now it uses PyObject_Hash() as object identification method. It’s qute slower than past revisions, but still it works faster using lazy initializations. Another important change made to the serialization algorithm, is the fact that now it is skipping callable objects, so function will not be serialized on any manner.

On the distribution now you will find on the test folder directory, some interesting tests, such as utf-16 encoding test and ascii encoding test. All serializations now are using the Python embeded Unicode codecs for deserialization, that is not the case for serialization, it is using the LibXML2 codec, so it can handle more encodings.

I must strongly thank to pyxser users for their feedback.

Copyright © 2010 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2010. | Permalink | No comment
Post tags: