There are various libraries for doing parallel computing based on SMP techniques, including GPU usage and similar stuff. But its basis are a simple annotated function pair, where par and pseq can be viewed as pthread_create and pthread_cond_wait. As you know I’m trying to solve the Project Euler set of challenges using Haskell to practice and gain good experience with this language. So, you can review the code of the problem 104 and see how par and pseq are being used. On the code you can see two functions using parallelism, isValidDig and isValidPan.

-- | Checks if the given number x have valid last n digits
-- and last n digits
isValidDig :: Integer           -- ^ Number to Check.
              -> [Integer]      -- ^ Sequence to Check.
              -> Bool           -- ^ Is valid or not.
isValidDig x xs
  | length (digits 10 x) < length xs = False
  | otherwise = let ds = digits 10 x
                    l = length xs
                    s = sort xs
                    r = sort $ take l ds
                    t = sort $ take l $ reverse ds
                in r `par` t `pseq` (r == s && t == s)


-- | Checks sequentially if the given range covers the
-- problem of pandigital sequence of digits using the reqDigs
-- sequence.
isValidPan :: Integer           -- ^ Number to check.
              -> Bool           -- ^ True when is valid.
isValidPan x = s `par` r `pseq` isValidDig r s
               where r = fib x
                     s = [1..9]

Due to the nature of the small computations which are sent to another thread by par, the performance gained on the resulting execution — using the RTS option N — is not so good, but enough to reach 8% less time than the sequential one. If you have some experience using POSIX threads you will see that par annotation function have a very similar usage to pthread_create, sending the computation of the r and s expressions to another thread — where lazy evaluations using $ instead of $!, which prevents the immediate evaluation — and the pseq annotation function acts as pthread_cond_wait where both threads are meet on the final evaluation — like using pthread_join. So, you do not need to worry about synchronization, because that task is already done by the annotation functions par and pseq. You can use that pair of annotations many times as you need — and cores that can use your operating system.

The problem with POSIX threads is the fact that they are a complex set of calls, this seems to be much simpler and elegant. There are other interesting approaches, like the Par Monad. It is very similar to use standard thread calls, but with the simplicity and robustness of using Monads and Haskell, with its approach you do not need to think which expressions should be evaluated first, with explicit parallel functions like spawn and fork, you have more control of how and which calls will be parallelized — not so compiler driven and more expression driven. On further experiments I will try solve some problems using the ParFunk hackage, taking advantage of the GPU driven computations.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

Spam is the use of electronic messaging systems (including most broadcast media, digital delivery systems) to send unsolicited bulk messages indiscriminately. While the most widely recognized form of spam is e-mail spam, the term is applied to similar abuses in other media: instant messaging spam, Usenet newsgroup spam, Web search engine spam, spam in blogs, wiki spam, online classified ads spam, mobile phone messaging spam, Internet forum spam, junk fax transmissions, social networking spam, television advertising and file sharing network spam.

This experiment requires a fake electronic mail account, a well known internet domain — which is subject of constant web scrapping seeking for electronic mails — and the proper software development tools, something flexible like Linux, rather than those very closed platforms like Windows. If your domain is subject of spam: “Send the same message indiscriminately to (large numbers of recipients) on the Internet”, this experiment probably will solve your common problems with SPAM and will let you build the proper filters on your SPAM filter daemon or service.

Start creating the fake account. For example spam-booby@example.org. This email address will be scrapped by those internet bots that are seeking for email addresses. Create a fake email client, to review and fetch that folder, that will be seriously affected by those SPAM bots. On your well known domain site, for example www.example.org, put your fake email address that will be subject of web scrapping in a hidden HTML anchor element:

<a href='mailto:spam-booby@example.org'
   style='display:none;'
   title='just fall here fu**ing bot'>
  spam-booby@example.org
</a>
<a href='mailto:spam-booby@example.org'
   style='visibility:hidden;'
   title='just fall here fu**ing bot'>
  spam-booby@example.org
</a>

Those mail links will not be displayed on the web browser of your customers, so, you will be safe, but those bots are not smart enough to process the visibility: hidden or display: none CSS properties. If those bots are smart enough to process that CSS property, well, just use some JavaScript trick to hide them, for example with jQuery that should be like:

<a href='mailto:spam-booby@example.org'
   class='please-hide-me'
   title='just fall here fu**ing bot'>
  spam-booby@example.org
</a>

<script type='text/javascript'><--//
$(document).ready(function () {
    $('a.please-hide-me').hide();
});
//--></script>

That will ensure that those email links will be hidden for your customers using a real browser. Wait one week, and you will be collecting SPAM on the fake spam-booby@example.org email account. Also, I know that those spam bot constructors are so bad software developers that will not consider the RFC2606 referring to example domains, and will use the email addresses on the present article in their spamming tasks.

This is the first phase, on the second phase, I will try to process the collected data from those captured emails. I will begin this experiment with certain domain, and I hope that you will enjoy the results. For the second phase I’ve chosen Python as primary language, and some tools to download and fetch the email from the fake account, some Bayesian filter classifiers and certain key/value database to speed-up the mail processing functions. I’ve started collecting data from today :)

© Daniel Molina Wegener for coder . cl, 2011. | Permalink | One comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

python-mode

Emacs is configured through the Lisp dialect Emacs-Lisp. Core configuration directives are processed from the ~/.emacs file. To enable the python-mode, you just need to load the python-mode each time that your Emacs editor opens a Python file.

;;; auto load python-mode
(autoload 'python-mode "python-mode" "Mode for editing Python source files")

;;; enable python-mode on .py files
(setq auto-mode-alist
      (append '(("\.py" . python-mode)
                ) auto-mode-alist))

;; auto font lock mode
(defvar font-lock-auto-mode-list
  (list 'python-mode))

Also i known that each Emacs Mode, has some hooks, and python-mode isn’t an exception. To customize the python-mode you can use the python-mode-hook variable:

(add-hook 'python-mode-hook 'dmw-python-mode-hook)

Where, in this case, dmw-python-mode-hook is a symbol pointing to a function called dmw-python-mode-hook:

;; python mode hook
(defun dmw-python-mode-hook ()
  (load "py-mode-ext")
  (load "pyp")
  (require 'pycomplete)
  (setq py-indent-offset 4)
  (setq py-smart-indentation t)
  (setq py-continuation-offset 4)
  (setq indent-tabs-mode nil)
  (setq py-pychecker-command "~/bin/pylint_etc_wrapper.py")
  (setq py-pychecker-command-args (quote ("")))
  (autoload 'pymacs-load "pymacs" nil t)
  (autoload 'pymacs-eval "pymacs" nil t)
  (autoload 'pymacs-apply "pymacs")
  (autoload 'pymacs-call "pymacs")
  (setq interpreter-mode-alist(cons '("python" . python-mode)
                                    interpreter-mode-alist))
  (eldoc-mode 1)
  (define-key py-mode-map [f12] 'pyp)
  (define-key py-mode-map "C-cC-w" (lambda ()
                                       (interactive)
                                       (command-execute 'py-pychecker-run)))
  (define-key py-mode-map [C-S-iso-lefttab] 'py-complete)
  )

python-mode-hook

This python-mode-hook function or hook function, enables a series of features for you python-mode. First, it loads py-mode-ext and pyp, and also loads pycomplete. Those tools are part of the PAGE – Python Automatic GUI Generator package. py-mode-ext adds some nice functions and commands, like py-call-pdb, to begin debugging Python. pyp adds a command/function that prints the current expression, or prompts for an expression to print, with the complete code hierarchy as prefix:

def main():
    if len(sys.argv) != 3:
        print "No dialup numbern"
    txtmsg = sys.argv[2]
    number = sys.argv[1]
    number = number.strip()
    for n in [' ', '+', '(', ')', "s", "t", '-']:
        number = number.replace(n, "");
    number = '+' + number
    skype = Skype4Py.Skype()
    skype.Attach()
    ### this line was generated with the pyp
    ### command that has the line
    ### (define-key py-mode-map [f12] 'pyp) on the
    ### python-mode-hook
    print 'main: skype =', skype    # dmw   pyp
    message = skype.CreateSms(Skype4Py.smsMessageTypeOutgoing, number)
    message.Body = txtmsg
    message.Send()

Finally the pycomplete module has a very nice tool to enable Python auto-completion. For example the following code:

class ParentObject:
    parent1 = None
    parent2 = None
    parent3 = None
    def __init__(self, m1, m2, m3):
        self.parent1 = m1
        self.parent2 = m2
    def child(self, m1, m2, m3):
        self.parent3 = ChildObject(m1, m2, m3)
    def nested(self, m1, m2, m3):
        self.parent3.child4 = NestedChild(m1, m2, m3)
    def subnested(self, m1):
        self.parent3.child4.nested4 = SubNestedChild(m1)
        ### here I press ctrl + s + tab
        ### this will run py-complete as the line
        ### (define-key py-mode-map [C-S-iso-lefttab] 'py-complete)
        ### is defined on the python-mode-hook
        str<Ctrl+S+Tab>
    def __repr__(self):
        return "n-> " + repr(self.__dict__)

This will display a window as follows — where you just need to press mouse-2 or the middle button on your mouse to select one completion:

pychecker

Python static checkers are cool, also are tools that you must use in your daily Python programming tasks. For example, on the Emacs Wiki, you can find this article about using Emacs as Python IDE. There I’ve found the pylint_etc_wrapper.py script, which runs three Python static checkers at once: pylint, pychecker, pyflakes and pep8. Running the pychecker script using the ctrl + cctrl + w key sequence, you will obtain a window similar to the next one, will errors reported from those tools:

I hope that you will enjoy hacking Python from Emacs, it’s a great editor. Also you can use the Emacs Code Browser with Python, I hope that you can find it very interesting.

© Daniel Molina Wegener for coder . cl, 2010. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

On the Perl language, prototypes usually are absent, since are not required. you just create a function — anonymous or not — by writing subs:

sub hello_world {
    return "hello world";
}

The principle of perl subs, is that you call them with an unique argument and that argument is a list, so that’s why calling functions in perl do not require the form function(arg1, arg2, arg3). For example you can call the hello_world function with four different forms:

print hello_world;        # the mos common form
print &hello_world;       # code reference style
print hello_world();      # the c derived form without arguments
print &hello_world();     # code reference style form without arguments

Like many languages, Perl provides for user-defined subroutines. These may be located anywhere in the main program, loaded in from other files via the do, require, or use keywords, or generated on the fly using eval or anonymous subroutines. You can even call a function indirectly using a variable containing its name or a CODE reference.

The last two ones are just indicating that the argument list is empty. Have you tried to pass an array or list as argument?

sub hello_who {
    my ($who) = @_;
    print "hello ${who}n";
}
my @nekos = ('neko', 'core', 'test');
hello_who 'neko';
hello_who @nekos;
hello_who (@nekos);

The code above will just print "hello neko" three times, since the argument handling is just requesting the first argument, and it’s equivalent to call my $who = shift;. So, to pass an array or list as argument, you need to pass its reference — which is handled internally as scalar type in perl:

sub hello_who_iterator($) {
    my ($who) = @_;
    foreach my $i (@{$who}) {
        print "hello ${i}n";
    }
}
my @nekos = ('neko', 'core', 'test');
hello_who_iterator @nekos;
hello_who_iterator @nekos, "hola";

The prototyped function above will allow you to handle one argument as reference and ignore the remaining argument, if any. The same happens to hashes, so you can pass three references without problems:

sub hello_world_three($$$) {
    my ($a, $h, $s) = @_;
    my @hello_l;
    foreach my $i (@{$a}) {
        if ($h->{$i}) {
            push @hello_l, "Hello ".$s.' '.$i.' '.$h->{$i};
        }
    }
    return join("n", @hello_l)."n";
}
my %saludos = ('Juan' => 'el "Gato"',
               'Pedro' => 'el "Epidemia"',
               'Diego' => 'el "Conejo"',);
my @saludar_a = ('Juan', 'Pedro');
print hello_world_three(@saludar_a, %saludos, 'Mr.');

But what happens if I pass directly those three arguments, without using their references? The example above will print the following text:

Hello Mr. Juan el "Gato"
Hello Mr. Pedro el "Epidemia"

If we replace the code to catch directly those arguments, and not their references, we will not get the same result and the function will not do anything…

sub hello_world_three {
    my (@a, %h, $s) = @_;
    my @hello_l;
    foreach my $i (@a) {
        if ($h{$i}) {
            push @hello_l, "Hello ".$s.' '.$i.' '.$h{$i};
        }
    }
    return join("n", @hello_l)."n";
}
my %saludos = ('Juan' => 'el "Gato"',
               'Pedro' => 'el "Epidemia"',
               'Diego' => 'el "Conejo"',);
my @saludar_a = ('Juan', 'Pedro');
print hello_world_three(@saludar_a, %saludos, 'Mr.');

The example above applies the shift function to the first argument, so the function just receives the @saludar_a argument, so %h and $s arguments will be empty. If you want to pass an array or a hash to a function, you must do it by passing their reference.

Any arguments passed in show up in the array @_. Therefore, if you called a function with two arguments, those would be stored in $_[0] and $_[1]. The array @_ is a local array, but its elements are aliases for the actual scalar parameters. In particular, if an element $_[0] is updated, the corresponding argument is updated (or an error occurs if it is not updatable). If an argument is an array or hash element which did not exist when the function was called, that element is created only when (and if) it is modified or a reference to it is taken. (Some earlier versions of Perl created the element whether or not the element was assigned to.) Assigning to the whole array @_ removes that aliasing, and does not update any arguments.

references

• perlsub(1)
• perlref(1)

© Daniel Molina Wegener for coder . cl, 2010. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

daemon configuration

The daemon is quite easy to configure, you must setup the snmptrapd.conf(5snmp) file. For this configuration, you must read your desired configuration options, such as logging and execution. For example if we have a community called inetsnmp, we can configure the file to allow traps from that community and also configure a trap handler written in perl.

# configure inetsnmp community to allow logging,
# execution of handlers and network traffic.
authCommunity   log,execute,net         inetsnmp

# perl embeded handler.
perl do "/usr/share/snmp/handler/trapdembed.pl"

This configuration is not enough, the snmptrapd(8) daemon uses hosts.allow(5) facility, so we need to add the proper rule in that file:

snmptrapd : 192.168.100.0/255.255.255.0 : allow
snmptrapd : 10.10.10.0/255.255.255.0 : allow

the embeded perl handler

#!/usr/bin/perl -w
#

# strict perl is better
use strict;
use FileHandle;
use Data::Dumper;
use NetSNMP::TrapReceiver;

# we create a sample/demo log file...
my $log = FileHandle->new("/var/log/snmptrapsample.log", "a+") ||
    die "Failed to open log file";

# how we process an SNMP variable
sub process_var {
    my ($var) = @_;
    my %res;
    my $name = "$var->[0]";
    my ($vt, $vv) = split /: /, "$var->[1]";
    $res{'oid'} = $name;
    $res{'type'} = $vt;
    $res{'value'} = $vv;
    return %res;
}

# our default receiver
sub default_receiver {
    my ($pdu, $ivars) = @_;
    my %vars;
    $log->print(Dumper($pdu));
    foreach my $k (@{$_[1]}) {
        $vars{$k->[0]} = process_var($k);
    }
    $log->print(Dumper(%vars));
    $log->flush;
}

# every OIDs pass through the default_receiver
NetSNMP::TrapReceiver::register("all", &default_receiver) ||
    die "Failed to laod Sample Trap Receivern";

# status message...
print STDERR "Loaded Sample Trap Receivern";

This perl(1) handler will allow you basically to create a handler which is capable to process SNMP notifications creating two main variables on the handler itself $pdu which is the reference to the %pdu hash and %vars hash. Both hashes contains the proper data to process the request as you want:

%pdu = {
    'notificationtype' => 'TRAP',
    'receivedfrom' => 'UDP: [10.10.10.1]:53951->[192.168.100.5]',
    'version' => 1,
    'errorstatus' => 0,
    'messageid' => 0,
    'community' => 'public',
    'transactionid' => 1,
    'errorindex' => 0,
    'requestid' => 1012897136
};


%vars = {
    'IF-MIB::ifDescr' => {
        'value' => 'eth0',
        'type' => 'STRING',
        'oid' => 'IF-MIB::ifDescr'
    },
    'IF-MIB::ifAdminStatus.1' => {
        'value' => '1',
        'type' => 'INTEGER',
        'oid' => 'IF-MIB::ifAdminStatus.1'
    },
    'DISMAN-EVENT-MIB::sysUpTimeInstance' => {
        'value' => '(0) 0:00:00.00',
        'type' => 'Timeticks',
        'oid' => 'DISMAN-EVENT-MIB::sysUpTimeInstance'
    },
    'IF-MIB::ifIndex.1' => {
        'value' => '1',
        'type' => 'INTEGER',
        'oid' => 'IF-MIB::ifIndex.1'
    },
    'SNMPv2-MIB::snmpTrapOID.0' => {
        'value' => 'IF-MIB::linkUp',
        'type' => 'OID',
        'oid' => 'SNMPv2-MIB::snmpTrapOID.0'
    },
    'IF-MIB::ifOperStatus.1' => {
        'value' => '1',
        'type' => 'INTEGER',
        'oid' => 'IF-MIB::ifOperStatus.1'
    }
};

Where each OID or variable, can be treated by using the NetSNMP::OID package. Fora stdin handler, if you don’t know about perl(1), the difference is made on the snmptrapd.conf(5snmp) file, instead of configuring a global perl script which itself registers which OIDs will handle, you need to configure a global trap handler or each handler for each OID that you want to handle:

# configure inetsnmp community to allow logging,
# execution of handlers and network traffic.
authCommunity   log,execute,net         inetsnmp

traphandle     default           /usr/share/snmp/handler/defaultstding.py   default
traphandle     IF-MIB::linkUp    /usr/share/snmp/handler/ifuphandler.py     up

This will make your script or application to receive the OID data from stdin as follows:

router
UDP: [10.10.10.1]:37745->[192.168.100.5]
DISMAN-EVENT-MIB::sysUpTimeInstance 0:0:00:00.00
SNMPv2-MIB::snmpTrapOID.0 IF-MIB::linkUp
IF-MIB::ifIndex.1 1
IF-MIB::ifAdminStatus.1 up
IF-MIB::ifOperStatus.1 up
IF-MIB::ifDescr eth0

And also will require that you will enable in some manner the processing of that data as plain text.

Good luck configuring SNMP traps :)

© Daniel Molina Wegener for coder . cl, 2010. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

#!/usr/bin/env python
# skype_call.py
import sys
import Skype4Py

def main():
    if len(sys.argv) != 2:
        print "No dialup numbern"
    number = sys.argv[1]
    number = number.strip()
    for n in [' ', '+', '(', ')', "s", "t", '-']:
        number = number.replace(n, "");
    number = '+' + number
    print number, "n"
    skype = Skype4Py.Skype()
    skype.Attach()
    skype.PlaceCall(number)

if __name__ == '__main__':
    main()

Then, on the Kontact configuration, on the address book settings, we configure the path to our script, called skype_call.py, in this case located at my home directory, on my personal scripts folder.

One time configured contact, you can call your contacts just by clicking on their phone numbers. Remember that skype needs the format +country_code—area_code—phone_number, like +56 (2) 555 55 55.

Happy internet calling ;)

© Daniel Molina Wegener for coder . cl, 2010. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

My current fragment of my userContent.css, located at my profile directory for Firefox, usually ~/.mozilla -> firefox -> profile_name -> chrome, is as follows:

/* gmail fonts */
@-moz-document domain(mail.google.com)
{
    .gs .ii, textarea.dV {
        font-family: "Lucida Sans Typewriter" !important;
        font-size: 8pt !important;
    }

    .editable , .editable * {
        font-family: "Lucida Sans Typewriter" !important;
        font-size: 8pt !important;
    }

    .tr-field, .editable * {
        font-family: "Lucida Sans Typewriter" !important;
        font-size: 8pt !important;
    }

}

/* google groups fonts */
@-moz-document domain(groups.google.com)
{
    textarea, textarea * {
        font-family: "Lucida Sans Typewriter" !important;
        font-size: 8pt !important;
    }

    textarea.wdth100, textarea.wdth100 * {
        font-family: "Lucida Sans Typewriter" !important;
        font-size: 8pt !important;
    }

    textarea.padall5, textarea.padall5 * {
        font-family: "Lucida Sans Typewriter" !important;
        font-size: 8pt !important;
    }
}

Editing mail in fixed with fonts is great, mainly for code an similar tasks. The problem with variable with fonts is the fact that you can not setup the proper spacing or line with in chars while you are trying to preserve the Netiquete. Sorry, and you can call me old, but I still think that those rules must apply to electronic communications. Also, the Netiquete was standarized: "RFC1855 – Netiquette Guidelines".

© Daniel Molina Wegener for coder . cl, 2009. | Permalink | 3 comments | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

Phishing is a criminal activity. I’ve recently received an electronic mail with one of those phishing attempts. Surely I’ve ignored since I know how to read the electronic mail headers and some other useful information that comes in electronic mails. The Wikipedia refers to it as:

In the field of computer security, phishing is the criminally fraudulent process of attempting to acquire sensitive information such as usernames, passwords and credit card details by masquerading as a trustworthy entity in an electronic communication.

This time — I usually receive those electronic mails — I’ve received an important announcement from a Chilean bank, but really it was a phishing attempt. How do I know that? It appears to be quiet real, but what is behind all those pretty and trustful words?

The electronic have a plain text format. Yes! is not a binary format like Micro$oft Word or another kind of electronic format. This format has standard basis from the RFC 5322, and looks pretty similar on every message you sent and you receive. As the RFC 5322 standarizes the message format, the message format is as follows:

A message consists of header fields, optionally followed by a message body. Lines in a message MUST be a maximum of 998 characters excluding the CRLF, but it is RECOMMENDED that lines be limited to 78 characters excluding the CRLF. (See section 2.1.1 for explanation.) In a message body, though all of the characters listed in the text rule MAY be used, the use of US-ASCII control characters (values 1 through 8, 11, 12, and 14 through 31) is discouraged since their interpretation by receivers for display is not guaranteed.

Well, message headers are an important task while we are examining the message. You can view the original message by saving your email as plain text and looking at it with any text editor. Find a menu with View Source, View Message Source, View Raw Message, Save As (…plain text), and similar options to take a look on the message source. Well, this phishing mail has some interesting headers, such as the Received header, which indicates how the mail message was received by MX servers or mail servers:

Received: (qmail 12145 invoked by uid 1552); 14 Sep 2009 08:38:10 -0000
Received: from virtual1.webair.com (virtual1.webair.com [216.130.161.111])
    by mail07.ifxnetworks.com with SMTP id 9f9zhdib52zaavt9vaiekajna6;
    for dmw@unete.cl;
    Mon, 14 Sep 2009 08:38:10 +0000 (GMT)
    (envelope-from monitor@santander.cl)
Received-SPF: None; receiver=mail07.ifxnetworks.com;
    client-ip=216.130.161.111; envelope-from=<monitor@santander.cl>;
    helo=virtual1.webair.com
X-Avenger: version=0.7.9; receiver=mail07.ifxnetworks.com;
    client-ip=216.130.161.111; client-port=2914;
    syn-fingerprint=32768:58:1:60:M1380,N,W0,N,N,T FreeBSD 4.8-5.1 (or MacOS
    X); data-bytes=0

My electronic mail address which received the phishing mail, is hosted at IFX Networks, so the final receiver — from top to bottom — was the ****.ifxnetworks.com server. Electronic mail routes can be longer that this one, but you can read as final receiver from top to the initial receiver to bottom. WOW! What a surprise!, the initial client is 216.130.161.111, let see from where comes that IP address…

[www@quake ~]$ geoiplookup 216.130.161.111
GeoIP Country Edition: US, United States

[www@quake ~]$ nslookup 216.130.161.111
Server:         200.62.2.180
Address:        200.62.2.180#53

Non-authoritative answer:
111.161.130.216.in-addr.arpa    name = virtual1.webair.com.

WOW!, the client comes from USA, not a Chilean sender! and matches with Spam Filter receiver at IFX: helo=virtual1.webair.com. Why a Chilean bank wants to send an important advice from foreign servers? it smells like pure phishing. Now let me see what are indicating the NIC servers from that domain:

[www@quake ~]$ whois webair.com

Whois Server Version 2.0

Domain names in the .com and .net domains can now be registered
with many different competing registrars. Go to http://www.internic.net
for detailed information.

WEBAIR.COM.BR.GAROTAEXECUTIVO.COM
WEBAIR.COM

Hhhmmm… looks like a Brazilian domain ;). Looking at the real entry: GAROTAEXECUTIVO.COM.

Registrant:
   Executivo
   Rua: Executivo 3333
   Sao Paulo, Sao Paulo  01313000
   BR

   Registrar: DOTSTER
   Domain Name: GAROTAEXECUTIVO.COM
      Created on: 24-OCT-07
      Expires on: 24-OCT-09
      Last Updated on: 04-OCT-08

   Administrative, Technical Contact:
      Club, Executivo  postmaster@postmaster.com
      Executivo
      Rua: Executivo 3333
      Sao Paulo, Sao Paulo  01313000
      BR
      551122334455

Yeah! it’s a Brazilian one! Now, looking for all those URLs to make me fall in the fraudulent activity of phishing, where I’ve found three interesting URLs:

• hxxp://www.santander.cl/estilos/2008_08/bitmaps/santander.gif
  WOW!, it points to an image at the Chilean bank — I’ve replaced the t for x intentionally — and surely this will make some believe some people that the electronic mail is quiet real.
• hxxps://www.officebanking.cl/images/porque.gif
  WOW!, it points to an URL of the same bank but using the HTTPS protocol, this will make a possible client to believe that is pointing to the real bank, since many email clients asks for the site certificate. Interesting.
• hxxp://www.fpfa.esp.br/imagens/campeoes/st2.php
  WOW!, the full URL to the phishing site! I’ve not opened the URL, and I don’t know what is behind it :B

The piece of code with the link to the phishing web site in the body of the message:

<span class="style10">
                                            <a target="_blank"
href="http://www.fpfa.esp.br/imagens/campeoes/st2.php">
                        <img
src="https://www.officebanking.cl/images/ingresar.gif"
align="middle" border="0" width="65"
height="21"></a></span>

conclusions

• You can not trust in banking electronic mail until you strongly verify the electronic mail.
• You can not enter any site with your financial institution logos without verifying the URL bar.
• Google permits to download the message source.
• Use a good electronic mail client, for example Thunderbird has good advices on spam and phishing.
• Certainly the www.fpfa.esp.br site was cracked, and was used for phishing activities, and I’ve blocked it in my browser.
• Those virtual1.webair.com servers are widely opened to be used by spammers, and I’ve added them to my blacklist.

© Daniel Molina Wegener for coder . cl, 2009. | Permalink | One comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

This is not a security guide for Apache HTTP Server. Instead is a small guide that can be used as reference to protect some aspects of how the applications and pages are served. For security guides, you must look at other places. Well, I hope this approach would help a little in your administration tasks. All examples are not a copy/paste rules, you must think on them. I’ll never give you recipes… you always must think.

anonymous version

In Apache 2.X you can avoid of showing the server version and installed extensions, such as mod_php, mod_perl and others. This is an easy task, just add the both next directives — search for them and change their value.

#
# This hides the server signature on error pages.
# This means that the server version is not shown
# on error pages.
#
ServerSignature Off

#
# This will hide the server version and installed
# modules on the response headers.
#
ServerTokens Prod

limit requests

mod_rewrite is a good option to setup some filters on your server. If you enable this module, you can filter the request method and certain CGI enabled URLs.

mod_rewrite filters

If you serve dynamic web pages using some module as mod_perl or mod_php, you can filter only the used methods on your application, in example: HEAD, OPTIONS, GET and POST, and disable other methods, such as TRACE or WebDAV methods.

#
# this rule will block all request that differs from HEAD, OPTIONS,
# GET and POST
#
<IfModule mod_rewrite.c>
    RewriteCond     %{REQUEST_METHOD}       !^(HEAD|OPTIONS|GET|POST)$
    RewriteRule     (.*)                    [F,NS]
</IfModule>

You can build a better request block using directories, in example over upload directories.

#
# this rule will block every request method that differs from HEAD,
# OPTIONS and GET. also it will block any request in files that do
# not have the extensions jpg, png and gif. It also works with image
# directories ;)
#
<IfModule mod_rewrite.c>
<Directory "/var/www/site1/upload">
    RewriteCond     %{REQUEST_METHOD}       !^(HEAD|OPTIONS|GET)$       [OR]
    RewriteCond     %{REQUEST_FILENAME}     !^.*.(jpg|png|gif)$
    RewriteRule     (.*)                    [F,NS]
</Directory>
</IfModule>

You can play a lot with all those RewriteCond and RewriteRule. Also, you can block unwanted scripts, in example, if you are using PHP as your unique dynamic language to display your dynamic pages, you do not need to allow any Perl or Python scripts, you can block all other scripts that are unwanted on your site.

#
# this one will allow only PHP scripts, and also
# HTML pages and JavaScript
#
<IfModule mod_rewrite.c>
<Directory "/var/www/site1/">
    RewriteCond     %{REQUEST_METHOD}       !^(HEAD|OPTIONS|GET|POST)$       [OR]
    RewriteCond     %{REQUEST_FILENAME}     !^.*.(php|html|htm|js)$
    RewriteRule     (.*)                    [F,NS]
</Directory>
</IfModule>

And yet, there are a lot of useful rules that you can apply, just think on them to fit your needs. Well, also you can enable Apache to allow only preprocessed URLs, in example if you are using some kind of wrapper to deny direct access to all PHP scripts, why, because our application is a MVC based application.

#
# we deny access to all kind of scripts on our system
#
<IfModule mod_rewrite.c>
<Directory "/var/www/site1/">
    RewriteCond     %{REQUEST_METHOD}       !^(HEAD|OPTIONS|GET|POST)$          [OR]
    RewriteCond     %{SCRIPT_FILENAME}      !^.*.(jpg|png|gif|html|htm|js)$    [OR]
    RewriteCond     %{REQUEST_FILENAME}     ^.*.(py|pl|cgi|php|php3|shtml)$
    RewriteRule     (.*)                    [F,NS]
#
# we allow jpg, png, gif, js and html
#
    RewriteCond     %{SCRIPT_FILENAME}      !^.*.(jpg|png|gif|html|htm|js)$    [OR]
    RewriteRule     (.*)                    $1                                  [PT]

#
# and then we redirect all requests to the /index.php script ;)
# possibly the main controller on our application
#
    RewriteCond     %{REQUEST_METHOD}       !^(HEAD|OPTIONS|GET|POST)$
    RewriteRule     (.*)                    /index.php?$1                       [PT]
</Directory>
</IfModule>

You can play a lot with mod_rewrite rules and customize your application to allow users to have normal and browser based access to your application.

limit by location

Two helpful directives are Limit and LimitExcept. Those directives can allow you to limit the request method for certain Location.

#
# this will block other request method
# than HEAD GET POST OPTIONS, under Location
#
<Location />
    <Limit HEAD GET POST OPTIONS>
        Order allow,deny
        Allow from all
    </Limit>
    <LimitExcept HEAD GET POST OPTIONS>
        Order deny,allow
        Deny from all
    </LimitExcept>
</Location>

third party modules

Well here you have a list of modules that you can use. All of them will help you on protecting your application

• mod_security
• mod_evasive
• mod_clamav

disable unused modules

Make deep review on how many modules are you using. If you are not using certain module, just remove it from your configuration. In example, many sites have enabled the dav_module and dav_fs_module, when they really don’t need them. This will reduce all possible bugs that can be remotely attacked and will reduce the memory and processor usage ;)

on what can’t help?

Surely, there a lot of "responsibility of the user" issues on those those modules and rules can not help. In example, weak passwords, buggy applications, weak validations and wrong handled user privileges.

I’ve found an interesting paper on this topic, about the user authentication issue. It’s written by MIT researchers. Here is the abstract bellow.

Client authentication has been a continuous source of problems on the Web. Although many well studied techniques exist for authentication, Web sites continue to use extremely weak authentication schemes, especially in non enterprise environments such as store fronts. These weaknesses often result from careless use of authenticators within Web cookies. Of the twenty seven sites we investigated, we weakened the client authentication on two systems, gained unauthorized access on eight, and extracted the secret key used to mint authenticators from one.

We provide a description of the limitations, requirements, and security models specific to Web client authentication. This includes the introduction of the interrogative adversary, a surprisingly powerful adversary that can adaptively query a Web site.

We propose a set of hints for designing a secure client authentication scheme. Using these hints, we present the design and analysis of a simple authentication scheme secure against forgeries by the interrogative adversary. In conjunction with SSL, our scheme is secure against forgeries by the active adversary.

And here is the link to download it. I hope that all of those topics would help on your Web Site maintenance and development… And yet I know that I don’t cover some other issues, but that’s because I know that you can search for proper research and do security labs on your application ;)

© Daniel Molina Wegener for coder . cl, 2009. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

Working with Web Services is not an easy task. We must know about XML, WSDL and some other technologies and also the framework that we are using. In this article I will try to demonstrate how easy is the task of sending objects through Web Services. First of all I’m using SOAPpy as my Web Services framework and the second element in the recipe is pyxser my Python-Object to XML serializer and deserializer.

A single web service that should receive an XML string, now can convert that input into a Python Object. Let’s see the server code under SOAPpy framework ;)

#!/usr/bin/env python
#
#
import pyxser as ser
from testpkg.sample import *
import SOAPpy

def hello(request):
    ### so we take the input string as xml and we deserialize it
    req = ser.unserialize(obj = request, enc = "ascii")
    ### we create a response object
    reqo = TestAnotherObject()
    ### we assign equal .first_element properties
    reqo.first_element = req.first_element
    ### we set a custom .second_element property
    reqo.second_element = "client"
    ### we serialize the response object
    rep = ser.serialize(obj = reqo, enc = "ascii", depth = 0)
    ### we return back the response object as xml string
    return rep

### create the server
server = SOAPpy.Server.SOAPServer(("localhost", 8888))
### register the hello function
server.registerFunction(hello)

print "Starting server..."
### and let server run...
server.serve_forever()

The code above show a single server with one operation hello(). The pyxser serializer can not serialize/deserialize classes declared in the __main__ module. So we declare a class in an external module testpkg.sample with the code bellow.

__all__ = [
    'TestAnotherObject']

class TestAnotherObject:
    first_element = "123"
    second_element = "456"
    def __str__(self):
        return repr(self.__dict__)
    def __repr__(self):
        return repr(self.__dict__)

This module contains our serialization target data transfer object. Now, at the client side, we have a complete support to send and receive objects using pyxser ;).

#!/usr/bin/env python
#
#
#

import pyxser as ser
import sys, httplib
from testpkg.sample import *
import SOAPpy

HELLOWS_NS = "http://localhost/hellows/"

def GetHello():
    ### we create a request object
    reqo = TestAnotherObject()
    ### we set the object properties as we need
    reqo.first_element = "client"
    reqo.second_element = "server"
    ### we serialize the object using pyxser
    hstr = ser.serialize(obj = reqo, enc = "ascii", depth = 0)
    ### we create a SOAP proxy to our server
    server = SOAPpy.SOAPProxy("http://localhost:8888/")
    ### we call the server method and we get the xml string
    resp = server.hello(hstr)
    ### we convert the response xml into a python object and
    ### return it!
    return ser.unserialize(obj = resp, enc = "ascii")

if __name__ == "__main__":
    ### then we can manipulate the object easily
    newobj = GetHello()
    print newobj.first_element[0:2]

We can do more expertize work with pyxser. You can try using WSDL, and importing the complete pyxser 1.0 XML Schema as follows…

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<wsdl:definitions xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
        xmlns:tns="http://www.example.org/Hello/"
        xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
        xmlns:xsd="http://www.w3.org/2001/XMLSchema"
        xmlns:pyxs="http://projects.coder.cl/pyxser/model/" name="Hello"
        targetNamespace="http://www.example.org/Hello/">
        <wsdl:types>
                <xsd:schema targetNamespace="http://www.example.org/Hello/"
                        xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
                        xmlns:tns="http://www.example.org/Hello/"
                        xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
                        xmlns:xsd="http://www.w3.org/2001/XMLSchema"
                        xmlns:pyxs="http://projects.coder.cl/pyxser/model/">
                        <xsd:import
                                namespace="http://projects.coder.cl/pyxser/model/"
                                schemaLocation="pyxser-1.0.xsd" />
                        <xsd:element name="obj" />
                </xsd:schema>
        </wsdl:types>
        <wsdl:message name="HelloRequest">
                <wsdl:part element="tns:obj" name="request" />
        </wsdl:message>
        <wsdl:message name="HelloResponse">
                <wsdl:part element="tns:obj" name="response" />
        </wsdl:message>
        <wsdl:portType name="Hello">
                <wsdl:operation name="Hello">
                        <wsdl:input message="tns:HelloRequest" />
                        <wsdl:output message="tns:HelloResponse" />
                </wsdl:operation>
        </wsdl:portType>
        <wsdl:binding name="HelloSOAP" type="tns:Hello">
                <soap:binding style="document"
                        transport="http://schemas.xmlsoap.org/soap/http" />
                <wsdl:operation name="Hello">
                        <soap:operation
                                soapAction="http://www.example.org/Hello/Hello" />
                        <wsdl:input>
                                <soap:body use="literal" />
                        </wsdl:input>
                        <wsdl:output>
                                <soap:body use="literal" />
                        </wsdl:output>
                </wsdl:operation>
        </wsdl:binding>
        <wsdl:service name="Hello">
                <wsdl:port binding="tns:HelloSOAP" name="HelloSOAP">
                        <soap:address location="http://www.example.org/" />
                </wsdl:port>
        </wsdl:service>
</wsdl:definitions>

So we can use integrate our Python web services by calling with other platforms and let the other platforms know the pyxser serialization model to allow us to develop more complex Web Services…

Certainly the pyxser package comes with with both standard and C14N XML schemas, and both standard and C14N XML DTDs — or document type definitions — so you can use them in your Web Services development.

© Daniel Molina Wegener for coder . cl, 2009. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)