The data is lifted from its original type when it is stored as set. We can have a basic and plain example on how operates the map over the data lifting its original type. For example we have a plain text file with lines containing numbers separated by spaces and we want to extract the product for each line of those numbers. Since the data is plain text and it is not a numeric type, we need to lift those numbers and convert them to type that we can operate — like Integer types. So, map implies data extraction and data conversion, to operate that data with reduce. Following our plain text file example, we have a plain text file as follows.

4 3 1 4 5
2 1 3 2 4
3 4 4 2 1
2 1 3 4 1

Now, we can use map to make the proper data conversion to lift the input data from the plain text file and process each line obtaining lists of integers instead of string characters as follows.

module Main where

import System.Environment
import Data.String.Utils

liftLine :: String -> [Int]
liftLine l = fmap ( read . strip ) $ split " " l

main :: IO()
main = do [i] <- getArgs
          f <- readFile i
          let rows = fmap liftLine $ lines f
              in putStrLn $ show rows

In Haskell there are various map functions, here we using the most basic map function called fmap, which is implemented using Functor type class with instance types or anonymous types. So, the liftLine function extract a list of Integers from each line splitting the line using the space character, then applies strip to each tokenized string and reads its input converting it to Integer, and since it is using fmap, it will return a List of Integers. Where map is used again to lift each line, so, we are using fmap Functor as our lifting map function to extract the data as Integer from its plain text representation.

Now we need to process that lifted data using map, to obtain our required product for each integer list. So, we need to use the reduce function, which has two variants in Haskell, foldl and foldr, and for our convenience, we will use foldr, where reduce has a Monoidal behaviour due to its typed nature and can be expressed as .

module Main where

import System.Environment
import Data.String.Utils

liftLine :: String -> [Int]
liftLine l = fmap ( read . strip ) $ split " " l

reduceProduct :: [Int] -> Int
reduceProduct l = foldr (\ x y -> x * y ) 1 l

main :: IO()
main = do [i] <- getArgs
          f <- readFile i
          let rows = fmap liftLine $ lines f
              prds = fmap reduceProduct rows
              in putStrLn $ show prds

So, those are the very basic principles of MapReduce, which is used to calculate the PageRank on Google and get the site ranking on search results. So, this example is not distributed and it is a plain example of how it operates, just think on a distributed map and a distributed reduce operating with a very large amount of data to be processed. And there are various frameworks to work with MapReduce as model, because the original Google framework is not open source and you cannot work with it. As example of available frameworks supporting MapReduce as model you have Hadoop, Elastic MapReduce and Holumbus, so you can use this model with those frameworks reducing computation costs if you do not want to process your data with linear computations.

Copyright © 2012 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment
Post tags:

So, the language syntax was formalized using an EBNF grammar specification, we can understand it as follows.

<eol>            ::= "\n" | "\r" | "\r\n"
<white-space>    ::= " " | "\t"
<assign-sign>    ::= "="
<variable>       ::= ["a"-"z"]
<number>         ::= ["0"-"9"]+
<print-literal>  ::= "print"
<lock-literal>   ::= "lock"
<unlock-literal> ::= "unlock"
<end-literal>    ::= "end"
<assign-var>     ::= <variable> ( <white-space> )+ <assign-sign> ( <white-space> )+ <number> <eol>
<print-var>      ::= <print-literal> ( <white-space> )+ <variable> <eol>
<lock-inst>      ::= <lock-literal> <eol>
<unlock-inst>    ::= <unlock-literal> <eol>
<quantum-seqs>   ::= ( <number> ( <white-space> | <eol> ) )+
<program>        ::= ( <assign-var>
                     | <print-var>
                     | <lock-inst>
                     | <unlock-inst> )+
                     <end>
<program-files>  ::= <quantum-seqs> ( <program> )+

And it runs a parallel and concurrent context, using locks with lock and unlock primitives to gain exclusive execution of each program. So, I have used Haskell ADT to hold the instruction modifying the PInstruction structure as follows.

data PInstructionType = IsPLock
                        | IsPAssign
                        | IsPPrint
                        | IsPEnd
                          deriving (Show, Eq)

data PInstruct = PVAssign PVariableAssign
                 | PPLock PProgramLock
                 | PVPrint PPrintVar
                 | PEP PEndP
                   deriving (Show, Eq)

data PInstruction = PInstruction {
  inst           :: PInstruct
  , itype        :: PInstructionType
  , vend         :: Bool
  } deriving (Eq)

The instruction element becomes a Haskell ADT holding a particular instruction which holds which data should be processed using a single function that runs each step of the DSL program using output and a program context which is changed according each instruction. For that task and to differentiate between output and context related instruction, like variable assignment and locking instructions, I have used an Either construct, which allows me to return even a Haskell string to be printed on the screen or the modified program context itself.

data PProgramSet = PProgramSet {
  vars         :: Map.Map Char Int
  , lock       :: Bool
  , ilock      :: Int
  , idx        :: Int
  , pidx       :: Int
  , progs      :: [ PProgram ]
  }

runInstruction :: PProgramSet -> PInstruction -> Either String PProgramSet
runInstruction s i | itype i == IsPAssign = Right
                                            $ runAssign s
                                            $ uwAssignIns ( inst i )
                   | itype i == IsPPrint  = Left
                                            $ runPrint s
                                            $ uwPrintIns ( inst i )
                   | itype i == IsPEnd    = Left ".!"
                   | itype i == IsPLock   = Right
                                            $ runLock s
                                            $ uwLockIns ( inst i )
                   | otherwise = Left ".!"

Where the program context is held on a PProgramSet instance, using a Haskell Data.Map instance that holds the variable name as char and its value as integer, where the instruction ADT is unwrapped using the uw set of function, and each specific statement is handled using guards which are verifying the instruction type itype.

So, the instruction collection is processed using a recursive function which handle end statements and stores them in a PProgramSet, returning the collection, that collection is passed to the instruction runner which handles each instruction step as a context modifier or an output statement.

buildProgram :: PProgram
                -> [ PInstruction ] -> [ PInstruction ]
                -> [ PProgram ] -> [ PProgram ]
buildProgram n [] [] [] = buildProgram n (tasks n) [] []
buildProgram n [] [] ns = ns
buildProgram n [] ms ns = ns
buildProgram n (x:xs) ms ns = if itype x == IsPEnd
                                 then let pp = createPProgram (taskSet n) ms
                                          nns = (ns ++ [pp])
                                          in buildProgram n xs ms nns
                                 else let nms = (ms ++ [x])
                                          in buildProgram n xs nms ns

runPrograms :: PProgramSet -> IO ()
runPrograms s = if checkFinished s
                   then putStrLn ".!"
                   else let is = runStep s
                            ri = runInstruction s is
                            in case ri of
                                    Left  st -> putStrLn st
                                                >> runPrograms ( calculateIndex s )
                                    Right xs -> runPrograms ( calculateIndex xs )

main :: IO ()
main = do [f] <- getArgs
          c <- readFile f
          let prg = fromJust $ extractProgram $ parseProgram c
              grp = buildProgramSet $ buildProgram prg [] [] []
              in runPrograms grp

I have removed the IORef statements and replaced them by a context driven interpreter, just to make it fully functional instead of using state driven features of Haskell. So, the interpreter remains fully functional and I do not have used any state-full features, like StateMonad, IORef or MVar.

Copyright © 2012 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment
Post tags:

class Monad m where
      (>>=) :: m a -> (a -> m b) -> m b
      (>>) :: m a -> m b -> m b
      return :: a -> m a
      fail :: String -> m a

We can start saying that a Monad is bound to a type, where chained computations relates to the same type on each link. And jQuery is called first with a varying type argument called selector, which can be either a DOM object itself or a string holding a jQuery selector, throwing a DOM object collection as internal result, but it returns the jQuery instance as resulting object always. So, jQuery handles different types, disallowing the category boundaries.

Analysing its properties, a Monad has an endofunctor T : X → X covering its category. Where we have seen that jQuery holds different categories on its computations, and since jQuery provides different categories on different methods, the natural transformation μ : T × T → T cannot be meet, and it is also being broken by methods like remove() and detach(). Also, the natural transformation η : I → T, providing identity to Monad endofunctor is not supplied on any method of jQuery, where the jQuery instance is returned always as identity of itself, disallowing the category boundaries supplied by the selector and the monadic type covered by its definition.

So, with the cases above, we can say that the associativity law μ(μ(T × T) × T)) = μ(T × μ(T × T)) cannot be meet once the μ transformation is broken. And the left identity and the right identity which are specified by μ(η(T)) = T = μ(T(η)) are not provided by any method, since the jQuery instance returns itself always, instead of delivering the selector results. On certain cases, probably right identity is supplied on very specific cases, like the replaceWith() method, which is similar to its definition (>>=) :: m a → (a → m b) → m b or the return :: a -> m a operation, which binds both left and right identity without altering the results, no matter where it is placed.

With different types — or categories — you cannot cover a Monad, and with broken Monad laws and properties, you even cannot call Monad to jQuery. Also, the only one Monad operation supplied by jQuery is the >>= operator, where binds the right identity law, and once you call the >> operator related method the chain is broken, like calling each(), remove() and similar ones. So, expressing a chain in jQuery terms looking as follows.

    $('#my_div').
       addClass('visible-block').
       removeClass('invisible-block').
       live('click', __click);

The code above represents a chain of computations, that can be expressed with a monadic form as follows.

-- if jQuery is implemented in Haskell

    selectElement "#my_div"
        >>= addClass "visible-block"
        >>= removeClass "invisible-block"
        >>= liveEvent "click" __click

But what happens once you call the remove() method using jQuery. Seems that the monadic operation is broken by calling them, and the most important event is the fact the it cannot recover its original state disallowing the computation chain and recovering its original behaviour, and the fact that a Monad do not have a state associated with it, once you declare it — like using the State Monad. So, calling methods like remove() is similar to make use of the >> operation, but breaking the computation chain, without the proper recovery. At least a Monad can continue its operations, because it covers similar behaviours by design. In a simple example, once you delete a file using the IO Monad, you can continue reading files, once you remove a DOM object from its tree using jQuery, you cannot continue operating over the DOM tree, and you need to recall the jQuery instance, where a Monad should meet its laws, which can be expressed as follows.

return a >>= k = k a
m >>= return = m
xs >>= return . f = fmap f xs
m >>= (\x -> k x >>= h) = (m >>= k) >>= h

So, the laws above are not supplied by jQuery on all its operations, because the computation chain is broken in certain cases, disallowing a continous operation. For example with remove() you cannot implement return a >>= k = k a, and using remove it’s like using return a >> k b, breaking the chain and altering the functor results, despite the jQuery instance is returned always — as the proper Monad implementation requires — the selection remains empty after using remove() disallowing the further normal operations.

Copyright © 2011 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2011. | Permalink | No comment
Post tags:

The tokens are clearly defined and the Text.Parsec.Token module would be used, but instead I have defined the parser using the Parsec monadic primitives reading them as it would be implemented using BNF grammar parser, creating a simple data structure to hold the program instructions.

data PProgramLock = PLock
                  | PUnlock
                    deriving (Show, Eq)

data PVariableAssign = PVariableAssign {
  varName       :: Char
  , varValue    :: Int
  } deriving (Show, Eq)

data PPrintVar = PPrintVar {
  varn          :: Char
  } deriving (Show, Eq)

data PInstructionSet = PInstructionSet [ Int ]
                     deriving (Show, Eq)

data PInstructionType = IsPLock
                      | IsPAssign
                      | IsPPrint
                      | IsPEnd
                        deriving (Show, Eq)

data PInstruction = PInstruction {
  vassign        :: Maybe PVariableAssign
  , vlock        :: Maybe PProgramLock
  , vprint       :: Maybe PPrintVar
  , itype        :: PInstructionType
  , vend         :: Bool
  } deriving (Show, Eq)

data PProgram = PProgram {
  taskSet       :: PInstructionSet
  , tasks       :: [ PInstruction ]
  } deriving (Show, Eq)

Once I have defined the structure to hold the program, I have created the parser using the Monadic primitives from the Parsec package and its modules, mainly the GenParser Monad, allowing me to tokenize the program and store its structure in the data structures defined above. The core parser is defined as follows.

parseInstruction :: GenParser Char st PInstruction
parseInstruction = parseLock
                   <|> parseUnlock
                   <|> parseEnd
                   <|> parsePrint
                   <|> parseAssign

readInstructions :: GenParser Char st [PInstruction]
readInstructions = do instr <- many parseInstruction
                      return ( instr )

parseTasks :: GenParser Char st [PInstruction]
parseTasks = do instr <- readInstructions
                return instr

programParser :: GenParser Char st PProgram
programParser = do pTaskSet <- parseTaskSet
                   pTasks <- parseTasks
                   return PProgram {
                     taskSet = pTaskSet
                     , tasks = pTasks
                     }

parseProgram :: String -> Either ParseError PProgram
parseProgram inp = parse programParser "[Invalid]" inp

main :: IO()
main = do [f] <- getArgs
          c <- readFile f
          let prg = parseProgram c
              in putStrLn $ show prg

Each instruction line is handled by the parseInstruction monadic function, where it tries all different parsers, using literals first, and then more complex constructions, like the variable assignment and variable printing instructions. Those expressions are handled by simple combinatorial token parsers defined as follows.

parseLock :: GenParser Char st PInstruction
parseLock = do n <- string lockInstLiteral
               e <- readEol
               return $ createLockP PLock

parseUnlock :: GenParser Char st PInstruction
parseUnlock = do n <- string unlockInstLiteral
                 e <- readEol
                 return $ createLockP PUnlock

parseAssign :: GenParser Char st PInstruction
parseAssign = do dvar <- parseVar
                 dassign <- parseAssignment
                 dint <- parseInteger
                 deol <- readEol
                 return $ createAssignVarP dvar dint

parsePrint :: GenParser Char st PInstruction
parsePrint = do dprint <- string printLiteral
                dspace <- many $ char ' '
                dvar <- oneOf validLetterChars
                deol <- readEol
                return $ createPrintP dvar

So, if you follow the language, it is quite simple, probably it can be defined as follows.

<eol>            ::= "\n" | "\r" | "\r\n"
<white-space>    ::= " " | "\t"
<assign-sign>    ::= "="
<variable>       ::= ["a"-"z"]
<number>         ::= ["0"-"9"]+
<print-literal>  ::= "print"
<lock-literal>   ::= "lock"
<unlock-literal> ::= "unlock"
<end-literal>    ::= "end"
<assign-var>     ::= <variable> ( <white-space> )+ <assign-sign> ( <white-space> )+ <number> <eol>
<print-var>      ::= <print-literal> ( <white-space> )+ <variable> <eol>
<lock-inst>      ::= <lock-literal> <eol>
<unlock-inst>    ::= <unlock-literal> <eol>
<quantum-seqs>   ::= ( <number> ( <white-space> | <eol> ) )+
<program>        ::= ( <assign-var>
                     | <print-var>
                     | <lock-inst>
                     | <unlock-inst> )+
                     <end>
<program-files>  ::= <quantum-seqs> ( <program> )+

Which is the same implementation that I have provided to Haskell using the Parsec GenParser Monad.

Copyright © 2011 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2011. | Permalink | 5 comments
Post tags:

A source code example for this very basic reduction can be implemented in Haskell with the following code.

module Main where

addInt :: Int -> Int -> Int
addInt a b = a + b

fwrap :: (Int -> Int -> Int) -> Int -> Int -> Int
fwrap f a b = f a b

fwrapX :: (Int -> Int -> Int) -> Int -> Int -> Int
fwrapX f = f

main :: IO()
main = do print $ fwrap addInt 5 6
           print $ fwrapX addInt 5 6

Where fwrap and fwrapX are λ equivalent. The fwrap function has the form (λxyz.(x)yz)(λrs.t)mn and fwrapX has the form of the function application t. So, hlint in this case makes the proper suggestion about the obvious η-reduction as follows. I am using untyped lambda calculus expressions just to simplify the expressions, so you cannot get confused.

hlint.hs:9:1: Error: Eta reduce
Found:
  fwrap f a b = f a b
Why not:
  fwrap f = f

1 suggestion

Is very nice to work with a language capable of doing that kind of suggestions with its static analyzer. So, hlint is almost evaluating expressions in a manner that I have never seen in other static analyzers, making your code — probably — cleaner than others static analyzers. I hope that other static analyzers will include formal reduction suggestions in the future, so can have cleaner code and probably less redundant. Would be nice to have a similar static analyzer in Python, C, C++ and Java, I hope that some developer will include that kind checks in some of those languages. On C we have splint, which works great, but it doesn’t have support for formal reductions. At the other side, in Python we have pep8, pylint, pycheckers and pyflakes, but they doesn’t support formal reductions too. I think that certain imperative expression allows you to be analyze the code as functional code, so there should not be any problem applying this kind of theory.

There are many theoretical approaches that can be used while you are programming, you just need the proper knowledge to handle that theoretical background. Having only the object oriented approach will not make you a better programmer.

Copyright © 2011 Daniel Molina Wegener
Atribución-No Comercial-Sin Derivadas 2.0 Chile

© Daniel Molina Wegener for coder . cl, 2011. | Permalink | No comment
Post tags: