coder . cl » development

proview i/o layers

Daniel Molina Wegener — Sat, 03 Nov 2012 12:37:41 +0000

ProView provides a nice I/O API. All devices are controlled through a pair of API calls for reading and writing data from signals. You cannot use channels directly, so every channel should be bound to signals for I/O. All write requests are made using io_write and read requests are made using io_read, where both methods have their context in the ProView application. All devices that should be controlled through any ProView application should use the Process flag set to User2 with value 8. The problem is how to deal with I/O calls centralized in two API calls rather than using device specific I/O calls. Your code should be consistent enough to support that I/O method.

We are currently using Hierarchical State Machines, and once a state requires channel usage, the state machine stops its executions, runs the I/O calls io_read and io_write and continues its execution on the next scan cycle. Device synchronization is using single boolean variables holding read/write states as semaphores and reviewed on each task request. The entire framework is designed as it is running a PLC program, so you have a scan → read → write cycle. All state machines are pausing its execution on channel requests because they must not use the communication channel concurrently and we cannot use a locking mechanism.

We have logical composite modules to hold the device structure, where a device uses the same physical levels as rack, card and channel, but with its logical level as rack, card and signal. This allows a better organization and binding of devices along the project structure. Modularity is important, so we are using device classes to hold each device structure. Channels remain untouched, and we are writing and reading data exclusively from signals, despite we know some calls to access channels directly.

The device organization is pretty clear. Since we are using ModBus, we organize signal groups according to register groups in the device for each register block that we want to read or write. Even more for coil registers, where the are almost organized in groups, thanks to the class BaseComponent, which allows that kind of abstraction, rather than using ModBus modules and cards directly.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

y combinator in coffeescript

Daniel Molina Wegener — Thu, 01 Nov 2012 13:46:28 +0000

As you know the Fixed Point Combinator, or Y-Combinator, allows to implement recursive functions without using recursive calls, and maybe you are asking yourself what kind of magic is this. The Fixed Point Combinator can be easily implemented on any language with functional features with a consistent anonymous function implementation. The concept comes from Lambda Calculus, a formal system created by the mathematician Alonzo Church, to study functions, and also it allows the representation of computations. The Fixed Point Combinator, as any combinator used to compose functions, creates a composite function that is called recursively without doing recursive calls using an anonymous function to create the recursive call instance.

The Y-Combinator is expressed in Lambda Calculus as λg.(λx.g(x x))(λx.g(x x)), and once you apply that combinator on a function, compositing a new function, you have a pseudo-recursive construct that creates a new function using anonymous functions to make the recursive call, rather than doing the recursive call with the function name. For example if we apply the Y-Combinator to the identity combinator λx.x, we have a sequential reduction as follows.

(λg . (λx . g (x x)) (λx . g (x x))) (λm.m)
(λx . (λm.m) (x x)) (λx . (λm.m) (x x))
(λm.m) ((λx . (λm.m) (x x)) (λx . (λm.m) (x x)))
(λm.m) ( ((λg . (λx . g (x x)) (λx . g (x x)))) (λm.m)) { equal to (λm.m) (Y (λm.m)) }

The same approach can be implemented in any programming language that implements anonymous functions or lambda expressions properly, like JavaScript or Coffee Script — which compiles to JavaScript. So you can derive that combinator in that language, as you can do it in Lisp and Haskell, because JavaScript implements that functional feature without obstacles. In other languages like C Plus Plus 11x you cannot implement the Y-Combinator due to its anonymous function implementation, because it seems that does not allow anonymous function nesting and treating anonymous functions as first order citizens.


Y = (h) -> ((f) ->
  f f) (f) ->
    h (n) ->
      f(f) n

fact = Y((g) ->
  (n) ->
    return 1 if n < 2
    n * g(n - 1)
)

console.log (fact 5)

As exercise, trace the function calls and anonymous function declarations and compare it to the Lambda Calculus expression and trace. You will find the recursion on the nested anonymous functions rather than finding the recursive call in a language literal like a function name. Despite its theoretical background, this combinator can be used with other libraries, like Katy, to traverse node trees and apply transformations, and your code will probably look a little bit monadic, but still will be based in combinators.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

filtering companies

Daniel Molina Wegener — Tue, 30 Oct 2012 11:27:35 +0000

As you know, I use a well defined set of questions to filter where should I work. If the company does not pass the test, I usually ignore that job offering. For example if I ask “what is your current distributed version control system?” and their answer is “SVN”, they got banned immediately — you know what I mean. The set of questions is pretty simple, and since you are usually interviewed by someone who is hiring or filtering the candidacies, you can do the same asking questions. For example, I am not interested in the company name; it can be Microsoft, but if they tell me “we are using visual source safe”, I do not handle that job offering. For me what is doing and what is using a company is important, because that will tell you how good are they doing their job.

The set of questions is not fixed and depends on the company, and everything is asked on the interview, telling me what is used language and which framework, is not enough. I put special emphasis in the architecure and programming techniques, like the used paradigms. If they are using Ruby, I expect a multi-paradigm approach, not only object oriented. The same applies to other languages. If they tell me “we are using MVC”, I can assume that their architecture and application is small, and it is not integrated with third party applications, because it does not require an n-tier architecture.

The same applies to embedded systems. If they tell me “we are doing iOS applications”, I will ask how do they handle the development. For example, my favorite DVCS is Darcs, and then comes GIT, and finally Mercurial, in that order. Should I handle a development which is using Subversion? I just start thinking on how hard will be doing branch merges, and I got tired immediately. The same applies to the knowledge that shows the interviewer. Should I handle a job where everything that I speak is not understood? I think not. Usually if, they do not understand every technical term that I use, it will lead me to a team that is not applying technical knowledge correctly, and they are not handling technical requirements correctly. So, I expect accurate answers to my questions.

Asking questions is a good approach to identify the scope of the company, how do they handle the development and how do they work. Feel free to ask. You can avoid several headaches if you ask the proper questions, and you will see how good are they doing their job if you ask them early. If your questions are technical, just expect the right answer, not an answer that half fits the question. Technical questions are accurate, expect the same answer. Do not leave anything in doubt, they should answer almost everything that you are asking.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

hierarchical state machines

Daniel Molina Wegener — Mon, 29 Oct 2012 15:33:00 +0000

I need to work with a hierarchical state machine. The main problem that I am currently facing is how to pass messages or trigger events on each event transition. Since I am using C Plus Plus, I will use class methods. In this case, I will use a hardware device class method. On each state transition, an event will be thrown, and the given event should trigger an action — in this case a class method — that will complete the state transition. The advantage of the hierarchical state machine is the fact that you can manage composite hardware devices without problems, because each sub-device can be handled individually with its own state, but increases the complexity of the program.

On this case, I will iterate over the list of units on the scheduler — where each unit is a composite device — it will assign a new work state on each device, calling the run method on the device object. The run method will launch the child state machine of the required action, but the parent state machine only has few states storing the device status, so the processing scheduler knows which event should be triggered on each scan cycle. The required state transitions on the parent state machine, which are informational rather than hardware related, will be useful handling nested event handling.

Each cycle will determine whether read or write actions should be triggered, even if they are manual or automatic operations on each device, and only then the program will be able to call read and write once they are required. There are some actions not using the communication channel, so I can avoid calling read and write, enhancing the application performance a little, but still I will need to check the device state to check for some automatic reads, required to display the device state and ModBus holding registers.

You can find several models of hierarchical state machines, but I prefer to write one using events bound to class methods, where each class method drives an action on the device. This approach is simpler than others using signals and similar event handling methods, but I need to keep the source code as simple as possible, because increasing the complexity should increase the development time. Finally, the unit class will be only a gateway to the each hardware device, where each state will be triggered by the scheduler. Before this model, I was using a priority queue using messages, very similar to message queues, but is required on this system, to have direct access to the device state, and that message queue is not so synchronous as it is required, where a direct feedback from the device should be placed on each state transition.

Now I have finished the scheduler and the state machine processor, I just need to complete the integration between each component and I will deliver the new device management application. Also, the HMI has changed a little, and the new model allows some direct actions like resetting the device and its state.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

programming at fifty

Daniel Molina Wegener — Sun, 28 Oct 2012 13:32:50 +0000

There are three very objective posts about programming at fifty, probably with controversial opinions about being a programmer at fifty. Each post with some truth behind it, but in my opinion the solution to keep working as a programmer at fifty in a safe environment is to be more selective in the type of job that you choose. In the first post, I get the idea that you will be working — if you are working on the consumer applications market — in an ocean of APIs, and the fact that you can’t ever lose the hunger to know more, and almost every task as programmer depend on the involved market. Job filtering is an option, a real option. I currently do not handle any kind of project, and I use very consistent filters.

James Hauge uses the phrase oceans of APIs that’s true. Most architectures are very different, and you need to keep learning about them, and their changes to keep working on them. Mainly on the consumer applications market, even if they are web applications or desktop applications, you need to learn different frameworks, design techniques and the proper architectural patterns to reach a good design. Almost all programming interfaces are subject of changes, except those with a traditional approach, like system level calls and standard libraries, but the problem is not entirely the set of changes, the problem is how each company handles the set of changes.

It’s about skimming great oceans of APIs that you could spend years studying and learning, but the market will have moved on by then and that’s no fun anyway, so you cut and paste from examples and manage to get by without a full picture of the architecture supporting your app. [James Hauge, “Do You Really Want to be Doing This When You're 50?”, Programming in the
Twenty-First Century]

If you will be working with a set of interfaces which are new for you and your company, you must be sure that the company will bring you enough time to study that API, otherwise the company should send you to any good course covering that set of interfaces. If not, if you will be working with a tight schedule, I can guess that you will be using your own time to reach a good performance and meet the schedule. I think that it is the main problem of the consumer application market. Each company must face competitive rates and schedules to gain customers.

If you want to be a programmer at 55 still you can’t ever lose the hunger to know more, know better and know simpler. Once you lose that edge the technology steamroller keeps on coming closer and closer until you wind up flat doing something else for a living. [Andrew Wulf, “Yes I Still Want To Be Doing This at 56”, The Codist Programmer Think]

You are continuously learning. You have a bunch of interfaces to learn, but also you must learn new programming techniques and algorithms, like Map Reduce, with functional basis, or work with Big Data approaches, NoSQL databases, and you must keep learning and reading. So, programming for me is the continuous process of learning and applying your theoretical and practical knowledge.

Ravi Mohan describes two types of programmers. The programmer type A who seeks solutions for difficult problems, and the programmer type B who seeks solutions for not so hard but stick in a bunch of APIs that must learn. Where programming at fifty usually means that you will be on the group A, solving hard problems with longer schedules, but the market is reduced, and you must have some expertise to work on it. That expertise is not gained with previous job positions on you have several options to reach those job positions, like creating your own side projects or delivering some FOSS product.

The programming language really matters. Mainly because almost every programming language is currently being focused in a multi-paradigm approach and using mixed programming techniques from some programming language paradigms to solve programming problems, like how is faced the parallel programming using higher order functions, now called asynchronous I/O. That is why I have learnt programming languages which have a traditional scope, like C, C Plus Plus, Lisp, Haskell, Python and Java, because they are grouping almost every programming style and paradigm, and they help me to understand deep aspects of any other programming language and framework.

a working AST generator

Daniel Molina Wegener — Wed, 24 Oct 2012 20:41:42 +0000

As you know I am trying to participate on the last challenge on the blog La Sombra de Dijkstra, where you must provide a program that compares two Fortran IV programs and must determine if both programs are equivalent. I have finished the first component, the Abstract Syntax Tree of the program. This will allow me to run the program creating an interpreter, and also will allow me to analyze the program structure — the complete Abstract Syntax Tree as graph — and make some heuristics on it.

First I want to convert that AST into Lambda Calculus functions to check if both programs are beta equivalent regarding its Lambda Calculus. I know that the program is not functional, and probably I will not find the proper representation, but I want to try it. Then I will try to reduce the graph to the lesser amount of calls, for example, removing duplicate Goto statements, trying to simplify the graph. To do that, and to use the Functional Graph Library (FGL) on the graph analysis, I must implement the Labellable instance for each node to work with each node type directly, rather than using text labels — Show instances for now.

The resulting AST for the first example, where the code can be seen as in this link, is as follows.

Fortran IV, File A

The resulting AST for the second example, where the code can be seen as in this link, is as follows.

Fortran IV, File B

The main problem is the fact that Fortran IV analyzer should be able to analyze different programs with up to 1000 lines of code. I don’t know if Parsec — the Parser Combinator library that I am using — will be capable to process that amount of code. Probably I will request one of those example files. You can find the source here.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

programming requires thinking

Daniel Molina Wegener — Fri, 19 Oct 2012 11:58:01 +0000

Programming requires thinking. You must place instructions correctly, with a logical order, and you must prove that your program will produce the right behaviour. Even doing tests, your programs can fail, so you can use code hardening or similar techniques to avoid errors. The inclusion of a program verification is engaging, but it is limited to prove a limited set of features, even if that verification is made formally, you will need to make remarkably different types of tests to check the program correctness. So, the main problem with programming is reaching correctness, and how do you need to be concentrated to avoid programming mistakes, like passing null pointers to some function, because a program can compile, but cannot work.

You have programming paradigms, or programming models, where you have different forms to face a programming problem. Where not all problems can be solved using computers, because you have several complexity levels, and also you have several algorithmic complexity levels. You have some people stating that you cannot solve every programming problem using only one paradigm, and probably you should use different paradigms to solve different problems, and you have each paradigm exponent stating that using his beloved programming paradigm is better than using other paradigms, and even language exponents will tell you that their programming language is better than others.

What does the actually happens is the fact that the market leads almost every tendency on programming language usage. If you aim to work on a company with enterprise features, you should expect that the company will use a programming language with enterprise level support. Then you have enterprises using Java and NET, with few of them using JVM driven languages like Scala, or some languages like F#. And then you have companies using programming languages where you do not need to think too much — as they think that happens — like Visual Basic NET, because is cheaper to train programmers in a language which aims to be easy to use, rather than training experts in a programming language that is hard to learn. Tell me if I lie, “every corporate environment exposed to the internet has at least one bug appearing each time that you operate that environment”. Does actually works that working scheme of using programming languages that aim to be easy to use, productive and easy to learn?. Personally I do not see any difference. Everyone can learn programming, but not everyone should program, and the problem is not specifically the experience. I know programmers with more than ten years of experience doing the same mistakes for many years.

Programming requires to be concentrated, to have a pretty clear idea of what happens in the code, not specifically with one line of code, you must have the perspective of the current block, its algorithmic context and its environmental context. So, when you are programming you are using almost every feature of your memory. If you have not developed any aspect of your memory, probably is more feasible that your code will be more buggy. Logical thinking is required too, not forgetting that how is applied that thinking, the current block of code, its algorithmic context and its environmental context. Can you create a new thread in a web server?

Programming Context

I have worked with almost every programming paradigm, with languages like Java, C/C++, Haskell, Lisp, Python, Scala, JavaScript and Coffee Script among a large list of programming languages and domain specific languages. On each one, I never knew a skilled programmer that is not concentrated in his work, without distractions. A skilled programmer keeps his mind on the code, no matter its environment. Probably is harder to get concentrate on certain environments, but I never knew a programmer that cannot put his attention to the code. I must recognize that working as freelancer have enhanced my skills, I have almost no distractions, and I am currently more concentrated in my work. The quality of my code has increased; I was able to study new languages like Haskell, Scala, Coffee Script among others and I have learned their theoretical basis. Nobody is doing annoying interruptions of my work, and it is certainly flowing. Just imagine how productive I am without annoying interruptions.

I am doing it well. I have the top points on each job related social network, where the top is 5.0 I have 4.99, where the top is 10.0 I have 9.89, with more than 10 jobs on each one, with extremely gratifying recommendations, and I have worked with highly interdisciplinary teams, delivering exceptionally high quality products. I am doing a terrific job, and I will continue doing it because I have the ability to program and the ability to learn.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

program equivalence problem

Daniel Molina Wegener — Tue, 16 Oct 2012 22:10:12 +0000

Determining whether a program is equivalent to another using its source code is probably a little bit hard to solve the problem. Usually a program has a programming paradigm according to its programming language. Probably the main conflict finding equivalences between two programs is the fact that we cannot use an approach based on its execution and expected results. Recently on a blog post on programando.org a problem related to find the equivalence between two programs was proposed as programming challenge, but the problem is not related to algorithm equivalence, which seems that it is not solved yet. There are several approaches on the equivalence between two programs. Now in this post, I am just rambling about the ideas that are coming to my mind about this problem.

We have beta equivalences, which means that translating the code to Lambda Calculus functions, we can find if two programs can be derived into the same final function using beta reductions. Expressing labels in some functional language is not so hard as you think. Guy Steele already have proposed a grammar to express labels as are they presented on the Fortran code in the examples of the challenge. Even if, that syntax is not present on the functional programming language, you can use maps and graphs to hold both labels and code, because you can bind functions to variables in functional languages. So, expressing labels as maps and graphs is not a problem. So, each code segment in the Fortran code should have an unique representation as function, and you should have same reduction once you process the code.

Another idea comes to my mind. The complete code can be represented as a tree, and we have a common approach called Abstract Syntax Tree, which is a directed graph. Very similar to what I have designed to parse the L-System DSL made for a previous contest on programando.org. Also, two directed graphs are equal if they have the same order in their nodes. This makes a difference. So, it should be a kind of sorted directed graph, but using some kind of identities between graph nodes related to its composition, rather their node keys. We can use some kind of checksum or hash on each node, like the hash function on Python or the hash method on Java, but avoiding label names and variable names, holding only calls, constants and control statements. The final hash or checksum should be equal on both programs.

So, would be nice to implement a program meeting both equivalences, the beta reduction and the AST hash. Including the brute force testing algorithm interpreting the program and comparing the execution results. I think that will be implementing the problem in three stages, first I will check the AST, because I am forced to build it first to parse the program. Then I will try to implement the beta reduction and finally I will try to implement the interpreter. I really like how parser combinators can help on this kind of problems; you can quickly define a DSL when you need it.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | One comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

side projects

Daniel Molina Wegener — Fri, 12 Oct 2012 13:56:38 +0000

As a software developer, who has a side project — open source or not — is a software developer that can make products. If the developer can identify a community need, and produce a product, there is no problem for that developer to understand a specification and meet the desired specification. Also, the developer will be able to satisfy the correct solution and will be able to meet any observations about the requirements and the source code.

Failed or not, the developer with a side project are not a mere programmer that works coding and supplying solutions for a company. He is probably a skilled programmer that wants to cover some needs on his environment, and he will probably have excellent communication skills and will be able to work on teams of programmers. If their side projects are open source projects, they will be receptive to any review of his code. He should be able to write pretty clear documentation about his product, if not an acceptable document about how it works. Not necessary a successful project, but a project by his own initiative is just means that he wants to contribute.

So as a developer with his own side projects, he should be able to communicate with his customers, to supply the proper solution, and he will be able to solve their problems. That is what does means. No matter the project type that does he handle, on his development area, he will be skilled enough to develop his environment as a programmer. As a freelancer, I must be receptive to the requirements of my customers, and I am doing it well. I am skilled communicating and delivering software pieces highly appreciated by them.

Profile Points (4.99 of 5.0)

Start considering a filter once you try to get new projects. I do not handle any project that I receive. Usually I filter projects using some questions and restrictions to certain technologies, because I know that they will meet my skills, and I have at one’s fingertips that they will not be an obstacle. The main used filter the technical skills level of the project manager that wants to hire me. There are some pseudo-technical project managers that have no idea about programming and the required effort to build a product.

Another trick is to handle projects paid with hourly rates rather than fixed price projects. Hourly rates are almost the best option that I can handle. Fixed price projects are a problem, even if the requirements are pretty clear, they are always wanting changes, if they are not so good negotiating, they will accept any new changes without charging the proper bid, devaluing the project cost and your work. With hourly rates you really get what have your worked, and if you are a really skilled developer, that will not be a problem for your customer, instead to be a problem they will like that they will have the same requirements in less time that they think. So, if you are not skilled negotiating requirement changes, do not contact me…

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)

the problem with web development

Daniel Molina Wegener — Sat, 06 Oct 2012 15:17:57 +0000

If you are a programmer, you are probably working on web development. Almost every application that is being launched to the market is a web based application or a mobile application with a background web platform supporting it. From that point, almost every job option related to software development is strongly linked to web development, and I think that the web development market is a little bit messy, mainly due to the lack of technical knowledge of people that are hiring developers. Just naming the required technology enough times to make your resume to pass the filter in the job position is a good start point to getting hired. Is that enough to meet the job requirements?. On software development experience is not so required, and seems to be strongly required by some companies hiring developers.

Then you have people naming the required technology as many times and many years that position requires, but do they really have the required skills and required knowledge?. I have seen some resumes filled with technology names and languages names arguing strong experience with them, but nothing related to concepts, and the same applies to job positions. So you have hiring announces with the same message pattern everywhere: “we need people experienced with technology-name-here”. Also, those announces are avoiding people that can make a real difference with their current programmers, because they have pretty clear programming concepts and they will be able to learn that required technology as fast as they have passed the hiring process — even if they are not experienced with the required technology.

The theoretical background of the programmer matters. With the proper knowledge, the programmer should be able to learn any programming language that is being used on the market. A good programmer should be able to jump from Python to Java and backward, without any course, just few books related to the used technology. I have several resumes, because I am a skilled programmer that can handle several programming languages, so I have a resume for the Enterprise Java market, another for Embedded Systems market, and so on. If I place every language that I know in one resume, the required technology for that job position will not be named enough to make it pass the filter in the hiring process.

No matter from where comes the programmer, his breath in programming practices is what makes him a good programmer. Someone with a strong lack of interest on programming will not be as good as someone with real interest in programming. All is about programming, is the main task on software development. Someone without programming knowledge cannot build a good software design, because it will lack consistency related to the technology and environment. Also, will not be able to manage a project because he lacks the proper technical knowledge to cover a good design and programming practices, and programming is about continuous learning because everything is changing constantly, but the basis has been the same from the elder stages of software development.

So, I agree with the idea of proving how good a programmer is. Which method do I use to test a programmer? I ask for open source projects because I want to see his source code. If he does not have an open source project, he should have a side project, and he should be able to show me some library file with something impressive. Once I see his source code and resume, I start evaluating his theoretical knowledge through a conversation, just to check the clarity on some programming concepts — I am not interested on how many years of experience does he have, but I want to check that he will meet the requirements — and then I will start doing a live programming test. So he can show me his screen through Skype, TeamViewer or another similar application and will let me know how good is solving programming problems, having a conversation about the programming solution and check the applied concepts. Experience is not required, but deep programming practices are a must.

© Daniel Molina Wegener for coder . cl, 2012. | Permalink | No comment | Add to del.icio.us
Post tags:

Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0)