Did somebody say "It Depends"?

In a toast masters club, there is a role named "Ah Counter". The duty of this person is to listen carefully to the speakers during a session and mark the gap filler words and sounds that appear between meaningful phrases. They call them "clutch words". It includes sounds and words like "Ah", "Um", "So", "but", etc. In addition, if the speaker habitually puts words such as "kind of", "sort of", "you know", etc everywhere they are also regarded as clutch words. After the speech, Ah Counter provides a report of the number of clutch words uttered by a speaker. This helps a speaker to identify the most frequent clutch words used by him (mostly unknowingly) and to put a conscious effort during next speech to minimize them. I know that this feedback approach helps greatly from my experience. I was surprised by the high clutch word count reported regarding my initial speeches in the toast masters program (conducted in our company) because I did not utter any of them intentionally. However, after being cautious about them I was able to reduce the number significantly. During my last prepared speech I uttered only one or two clutch words.

Toast masters community says that the clutch word count is an indicator of the preparedness of a speech. The argument is that a speaker will tend to put those words to fill in gaps in his speech while thinking about the next thing to say if he is not well prepared. This is a sound argument and I realized the truth in it after noticing that I always get a significantly higher clutch word count in my unprepared speeches (there is a separate session in toast masters for quick topics) than I get in prepared ones. Almost every speaker in the club exhibited an improvement in clutch word usage throughout the program indicating that they are preparing better for speeches and are becoming better public speakers.

I think there is another meaning to the clutch word count too. In my opinion, a speaker may use a gap filler word like "kind of" or "sort of" when he is not sure about what he is saying. This form of clutch words can appear even in written forms like articles. For example, I used the term "gap filler words" in the beginning of the article when introducing clutch words. If I was not certain about the appropriateness of that term I would have written "sort of gap filler words". Ideally what I should have done in such a doubtful situation is to put some extra effort to verify the appropriateness of the term or to find a better alternative term. Instead of doing that I hide this uncertainty inside the term "sort of" so that I am not responsible even if the term that follows is not a good fit. This is none other than cheating. I am cheating to the listener or to the reader by concealing my laziness. After getting to know about clutch words, I noticed that many of my previous speeches and writings exploited this cheat trick and everyday I see other speakers and writers doing the same thing. The general rule is that if you use terms like "kind of" and "sort of" unnecessarily you really do not know what you are saying.

We all have heard people using the term "it depends" during technical discussions. Some of them put it in the beginning of every fact they talk about particularly when answering questions. Is there any meaning to this term whatsoever? As a listener, I already know that "it depends". Specifically I know that everything in the world depends on something else. I do not need to read Stephen Hawking's "A Brief History of Time" to understand it. What the heck is the need for a speaker to utter this term? I guess the reason is the same as mentioned in the previous paragraph. They say "it depends" because they do not really know what they are talking about. After saying "it depends" one can say anything. He is shielded from any criticism or questioning on what he says because he dilutes it with the first two words and hence does not stand for it. I suggest to regard "it depends" as a clutch word in technical speeches. It is okay to be used when the dependency really counts where the speaker is responsible for explaining each dependency and its effect.

The toast masters program helped me to get rid of unnecessary words in public speaking and to identify unprepared and dishonest speakers (and writers). I hope this article helped you, the reader, to be cautious about this. Your feedback is much appreciated. So please use the Comments section.

Breain teasers

Here are two interesting problems I came across recently. Take my words; they are worth giving a try. You might not come up with a correct answer quickly. The key is to keep on trying and never to give up. Providing this sort of exercises to the brain from time to time is a very good practice particularly to fight against decline of thinking power with aging.

(1) The problem of circles

I read this problem in a web site. It is really interesting. The challenge is to find the maximum number of closed regions that can be formed by overlapping a given number of equal sized circles. Each region that counts must fall within at least one circle. The answer for the case with two circles is obvious; we can form 3 regions. It doesn't even deserve a pictorial illustration. 3 circles is also a pretty simple case. It is shown in the picture in the beginning of the article. The result is 7 regions. What about the case with 4 circles? Now the problem is getting more challenging. However, I guess anyone with average thinking ability can come up with the answer after few minutes of brainstorming. The answer is illustrated below.

What we have done is placing the forth circle between the 3 circles in the orientation for the previous case. The result is 13 regions. However, there's a little problem. How can one be certain that this is the orientation yielding the maximum number of regions? 13 is the correct answer in this case, but how can we prove that it is?

We clearly observe that the difficulty of the problem problem grows exponentially with the number of circles. Furthermore we do not have a proof even if we come up with a decent solution. The issue is that the ad hoc thinking we used for the simple cases just do not work when the problem gets more complex. Ad hoc methods generally do not generate proofs either. The problem asks us to come out of the frame and think smarter to arrive at an enlightening solution. Can we do it? Yes, it is possible. All one needs to do is to defeat the inertia to engage in thinking and continue with willpower. 

Here we go...My challenge is: What is the maximum number of regions that can be formed by overlapping 10 circles?

(2) The problem of Aluminium plates

This is a practical engineering problem. One of my friends who works in an Aluminium factory came up with this question regarding a problem encountered in their day to day operations. They get fairly big square shaped Aluminium plates and are asked to cut rectangular pieces of various sizes out of them. The requests come at different times so they do not know the sizes of all the pieces they need to cut from a single square. When they get a request for an Aluminium piece, they cut it from one of many available half cut Aluminium squares by guessing the best one to cut from. When few pieces are cut from a square it might look like this.

At a given time our friends have many plates with this type of shapes. The pieces are always cut with parallel edges to that of the initial square. Therefore there are no slant cuts. After a square is consumed to a level that is no more usable (no pieces with a given minimum size can be cut from the remainder) they add it to a recycling phase where those remainders are melted to make new squares. Understandably, this melting process is quite costly and optimum usage of an Aluminium square before recycling will definitely save them a good amount of money. Therefore the problem is to come up with a systematic way (an algorithm..forget this word if you are not a computer engineer) to cut pieces from Aluminium squares in an optimal way.

We can formulate the problem like this. Given all the shapes of remaining Aluminium squares and the size of the rectangular piece to cut, how to find the best square to cut from and the best place in it to cut the piece? Remember that figuring out the optimality criteria is also a part of the solution. After all what our friends in the Aluminium factory need is to cut more pieces from squares.

I will post my analysis and solutions to these problems in a later post. Until then, happy thinking.

My challenge for new year

I have just finished a workout and feel fresh and energized after the shower. During the workout, a question was stuck in my mind. What will happen to my health program during the next two weeks? The biggest festival in Sri Lanka falls next week. It is called the Sinhala new year (or "Sinhala Avurudu") which is aligned with an astrologically significant event. Everybody goes on leave for at least one week and it's party season. Everybody wears a new cloth in the auspicious time and follows certain rituals in an astrologically determined time. The intention is to renovate relationships with family, relatives and friends whom you might not have met for quite a time.

Well...No festival is complete without exotic food. Sinhala new year too is celebrated with lot of sweets that are unique to the season. Some of them are unique to areas as well. The good (or, may be the BAD) thing is that everyone gets unlimited access to all these food during the whole season. Let it be home, office, filling station or may be even the gym (he he). Food is exchanged between households so that everyone tastes every flavor. People wait for this season to fill their appetite with this fascinating collection of foods. My favorite is "Mun Kevum" which is made with flour, sugar and oil; a deadly combination..he he. In each season I usually eat more than 100 of it.

The problem is the conflict with my usual food discipline which is a part of my health program. Lot of oily sweets means lot of fat. I really don't want to end up jeopardizing all my long term health achievements during a period of two weeks. However, missing the exotic food doesn't sound like a smart option too because this is the only period to have them. Tom Venuto's (visit http://www.burnthefatblog.com for his excellent health blog which greatly helped me in shaping my health program) "Christmas Challenge" came to my mind when thinking about this. Recalling it, his main point was something like "extra effort to burn the extra fat". This is apparently a smarter approach since it doesn't ask you to be a food fanatic during the party season. Rather the advice is to account for the extra fat intake during the burning phase. Simply put, one has to do more exercise during festival seasons. 

Ok...The ask is...More exercises please. I do not get intimidated by that because I enjoy most workouts. I need to plan for more workouts for the coming weeks. Usually I do workout for about 5 days a week. Now I have to target 7 days a week, and more than one session per day when possible (Aim for the moon, that way, even if you miss you'll still be amongst the stars! - W. Clement). One target of writing this article is to put more pressure on my self to stick to this discipline. Something in written form is more effective than a thought, isn't it? Even more effective is when you write it in a public place. Now I'm accountable. I think one more thing I can try is eating only the foods that I'm crazy about and keep my self away from others. It might help to keep the damage within accountable limits.

I'm ready for the season. I will pamper my taste buds with the exoticness of new year food and will stay more active to compensate. Welcome new year!!    

If you don't swim, you miss!!

I'm trying to write an inspirational story. It is based on a recent experience of mine. I stole my theme from Osho. The heading of his one beautiful article was "If you swim, you miss". I twisted the title to match my topic.

I have been trying to learn swimming at least for past 7 years. I made several attempts from time to time. Each time I started with much enthusiasm and tried hard to learn the basics during the first few days. However, I hardly made any progress beyond floating in the water and drifting couple of meters as long as I could hold one breadth. I thought that swimming is not my thing and there's something terribly wrong in my body which hinders me from becoming a swimmer. This thought lead me to give up every time.

However, many incidents compelled me to give another try to learn it. Whenever I got into a swimming pool, may be after a pool side party or in a hotel during a trip, I had to keep my self in the shallow end standing on the pool like a kid which posed me in utter disgrace. Last year I bought a new swimming pool membership and thought to me, "This time you are either going to learn swimming or drown in the pool and die". This time I made a change; I got the help of a professional trainer.

I started with the same desperate condition. However, my coach gave me the most important advice in learning swimming: "Don't expect progressive results and never give up". This meant that I should not be discouraged if I do not see any progress within days of training. I asked him how long should I be trying to become a swimmer with basic skills. He said "about 30 days". At that point I realized why I was not successful in my previous attempts. In any of those attempts I did not try even closer to 30 days.

Having received the correct advices I started training with a huge will power. I did not care whether people laugh at me or whether I look ridiculous making lot of sounds and drinking pool water. I believed that there should be a light in the end of the tunnel. Even in the twentieth day of my training I did not perform much better than my first day's workout. However, my coach was correct!! When it came close to thirty days...suddenly and totally unexpectedly...I could swim a good length while performing all the basics well. I could not believe it for a moment. It was one of my happiest days in that year. After all those years of struggling I became a swimmer.

It is interesting to note that many changes in the human body take place in leaps but not as ramps. Many people misread this behavior as unresponsiveness. A medical book I recently read, "The secrets of miracle doctors", suggests that this fact is true for many health aspects. What you need to do is keep making small quantitative changes even if you don't see any response. Those small changes add up and trigger a significant qualitative change somewhere down the line. It might be an amazing coincidence that Marx and Engels say that "Continuous quantitative changes lead to sudden qualitative changes in the society". This might be what keeps the agitation of courageous socialist leaders even when they get almost no response from the society.

Learning swimming inspired me to try out the same strategy in other things too. Even if none of them works, I'm a happy regular swimmer now.

Automatic log file analysis

keywords: log data extraction, record expert knowledge, mind maps, expert systems, Application Verifier

I'm currently engaged in a research on automatic log file analysis. I came across this idea during my MSc research on software quality verification. When it comes to black box testing, there are many handy tools that analyse a certain aspect of an application. These aspects may be CPU utilization, memory consumption, IO efficiency or low level API call failures. One prominent problem associated is the requirement for expertise for using these tools. Even for experts the process takes a lot of time. For example, I have been using a free Microsoft tool called Application Verifier which keeps an eye on an application's virtual memory errors, heap errors, access failures due to improper access rights, incorrect usage of locks (which may result in hangs or crashes), exceptions, corrupt Windows handles, etc. It is a very useful tool to capture application errors that are impossible or extremely difficult to identify in a manual QA process. Even with experience it takes me about 2 days to test a product with this tool before a release. Given the hectic schedules close to a release, what happens more often than not is that I do not get a chance to do this test. One other problem is that there is no good way to record my analysis knowledge so that someone else or "something" else can perform the analysis if I'm busy with other stuff. Sequential text, which is the popular form of recording knowledge is not a good option in this case due to several reasons. First, it is difficult to write documents in sequential text form (I think most developers agree with me in this). Then it is difficult for someone to understand it due to the inherent ambiguous nature of natural language. Furthermore, a program (this is the "something" I was referring to) cannot understand it for performing an automated analysis.

Almost all the analysis tools that are out there generate some form of a log file. Big majority of them are text files; either xml or flat text. If we can come up with a mechanism to extract the information from these log files then the analysis procedure can be partly automated. The challenge here is to devise a scheme that can deal with the wide variety of proprietary structures of these log files. Though there are a bunch of tools available for log data extraction all of them are bound to a specific log file structure. All the log analysis tools I found are web log analyzers. They analyze the logs generated by either Apache web server or IIS. One cannot use them to analyze any other log file. An additional restriction is that the reports generated after the analysis are predefined. One cannot craft customized reports for a specific need.

There's one more dimension to highlight the importance of automated log file analysis. The majority of software products themselves generate log files. These logs are analyzed by product experts in troubleshooting. Each product has its own log file format and the knowledge required for reading the logs and making conclusions lies only within a limited group of product experts. With the maturity of a product, it is highly likely that some troubleshooting patterns emerge over time. However, there is no means for recording the knowledge on these recurring patterns for later use of the same expert, others or an automation program.

The tasks associated with log file analysis are information extraction, inference, report generation and expert knowledge recording. What I'm working on is a unified mechanism to automate all these tasks. I'm trying to do it with a new simple scripting language based on mind maps. I will write more about the solution in future with the progress of my research. Please keep me posted (dilj220@gmail.com) about:

• Any automated log analysis tool known to you
• Any other reason or scenario that comes to your mind for automated log file analysis
• The features that you expect as a developer / QA engineer / product expert / manager from an automatic log file analysis tool

Software Quality Verifier Framework

I completed my MSc thesis couple of weeks back. My project was developing a Software Quality Verification Framework. Given the value of early bug detection in the software life cycle, the framework addresses both white box and black box testing.

White box testing
White box testing is implemented in two phases.
1. Commit time analysis - In this, the code is automatically analyzed when the developer tries to add new code or code changes to the code repository. Quality verification is done employing tools against a predefined set of rules. The commit is rejected if the code does not conform to the rules. The developer is informed with the reasons for rejection in the svn client interface. This functionality is implemented using svn hooks. Example output is as follows.

2. Offline analysis - A more thorough analysis is performed in an offline fashion, in the context of a nightly build, for example. Results of the analysis is displayed in a dashboard which shows various analytics and provides violation drill-downs to code level. Automatic emails can be configured to acknowledging various stakeholders on the overall health of the system and developer technical debt. This is implemented using a tool named Sonar (http://www.sonarsource.org/).

Black box testing
It was identified during the research that quite a number of tools have become available recently for evaluating a software product during its run without looking at the source code which generated it. Different tools evaluate a product on different aspects such as memory usage (corruptions, leaks), IO usage, operating system calls, performance, access right violations, etc. However, there's no tool that combines the results generated by these individual tools to automatically generate a product health profile like Sonar does with the white box testing tools. There are two main problems associated with the approach of using individual tools manually to perform tests.
1. Tool usage requires expertise and also is laborious
2. There's no way to record or automate a once identified troubleshooting (or evaluating) procedure

I thought about different solutions for this. Noting that almost all the tools generate a textual output in the form of a log file, I decided to implement a way to automatically extract the information of interest in a given context from those log files and generate reports for consumption by various parties like project managers, developers and technical leads. The output was a simple scripting language based on mind maps. The developers can write scripts in this language to extract information from various log files, derive conclusions based on them and generate reports.

Following is the architecture of the framework. I will blog more about the framework later.

Software Requirements - Dilbert

This must be every developers' dream

Subscribe to dilbert from: http://www.dilbert.com

It does really worth!