Wednesday, April 29, 2009

Shooting without Aiming

Cross post from IRefactor

During my lectures on “Refactoring & Design” I am frequently amazed to hear the following ideas:
“Doesn’t Design contradict Agile? Isn’t Agile about gaining speed, while Design about gaining bureaucracy?” - some ask.
“Why bother with Design? Eventually It’s impossible to do any design when using Agile methods, like TDD” - others continue.

I often compare the aforementioned questions to “Shooting without Aiming”. It won’t occur to shoot at a target without first aiming at it. The rule is true, especially when the time is short and every shot counts; You will spend an extra minute just to aim it well!

Shooting after Aiming is “life saving”!
So, How does it even come to any mind to code without thinking(thinking=Design)?

Please, don’t take my word on it. See what Robert C. Martin has to say about The Scatology of Agile Architecture. Robert C. Martin knows a thing or two about Agile Development. After all, he was the originator of the meeting that led up to The Agile Manifesto.

Friday, April 24, 2009

Dude, I blew up the Demo!

Cross post from IRefactor

I am sure, we are ALL familiar with the situation:

Morning… The sun is shining, the birds are chirping… You are sitting in front of your computer, sipping a delicious cup of coffee. Then, in the corner of your eye, you spot movement; your VP Marketing approaches you with a big smile on his face:
“Jonathan”, he says, “Just got important news, we have a big opportunity! We have been requested to demonstrate our super complex web analysis capabilities to a huge potential client. Could we build a quick demo?”
You sigh and start coding…

You don't sleep neither eat; you copy and paste; you code; you build and execute and after stressful (but, yes, enjoyable) five days you provide a top-notch Demo.

The new “toy” becomes the hottest news in the office.
It’s cool, it's fast, it's colorful and it demonstrates an innovative functionality and thinking!

The VP Marketing is in heaven; he presents the demo to the client and gets an enthusiastic response.

“Jonathan”, his eyes are gleaming, “They are excited! They just need a small feature - export to excel, to evaluate it a little bit more.”
You return to the operating table and add some “quick & dirty” code to export the analysis to excel.
“Jonathan”, your VP Marketing, “Great work! Could you also add a small feature of notification by email?”
A few days later…
“Jonathan, Well done!, Could you also add…”
A year later, you find yourself maintaining the demo and cursing that cheerful morning you had agreed to develop the goddamned application!

If you ask your VP Marketing what happened, he will honestly say: “Dude, I blew up the demo!” Remember the Honey, I blew up the kid!? Your VP Marketing, "accidentally", blew up your five days old “child” into a giant “monster”.

It’s common for a small-mid size companies to turn their “demo” applications into production ones. The “time to market” is crucial; once the demo was introduced successfully, the features are added patches over patches resulting in a House of Cards AntiPattern.

House of Cards AntiPattern – a continuous patch (card) over patch (card) which is done in order to correct a bug or to add a focused feature without design or refactoring considerations.

Even in a small demo application, there is a place for a careful examination of the developed features. You can make assumptions; you can speed-up the UI development, but you need to design the core features, as you develop the production code:

  • Separate the Domain from the Presentation, using MVP pattern for example.
  • Use Fa├žades to shadow the BL.
  • Provide a well organized BL. (Don’t try to address all the future possibilities and requirements. Just provide good object oriented basis).
  • Don’t duplicate code, don’t provide lengthy and hard to read methods.
  • Provide Unit Tests and test the BL as much as possible.
  • Deal with “considerable”exceptional situations (It’s OK to decide not to deal with uncommon demo scenarios).
Notice, it is perfectly fine to make assumptions and to apply some limitations during the demo development. However, building the demo correctly will ease the move to the production, even if a certain feature is redesigned or redeveloped.
Undoubtedly, it will ease the development of any additional “demo” features for more potential clients.

As for the managers, try to remember; unless you want to make excuses for unmaintainable and hard to change code, you need to allow your development teams to work a little bit longer, just to produce a better demo!

Saturday, April 18, 2009

Separate Domain from Presentation – Part I

Cross post from IRefactor

Visual Studio can be a swiss knife in the hands of a Software Developer.

You want a fully functional application within several hours; here you go sir:
  • Create Windows Forms Application.
  • Add relevant Data Sources (in Figure 1 below: Courses table).
  • Drag generated entities from the Data Sources onto relevant Forms.
  • Add minimum code to show the Forms in the required order.
  • Compile & execute.
Walla, I present you an “enterprise application”!
Though it looks amateur, it allows fully functional CRUD operations.

Figure 1

But, the power can corrupt…

Lately, I came across a few projects implemented using the above approach. Needless to say, the projects were “spaghetti code” - all the UI elements, visual states, business logic & data access, were centralized within a few Forms.

(Don't think it's a rare situation! There are enough companies that try to "save" time using such a “quick & dirty” approach).

I am a pragmatic Software Engineer and I do believe that the “quick & dirty” approach is vital in some cases.
If you present a demo application that demonstrates your capabilities; go ahead, use the necessary tools to implement it as fast as possible.

However, if you develop a real life applications using the above approach it’s a sure road to hell. Slowly but surely the “fast development” takes more and more time:

  • The customer’s button event click is very similar to the employee’s event click, but correlate and update different UI elements » you duplicate the logic in both places.
  • New requirement add new conditions to some wizard flow » you add an “if” statement to address the conditions. Sometimes you add a “switch” statement due to the fact that many “ifs” already exist.
  • New requirements impose saving more data » you add more parameters to your methods. You also go further and add those parameters to each and every place (form/method) where the change is required.
  • The classes and methods become huge, they are overcrowded with UI elements, visual states transformations and business logic. Next time you try to read the code, you will need at least a few hours to concentrate and walk through the mess.

The Project Manager responsible for the aforementioned projects, said to me:

“Any change request I want to apply, come back with unrealistic time estimations. When I ask the developer why it takes so long, he answers: My Gosh – do you know what it takes to change that code? Do you understand how long I need to retest the application, just to verify I didn’t break anything?”

It seems obvious that if in the first place, the project had been written in clear separation of UI, Business Logic and Data Access Layers, it would have been easier to maintain the product lifetime. However, clearly it is not the case here, so what can we do?

One of the tools we can apply here is: Refactoring.

Refactoring is a process of improving software’s internal structure without changing its external behavior. Refactoring ensures completeness of the external behavior, by executing automatic unit tests. We improve the code in incremental small steps (refactoring steps) and each change is followed by a compilation and unit tests execution.

In the following posts I will demonstrate refactoring called “Separate Domain from Presentation”, which obviously breaks the tight coupling between UI and BL. After finishing the demonstrations of the refactoring process itself (it will take me several posts to do it) I will also explain how to create the suite of unit tests, which allow validation of external behavior completeness.


Remarks:

  • I will refactor the application from figure 1 (above).
    The application contains one View (FrmMain) and one domain object (CoursesDS).
  • Though the sample application contains one Form and one DataSet, the same technique can be applied in more complicated situations. Just identify the groups of objects that fit together and apply for each one of them the following steps of separation.
  • It’s advisable to download the sample IRefactor.CoursesView project and go through the refactoring steps while reviewing the code.
  • In the next posts I will introduce how to enhance the separation using refactoring towards MVP (or MVC) patterns.
  • It’s harder to write about the refactoring process, than actually implement it. Don’t be intimidated, it takes longer to explain each and every small step (and provide screenshots) than to do it in practice. To Separate Domain from Presentation (up to MVP) takes me about 10 min of work.

Refactoring steps:

  • When the Irefactor.CoursesView project was defined initially, Visual Studio generated a DataSet called CoursesDS for the Courses Data Source. As you can see, the Irefactor.CoursesView project mixes UI (FrmMain) elements and Domain (CoursesDS) elements.

    Figure 2
  • Let’s start by removing the CoursesDS domain object from the UI (View).
    Create a project IRefactor.Common and copy there CoursesDS DataSet.
    (Drag the CoursesDS from project to project, Visual Studio will do the rest).

    Due to the fact that CoursesDS is a VS auto generated class, one should also copy the connection string definition which is stored in the Settings of the IRefactor.CoursesView project. Add Settings.settings to IRefactor.Common Properties folder and copy the connection string.

    Figure 3
  • Compile the Solution and execute the Unit Tests.
  • Change the FrmMain (View) to use the newly created IRefactor.Common.CoursesDS DataSet.
    Open the FrmMain’s designer. Remove the coursesDS (the instance of the domain object: IRefactor.CoursesView.CoursesDS DataSet – see red arrow 1).
Figure 4

  • On the Toolbox window, under the recently added IRefactor.Common Components you will see the IRefactor.Common.CoursesDS DataSet.
    Drag this CoursesDS to the FrmMain Form.
Figure 5
  • Dragging the IRefactor.Common.CoursesDS DataSet onto the FrmMain surface generates an instance of the class. Rename it to coursesDS (to match the previous name).
  • Repeat the three last steps also for IRefactor.CoursesView.CoursesTableAdapter (see Figure 4, red arrow 2)
  • Select the BindingSource (see Figure 4, bsCourses) and change its DataSource to coursesDS instance (coursesDS now comes from IRefactor.Common project). Also change its DataMember to point the Courses table.

Figure 6


  • In FrmMain code behind file add reference to the IRefactor.Common namespace and change the CoursesDS to a fully qualified name.

  • // add reference
    
    using IRefactor.Common;

    // ...

    private void coursesBindingNavigatorSaveItem_Click
    {
    (object sender, EventArgs e)
    // ...

    // change the coursesDS to IRefactor.Common.CoursesDS
    IRefactor.Common.CoursesDS.CoursesDataTable changes =
    this.coursesDS.Courses.GetChanges()
    as IRefactor.Common.CoursesDS.CoursesDataTable;
    // ...
    }
  • Compile the Solution and execute the Unit Tests.
  • Go over IRefactor.CoursesView and remove all instances of the previous domain object IRefactor.CoursesView.CoursesDS DataSet. Also, delete the CoursesDS.xsd file from the project.
  • Compile the Solution and execute the Unit Tests.
We have successfully completed the first steps. We have separated the UI project called IRefactor.CoursesView from the Domain Project called IRefactor.Common. Now it’s time to become serious and continue the refactoring towards the MVP pattern.


The Hunt for a Software Engineer

Cross post from IRefactor

 

This week I had interesting discussions regarding hiring and interviewing professional personnel.

The reason it was so interesting, is the fact that the discussions were raised simultaneously by two friends of mine, each from his unique point of view.

David is a R&D Manager who leads successfully a complicated software project written in .NET technologies.

David tries to hire a Software Engineer with specific experience in Smart Clients and Complicated Servers applications but without any success.

David complaints that most of the candidates he meets have a superficial knowledge and don’t meet his expectations.

Daniel is a talented Software Engineer whose major experience is in ASP.NET, willing to change his place of work. Daniel is a very dedicated person; he attends almost each and every summit he can on .NET technologies. He also reads quite a lot of blogs and articles in order to enhance his knowledge.

Daniel had already attended more than dozen interviews, most of which he failed because he was unable to solve some “tricky” (according to his words) questions.

By “tricky”, he gives an example of: “Given a uniformly distributed function between 0 to 1 write a function that generates unique numbers from 1 to N in complexity of O(…) and …” and that’s there he loses his nerve.

“What the hack”, he says to me, “The company even doesn’t deal with any statistical or mathematical computations, all it does is pretty standard enterprise application; retrieve, show and update”.

On the one hand it is ridiculous that Daniel cannot pass interviews, but on the other hand it is also true that we lack good Software Engineers.

But wait, could it be that the problem is within our industry (or us)?  

·         Understanding that there is a need to hire a new Software Engineer mostly comes too late (in the project’s schedule).

·         As a consequence, due to time constraints, there is no time to train the new employee. Therefore the employee must meet very rigorous requirements.

·         In order to ensure those requirements (that vary a lot from company to company, based on their line of business), the interviewer tries to infer candidate’s competence using one of the following tools:

o   Ask domain specific question, based on the interviewer’s (or his company) experience or immediate needs.

o   Ask “quiz” questions, question that can be solved using some tweaking.

The interviewer forgets Peopleware which clarifies: “Major problems of our work are not so much technological as sociological in nature.” Failing to answer a tweaking (“a-ha”) question isn’t a virtue you would like to judge on.

·         Finally, candidates that somehow meet the requirements are being hired, which deepens and narrows their experience in the long running term.

 

A good candidate doesn’t need to know how to tweak a domain specific problem within an hour. Moreover, a good candidate doesn’t need to be a specialist in Smart Clients applications.

Don’t get me wrong, if such a candidate appears during the interviewing process I will gladly hire him (if I need such a skill for long term).

However, if your projects and recourses are planed correctly, you can loosen your requirements a bit.

I have always believed that a Software Engineer should be as versatile as possible and therefore the minimum bare requirement I seek is a very strong software engineering background.

In my opinion, a Software Engineer must understand in different “tools of trade”. He needs to know why it is better to apply this pattern or that pattern, when it is suitable to use Smart Client applications and when it is not, how to build good infrastructure and clean software and etc… etc…

Having good software engineering skills, doesn’t mean you don’t need to specialize vertically in a specific field.  With the correct mentoring and coaching, candidate’s experience in several disciplines he chooses, will evolve with years (and this can be a whole another post) and I even expect it to happen.

And that’s how, my conversations with David and Daniel this week ended by discussing how do I find this bare minimum in the candidates, using technology questions and without any tricky (“a-ha”) approaches (of course there are also non technology question that I ask during the interview in order to review candidate’s soft skills).

Here are the virtues that I try to pursue during an interview:

·         A candidate must show a strong understanding of Software Engineering principles

·         A candidate must be able to learn and evolve with the time.

·         A candidate must have a passion for the trade.

 

The last virtue is extremely important; how many people do you know that work in the hi-tech industry, just because it seems to be “a shiny, nice place”? You probably met such people in the past when they complained to you not to discuss any work issues during the lunch time.

 

 

And here is the basic skeleton of the interview:

 

Previous Experience

I ask the candidate to choose an example from his previous work and to explain its design.

I expect the following:

·         Problem explanation – clear and concise explanation of the domain problem.

·         Solution explanation – short conceptual model or class diagram explaining the solution, including alternatives that were considered, technological pitfalls that were overcome and etc…

It’s amazing to see that many candidates don’t know how to communicate simple technological problems, don’t know what they really tried to solve in the first place and why the solution was chosen the way it was.

 Design

I ask a design question from a well known domain.

A well known domain helps to remove obstacles when introducing to the candidate some domain specific logic that might come from the interviewer company’s experience.

An example of such a question will be:

“Please design the MSN Messenger” or “Please design the Linkedin service”

During the interview I add more and more constraints to the question.

I also provide some leads that may (or not) help to solve the question.

I expect the following:

·         An ability to adjust the design with each added constraint.

·         An ability to learn from my leads and to change the solutions accordingly.

 

Technology Questions

No tricks here, no “a-ha” questions\answers.

I really don’t want the candidate to go back home following an interview with me and suddenly to have the “a-ha” answer for the question I asked.

I just try to reveal how deep the candidate understands the “tools of trade” (based on his experience).

Here are a few examples:

·         What is the difference between class and struct?

The question yields in 70% (there are still people that don’t know the difference?) a correct answer that a class is a reference type sitting on the heap and a struct is a value type sitting on the stack.

But, wait, what does it really mean?

How that helps you to build better software in .NET technologies?

(Here is a quick hint: in .NET everything is a struct or a class. Basic int is a struct; why is that?)

·         What is the difference between a thread and a process?

The question drives a broad discussion over why and when we need threads, what are the synchronization methods available in .NET framework, what are their pros and cons and etc…

·         How do you retrieve a record from a Database?

This question also drives a broad discussion about connected and disconnected modes, dataset\datatable, typed dataset\datatable, XML, ORM, optimistic and pessimistic locks and etc…

 

By adding more and more constraints on the questions asked I evaluate how deeply the candidate knows his “tools of trade”. He doesn’t need to know all the answers, but you will see; a passionate Software Engineer will know quite a few things under the hood and even if he won’t know, he will try to figure it out, discussing with you all the possibilities and alternatives.