Walter Almeida's Blog

Presenting a great tool: ReSharper

As I describe in the Golden Rule #2, refactoring is a first class citizen of an agile software development process. However refactoring can be painful without appropriate tooling. I have been using the refactoring tools included in Visual Studio since version 2005. This was until I tried and adopted ReSharper.

Visual studio provides us since version 2005 with great refactoring tools. However you start to see limits of these embedded tools once you start large refactoring work involving multiple projects and cross references. I recently started the first refactoring phase of a large project I am working on at the moment. After a first half through, I decided on trying the ReSharper tool. And this was the best choice I possibly could have done! For information I am talking here of ReSharper version 4.1, and I am developing on Visual Studio 2008 SP1. So here is my feedback on the tool:

First of all, I must say that ReSharper is amazingly well integrated into Visual Studio, as you will see in the snapshots later. Very easy to install and then fully integrated into Visual Studio. 

Then I should clarify that ReSharper is more than just a refactoring tool. It includes tools for code analysis, coding assistance, code cleanup, code generation, obviously refactoring and even more… I haven't tried all the included tools yet but I will say a word about code analysis and coding assistance. ReSharper is doing some things that were intended by FxCop (now included in Visual Studio as code analysis), but in a much better way. FxCop was an attempt to guide developers toward writing clean code and following best practices. But it was IMHO a failed attempt, and FxCop was perceived by the developers more as a hassle than real help… Those who have tried the tool can understand what I mean. ReSharper has another very impressive approach to deal with code analysis and gives to the developer hints and advices directly on the source code view! And it is presented as hints rather that error messages. It might seem a detail but it makes a huge difference! Well I guess that FxCop provides many more rules than ReSharper and is not intended for the same purpose, but the ReSharper concept could maybe be used for competing with the Visual Studio code analysis.

Things I experienced are still missing in ReSharper are: renaming in XOML files for WF workflows. And I am not sure yet how renaming works globally with XML configuration files… So if you use dependency injection and configuration in XML then you might have to handle the renaming by hand. Therefore this is one advantage of configuration the DI cotainer dependencies though code versus XML, but that's out of scope here… Another thing that does not properly support renaming is data source definition files.

 

I will now give some examples of ReSharper features I found interesting while refactoring my project. 

First example is the nice way ReSharper highlights (light grey) the unused or redundant parts of your code and propose hints and operation to remove it. In this example, ReSharper proposes you to simplify your code by removing a redundant qualifier: 

 

And in the next example, ReSharper proposes an option to clean up the using directives by removing the unused ones. By the way you can automate this cleaning process on the whole code:

 

 

 

ReSharper also highlights potential runtime problems like in next example where you get the hint that you might get an exception from an API call. Up to you then to judge if your need to add error handling code:

 

 

One great feature is that ReSharper allows you to rename namespaces or portion of namespaces. This is something very useful as reorganizing namespaces is a very common refactoring scenario. I have validated that this refactoring works amazingly well, included across multiple projects:

 

 

ReSharper provides an improved way to navigate usage of method, or class:

 

Please note that ReSharper is able to navigate usages in comments! It is also able to rename into comments. So if you rename a class, you can ask ReSharper to rename the name of the class when it is used in comments!

Other example: ReSharper gives you hints to clean your code (more than hints: it is able to do the work for you):

 

 

Improved Intellisence, with here an example of highlighting obsolete methods:

  

A last example: when a reference is missing, ReSharper highlights it for you and gives you the option to add the missing reference. Well Visual Studio does it too. But ReSharper does it a more visible and accessible way that the very little underscore you need to click with Visual Studio:

 

To conclude I was really convinced by the tool which was of great help for my refactoring phase! And I am discovering more and more functionalities every day…

Golden Rule #2: Don’t be afraid of refactoring!

Here I am, back after more than a month publishing the first golden rule. I said to myself today that I should keep up to speed with this subject in order to finally end-up with a good number of rules as I find the exercise interesting and opening great paths for thinking and reflecting on limited but compelling subjects.

So the Golden Rule #2 is about refactoring code and software architectures. One may wonder: "Why are you talking about continuous refactoring as a Golden rule??!! After all if you follow development best practices and apply all golden rules, aren't you then expecting that the design and code produced are of good quality, therefore not requiring refactoring?!"

Introduction - what is all the fuzz around refactoring?

Refactoring is often associated to "repairing" legacy code or poorly designed applications. Even though it is sometimes what refactoring is used for, refactoring is also first class citizen of a full agile process of developing high quality applications. I would even go one step further: if a reasonable sized project is achieved with no refactoring phase during its implementation, then there is probably something wrong and a flawed underlying design that will bring you trouble on the path ahead.

Software development is a complex discipline, and the vast number of patterns, tools, frameworks and best practices leads to numerous ways for implementing one scenario. It is all about pros and cons, compromises, and identifying the potential future points of evolutions of the business need in order to be ready for it. 

Even though it is always recommended to do a design first and coding later, I don't believe one could or should do a fully detailed design before writing the first line of code. The key winning idea is to be pragmatic, agile and iterative. Any design should be validated by proper implementation and validation, to then be adapted, enhanced and implemented again. There is a limit to paper diagrams and even experienced designers cannot fully predict the behaviour of the future implementation of there designs in terms of performance, security, usability and fitting in the target infrastructure, nor can predict that the frameworks and techniques that they theoretically plan on using will actually be relevant perform/provide a service as excepted; one exception is if they already designed and developed a very similar application, following very similar constraints and non-functional requirements, and targeted to a very similar platform. Added to these technical only aspects, business needs can also evolve, during the lifecycle of the project, get re-scoped and adapted and therefore require changes in the application technical design. The consequence of this is that refactoring phases will be mandatory to iteratively build the software solution.

 

Another point is that a successful software architecture and design is the result of a collaborative work between several people, architects, analysts, and developers. So, all participants should be involved in the process as soon as possible thus requiring starting the development phase early for efficient and productive feedback from the development side ASAP. As a consequence, refactoring phases will have to be planed and performed until reaching an acceptable quality level.

So why do you need refactoring?

Including refactoring phases in your development process and lifecycle answers to the following needs:

• You need to make sure that your software design and architecture properly address evolving business needs: Refactoring phases can be necessary if a major business evolution requires rethinking of part of the design of the application. The application should always be designed with flexibility in mind and requires that aspects of potential evolutions are properly identified to know where to design and build flexibility.

• In an agile process, refactoring is an essential technique to iteratively build the most appropriate software architecture and design, while gaining in maturity iteration by iteration. This is a key for agile development where functional slices of the application are frequently delivered and most of the times require partial refactoring to move from one iteration to the other and increase quality and relevance of the design.

• Late discovery of functional or technical constraints can lead to the need for a partial refactoring of the application to address this unexpected event. By the way, an agile, highly iterative process can help reducing the risk by early addressing all the aspects of the application, and especially the aspects that are considered high risk.

• Results of test campaigns can reveals areas that need refactoring, for either technical reasons (poor performance, low scalability, etc.) or functional reasons (inappropriate functional behaviour revealed by the UAT that has an impact on technical design). Again: and agile iterative process will help address early these issues.

Rigorously including refactoring phases in your development process will therefore help you increase the quality and relevance of the overall application design, making the application more maintainable, robust and flexible. Moreover, being able to regularly and efficiently perform refactoring sessions without breaking the whole application and entering uncontrolled regression phases will be the undisputable proof that your software design is effectively of high standard.

 

Can I include refactoring in my development process seamlessly and for free?

Ok, refactoring is not a simple thing and it is definitively not a painless, safe action to perform if you are not prepared for it.

First condition to include regular refactoring phases as part of your development process is to actually have a well designed and architected application. And refactoring becomes a way to iteratively increase the quality of this design. But you can't apply proper refactoring as part of the development process for commando like or any other kind of quick and dirty type of programming.

Second condition is that you should have a proper set of unit tests covering your full code base. Refactoring is about rewriting and restructuring code thus meaning creation of new bugs. And you don't want to break what was working before. Therefore a set of unit test with high code coverage will help you (yet not always suffice) verify the quality of the refactored code and make corrections if required. Important refactoring will also require the re-writing of the unit tests, which is ok: just make sure that your new set of unit tests still covers the same functional use cases.

Refactoring can be painful if you don't have the proper tooling to assist you and automate repetitive operations. And the pain can be even worth if you need to refactor across several projects. Visual Studio includes basic refactoring tools that are really useful and a big step forward since version 2005. But you can go even a step further and use commercial refactoring tools. The one I use (and love!) is ReSharper, and you can read my post about it for more details.

Another thing you need is a refactoring compliant source code control system that allows you to seamlessly move and rename files while keeping the integrity of source code repository and version history. And I haven't found such a tool yet. To be honest I haven't tried too many tools as I am quite happy with the Microsoft Team Foundation Server package for building a full continuous integration environment. Unfortunately, TFS does not quite like renaming and moving files and can act weird when you try to do so. For large refactoring, the solution is to start on a new, clean TFS project that is built on the new file structure implied by your refactoring. You still keep the full version history though the previous TFS project but lose the connection with your new project. Another advice to limit the issues you can have with your source control while refactoring is the following: try to avoid to a too deep hierarchy of projects, trying to mirror the namespace tree. Just save your projects on a flat, or as flat as possible, file structure. You still can have a hierarchical view in Visual Studio by creating solution folders. This way, it will dramatically make moving or reorganising namespaces much easier! Don't forget that reorganisation namespace is one of the most common thing you will do while refactoring...

Last point to mention is that it will be useful to have an automated process for updating documentation. Refactoring implies changes in the documentation and documentation is not a good candidate for manual update because it will then never be updated. Solution for the documentation is the use of tools that automatically reflect on the code comments to generate the detailed documentation. So rule is: comment properly your code and run a tool on your code base. The tool I use is Sandcastle. And for more high level class diagrams, you can choose a UML tool that can reverse code-engineer UML diagrams from you code. Visual Studio embeds one very basic tool. I personally use Enterprise Architect that I will describe in a future post. Key element is that you should be able to automatically update the technical documentation if you want to have technical documentation at all!

 

So am I ready to go?

You must be aware that refactoring cycles are time consuming, require courage and patience for the extra effort induced. It therefore requires discipline and a well accepted and applied process that includes these "technical" refactoring phases, apparently not bringing any additional features or business value, but really mandatory in the medium to long term for creating a robust and flexible applications. Therefore the sole idea of technical refactoring being part of the software application development lifecycle is something that needs to be strongly supported by the project management board, and more importantly: really applied during the project lifetime. And this is probably the most difficult part as management teams can have difficulties understanding the need of spending a lot of time, effort and money for something that apparently (but apparently only) does not bring any added value to their business application. And this difficulty can be caused by a lack of understanding of both parties and a low trust given to others.

This leads us to another Golden rule on which I will write later. This Golden rule is not really a development golden rule but a more high level, project level, strategic golden rule: go for collaborative and responsibility driven team organisation rather than for a hierarchic centric organisation. This rule says the following: to make a project successful (be it a software project or any kind of project BTW), you should build a team with well defined responsibilities and skills and trust everyone in the team for his/her competencies and believe and take into consideration the well-founded of their advice. This is the best and more human way IMHO to have a real success on your projects and long term involvement and efficiency of your overall project team. But more on this later, so keep connected to the Golden rules series!!

I Love .NET Rocks! Or what's great about Podcasts

 

I discovered quite recently the concept of podcasts as a regular consumer, and I must say I am now addicted to it, and I now listen to podcasts all the time: while commuting from and to work, when I go to the gym, when I go running, unfortunately haven't found yet a way to listen while swimming...

I have always been looking to ways to make unused time useful, especially commuting time. I have tried a lot, reading books, looking at videos and webcast on my portative Archos player. And I have found nothing more convenient than audio podcasts! A lot does to it, it is weightless, you can listen in any position, any light condition, and it is so easy to just plug-in your MP3 player when you're back home to get automatic downloads of the new episodes of your favourite podcasts!

I have started my podcast experience by listening to .NET Rocks! Can't even remember the first show I listened but I LOVED it. Since then I have subscribed to other technical podcasts that I regularly listen like Hanselminutes and Software Engineering radio. I have tried once the Polymorphic podcast but was never really convinced. And I must say my favourite remains still .NET Rocks! Great contents, great guests and … real fun! Richard and Carl definitively know how to entertain the audience and make even very technical matters very enjoyable: you can be sure to have several laughs during the listening, which doctors say is very good for your health!

I like Hanselminutes podcasts too, the shows are a bit shorter, making them sometime more convenient to listen if you don't have that much time and don't want to split the listening.

The Software Engineering radio have some good podcasts too. Interesting thing is that this podcast is not specific to any technology, bringing yet another view to some subject (.NET Rocks! and Hanselminutes are specifically shows around Microsoft technologies). However… the shows can be a bit boring in that you will not laugh at all. Gosh, these guys are so serious!! Well anyway some good contents there too, and good content is what is the most important.

By the way, I so like .NET Rocks! shows that I have added a .NET Rocks! tag to my blog to easily link the posts I do sometimes on shows I really liked or want to react on. So check this out here!

So now, after a few months of listening to podcasts, here is my feedback of what I found great in podcasting:

It is definitely a good way to keep informed in an audio format on many aspects of your business or subjects that matter to you: the trends of the markets, technical subjects, business oriented talks, returns on experience etc… It is also a way to hear the voices of big names anyone dedicated to software engineering has probably read articles or books from, and that gives a bigger perspective, other view on the person, and makes you feel as an individual part of a real world made of real people who also can have a laugh while talking about serious, professional matters. Wow, that a long sentence! I guess you get the point: we technical guys indeed tend to read a lot on books or internet, driving us to evolve in a virtual world. And I really fell, the audio, and even more video, gives more life and joy to our business making it more real and lively! You don't feel so lonely anymore and realise that, Hey! There are other freaks out there happening to just love talking for hours about software!!

The other interesting thing about audio/video podcast is that the subjects treated are treated in a very different way than how they would be covered on writen articles. And this is because podcasts are usually interviews and thus are a discussion where a matter is really discussed differently than if one single (or even several) writer would do in a book or magazine: it is more interactive and lively and very open to talks that can easily go beyond the initial subject, thus bringing to the picture yet another perspective, maybe a personal feeling, a real world experience or… just a laugh! You then learn a different way, a more enjoyable way and from a different perspective bringing therefore some added value you would not get another way!

So try by yourself, enjoy the listening, and one or two morning laughs while commuting!

Is the Microsoft Entity Framework more than just an ORM?

I recently listened to the show #369 of .Net Rocks! And read the reaction of Frans Bourma (LLBLGenPro) on the subject.

I wanted to react and give my views on the subject. 

For an overview of what Object Relational Mapping is, please read my previous post on the subject.  

This post assumes that the reader will already have the knowledge of the Microsoft Entity Framework concepts. If it is not the case please read the ADO.NET Entity Framework Overview as a pre-requisite.

Quick introduction to the Microsoft Entity Data Model (EDM)   

The MS Entity Framework is composed of several parts, one of which is the Entity Data Model (EDM). The EDM is used for modelling at a conceptual level the business space, and actually corresponds to the "object space" of the Object/Relational mapping. The EDM is the model describing the set of objects mapped to relational tables on the database side. Then comes the mapping layer, being the bridge or glue between the EDM, or "object side" and the tables in your relational database. Finally, the Object Relational Mapping engine does the trick to manipulate data, persist changes to the database and feed the EDM objects at runtime. An important aspect that will be essential for the main subject of the post is the following: the EDM model is decoupled from the mapping layer and from the ORM engine. In other words the EDM model has no knowledge that there is a mapping and an ORM engine at all, and can be used or re-used without the presence of the mapping and ORM engine. This is an essential pattern for working in a remote/disconnected scenario where a service is responsible for manipulating and persisting data (and thus is using the mapping layer and ORM engine), and the client solely use the data provided by the services, through the form of EDM objects, with no knowledge on how and where the data is stored. Please note that there is nothing conceptually new here and this is the basis of most existing ORM tools of the market, adapted to the Microsoft terminology.

So now that the introduction is made, here comes the main subject of this post.

The Microsoft Entity Framework entering the ORM tools market

For what I've understood from the .Net Rocks podcast, Microsoft could be having more interest on the EDM model than on the ORM engine. You could imagine that this is because they did not manage to catch-up with the existing ORM tools maturity or for a more strategic reason, but that's out of subject here.

Fact is they are many ORM vendors, doing and mastering this job for years and being very mature. The EDM specifically could be on its side more interesting for MS because it is platform oriented and could become a new Microsoft standard, ideal for making applications inter-communicate though this EDM full object oriented, strongly typed object model. Let be clear: the Entity Framework is not a platform, however the Entity Framework and more specifically the EDM model can become an important glue element of the Microsoft platform, becoming the de-facto standard for inter-connecting Microsoft applications and services and any third party application wanting to leverage the EDM opportunity and offer a communication interface to the outer world though EDM entities.

  

A new business opportunity for third party vendors?

What I would extrapolate from Stephen Forte talk in the podcast (and that engages only me) is that there is an opportunity for ORM vendors to build on top of EDM and provide advanced and efficient ORM engine core functionalities: a third party ORM engine replacing the Microsoft one, manipulating and persisting data all through EDM entities. There is also an opportunity for Microsoft Partners to build tools or designers that could assist creating a EDM model and map it to the database + making use of an ORM engine (being the Microsoft one or any other "EDM Compliant" ORM engine). In this scenario, third parties ORM vendors would be using the EDM as the data model used to manipulate, transfer data, between applications and services. Another clarification: the EDM is indeed not a new, revolutionary concept, and every ORM tool in the market is including such a concept: the entities model is indeed the actual object oriented side of the object/relation mapping layer and is an essential part of any ORM tool.

    

Closer look on the EDM on a broader picture

The EDM model becomes a data contract in the WCF sense, making easier and efficient to exchange data between systems, regardless of where the underlying data is stored and whether or not we are using ORM engine to concretely retrieve these EDM entities. For instance you could most probably use and EDM model to represent data in an inter-application communication scenario and not link to any database in the back-end. The EDM and the Entity Framework ORM engine are loose-coupled and you can use one or the other independently. Well at least you can use the EDM independently from the EF ORM engine, I am not too sure of the opposite.

And the EDM is even more than just a data contract: it comes with tools and will probably come with more. One is the Entity SQL, based on SQL syntax oriented toward object queries on the EDM model. And again this is nothing new: for instance Llblgen provides a full framework around its entity model, adding extra functionalities as compared to a basic object model or to any model you could build from scratch by hand, in a reasonable amount of time.

So back to the point: using the EDM as a data contract, it removes the need for applying a DTO pattern for interoperability between systems and especially when exposing services and wanting to make your clients unaware and decoupled from your inner implementation and especially from the ORM tool you are using. Well to be more precise: if the EDM becomes a "Microsoft standard", any one agreeing on that could consume a service delivering EDM entities. But you should be aware that the interoperability will anyway not be complete as using EDM will probably tight you to the Microsoft platform and "Microsoft Entity Framework aware" clients. I will write another post about the DTO pattern in a SOA world and when to use or not use it. DTO is yet another mapping layer and one can wonder why after mapping from the relational world to the object world we still need another translation. My recommendation would be that one should not introduce another mapping layer is there is no absolute need for it. You might need it or might absolutely want to forget about it, depending on the scenario you're in, there is no single solution to all problems.

So is the Entity Data Model much bigger than a just an ORM?

So Yes, for what the EDM is concerned and what you can hear in the podcast as being "more than just ORM", you can achieve the same result with other ORM tool and I am actually doing it on my projects with the LLBLGen Pro generated entities model layer. What however the EDM might provide as a plus in the future is integration in the whole MS suite and become the missing glue link to seamlessly connect systems though the strongly type, object oriented EDM data model. Well let´s be more precise: the missing link in systems based on Microsoft tools and technology.   

So to insist on this point, I can understand Frans Bourma when he is reacting strongly on the sentence "The EF is much more than an ORM" because no, there is no magic and what MS say they can do with the entity framework and the EDM could be achieved with another ORM tool and its own entity model. It is just after all a matter of convention and all actors standardizing on the same framework (and for instance here the EDM part of entity framework). But Yes MS can bring something more with the EDM that goes beyond the ORM aspect, just by the fact that they can push for using the EDM as the data contract layer for all of their products. They can do it because they own the product! And this is what makes the idea powerful: for instance when a company start investing in Microsoft, it usually spread the whole SI and range of applications: start installing MS servers and desktops windows OS and you end up using Exchange server and SharePoint server and office and more and more, rather that competing products. And why? Just because all integrate so well: it is all coming for a single vendor who actually makes the integration of all its range of applications as easy and straight-forward as possible! Same thing can happen with the EDM on a more technical, low level I must say, but following the same driving idea.

      

To conclude …   

To conclude, what I can foresee is that what will bring most to Microsoft strategically speaking is the EDM layer of the Entity Framework. Because this it becomes part of the communication layer, behind the data container for interconnecting the various MS applications. Then how the ORM engine is internally implemented is not of so much importance. And I guess they will be more than happy at Microsoft if a third party ORM engine could be used, and build on top of the EDM and proving very efficient. It will just go in their direction of promoting the entity framework and especially the EDM layer and through the EDM the maturity of their platform and catalogue of products and they will be in a position to market and sell how easy interconnecting them is.