Walter Almeida's Blog

The Golden rules of Software Development series

Last night I thought about an idea for a "post series". I often have coming to my mind quick thoughts about short tips and best practices related to software development. Nothing big, just some general thoughts that could be used as the "tip of day" kind of scenario. I have the tendency to think a lot and the best way for me to release this energy and free my mind is to write down my thoughts and share them. 

 

Good thing about writing down your thoughts and ideas and sharing them is that it forces you to formalize them which often is a more difficult task than first thought but prove to be a very creative activity. I indeed fell and realize that writing down and sharing my own thoughts uses and converts simple, volatile brain energy to fully constructed and usable ideas, and concepts and therefore leverage their power.

 

So I decided upon starting today a series of post called "The golden rules of software development". It will be a series of short posts, about general software development best practices, in random order. First post will for instance be titled "Software Development Golden Rule #1 : Don't write code!"

 

Hope you enjoy the reading and please feel free to comment and react on them!

 

Click here for access to all Software Development Golden Rules available.

Object-Relational Mapping (ORM) - Overview

Today's talk is about object-relational mapping, what it is, why it is an essential part of software applications. I will then specifically talk (in a later post) about the ORM framework/tool I am regularly using on my projects and provide consultancy on to my clients: LlblgenPro

I don't pretend here to be an expert on ORM tools inner details and design complexity, I am posting here as an ORM tool user and express my return on experience on this matter.

     

It's all about data

The vast majority of software applications are dealing with data and data is the core business value of these applications. May you lose or corrupt this data and you are out of business. In some cases, data is an even more critical asset that also needs to be protected against theft and usage by malveillant people.

    

Data is relational

In the vast majority of cases data is stored in relational database systems. Reason for this is first historical: database systems have been developed over years, and evolved to very performing, scalable, reliable, widely used systems. For such a critical asset of any business, this is not something that will change easily, even though object databases have emerged and are used is some cases. Secondly, the relational representation of data along with the commonly used query language (SQL) are well suited to querying and consolidating large amount of data in a concurrent multi-user mode, computing aggregate information, insuring transactional operations and enforcing integrity of data.

     

Code is object oriented

Object oriented development is now widely accepted and used for its well-known benefits that we will not detail here. Through object oriented design and techniques, code is made a close abstraction of real world and business domain concepts. Many design patterns have been identified that help solve common design problems and leverage the power of object oriented design to build robust, flexible solutions.

    

Impedance mismatch and data access layer

Now we can spot the problem/challenge and the mismatch that exists between application code that is object oriented and need to work on data represented and stored in a relational format. And from a separation of concern point of view we want this mismatch to be encapsulated and abstracted from both the database layer and the application code layer. It is indeed a best practice from a software architecture point of view to separate concerns and loose couple the various aspects of a software project (more on this in a future post). In our specific case we want to have an independent relational database layer created, maintained and optimized by a database specialist who does not need to be concerned at all by the way his data will be accessed and used; and at the same time ensure that application developers can use their best creativity and knowledge on how to efficiently build flexible, robust and maintainable application code in an object oriented fashion without having to deal with the technical intricacy of converting back and forth relational data to objects. And this is where we introduce the data access layer, which purpose is to encapsulate all the tricky details and logic needed to perform the bi-directional transformation between relational data and object oriented application data. From now on, we will name object oriented application data after the denomination of business objects or entities.

          

OK, so can't I develop my own data access layer?     

As developers, most of us have one day developed our own data access layer, taking care of communicating with the database, retrieving and exposing data to the business layers and performing the common CRUD (create/update/delete) actions against the database. After all, this at first does not seem like such a big deal does it? But it very quickly appears that writing proper, efficient and robust data access code is really a hell of a work. For instance how about managing primary keys / foreign keys relationship? How about dealing with strongly typed data rather than simple raw data tables? What about server side pagination? What about complex retrieval of data across multiple tables?

Other fact is that the data access code is highly technical plumbing code that is good candidate to be made generic and thus legitimate the use of a common set of tool, framework and runtime libraries to perform the work for us and concentrate on the core business implementation. So here we are: after writing one's own data access layer on a real, reasonably sized project, one will quite fast conclude that leveraging an existing ORM tool is relevant!

     

So tell me about ORM tools!

One important point to note is that a data access layer does not necessarily have to be ORM layer; in other words, a basic data access layer that only provide encapsulation of the database connections, database commands building and requesting data that is then used by the business layer (probably as ADO.NET datasets) is not a proper ORM layer. The ORM layer goes one step further than a basic data access layer by providing (or connecting to, through a mapping configuration) a business entity model layer on top of the actual data access code and most probably advanced functionalities that can include various interesting stuff like object oriented query language (that could now be exposed through the Microsoft Link extension to .NET languages), web service and distributed application support, auditing, database schema change management, multi-databases, support for data-binding etc.

Using an ORM can therefore save an awful lot of work and give you for free many extra functionality that you can potentially use to improve or extend your applications. The only matter then is to select one ORM tool from the many existing.     

You can see from what exists that ORM tools can be of different nature, you will find:

1. Simple entity objects and data access code generator (which I believe are not really fully fledged and powerful ORM tools). Example could be Code Smith templates.

2. Entity object layer code generator + runtime libraries for data access. Example of such a tool is LlblgenPro.

3. Mapping configuration to connect an existing data store to an existing entity object layer + runtime libraries for data access. Example of such a tool is NHibernate

 

Please note that an ORM tool can also be data centric, model centric or both. A data centric ORM tool will need an existing database schema and generate its entity business model from it; a model centric ORM tool will be able to generate a database from an object entity model. And hybrids ORM tool will be able to connect an existing entity object model and existing database through a configuration mapping file. 

I don't personally find the model centric approach and generation of database relevant. The database should IMHO be designed separately. There are techniques to design efficient well performing databases and I don't believe an automatic generation of the schema is appropriate, except maybe for quick proof of concept prototype or very low load applications. Designing an appropriate database schema is too much dependant on the application business domain and the usage that will be done of it, and the load that will be supported by the application.  

I personally find the data centric approach being the best compromise for most applications. The constraint is that your business model should then be quite close to your data model, which I believe in most business application is the case. It's maybe not the case for warehouse databases designed and optimized for reporting purposes, but there are other techniques than business application development to manage their data and more a appropriate scenario would then to use a specialized data reporting server application (like Microsoft SQL Reporting Services). Another case for which the entity data model might not fit the database schema could be very high load databases for which the database schema has been deeply optimized for performance.

I can see too ways to overcome the constraint of having close entity model and database schema and allowing some divergence between the models. One is to use a hybrid ORM tools allowing connection though mapping configuration between an entity object model and a database schema. However what I don't like about this solution is that you have to write your own entity data model where a data centric ORM tool could automatically generate it for you. Other option is to use a data centric ORM tool and add to the database schema database views that could encapsulate and hide for you the differences between your models. Well of course then you have to handle updateable view or use triggers to manage write operations on the view but hey, that would be the subject of another post!

  

So you now know my preferences. Of course every project has its own specificities and constraints and the choice of an ORM strategy must be revised for every new project. But I always try to save myself the most development work I could, and therefore anytime I can I go for a data centric ORM, because of its ability to generate for you both the entity object model and the actual data access layer, saving you a huge development effort! And my favorite ORM tool, of which I will talk about in a future post, is LlblgenPro.

About my Blog

The intent of this post is to describe the main contents and goals of my blog and what I will be posting about.

   

Main Topic and scope

As the title of my Blog states, the main topic covered is "Business Oriented Software Solutions". It's all about delivering fast time to market, good quality software that answers to business needs and result in a return on investment. 

The type of software projects that are in scope of what I call "Business Oriented Software Solutions" are commercial projects, and are not included research & development, educational projects and any other projects for which there is no associated direct business profitable value. However, all posts that are technical and about development methodologies and quality are still applicable to all projects.

Also not included in "Business Oriented Software Solutions" are all RAD projects and disposable software applications. Disposable software applications can be proof of concept prototypes or software application that are intended for a single, on-off purpose, and discarded away after use. "Business Oriented Software Solutions" are software applications that are designed and intended to be used on the long term and will most probably have to evolve and integrate new or changed business needs.   

This Blog being about software development, there will be a great deal of technology talks. Even though part of the posts will be technology agnostic, another part will be specific to Microsoft and Microsoft partners technologies. This is because I made the choice of using Microsoft centric technologies for the implementation of my software projects and it is therefore the technology platform I am expert on and will be posting about.

   

Target audience

This blog will cover all activities of a software development project and therefore is targeted to multiple audiences. Depending on your level of implication in software projects, you might be interested in certain posts only. Click on your profile below to filter posts of most interest to you.

   

Posts for business owners: As a business owner you will be interested in posts that reunite software solutions and business and explain how software solutions can bring your business added value and exactly fit to your business rather than your business be constrained and limited by a software solution. These posts do not require specific technical knowledge and yet give a nice and clear understanding on software and technical matters from a business perspective.

   

Posts for technical project managers: As a technical project manager you will be interested in all technical and process & development practices posts

   

Posts for functional project managers: As a functional project manager you will be interested in process & development methodology posts and business related posts. You will be especially interested to read about how to drive a project in a feature driven way and through highly iterative delivery cycles.

   

Posts for developers: As a developer you will be interested in all technical posts and in the process & development methodology posts that are related to your daily activities. May be of interest for you two to quickly go through all other posts to get an overall view on how you job fit in the total project landscape.

   

Posts for software architects: As a software architect you will be interested in all technical posts and especially these posts related to software architecture, development practices, tools and frameworks and design patterns.

    

Overview  

This blog is about "Business Oriented Software Solutions" or: how to deliver fast time to market, good quality software that answers to business needs and therefore result in a return on investment.  

This could seem quite obvious and not relevant to be the object of a complete blog: after all, who wants to build and late deliver bad quality software that is useless, not driven by any business case and not bringing any added value? Well, yes, one could tell that software development can be an exclusively creative and fulfilling, passionate activity for some individuals, but as stated above, less focus here on software developed as part of a commercial company strategy. And still: as a passionate individual you may don't care of your creation bringing any added business value and keep it useless yet exciting, but you most probably still want to efficiently produce high quality code!

So obvious answer is: NO, of course no-one wants to create these useless, expensive, poor quality software applications, so why this blog?? Thing is that developing and delivering these Business Oriented Solutions can prove to be tricky and the purpose of my Blog is to gather and consolidate information to provide guidance on how to best choose, use, re-use and compose processes, technologies, patterns, frameworks, tools and applications to deliver fast time to market, feature oriented, high quality, robust and performing software solutions.

I believe that the key to efficiently build Business Oriented Software Solution is to concentrate and excel in two complementary directions:

• Define the goals to aim at and set these goals as your priority
• Clearly understand the various areas of expertise you need to cover to achieve your goals and make sure you have skilled individual on board in each of these areas. 

And for our matter, Building Business Oriented Software Solutions is a real big challenge and the difficulty is to succeed in three main goals: create solutions that are business driven, high quality and fast time to market. 

And building Business Oriented Software Solutions involve mastering complementary skills in three areas of expertise: the business domain/functional space, technology & development practices and management & project methodologies.

  

The three goals for success

One can list more goals to aim at to successfully deliver Business Oriented Software Solutions, and this list can depend on the specificity of a project and/or business domain. However I believe than the main three goals to achieve that are essential and drive success in most cases are to build solutions that are:  

• Business driven/feature driven: solutions answering a real business need and seamlessly adapting to the business changes and evolutions.
• High quality: solutions technically properly designed and implemented making the solution robust, performing, maintainable and seamlessly welcoming evolutions and new features.
• Fast time to market: solution that have the ability to be delivered or more be iteratively delivered as fast as possible, in order to compete efficiently on the market place.

     

The difficulty comes from the interdependency between these three targets. It is for instance easy to understand that reaching fast time to market result while keeping a high level of quality is not so easy to achieve. This is part of the reason why it's common place to have software that are early on the market place but of poor quality, or software that are of good quality, delivered on time but not aligned with the business requirements and priorities, or even worth: software that is not answering the business initial needs, of poor quality due to the pressure involved by fast time to market constraint and delivered very late as a consequence of the two first problems.

The posts will be written with this in mind and will be oriented toward solutions and guidance to succeed in meeting the three targets.

     

The three areas of expertise involved

Various disciplines are to be conducted as part of as software project and these disciplines require different set of skills and abilities that we can distribute in three areas of expertise:  

• Business domain & Functional space: Know and understand the functional space and business domain that is yours. Should not be constrained by technology or methodologies, as business need is the driving force.
• Technology & development practices: Excel in efficiently using technology and composing the right frameworks, tools and patterns to answer a business need in a robust and flexible way.
• Management and project methodologies: Master appropriate management and project methodologies to efficiently and iteratively delivery software with accordance to business needs and technology constraints.

   

Please note that in a software project there are more disciplines than areas of expertise. If we take RUP as an example, the disciplines are: business modeling, requirements, analysis & design, implementation, test, deployment, configuration & change management, project management and environment. RUP is the IBM iterative software development process (http://en.wikipedia.org/wiki/IBM_Rational_Unified_Process) .

You need to be aware that no matter how small or big is your team you will have to master skills in all these three areas of expertise. And the challenge in succeeding in all three areas lies in the inherent difficulty of cross-cutting these three distinct aspects of a software development project that require complete different abilities and most usually different persons who need to communicate and understand each-other and each-other concerns.

      

Outcome of the posts  

Each of the posts that are relevant to creating Business Oriented Software Solutions will be started with tables indicating what goals the discussed topic can help achieve and what area of expertise is concerned and the posts will be tagged accordingly. This way the reader will have a mapping between goals and areas of expertise and a set of relevant posts, ideas, and guidance to succeed in efficiently building Business Oriented Software Solutions.

So here we start, I will now start blogging, I hope you will enjoy the ride!