Thursday, 3 October 2013

Executing Razor Views outside of a web application with NonHttpRunTimeRazorSupport

NonHttpRunTimeRazorSupport (catchy) is a small library that allows you to use ASP.Net MVC Razor views to render content outside of the context of a running ASP.Net application  with full support for Layouts, _ViewStart inclusion, Partials, built-in HtmlHelpers etc.


Introduction

Razor views are a really nice mechanism for generating web page content within an ASP.Net MVC application. They're so great in fact, that it's a shame to use them just to render HTML to display in a browser. What if we could use them to generate other content within our application, like email messages?

Many applications need to send emails. For something simple, like a one-off message to a site administrator, you might be able to get away with using a StringBuilder to generate the content of the email. If you need to send a range of rich HTML emails and want developers to be able to modify them easily, a full-blown template system like Razor is a much more flexible and maintainable mechanism.

Another common requirement is to send emails in an "offline" scenario. By "offline", I mean outside of the context of an HTTP request in a running ASP.Net application. Maybe you want to send emails within a console app, Windows service or an NServiceBus / MassTransit / Rebus message handler.

NonHttpRunTimeRazorSupport lets you use all of Razor's features outside of a running web app.

There are loads of other things you could use this library for - but email is probably one of the most common requirements it supports.

How it Works

The library is on GitHub at https://github.com/danmalcolm/NonHttpRunTimeRazorSupport. If you know about git, go ahead and clone the repo on your development machine, otherwise you can download a zip file of the source or browse the code directly on GitHub.

This solution uses RazorGenerator, a great tool that allows you to pre-compile Razor views at design time. As you edit your Razor views in Visual Studio, a pre-compiled view (an accompanying C# class file a bit like a web forms Something.aspx.Designer.cs file). Pre-compiled views can be used instead of .cshtml files that are compiled on-the-fly by RazorViewEngine. One thing that the RazorGenerator project is missing is an example of how to render pre-compiled Razor views in the "offline" context we introduced above*.

Razor View templates, whether compiled at design-time by RazorGenerator or at runtime by RazorViewEngine, are instances of WebViewPage. Views of this type are designed to execute in the context of an HTTP request. The standard built-in RazorViewEngine assumes a web request and other supporting infrastructure offered by HttpRuntime to be present when locating and rendering Razor views. 

The main purpose of NonHttpRunTimeRazorSupport is to work around this limitation. A customised ViewEngine and fake implementations of HttpContext (and it's HttpRequest, HttpResponse members) are used to support the execution of Razor Views.

Needless to say, you have to write your templates with this "offline" context in mind. Putting something like @HttpContext.Current.Request.ServerVariables[...] in the middle of your template would result in an exception. Note that the library attempts to provide help here by throwing an exception with an informative message if  any non-supported HttpContext (and family) members are accessed.

Getting Started

See the README for details on how to get up and running.

The repository includes a simple EmailDemo console app, that demonstrates how you might handle sending emails in your application.

The EmailNotificationService class uses Razor templates to render email content, using a convention to locate views based on the name of the model type. Here's how the views are structured:


This foundation would allow you to support a range of emails without adding a great deal of code. For each message type, you would simply need to add a model and Razor templates to render the subject, text and html body content. Here's the code for EmailNotificationService in full:

Look at EmailNotificationService.cs on github if you can't see the code in-line above.

Alternatives

There are other libraries that allow you to use Razor templates to generate email content, such as MvcMailer and ActionMailer, but, they don't fully support sending Razor-templated emails in an "offline" scenario ("Email sending from a background process" is still listed as an upcoming feature on the MvcMailer Wiki page and the standalone version of ActionMailer doesn't appear to support layouts and other Razor features.

What Next?

I'll consider adding a NuGet package once I've got feedback from using this approach on some other projects. Any comments or feedback would be great.

I came up with this solution while working on a project that was already using RazorGenerator. I'm interested in exploring whether it would be possible to bypass RazorGenerator and compile the views on the fly. Compiling Razor views from templates stored in a data store could also be interesting....

Happy mailing!

* I should point out that views compiled with the RazorGenerator "Template" generator can run outside of the context of an ASP.Net application. They are not the same thing as standard MVC Razor views - the declarative Layout property, _ViewStart pages, partials, HtmlHelper / UrlHelper etc are not available.


Saturday, 21 September 2013

ControllerPathViewEngine - Structuring View Folders based on Controller Namespace Hierarchy in ASP.Net MVC

Introducing ControllerPathViewEngine

ControllerPathViewEngine is an ASP.Net MVC ViewEngine that allows you to structure your views in folders that match the namespaces of your controller, like this:



If you know about git, go ahead and clone the repository on your development machine, otherwise you can download a zip file of the source. If you just fancy a look around, you can browse the code directly on GitHub.

The Problem

One thing that has always annoyed me about ASP.Net MVC is the flat Views folder structure.

When an ASP.Net MVC controller action renders a view, the built-in view engines will, by default, expect your Views folder to be structured based on the following pattern:

Views/{controller name}/{action name}.{extension}

There's no support for structuring views in folders that relate to the namespaces of their related controllers.

In a simple project with a handful of controllers, this probably won't be a problem. However, on a larger project with a lot of controller code, you'll probably find yourself organising your controllers within a deeper namespace hierarchy.

A nested hierarchy makes a complex application manageable by providing an isolated context to the things it contains, keeping things focussed and easier to name. With the flat view structure we lose some of this context and it becomes difficult to relate view to controller.

And don't even think about having more than one controller with the same name! A flat views folder structure is going to lead to naming clashes, unrelated views sharing the same folder and all sorts of shenannigans.

ASP.Net MVC Areas could alleviate this to some extent by allowing you to group and manage related controllers, views and their related routes in a more modular fashion. They don't really solve the core problem though and using an area only adds a single level to your Views folder structure, based on the following default pattern:

Areas/{area name}/Views/{controller name}/{action name}.{extension}

The Solution

Anyway, that's enough moaning justification - writing the above has taken longer than it took to write the code!

A quick look at the ASP.Net MVC source code led me to an extremely simple solution. We simply intercept the call to FindView and FindPartialView and cheekily switch the folder path for the controller name during the call to the base implementation. This allows us to leverage the core caching and view finding functionality.

The project's README contains some background on implementation and the logic used to drive the view folder structure. See the code itself and unit tests to learn more about the implementation.

Alternatives

I had a quick look around at some other approaches, including this one. I had a couple of problems with this implementation however (in fairness to the writer of the post, the main intention was to demo view engine extensibility). Firstly, the cheesy %1 placeholder used in the view location format strings shows up in the exception thrown when a view cannot be located, which doesn't really help the developer work out which view file is missing. Secondly, I'm not sure that it would support the caching functionality that VirtualPathProviderViewEngine uses to optimise view retrieval when dealing with controllers with the name name (a bit of an edge-case but there's potential for a big WTAF there).

Hope you find this useful. Comments and suggestions are welcome as usual.










Friday, 26 July 2013

Searching all local drives for files using Windows Powershell

There's often a "forensic" side to software development. Unfortunately source control and continuous integration aren't going to be in place on all projects that you work on and, from time to time, you'll need to track down some elusive source code. I remember once helping to track down the source code for an important project on an old hard drive in the company owner's garage!

At the moment, I'm helping to pin down local copies of source code on some developers' PCs and needed a quick way of searching for all Visual Studio solutions on their hard drives. The following Powershell script searches recursively through all folders on all local drives for files with the ".sln" extension and outputs to a file.

Look at the script on github if you can't see the code in-line above.

The main challenge was getting the list of available drives. "Get-PSDrive -PSProvider FileSystem" will get you all local drives, but includes CD / DVD drives which might not be available. Filtering this list further using "Where-Object -Property Free" will restrict the list to available drives.

Wednesday, 19 June 2013

Using Razor to Customise the Templates Used by Html.Display, Html.DisplayFor, HtmlEditor and Html.EditorFor


I recently added a new public repository on GitHub that might be useful to ASP.Net MVC developers:

https://github.com/danmalcolm/mvc-razor-display-and-editor-templates

Essentially, this project provides Razor (C#) alternatives for all of the built-in templates used by the Html.Display / Html.DisplayFor and Html.Editor / Html.EditorFor extension methods.

Writing your own templates isn't that hard but if you wanted to introduce changes across all of the templates in the framework, you'd have quite a lot of templates to write from scratch and there's a bit of subtlety here and there in how they work.

I hope that this project provides a useful starting point for developers wanting to introduce changes to templates within their application.

Some examples of things you can do once you have the templates in place:
  • Change HTML in line with your project's CSS rules
  • Introduce visual effects, such as graphics / words for displaying boolean values etc.
  • Introduce parameterised variations using additionalViewData parameters, e.g. @Html.Editor("EnableWeapons", new { icon = "warning" })
  • Change surrounding HTML generated by Html.DisplayForModel to support your project's CSS layout framework
Sorry, I forgot to mention the date-pickers! You'll be able to add date pickers to your date editors.

In line with the built-in ASP.Net MVC editor templates, HTML5 input elements are generated for some data types.

There's a full write up in the README visible on the link above.

I've also added some additional extension methods. You can now use Html.EditorSection and
Html.DisplaySection to display individual properties of your model together with some surrounding HTML, labels and (for editors) validation messages. There's another post on the way with more about this.

Sunday, 21 April 2013

Testing your Database with TinyNH, the Simple NHibernate Infrastructure

TinyNH is a demo project and series of articles designed to help you get your project up and running with a simple solid NHibernate persistence infrastructure. See contents for a full list of articles.

Introduction

So far, you've:
  • Created your domain model entity classes
  • Added a ConfigurationBuilder to initialise your NHibernate Configuration and class mappings
  • Added a ConfigurationStore as an application-wide point of access for your Configuration and ISessionFactory
You'll now be thinking about checking whether everything you've done actually works. Is NHibernate set up correctly for your target database environment? Are all of your properties and relationships in your model mapped so that your entities can be loaded and saved as expected?

One way to do this would be to:
  1. Write some application code
  2. Manually test that you can load and save your entities
  3. Fix any issues that come up
  4. Restart the application and try again
  5. Repeat steps 1-4 until everything works properly. 
All very well, but quite a burden, especially on a larger project, and it's a process that you'd need to repeat often if making changes to your model.

Automated Database Testing

TinyNH demonstrates how to write automated tests to validate your NHibernate set up and persistence infrastructure.

End-to-end database tests interact with, like, a real database in exactly the same way as your application does and can drastically reduce the amount of manual testing you'll need to do. They provide rapid feedback after you have made changes to your entity classes, tweaked your NHibernate mappings or upgraded NHibernate and any related components.

Database Test Examples 

You can review the database tests in the TinyNH.DemoStore.Tests project:


Here’s an example of a test used to validate the saving of Category entities to the database:

Look at CategoryPersistenceTests.cs on github if you can't see the code in-line above.

The test essentially "round-trips" a Category entity object to and from the database, comparing the original saved object with a different loaded version. You'll notice that several assertions are used to validate that each property has been saved and loaded correctly. This could become quite tedious for entities with lots of properties. The invention of a reliable mechanism for comparing the structure of 2 objects has been left as a creative exercise for the reader for now - a demonstration of some techniques may be added to TinyNH in the future.

We also include some very basic validation of our configuration in ConfigurationTests.cs. This is quite a coarse test that tells us if the Configuration and ISessionFactory are created correctly. This is a great way to flag up any errors with your configuration and entity classes when first developing your model, such as missing empty constructors or virtual modifiers on public members. Running this test is a lot quicker than hitting F5 and waiting for all of the other parts of your application to initialise in addition to NHibernate, only to find that you've got one more non-virtual property to fix.

Database Testing Infrastructure

The DatabaseTests base class provides some useful shared set up functionality to our database test classes. It does two useful things:
  • Database setup - A dedicated test database is automatically created for use by our tests
  • Data cleanup - Unwanted data is deleted before each test runs, allowing tests to begin with the database in a consistent state.
Let's look at the database and schema set up first, which happens once per test run:

Look at the static DatabaseTests constructor in DatabaseTests.cs on github if you can't see the code in-line above.

A ConfigurationStore is used to initialise and make available a shared Configuration and ISessionFactory throughout the test run. Initialising these objects takes a bit of time and using a ConfigurationStore benefits our tests in the same way as it would a regular application.

We also make use of ConfigurationStore's "configurationReady" hook to trigger creation of the database after the Configuration has been initialised. The SetupDatabase method first drops and creates the database that NHibernate will connect to when running the tests. The DatabaseSetUpHelper.cs class is used to manage database creation. It works by using a special "setup" connection string to connect to the master database on the server where the tests are running, then runs some SQL to drop and recreate the database.  We then apply the schema to this new empty database. These tests demonstrate a code-first approach, where the database schema is generated from our domain model and NHibernate mappings, using the NHibernate SchemaExport tool. Below, we discuss alternatives for setting up the database schema.

With the database in place, we also need to look at cleaning up the data that is generated by our tests. The ResetDatabase method is called before each individual test runs, so that each test starts with the database is in a known clean state.

Look at the SetUp and ResetDatabase methods in DatabaseTests.cs on github if you can't see the code in-line above.

As you can see, tables are cleared using a simple direct SQL script. See Using a SQL script to reset data in database integration tests for further background on this approach and Generating a script in SQL server to delete data from your database for an interesting way of generating a reset script for a larger database.

Not bad for a 74 line base class!

Code-First or Database First

There are 2 approaches to domain model design and schema creation.
  • Code-first - You design your domain model entities as C# classes, then generate the database schema needed to support these entities.
  • Database-first - You prefer to design your entities as database tables, then think about writing the classes that map to them.
A code-first approach works really well on greenfield projects and feels like a purer OO approach but it really comes down to personal preference. A hybrid approach is best: code first, but don’t forget about the database! Developers who understand both good OO design practices and the mechanics and limitations of relational databases will probably make domain models that work best in the real world.

TinyNH demonstrates a code-first scenario - in that it demonstrates how you can generate a database schema from domain model entities you have designed as C# classes. 

If you are working database-first and / or have an existing database schema to work to, you can replace the SchemaExport mechanism with an alternative means of setting up the structure of your database.

Generated vs Versioned Database Schemas

If you develop your model in a code-first fashion, automatically generating your database schema at runtime is a useful short cut that will serve you well during the earlier stages of a project's development.

However, there will be a point where you need to bring your database creation under version control, ideally using a database migration tool to apply a sequence of change scripts. At this point, both your application and tests should be running against a schema that has been created from version controlled database migration scripts, not a schema generated at runtime using your ORM! 

Unfortunately, this is outside of the scope of TinyNH at present. If you are familiar with evolutionary database design and migration scripts, you would probably find it simple to swap out the SchemaExport mechanism used here.

Do you need database tests?

Hopefully, the above demonstration has shown that automated database testing needn't be difficult or complex. It is a practice that would benefit any project.

Here are some guidelines to help you decide whether you want to include database tests in your application:
  • Familiarity / Expertise - If you’re learning NHibernate at the moment, aren’t familiar with automated testing tools and practices and are worried that your head might explode, you may wish to put test development on hold and write something dirt-simple to get you started. A console application that loads and saves your objects and writes to the console is a good way to start.
  • Model Complexity - If you have a larger model with some complex mappings, tests are an extremely useful way to check that you have mapped your model correctly. If you have a simple model that is easy to manually test within your application, then you might prefer not to bother.
  • Project Lifetime - If your application is an ongoing project that is likely to evolve over time, you’ll get a better pay-off from the time invested in setting up tests. You might be happy that manual testing is enough for a simpler throw-away demo or prototype project.
  • Test Database Environment - Automated database tests are harder to write if you have restricted access to your test database, if a test environment isn't permanently available, if you don’t have control over the data, or if it is shared by multiple users. 

Testing Framework and Structure

TinyNH uses NUnit, a widely used unit testing framework. You can adapt the tests to your preferred framework if you wish.

There is a single Tests project in the TinyNH solution. This contains an XXXTests.cs file for each class under test, with the test classes being located in a parallel namespace hierarchy within the Tests project. For example, TinyNH.DemoStore.Tests\Core\Domain\EntityTests.cs is used to test TinyNH.DemoStore.Core\Domain\Entity.cs.

We have Category attribute on any tests that access the database. Database tests are a lot slower than “textbook” unit tests, which run entirely in-memory and do not access databases, network, file system etc. When running your tests, you may prefer to run the faster unit tests first.

Tests for Other TinyNH Elements

TinyNH isn't a separate framework that you make use of by referencing a dll, it’s a template project containing elements that you copy into your own project. You’ll notice that TinyNH contains tests for these elements, such as the base Entity class, ConfigurationBuilder and ConfigurationStore. If you copy Entity.cs and ConfigurationBuilder.cs into your project, you may be wondering whether you should copy over EntityTests.cs and ConfigurationTests.cs too.

If you already have tests in place for other parts of your project, then you may wish to copy all of the tests over from TinyNH tests across so that any code you've added from TinyNH is covered by tests.

How to do it

If you don't already have a test project in your solution, add a Tests class library project to your solution

Add a reference to NUnit (or preferred testing framework).

Add the DatabaseSetupHelper.cs class to a suitable location within your solution. This isn't used only for testing so it needs to be available to application code as well as test code. Maybe you have somewhere within your project for shared utility functionality.

Establish the correct namespace within your test project for your database tests and add the DatabaseTests.cs base class. Modify the resetSql constant within the ResetDatabase method in line with your database schema.

Add ConfigurationTests.cs to the same namespace as DatabaseTests.cs.

Create some end-to-end load-and-save test for the relevant entities in your solution, based on examples in ProductPersistenceTests.cs.

Managing Configuration and ISessionFactory Access with TinyNH, the Simple NHibernate Infrastructure

TinyNH is a demo project and series of articles designed to help you get your project up and running with a simple solid NHibernate persistence infrastructure. See contents for a full list of articles.

Introduction

So far, we've created some domain model entity classes and added a ConfigurationBuilder class to configure NHibernate.

To start working with your objects, you first need to create your Configuration, then use it to build an ISessionFactory. In a really simple application, like the TinyNH ProductImporter console application, a couple of lines of code is sufficient:

Look at ProductImporter/Program.cs on github if you can't see the code in-line above.

However, there are a number of important points to consider if you are using NHibernate in the context of a web application or any other long-running service:
  • Configuration and ISessionFactory instances are expensive to create and should only be created once during the lifetime of your application.
  • We need to ensure that Configuration and ISessionFactory initialisation happens only once and in a thread-safe manner, even in a multi-threaded environment such as a web application.
  • Initialisation can fail due to database outages. Specifically Configuration.BuildSessionFactory will fail if the application cannot connect to the database. Do you want your web site to become unavailable, just because the database happened to be down for a couple of seconds, coinciding with when your application started up? 

The TinyNH ConfigurationStore is designed to take care of the above concerns.

Here it is:

Look at ConfigurationStore.cs on github if you can't see the code in-line above.

ThreadSafeInitializer is a utility class that is used to initialise the Configuration and ISessionFactory in a thread-safe manner. It is similar to System.Lazy but it will retry initialisation if a previous attempt fails - see my article about System.Lazy's exception caching features for background.

How to do it

If you need it, add this functionality to your solution as follows:

  1. Add the ConfigurationStore.cs class to your solution, within the same namespace as the ConfigurationBuilder class described in the previous article.
  2. Add ThreadSafeInitializer.cs to a suitable location in your solution. You may have a "slot" used for common utility classes or you could add it to the same namespace as ConfigurationStore.
  3. Add some code that runs during start-up of your application to initialise your ConfigurationStore.

In our example web application (TinyNH.DemoStore.Admin project) we do this in Global.asax. In keeping with our super-simple approach, we make the ConfigurationStore available as a public property of our Global application class.

Look at Global.asax.cs on github if you can't see the code in-line above.

You might shudder at the thought of accessing dependencies via static properties and prefer to use an IoC container to manage dependencies between objects. The ConfigurationStore is still a useful mechanism to ensure thread-safe initialisation of your Configuration / ISessionFactory during application start-up and can be used to provide the instances to your IoC container.

Configuring NHibernate with TinyNH, the Simple NHibernate Infrastructure

TinyNH is a demo project and series of articles designed to help you get your project up and running with a simple solid NHibernate persistence infrastructure. See contents for a full list of articles.

Introduction

The previous article covered the implementation of your domain model entity classes. The next thing that you need to do is give NHibernate the information it needs about your object model and tell it how to map this from the database. To do this, we need to initialise a Configuration object.

Where Does NHibernate-specific Code Belong?

You first need to make a decision about where you want to locate the NHibernate-specific code in your solution. In TinyNH, we keep our project structure simple. Our entity classes live in the TinyNH.DemoStore.Core.Domain namespace and our NHibernate code is located in the “child” NHibernate namespace TinyNH.DemoStore.Core.Domain.NHibernate:


It is, of course, possible to use a separate project if you don't like the idea of your entities “knowing” about NHibernate.

ConfigurationBuilder

In keeping with  the Single Responsibility Principle, TinyNH uses a dedicated ConfigurationBuilder class, which creates the NHibernate Configuration. It’s just a simple class, which you can see below:


Please look at ConfigurationBuilder on github if you can't see the code in-line above.

There are two things to explore further here: a preference for configuration by code and a mechanism for overriding default configuration properties.

Prefer Configuration by Code instead of Configuration Files

There are a range of configuration properties available in NHibernate and, traditionally, these have been stored in configuration XML files or a section within your app.config / web.config (NHibernate originates from the Java world, I think somebody once wrote that Java is a DSL used to convert XML files into stack traces).

Configuration files are useful if you need to be able to change aspects of the configuration at runtime, for example if your application ran on a range of database platforms, then you would need to introduce configuration files to configure the “driver” and dialect “properties” for each deployment.

However, if you don’t need this flexibility, there are benefits to pushing as much NHibernate configuration as possible into code. In TinyNH, all configuration is done by code within our ConfigurationBuilder class and the only thing that is configured from the “outside” is the connection string. If we do it this way, we don’t need separate NHibernate configuration files, or a separate configuration section in our web.config or app.config file. This can be awkward to maintain if you have multiple applications in your solution. It also ties you down to a "static" configuration file and makes your code less testable.

Configuration Overrides

There are scenarios where we might want to make small changes to our standard NHibernate Configuration. One example is enabling certain features for testing and diagnostic purposes, such as asking NHibernate to maintain statistics about database activity or to log all SQL statements it executes using nice readable indented SQL.

Using a single ConfigurationBuilder and performing all configuration by code needn't restrict us when we do need to vary aspects of our configuration. ConfigurationBuilder's optional "customize" constructor parameter allows the NHibernate configuration to be customised on a per-application basis. For example, when we create a Configuration object for use in our automated tests, we enable a couple of useful features that we wouldn't want to have running as standard in a production environment:

Please look at DatabaseTests.cs on github if you can't see the code in-line above.

Class Mapping Options

TinyNH's main goal is to help you to put things in the right place. It doesn't prescribe any single approach to configuring your mappings.

For simpler projects, NHibernate’s built-in mapping by code API provides you with a nice concise way to configure your mappings. The ConfigurationBuilder in TinyNH sets up mappings using this mechanism, via a separate ModelMapping class. The properties of our entity classes are configured explicitly, with a few conventions handled by the BeforeMap[something] extension points.

How to do it

It's time to add the NHiberate configuration functionality to your solution:

  1. Copy ConfigurationBuilder.cs and ModelMapping.cs from TinyNH to a suitable location and adjust the namespaces as appropriate.
  2. Modify the configuration and mapping in line with your preferred approach to mapping and the structure of your domain model entities. 
Remember to start simple and consider mapping a single entity class to begin with.


Final Thoughts

You might be feeling a little short-changed here. You'll need to do most of the thinking around setting up your configuration and mapping yourself, all this article has done is told you where to put it! However, having a clearly identifiable "home" for your NHibernate configuration that isn't embedded deep within your application start-up code, or tied to specifically named configuration files has a number of useful benefits, which we'll explore in the future.

Creating your Domain Model with TinyNH, a Simple NHibernate Infrastructure

TinyNH is a demo project and series of articles designed to help you get your project up and running with a simple solid NHibernate persistence infrastructure. See contents for a full list of articles.

Introduction

The raison-d'ĂȘtre of our infrastructure is to load and save instances of our objects from and to the database, so let’s start with our domain model.

Background

If you are using an ORM, then you are using a domain model architecture. Your domain model consists of classes that capture the data and behaviour of "things" in your business domain.

The "things" that you model within a domain are commonly described as “entities” and we'll be using this term throughout these articles. To be more precise, the term "entity" is being used here to describe "things" in our business domain that that have some concept of identity and ongoing life-cycle.

Our Model

TinyNH uses a simplified e-commerce catalogue model, containing Product, Category and Supplier entity classes.


We've chosen to locate our entity classes within the Domain namespace of a TinyNh.DemoStore.Core C# class library project. This project contains core functionality that is shared by 2 applications in our solution: the catalogue admin web application and a product importer console application. For a simpler solution, there’s no reason why your solution couldn't consist of, say, a single ASP.Net web application containing controllers, views etc, along with your domain model.

Here's the Product class:

Look at Product.cs on github if you can't see the code in-line above.

I said it was simple... TinyNH’s focus is on “where to put things” when setting up NHibernate for your model, not on the design of your entities or the specifics of your NHibernate mappings. Our entities don't have much in the way of behaviour either. For more about developing behavioural domain models, Eric Evans' Domain Driven Design book and the related discussion group are excellent resources.

Entity Base Class

Domain model entities usually have some shared characteristics that can be extracted to a common base class. TinyNH includes the Entity abstract base class that is inherited by all entity classes in our domain model. Here's the code in full:

Look at Entity.cs on github if you can't see the code in-line above.

This class provides a common Id property and an Equals / GetHashCode implementation.

Id Property

Our Product, Category and Suppliers are root entities - they exist independently, have their own identity and can be queried directly from the database. As such, they need an identity field. On a new project, where you have full control over the database schema, it is a common practice to use surrogate key as an identifier, an an arbitrary “meaningless” value generated by your application / database, rather than a real-life property from the application domain.

We have chosen the type System.Guid for our Id property. We will configure NHibernate to generate our Id values using the Guid.Comb strategy. Guid.Comb is a "special kind of GUID", that performs well when used as a primary key in a relational database (see Jimmy Nilsson's article for background).

If you really don’t like GUIDs (do users of your application get cross when you ask them to read out GUID values over the phone?), you can use ints for your Ids instead and get either NHibernate or the database to supply the values as objects are saved.

Equals Implementation

The default implementation of Equals for reference types is based on reference equality. 2 different Product instances with the same Id value would not be equal if compared using object.Equals method. Within the scope of your application, you would probably consider these two entity object instances to be logically equal, i.e., they are 2 different objects materialised from the same database record and represent the same entity in your business domain.

Our entity base class overrides the Equals method to compares objects based on their identity. If two objects are of the same type and have the same Id value, then they are considered equal. There are a few issues to consider around NHibernate-generated proxies and unsaved objects - see the comments in the Equals override for full details. Overrides of the == and != operators have not been implemented, it is not recommended for mutable reference types.

Note that if you have 2 unsaved objects (with no Id value to act as an identifier), there is no attempt to determine whether they might still be logically equal. If you can think of scenarios where this might introduce a problem, then you need to read up on a more involved approach to equals and gethashcode implementation. See Sharp Architecture’s entity base class for another example, then read this detailed discussion.

But before you do, here’s a test: Can you suggest a scenario within your application where you need business key equality to support identification of 2 different unsaved objects that represent the same logical entity? Don’t introduce this requirement into your project if you don’t need it.

GetHashCode Implementation

Because all objects that are equal should have the same hash code, our entity’s hash code is generated using a combination of the entity's type and value of its Id property.

It is important to be aware that the hash code returned by Entity.GetHashCode will change when an entity is saved and given an Id value (from this point onwards, the hash code will be stable however, so there's light at the end of the tunnel during these trying times...). Therefore, we should bear in mind the following rule from Eric Lippert's Guidelines and rules for GetHashCode: "the integer returned by GetHashCode must never change while the object is contained in a data structure that depends on the hash code remaining stable".

This means that unsaved entities should probably not be used as a key in a hashtable if they are going to be saved at some point while the hashtable is in use. In practice.

Note that the complications around entity equality and hash codes are not specific to NHibernate. A correct implementation is important for domain model entities regardless of how they are saved to a data store. You'll get very funny results when using LINQ operators like Distinct if you don't implement it.

How to do it

It's time to add some code to your project.

1. Choose a suitable place in your solution for your domain model entities.

2. Copy and paste the Entity.cs class from TinyNH into your project and modify the namespace accordingly.

3. Start adding your own entities. Note that you’ll definitely feel more “in control” if you first get your NHibernate infrastructure running end-to-end with the simplest possible scenario to start with, so consider starting off with a single entity to start with.


Introduction to TinyNH, A Simple NHibernate Infrastructure

TinyNH is a demo project and series of articles designed to help you get your project up and running with a simple solid NHibernate persistence infrastructure. See contents for a full list of articles.

It isn't a framework, it's more like a template project that demonstrates some key infrastructural elements that you're likely to need. Each element is accompanied by an article (on this blog) that describes what it does and why, helps you to decide whether or not you need it and, if so, tells you how to add it to your project.

Why?

You’re working on your latest project and, believe it or not, you have a few things stored in a relational database that your users need to view, edit and save. You’re weighing up the options available for handling data access.

You briefly consider hand-coding a data access layer using raw ADO.Net. Although there are some advantages to keeping things really simple, it’s hard to justify this approach, given your experience with practices and frameworks that will get the job done a lot quicker.

You then look at micro ORMs, like Dapper.Net and Massive, which offer a lightweight, efficient and fast mechanism for mapping database records to objects (nice overview here). Dapper Extensions even adds support for inserts and updates. However, your project has a reasonably complex object model. It’s small, but it’s a real domain model nevertheless, with a few associations between objects and a collection or two. Dapper.Net would do the work of generating the inserts and updates, but you’d have to write some fairly fiddly code to ensure that parents, child objects and items in collection were saved in the correct order.

You really want a solution that can handle the complexity of efficiently loading and saving a graph of connected objects. Which leads you to a full-featured ORM, such as Entity Framework or NHibernate.

NHibernate is your weapon of choice when it comes to full-on ORM action. A quick “install-package NHibernate” and you have NHibernate referenced, but what next? You realise that, in addition to your object mappings, you have a fair bit of infrastructure code to write to get things up and running, things like:
  • Initialising the configuration
  • Setting up the configuration and session factory in a thread-safe manner
  • Testing your configuration and mappings
  • Managing sessions and transactions
Using NHibernate to manage a handful of classes is starting to feel like a bigger step than it should. Many projects you've worked on have accreted layers of abstraction and other infrastructure around NHibernate that seems over-the-top for a simpler model. You’re determined to keep things simple and start picking out the bare minimum you need for your project, assembling pieces from various projects, documentation and blog articles. You feel like you're having to think a little harder about this than you should...

Meanwhile, another team is asking for your advice in a similar scenario. They're familiar with NHibernate, but, given that you're having a "where do I start" moment yourself, you know that they'd benefit from a coherent end-to-end guide to getting their project up and running.

Overview

The "you" above was me. What I have tried to do with TinyNH is to identify the core infrastructural elements that you need when working with NHibernate and distil this into several small (tiny!) pieces of infrastructure code that you can add directly to your project. This includes:
  • Model - Creating your domain model
  • Configuration - A flexible approach to NHibernate configuration and mapping
  • Start-up - A mechanism to ensure thread-safe NHibernate initialisation
  • Context sessions - Efficient NHibernate session management in a web application
  • Automated Testing - Test your database without hitting F5!
  • Developer friendly database set up - Automated local database set up for each developer.
TinyNH is not a library or framework. It is a guidance project that provides a demo of some infrastructure code that you’ll need to add to your own solution using some traditional copying and pasting. 

Keeping Things Simple

NHibernate and Entity Framework have a reputation for being "overly complex" or "heavy and complicated". There is a feeling that the investment required to work effectively with an ORM might not pay off on simpler projects.

The development of TinyNH was motivated by the belief that a full-blown ORM can be used successfully on simpler projects. The code used to manage NHibernate is pretty small and easy to follow. It includes a 49-line class to create our configuration, a 53-line class to manage NHibernate start-up, a couple of 50-60 line classes to manage sessions in web applications and a 75-line entity base class. These numbers include comments and whitespace etc, the number of logical lines of code is probably around 30-40% of the above.

Has the "it's too complicated" viewpoint been partly reinforced by experiences with overly complex abstractions around NHibernate in "enterprisey" solutions? TinyNH is based on the opinion that you do not, by default, need to "hide" NHibernate with things like UnitOfWork and IRepository abstractions. Experienced NHibernate and TDD experts can make informed decisions about where persistence ignorance might genuinely benefit their solution and add a lightweight layer over the top of TinyNH if they need it.

Let's not pretend that there aren't gotchas, limitations and best-practices that take time to learn when working with an ORM. TinyNH offers less experienced NHibernate users a short cut to getting their project up and running with the essential pieces in place, allowing them to focus on the underlying mechanics of the framework.

What Next?

Read getting started to find out how to run the TinyNH project, then work your way through the articles, starting from the contents page.

I hope that TinyNH becomes a useful resource for both new and experienced NHibernate users.



Getting Started with TinyNH, a Simple NHibernate Infrastructure


TinyNH is a demo project and series of articles designed to help you get your project up and running with a simple solid NHibernate persistence infrastructure. See contents for a full list of articles.


Getting the Code

TinyNH is available on GitHub at: https://github.com/danmalcolm/TinyNH.

If you know about git, go ahead and clone the repository on your development machine, otherwise you can download a zip file from https://github.com/danmalcolm/TinyNH/archive/master.zip. If you just fancy a look around, you can browse the code directly on GitHub.

See the README for further details on getting the solution up-and-running on your computer. Note that the solution is currently compatible with Visual Studio 2012 only.

Contents

The src folder contains a single Visual Studio Solution:



The solution contains 4 projects:

  • TinyNH.DemoStore.Admin - Demonstrates NHibernate setup for a web application
  • TinyNH.DemoStore.Core - Contains some shared elements within the solution, namely our object model and NHibernate infrastructure
  • TinyNH.DemoStore.ProductImporter - An example of working directly with NHibernate in a simple "throwaway" data importer app.
  • TinyNH.DemoStore.Tests - Contains some examples of automated database tests.

Contents - TinyNH - A Simple NHibernate Infrastructure

TinyNH is a demo project and series of articles designed to help you get your project up and running with a simple solid NHibernate persistence infrastructure. See the introduction for further details.

Here are the articles in the series:


Tuesday, 16 April 2013

Ambient Session and Transaction Management with TinyNH, the Simple NHibernate Infrastructure

So far, you've:

- Created your domain model entity classes
- Added a ConfigurationBuilder to initialise your NHibernate Configuration and class mappings
- Added a ConfigurationStore as an application-wide point of access for your Configuration and ISessionFactory
- Maybe added some automated database tests

It’s time to start writing some code to load and save our objects!

When working with NHibernate, the central point for accessing your objects is the NHibernate ISession. You can load individual objects using the Get method, query using a range of mechanisms and save new / modified objects to the database. The session really is where it all happens.

It is also a good practice to execute your work in the scope of a transaction. This applies even when you aren't saving any changes to your data. A transaction is an important consistency boundary when reading data, as well as writing it (see this article for further information).

In a very simple application, you can work directly with an ISessionFactory and create your own session and transactions. The TinyNH TinyNH.DemoStore.ProductImporter console application provides an example of how you might work with NHibernate in a simple "throwaway" application. Here, the session and transaction are managed explicitly:

Look at ProductImporter/Importer.cs on github if you can't see the code in-line above.

Context Sessions

The application code shown above takes full responsibility for the session life cycle  from opening the session and beginning a transaction to committing and rolling back the transaction.

In other scenarios, it makes sense for the session and transaction to be managed externally to the code that uses the session to do its work. To do this, we introduce a mechanism that makes an “ambient” session available in the background to anything that needs to use it. The life cycle of this session is tied to a given scope or context. One obvious scope would be the lifetime of a web request within an ASP.Net application. Other examples include an NServiceBus handler or similar command handler, anywhere where objects are loaded and saved within a logical “unit of work”.

Sharing a session within a given context brings a number of benefits:
  • You don’t want to have repetitive session and transaction management code scattered throughout your application code.
  • It is less efficient for separate pieces to start and close their own session when the work could be completed within a single shared session.
  • NHibernate's first-level cache can be leveraged more effectively, reducing the number to trips to the database if separate pieces of code need the same object
  • NHibernate's lazy loading mechanism - where an object’s related objects are loaded on-demand as properties and collections are navigated - requires a session to be kept active in the background, perhaps after the method responsible for loading the objects has exited. For example, an ASP.Net MVC controller action method may load some objects and assemble a  model for display in the view. When the view is rendered, child objects associated with objects loaded by the controller may need to be loaded as they are accessed within the view, a good example of where a session needs to be managed outside of the code that uses it. 
Bear in mind that there may be scenarios where certain objects absolutely should be saved in a separate session and transaction, e.g. you are logging user actions at the beginning of a web request and you don’t want this to be rolled back as a result of an error occurring further down the line.

Context Session Management

We need two mechanisms in place to make our ambient session available during a given context:
  • Somewhere to store the current session
  • Something to co-ordinate everything, starting the session / transaction when the scope begins, committing the transaction when the scope ends or rolling back the transaction if an error occurs.
The first element is built-in to NHibernate, in the form of ISessionFactory's GetCurrentSession method. See the documentation about context sessions for background.

NHibernate must be configured with the strategy that it will use to store the context session. ConfigurationBuilder, responsible for initialising our NHibernate Configuration initialises our configuration to use CallSessionContext, a sensible default for tests and offline applications. This is overridden in our web application start-up code to use an implementation of ICurrentSessionContext for web applications:

Look at ConfigurationBuilder.cs on github if you can't see the code in-line above.

LazyWebSessionContext is TinyNH's custom implementation of ICurrentSessionContext. It supports a mechanism whereby starting a session and transaction is deferred until it is actually used. This ensures that we don't pay the price of opening a database connection and starting a transaction during a request that doesn't need to access the database.

NHibernate can't get the current session by itself however. The next piece in the jigsaw is a mechanism to coordinate the session in line with the lifetime of our context.

This is handled by LazyWebSessionContextModule.cs, an HttpModule:

Look at LazyWebSessionContextModule.cs on github if you can't see the code in-line above.

An HttpModule is not the only mechanism available. You could add session management functionality directly to the Application_BeginRequest, Application_EndRequest and Application_Error methods of your global.asax file.ASP.Net MVC ActionFilters and the Controller class itself also provide hook methods that could be used in a similar way. An HttpModule is a useful one-size-fits-all mechanism however, you may be running MVC controllers, Web API controllers and ASP.Net handlers in your application.

Accessing the Session

With the above in place, NHibernate, LazyWebSessionContext and LazyWebSessionContextModule will work together to manage your sessions and transactions. It remains to add a point where your application code can access the current session. In keeping with it's structurally simple architecture, TinyNH adds a CurrentSession property to Global.asax.cs and an additional convenience property to a shared controller base class AppController.cs. There are of course other mechanisms that could be used to make this dependency available to classes that need it - just configure your IoC container to resolve the ISession from the ISessionFactory (an example of how to do this may be added at some point).

How to do it

If you are using NHibernate in a web application, you can add lazy context session management as follows:

1. Add LazyWebSessionContext.cs and LazyWebSessionContextModule.cs to your project, within the same namespace as the ConfigurationBuilder class and other NHibernate infrastructure code.

2. Modify your web application start up code, using ConfigurationBuilder to configure NHibernate to use LazyWebSessionContext - see Global.asax.cs

3. Configure your application to use the LazyWebSessionContextModule HttpModule by adding the module to the configuration/system.webServer/modules section - see web.config for details.

If you are using IIS6 or need to run your web application in the older Visual Studio Development Server, then the module needs to be added to the configuration/system.web/httpModules section instead.

4. Define a point where your application code can access the context session as outlined above.

Thursday, 24 January 2013

Be aware of exception caching when using System.Lazy<T> to initialise shared objects

System.Lazy<T> gives us a nice declarative way of lazily creating object instances. The class is useful for a couple of reasons.

Firstly, the wrapped object is only initialised (lazily) when the Value property is accessed, allowing work to be deferred until it is actually needed, and could be used to improve an application's start up time. Quoting from this Martin Fowler article is mandatory here:
As many people know, if you're lazy about doing things you'll win when it turns out you don't need to do them at all.
In addition, Lazy<T> can be configured to control initialisation of the value when the Value property is accessed concurrently by multiple threads. This prevents nasty bugs from arising due to different threads getting different copies of an object, or multiple threads running non-thread-safe code concurrently.

Lazy<T>'s thread safe behaviour makes it ideal for managing the initialisation of an object that is costly to create. A trivial example is shown below:


Here are some sort-of-real-world example scenarios where we might consider using a thread-safe Lazy<T> to initialise a shared object within our application:

  • Loading some information from a database when the application starts and storing it for use throughout the application
  • Working with a third party library or framework that requires you to maintain a single application-wide instance of an object instance. For example, it is intended that single NHibernate Configuration and ISessionFactory instances are shared by all threads within an application.
If you are doing something similar to the above and are thinking of using Lazy, then you need to be aware of its thread safety modes and exception caching mechanism.

First, let's look at thread safety in more detail. To illustrate the thread safety options, here are some examples of how you can create instances of Lazy<T>:

There are three LazyThreadSafetyModes that you can choose from:

  • None - The Lazy<T> instance is not thread-safe and multiple threads can initialise the value concurrently. Returning to the example scenarios above, this would result in multiple threads running code to connect to the database or several NHibernate Configuration and ISessionFactory instances being created if a web application were under load.
  • ExecutionAndPublication - Locking is used to ensure that only a single thread initialises the value. Any other concurrent threads accessing the Value property will wait and receive the value initialised by the chosen thread. 
  • PublicationOnly - This one sounds quite fun... If the value has not yet been initialised, all threads that access the Value property concurrently get to have a go at initialising the value. The first to complete is the "winner" and all other threads accessing the Lazy<T>'s Value property will get the value initialised by the winning thread. 

ExecutionAndPublication is the most useful mode if you want to ensure that only a single thread is allowed to execute some costly initialisation functionality. As we saw in the Lazy<T> constructor examples above, Lazy<T> uses this thread safety mode by default, unless you specify another mode explicitly.

So far, so good. But what happens if something goes wrong during initialisation of a lazy value and an exception is thrown? The way that exceptions are dealt with varies according to thread safety mode. This is the full description of ExecutionAndPublication from MSDN (with my emphasis on exception caching):
Locks are used to ensure that only a single thread can initialize a Lazy<T> instance in a thread-safe manner. If the initialization method (or the default constructor, if there is no initialization method) uses locks internally, deadlocks can occur. If you use a Lazy<T> constructor that specifies an initialization method (valueFactory parameter), and if that initialization method throws an exception (or fails to handle an exception) the first time you call the Lazy<T>.Value property, then the exception is cached and thrown again on subsequent calls to the Lazy<T>.Value property. If you use a Lazy<T> constructor that does not specify an initialization method, exceptions that are thrown by the default constructor for T are not cached. In that case, a subsequent call to the Lazy<T>.Value property might successfully initialize the Lazy<T> instance. If the initialization method recursively accesses the Value property of the Lazy<T> instance, an InvalidOperationException is thrown.
This is quite subtle and I'm glad I RTFMed this.

To clarify:

  • If you create your Lazy<T> using an initialisation method (i.e., a "valueFactory" parameter), e.g. var lazy = new Lazy<T>(() => Initialise()) and that delegate throws an exception, no further attempts will be made to initialise the value. The original exception will be thrown every time the Value property is accessed. 
  • If you don't use a delegate (i.e., you are specifying that Lazy<T> will initialise an instance of target type T using its default constructor), an exception thrown during initialisation will not be cached and further attempts will be made to create the value until initialisation succeeds.

Let's consider this in the context of our sort-of-real-world example scenarios.

Firstly, imagine a web application that uses a Lazy<T> with an initialisation method that loads data from a database to initialise an object. If an exception were thrown by the initialisation method, perhaps due to a temporary connectivity issue, the same exception would be thrown every time the lazy value were accessed. Every request that required access to the lazy value would result in a server error.

The second scenario, using Lazy<T> to initialise an NHibernate Configuration or ISessionFactory, could also potentially cripple the application for similar reasons. An exception will be thrown if the database specified in the configuration is not available when Configuration.BuildSessionFactory is called and no further attempts would be made to build the session factory, even when the database became available.

So, let's conclude with ... a top tip: You must not use a thread-safe (ExecutionAndPublication) Lazy<T> with a initialisation method ("valueFactory" parameter) if there is any risk of the method failing. If it fails, it won't get the chance to run again, which could bring down your application.

If this helps just one or two developers out there I'll be happy. Let me know if this has helped you!

Some alternatives and work-arounds will be explored in a future post.

Thursday, 10 January 2013

Generating a script in SQL server to delete data from your database

In my previous post, I described how a simple SQL script could be an effective way of resetting your database to a known state between tests in your database integration test suite.

Writing a SQL reset script manually for a large database is tedious though. Imagine having to go through 100 tables and work out what order to delete records in... Fortunately, most database engines expose data about tables and the relationships between them in system tables. You can use this information to derive the dependencies between your tables and automatically generate a sequence of delete statements in the correct order. In Isolating database data in integration tests, Jimmy Bogard demonstrates some C# code that uses the sysobjects and sysforeignkeys tables in SQL Server to do this.

In some cases, you might prefer to have a SQL script that does a similar thing. No code to compile or connection string to configure, just a script that you can copy and paste into SQL Server Management Studio (or your client tool of choice) and execute against your application's database.

The following script analyses the tables and relationships in the current database and generates a sequence of “delete from [table]” statements in the correct order. If it can’t resolve the delete order for all tables, it reports the foreign key constraints that are preventing it from doing so.

Here it is:


Some database schemas may need you to hand-code some parts of your SQL reset script. For example on one project, we had to modify the generated script to accommodate a self referencing constraint:


The number of customisations required will vary from project to project. Hopefully the script will do 90% or more of the work for you.

Why not run the script against your application database and see whether it could help you? I’d be keen to hear from you if you have any comments or ideas.

Wednesday, 9 January 2013

Using a SQL script to reset data in database integration tests

Automated database tests need to be isolated and repeatable. It’s a lot simpler if each test begins with the database in a known state. The simplest way to do this is to delete unwanted data from the database before each test. As one of my colleagues likes to say: “We need to flatten the database”.

On a recent project, I went through a number of options for resetting the database, similar to those outlined by Jimmy Bogard in Isolating database data in integration tests. I also spent some time attempting to query and delete test objects via features of our ORM before realising that it wasn’t really set up to help me delete a complex graph of inter-related objects within a single unit of work.

Based on this experience, I am pleased to share a top tip: Direct SQL is the fastest and simplest mechanism for resetting your database and is probably the one that you should try first.

So how do we generate and run our SQL script? Jimmy describes a way of calculating the dependencies between tables at run time in some C# test infrastructure code and executing a bunch of “delete from xxx” statements. Nice. An automated solution like this will even accommodate changes to the database schema, er, automatically. However, there are some drawbacks. Diagnostics / analysis options are limited. If a schema / mapping change broke the assumptions in the code and made it impossible to generate the delete scripts in the correct order, we’d probably need to fire up the debugger and step through the code to work out which constraints were preventing the dependency order from being established.

Another option is to develop a SQL script yourself and execute it when preparing to run each test. It’s very simple and works as follows:


This approach has a couple of advantages:
  • It is clear at-a-glance how the database is being reset
  • You may need to go beyond several “delete from [table]” statements and get down and write some bespoke logic. Maybe you need to preserve a couple of records in a certain table. Perhaps there is one table in your 100 table system where you need to drop constraints, delete the data and re-create the constraints. A raw SQL script allows for manual intervention.
The main drawback, of course, is that you will need to update the script as the structure of your application's database changes, but you probably won't find that too annoying in practice.

Note that we don't have to hand-code our SQL from the ground up. See the next post for details of a TSQL script that you can run directly in SQL Server to help generate your reset script.