First of all I have to explain to everybody what I’m talking about:

With Visual Studio 2010 Microsoft introduced a new way to deploy extensions to Visual Studio. Instead of creating a setup.exe that copies files, writes to the registry and calls devenv.exe with the /setup option one only needs such a VSIX file that does all that is necessary to install an extension.

When working with the T4 (Template Transformation Toolkit) system one creates tt files with some kind of script that is used to generate source code within a user’s Visual Studio project. The Microsoft DSL Tools do the same: every DSL project contains a number of tt files that will generate all the code. If you take a look at such a file, you will only see two lines:

<#@ Dsl processor="DslDirectiveProcessor" requires="fileName='..\DslDefinition.dsl'" #>
<#@ include file="Dsl\Diagram.tt" #>

The magic happens with the include command. The real template code sits in the included file and not within every single project. When installing the DSL Tools extension to Visual Studio all these include files get installed somewhere where they can be found by the tt engine.

For my own DSL Languages I would like do the same: the user projects should only contain such two line tt files and include the rest of the template from other files.

How to deploy tt template files?

The following steps will show you how to deploy additional files with you DSL extension.

I will start from a newly created Domain-Specific Language Designer project. Such a project contains a Dsl and a DslPackage project where the DslPackage project creates besides of the DslPackage.dll a vsix file to install the designer onto another machine.

Our goal will be to add some tt files to the vsix and make them includable on the installed machine.

1. Add a folder to the DslPackage project and name it TextTemplates (or any other name)
2. Add the tt files to this folder.
   • In the properties windows clear the “Custom Tool” property; otherwise Visual Studio will try to execute these templates within the DslPackage project.
   • set “Build Action” to “Content”
   • set “Include in VSIX” to “True”
3. Add a file called Additional.pkgdef to the DslPackage project with the following content and
   • set “Build Action” to “Content”
   • set “Include in VSIX” to “True”
     

     [$RootKey$\TextTemplating\IncludeFolders] [$RootKey$\TextTemplating\IncludeFolders\.tt] "IncludeMyDsl"="$PackageFolder$\TextTemplates"

     Replace “IncludeMyDsl” by a unique name for your DSL but make sure it starts with the word “Include”.

4. Edit the source.extension.tt in the DslPackage project and add the following line within the Content tag:

   <VsPackage>Additional.pkgdef</VsPackage>

With these changes the tt files from the TextTemplates folder get packaged into the vsix file and will be accessible using the <include> tag on the machine where this extension is installed.

Let’s start with a short introduction to DirectiveProcessors. DirectiveProcessors are used in .tt-files to provide data to the template. The DirectiveProcessor can take parameters from the declaration in the .tt-file and adds some code to the compiled template. This code (often properties) can be used from within the template. For example the DSL Tools are generating one DirectiveProcessor for each DSL model. In a template form the DSL Tools you will see a line like the following:

<#@ Language1 processor=“Language1DirectiveProcessor”
requires=“fileName=’Sample.mydsl1′” #>

This line uses the Language1DirectiveProcessor and provides it with one parameter (fileName). The DirectiveProcessor adds code to the template to open the given file and creates (in this case) a ExampleModel-property to use in the template code.

I created two DirectiveProcessors to use the data of simple Xml-files and Visual Studio Project files in the templates. The code of the two DirectiveProcessors is very straight forward and maybe one could advance it (e.g. make it compatible with templates written in Visual Basic).

Using the XmlFileDirectiveProcessor

Add a line like the following to your .tt-file:

<#@ XmlFile processor=“XmlFileDirectiveProcessor” FileName=“example.xml” #>

Inside this template you can access the (full qualified) filename via the this.XmlFileName-Property and the Content of this file (as a XDocument) via the this.XmlFile-Property.

Using the VsProjectFileDirectiveProcessor

Add this line to your template:

<#@ ProjectFile processor=“VsProjectFileDirectiveProcessor” FileName=“x.csproj” #>

A property named this.ProjectFile will be added to the template. This property provides you with an instance of the VsProjectFile-class containing the project file contents. This class contains only very little functionality (feel free to add some more and submit it back to me!). Just take a look at the source code of this class. The method GetAllFiles() returns a string array of all files found in the project (supporting C++ and C# project files – .vcproj and .csproj).

Setup

An entry in the registry is needed to allow the T4 system to find custom DirectiveProcessors. Just add the following entries to your registry and don’t forget to point to the right location of the JaDAL.dll.

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0Exp\
Configuration\TextTemplating\DirectiveProcessors\VsProjectFileDirectiveProcessor]
“Class”=”BenjaminSchroeter.Dsl.DirectiveProcessors.VsProjectFileDirectiveProcessor”
“CodeBase”=”D:\\JaDAL\\bin\\Debug\\JaDAL.dll”

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0Exp\
Configuration\TextTemplating\DirectiveProcessors\XmlFileDirectiveProcessor]
“Class”=”BenjaminSchroeter.Dsl.DirectiveProcessors.XmlFileDirectiveProcessor”
“CodeBase”=”D:\\JaDAL\\bin\\Debug\\JaDAL.dll”

These registry keys are for the Experimental Hive of Visual Studio. To register the DirectiveProcessors for the normal Visual Studio instance the registry key starts with HKEY_LOCAL_MACHINE\Software\Microsoft\VisualStudio\9.0
\TextTemplating\DirectiveProcessors\

For some reason I want to know the filename of the model used in the tt-file. The default implementation provides you only with a reference to the model but with no chance to get its filename.

In your tt-file you will find something similar to

<#@ Language15 processor="Language15DirectiveProcessor"
     requires="fileName='Sample.mydsl1'" #>

This will use the Language15DirectiveProcessor to load the given file and provide your template with a global ExampleModel variable (of cause the name depends on your DSL and you can change it with the provides attribute in the tt-file).

To change the Language15DirectiveProcessor you can create a partial class in your DSL and add some code to it:

partial class Language15DirectiveProcessor
{
    protected override void GenerateTransformCode
     (string directiveName,
      StringBuilder codeBuffer,
      System.CodeDom.Compiler.CodeDomProvider languageProvider,
      IDictionary<string, string> requiresArguments,
      IDictionary<string, string> providesArguments)
      {
        base.GenerateTransformCode(directiveName,
                    codeBuffer, languageProvider,
                    requiresArguments, providesArguments);           

        codeBuffer.AppendFormat(
           "public string {1} = @\"{0}\";",
             requiresArguments["FileName"],
             providesArguments["FileName"]);

       codeBuffer.AppendLine();
    }

    protected override void InitializeProvidesDictionary
    (string directiveName,
      IDictionary<string, string> providesDictionary)
    {
        base.InitializeProvidesDictionary(directiveName,
                            providesDictionary);

        providesDictionary.Add("FileName","FileName");
    }
}

This adds a new global variable FileName to the tt and initializes it with the model file name.

Attention: You see the variable in this example is generated by writing a line of C# code to the codeBuffer. This may be a bad idea if you want to use this DirectiveProcessor for templates where the template language is set to VB. The better way is to create the code using the Emit and CodeDom API, but I was to lazy to do it that way.

If anybody ports this code to work with VB templates by using the CodeDom or plain VB code, please post a comment here.

Further reading: The process of creating your own DirectiveProcessor and everything you need to understand the code above can be found in the msdn: Creating Custom Text Template Directive Processors.

Maybe you do this already with a codegenerator of some kind or by using a scripting language like PHP or Python. Maybe you have reasons not to generate it and write it by yourself. Very often these reasons are bad reasons: you don’t know any adequate tool, you don’t want to buy one or you don’t have the time to learn it.

CodeSmith

The first time I had to generate source code we wanted to generate a library to access the data of an external software in a type safe way with real properties and not throwing around with strings. The input was a xml configuration file of this software.

For this aim we used a freeware code generator named CodeSmith. CodeSmith is a templating engine using C# or Visual Basic as embedded language and a syntax that is very similar to ASP. You can use all .Net framework classes and function within your template, load your own or 3rd party libraries into it and use all these classes as parameter of the template.

A few years later we had to generate code again and discovered that CodeSmith became a company that sells an enhanced standard and professional version of the well known tool. They added a very powerful IDE to create and edit your templates and for the professional version Visual Studio integration and an API to use the engine with your templates in your applications.
After buying the professional version we discovered that we cannot use CodeSmith for our needs. We wanted to build a product that generates code for the end-user on his machine but you are not allowed to distribute the CodeSmith libraries that are needed when you use the API. It took a long time for me to imagine a good usecases for an API like this if I cannot give my software away (or every user of my software has to buy a $399 licence of CodeSmith).
Fortunately the old freeware version was capable of this. It can compile your template to C# code that runs without any external reference or library. You can add this code to your application and that’s it. You do not have the fancy IDE (or you can buy a CodeSmith licence and use the IDE but process your template with the freeware version – the syntax is nearly the same) and maybe not all of the features of the new version but it is free and works. The freeware version is still downloadable on their webpage.

T4 – Text Templating Transformation Toolkit

Visual Studio 2008 (aka. Orcas) comes with its own templating engine that is a free part of every Visual Studio Professional or Team System installation (free as in: you paid for it with your Visual Studio licence). This engine is powerful with C# (or even C# 3.0) and Visual Basic as language and access the whole .Net framework and external assemblies as well.

Unfortunately there is no editor integrated in Visual Studio for the TT files. And even worse, you cannot use the engine out of the box. The DSL Tools use it but if you want to do so, you have to write code and access the public interfaces. But for both drawbacks there are solutions.

You can download a beta version of the T4 Editor. It integrates in Visual Studio (the download for the Beta 2 works fine with the final version of Visual Studio for me, too) and has syntax highlighting for template files. It comes also with some Intellisense, but only for the template syntax and not for code you write inside your templates.

Then there is TTxGen that allows you to use every file you want as input for a template in a Visual Studio solution. It adds a template to the input file and generates the output from the input using the template. It’s really that easy. TTxGen is a very lightweight thing (only 45 kByte for the installer) because it delegates most of the work to the T4 stuff from Microsoft and adds only a few menus to Visual Studio.
I had some trouble with the installation of the February 2008 CTP release. You can find my workaround at the discussions over at MSDN Code Gallery.

For one of my DSL Tools add-on libraries I use TTxGen to generate code from a xml file. Later I will post an article with the source code of the template that can be used as an example.