For the past several months, I've been experimenting with ways to use .NET reflection with the SharePoint API.  As I've mentioned before, I believe that .NET attributes can not only make it easier to build great SharePoint applications, but can also provide a foundation for building better development tools.

In this series of posts about the evolving SharePoint Reflection Framework, I'll continue to share the results of my research and I'll include links to sample code directly within each post.  As the code matures, I'll consider making it available on CodePlex so that others may contribute.  For now, let's just treat it as a "thought experiment" as my friend Andrew Connell likes to call it.

The sample code and the framework assembly can be downloaded here:

http://www.johnholliday.net/downloads/JohnHolliday.SharePoint.ContentTypes.zip.

There is also a short screencast (27 minutes) that shows how to use the framework which you can view here:

 

Now let's take a closer look at how we can apply .NET attributes to simplify working with content types. 

Some have asked, and you might also be wondering "Why do we need this?   Why not just use CAML to declare the content types and then simply publish them using a feature?  Why go to all the trouble of using this non-traditional approach?"

Well, there's nothing wrong with the traditional CAML-based approach.  There is ample support for it and with a few simple tools, it's pretty straight-forward once you get the hang of it.  But there are a number of limitations with the "code + markup" paradigm when it comes to reusability and extensibility.  The markup is "brittle" in the sense that subtle mistakes are hard to find.  True, you can preload the schema to help you remember what attributes go where, but there is no true intellisense and no strong typing to prevent those mistakes.  If you want to extend an existing content type, you have to  visit multiple files and switch back and forth between your event handler code and the markup.  You have to spend a good deal of time making sure that the code matches the properties that have been defined in the markup.

The unfortunate reality is that working directly with XML still requires a significant paradigm shift in the way most developers think.  With all the buzz about improving the SharePoint "developer experience", I keep thinking there must be a better way.

So, there are several goals for this experiment:

1. We want to make it easier to declare content types in code.
2. We want to eliminate the need to work directly in CAML or XML.
3. We want to use the same coding idioms we are used to when working with other code.
4. We want to make it easier to write custom event receivers for our content types.
5. We want to make it easier to associate sub-components like document templates and XML documents with our content types.
6. We want to be able to build libraries of reusable components that can be leveraged across multiple projects easily.
7. We want to enable DRM for content type components at the assembly level so that developers can protect their intellectual property.

The SharePoint Reflection Framework

For this experiment, we'll construct a content type for an expense report in C#.  But we also want to work with the expense report content type in the same way we might work with other C# classes.  For instance, we might want to create a library of reusable content type classes for use in financial solutions. 

To achieve this, we need to have a set of attributes that we can attach to a class to "declare" the content type and its properties.

We can start simply by deriving a ContentType class from System.Attribute.  To control where and how it is used, we can mark it so that it can only be applied to classes and only once per class.

[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]public class ContentType : Attribute{    public string Name { get; set; }    public string BaseType { get; set; }    public string Description { get; set; }    public string DocumentTemplate { get; set; }    public string Group { get; set; }    public bool Hidden { get; set; }    public bool Sealed { get; set; }    // ...}

namespace JohnHolliday.SharePoint.SampleContentTypes{    [ContentType(Name = "Expense Report",        BaseType = "Document",        Description = "A worksheet for recording expenses.",        Group = "SharePoint Reflection Framework Samples",        Hidden = false]    public class ExpenseReport    {    }}

Notice that we are basically specifying the same properties we would have to add to a <ContentType> element in CAML.  The difference is that here we get strong typing and intellisense support.  Notice also that we don't have to worry about constructing the content type identifier.  Instead, we just supply the base type and let the framework handle the details.

Next, within the content type declaration, we need to specify the fields that make up the type.  Here again, .Net reflection comes to our aid to make things easier.  What we really want is to declare the fields of the content type in the same way that we might declare properties for any C# class.  So, if we wanted to add a description field to the expense report content type, we would naturally want to do something like this:

[ContentType(Name = "Expense Report",    BaseType = "Document",    Description = "A worksheet for recording expenses.",    Group = "SharePoint Reflection Framework Samples",    Hidden = false]public class ExpenseReport{    private string m_description=string.Empty;    [FieldRef("Description")]    public string Description {        get { return m_description; }        set { m_description=value; }    }}

Here, we can simply make use of a second attribute included in the framework to markup the properties we want to map to SharePoint fields.  The FieldRef attribute is provided for this purpose.  The framework supplies the necessary code to examine the ExpenseReport class, locate the mapped properties, determine their field types and link them automatically to the content type on creation.  For instance, if we add a DateTime property to the ExpenseReport class, we want it to surface in SharePoint not as text, but as a date field.   Similarly, the other field types can be automatically determined from the underlying property type.

Enumerations

We can extend this idea further to include built-in support for CHOICE fields if an enumerated type is mapped to a field.  For example, we might need to declare a ProjectTypeEnum enumeration and then map it to a ProjectType property as shown below. 

public enum ProjectTypeEnum{    Consulting,    Training,    Programming,    Sales,}private ProjectTypeEnum m_projectType;[FieldRef("ProjectType", DisplayName = "Project Type", Required = true,    Description = "The project type for which expenses are being submitted.",    Group = "Sales")]public ProjectTypeEnum ProjectType{    get { return m_projectType; }    set { m_projectType = value; }}

In this case, the framework detects that the underlying data type of the property is an enumerated type and it then generates the appropriate choices automatically, as shown here.

We can do a lot more with field references, but let's move on to the question of how to create the actual content type within the SharePoint environment.

Creating Instances

Typically, we'll be writing a FeatureActivated feature receiver or building a command-line utility, and we need to create a content type as part of the solution.  Ideally, we'd like to end up with an instance of SPContentType that we can work with.  So we really only need the framework to handle the dirty work of talking to the SharePoint API and then returning a properly constructed SPContentType object.  One way to implement this pattern is to provide a factory method on the ContentType attribute class that takes a reference to our implementation class and then gives us back an SPContentType object.  For the expense report content type, we might call it like this:

using (SPSite site = new SPSite("http://litwareinc.com")) {    using (SPWeb web = site.OpenWeb()) {        // Create the expense report content type.        SPContentType ctExpenseReport = ContentType.Create(web, typeof(ExpenseReport));        if (ctExpenseReport != null) {            // do something with it        }    }}

Note that the ExpenseReport class is completely independent of the SPContentType object it was used to create.  It can be implemented in a separate assembly.  Using this approach, we can easily build libraries of content types that can be reused across multiple solutions.  But there are still a couple of additional requirements we need to fulfill.  Namely, how do we handle item event receivers?  And what to do about the nested sub-components of a content type, such as document templates and XML documents?  These are critical requirements, because we want to apply the same coding paradigm we are used to for other C# classes.  We don't want to have to revert to using XML markup in order to specify event receivers and other elements.

Item Event Receivers

To specify event receivers for a content type in code, we need to add the appropriate entries to the EventReceivers collection of the SPContentType object.  When working with XML markup, this is done by creating a special XML document that specifies which event we want to capture and the assembly and class in which the event receiver method is implemented. 

To deal with event receivers in our ContentType attribute, we can take advantage of the fact that SharePoint provides an abstract class called SPItemEventReceiver that declares all of the event receiver methods.  By simply deriving our ExpenseReport class from SPItemEventReceiver, we get the correct method declarations in the base class and then we can selectively override the ones we want.  Instead of telling the framework explicitly which methods we are implementing as we would have to do with CAML, we can let the framework do the work for us and infer which methods have been implemented by examining the class via reflection.  So all we have to do to handle the ItemAdded event, for example, is add the derivation and override the ItemAdded method as shown in the following code.

[ContentType(Name = "Expense Report",    BaseType = "Document",    Description = "A worksheet for recording expenses.",    Group = "SharePoint Reflection Framework Samples",    Hidden = false)]public class ExpenseReport : SPItemEventReceiver{    // ...properties    public override void ItemAdded(SPItemEventProperties properties)    {        base.ItemAdded(properties);        string message = string.Format("Expense report added to list: {0}",              properties.ListTitle);        EventLog.WriteEntry("SharePoint Reflection", message);    }}

Still, no CAML.  We are working entirely within a C# class and we have captured all of the information needed to create the content type and to install an event receiver for the ItemAdded event.  The framework looks at the ExpenseReport class and sees that only one of the abstract classes was implemented, so it creates an event receiver for that method only.  It knows which assembly the class is declared in, so it automatically assigns the assembly and class names.  If we want to add an event receiver for another event, such as ItemUpdated, we can simply add another override and implement the method.  The framework automatically registers it for us.

Document Templates and XML Documents

The thing about document templates and other kinds of files that we might need to associate with a content type is that they are files.  Ideally, we'd like to work with them as files without having to jump through hoops just to get SharePoint to recognize them.  Fortunately, Visual Studio has a nice feature that lets us access and work with files easily from within our code.  These are embedded resources.  If we change the build action for any given file within our project to EmbeddedResource we can reference it using a path expression of the form <folder name>.<sub folder name>.<...>.<filename>.<extension>.  So if we have a project folder named "DocumentTemplates" and a file in that folder named "ExpenseReport.xlsx", and both are part of a project named "SampleContentTypes", we could refer to the resource using the path "SampleContentTypes.DocumentTemplates.ExpenseReport.xlsx".

By extending the ContentType attribute to recognize and locate embedded resources, we can easily add document templates and other resources directly to our class declaration without having to worry about how they actually get copied into SharePoint.  Here again, we can delegate that responsibility to the framework and just focus on the content type implementation.

For consistency and to distinguish document templates specified as embedded resources from those specified with legitimate urls, I've added a "res://" prefix which the framework recognizes.  It then treats the url as a reference to an embedded resource and copies the file into the proper location when the content type is created.  Since it already knows where the resource is located, we can drop the assembly name and just specify the assembly-relative path to the resource file.  We end up with a declaration that looks like this:

[ContentType(Name = "Expense Report",    BaseType = "Document",    Description = "A worksheet for recording expenses.",    Group = "SharePoint Reflection Framework Samples",    Hidden = false,    DocumentTemplate = "res://DocumentTemplates.ExpenseReport.xlsx")]public class ExpenseReport : SPItemEventReceiver{    //...}

Future Directions

As you can see, there is a lot of promise to this approach, but we've only begun to scratch the surface.  I'm currently working on one-way data binding of list items to properties in the implementation class whenever an event receiver method is called.  This would essentially let you reference the mapped properties just like other properties in your event receiver methods.  It would be one-way only because there might be times when you don't want those properties to get copied back into the list item automatically.  For that, you can always call into the SharePoint API.

There is a lot more we can do with content types as well as with other SharePoint objects that can really make our lives easier as SharePoint developers.  Give the code a spin and tell me what you think.  In the coming weeks, look for additional posts that will go into more capabilities of the framework. 

Stay tuned.

From the title of my blog, one might surmise that I'm kind of into reflection. (Ya think?)  Well, although I haven't blogged about using .NET reflection for SharePoint development since back in 2006 (see my article "Use .NET Reflection to Create SharePoint Sites and Site Definitions"), I've continued experimenting with the idea and may have actually come up with something useful.

My original article dealt with the problems associated with writing site definitions for WSS 2.0.  Since then, with all the new capabilities they've stuffed into WSS 3.0, the basic problem still remains - how to reduce the ever-increasing "surface area" faced by SharePoint developers?  I still find it somewhat disconcerting to write XML markup along with my managed code while keeping everything in sync.  Every time I conduct a training class for SharePoint developers, they all echo the same sentiment.

So, what if you could write a SharePoint Feature like this?

All C# - all the time.  No CAML to speak of. 

Of course, there's the question of how to get SharePoint to understand it.  There's also a bit of plumbing required under-the-covers to make everything work.  But the end result is somewhat compelling, isn't it?  

If you're intrigued, then click here to read the entire article.  I've basically created a SharePoint Reflection Framework that I've started to use in my own development work.  It's getting to the point now where I'm feeling the need to share the fruits of my labor.

And don't resist the urge to give me some feedback!  I'm always on the lookout for beta testers!

Stay tuned.

JFH

Boy, news travels fast on the Internet!  Awhile back, my fellow SharePoint MVP Carlos Segura Sanz came up with a pretty nifty tool that lets you create CAML queries using T-SQL syntax.  It's called YACAMLQT for "Yet Another CAML Query Tool", and you can read all about it here. 

Basically, it lets you write a query like this:

WHERE ContentType="My Content Type"
OR Description<>null
GROUPBY Title DESC
ORDERBY _Author, AuthoringDate, AssignedTo ASC

YACAMLQT then parses the text and produces a CAML query that looks like this:

Pretty cool, eh?

So I got to thinking, "Why not extend the tool to generate CAML.NET code?".  Then you would have the convenience of T-SQL with the power and extensibility of C#.

Well, wouldn't you know it?  Carlos was already way ahead of me.  In fact, he had already implemented it.  Here is a screenshot of the latest version of YACAMLQT, that now has an extra tab containing the generated CAML.NET code!

I'm working with Carlos to extend his tool to support some additional feature, like adding a SELECT clause so you can easily specify the view fields.  I'm also looking at ways to incorporate the YACAMLQT parser directly into the CAML.NET framework so you could do things like pass a T-SQL string to an existing CamlQuery instance to take advantage of the built-in data binding features of CAML.NET.

Look for updates to appear soon on both our blogs, and also on the CAML.NET Project on CodePlex.  In the meantime, you can download the updated YACAMLQT tool from the following link:

 YACAMLQT.zip (15,07 KB).

This session will introduce the CAML.NET.NET framework in detail, showing how you can use it to take full advantage of the CAML.NET query schema in your WSS and MOSS applications.

• Presented by: John Holliday
• Date: Thursday, August 23, 2007; 6-8 PM
• Location: MODIS Building, Downtown Jacksonville; Conference Room A, 2nd FL
• Meeting Location Map: http://floridaofficegeeks.org/JOG/Pages/Meeting%20Logistics.aspx
• For more information or to RSVP: http://www.clicktoattend.com/invitation.aspx?code=119500

In Part 1 of this series, I introduced the CAML.NET Framework, which provides a way for C# developers (and eventually VB developers) to construct CAML queries and other data structures using a strongly-typed .NET language instead of XML.  This has the added benefit of enabling CAML queries to be packaged easily into .NET assemblies that can then be called from any .NET application.  It does this using a combination of static classes, generic methods and a base class library for automatic data-binding to and from SharePoint list items.

The use of a strongly-typed CAML query language is compelling for a number of reasons.  First of all - it's strongly typed, which means it is easier to avoid and detect errors in the code.  Second - CAML.NET makes it easy to reuse the compiled queries and incorporate them into other projects.  It's also easier to share them with other developers who may not understand the CAML syntax, but need to retrieve data from one or more SharePoint lists as part of their solution.

Having a strongly-typed language for expressing CAML queries is one thing.  Using the language effectively is another.  It would be nice if we could also share the essential elements of a CAML query with non-technical colleagues so that they can understand more precisely what the query is supposed to do.  Currently, the only way to do that is to use a whiteboard or a Visio diagram to sketch the query components and then use that diagram separately as the basis for implementing the query.

A New Tool for CAML Developers

In this post, I will introduce a new tool that further simplifies the developer experience when working with CAML and provides the basis for a new breed of developer tools aimed at bridging the gap between mainstream .NET development and SharePoint solution development.

The CAML Query Designer add-in for Visual Studio 2005 consists of the following components:

• A graphical design surface for designing CAML queries,
• A CAML.NET explorer tool window for browsing the query structure,
• A custom code generator that tranforms the query diagram into C# code.
  

Recall the CamlQuery class which is included in the CAML.NET package.   In Part 1, I gave an example of a derived query that retrieves list items by matching the name of its content type with an arbitrary string passed as an argument to the class constructor. 

    public class QueryByContentType : CamlQuery
    {
        public QueryByContentType(string typeName) : base(
            CAML.Where(CAML.Eq(CAML.FieldRef("ContentType"), CAML.Value(typeName)))
            )
        { }
    }

To build this same query using the CAML.NET Query Designer, we  start by creating a CAML object model using the CAML.NET Visual Studio Project Item template.

This creates a new .caml file and opens the design surface.

 The toolbox contains several objects we can drag onto the design surface.  We'll start with a Query object, and then add the fields we are interested in.  In this case, we'll specify the name of the query as "QueryByContentType", and then we'll add the ContentType field as a field of interest.

At this point, we haven't declared a relationship between the query and the field.  We've only added the field to the model.  We can relate the field to the query in two ways: we can link the field directly to the query using a connector; or we can add a qualifying statement to the query and then reference the field from the statement. 

Note: Connecting fields directly to the query says that the query should limit the result set to only those fields.  If there are no fields connected to the query, then the result set will include all fields associated with each item.

Next, we need to specify that the query should match all list items where the content type field matches a given expression.  To do this, we drag a Comparison operator from the toolbox onto the design surface and associate it with the ContentType field.   The Comparison operator can only be attached to the Query object.  To associate it with the ContentType field, we can simply select the field from the dropdown list in the properties pane.

Next, we could enter a literal expression into the Expression property of the comparison operator, but what we really want is to parameterize the expression so that the query can be applied to more than one content type.  To do that, we add a Query Parameter object to the model and give it the name "typeName" so we can reference it from the comparison operator.  As before with the Comparison object, the Query Parameter object can only be added to a Query object.

Now, the query is complete.  At this point, we can simply save the model.  This does two things:

• creates a folder within our Visual Studio project to help keep the queries organized,
• generates a C# class for each Query object in the model

With the where clause defined, we can then further refine the query by limiting the result set to just the Title and Author fields.  To do this, we can drag two additional Field objects onto the model and connect them directly to the query object.

Since we are explicitly including the Author field in the result set, we might as well limit the result set to only those items where the Author field is not null.  We can do that easily by replacing the Comparison object with a Logical Join and then adding a Condition to specify the desired relationship.

The following screenshot shows the regenerated code.

Although this is a rather trivial example, it's easy to envision using the tool to construct much more complex queries.  Using the CAML.NET Query Designer significantly reduces the overhead associated with building and maintaining CAML queries, and will hopefully inspire the development of reusable CAML query libraries.  It is also possible to involve a wider range of skillsets in the development process because the diagrams can be easily printed and shared with business analysts and other developers.

There is still a bit of work to be done on the designer.  Some of the remaining features include:

• GroupBy and OrderBy clauses
• Importing fields and site columns directly from SharePoint
• Generating the raw CAML alongside the CAML.NET code

I'm hoping to have a community edition available for download *real soon*.  If I could just figure out how to live without eating and sleeping.  In the meantime, you can see a brief Camtasia demo of the CAML Query Designer in action by clicking here.  

For more information about the CAML.NET Framework and to stay abreast of new developments, visit http://codeplex.com/camldotnet.

As always, your comments and feedback are most welcome.

-JFH

We had a great turnout for the South Florida code camp down in sunny Miramar over the weekend!.

It's great to see first-hand the level of excitement and commitment around the new Office Server platform. Especially around SharePoint and related technologies. I had the pleasure of riding down in Andrew Connell's "geek mobile" along with Dennis Bottjer and new friends David Strommer and Roy Ashbrook (who really liked my presentation on Document Management).

I presented two sessions, and I promised to provide the slides, so here they are:

Building Document Management Solutions using Windows SharePoint Services 3.0 Content Types

Advanced Code Generation using CodeSmith 4.0

Technorati : codecamp, microsoft, office, sharepoint
Del.icio.us : codecamp, microsoft, office, sharepoint

Registration for the 3rd annual Atlanta Code Camp is now open!

Here's the link to register (you'd better hurry):
https://www.clicktoattend.com/invitation.aspx?code=113135

The 3rd annual Atlanta Code Camp will take place on January 20th. The event is completely free and lunch is included. Doors open at 7:30am at the Decatur campus of DeVry University. Here's the address:

250 North Arcadia Ave
Decatur , GA 30030

If you are coming in from out of town, the event organizers are recommending the Holiday Inn in Decatur. Ask for the DeVry University rate to get a $99/night rate. Parking is extra and costs $7/day.

Holiday Inn
130 Clairemont Ave
Decatur, GA 30030
404-372-0204

And be sure to catch my session, "Building Document Management Solutions using Windows SharePoint Services v3.0 Content Types". Here's the session abstract:

Windows SharePoint Services 3.0 introduces a new concept called "Content Types" that will dramatically alter the Enterprise Content Management landscape. On the surface, Content Types make it easier to encapsulate field definitions into a nice neat package, but there is a lot more going on under the covers. In this session, we'll take a deep dive into the world of content types and see how to declare them using XML and how to create them programmatically. Then we'll apply content types to create a real-world document management solution with custom policies that we'll use to control every stage of the document lifecycle. At the end of this session, you will have a thorough understanding of what content types are with a deeper appreciation of the value they bring to document management solution development.

Hope to see you in Atlanta!

Technorati : atlanta, code camp
Del.icio.us : atlanta, code camp

Well, I just got back from about 12 hours of great fun at the Second Annual Jacksonville Code Camp. I've got to hand it to the guys at JaxDug! They really went all out and created an agenda that was jam-packed with great information, insights, tips and tricks for just about every area of the information technology business - from professional development to success in life.

From the very start, you could tell this was going to be a pivotal event. The tone of John Spence's keynote was powerful and upbeat with great insights on being a better IT professional (really on being a better person) from the perspective of a highly-respected business improvement consultant who has presented workshops to more than 300 of the world's best known organizations. From there it just got better.

I gave a talk on Advanced Code Generation Using CodeSmith, and another on Using .NET Reflection To Create SharePoint Sites.  Then I got to hang out with fellow 'SharePoint Guys' Andrew Connell, Stacy Draper and David McNamee. I had the great pleasure of attending two of Andrew's talks - one on creating ASP.NET 2.0 web parts and the other on making the most of some new and exciting developer tools - both outstanding! Thanks, Andrew for shining the light!

There were so many great sessions - and the level of enthusiasm was so high. I was in geek heaven! Then at the end of the day, I got to share an "Ask the Experts" SharePoint panel session with David, Stacy, Andrew and Duray Akar as we fielded some pretty tough SharePoint questions. David always brings great breadth of perspective and Duray demonstrated once again why people tend to listen when he talks about SharePoint. You can tell it's coming from real experience.

All in all, a great way to spend a Saturday. Thanks again to all the folks who participated!

The Jacksonville CodeCamp is just around the corner. In fact - it's happening this weekend! If you haven't registered already, here is the link. This is going to be a great event. We have seven (7) Microsoft MVPs, not to mention several Microsoft employees with lots of great info about VSTS, VSTO, VSTA, VISTA, MOSS, WSS and just about every other acronym you can think of! I'll be presenting on SharePoint and CodeSmith, so be sure to check out my sessions.

I'll look forward to seeing you there.

Technorati : CodeCamp, DotNet, Jacksonville, Microsoft
Del.icio.us : CodeCamp, DotNet, Jacksonville, Microsoft

CodeSmith has a great feature that allows you to use XSD schemas to load XML properties into code templates. However, the classes that CodeSmith generates from the XSD are not identical with those produced by XSD.exe. This can lead to problems when writing custom templates or when converting CodeSmith templates into standalone assemblies that depend on the XML properties.

I saw a recent post in the CodeSmith forums where other developers were also struggling with this issue. Eric Smith (the creator of CodeSmith) responded that a command line tool for generating compatible classes from an XSD will be included in version 4.0 of CodeSmith, but there is no set date for its release....so I decided to take matters into my own hands.

XSD2CS (download here) is a Custom Tool Add-In to Visual Studio that converts an XML schema into CodeSmith-compatible classes in C#. Using this tool, you can add an XSD to a Visual Studio project and then automatically generate the same classes that CodeSmith uses, allowing you to use both the CodeSmith explorer and Visual Studio to develop your standalone templates.

To setup the XSD2CS custom tool, you must have CodeSmith version 3.2 installed. Navigate to the XSD2CS installation folder and edit the setup.bat file so that it points to your CodeSmith install directory as well as to your .NET Framework bin directory. Run the setup.bat file to copy the CodeSmith.Engine.dll, export the types in XSD2CS.dll to COM and add the appropriate Registry entries for this custom tool.

Once you've created your XSD file, add the file to your project and set the Custom Tool property to "CodeSmithXsdConverter" (no quotes). After you've done that, you'll get an associated .cs file. To see it, make sure that Show All Files is selected at the top of the Solution Explorer. From then on, whenever you change the contents of your XSD file, the classes will be regenerated.

Technorati : Code Generators, CodeSmith, Custom Tool, XmlProperties, Xsd
Del.icio.us : Code Generators, CodeSmith, Custom Tool, XmlProperties, Xsd