OakLeaf Systems

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Updated SQL Server Data Services (SSDS) Test Harness: Northwind REST and SOAP Uploads

The SSDSNwindEntitiesCS.sln sample project for my “Test-Drive SQL Server Data Services” cover article for the July 2008 issue of Visual Studio Magazine has been updated to SSDS Sprint 3, adds a couple of features, and has a few start-up bugs excised.

New Features:

• The program attempts to create an authority if you haven’t added one.
• A splash screen opens during the initial query for ACE container and entity object because the process is very slow with a moderate number of objects.

Updates:

• The new 'application/x-ssds+xml' MIME type replaces ‘application/xml’.
• The containerUri = response.Headers[HttpResponseHeader.Location]; instruction which now returns null has been replaced with concatenated constants as the containerUri value. This eliminates a problem when creating new containers with REST operations.

Download: You can download the SSDSNwindEntitiesCS project here (114-KB SSDSNwindEntities.zip file at Skydrive).

My Updated SQL Server Data Services (SSDS) Test Harness: Northwind REST and SOAP Uploads of July 27, 2008 has more details and screen captures.

Added: July 28, 2008

Stephen Forte’s Impedance Mismatch Post Elicits Responses from Domain-Driven Designers

The “Stephen Forte Embraces the Impedance Mismatch” topic of my LINQ and Entity Framework Posts for 7/14/2008+ post quotes Steve’s Impedance Mismatch post of June 27, 2008, which eschewed “object-fist” in favor of “data-first” design.

I assumed that Steve’s post would brew a storm of controversy, especially since he’s a member of the Entity Framework’s new Advisory Council. However, it turned out to be a tempest in a teapot as described in my Stephen Forte’s Impedance Mismatch Post Elicits Responses from Domain-Driven Designers of July 27, 2008.

Bidirectional Serialization of LINQ to SQL Object Graphs with Damien Guard’s T4 Template in VS 2008 SP1

LINQ to SQL developer Damien Guard, who joined the DPG in May 2008, announced the availability of a customizable T4 template to generate LINQ to SQL classes from the *.dbml file in his LINQ to SQL T4 template reloaded post of July 23, 2008.

A couple of minor modifications to the L2ST4.tt template file and addition of a reference to and using directive for the System.Runtime.Serialization namespace enables full-object-graph serialization of LINQ to SQL entities. This brings parity to both VS 2008 OR/M tools in the serialization department.

Read more at Bidirectional Serialization of LINQ to SQL Object Graphs with Damien Guard’s T4 Template in VS 2008 SP1 of July 24, 2008 (updated 7/27/2008) and download the sample C# project.

Mike Taulty Posts Five ADO.NET Data Services Screencasts

Following are titles with links to the five Astoria screencasts that Mike posted on July 25, 2008:

• ADO.NET Data Services (VS08 Sp1 B1) - Querying
• ADO.NET Data Services (VS08 Sp1 B1) - Surfacing Data
• ADO.NET Data Services (VS08 Sp1 B1) - Silverlight Client
• ADO.NET Data Services (VS08 Sp1 B1) - AJAX Client
• ADO.NET Data Services (VS08 Sp1 B1) - .NET Clients

Ryan Dunn Makes IE Display SQL Server Data Services New application/x-ssds+xml MIME Type

Internet Explorer now prompts you to download SQL Server Data Services (SSDS) content that uses the new (post-Sprint 3) application/x-ssds+xml MIME type instead of application/xml.

Ryan’s Rendering POX for SSDS in Internet Explorer post of July 25, 2008 gives you the magic Registry hack to make IE display SSDS’s POX wire format.

Matthieu Mezil Shows You How to Instantiate a New EntityKey

Entity Framework uses EntityKeys to represent unique entities in its cache. EntityKeys are immutable, so Matthieu shows you how to instantiate new ones in his EntityReference change key post of July 25, 2008.

Phani Raju Demonstrates Using the AJAX 4.0 Preview’s DataView Control with ADO.NET Data Services

Phani’s ASP.NET AJAX 4.0 Master-Details View with the DataView and ASP.NET AJAX 4.0 Master-Details View with the DataView , Part 2 posts of July 24 and 25, 2008 use the ASP.NET AJAX 4.0 CodePlex Preview 1’s templating engine and DataView control to build master-details pages.

Phani uses Astoria and its AJAX Client Library for ADO.NET Data Services from CodePlex as the data source.

Jim Wooley: Quickly create an IEnumerable<T> from a DataReader

Jim’s Filling an object from a DataReader with LINQ using DataContext.Translate post of July 25, 2008 shows you how to use the DataContext.Translate<T>() method to create an entity set from a database table without mapping for the DataContext.GetTable() method.

Eric White Analyzes Responses to his “Are Developers Using LINQ?” Post

Eric received many detailed responses to his Are Developers Using LINQ? post of July 23, 2008. In his Are Developers Using LINQ? (Part 2) post of July 25, 2008, he describes these four common LINQ usage scenarios:

• Using LINQ to Objects and LINQ to XML … .
• Using query expressions … to access objects and XML.
• Using LINQ to SQL or LINQ to Entities to access a database.
• Using LINQ implemented using an IQueryable provider to something other than a SQL database.

and noted that there were only a few negative responses.

Keith J. Farmer points out in a comment to the first post that folks still call LINQ “link,” consider LINQ to SQL to be the only implementation, and believe SQL Server is the only LINQ-enabled data provider.

Ruurd Boeke Joins Microsoft’s Silverlight (not Entity Framework) Team

Ruurd notes in his I’m joining Microsoft!! post of July 25, 2008 that:

I was torn between working on EF (which, I’ve been very involved with lately) and on Silverlight. In the end, Silverlight won, because I strongly believe it to be the strongest contender in the client space.

I’m disappointed that Ruurd didn’t choose to join the Entity Framework team. His six-part Introducing Entity Framework Contrib: Easy IPoco implementation V 0.1 series used Postsharp to “[a]utomatically implement the IPoco interfaces. The project is aimed at helping you build your domain layer in a more persistence ignorant way than is possible at this moment.”

Note: Ruurd’s eight-part Workflow as controller: Introducing <M,V,C> where M: ViewModel, V : WPF, C : WF describes an inventive framework that uses WF in an unconventional role, although he won’t call his tutorial a framework.

Jeff Currier Defends SQL Server Data Services’ Missing PUT/POST Response Bodies

Mike Amundsen’s REST PUT/POST Responses contain ETag, but not body - Oops! thread of July 24, 2008 in the SQL Server Data Services (SSDS) - Getting Started says:

I note that the PUT results in a return that includes an ETag (yay!), but that there is no body returned for this PUT. I think this is a problem. ETags act as a version hash for the document body. Yet this Response has no body!

Jeff’s What happened to SSDS PUT/POST responses? post of July 25 replies:

Prior to this rollout the only mechanism that we had to communicate version information to you was via the entity body of the response (specifically the version element in the body).  However, now that we have true ETag support we now can simply return this value back to caller via the ETag header per the Http spec.

Jeff attributes the reason for the change to future ingress and egress charges; returning the body would increase downloaded bytes.

Beth Massi Posts Four WPF Forms Over Data Video Segments to the Code Gallery

Her WPF Forms over Data: 2 More Videos! post of July 24, 2008 announces #3 and #4 in the following series about data binding in WPF:

1. How Do I: Create a Simple Data Entry Form in WPF?
2. How Do I: Display Data in a List in WPF?
3. How Do I: Create Lookup Combobox in WPF?
4. How Do I: Hook Up and Display Validation in WPF?

Click the Watch Video link to substitute MediaPlayer for the miniature Silverlight window. VB and C# sample code is downloadable.

Charlie Calvert Continues with Aggregate Operators in his Revived LINQ Farm Series

Charlie’s LINQ Farm: More on the LINQ Aggregate Operators discusses the following LINQ aggregate operators and their overloads:

• Count and LongCount
• Min and Max
• Average
• Sum
• Aggregate

Jeremy D. Miller and Ward Bell Discuss Object/Relational Modeling and the Entity Framework in a Two-Part Podcast

The first ALT.NET Object-Relational Mapping podcast with Jeremy D. Miller and Ward Bell covers Object/Relational Mapping (O/RM) tools and when to use them. Topics include:

Posted by Roger Jennings (--rj) at 8:38 AM 0 comments  

Labels: ADO.NET Data Services, Ajax, ASP.NET Dynamic Data, Astoria, DataContractSerializer, EF v1 Manifesto, Entity Data Model, Entity Framework, Entity Framework v2, Entity SQL, eSQL, LINQ, LINQ to Entities, LINQ to SQL, NHibernate, O/RM, O/RM Persistence Ignorance, Persistence Instance, POCO, Silverlight 2, SQL Server Data Services, SSDS, WPF

Note: This post is updated daily or more frequently, depending on the availability of new posts.

Danny Simmons: Serializable Entity Framework Object Graphs Coming in Next CTP

Hidden in an innocuous "WCF & Entity Framework not working as expected" thread started on October 31, 2007 in the ADO.NET Entity Framework and LINQ to Entities (Pre-release) forum is this bombshell dated March 28, 2008:

[W]e realize that it would be nice to have graphs of entities just serialize by default without these kinds of gymnastics, but there are some hard problems with doing this using WCF in an interoperable way given that EF relationships are always bi-directional which means that EF always has graphs not trees and the cycles in graphs cause problems with the default WCF (DataContract) serializers.  Happily we've been working with the WCF team, and they came up with a great way to deal with this, so the next CTP of the EF will serialize entity graphs by default. [Emphasis added.]

If true, this new capability will obsolete Danny's EntityBag (Perseus) project. It wouldn't appear to eliminate the need for Ruurd Boeke's serialization to a lightweight client project, at least for Silverlight clients, but it might affect the market for IdeaBlade's DevForceEF product.

The inability of the DataContractSerializer to handle cyclic references appeared to be a major stumbling block to enabling an "out-of-the box multi-tier story" for LINQ to SQL. Hopefully, Matt Warren and his C# team will take another look at enabling n-tier deployment for LINQ to SQL projects.

Thanks to Julie Lerman for the head's up: Hallelujah - Entity Graphs will be XML serializable.

Added: 3/28/2008

Performance Problems Fixed for VB Queries with Filters Over Nullable Columns

Tim Ng reports in the VB Team's LINQ to SQL (and LINQ to Entities) performance improvements post of March 28, 2008 that LINQ to SQL and LINQ to Entities queries are "now smart enough to pass three-value logic from VB to SQL." This means you'll get

SELECT [t0].[CustomerID] 
FROM [dbo].[Orders] AS [t0] 
WHERE [t0].[EmployeeID] = @p0 
-- @p0: Input Int (Size = 0; Prec = 0; Scale = 0) [123]

instead of

SELECT [t0].[CustomerID] 
FROM [dbo].[Orders] AS [t0] 
WHERE (COALESCE( 
 (CASE 
 WHEN [t0].[EmployeeID] = @p0 THEN 1 
 WHEN NOT ([t0].[EmployeeID] = @p0) THEN 0 
 ELSE NULL 
 END),@p1)) = 1 
-- @p0: Input Int (Size = 0; Prec = 0; Scale = 0) [123] 
-- @p1: Input Int (Size = 0; Prec = 0; Scale = 0) [0]

when you filter on the nullable Orders.EmployeeID field.

Added: 3/28/2008

Ajax World: "Microsoft SSDS - Dumb Name, Brilliant Idea!"

Tim Negris' AjaxWorld Magazine article of March 17, 2008 has this deck: "A Google News search for Microsoft SSDS, the SQL Server Data Services it announced last week, yields just six articles." If Tim had broadened his search beyond New articles, he would have found hundreds of SSDS blog posts by March 17.

Tim explains the article's title:

To understand the potential impact of Microsoft's cloud strategy and SSDS service initiative on Oracle, IBM/DB2, Sun/MySQL, Ingres, Postgres and other relational or post-relational database management systems, it's important to note that SSDS has nothing to do with SQL. SQL Server Data Services is a dumb name because, while the service is apparently based at the storage level on Microsoft's SQL Server technology, it doesn't use SQL (Structured Query Language) or a variant of it to query the information in the database. It uses a new Microsoft language called LINQ.

He goes on to say:

LINQ enables full text search, page search, and presumably things like picture and audio search against items stored in the Microsoft SSDS database.

and notes earlier:

Keep in mind that "data" [stored by SSDS] can mean way more than inventory or contact records in columns and rows of text and numbers. Data can mean documents, e-mails and web pages, or music and video.

Neither of the preceding two excerpts are correct, at least for SSDS v1.

SSDS's only connection with AJAX that I can come up with is the promise of a JSON wire format for a future version.

Thanks to the SQL Server Data Services team for the heads-up: Interesting take from Rick Negris over at AjaxWorld magazine.

Added: 3/28/2008

Bryan Dawson Compares Performance of Entity Framework' eSQL and LINQ to Entities with SqlClient and T-SQL

Bryan follows up his two previous performance-related posts (Exploring Performance Part 1 of February 4, 2008 and Exploring Performance Part 2 of February 11, 2008) with ADO.NET Entity Framework Performance Comparison of March 27, 2008.

Bryan analyzes execution time of conventional and paging queries with no tracking for object queries. His graph shows the Entity Framework is about 4 to 5 times slower than SqlClient for paging queries. However, I believe Bryan's calculations are wrong for Entity SQL and LINQ to Entities paging queries.

My calculations, based on 100 tries with 64 ms. and 87 ms. for Entity Client and LINQ to Entities respectively appear in the Corrected column:

The ratios between execution times for Standard Query and Paging Query don't correspond, but Standard Query and Corrected ratios track closely.

Moral of this story: It pays to apply reasonableness tests before publishing performance results.

Added: 3/28/2008

Noam Ben-Ami Explains Simple Stored Procedure Mapping and Raises a Red Flag

Noam's Stored Procedure Mapping of March 26, 2008 (which appeared in the ADO.NET Team Blog on March 28) provides a script to create simple Product and Order tables as well as INSERT, UPDATE, and DELETE stored procedures for the Product table. Noam adds this caveat in his instructions:

At this point it is appropriate to note the first stumbling block: At this time, the Entity Framework requires you to map all three stored procedures. You cannot map only the delete stored procedure. In addition, the entity type must still be mapped to a table, view, or query view.

The requirement to map all three updated stored procedures is unadvertised but mentioned in a newsgroup post. Noam says the following about this requirement:

We hope to provide a solution for this issue in future versions of the Entity Framework, but for now, this is what needs to be done for associations whose target multiplicity, with respect to the type being mapped, is 1 or 0..1. In other words, the “reference” end of the association needs to be mapped to the keys of the target type.

I've never heard of a restriction that prevents executing update stored procedures against an Entity instance mapped to a FunctionImport. I've retested my initial test code that executes the INSERT, UPDATE and DELETE stored procedures and don't encounter a problem.

Added: 3/28/2008

Migrating to SQL Server Stored Procedures with the EDM Designer December CTP 2

Migrating the data source for Entity Framework (EF) EntitySets and AssociationSets from dynamic Entity SQL (eSQL) to dynamic SQL (T-SQL) commands to stored procedures (sprocs) with the Entity Data Model (EDM) Designer December CTP 2 isn't a walk in the park.

There's little in the way of official or unofficial documentation that specifies EF's requirments for supporting sprocs as data sources and data modification agents. What little information is available is buried in ADO.NET Team members' replies to a few users questions in the ADO.NET Entity Framework and LINQ to Entities (Pre-release) forum.

Earlier ADO.NET Team posts don't cover the EDM Designer's role in the process. For example, How to Map Stored Procedures using the ADO.NET Entity Framework of September 14, 2007 covered only the schema syntax and didn't discuss associations or updates. The earlier Using Stored Procedures for Change Processing in the ADO.NET Entity Framework of March 8, 2007 is more than a year old.

My Migrating to SQL Server Stored Procedures with the EDM Designer December CTP 2 post provides illustrated step-by-step instructions for moving the Products, Supplier, and Categories EntitySets from default eSQL queries and commands to a set of T-SQL stored procedures for EntitySet hydration and Entity instance updates, and covers mapping Associations as well as stored procedures for populating AssociationSets.

A future post will show the code required to test the migration.

Added: 3/27/2008

The "Ruurd Boeke Plans a Entity Framework Variant with a Lightweight Client" topic of  LINQ and Entity Framework Posts for 3/17/2008+ describes Ruurd's objectives for a disconnected Entity Framework component with a lightweight client that can be used in a Silverlight project:

So, the end result should be an easy way to serialize and deserialize objects on the client and server, and a way to build up the object correctly on the server so it can be attached to the context again.

Ruurd's approach differs considerably from Danny Simmons' Perseus: Entity Framework EntityBag project on the MSDN Code Gallery, which I described in the "Retrieve and Update Entities Across Tiers with WCF and the EntityBag" sidebar for my "Model Domain Objects with the Entity Framework" for the March 2008 issue of Visual Studio Magazine.

His Relation support for n-tier EF post of March 27 provides code and message examples for serializing Entity updates with WCF's DataContractSerializer. You can compare his messages on the wire with those for Perseus that I described in my Logging EntityBag's SOAP Request and Response Messages post of January 29, 2008.

Ruurd concludes:

The API I have created is monstrous. Also, I think there are quite a few situations where this will break horribly.

Once I get just a little happier about it, I will do a screencast building a server using Entity Framework and a Silverlight client that is editing.

Added: 3/27/2008

Colin, an EF developer, starts his new blog with ADO.Entity Framework: Stored Procedure Customization post of March 26, 2008 which provides documentation for the Entity Framework Extensions described in this week's "Entity Framework Extensions Library Available for Download from MSDN Code Gallery" topic.

His post describes how to use the extensions to simplify running arbitrary stored procedures and T-SQL commands as store commands against an ObjectContext, materialize them as objects, track changes, and deal with multiple result sets. The post describes the following coding patterns:

• Creating and executing store commands
• Materializing typed results
• Tracking results
• Multiple result sets

Alex James waxes enthusiastic about EF Extensions in his Colin Meek talks about his EF Extensions post of the same date:

Here is a little snippet that illustrates what is possible [with a typed materializer]:

var results = context
.CreateStoreCommand("GetCategories", CommandType.StoredProcedure)
.Materialize<Category>()
.Bind(categorySet);

What this does is execute the GetCategories stored proc, turning results into Category objects and binds them to the categorySet entityset so that changes can be tracked/managed by the Entity Framework.

Update 3/27/2008: Diego Vega adds more praise for the extensions in his Entity Framework Extensions (EFExtensions) Project available in CodeGallery post of March 27, 2008.

If you're populating entities with stored procedures, be sure to read Colin's post, download the EFExtensions and EFExtension Samples and give the sample project a test drive.

Added: 3/26/2008

Freek Leemhuis Scorns LINQ to SQL and Others Pile On

LINQ to SQL problems, which Freek posted on March 26, 2008, takes Anko Duizer's LINQ to SQL - TOP 5 of problems post of March 14, 2008, and contributes explanations of why Freek considers the following problems serious:

1. LINQ to SQL is designed for a two tier scenario. LINQ to SQL does not fit natural[ly into] a multi layer architecture.
2. It is not possible to tune the SQL Server queries generated by LINQ to SQL.
3. LINQ to SQL generates dynamic SQL. The technology strongly motivates use [olf] dynamic SQL instead of stored procedures.
4. LINQ to SQL encourages ... "SELECT *" quer[ies.]
5. The cache mismatch problem. LINQ to SQL materialize[s] your objects "once" in the cache, this can give unexpected query results.

Paul Gielens adds #6 in his LINQ to SQL - TOP 5 + 1 of problems post of March 25:

#6 LINQ to SQL is intended for direct 1-1 mapping of a Microsoft SQL server database to .NET classes. LINQ to SQL is not a natural fit in typical enterprise scenarios.

LLBLGen's Frans Bouma pronounces judgment in his detailed comment to Paul's post:

Item 3, 4 and 5 for example are noise or wrong. ... Your #6 point is also noise, sorry :). ... Point 2 is valid.

and recites his usual litany of complaints about LINQ to SQL. Be sure to read the rest of the comments.

Note: Freek has written about LINQ to SQL previously. Click here for his posts tagged LINQ.

More Reasoned Views of LINQ to SQL Issues

Freek also includes a reference to Bob Beauchemin's MHO: LINQ to SQL and Entity Framework: Panacea or evil incarnate? Part 1 post of February 13, 2008. However, Bob generally debunked #2, #3, and #4 in this post. See the "Bob Beauchemin Starts a Series About LINQ to SQL and Entity Framework" of LINQ and Entity Framework Posts for 2/25/2008+.

Similarly, Bob's later posts offer rational evaluations of other topics. See the "Bob Beauchemin Addresses LINQ to SQL/Entity Framework Worry Points 4, 5 and 6" topic of LINQ and Entity Framework Posts for 3/6/2008+.

As to #1, LINQ to SQL fits without difficulty into a multi-layer architecture. The issue is its problems in with n-tier architecture. Keith Farmer made a strong point in his LINQ and 3-Tier Dogma post of November 6, 2007:

I know someone somewhere is going to accuse me of heresy for what I'm about to state, but it's something I've wanted to get off my chest for a couple years, since I first encountered LINQ to SQL, and started to realize what it meant...

The 3-Tier Model is just a pattern.

And, like all patterns,

It is meant to serve our needs, not the other way around.

Don't get hung up on academic concerns such as how many tiers you have in your system.  You'll just waste the time you could spend doing things you actually need to do.  Such as watching Avatar or playing Warcraft.

As evidenced by the amount of work being devoted to change tracking in disconnected (i.e., n-tier) scenarios, the n-tier issue might not remain for long.

Added: 3/26/2008

Matthew Hunter Updates LINQ to SQL Entity Base 1.0 to Beta 1.2

LINQ to SQL Entity Base 1.0 now supports change tracking for deletions and can perform cascade deletions. His New Version of LINQ to SQL Entity Base Class (Beta 1.2) post of March 26, 2008 adds:

I guess from my point of view it's feature complete for ASP.NET use, assuming you store the entities in the session.

Still have to work on serialization for WCF and other uses.

Here's the CodePlex link: LINQ to SQL Entity Base Class Version 1.0 Beta 2.0

The "Matt Hunter: Another Disconnected LINQ to SQL Project" topic provides more details on LINQ to SQL Entity Base.

Added: 3/26/2008

Update 3/28/2008: Implementing Deletion Change Tracking adds a SetAsChangeTrackingRoot() method to establish the baseline original values of an object and implements cascading deletions on the client.

Avi Wortzel Demonstrates a Complete Mock Object for LINQ to SQL

Israeli developer Avi Wortzel's Testing LINQ to SQL with Mock Object – a complete solution post of March 25, 2008 explains:

I decided to implement my own solution which includes two parts:

1. The ability to create database states [to] test the queries against ... (without the need of active SQL Server [o]n the air).

2. The ability to mock DML operations without actually execut[ing] them (and to create a database-less tests [= Tests that don’t have an active instance of SQL server to run]).

For the first part I used the Repository pattern (as the other guys suggested) and I referred the class which represent[s] the table as an in-memory one. It’ll give us the ability to query it with LINQ to Object. To achieve the second part I mocked the SubmitChanges() method from the DataContect class.

It appears to me to be a workable approach for removing SQL Server from the test loop. Avi's downloadable example uses Oren Eini's Rhino Mocks framework.

Added: 3/26/2008

Doug Rothaus: VB's XML Axis Properties and LINQ to XML for Accessing Descendent and Ancestor Elements

His VB XML Cookbook, Recipe 2: Descendants and Ancestors post of March 19, 2005, which didn't appear in the Visual Basic Team blog until March 25, "shows how you can access descendant and ancestor elements in an XML document using Visual Basic, XML Axis properties, and LINQ to XML objects."

Julie Lerman: Build Before Changing Entity Framework XML Files to Resources

Her Tip for working with embedded Ent. Framework models with Beta3 of March 25, 2008 explains Beta 3's problems with moving *.csdl, *.msl and *.ssdl files from the execution folder to project resources.

K. Scott Allen Explains Outer Joins with the LINQ GroupJoin Operator

Scott digs into creating the equivalent of SQL's LEFT OUTER JOIN with the LINQ GroupJoin, SelectMany, and DefaultIfAny operators in his Inner, Outer, Let's All Join Together With LINQ post of March 24, 2008.

Colin Meek posted ADO.NET Entity Framework Extensions to the ADO.NET Entity Framework & LINQ to Relational Data section of the MSDN Code on March 24, 2008.

The undocumented extensions claim to simplify "various tasks in the Entity Framework including store command execution, entity state management and stored procedure mapping."

Update 3/26/2008: Documentation is available. See the "Colin Meek: EF Extensions for Custom Stored Procedures with the Entity Framework" topic.

Anything that simplifies stored procedure mapping is appreciated.

Thanks to Julie Lerman for the heads-up.

Live Demo of CSDL, MSL, and SSDL File Content for Entity Framework Inheritance and Other Scenarios

Asad Khan's downloadable Entity Framework Mapping Helper project from the MSDN Code Gallery is an interactive Windows form that displays the XML content of the CSDL, MSL, and SSDL layer for a variety of different scenarios. These include combinations of inheritance types (table per hiearchy, table per type, table per complex type, or mixtures of all three) with entity splitting, abstract types, complex types, and associations.

The project has self-contained *.csdl, *.msl and *.ssdl files for what appears to be a country club, so all you need to do is mark one of the inheritance check boxes and click the Display Mapping button to start.

Julie Lerman's EF Mapping Helper on Code Gallery post of March 24, 2008 provides a more detailed description and several screen captures of the Mapping Helper in action.

Andy Conrad: New Features in Store for ADO.NET Data Services V2

According to Andy's Astoria team is hiring! post of March 24, 2008:

We have positions open for Developers and Program Managers.  Currently, we are finishing up V1 and planning for V2, so it’s a great time to join the team.  For V2, we are thinking about expanding the project into new areas (e.g. offline support, sync, dynamic language support) so there will be plenty of new and exciting work.

Adding resources to the Astoria team also bodes well for its future as the on-premises version of SQL Server Data Services.

Eugenio Pace Continues His LitwareHR Series with Data Caching Episode

Eugenio ported the LitwareHR hosted multi-tenant demo app from SQL Server 2005 to SSDS, as described in his SQL Server Data Services - SSDS - New version of LitwareHR post during MIX 08.

His LitwareHR on SSDS - Part III - Data access enhancements 1: caching post of March 24, 2008 describes client-side caching at the Web services layer with a CacheManager (based on the Enterprise Library's caching block) and Repository to get and store cached and uncached items.

Here are links to earlier episodes:

The LitwareHR on SSDS - Part II - The data access layer post explains that the new client data model for SSDS implements an SSDS SOAP Client, ProxyFactory and Repository<T> stack, which replaces the previous SQL Server-based version's Data Access Application Block (DAAB) and MultiTenantDataAccess layer.

His LitwareHR on SSDS - Part I - Multi-tenancy & Flexibility post of March 14, 2008 explains SSDS's Authority-Container-Entity (ACE) storage architecture and how LitwareHR uses it to store individual Position and Resume entity sets for each tenant.

Justin Etheredge Continues Improvement to LINQ to Mock (Moq) Implementation for ASP.NET MVC

The Grouping expectation verification in Moq post of March 24, 2008 documents Justin's continued effort to use LINQ to Mock (a.k.a. Moq) to provide a complete mocking framework for the ASP.NET Model-View-Controller add-in.

Click here for more about LINQ to Moq and ASP.NET MVC, plus Justin's previous episodes.

LINQ to Firebird (Entity Framework Provider for Firebird) Is Progressing

Jiří Činčura's Entity Framework and Firebird (Part 2) post of March 24, 2008 describes the progress to date on creating an Entity Framework-enabled data provider for the Firebird open-source database (formerly Borland's InterBase RDBMS). The developer, jirka, says:

We're moving, slowly, step-by-step. And new functionality is coming in view. ... Yes, it's a "LINQ to Firebird", in fact LINQ to Entites connected to Firebird.

Here's the initial post of March 20, 2008: Entity Framework and Firebird.

Another from the How Did I Miss That? Department:

Matt released v1.0 Beta 1.1 of his LINQ to SQL Entity Base - Disconnected LINQ CodePlex project on March 20, 2008. Matt describes his project:

LINQ to SQL Entity Base is a simple base class that is primarily designed to support LINQ to SQL in a disconnected way, which is one of the shortcomings of the LINQ to SQL technology at the present time. This is highly useful in an n-Tier or ASP.NET environment where disconnected functionality is relavent.

LINQ to SQL Entity Base includes change tracking support in a "disconnected from data context" mode.

LINQ to SQL Entity Base also supports automated re-syncing of entities to a data context. This includes all entities under a entity tree which is basically all parent --> child type relationships.

There's more background information on LINQ to SQL Entity Base in Matt's .Net Blog for Software Developers blog.

Benjamin Eidelmann: A Disconnected LINQ to SQL Project

From the How Did I Miss That? Department:

Jose Marcenaro, Benjamin Eidelman and Ricardo de Guzmán of Argentine developer Tercer Planeta have posted the Unplugged LINQ to SQL Generator project (ULINQ) to CodePlex. Benjamin says:

Fitting the LINQ to SQL technology in your own N-tier application framework may require writing custom code for the entities.

This project implements a custom tool for generating code from a DBML model, which you may use instead of the default MSLinqToSQLGenerator in C# or VB projects.

Click here to read background details on the ULinqToSQLGenerator from the Tercer Planeta blog.

Posted by Roger Jennings (--rj) at 4:21 PM 0 comments  

Labels: ADO.NET Data Services, ASP.NET MVC, Astoria, DataContractSerializer, EDM, EF Extensions, Entity Data Model, Entity Framework, Entity Framework Beta 3, Entity SQL, EntityBag, Firebird, LINQ to Firebird, LINQ to Mock, LINQ to SQL, LINQ to SQL Entity Base, LINQ to XML, Serialization, SQL Server Data Services, SSDS, VB 9.0

Rick Strahl's DataContractJsonSerializer in .NET 3.5 post of December 29, 2007 describes the .NET Framework 3.5's new DataContractJsonSerializer (DCJS) class from the System.Runtime.Serialization.Json.DataContractJsonSerializer namespace, which contains the serializer and related objects for the lightweight JavaScript Object Notation (JSON) transport (RFC 4627). JSON is one of ADO.NET Data Service's two wire formats, as noted in the "Julie Lerman: Astoria to Become ADO.NET Data Services" topic of Link and Entity Framework Posts for 12/10/2007+. (The Astoria Team intends to add plain old XML (POX) format to v1.0 by RTW.)

Quick Summary: JSON serialization with DCJS requires about 25% to about 40% fewer bytes than the XML DataContractSerializer (DCS) class to represent moderately complex objects, such as Northwind Order and their their associated Order_Detail objects, and takes about 25% longer than the DCS class to serialize and deserialize.

However, Windows Communication Foundation (WCF) serializes JSON streams to an internal XML Infoset wire format that requires about 50% more bytes than DCS to hydrate an object across a process boundary. Adding type="JSONDataType" attributes to each member's element contributes the additional bytes.

The added time to serialize a moderate-sized object collection and the dramatic increase in the size of WCF messages indicates that JSON serialization isn't as lightweight as one would expect from its simple syntax.

You can download the VB 9.0 code for the JsonSerializationWinVB.sln test harness project  here. The project requires Visual Basic 2008 Express or Visual Studio 2008 Standard or higher but doesn't need a database connection. (Mock generic data objects are provided.)

The Test Harness

Rick's sample code uses simple, lightweight objects and I wanted to verify whether the DCJS class actually was lighter in weight and as performant or better than the .NET Fx 3.0's DataContractSerializer (DCS) class with more complex classes containing a variety of data types, including Nullable<T>, and generic collections. So I created the following two classes with the LINQ In-Memory Object Generation (LIMOG) utility that's described in my Serializing Object Graphs Without and With References post of November 21, 2007.

Here are C# versions of the class definitions for the Order and Order_Detail objects used in the tests:

[DataContract(Name = "Order", Namespace = "")] 
public class Order { 
 [DataMember(Name = "OrderID", Order = 1)] 
 public int OrderID { get; set; } 
 [DataMember(Name = "CustomerID", Order = 2)] 
 public string CustomerID { get; set; } 
 [DataMember(Name = "EmployeeID", Order = 3)] 
 public int? EmployeeID { get; set; } 
 [DataMember(Name = "OrderDate", Order = 4)] 
 public DateTime? OrderDate { get; set; } 
 [DataMember(Name = "RequiredDate", Order = 5)] 
 public DateTime? RequiredDate { get; set; } 
 [DataMember(Name = "ShippedDate", Order = 6)] 
 public DateTime? ShippedDate { get; set; } 
 [DataMember(Name = "ShipVia", Order = 7)] 
 public int? ShipVia { get; set; } 
 [DataMember(Name = "Freight", Order = 8)] 
 public decimal? Freight { get; set; } 
 [DataMember(Name = "ShipName", Order = 9)] 
 public string ShipName { get; set; } 
 [DataMember(Name = "ShipAddress", Order = 10)] 
 public string ShipAddress { get; set; } 
 [DataMember(Name = "ShipCity", Order = 11)] 
 public string ShipCity { get; set; } 
 [DataMember(Name = "ShipRegion", Order = 12)] 
 public string ShipRegion { get; set; } 
 [DataMember(Name = "ShipPostalCode", Order = 13)] 
 public string ShipPostalCode { get; set; } 
 [DataMember(Name = "ShipCountry", Order = 14)] 
 public string ShipCountry { get; set; } 
 [DataMember(Name = "Order_Details", Order = 15)] 
 public List<Order_Detail> Order_Details { get; set; } 
} 

[DataContract(Name = "Order_Detail", Namespace = "")]
public class Order_Detail {
 [DataMember(Name = "OrderID", Order = 1)]
 public int OrderID { get; set; }
 [DataMember(Name = "ProductID", Order = 2)]
 public int ProductID { get; set; }
 [DataMember(Name = "UnitPrice", Order = 3)]
 public decimal UnitPrice { get; set; }
 [DataMember(Name = "Quantity", Order = 4)]
 public short Quantity { get; set; }
 [DataMember(Name = "Discount", Order = 5)]
 public float Discount { get; set; }
}

LIMOG adds the Name, Namespace, and Order attribute/value pairs; DCJS processes Name and Order attribute values and ignores Namespace values. I wrote custom SQL commands to generate object initializers for Northwind's last 20 Orders rows with their associated Order Details and Customer rows. A few of the orders are missing Nullable<DateTime> ShippedDate values.

Following are the test harness's four operating modes. Click the image for a full-size capture.

1. JSON (DCJS) Serialization with Parent (Order) Objects Only 

Representative deserialized values appear below the column headers.

2. JSON (DCJS) Serialization with Parent and Child  (Order_Detail) Objects

LINQ queries populate the 1:many Order_Details associations.

DateTime values serialize as "/Date(MillsecondsSince1970-01-01T00:00:00.000 +/- HHMM)/" strings, where HHMM is the offset from GMT in hours and minutes. (RFC 4627 doesn't specify a serialization format for date/time values, which has resulted in free-form encoding by various implementers. Stand-Alone JSON Serialization covers the current .NET implementation for ASP.NET AJAX services created in WCF.)

3. XML (DCS) Serialization with Parent (Order) Objects Only

4. XML (DCS) Serialization with Parent and Child (Order_Detail) Objects

XML Infoset Wire Format for WCF Implementations: Parent (Order) Objects

WCF implementations serialize JSON streams to a "internal" XML Infoset format defined by the Mapping Between JSON and XML white paper. The test harness uses the JsonReaderWriterFactory 's CreateJsonReader method to instantiate an XmlDictionaryReader object whose ReadOuterXml method delivers the wire format as a string.

The semi-formatted text of the Infoset follows the report of its length below the deserialized data:

Note: I could find no documentation for the JsonReaderWriterFactory's CreateJsonReader method or creating the required XmlDictionaryReader object; I believe that this project contains the only publicly available example. (Search Google for CreateJsonReader.) Here's an excerpt from the test harness code that's based in part on Rick Strahl's example:

Dim ser As DataContractJsonSerializer = _ 
 New DataContractJsonSerializer(GetType(List(Of Order))) 
Dim ms1 As MemoryStream = New MemoryStream() 
ser.WriteObject(ms1, lstOrders) 
Dim json As String = Encoding.UTF8.GetString(ms1.ToArray()) 
txtBytes.Text = json.Length().ToString() 
ms1.Close() 
Dim ms2 As MemoryStream = New MemoryStream(Encoding.UTF8.GetBytes(json)) 
' Deserialize into generic List 
ser = New DataContractJsonSerializer(GetType(List(Of Order))) 
Dim jsonOrders As List(Of Order) = _ 
 TryCast(ser.ReadObject(ms2), List(Of Order)) 
' Serialize JSON MemoryStream to WCS XML wire format 
Dim xdrJson As XmlDictionaryReader = _ 
 JsonReaderWriterFactory.CreateJsonReader(ms2, _ 
 XmlDictionaryReaderQuotas.Max) 
xdrJson.Read() 
Dim xml As String = xdrJson.ReadOuterXml() 
xdrJson.Close() 
ms2.Close() 

XML Infoset Wire Format for WCF Implementations: Parent and Child (Order_Detail) Objects

As Rick mentioned in a comment to his post:

@Roger - awesome work checking out the perf considerations. I suppose it's to be expected that JSON is slower since the parsing of the JSON objects is necessarily a bit more complex than parsing the more XML so I'm not terribly surprised that it's slower than XML.

OTOH, you're not likely to use this on two way WCF communications, but only for AJAX/REST scenarios coming from a browser most likely and in that scenario there's not much choice since JSON is so much easier to use on the client than XML.

Atom10FeedFormatter<TSyndicationFeed> and Atom10ItemFormatter<TSyndicationItem> generic objects (for classes derived from SyndicationFeed) are likely to be used for two-way WCF communications with ADO.NET data services. Magnus Mårtensson shows you in his Create your own Syndication Feeds with .NET Framework 3.5 post of November 22, 2007 how to use these two objects (and their RSS 2.0 equivalents) to create and serialize SyndicationFeeds. Guy Burstein offers a similar demonstration in his How To: Create a RSS Syndication Service with WCF post of December 3, 2007.

Added: 1/1/2008

Size of Serialized Messages and Serialize/Deserialize Execution Times

Following is a summary table of the data collected for the four operating modes. Message sizes don't include whitespace. Execution times are the average of five tests conducted by reopening the form for each execution (to eliminate the effects of data caching.)

The preceding data was for a Gateway S-5200D with a dual-core Pentium 2.6 GHz processor running Windows Vista Premium as a virtual client with 1GB RAM assigned under a Window 2005 R2 Virtual Server.

Download the VB 9.0 source code for the JsonSerializationWinVB.sln test harness project  here and give it a try on your development machine.

Updated 12/31/2007: Minor edits and additions

Posted by Roger Jennings (--rj) at 5:01 PM 0 comments  

Labels: ADO.NET Data Services, Ajax, Astoria, DataContract, DataContractJsonSerializer, DataContractSerializer, LINQ, LINQ to Objects, Mocking Frameworks, Northwind, Test Harnesses, VS 2008

Note: Happy New Year! There probably won't be many updates to this linkblog post.

JSON vs. XML DataContract Serialization: Downloadable Test Harness

This VB 9.0 test harness compares the message size and performance of moderately complex objects serialized and deserialized by the NET Framework 3.5's new DataContractJsonSerializer (DCJS) class and the .NET Fx 3.0's XML-based DataContractSerializer (DCS) class.

Here's a screen capture of the test harness's second mode, which displays the formattted JSON string for Northwind Order and Order_Detail objects:

Click the image for a full-size screen capture.

You can download the VB 9.0 code for the JsonSerializationWinVB.sln test harness project  here. The project requires Visual Basic 2008 Express or Visual Studio 2008 Standard or higher but doesn't need a database connection. (Mock generic data objects are provided.) The code is based on Rick Strahl's original version described below.

Rick Strahl Dissects .NET Framework 3.5's JSON Serializer

Rick's DataContractJsonSerializer in .NET 3.5 post of December 29, 2007 delves into the System.Runtime.Serialization.Json.DataContractJsonSerializer namespace, which contains the serializer and related objects for the lightweight JavaScript Object Notation (JSON) transport (RFC 4627). Rick observes that this serializer produces "Plain Old JSON" (POJ?) text without the proprietary __type="TypeName" attribute that ASP.NET 2.0 AJAX Extensions introduced.

To run Rick's code, you must add references to System.Runtime.Serialization (v3.0 for WCF) and System.Service.Model.Web (v3.5 for JSON), and the following directives:

using System.Runtime.Serialization.Json;
using System.Collections.Generic;
using System.IO;
using System.Text;

Add a using System.Runtime.Serialization; directive if you want to decorate classes with [DataContract] instead of [Serializable] attributes and add the [DataMember] attribute to public fields or properties you want to serialize.

Added: 12/30/2007

Paul Yuknewicz: ASP.NET LinqDataSource Guided Tour

The VB Team's LINQ for the Web Using VB (By Paul Yuknewicz) post provides an illustrated guided tour of the LinqDataSource component and LINQ to SQL behind a simple ASP.NET 3.5 Web site. The post is similar to Scott Guthrie's LINQ to SQL: Part 5 - Binding UI using the ASP.NET LinqDataSource Control post of July 16, 2007 except that VB substitutes for C#.

Added: 12/28/2007

Paul Vick: Visual Basic 9.0 Spec (Almost) Cooked

Paul Vick's (Almost) final VB 9.0 language specification posted article of December 27, 2007 (almost) says it all. You can download the latest edition (sans final copy edits) here. According to Paul, the spec covers the following new VB 9.0 features:

• Friend assemblies (InternalsVisibleTo)
• Relaxed delegates
• Local type inferencing
• Anonymous types
• Extension methods
• Nullable types
• Ternary operator
• Query expressions
• Object initializers
• Expression trees
• Lambda expressions
• Generic type inferencing
• Partial methods
• XML Members
• XML Literals
• XML Namespaces

Added: 12/27/2007

.NET Rocks: Joe Duffy on the Parallel Extensions and PLINQ

Carl and Richard interview Joe Duffy, the PM behind the Task Parallel Library (TPL), Parallel Extensions for .NET (PFX), and Parallel LINQ (PLINQ), about parallel (concurrent) programming and multi-core processors in Joe Duffy on the Task Parallel Library of December 24, 2007.  See the "Parallel Extensions" topics in LINQ and Entity Framework Posts for 11/26/2007+ and my PLINQ Gathers Momentum post of December 8, 2006 for more background.

Note: The PFX Team now has a Parallel Programming with .NET blog that has several posts on the use of PLINQ. Links to this blog will be added to future posts.

Added: 12/27/2007

Mike Taulty Discovers that WebDataGen.exe Doesn't Recognize LINQ to SQL Associations

Mike's ADO.NET Data Services - LINQ to SQL and Associations post of December 24, 2007 points out that when you run WebDataGen.exe against the URL of a .NET Data Source based on LINQ to SQL associations are ignored. The metadata that exposes associations as a Property of the String data type. Using an Entity Framework data source designates associations with a typed NavigationProperty and a corresponding Association element group.

This obviously prevents writing LINQ to REST queries that traverse associations. Defining LINQ to SQL entities with hand-written classes, as outlined in Andy Conrad's Linq to REST post of December 10, 2007, enables associations. You can then substitute LINQ queries, such as

var query2 = from c in context.CreateQuery<Categories>("Categories")
             where c.CategoryID == 1
             select new { c.CategoryID, c.CategoryName, c.Products };

for WebDataQueries like

WebDataQuery<Categories> query =
    context.CreateQuery<Categories>("/Categories(1)?$expand=Products");

However, I've found that any attempt to filter collections with Where criteria based on child member values, such as

var query = from c in context.CreateQuery<Customer>("Customers")
            where c.Country == "USA"
            && c.Orders.Any(o => o.Freight > 50)
            orderby c.CompanyName
            select c;

fail with a 400 - Bad Request error because the query string isn't composed properly. (Commenting the Freight amount test enables the query to execute.)

It's not clear to me how restricted LINQ to REST query syntax will be at RTM. It's likely that many Astoria users will load large chunks of data into memory-resident IEnumerable() sequences (probably generic Lists) and query them with client side LINQ to Objects expressions.

Mike Taulty Suggests Cure for Error 7005: 'Namespace' Attribute Is Invalid

The solution in Mike's ADO.NET Data Services and CLR Namespaces post of December 24, 2007 is to "put the types that you're exposing (i.e. the T that you have fed into WebDataService<T>) into a CLR namespace."

So far, I haven't encountered that exception.

A Christmas Present from Milan Negovan: A LINQ Cheat Sheet

You can read and download the two-page Cheat Sheet for Standard Query Operators (method call syntax) at Milan's ASP.NET Resources site.

Mike Flasko Proposes Eliminating Parent Key Duplication with Deep Addressing

Mike Flasko's Design Notes: URI Containment in Astoria post of December 21, 2007 (missing from previous linkblog post) observers that parent keys are duplicated in following case b:

a) entities are always addressable through top-level entitysets, e.g. /OrderLines(1,2)

b) deep addressing through container has redundant information, e.g. /Orders(1)/OrderLines(1,2)  (the container id, “1”, is repeated)

The plan is to annotate the CSDL and CLR object model to indicate a parent/child relationship to eliminate the repetition.

Posted by Roger Jennings (--rj) at 3:40 PM 0 comments  

Labels: ADO.NET Data Services, Astoria, DataContractSerializer, JSON, LINQ, LINQ to Objects, LINQ to REST, LinqDataSource, Parallel LINQ, ParallexFX, PLINQ, Serialization, Task Parallel Library

Updated: 11/25/2007 and 12/12/2007 (Added C# class definitions, plus C# and VB object initialization code. See below.)

Windows Communication Foundation (WCF) defaults to the DataContractSerializer (DCS) for serializing services' object graph payloads to well-formed XML Infosets. DCS is capable of serializing 1:many associations of an object graph, such as Customers -> Orders -> OrderDetails, into a hierarchical XML Infoset, which has a structure that any WS-* SOAP client should be able to deserialize correctly.

An alternative to DCS is the NetDataContractSerializer (NDCS), which can serialize graphs with 1:many and many:1 (bidirectional) associations by preserving association references in ID/IDREF pairs of Id and Ref values. NDCS delivers graph fidelity at the expense of interoperability because it requires clients to support CLR types. A factor in lack of interoperability with other vendors' Web service toolkits is the complexity of schemas genrated by NDCS and Microsoft's patent that appears to cover NDCS's method of serializing/deserializing objects.

Note: The Entity Framework (EF) maps associations to Navigation Properties .

The online help topics for DCS and NDCS use trivial classes with two or three members; no help topics that I've seen demonstrate serializing generic lists. To verify that DCS and NDCS behave as expected with object graphs of moderate to high complexity generated from relational databases, I used my LINQ In-Memory Object Generation (LIMOG) utility. LIMOG autogenerates C# and VB classes designed for serialization with DCS and NDCS. LIMOG also creates object initializers to generate customized List<T> or List(Of T) instances for serialization and deserialization. The three default data sources for classes are Northwind, AdventureWorks, and AdventureWorksLT (Lite) sample databases. Click here to download the AdventureWorks (OLTP)'s entity-relationship diagram in HTML or Visio format; click here to open an HTML entity-relationship diagram of AdventureWorksLT.

Note: The original impetus for creating the LIMOG app was to generate mock objects that could substitute for entities generated by LINQ to SQL and the Entity Framework and speed unit testing. Using mock objects also frees the unit test from infrastructure dependency, which conforms to the separations of concerns principal of software design. Serializing files generated by LIMOG can provide provide payload size and complexity estimates for WS-* Web services.

Here's a screen capture of the LIMOG utility after generating the VB code for the Order class and a List(Of Order) instance with 100 members (click for full-size image):

Note: Marking the Cyclic References check box adds the [DataMember] attribute to members representing many:1 associations.

Serializing Partial Object Graphs with DCS

DCS uses opt-in serialization; DCS serializes classes decorated with a [DataContract] attribute and properties with a [DataMember] attribute only. These two attributes have properties that let you customize the resulting document, such as specifying object and member names and member sequences. (DCS sequences properties alphabetically by default.)

By default, DCS doesn't preserve references by default so it creates duplicates of objects that normally would share a common pointer to the same object in memory (i.e., ReferenceEquals = True). This means that the default DCS implementation can't handle cyclic references (cycles) created by bidirectional 1:many and many:1 relationships. I raised this issue in my LINQ to SQL and Entity Framework XML Serialization Issues with WCF - Part 1 post of October 9, 2007.

DCS offers a PreserveObjectReferences property to solve the preceding problem. However, there are no commonly observed standards for serializing and deserializing the ID/IDREF attributes in XML Infosets required to handle bidirectional references, so Microsoft's XML team purposely made it difficult to set DCS's PreserveObjectReferences property to true.

LINQ to SQL handles this problem by not applying the <DataMember()> attribute to many:1 properties, such as Order.Customer, Order.Employee, Order.Shipper, Order_Detail.Order or Order_Detail.Product. Thus the serializer doesn't follow the paths necessary to include the Products, Categories, Suppliers, Shippers, and Employees classes in the resulting document, which has this structure:

<ArrayOfCustomer xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Customer>
...
<Orders>
<Order>
...
<Order_Details>
</Order_Detail>
...
</Order_Detail>
...
</Order_Details>
</Order>
...
</Orders>
...
</Customer>
</ArrayOfCustomer>

VB code to serialize the CustomerList tree is:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Create)
Dim dcs = New DataContractSerializer(GetType(List(Of Customer)))
dcs.WriteObject(fs, CustomerList)
fs.Close()

and to deserialize it is:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Open, _
FileAccess.Read)
Dim dcs = New DataContractSerializer(GetType(List(Of Customer)))
CustomerList = CType(dcs.ReadObject(fs), List(Of Customer))
fs.Close()

Click here to open a 39.1 kB text file with autogenerated VB code for a set of Northwind classes suitable for serialization with DCS. (The class code is autogenerated from database metadata by a custom VB project, not by LINQ to SQL.) Notice that the [DataMember] attribute is missing from the last one to three members of several classes.

Added 12/13/2007: The C# version of the DCS-formatted class is here and NCDS-formatted class is here.

Click here to open the 125-KB, 5,012-line XML file generated from 100 Order and related Customer and Order_Detail objects by a test harness that executes the preceding functions.

Added 12/13/2007: The C# version of the three functions that generate the three-tier object graph are here and the VB version is here.

Click here to open its 69-line, easily read XSD schema generated by Visual Studio 2008's schema inference engine. (Schema files are renamed with an xml extension so they open in IE.)

EF Beta 2 doesn't serialize many:1 or 1:many associations. It's up to the programmer to write code to download EntityCollections<T>, which correspond to LINQ to SQL's EntitySets. If the client application depends upon many:1 associations, for example to replace surrogate (usually numeric) foreign key values with readable names, it must request the missing EntityReference. EF hides foreign key values whether or not they are surrogate keys, which makes reconstituting many:1 associations unnecessarily difficult.

In either case, you must regenerate the associations with code. The entity classes contain a single read-only property, MemberName As EntityCollection(Of EntityType) for 1:many associations. Many:1 associations generate a read-write MemberName As EntityType property and a MemberName As EntityReference(Of EntityType).

All three properties are decorated with <XmlIgnoreAttribute(), SoapIgnoreAttribute()> attributes and lack <DataContract()> or <DataMember()> attributes, so the WCF doesn't searialize them. Therefore, SOAP clients don't receive the information required for a SOAP request to obtain the entities.

LINQ to Objects simplifies writing the code for client-side reconstructions of associations. However, the code is very "chatty" and thus isn't well suited to service-oriented architecture.

Serializing Complete Object Graphs with NDCS

XML Infosets are well-suited to representing acyclic tree data structures, such as the preceding Customer -> Order -> Order_Detail example, but have serious problems with representing edge-labeled directed graphs that result from the cyclic references (cycles) introduced by adding many:1 relationships. Cyclic references lead to endless loops and an XML document of infinite size. NDCS handles serialization of graphs with cycles by assigning an integer ID value (Id attribute) to each unique element when it's first encountered and then substituting a REF[erence] (Ref attribute) as a pointer to the element when it's encountered again.

Code to serialize the CustomerList graph with NDCS is similar to that for DCS except that DCS specifies the generic class name in the constructor while NDCS's XML document contains the CLR type declaration (System.Collections.Generic.List`1[[NwindObjects.MainForm+Customer, NwindObjects, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]). Here's the NDCS serialization code:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Create)
Dim ndcs = New NetDataContractSerializer()
ndcs.Serialize(fs, CustomerList)
fs.Close()

Deserialization code also is similar to that for DCS:

Dim fs As FileStream = New FileStream("NwindGraphDCS.xml", FileMode.Open, _
FileAccess.Read)
Dim ndcs = New NetDataContractSerializer()
CustomerList = CType(ndcs.Deserialize(fs), List(Of Customer))
fs.Close()

It's easier to view the serialized XML Infoset than to describe its structure, so click here to open the 410-kB, 15,299-line XML file with all 2,626 references preserved. The autogenerated 39.5 kB text file for classes designed for serialization with NDCS is here.

The primary and imported XML schemas inferred for the XML document are available here (203 KB, 7,136 lines) and here (600 bytes). Whitespace added by VS 2008 contributes 90% of the primary schema's original 2 MB file size as inferred; whitespace has been removed from the linked version.

Note: This is a very large schema for a SOAP WSDL file and it uses the xs:import feature that some Web service clients don't support in WSDL files.

Using NDCS for WCS Serialization

Unfortunately, it's not so simple to substitute NDCS for WCF's default DCS on either the service or client component. The Indigo team deliberately obfuscated the substitution process by requiring additional user-written code to enter the serialization and deserialization pipelines, remove the DataContractSerializerOperationBehavior objects from and create and add NetDataContractSerializerOperationBehavior objects to each Operation (see Configuring and Extending the Runtime With Behaviors and WCF's NetDataContractSerializer by Aaron Skonnard.) Autogenerated service proxies also have problems with the service's CLR type declaration for generics (shown above).

Microsoft's party line appears to be: If we can't serialize LINQ to SQL or EF objects in an interoperable manner we'll enforce unidirectional serialization by not serializing many:1 associations (LINQ to SQL) or not serializing any associations (EF). This policy is likely to result in EF sharing LINQ to SQL's failure to provide an "out-of-the-box multi-tier story."

Note: NCDS appears to serialize NHibernate's Bag objects, which provide custom collection types that implement IList<T>. Tim Scott and Greg Banister's Remoting Using WCF and NHibernate post of May 9, 2007 describes a unit test that serializes an NHibernate.Collection.Bag MemoryStream and deserializes it back to the same type. However, this test, which is the in-memory analog of the preceding code snippets, doesn't duplicate WCF serialization. There is nothing in the post that suggests that Bags contain cyclic references. (See Frans Bouma's comment of May 28, 2007 to this post.)

Note: The preceding section was added on November 26, 2007

Probable Patent Coverage of NDCS Serialization

The process by which the NDCS converts a cyclic edge-labeled directed graph to an acyclic edge-labeled directed tree appears to be covered by US Patent 20040239674 "Modeling graphs as XML information sets and describing graphs with XML schema" published on December 2, 2004 and assigned to Microsoft Corp. The inventors are luminaries of Microsoft's early XML Web services efforts: Tim Ewald, Don Box, Keith Ballinger, Martin Gudgin and Stefan Pharies. Here's the abstract of the patent:

Systems and methods for modeling graphs as XML information sets and describing them with XML schema. An edge labeled directed graph is converted to an edge labeled directed tree representing some of the edges directly and some of the edges indirectly. A graph is completely traversed such that all nodes are visited and all edges are traversed. Nodes are included by value initially and then by reference. A schema is provided that describes the structure of an XML tree tha[t] contains graph data.

The patent provides a detailed description of the serialization process with a simple XML Infoset example.

Obviously, intellectual property issues cloud interoperability of services that use NDCS serialization.

Additional References:

• Juval Lowy: Programming WCF Services (O'Reilly)
• Oakleaf Systems Blog: Serializing Cyclic LINQ to SQL References with WCF
• Aaron Skonnard's "Service Station" July 2006 column in MSDN Magazine: "Serialization in Windows Communication Foundation"
• Aaron Skonnard's "Service Station" December 2007 column in MSDN Magazine: "Extending WCF with Custom Behaviors"
• Nicholas Allen's Indigo Blog: Serializing Object Graphs
• James Kovacs' Weblog: Going Around in Circles with WCF
• Steve Seymour's Weblog: Using Interfaces as (WCF) Parameters (Part 1)
• Steve Seymour's Weblog: Using Interfaces as (WCF) Parameters (Part 2)
• Windows Communication Foundation ("Indigo") forum: NHibernate proxy serialization using NetDataContractSerializer in WCF service

Posted by Roger Jennings (--rj) at 5:30 PM 0 comments  

Labels: DataContractSerializer, Entity Data Model, Entity Framework, LINQ, LINQ to Entities, LINQ to Objects, LINQ to SQL, NetDataContractSerializer