WPF : Dynamic Search Driven List Results

At work at the moment I am working on a way way cool customisable search that basically allows user to pick source entities and related entities and then pick what fields they would like to show, this is similar to Microsoft’s CRM Dynamics product, which allows very very sophisticated searches to be produces, by the use of a nice search UI.

Here is what CRM looks like. The search we actually built at work is even better than this, but for NDA reasons I can not show.

image

From our search we create a strongly typed Query object, which is sent across a WCF service boundary and when received at the other end, in converted to dynamic SQL, and is the  run against a SQL database. Yes that right we created our own LINQ effectively. Its call GNRSQL.

Anyway as a side effect from being able to search for anything from anywhere, we also needed the ability for our search results grid to dynamically adjust to any results set. To compound this our results objects are hierarchical in nature, and we are using the Infragistics XamDataGrid, which wants to show things in a hierarchical manner. Which is not what we wanted, so we needed to flatten the results from a hierarchy to a flat structure.

Your 1st thought might be, oh just send a DataTable across the WCF boundary, this is not a good idea, the serialization of DataTable(s) and WCF is very strange, and also incredibly heavy. Basically the namespaces get messed up and so do some of the internal field names, and my god the serialization is so far from pretty.

So we had to try some other things.

My next idea was to use IEnumerable and use LINQ to pluck out only those properties that we wanted to show based on the actually results obtained. Our results contained metadata that we could use to construct a LINQ query which would return a flattened IEnumerable of anonymous types, that had only the properties we wanted. Choosing of properties was driven by examining the actual results metadata. And all this has to happen at runtime. Doing this at design time is dead simple we can just do something like (assuming we have a list of results already)

var newFlattenedResults = (from x in results  select new { ID = x.IDField, DateOfOrder=x.OrderDate });

But how could you create something that could be used in this manner, and tailored to suit the properties returned by the search, but done at runtime. The rest of this post will show you how.

Dynamic Assemblies

.NET comes equipped with the ability to produce dynamic assemblies on the fly at runtime. So lets start there and have a look at a helper class for making this process easy.

   1:  using System.Reflection;
   2:  using System.CodeDom.Compiler;
   3:  using Microsoft.CSharp;
   4:  
   5:  using System;
   6:  using System.Collections.Generic;
   7:  using System.Text;
   8:  
   9:  namespace Dyno
  10:  {
  11:      /// <summary>
  12:      /// Utlilty class. Compliles and runs assemblies 
  13:      /// on the fly, with the option to invoke a method 
  14:      /// and return the results from the chosen
  15:      /// method
  16:      /// </summary>
  17:      public class DynamicCompiler
  18:      {
  19:          #region Public Methods
  20:          /// <summary>
  21:          /// Compiles the code and either returns the compiled code as a Type
  22:          /// or creates compiled code as a Type, and invokes the created Types
  23:          /// method. Where the method is picked by the user
  24:          /// </summary>
  25:          /// <typeparam name="T">Type of T to use</typeparam>
  26:          /// <param name="code">code to compile</param>
  27:          /// <param name="nameSpace">namespace to use for compiled code</param>
  28:          /// <param name="classToLoad">class name</param>
  29:          /// <param name="methodToRun">method to invoke (optional)</param>
  30:          /// <param name="ShouldImvokeMethod">true if you want to 
  31:          /// invoke the method</param>
  32:          /// <returns>Type of T to use</returns>
  33:          public T ComplileAndRun<T>(String code, String nameSpace,
  34:              String classToLoad, string methodToRun, Boolean ShouldImvokeMethod)
  35:          {
  36:              try
  37:              {
  38:                  String lcCode = code;
  39:  
  40:                  var provider = new CSharpCodeProvider(
  41:                          new Dictionary<String, String>()
  42:                              { { "CompilerVersion", "v3.5" } });
  43:  
  44:                  CompilerParameters parameters = new CompilerParameters();
  45:  
  46:                  // Start by adding any referenced assemblies
  47:                  parameters.ReferencedAssemblies.Add("System.dll");
  48:                  parameters.ReferencedAssemblies.Add(
  49:                      typeof(Demo.Data.Person).Assembly.Location);
  50:                  parameters.ReferencedAssemblies.Add(
  51:                      typeof(System.Linq.Enumerable).Assembly.Location);
  52:  
  53:  
  54:                  // Load the resulting assembly into memory
  55:                  parameters.GenerateInMemory = true;
  56:                  // Now compile the whole thing 
  57:                  //Must create a fully functional assembly as the code string
  58:                  CompilerResults compiledCode =
  59:                      provider.CompileAssemblyFromSource(parameters, lcCode);
  60:  
  61:                  if (compiledCode.Errors.HasErrors)
  62:                  {
  63:                      String errorMsg = String.Empty;
  64:                      errorMsg = compiledCode.Errors.Count.ToString() +
  65:                                 " n Dynamically generated code threw an error. n Errors:";
  66:  
  67:                      for (int x = 0; x < compiledCode.Errors.Count; x++)
  68:                      {
  69:                          errorMsg = errorMsg + "rnLine: " +
  70:                                     compiledCode.Errors[x].Line.ToString() + " - " +
  71:                                     compiledCode.Errors[x].ErrorText;
  72:                      }
  73:  
  74:                      throw new Exception(errorMsg);
  75:                  }
  76:  
  77:                  Assembly assembly = compiledCode.CompiledAssembly;
  78:  
  79:                  // Retrieve an obj ref – generic type only
  80:                  object instance = assembly.CreateInstance(
  81:                      nameSpace + "." + classToLoad);
  82:  
  83:                  //invoke the method if needs be, and get results 
  84:                  //back from method invocation
  85:                  if (ShouldImvokeMethod)
  86:                  {
  87:                      if (instance == null)
  88:                          return default(T);
  89:  
  90:                      T result = (T)instance.GetType().InvokeMember(
  91:                             methodToRun, BindingFlags.InvokeMethod,
  92:                             null, instance, new object[0]);
  93:  
  94:                      return result;
  95:                  }
  96:                  else
  97:                  {
  98:                      return (T)instance;
  99:                  }
 100:              }
 101:              catch (Exception ex)
 102:              {
 103:                  Console.WriteLine(String.Format(
 104:                      "An exception occurred {0}", ex.Message));
 105:                  return default(T);
 106:              }
 107:          }
 108:          #endregion
 109:      }
 110:  }

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }So using this helper class we can construct an in memory assembly and call a method within an object within it, or simply return the newly created object within the dynamic assembly.

So lets continue our journey, and look at a small XAML app, where there is a ListView that starts with the following columns

image

Where the XAML for this screen looks like this

   1:  <Window x:Class="DynamicLINQ.Window1"
   2:      xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   3:      xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   4:          WindowStartupLocation="CenterScreen"
   5:      Title="Window1" Height="300" Width="700">
   6:      <DockPanel LastChildFill="True">
   7:          <StackPanel DockPanel.Dock="Top" Orientation="Horizontal"
   8:                      HorizontalAlignment="Stretch" Background="CornflowerBlue">
   9:              <Button x:Name="ShowAll" Content="Show All" Margin="5"
  10:                      Click="ShowAll_Click"/>
  11:              <Button x:Name="ShowSome" Content="Show Some Columns" Margin="5"
  12:                      Click="ShowSome_Click"/>
  13:          </StackPanel>
  14:  
  15:  
  16:          <ListView x:Name="lvItems" >
  17:              <ListView.View>
  18:                  <GridView>
  19:                      <GridViewColumn Header="Age"
  20:                                      DisplayMemberBinding="{Binding Age}" />
  21:                      <GridViewColumn Header="FirstName"
  22:                                      DisplayMemberBinding="{Binding FirstName}" />
  23:                      <GridViewColumn Header="MiddleName"
  24:                                      DisplayMemberBinding="{Binding MiddleName}" />
  25:                      <GridViewColumn Header="LastName"
  26:                                      DisplayMemberBinding="{Binding LastName}" />
  27:                      <GridViewColumn Header="LastName"
  28:                                      DisplayMemberBinding="{Binding LastName}" />
  29:                      <GridViewColumn Header="ID"
  30:                                      DisplayMemberBinding="{Binding ID}" Width="230" />
  31:                      <GridViewColumn Header="DOB"
  32:                                      DisplayMemberBinding="{Binding Dob}" Width="130" />
  33:                  </GridView>
  34:              </ListView.View>
  35:          </ListView>
  36:  
  37:  
  38:      </DockPanel>
  39:  </Window>

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }You can see that the initial columns for the displayed results are static at this point, and are showing the results of being bound to a List<Person> objects, where a Person looks like this

   1:  using System;
   2:  using System.Collections.Generic;
   3:  using System.Linq;
   4:  using System.Text;
   5:  using System.ComponentModel;
   6:  
   7:  namespace Demo.Data
   8:  {
   9:      public class Person : INotifyPropertyChanged
  10:      {
  11:          #region Data
  12:          public Int32 age;
  13:          public String firstName;
  14:          public String middleName;
  15:          public String lastName;
  16:          public AdditionalData personData;
  17:          #endregion
  18:  
  19:          #region Public Properties
  20:          public Int32 Age
  21:          {
  22:              get { return age; }
  23:              set
  24:              {
  25:                  age = value;
  26:                  NotifyPropertyChanged("Age");
  27:              }
  28:          }
  29:  
  30:          public String FirstName
  31:          {
  32:              get { return firstName; }
  33:              set
  34:              {
  35:                  firstName = value;
  36:                  NotifyPropertyChanged("FirstName");
  37:              }
  38:          }
  39:  
  40:          public String MiddleName
  41:          {
  42:              get { return middleName; }
  43:              set
  44:              {
  45:                  middleName = value;
  46:                  NotifyPropertyChanged("MiddleName");
  47:              }
  48:          }
  49:  
  50:          public String LastName
  51:          {
  52:              get { return lastName; }
  53:              set
  54:              {
  55:                  lastName = value;
  56:                  NotifyPropertyChanged("LastName");
  57:              }
  58:          }
  59:  
  60:          public AdditionalData PersonData
  61:          {
  62:              get { return personData; }
  63:              set
  64:              {
  65:                  personData = value;
  66:                  NotifyPropertyChanged("PersonData");
  67:              }
  68:          }
  69:          #endregion
  70:  
  71:          #region INotifyPropertyChanged region
  72:          public event PropertyChangedEventHandler PropertyChanged;
  73:  
  74:          private void NotifyPropertyChanged(String info)
  75:          {
  76:              if (PropertyChanged != null)
  77:              {
  78:                  PropertyChanged(this, new PropertyChangedEventArgs(info));
  79:              }
  80:          }
  81:          #endregion
  82:      }
  83:  
  84:      public class AdditionalData : INotifyPropertyChanged
  85:      {
  86:          #region Data
  87:          private Guid id;
  88:          private DateTime dob;
  89:          #endregion
  90:  
  91:          #region Public Properties
  92:          public Guid ID
  93:          {
  94:              get { return id; }
  95:              set
  96:              {
  97:                  id = value;
  98:                  NotifyPropertyChanged("Id");
  99:              }
 100:          }
 101:  
 102:          public DateTime Dob
 103:          {
 104:              get { return dob; }
 105:              set
 106:              {
 107:                  dob = value;
 108:                  NotifyPropertyChanged("Dob");
 109:              }
 110:          }
 111:          #endregion
 112:  
 113:          #region INotifyPropertyChanged region
 114:          public event PropertyChangedEventHandler PropertyChanged;
 115:  
 116:          private void NotifyPropertyChanged(String info)
 117:          {
 118:              if (PropertyChanged != null)
 119:              {
 120:                  PropertyChanged(this, new PropertyChangedEventArgs(info));
 121:              }
 122:          }
 123:          #endregion
 124:      }
 125:  
 126:  }

So now let us see what would happen if we simulated some other search results coming back, from somewhere else, that should alter what is shown in the search results ListView.


   1:  private void ShowSome_Click(object sender, RoutedEventArgs e)
   2:  {
   3:      DataLoader dl = new DataLoader();
   4:  
   5:      //show some columns from Dynamically compiled LINQ
   6:      //query
   7:      IEnumerable results =
   8:          dl.ObtainFlattenedResults(lvItems,
   9:              people, WritePropertyType.Some);
  10:  
  11:  
  12:      lvItems.ItemsSource = results;
  13:  
  14:  }

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }


.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }


.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }Where people is simply a list of Person objects set up with some Person objects in it (obviously I am faking the search results part, for the sake of this article, the Person objects, would actually be some search results or something coming from the results of a search, but for this example it doesn’t matter, as I am just trying to show you how to dynamically work with LINQ at runtime)

   1:  private List<Person> people = new List<Person>();

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; } .csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }You can see from the above code that we use a method called ObtainFlattenedResults() which is available in the following helper class

   1:  using System;
   2:  using System.Collections;
   3:  using System.Collections.Generic;
   4:  using System.Linq;
   5:  using System.Reflection;
   6:  using System.Text;
   7:  
   8:  using Demo.Data;
   9:  using System.Windows.Controls;
  10:  using System.Windows.Data;
  11:  
  12:  
  13:  namespace Dyno
  14:  {
  15:  
  16:      public enum WritePropertyType { All = 1, Some };
  17:  
  18:      public class EnumerableResultsCreator
  19:      {
  20:          #region Data
  21:          private readonly String generatedNamespace = "DynamicLINQ";
  22:          private readonly String generatedClassName = "ResultsWriter";
  23:          private readonly String generatedMethod = "GetResults";
  24:          private List<Person> originalResults = null;
  25:          private WritePropertyType currentPropertyType =
  26:              WritePropertyType.All;
  27:          #endregion
  28:  
  29:          #region Ctor
  30:          public EnumerableResultsCreator()
  31:          {
  32:  
  33:          }
  34:          #endregion
  35:  
  36:          #region Private Methods
  37:  
  38:          /// <summary>
  39:          /// Writes out code for class to dynamically compile
  40:          /// </summary>
  41:          private String WriteDynamicClass(WritePropertyType currentPropertyType)
  42:          {
  43:              StringBuilder code = new StringBuilder();
  44:              code.AppendLine(" using System; ");
  45:              code.AppendLine(" using System.Collections.Generic;");
  46:              code.AppendLine(" using System.Collections.ObjectModel;");
  47:              code.AppendLine(" using System.Linq;");
  48:              code.AppendLine(" using System.Text;");
  49:              //Where the DataLayer objects live
  50:              code.AppendLine(" using Demo.Data; ");
  51:              code.AppendFormat(" namespace {0}", generatedNamespace);
  52:              code.AppendLine();
  53:              code.AppendLine(" {");
  54:              code.AppendFormat(" public class {0}", generatedClassName);
  55:              code.AppendLine();
  56:              code.AppendLine(" {");
  57:              code.AppendFormat("{0}",
  58:                  this.WriteCodeProperties(currentPropertyType));
  59:              code.AppendLine();
  60:              code.Append(" }");
  61:              code.AppendLine();
  62:              code.Append(" }");
  63:              code.AppendLine();
  64:              return code.ToString();
  65:          }
  66:  
  67:  
  68:          /// <summary>
  69:          /// Either writes a LINQ query string that returns an anomomous
  70:          /// set of object with ALL properties, or just some of the properties
  71:          /// </summary>
  72:          private String WriteCodeProperties(WritePropertyType currentPropertyType)
  73:          {
  74:              StringBuilder builder = new StringBuilder();
  75:              builder.AppendFormat(
  76:                  "n public Object {0}(List<Person> results)",
  77:                      generatedMethod);
  78:              builder.AppendLine("{");
  79:  
  80:  
  81:              switch (currentPropertyType)
  82:              {
  83:                  case WritePropertyType.All:
  84:                      //OK this is a static query that could have been done in  standard LINQ
  85:                      //but this techniqe can be used to create dyanamically created
  86:                      //LINQ queries at runtime
  87:                      builder.AppendLine("var x =(from r in results select new { ");
  88:                      builder.AppendLine("         Age = r.Age,FirstName = r.FirstName,");
  89:                      builder.AppendLine("         MiddleName=r.MiddleName, LastName = r.LastName,");
  90:                      builder.AppendLine("         ID = r.PersonData.ID, Dob = r.PersonData.Dob });");
  91:                      builder.AppendLine("return x;");
  92:                      builder.AppendLine("}");
  93:                      break;
  94:                  case WritePropertyType.Some:
  95:                      //OK this is a static query that could have been done in  standard LINQ
  96:                      //but this techniqe can be used to create dyanamically created
  97:                      //LINQ queries at runtime
  98:                      builder.AppendLine("var x =(from r in results select new { ");
  99:                      builder.AppendLine("         Age = r.Age,FirstName = r.FirstName,");
 100:                      builder.AppendLine("         LastName = r.LastName});");
 101:                      builder.AppendLine("return x;");
 102:                      builder.AppendLine("}");
 103:                      break;
 104:              }
 105:  
 106:  
 107:              return builder.ToString();
 108:  
 109:          }
 110:  
 111:          /// <summary>
 112:          /// Creates the new columns for the Dynamically retrieved
 113:          /// results 
 114:          /// </summary>
 115:          private void CreateListViewColumns(IEnumerable result, ListView lv)
 116:          {
 117:              GridView gv = new GridView();
 118:  
 119:              IEnumerator e = result.GetEnumerator();
 120:              while (e.MoveNext())
 121:              {
 122:                  Type t = e.Current.GetType();
 123:                  PropertyInfo[] infos = t.GetProperties();
 124:  
 125:                  foreach (var item in infos)
 126:                  {
 127:                      GridViewColumn column = new GridViewColumn();
 128:                      column.Header = item.Name;
 129:                      Binding binding = new Binding(item.Name);
 130:  
 131:                      column.DisplayMemberBinding = binding;
 132:                      gv.Columns.Add(column);
 133:                  }
 134:                  break;
 135:              }
 136:  
 137:              lv.View = gv;
 138:          }
 139:  
 140:  
 141:          #endregion
 142:  
 143:          #region Public Method
 144:          /// <summary>
 145:          /// Create the dynamic Type invoke its method
 146:          /// and get the results
 147:          /// </summary>
 148:          public IEnumerable ObtainFlattenedResults(
 149:              ListView lv,
 150:              List<Person> originalResults,
 151:              WritePropertyType currentPropertyType)
 152:          {
 153:              if (originalResults.Count == 0)
 154:                  return null;
 155:  
 156:              this.originalResults = originalResults;
 157:  
 158:              //Create the DynamicCompiler
 159:              DynamicCompiler compiler = new DynamicCompiler();
 160:  
 161:              //Get the newly compiled object
 162:              Object dynaClass = compiler.ComplileAndRun<Object>(
 163:                  this.WriteDynamicClass(currentPropertyType),
 164:                  generatedNamespace, generatedClassName, "", false);
 165:  
 166:              //invoke its method to get the result
 167:              IEnumerable result =
 168:                  (IEnumerable)dynaClass.GetType().InvokeMember(
 169:                          generatedMethod, BindingFlags.InvokeMethod,
 170:                          null, dynaClass, new object[1]
 171:                              { this.originalResults });
 172:  
 173:              //create new ListView columns
 174:              CreateListViewColumns(result, lv);
 175:  
 176:  
 177:              return result;
 178:  
 179:          }
 180:          #endregion
 181:  
 182:      }
 183:  }

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }So what this class does, is construct a new code file, that is compiled using the DynamicCompiler helper class that I showed you earlier. What we then do is get an actual instance of an object back from the dynamic compilation process, which we can then use to invoke a method on.

Here is what the compiled code will look like

   1:  using System;
   2:  using System.Collections.Generic;
   3:  using System.Collections.ObjectModel;
   4:  using System.Linq;
   5:  using System.Text;
   6:  using Demo.Data;
   7:  namespace DynoGen
   8:  {
   9:      public class ResultsWriter
  10:      {
  11:  
  12:          public Object GetResults(List<Person> results)
  13:          {
  14:              var x = (from r in results
  15:                       select new
  16:                       {
  17:                           Age = r.Age,
  18:                           FirstName = r.FirstName,
  19:                           LastName = r.LastName
  20:                       });
  21:              return x;
  22:          }
  23:  
  24:      }
  25:  }

But Beware

As we are effectivly creating a new dynamically created Dll in memory in our current AppDomain this could lead to the current AppDomain getting bigger and bigger. So it is a good idea to do all this in a new AppDomain, but how do we do that?

Well that is actually pretty simple, we can do this

   1:  using System;
   2:  using System.Collections.Generic;
   3:  using System.Linq;
   4:  using System.Text;
   5:  
   6:  using Demo.Data;
   7:  using System.Windows.Controls;
   8:  using System.Collections;
   9:  using System.IO;
  10:  using System.Runtime.Remoting;
  11:  using System.Reflection;
  12:  
  13:  namespace DynamicLINQ
  14:  {
  15:      /// <summary>
  16:      /// Obtains dynamic results using a seperate AppDomain that is 
  17:      /// unloaded when the work is finished with the Dyanamic Assembly
  18:      /// </summary>
  19:      public class DataLoader
  20:      {
  21:          public IEnumerable ObtainFlattenedResults(
  22:              ListView lv,
  23:              List<Person> originalResults,
  24:              WritePropertyType currentPropertyType)
  25:          {
  26:  
  27:              //Create new app domain            
  28:              AppDomain domain = AppDomain.CreateDomain("ResultsDomain");
  29:              IEnumerable results = null;
  30:              Object instance = null;
  31:  
  32:  
  33:              try
  34:              {
  35:  
  36:                  //load search assembly
  37:                  domain.Load(File.ReadAllBytes(String.Format(@"{0}{1}",
  38:                      Environment.CurrentDirectory,
  39:                       "Dyno.dll")));
  40:  
  41:                  //Get the EnumerableResultsCreator instance
  42:                  ObjectHandle instanceHandle = domain.CreateInstance("Dyno",
  43:                                      String.Format("{0}.{1}"
  44:                                      , "Dyno"
  45:                                      , "EnumerableResultsCreator"));
  46:  
  47:                  if (instanceHandle != null)
  48:                      instance = instanceHandle.Unwrap();
  49:  
  50:                  //call method to get IEnumerable results
  51:                  if (instance != null)
  52:                      results = instance.GetType().InvokeMember
  53:                          ("ObtainFlattenedResults",
  54:                                  BindingFlags.InvokeMethod,
  55:                                  null,
  56:                                  instance,
  57:                                  new object[3] {
  58:                                      lv
  59:                                      , originalResults
  60:                                      , currentPropertyType }
  61:                          ) as IEnumerable;
  62:              }
  63:              catch (Exception e)
  64:              {
  65:                  Console.WriteLine("Problem constructing the " +
  66:                      "EnumerableResultsCreator instance");
  67:              }
  68:              finally
  69:              {
  70:                  AppDomain.Unload(domain);
  71:              }
  72:              return results;
  73:          }
  74:      }
  75:  }

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }This takes care of managing a new AppDomain and gets the results from the dynamic assembly, and then unloads the new AppDomain. This ensures the primary applications AppDomain doesn’t get any unwanted code in it.

So you can see that we are creating an actual class that accepts a List<Person> (the fake search results) and uses some LINQ to grab out ONLY the properties we want to use all rolled up in an anonymous which is selected as a single flattened search result for the search results grid. Remember this is all happening at RUNTIME, so we can do what we like, we can manipulate the results as much as we like, as the code we compile is just a String.

The other thing that happens is that the WPF ListView columns are altered to only show the necessary  columns based on the newly flattened search results.

Some grid works nicer with anonymous types, though so far the WPF Toolkit DataGrid failed, and so does the ListView, that is why we need to do the trick with re-creating the columns based on the new results. At work we are using the Infragistics xamDataGrid, and it works straight out of the can with anonymous types, we do not have to mess about with columns at all, it just knows what to do.

Here is the results

Showing ALL Columns Of Results

image

Which is done using the following logic

   1:                  case WritePropertyType.All:
   2:                      //OK this is a static query that could have been done in  standard LINQ
   3:                      //but this techniqe can be used to create dyanamically created
   4:                      //LINQ queries at runtime
   5:                      builder.AppendLine("var x =(from r in results select new { ");
   6:                      builder.AppendLine("         Age = r.Age,FirstName = r.FirstName,");
   7:                      builder.AppendLine("         MiddleName=r.MiddleName, LastName = r.LastName,");
   8:                      builder.AppendLine("         ID = r.PersonData.ID, Dob = r.PersonData.Dob });");
   9:                      builder.AppendLine("return x;");
  10:                      builder.AppendLine("}");

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }Showing Selected Columns Of Results (see we only get some of the columns)

image

Which is done using the following logic

   1:                  case WritePropertyType.Some:
   2:                      //OK this is a static query that could have been done in  standard LINQ
   3:                      //but this techniqe can be used to create dyanamically created
   4:                      //LINQ queries at runtime
   5:                      builder.AppendLine("var x =(from r in results select new { ");
   6:                      builder.AppendLine("         Age = r.Age,FirstName = r.FirstName,");
   7:                      builder.AppendLine("         LastName = r.LastName});");
   8:                      builder.AppendLine("return x;");
   9:                      builder.AppendLine("}");
  10:                      break;

.csharpcode, .csharpcode pre { font-size: small; color: black; font-family: consolas, “Courier New”, courier, monospace; background-color: #ffffff; /*white-space: pre;*/ } .csharpcode pre { margin: 0em; } .csharpcode .rem { color: #008000; } .csharpcode .kwrd { color: #0000ff; } .csharpcode .str { color: #006080; } .csharpcode .op { color: #0000c0; } .csharpcode .preproc { color: #cc6633; } .csharpcode .asp { background-color: #ffff00; } .csharpcode .html { color: #800000; } .csharpcode .attr { color: #ff0000; } .csharpcode .alt { background-color: #f4f4f4; width: 100%; margin: 0em; } .csharpcode .lnum { color: #606060; }Here is a small demo project

dynamiclinq1.zip

Now I know this is a very specific business case, and not many people will do this exactly, but it did lead to some very interesting code, that I thought I could share with you, it does show you how to work with dynamically compiled assemblies.

IMPORTANT NOTE

For this example I could have used the Dynamic LINQ library Dll, which is discussed over at Scott Guthrie’s blog.

http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx

However this technique is a powerful one, and could be used for a great many use cases, where you need to run some dynamic code that you only know about at runtime. It just happens to be I chose to use LINQ as the demonstration tool, at work we actually use the Dynamic LINQ library Dll, but the rest is the same.

About these ads

31 thoughts on “WPF : Dynamic Search Driven List Results

  1. brad says:

    Instead of all that dynamic c# stuff, i think you could create an expression tree (the backbone of linq) and then compile that tree and execute it. If you would like a quick example, let me know.

  2. sacha says:

    Brad

    I think you are missing the point, the LINQ stuff has to be decided and worked out at runtime, as I do not know what the expression tree will look like until a search is run (runtime). See what I mean.

    What would the expression tree look like, what properties would it include, these would change at runtime depending on metadata.

    I fully understand how LINQ works, it just doesnt work here.

    On the other hand Entity SQL (which allows dynamic expressions to be created) also uses strings.

    And the extensions that were released out of bands (Dynamic Library as seen on Scott Guthries blog) also only allowed a string for a Where() extension method.

    Of of which are not good enough.

    I needed to create a LINQ call at RUNTIME, which I knew nothing about at design time.

  3. Ben says:

    Interesting article.

    I’m curious does this method lend itself to memory leaks though, if code is being compiled dynamically each time a button is clicked?

  4. sacha says:

    Ben

    Yeah you would not want to use this approach all over the place. But we have used it evry sparingly, and have seen no adverse effects from this approach, bith in terms of memory usage and time

  5. Georg Begerow says:

    Realy nice article.

    We used a similar approach some years ago (with .Net 1.1 :-)) to compile user defined filters. This filters searched a collection of many thousand items in memory for matching items on multiple criteria for every logedin user, so they had to be fast. The application is still running on .Net 1.1 and still fast enough.

    One point we learned the hard way, was not to hardcode the filter source code in the application. We finaly used a combination of a filter base class together with a homebrewn template engine. Nowadays i would suggest using T4. This way you can easily generate and debug the generated code outside your application.

    Georg

  6. sacha says:

    Georg

    Yeah we actually create proper .NET types for our query, this dynamic stuff is just to trim the result set so to speak.

    Gad you like it though.

  7. Jean-Philippe Leconte says:

    I fail to see why you couldn’t use the Dynamic LINQ library. The only thing I used that is not provided by it is Join, and it can easily be implemented over the existing Dynamic LINQ code.

    CodeDOM is an interesting approach, but the Dynamic LINQ does it without “leaking” new code in your AppDomain (unless you use a Select that returns a new “anonymous” struct, which makes Dynamic LINQ emit a dynamic class).

  8. sacha says:

    Jean-Philippe

    I have used the Dynamic LINQ API before but I thought it was only able to be used to create Func that could be used on Where() extension methods where we already had a IEnumerable already.

    I could be wrong, have you got an example that would do what I am doing here returning an anonmous type using the Dynamic API.

    I will give it a go, I have had another think about this, and yeah it may actually be possible using Dynamic LINQ after all.

  9. Jean-Philippe Leconte says:

    To return an anonymous type using Dynamic LINQ, just use the Select ( string selector ) extension, which supports the new keyword.

    from r in results
    select new
    {
    Age = r.Age,
    FirstName = r.FirstName,
    LastName = r.LastName
    };

    becomes

    results.Select ( “new ( Age, FirstName, LastName )” );

    to rename properties, you can use the “as” keyword :

    results.Select ( “new ( Age, FirstName + ” ” + LastName as Name )” );

  10. Jean-Philippe

    I think this approach will actually work after all, still I have now given folks another option to think about.

    Its all good.

  11. Pablo Alarcón says:

    As always a great post Sacha, thanks a lot for contributing that much.

    As Ben has previously said, I expect a little “memory leak” each time a dynamic assembly is emitted.

    When emitting the class you’re generating a new Assembly and loading it into the current AppDomain, each time. Assemblies cannot be Unloaded, you have to Unload the AppDomain they reside in, so here it comes the “memory leak”.

    What I did in the past is to explicitely create an AppDomain where to inject dynamic assemblies and Unload it after execution. There were some difficulties, speaking from memory you had to use “inter-AppDomain remoting” to invoke the emitted methods, Dynamic class must derive from MarshalByRefObject and so on.

    This was a typical problem with “Plug-In” scenarios in .NET 1.1, Google has info for sure ( as for everything :) )

  12. sacha says:

    Pablo,

    This post is actually a trimmed down post of what we do at work, we actually do unload the AppDomain.

    We use something simliar to this

    string fileName = @”C:theFolderprintlib.dll”;
    string folderName =@”C:theFolder “;
    Assembly assemb = Assembly.LoadFile(fileName);
    AppDomain otherDomain = AppDomain.CreateDomain(“OtherDomain”, null,
    folderName , null, true);
    object t = otherDomain.CreateInstance(assemb.FullName,”PrintLib”);
    AppDomain.Unload(otherDomain);

    Thanks for your comments

  13. Pablo Alarcón says:

    Cool, but when publishing code for others to learn it’s safer to cover every “don’t this in production code” bit known, even if it’s just a little mention to avoid polluting the example :)

  14. Pablo Alarcón says:

    Great, now let’s hope Google shows this as an example to do CodeDom emitting and less people suffers that common memory leak problem.

  15. Stephan Keil says:

    Sorry, but I don’t get the trick of using a separate AppDomain.Can you please explain which kind of memory leaks it avoids?

    IMO the AppDomain – as you use it – still leaks in dynamic assemblies and the type objects in these assemblies:

    1. ‘domain.Load()’ loads the dyno.dll into the temporary AppDomain. But as a side effect of returning the loaded assembly, the AppDomain.Load() also serializes/deserializes dyno.dll into the current (“main”) AppDomain.

    2. When calling ‘instanceHandle.Unwrap()’, the ‘EnumerableResultsCreator’ object is deserialized/serialized into the current (main) AppDomain (for this to succeed the download code correclty contains the necessary [Serializable] attribute which is omitted in the snipped of your post).
    Thus the now unwrapped ‘EnumerableResultsCreator’ instance (along with its type object) lives in the main AppDomain and so do the classes dynamically compiled during ‘ObtainFlattenedResults’.
    To see the effect you can set a breakpoint right after the ‘results = …’ in DataLoader.ObtainFlattenedResults (line 62 in your code snippet) and watch AppDomain.CurrentDomain.GetAssemblies() growing. For each call to DataLoader.ObtainFlattenedResults() a new assembly containing the dynamically compiled types is added to the domain.

    As I understand it, you simply cannot unload an AppDomain and still use an object residing in that domain.
    Either the object does not “really” reside in the AppDomain, but has been serialized/deserialized into the calling AppDomain (which needs the respective class to be [Serializable], as your EnumerableResultsCreator); or the object is accessed through a proxy (which needs the respective class to derive from MarshalByRefObject). In the latter case the proxy is “dead” after unloading the AppDomain.

    A real nice feature to support dynamic applications would be that Type objects and the code associated with them would be (optionally?) subject to garbage collection. I.e. the Type and code is unloaded as soon as the last instance of that type is gone. But – AFAIK – this is not possible in .NET/CLR.

  16. sacha says:

    Stephan

    These are some excellent comments.

    And to be honest I was under the impression that you could unload and appDomain and it would also unload any items that were dynamically loaded into it.

    I shall have to look into this a bit further. Ill try and do that soon and write my findings here.

    Thanks for pointing this out.

  17. Stephan Keil says:

    Hm, my previous post doesn’t show up, so here is my comment again:

    IMO your AppDomain does not avoid the leaks, at least if you mean the leaking of temporary assemblies and Type objects.
    Please correct me if I am missing something:

    1. While loading the Dyno.dll assembly (line 37: ‘domain.Load(…)’) into the temporary AppDomain, the assembly is also loaded into the “main” (current) AppDomain. I suppose that is because AppDomain.Load() returns the loaded assembly, which actually serializes/deserializes the assembly from the temporary to the main AppDomain.

    2. When unwrapping the object handle (line 48: ‘instance = instanceHandle.Unwrap()’), the same occurs with the created EnumerableResultsCreator. I.e. it is copied by (de-)serialization from the temporary to the main AppDomain (for this to succeed, the downloaded code correctly has a [Serializable] attribute on EnumerableResultsCreator, which is missing in the code in your post).
    All subsequent accesses to the instance then occur in the main AppDomain, thus the temporary AppDomain is actually useless.
    For each call to DataLoader.ObtainFlattenedResults(), the invocation of EnumerableResultsCreator.ObtainFlattenedResults() (through reflection, see line 52: ‘results = instance.GetType()…’) finally uses CSharpCodeProvider.CompileAssemblyFromSource() to compile the dynamic code. The dynamic code actually contains two class types: ‘ResultsWriter’ and the anonymous type from the linq expression.
    CompileAssemblyFromSource() creates a dynamic assembly for these types. And as the EnumerableResultsCreator ‘instance’ resides in the main domain, the assemblies are also loaded into that domain. So there is one dynamic assembly with two types for each DataLoader.ObtainFlattenedResults() call => leak.
    You can nicely inspect the effect by setting a break point after ‘results = instance…’ (i.e. line 62 in your code snippet) and watch AppDomain.CurrentDomain.GetAssemblies() growing.

    As I understand it there are two options for using an object accross AppDomains:
    Either the object is serialized/deserialized from the source domain to the target domain (which needs the respective class to be [Serializable] as in your example), resulting in actually two instances, one per AppDomain. Here unloading the source AppDomain has no effect since the target AppDomain works on a copy of the object.
    Or a proxy for the object is created in the target domain (which needs the respective class to derive from MarshalByRefObject). In this case, unloading the source AppDomain invalidates the proxy, so each subsequent call to the proxy will result in an exception.

    To develop a dynamic data library for the company I work for (see http://www.quinlogic.com) I was also looking for a way to dynamically create types (classes) which only exist temporarily. For my understanding this can be achieved with a temporary AppDomain, if all accesses to the “dynamic” types are only made in that AppDomain.
    This is not very convenient. My favourite approach would be if a dynamically created Type (and its associated code) is (optionally) subject to garbage collection. I.e. the type and its code is collected if the last instance of that type has gone. To my knowledge this is not possible with .NET/CLR.

  18. I know this is an old thread; but, did the memory leak issue ever really get resolved? We have implemented the complete System.Data.DataSet classes on Silverlight, and conversion back to Web using a very similar technique.

    The last few comments sounded like you couldn’t really keep generating assemblies, even in a different AppDomain, without memory leaks.

    If ANY one has more info, please post.

    Thanks,
    Steve

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s