The customer wants to see relationships between multiple data types, groups of data, drill down into it, have a geographical view, a temporal view and a conceptual view. And much more of course. That’s the moment you realize you spent ten years of your life creating all possible variations on the subject ‘diagramming’ and now you actually need a ‘diagramming database’. A whole different beast, not the usual yada-yada, what does it entail?

The thing is, relational database systems are not very suitable to act as repositories for graph-like structures but at the same time they are unavoidable because the whole world runs on them. Sure, there is something like an OO database but it would be even less suitable for graphs. So, you choose for a RDBMS even if you have some back-thoughts. At least RDBM systems are well supported on every API level and you get tons of fun tools which on the long run could be useful. In the case of SQL Server you know things like reporting services, StreamInsight, OLAP and whatnot will show up at the end of the road.

How do you store a graph-like system whose nodes are of an arbitrary data type? Say, you wish to store relationships between telephone numbers, file locations, car types, chat messages and credit card numbers. It wouldn’t be very wise to create data tables for each data type since you know at some point the customer will come along and ask for mobile numbers, addresses, book titles and cooking recipes! Well, there are not that many solutions in this case; you can use binary serialization of some data blob and store it as varbinary or you can use string-like serialization (which includes XML). Obviously, binary stuff in your database is not a good thing if you wish it to be searchable and ultimately open to other integrations. There is, hence, no other choice but to go for XML structs which at least is a well-supported standard across the while industry.

What about the relational structure? Easy enough, you just store the ‘from’ and the ‘to’ (sink, source, target, drain, start, end, initial, final…) in a two-field table. Voila, you’ve got a graph-database on top of an RDBMS.

Well, not really.Obviously having the data in place is just the beginning; you need to calculate a spanning tree, Hamiltonian cycles, Eulerian paths, shortest distance…and more fun pseudo-math stuff. If you want to use the standard algorithms it means you will put a lot of pressure on the data exchange with your client (say, a WCF service). Argh, you realize the only way is to actually do as much as possible on the database side. Through T-SQL or use .Net assemblies inside the SQL process? Not an easy choice. You start to wonder if creating a server is not a better option. But this means…writing a query processor, heavy threading programming and more wizardry which goes far beyond one’s standard toolbox. So…let’s assume T-SQL can do it. Actually T-SQL is not too bad if you are a bit fluent with it you can go quite far into this direction.
Status: a graph-database consists of two tables and a lot of smart stored procedures.

But hold on, the customer wants it also multi-user and multi-application. That is, the total data is not always ‘on’, only a subset of if in function of the application the current user is using. More branching, more subtleties in the stored procedures.
The situation is actually comparable to having heaps of XML documents where each user and each application adds attributes to the standard XML to decorate it with the necessary application-options and security settings. On top of that lots of tuples describe relationships between these XML documents. Did I mentions the relationships are also user and application dependent?

Maybe I’m giving you the impression that it’s an hopeless complexity and an overly open system which tries to embrace too much flexibility. It’s not that bad. In fact, we have a working prototype which includes a full-fledged client application (both for Silverlight and for WPF). The feeling sticks with me, however, that as far as abstract beauty is concerned a true graph database server would be better. This exists in Java but not in .Net and even if it would, I wonder how well adjacent systems (like reporting and such) would integrate with it. So, the current prototype is a balanced solution between various poles and constraints. In a next phase we’ll develop connectors to StreamInsight, file system, legacy event aggregators and more. The purpose being to be able to store and analyze graphs which cross the boundary of the local storage and also capture real-time events (through, for example, StreamInsight). Big fun and more challenging stuff is ahead.

L2 G2 is our biggest product and has been used by companies to create ‘big’ applications in diverse markets. Because it is based on Unity for dependency injection and because Microsoft waited quite a while to release a Silverlight analog, it took a while to create a Silverlight analog of G2. We baptised it ‘L2′. Of course we could have used alternative injection libraries (many have popped up this year in fact) but we wanted to stay as parallel as possible to G2 in order to enable our customers to move with ease between G2 and L2. So, it’s there and you can test it online here. It’s still in beta and we welcome any feedback. You can also download the code and the core library, see the L2 product page for more.

S#scripting library. This little jewel allows you in effect to alter an application at runtime much like macro’s. In fact, you can compare it to VBA in office except that our S# library will work in any .Net context (including XNA, Silverlight, WPF and WinForms). I will talk about S# more later this week in another blog posting and Denis Vuyka will also publish soon more information regarding S# and, another kid on the stage; Processing#.

iSee! In addition to this we have been experimenting with novel approaches to diagramming and how to take full advantge of Silverlight and WPF animation capabilities. This lab project which has been now already for one year in the back of our minds is ready to be tested as well. Head over to our iSee demo and let us know what you think. iSee is all about animating visual elements and you can apply it anything which inherits from the FrameworkElement base class, that is…as good as anything you’ll find in WPF and Silverlight. Furthermore, iSee is really easy to use. It boast a fluent API and a very intuitive series of methods. The fluent things means you can chain actions ad infinitum.

For example, the following line of code:

Attach(image, true).Scale(3, 3, Duration.Automatic)
.Move(new Point(145,150)).Scale(0.3, 0.3, Duration.Automatic).Play();

will attach a given image to a canvas, scale it with a factor of three, subsequently move the image to another location and un-scale it to the original proportions. Easy! It would take a substantial amount of work to code this in C# or to create an equivalen storyboard in Expression Blend.

We also wanted to enable certain diagramming scenario’s and have integrated diverse methods which allow you to create diagrams, mindmaps and data visualization with just a few methods. In the demo you will see a few examples of this.

SPGLast but not least, I’m also proud to announce our new Silverlight property grid. Much like L2 is an analog of G2, so is the Silverlight property grid (we call it ‘SPG’) an analof of our WPF property grid (‘WPG’). You can read about it on Denis’ blog and watch this space for more in the next few weeks.

All these products represent a considerable amount of work (testing, demo versions, docs, samples…) and we are still in the process of packaging them. Also, for prices and official releases you’ll have to wait a bit, we’re working on it. Meanwhile let us know what you think. The Silverlight market is booming these days and there are excellent products around, still we think that our set of products fill a gap and represent a unique spectrum of solutions you’ll not find elsewhere.

We’ve been busy. The screenshots reveal quite a bit the upcoming features but let me be explicit:

• Groups: shapes can be embedded in groups ad infinitum (this is an undoable action and copy/cut/paste works with groups as well, of course)
• Flowcharting: better support for standard flowcharting
• Arrows and connections: better support for databinding your stuff (i.e. data backend) to connections and related adorners
• Rotations: rotating shapes and groups (undoable action)
• Unity 2.0: Unity merges with the ObjectBuilder2. Nothing shacking here but it will simplify the underlying data exchange pipeline of G2.
• Library support: shapes, groups, layers, pages and whole diagrams can be saved to libraries and organized in categories. This features is based on an adapter mechanism, which means you can have libraries in a database backend or accessible via WCF etc.
• Alignments: aligning shapes is now possible. In conjunction with the snap & grid features this is particularly useful to have a nice layout.
• Snap & grid: everything has become snap-able.
• Linking: better support inside shapes to (hyper)link data; either diagram elements or data entities coming from your backend.
• XML: better support for XML import/export, extending the already very solid and beautiful data exchange pipeline of v1.0
• Theming of whole diagrams: apply a style to a complete diagram (cfr. Microsoft PowerPoint)
• Auto-connect shapes: auto-add or auto-connect shapes via the adjacent arrows appearing when hovering over shapes (cfr. Microsoft Visio)
• Bug fixes and squadrillions of little changes to make G2 better, faster, more useful, more business and data oriented, more beautiful.

No date fixed yet to release v2.0 though it’s pretty stable already now. So many things are going on these days… I sincerely hope to deliver a big headline one of these days regarding our future (and G2′s future in particular). In any case, I feel immensely proud about G2′s wonderful architecture and the breadth of applications it can support, the scope of features and the many high-tech subtleties which you will not find in any other diagramming package. It leaves the competition beyond the diagramming horizon.

Six degrees of freedom is everywhere around us; it’s how your brain is wired, it’s the law ruling the internet growth, it describes the strucutre of movies and movie stars relationships, it tells how you are connected to your friends. Twitter much like any other seemingly random database (LinkedIn, IMDB, , Disease database, Facebook…) is proving the six degrees behavior. Recently someone took our Graphite control for Silverlight to visualize Twitter relationships and created Twampoline, a Twitter visualizer. It shows so-called cliques and densities, however if you have a busy network of followers you will have difficulties to see through the clusters. The layout algorithm used should be reparametrized in function of the graph scale.

Aside from this beautiful visualization idea, I have never understood the whole Twitter hype and wonder what the next superlative of all this will be?. From websites, to wiki’s, to blogs, to twitter, there has been a continuous decrease of content and informative value. The next big thing will be single words or acronyms summarizing one’s mood and thoughts? Or maybe Google’s Wave will bring  Web 3.0 to the world but I doubt it.

To all customers of Graphite, head over to the following articles which compile the tips and solutions related to the customization and integration of Graphite:

• General overview, event logic, import/export, etc.
• How to template the VisualNode of Graphite
• How to change the layout algorithm of Graphite
• Graphite: differentiating between parents and children

Remarks about and requests for particular uncovered topics, just let me know.

It’s been a while since I jotted something down on this blog…it’s been hectic for months now and the pace doesn’t seem to slow down but I think I have a few interesting things to mention.

Since the end of last year G2 has grown bigger, stronger, more popular and various big customization projects are on our workbenches. At the same time, Graphite continues to be our bestseller but most customers use it ‘as is’ without much alterations. There are however a few lines of evolution. On the one hand, Silverlight becomes more and more popular and seems to have a better market penetration than WPF. This means that also Silverlight diagramming solutions are increasingly asked for. In this context, Graphite allows an easy path to data visualization but misses the flexibility and breadth of a true diagramming framework. Equally well (and enforced by the Silverlight wave), more and more customers wish to have a dual solution; all the advantages of diagraming in WPF and the possibility to bring as much as possible on the web (Sharepoint integration, read-only viewers and so on). These evolutions and the obvious gap in our products related to this meant a necessary investment in Silverlight and a common diagramming core. Now, hold your horses and don’t expect the sky since Silverlight still remains a subset of the .Net framework and even though Silverlight v3 broadens the horizon I don’t think any day soon you’ll have a full-compatibility tale (except through XBAP, but that’s another story). In any case, here is the good-news show and some notes from the field where diagramming meets Silverlight, the Entity Framework, MySQL and WCF.

Before describing the juicy technical details let me give you an overview of the application (screenshot above) we’ve been working on recently. It’s a Silverlight application which allows one to visualize any kind of relations between different types of entities; persons, documents, sites, etc. Any kind of relation means that it allows multiple links and generic graphs but can also handle tree-like data and multiple diagram layouts. The database uses just a few tables to store the graph structure and the actual data (persons, documents, metadata) can reside in the database but could also be stored in another database, in XML files, you name it. The application also allows one to edit relations and to edit the entities; you can draw links and it will be saved in the database, you can change or add entities and so on. Finally (and this is what this blog items is mostly about) this Silverlight application can also be used as a WinForm application and accesses data through a local self-hosted WCF service. The WCF service is a RESTful ADO.Net data service which can also handle standard HTTP requests and thus acts as a local webserver. Minor features of this application is the integrated security system which can be connected to the ASP.Net membership mechanism, to ActiveDirectory services or any legacy single-sign on system you have in-house. Let me emphasize that this application boasts a full-fledged diagramming framework for Silverlight. which in many ways extends/surpasses our current Graphite product. For example, just for fun I developed in less than a day a reasonable mindmapping application for Silverlight (screenshot below).

Entity Framework

First off, I have been happily surprised by the Entity Framework (EF). I know, it’s got a long list of shortcomings and I’ve walked the whole avenue of POCO adapters, detached entity tricks, constrained Visual Studio designer and more. However, in the context of Silverlight I was pleased to experience the ease with which it worked on top of MySQL and through the ADO.Net data services (aka Astoria). Initially I wrote the whole framework for SQL Server and had later on to move to MySQL…oh miraculus! Only minor changes were necessary on the database level (no Guid’s in MySQL, no triggers e.g.) and one needs at least MySQL v5 to make it work. Note that this happy message has more its origins in the hard work which went into developing the .Net driver for MySQL rather than how open Microsoft is towards other relational database systems. In fact, I can also claim the same for other systems: EF works well with SQLite and with SQL Server Compact v3.5. There are pro’s and con’s with each and you need to tweak your model because of datatype differences, missing features (views, stored procedures etc.) but I expected a complete no-go orginally.

ADO.Net Data Services

Concerning ADO.Net data services. It works flawless with the EF, whatever the underlying database-dependent model is. In fact, if the data type signature is identical in different database systems, you only need to change one word in your code to switch from one to the other:

1
2
3
4
5
6

[System.ServiceModel.ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class CrystalMsSqlService : DataService<crystalmysqlcontext>
{
...
}
</crystalmysqlcontext>

This code is the generator of a REST WCF service on top of a EF model. The CrystalMySQLContext is the data context generated by the EF. If you wish to switch to, let’s say, SQL Server you only need to replace the data context specification and you’ll be safe for the rest of the journey.
Note, as a side note, the ServiceBehavior attribute which propagates exceptions to the surface and which is not added by default in Visual Studio. You will also want to add a try-catch in the code to convert the exceptions to a DataServiceException like this:

1
2
3
4
5
6
7
8
9
10
11
12

public static void InitializeService(IDataServiceConfiguration config)
{
    try
    {
        ..
        config.SetEntitySetAccessRule("SomeTable", EntitySetRights.All);
    }
    catch (Exception e)
    {
        throw new DataServiceException(e.Message);
    }
}

It will make debugging much easier, trust me.

Why ADO.Net data service (ADS)? You need to know that Silverlight only likes asynchronous data exchange with the outside world. Now, in order to expose data via WCF you can go via a artisanal, hand-crafted WCF service but ADS makes things easy and turns an EF into a RESTful service in a snap. In effect, it means you have all your tables and views available in Silverlight as if they were on the client. The price for this is the asynchronous behavior, which at times can be frustrating; you cannot have a data request and a data related action in the same C# method. A wisdom which I can give to you here is: don’t try to launch inner join requests from Silverlight to the REST, it doesn’t work. If you wish to do this you need to create a view in you database and make this view accessible in ADS as if it was a table. This simple fact makes SQL Server Compact immediately a bad choice because it doesn’t support views (SQL Express will do however).

WCF (ah, so much beauty!)

One of the things which kept me busy for a long time was how to demonstrate/ship a Silverlight demo without having to ship a database and without having to buy an expensive SQL Server license? Imagine you have a beautiful Silverlight application for a customer but you don’t want a whole setup involving a SQL Server database deployment (logins, connection strings etc.) and you also don’t want a complicated IIS setup wherein the Silverlight, WCF, XAP…reside. What to do? At first I tried to develop a local WCF proxy which would go to a file based data system. This meant a self-hosted WCF service, some sort of database on the client and a separate local webserver. This worked well up to the point that Silverlight kept on complaining about security issues. No mystery here: Silverlight doesn’t want to access data sources if they are not coming from the precise (includes the port number!) same origin as the xap (i.e. the Silverlight application).

Easy enough, I thought, but read on. You only need to supply the client with the clientaccesspolicy.xml or the server with the crossdomain.xml. Extend the WCF service like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

[ServiceContract]
public interface IWebServer
{
    [OperationContract, WebGet(UriTemplate = "/clientaccesspolicy.xml")]
    Stream GetSilverlightPolicy();
    [OperationContract, WebGet(UriTemplate = "/crossdomain.xml")]
    Stream GetFlashPolicy();
...}
public class CrystalMsSqlService : DataService<crystalmysqlcontext>, IWebServer
{...
public Stream GetSilverlightPolicy()
{
    const string result = @"< ?xml version=""1.0"" encoding=""utf-8""?>
                                <access -policy>
                                    <cross -domain-access>
                                        <policy>
                                            <allow -from http-request-headers=""*"">
                                                <domain uri=""*""/>
                                            </allow>
                                            <grant -to>
                                                <resource path=""/"" include-subpaths=""true""/>
                                            </grant>
                                        </policy>
                                    </cross>
                                </access>";
    return StringToStream(result);
}
 
public Stream GetFlashPolicy()
{
    const string result = @"< ?xml version=""1.0""?>
                                < !DOCTYPE cross-domain-policy SYSTEM ""http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd"">
                                <cross -domain-policy>
                                    <allow -access-from domain=""*"" />
                                </cross>";
    return StringToStream(result);
}
...
}
</crystalmysqlcontext>

It doesn’t work. Or at least, you can check that the WCF service does supply the policies but for some reasons Silverlight in my case still didn’t want to query the WCF service.

Now, because REST allows a client to query a WCF as if it’s a website and because one can serve both ascii and binary data through WCF/REST the obvious next step was to extend the WCF service with methods which would mimic a webserver behavior. Easy to do in fact by extending the service with more methods and templating the URL requests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

 
[ServiceContract]
public interface IWebServer
{
[OperationContract, WebGet(UriTemplate = "/clientaccesspolicy.xml")]
Stream GetSilverlightPolicy();
[OperationContract, WebGet(UriTemplate = "/crossdomain.xml")]
Stream GetFlashPolicy();
[OperationContract, WebGet(UriTemplate = "/app.htm")]
Stream GetApplicationPage();
 
[OperationContract, WebGet(UriTemplate = "/{pagename}")]
Stream GetSitePage(string pagename);
[OperationContract, WebGet(UriTemplate = "/Images/{imagename}")]
Stream GetImage(string imagename);
[OperationContract, WebGet(UriTemplate = "/Crystalizer.One.xap")]
Stream GetCrytal();
 
}
public class CrystalMsSqlService : DataService<crystalmysqlcontext>, IWebServer
{...
public Stream GetSitePage(string pagename)
{
    try
    {
        WebOperationContext.Current.OutgoingResponse.ContentType = "text/html";
        string fileContents;
        string path;
        path = string.Format(@"WebRoot\{0}", pagename);
 
        using (StreamReader sr = new StreamReader(path))
        {
            fileContents = sr.ReadToEnd();
        }
        return new MemoryStream(Encoding.UTF8.GetBytes(fileContents));
    }
    catch (Exception)
    {
        return null;
    }
}
 
public Stream GetImage(string imagename)
{
    byte[] fileContents;
    fileContents = File.ReadAllBytes(string.Format(@"WebRoot\Images\{0}", imagename));
 
    // byte[] resp = Encoding.ASCII.GetBytes(string.Format(HttpHeader, "image/png", fileContents.Length, fileContents));
    return new MemoryStream(fileContents);
}
...}
</crystalmysqlcontext>

This now turns your RESTful ADS service into a webserver which serves both the data, the site pages and site images.
One last trick is the fact that because now your WCF service actually implements two interfaces the default endpoint configuration of ADS will not work. To solve this in the self-hosted scenario you can add another endpoint as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

private static void HostWithWebServiceHost(Uri baseAddress)
{
 
    WCFHost = new WebServiceHost(typeof(CrystalMsSqlService), baseAddress);
    WebHttpBinding binding = new WebHttpBinding();
    WCFHost.AddServiceEndpoint(typeof(System.Data.Services.IRequestHandler), binding, "dataservice");
 
 
    WCFHost.AddServiceEndpoint(typeof(IWebServer), binding, "").Behaviors.Add(new WebHttpBehavior());
    ServiceMetadataBehavior smb = new ServiceMetadataBehavior {HttpGetEnabled = true};
    WCFHost.Description.Behaviors.Add(smb);
 
    WCFHost.Open();
}

At this point the Silverlight application will run at the client without any server or internet connection.. You can go a step further and even host the Silverlight application in a Windows form which at the same time hosts the WCF service, but that’s really a no-brainer. You simply add a webbrowser control in a WinForm and call the HTML page which contains the Silverlight control and you’re done. Note that by doing this you have a Silverlight 2 application which runs as if it’s a regular WinForm application. Still, because the WCF service serves ANY client you can also access the Silverlight application through a standard browser. Actually, this fact helps to debug the Silverlight code by attaching the debugger to the browser process. I have not found any other way to debug Silverlight if it’s not part of a Visual Studio web application (as in my case).

In my final ‘show it to the customer without deployment’ solution, I removed the database from the client setup and kept the MySQL on a publicly available location. The only things you need to watch in this case is the fact that by default EF will store the database connection credentials in the web.config. You need to create a EF compiled connection with the EntityConnectionStringBuilder and override the CreateDataSource method of the ADS service:

1
2
3
4
5
6
7
8
9
10
11
12

protected override CrystalMySQLContext CreateDataSource()
{
    EntityConnectionStringBuilder entityBuilder = new EntityConnectionStringBuilder();
    entityBuilder.Metadata = @"res://*/MySql.CrystalMySQLModel.csdl|res://*/MySql.CrystalMySQLModel.ssdl|res://*/MySql.CrystalMySQLModel.msl";
    entityBuilder.Provider = "MySql.Data.MySqlClient";
    entityBuilder.ProviderConnectionString = "server=xxx;user id=xxx;password='xxx';persist security info=True;database=illumineo";
 
    EntityConnection con = new EntityConnection(entityBuilder.ToString());
 
    CrystalMySQLContext ctx = new CrystalMySQLContext(con);
    return ctx;
}

MySQL

During the testing and deployment phase of this project I encountered various difficulties which were all related to the fact that all the assemblies you assume are not necessarily on a blank machine. Adding these assemblies to the bin directory doesn’t solve these issues, the great trick here is a configuration you need to add in order to tell the CLR these assemblies are available. During the installation of a database driver the machine.config is altered but this is not the case if you deploy a self-hosted service and the binaries are not in the GAC:

1
2
3
4
5
6
7
8
9

< ?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <dbproviderfactories>
      <clear />
      <add name="SqlClient Data Provider" invariant="System.Data.SqlClient" description=".Net Framework Data Provider for SqlServer" type="System.Data.SqlClient.SqlClientFactory, System.Data, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" />
      <add name="MySQL Data Provider" invariant="MySql.Data.MySqlClient" description=".Net Framework Data Provider for MySQL" type="MySql.Data.MySqlClient.MySqlClientFactory, MySql.Data, Version=6.0.2.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d" />
    </dbproviderfactories>
 
</configuration>

Silverlight only likes it async

Concerning the asynchronous behavior of Silverlight I can only say: you need a different mindset to live with it. In my Silverlight application I invested quite some time to create an MVC pattern on the basis of asynchronous data access. Because Silverlight handles templating/styling differently and doesn’t have a databinding mechanism which support element names all of this was tricky at times but in the end I think I was able to cook a very clean and felxible Silverlight framework which in fact has not much affinity with diagramming and could be used as a generic framework (just like our O2 framework for WPF).

Commercial rands

I mentioned in the introduction that one of the aims of this project was to have a common core between WPF and Silverlight. Bad news here: you cannot share assemblies between Silverlight and WPF, you cannot share template, you cannot share databinding patterns, you cannot…well, lots of things. You can find on the net different ways to handle these difficulties (pragma’s in your code for instance) but I haven’t found any of these satisfactory. So, the end result in this context is a big deception, until Microsoft brings on stage a clear strategy targetting this scenario you have to live with duplicated code. For our diagramming aims this means we have a completely different Silverlight library which covers a lot of the features available in G2 but is nowhere near identical in code as G2. On the surface (read: our commercial offering) we now have

• an enterprise level diagramming framework (product: G2) for WPF and a MEF-based framework for WPF applications (product: O2).
• a very complete diagramming framework (product: L2) for Silverlight and a MVC-based framework for Silverlight applications

which in look-and-feel will give you pretty much the same experience. A BPM modeller or a workflow application in Silverlight with the same functionality in WPF…it’s all possible but not on the basis of a common core. In commercial terms this means that a customer wishing to have a dual Silverlight/WPF product needs to pay twice for customizations but can share a common data access layer hidden behind a WCF facade. Speaking about commercial moves I should also mention that The Orbifold has acquired recently two new products which were previously in the open source realm: Denis’ property grid for Silverlight and WPF, as well a Petro’s S# scripting framework (aka Script.Net). More about this in the near future.

There is so much more to say about our other projects and our latest WPF magic but I’ll stop here and leave if for another blog post.

How to picture a 5-dimensional sphere or a see the correlation of  112-dimensional weather data? The answer was found in 1885 when parallel coordinates were invented and later refined in 1959 through  Inselberg. The idea is actually very simple, given some N-dimensional space of data you draw N parallel lines and each line has its own scale. Plot a given data point on these lines by creating a line between the N point of this multi-didmensional point. In this way you get a line per data point and the series of given data points are mapped to a series of lines on the graph. The situation being much like projective geometry where we also find a back-and-forth thinking between lines and points in space. The good thing about such a representation is that if points have some accumulation in a particular dimension you will see it because lines will accumulate on the axis representing this dimension (see adjacent image). This kind of correlation can usually only be detected through statistical analysis, but what do you do if you want to record or supervise data which continuously comes in from real-time systems? This would mean a continuous mathematical analysis of the sampled data and this would entail some very custom algorithm bound to the particular context.

A customer of ours recently challenged us to implement the parallel coordinates visualization in WPF. While quite straightforward at first sight it proved to be more difficult than we initially thought for two reasons:

• non-numeric data: what to do if you have some string data in one of the dimensions? Say, you want to visualize in one of the dimensions a timestamp or telephone numbers or names of patients.
• how to emphasize the accumulation points in the graph? While in the screenshots above you can actually see well the discrete tendence in the second dimension, the screenshot on the right bestows you with a challenge to detect it.

The first problem was tackled through an internal mapping of the data which spreads the non-numeric data across an axis. The drawback of this is that at various levels one has to switch between the actual

data and the internal representation for labels, line hit during hovering, the histogram (see below) and any calculation in general. The second was more difficult and entailed some analysis of the data and various visual alternatives to attract the attention of the user (human interpreter of the data). The end-result is an histogram which like a rotated bar chart on the axis graphs the relative amount of crossings on the axis. The histogram actually shows also the opposite of an accumulation, since (see the second axis in the screenshot below) an equal spreading of the histogram’s rectangles means an equal distribution of the lines across the dimension axis.

Other minor features we added to this data visualization are; the possibility to hover over the graph and see the actual data of the multi-dimensional points, datagrid binding and editing, customization of look & feel through dependency properties, swapping data axes, scaling data vertically on a particular axis.

Probably the most satisfactory moment for myself during the development was at the point when it became clear that just any CSV/spreadsheet data can be bound to the parallel graph visualization. Indeed, due to the internal mapping mechanism one can present any type of tabular data and as a proof of concept I mocked some random spreadsheet in Excel which was subsequently visualized without a hitch.

Stocks & shares anyone? Let me know.

We (Denis Vuyka 1, Denis Vuyka 2 and myself) have completed a fully functional mindmapping application on top of G2 and have used it in the past months internally for various things. This application is, however, not a ‘product’ or anything you can buy, it’s only a proof of concept.It’s a test application, a benchmark, a toy model, a fun thing to play with and see how G2 handles various things. Much like the class designer and the database table designer I talked about earlier, it’s a way to show off the potential of G2 to you, even though I think this mindmapping application could be a stepping stone to something bigger.

This said, the development was a lot of work; the devil is in the details as everyone knows. G2 is a framework on top of which you can create an application, attach meaning to nodes and links, handle events and trigger some mechanism in the G2 core and so on, all this is conceptually quite easy. Making things smooth on the UI level, making custom controls to manipulate the data coming or going to G2 is on the other hand a tedious process of UI-design, usability testing/trying, debugging, handling special cases (large images, bad alignment of text, broken serialization due to unicode characters…) and more. This journey has been rewarding, once you have the idiosyncrasies of WPF in your fingers and can navigate blindly through G2, the whole process feels very productive.  For example, creating a thought-node which binds to some RSS and injects blog-titles into the map is fun and easy, the same for custom nodes binding to stocks & shares data (see the screenshots). Creating thought-nodes with clickable icons or showing some custom data is a no-brain activity. In fact, because the stuff shown on a node is presented through WPF’s databinding you can add pretty much anything you like to the node’s visual.

Stocks & shares

Mindmap on 'rendering'

About The Orbifold

The classic mindmapping layout.

I’ll review a bit the features of the application which obviously reflect the features and capacity of G2 as a basis for diagramming in general.

Standard features

Menu items

• Saving to the internal binary format (*.g2d) and to XML (*.xml). You can alter the generated XML and open it up again. At some point I’ll publish the XML data format but you can experiment and find out yourself as well.
• Opening diagram from binary and XML formats. Both formats support multiple pages inside a single document.
• Creating a new document (CTRL+N) will request by default if you wish to save the current document first.
• CTRL+S and CTRL+O are bound and subsequent savings will first try the current file path. If you wish to save it elsewhere you can use SaveAs (no input binding here), but all this is pretty standard.

Note that a document (a mindmapping document in this demo) consists of some metadata and one or more pages. A page is much like a sheet in Microsoft Office Excel or Visio. Internally a page is furthermore subdivided into layers but in this demo you will not be able to have mutliple layers. In fact, it’s rather difficult to conceptually give meaning to layers in a mindmapping diagram.

Multi-page tab

Page can be added through the toolbar item entitled ‘Add page’ and removed or renamed through the tabpage context menu. A document has at least one page, which means you cannot delete the page if it’s the only one in the document. See also the know issue related to the tabs below.

Cut, copy, paste and delete (CCPD) need little explanations, I guess. Mind though that when navigating the diagram with the keyboard the focus is on the text inside the shapes. This means that at that moment CCPD has its effect on the text and not on the shapes. In order to CCPD shapes the focus needs to be on the diagram level. Just use the traditional rubber-band (marching ants, selector) to select shapes (use CTRL or SHIFT to add to an existing selection) and CCPD then. In this context I need to note that if you select multiple shapes which have a parent relationship (like in the screeenshot below) the upper parent will be used and hence all children underneath will be used in the CCPD process. This system ensures that no shape becomes an orphan in the CCPD process.

Illustrating the obvious

The O2 docking environment is an in-house development based on MEF, AvalonDock and Denis’ wonderful propertygrid (though in this demo it’s been hidden). The View menu allows you to bring up the various panels of the demo. Most notably, the Output window will display internal messages or (worse) exceptions and the internal browser is just a WPF Frame (hence, wrapped IE ). The docking layout can be saved via the Tools menu and re-opened later on.

Nothing spectacular but anyway, you can select a custom background for a diagram which in some cases can really make a diagram more attractive. With WPF there are so many possibilities…

Custom diagram backgrounds

Editing the diagram metadata (author, date and title), zooming and assigning an icon can all be done through the Diagram elements panel. There is also a thumbnail view which in a later phase will allow you to navigate the diagram, but I haven’t put much focus on this lately. Zooming can also be achieve via the mousewheel and pressing the CTRL key (quite standard nowadays). To reset the zoom factor just press/click the mousewheel (yes, you can press the wheel). An addition which I have overlooked as I’m writing this is the SHIFT with the mousewheel which in most applications allows you to scroll horizontally.

Diagram elements panel

Mindmapping specific features

Besides the features which are available to every G2 diagram type (or part of the O2 environment), there are also a few things specific to mindmapping which we’ll highlight below.

Mindmapping toolbar

One of the very productive features of WPF is the possibility to style or template a control. When you select a thought-shape you can alter its appearance through the dropdown in the toolbar (see screenshot). For this demo I created some custom styles which suit different data bucket but, as I described above, the possibilities are limitless and easily implemented. Take this together with the databinding and you have a mindmapping framework for every possible (business) context.

An overview of some styles defined in the demo

• CTRL+R rotates the diagram
• CTRL+ENTER adds a child node. Some dummy text is auto-generated which eases the testing process and easily creates maps in no time. As this is some kind of ‘random book-title generator’ you can have sometimes rather strange or funny titles
• CTRL+T switches between editing the text or thought title inline and editing the body text
• CTRL+M expands/collapses the children of the selected node
• CTRL+ARROW allows you to navigate the map through the keyboard. This togheter with CTRL+ENTER makes it easy to create dummy maps in no time.

Drag-drop. If a node is selected and you dragdrop an URL it will be attached to the selected node. If you dragdrop an image from the file explorer this will create a new child node to the selected node. Nothing will be created if no node is selected!

The thought editor. The data displayed in a shape is, as said earlier, just a data bucket bound to the visual in the diagram. For example, below is the definition of the ‘Shares’ shape template wherein you can see the different data bindings. So, in effect there is just a collection of data islands which you can bind both to the shape in the diagram but also to some editor. Altering the data in one will change the data everywhere else, which demonstrates brilliantly the power of WPF’s databinding. The though editor is a dockable panel which allows to change the data island and it’s rather self-descriptive. You can add some URL’s or some files to a thought through the listviews, change the body text, add an icon or change the title inline. Obviously the whole editor would be even more useful if RTF-ed or HTML-ed but I did not want to waste too much time on this. Ultimately, it’s just plugging in some open source or commercial editor since the data bucket only cares about the string result.

The 'Shares' template

One word of warning for those thinking that adding shapes and styles is only a matter of adding properties and changing the XAML. The underestimated part behind CCPD and saving documents is the whole (de)serialization process; making sure databinding works after deserialization, picking up the styles from the assembly’s resources, disassembling the layout organization, figuring out how to maintain the mindmap hierarchy and so on. To the rescue, the data exchange pipeline (aka. G2D) inside G2 maximizes flexibility through some smart usage of Unity (as explained elsewhere on this blog) which allows us to really change the import/export of XML in a snap, embrace data changes with ease and be prolific without having to step through the (de)serialization on a low level (anymore).

Things left out

• Other types of diagrams are not possible in this demo (though the XML manipulation will not stop you from getting there).
• Saving data to a WCF endpoint or database backend.
• Dragging thought-shapes and altering the map via the mouse.
• The possibility to mix free shapes and thought-shapes.
• Layers (see also my remark above).
• Saving/opening sub-document elements.
• Active shapes (RSS shape, Google search shape and other ‘intelligent’ shapes).

All these things are possible with G2 but I think the current demo is sufficient (for now) as other diagram types need attention as well…

What’s the point?

To paraphrase my compatriot; ceci n’est pas un logiciel. It’s a demo (I’d almost say a fancy UI facade on top of) showing what can be done with G2. The diagrams I talked about in other blog postings will be integrated in the near future to complete the demo so that in the end a more ‘covering’ picture appears of what G2 is. There as been an enormous interest in G2 and Graphite in the past months but unfortunately we have not been able to deliver a downloadable demo on time. I hope this first release already satisfies a bit the hungry (customer) minds out there.

You can do as you wish with this demo, there are no constraints. Of course, any feedback is welcome. If you really have the burning wish to turn this demo in something you think is worth selling you can drop me a mail, custom solution requests and business opportunities are welcome as well.

The next download of the G2 demo wil contain the other diagramming types, but no schedule on the table for now. We are developing custom diagramming controls for customers, elaborate data visualizations in our lab, handle heaps of mails and questions, have a fulltime job next to all this and take care of our family. Acrobatic challenges at times. Most probably you’ll see signs on this site of our other projects before the next G2 drop.

Download the February G2 mindmapping demo.

Known issues

.Net v3.5 SP1 is required to run this demo. You can download all this from MSDN.

The virtual double. This occurs when you load a document and the page tab control shows a double of some page. When you click on either when the other virtual double will disappear. This is a bug in the databinding for which I was unable to find an answer or trace the cause.

Doubling of pages on load

Sub-document file types. The open/save dialogs present you alternative file extensions (*.g2s, *.g2b, *.g2p). These types are not supported in this demo, though the serialization is inside G2. These extensions allow one to save only a page, a subset of shapes or just one single shape to a file. These types are useful to load templates and re-use single shapes in other documents.

Other file types

Undo/redo. The undo/redo mechanism is not everywhere properly bound to the actions. This is not a bug in G2 but rather a badly bound UI to the underlying core. Sorry.

Semantic note: the whole nine yards: means “completely, the whole, everything.

Althoug it might look like a piece of cake to display this kind of information and it seems like a simple shift from the class shape discussed earlier, it was quite a challenge to dot it. Not so much the diagramming part but the data grid inside the shape was somewhat difficult. While there is a datagrid for WPF it would be a heavy load on the diagramming surface to include it in the shape containers. So, the only option was to create a home-made lightweight datagrid. In addition, it had to have an inline editing capability in the form of comboboxes, textboxes, checkboxes and more. Finally, I also wanted the whole shape to have different view states; totally collapsed, names only and the full datagrid view.

In the picture you can see a sample diagram with the various features;

• the User & Countries tables are ‘full’, the Credential table is in a collapsed state and the Address table shows only the names of the fields. The upper-right icon allows you to cycle through the different views. This is not mendatory, however, as the icon simply launches a routed command which can be captured anywhere and you can call the ShowCollapsed(), ShowNamesOnly and ShowFull() methods on the original source to changes the view as you like.
• the lines in bold in the grid show either the active record or the primary key(s)
• double clicking allows one to edit the fields
• the labels can contain in general the info you like and it’s a generic feature of G2 rather than part of the database related shape info.

Although I did not fully develop things, one can go the full nine yards and turn this sample into either a database visualizer or a database editor (or both) much like in SQL Server’s management studio. Using the integrated analysis service of G2 one could also traverse the diagram and create scripts or XML on top of it. Well, you got the point I guess.