Blog Archive

Home
Javascript

Compilers are what make the software world function.  Really, where would we be without ‘em?  But compilers are like clans – you stick with the family.  Just like Romeo and Juliet, you can’t marry your enemies sister’s cousin, it just ends up in tragedy.  For the longest time, the .Net and C# world has been considered an “enterprise” thing, and by that we do not mean Captain Kirk and Spock:  .Net is for corporations, not the web.  It’s not Nodejs.  It’s slow.  It’s for the nerds who ain’t the cool nerds.  So take your compiler and stay on your side of the street.

Well, these days technology is a lot like the Berlin wall coming down, and like the union of the eastern and western European countries, the lines are blurred with static compiled languages of C# and Javascript.  You see, these days you compile C# code to Javascript.  In fact, a dirty secret is that you could do that for many years now.

DuoCode Is The New Kid On The Block

A new entry in this field is DuoCode.  From the site:

“DuoCode is an alternative compiler, powered by Microsoft® Roslyn, and integrated in Visual Studio.

It magically cross-compiles your C# 6.0 code into high-quality readable JavaScript code, enabling rapid development of web applications utilizing the extensive features of the C# language, the Visual Studio IDE, and the .NET Framework base class libraries.

Development in C# with Visual Studio brings great productivity wins, thanks to strong-typing, code completion, compile-time error checking, static analysis, code navigation and refactoring.

Develop HTML5 applications using strongly-typed and documented class-definitions of the entire DOM class library (including HTML, CSS, SVG and WebGL definitions).”

DuoCode works with Visual Studio and will compile assemblies from multiple projects to Javascript.  DuoCode claims to support LINQ, classes, Generics, lambda expressions, extension methods, and many features that are the strengths of C#.

The Question:  Should All C# Capabilities Be Ported To Javascript?

Does Javascript need type checking and casting?  Yes, no need to check that a int is not a string, but tying yourself to a compiler for Javascript is going to introduce a different type of workflow for your client side development.  Part of the refreshing aspects of Javscript is not having to compile constanctly.  ”You have to ‘build’ your website, hehe …” meant you have to sit and wait for Visual Studio to compile and deploy before you could debug.  One of the main reasons for adpating MVC and leaving ASP.Net Webforms behind was to get away from the stilted, awkward development process when you wanted to examine an issue with your app’s UX.  Compiling your Javascripts feels a bit like a step backwards.

That said, LINQ and lambda expressions are a really good thing.  They’re like power tools for your boilerplate code. Javascript, while having tools like underscoreJS and lodash, is still an open field in this respect.  That’s not to say that there are no alternatives and efforts to build those type of capabilities; indeed, there are hundreds of open source projects and other efforts to create better functionality, and with EM6 on the horizon things like iterators will become part of the new Javascript specification.

Quite Hemming and Hawing – Why Would I Need This?

Ok, under what circumstances will you need this capability, and more importantly, is the generated Javascript any good?  In response to the former query, you may have a series of objects that are tested, run on your server, and you want those to

also run in your UX.  Maybe you have a state machine and a process that you want to run on a mobile device and instead of transpiling it to Mono or Swift you want to go the HTML 5 route.  Or, perhaps you want to port a portion of your code base to nodeJS for message brokering.  This could help tremendously.

Yet lurking in the background here is this:  what does the ported Javascript look like, and if there are issues where do you go to fix them?  Are they a Javascript issue, or are the origins from C#, compounded by the code compiling to Javascript?

So what do you think?  

treebeard-the-ent-e1269783419659Organizing data so that you sort, filter or manipulate items efficiently is critical for front end applications.  These days its not enough to have a pretty design – it’s gotta like do some stuff.  Just sayin’.  It would be silly if you had to constantly send requests back to a server just filter them, or sort them in a new order.

Sometimes you might be asked to support hierarchical or nested data.  A perfect example is a project plan: there are levels and nested levels; there is an implied relationship between the parent levels and their children; the order of the list items implies a precession, or a progression.  Consider also thate each list item not only has a task, but also has the duration for that task, and you have to total all the durations for an estimate.  Perhaps your crazy manager asks you to only fetch the tasks where a certain team member works, oh, and by the way, exclude all the tasks that are parent level tasks.  As you can see, there is a need for a structured approach that you can easily re-use.

Simple Nested Structures May Not Be Enough

In general, creating nested data structures is fairly straight forward.  Let’s continue with our scenario of a project plan:


var Task = function (options) {
    var _options = options || {};

    var _description = _options.description || "";
    var _assignedTo = _options.assignedTo || "";
    var _duration = _options.duration || 0.0;

    var _tasks = [];

    return {
        description: _description,
        assignedTo: _assignedTo,
        duration: _duration,
        tasks: _tasks
    };
};

We will represent each project item as a Task that has a description, the team assigned to the task and a duration. The array “_tasks” represents the sub tasks or children of a task. You can indeed represent a hierarchy by simply adding this array and achieving what ever nesting levels that you desire.  This jsFiddle illustrates what that would look like.

While this does the job, how are you going to provide filtering? We need a way to walk the hierarchy; furthermore, we don’t want a one time function for type of traversal. Also, how do we handle the times when we need to affect a change only to items that are two levels deep. Simple recursion will not be enough here.

TreeNodes Can Provide Help

What we need is a data structure that will tell us where we are in the hierarchy of data items; who the parent of an item is, and if there are any children for a particular item. Once we can obtain these, performing operations are much easier, and we can answer questions like “What are all the tasks that over 10 hours” or “what are all the tasks for ‘POC Development’”.  Being able to walk the hierarchy will also give us the capability to automatically calculate the overall hours for a project.

var TreeNode = function (options) {
        var _options = options || {};

        var _value = _options.value || {};
        var _children = _options.children || [];
        var _parent = _options.parent || null;

        //  Public methods
        var isRoot = function () {
            return (_parent == null);
        };

        var level = function () {
            return isRoot() ? 0 : _parent.level() + 1;
        };

        var addChild = function (value) {
            var childNode = new TreeNode({ value: value, parent: this });
            _children.push(childNode);

            return childNode;
        };

        return {
            value: _value,
            children: _children,
            parent: _parent,

            //  public methods
            isRoot: isRoot,
            isLeaf: isLeaf,
            level: level,
            addChild: addChild
        };
    };

Let’s dissect this a bit. A TreeNode will wrap an object; more specifically “_value” will be the container for our Task object. “_children” is an array of TreeNodes that each wrap another Task object. A TreeNode can have a “_parent”, so naturally any item the _children array will “point” back a single TreeNode. This establishes our relationships between tasks and sub-tasks. In some scenarios you have a TreeNode and need to a property for that TreeNode’s parent, and “_parent” offers that convenience.

The function “isRoot()” tells us that we are at the top of a hierarchy. “level()” tells us what level a TreeNode sits at. Looking at this code you’ll note that it relies on “isRoot()” to determine whether to return a 0 or simply increment the “_parent.level”. To create your hierarchy, simple call TreeNode.addChild(new Task);

If you look at the jsFiddle you’ll see that we have a hierarchy established, and the path of “Check Team Schedule” can represented as tasks[3].tasks[0].tasks[2].  If we were to set a variable to tasks[3].tasks[0].tasks[2] we are disconnected from our hierarchy, as the individual items sitting in each “tasks” array are unaware of their parents.  In order for us to start any filtering or querying, we will first have to pull the tasks apart and re-assemble them as TreeNodes.  Luckily we can create a function to build a tree from our Task objects:

var treeFuncs = function(){
    var buildTaskTree = function (sourceTasks) {
        var tree = new TreeNode({
            value: "ROOT"
        });

        var taskStack = [];
        var treeNodeTargetStack = [];

        taskStack.push(sourceTasks);
        treeNodeTargetStack.push(tree);

        while (taskStack.length > 0) {
            var currentTask = taskStack.pop();
            var targetNode = treeNodeTargetStack.pop();

            var length = currentTask.length;
            for (var i = 0; i < length; i++) {
                var task = currentTask[i];
                var childNode = targetNode.addChild(task);

                if (task.tasks.length > 0) {
                    taskStack.push(task.tasks);
                    treeNodeTargetStack.push(childNode);
                }
            }
        }

        return tree;
    };

    return {
        buildTaskTree: buildTaskTree
    };
};

The function buildTaskTree accepts an array of Task objects. Next, we will create a root for our TreeNodes, and all other child TreeNodes will be added to this variable. Instead of using recursion, we will create two stacks “taskStack”, where we will store all the Task objects, and “treeNodeTargetStack” will hold the current TreeNode that will receive new children. Note that we are “seeding” the “taskStack” with the incoming array of Task objects. “targetTreeStack” will have the “tree” variable that is the root of the tree that we are building.

The while loop will always pull from the taskStack; more specifically, the first pass will have the top level tasks “Discuss Product Concept”, “Create Mockups”, “Setup POC environment”, “POC Development” from our project plan. We look at each item and add it to the current position of the tree. If the current Task has children we will want to do two things:

  1. Push the child tasks array onto the “taskStack”. When we execute the loop again, it will pick up this new entry and continue our processing.
  2. Push the newly created child TreeNode onto the treeNodeTargetStack so that we maintain the hierarchy between the tasks. The next time we execute the while loop, we are the current TreeNode and add to its children. This maintains our hierarchy relationships.

That’s it. Each subsequent pass of the while loop drives deeper into Tasks hierarchy as each child is added to “taskStack”. When we’re done, we return “tree” as our TreeNode structure.

How To Put TreeNodes To Good Use

Great. We have TreeNodes. Now what? Well, our goal was to create structure that we could traverse and always know who our parent was, and who our children were, as well as creating the ability to determine what level in the hierarchy we are located.  To find answers to questions that require us to filter, sort, etc we will visit each node on the TreeNode structure, and at each stop we will perform a function that will act as our query and record the result for that node.  When the trip is done we’ll have a subset of our tree.

First the function that walks the TreeNode:


var traverseAndApply = function (treeNode, func, functionArgs) {
        var treeNodeStack = [];
        var rootArray = [];
        rootArray.push(treeNode);

        treeNodeStack.push(rootArray);

        while (treeNodeStack.length > 0) {
            var currentNodeList = treeNodeStack.pop();
            var length = currentNodeList.length;

            for (var i = 0; i < length; i++) {
                var currentNode = currentNodeList[i];
                if (currentNode.parent != null) {
                    func(currentNode.value, currentNode, functionArgs);
                }

                if (currentNode.children.length > 0) {
                    treeNodeStack.push(currentNode.children);
                }
            }
        }
    };

The function traverseAndApply accepts three arguments. The argument “func” is a function, and this is the function that will executed at each node in the TreeNode. “functionArgs” are the items that will be pass into the “func”. The remainder of his function may look familiar – the approach is the same as in buildTaskTree. Again, we are avoiding recursion and are using the stack approach. We push the TreeNode to an array, then push the array onto a stack. As we loop, we pull each TreeNode, fire off “func”, then progress to the next.

So let’s put this to use. Our first example will be to find all tasks that are the top level tasks, in the parlance of our TreeNode, where the TreeNode.level() is equal to 1.


var treeNode = _treeFuncs.buildTaskTree(_tasks);

        var matchTopLevelArgs = {};
        matchTopLevelArgs.results = []

        var matchTopLevelFunc = function(value, node, funcArgs){
            if(node.level() == 1){
                funcArgs.results.push(value);
            }
        };

        _treeFuncs.traverseAndApply(treeNode, matchTopLevelFunc, matchTopLevelArgs);

And now let’s find all of Frodo’s tasks that are not top level tasks:

var treeNode = _treeFuncs.buildTaskTree(_tasks);

        var frodoTasksArgs = {};
        frodoTasksArgs.results = [];

        var frodoTasksFunc = function(value, node, funcArgs){
            if((value.assignedTo.indexOf("Frodo") > -1) & (node.level() != 1)){
                funcArgs.results.push(value);
            }
        };

In our jsFiddle you’ll find working versions of the above filter examples. Experiment around and see what you can come up with.

Future Considerations

Now that we have the basics down you can see that we have a very powerful tool. No doubt you have noticed that the hours for our project don’t make sense, as normally the duration of a task is comprised of the duration of any sub-tasks. Since we can visit each node, we may be able to devise a way to create a function that can calculate the duration for each level appropriately.

Another idea could also be the ability to change a task to sub-task or maybe make a sub-task into a new parent.  Our ability to manipulate the TreeNode and it’s underlying data can go a long for us.  So that’s what we will explore next time.  Until then, play the video below and enjoy some tree music.

This post is in our Javascript Primer series, a collections of articles aimed at .Net and back-end developers who are transition to client-side UX development.  In a previous installment, the topic of publish-subscribe design pattern was featured, and this pattern fostered loosely coupled ViewModels.  It is recommended that you read that post as well, as this edition of our primer builds on those concepts.

A basic tenant to good programming is to break down activities to small components.  Smaller functions are easier to maintain.  In many instances the information that the users are working with is best presented in logical groups, dissected so that an improper decision can never be made.  A rich user interface may force you to break a larger object into several ViewModels.  But guess what?  If you need to break information down, will also need to re-assemble this information to transmit back to mother ship to be stored in your database.

Challenges for the ViewModels

There are several challenges ahead for you if need your ViewModels to communicate, and you want to retain the agility provided to you with loose coupling.  If you recall the architecture we established with the publish and subscribe design pattern, all our “pieces” are highly independent from one another.  But we now have a complication.  Independence also means blissful ignorance.  Each ViewModel only knows how to contribute their small portions of information for glorious “mother object”.  The best it can is yield up what it has and somehow, someway, something will piece things together.

Let’s consider a code example:

We have simple project tracker that has a name, a roster of workers and a calendar that displays each workers start date.  Each tab is serviced by a ViewModel.  The “Project Info” tab shares the project name information with “Team”.  The tab “Team” shares the project team data with both the “Project Info” and the “Timeline” tabs.  This is accomplished using the publish and subscribe pattern that was detailed in this post.  You should review that post is your are unfamiliar with postaljs or with the publish and subscribe design pattern.

The challenge here is that we want a “Save” button that will easily gather the data from each tab and save it our project object.   We also want this controlling activity to be flexible and allow each ViewModel to supply the data needed without too much orchestration: in other words, we just ask the ViewModel “What do you have for a project object”, and each ViewModel fulfills its responsibility.

Pipe and Filter Pattern For Chaining Events or Commands

Many of you with a background in server side application development may recognize the pipe and filter pattern.  Simply said, it is a way to chain a set of operations that are to be performed in succession.  This diagram is a depiction of the pipe and filter:

PipesAndFilters

The “Filter” is a function that performs one thing.  Our pipe is the “orchestrator” or directory in that it will forward to each filter as ordered.

For our purposes we will continue to use postaljs as our communication mechanism.  We’ll need a pipeline that will keep track of which filter to execute and forward data to each subsequent filter of the process chain.

var pipeline = {
  index: 0,
  filters: ["edit.getViewOneData","edit.getViewTwoData","edit.finalDestination"]
 };

The property “filters” contains the names of the postaljs topics that will kick off a function.  In this case we will need 3 subscriptions that listen for items listed in the “filters” array.  Each process step, or filter, will be executed in the order that listed in the “filters” array: in this case, “edit.getViewOneData” will be executed first.  The property “index” tracks which filter is being processed.  As each filter is executed, it will increment index, then use index to access the next topic in the chain of events.  Starting off the process is accomplished in this fashion:

// Now start up the pipeline, fire off the first step that will execute the first filter
 postal.publish({
  channel: "pipenfilter",
  topic: pipeline.filters[pipeline.index],
  data: pipeline
 });

//  A sample subscription.  This will be executed first since it is listed first in the pipeline.filters array
postal.subscribe({
  channel: "pipenfilter",
  topic: "edit.getViewOneData",
  callback: function (pipeline, env) {
    fetchViewOneData(pipeline);
}
 });

//  After filter performs, it needs to call the next filter in the chain<
var fetchViewOneData = function(pipeline){
  // ... perform activities with some data

  //  To forward to next step, increment the index
  pipeline.index++;

  //  Now forward to the next filter
  postal.publish({
    channel: "pipenfilter",
    topic: pipeline.filters[pipeline.index],
    data: pipeline
  });
};

For our particular scenario with the project UX above we will have to do a few simple things:

  1. The ViewModels “ProjectInfo”, “Budget” and “Team” will need subscriptions to a “pipenfilter” channel
  2. Each ViewModel will need a corresponding filter function, so the “Team” ViewModel will get a “fetchTeamFilter”
  3. Add an endpoint in our $(document).ready section to act as a controller for the pipeline process.  Since we don’t have a controller this logic can sit here.  Naturally for cleaner production code you’ll want to be a bit cleaner and place this in a controller object.

All of the addition changes are noted with a “New pipe and filter” comment, so you can search through the Javascript to see where the changes have been made.  Most of the updates are just a few lines, and the nice thing is that the code to forward to the next filter is the same each time.  Not much exciting stuff to be seen other than incrementing the index and passing the data to the next topic.

Where No One Has Gone Before (just sayin’)

Now that you can synchronize your data, assemble it back into a proper object, you can begin your next journey.  There’s many long term benefits here that you will begin to realize.  What if you need to add an additional ViewModel, or what if you need to change which ViewModel display the groupings of data?  Without our pipe and filter solution, you would need to change a lot of things.  With what we have described here, the most that have to do is to add an additional filter name to the pipeline list and make sure that your additional ViewModel functioned properly.  The other ViewModels are left untouched.

Another benefit is also be that you can control what functions are fired while allowing the ViewModels to do their thing.  What if you had a ViewModel that published changes to other ViewModels after it finished performing calculations.  By ordering the filters in the pipeline accordingly, you can allow the ViewModels to do that hard work and be assured that all aspects of your data is up to date.  Doing more with less code means you can test and maintain this code with confidence.

This is a continuation of the Javascript Primer series where we focus on topics essential for .Net or back-end developers or who are transitioning to Javascript and client side development.  Many of the topics covered in this series will be related to producing web apps running in the browser.  Some design patterns and practices are also application to NodeJS / server-side Javascript apps as well.

Keeping IT Together

One complaint that many have when embarking on applications of medium complexity is that organizing your code is difficult; indeed, even with frameworks you can still end up with giant god object that contains all data-bindings, algorithms, helpers methods.  If you take the attitude that Javascript is a second class, toy language where you simply store jQuery animation methods and AJAX calls, you quickly end up with files that can be 3-4000 lines long!  Maintaining that is beyond torture.

A common design pattern used to alleviate this issue is the Model View ViewModel (MVVM) pattern.  The goal of MVVM is to divide up the logic of your application into UX, business logic, and data-binding activities into discrete components.  KnockoutJS is a popular framework that supports this approach.  Addy Osmani’s “Design Patterns for Javascript” defines the primary components of MVVM, and it is great primer to read if you are unfamiliar with these terms.  For our purposes:

Model will contain the data elements, such as FirstName, LastName, etc.  Essentially this is a building block for your front-end that will “model” entities such as user, customer, project, etc.

ViewModel will contain the behavior for the UI.  This can be business logic, formatting of data, and bridge the gap between data coming from a server and what is useful for the user to interactive with.  The ViewModel will also use the Models.

View will be the html document.  This will also contain the directives for what to present to the user, and will interact with the ViewModel.

We Still Have A Problem

So even with the MVVM pattern at our disposal we still have to fight bloated ViewModels.  From here out we will use a sample project for our discussion where we want to maintain project information comprised of project name, project description, and project notes, team members and a simple timeline / calendar function.  Sounds straight forward enough.  On our UI we will have a tab for grouping of information, and as information changes on a tab, we want that information available in each subsequent tab.  Again, nothing too earth shattering here.  It should look something like this JSFiddle below.  (NOTE:  For the sake of this exercise all code has been placed in the Javascript section of our JSFiddle.  You should split your code into modules so they can easily managed).

Each tab on this screen has logical groupings of information.  For the sake of argument let’s say that we decide to implement this UI with just one ViewModel for all three tabs.  For databinding we will be using KnockoutJS,  Starting out we have just one ViewModel – ProjectViewModel – that handles all updates; more specifically, we can take advantage of the data-binding facilities here, so when a team member is added, we can detect this change and automatically execute and update to the calendar.

We have three tabs – “Project Info” is pretty simple, and it uses the computed observable teamSize to display the count of the project team. You note that any changes to “Project Name” will be reflected on the “Team” tab.   The tab Team allows you to add a team member.  To support updating the calendar with the team member start date we have create subscription to the projectTeam observable array.  The tab “Timeline” has a calendar that displays the team member start date, and this is updated each time we add a new team member.  Also, any changes to the number of team members will be reflected on the first tab, “Project Info”.

Indeed this works well, since we have the advantage of one ViewModel and can benefit from easily sharing one set of data between all three tabs, But we still have an issue should we start adding more functionality and increase the properties and functions in the ProjectViewModel.  Soon maintaining the ViewModel will become difficult, and testing discrete portions of functionality will rapidly become difficult.  We need to be able to isolate or reduce the impact that the alterations made to the ViewModels have on each other.  This means adhering to the principal of “loose coupling”.

So what does it mean for us to create a ViewModel per tab?  The biggest issue now will be communicating between each ViewModel, and as always, our mantra should be “Keep it simple.  Please.  Pretty please”. So how do we achieve communicating data updates without creating three ViewModels that are still highly dependent on the others?  Our solution is to implement a subscribe and publish message architecture.  While this sounds intimidating, this simply means we will create each ViewModel with a method – a “subscription” – that will wait for a message that has the data we need.  As a corollary, each ViewModel need a way to “publish” any changes to data that occurs.

Super Simple Message with PostalJS

PostalJS is a very simple messaging framework that facilitates sub / pub messaging in a unique way.  In particular, PostalJS allows us to set up a subscription that remains “ignorant” of how the message is generated.  There is no “pre-registration processes” to map out a route between ViewModels; on the contrary, your Javascript object simply declares that will update when a subscription arrives.  This also means that several ViewModels can subscribe to an event and the publisher doesn’t care.

Here is a brief sample taken from github:

var subscription = postal.subscribe({
        channel: "orders",
        topic: "item.add",
        callback: function(data, envelope) {
            // `data` is the data published by the publisher.
            // `envelope` is a wrapper around the data & contains
            // metadata about the message like the channel, topic,
            // timestamp and any other data which might have been
            // added by the sender.
        }
    });

postal.publish({
        channel: "orders",
        topic: "item.add",
        data: {
            sku: "AZDTF4346",
            qty: 21
        }
    });

Looks pretty cool.  And simple.  So for us, our ViewModels will remain ignorant of each other, and only are aware that someone, something has sent data to it.  Another nice by product is that should one of the ViewModels fail, the impact is greatly reduced.  So should the ViewModel with the calendar fail, the other tabs could still collect data and function.  Better yet, adding new features to a ViewModel is going to be easier to achieve since we can test in isolation, and if the new features do not interrupt our sub / pub messaging, we can feel confident that new changes can be rolled into production without bringing down the entire application. Let’s look at the results, then examine the changes to our app’s architecture.

A Brave New World

If you open the Javascript tab of the JSFiddle, you’ll see that we have broken up the ProjectViewModel into 3 ViewModels:  TeamViewModel, TimelineViewModel, and of course the remants of ProjectViewModel.  We’ll take the updates on at time, focusing on the feature in the UI and work back to the code.

Project Name As before, entering the project name in the “Project Info” tab updates the title in the “Team” tab.  Same as before, the update happens when the input box loses focus.  Now let’s focus on the code.  Open up the Javascript in the JSFiddle, and you’ll see some drastic changes.  In the ProjectViewModel we have created a Knockout trigger that publishes a message:

</pre>
<pre data-language="js">_projectName.subscribe(function(newValue){

        //    Now use PostalJS to push message out
        postal.publish({
            channel: "project",
            topic: "edit.projectNameChange",
            data: {
                projectName: _projectName()
            }
        });
    });

So what we are doing here telling Knockout “Anytime there is a change to the observable _projectName, fire off a function that publishes a piece of data with the new value of _projectName”.  That’s it.  ProjectViewModel has fulfilled its responsibility and told anyone who will listen that a change has taken place.  Now find the TeamViewModel object.  Here we have a subscription that waits for a message containing _projectName updates:

</pre>
<pre data-language="js">var projectNameChangeSubscription = postal.subscribe({
        channel: "project",
        topic: "edit.projectNameChange",
        callback: function(data, envelope) {
            _projectName(data.projectName);
        }
    });

We get the data, and merely update an observable on the TeamViewModel.  It should be apparent by now that neither ProjectViewModel nor TeamViewModel “know” about each.  ProjectViewModel says “_projectName update ready” and who can handle it, does so. TeamViewModel just grabs the update, uses it.  That’s it.  The messaging works by creating a “channel” – in this case “project” – further defines the message with a topic – “edit.projectNameChange”.  This means you can a single channel that support multiple topics and narrowly define your event messages.

ProjectTeam Update Play around with the “Team” tab and add some team members with different dates.  The operates just as before.  Under the hood, the code in TeamViewModel that handles the publishing is:

</pre>
<pre data-language="js">_projectTeam.subscribe(function(){
        //    this is how postaljs publishes events
        postal.publish({
            channel: "project",
            topic: "edit.teamUpdate",
            data: {
                projectTeam: ko.toJS(_projectTeam)
            }
        });

    });

As before, we are asking KnockoutJS to watch for changes to the _projectTeam array, and when a change occurs, we publish a different message.  The portion of code – topic: “edit.teamUpdate” distinguishes this event from the previous on.  We simply take our observableArray and convert it a plain Javascript script array and throw out to whoever wants it.  If you look in both ProjectViewModel and TimelineViewModel you’ll see subscriptions that handle this topic.  This brings us to another important aspect of messaging and postaljs’ strength:  should we want to add more ViewModels that need _projectTeam updates, we only need to subscribe to the topic “edit.teamUpdate”.  We could subscribe 50 more times if we wanted.  The publisher / source ViewModel does not need any alteration, and this achieves our goal of loose coupling.

What Are The Other Benefits?


Looking at our ViewModels, you may have noticed that we are exposing less in the return statements, and with each ViewModel handling less functionality they become simpler to read and maintain.  Imagine that the needed to support 20 different data entry input boxes – one monster ViewModel would have a huge return statement.  It makes it hard to keep things straight in one “god” class.

With PostalJS and event messaging we no longer need to expose an entry point in our objects for data to be passed.  The subscriptions allow the ViewModel to handle receiving update messages internally.  A quite common scenario would be responding to an AJAX update.  With PostalJS you can avoid calls to each ViewModel when new data arrives from the server.  Simply publish to the appropriate topic and let the ViewModels respond as they need.

“This. Is Not. For. Practice!” – a phrase used to emphasize that you must get things right.  Mentors have used it to mean “Get serious, now!”.

For years jQuery has been a tremendous productivity boost for client-side development; so successful, that it gave rise the Single Page Application (SPA) where a good portion of your application logic was now written with Javascript. Soon, so soon, we all learned that the increased complexity of our client side solutions required good organization, a better architecture.  Time to get serious about clean code.

If you are .Net developer, then many times you can fall into the ECMAScript trap: all ECMAScript must be the same.  What I know from C# must be applicable to Javascript, ’cause it looks the same. Sad news, it’s not. Javascript executes in a much different environment. And to make things more confusing, close cognates such as this are not close enough. “this” in C# is a reference to the current object. “this” is Javascript is , well, it all depends. “this” in Javascript changes context dependent on many factors.  A good reference is here at Mozilla.  Let’s create a quick summary, and where relevant we’ll talk relevant areas that impact programming practice:

Global Context

When you are outside any function, this is referred to as the global context.  ”this” will refer to document.  So if you fire up the console and perform this test:


console.log(this == document); // true.

And in web browsers this also equals the window.  So:


console.log(this == window); // true.

Yikes!  Wait there’s more.  Try this little piece:


var yikes = "ouch";

console.log(window.yikes); // ouch

window is our global memory space.  So think back to any client side code you have written, and this should be very familiar to you:


<script type="text/javascript" src="scripts/jquery.min.js"></script>

This loads jQuery into the global memory space.   Go look at window and you will see “window.$” and “window.jQuery”.  You guessed it, jQuery is now loaded into the global namespace.

This may seem like a digression, but we need to pause at this very important point.  You must be really careful what you put into this global space.  As you grow in Javascript capability you will eventually develop your own utils or libraries.  So consider this very innocent utility file:

  function objectToArray(sourceObj) {
    var results = [];

    for (var key in sourceObj) {
      if (sourceObj.hasOwnProperty(key)) {
        var newObj = sourceObj[key];
        newObj.objectKey = key;
        results.push(newObj);
      }
    }

    return results;
  };

function localize(t) {
  var d = new Date(t + " UTC");
  return d;
}

function parseDecimal(d, zeros, trunc) {
  if (d == null)
    return 0;

  if (typeof d == "number") {
    return parseFloat(d);
  }

  d = d.replace(/[a-zA-Z\!\@\#\$\%\^\&\*\(\)\_\+\-\=\{\}\|\[\]\\\:\"\;'\<\>\?\,\/\~\`]/g, "");
  while (d.indexOf(".") != d.lastIndexOf("."))
  d = d.replace(/\./, "");

  if (typeof zeros == 'undefined' || zeros == "") {
    return parseFloat(d);
  }
  else {
    var mult = Math.pow(10, zeros);
    if (typeof trunc == 'undefined' || (trunc) == false)
      return parseFloat(Math.round(d * mult) / mult);
    else
      return parseFloat(Math.floor(d * mult) / mult);
    }
};

All of these functions are now attached to the global namespace, our what is called “polluting the global namespace”.  Most likely a utility class will have 20 – 30 functions, not the mere three listed above.  These functions will remain in memory until the user refreshes or navigates away from the page.  With a small app, this will be no problem, but anything with a moderate degree of code will soon balloon and consume memory.  And in some cases, such as a mobile friendly websites, you want to conserve as much memory as you can, only loading the objects that you need then allowing them to be garbage collected.

Wow – we’ve only covered “this” in a global context, and if your head isn’t a little numb, don’t worry, do another shot and see if it gets worse!

Function Context – Simple Call

Here we get a strange, as this is the rule:  ”this” will depend on how the function is called.  Consider this code:


var myFunc = function(){
   return this;
 };

myFunc() == window;

Since we are not using strict mode “this” must refer to something so it will default to the global object.  Changing the code to:


var myFunc = function(){
  "use strict";
  return this;
 };

myFunc() == "undefined";

Function Context – As An Object Method

The rule here is “this” refers to the object that the method is called on.  For example:


var mathDoer = {
  someNumber: 13,

  addSomeNumber: function(addMe){
    return this.someNumber + addMe;
  };
 };

mathDoer.addSomeNumber(7); // returns 20

You’ll note that both mathDoer.someNumber and mathDoer.addSomeNumber() are public in scope.  When first developing in Javascript it may be tempting to create your objects as depicted below, and while it is legitimate, it will lead to clutter very quickly:


var mathDoer = {};

mathDoer.someNumber = 13;

mathDoer.someArray = [];

mathDoer.addSomeNumber = function(){ ...};

Imagine this is a none trivial object, and you will have a cluttered mess.  Also, all your methods and properties are public, which you may not want to expose.

Function Context – As A Constructor

When used in a constructor “this” will refer to the object that is being instantiated.  This is best illustrated with the following:


var mathDoer = function(){
this.someNumber = 13;

this.addSomeNumber = function(addMe){
    return this.someNumber + addMe;
  };
 };

var mathIsFunObj = new mathDoer();

mathIsFunObj.addSomeNumber(7); // returns 13

Many of you may be coming to Javascript via KnockoutJS, a data binding library that provides two way binding between the DOM and Javascript objects.  Many of the KnockoutJS samples use this “style” for creating ViewModels:


var mathDoerViewModel = function(){
  this.justAnInt = 5;
  this.someNumber = ko.observable();

  this.addSomeNumber = function(addMe){
    return addMe + this.someNumber();
  };
};

var vm = new mathDoerViewModel();
ko.applyBindings(vm);

Again, our observables and functions are public in scope.  If you have a separate file for this ViewModel and simply include it with the HTML your “vm” variable is now added to the window object and part of the global memory space.  This can be considered good or bad.  From a diagnostics perspective, you can sit at the console and see what our ViewModel is doing by simply issuing commands like “vm.someNumber(45);”  Good for debugging.  This is also considered bad by others for two reasons: everything is public, and this can lead to bloated objects that are hard to use; and, you have added yet another item to the global memory space.  Imagine you have several ViewModels for a single page.  Things can get confusing very quickly.

So how can you make items private?  Remember that the scope of “this” when used in a constructor is a reference to that object – in our case that object is mathDoerViewModel.  If you were merely to declare a variable in the constructor, the scope of that variable would be local to the constructor.  This “hides” the variable from contexts that are outside the scope of the constructor.  That would look like this:


var mathDoerViewModel = function(){
  var _justAnInt = 5; //this makes the variable private

  //  function is local in scope
  var privateFunction = function(){

    //  our locals are available as well a methods and properties from the object ("this")
    return _justAnInt + this.someNumber();
  };

  this.someNumber = ko.observable();

  this.addSomeNumber = function(addMe){
    return addMe + this.someNumber();
  };
};

From the console still can interact with mathDoerViewModel.someNumber() and mathDoerViewModel.addSomeNumber(). We also have private functions and variables now: _justAnInt and privateFunction. For the .Net developer these conventions should have more familiar feel.

When you scour the web for Javascript code, you’ll find that many of the Javascript heavies will use a different standard for variable / scope hiding. We touch on that briefly here before we move on.  In Javascript, your constructor returns a reference to the object you are creating.  In other words, your constructor will return a version of “this” that refers to the objects properties and functions.  However, your constructor can also return an object.  This gives you great flexibility, and makes the  “Module Reveal Pattern” possible.  First here is what our ViewModel would look like using the Module Reveal Pattern:


var mathDoerViewModel = function(ko){

  var _justAnInt = 0;
  var _someNumber = ko.observable();

  var privateFunction = function(){
    return _justAnInt + _someNumber();
  };

  var addSomeNumber = function(addMe){
    return addMe + _someNumber();
  };

  var setJustAnInt = function(value){
    _justAnInt = value;
  };

  var showMeTheInt = function(){
    return justAnInt;
  };

  //  We return an object that refers to items we want to be public
  return {
    someNumber: _someNumber,
    addSomeNumber: addSomeNumber,
    showMeTheInt: showMeTheInt
  };
};

Looks a little different. In our constructor we are creating local variables and local functions, and in fact our observable – _someNumber – is now a local variable. But we “reveal” the items that we want to be public with the “return” statement. So in this case, we return an object with someNumber that references _someNumber, and references addSomeNumber. As a result, we can create:

  var vm = new mathDoerViewModel(ko);

  // Set value of someNumber
  vm.someNumber(35);
  vm.addSomeNumber(5); // returns 40

  //  Let's use our setter
  vm.setJustAnInt(12);
  vm.showMeTheInt(); // returns 12

Notice that we are passing in a “ko” variable in the constructor call. This serves many purposes: firstly, it makes the ViewModel modular, and allows us to load the module asynchronously with AMD tools like requirejs or curljs; secondly, the constructor signature lets us know that we are depending “ko” to make our ViewModel work; finally, it gives us a tiny speed boost as now that “ko” is local, since Javascript does not have to search up the stack to find it.

You should avail yourself of the fantastic book “Learning Javascript Design Practises” by Addy Osmani.  He’s been kind enough to place a copy online.  This will help you understand some of the code that is that would otherwise seem very strange compared to what you are used to in C#.

Function Context – As A DOM Handler

Moving on the DOM, “this” takes on a different context.  You’ll most likely be familiar with this common jquery statement:


$(".apparea").on("mouseover", ".milestone-border", function (event) {
//   here this means the DOM element with a class of "milestone-border"

$(this).addClass("selected-children");
 });

The rule here is that “this” means the DOM object that fires the handler.  So in this case “this” is equal to event.currentTarget.

Go Forth And Create

ImageBy now your head should hurt a little, with all these nuances it’s wonder that you don’t feel more beaten up.  That’s where the complexity of web development is under appreciated.  Hopefully we have covered enough here for you to save some time debugging, designing, or trying to understand some the vast Javascript frameworks, utils and libraries that are out there.  If you are coming from .Net and are new to Javascript, you avoid the trap that many have fallen into of creating monstrous Javascript objects of pure public functions and properties jammed into a single file.  This will lead to a deep valley of frustration when it comes time to refactor your code after you realize you have a hard time maintaining things.  If you can keep your code modular and have things reside in separate files, you’ll be better off.

We covered “this” in order to reveal some deeper yet vital nuances to Javascript.  Mastering “this” and the related topics above are going to help.  A lot.  This brings Javascript from being a “toy” to  really productive platform because you can avoid some pitfalls that slow you down.  In future posts we continue to build on this foundation where we focus on modular design and asynchronous loading.

Some gifts just keep on giving, and many times things can just take on a momentum that grow beyond your expectation. Bob Sherwood wrote to Sensei and pointed out that DataTables.net supports multiple column sorting. All you do is hold down the shift key and click on any second or third column and DataTables will add that column to sort criteria. “Well, how come it doesn’t work with the server side solution?” Talk about the sound of one hand clapping. How about that for a flub! Sensei didn’t think of that! Then panic set in – would this introduce new complexity to the DataTablePager solution, making it too difficult to maintain a clean implementation? After some long thought it seemed that a solution could be neatly added. Before reading, you should download the latest code to follow along.

How DataTables.Net Communicates Which Columns Are Involved in a Sort

If you recall, DataTables.Net uses a structure called aoData to communicate to the server what columns are needed, the page size, and whether a column is a data element or a client side custom column. We covered that in the last DataTablePager post. aoData also has a convention for sorting:

bSortColumn_X=ColumnPosition

In our example we are working with the following columns:

,Name,Agent,Center,,CenterId,DealAmount

where column 0 is a custom client side column, column 1 is Name (a mere data column), column 2 is Center (another data column), column 3 is a custom client side column, and the remaining columns are just data columns.

If we are sorting just by Name, then aoData will contain the following:

bSortColumn_0=1

When we wish to sort by Center, then by Name we get the following in aoData”

bSortColumn_0=2

bSortColumn_1=1

In other words, the first column we want to sort by is in position 2 (Center) and the second column(Name) is in position 1. We’ll want to record this some where so that we can pass this to our order routine. aoData passes all column information to us on the server, but we’ll have to parse through the columns and check to see if one or many of the columns is actually involved in a sort request and as we do we’ll need to preserve the order of that column of data in the sort.

SearchAndSortable Class to the Rescue

You’ll recall that we have a class called SearchAndSortable that defines how the column is used by the client. Since we iterate over all the columns in aoData it makes sense that we should take this opportunity to see if any column is involved in a sort and store that information in SearchAndSortable as well. The new code for the class looks like this:


public class SearchAndSortable
{
	public string Name { get; set; }
	public int ColumnIndex { get; set; }
	public bool IsSearchable { get; set; }
	public bool IsSortable { get; set; }
	public PropertyInfo Property{ get; set; }
	public int SortOrder { get; set; }
	public bool IsCurrentlySorted { get; set; }
	public string SortDirection { get; set; }

	public SearchAndSortable(string name, int columnIndex, bool isSearchable,
								bool isSortable)
	{
		this.Name = name;
		this.ColumnIndex = columnIndex;
		this.IsSearchable = isSearchable;
		this.IsSortable = IsSortable;
	}

	public SearchAndSortable() : this(string.Empty, 0, true, true) { }
}


There are 3 new additions:

IsCurrentlySorted - is this column included in the sort request.

SortDirection - “asc” or “desc” for ascending and descending.

SortOrder - the order of the column in the sort request. Is it the first or second column in a multicolumn sort.

As we walk through the column definitions, we’ll look to see if each column is involved in a sort and record what direction – ascending or descending – is required. From our previous post you’ll remember that the method PrepAOData is where we parse our column definitions. Here is the new code:


// Sort columns
this.sortKeyPrefix = aoDataList.Where(x => x.Name.StartsWith(INDIVIDUAL_SORT_KEY_PREFIX))
								.Select(x => x.Value)
								.ToList();

// Column list
var cols = aoDataList.Where(x => x.Name == "sColumns"
							& string.IsNullOrEmpty(x.Value) == false)
						.SingleOrDefault();

if(cols == null)
{
	this.columns = new List();
}
else
{
	this.columns = cols.Value
	.Split(',')
	.ToList();
}

// What column is searchable and / or sortable
// What properties from T is identified by the columns
var properties = typeof(T).GetProperties();
int i = 0;

// Search and store all properties from T
this.columns.ForEach(col =>
{
	if (string.IsNullOrEmpty(col) == false)
	{
		var searchable = new SearchAndSortable(col, i, false, false);
		var searchItem = aoDataList.Where(x => x.Name == BSEARCHABLE + i.ToString())
									.ToList();
		searchable.IsSearchable = (searchItem[0].Value == "False") ? false : true;
		searchable.Property = properties.Where(x => x.Name == col)
										.SingleOrDefault();

		searchAndSortables.Add(searchable);
	}

	i++;
});

// Sort
searchAndSortables.ForEach(sortable => {
							var sort = aoDataList.Where(x => x.Name == BSORTABLE + sortable.ColumnIndex.ToString())
				.ToList();
sortable.IsSortable = (sort[0].Value == "False") ? false : true;
sortable.SortOrder = -1;

// Is this item amongst currently sorted columns?
int order = 0;
this.sortKeyPrefix.ForEach(keyPrefix => {
	if (sortable.ColumnIndex == Convert.ToInt32(keyPrefix))
	{
		sortable.IsCurrentlySorted = true;

		// Is this the primary sort column or secondary?
		sortable.SortOrder = order;

		// Ascending or Descending?
		var ascDesc = aoDataList.Where(x => x.Name == "sSortDir_" + order)
								.SingleOrDefault();
		if(ascDesc != null)
		{
			sortable.SortDirection = ascDesc.Value;
		}
	}

	order++;
	});
});

To sum up, we’ll traverse all of the columns listed in sColumns. For each column we’ll grab the PorpertyInfo from our underlying object of type T. This gives only those properties that will be displayed in the grid on the client. If the column is marked as searchable, we indicate that by setting the IsSearchable property on the SearchAndSortable class. This happens starting at line 28 through 43.

Next we need to determine what we can sort, and will traverse the new list of SearchAndSortables we created. DataTables will tell us what if the column can be sorted by with following convention:

bSortable_ColNumber = True

So if the column Center were to be “sortable” aoData would contain:

bSortable_1 = True

We record the sortable state as shown on line 49 in the code listing.

Now that we know whether we can sort on this column, we have to look through the sort request and see if the column is actually involved in a sort. We do that by looking at what DataTables.Net sent to us from the client. Again the convention is to send bSortColumn_0=1 to indicate that the first column for the sort in the second item listed in sColumns property. aoData will contain many bSortColum’s so we’ll walk through each one and record the order that column should take in the sort. That occurs at line 55 where we match the column index with the bSortColumn_x value.

We’ll also determine what the sort direction – ascending or descending – should be. At line 63 we get the direction of the sort and record this value in the SearchAndSortable.

When the method PrepAOData is completed, we have a complete map of all columns and what columns are being sorted, as well as their respective sort direction. All of this was sent to us from the client and we are storing this configuration for later use.

Performing the Sort

[gigya src="http://listen.grooveshark.com/songWidget.swf" width="204" height="40" flashvars="hostname=cowbell.grooveshark.com&widgetID=23379337&style=water&p=0" allowScriptAccess="always" wmode="window" ](Home stretch so play the song!!)

If you can picture what we have so far we just basically created a collection of column names, their respective PropertyInfo’s and have recorded which of these properties are involved in a sort. At this stage we should be able to query this collection and get back those properties and the order that the sort applies.

You may already be aware that you can have a compound sort statement in LINQ with the following statement:


var sortedCustomers = customer.OrderBy(x => x.LastName)
                              .ThenBy(x => x.FirstName);

The trick is to run through all the properties and create that compound statement. Remember when we recorded the position of the sort as an integer? This makes it easy for us to sort out the messy scenarios where the second column is the first column of a sort. SearchAndSortable.SortOrder takes care of this for us. Just get the data order by SortOrder in descending order and you’re good to go. So that code would look like the following:


var sorted = this.searchAndSortables.Where(x => x.IsCurrentlySorted == true)
.OrderBy(x => x.SortOrder)
.ToList();

sorted.ForEach(sort => {
    records = records.OrderBy(sort.Name, sort.SortDirection,
                         (sort.SortOrder == 0) ? true : false);
});

On line 6 in the code above we are calling our extension method OrderBy in Extensions.cs. We pass the property name, the sort direction, and whether this is the first column of the sort. This last piece is important as it will create either “OrderBy” or the “ThenBy” for us. When it’s the first column, you guessed it we get “OrderBy”. Sensei found this magic on a StackOverflow post by Marc Gravell and others.

Here is the entire method ApplySort from DataTablePager.cs, and note how we still check for the initial display of the data grid and default to the first column that is sortable.


private IQueryable ApplySort(IQueryable records)
{
	var sorted = this.searchAndSortables.Where(x => x.IsCurrentlySorted == true)
										.OrderBy(x => x.SortOrder)
										.ToList();

	// Are we at initialization of grid with no column selected?
	if (sorted.Count == 0)
	{
		string firstSortColumn = this.sortKeyPrefix.First();
		int firstColumn = int.Parse(firstSortColumn);

		string sortDirection = "asc";
		sortDirection = this.aoDataList.Where(x => x.Name == INDIVIDUAL_SORT_DIRECTION_KEY_PREFIX + "0")
										.Single()
										.Value
										.ToLower();

		if (string.IsNullOrEmpty(sortDirection))
		{
		sortDirection = "asc";
		}

		// Initial display will set order to first column - column 0
		// When column 0 is not sortable, find first column that is
		var sortable = this.searchAndSortables.Where(x => x.ColumnIndex == firstColumn)
											.SingleOrDefault();
		if (sortable == null)
		{
			sortable = this.searchAndSortables.First(x => x.IsSortable);
		}

		return records.OrderBy(sortable.Name, sortDirection, true);
	}
	else
	{
		// Traverse all columns selected for sort
		sorted.ForEach(sort => {
		records = records.OrderBy(sort.Name, sort.SortDirection,
		(sort.SortOrder == 0) ? true : false);
		});

		return records;
	}
}

It’s All in the Setup

Test it out. Hold down the shift key and select a second column and WHAMO – multiple column sorts! Hold down the shift key and click the same column twice and KAH-BLAMO multiple column sort with descending order on the second column!!!

The really cool thing is that our process on the server is being directed by DataTables.net on the client. And even awseomer is that you have zero configuration on the server. Most awesome-est is that this will work with all of your domain objects, because we have used generics we can apply this to any class in our domain. So what are you doing to do with all that time you just got back?


ActiveEngine Software by ActiveEngine, LLC.