Introduction to the MEAN Stack, Part Two: Building and Testing a To-do List

In last week’s blog post, I showed you how to install all of the basic tools that you need to get up and running with the MEAN Stack. Didn’t catch that one and need help getting started with the MEAN Stack? You can find everything you need in Introduction to the MEAN Stack, Part One.

This time we’re going to take that shiny new MEAN Stack and put it to good use by building a very simple to-do list. We’ll also set up a basic automated testing suite. This process will take 12 steps and you can follow along here. I will link to the diff on github for each step below as well. The mean-stack-todo repo started off as a direct duplicate of the mean-stack-skeleton repo from last week.

Over the course of this tutorial, you’ll see some of the key features and concepts that make the MEAN stack such a powerful development tool. Here are a few highlights to keep in mind:

  • The MEAN stack comes with a very powerful suite of testing tools. Javascript’s flexibility makes writing tests extremely easy, Jasmine syntax makes your tests into a set of stories that help lower the barrier to understanding your code base, and tools like Karma and Nodeunit enable you to have unit test coverage of your entire stack. Even if you decide to not write unit tests for your application, AngularJS’s end-to-end test runner enables you to fully automate your integration testing.
  • You can include dynamic Javascript directly into your templates. The first rule of writing AngularJS code is that your Javascript should never reference the DOM. In the AngularJS world, $(‘input#inputName’).val(‘vkarpov’) is (almost) always the wrong thing to do. Your Jade templates should be the definitive guide to all of your app’s UI/UX decisions, while your client Javascript should simply maintain the state of your data.
  • You use the same language all the way through. MongoDB stores your objects in a format that is very similar to JSON and the database shell uses Javascript-like syntax. NodeJS queries MongoDB using JSON queries, then passes the resulting objects to AngularJS as JSON objects. What does all this mean? All Javascript, all day. This streamlines the code itself and makes coordinating development across your team way easier.

1) Create some sample data for the server to display (diff)
Our to-dos are going to have a description, a due date, and a boolean flag to indicate whether or not they’re already done. Our app.js file shows that when the user accesses the GET / route, ExpressJS will call routes/index.js “index” function, which in turn renders views/index.jade. Let’s add a couple of sample to-dos to index.jade’s template parameters.

2) Modify the index.jade template to display our sample data using ExpressJS templates (diff)
Let’s create a very simple layout to display our to-dos. We’ll have to separate our to-dos into two lists: one for completed to-dos, and one for outstanding to-dos. It’s very important here to note that by default, ExpressJS uses Jade for templating. Jade is a controversial templating library which uses a simplified and more human-readable version of HTML syntax. For example, let’s say we have the following HTML:

</pre>
<ul class="span6 text-center" id="tools-list">
	<li><a href="http://www.angularjs.org">AngularJS</a></li>
	<li><a href="http://www.expressjs.com">ExpressJS</a></li>
</ul>
<pre>

In Jade, this looks like:

ul#tools-list.span6.text-center
  li
    a(href='http://www.angularjs.org')
      | AngularJS
  li
    a(href='http://www.expressjs.com')
      | ExpressJS

3) Add Bootstrap to our project using Bower (diff)
You’ll want to scroll all the way to the bottom of the diff, because this diff includes all of Bootstrap and jQuery. In addition to changes in the layout, you’ll notice that we take advantage of some of Jade’s inheritance features. The index.jade template inherits from layout.jade using the “extends layout” on the first line. In this commit, we added “block head” to layout.jade, and then in index.jade we overwrote “block head” to include bootstrap in our template.

As an aside, you’ll want to install bootstrap via

bower install bootstrap-css
bower-install bootstrap-javascript

because “bower install bootstrap” only gives you an un-compiled version of Bootstrap. This is pretty silly considering the fact that Twitter maintains both Bower and Bootstrap. Fate, it seems, is not without a sense of irony.

4) Use AngularJS to make our template dynamic (diff)
One big problem with our to-do list right now is that there is no way to move a to-do from the outstanding list to the completed list, or vice versa. A Jade template is compiled once and then served up as static HTML to the user. However, if we use AngularJS’s two-way data-binding as a templating tool, our template updates automatically to reflect the state on the client side. Once you’ve implemented this, items will automatically move to the completed list when you click the checkbox on the outstanding list.

5) Install AngularJS’s end-to-end test runner and write a simple test (diff)
One of AngularJS’s original co-authors was Misko Hevery, my former mentor at Google whose sole mission in the professional world is to make sure every piece of software has 100% test coverage. As you might expect, AngularJS was built with ease of automated testing as a core design consideration. Angular-scenario is a part of AngularJS that essentially hijacks an iframe to perform tests on your site as the user would see it. After this commit, run “node app.js” and navigate to http://localhost:3000/tests/e2e/runner.html. You should see something like this:

screenshot-linux

This page will run the test we defined in public/tests/e2e/scenarios.js. As you can see, Jasmine syntax does a pretty good job of making tests read like stories. Our test says when we navigate to “/,” we should see two to-dos in the outstanding list and one in the completed list. Then we click on a checkbox and expect that one of the to-dos moved to the completed list. You can install angular-scenario via bower

bower install angular-scenario

6) Write code for a REST-ful path to let the user add a to-do and a unit test (diff)
For this step, you need to install nodeunit with

npm install -g nodeunit

You can run the test suite with

nodeunit nodeunit/*

from the repo root directory. You’ll notice that the route is defined as a function that returns a function. The purpose of this is two-fold. First, it lets you write a proper unit test for this function. The core tenant of writing unit tests is that if you introduce a bug to a clean code base, the only tests that should fail are the ones that test the newly broken function. If the to-do objects were not passed to the route as a parameter, the route would have a hidden dependency that we wouldn’t be able to mock out, so bugs in the to-do object itself could potentially break the route’s test as well. This would be especially bad if to-dos were a complex object rather than a simple list. The secondary purpose for defining routes in the manner described above is that this syntax allows you to easily see which functions depend on a given module, which is very important if you have to change how a module works.

7) Open up the REST-ful path for adding a to-do (diff)
In line with the argument in step 6, we’ll move the todos array into app.js for now and inject it into our GET / and POST /todo.json routes. This pattern is called dependency injection.

8) Create a basic form to add a to-do (diff)
AngularJS gives us the ability to override the default form-submission action using the ng-submit directive, so we can perform the POST /todo.json action without refreshing the page by using AngularJS’s $http service.

9) Add angular-ui-bootstrap and use its datepicker (diff)
Fair warning, this commit is a little difficult to read because it includes all of angular-bootstrap. You can install it through bower using

bower install angular-bootstrap

One critical feature that was hardcoded into the previous commit was the due date of the new to-do. It was set to one week from now and there was no way to change it. Thankfully, the angular-bootstrap module includes, among other features, a nice datepicker directive. The hardest part is setting up an AngularJS module so that we can include angular-bootstrap. AngularJS uses modules to organize dependencies, so we need to create a TodoModule in public/javascripts/TodoModule.js and explicitly tell AngularJS that this module depends on the ui.bootstrap module (in addition to including the actual ui-bootstrap Javascript file). Once this is done, adding a datepicker input with two-way data-binding to a Javascript date becomes a trivial one-liner:

datepicker(ng-model="newTodo.due")

10) Set up a MongooseJS model (diff)
MongooseJS is a schema and validation tool for NodeJS and MongoDB. You use MongooseJS essentially the same way you would use an ORM in other development environments. Here we define a basic schema which tells MongooseJS the form that objects in the todos collection must take and define a Todo “model” which allows us to perform ORM-like operations on the todos collection. If you want to learn more about MongooseJS, I’ve written about it in much greater detail here and here.

11) Use the Todo model for database persistence and then fix our tests (diff)
Now that we have an object that lets us save and query a todos collection in MongoDB, we can make our GET / route query for all to-dos and our POST /todo.json route add a new to-do to the collection. Note that the function to save a new to-do can fail – if the posted object takes on an incorrect form (e.g. the “description” is empty), we’ll return an error object. For more information about handling this error object, you can read my post How to Easily Validate Any Form Ever Using AngularJS.

12) Tie up some loose ends: persist checking and un-checking, and reload data periodically (diff)
If you’ve been paying close attention, you may have noticed that up until now we missed a very important feature: we never actually update the backend when we mark a to-do as done. You would also be more observant than I was when I was outlining the steps for this post, because I missed that until I got to this step. However, like most of the tasks in this post, it is pretty trivial. We open up a PUT /todo/:id.json route that updates an existing to-do, create a function called “update” in TodoListController that uses $http to perform a PUT, and use the ng-change directive to call “update” every time the checked/unchecked status of a to-do changes.

Finally, we can add one more feature that I find tragically lacking in my personal to-do list app of choice, Todoist: periodic syncing with the server. I find it really frustrating to check off something as done on my laptop and then have to refresh the page on my Ubuntu box when I get back home. The simplest way to improve this is to add a dependency to AngularJS’s $timeout service and set it to ping the server for a full to-do list every 30 minutes.

[Corrected on 8/2/13 : Using $timeout causes the E2E test runner to hang until all outstanding calls to $timeout finish. This is a bug that the AngularJS team is currently working on. Until that’s fixed, I recommend avoiding using $timeout in favor of Javascript’s native setInterval (diff), or, better yet, your own custom AngularJS service that wraps setInterval. ]

Conclusion
Hey, guess what? You’ve just successfully built a to-do list app! How badass is that? What’s even better is that in some ways this fancy new app you built is in some ways more useful than commercial offerings like Todoist. [Note: To any Todoist team members who read this- I love many things about your software but hot damn there are some really annoying issues with it that I think could be improved]. Hope you found this little code adventure useful. By the way, I’ve been toying with the idea of building a more sophisticated to-do list/task management app in the near future that addresses a lot of the issues I’ve had working with Todoist, Wedoist, Asana, Jira, Github, Trello, Streak, etc. Would you guys be interested in something like this? Something along the lines of using Github to successfully coordinate tasks that aren’t actually code-related. What sort of features would you want to see in something like this? What annoys you about existing to-do or team management tools? Feel free to discuss in the comments section.

Have any questions about the code featured in this post? Want to suggest a better approach? Go ahead and leave a comment below, or tweet me at @code_barbarian. I’m always down for both unfettered praise and horribly vicious flame wars. Until then, feel free to rant about which task management apps you like and which make you want to feel like this in the comments. You know what’s better than a task management app? A guy like William Kelly (@idostartups). Major props to him as always for helping make this post happen.

Advertisements

Introduction to the MEAN Stack, Part One: Setting Up Your Tools

I’ve received several emails asking for instructions on how to set up a basic MEAN stack app. I’m going to take it one step further and give you guys a two-part post that will walk you through creating your first MEAN stack app- from installing the tools to actually writing the code. In Part One we’ll go through the setup and installation process. Next in Part Two we’ll walk through the steps for building a very simple to-do list. Part One consists of seven steps, although only the first two are are strictly necessary.

We’ll start by installing all of our tools. NodeJS and MongoDB are designed to be as close to operating system independent as possible and we will be covering three of the most common operating systems here – OSX, Windows 7/8, and Ubuntu. The one tool that you’ll need to get started is a bash shell (the standard linux command line). This tool takes different names on different operating systems, but they are all effectively the same for the purposes of this tutorial. If I use the term terminal, shell, or command line, I’m referring to a bash shell window. If you’re on a Mac or an Ubuntu machine, you’re looking for the Terminal. Windows doesn’t have one by default, but there are several alternatives. The git installer for Windows comes with “git bash,” which is the tool that I currently prefer.

If you’re on OSX, I highly recommend you install Brew to help with this process. Brew is a phenomenally useful tool for installing programs directly from your command line. For example, if you wanted to install git the hard way, you would have to open up your browser, google “git,” click on a few links, download an installer, and run it. With brew, you would simply open up the Terminal, type brew install git , hit enter, and you’re done. Brew will help a lot with setting up MongoDB and NodeJS.

1) Installing MongoDB-
First, we’re going to install MongoDB.
OSX: Open up your Terminal window and run

sudo brew install mongodb

Ubuntu: Open up your shell and run:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list
sudo apt-get update
sudo apt-get install mongodb-10gen

Windows: Go to http://www.mongodb.org/downloads and download the latest version of MongoDB for Windows, which should be a plain zip file. Once the download is done, extract the contents of the zip file to a memorable directory, such as ‘c:\mongodb’.

That’s it! After the installation process is done, you should be able to run the command

mongod

directly from your command line on Mac and Ubuntu. On Windows, you’ll need to run it as

/c/mongodb/bin/mongod

Think of mongod as your local MongoDB server- you need to have a mongod process running in order to be able to use MongoDB. You can now leave mongod running, open up another terminal, and run

mongo test

or

/c/mongodb/bin/mongo test

on Windows. This should open up the MongoDB shell. Hit ctrl-c to close the MongoDB shell.

2) Installing NodeJS and npm-
Next we’re going to install NodeJS and npm (node package manager).
Mac: Open your terminal and run

sudo brew install node
curl http://npmjs.org/install.sh | sh

(Instructions from http://madebyhoundstooth.com/blog/install-node-with-homebrew-on-os-x/)

Ubuntu:

sudo apt-get update
sudo apt-get install python-software-properties python g++ make
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs npm

Windows: Download the installer from http://nodejs.org/download/. I recommend using the installer rather than the binary to save yourself the extra work of adding the binary location to the system path.

Once you have successfully installed NodeJS, you should be able to run the “node” and “npm” commands from your terminal. When you run “node,” you should see a single “>.” This is the node shell, hit ctrl-c to close it.

When you run “npm” with no arguments, you should see a bunch of usage information. Keep in mind that “npm” often requires root permissions to run. If an npm command fails for unclear reasons, try running it through “sudo.”

At this point, you’ve done essentially all you need to do to run a MEAN stack application. You can simply clone/fork https://github.com/vkarpov15/mean-stack-skeleton , start a “mongod” process, navigate to the git repo, and run

npm install -d

You should then be able to run

node app.js

to start your server. However, I recommend at least reading through the rest of the below steps, as they will explain in a bit more detail some of the tools that you will be using. You can follow along step by step from the mean-stack-skeleton repo commit log, because, conveniently, the commits correspond one-to-one with steps 4-7.

3) Installing ExpressJS-
Now that we have MongoDB and NodeJS, it is time to install ExpressJS- this is pretty trivial now that we already have npm installed from step 2. In Mac, Linux, and Windows, you should open up a terminal and run

npm install express -g

The “-g” flag means that the package will be installed so you can run it from your terminal.

We’re using ExpressJS here because it adds some extremely useful tools for web development that NodeJS lacks. Contrary to what you may have heard, NodeJS is not a fully featured web server. NodeJS is simply a tool for doing I/O in an event-based concurrency framework with Javascript. It may be sufficient for a trivial server that prints “Hello, World” for every HTTP request, but it makes more sophisticated web development more difficult than it has to be. ExpressJS provides a familiar MVC (Model-View-Controller) framework for you to work with.

4) Creating an ExpressJS application-
Now that we have all of our server tools in place, let’s create an ExpressJS application. Start by running the command

express mytestapp
cd mytestapp
npm install

This should create a “mytestapp” folder in your current directory that will contain your application. You should be able to now run

node app.js

from the “mytestapp” folder. If you point your browser to http://localhost:3000, you should see a simple “Welcome to Express” screen.

The “mytestapp” directory will contain a few sub-directories: routes, views, public, and node_modules, as well as app.js and package.json files. Here’s a brief description of what these files and directories do:

  • app.js: The main point of entry into your web server. This file defines the port that your application listens for requests on, includes dependencies via the “require” function, and defines handler functions for different REST-ful paths your clients can access.
  • package.json: Defines internal dependencies for your application. Running “npm install -d” (which we will do shortly when we modify this file) installs all the dependencies listed in the “dependencies” file.
  • routes: The routes directory will contain Javascript handlers for REST-ful paths defined in app.js. For example, if you open index.js, you’ll see the handler for requests to the “/” path, which renders the “index” template that resides in “views/index.jade”.
  • views: The views directory will contain templates defined in the Jade language. Jade is a cleaner and more human-readable version of HTML with several extraordinarily useful features, such as inheritance and partials. Your routes will access these views via the “res.render” function that you saw in routes/index.js.
  • public: The public directory is usually used for images, client-side Javascript, and other static assets. ExpressJS will route requests that correspond to files under the public directory directly to the file. For example, if you run
    node app.js

    and point your browser to http://localhost:3000/stylesheets/style.css, you’ll see that Express returned the contents of the “public/stylesheets/style.css” file.

  • node_modules: The node_modules file contains source code for tools installed via npm. You can feel safe ignoring its contents for now.

5) Installing drivers and MongooseJS-
Now we’re going to install the MongoDB driver for NodeJS, as well as another useful tool called MongooseJS. MongoDB is organized with language-specific drivers that interface with the core database, so in order to use MongoDB with NodeJS, we need the NodeJS driver. MongooseJS is an ORM for NodeJS and MongoDB that is relatively similar to Ruby on Rails’ ActiveRecord. For more discussion of MongooseJS, see my original introductory post about the MEAN Stack, as well as the MongooseJS-specific follow-up Mistakes You’re Probably Making With MongooseJS, And How To Fix Them.

Open up the package.json file in “mytestapp” using your text editor of choice. I prefer SublimeText on Mac and Windows, and gedit or vim on Linux, but anything like Notepad or Textmate should work fine. The package.json file should contain a list of dependencies which looks like this:

"dependencies": {
  "express": "3.0.3",
  "jade": "*"
}

We’re going to add two more lines

"dependencies": {
  "express": "3.0.3",
  "jade": "*",
  "mongodb": ">= 0.9.6-7",
  "mongoose" : ">= 3.6"
}

Next run

npm install -d

If you’re on OSX or Ubuntu, you may have to run “sudo npm install -d” to get it working. Now that we have both the NodeJS MongoDB driver and MongoDB, we can create a connection to MongoDB via Mongoose in our app.js file:

var Mongoose = require('mongoose');
var db = Mongoose.createConnection('localhost', 'mytestapp');

You can see the diff for this step on github here. Run “node app.js” now –  your “mongod” process should output a message that looks something like “[initandlisten] connection accepted from 127.0.0.1,” which indicates that your app has successfully connected to your mongod process.

6) Use Bower for adding AngularJS-
Bower is another npm tool that we will use to add AngularJS to our app. It is essentially an npm for client-side Javascript. For example, if you want to add jQuery to your project, Bower enables you to do so from your terminal with the command “bower install jquery.” Install it with

npm install bower -g

from your terminal.

Bower has one minor quirk that you need to be aware of – it will install components into a “bower_components” directory by default, which unfortunately is not under the public directory. From your “mytestapp” directory, create a directory called “vendor” under “public/javascripts.” You can do this by running the command “mkdir public/javascripts/vendor.” Next, in the “mytestapp” directory, create a plain text file called “.bowerrc” that contains:

{ "directory" : "public/javascripts/vendor" }

This configuration file will tell Bower to install tools into the “public/javascripts/vendor” directory. You can see the diff for this step on github here.

7) Installing AngularJS-
Now lets install AngularJS v1.0.6 using “bower install angular#1.0.6.” You should see a public/javascripts/vendor/angular directory that contains “angular.js,” “angular.min.js,” and “bower.json” once you’re done.

To be continued
Congratulations, you’ve now officially completed Part One and set up the MEAN Stack on your machine! Part Two will be posted in a week, and in it we’ll go step-by-step for building a simple to-do list application using the MEAN Sack. In the MEANtime (pun very much intended), I encourage you to play around with these tools and see if you can build anything cool with them. Could you whip up a sweet little MEAN Stack application before I get a chance to drop the next part of the guide? I certainly think you guys can figure it out. It’s always great to hear about the awesome ways people are using the MEAN Stack, so if you decide to make something please go ahead and share it with everybody in the comments. Maybe it will turn into a real product and you’ll make billions of dollars and commission the construction of a solid gold yacht despite the insistence of your engineers that it will totally sink but you build it anyway because you’re so rich it doesn’t matter if it ends up on the bottom of the ocean, laws of physics be damned. Or maybe the other readers will just say “sweet app bro.” Either way, what are you waiting for?!

Want to learn more in advance of Part Two? Howtonode.org has a good tutorial on how to use the NodeJS MongoDB driver and ExpressJS to build a simple blogging platform at http://howtonode.org/express-mongodb. In a previous post I described how integrate MongooseJS validation and AngularJS, which will be useful for the next post. Thanks as always to my partner William Kelly (@idostartups) for his help writing this post, and for getting MEAN Stack shared across all of the internets.