Forbes Lindesay

Different people seem to go with vastly different release strategies for software. When and how often you release has knock on affects on how you test and how you develop or consider new features.

Assumptions

1. Releasing a new version of your software is fast and easy.
2. Downloading an updated version of your software is fast and easy.

Both these points should almost always be under your control. If they aren’t true, do everything in your power to make them true. If you’re going through an App Store, point 2 should be implicitly true. Point 1 is usually true in that submitting is pretty quick and easy. It might take a few weeks to actually get reviewed and published, but there’s nothing to stop you following up with a new release in the mean time: it’s a pipeline.

If you manage the whole software stack, make downloading updates fully transparent (think Google Chrome). If it’s a library, then use and rely on SemVer. Make uploading a new version a single command line message: foo publish v1.0.0

Bugs & Regressions

The problem with frequent releases is that it’s difficult to prevent bugs slipping into your software. You can help that with automated tests. A module is never finished unless it has a series of automated tests. I don’t care how big or small your software product is; running a manual test twice is doing it wrong (Golden Rule No. 1).

Some things are legitimately difficult to test. If you have a complex project run on many platforms in many different situations there will be bugs slipping through the cracks. People try and solve this by making the release cycles longer to allow for more time to test. The issue is, it still allows bugs through the cracks. Unless your software bugs might cost lives (or millions of dollars) people will forgive the occasional bug. In fact, they’ll forgive a lot of bugs, provided those bugs are fixed quickly.

Always fix bugs before writing new features (Golden Rule No. 2). This is important for multiple reasons. Nothing will annoy your users more than having something that doesn’t work remain broken while being told about exciting new features they’re not going to use.

Users will also normally forgive bugs if they know that releases typically come out every few days and always fix bugs before adding features. This brings me to my closing argument.

Conclusion

Someone is always waiting for your bug fix or new feature. If you wrote code this week, you should be planning to release this week (Golden Rule No. 3).

SemVer or Semantic Versioning is an important tool to help us build large modular systems. Most modern programming languages have a package manager of some description to go with them. This lets you install dependencies easilly, and it allows those dependencies to have their own dependencies and so on. The advantage of this is that modules can share code, which saves time and speeds up bug fixing.

For the Library Consumer

OK, so lets imagine your building a large web-app and you’ve got lots of dependencies because you know that’s better than re-writing or copy and paste programming. You have broadly 3 options for how you lock down your dependency versions:

1. Use * to always get the latest
2. Use an exact version number to always stay on the same version
3. Use some kind of range specifier.

Option 1 is bad news. If you use option 1 then every time someone changes their module, yours might break. It means you automatically always get the latest version and you don’t keep any information about what version your app was actually tested on.

Option 2 has two issues. Firstly there’s a false sense of security. You may have used exact specifiers for your dependencies, but the dependencies of your dependencies probably haven’t, so this doesn’t really fully protect you. The other problem is that you won’t automatically get bug fixes. You’ll only get bug fixes if you explicitly upgrade.

Option 3 is usually the best option. If you write your version specifier as ~1.0.0 you will get the latest version that’s been released and begins 1.0. This means that if the versions that have been released were:

• 0.9.0
• 1.0.0
• 1.0.5
• 1.1.0

You would get version 1.0.5. The key to this semantic versioning is that the first number represents large or breaking changes, the second version represents new features that might cause minor backwards incompatabilities, but probably wont. The third number represents bug fixes. As such, this specifier leads to you getting all bug fixes automatically, but forces you to manually update if you want new features.

For the Library Author

As a library author the rules are simple:

1. Increment the third number for bug fixes. Never introduce any breaking changes without incrementing another number.
2. Increment the second number for new features that don’t change the API. This should mean that 99% of users won’t see any breaking changes, but if someone’s relied on an odd corner case, or done something a little wierd, it might break.
3. Increment the first number every time there’s a major breaking change. If you expect most people will need to do a little bit of manual work to upgrade, then increment this number.

What could go wrong?

You might ask yourself, “Does it really matter that much?”. The answer is that it does. If you break things without incrementing the correct version number, then other software that relies on semantic versioning will break. If you frequently increment too high a number, your bug fixes will be slow to propagate and people will run into errors that have been fixed for ages.

Safety First

There’s one problem I keep seeing from people who normally follow SemVer. What happens when you realise your back room project has become hugely popular and that your hastily concoted API just isn’t cutting it anymore? You decide it’s time for a total redesign. You’re going to create a totally new version and everyone’s code will break unless they manually change things to be compatible with the new API.

When you’re faced with doing this, you have to be prepared. You will get users who get stung by the change. There will be people who used * as their version specifier when they depended on your library. These people have never learned about SemVer so they just feel cheated that you broke their module. Expect to have to patiently explain SemVer to a few people in a couple of GitHub issues. Perhaps link to this, or another, blog post.

When faced with this prospect, I see lots of library authors going “why not release it under a new name?”. This is what @mishoo decided to do with uglify-js. Instead of publishing it as uglify-js@2.0.0 he published it as uglify-js2. The long term problem with starting out this way, is that there are still a number of modules that depend on the uglify-js2 version, even though he did ultimately switch back to publishing as uglify-js@2.x.x. You might argue that this was a problem that could have been avoided if he just stuck to his original plan of calling it uglify-js2.

There are 2 key problems with this:

1. Discoverability
2. The future’s a whole lot longer than the present.

Discoverability is a problem because all the people using your old version may never find out about the new version. Tools like gemnasium and david won’t report the version as out of date, because it won’t know about the new version. This means people will miss out on awesome features. Perhaps worse though are the newcommers. They may find the old version and stop there. For example, a google search for uglifyjs displays the original github repository for uglify-js before the new one for uglify-js2. This is because @mishoo still hasn’t fixed the mistake of creating a whole new GitHub repo for the new version (rather than making use of branching). Many newcommers would just see that first result and use it. Even the ones who see the second result, might see that the first is more popular and assume that the second is some botched attempt at creating a better copy of UglifyJS and just use what appears to be the original, more popular, official version.

The second problem is that I envisage uglify-js being one of the most popular JavaScript minifiers for at least another couple of years. It’s fully possible that if we keep using JavaScript, and uglify-js stays up to date, it could end up being the most popular minifier for another 10 or 20 years. That’s a long time for us to live with adding a 2 to the end of the name. It might not seem like much, but everyone adding a 2 to the end forever more is a definite pain, and can definitely be avoided. The small pain of breaking a few apps/code briefly now vs. slowing down software development forever is, in my opinion, not a tough choice.

Teaching

It’s important to use SemVer, even if those around you sometimes get it wrong. If we don’t, then people won’t learn. If you’re writing a library, and you don’t let people get stung because they didn’t follow SemVer, then those people will never learn the valuable lesson about how to specify version numbers. Most of the people who get stung probably won’t be big professional developers, they’ll be people just starting to learn, and it’s better that they learn now than later.

If your consuming a library, you should use SemVer, and trust your unit tests (make sure you always write some). This way you’ll find out, and can teach the library authors about SemVer when they publish versions with breaking changes.

Every rule is made to be broken

I break this rule myself, for two reasons.

1. If I know a version is broken, I may use absolute version numbers to avoid the new version until it’s fixed. If I do this, I’ll always open an issue on the library author’s GitHub repo, or send them an e-mail if I can’t find the GitHub repo.
2. For my development dependencies I often just use *. This rarely results in breaking changes; It’s always me, or another library author, who discovers them, not an end user; and it’s a pain to always have to worry about updating them manually.

Other than this, if you see a library of mine that doesn’t use SemVer, it’s because I wrote it before I knew about SemVer.

Debugging a stack overflow exception is usually pretty simple, providing you have a stack trace. Just look for where the loop is and your sorted. That’s no help of course if you don’t have a stack trace. The gods of C# didn’t see fit to provide you with a stack trace when you get a Stack Overflow Exception.

The workaround is to somehow put a break-point just before stack overflow is triggered. This can be done with a conditional break-point. Start by creating a new break-point then right click on it:

Click Condition, and then in the box that appears, type something like:

(new StackTrace()).GetFrames().Length > 60

You may also need to add the following using directive to the top of your page:

using System.Diagnostics;

There you have it, that break-point will only be triggered if the stack depth is over 60. You can increase it if that doesn’t give you enough information. The maximum length (the one that causes a stack overflow) is pretty enormous.

Text files may seem like the most fanastic, ubiqutous and simple format. Almost all programmes seem to be able to cope with reading and writing them interchangeably, which is great. They aren’t quite a simple as you might think. Back in the dark ages of computing, when space was at a premium and even text files seemed large, most people in compuging spoke English, so they thought that a single Byte would be enough to store all the letters. This lead to a nice, efficient, simple encoding called ASCII.

What happened to just ASCII

It turned out that one byte doesn’t store enough characters (it only gives you 256 and that includes lowercase and uppercase letters and all punctuation). To fill the gap, hundreds of different formats sprang up. They were all slightly different. Many of these will work fine even if you interpret them as ASCII but some won’t.

This was no good, you had to try and guess the format of any text file before you could read or understand it. A sollution was needed that could match ASCII for small file size most of the time, but could also encode all the many characters of foreign cultures (and ancient ones).

Enter UTF8

A format was concieved that used one byte for the most commonly used symbols, but used multiple bytes when more were needed. This format was called UTF8. The 8 stands for the fact that 8 bits are used most of the time. One bit in that first byte is used to indicate whether or not the character is a two big character. If it is, a second byte will need to be read to determine what that character is.

UTF8 is simple, high performance, and can encode practically any character just fine. It’s by far the most popular, so the web is using it almost exclusively.

Almost

Yes, I said almost. Web browsers don’t default to UTF8 mode, they instead try and guess by default. They guess wrong, so you have to tell them. Always add the following to the head of any HTML file you ever write, and you’ll be fine:

<meta charset="utf8" />

If you forget to do that, or think you don’t need to use UTF8, bad things will happen to you. If you (or someone you know) is responsible for writing software which either reads or writes text files, tell them to do so in UTF8 by default, and only use other formats when explicitly told they have to.

The BOM

There is one little problem with using UTF8 (usually on windows). It’s called the BOM (Byte Order Mark). In order to deal with the difficulty of guessing the file type, some windows users of UTF8 decided to add a couple of special characters to the start of each document in order to indicate that the file was UTF8. Most text editors will understand these characters and not display them. Most other computer programmes will fail when they see them. If you get a couple of wierd looking characters at the start of your document, that’s why. If you’re building a text editor, please get rid of those characters.

Browserify now supports standalone builds thanks to integration with my umd (universal module definition) library. Universal module definition is a simple tool to help you (as a library author) deal with the fact that all your users are probably stuck working on legacy systems with legacy build managers (such as AMD).

The goal on your part is to write your code once, in the easiest module system, and have all your users get a copy that’s “optimised” for them. You want to have one file that will work in all the build systems out there. By using the umd library or the standalone option in browserify, you get just this.

Creating a Package

Consider an example package made of two files:

beep.js

var shout = require('./shout.js');
module.exports = function beep() {
  console.log(shout('beep'));
}

shout.js

module.exports = function shout(str) {
  return str.toUpperCase() + '!';
};

If you’re in the browserify/CommonJS world, you can just require('./beep.js')(); and 'BEEP!' will get logged. If you’re not though, you’ll want a standalone build:

Using the Command Line

$ browserify beep.js --standalone beep-boop > bundle.js

Using teh API

var fs = require('fs');
var browserify = require('browserify');
var b = browserify('./beep.js');
b.bundle({standalone: 'beep-boop'}).pipe(fs.createWriteStream(__dirname + '/bundle.js'));

You’ve now generated the bundle and can provide it to your users.

Consuming a bundle

Depending on the environment you’re in, there are a number of ways you can consume the package:

CommonJS

If you’re in a CommonJS like environment that’s just not quite fully browserify compatible (perhaps you’re using clever transforms), you can just requiure('bundle'):

app.js

require('bundle.js')();

Logs:

BEEP!

RequireJS

If you’re using RequireJS it works just as well:

index.html

<script src="require.js"></script>
<script src="app.js"></script>

app.js

require('bundle.js', function (beep) {
  beep();
});

Logs:

BEEP!

SES (Secure Ecma Script)

If you’re using Secure Ecma Script you can just call ses.makeBeepBoop:

ses.makeBeepBoop()();

Logs:

BEEP!

Global/Window

If you’re not using a module system at all, you can still access the package. If you’re in a browser it will be at window.beepBoop and if you’re in another environment it’ll be at global.beepBoop. You don’t need to be explicit about he window prefix though:

index.html

<script src="bundle.js"></script>
<script src="app.js"></script>

app.js

beepBoop();

Logs:

BEEP!

Bonus Feature

This UMD implementation is highly robust. It’s clever enough to prevent any UMD definitions inside files you’ve browserified from getting confused. They won’t see define, ses, bootstrap etc. so they’ll just use the CommonJS option (assuming they have one).

Extending

If we don’t support your chosen module system yet, I’m happy to extend things so that UMD does. Just submit a pull request for umd/template.js that adds support for your library and I’ll be sure to accept it (providing it’s not going to break any of the other sollutions.

I’ve spoken about package management in the client side JavaScript sphere before. I’ve extoled the virtues of component and broadly avoid mentioning browserify. A lot has happened since then though. Both have shiny new websites and browserify has hit version 2.

Why didn’t I talk about Browserify v1?

Browserify v1 has been around for a long time. It has an almost magical ability to take JavaScript code that was written to run on a server using node.js and make it work on the client. That’s exactly the issue though. It was a magical black box that seemed to do more and be more complex each time you looked at it. For a long time it was plagued with bugs when running on windows. It attempted to do all the things that should be seperate extensions internally in the one module.

Doing too much is a problem. If I rely on something I have no understanding of and it ceaces to exist, I’m completely stuck. Component, by contrast, is fantastically simple. You list where all the JavaScript files are and then it concatenates them into one long file with the standard require logic wrapped around them. It couldn’t be simpler.

This simplicity made it easy to extend. I was able to build a streaming version and a website which serves up fully compiled modules with ease. It was always easy to see what was going on and to debug.

How does Browserify v2 fix these issues?

Browserify v2 is made up of a collection of small modules. Each module is inherently streaming, which is perfect for building things like a website to let people download pre-compiled modules. All browserify does is handle the require side of things. It detects calls to require so it knows which files are needed, and then it concatenates them into one long file with the appropriate thin require logic wrappped around them.

It has clever fallback logic when it doesn’t find a module so that you can include multiple bundles where some bundles can depend on things in bundles which were loaded earlier. This means that browserify can make excellent use of CDNs and caching.

Creating a module

When creating a module, the main downside to component is fairly obvious. For every JavaScript file you create you have to add an entry to scripts in component.json. The other downside is that if you’re writing for both server side and client side you have to add both component.json and package.json with broadly very similar contents.

Both component and browserify allow you to support different code for client and server. Component does this by having an entirely seperate module definition file with its own main field. Browserify does it by adding a simple "browser" field wich overrides "main" when in the browser.

Publishing and Installing a Module

When publishing a module the main issue is finding a name for it. Component uses GitHub as it’s registry while browserify uses npm. npm has a flat namespace and you can name modules anything you want.

To publish you run:

npm publish

To install you run:

npm install my-module

And then in your code you load it using:

var mod = require('my-module');

This is great, providing nobody already has ‘my-module’ as a name. Having said that, if they do there are two ways it can still end well:

1. They might have already built a module that does what you want done
2. They might have built a module but then decided they don’t want it, and they may be willing to unpublish their module to make room for yours (this happened to me).

By contrast, GitHub enforces a strict two-tier namespace. The first level is your username and the second is your module name. You’re guaranteed to be able to get any module name you want, but you have to prefix it with your username.

To publish, just push to GitHub

To install you run:

component install ForbesLindesay/my-module

And then in your code you load it using:

var mod = require('my-module');

Notice that the username prefix is dropped once it’s installed. This works really well. It’s worth noting that you can still get this behaviour using npm. You can use npm to install anything from a GitHub repository providing it has a package.json file. If the package.json file has a name property set to my-module you can still do:

npm install ForbesLindesay/my-module

And then in your code you load it using:

var mod = require('my-module');

So in essence, npm is just as magic, but has the advantage of working for server side code as well

Finding Modules

One of the hardest thing with any new module system is finding the modules that do what you need them to. Ultimately many people just end up searching GitHub using Google. Once most people converge on one module system, this will likely become an acceptable solution, but at the moment it’s completely inadequate. Most of the libraries you will find will not work with your chosen package manager.

Both browserify and component do come with a dedicated search system. Neither come up to scratch though. Component’s is a new website, it’s very shiny, but doesn’t give you any information about the browser compatibility or quality of the components. It has a search in the bottom left corner, which is just a basic text based search. It’s difficult to browse through and for the moment I still prefer the old site at https://component.jit.su/ which at least makes the licence very clear and categorises the modules.

Browserify’s is initially just a blank page with a search box, somewhat un-inspiring as a starting point. when you start typing you get a simple list of modules and descriptions. This works fairly well but again lacks info on licenses. It does show test results for browser compatability when available from http://ci.testling.com and I’d really like to talk to them at some point about adding support for https://jepso-ci.com

What both of them currently seem to lack is any sense of the fun that can be had browsing really cool modules by category. There are awesome things out there, and sometimes seeing something you hadn’t thought of can spark and idea for something else that would be cool to build.

Testing

Disclaimer I built https://jepso-ci.com so I’m biased

It’s curicial when your writing code for the browser that you have unit tests which can be automatically run across all the browsers you want to support. https://jepso-ci.com can run any tests that will run in a real web-browser so it’s great for testing both component and browserify based code. At the moment jepso-ci does not have support for automating the build step for browserify, but this will be comming soon. You can track the progress in this issue

http://ci.testling.com is currently the only other major cross browser continuous integration platform. It’s built by the same people who built browserify, so naturally the two work well together. It has no support for easilly running the same tests in a real local web browser though, and doesn’t seem to provide any information about the html of the page the tests will be run on.

Style Sheets etc.

Component aims to view modules as more than just JavaScript (hence the name) and includes CSS and other files. In the long term this is definitely something the world will need. There’s a difficulty for now though in that CSS doesn’t have any nice systems for modularity. It’s one global namespace and it’s really ugly. It has no variables, no parameters, no opportunities for configuration. As such I would suggest that component’s CSS support is essentially degenerate and insufficient.

Browserify views modules exactly like server side JavaScript does. They’re some JavaScript code, joined together with require calls. You can easilly add a build step that converts a text file into JavaScript, and CSS just isn’t ready for a package manager yet anyway.

This process sucks there should be a better way of doing it, but here are the steps to permanently add an environment variable on Windows.

Step 1

Right click on your computer in the file browser and select properties:

Step 2

Click “Advanced system settings” in the left hand side bar

Step 3

Click “Environment Variables”

Step 4

Click new. For most things you’ll want “User variables for Username” and not “System variables”

Step 5

Enter a name and value and click OK

Step 6

Are you prepared for the worst part? Now you have to restart your computer…

Temporarily Setting Environment Variables

Sometimes you don’t need the value to persist:

If you’re in powershell

SET:

$env:VARIABLE_NAME = 'variable value'

GET:

$env:VARIABLE_NAME

If you’re in a standard command prompt

SET:

SET VARIABLE_NAME = variable value

GET:

SET VARIABLE_NAME

When it comes to asyncronous operations in JavaScript there’s only one sane way to handle them, and that’s promises. The reason for this is that they’re composible:

all(asyncThingA(), asyncThingB())
  .spread(function (resultA, resultB) {

  });

The all function and the spread function are both really easy to build, and the above code simply says wait for both async operations to complete, then give me them. Writing the same thing using code based off event emitters or callbacks is a complete nightmare (I’m not even going to go into how you’d code that).

Promises are great for lots of things then, but there’s oen thing they haven’t yet got sorted out…progress. How on the earth should we handle progress?

Progress in event emitters

Event emitters have the upper hand when it comes to dealing with progress. They can just emit a progress event which you can handle.

function doAsyncOperation() {
  var emitter = new Emitter();
  var progress = 0;
  setInterval(function () {
    if (progress < 1)
      emitter.emit('progress', progress);
    progress += 0.1
  }, 100);
  setTimeout(function () {
    emitter.emit('end');
  }, 1000)
}

var op = doAsyncOperation();
op.on('progress', functoon (p) {
  console.log('Operation is ' + (p * 100) + '% done.');
});
op.on('end', function () {
  console.log('Operation complete');
});

This actually works great. It’s simple, easy to understand, and does what you need. There are two problems with adopting the same approach in promises:

• Errors
• Composition

Errors

What happens when an exception is thrown in a progress handler?

Just throw the error

With event emitters it will simply throw and bring down your application (browsers will carry on despite the error, but node.js apps won’t, and neither will windows 8 apps). That seems fine if you come from a world of event emitters, but jarring if your used to the promises behaviour of forwarding the error:

makePromise()
  .then(funciton success(res) {
    throw new Error('This error gets caught');
  },
  null,
  function progress(p) {
    throw new Error('This error crashes your app');
  })
  .then(null, function errorHandler(err) {
    //handle error
  })
  .done();

The thing here is that it’s not as simple as saying “Never throw in progress handlers” because progress handlers aren’t really critical bits of code. You don’t really need them for your application. They might never get fired because your async operation might not support progress. Then you re-factor and decide to make the async operation support progress and your application crashes.

Ignore the error

You could just ignore the error. We’ve already said that progress handlers aren’t that critical to your application, so why bother notifying anyone of the exception? To answer that, imagine you’re developing an application for sharing images you have the perfect UX except for the fact that the progress bar doesn’t move. You spend hours trying to figure out what’s going wrong, only to find an error being silently ignored coming from your progress handler that would’ve made the problem obvious and easy to fix.

Never silently ignore an error unless it’s deliberate and acompanied by a note along the lines of “This error doesn’t matter because …”

Remaining options

So the remaining options are:

1. Log the error, without throwing an exception
2. Make the exception handleable as part of the promise
3. Accept the problems with just throwing and just throw

None of these are completely un-acceptable, but none of them are ideal.

Composition

Parallel composition

What happens to progress in the example at the start of this post? Imagine the following:

AsyncThingA emits:

10ms Progress: 25%
20ms Progress: 50%
30ms Progress: 75%
40ms Progress: 100%

AsyncThingB emits:

30ms  Progress: 25%
60ms  Progress: 50%
90ms  Progress: 75%
120ms Progress: 100%

If we just emit both sets of events we might get:

10ms  Progress: 25%
20ms  Progress: 50%
30ms  Progress: 75%
30ms  Progress: 25%
40ms  Progress: 100%
60ms  Progress: 50%
90ms  Progress: 75%
120ms Progress: 100%

That’s not going to lead to a nice smooth progress bar. What if we take the average?

10ms  Progress: 12.5%
20ms  Progress: 25%
30ms  Progress: 37.5%
30ms  Progress: 50%
40ms  Progress: 62.5%
60ms  Progress: 75%
90ms  Progress: 87.5%
120ms Progress: 100%

That’s more like it, it seems we have a winner. We have a problem if our progress isn’t represented as a percentage. Consider what would happen if we had:

10ms Progress: Stage A complete
20ms Progress: Stage B complete

Clearly the previous approach won’t work. So from that we need to either:

1. Restrict progress values to a number between 0 and 1
2. Support customised progress composition

Serial composition

What about when our promises are in serial rather than parallel:

doAsyncThingA()
  .then(function (res) {
    return doAsyncThingB(res);
  })

Consider the functions are as they were in the parallel execution:

AsyncThingA emits:

10ms Progress: 25%
20ms Progress: 50%
30ms Progress: 75%
40ms Progress: 100%

AsyncThingB emits:

30ms  Progress: 25%
60ms  Progress: 50%
90ms  Progress: 75%
120ms Progress: 100%

If we just pass all progress events straight through we might get:

10ms  Progress: 25%
20ms  Progress: 50%
30ms  Progress: 75%
40ms  Progress: 100%
70ms  Progress: 25%
100ms Progress: 50%
130ms Progress: 75%
160ms Progress: 100%

That’s better than the parallel progress we had, but still pretty useless. In the same veign as before, we could assume they take roughly equal time and get the following:

10ms  Progress: 12.5%
20ms  Progress: 25%
30ms  Progress: 37.5%
40ms  Progress: 50%
70ms  Progress: 62.5%
100ms Progress: 75%
130ms Progress: 87.5%
160ms Progress: 100%

It’s not going to be completely smooth, but it’s pretty good as an aproximation. Imagine we replace AsyncThingB with as syncronous operation of virtually 0 time. Our previous aproximation now yields:

10ms  Progress: 12.5%
20ms  Progress: 25%
30ms  Progress: 37.5%
40ms  Progress: 50%
40ms  Progress: 100%

This is actually typically true of quite a lot of the operations in a promise chain.

A solution

This problem really needs solving, fast. Parallel composition is not too urgent, because there’s nothing else that doesn’t completely suck for parallel composition anyway. Serial composition needs to get fixed though. One option:

Take a progress-transformer as the third argument to .then(cb, eb, progressTransformer). progress-transformer can be one of three things:

• false - the value false indicates that progress propogation should stop at this point.
• a function - The progress values from the previous promise are passed to this function, propogation can be stopped by returning false, making false an invalid value for progress.
• Anything else - Anything else is also acceptable, and results in progress being propogated (effectively assuming that the current operation will be syncronous).

doAsyncOperationA()
  .then(function (res) {
    return doAsyncOperationB(res);
  }, null, divideByTwo);
function divideByTwo(v) {
  return v / 2;
}

Only the progress from operationA will be passed through the transformation, so the resulting promise yields:

10ms  Progress: 12.5%
20ms  Progress: 25%
30ms  Progress: 37.5%
40ms  Progress: 50%
70ms  Progress: 62.5%
100ms Progress: 75%
130ms Progress: 87.5%
160ms Progress: 100%

Which is the best result we came up with for serial composition.

doAsyncOperationA()
  .then(function (res) {
    return doSyncOperationB(res);
  });

Yields:

10ms Progress: 25%
20ms Progress: 50%
30ms Progress: 75%
40ms Progress: 100%

Which is perfect because it’s just the progress of A which is the same as the progress of the whole because B is sycronous.

A Solution to Parallel Composition

Parallel composition is a tricky one, I’d suggest that the all function will have to do some extra work. If we let progress be any value, then the extra work should probably just be agregating all the progress into a big array. If we force progress to be a number, we should probably just take the average of that number.

API

The behaviour we have specified above should help to inform our decisions regarding the progress API. Having a progress handler as the third argument of .then strongly implies that errors should be forwarded down the promise chain. Having it as a separate ‘event’ like structure would suggest it should just throw.

Option A

Make the third argument of .then be the progress handler. Forward exceptions down the promise chain. Propogate all progress events if the argument is not a function and not false. If the argument is false, don’t propogate anything.

Option B

Separate out propagation compltely from handling. Essentially this lets us take the event-emitter aproach to handling progress.

A promise supports progress if it has a .progress function.

If .progress is called with an argument of type function then it registers that function to recieve progress updates.

If .progress is called with anything other than a function then all registered functions are called with that value.

If .progress(fn) is called after .progress(val) then fn is called in the next tick with the most recent value.

Passing true as the third argument of .then triggers basic propagation, but all other propagation must be done manually. The following two bits of code would therefore be equivalent:

var p2 = p1.then(cb, eb, true);

var p2 = p1.then(cb, eb);
p1.progress(p2.progress.bind(p2));

Conclusion

I’m actually in favour of Option B. It’s simple to impliment, and I feel that it probably covers the 80% case. By letting people manually write to the progress of a promise, they can artificially do whatever they want with progress, but we cover the case of one async operation that’s followed by a sync operation by just passing true as the third parameter.

I’ve had a need to create in-line editing components for a project I’ve been working on lately. The required behaviour is simple. There are areas of the page that are editable. When you click on them they turn into a control that lets you edit them, and when you click away, the results are saved.

I came across jEditable which seems to do the trick, but I really don’t like the style it’s built in. It’s tightly integrated with jQuery in terms of its api. That could easily be fixed with a simple wrapper though. The more problematic aspect of it though is that it seems highly opinionated about how you update the server etc. I set out to create something simpler and smaller.

I’ve created a github organisation called componitable to store these little inline editable widgets. If you want to help the development effort, talk to me about getting involved.

Once installed with component, you can use them simply:

var text = require('editable-text');
text('[data-editable="text"]', {maintainSize: true});

Would take all elements that have a data-attribute of data-editable="text" and make them click-to edit text-boxes. The maintain-size option makes the text box the same size as the text that is being edited. That’s worth doing if you’re editing text in a table and need it to not re-size the table all over the place.

Editable Dates

Because I’ve tried to keep the library un-opinionated, the default formatting of date is just one of the toString methods, but I let you specify your own formatting however you want, e.g.

var date = require('editable-date');
date('[data-editable="date"]', {
  maintainSize: true,
  displayFormat: function (date) {
    var months = ['January', 'February', 'March', 'April', 'May', 
      'June', 'July', 'August', 'September', 'October', 'November', 'December'];
    var special = {
      1: 'st',
      2: 'nd',
      3: 'rd',
      21: 'st',
      22: 'nd',
      23: 'rd',
      31: 'st'
    };
    if (date) {
      return date.getDate() + (special[date.getDate()] || 'th') + ' ' + months[date.getMonth()] + ' ' + date.getFullYear();
    } else {
      return '';
    }
  },
  parseDate: function (dateStr) {
    if (!dateStr) return null;
    else return new Date(dateStr.replace(/(\d)[a-z]{2}/g, '$1'));
  }
});

Show me the money

Formatting for number fields is a little complex, so you might want to use a library like format-number which formats them neatly:

var formatNo = require('format-number');
var number = require('editable-number');
number('[data-editable="money2"]', {maintainSize: true, format: formatNo({prefix: '£', padRight: 2, truncate: 2})});

Updating the server

I’ve tried to stay un-opinionated about how you update the server, but I’ve given you some tools to help you do it in a sane way. The editable component will emit an update event with a value whenever a field is updated (regardless of what type the field is). Here’s an example of how I handle updates in my app:

var editable = require('editable');

editable.on('update', function (element, value) {
  var path = editable.attribute(element, 'path');
  var field = editable.attribute(element, 'field');
  var data = {field: field, value: value};
  post(path, data, function (err) {
    if (err) {
      alert('Error updating database.');
      throw err;
    }
    if (editable.attribute(element, 'requires-refresh') == 'true') {
      location.reload();
    }
  });
});

That lets me set data-attributes for path, field and requires-refresh anywhere up the tree of parent items and have that information used when updating the server.

Conclusion

This set of libraries are very much in a state of flux at the moment (none of them even have proper version numbers yet). If you use them, it’s up to you to test them thoroughly, but I thought I’d give the world a peek, so hopefully anyone else trying to solve the same problems can see how I’ve done it.

Embedded JavaScript (hereafter referred to simply as EJS) is a templating engine, much like moustache, which I covered in an earlier post. At first glance, this might seem like a painfully verbose syntax by comparison to the exceptionally terse moustache, but I’d like to argue that it more than makes up for it in power and ease of use. I’d then like to conclude by talking about asynchronous templates in their various forms.

So what is it?

Broadly speaking, it does exactly what it says on the tin. Take an html document which has some JavaScript embedded in it and then turn it into a template. Unlike moustache, EJS is a two stage process. First, you compile your template, then you provide it with some local variables and execute it. Where the power comes, is that this isn’t limited to some simple constrained sub-set of JavaScript, it’s literally any (synchronous) JavaScript. That means you can write a for loop as follows:

<ul>
<% items.forEach(function(item){ %>
    <li><%= item.name %></li>
<% } %>
</ul>

If that looked strange and unfamiliar to you, fear not. EJS lets you use ANY JavaScript, so you can do your loop in any way you feel comfortable with, perhaps you’d prefer:

<ul>
<% for(var i = 0; i<items.length; i++){ %>
    <li><%= items[i].name %></li>
<% } %>
</ul>

Both those templates would print out the exact same html, and have very similar performance characteristics, so you really can use whatever you’re most comfortable with.

If Then Else

Just in case you’re not getting it yet, here’s an example using if and else to display a login/logout button.

<% /* Comments work as you would expect */ %>
<button id="toggleLogin">
<% if(user.isLoggedIn) { %>
  Log Out
<% } else { %>
  Log In
<% } %>
</button>

But I don’t trust users!!

Hmm, if you’ve been thinking really carefully, you’ll have noticed that I outputted text from a javascript value directly into my html in my for loops. Depending on just where that value came from, I might want to be real careful doing that. To help us, EJS offers two ways of outputting a value from javascript.

Unsanatised, Dangerous, only do this if you know where this variable has come from and you’re CERTAIN it’s safe:

<%- text %>

If you don’t need to be able to output html, you should always sanitise your outputs. This also has the advantage that symbols like greater than or less than will render as expected:

<%= text %>

A word about semi-colons and line comments

Because of the way EJS is compiled, you’ll have a much more reliable system if you use block comments /* comment */ instead of line comments //comment.

EJS may also do weird stuff with semi-colons, so it’s best to omit them where they’re optional (which is most places in JavaScript).

You should always be explicit about blocks in EJS too.

Good:

<% if (true) { %>foo<% } %>

Bad:

<% if (true)  %>foo

Go make cool stuff

So, that’s pretty much all there is to it. It’s a fantastic bit of kit that I’m sure you’ll absolutely love and it’s available both for the client side http://embeddedjs.com/ and as an npm module for the server (npm install ejs) if you’re using node.js (which you should be as it’s great).