Introducing grunt-each

A couple weeks ago I published version 1.0.0 of grunt-each, a plugin for grunt that runs arbitrary actions on file contents and writes the results to a destination. As with (almost) all of my modules, I wrote it because I had a use case that no existing module (that I could find) would solve. Necessity is the mother of invention, as they say.

Why do you need grunt-each?

There are tons (literally 1,722 at the time of this writing) grunt plugins published on npm that solve hundreds of different tasks. What use case could possibly exist that isn't covered by one of them? And why not publish a more specific module that solves that one task, rather than something generic?

We are in the process of trying to reduce the size of our angular-based javascript bundle and have considered using webpack to create page-specific bundles, rather than one large bundle. This requires converting non-dependency-tree style javascript into a dependency tree, and ideally each piece (controller, directive, template, etc.) will require whatever other pieces it needs. The problem with this is that we write our templates as plain html, and then process them with grunt-angular-templates to create a javascript file that caches our templates when the app bootstraps (not uncommon . . . this is pretty widely agreed to be best practice). So there's not a simple way to do something like this:

require('templates/some-dependent-template');  
require('controllers/some-dependent-controller');  
<div ng-controller="SomeDependentController">  
  <div ng-include="'some-dependent-template'"></div>
</div>  

Not only is it weird and non-semantic, but also it (probably) will make syntax highlighting not great (since you're writing javascript in an html file, but not in a script tag). Furthermore, when grunt-angular-templates processes the files, the end result would be something like this:

angular.module('app').run(['$templateCache', function($templateCache) {  
  $templateCache.set('my-template', 
      "require('templates/some-dependent-template');"
    + "require('controllers/some-dependent-controller');"
    + "<div ng-controller=\"SomeDependentController\">"
    + "  <div ng-include=\"'some-dependent-template'\"></div>"
    + "</div>"
  );
}]);

Obviously, that's not going to work. Of course, there is a webpack loader for angular template caching, but it seems to be geared toward templates required by directives where requiring other files is not a problem (because they are javascript). It's unclear from the examples (to me anyway) how you would require a template from another template (or really, require anything from a template). Maybe that's because that kind of thing is frowned upon. If you're doing strictly componentized angular, maybe you'd never have this problem. At any rate . . . we do. What we need is a way to grab the contents of each file and pull any requires outside the angular.module piece that grunt-angular-templates creates. We need a way to loop over a set of files and perform arbitrary actions on the contents.

That's the answer to the first question above. The answer to the second is "Why not make it generic?" There's a gulp-foreach plugin and a gulp-each plugin that do comparable things for gulp builds, so I was surprised to find that neither grunt-foreach nor grunt-each was taken on npm. And while there are plenty of specific plugins, I imagine there will be enough people interested in a generic solution to make this worth while.

So how does it work?

Grunt-each accepts any of the standard grunt file formats, but instead of doing something with the file contents like most plugins, it passes that content to an action (or multiple actions) you specify, allowing you to do whatever you need to do with that content. There's some flexibility built in to this. Actions can be synchronous or asynchronous, they can be a single literal function, an array of functions, or even a string or array of strings. When an action is a string, grunt-each will try to require the string. This is significant for a few reasons:

  1. It allows actions to be published on npm. I could publish our "require things inside angular templates" action and then other people could install it and use it in their projects easily.
  2. It allows complicated actions that would take up a lot of space to be abstracted into external files. Gruntfile.js is, theoretically, a configuration file, so hundreds of lines of file transformation logic may not belong there.
  3. It allows actions to be abstracted to other files where they can be reused. Possibly, this won't be that significant. At least in our case, I don't foresee us needing to process html in the same way anywhere else in the code base. But if you needed to, you would be able to. I try never to make assumptions about how my modules will be used. Developers have lots of great ideas that I can't anticipate.
  4. It makes them easier to test in isolation. I don't typically write tests for ad hoc grunt tasks, but you could, and it would be a lot easier to require the action separately and control what's passed to it than to try to do it through grunt.

You can also mix literal functions and strings as necessary. Synchronous actions accept a file parameter that has the properties contents, name, and origContents (more about origContents later) and return the modified contents. Asynchronous actions accept the same file object as well as a callback. The callback accepts an optional error and the modified file contents.

How do I use it?

I'm making the assumption here that you already have a Gruntfile and run grunt tasks. If you don't, you may want to start here.

Step 1

npm install --save-dev grunt-each

Step 2

Add an each section to your Gruntfile. each is a multitask, so configure as many targets as you need and specify files on which to operate in any of the normal grunt formats.

// Gruntfile.js
module.exports = function(grunt) {  
  grunt.loadNpmTasks('grunt-each');

  grunt.initConfig({
    each: {
      dev: {
        files: [{
          expand: true,
          cwd: 'app/templates/',
          src: '**/*.html',
          dest: 'assets/',
          ext: '.js'
        }]
      }
    }
  });
};

Step 3

Write actions that do what you need and add them to options.actions (as described above). Multiple actions are called in succession with each action receiving the modified content of the previous as the content property (note that this is not guaranteed to be a string, if you return something other than a string from an action). The origContent property will always have the original file content, regardless of modifications. Probably you won't need this, but it's there if you do.

// Gruntfile.js
module.exports = function(grunt) {  
  grunt.loadNpmTasks('grunt-each');

  grunt.initConfig({
    each: {
      dev: {
        files: [{
          expand: true,
          cwd: 'app/templates/',
          src: '**/*.html',
          dest: 'assets/',
          ext: '.js'
        }],
        options: {
          actions: [
            // This action let's you write something like
            // <script>
            //   require('controllers/foo');
            // </script>
            // at the top of an html file
            function(file) {
              var pieces = file.split('</script>\n');
              var dependencies = pieces[0].split('\n').slice(1).join('\n');
              return {
                dependencies: dependencies,
                html: pieces[1]
              }
            },
            // It's not strictly necessary to write this
            // as two actions, or to do it asynchronously;
            // I'm doing it to illustrate how it works.
            function(file, cb) {
              var script = `angular.module('app').run(['$templateCache', function($templateCache) {
                $templateCache.put('${file.name}', "${file.contents.html}");
              }]);`;
              cb(null, [file.contents.dependencies, script].join('\n\n'));
            }
          ]
        }
      }
    }
  });
};

Now when I run grunt each:dev, all of my templates will be run through these two functions and the final result written to assets/. Basically grunt-each handles the file IO for you but leaves the specific file transformations up to you, meaning you can literally do anything you want with files in your code base without needing to search for (or write) a specific plugin. Note that I'm not even using grunt-angular-templates anymore in this example.

If there's a specific plugin that achieves whatever thing you need, it's probably better to use that, as you'll get more useful support for issues you face, and it will be optimized (theoretically) for that use case. But if, like me, you find yourself facing a problem for which there is no existing grunt plugin, and you just need a generic solution for modifying files, look no further than grunt-each.

comments powered by Disqus