jsmake: a simple javascript preprocessor in perl

In the previous post I listed some javascript preprocessors you could find out there...
But could not find any written in perl...
And could not find a preprocessor that could handle package dependencies...
Therefore,I've taken a day to hack a command line utility to use as a basic preprocessor for javascript.
Only tested on my mac, it's a dirty, huge perl file but it might help some people out there until there's a real project to build a javascript preprocessor in perl.

Functionality

This utility can:

- compress the javascript files with 3 different algorithms:

Javascript::Mini
Javascript::Packer
Google online closure application

- load and merge files in order of dependency:

jsmake follows package rules used by perl or actionscript language.
It transpose the package name into a folder tree. Therefore:

Core.Util.Array

is understood as Core/Util/Array.js.

You can specify the dependeny at the very first line of your javascript file in a comment as follow:

/*!require: Core.Util.String,Core.Util.Queue */

jsmake will load all the dependencies in all the files in the right order.

- parse the javascript source file for some directives.

ifdef, ifndef, end, define are understood:

/*@ifdef DEBUG */
console.log("debug mode");
/*@end */


/*@ifndef DEBUG */
console.log("not in debug mode");
/*@end */

or if you like # better:


/*#ifndef DEBUG */
console.log("not in debug mode");
/*#end */

/*@define VERSION 1.06 */

How to use

on the terminal, just type:

$ jsmake

if you do not need any options just make the file executable.

There are obviously command line options:

input

$ jsmake -input ../lib

This tells where to look for the javascript to be merged/preprocessed/smashed.

output

$ jsmake -output ../src/merged.js

This tells where to output the final merged output.

compression

$ jsmake -compression name


Example:
$ jsmake -compression mini

This tells what kind of compression it should use.
There are 3 different compressors for now:

• packer
• mini
• closure

Closure use google online closure service with the least aggressive algorithm.
So... you just need to be connected to the internet to use this one.
packer and mini requires perl modules to be installed, respectively JavaScript::Packer and JavaScript::Minifier.

Using the compression will output 2 files:
the one uncompressed and the one compressed prefixed with the name of the compression type, so it might be output.js and closure-output.js

packages

$ jsmake -packages name separated by a space

Example:
$ jsmake -packages Core.DOM Core.Util.String

If you do not specify a package jsmake will merge all the files that contain the !require: directive.
By specifying the package name,jsmake will only merge the file necessary for this package to work as defined in your dependencies.

vars

$ jsmake -vars name separated by a space

Example:
$ jsmake -vars DEBUG STRICT_MODE

By specifying the name of some variables, you will make them come true.
Therefore all the ifdef or ifndef will be stripped down accordingly.
if you want a STRICT_MODE but no DEBUG messages, you will write:
$ jsmake -vars STRICT_MODE

define

$ jsmake -define name=value variable name=value

Example:
$ jsmake -define VERSION=6.01 AUTHOR=myself

vars allows you to overwrite the one inline in your javascript file.
So if you define in your javascript /#define VERSION 3.01 / but at the command line
set the variable to 6.01, the final output will be 6.01
Will look into merging vars and define functionality into one as it is confusing...

That's it!!

so if we go for a full option set up, you can get:

$ jsmake -input ../lib -output ../src/merged.js -packages Core.DOM Core.Util.String -vars DEBUG -define VERSION=6.12 AUTHOR=myself -compression mini

This will create two files merged.js and mini-merged.js compressed, cleaned out and with all the dependencies.

Download & Doc

http://code.google.com/p/jsmake-preprocessor/

Hope this is useful too somebody and that you won't be too hard on my perl capacities. Comment to improve the code welcome!

Why bother with javascript preprocessors you say?

Javascript is a dynamic language that offers many features to create rich online applications and is now used as a key language in heavy ajax based online applications.
Although enhanced by profiling, debugging tools like Firebug for Firefox, javascript developing in itself still miss features offered since a long time by other static languages.One of this feature being a preprocessor syntax that could ease development.

A preprocessor will allow you to add extra code (like debug statements, conditional flow) that will be stripped down before going live. Some will also give you the opportunity to compress/obfuscate the final output or even merge multiple files into one or why not create different built for different environment (why should you have IE related branching code that might increase file size, slow down the UI experience when you are developing solely for an iphone/ipad version of a website?).

So that you can grab the concept of a preprocessor for those of you who did not use any, here is a short example:

/*@define VERSION 1.1 */

/*@require Base.js,Math.js */

/*@ifdef DEBUG */
console.log("debug mode. Version:"+/*@=VERSION */);
/*@end */

function sum(num1,num2) {

/*@ifdef STRICT_MODE */
        if(num1===undefined || num2===undefined){
              throw new Error("the function requires 2 parameters");
       }
/*@end */

/*@ifdef DEBUG */
      console.log(num1+num2);
/*@end */

      return num1+num2;
}
This may sound rather hard to read but you will be able to create different builts.

The "require" directive will merge the two files listed into this one.
When you call the preprocessor of your choice on the command line let's say, just give it some options:

jspreprocessor -vars DEBUG -output merged.js -compression mini

You will get a merged.js file containing the following output:

//..contents of the js files required goes here
console.log("debug mode. Version:"+1.1);
function sum(num1,num2) {
     console.log(num1+num2);
     return num1+num2;
}

but compressed on top of that!!

The preprocessor stripped down the javascript to minimize overhead and unnecessary code lines (lightweight javascript file).

Do you see how a preprocessor could come in handy??


Hopefully, below is a list of preprocessors that exist and could help you in your future development if you do not already use them.

Javascript Preprocessor

language:javascript

http://www.bramstein.com/projects/preprocess/

A preprocessor combine with Apache Ant or Rhino Javascript Engine or simply call from within javascript!

jscompiler

language:none (online)

http://jscompiler.org/usage.php 

an online preprocessor using java like annotations.

JS Built Tools

language:Java/xml (Ant suit)

http://code.google.com/p/js-build-tools/

a collection of Ant tasks that contains a preprocessor, a compressor (YUI) and a doc generation based on MoxieDoc.

Jsmacro

language:python

http://github.com/smartt/jsmacro

a preprocessor following a syntax closed to C.

Sprocket

language:ruby

http://github.com/sstephenson/sprockets

a preprocessor written in ruby mainly intended to join multiple files into one through "require" and "provide" directives. work well with Pdoc.

Juicer

language:ruby

http://github.com/cjohansen/juicer

Focus on combining js and css files, compressing them with different tools.

Js preprocessor

language:ruby

http://js-preprocessor.com/

Focus on combining js and css files, compressing them with different tools.

-----
I certainly forgot some preprocessor out there but don't hesitate to comment about them and give feedback on the one listed above if you use them!