Monday, July 7, 2014

A Memory Leak in IE-9

I recently read David Glasser's blog-post "A surprising JavaScript memory leak found at Meteor" about a JavaScript memory-leak in Chrome. That blog-post is about a year old so I wanted to check if things might have improved since then.  I tried a simplified version of his example in IE-9 and surprisingly could reproduce the leak with even fewer statements.

Here's my modified, simplified version of David's example:
 
  var variableOfOuterScope = null;

 function runManyTimes() 

 {var previousValue    = variableOfOuterScope ;
  variableOfOuterScope 
  { bigArray:       new Array (2000000).join('*')
  , aRefToAFunction: function () {}
  };
  // previousValue = null;
  // Un-commenting the above line will fix the leak.
 };
  
 for (var i=0; i < 30 ; i++)
 { runManyTimes () ;  
 }
 // See in Windows Task Manager how the memory used  
 // by IE9 has grown significantly, to about 240 mb
 // after running the code above.


After running the code it seems the browser now uses about 240 mb instead of the 24 mb it used before running it. This is not good, memory is bursting at the seams! Why does it  happen? Why does the memory leak?  I assume it goes something like this:

When you call runManyTimes for the first time it creates a new "context" which the anonymous function inside it holds on to. This context refers to variables that exist outside the anonymous function, so that if it wanted to, it could refer to their values which were in effect at the time runManyTimes exited.  This context refers to the variable 'previousValue' whose value when runManyTimes returns first time is null. But the value of variableOfOuterScope now contains a very big Array. By the time you have called it the 2nd time,  previousValue points to that very big Array and  variableOfOuterScope then contains yet another big Array.

Each time you call runManyTimes  this chain of very big Arrays gets one bigger.  But if you un-comment the fix-line and set previousValue to null, this chain is broken, and  garbage gets collected.

One more thing is required to cause the leak to happen: The  variableOfOuterScope  must contain a REFERENCE to the anonymous function. If you just created a simple inner function without referring to it, there is no leak. Saving the anonymous function to a field in  variableOfOuterScope  means each of its values refers to the anonymous function (created on a specific call  of runManyTimes) which holds the context which refers to the previous value of  variableOfOuterScope and so on. Without that the variableOfOuterScope  would not refer to the function which holds on to the context, which refers to the previous value, which refers to the previous value and so on. But since it does, it now refers to the whole chain of the big arrays, thus consuming the memory.


Ah, but then, I ran the example on Chrome v. 31. There was no memory leak, even without setting previousValue to null!  Chrome is able to prevent the leak on its own. IE-9 is not. Maybe Chrome realizes that the anonymous function is not referring to any variables, so it need not create a "context" for it.

The moral of this story then is: Beware of memory-leaks in  JavaScript, when creating functions within functions.  It is fairly easy to detect the leaks by observing the amount of memory your browser consumes, when running your application. Don't worry too much about what causes them exactly, because results might differ on different browsers. Try to reset variables whose values you no longer need to null, and that may fix it.  Help the garbage-collector a bit!


 © 2014 Panu Viljamaa. All rights reserved

Tuesday, July 1, 2014

Partial Partial Application

"Partial Application" (PA) means applying a function to only part of its arguments, and getting back a function which can take in the remaining ones. This post is about it and what I like to call "Partial Partial Application" -  meaning the resulting function can do partial application on its own, as well.

In a previous blog-post I wrote about "currying", a related concept. I posted that currying is the process of implementing a multi-argument function as a combination of single-argument functions, and calls to them. I'll discuss the difference and relationship of Partial Application and Currying, briefly.

Partial Application is something your program can do. "Currying" is something you, or your compiler or interpreter does. So PA seems like a  more straightforward concept to implement. And yes, there is an accompanying code-file that does it for you.

Let's start by asking:  If we had an implementation of the function 'applyPartially()' or applyP() for short, how would we use it? Let's write some test-cases! Ah, this is Test-Driven-Programming, we write the test-cases first?  Well maybe something like that. But let's start by writing the function applyP(), a 'dummy' version of it first. That dummy version says something about what it should do when it grows up, what kind of arguments it expects etc.

  function applyP (baseFunction, a, b, c, d, e)
  { 
   /**  ... NOT READY   
    Return a function that is like the argument 
    'aFunction' except that its leading arguments 
    will be "fixed" to be those given by the rest 
    of MY arguments. So, my result is a function
    which will (typically) take FEWER arguments 
    than my 1st argument, 'aFunction'.  
    */
   ...
  }

The 1st argument  of applyP( ) is called  baseFunction because the result of applyP() will be  derived  from it.

Now let's write one test for it:

 function f2(x,y)
 {return x + y.
 } 
 var f2b     =  applyP (f2, 1);
 ok (f2b (2) == 3);

Above we partially apply f2 with '1' as the fixed value of its first argument, and get a function f2b as the result.  Then we call f2b() passing only one argument '2' to it. Then verify that the result is 3 with our simple test-utility function 'ok()'. There, almost done. Should we go further? No, unless we can answer the following question:

What is it good for? Absolutely nothing? No. Partial Application is good whenever some arguments of a function come from a small fixed set. That usually means you will be calling the function many times with the same values from that fixed set. In such a case it makes sense to partially apply the function to those frequently used values. That will give you new functions that are simpler to call and easier to understand. They will be simpler because they take fewer arguments. They will be easier to understand because they can be given more meaningful names.

A typical example is any function that takes a boolean as one of its arguments, indicating a caller-preference of some kind.  Example:

  function runTest (haltOnError, testCase)
  { /* Run the test and if 1st argument is true
       and test-case fails, throw an error. If
       1st arg is false just return a boolean
       telling whether the test succeeded or not.
     */
    ...
  }

Now if you call runTest() many times,  sometimes with true,  sometime with false,  it might be a good idea to refactor it into two single-argument functions, by using  Partial Application:

   var runTestHalt   =  applyP (true,  aTestCase);
   var runTestNoHalt =  applyP (false, aTestCase);
   ...
   runTestHalt (myTestCase);
   //   runTestNoHalt (myTestCase);

The new partially applied functions are easier to understand because their names can express more about their semantics, because the semantics is simply simpler.There are fewer or no IF-THEN conditions you need to understand.  If you instead use the original test-function, your code would look like this:

   runTest (false, myTestCase) ;

Looking at the runTest() -call above it's not clear at all what is the meaning of the first argument. We can see it is false, but what does that MEAN?  It's hard to remember, takes some effort to look it up, and easy to pass a wrong value to it. Maybe you changed its meaning, and forgot to change some places where it was already called. When the error hits the fan in production, start debugging.

The First Rule of O-O Programmers:  Beware of IF-THEN !

In the accompanying code-file I provide an implementation of ClassCloud.applyP() which does partial-application for you. More than that, it returns functions that can successively partially apply themselves, to further arguments. Here's an excerpt from the unit-tests inside the code-file that show how it can be used:

   function f2 (x,y){return x + y}

   ok (applyP(f2, 1, 2)          == 3);
   ok (applyP(f2, 1   ) (2)      == 3);

   ok (applyP(f2      ) (1) (2)  == 3);
   ok (applyP(f2      ) (1,2)    == 3);


The last two tests above are interesting because they show that we can do partial application by providing ZERO arguments to fix, even though the function really needs more to calculate its value. Why would we do partial application with zero fixed arguments?  The last two lines  give us the clue: applyP(f2) returns a function that can be called exactly like its argument-function f2:
 
   ok ( applyP (f2) (1, 2) === f2 (1, 2) );

So it can do the same thing. But, it can also do MORE:  It can apply  ITS arguments partially, in separate calls as shown above and here again, still returning the same result as f2:

   ok ( applyP (f2) (1)(2) === f2 (1, 2));

We could say that applyP(f2) is not just a different version of f2(), it is a BETTER version of it. It can do everything  f2()does, and then some:  It can take its arguments incrementally.

applyP() as shown by above examples does partial application. But depending on how many arguments you give it, there may be more  partial application left you can do with its result.  Hence the title of this blog-post: Partial Partial Application. The results of applyP() can take care of the further rounds of partial application themselves, and they can do that in multiple stages as well - partially. applyP() can do partial application partially, or fully.


So how is Partial Application related to currying?  My previous blog-post was about currying. In that post I showed  how a function for calculating  max. of two numbers could be implemented in the "curried form" as two single-argument functions, one inside the other, the inner function returned as the result of the outer one:

  function max(a)
  { return maxB;
    function maxB (b)
    { return a < b ? b : a;
    }
  }

Note that the above function is not doing "currying". It is the result of you (or me) having done it. Making your compiler do it automatically is possible but a lot of work; it basically means implementing your own PL. Some PLs do it for you.  Whereas creating a function like applyP() should be within the realm of the practical, as shown by the accompanying, rather small file of JavaScript.

If you look at that code you will see that applyP()is mostly implemented by applyPBasic()which contains AND RETURNS the inner function newFunk(). That is basically the curried form shown for max() above. Currying is the way to implement Partial Application!

There's one more thing to know about ClassCloud.applyP():  It relies on the number of declared arguments of the base-function. If you use it with functions that rely on varying number of arguments, your results might vary (pun intended!). If you call it with more than the declared number of arguments you get an error, which is useful for error-checking. It checks that your assumptions about how many arguments a function takes are correct.  

You can copy the MIT-licensed implementation of applyP() including its tests from http://panuviljamaablog.blogspot.com/2014/07/classcloudapplypjs.html .


 © 2014 Panu Viljamaa. All rights reserved


ClassCloud.applyP.js

/***********************************************************

Implements the JavaScript -function 'ClassCloud.applyP()' 
which creates "Partially Applied" versions of any function, 
to "fix" part of their arguments. See the tests -section
below for how to use applyP(). 
URL: http://panuviljamaablog.blogspot.com/2014/07/classcloudapplypjs.html 
*********************************************************** 
   
This software is made available under
"The MIT License", http://mit-license.org/.

Copyright © 2014 Panu Viljamaa, ClassCloud LLC

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

 The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 */


function ClassCloud  ()
{ /* This is the only global created by the loading 
     of this js-file. After loading this file you can 
     get a reference to the applyP() -function like this:
     var applyP = ClassCloud.applyP;
   */
}

(function anonymous  ()
{ "use strict" ;
  var _PREVIOUSLY_GIVEN_ARGS;
  var _MISSING_ARGS_L       ;
  ClassCloud . applyP = applyP;
  return;


  function applyP (aFunction, a,b,c,d,e)
  {
    _PREVIOUSLY_GIVEN_ARGS  = [];
    var argsArray = [].slice.call(arguments);
    var fargs       = argsArray.slice(1);
    var neededArgsL = numbOfArgs(aFunction);
   
    if (fargs.length > neededArgsL)
    { var emsg =
        "Too many args given to ClassCloud.applyP() "
        + fargs.length  + ". "
        + "The base-function only expects " 
        + neededArgsL   + ": \r\n"
        + aFunction;
        throw emsg;
    }
   
    var result      
     = applyPBasic (aFunction, fargs, neededArgsL);
    return result;
    
      /**
       Return the 1st argument partially applied
       to the rest of the arguments.  
       */
  }

   function numbOfArgs(aFunction)
   { var s  = aFunction.toString();
     var p1 = s .split(/\)/) [0];
     var p2 = p1.split(/\(/) [1];
     if (p2.match(/^\s*$/) )
     { return 0;  // IE7 fix
     }
     var argNames = p2.split(/,/);
     return argNames.length;
   }


  function applyPBasic (topFunk, fargsArray, missingArgsLength  )
  {/*  
      'missingArgsLength' does NOT yet include the ones that
      may now be given in fargsArray
    */
    var knownArgs           = _PREVIOUSLY_GIVEN_ARGS;
    var argsKnownNow        =  knownArgs.concat(fargsArray);
    _PREVIOUSLY_GIVEN_ARGS  = argsKnownNow;
    _MISSING_ARGS_L         
       = missingArgsLength - fargsArray.length;
   
    var mislen = _MISSING_ARGS_L
    if (mislen == 0)
    { var result = topFunk.apply(this, argsKnownNow);
      return result;
    }
    return newFunk;

    function newFunk (y,z)
    { var result;
      var myArgsArray = [].slice.call (arguments);
      var previouslyGivenArgs = _PREVIOUSLY_GIVEN_ARGS;
      var argsKnownNow 
        = previouslyGivenArgs.concat(myArgsArray);
       
      var weKnowAllArgs
        = (_MISSING_ARGS_L - myArgsArray.length) <= 0;

      if ( weKnowAllArgs )
      { result = topFunk.apply(this, argsKnownNow );
        return result;
      }
      var nextLevelF
         = applyPBasic (topFunk, myArgsArray, _MISSING_ARGS_L);  
      return nextLevelF;
    }
  }
}) ();



// ====================  TESTS: ======================


(function anonymous2 ()
{ "use strict" ;
  if (false)   
  { /*  Change false above to true, to skip
        running these tests, which makes sense
        if you include this in the production
        version of your software. You could
        of course remove these tests alltogether.
        You can modify this source-code except
        as restricted by the MIT license given
        at the top.
    */
   return
  }

  var applyP = ClassCloud.applyP;

  testF2();  // Simple but interesting
  testF3();  // N arguments.
  testF1();  // Trivial case
  testF0();  // Pathological case

 return;
  
  
  function testF2 ()
  { 
     ok (applyP (f2, 1, 2)          == 3);
     ok (applyP (f2, 1   ) (2)      == 3);
     ok (applyP (f2      ) (1) (2)  == 3);
     ok (applyP (f2      ) (1,2)    == 3);
     /* 
        f2() sums up its TWO arguments.
        The last 2 cases above are interesting because 
        they show that we can do partial application 
        without passing in ANY arguments. See the 
        blogpost for more explanation. 
        */
     function f2 (x,y){ return x + y}
  }

  function testF3 ()
  {
     ok(applyP (f3      ) (1)(2)(3) == 6); // a)
     ok(applyP (f3      )   (1,2,3) == 6); // b)
     
     ok(applyP (f3, 1   )    (2)(3) == 6);
     ok(applyP (f3, 1   )     (2,3) == 6);
     
     ok(applyP (f3, 1, 2)       (3) == 6);
     ok(applyP (f3, 1, 2, 3)        == 6);  // c)
     /*
       Above shows that there are 3 ways you can  
       inject the arguments into the calculation: 
       a) You can give them ONE BY ONE to the result 
          of applyP().
       b) You can give one or more of them in a SINGLE
          call to the result of applyP().
       c) You  can give them ALL when calling applyP(). 
      */
     return;
      
     function f3 (x,y,z)   { return x + y + z }
  }
  
  
  function testF1 ()
  {
   var f1P =  ClassCloud.applyP (f1);
   f1P ()  ;  
   ok( f1P()()()().constructor == Function)  ;
   // As long as you call f1P with TOO FEW arguments
   // you will be getting a curried function back.
   
   ok (applyP (f1) (55) == 55)  ;

   var emsg = null;
   try 
   { applyP (f1, 1, 2)  ;
     /* Above fails because you are trying to
        apply more arguments than the base
        function has declared to itself. 
        Calling it with too many args would
        be of no use so we treat it as error.
    */
   } catch (e) 
     { emsg = e;
     }
   ok (emsg != null);  
   return; 

   function f1 (x)   { return x}     
  } // end testF1.


  function testF0 ()
  {  
   ok (ClassCloud.applyP (f0) == 123);
       
   var emsg = null;
   try 
   { ClassCloud.applyP (f0)()   ;
     /* Above fails because ClassCloud.applyP(f0)
        return 123, which is not a function.
      */
   } catch (e) 
     { emsg = e;
     }
   ok (emsg != null);  
   
   function f0 ( )   { return  123; }
     
  } // end testF0
  
  function ok (bool)
  { if (! bool)
    { throw "ERROR. ok(bool): bool == " + bool ;
    }
    /**
     Our simplest possible testing-utility
     */
  }
}) ();

Saturday, June 28, 2014

Currying Explained Simply

I know there are many articles about "currying" on the web, especially aimed at JavaScript programmers.  Do we need another one?  Well last time I looked I didn't find a really easy-to-understand article about it, so let me try my best. It can be a confusing topic and I admit I have been confused myself, at least dazed. Maybe I can help you if you feel that way too.

The reason it is confusing is that most articles about it are based on the unspoken assumption that you NEED currying, regardless of your PL. But you rarely do in JavaScript.  I'll try to explain in this blog-post why not. And then you need to understand what currying is NOT, why it's not the same thing as "partial application". But I'll focus on explaining "currying" in the simplest way possible.

It is confusing to try to understand something if you don't know why you need it. And even more so if you really don't need it.  So here we go ...

Imagine your programming department has the rule that functions can only take max. one argument. You think, who would make such a silly rule. But there are programming languages where a function can take just one argument. Wink wink: Haskell, F#.  Then your manager asks you to write a piece of code that calculates the maximum of two numbers:

  function maxOfTwo (a, b) 
  { // Return a if it is bigger than b, 
    // else return b.
    ...
  }

You're in a pickle. You're supposed to return max of two numbers, but your function can only take one argument. What to do? The answer, the secret sauce:  currying.  Curry to the rescue. Can you do it? You can, if you can write something like this:

  max (5) (22);   //  ==  22

Notice above you are not passing multiple arguments to a function.  You are doing something else. You are calling max(5) and it seems to be returning something you can pass (22) into.  This is only possible if  max(5) RETURNS A FUNCTION. Yes, functions can return functions, this is JavaScript!

So how would you write the above function 'max'?  Like this:

  function max(a)
  { return maxB;
    function maxB (b)
    { return a < b ? b : a;
    }
  }

Try it.  You see it works. "max(5)" returns the function 'maxB' and if you pass 22 to that, you will get the result 22.

The process of implementing 2- or more argument functions by combining 1-argument functions in this manner is called "currying". 

So is currying useful?   How often do you have 1-argument functions laying around you  want to combine in this manner, to do the job of a 2-argument function?  Maybe you do, but typically it is much simpler to write a 2-argument function to start with - if your language allows it. 

Haskell and F# don't because "pure functions can only take one argument".  But they "fake it".  If you write the equivalent of  max(a,b) in them, they automatically, behind the scenes, turn it into the form of our single-arg max(a) above. Why? That would be another blog-post. But if you program in JavaScript you don't need to know. 

If you program in Haskell or F# you never need to DO currying, it is done for you. If you program in them it's good to KNOW about it, so you know what goes on "behind the scenes".  

In JavaScript, you can forget about it. We rarely start with a two-argument function which we want to turn into two 1-argument functions.  If we already have a 2-argument function we just use it. 

The "pattern" in the max -example above is in fact a very useful JavaScript idiom. You could call it "currying" if that is the thought-process that led you to write it in that manner. But note that executing the function max() does not DO currying. The function  max() is the RESULT of you  doing currying, writing a two-argument function in the "curried form". And it is not doing 'partial application' either. That will be the topic of my next blog-post, I hope.

So what is "currying"?  Let me offer this definition:
 
"Currying is the process by which a function of N arguments is implemented as N single-argument functions such that first of them takes in the first argument and returns a function which takes in the 2nd argument and so on, until the Nth single-argument function finally returns the value of the multi-argument function being implemented."


 © 2014 Panu Viljamaa. All rights reserved

Thursday, June 12, 2014

Virtual Virtualization

I find it interesting and useful that we can run virtual machines inside virtual machines. I'm probably not the only one. Here's my story.

I recently bought a new PC running Windows 8. Rather than replace my older Vista-PC with it, I use the new machine over a Remote Desktop -connection, from the old machine.

That way I don't have to REPLACE the old one with the new, but can combine the processing power of both. I can use just a single keyboard and the same set of monitors for both. I also get to keep the existing applications and peripherals on the older machine, without compatibility issues, or having to buy new licenses for them.

The new machine running Windows-8  (which I use over remote-desktop-connection) runs a virtual machine running Windows-7, which I log into. From that Windows-7 session I take a VPN remote-desktop-connection to a PC on a client's site running also Windows-7. From there I take a remote connection to a physical server running a virtualized Windows -server. In that remote Windows Server -session I run a Java virtual machine.

A remote-desktop-connection is not usually considered a "virtual machine" but it kind of is, conceptually. It effectively  transforms my physical PC to look and behave like a machine that seems to virtually exist on it.  From my point of view it is a virtual machine, running on my physical (OR virtual) machine.

What about JavaVM? Is it really a virtual machine? Yes, it's name is officially Java Virtual Machine. It is one, it just doesn't typically run an OS. It just runs applications. Imagine that your PC would run just one or two applications on it's BARE METAL, no OS in between. That would make those applications  run and start fast. JavaVM is a virtual version of that scenario.  A Virtual Machine is not the same thing as an Operating System. An OS needs a machine, virtual or physical, to run on. But note, there is something called JavaOS.

If instead of Java I would develop on something like Pharo Smalltalk it would be even clearer that I was developing applications for a virtual machine ON a virtual machine. Pharo-VM runs its own virtual desktop and has the ability to save "snapshots" of the state of that VM on disk, and keep multiple copies of them around.

So, I have multiple layers of virtual machines running on virtual machines. And it all works great. Diagrammatically my configuration looks like this:

-> Windows Vista
-> Windows-8
-> Windows-7
-> Windows-7
-> Windows Server
-> Java Virtual Machine

You could think that having all those layers would affect performance negatively. But there are actually four physical machines involved whose combined processing power helps to make performance good. A fast network is needed of course as well.

One more benefit of this setup: Running different things on different machines which are more or less isolated from each other, and which can be restored to an earlier version if needed, improves security.

The main points of this article: If virtual machines are good, then it's probably also good to run them inside a virtual machine, at least in some cases. If "bare metal" is good, then it's probably also good to virtualize the good aspects of it, as exemplified by Java Virtual Machine. If something is good, then probably also a virtualized version of it is.

 © 2014 Panu Viljamaa. All rights reserved

Wednesday, April 9, 2014

How to Enhance JavaScript Built-in Classes

In this article I discuss how to improve upon JavaScript built-in classes such as Array and Object and others and also how not to do it. A general utility for "enhancing classes" is provided and a concrete example of enhancing Array.prototype is presented.  An obvious application of this approach is to  make older and newer browsers work alike. Links to source-code are provided. 

 About terms used:  JavaScript is a prototype-based language. We can call its constructor-functions "classes". Why?  Because, they are things which 1) Create instances and 2) Describe the common properties of all instances created by the same constructor.  A 'method' is a property of an object whose value is a function. For more on what terms  'class' and 'method' can mean in JavaScript, see my previous blog-post  "Does JavaScript have classes and methods?"  .

Our main purpose is to present a way to enhance JavaScript base-classes in general. But to demosntrate the benefits of our approach we present source-code for four particular enhancements to  Array.prototype.map() in particular, which make it easier to use map() in practice, requiring less code to do so..

So two related topics are covered:
 1) How to enhance JavaScript base-classes in general
 2) How to enhance Array.prototype.map() in particular.

The presentation is in the Design Patterns -format: Problem, Forces, Solution, Implementation, Example, Consequences, Try-it-out.  This is a work in progress so any comments, suggestions and corrections are welcome.

1. PROBLEM

JavaScript is an evolving language, which means some of its newer features are not available on older browsers. Newer versions of the ECMA -standard introduce powerful functional-programming features like map() and reduce(). But those do not work on older browsers. And as good as the latest standards are, we might still sometimes like to improve them to our taste.

Here's an example.  The ECMAScript 5 Array.prototype.map() syntax is lengthy, because JavaScript does not provide a shorthand notation for anonymous functions. Can we create our own, easier to use implementation of map()?

The purpose of map() is to avoid  boilerplate code that is required by a for -loop. But having to write a full function definition for even the simplest uses of map() means that using map() may not be much shorter than writing a for-loop after all.

You typically use map() like this:

  [1,2,3].map(function (x) {return  x * 10});  

I would prefer to be able to write it in a shorter way:

  [1,2,3].map( 'x * 10' ) ;  

And I also can't use map() at all if my code is run on older browsers that don't provide it.



2. FORCES

In a previous blog-post I discussed the function F() whose purpose is to create functions out of  String -expressions. That allows us to write the above example more succinctly as:

  [1,2,3].map (F('x * 10'));    

A problem I have with this is that function name 'F' is not very descriptive. The reason for that is that our purpose is to make the code shorter to make map() easier to use. Therefore I made the name of the  function 'F' as short as possible. Also there's still two levels of parenthesis combined with the single-quotes make the code look more complicated than I would like to.

An obvious solution to these problems would be to modify map() so that it accepts a String-argument directly, and uses F() internally to turn it into a function. Then we can write:

  [1,2,3].map ('x * 10') ;   
 
And thus finally we have arrived at tyhe problem this article is about: How can we modify map() and  similar functions to work the way we want them to, allowing us to write less code, and make it work on all browser versions?

One solution is to add our own implementation of  map() as a method of Array.prototype, thus providing it for older browsers as well. This is the "Polyfill" -approach suggested for instance at the MDN web-site .

At first I thought using a 'Polyfill' is a great idea - because I can replace the built-in implementations of map() etc. with my own, and make it work on  browsers. But I soon discovered a big problem. I added the function map() to Array.prototype as a property. Then on IE-8 the new map() -method had become an enumerable property of every Array in my program. Why? Of course! Every instance of Array inherits its properties from the prototype. Sorry Charlie, no cigar. Trying to loop over any array now included my map() -implementation as one of the elements of any Array.

That meant I would need to add a test to every Array-loop to exclude my map-method. And I would also need do the same for every loop in every library I was using, to be safe. Making such checks inside every for-loop would be tedious and error-prone. It would also make my loops slower. 

An obvious problem with the Polyfill -approach is also that it can break code in the libraries you use, if those rely on the standard behavior behavior. You should not modify parts of code that other parts of code may depend on not having been modified. 


3. SOLUTION

If we can not safely modify the existing Array.prototype the only option seems to be to create our own version of it. Let's imagine we can do it like this:

var Array2 = ClassCloud.enhanced(Array, {map: function () {...}); 

This will return a constructor which will create a wrapper-object around any Array given as argument to it. Let's further imagine that such a constructor can be called without the 'new' -keyword, to make our code even shorter. Then we can write:

 Array2([ 1,2,3] ) . map( ' x * 10 ' );

Array2.map() loops internally over the basic array given as argument when the Array2 -instance was created.

We can make calls to Array2() even shorter of course by assigning it into a short variable:

 var A = Array2;

Then we can write most succinctly:

 A([1,2,3]).map('x * 10');    // 1.

Contrast the above with our first solution:

 [1,2,3].map(F('x * 10'));    // 2.

They are still much the same length. So is there much improvement after all?  Yes.  The difference is that Example 1 works on older browsers too.

And 'A' above is a class, whereas 'F' is just a function. This same 'A' can give us other functions missing from older browsers, reduce() for example. Since we now have an easy way to create enhanced Arrays, it is easy to see that we can apply to improve other system classes as well. We could enhance every Object with any methods we choose and write for example:

 O( {x:1, y:2} ) . map ( ... ) 

So this solution makes it possible to write our own interpretation of what it means  to "map" over Objects as well. We have a general way to improve any base-class in JavaScript, and in a way that works on older browsers too. Obvious candidates for enhancement include for instance  Math, Boolean, RegExp and Function.

Now that we know what we want, all that remains is to come up with an implementation. 



4. IMPLEMENTATION

The Array2() -function used previously must be a constructor that takes a an Array as argument, and returns an instance of Array2, with some enhanced behaviors.  It doesn't need to have all methods of a standard Array, because we choose to use it only when we need its enhanced functionality. For instance we might give it have just the method map() so we can use a String-expression in place of a function as the argument of it. But we can also include some pre-existing methods of Array if we think we need them.

To create our Array2 we need the function ClassCloud.enhanced() , which can be used like this:

var ap = Array.prototype;
var methodMap =
{ pop:      ap.pop
, push:     ap.push
, concat:   ap.concat
, map:      mapCC  
};

var Array2 = ClassCloud.enhanced (Array, methodMap);
var a2     = Array2 ([1,2,3]);
var a2b    = a2.concat([1,2,3]);

The above reuses the methods pop(), push() and concat() from the built-in Array, and creates a new method of its own  'map()',  implemented by the function 'mapCC'.

SIDE-NOTE:  If we targeted newer browsers only, we could add EVERY method of the Array.prototype automatically to our new class. But that would not work on older browsers because on them we don't know what are all the methods of the standard Array. We can't enumerate them, partly to the same reason why adding our own methods to Array.prototype is a bad idea.The reason why older browsers can't enumerate methods is explained in this answer on StackOverflow.

Internally the function  ClassCloud.enhanced() will create wrapper-methods which call the methods given in the methodMap -argument, in a way that the  pseudo-variable  'this'  inside them is bound to the data given when Array2 is called.  The same thing will be done to the newly added methods similarly.

One added requirement is that when an original method like Array.prototype.concat() returns an instance of Array, the wrapper-method must return an instance of Array2.  That will allow us to chain together a set of iterators we have defined for Array2. So we can compose applications of our functions in this way:

  var a2b  = Array2 ([1,2,3]) . map(funkA) . map(funkB) ... ;

I hear you thinking:  You are chaining together function-calls. How about doing the same to  the functions themselves? We could then reuse such  composed functions anywhere. And yes ... ClassCloud.enhanced() can be used to enhance any built-in JavaScript constructor. Therefore we could use it to enhance the Function.prototype for instance. We leave it as an exercise to implement Function2 which could be used like this:

  var funkD =  Function2(funkA) . then(funkB) . then(funkC) ;

The implementation of the basic enabling method ClassCloud.enhanced()  is given in the code-listing  at (hyperlink).



4. EXAMPLE

As a more concrete example of how to benefit from ClassCloud.enhanced() I next describe  my enhancements to map(),  implemented with the function mapCC().  You can copy its source-code from [hyperlink].

mapCC() accepts the same arguments and can be used exactly like the standard ECMAScript 5 map() described at MDN web-site . But it enhances the standard map() in four ways described in the four sections below.  For actual examples of using this enhanced funcitnality see the test-code at [HYPERLINK].


4.1  String argument 

In addition to a function, mapCC can take a String that evaluates to an expression, which gets converted to a function. That means you need to do less typing, and your code becomes faster to read. This means we can write:

  [1,2,3].map( 'e * 10' );  

No "return" statement is needed within the string, which makes it short to write. It must contain a JavaScript expression, whose value is returned.  The above is a variable that gets as value the elements of the array of successive calls. There are two more variables you can use within the string. 'i' will be the index of the iteration on each round and 'a' contains the array being iterated over. For examples of their use see the unit-tests provided in their own document.


4.2  Useful default for 'this'

According to the MDN documentation  'map' takes an optional 2nd argument 'thisArg' for which it is said  "... If a 'thisArg' parameter is provided to map, it will be passed to callback when invoked, for use as its this value.  Otherwise, the value undefined will be passed for use as its this value."

However, on my IE-10 it seems if I don't pass in that argument, the value of 'this' will be the window-object. That is one more reason to define your own implementation for map() rather than rely on built-in implementations which may differ from browser to browser.

For our mapCC() if no 'thisArg' is passed in, we automatically create a new object {}that will be the value of 'this' during a call. So our callback function can always use 'this' for instance to pass data from one call to the next. As an example we can simulate another ECMAScript-5 function 'reduce()' with our map():

 a2 = Array2([1,2,3]).map
 ( function (each)
   { var previousResult  
     = this.previousResult   
     ? this.previousResult  
     : 0;
     var result = each + previousResult;
     this.previousResult = result;
     return result;
   }
 )

Having executed the above the following tests will pass:

 a = a2.data;
 ok (a[0] == 1);   // 1 + 0 
 ok (a[1] == 3);   // 2 + 1 
 ok (a[2] == 6);   // 3 + 3 

Above 'a2'  contains an instance of our Array2 which in our implementation (not visible above) "wraps" the instance of Array it "represents" into its field 'data'  for easy "unwrapping".  It's a bit like a monad really. Even if it's not really a monad, according to the official definition, this points to the direction of how they could be implemented in JavaScript. An alternative would be to provide a method at() to access the elements of Array2, so we could write:

 ok (a2.at(0) == 1);   // 1 + 0 
 ok (a2.at(1) == 3);   // 2 + 1 
 ok (a2.at(2) == 6);   // 3 + 3 



4.3 Selecting specific elements

If the callback-function returns undefined, that will not be added to the result-array. This means you can decide to skip some elements of the original array. For instance we can select all odd numbers from a list:

 a2a  = Array2  ([1, 2, 3, 4, 5]);
 a2b  = a2a.map ('e%2 ? e : undefined'); 
 d    = a2b.data ;

 ok (d[0]     == 1);  // 1%2 != 0 so result is not undefined
 ok (d[1]     == 3);  // 3%2 != 0 so result is not undefined
 ok (d[2]     == 5);  // 5%2 != 0 so result is not undefined
 
 ok (d.length == 3);  



4.4 Stopping iteration early

Sometime you are interested to if there's at least one element in a collection that fulfills a specific condition. For instance you might want to know whether there are ANY odd numbers  in a list.

When using Array2().map()  we can stop the iteration early inside the callback-function by setting

  this.cut = true;

This means you can stop the iteration as soon as you have enough elements processed.
 a2 = Array2([22,98,3,100]).map
 ( function (each)
   { var result = each + previousResult;
     this.previousResult = result;
     return result;
   }
 )

Having executed the above the following tests will pass:

 a = a2.data;
 ok (a[0] == 1);   // 1 + 0 
 ok (a[1] == 3);   // 2 + 1 
 ok (a[2] == 6);   // 3 + 3 

This can be used to find the first element that fullfills some criteria. No matter how big the array you are iterating over is, you can stop after 99 elements. Or you can use this to stop iteration as soon as you know there can be no more good results. Our choice of the filed-name 'cut' was inspired by the language Prolog, which uses "cut" for just that purpose: stop searching for more answers.



5. CONSEQUENCES

Above we have seen that it is bad to modify system classes directly, even if that modification only consisted of  adding new methods to them. By using the pattern presented we can pick and choose the methods for our alternate system from multiple sources, including our own additions without having to modify system classes.

But the pattern is called Alternate System-Class, not "Pick and choose your methods". Why? It could seem that since we can add any methods to our alternate system-class, there would be little connection between our alternate class, and any specific system-class. Yet there is. The enhanced() -method takes the chosen system-class as its first argument because that allows it to "wrap" the results returned by the original method into instances of the alternate system-class.

That allows us to chain multiple operations involving our new methods into a call-chain without requiring us to create new instances explicitly. If that starts to sound bit like "monads", you may be right.  Since the provided source-code is so short, you can study it in detail, and insight into the benefits of "monadic" programming.


6. TRY-IT-OUT

To use the JavaScript module ClassCloud described above you need to 3 files on your own system, in the same directory, with the content from the 3 source-code pages related to this article.

The three files needed are:

1. The HTML-file ClassCloud_enhanced.htm. Copy the contents of it from the lilnked page ClassCloud_enhanced.htm . 

2. The JavaScript -file ClassCloud_enhanced.js. Copy the contents of it from ClassCloud_enhanced.js 

3. The JavaScript -file ClassCloud_enhanced_TEST.js.  Copy the contents for it from ClassCloud_enhanced_TEST.js. 

The two JavaScript -files will get loaded when you open the HTML-file in your browser. The first JavaScript -file contains the implementing source-code.  The 2nd JavaScript -file contains tests which serve as examples of how to use this module.

If for some reason the tests don't pass, you should see an alert-box on your browser. That might be because our code does not run on your browser as we intended, does not run in conjunction with your other code, or because you have modified this code or its tests, to accommodate your own modifications and tests for them.

_________________________________________________________
Copyright © Panu Viljamaa 2014. All rights reserved.