Helper functions to perform simple token replacements in a string

Question

This script provides function T9r, which has some methods to detect, parse and replace tokens "{{ some_token }}" in a string with properties on an object.

My use case, to have "composable" json objects or strings, (used for configs), that can be populated with real values at runtime.

It's been a while since i worked in JS and i just wanted to get a gauge on my style, and general readability of my code, as well as see if there are any improvements i could make to the script.

Example Usage:

var str = '{{ App_Root }}/apple/sauce/{{ OK }}';
var context = { app_root: ':-)', Ok: '' };
T9r.replaceTokens(str,context);

Module:

// T9r ~ "T{oken Parse}(9)r"
const T9r = function(){};
/**
 A helper function to process and replace
 all tokens within a string.
*/
T9r.replaceTokens = function(Str, Context){
 return T9r.parseTokens( T9r.extractTokens(Str), Context, Str );
};
/**
 Will match all tokens "{{ some_token }}, {{some_token}}, 
 {{ Some_ToKen }}" within a body of text, extracting all 
 matches ( Tokens ), while pruning each match, removing 
 the opening and closing curly brakets, as well as strip out any 
 whitespace, so we have text that can be used to lookup props
 on an object.
 Will return an empty array, if no tokens are found.
*/
T9r.extractTokens = function( str, pattern){
 pattern = pattern ? pattern : /\{([^}]+)\}/ig;
 var matches = str.match(pattern);
 if( ! matches ) return [];
 return T9r.pruneTokens(matches); 
};
/**
 Returns the count of Tokens that exist within a 
 string body.
*/
T9r.tokenCount = function( str, pattern){
 pattern = pattern ? pattern : /\{([^}]+)\}/ig;
 return str.match(pattern).length;
};
/**
 Removes the leading and trailing wrapping-chars from
 a token match, as well as strip out all whitespace.
*/
T9r.pruneTokens = function(Tokens){
 Tokens.forEach(function( token, idx, tokens ){
 tokens[idx] = token.slice(2,-1).replace(/\s+/g,'');
 });
 return Tokens;
};
/**
 Checks to see if some reasonable version of a token exists,
 within our context and returns the actual match. Otherwise returns
 null.
*/
T9r.recognizedIn = function(token, context){
 if( context[token] ) return token;
 if( context[token.toLowerCase()] ) return token.toLowerCase();
 if( context[token.toUpperCase()] ) return token.toUpperCase();
 // Last ditch effort to find matches
 for ( prop in context ) if(token.toLowerCase() === prop.toLowerCase()) return prop;
 return null;
};
/**
 Will loop through a set of tokens, replacing all matches within a 
 string body, with the values supplied via a context. 
*/
T9r.parseTokens = function(Tokens , Context, Str ){
 Tokens.forEach( function(token, idx, Tokens){
 var TOKEN = T9r.recognizedIn(token, Context);
 if( TOKEN !== null ) 
 Str = T9r.parseToken(TOKEN, Context[TOKEN], Str);
 });
 return Str;
};
/**
 Will automaticly escape Character Classes, for use by the RegExp
 Contructor. For when composing RegExps dynamicly the symantics 
 of (\s) and other character classes can become very messy.
 "/\s/"+some_var+"/\s/" must be written as "/\\s/"+some_var+"/\\s/"
 You end up having to perform extra escaping, not for your pattern but 
 for the RegExp constructor, this becomes very messy, is 
 easy to forget and a little tricky to debug.
*/
T9r.escapeCharacterClasses = function(string){
 return string.replace(new RegExp(/[\\s|\\S|\\w|\\W|\\d|\\D]/, "g"), '\\s');
}
/**
 Wraps the value of a variable into a Regex that selects the whole
 token, including the curly brackets.
*/
T9r.makeTokenPattern = function(variable, left, right, flags ){
 left = left ? left : "\{{2}"; 
 right = right ? right : "\}{2}"; 
 flags = flags ? flags : "ig";
 return new RegExp(T9r.escapeCharacterClasses( 
 left + ".?\s*(" + variable + ").\s*?" + right), 
 flags
 );
}
/**
 Within a string body, does the actual replacement of all instances of a token, with a 
 supplied value.
*/
T9r.parseToken = function(Token, Value, Str){
 return Str.replace(T9r.makeTokenPattern(Token), Value);
};
module.exports.T9r = T9r;

• 200_success: Rev 6 has been rolled back to Rev 4. Please see What to do when someone answers .

Answer 1

A few notes:

• Your escapeCharacterClasses function is broken. In multiple ways. For one, I get an error because new RegExp doesn't accept the flags argument ("g" in this case), if the first argument is a regex literal. (Edit: Well, it's broken in my outdated browser, at least. According to the comments, ES6 allows flags to be set on literals, but my browser doesn't allow it - despite handling other ES6 features fine.) You can only supply the flag argument if the pattern is a string.
  Secondly, the pattern makes no sense as far as I can tell. You have a big character class, [...], but you're using | as though you meant it to be a branching expression instead, i.e. (a|b). So what you have (if it could run at all) is a regex that matches \, s, S, w, W, d, D, and | as individual characters, and replace them with \\s. Which makes no sense as far as I can tell.
  Even if it was a branching statement, it would, as far as I can tell, replace everything with \\s. A pattern like /(\s|\S|\s|\W|\d|\D)/g matches, well, everything. Again: Makes no sense.

• JavaScript convention is that functions and variables are camelCase and only constructors are PascalCase. You're mixing things: In most function the arguments are camelCase, but in parseTokens they're PascalCase, etc.. Be consistent.

• Also make others be consistent. JavaScript is case-sensitive, so I'd recommend making the tokens case-sensitive too. I.e. your example of replacing {{ App_Root }} with app_root shouldn't work.

Anyway, all of this can be made a lot simpler. You can actually do this in one go, if you want:

function t9r(template, interpolations) {
 return template.replace(/\{\{\s*([^}\s]+)\s*\}\}/g, (_, token) => interpolations[token] );
}

That'll replace any {{ token }} stuff in the string with a property from the interpolations object.

Better yet, you can be stricter about it, and require tokens to only consist of word characters and digits - like a property name. The one above is a bit too liberal in what it allows, if you ask me. And you add the option to throw an error if something doesn't have an interpolation. And use some ES6 interpolation while we're at it to generate the error message:

function t9r(template, interpolations, throwErrors) {
 throwErrors = throwErrors === false ? false : true; // default to true
 return template.replace(/\{\{\s*(\w+)\s*\}\}/g, (marker, token) => {
 if(throwErrors && !interpolations.hasOwnProperty(token)) {
 throw new Error(`Missing interpolation for '${token}'`);
 }
 return interpolations[token] || marker;
 });
}

Now the tokens must be only word characters. And if you explicitly set throwErrors to false, you just get the string back with all possible replacements made, and the impossible ones untouched (i.e. still written as {{ foo }}).

If you want something that just pulls out the token names present in the template string, you can do:

function t9rTokens(template) {
 var tokens = [];
 template.replace(/\{\{\s*(\w+)\s*\}\}/g, (_, token) => {
 tokens.push(token);
 });
 return tokens;
}

Of course, in that case it'd be much better to move the regex pattern into a reusable constant, rather than repeat it in two functions.

• Casimir et Hippolyte: Writing \w[\w\d]* doesn't make sense since \w contains \d already.
• Flambino: @CasimiretHippolyte whoops, right. I had in my head that \w = [a-z_] only
• Casimir et Hippolyte: Other thing, writing new RegExp(/pattern/, flag) throw an error until ECMAScript 5, but is allowed starting with ECMAScript 6.
• Flambino: @CasimiretHippolyte Interesting. I just tried it in my browser while reading the question, and it failed. Didn't expect it to have changed with ES6
• Casimir et Hippolyte: Actually, if you want to test ES6, you need to use nodejs or firefox 39.