Net health monitoring

Question

This is a little net health monitoring project I made in NodeJS. It pings Google every second and stores the result (true/false) along with the date in a database (MongoDB). I've collected enough data and I want to represent it for the past week.

I created a 2x2 array of 7 days and 24 hours each. I go through each record and map it to the appropriate day and hour of the week. And I give that hour a little score: if true +1, if false -5. And it creates a little matrix like this:

[
 [2015, 1724, 1733, 1679, ...
 [1818, 1909, 1614, 1829, ...
 [1632, 1778, 1726, 1657, ...
 ...
]

The only problem is computation takes ~10-15 seconds and lots of CPU for going through ~200k records of just 3 days.

Is there some scope for improvement here?

days = days || 3;
var counter = 0
this.find({
 date: {
 // Date greater than (Now - X days)
 $gt: new Date(Date.now() - days * 24 * 3600 * 1000),
 },
}, function(err, datums) {
 var week = new Array(days);
 for (var i = 0; i < week.length; i++)
 week[i] = new Array(24);
 // Empty Week Matrix (7x24)
 datums.forEach(function(data) {
 counter++;
 var date1 = data.date;
 var date2 = new Date();
 var timeDiff = Math.abs(date2.getTime() - date1.getTime());
 var diffDays = Math.ceil(timeDiff / (1000 * 3600 * 24));
 //stackoverflow.com/q/3224834/1266650
 var day = days - diffDays;
 if (day < 0) return;
 var hour = data.date.getHours();
 if (!week[day][hour]) week[day][hour] = 0;
 if (data.isNetAlive)
 week[day][hour]++;
 else
 week[day][hour] -= 5;
 });
 console.debug(week);
 console.debug(counter, 'records');
});

Answer 1

You might want to move out 1000, 3600, 24 to "constants" so that they're properly described. Something like:

var HOURS_A_DAY = 24;
var MILLISECONDS_A_SECOND = 1000;
var SECONDS_AN_HOUR = 3600;

In addition, you can name your variables properly so that they're easily understood. timeDiff could be timeDifference, diffDays could be daysDifference.

Creating date objects per iteration is costly. Try operating using milliseconds instead. Make data.date return milliseconds for date1. Then use Date.now() on date2. Also, if possible, avoid doing getTime() and getHours(). Try calculating them manually.

Next, Math.abs can be done using a bitwise operation. The same could be done for Math.ceil, just look around the internet. Here's an example of a bitwise Math.abs, where n is the number to absolutify.

return (n < 0) ? (~n + 1) : n

Bitwise isn't the solution to all math performance problems and highly depends on the JS engine running. Some bitwise operations are faster than their high-level counterparts, some the other way around. Highly depends on the combination, so try checking which ones work for you.

if (!week[day][hour]) week[day][hour] = 0;

Property access and condition evaluation can be costly. Move this out to the for-loop that builds your matrix. This has nothing to do with the data crunching. It's just initializing a hole in the array to 0.

Lastly, for a more beautiful code, let's make everything less redundant and more consistent-looking.

week[day][hour] += data.isNetAlive ? 1 : -5;