Link to blog all posts was migrated Blogger
Client side error logging
Everybody uses some analytics service to know how users uses its site. Some programmers even use custom events for this.
Generally it is good idea to know also if your user go out because of js error on your site.
Error logging on client
When something happen on development site programmer usually open developer console and look at errors, but if this happen for remote user we cannot ask each to open console and send us errors.
window.onerror — it is js DOM event handler that catch all programming errors. Some examples of errors:
// syntax error <script>{</script> // uncaught exception throw "some message"
(i think anyone can suggest even more)
window.onerror handler has 3 arguments:
window.onerror = function(message, url, line) { // message - it is some description of this error // url - it is url of script that cause error // line - it is line number in script where error happen }
Simple example at jsfiddle on clicking button we can see in console messages:
So this is all we need to handle all errors. Second thing it is how to report errors.
On client we have a lot of choices where and how to send requests. For example i will use my tracking example from previous post. I need to include loading code in head as early as possible to get errors in other scripts:
<script> window.onerror = function(message, url, line) { __it("error", message, url, line) } </script>
Of cource i understand that almost everyone prefer to use something more popular and advanced than just analytics from some blog post. Google analytics has special category for exceptions and someone will write something like this:
window.onerror = function(message, url, line) { ga('send', 'exception', { 'exDescription': 'Error "' + message + '" at '+ url + ' on line ' + line }); }
As a basic and free solution this is enough. But also need to say that exists a lot of paid services that solve similar problem but with some nice interface and more advanced features (I do not want publish its url, but if someone need i will add).
Jenkins ♡ node.js
With my colleague (from InGo) we are building new small startup which will help (we hope) to people to solve «problem of 1000 tabs opened» (usually to read them later).
Our server are built with node.js express application and for CI we are using Jenkins. This post is about how we make Jenkins understand node.js test tools results.
First I have installed some plugins:
- AnsiColor — this one to have a nicer console output when it is available
- Jenkins Cobertura Plugin — code coverage
- Jenkins Violations plugin — for jshint/jslint warnings
- xUnit plugin — test’s results
To do not spend time each time to edit jenkins job I created Makefile and added tasks to it.
Test results with xUnit plugin
For tests we are using mocha and should.js (and supertest for http requests). This is a job to get xUnit compatible xml:
test-jenkins-xunit: @NODE_ENV=test ./node_modules/.bin/mocha \ --recursive \ --check-leaks \ --reporter xunit \ $(MOCHA_OPTS)
In build steps I add shell script step:
make test-jenkins-xunit 1> results/xunit.xml
And in post build steps:
JsHint warnings with Violations plugin
First again Makefile task:
test-jenkins-jslint: ./node_modules/.bin/jshint $(JS_FOLDERS) --jslint-reporter
And how to use it in jenkins build step:
make test-jenkins-xunit 1> results/jshint.xml || exit 0
I added ‘exit 0’ because it return 1 if anything found and build step will fail.
Exists one problem with Violations plugin and xml that produced by jshint. Because it uses relative path you cannot see in job report where actual problem is. I saw several posts about it and nobody publish solution. After previous build step i have added new one that fix xml:
sed -E "s?<file name=\"(.*)\?\">?<file name=\"$WORKSPACE/\1\">?" results/jshint.xml > results/jshint-fixed.xml
This will replace relative paths in name attribute to absolute that Violations Plugin can handle.
Now after build step:
Code coverage with Cobertura plugin
Makefile task:
test-jenkins-cov: @NODE_ENV=test ./node_modules/.bin/istanbul cover --report cobertura --dir ./results ./node_modules/.bin/_mocha -- \ --recursive \ --check-leaks \ --reporter $(REPORTER) \ $(MOCHA_OPTS)
Build step is very simple:
make test-jenkins-cov
Last thing after build step:
Now I press ‘Save’ and run this job to see reports (I hope You don’t forget to add repository to this job). Click on job name:
When I click on any of this charts i can see more detailed report. This one from Violations Plugin that you can see when click on file in report (and you will not see this if you do not fix problem with relational paths).
That is all! I hope this post will help to somebody. I decide to write it because there is no similar post that describe whole process.
Making web tracking
A few weeks ago, I had a task to track if users at a partner site open our pop-up. There are a lot of solutions, but the main idea is simple. When an event happens, we send a request to our server.
Need to say that I just need to send get request and make it as most universal as possible.
One of the best solutions (that I chose) is to use a transparent image (remember how to make transparent png working in ie6, no? but I remember =). It is more than universal — img tag appeared very long time ago!
When someone include our image in markup of page via img tag, browser will load it — that is what i want. So lets take 1px transparent gif (small and good supported) — I did not have such so I googled it and found… several. I took the smallest one — 35 bytes.
I will use Lift framework as we use it at InGo. First I need bytes of this image:
Files.realAllBytes(Paths.get("./empty.gif"))
And save it somewhere
val emptyGif: Array[Byte] = Array(71, 73, 70, 56, 57, 97, 1, 0, 1, 0, -128, -1, 0, -1, -1, -1, 0, 0, 0, 44, 0, 0, 0, 0, 1, 0, 1, 0, 0, 2, 2, 68, 1, 0, 59)
Now i need rest endpoint to return my image:
object TrackApi extends RestHelper with Loggable { val emptyGifResponse = InMemoryResponse(emptyGif, headers, Nil, 200) serve { case "track" :: "empty" :: Nil Get req if req.path.suffix == "gif" => emptyGifResponse } }
That is it! Working and powerful web analytics platform … almost.
Basic working case it is
<img alt="" src="//example.com/track/empty.gif" />
This is enough to know that someone open some page. From request headers we can get all required information about client: Referer, User-Agent, etc.
I almost forget to say about how to prevent browser from caching our image — minimal set of headers:
val headers = List( "Expires" -> "0", "Cache-Control" -> "no-cache, no-store, must-revalidate", "Pragma" -> "no-cache", "Content-Type" -> "image/gif" )
If we want to limit with server-side solution then we can set response header Set-Cookie and identify user each time. But this can be problematic sometimes because, in general, we cannot trust request headers. It was not my case and add small js library to be included in site of our partner.
First we need some good way to load our js library to do not interfere client site. Remember how each big site load its own scripts, create script tags and insert them somewhere in page and load async, let make similar or maybe in the same way:
// i will expose only 2 public objects to host object, // this is because i want to be sure that client can change it without problems if it is required // SuperAnalyticsObject - first exposed object. it contains name of tracking function - second exposed object window.SuperAnalyticsObject = name; // first need to check if library was not loaded already in some way // if it is not then create small mock object that collect tracked events before script loaded window[name] = window[name] || function () { (window[name].q = window[name].q || []).push(arguments) } // now construct script tag and insert it at the top script = document.createElement(scriptTag), firstScript = document.getElementsByTagName(scriptTag)[0]; script.async = 1; script.src = src; firstScript.parentNode.insertBefore(script, firstScript);
Now let make it friendly to js minifier (I am using closure usually — it shows the best results) and save variable values for this small script:
(function (window, document, scriptTag, src, name, script, firstScript) { window.SuperAnalyticsObject = name; window[name] = window[name] || function () { (window[name].q = window[name].q || []).push(arguments) }; script = document.createElement(scriptTag), firstScript = document.getElementsByTagName(scriptTag)[0]; script.async = 1; script.src = src; firstScript.parentNode.insertBefore(script, firstScript); })(window, document, 'script', '//example.com/public/tracking.js', '__it');
With this code snippet I will load my js script and set name of exposed function.
Now about tracking script.
First I need some utility functions to read and write cookie (I want to know that it is the same user) and generate user id.
Id generation is the most simple function there (just funky random string good enough to be user id):
var generateId = function () { return Math.round(Math.random() * new Date().getTime()) + '.' + new Date().getTime(); };
Reading the cookie is also not hard — all cookies available for this page are stored in document.cookie as string name=value;*. Lets parse it:
var getCookieValue = function (name) { var res = [], cookies = document.cookie.split(";"); name = RegExp("^\\s*" + name + "=\\s*(.*?)\\s*$"); for (var i = 0; i < cookies.length; i++) { var m = cookies[i].match(name); m && res.push(m[1]) } return res; };
How to set cookie:
var setCookie = function (cookieName, cookieValue, path, domain, expires) { var removeWWW = function (a) { 0 == a.indexOf("www.") && (a = a.substring(4)); return a.toLowerCase(); }; // i remove www. prefix because i want cookie to be available on all subdomains domain = domain || removeWWW(document.domain); expires = expires || 63072E6; // 2 years in miliseconds by default path = path || '/'; // cookie is restricred in length cookieValue && 2E3 < cookieValue.length && (cookieValue = cookieValue.substring(0, 2E3)); cookieName = cookieName + "=" + cookieValue + "; path=" + path + "; "; expires && (cookieName += "expires=" + (new Date((new Date).getTime() + expires)).toGMTString() + "; "); domain && (cookieName += "domain=" + domain + ";"); document.cookie = cookieName; };
Now lets load our empty gif — it core of tracking. Just load gif:
var loadImage = function (query, callback, src) { callback = callback || function () {}; src = (src || baseUrl) + '?' + query; // Image it is DOM representation of <img alt="" /> set it height and width var img = new Image(1, 1); img.src = src; // for me it does not matter if it loaded or it is not, i just should try to load img.onload = function () { img.onload = null; img.onerror = null; callback(); }; img.onerror = function () { img.onload = null; img.onerror = null; callback(); }; };
That is more than enough to make it working, finally lets describe default query parameters and load events that were generated before script loaded.
// this function i will expose it can be called with any number of arguments first one is required it is name of event var track = function () { var values = Array.prototype.slice.call(arguments, 0), name = values.shift(); // first required parameter it is user id that we take from cookie var q = 'u=' + encodeURIComponent(track.options.id) + // second required parameter it is name of event '&e=' + encodeURIComponent(name) + // third required parameter it is host where event was generated '&h=' + encodeURIComponent(location.host) + // fourth parameter it is just random number to be suer browser will not try cache request '&r=' + (1 * new Date()); // other parameters it is arguments of event any relevant information that i will want to send for (var i = 0; i < values.length; i++) { q += '&a=' + encodeURIComponent(values[i]); } loadImage(q); }; track.options = {}; // try to read user id var id = getCookieValue(cookieName)[0]; if (!id) { // and create one if it is empty id = 'IT-' + generateId(); setCookie(cookieName, id); track.options.init = true } track.options.id = id; // how to expose my function in host object var name = window.SuperAnalyticsObject || '__it'; // load all already generated events if(window[name] && window[name].q) { var len = window[name].q.length; for(var i = 0; i < len; i++) track.apply(undefined, window[name].q[i]); } window[name] = track;
And let generate event:
__it('popupOpened');
That is all, fully functional code to make web tracking. As an improvement, we can record number to tried and request frequency to do not kill our server. There is a lot of things that can be done and improved, but similar code solve my task.
I took most ideas from stackoverflow and from already working analytics scripts.