tracking authenticated Drupal traffic in Varnish

As part of the performance and load testing suite I am building for a client, they asked that it accurately reflect the percentage of traffic which is logged in. Sure, no problem, we told them. But how do we get this? Really, their setup provided no easy way to collect this info (multi-site with ~200 sites). The Google Analytics module *could* be used, as it provides custom variables and we could then use a token with the users role. But, that would require building out a feature, deploying it, and enabling. We asked them for statistics on how many users post comments each day (primary authenticated action on the site), but they couldn't give us reliable data. Then, the 'easy' solution came to me. Every last bit of traffic to these sites flows through two Varnish servers. All authenticated traffic in Drupal has a session cookie. Why not just add this to the logs?

Varnish 3.0.3 to the rescue. Using a little bit of logic and the std.log() function, we now have the data we need in our logs (FWIW, we use Logstash\Redis\Elasticsearch\Kibana for aggregating all of the web traffic data). Implementing this is actually really easy.

In your vcl file, add import std; near the top. This imports the 'standard' module for Varnish.

Then, in the vcl_deliver section, add :

if (req.http.Cookie ~ "S?SESS[a-f0-9]{32}=([a-zA-Z0-9\-=]+).*") {
  std.log("sessionid:true");
} else {
  std.log("sessionid:false");
}

Note that we are not logging the actual session name. Though that would give us insight to the actions of a specific user, it is also a bit of a security risk, if an unauthorized person gained access to our logging tools.

We use varnishncsa for formatting the logs, but with a custom JSON format that elasticsearch likes. For us, it was adding a key:value pair , \"sessionid\": \"%{VCL_Log:sessionid}x\" to the JSON structure, yours may be as simple as appending %{VCL_Log:sessionid}x to the LOGFORMAT variable in /etc/init.d/varnichncsa

Hat tip to http://lassekarstensen.wordpress.com/2012/06/15/varnishncsa-and-std-log/ for the original idea.

Comments

Excellent post, mate! Thanks for the useful information

Add new comment