Graphing search engine referrals with AWStats and MRTG

From my previous post, AWStats tip: creating static pages (and why it’s a good idea):

for instance, I’m currently using MRTG to plot a graph of Google and Bing referrals, using the AWstats-generated static pages as input

So… anyone curious? πŸ™‚

Note: again, beware of word wrapping below. I’ve added an empty line between any “true” line of text; if you see two or more lines together, it’s supposed to be just one.

cat /root/bin/



GOOGLE=`grep \>Google\</a\>\</td\>\<td\>
/var/www/htdocs/AWstats/ | awk -F"<td>" '{print $2}' | cut -d "<" -f 1`

if [ -z $GOOGLE ]; then



echo $GOOGLE

BING=`grep \>Microsoft\ Bing\</a\>\</td\>\<td\>
/var/www/htdocs/AWstats/ | awk -F"<td>" '{print $2}' | cut -d "<" -f 1`

if [ -z $BING ]; then



YAHOO=`grep \>Yahoo\!\</a\>\</td\>\<td\>
/var/www/htdocs/AWstats/ | awk -F"<td>" '{print $2}' | cut -d "<" -f 1`

if [ -z $YAHOO ]; then



echo $(($BING + $YAHOO))

uptime | awk -F"up " '{print $2}'

uname -n

EDIT in April 1, 2011: fixed the script so that it deals with non-existing entries (typically on the first day of the month).

By the way, this is a FreeBSD box; that’s the reason why bash is in /usr/local/bin. In Linux, it’s probably in /bin.

The mrtg.cfg file is mostly standard: it invokes the script shown above, and labels “I” as hits from Google, and “O” as hits from Bing and Yahoo! put together.

If you want to see what the result looks like, look here.

If you’ve been paying attention, yes, both values will reset to zero the first day of every month. That’s exactly how I want it: I want to see if every month it “rises” faster than the month before, or slower, or whatever. πŸ™‚

4 great WordPress plugins

I have been blogging with WordPress since 2004, and during all these years I’ve maintained several blogs; some of them exist to this day, some were eventually shut down, and some “lie dreaming until the stars are right.” I hadn’t started a new one in quite a while now, and so, when I created Winterdrake a couple of weeks ago, I did what I usually do in terms of which plugins to install: instead of just copying the /wp-content/plugins/ directory from an existing blog, I start afresh. I see what plugins are the most popular and/or have the most positive votes, both for each requirement (e.g. one to edit your comments, one to show related posts, one for caching, one to improve SEO, and so on) and in general: by doing the latter, I can (and do) find something extremely useful that I didn’t even know existed.

The result is that this blog currently has an almost completely different set of plugins from my other blogs (which will eventually be changed so they use most of these ones, too), and I’d like to share a few of them, that old WP bloggers such as myself may not even know exist:

  • WordPress SEO: forget All-In-One SEO and other SEO plugins: this one is so much better, it’s not even funny. It also takes care of XML sitemaps for you, so no need to install a separate plugin for that.
  • W3 Total Cache: this one is relatively well-known, so this may not be news to you, but if you’re using any other caching plugin (e.g. WP Super Cache), switch to this one. (And if you’re not using a caching plugin at all, you should.) Besides the incredible improvements in terms of speed, you can also use it to minify HTML, CSS and JavaScript automatically, and keep static files automatically on a CDN, or simply in a “” virtual host (which is quicker since it won’t have PHP and such enabled, and the fact that it’s a different host allows browsers to open more connections, as they by default limit the number of simultaneous connections to each host), accessible by FTP. I’m using the latter option on this very blog.
  • Fluency Admin: doesn’t affect your readers directly, but makes your blog’s Administration section so much more pleasant to use and look at, that I almost cringe when I enter the same section for my other blogs where I haven’t installed this plugin yet.
  • Jetpack: a relatively recent “package” of useful plugins distributed by WordPress, so that self-hosted blogs (such as this one) can have several interesting features until now unique to blogs hosted on WordPress.Com. Currently I’m only using two of them: the WordPress.Com stats (which have several advantages over Google Analytics, though I’m using that as well), and After the Deadline, which adds grammar and style checking to the already existing spellchecker, and usually spots something I’ve missed, such as using a short word such as “is” twice in succession.

Hope you find this useful. Two weeks ago, the only one of these I knew about was W3 Total Cache (and that one I’m already using on my other blogs), and I’m in love with the other three right now. πŸ™‚

The (Eventual) End of SEO?

I’ve just read a thought-provoking article, Has SEO Peaked?, which links to another article called Someday, the sun will set on SEO β€” and the business of news will be better for it, and I’d like to share my thoughts on that subject.

One of the questions raised in the above articles is this: if search engines were perfect, then everyone could use the “I’m feeling lucky” option on Google (which takes you immediately to the search’s first result), or its equivalent on another engine. After all, the first result would be the best one, wouldn’t it? The fact that very few people ever do that is proof that people don’t trust search engines, because they often fail. Either they put a bad quality page on top, or one that isn’t really what the user was searching for.

But what if they didn’t? (more after the break)

Continue reading The (Eventual) End of SEO?