Good and bad news (and some other stuff)

Chaos (ZX Spectrum, 1985)
Chaos (ZX Spectrum, 1985)

First, the good (great, in fact) news: Julian Gollop, author of Chaos, Laser Squad, and the original X-COM: UFO Defense (known in Europe as UFO: Enemy Unknown; note that the recent Firaxis remake combines the two names) has announced that he’s remaking Chaos, and his ideas so far (the previous link goes to his development blog) seem great.

 

The Lords of Midnight (ZX Spectrum, 1984)
The Lords of Midnight (ZX Spectrum, 1984)

And now the bad news: Mike Singleton, creator of The Lords of Midnight, Doomdark’s Revenge, and Midwinter, has sadly died. He was in the middle of remaking The Lords of Midnight for iOS and Android, among other platforms. Chris Wild, who was doing the remake with Singleton, has announced that he will complete the remake, although it will understandably have fewer changes / improvements than intended.

It is interesting to note that the two guys mentioned above wrote, between them, my favorite 8-bit games of all time.

Beyond that… Gaming-wise, I haven’t had a lot of time for playing in the past few months. I returned to The Lord of the Rings Online (LOTRO) “for real”, and I’ve been enjoying the game a lot, although I don’t play it more than a couple of nights a week. The good part is that I’ve been doing it with a couple of friends, and we only play it when we’re together, making it more like a “normal” RPG, instead of an MMO.

Isaac Asimov - Tales of the Black Widowers
Isaac Asimov – Tales of the Black Widowers

As for books, I’ve just started reading Isaac Asimov’s Black Widowers series. It’s interesting to see how Asimov was able to successfully create something that 1) was outside his “normal” thing (science fiction), and 2) has all its (short) stories following the same format, yet without making them repetitive or monotonous.

In terms of personal projects… no big news here. I’ve been working on and off on something for about a year, but it’s not something to be shared with the world. 🙂 Otherwise, I’ve recently improved my online Fantasy Name Generator, which now supports new “types”, including hobbit names, science fiction names, and even modern, English-language names. More to come soon… I hope.

New ‘disposable email’ service: Mail60

Mail60 is a ‘disposable email service’; perfect for receiving confirmation emails from places you don’t trust not to spam you in the future. Mail60 mailboxes are automatically erased after 60 minutes, so you can simply create one, use that email address somewhere, receive the email(s) you’re expecting, and then simply forget about the mailbox. For more detail, see the FAQ.

The idea for it came from reading the comments, about a week ago, in PZ Myers’ wonderful blog Pharyngula, where people were talking about an internet poll they wanted to vote on, but the site required registration, and it was a right-wing paranoia site, so it wasn’t a place they really wanted to be members of. One commenter, then, suggested using a “disposable email service” such as Mailinator. That was the first time I heard of those. I found the idea intriguing, and thought about how I could implement such a thing. It looked doable, so I started programming it in my free time, and Mail60 is the result.

I intentionally wanted to keep this simple, so I didn’t go for features such as “create a mailbox automatically when receiving mail on a non-existent address” or “forward email to a real mailbox for X days and then stop”. Also, since mailboxes are so ephemeral, features such as filters, address books or folders don’t really make sense. And, of course, the only way to allow instant creation of mailboxes with no verification whatsoever and yet prevent abuse was to disallow email sending. But for the most common use I foresee — receiving confirmation emails –, that’s not a problem.

For the techies out there, I’m using PHP, MySQL, Postfix, DBMail, and Hastymail for accessing mailboxes. The (virtual) server runs FreeBSD.

P.S. – yes, this is the “new project” I mentioned a few days ago. 🙂

New project in development

You may have noticed that there haven’t been any news posts here for more than a week. No, I’m not “bored” with my favorite subjects (as if I could); the reason for that is that I’ve been relatively busy with a new project of mine, which I’ve been doing in my free time. It’s still too soon for it to be revealed, but I can say the following: it uses PHP and MySQL (doesn’t everything, these days?), and has to do with email. It’ll provide a service that isn’t exactly novel (though a lot of non-tech users don’t even know such a thing exists), but I don’t think I’m doing it exactly like the “competition” does, which means it’ll be unique in some ways. I hope.

I hope to have something to show to the world in a week or two. But I’ll try to write a few new posts here, until then. I have a draft of the second entry in the Conversion Wars series almost finished (and it has been that way for a while, now), so that’ll probably be the next one.

The “7-day-long Bing trial”: results

About a week ago, I mentioned that I would be trying out Microsoft’s Bing as my default search engine in my main browser (Firefox 4.0) on my main computer, for at least seven days. So, the week has passed… what are my impressions of Bing, when I had used Google for years?

A Bing search

Continue reading The “7-day-long Bing trial”: results

AWStats + MRTG search engine referrals script updated

Just a brief note: I’ve just updated my script for graphing referrals from Google, Bing and Yahoo!. I intended it to reset each first day of a month, so one can see whether each month is “better” than the one before… however, it didn’t handle stats without any referrals at all from those search engines, which is likely to happen every time the current month changes. It’s fixed now.

AWStats tip: removing ‘chromebar’ from search keyphrases

Being still less than a month old, it’s normal that I keep looking at this blog‘s stats quite often, and one of the most interesting bits — especially if you care about SEO — is the top search keyphrases list. I use both AWStats and Google Analytics for this site, and, concerning the former, I had been curious for a while about why the top search leading people to my blog was apparently for “chromebar“.

Now, I’m pretty sure I had never mentioned that term on this blog, so I thought AWStats might have been doing something wrong. A quick grep on my logs, and where “chromebar” came from became obvious: it’s from the StumbleUpon add-on for Google Chrome. The whole combination was necessary; StumbleUpon users with other browsers weren’t triggering anything like this.

Now, here’s what the “referrer” part of a hit from StumbleUpon, using Chrome with the SU add-on, looks like:

"https://www.stumbleupon.com/toolbar/litebar.php?device=chromebar&version=chromebar%202.9.8.1&ts=1301274701"

While a hit’s referrer from SU with another browser looks like this:

"https://www.stumbleupon.com/refer.php?url=http%3A%2F%2Fwinterdrake.com%2Fbad-comic-panels-3-its-a-membership-card-in-a-subversive-communist-front-organization%2F"

See the difference? The Chrome version has a query string: ?device=chromebar&version=chromebar%202.9.8.1&ts=1301274701 , while the other one doesn’t. And “chromebar” appears at the beginning of that query string.

Now, StumbleUpon is listed as a “search engine” in AWStats, in a file called lib/search_engines.pm , and that file optionally specifies which part of a query string (e.g. “q=“) from the referrer is the actual search. For some search engines, however, no such part is specified — meaning that they don’t provide it in the “referrer” part of the hit. Such is the case with StumbleUpon, where the query string part is an empty string.

But there seems to be a bug (or maybe it’s an intended feature?) in AWStats here: if no part of a query string is specified, and yet there is a query string, AWStats seems to use the first part it catches. As you can see above, that was ?device= , and it was always set to chromebar.

The easiest way around this problem (I don’t know if the AWStats authors will consider this a bug or not; I’ll try to report the problem in the near future) is to edit lib/search_engines.pm and add a non-existing query string part to StumbleUpon. Open that file with a text editor and look for this line:

'stumbleupon','',

and change it to:

'stumbleupon','qwerty=',

Presto! No more “chromebar” entries in your stats in the future. (It won’t delete current entries, though, unless you clear your AWStats cache files for that month and generate new stats.)

More on Google vs. Bing, and the “7-day-long Bing trial”

After my last couple of posts looking at the comparative hits between Google and Bing and investigating the reasons for the huge discrepancy (Bing bringing me a puny 2-3% of Google’s hits), I’m still curious about this subject. So far, it appears that Bing brings me 2-3% of what Google brings because it has 2-3% of the users Google has… which seems too low a number to me. Yes, “google” as a verb has entered everyday language (“I’ve just googled for it”), and it’s the default browser on most non-Microsoft browsers (though I’ve heard that a couple of them changed their defaults to Bing recently), but, well, Internet Explorer is still widely used (unfortunately), and it certainly defaults to Bing.

Further investigation may be in order, but, unless both companies reveal their search volume, there’s no way to know with certainty. We can look at other statistics such as Alexa, which are certainly far from definitive, and which lists google.com as #1 worldwide (and remember that there are many other “international” Googles), and Bing as #20 worldwide (#14 in the US). Google (again, just the “.com” version) is listed as having a “reach” of 49.05000, while Bing supposedly has a “reach” of 4.77600. This puts Bing as 9% of Google, but if you add in the international Googles, the 2-3% actually sound like a plausible result.

Still, all of this, and we haven’t actually asked an important question: how good actually is Bing, for users? Are people — even Internet Explorer users — using Google out of habit / popularity, or is Bing in fact a worse search engine — say, providing worse, less relevant search results, giving you exactly what you want at the #1 position less often, or not even finding something that exists when Google does? Only one way to find out: my 7-day-long Bing trial. (Actually, 8-day-long, since I started it yesterday, but I think that ending on a Saturday is nicer than on a Friday).

Until April 2, I’m going to be using Bing as the default search engine on my main browser (Firefox 4), and I’ll post my impressions after that time. If you’re interested in SEO, or simply curious, I’d suggest you try the same — learning is always good, isn’t it? 🙂 To do it in Firefox:

  • for the search engine box (typically on the right), it’s easy: just click on the little “down arrow” in the box, and choose Bing.
  • for the URL box (typically on the left), go to about:config, accept the so-called “risk” if it’s your first time doing so, search for keyword.URL, and change it to “https://www.bing.com/results.aspx?q=“, without the quotes. Yes, it ends with an equal sign. To switch back to Google, change it to
    https://www.google.com/search?ie=UTF-8&oe=utf-8&q=“. I believe this is unchanged if you’re using Firefox 3.x instead of 4 (though you really ought to upgrade, Firefox 4 is great).

Search engine hits: Google vs. Bing (part 2: positions)

In my previous post, Search engine hits: Google vs. Bing (part 1), I showed how three of my sites — a recent US-hosted blog in English, a popular Portugal-hosted forum in Portuguese, and a search engine-weak US-hosted aggregator in English — had a much greater proportion of search engine hits from Google than from Bing — about 100 Google hits for each 2-3 Bing hits. Naturally, the next step would be to investigate whether this is just a matter of quantity — in other words, Bing has 2-3% of the users Google has –, or whether there are other factors affecting this, such as the three sites having better rankings in Google than in Bing. This, then, is the subject of this post.

The method I’ll be using is the following: for each of the same three sites used in the last post, I’ll be searching for a number of phrases that I think should, reasonably, lead a user to the site. I won’t look at my stats to get the top phrases from there; as most hits come from Google, I’d just be using phrases that I already know work for that engine. And, for each phrase, I’ll note the position my site came in — it at all.

Remember that a lower number is better. Also, a caveat: I am not really testing how “good” each search engine is, only how good it is at sending people to my sites from appropriate queries. I have, however, avoided “gaming the system”; the queries are reasonable and not “custom-designed” to lead people to where I want, and the pages / sites are relevant to them.

Note: all queries were as written below; i.e. without quotes.

Results

Winterdrake:

For this very blog, I searched for things I know I posted about — but not the exact titles. Instead, I tried strings that I myself would use if searching for those subjects:

[table id=1 /]

DragonBall-PT:

For that MyBB forum with mostly Brazilian and Portuguese users, I didn’t go for particular posts, just some strings that a Portuguese-speaking person could possibly use, and I counted the first forum post that appeared. I tried to diversify; hence the “dragonball”, “dragon ball” and “dbz”. I also tried both Google.com and Google.pt, which seem to prioritize pages differently (perhaps because that site is hosted in Portugal); Bing.pt, however, was exactly the same as Bing.com in terms of results, only with the user interface in Portuguese.

[table id=2 /]

Planet Atheism:

And, finally, my FeedWordPress-based blog aggregator. As I explained in the previous post, PA doesn’t ever show posts individually; there are only the main page (and, page 2, page 3, etc.), author pages, and archives by date. On all of these, the post links go to the original blog posts; therefore, search engines treat all of PA as non-original content (and this is by design); because of that, searching for something in a particular post is mostly useless. My alternative was to search for more general terms:

[table id=3 /]

(1) category pages that included the post were indexed, but not the post itself
(2) not in the first 100 results, at least

Conclusions

For Winterdrake, Bing was a pleasant surprise, winning 5.5-0.5. Not only did it apparently index every post in the site, even some very recent ones, but it also returned a first page result for everything I threw at it — including two “number ones” that Google doesn’t even seem to currently index, for some reason. Therefore, a Bing user actually has a greater chance of getting to Winterdrake — assuming he or she is interested in something on it — than a Google user. In other words, Bing wins on “quality” — but still only sends me 3% of the users Google does. Are there really so few people using Bing? I had figured something like a 10-20% market share, not 2-3%…

As for the few non-indexed posts on Google, when the category pages have been indexed after the posts were written, I assume the reason for that is the fact that Winterdrake is still very new, and has almost no incoming links. In a couple of months, I expect that new posts will be quickly indexed by Google, as Bing already does. Still, there’s currently an advantage for Bing here.

DragonBall-PT gets mixed results. Compared to Google.com, Bing wins 3-2. Compared to Google.pt, however, Bing loses 1.5-3.5, meaning that “which” of the Googles people use is an important factor. Still, if we assume that most people use “their” national Google (I don’t, but I’m weird…), Google wins this time, and that was reflected in the results on the last post: Bing sent me only 2% of the people Google (all of them put together) did.

Finally, for Planet Atheism, Bing is a big disappointment, losing 0-3 to Google. 🙁 Either because it “hates” non-original content to a much greater degree that Google does, or because it is less “intelligent” in adapting to / understanding similar terms, it failed to show PA in the first 100 results for two thirds of the search strings I used, while Google did great with two of them, and even had a reasonably decent result for an incredibly competitive, 1-word term such as “atheism”.

Another thing I noticed is that most of the first couple of results pages in Bing for my atheism-related queries actually showed anti-atheism blogs (“atheism sucks”, “stop atheism”, and several others). That’s like searching for “democratic party” and getting mostly Republican propaganda pages. 🙂 I don’t want to sound paranoid… it’s probably just an algorithmic thing, instead of there being a less-than-honest religious programmer at Microsoft… 😉

EDIT: I’ve just noticed that the three tables above look quite bad on the RSS feed. If you want them more readable, please see the original blog post. I apologize for the inconvenience.

Search engine hits: Google vs. Bing (part 1)

Having been interested in SEO for a long time, one of the things I naturally do is look at how many people are coming from the several search engines. While many people seem to care only about hits from Google, I think that 1) monopolies are bad, and 2) precisely because so many people optimize only for Google, there may be an “untapped market” of people who use other search engines. So, I’ve been doing a little investigating, and here’s what I found. Today, I will only be comparing Google with Microsoft’s Bing; I may look at Yahoo! or Ask.com some other time.

So, let’s begin with this very blog, Winterdrake. Hosted in the US, it’s a very young site, having only launched on March 3, 2011… which means that it’s just 22 days old today. Naturally, one shouldn’t expect a lot of search engine hits on such a recent site (and there aren’t), but it allows us to look at yet another interesting factor: which search engine is quicker to index and send hits to new sites? Let’s look at a (terrible-looking; I’m no graphic designer, and it shows) chart:

Winterdrake - Google vs. Bing hits, March 2011
Winterdrake - Google vs. Bing hits, March 2011
Bing hits: 3.36% of Google hits

Not very good in terms of Bing hits, is it? Let’s look at another, more popular and established site: DragonBall-PT, a Portuguese-language forum, hosted in Portugal, launched in 2007, with mostly Brazilian and Portuguese users, which has a few thousand hits per day:

DragonBall-PT - Google vs. Bing hits, March 2011
DragonBall-PT - Google vs. Bing hits, March 2011
Bing hits: 2.12% of Google hits

As you can see, while, as an absolute number, there are a lot more hits from Bing (a little below 2000), the proportion in relation to Google is even worse.

This can be explained in several ways. First, Bing seems to be more optimized for the US and/or English language sites, while Google appears to be more “international”. Another possibility, of course, is that almost no one in Portugal or Brazil uses Bing. But, to discount the “optimized for the US/English” factor, let’s look at a third site, one that is 1) relatively old, and 2) in English, and hosted in the US.

Planet Atheism is the world’s top aggregator of atheism-related blogs. It’s (by design) not good for search engines, as all its content is duplicated from the member blogs (and every member either asked to join or accepted an invitation; no one is aggregated there without permission), and PA doesn’t show individual posts ever. Click on a post title, and you are taken to the post on the original blog. In other words, PA is by design “below” every single one of its blogs in terms of search engine positions; most hits come from people who actually search for PA itself, or for “atheist blog(s)” or something like that.

On the other hand, it’s an established site (launched in 2006), hosted in the US, and in English. Let’s see how it looks like:

Planet Atheism - Google vs. Bing hits, March 2011
Planet Atheism - Google vs. Bing hits, March 2011
Bing hits: 2.20% of Google hits

Ouch. Not very good for Bing, is it?

So, we’ve shown that Bing is sending about 2-3 hits for every 100 sent by Google, a very small percentage. But now for the million dollar question: why? Logically, there are two possibilities:

  1. Bing is performing “worse”: that is, either doesn’t index sites as well as Google, or it is giving them worse positions — possibly out of the first page for results Google shows in the first 10 results; or
  2. Bing is actually performing “as well” as Google (or possibly even better), but it has only about 2-3% of the users Google has.

Note that, in either case, Google has a lot more users; the question is whether, for the same number of users, Bing is performing worse, better, or the same. And I think we can find out… but this post is getting a bit long, so I’ll leave that for Part 2.

Ah, spammers, spammers…

I’m a bit torn. I’ve been looking at the pathetic attempts at automated spam in this blog, and my “I like an intellectual / technical challenge” side just wants to write a post detailing what they’re doing wrong and how they could easily make their tools much more effective.

On the other hand… we’re talking about spammers. The scum of the earth. The only creatures in the world that make lawyers and politicians look like decent, lovable human beings. Now, some may argue that what I could suggest in a couple of minutes wouldn’t be exactly rocket science, that they’d surely have thought of it already… but my point is that they haven’t. Both their methods and the comments themselves that they try to post are terrible and ridiculously easy to detect; anyone with half a brain could do a lot better.

In fact, if they’d just… ah, crap. I just can’t. It’d be like handing a loaded gun to a kid. The biggest asshole-ish jerk of a kid, and a brain-damaged, glue-eating one as well, but still, in some ways, a kid. Who could do a lot of damage with it. But, to get some idea, my “brilliant” suggestion for making comment spam tools much more effective — which would not only hugely improve the odds of beating Akismet, but would also be accepted by many less attentive blog owners — would take… a line and a half of text. This is not because I’m some sort of genius… it’s because they’re morons.

Interesting, email spams, while still typically primitive, are actually much more advanced than comment spam tools. There’s actually a bit of thought put into them. Comment spam tools, on the other hand, are the equivalent of a burglar wearing a Beagle Boys-like mask and a stereotypical black and white striped prison suit, and going in broad daylight from house to house, knocking on each door. 🙂