Unusual Callsign Activity at QRZ

When QRZ first started, the vanity callsign system didn't even exist. Since its inception, it's been pretty much a headache to keep straight in the database. You seem like a fairly intuitive person. Perhaps you might have noticed that the "Previous Callsign" field is one in which the user is allowed to modify at will. So, wouldn't you know that both you and K7NPS have NF7D listed as the previous callsign!

Since K7NPS comes before NF8V in the alphabet, his gets chosen as the winner in a two-way tie.

And since you asked, yes there probably is a complicated software solution that would figure this out by examining certain FCC records which may or may not yield the desired results. Note, I haven't written that one yet.

The simpler solution would be for you to ask K7NPS if he might be willing to give up the "previous" listing since yours is more likely to get QSL's. Just a thought.

Things are never as easy here as they might seem....

 

Since we've started talking about feeds and speeds, note that a typical web page load can result in more than a dozen hits on the server, one each for the HTML, each picture, the javascript, the CSS, and so on. It's busier than you might imagine.

Then, there is the issue of what we call Virtual Memory. In practical terms, Virtual memory is the amount of physical ram you have plus the amount of swap space you have allocated, plus a few more megabytes for shared system files, etc. Our server has 8 GB of physical memory plus 8 GB of swap configured. Swap is a killer, however, because you don't really ever want to use it. You're 10,000 times better off if you can just hold it all in RAM. Once the demands of the programs in your system begin to exceed the amount of physical memory then the excess demand starts going to disk. The problem is that what's on disk needs to be swapped into and out of RAM an an increasingly alarming rate, right up until the point at which the swapping is taking up most of your system's available I/O throughput. This is called 'thrashing'. It's ugly.

The good thing about thrashing, if you can call it that, is that the system won't die but rather it just slows down. At least its democratic about it.

The onset of thrashing can be like the proverbial straw that broke the camel's back. The QRZ web server serves many more than your calculated 6000 hits per minute. After all, it's a 64-bit, quad processor system. at the time of the slowdown, we were running 400 Apache web server processes, each having an average virtual memory allocation of about 40 megabytes (that's 16 GB) and a real memory use of about half that. So you see, it was right on the hairy edge of the precipice. All it took was for some bozo to come along and fire a few thousand nonsensical hits at it to cause the slowdown to start.

In computer performance, something that I learned to manage in my 18 years of systems engineering at Sun, nothing is effective at managing performance as good software. That's my approach to this problem and that's what we're doing right now. I have firewalled off Google, Yahoo, and Inktomi and have put new throttling code into the server that detects when the same IP is hogging the server. In the past 24 hours things have been pleasantly quiet and so it looks like I made the right choice.

So that's probably more than you wanted, but, it was fresh on my mind.

.

 

About the Lookup Counters

I've always liked the lookup counters because they do provide a way for us to see which callsigns are popular. I think that everybody can see this. Sure, some of them are invalid because of abuse, but that's a very small percentage of the 1.2 million calls we keep online. In normal, day-to-day operations, you can tell the difference between someone who has perhaps a few dozen lookups versus someone who has a few thousand. It's a piece of information which I would call 'handy', if even taken only at face value.

Until yesterday, the count would increase by one each time you refreshed a given page. That has stopped now by the use of a tracking system that increments the counter only if you haven't seen it in the past hour. This stops the bozo updates dead in their tracks.

In addition, we are now tracking how many times you look up the same callsign in a given period and if it looks like you're doing continual refreshes we cut you off from that callsign.

On top of that, we also track the total number of all callsigns you lookup in a given period and if you exceed our "maximum lookup threshold", we'll cut you off from the entire database for a while.

In addition to the Google and Yahoo robots cruising the database, and the LCB's (lookup count bozos), there are also a few programs (bots) out there that are doing large numbers of lookups, perhaps some kind of log verification, that were also slowing things down. All of these have been stopped or severely impeded by the new changes in our code. What ordinary users should appreciate is that all of these countermeasures are invisible and never noticed by those who casually browse the database.

Make no mistake about it, we're in a battle to stop people from programmatically accessing the QRZ web pages. Our web pages are designed for people, not robots, and we reserve the right to limit or deny access to those which we consider an abuse of our free service.

.

 

Our forums are still searched by Google/Yahoo/etc. Only the callsign database is blocked from search. I agree about VB searches - they stink.

 

I would love to, but I can hear the screaming protests already. A very large percentage of our visitors never login.