ad: SteppIR-1

Help for LOTW

Discussion in 'Discussions, Opinions & Editorials' started by W5INC, Jan 2, 2013.

Thread Status:
Not open for further replies.
ad: L-HROutlet
ad: l-gcopper
ad: l-rl
ad: l-Waters
ad: L-rfparts
ad: l-innov
  1. W5INC

    W5INC Ham Member QRZ Page

    It looks like the ARRL is moving ahead with it's LOTW program. Upgrading to newer software that should decrease the time it takes to upload logs to the web site. I wonder if the ARRL in it's planning stages for this project, took into account the number of logs that would be downloaded to the site at any given time?

    From looking at the que time for downloads @ LOTW it looks like there are over 32,000 logs waiting to be downloaded to the FTP site. That is why it takes close to 11 days for your downloaded logs QSO information to show up on the web site. As LOTW becomes even more popular these numbers should be ever increasing, IMO. I'm not an IT guy, but 32,000 logs, containing mostly "text" information and not large graphic files should be able to be sent to the FTP site quicker then that. Can any IT gurus in the Forum here shed some light on the download speed attained in today's website World?
  2. KF6ABU

    KF6ABU Ham Member QRZ Page

    Yes. It's a lot more complicated then that. There are many processes that run for each entry in each log.
    Comparing it to the download speed of an FTP is not relevant.
  3. N7SMI

    N7SMI XML Subscriber QRZ Page

    The backlog is not due to bandwidth - it's due to processing speed of the QSOs. There are currently 465 million QSOs in their database. Those 32,000 log files in the queue hold another 9 million QSOs. Each QSO has to be compared to the 465 million already in there and then added to the database appropriately. Admittedly, this is not a terribly complex endeavor, but they have indicated that there are hardware constraints that are limiting their processing throughput. New hardware is to be tested and installed in the coming weeks and it should resolve the backlog... for now.
  4. AB1QP

    AB1QP Premium Subscriber QRZ Page

    LoTW was struck by a number of adverse events over the last 6 to 8 weeks that apparently have brought it to its knees.

    First, it allegedly ran out of storage capacity, crashing the system, without alarming or otherwise yelling for help( hard to believe, isn't it?)
    Then in the process of restoring the system, the admin discovered an existing bug that was allowing submitted logs to be overwritten before processing( yes, true loss of data)

    The ARRL, in a loosely worded message, said that lost logs should be resubmitted, which was no doubt interpreted incorrectly and caused an significant increase in traffic. Can't really blame the users, it would be difficult to tell if your logs were among the lost or not. Then came a few large contests, and end of year log catch up........blooey.

    Since then, the system has been consistently losing ground - input exceeding processing, or in other words, the delay between submitting a log and it being processed, growing longer over the last 5 or 6 weeks).

    ARRL has been exceedingly tight-lipped on the root cause, and claims the hardware upgrade in progress now will remedy the issue.
    I am sure things will improve, eventually, but I think the root cause and true fix will not be revealed.

    Note also, that while improvement is promised, there is no level of service specified - so all we can expect is that the delay, now at 11 or 12 days, will be , well, less than 11 or 12 days after the fix.

    I have written my ARRL division director, asking him to pass on my feelings on the care and feeding of LoTW , and urge other interested users to do likewise.
    In fairness, it needs to be mentioned that we are talking about log processing time - not other LoTW functions, such as searching, reporting, or award tracking, accreditation, or processing. Those seem to be working just fine.
  5. KF6ABU

    KF6ABU Ham Member QRZ Page

    11 or 12 days, to get a QSL for free is awesome. It's a great service.
  6. AB1QP

    AB1QP Premium Subscriber QRZ Page

    As it gets worse, I suppose that will make it greater.
    Aim low.
  7. WH2HAO

    WH2HAO QRZ Moderator QRZ Page

    ABU, I think you are one of the LoTW workers. I noticed that no matter what anyone states as fact against LoTW , you will defend it. If I said it takes 120 days to confirm a QSO, you would say, that's good, and it's free !! lol. Just an observation, no judgement. Just an observation.
  8. KF6ABU

    KF6ABU Ham Member QRZ Page

    Yeah, anything faster then the bureau, and most direct international mail round trip, for free, processed automatically by my log book is super low aim.
  9. KF6ABU

    KF6ABU Ham Member QRZ Page

    No problems, dude. It’s a pretty accurate observation.

    120 days would be super lame, and uncalled for. With that being said.. you pretty much nailed it. :p LoTW saves so much time. I don’t have to have cards, or send them, or get envelopes to send them in, or stamps, or dollars, or rectify my log book to my received cards, or have the cards checked by a card checker.

    Anything under 30 days is pretty cool in my book. I think it's a great service, and far easier, and cheaper than sending QSL cards and a dollar to someone.

    I think the difference between me, and a lot of people who have cry baby attacks is I know what technology is and its use and deployment on a very large scale, and how it works beyond hooking up a printer to my computer, or configuring some settings. In that respect I know what a large project LoTW is. I also know they don't have much IT support. I also know it’s a free of charge service. With that in mind, I am able to make logical, educated conclusions and assumptions about LoTW.

    Some people are some sort of jr. tech support and feel they can make valuable or valid assumptions, and that’s cool and all, but that doesn't make them valid. Those are who most of my replies are geared towards. Also towards those that expect a program with no price point to work to their specifications - or else, even when its already far superior to anything else currently available.

    AND........ No, I dont work for ARRL / LoTW / get anything from them in anyway other then their product like everyone else does.
  10. WH2HAO

    WH2HAO QRZ Moderator QRZ Page

    Very well stated..
  11. KE3FG

    KE3FG Ham Member QRZ Page

    Can you explain to us how eQSL processes our QSOs in seconds?
  12. KF6ABU

    KF6ABU Ham Member QRZ Page

    I can. It doesn't do everything that LoTW does - which takes longer.
  13. KE3FG

    KE3FG Ham Member QRZ Page

    If you are referring to awards, eQSL has eDX100, eWAS, along with a few others that don't interest me. Does LOTW have a lot more subscribers than eQSL? You haven't demonstrated to me that there should not be similar processing times for the two systems.
  14. KF6ABU

    KF6ABU Ham Member QRZ Page

    I was not.

    I wasn't attempting to, just like I am not going to explain database terms to you, it's pointless. Basing it on subscribers, since eQSL has more, shows your lack of realistic knowdledge, which is perfectly fine. If you knew how it worked, you would not have any questions unless you are trying to be difficult. If you don't know how it works, or its differences, no amount of qrz forum LoTW tutorials are going to impart that knowledge on you, because it's probably impossible to give you that.

    I'm sorry you don't know how it works, or have the technical expertise to know the differences. I can assure you the differences are vast between the two systems, which accounts for processing time. If that's not a sufficient explanation, a MCDBA would be a good place to learn the basics.
  15. KE3FG

    KE3FG Ham Member QRZ Page

    Parts is parts, and BS is BS.
  16. KF6ABU

    KF6ABU Ham Member QRZ Page

    I guess I didn't write it plainly enough, Jim. I can re-write it in simpler terms if you would find that helpful?
  17. KE3FG

    KE3FG Ham Member QRZ Page

    If all you can do is insult people who ask questions, then you don't know what you are talking about.
    Both eQSL & LOTW are database systems that process QSO data. One does it in seconds, the other takes weeks to do very similar functions.
  18. AB1QP

    AB1QP Premium Subscriber QRZ Page

    To expand just a little..... LoTW has a completely different and much more stringent set of authentication/security processes using signed tq8 certificates, and that adds considerable overhead.
    The sequence of processing files is totally different, but the significant thing is that eQSL can discard dupes as part of the upload process, LoTW apperntly needs to ingest the file before it can do that.
    I understand they have commisioned a project to improve the process, but that is incidental to the current crisis.
  19. W4PG

    W4PG Super Moderator Staff Member QRZ Page

    I've never really understood the complaints about LoTW being "slow." Slow?? Compared to what??

    I just got a batch of QSLs from the BURO that contained cards for QSOs over 2 years old!! 2 YEARS!!! Shall I write my ARRL director and threaten to cancel my membership because of the lousy service?? I mean, 2 YEARS!!!

    You mean I can get a confirmation of my DX contact in a few days now via LoTW, but that's not good enough?? Obviously, I'm missing something.

    Then again, for the life of me, I cannot figure out why DXpeditions often hold LoTW uploads for 6 months or more . . . are there folks out there who JUST HAVE TO HAVE a quick confirmation, such that they will gladly send many times the cost of a QSL card just to get one back sooner than waiting 6 months for LoTW?? THAT is something I cannot understand at all!

    My LoTW hit rate is MUCH higher than eQSL and I upload all contacts to both. I'm primarily a DXer, so if eQSL is so easy to set up, why do I get so few confirmations compared to LoTW?? I think the answer is clear - eQSL doesn't count for much of anything!! While I do like to get the digital pictures of eQSLs to display, it ain't 5BDXCC or regular DXCC, which are ARRL awards and the standard for DX awards.

    I'd go on, but I have to go write my ARRL director about the slow response of the buro!! I mean REALLY!!!

  20. KF6ABU

    KF6ABU Ham Member QRZ Page

    One does not even need technical expertise to know part of the difference between their processing time.
    You could ponder, if you choose to seek answers, how many dxpeditions upload to eQSL vs LoTW, or how many people submit contest logs to eQSL vs LoTW, and how that would affect queue processing. Of course, such a simple quandary has probably already occurred to you - but that does not explain why you can't grasp LoTW's back log.

    I didn't insult you, you sought answers, and then answered with snark because you are not knowledgeable about the differences. But instead of actually trying to find that out, you chose to play the I am right because I don’t really know what I am talking about, but since you won’t explain databases to me, you are a super big meany head for allowing me to display I’m not really sure what I am talking about.

    I am not sure of anything other than learning “how stuff works” would make you feel better, and again, It’s rather pointless to attempt to explain that to you in the constraints of an internet forum. I tried the short answer, which didn’t help you, and the long answer is too technical most likely.
Thread Status:
Not open for further replies.

Share This Page

ad: elecraft