Goodbye FT8, Hello Olivia, The MAGIC Digital Mode For HF!

IP isn't a base-band modulation scheme. It's a data protocol that is encoded above all that.

The underlying transport media (e.g., Ethernet, 802.11) absolutely require clock recovery from the signaling line. Just go read up on the specifications. The receivers in those schemes depend 100% on clock recovery, and the ethernet modulation (various Manchester encodings and its derivatives) are so dependent on clock recovery that the clock runs with very strict zero-crossing requirements -- even much more than ham encodings like Baudot RTTY.

You're kinda making my point for me.

Part of the problem is that so many people have never had to work on data modem implementations, and so they have never seen what happens when the clock recovery is a little off, or starts to drift. The result on the effective minimum SNR needed for reliable decoding can be catastrophic. So yeah, the clock recovery is fully half the battle. Like I said, disconnect your computer from its time sources, then set your clock (seconds included) to a truly random time, and then try to make QSOs with FT8 and you will see how critical clocking is to signal recovery.

 

Yes, it absolutely does. The problem is that you apparently don't understand how serial data communication modulations work. That's unfortunate, but it doesn't make you correct.

Think the problem through instead of just making random statements. Every data modulation has both clock and data. The data are the states (the ones and zeroes) and the clock determines where modulation takes place (the transitions between ones and zeroes). The clock used in the receiver has to be aligned with the transmitter data in the same places chosen by the transmitter, or the decoding will be unreliable or unusable. There are only two ways to do that -- either recover the clock from the signal, or communicate it through some other data channel.

You say you don't need to have an external communication channel to synchronize your clock, but you described exactly that in prior posts. Stop and think... where did the value of the clock in your PC come from? Once you answer that honestly, you will understand that I am correct.

 

Been playing with FT8 the last few days. It is so crowded it seems like there is a need for a local US and a DX frequency.

 

Certainly, and GPS should give you a clock source that is just as good or better than WWV, and about as good as any NTP source you could choose from the 'net.

 

Can you post links to these papers? From what I have read, all the JT modes include timing information. For example, in Joe's paper, "EME with JT65":

http://physics.princeton.edu/pulsar/K1JT/WA50_June05.pdf

Even the latest, FT8, uses syncronisation, but using a different method that I have been learning more about called "Costas Arrays", usually used for radar or sonar reception:

https://wsjtx.net/home/ft8-mode.html

https://en.wikipedia.org/wiki/Costas_array

Where does Joe describe these modes as being designed to recover only "half" of the data? Time periods and frequency are reciprocals, we all need a frequency "source" to put us on the air at the right frequency to have an any type of RF communications (realistically).

 

The word "half" is obviously my word. I have been looking for the older paper that had the explicit footnote, but my link to it no longer works. When I find a live link, I will post it.

I have quoted the relevant portion of that document in other posts in earlier discussions on this subject, so if you search QRZ, you'll find the quote.

What I got out of the part that you quoted was that the decoder was correcting for Doppler or other such frequency distortion, since it is an EME paper. Scattering a fixed data pattern through the data envelope would seem to make sense for that purpose, and in that application. It's not clear from that one paragraph what exactly the algorithm is doing. If the demodulator was able to do clock recovery, there would be no need to synchronize the PC clock with true UTC. Each transmission could simply begin any time subsequent to the prior transmission from the other station.

 

I've been using FT8. I like it, but like we all know, it's just JT65 on speed, and that's good if you just want quick contacts. I've tried Olivia, but it is very slow. PSK is far more efficient. Particularly BPSK63 and 125. However, I'm wondering why are folks not using FSQ? If you want to ragchew, this is an ideal mode for one-on-one or one-on-many.

K5USB

 

You're assuming that WSJT-X couldn't easily be written to recover clock from an FT8 signal - which I don't believe is a valid assumption. Using an external clock source is a stroke of genius that keeps the multiple FT8 transmissions synchronous and allows you to decode 20 or more at time, instead of the one asynchronous 1 kHz wide Olivia transmission.

 

Not at all. The more I re-read Joe's papers, the more I think I see why he wanted to try the approach he did. I think the limiting factor that might be preventing or delaying the feature you describe is CPU power. If I understand correctly, what he appears to be doing is to use the PC clock as the actual "clock," and then he runs a whole bunch of math over the signals to fine-tune the clock alignment so as to achieve the "best fit" of the data received to a valid message. I'm still looking at his code (FORTRAN! ), trying to find the main pieces to confirm my suspicions.

Again, I'm not saying it's "bad" or anything like that. In fact, it's a very unique approach to decoding. I also find all you overzealous FT8 people very useful as beacons for antenna testing.

All I'm saying is that comparing modes needs to be apples-to-apples, and this mode isn't an apple. Well, at least that's all I was trying to say, but then the FT8 religious nuts start yelling at me, and I have to be more ... detailed.

Similarly, the only thing preventing this is the CPU power of the machine, and the support in the software. PSK modes have been doing multiple, asynchronous decoding with fully independent clock recovery for almost 20y now.

 

From the JT65 paper, "The JT65 Communications Protocol" by Joe Taylor:

The "clock" is recovered from the signal - sync is established by detecting the pseudo-random sync pattern, and using it to chop it up in the 63 channels. The word "clock" only appears once in the paper, and is shown in the above fragment, and refers to using the sync pattern to correct for errors in "frequency calibration and clock settings." Let me stress that - the embedded sync pattern is used to correct for clock errors. The clock is recovered from the signal. Not "out of band". Frequency error, clock error, Doppler shift is all accounted for in the protocol and decoder.

The PC clock issue is around convention - knowing when to transmit, and when to receive when dealing with signals below the noise floor. Much like the quite quiet periods used on 500kHz years ago to listen for distress calls:



The clock isn't used in decoding the message because it's "missing" from the message - it's embedded in the signal. It IS used to know when to listen for a message (or when to transmit).

Another way to look at it... the decoder can decode multiple signals in its bandpass, with varying frequency AND time offsets, even with signals overlapping each other. To do this, the sync signal must be detected in each one of these signals and independently used for the decoding - the offsets by frequency and time are exploited to be able to decode these overlapping signals. It HAS to use the embedded sync information to decode the individual signals.

 

Without a calendar and clock, synchronized to a standard, how would you know when to join the Saturday afternoon Drake Net? Heck, without the agreed upon Julian calendar, how would you know what Saturday even is? Frequency is the same thing, 14.070 Mhz is based on a defined convention (what a Megahertz is), and calibration to that standard's source.

The only difference between joining a Saturday afternoon HF net by looking at your calendar and clock (which have been externally calibrated - someone has to tell you what day or time it is if you don't know), and the timing used by the JT modes is resolution and accuracy. You don't know it's Saturday unless you get that info externally, and can stay synced.

 

I already addressed this several pages ago, and the analogy is too flawed to be useful. You're talking about macro-level scheduling, and I'm talking about bit-level clocking. Both are useful in context, but they are not comparable beyond a broad reference to "time."

 

Dude, you started out trolling us FT8 religious nuts by saying things like:

​

So, come on. While it's not an apples to apples comparison, but it's not the sacrilege you're making it out to be either.

This is a valid point, however, with Olivia the bandwidth it requires is also an issue, unless you have a receiver and software that can consume way more than the 3KhZ most mortals can handle.

 

The only difference is resolution and accuracy. I have a wind-up ships clock here, as well as one that monitors the hyperfine transition of electrons in rubidium-87. Both keep time adequately to join the Drake net. The windup clock would work just fine for the send/receive windows in JT8 (if I could interface it to a computer) for probably 12-24 hours - it drifts about 15-30 seconds a month when properly tuned up. The rubidium clock should be good for probably 5000 years if it would run that long. Both would work just fine for an operating session of a few hours dispute being many orders of magnitude different in their accuracy's. Not flawed at all, it's about scheduling at different resolutions. You can calibrate a PC clock - by hand, listening to WWV - accurately enough to work using JT mode, and (at least mine PC) it keeps that time for hours. I know, because I have done it.

 

How so? I made a simple statement of fact. Technical fact, nonetheless. Of all the stuff I have written, you pick that as the thing to get wound up about?

Sacrilege? Now who's overreacting? I have been fairly careful to stick to the technical issues. If you are offended, then that's your problem to solve.

Look, if I buy a car, and the sticker says 30-mpg overall fuel economy, I would want to know if that number came from playing games like some manufacturers do these days. If it can only get 30mpg when it is being pushed along by another car, I feel like the 30mpg sticker isn't very honest, even if the actual fuel consumption of the test vehicle under the test conditions is 30.000mpg, because the underlying assumptions aren't realistic for comparison to other vehicles. I don't see how this is any different. I simply want people to understand what they have and what they don't, and I want others to be honest in their assertions. Comparing FT8 with Olivia using SNR numbers isn't honesty, because it's not a complete story. And that's all I'm trying to say (for like the 10th time in the 2nd or 3rd thread).

IIRC, Olivia has a couple of different 250Hz modes that are fairly common. You just click the desired configuration and away you go. Contestia (close cousin to Olivia) has a 125Hz mode. A 3kHz channel can handle several of those. Heck, I have seen Olivia decode fine with PSK31 stations blasting away on top of their MFSK. It's very robust that way. And while we're on that subject, this is the kind of number-bending that causes guys like me to be skeptical of quotes of FT8's capabilities and accomplishments. People make comparisons (just like yours) to other modes without even getting the other mode's details right.