Decode off-the-air Winlink message - request for programming help

Discussion in 'Working Different Modes' started by KX4O, Aug 6, 2019.

ad: L-HROutlet
ad: l-rl
ad: Subscribe
ad: abrind-2
ad: L-MFJ
ad: Left-3
ad: Left-2
  1. KX4Z

    KX4Z Ham Member QRZ Page

    THANKS!. I'll try. See where this goes....
    Gordon
     
  2. KX4Z

    KX4Z Ham Member QRZ Page


    Finding the START of the message data
    (after you have identified the FC EM statement):
    Search forward (it will be fond in a ###PAYLOAD2 line) look for "00, 02"
    (it will follow the "subject" of the email printed out as the beginning of the ###Payload2: line)
    The next hexadecimal (2 printed digits, but 1 sent binary) gives the number of bytes in the packet.
     
  3. KX4Z

    KX4Z Ham Member QRZ Page

    AN IMPROVEMENT: It turns out that WINLINK FC protocol appears to ALWAYS use an "offset" value of 03. Hence, an even better string for which to search for the beginning of the data message is: 00,30,00,02 in the PMON HEX 1 output (and likely also in the PMON HEX 2 output).

    I am now able to find both the unencoded size of the file AND I am able to move to the proper start of the compressed data. The c-code to read the /home/pi/capturefile and do this is attached (with .txt added to the .c file to make QRZ happen). It will compile with

    gcc <cfilename> -o <yourdesiredexecutablefilename>

    That is enough for one night. But now I have as many as three files that I've captured that appear to have all or part of a winlink message. I'm traveling all of tomorrow, but I may be able to take enough raspberry pi with me, in order to continue working on this. Two files are attached. Remember to move them to /home/pi and take off the .txt that I was forced to use to upload here.

    Now I find the size of that packet, append the text onto the output file, move to the next packet, find size, append text to output file, and continu until you reach the 0x04 character, which is the end of the file. Terminate the outputfile, and send to lzhuf and out should pop the message.

    Whew.....what a long trail!!! But in theprocess I have gained a ton of skills and brought a tiny spark of c-programming back to life in these old bones. The grandkids had a great time in the pool ---they are almost 2 and almost3....and still swimming with floaties, etc, but the younger one was a tiny bit more brave today. In Florida, it is extremely important to get kids able to SWIM as fast as possible, because there are SO MANY swimming pools in Florida....major hazard for young, non-swimmers.


    Gordon
     

    Attached Files:

    KX4O likes this.
  4. KX4Z

    KX4Z Ham Member QRZ Page

    John --- I'm properly storing the "unencoded file size" into the file I'm creating for lzhuf.exe, in little-endian order.

    BUT WHAT ABOUT THE DATA ITSELF?
    when I pull in the data from the packets, do i need to reverse them for little-endianess?

    In other words, if I am working on
    "48," <---ascii in the case of PMON HEX 1,
    and I convert that to numbers,

    4, and 8, meaning the value received was 0x48 = (4*16)+8 = 72 (decimal)

    when I store it out into the file for lzhuf.exe to read, do I store it as the binary 72, (normal)
    or do I need to reverse the upper and lower 4-bits of the 8-bit binary number that goes into the file?

    THanks
    Gordon
     
  5. KX4O

    KX4O Ham Member QRZ Page

    Endianess only applies to byte order of multi-byte numbers. Treat the two nibbles as the byte straight away. 48 = 01001000.
     
  6. KX4Z

    KX4Z Ham Member QRZ Page

    THANKS!
    I was busy with a Tenn Exercise and traveling much fo the day, but I finally had time to debug the code I wrote. I am always AMAZED at how many errors I can make Simple things like atoi() -- never realized some of its intricacies!!!

    The SCS people in PMON HEX 1 print out the DATA in CAPITAL ASCII -- and you have to convert to lower in order to make a conversion. I had to learn how to use sscanf for some features.
    Then I had much confusion figuring out how to find the END OF LINE -- turns out it is simply detected when you getc a decimal 13 from the array. With those corrections, I am now

    a) reading the unencoded file size properly from the FC EM propocal
    b)printing out in little-endian fashion the unencoded size, padded to 8 bytes
    c) properly detecting the actual start of the data string as explained above
    d) properly detecting the FIRST tranche of characters (often of size FA ) and detecting the number of bytes in the entire tranche
    e) stopping my processing when I hit the end of the TRANSMITTED PACKET (which may be shorter than the tranche)
    f) writing out all the above in what I think is correct form to /home/pi/outputfile

    Next I have to generalize the detection above to step to the next transmitted packets to capture the remaining portions of the tranche of data. Requires some thinking on my part how to track where I am at each point. Something real experts probably do easily......

    I don't seem to have as pretty a set of captures as you have....I'm hearing HUMMMM on my transmitted signal and not sure why that is....very very old pactor modem on that one....
    And I am currently working on PMON HEX1. but since you have a better capture, I may recode the capture of the FC EM statement and size, in order to be able to handle yhour PMON HEX 2 files. I think that is the only change needed.

    More programming to be done!

    Gordon
     

    Attached Files:

    KX4O likes this.
  7. KX4Z

    KX4Z Ham Member QRZ Page

    Well, after more hours of novice programming, i finally have a system that APPEARS to properly handle incoming packets, moves them into a char array, and then packages them back out as a file that I *thought* would be appropriate for lzhuf to decode. As you might guess, a TON of trial and error was needed for me to get that much done......a zillion little programming and logic-type errors....and there may still be errors of comprehension of course.

    Then I got ready to test it.....and could not get an lzhuf to work. segmentation faults, inability to compile (claiming it needed some library or something for _start something). Several hours went into trying to figure out why that wasn't working.....until I finally found a very simple set of code on a .se web site that compiled perfectly and RAN without segmentation faults etc. It can compress and decompress a test file.

    But....the output from my readcapture does not decode with the working lzhuf. I've compared the files I'm creating....with the compressed files that the lzhuf creates.....and they "look" identical. I tried various numbers of 00 00 00 etc in the start of the file, after the little-endian "size".....and it still doesn't work.

    So what I'm going to have to do next is take a KNOWN file received over winlink....and compress it with the WORKING version of lzhuf that I created....and compare it to the compressed version RECEIVED OVER THE AIR at my house (where I am not at the moment....) and then I may spot the error.

    But I'm quite happy, having finally actually gotten some code that can (verified by a zillion print statements -- producing 100 kB of test output to examine) read the files I captured over the radio, and properly find the size, the beginning of the file, and move the packets out.....


    So...it may be a bit longer but I'm encouraged that this software will eventually work. Software rarely works when you are "close" in my experience....so I have to find out what I'm missing.


    Gordon
     
  8. KX4O

    KX4O Ham Member QRZ Page

    Did you try compiling my version?
    https://www.hamradio.me/graphs/WinlinkTests/lzhuf.c
     
  9. KX4Z

    KX4Z Ham Member QRZ Page

    That code takes captured pactor retrieved in PMON HEX 1 format (so you can still see the headers, etc in easily readable text) and handles the rest of the PAYLOAD2 that comes out in hexadecimal ascii representations of binary.
    it produces an output file /home/pi/outpufile, which you can then send to lzhuf d outputfile <your choice of reconstructed filename>

    it expects input in a file /home/pi/capturefile
     
  10. KX4Z

    KX4Z Ham Member QRZ Page

    yes, and it would not compile (at least not today). something about missing a _start in some incomprehensible library.....i worked on it for a while and could not fix it so moved on.

    I don't understand all of those issues.....it said that there was some .o file that was missing something

    Gordon
     

Share This Page