Colorado Ultraviolet Transit Experiment (CUTE): S-band transmissions

[nsdecicco]

Introduction

This is a continuation of issue #297, "Add support for the Colorado Ultraviolet Transit Experiment (CUTE)", with a focus on adding support for CUTE's S-band transmitter as opposed to its UHF transceiver.

CUTE S-band transmitter description

Band	13cm (S-band)
Mfg./Model	BCT, TX only
Frequency	2.402 GHz
Modulation	BPSK
Baudrate	500 kilobaud
Framing	CCSDS
Antenna	High-gain patch
Transmit power	~2W

Demodulation

The BPSK demodulator that currently exists in gr-satellites has been found to work to demodulate CUTE S-band transmissions without modification.

Deframing

gr-satellites has two CCSDS deframers:

1. CCSDS Reed-Solomon Deframer (implemented in python/components/deframers/ccsds_rs_deframer.py)
2. CCSDS Concatenated Deframer (implemented in python/components/deframers/ccsds_concatenated_deframer.py)

Both of these assume that a forward error correcting (FEC) code is used. CUTE does not use an FEC, so neither of these blocks will work as-is. However, the former is very close to being usable as-is if the FEC block is bypassed.

The blocks internal to the CCSDS Reed-Solomon deframer are initialized in ccsds_rs_deframer.py at lines 62 through 73:

gr-satellites/python/components/deframers/ccsds_rs_deframer.py

Lines 62 to 73 in 7f193ab

	self.slicer = digital.binary_slicer_fb()
	if precoding == 'differential':
	self.differential = digital.diff_decoder_bb(2)
	deframe_func = (sync_to_pdu if scrambler == 'CCSDS'
	else sync_to_pdu_packed)
	packlen_mult = 8 if scrambler == 'CCSDS' else 1
	self.deframer = deframe_func(
	packlen=(frame_size + 32 * rs_interleaving) * packlen_mult,
	sync=_syncword, threshold=syncword_threshold)
	if scrambler == 'CCSDS':
	self.scrambler = ccsds_descrambler()
	self.fec = decode_rs(rs_basis == 'dual', rs_interleaving)

These blocks are then connected together on lines 75 through 86:

gr-satellites/python/components/deframers/ccsds_rs_deframer.py

Lines 75 to 86 in 7f193ab

	self._blocks = [self, self.slicer]
	if precoding == 'differential':
	self._blocks.append(self.differential)
	self._blocks += [self.deframer]
	self.connect(*self._blocks)
	if scrambler == 'CCSDS':
	self.msg_connect((self.deframer, 'out'), (self.scrambler, 'in'))
	self.msg_connect((self.scrambler, 'out'), (self.fec, 'in'))
	else:
	self.msg_connect((self.deframer, 'out'), (self.fec, 'in'))
	self.msg_connect((self.fec, 'out'), (self, 'out'))

So, these blocks are connected as such:

   self.slicer               self.differential              self.deframer              self.scrambler               self.fec

                                                         If scrambler == 'CCSDS'
                                                         ╭─────────────────────╮    ╭───────────────────╮
                                                   ╭────>│     sync_to_pdu     │───>│ ccsds_descrambler │───╮
                                                   │     ╰─────────────────────╯    ╰───────────────────╯   │
╭──────────────────╮       ╭─────────────────╮     │                                                        │     ╭───────────╮
│ binary_slicer_fb │──────>│ diff_decoder_bb │─────┤                                                        ├────>│ decode_rs │────> self.out
╰──────────────────╯       ╰─────────────────╯     │                                                        │     ╰───────────╯
                             Only enabled if       │     ╭─────────────────────╮                            │
                              precoding ==         ╰────>│ sync_to_pdu_packed  │────────────────────────────╯
                              differential               ╰─────────────────────╯
                                                         If scrambler != 'CCSDS'

The problem w.r.t. CUTE is that this block does not provide a way of disabling (disconnecting) the decode_rs (self.fec) block.

I was able to successfully decode a recorded CUTE CCSDS transmission over S-band with gr-satellites with the following combination of blocks:

       ╭──────────────────╮    ╭─────────────────╮    ╭─────────────╮    ╭───────────────────╮
in ───>│ binary_slicer_fb │───>│ diff_decoder_bb │───>│ sync_to_pdu │───>│ ccsds_descrambler │───> out
       ╰──────────────────╯    ╰─────────────────╯    ╰─────────────╯    ╰───────────────────╯

I.e., this is equivalent to the gr-satellites CCSDS Reed-Solomon block with scrambler == 'CCSDS', precoding == differential, and the decode_rs block removed.

Modifying the CCSDS Reed-Solomon block to be compatible with CUTE S-band transmissions

Modifying the CCSDS Reed-Solomon decoder block to allow disabling forward error correction can be accomplished relatively easily, by adding a new rs_en parameter to the constructor:

diff --git a/python/components/deframers/ccsds_rs_deframer.py b/python/components/deframers/ccsds_rs_deframer.py
index 2f9bcdc..82b2f8c 100644
--- a/python/components/deframers/ccsds_rs_deframer.py
+++ b/python/components/deframers/ccsds_rs_deframer.py
@@ -31,14 +31,15 @@ class ccsds_rs_deframer(gr.hier_block2, options_block):
         frame_size: frame size (not including parity check bytes) (int)
         precoding: either None or 'differential' for differential precoding
                    (str)
+        rs_en: If Reed-Solomon should be enabled or not (bool)
         rs_basis: Reed-Solomon basis, either 'conventional' or 'dual' (str)
         rs_interleaving: Reed-Solomon interleaving depth (int)
         scrambler: scrambler to use, either 'CCSDS' or 'none' (str)
         syncword_threshold: number of bit errors allowed in syncword (int)
         options: Options from argparse
     """
-    def __init__(self, frame_size=223, precoding=None, rs_basis='dual',
-                 rs_interleaving=1, scrambler='CCSDS',
+    def __init__(self, frame_size=223, precoding=None, rs_en=True,
+                 rs_basis='dual', rs_interleaving=1, scrambler='CCSDS',
                  syncword_threshold=None, options=None):
         gr.hier_block2.__init__(
             self,
@@ -66,7 +67,8 @@ class ccsds_rs_deframer(gr.hier_block2, options_block):
                         else sync_to_pdu_packed)
         packlen_mult = 8 if scrambler == 'CCSDS' else 1
         self.deframer = deframe_func(
-            packlen=(frame_size + 32 * rs_interleaving) * packlen_mult,
+            packlen=((frame_size + (32 * rs_interleaving if rs_en else 0)) *
+                     packlen_mult),
             sync=_syncword, threshold=syncword_threshold)
         if scrambler == 'CCSDS':
             self.scrambler = ccsds_descrambler()
@@ -78,12 +80,20 @@ class ccsds_rs_deframer(gr.hier_block2, options_block):
         self._blocks += [self.deframer]
 
         self.connect(*self._blocks)
+
+        if rs_en:
+            out = (self.fec, 'in')
+        else:
+            out = (self, 'out')
+
         if scrambler == 'CCSDS':
             self.msg_connect((self.deframer, 'out'), (self.scrambler, 'in'))
-            self.msg_connect((self.scrambler, 'out'), (self.fec, 'in'))
+            self.msg_connect((self.scrambler, 'out'), out)
         else:
-            self.msg_connect((self.deframer, 'out'), (self.fec, 'in'))
-        self.msg_connect((self.fec, 'out'), (self, 'out'))
+            self.msg_connect((self.deframer, 'out'), out)
+
+        if rs_en:
+            self.msg_connect((self.fec, 'out'), (self, 'out'))
 
     _default_sync_threshold = 4
 

Exposing this new rs_en parameter to the user can be accomplished by then editing grc/components/deframers/satellites_ccsds_rs_deframer.block.yml. However, from an end-user perspective, this might be confusing: a user would not likely suspect that the CCSDS Reed Solomon deframer block would allow disabling a major part of its namesake, and might expect to find a separate CCSDS deframer block lacking Reed-Solomon decoding. I was able to accomplish this (i.e., create a separate block) without duplicating (Python) code by creating another .block.yml file, grc/components/deframers/satellites_ccsds_deframer.block.yml, which instantiates a ccsds_rs_deframer with rs_en forcibly set to False. This same parameter in the CCSDS Reed Solomon deframer block (satellites_ccsds_rs_deframer.block.yml) stays hidden from the user: rs_en is forcibly set in this file to True.

However, I am not sure how exactly to implement this in _deframer_hooks() in gr-satellites/python/core/gr_satellites_flowgraph.py. I think the right way of doing this is to add a line using the set_options() routine, like so:

diff --git a/python/core/gr_satellites_flowgraph.py b/python/core/gr_satellites_flowgraph.py
index 621c05f..6e8146c 100644
--- a/python/core/gr_satellites_flowgraph.py
+++ b/python/core/gr_satellites_flowgraph.py
@@ -441,6 +441,7 @@ class gr_satellites_flowgraph(gr.hier_block2):
         'Swiatowid': deframers.swiatowid_deframer,
         'NuSat': deframers.nusat_deframer,
         'K2SAT': deframers.k2sat_deframer,
+        'CCSDS': set_options(deframers.ccsds_rs_deframer, rs_en=False),
         'CCSDS Reed-Solomon': deframers.ccsds_rs_deframer,
         'CCSDS Concatenated': deframers.ccsds_concatenated_deframer,
         'LilacSat-1': deframers.lilacsat_1_deframer,

Decoding

All data transmitted by CUTE over its S-band link is contained within a 2048 CCSDS transfer frame. Each transfer frame is prefixed by a 4-byte sync word. A CCSDS packet within a transfer frame may span multiple frames.

Sync word (0x352EF853†)	2048 bytes
	Transfer frame
	Transfer frame header (6 bytes)	CCSDS packet (may be multiple in a transfer frame)					FCEF (2 bytes)
		Header		Payload
		CCSDS Primary Header (6 bytes)	CCSDS Secondary Header (6 bytes)	CCSDS Primary Header (6 bytes)	Payload	Checksum (2 bytes)

^† The sync word has this value after descrambling

If the payload is longer than 2048 - 24 = 2024 bytes, the next transfer frame will not begin with CCSDS primary or secondary headers: it will continue with payload bytes that "overflowed" the previous transfer frame.

I am not sure how to deal with the problem of packets that span multiple transfer frames. I suppose I could do similar to what I did for CUTE's UHF packets, which is to say, implement a stateful telemetry decoder, but the idea of re-assembling frames in a transport seems more appealing in this case.

Progress thus far

As can be seen from the links earlier in this issue report, I have pushed two commits to lasp/gr-satellites:

• 91fd8baf: Allow disabling Reed-Solomon FEC in CCSDS deframer
• 0c28a305: Add S-band transmitter to CUTE.yml

I am attaching a zip file, CUTE S-band flow graphs.zip, containing two flow graphs:

• gr_satellites_s_band_test_manual.grc - flow graph using discrete GNU Radio / gr-satellites blocks to decode CUTE S-band CCSDS data
• gr_satellites_s_band_test.grc - flow graph using new ccsds_deframer block introduced in the above commit.

A̵ ̵t̵e̵s̵t̵ ̵I̵&̵Q̵ ̵f̵i̵l̵e̵ ̵c̵a̵n̵ ̵b̵e̵ ̵f̵o̵u̵n̵d̵ ̵a̵t̵ ̵t̵h̵i̵s̵ ̵l̵i̵n̵k̵.

EDIT: the correct file is called CUTE_Sband_raw_4Msps_PRBS_recvd_trimmed.bin--the above link points to a 70cm (UHF) CUTE beacon, not S-band data.

To-do

• Determine how best to re-assemble CCSDS packets that span multiple transfer frames
• Write a telemetry decoder for S-band data

[K4KDR]

Thank you for the VERY thorough explanation! I "thought" that I had replicated your steps to add the new block [CCSDS deframer (sans Reed-Solomon)] to my gr-satellites local install, but unfortunately that block continues to be 'not found' on my system.

Your notes mention:

- Add satellites_ccsds_deframer.block.yml to the CMakeLists.txt

... but unless I am looking in the wrong place, I do not see any recent changes to that file nor any reference to the new block inside it.

I'd be so grateful for a push in the right direction to allow me to have this new block available for testing in my local install so that I might follow along with the developments here.

Many thanks!

-Scott, K4KDR

[nsdecicco]

Hi Scott,

Please note that I pushed these changes to the lasp/gr-satellites fork, as I don't have direct commit access to gr-satellites. (I did open a pull request recently at Daniel's repository here for some CUTE UHF-related code, which he merged earlier this week. So this code will most likely find its way into this (upstream) gr-satellites repo the same way.)

So, you'll need to clone that fork like so:

$ git clone https://github.com/lasp/gr-satellites.git

and build/install as you would normally. (Please let me know if you need help building this code.)

It might also be worth noting that I was able to cherry-pick these commits over to the maint-3.8 branch and test these with GNURadio 3.8 as well if you're using that version. (I could explain how to do that as well if you need.)

Keep in mind that it's not enough to re-run 'make' after pulling these code changes: you will need to re-generate makefiles with CMake for the changes in CMakeLists.txt to propagate.

Anyway, if you could share some more details of what you tried, I'd be able to assist better.

Your notes mention:

• Add satellites_ccsds_deframer.block.yml to the CMakeLists.txt

... but unless I am looking in the wrong place, I do not see any recent changes to that file nor any reference to the new block inside it.

Here's that file:

https://github.com/lasp/gr-satellites/blob/91fd8baf2c90bca1580d7148209b157cbedfd13c/grc/components/deframers/satellites_ccsds_deframer.block.yml

And the corresponding change to the CMakeLists.txt:

lasp@91fd8ba#diff-15bc7ae52de968dcdd55de32a820963f47ab4fc763aaaef4df0c8e76854328a8

Thanks,

Nick

[K4KDR]

Many thanks for the reply!

#1, somehow I missed that the 'CMakeLists.txt' file in question was not in the root, but was 'grc/components/deframers/CMakeLists.txt'. So, I have copied over your version. The new block is now available in my system!

#2, I am not quite done as the there is a complaint about the new argument:

got an unexpected keyword argument 'rs_en'

... clearly I have not properly incorporated all your additions to allow that to be interpreted. I will track that down next.

Thanks again!

[K4KDR]

Works perfectly w/ your sample I/Q file! Very sorry that I didn't copy over your changes properly the first time.

FYI, system here is Ubuntu 18.04, GNU Radio 3.8.1.0, gr-satellites branch 'maint-3.8'

No idea if I will be able to downlink your 2.4 GHz signal with enough SNR for decodes, but it'll be great to have the decoder incorporated into gr-sats. Thanks for the help & best of luck with the project!

[nsdecicco]

Hi Scott,

Glad you managed to figure things out, and thanks for sharing that screenshot!

However, your screenshot showed me that I actually put a link to the wrong I&Q file 😬

The correct file is called CUTE_Sband_raw_4Msps_PRBS_recvd_trimmed.bin

I'll edit the original issue report to have the new link.

CUTE is using BPSK modulation, and you should see a spectrum evenly centered about 2.402 GHz. It should look something like this:

No idea if I will be able to downlink your 2.4 GHz signal with enough SNR for decodes

LASP has a 4.2m(!) dish, and even with that we struggle with our S-band link. I should have noted, we're wont be configured at launch to beacon over S-band, because successfully closing that link requires the spacecraft to actively point its patch antenna towards the ground, which means that we can't collect science data at the same time. So, our conops has us send commands to the spacecraft during passes to schedule active pointing on future passes for S-band downlink.

But--we will be beaconing every 16 seconds on UHF, at 437.250 MHz--see my other issue report for details on that :)

Thanks for the help & best of luck with the project!

Thanks!

Nick

[K4KDR]

Thanks for updating the sample I/Q file location. Fortunately for me, that decodes properly in GRC as well.

Appreciate the UHF beacon info; I'll be listening when the time comes!

[daniestevez]

Hi Nick!

Thanks for this very detailed report.

Indeed gr-satellites is currently missing the capability to deframe CCSDS uncoded frames, and this is a nice addition. May I suggest to call the new GRC block "CCSDS uncoded deframer" instead of just "CCSDS deframer"? This is how the TM Synchronization and Coding Blue Book refers to these frames (see Section 11.3, for instance).

At first I was thinking that it was best to create a new Python block, but I like better your solution of adding an optional parameter to the RS Python block and then creating a new GRC block, since the RS block already has some options (precoding and scrambler) that are still useful for uncoded frames, and creating a new Python block would cause too much code duplication.

You are right that the correct way to add a hook is

'CCSDS': set_options(deframers.ccsds_rs_deframer, rs_en=False),

Again, I would suggest using "CCSDS uncoded" instead of "CCSDS" here.

Something you might want to do if you have time is adding documentation for this new deframer. See "CCSDS Deframers" here. Just a mention of the existence of the uncoded deframer in that section would be fine, since the RS and concatenated deframer are treated together. Also see the large bullet list here. An item for the uncoded deframer should be included in that list.

By the way, I found this note a bit confusing:

† The sync word has this value after descrambling

The syncword is not descrambled, and the CCSDS synchronous scrambler is reset at the beginning of the frame (just after the syncword), so even if I wanted to descrambler the syncword, I wouldn't know what the output would be, since the descrambler is not initialized at this point.

Regarding defragmentation of packets, I think this is better done in a transport. However, there is something I don't understand well. How do you know if the frame you get contains the first fragment or it is one of the subsequent fragments? Is the Transfer Frame header a TM Space Data Link Primary Header, containing the First Header Pointer? If so, then I understand.

Will the S-band transmitter be used to downlink files? If so, you might want to take a look at the file receiver infrastructure in gr-satellites, which is able to reassemble files that are transmitted in chunk (and it is rather general regarding the format of the chunks).

I think I have addressed all the points in your comment, but if I've missed something, don't hesitate to insist, since that's a long comment, so I might have forgotten about something.

[nsdecicco]

Hi Daniel,

Apologies for the slow response, and thanks for your feedback.

May I suggest to call the new GRC block "CCSDS uncoded deframer" instead of just "CCSDS deframer"? This is how the TM Synchronization and Coding Blue Book refers to these frames (see Section 11.3, for instance).

[...]

You are right that the correct way to add a hook is

'CCSDS': set_options(deframers.ccsds_rs_deframer, rs_en=False),

Again, I would suggest using "CCSDS uncoded" instead of "CCSDS" here.

[...]

Something you might want to do if you have time is adding documentation for this new deframer.

I've gone ahead and made those changes--you can find these in commit c0c014c4. I wasn't very verbose with the documentation that I added--please let me know if you would like to see more verbiage, or if I missed anything. I could create a pull request for this code now, or defer that until later (per your preference).

By the way, I found this note a bit confusing:

† The sync word has this value after descrambling

The syncword is not descrambled, and the CCSDS synchronous scrambler is reset at the beginning of the frame (just after the syncword), so even if I wanted to descrambler the syncword, I wouldn't know what the output would be, since the descrambler is not initialized at this point.

Interesting. My remark (that this is the value of the sync word after descrambling) was based off an assumption. In the 'Sync and create PDU' block, the default sync word is 0b00011010110011111111110000011101, or

0001 1010 1100 1111 1111 1100 0001 1101
   1    A    C    F    F    C    1    D

which works, whereas BCT claims that the sync word is supposed to be

0011 0101 0010 1110 1111 1000 0101 0011
   3    5    2    E    F    8    5    3

which is a good amount different, and doesn't work if entered as the sync word in said block. So, I assumed that 0x352EF853 must have been the value of the un-scrambled sync word, assuming further that scrambled this appears as 0x1ACFFC1D.

Regarding defragmentation of packets, I think this is better done in a transport. However, there is something I don't understand well. How do you know if the frame you get contains the first fragment or it is one of the subsequent fragments? Is the Transfer Frame header a TM Space Data Link Primary Header, containing the First Header Pointer? If so, then I understand.

Yes, the transfer frame header contains a first header pointer. I am attaching the output of some C code I wrote a while back to decode a stream of S-band data, which should hopefully clarify things. Note that I cut the output short at an arbitrary point as somewhere shortly thereafter an erroneous bit caused the output to get screwy.

Will the S-band transmitter be used to downlink files? If so, you might want to take a look at the file receiver infrastructure in gr-satellites, which is able to reassemble files that are transmitted in chunk (and it is rather general regarding the format of the chunks).

Sort of... the CUTE instrument can generate a few different data products:

• Full-size 2D image frames
• Cropped 2D image frames
• 1D spectra

Most of these data products are significantly larger than the transfer frame size: the CUTE detector is a Teledyne E2V CCD42-10, which has a resolution of 2048x512, and our ADC resolution is 16-bits. So, not counting the image header, the raw 2D frames are 2 megabits in size, which means that they take about 1024 transfer frames to get down.

But thanks for pointing out that part of the code--I'll give that a look.

[daniestevez]

Hi,

Your commit c0c014c looks perfect. You can send that in a PR right away if you want.

So, I assumed that 0x352EF853 must have been the value of the un-scrambled sync word, assuming further that scrambled this appears as 0x1ACFFC1D

Apparently 0x352EF853 appears as the ASM for embedded data stream in Section 9.6 of the Blue book my guess is that in this transmitter it might be used internally when moving frames from the software to the transmitter, but it isn't sent over the air.

I am attaching the output of some C code I wrote a while back to decode a stream of S-band data, which should hopefully clarify things.

Thanks. That makes sense. It's pretty standard CCSDS stuff.

[daniestevez]

On a related note. Now that Github has added support for discussions in the repositories, I have started using them, and thought of some guidelines.

From what I have seen so far (I'm a newcomer to this feature), discussions are somewhat similar to issues, but they can't be closed (which kind of makes sense, since you can call an issue "solved", but in a discussion anyone could come later with an interesting comment), and they support answering to particular messages (so you can have threads instead of just a single line of comments).

I have transferred to the discussions the other issues that were open about satellite teams, since they haven't had any comments in months and I think the discussions section for satellite teams is a better place to gather all this information.

Do you it's good to transfer your two issues to the discussions section? I think these are very good documentation, and in the discussions they don't risk being mixed with many other issues. If you think this is good, I'm not sure if you can do it yourself (there's a link called "Convert to discussion" on the right). If not, let me know and I'll do it.

[nsdecicco]

Hi @daniestevez, apologies for the delay in responding to this.

Do you it's good to transfer your two issues to the discussions section? [...] If you think this is good, I'm not sure if you can do it yourself (there's a link called "Convert to discussion" on the right). If not, let me know and I'll do it.

Sure, that sounds good to me! However, I don't see a 'convert to discussion' button, unless I am looking in the wrong place. It would make sense that an unprivileged user such as myself would not be able to do this (to prevent abuse), so I suspect that you'd have to be the one to do it.

[daniestevez]

Yes. That makes sense. I have just converted both issues to discussions.