1 Introduction

MUSCAT is a new camera for millimetre and submillimetre continuum astronomy at the 50 m Large Millimetre Telescope Alfonso Serrano (LMT). The instrument has been developed as part of a collaboration between El Instituto Nacional de Astrofísica, Óptica y Electrónica (INAÓE), Puebla, Mexico, and Cardiff University, Wales, UK, to strengthen ties and facilitate knowledge transfer between researchers across the two countries. The superior angular resolution and field of view of the LMT, combined with the large number of background-limited detectors in MUSCAT, will allow us to investigate local star formation processes across the Milky Way and galaxy evolution in general over the last \(\sim\) 12 billion years [1].

The MUSCAT instrument consists of a focal plane array of 1458 non-polarisation-sensitive horn-fed aluminium lumped element kinetic inductance detectors [2], continuously cooled at 130 mK with a novel miniature dilution refrigerator [3]. Optical coupling to the telescope is through a series of fully reflective cold and warm optics, and the 1.1 mm optical passband (250–300 GHz) is defined by a combination of the waveguide cut-off of the feedhorns and low-pass metal mesh filters within the cryostat. The detectors are spread across three silicon wafers (sub-arrays) with two CPW feedlines (readout channels) per sub-array, leading to a total of six RF multiplexed readout lines, each with 243 capacitively coupled resonators, two cryogenic RF attenuators and an LNA. The resonance frequencies span an approximate bandwidth of 500 MHz, nominally centred at 750 MHz on each line.

The detector readout electronics and FPGA interface are based on the BLAST-TNG system [4] and consist of six sets of ROACH2 boards with MUSIC DAC/ADC cards, a custom RF mixing unit, and a timing and synchronisation distribution system. The digital electronics and programmable components are initialized and managed by the Python Control Program, PCP, which provides tools for the recording and inspection of frequency sweeps and time-ordered data, as well as the automated retuning of readout tone frequencies and amplitudes, as required. The MUSCAT backend software is integrated with the existing LMT Instrument Control System and will listen for instructions from a telescope operator to perform observations of various types, feeding back results of quick look analysis for telescope pointing, focussing and mapping observations.

MUSCAT has undergone extensive testing in the Cardiff lab and has been shown to be fully functional and fit for purpose [5, 6]. The cryostat, electronics rack and all accessories have been shipped to the telescope and are currently being installed into the receiver cabin, where they will undergo a final performance verification prior to commencing science observations. Details of the instrument electronics and control software, and the key results and findings of the in-laboratory testing, are given here.

2 Hardware Overview

One of the main advantages of a KID-based camera over other contemporary low-temperature detector systems is the reduction in cryogenic readout complexity afforded by the natural frequency division multiplexing capabilities of the detectors. The RF probe tones for every detector in a readout channel are combined into a frequency comb that propagates through a single coaxial line into and out of the cryostat via a cryogenic low noise amplifier (LNA). The vast majority of the cold electronics wiring for MUSCAT is actually for controlling the cryogenic refrigerators and thermometry, with only six twisted pairs required for the LNA biases and six pairs of coax lines for the RF detector readout. The coaxial input/output ports, the cryogenics controller box and a small box to distribute the LNA DC biases are all mounted to the top plate of the cryostat, alongside the optical window, a vacuum port, a mechanical heat switch, and the pulse tube cooler. The remaining electronics are all contained within a single 19-inch rack that stands next to the cryostat. A bundle of flexible 2-m coaxial SMA cables links the cryostat to the RF unit in the rack, along with Ethernet and DC power cables for the cryogenics controller and LNA biases. A schematic is presented in Fig. 1.

Fig. 1
figure 1

A schematic of the MUSCAT electronics showing internal interconnect between the various components, as well as external connections to the cryostat and the telescope infrastructure

Full size image

Power is provided to the rack through a single phase 230 V input into a 2200 VA UPS that should hold for up to one hour in case of mains supply interruption. In addition to the UPS, the rack houses the six ROACH2 boards in their 1U standard enclosures, the custom 3U RF mixer unit, the 1U timing and synchronisation signal distribution amplifier, two custom 2U AC-DC power supplies, three computers, a 2U unit for miscellaneous electronics, and a storage drawer. Powered USB hubs and gigabit Ethernet switches are mounted to the back of the rack, and there is provision for a screen and keyboard to be mounted to the side if a local workstation is required during commissioning.

The three MUSCAT computers each serve different purposes. muscat-cryo is a Windows computer that runs NI Labview to interface with the CompactRIO FPGAs in the cryo controller. It maintains the continuous cooling cycle, logs the various cryostat internal temperatures, and runs the PID temperature controller for the focal plane array. muscat-mux is an Ubuntu Linux machine that controls the multiplexing readout electronics and records raw observation data based on commands received from main LMT Instrument Control System. It performs the resonator tuning, timestream recording and monitoring, and data backups. muscat-da is a Fedora Linux machine for data analysis which monitors all incoming observations, performs automatic quick look analysis (QLA) and feeds back the results to the telescope operators. It also acts as the gateway machine to the outside world for the rest of the MUSCAT computers. MUSCAT has a private gigabit Ethernet LAN for communication between the three computers, the cryo FPGAs, the ROACH2 PowerPCs, and any other peripheral devices such as laptops, spectrum analysers and VNAs. There is also a dedicated private gigabit LAN for streaming the high-speed detector data from the ROACH2 FPGAs to muscat-mux.

A GPS locked 10 MHz reference clock and a one pulse per second (PPS) signal are fed into a Meinberg dual TTL signal distribution unit (SDU) and split out to the RF mixing unit and the ROACH2 boards, respectively. Each 10 MHz signal provides a stable reference for an RF signal generator to drive the FPGA clock, DAC/ADC clocks and mixer local oscillators. The PPS signal enables precise synchronisation between the detector data streams and the telescope pointing information. During the in lab-testing, the 10 MHz and PPS signals were sourced from an SDR-Kits GPSDO and an Adafruit Ultimate GPS housed in the miscellaneous electronics box, both fed from an external GPS antenna. At the LMT, the signals will be fed from a local OctoClock-G CDA-2990 GPSDO. Alarm signals from the SDU are monitored by a microcontroller in the miscellaneous box, and users are notified in the case of any signal loss that will compromise timing synchronisation.

Fig. 2
figure 2

A block diagram indicating the complete RF chain of MUSCAT, from the digital system, through the RF box, to the cryostat and back

Full size image

3 Cold Readout Electronics

The six cryogenic RF readout lines in MUSCAT are nominally identically and consist of hand-made, pig-tailed, sections of 50\({\Omega }\) 2.2-mm stainless steel inner-outer semi-rigid coaxial cable, solder terminated with SMA connectors. Pairs of cables run between each cold stage, i.e. 300 K to 50 K, 50 K to 4 K, 4 K to 300 mK, and 300 mK to the focal plane array. SMA bulkhead feedthroughs act as heat sinks for the outer conductors and BLK-18-S+ in-line coaxial DC blocks on the cold-facing side of each stage inhibit heat flow through the centre conductors. This cost-effective arrangement reduces the heat load on the various refrigeration stages of MUSCAT. However, we have found that the solder joints on the SMA cables are occasionally prone to failure during thermal cycling due to the high rigidity of the stainless steel and that, rarely, tension changes due to thermal contraction can damage the DC blocks. We have also seen LNA output ports damaged in this way and would advise the use of lower rigidity cables for future instruments.

MUSCAT has 20 dB and 10 dB in-line attenuators on the RF input lines at 4 K and 300 mK, respectively. These serve to reduce thermal noise on the input RF signals relative to equivalent 30 dB attenuators at room temperature. We have found that the higher-performance BW-series attenuators operate suitably within specifications at ultra-low temperatures but that low-cost alternatives can have unpredictable properties.

The six low noise amplifiers in MUSCAT are 0.5–3 GHz SiGe LNAs from ASU and are mounted on the warm-facing side of the 4K stage. They have over 30 dB gain and noise temperatures below 7K in the current 500–1000 MHz readout band and are similarly rated out to 3 GHz making them suitable for any future upgrades of the MUSCAT focal plane arrays. The DC bias for each LNA is fed via a manganin twisted pair from a port at the top of the cryostat. The bias levels were tuned with a custom six-channel variable voltage source, and however, this source introduced considerable noise into the system, so a fixed voltage source based on a single ADM7160 ultra-low noise linear regulator driven from the 5V output of the readout rack was implemented in the final DC bias box. A schematic of the complete RF chain, including the cryogenic system, can be seen in Fig. 2.

The overall complex transmission of the MUSCAT cryogenic RF lines has been measured by vector network analyser, and the performances are identical to within 1 dB of power. As shown in Fig. 3, there is an overall drop in transmission towards higher frequencies, primarily due to losses in the coax cables and connectors, but the average transmission through the full cryogenic system, across the readout band, is − 7 dB.

Fig. 3
figure 3

Left: VNA measurements of S21 transmission magnitude of the six MUSCAT cryogenic readout channels. Right: Frequency comb output power for five out of the six warm electronics readout systems, each with 250 evenly spaced, nominally equal amplitude tones, 750 MHz LO and 0 dB attenuation. Vertical lines indicate the nominal readout frequency band

Full size image

4 Warm Readout Electronics

4.1 Digital System

The detector readout system is based on the BLAST-TNG firmware that runs on the ROACH2 FPGAs. The firmware has a rich heritage in low temperature superconducting resonator readout systems [7], especially KID systems, and the latest ROACH2 port has been successfully deployed on the BLAST-TNG [8] and OLIMPO [9] balloon-born telescopes, with ground-based cameras and spectrometers such as MUSCAT, TOLTEC [10] and SUPERSPEC [11, 12] coming online in the near future.

The FPGA firmware implements large circular memory buffers for outputting the complex frequency division multiplexed waveforms that feed into the cryostat. The signals returned to the FPGA are spectrally decomposed in the firmware and variations in the complex transmission for each tone in the frequency comb are decimated and packetised along with the PPS timestamp and GPIO data and then streamed continuously to the host computer over Ethernet at a sample rate of 488.28125 SPS. The frequency and dissipation response of each detector are estimated in post-processing by comparing the recorded changes in the I and Q values with the changes measured during a controlled frequency sweep.

The DACs and ADCs on the MUSIC card are clocked at 512 MHz with inputs split from one output of a dual channel programmable signal generator in the RF mixing unit, and the 256 MHz FPGA clock is derived from the DAC clock on the card. The DAC outputs are coupled to special purpose high-order 256 MHz anti-alias filters from ASU. The TTL level PPS signal and GPIO inputs are fed into the FPGA through the AUX IN port and the GPIO header on the ROACH2 board.

4.2 RF System

To achieve the required detector readout frequency range of 500-1000 MHz, the baseband (0–256 MHz) DAC outputs are mixed up with a local oscillator (LO). The RF signals returning from the cryostat are mixed back down with the same LO prior to being sampled by the ADCs. Quadrature mixing, based on appropriately phase-shifted I and Q signals for optimal image rejection, is utilised to double the usable bandwidth of the system. For each of the six RF readout channels, nominally identical mixer circuits based on commercially available connectorised components were assembled and slotted into the RF mixer unit in the MUSCAT rack. Each circuit comprises of an AM0350 quadrature modulator and an AD0540 quadrature demodulator from Polyphase Microwave for the RF mixing, a Windfreak SynthHD RF signal generator for the LO and digital clock signals, and components for programmable gain control on both the RF output and RF input lines, as well as connectors for all of the required DC power and USB signal lines. A schematic of the complete RF electronics chain can be seen in Fig. 2.

Frequency sweeping is achieved by stepping the LO frequency of the mixers over a range of roughly 200 kHz, which in turn steps the entire frequency comb, so that all \(\sim\) 250 resonators per channel are swept in a single pass. The LO outputs have in-line VLF-800+ low-pass filers for rejection of harmonics that would otherwise mix with baseband signals and reduce the system dynamic range.

Gain control on each RF input to the cryostat is implemented with a RUDAT-6000-60 programmable attenuator with 0.25 dB step size. A band-pass filter on the output is added to further improve the dynamic range, which is composed of ZX75LP-1050-S+ low-pass and ZX75HP-396S-S+ high-pass coaxial filters. Gain control into the quadrature demodulator on the return from the cryostat includes a ZX60-P105LN+ low noise amplifier, an identical programmable attenuator and an identical band pass filter.

Ignoring the limits set by the inline coaxial filters, the highest readout frequency achievable with this RF mixing system is \(\sim\) 4000 MHz (set by the quadrature demodulator) and the maximum bandwidth per channel is \(\sim\) 600 MHz (also set by the quadrature demodulator) The present high-frequency limit for detector readout in MUSCAT is \(\sim\) 3000 MHz (set by the LNAs), and the overall bandwidth limit is \(\sim\) 500 MHz per channel (set by the MUSIC DAC/ADC card).

4.3 LO Leakage

DC offsets on the inputs to the quadrature modulator cause power from the LO to leak into the RF output port. The LO input power of +5 dBm is considerably higher than the typical − 25 dBm tone level coming out of the modulator, so it is important that leakage is minimised to prevent the LO power from dominating the RF dynamic range. The modulators have differential I and Q inputs, and however, the MUSIC DAC outputs are single ended, so care has been taken not to introduce DC offsets by improperly transforming to differential inputs. A cost-effective solution was to terminate the \(\bar{\mathrm {I}}\) and \(\bar{\mathrm {Q}}\) negative inputs on the modulator to ground through a resistor equal to the resistance seen to ground from the I and Q positive ports. We soldered small surface mount resistors to board-mount SMA connectors between the centre pin and outer section and plugged them into the negative differential ports. The measured resistances were typically 5.15\({\Omega }\), so 5.1\({\Omega }\) resistors were used, which reduced the LO leakage power from 0 dBm with the port left open to − 45 dBm.

4.4 Image Rejection

As mentioned above, the baseband readout tones are generated in quadrature with 90-degree phase offsets between I and Q in order to provide image rejection so that both sidebands can be utilised after mixing with the LO. However, amplitude and phase imbalances in I and Q at the modulator inputs reduce the rejection ratio, and we have measured considerable variations across the six RF readout channels. At frequencies close to the LO frequency, the ratio was within the specifications of the modulator (40 dB relative to the power at the desired frequency), but it rapidly deteriorates further out with rejection ratios as low as 10dB at frequencies 200 MHz from the carrier. By applying frequency-dependent amplitude and phase offsets between I and Q in the multiplexed waveform, we are able to compensate for gain and group delay variations in the IQ signal chain and have achieved rejection ratios greater than 40 dB across the full band. The RF image tone power of an arbitrary frequency comb was measured in the laboratory with a spectrum analyser, and the correction coefficients were adjusted to minimise the power. The coefficient values were then interpolated as a function of baseband tone frequency and are applied automatically whenever a new waveform is generated, such as during detector tuning. The measurement procedure was performed once in the laboratory for each RF channel following the system build, and the coefficients should remain valid as long as the circuit remains unchanged.

4.5 System Noise

During laboratory testing of MUSCAT, timestreams were recorded under a range of conditions and the noise power spectral densities were estimated. With the cryostat window open and the camera looking out into the 300 K environment, the readout was tuned onto the detectors and noise was measured for each KID. Additionally, noise was measured with the readout tuned away from the resonance as a means to estimate to electronic system noise. The mean value of the phase noise power spectral density at sampling frequencies above 10 Hz was typically 10 to 20 dB higher in the on-resonance data. Given that the depths of the resonators are on average 6 dB and assuming a linear scaling of phase noise with measured tone power, this indicates that the readout system noise is lower than the inherent resonator noise and thus does not significantly impact the detector readout, as shown in Fig. 4.

Fig. 4
figure 4

Left: Average phase noise power spectral density at sampling frequencies higher than 10 Hz for each resonator with the readout system tuned on resonance. Right: The same but tuned-off resonance

Full size image

5 Software and Control System

At the top level, the MUSCAT readout software system is run from a Python script on the muscat-mux computer that responds to commands from a local user or from a telescope operator via the LMT Instrument Control System. All top-level actions are logged in a ’control log’, and all observation runs are logged in an ’observation log’ which is regularly parsed by the data analysis system for automated quick look analysis. Users are updated with results from the QLA and notified if any alarms are triggered, such as high cryostat temperatures, or issues with the timing and synchronisation systems.

Actions involving the detector readout electronics are handled by PCP, the Python Control Programme which was developed in collaboration with the SUPERSPEC project to perform multi-tone resonator readout with ROACH2 hardware running the BLAST-TNG firmware. It is adapted from KIDpy at the low-level FPGA interface layer. PCP runs all six readout channels in parallel, using Python threads to handle the programmable hardware control, and a system of asynchronous Python sub-processes for logging resonator I and Q time streams.

On startup, system parameters related to file-system organisation, networking, hardware devices, firmware registers and log file formatting are loaded from configuration files. Resonator-related readout parameters such as baseband tone frequencies, amplitudes, phases, local oscillator frequencies and attenuator values are loaded from ’toneslist’ files which can be updated and reloaded during operation as test conditions change.

5.1 Detector Tuning

Prior to most observations, the readout tones will need to be tuned to the KID resonance frequencies. This is because the KIDs move around in frequency depending on the background loading or effective sky temperature, which can vary significantly depending on atmospheric conditions and target elevations. Frequency sweeps are performed to identify the ideal tone frequencies for each resonator. The point on the resonance at which the complex transmission varies fastest with respect to the sweep frequency is chosen as this point gives the largest signal response for a given change in resonance frequency. Following the sweeps, the tones lists are updated and the multiplexed waveforms are regenerated and uploaded to the FPGAs. A second set of sweeps are recorded prior to the streams starting, and the resonators are fitted against a model to derive the calibration parameters necessary for the frequency and dissipation response estimations. We find that a 1kHz step size and a 250 kHz span (251 points per sweep) are sufficient to capture all of the resonances. Typically it takes 30 s per sweep and 30 s to reload the waveform, or 90 s in total to retune. While this could be sped up by reducing the number of sweep points, the time taken to load the waveform to the FPGA via the ROACH2 PowerPC cannot be improved upon.

The readout tone power is also tunable and is maximised to a level just below resonator bifurcation that is established by fitting a nonlinear resonator model to sweep data taken as a function tone power. The detector power handling is not expected to change over the course of a night to the point where the tuning is lost, so this time-consuming process should not be required regularly during usual operations. In cases where the resonator sweeps are not well described by the model, the power level is chosen that gives the highest resonance frequency. This generally avoids both the bifurcation and TLS-dominated regions of the power sweeps.

5.2 Recording Observations

Every UDP data packet received from the firmware on a ROACH2 board contains the data for one sample of time-ordered data and always holds 1024 pairs (I and Q) of 32-bit values, even when fewer than 1024 tones are being read out. So, the ethernet data rate from each of the six ROACH board is therefore \(2 * 32 * 1024 * 488.28125 = 32.0\) Mbps which gives 192.0 Mbps in total. Once received, data packets are stripped of redundant values, leaving \(\sim\) 1500 pairs of 32-bit values to be saved to storage. The data rate to storage is therefore \(\sim\) 6 MB/s, which is well within the spec of the mirrored RAID array of SSDs on the muscat-mux computer. These raw data are stored in the Dirfile format, with timelines stored in binary format and sweep data in ASCII format. The dirfiles are named according to timestamps taken at the start of the recording and the observation log stores these names along with relevant observation metadata. Metadata for in-laboratory testing include links to files relating to beam mapping, blackbody load testing and FTS spectra measurements. LMT metadata such as programme-type, observation-number and scan-number and source-name are included in the commands received from the telescope control system and are used to match the dirfiles with the telescope pointing information files. The LMT control system is also able to request a retune of the readout tones on-demand, such as before and after an observation, or even mid-observation if required.

5.3 Data Analysis

A real-time interactive data display system based on the KST plotting software [14] has been developed that can display timestreams of incoming detector data with millions of data points with practically zero latency. Raw I and Q values, derived frequency and dissipation responses and the associated power spectral densities are available along with real-time filtering and annotation tools. This has been very useful for in-laboratory testing and will aid with initial commissioning activities at the site, such as alignment to the telescope and detector noise measurements.

When the camera is up and running at the telescope, the QLA system will provide feedback on observations to the operators. Simple on-the-fly map making, beam fitting analysis and beam parameter estimation have been implemented and tested successfully in the laboratory.

Processing of science observations will be done offline and no complete data processing pipeline currently exists for MUSCAT, although astronomers may be able to make use of the Toltec Citlali pipeline [13] as many key features of the MUSCAT data will be shared with that of Toltec.

6 Future Work

MUSCAT is currently on site at the LMT and being unpacked from its shipping crates. Following the initial hardware inspection, the system can be re-assembled, the cryostat can be cooled down, the performance of the detector readout system can be tested, and the interface to the LMT control system can be verified. Initial work will take place to align muscat to the telescope, and then, observations can be made to correct for pointing offsets and focussing adjustments. Detailed beam maps of bright point sources will be taken to verify the optical performance, and a noise and responsivity analysis will be undertaken to estimate the ultimate sensitivity and mapping speed. Variations in detector resonance frequencies with changing sky temperatures will need to be studied as a function of telescope elevation and atmospheric conditions. Sky dip measurements may be required prior to mapping observations in order to find the optimal detector tuning range for the readout for a given elevation range.

7 Conclusion

The MUSCAT camera for millimetre astronomy and its associated readout electronics have been tested in the laboratory, and the performance has been deemed suitable for deployment. The design of the readout system and some key results of the in-laboratory testing have been presented here. MUSCAT will soon begin its installation at the LMT and we anticipate that good quality science data will be in hand before the end of the current good weather season.