Introduction

In optics and photonics, partially coherent light is the norm rather than the exception and accounts for emission processes in stars, LEDs, thermal emitters, photovoltaics, luminescent and scintillating materials, as well as natural light for sensing the environment1. The partial coherence of light naturally emerges in various physical phenomena, such as light propagation in turbulent media and astronomy2. Partially coherent light is also used in advanced imaging, sensing, and communication modalities, such as optical coherence tomography, ghost imaging, stellar interferometry, and low-power optical trapping, to only name a few2. Partial coherence describes statistical correlations between degrees of freedom of a light field (such as spatial, spectral, polarization, etc.)2,3,4. This general description is particularly relevant in understanding phenomena that involve coupled degrees of freedom, such as polarization (meta)optics5,6 and imaging7, cross-spectral purity8, cylindrical vector beams9, and “classically entangled” photonic states10.

The coherency matrix \(\rho\)1,11 (or its quantum optical analog, the density matrix12) is generally used to characterize such partial coherence over arbitrary channels of a photonic system. Of particular interest is the basis of so-called “natural modes”13,14,15. We can express any spatially partially coherent optical field near some wavelength as a linear superposition of these modes, which have the important physical property that they are mutually incoherent (i.e., completely uncorrelated). Equivalently, any spatially partially coherent field can be decomposed into orthogonal and mutually incoherent parts. This decomposition is mathematically equivalent to finding the basis that diagonalizes the matrix \(\rho\)13,14,15. Methods to reconstruct \(\rho\) for few polarization-spatial channels have been demonstrated via projective measurements (e.g., for 4\(\times\)4 polarization \(\times\) spatial degrees of freedom16). Despite the ubiquity of partial coherence in optical phenomena, there is no general, scalable method to measure \(\rho\), nor apparently so far any physical method that separates it into its mutually incoherent parts.

Meshes of Mach-Zehnder interferometers (MZIs)17 have proven very effective at manipulating18 and measuring19 coherent multimode light. MZI meshes have been used to implement inference20 and training21 in optical neural networks, heuristic algorithms for combinatorial optimization22, simulation of quantum transport23, free space optical control18, and universal linear optics24,25. Central to these works is the fact that MZI meshes are universal linear photonic processors24. Specifically, self-configuring MZI networks can automatically learn unitary operators for coherent light processing24,26 and establish optimal communication channels27,28. However, the existing literature on MZI meshes predominantly concentrates on coherent light processing, largely overlooking the expansive potential in processing and analyzing incoherent or partially coherent multimode light.

Here, we propose a general method using self-configuring optics—“partially coherent light analyzers” (PCLA)—to fully measure the coherency matrix of partially coherent light near some wavelength; this method additionally separates the light into its mutually incoherent orthogonal components, whose powers appear separately in the output waveguides. Our method performs sequential power optimization over the N output channels of a self-configuring network, thereby learning the coherency matrix eigenvectors and eigenvalues. In this process, the unitary network is then also implementing the linear transform that diagonalizes the coherency matrix. If we separately calibrate the network19, we can deduce this diagonalizing transformation (and hence the eigenvectors) from the resulting network settings by simple arithmetic. Together with measurements of the relative output powers, which give the matrix eigenvalues, this process therefore measures this matrix. If we run our process in reverse, injecting mutually incoherent light backwards into the different output ports, we can generate arbitrary partially coherent fields emerging backwards from the input ports, without fundamental beamsplitting losses.

We illustrate our method in three distinct settings: (1) analyzing partially coherent environmental light from a scene; (2) generating partially coherent light with an arbitrary coherency matrix; (3) analyzing incoherent mixtures of single photons on an integrated photonic network. Our method therefore paves the way to full characterization, processing, and generation of partially coherent light, addressing significant untapped opportunities in fields such as environmental and astronomical sensing, quantum optics, and advanced imaging, in each of which partial coherence plays a fundamental role.

Results

Self-learning partially coherent light analyzers (PCLA)

We first describe the physics and learning procedure of PCLA in processing partially coherent light. Our approach can in principle process partial coherence over many spatial degrees of freedom of a light field and in various settings, with some examples shown in Fig. 1a. We collect the input light into N spatial “channels” or waveguides into the PCLA, using grating or other input couplers. Polarization splitting couplers that route different input polarizations to waves in the same polarization in different waveguides would add the ability simultaneously to process polarization degrees of freedom also24. Throughout this paper, we assume that the coherence length of the sources is much longer than any path length differences in the system.

Fig. 1
figure 1

Measuring and processing partially coherent light with self-configuring optics. a Partial coherence of light is observed in many photonic systems: between spatial modes in thermal light emission or luminescent materials (pumped by optical light or high-energy particles); partially polarized light in environmental sensing; or as incoherent mixtures of pure states in quantum optics. b Partial coherent light analyzer (PCLA): self-configuring networks can automatically analyze these different forms of multimode partially coherent light near some wavelength of interest. Input multimode partially coherent light is coupled into N waveguides that then feed a cascade of self-configuring layers (e.g., each a diagonal line or binary tree of Mach-Zehnder interferometers (MZIs)). These layers then learn a decomposition of the corresponding density matrix \(\rho\) via a sequential, layer-by-layer power optimization method relying on measurement and feedback. Each node of the array is a 2\(\times\)2 MZI. c The learned decomposition separates the mutually incoherent modes (eigenvectors of \(\rho\)) to generate outputs (1) from the “top” waveguide of each self-configuring layer. The resulting network settings give the eigenvectors. The output power of each such waveguide (2) gives the corresponding eigenvalue of \(\rho\); and (3) the number of output ports with non-zero power corresponds to the number of mutually incoherent modes (up to a maximum of N)

Full size image

Our PCLAs consist of a cascade of up to N self-configuring layers, such as diagonal lines24,26 (resulting in a triangular mesh29), binary tree layers19,26, or hybrid architectures19,30, all constructed from 2\(\times\)2 programmable interferometer blocks. Self-configuring layers can be defined topologically as ones in which there is one and only one path through these blocks from the “top” output (Fig. 1b) of the layer to each input to the layer19.

For concreteness, we consider integrated self-configuring layers, with the 2\(\times\)2 blocks implemented using integrated MZIs. Such MZIs are made from two phase shifters (\(\theta ,\phi\)) and two 50:50 directional couplers (Fig. 1b)19,26,31,32. Each layer has a single (“top”) output whose power is measured with a photodetector. That measurement is used to update the settings of that layer via electronic feedback. The photodetector could be an external or integrated photodiode and could also be designed just to sample a sufficient amount of power during measurement, leaving the majority of the separated output power for other purposes. Specifically, each layer optimizes (e.g., maximizes) the power at each detector by tuning the parameters (e.g., phase shifters) of that single layer. In the self-configuring geometry, the power output of a given layer is independent of the parameters of all subsequent layers, thereby reducing the number of degrees of freedom for each subsequent power optimization. The power optimization is sequential: the power output of the first layer is first maximized, then that of the second, and so forth.

Once the sequential power optimization has converged, the PCLA has learned a modal representation of the spatially partially coherent input light field corresponding to mutually incoherent modes13,14,15 (see Fig. 1c). These mutually incoherent modes do not produce interference patterns when mixed with a tunable phase (a feature that can be further checked experimentally with an analyzer network after the PCLA, as discussed later in this paper). In the process, the PCLA has learned the coherency matrix eigenvectors, which can then be deduced directly from the resulting settings of the (calibrated19) network elements. The corresponding eigenvalues \({\lambda }_{i}\) can be measured by reading out the average values of the output powers. Additionally, the number of output ports with non-zero average power corresponds to the number of such mutually incoherent modes, or the rank of the coherency matrix.

We now describe the PCLA learning procedure. Let us denote \({\rho }_{{\rm{in}}}\) as the coherency matrix of the input field. The coherency matrix is Hermitian semi-positive and can therefore be diagonalized as:

$${\rho }_{{\rm{in}}}={UD}{U}^{\dagger }$$

where \(U\) is the orthogonal basis of mutually incoherent eigenmodes and \(D\) a positive diagonal matrix corresponding to the average power in each mode \({\lambda }_{i}\ge\) \(0\). Characterizing \({\rho }_{{\rm{in}}}\) entails measuring the unitary operator \(U\) and the eigenvalues \({\lambda }_{i}\). We assume that power averaging is performed over timescales much longer than the coherence time of the measured sources.

A linear operation UPCLA on these channels transforms the coherency matrix as: \({\rho }_{{\rm{out}}}={U}_{{\rm{PCLA}}}\,{\rho }_{{\rm{in}}}{U}_{{\rm{PCLA}}}^{\dagger }\)11 (where \({\rho }_{{\rm{out}}}\) is the coherency matrix of the network output y). Each step of the algorithm consists in the maximization of the ensemble averaged power at the output port of one of the self-configuring layers. At step k, the network optimization is the following:

$$\mathop{\max }\limits_{{S}_{k}}{\left({\rho }_{{\rm{out}}}\right)}_{{kk}}={\lambda }_{k}$$

where \({\lambda }_{k}\) is the k-th largest eigenvalue of \({\rho }_{{\rm{in}}}\) (ordered such that \({\lambda }_{1}\ge \ldots \ge {\lambda }_{N}\)), and \({S}_{k}\) is the set of tunable parameters (phases) in the k-th self-configuring layer, corresponding to a set of MZI denoted \({M}_{k}\). This equality is a direct consequence of the min-max or variational theorem of linear algebra, whose conditions are naturally enforced in self-configuring networks due to the mutual orthogonality of the self-configuring layers24. We can then optimize the network settings sequentially for one layer at a time, and the relative power at output node k gives \({\lambda }_{k}\).

The PCLA therefore “diagonalizes” the coherency matrix \({\rho }_{{\rm{in}}}\), such that \({U}_{{PCLA}}={U}^{\dagger }\). Consequently, reading out the network parameters and output powers fully characterizes the coherency matrix. More details of the proof can be found in Section S1 of the Supplementary Information (SI). In the following, we illustrate this method in several settings where partial coherence of light is essential.

Once configured, one can know the values of the phase delays in the phase shifters by reading the applied voltages (or other control variables). Approaches to the necessary calibration of the phase shifters include progressive methods presuming 50:50 beamsplitters19,33, methods of setting up and calibrating “perfect” meshes even when the fabricated beamsplitters are not 50:5031, and approximate methods based on global optimization34. Therefore, once the PCLA has performed the sequential power optimization, we can read off these voltages or control values and deduce exactly the unitary matrix \({U}_{{\rm{PCLA}}}\) represented by the mesh19.

Environmental light processing with self-configuring Mach-Zehnder interferometer arrays

We now show how PCLAs can be used to analyze and process partially coherent light fields impinging on the PCLA from a scene, as shown in Fig. 2a. As a practical matter, the behavior of circuit components such as input couplers, waveguide beamsplitters and phase shifters will depend on the wavelength to some degree, but we presume that the spectral bandwidth of the input light is narrow enough or has been sufficiently filtered that we can approximately neglect such dependence for our discussion. The interferometer meshes themselves can be constructed with path lengths that are all essentially equal for all interfering components19, so the behavior of the meshes is otherwise essentially independent of wavelength.

Fig. 2
figure 2

Environmental light processing with self-configuring Mach-Zehnder interferometer arrays. a A PCLA can process environmental light input and learn its decomposition into mutually incoherent modes (eigenvalue decomposition of input density matrix \({\rho }_{{\rm{in}}}\)). b An incident wavefront of partially coherent waves is sent through a triangular array of MZI. A sequential power optimization learns a decomposition of the incident wavefront into mutually incoherent modes. c 4-mode example input and output coherency matrices, the latter resulting from the sequential power optimization. d Top: The power optimization routine sequentially maximizes the power at each output channel (with channel index i from N = 1 to 4), mapping each output sequentially to the corresponding coherency matrix eigenvalue (\({\lambda }_{i}\)). Inset: Matrix product showing that \({U}_{{\rm{PCLA}}}\) learns \({U}^{\dagger }\) (up to a diagonal matrix of phases). Bottom: Resulting fidelity over power optimization iteration. e Analyzer circuit, consisting in a single layer of nodes that can be configured in either one of three gates: identity, swap, and Hadamard gates (mixing). The output signals are then analyzed via balanced homodyne measurements

Full size image

We consider N “channels” of input light, whose fluctuating amplitudes are denoted by an N-dimensional vector x. The partial coherence of these channels is described by the coherency matrix ρin1, such that \({\left({\rho }_{{\rm{in}}}\right)}_{{ij}}=\langle {x}_{i}{x}_{j}^{* }\rangle\), where 〈⋅〉 denotes ensemble averaging (e.g., time averaging if we presume stationary ergodic fields1). Such quasi-monochromatic fields with fluctuating complex amplitudes may be generated by imaging incoherent light sources (which acquire partial coherence via the van Cittert-Zernike theorem1), fluctuating currents in (spectrally filtered) light emitted devices35,36, and lasers described by the van der Pol oscillator model1. Potential experimental implementations of our scheme are described in the SI, Section S6. For illustrative purposes, each node of the network in a 4-channel triangular array example (Fig. 2b) is labeled with the corresponding output port optimization color (with \({M}_{1}\) shown in orange, \({M}_{2}\) in purple, and \({M}_{3}\) in green, respectively).

We demonstrate the validity of our approach with numerical experiments in Fig. 2c, d with a 10-channel fluctuating input field, simulating light propagation with fluctuating amplitudes through a triangular array. As the power optimization is carried out, each channel’s output power gives the corresponding eigenvalue of \({\rho }_{{\rm{in}}}\). The corresponding unitary fidelity (defined as \(F={\left\langle \left|{U}_{{\rm{PCLA}}}U\right|,{\rm{Id}}\right\rangle }_{{\rm{HS}}}\)22, where 〈⋅〉HS is the Hilbert-Schmidt dot product and Id is the identity matrix) increases throughout the power optimization and reaches values > 0.99 after convergence, thereby showing the PCLA learns the eigenvalue decomposition of the coherency matrix with great accuracy.

Once configured, the fields in different output channels of this network should be mutually incoherent; if we then attempt to interfere each pair of outputs, we should see no interference between them as the relative phase of those outputs is varied. To test such mutual incoherence, one can use an additional output analyzer layer of MZIs, as in Fig. 2e, after the coherency diagonalization circuit \({U}_{{\rm{PCLA}}}={U}^{\dagger }\). To interfere any two outputs, the MZI nodes can be appropriately configured as (1) identity; (2) swap; (3) or mix (i.e., 50:50 splitter, as in Hadamard gates), shown in Fig. 2e, onto an output photodetector. Scanning the relative input phase using the analyzer input phase shifters should then produce no interference fringes (see Fig. 1c), which is equivalent to performing balanced homodyne measurements, yielding a zero-mean power. Details of the parameters and methods used in this numerical experiment can be found in Section S2 of the SI. A detailed schematic of the experimental apparatus to analyze partially coherent light with a PCLA is shown in the SI, Section S6.

PCLAs can also be used to generate multimode partially coherent light described by an arbitrary coherency matrix. Running the self-configuring network of this section in the backwards direction, as shown in Fig. 3, we illuminate its output ports with mutually incoherent sources with average powers corresponding to the desired eigenvalues of the coherency matrix \(\lambda_i\). The resulting coherency matrix emerging “backwards” on the input side is that of partially coherent light and described as in Eq. (1), choosing \({U}_{{PCLA}}={U}^{\dagger }\). Knowing the natural mode decomposition of the coherency matrix of a desired partially coherent light field of interest, one can therefore use PCLAs to generate such a light field using mutually incoherent sources of variable power.

Fig. 3
figure 3

Generation of multimode partially coherent light with PCLA. To generate multimode partially coherent light described by coherency matrix \({\rho }_{{\rm{in}}}\), the PCLA is run backwards and illuminated with mutually incoherent sources. Their amplitude is set, for example with modulators as shown, so that their average power matches the eigenvalues \({\lambda }_{i}\) of the coherency matrix

Full size image

Processing incoherent mixtures of delocalized single photons with PCLA

We now further generalize our method to analyzing incoherence in quantum optical systems, generally described by a density matrix \({\rho }_{{\rm{in}}}\), and illustrated in an integrated photonic network where single photons propagate (Fig. 4a).

Fig. 4
figure 4

Processing incoherent mixtures of delocalized single photons with PCLA. a An incoherent mixture of delocalized single photons (described by coherency matrix \({\rho }_{{\rm{in}}}\)) is sent through a PCLA. The result of projective measurements is used to sequentially optimize the power at each output mode. The result is a modal decomposition into mutually incoherent modes, where the number of pure quantum states in the mixture corresponds to the number of output modes with non-zero average power. b Experimental proposal, where single photons emitted at random times (therefore mutually incoherent) are mixed through a random unitary transformation and subsequently analyzed by the PCLA. c 10-mode example (with a total of 7 pure states in the mixture): the output clicks resulting from the sequential power optimization are ordered and map to the eigenvalues of \({\rho }_{{\rm{in}}}\). Inset: Matrix product showing that \({U}_{{\rm{PCLA}}}\) learns eigenvectors of \({\rho }_{{\rm{in}}}\) (up to a diagonal matrix of phases)

Full size image

We consider incoherent mixtures of single photons delocalized over N waveguide ports (Fig. 4a). The input mixed state is described by ρin = ∑ipi|ψi〉〈ψi|, with 0 < pi < 1. Note, as is typical with mixed states, that the different |ψi〉 and the corresponding optical waves arriving at the PCLA need not be orthogonal to one another. The PCLA imparts a unitary transformation to the wavefunction |ψout〉 = UPCLA|ψin〉, which corresponds to the following operation on the density matrix, similar to that on the coherency matrix in the \({\rho }_{{\rm{out}}}={U}_{{\rm{PCLA}}}\,{\rho }_{{\rm{in}}}\,{U}_{{\rm{PCLA}}}^{\dagger }\). Detectors on the PCLA output measure “clicks” corresponding to single photons, and the average power at a given output channel k is given by (ρout)kk. Therefore, a sequential power optimization analogous to that of the previous sections can be carried out to analyze incoherence of this quantum optical system.

The nature of this incoherent mixture of single photons is responsible for stochastic fluctuations of the output power. Specifically, stochasticity arises from two sources: (1) the classical incoherent mixture, from which each pure state |ψi〉 can be “picked” with probability pi; (2) projective quantum measurements on the output, with probability of clicking at output port k given by |〈k|UPCLA|ψi〉|2. Both effects can be modeled by a categorical distribution, and more details on our numerical implementation can be found in the SI, Section S3.

Both sources of randomness are simulated in the thought experiment shown in Fig. 4b, where a random unitary transformation is first imparted to single photons emitted at random times (thereby providing incoherence of the mixture on the output). In this example, we propagate a mixture of 7 pure states through the PCLA and perform a sequential power optimization as described in the previous sections. While the input density matrix was mixed, the output of the PCLA after optimization is ordered with decreasing mean power. The number of channels with non-zero mean power corresponds to the number of pure states in the mixture, and the PCLA has learned the modal (diagonal) representation of the density matrix that outputs mutually incoherent modes.

Discussion

We further discuss potential applications of our methods and experimental considerations for their realization. We have shown that PCLAs, which consist of self-configuring networks with sequentially optimized power outputs, can be utilized to automatically analyze the classical and quantum partial coherence of multimode optical light fields. Quite generally, our methods highlight the interplay between coherence and multimodal coupling in the analysis of partially coherent light fields.

The proper operation of PCLAs relies on a few key assumptions we have made about the light fields described by the input coherency matrix \({\rho }_{{\rm{in}}}\) and the network architecture: (1) the coherence length of the source must be greater than any path length differences between interfering beams in the MZI mesh; (2) the input light is quasi-monochromatic (such that we can neglect the wavelength-dependent behavior of the MZI mesh) and with fluctuating complex amplitude.

Our method also displays a few distinctive advantages compared to tomographic reconstruction of the coherence function16. Once the PCLA’s learning algorithm has converged, it will naturally separate the input light field into its mutually incoherent components. To put it differently, the PCLA acts as an “unscrambler” of partially coherent light into its mutually incoherent parts. The decomposition process is “lossless” (other than for practical coupling losses and background absorption and scattering losses in waveguide components); there is no fundamental beamsplitting loss in this system. In the SI, we also show that for a uniform loss across modes, the PCLA will still reconstruct the actual coherency matrix, with eigenvalues rescaled by the loss factor (see Section S3 of the Supplementary Information), which can be calibrated out by measuring the loss across each channel separately. However, due to the variational nature of our method, even in the presence of uniform losses, the unitary basis that diagonalizes \({\rho }_{{\rm{in}}}\) is learned by the PCLA. Further connections of our method to other modal representations of partially coherent light fields are discussed in Section S4 of the Supplementary Information. We also numerically analyze the influence of additional sources of noise, such as detection noise, on the performance of the algorithm. We still find fidelities >0.9 with relatively small number of algorithm steps (20) and signal-to-noise ratios on the order of 1.

In our numerical experiments, gradients of the time-averaged output powers were calculated using automatic differentiation and optimized with stochastic gradient descent37. In experimental implementations, various gradient calculation or measurement techniques could be used, such as in situ back-propagation36,38 or dithering18,28. Alternatively, methods such as physical gradient descent39 or gradient-free physical gradients40 could be used.

We also envision that PCLAs may find applications in partially coherent light holography and imaging. Once configured, the PCLA has stored the information of the incoming field distribution. A phase conjugated “hologram” of the fields is generated upon illumination of the PCLA from the backside with power-modulated incoherent sources, as shown in Fig. 3. If the incoming partially coherent light field is generated by an object, this process generates a phase-conjugated image of that object at the same position. Note that this goes beyond conventional holography in that it effectively encodes separate holograms for each of the mutually incoherent components of the original field and allows their simultaneous reconstruction.

In conclusion, we have shown that self-configuring photonic networks, such as triangular arrays of MZIs, can automatically learn and measure the coherency matrix of a multimodal light field across N channels. Our method generalizes to quantum optical systems, as long as enough degrees of freedom are available to implement arbitrary unitary transformations on their Hilbert space. We envision that this method will be experimentally relevant in processing, imaging, and analyzing the classical and quantum coherence of light and matter in all systems and applications where spatially partially coherent light emission is of importance.

Materials and methods

Learning process

The PCLA learns the coherency matrix of the input field in a step-by-step process: (1) Optimize the output power of the first layer, which aligns the input field with the largest eigenvalue of the coherency matrix; (2): Fix the first layer and optimize the second layer to align with the next largest eigenvalue, and so forth. The PCLA network ultimately diagonalizes the coherency matrix, separating the light field into its mutually incoherent components.

Numerical implementation of PCLA learning algorithm

The implementation of the learning algorithm is performed in a triangular mesh29, where the mesh settings are calculated to achieve the desired unitary transformations. The mesh is constructed by calculating the settings of the MZIs progressively, starting from the first layer, and ensuring that the matrix representing the mesh is a product of 2 × 2 matrices corresponding to each MZI.

Section S1 of the Supplementary Information provides a framework for understanding how partial coherent light analyzers work and how they can be implemented using self-configuring photonic networks, specifically with MZI arrays. The learning algorithm ensures that the network can separate and analyze partially coherent light fields effectively.

PCLA algorithm for quantum optical systems

The sequential power optimization algorithm used for classical partially coherent light can also be applied to quantum optical systems. The main difference lies in how randomness is accounted for, particularly in the output measurements. To calculate gradients effectively during optimization, we use a reparametrization trick, allowing us to perform automatic differentiation (see Section S3 of the Supplementary Information).

The algorithm to simulate average power measurements works as follows: (1) Randomly select states from the incoherent mixture based on their probabilities; (2) Simulate projective measurements on these states; (3) Average the measurements to determine the power at the output port.

Influence of losses

We consider the effect of losses on the accuracy of the PCLA algorithm in reconstructing the density matrix. We assume that losses are uniform across all channels and elements of the mixture. This assumption allows us to model losses as a uniform reduction in the amplitude of the quantum state, leading to a mixture of the transformed state and a vacuum state.

The resulting output density matrix reflects this loss, but the algorithm still identifies the correct eigenvector corresponding to the largest eigenvalue, albeit with a rescaled magnitude. In practical applications, the loss factor can be measured and used to adjust the estimated eigenvalues. More details can be found in Section S3 and S6 of the Supplementary Information.