2023/01/26 Meeting notes - GRS1747-312 dataset

[Victoria-Samboco]

In this meeting we discussed about the GRS1747-312 dataset that we already started the discussion here #20.
This is a 1GC calibrated observation . This is a 15 min observation from L-band.

On the dataset I ran the SolarKAT pipeline (that runs from SELFCAL to Sun imaging).
This resulted in the image of the Sun that can be found here #20. From this image it can be seen that there is a high level of aliasing, which can be resolved by running wsclean with the -use-wgridder parameter.

Wsclean was run with the parameters below.

wsclean -name img_sun/sun -data-column CORRECTED_DATA -size 10240 10240 -channels-out 8 -niter 0 -mgain 0.9 -weight briggs -1.0 -scale 1.1asec -join-channels -subtract-model -auto-threshold 1 -pol I -padding 1.3 -nwlayers-factor 1 -no-update-model-required -auto-mask 3 -temp-dir 1671435077_sdp_l0_1024ch_GRS1747-312.ms
The -save-source-list parameter was added to generate the file containing the list of sources to then predict the SUN-MODEL_DATA with crystalball.

It was also created a ds9 region (sun.reg) that contains the Sun to then peel it off:

• first in the CORRECTED rephased DATA that will allow us to clearly observe whether the Sun is being subtracted in its entirety or not (evaluate the level of peeling)-next step
• and then do the subtraction and peeling of the Sun in the original MS (before rephasing it)-next step

[Victoria-Samboco]

Ahmm... after a while of quatical running to peel the Sun from the rephased MS here are the results I got ...

where the first upper left image is the psf, the upper rigth is the dirty image; the bottom left is the residual image and the button right is the model image. The images appear like this, completely clean, apparently the sun has been subtracted, but I'm not sure if that's really what happened...I'm confused.

quartical_2:
      cab: quartical
      info: "Peel off the Sun from the MODEL_DATA"
      params:
        input_ms.path: '{recipe.ms}'
        input_ms.is_bda: false #true
        input_ms.weight_column: WEIGHT_SPECTRUM
        input_ms.time_chunk: '40'
        solver.terms: [dE]
        dE.type: diag_complex 
        dE.freq_interval: '64'
        dE.direction_dependent: true
        solver.iter_recipe: [100]
        input_model.recipe: MODEL_DATA~SUN_MODEL_DATA
        output.overwrite: 'true'
        output.products: [corrected_residual]
        output.columns: [DD_DATA]
        output.subtract_directions: [1]
        dask.threads: 16

[IanHeywood]

That does look suspiciously clean! Can you please post the residual for the image you made to create the clean component model for the Sun? Any structure that's in that residual image won't be in the model so we should expect that to remain in the peeled image also.

Can you please also check and post the times for the solution interval(s)? If it's solving every time slot and every 64 channels then it might be overfitting.

I'm not super familair with QuartiCal yet but it looks like there is no simultaneous G term being solved for, just a dE term. I'm assuming this then just behaves like regular self-calibration but with a model that only consists of MODEL_DATA - SUN_MODEL_DATA, which might be not what you want to do. What is the contents of the MODEL_DATA column?

Sorry for the numerous questions, I'm sure we can figure this out between us.

Thanks.

[Victoria-Samboco]

That does look suspiciously clean! Can you please post the residual for the image you made to create the clean component model for the Sun? Any structure that's in that residual image won't be in the model so we should expect that to remain in the peeled image also.

Here is the residual image for the Sun

Can you please also check and post the times for the solution interval(s)? If it's solving every time slot and every 64 channels then it might be overfitting.

I'm seeing now that I made a mistake, to peel the Sun I didn't set the solution interval...

I'm not super familair with QuartiCal yet but it looks like there is no simultaneous G term being solved for, just a dE term. I'm assuming this then just behaves like regular self-calibration but with a model that only consists of MODEL_DATA - SUN_MODEL_DATA, which might be not what you want to do. What is the contents of the MODEL_DATA column?

For the MODEL_DATA column, I'm not sure but I assume it contains the Sun model after the rephasing the MS. Because at the meeting it was suggested that I should do a test that consisted of peeling the Sun first from the rephased MS.

Sorry for the numerous questions, I'm sure we can figure this out between us.

Thanks.

[IanHeywood]

Some thoughts on how to proceed based on the chat just now...

• I'm assuming the solar imaging step begins after selfcal, with MODEL_DATA containing model visibilities for the target area (not including the Sun).

• Note that this field is a problematic one, in that it's close to the Galactic plane and thus contains extended structures, and also Sgr A is visible to the north about half way between the phase centre and the Sun. The initial images need to be large enough to encompass Sgr A.

• Following rephasing of each scan (only one in the case of this field) to the position of the Sun, I think we should experiment with deconvolution of the Sun to get a clean component model. As discussed a sensible option might be to have the script that finds the position of the Sun also generate a circular DS9 region file centred on that position with a diameter of 0.5 degrees. This can be turned into cleaning mask for the Sun using breizorro.

• Try deconvolving the Sun with a few thousand iterations using this mask and then examine the residual. If there are significant sunspot features remaining in the residual then the cleaning is too shallow, and the number of iterations should be increased. You can use CARTA's profile plots to examine the general background level inside the cleaning region and how that compares to the zero level and the general background outside that region to consider suitable cleaning thresholds. Please post residuals here for discussion if necessary. Note that the solar imaging should be performed with -no-update-model-required enabled inside wsclean or it will overwrite the MODEL_DATA column.

• Also be sure to deconvolve in a number of sub-bands to capture any spectral behaviour (which may or may not be quite odd out there in the primary beam sidelobe badlands).

• Once we're satisfied with the deconvolution we'll also have a model for the Sun (on a per-scan basis) in the form of model images or the source list text file. This should be predicted into a custom column in the MS (e.g. SUN_MODEL_DATA) using either wsclean in predict mode for the model images (my preferred method), or crystalball for the source list (Oleg's preferred method). For the latter you can just use the half-degree region file created for the deconvolution.

• Before even bothering to try peeling I would just try subtracting the solar model. This can be done by rephasing the visibilities to the target centre (making sure all columns including models are being rephased), subtracting SUN_MODEL_DATA from CORRECTED_DATA, e.g. with something like this, and then re-imaging the CORRECTED_DATA to see if the sidelobes from the out of field solar emission have been mitigated.

• I think there are a few things amiss with the peeling parset as posted above, but we can of course discuss those. But first I suggest we should get the deconvolution (to model of the Sun) correct, and try the straighforward subtraction before rushing into the peeling step.

Cheers.

[Victoria-Samboco]

Okay, will try it. Thanks for the notes. I will be sending here feedback, images and doubt of each step.

…

On Mon, 6 Feb 2023, 14:28 IanHeywood, ***@***.***> wrote: Some thoughts on how to proceed based on the chat just now... - I'm assuming the solar imaging step begins after selfcal, with MODEL_DATA containing model visibilities for the target area (not including the Sun). - Note that this field is a problematic one, in that it's close to the Galactic plane and thus contains extended structures, and also Sgr A is visible to the north about half way between the phase centre and the Sun. The initial images need to be large enough to encompass Sgr A. - Following rephasing of each scan (only one in the case of this field) to the position of the Sun, I think we should experiment with deconvolution of the Sun to get a clean component model. As discussed a sensible option might be to have the script that finds the position of the Sun also generate a circular DS9 region file centred on that position with a diameter of 0.5 degrees. This can be turned into cleaning mask for the Sun using breizorro. - Try deconvolving the Sun with a few thousand iterations using this mask and then examine the residual. If there are significant sunspot features remaining in the residual then the cleaning is too shallow, and the number of iterations should be increased. You can use CARTA's profile plots to examine the general background level inside the cleaning region and how that compares to the zero level and the general background outside that region to consider suitable cleaning thresholds. Please post residuals here for discussion if necessary. Note that the solar imaging should be performed with -no-update-model-required enabled inside wsclean or it will overwrite the MODEL_DATA column. - Also be sure to deconvolve in a number of sub-bands to capture any spectral behaviour (which may or may not be quite odd out there in the primary beam sidelobe badlands). - Once we're satisfied with the deconvolution we'll also have a model for the Sun (on a per-scan basis) in the form of model images or the source list text file. This should be predicted into a custom column in the MS (e.g. SUN_MODEL_DATA) using either wsclean in predict mode for the model images (my preferred method), or crystalball for the source list (Oleg's preferred method). For the latter you can just use the half-degree region file created for the deconvolution. - Before even bothering to try peeling I would just try subtracting the solar model. This can be done by rephasing the visibilities to the target centre (making sure all columns including models are being rephased), subtracting SUN_MODEL_DATA from CORRECTED_DATA, e.g. with something like this <https://github.com/IanHeywood/oxkat/blob/dev/tools/sum_MS_columns.py>, and then re-imaging the CORRECTED_DATA to see if the sidelobes from the out of field solar emission have been mitigated. - I think there are a few things amiss with the peeling parset as posted above, but we can of course discuss those. But first I suggest we should get the deconvolution (to model of the Sun) correct, and try the straighforward subtraction before rushing into the peeling step. Cheers. — Reply to this email directly, view it on GitHub <#21 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWYM2BNAJB2ZGJOQ2A66ORTWWDU6BANCNFSM6AAAAAAUHT3OPE> . You are receiving this because you authored the thread.Message ID: ***@***.***>