View Issue Details

IDProjectCategoryView StatusLast Update
0002161CCdciel[All Projects] Generalpublic19-08-12 23:10
ReporterAndre KovacsAssigned ToPatrick Chevalley 
PrioritynormalSeverityfeatureReproducibilityhave not tried
Status assignedResolutionopen 
PlatformMacOSmacOSOS Version10.13
Product Version0.9 
Target Version1.0Fixed in Version 
Summary0002161: Add support for Autofocus per FWHM and automatic inclusion of FITS headers for photometry
DescriptionHello,
I'd like to be able to use CCDciel for photometry, but I'm missing features to autofocus per FWHM (instead of HFD, to aim 8-10 FWHM per pixel for color cameras and 3-5 FWHM per pixel for mono cameras), and the automatic inclusion of required FITS headers (DATE ['YYYY-MM-DDThh:mm:ss' / Date data written], OBJRA and OBJDEC with float precision ['hh mm ss.ss'], AIRMASS [x.xxE+xx], PIXSIZE1/PIXSIZE2 [exactly the same as XPIXZ/YPIXZ], XBAYROFF/YBAYROFF [X/Y offset of Bayer array], BAYERPAT [Bayer color pattern], SCALE [image scale in arcsecs per pixel]).
Regards,
Andre
TagsNo tags attached.

Activities

Andre Kovacs

19-07-31 16:24

reporter   ~0005805

Hello,
I forgot to ask for some file naming options:
1- the addition of the option to include frame type (light, bias, dark, flat) in the name, via the Files -> File name options configuration pane (just like done in the Files -> Folder name options);
2- the change of the frame sequence number to a 3 digit pattern (001, 002, ...), for better file ordering;
3- and the use of the Separator to replace spaces in the object name, instead of the removal of any space present in the name.
Regards,
Andre

Patrick Chevalley

19-07-31 19:49

administrator   ~0005807

Hi Andre,

I already do a few change but also need more information.

- add FITS headers for PIXSIZE1, PIXSIZE2, XBAYROFF, YBAYROFF, BAYERPAT, SCALE, add decimal position for OBJRA, OBJDEC. the keyword DATE and AIRMASS are already present.
https://github.com/pchev/ccdciel/commit/86bb66013992cab26e02a2414dc84c9feff8c182

- add and option for fixed sequence number length, can be selected between 3 and 5:
https://github.com/pchev/ccdciel/commit/e6fba807543841d5c0db3aa7d7d55eb970c446ab

- use separator to replace blank in object name:
https://github.com/pchev/ccdciel/commit/7cf57a5a81ee1c673f95b4b9c0412cf031f5b76c

Now the question.

1) I understand why you need defocused image for photometry, but I not see how auto-focus can help here? Can you not use "manual focus" option and move the focuser until you get the required FWHM?

2) The frame type is included in the filename in place of the object name when the frame type is not Light. This is because it make no sens to use "M42_Bias" as part of the file name. Do you really require something like "M42_Light" instead of "M42"?


Andre Kovacs

19-07-31 23:06

reporter   ~0005810

Hi Patrick,

Thank you very much for your time.

Regarding your questions:
1) I understand why you need defocused image for photometry, but I not see how auto-focus can help here? Can you not use "manual focus" option and move the focuser until you get the required FWHM?
AK: Yes, you're right, but the defocusing process is currently really time consuming (takes me 20 mins), and almost never is spot on (I would like to aim for 9 FWHM per pixel, to have some slack for focus drifting, but I usually end getting around 8 or 10 FWHM per pixel). So, I'm looking for an automated way of doing this, to speed up the setup of my photometry sessions.

2) The frame type is included in the filename in place of the object name when the frame type is not Light. This is because it make no sens to use "M42_Bias" as part of the file name. Do you really require something like "M42_Light" instead of "M42"?
AK: Sure, the information is redundant, since I'm storing each frame type in its own subdirectory, but it's helpful when I have an specific frame for inspection, since I have to check the file's directory (or it's FITS header) to know which frame type it corresponds to.

I hope that answered your questions.

Regards,
Andre

Patrick Chevalley

19-08-01 14:28

administrator   ~0005812

1) This can be implemented as a variant of the Vcurve focusing method (this is impossible with the dynamic or iterative method).
Do you already use the Vcurve focusing?
Can you try to calibrate it, set the "Start focus" to 20, the "Near focus" to 9.
Then do some focus operation and note how reliably it go to HFD 9 before to reach the focus.
This help me if you can upload the CCDciel log for the session you do this operation.

2) I still not understand. Below is the content of the directory (no subdirectory configured) after I do two light, bias, dark and flat. The configured filename is "Object name" and "Filter":
Bias_1.fits
Bias_2.fits
Dark_1.fits
Dark_2.fits
Flat_L_1.fits
Flat_L_2.fits
NGC6543_L_1.fits
NGC6543_L_2.fits

For me it is clear what each file content is. How do you need to name this files in this case?

Andre Kovacs

19-08-09 16:12

reporter   ~0005820

Hi Patrick,

Unfortunately, I had (and still having) many Exposure fail! messages during the calibration and autofocusing.
Sorry, but had a hard time getting the calibration done. I'm still learning how to use CCDciel. To get the calibration working, I had to change the "Maximum defocused HFR" to 50 and "Minimum starting point focuser movement" to 10. But the Vcurve Learning complained that the calibration was poor.

Sorry, but what do you mean by "Then do some focus operation and note how reliably it go to HFD 9 before to reach the focus."? After the calibration I changed the "Start focus" to 20, and the "Near focus" to 9, as you asked me, and I tried the Autofocus button, but the Exposure fail! keeps happening.
Also, I changed the Exposure time used by Autofocus, but it keeps reverting to the default 15s.

Regarding the file names, I'm used to have my light frames also named as light types, but I think I can handle the CCDciel naming convention if the names would be like:
Bias_001.fits
Bias_002.fits
Dark_001.fits
Dark_002.fits
Flat_L_001.fits
Flat_L_002.fits
NGC6543_L_001.fits
NGC6543_L_002.fits
In this case, what the L flag stands for?

Regards,
Andre

logs.zip (2,177,985 bytes)

Patrick Chevalley

19-08-10 14:56

administrator   ~0005823

The _L_ part is the filter name, you just not have this part with a OSC camera. So Flat_001.fits and NGC6543_001.fits.
You can add other information in the Preferences, but I like this automatic "object name"/"frame type" switch because it make no sens to add an object name for the calibration frames.

About the auto-focus I think there is some basic point to fix before we can go further.

The message "Exposure fail!" is because the images are very slow to download from the INDI server on your system. From the log it frequently take between 28-29 seconds for a 600x600 subframe, but for the control exposure used for focusing I have a hard limit to not wait for more than 30 seconds after the end of exposure, so a very small fluctuation can produce the error.

I not understand this slow performance, with the focusing subframe it must be very quick. As a comparison I see in the log a full frame 5208x3476 download in 35 seconds. Proportionally to the image size the subframe must download in 0.7 second.

Is the camera directly connected to the Mac? or remotely?
If remote, please look at : https://www.ap-i.net/ccdciel/en/documentation/faq#camera_image_is_very_slow_to_download_from_remote_computer

Another issue is it sometime lock on a hot pixel instead of a star, this is visible in the log as a steady hfd=1.7 independent of the focuser position.
To avoid that you can create a bad pixel map: https://www.ap-i.net/ccdciel/en/documentation/bad_pixel_map
or add a dark frame to the program: https://www.ap-i.net/ccdciel/en/documentation/dark_frame

The focuser calibration must start from a position as near as possible from the best focus. You frequently start near hfd=8 . Is you system oversampled and this hfd the best you can get using the manual focus tool?
In this case do your camera support binning? it can be good if you can set the binning to 2x2 for focusing.

There is also large random fluctuation independent of the focuser position when you are near this best position. This can be the source of error for failed calibration that otherwise complete normally.
If this is because of seeing it is better if you use a longer exposure time, maybe 2 or 3 seconds to average the seeing. Use a fainter star if this one get overexposed.

Patrick Chevalley

19-08-10 15:10

administrator   ~0005824

I forget your remark about the exposure time that not change.
This is probably because an event is not handled correctly with Cocoa.
Many of this problem are solved with the last build I am working on, please install this version before to continue:
https://vega.ap-i.net/tmp/ccdciel-0.9.59-1709-x86_64-macosx.dmg

Andre Kovacs

19-08-11 06:49

reporter   ~0005829

Hi Patrick,

The download timeout is due to the fact that I'm using a Canon DSLR that doesn't have ROI, and it takes more than 25s to download each exposure frame.

I managed to create a bad pixel map using ISO 100 and 10s (47 bad pixels), and adjusting the sigma threshold to 8.
Should I decrease the exposure time?
Also, I got an "access violation" error every time I clicked OK on the File -> Preferences -> preview tab.

Finally, I tested again the autofocus after creating the bad pixel map, but it's still failing due to the "Exposure fail!" error.

Thanks,
Andre

Log_20190811_011149.log (6,000 bytes)

Patrick Chevalley

19-08-11 11:26

administrator   ~0005831

As you cannot improve the hardware performance I make a change to increase the wait delay to receive the image to 60 seconds. This must fix this Exposure fail!" error.
Please install this new version:
https://vega.ap-i.net/tmp/ccdciel-0.9.59-1716-x86_64-macosx.dmg

For the bad pixel map you can use an exposure high enough to clearly show the hot pixel, 10 second must be good. A higher sigma make less pixel detected as bad.
The bad pixel map is applied automatically to any preview image so you can the the efficiency. Decrease sigma if you still see hot pixel.
When the image is processed for bad pixel you see the following in the FITS header: COMMENT Corrected with Bap Pixel Map

About the error in the Preferences, can you zip and upload the file ccdciel_CG-4 & OAG.conf located in ~/.config/ccdciel/

Andre Kovacs

19-08-11 20:17

reporter   ~0005840

Hi Patrick,

This 60s wait delay includes the exposure time or only the download time?

Here's the ccdciel_CG-4 & OAG.conf from ~/.config/ccdciel/

Thanks,
Andre

Andre Kovacs

19-08-11 20:19

reporter   ~0005841

The attached didn't upload.
Trying again.

ccdciel_CG-4 & OAG.conf.zip (3,774 bytes)

Patrick Chevalley

19-08-12 23:10

administrator   ~0005842

The 60 second delay is after the exposure time as elapsed, so to let time for the image to download from the camera to the Indi driver and then send to CCDciel.

I make different test with your configuration file but I find nothing abnormal and get no crash.

Issue History

Date Modified Username Field Change
19-07-30 19:33 Andre Kovacs New Issue
19-07-31 16:24 Andre Kovacs Note Added: 0005805
19-07-31 19:49 Patrick Chevalley Assigned To => Patrick Chevalley
19-07-31 19:49 Patrick Chevalley Status new => feedback
19-07-31 19:49 Patrick Chevalley Target Version => 1.0
19-07-31 19:49 Patrick Chevalley Note Added: 0005807
19-07-31 23:06 Andre Kovacs Note Added: 0005810
19-07-31 23:06 Andre Kovacs Status feedback => assigned
19-08-01 14:28 Patrick Chevalley Note Added: 0005812
19-08-01 14:28 Patrick Chevalley Status assigned => feedback
19-08-09 16:12 Andre Kovacs File Added: logs.zip
19-08-09 16:12 Andre Kovacs Note Added: 0005820
19-08-09 16:12 Andre Kovacs Status feedback => assigned
19-08-10 14:56 Patrick Chevalley Status assigned => feedback
19-08-10 14:56 Patrick Chevalley Note Added: 0005823
19-08-10 15:10 Patrick Chevalley Note Added: 0005824
19-08-11 06:49 Andre Kovacs File Added: Log_20190811_011149.log
19-08-11 06:49 Andre Kovacs Note Added: 0005829
19-08-11 06:49 Andre Kovacs Status feedback => assigned
19-08-11 11:26 Patrick Chevalley Note Added: 0005831
19-08-11 20:17 Andre Kovacs Note Added: 0005840
19-08-11 20:19 Andre Kovacs File Added: ccdciel_CG-4 & OAG.conf.zip
19-08-11 20:19 Andre Kovacs Note Added: 0005841
19-08-12 23:10 Patrick Chevalley Note Added: 0005842