|
|
focus3.png (734,783 bytes) focus2.png (751,697 bytes) |
|
|
I have now also a problem without any filter, both with local V-curve focusing and Dynamic focusing. Why is in attached zipped FITS image no star detected? At the mouse pointer is indicating the HFD /FWHM but not with CCD inspector?? ccdciel.png (658,747 bytes) astap.png (369,179 bytes) |
|
|
The HFD values at mouse position are reported as 14.6, but no star found with CCD inspector ccdciel-2.png (717,201 bytes) |
|
|
It not work if you start with this level of defocusing because the function MeasureImage use a window of fixed size of 14 pixels to detect the stars and trying with your image it require a window of at least 30. But I keep this value of 14 you initially set because if make too much false detection if too big. This is not a problem if in place dynamic focus is done only to refine the focus because the stars position are measured only on the first image, then it reuse the saved position with a bigger window to measure the defocused images. It work with the mouse pointer and double click because in this case it use the configured detection window instead of 14. This is not a easy problem because Astap work fine with this image but with my focused image with many stars it mark too much group of two or more stars as a single big star. |
|
|
Here an example of a focused image in Astap and Ccdciel. In Ccdciel I try to remove as much as multiple stars, big saturated stars, the cluster in the center, to try to keep only good quality stars for focusing. But this can probably not work when starting with a defocused image. focused_ccdciel.jpg (140,897 bytes) focused_astap.jpg (190,668 bytes) |
|
|
In Astap I have an adaptive ri, region of interest up to a maximum of 50 radius using a histogram. The problem image has a HFD of 14.6. It would be ideal if CCDciel could detect these HFD values without detecting double stars. It will required a ri of around 4*14.6 I agree Astap detects some double stars. That should not happen either. Let me see if I can improve this. First in Astap and then compare with CCDciel. Adaptive ri and no double star detection. I still don't understand why I had false star detection's last night. This should not happen. CCDCiel preformed fine a few days ago and suddenly not. |
|
|
The last change I do 4 days ago was to do more rejection to not select galaxies or group of stars for focusing. One of the change is to compare each star HFD with the median HFD and reject if the difference is more than 0.5*medianHFD. The other change is for the group of stars, it reject the position if the ratio HFD/FWHM is greater than 4. Because of the way we measure the FWHM by counting pixels this allow to detect a wrong HFD for a group of stars. |
|
|
Last night I returned to the previous version but had similar problems. Something to investigate Anyhow I'm experimenting. I think one of the problem lies in calculating the background value. Up to now it is the mean/average. For crowded star fields a star could easily be included in the mean. That's a big outlier. A median rather then mean approach will probably do better. Using the median as background, a star in the background will increase the standard deviation of the median background. That will raise the detection level for the star detection. |
|
|
Just for testing I replace the line 7895 fits.GetStarList... in MeasureImage procedure to retry using the configured window if the first detection with small window return no stars. This work and it detect a median HFD of 14.4. retry:=0; repeat fits.GetStarList(rx,ry,s,true); {search stars in fits image} nhfd:=Length(fits.StarList); if nhfd=0 then begin inc(retry); s:=Starwindow div fits.HeaderInfo.BinX; // maybe a defocused image, retry with configured star detection window rx:=img_Width-6*s; ry:=img_Height-6*s; end; until (nhfd>0) or (retry>1); retry_with_bigger_window.jpg (58,313 bytes) |
|
|
Yes that works much better ! M11 m11.png (907,246 bytes) |
|
|
This is the median routine I developed for removing deep sky objects and replace the area with the background color. (Astap, hold right mouse and pull a rectangle) It looks to peek of the value histogram. This seems to work better then the average background routine. With this implemented I assume you can relax the the ri of 14. function get_median(img :image_array;colorm,xmin,xmax,ymin,ymax:integer):integer;{find the median of a local area} var i,j,k,col,median_count :integer; begin if xmin<0 then xmin:=0; if xmax>width2-1 then xmax:=width2-1; if ymin<0 then ymin:=0; if ymax>height2-1 then ymax:=height2-1; for i := 0 to 65535 do histogram[i] := 0;{clear histogram} For i:=ymin to ymax do begin for j:=xmin to xmax do begin col:=round(img[colorM,j,i]);{get one color value} if nrbits=24 then col:=intensity2(col);{average the 3 colors} if ((col>=1) and (col<40000)) then {ignore black areas and bright stars} inc(histogram[col],1);{calculate histogram} end;{j} end; {i} result:=0; {for case histogram is empthy due to black area} median_count:=0; for i := 1 to 65535 do {get median but ignore 0} begin if histogram[i]>median_count then begin median_count:=histogram[i]; {find median} result:=i; end; end; end; |
|
|
Good result! this image look really difficult to measure and the result is very good. Yes, histogram peak is normally a good background estimate. Can we use the histogram width at some level to estimate the background deviation to only get stars above this value? |
|
|
Correction, what I did is not the median (middle) but the most common value. The histogram will be asymmetrical and have left width en right width . The width of the left (below) the most common value will indicate the standard deviation of the background value. Right (above) the most commonk value will indicate background deviation plus star noise. The whole image will be polluted by faint background stars. They should be included in the standard deviation of the background. I assume it is better to calculate the standard deviation from the corners of the regio of interest but let me experiment. We will have few hours clear sky now , but I can propose tomorrow a new HFD routine for CCDCiel. Is that okay? |
|
|
Yes sure I let you experiment and I not touch the code for now. For me it's cloudy with small rain for tonight and tomorrow I am away for two days, so you can take your time with that. |
|
|
Hello Patrick, Well it it was more work then anticipated. I did a lot of experiments. Attached a new cu_fit.pas. It has a new procedure getHFD2. It works only for image inspection and I only tested it using the image inspector button. See line 1905 To prevent double star detection, I added an asymmetry test. If the box of interest is asymmetrical (due to neighbour stars) the rs box is reduced and center of gravity and asymmetrical is retested in a repeat until loop. If the box gets too small it exits with hfd:=-1; If the star doesn't show a minimum distance_histogram, so the box is too small it is also aborted with an hfd:=-1 The background and standard deviation calculation is improved. More pixels are tested. I didn't use the median. The distance_histogram calculation uses now the subpixel function. That was also missing. I also removed FimageMin in getHFD2. I think it did more harm then good. Some important parts are marked with ## The findstarpos and getHFD are combined in getHFD2. It made it easier to program and only one center of gravity routine is required. I think it works fine, but please test it. Regards, Han |
|
|
Some last corrections. Found one and removed a FimageMin Noted that box size s has value 16. Try 30 to see it filters out most double stars. |
|
|
Removed an s:=30 used for testing. |
|
|
Hello Patrick, Today I did a lot of testing against attached test image. It will only detect all stars if you make variable s large as 60 and overlap maybe 20. I added a new nice feature to procedure getHFD2. A a drop-off test for finding the required ri. The routine works now fine both for dense star field and de-focused stars up hfd=25 diameter Two factors in the getHFD2 procedure are important: af:=min(0.9,ri/10) (distance_histogram[ri]<=0.1*distance_top_value {##drop-off detection}) I also removed all /FimageC in the getHFD2 procedure. Attached a test image with several de-focused stars and the latest cu_fits.pas I will now focus on some other things till you had time to look to it. Han |
|
|
Hello Han, I return and test your last version in cu_fits-4.zip This look very good with many stars detected but no group wrongly detected. I try s:=16 and s:=30 in Tf_main.MeasureImage , 16 give more detection on focused images in dense field but 30 detect larger defocused stars. With 30 there is no wrong detection in focused image, only less stars detected because of more rejection. Your test image with all size and shape is very useful for testing. Maybe the asymmetry factor can be relaxed a bit to allow for more collimation error? FimageMin is not important here, it only make the image values to start at zero. But we must remember to add it if we want to use the bg and vmax value returned by gethfd2, for example to check the star is not saturated. |
|
|
Hello Patrick, Yes that is something to play with. First I had af:=min(0.9,rs/15) then I changed it to af:=min(0.95,rs/10). The min(0.95,rs/10) gave less false star group detection's in crowded dense star images. The 0.95 indicates it will accept large defocused images with a +-0.05 unbalance. By the rs/10 is will accept a larger unbalance as soon RS is smaller then 10. For a rs is 5 is will accept an unbalance of +-0.5. You have to choose between accepting collimation errors or having false star group detections. My settings are empirical, so please try yourself. Maybe af:=min(0.9,rs/5) will also work well or even better. Anyhow the new routine shows a better performance in ASTAP, so i think the principle of the setup is good. Today I will focus on processing my images from the last two nights. Han |
|
|
Hello Han, I commit the actual state with the adaptation required in the other functions and using a default window of s=20. This allow to make further change from this state. https://github.com/pchev/ccdciel/commit/4a784482c13bd2e0cbe2426347fff0a6699e2786 Autofocus work good with the simulator using the new function as the first step to detect the stars, then using the standard gethfd() to make the defocused measurement. Unfortunately I will have bad weather for a few days more, so no real sky testing for me. |
|
|
I try a way to first detect the image median HFD using the large window to set the measurement window size "s". To help testing I commit in https://github.com/pchev/ccdciel/commit/4dd3c11591ea3f5d4cd33d0e653f7761c71049c9 With this change your file "no star detected.fits" show a median HFD of 14.5 from 6 stars in "Image inspection" (with configured star window of 80) It even measure 12 stars if I relax the asymmetry factor to 0.75. |
|
|
Hello Patrick, I tested the new code and yes all HFD measurement by ccd inspector look good. Both the file no-stars.fits and crowded star field are measured correctly. I noted you test the image by 60x60 boxes minus a small offset. This is fine. I do it a little different to catch all stars but you need only the brightest. I did a test this using an artificial star. I noted two major problems: - Dynamic testing, It test only below the start focus point. It should measure HFD below and above the start focus position. See last log. - If no star is selected, the SNR is reported around 17. This should be be below 3. I will send all logs by mail. I have no time anymore and will be away tomorrow. I can look to the topics again on Sunday. Regards, Han |
|
|
Han, The problem with the off center dynamic curve must be solved if the solution for 0001914 fix the problem with your focuser. You meant the snr reported by GetHFD2 ? In this case valmax and bg probably need to be scaled with FImageMin and FImageC before the computation. But I failed to make a test because when there is no star it exit before snr computation from line 1777: if sumval<=10*bg_standard_deviation then exit; Similarly a double click on an empty area exit without snr because FindStarPos valmax is zero. |
|
|
Hello Patrick, I'm just back. I refer to the log below. In getHFD2, at the beginning of the routine the SNR is set to zero, so I can't explain these values. I will test the mod with the artificial star tonight and have a look using the debugger. 2018-03-23T22:36:20.103 AutoFocus started, initial position: 28000 2018-03-23T22:36:21.850 Autofocus running, hfd=0.7 peak:812.6 snr:21.6 2018-03-23T22:36:21.866 ASCOM.USB_Focus.Focuser: Focuser move to 25200 - 400 Backlash compensation 2018-03-23T22:36:41.678 ASCOM.USB_Focus.Focuser: Focuser move to 25200 2018-03-23T22:37:01.911 Autofocus running, hfd=0.7 peak:597.9 snr:17.2 2018-03-23T22:37:01.927 ASCOM.USB_Focus.Focuser: Focuser move to 22400 2018-03-23T22:37:07.855 Autofocus running, hfd=0.7 peak:568.4 snr:16.6 2018-03-23T22:37:07.870 ASCOM.USB_Focus.Focuser: Focuser move to 22800 2018-03-23T22:37:13.720 Autofocus running, hfd=0.7 peak:627.9 snr:17.9 2018-03-23T22:37:13.736 ASCOM.USB_Focus.Focuser: Focuser move to 23200 2018-03-23T22:37:19.071 Autofocus running, hfd=0.7 peak:505.5 snr:15.2 2018-03-23T22:37:19.711 ASCOM.USB_Focus.Focuser: Focuser move to 23600 2018-03-23T22:37:25.062 Autofocus running, hfd=0.7 peak:637.7 snr:18.1 2018-03-23T22:37:25.077 ASCOM.USB_Focus.Focuser: Focuser move to 24000 2018-03-23T22:37:30.709 Autofocus running, hfd=6.8 peak:532.0 snr:15.8 2018-03-23T22:37:30.724 ASCOM.USB_Focus.Focuser: Focuser move to 24400 2018-03-23T22:37:36.793 Autofocus running, hfd=0.7 peak:569.2 snr:16.6 2018-03-23T22:37:36.824 ASCOM.USB_Focus.Focuser: Focuser move to 24800 2018-03-23T22:37:42.674 Autofocus running, hfd=0.7 peak:620.6 snr:17.7 2018-03-23T22:37:42.690 ASCOM.USB_Focus.Focuser: Focuser move to 25200 For code efficiency it is possible better not to use (1/af)* but divide the factor behind. old asymmetry:=( (val_00<af*val_11) or (val_00>(1/af)*val_11) or {diagonal asymmetry} {has shape large asymmetry?} (val_01<af*val_10) or (val_01>(1/af)*val_10) or {diagonal asymmetry} ((val_00+val_01)<af*(val_10+val_11)) or ((val_00+val_01)>(1/af)*(val_10+val_11)) or {east west asymmetry} ((val_00+val_10)<af*(val_01+val_11)) or ((val_00+val_10)>(1/af)*(val_01+val_11)) ); {north south asymmetry} New: asymmetry:=( (val_00<af*val_11) or (val_00>val_11/af) or {diagonal asymmetry} {has shape large asymmetry?} (val_01<af*val_10) or (val_01>val_10/af) or {diagonal asymmetry} ((val_00+val_01)<af*(val_10+val_11)) or ((val_00+val_01)>(val_10+val_11)/af) or {east west asymmetry} ((val_00+val_10)<af*(val_01+val_11)) or ((val_00+val_10)>(val_01+val_11)/af) ); {north south asymmetry} And some extra comments added: val_00:=val_00+ value_subpixel(xc+i,yc+j)-bg; {value top left} val_01:=val_01+ value_subpixel(xc+i,yc-j)-bg; {value bottom left} val_10:=val_10+ value_subpixel(xc-i,yc+j)-bg; {value top right} val_11:=val_11+ value_subpixel(xc-i,yc-j)-bg; {value bottom right} |
|
|
Did a field test with an artificial star. Both V-curve and dynamic focusing worked well!. The SNR problem happened only once. (Dynamic, slew to focus star, button autofocus) The program misses the focus star but still measures a constant signal value of about 600 or 700 = HFD=17 and produces log like below. Next time it all works normal, star is in view and flux is higher and follows focuser position. With th torche switched off, the focus routine is aborted immediatly after start on low SNR. So I assume it is something with the SNR calculation. So except for the small SNR problem, it looks good! Now waiting for clear skies for test with some stars. Strange log where star lock was not working: 2018-03-24T22:44:28.818 Autofocus start Dynamic curve 2018-03-24T22:44:28.896 Focuser temperature: 8.8 2018-03-24T22:44:28.912 AutoFocus started, initial position: 26705 2018-03-24T22:44:29.271 Camera frame x=1474 y=676 width=404 height=400 2018-03-24T22:44:30.846 Autofocus running, hfd=0.7 peak:659.7 snr:18.5 2018-03-24T22:44:30.862 ASCOM.USB_Focus.Focuser: Focuser move to 22505 - 400 Backlash compensation 2018-03-24T22:44:59.847 ASCOM.USB_Focus.Focuser: Focuser move to 22505 2018-03-24T22:45:05.681 Autofocus running, hfd=0.7 peak:687.2 snr:19.1 2018-03-24T22:45:05.697 ASCOM.USB_Focus.Focuser: Focuser move to 23105 2018-03-24T22:45:12.451 Autofocus running, hfd=0.7 peak:663.2 snr:18.6 2018-03-24T22:45:12.701 ASCOM.USB_Focus.Focuser: Focuser move to 23705 2018-03-24T22:45:19.659 Autofocus running, hfd=0.7 peak:799.1 snr:21.3 2018-03-24T22:45:19.674 ASCOM.USB_Focus.Focuser: Focuser move to 24305 2018-03-24T22:45:26.523 Autofocus running, hfd=0.7 peak:616.2 snr:17.6 2018-03-24T22:45:26.725 ASCOM.USB_Focus.Focuser: Focuser move to 24905 2018-03-24T22:45:33.449 Autofocus running, hfd=0.7 peak:620.8 snr:17.7 2018-03-24T22:45:33.480 ASCOM.USB_Focus.Focuser: Focuser move to 25505 2018-03-24T22:45:40.110 Autofocus running, hfd=0.7 peak:796.0 snr:21.2 2018-03-24T22:45:40.765 ASCOM.USB_Focus.Focuser: Focuser move to 26105 2018-03-24T22:45:47.614 Autofocus running, hfd=0.7 peak:663.5 snr:18.6 2018-03-24T22:45:47.645 ASCOM.USB_Focus.Focuser: Focuser move to 26705 2018-03-24T22:45:54.805 Autofocus running, hfd=0.7 peak:640.4 snr:18.1 2018-03-24T22:45:54.852 ASCOM.USB_Focus.Focuser: Focuser move to 27305 2018-03-24T22:46:01.685 Autofocus running, hfd=0.7 peak:747.3 snr:20.3 2018-03-24T22:46:01.701 ASCOM.USB_Focus.Focuser: Focuser move to 27905 2018-03-24T22:46:08.268 Autofocus running, hfd=0.7 peak:723.4 snr:19.8 2018-03-24T22:46:08.830 ASCOM.USB_Focus.Focuser: Focuser move to 28505 2018-03-24T22:46:15.912 Autofocus running, hfd=0.7 peak:683.1 snr:19.0 2018-03-24T22:46:15.928 ASCOM.USB_Focus.Focuser: Focuser move to 29105 2018-03-24T22:46:22.558 Autofocus running, hfd=0.7 peak:742.7 snr:20.2 2018-03-24T22:46:22.854 ASCOM.USB_Focus.Focuser: Focuser move to 29705 2018-03-24T22:46:29.812 Autofocus running, hfd=0.7 peak:702.3 snr:19.4 2018-03-24T22:46:29.827 ASCOM.USB_Focus.Focuser: Focuser move to 30305 2018-03-24T22:46:36.395 Autofocus running, hfd=0.7 peak:632.0 snr:17.9 2018-03-24T22:46:36.894 ASCOM.USB_Focus.Focuser: Focuser move to 30905 2018-03-24T22:46:43.945 Autofocus running, hfd=0.7 peak:788.2 snr:21.1 2018-03-24T22:46:43.961 Too small HFD difference, try to increase the number of point or the movement, or run the focuser calibration tool to measure this parameters. 2018-03-24T22:46:43.977 The focuser is now positioned at the best observed HFD position. 2018-03-24T22:46:43.992 ASCOM.USB_Focus.Focuser: Focuser move to 26705 - 400 Backlash compensation 2018-03-24T22:47:12.977 ASCOM.USB_Focus.Focuser: Focuser move to 26705 2018-03-24T22:47:19.139 Autofocus finished, POS=26705 HFD=0.7 PEAK:667.8 SNR:18.7 TEMP:8.8 Next run: 2018-03-24T22:47:58.077 AutoFocus started, initial position: 26705 2018-03-24T22:47:59.855 Autofocus running, hfd=12.8 peak:12849.5 snr:110.7 2018-03-24T22:47:59.886 ASCOM.USB_Focus.Focuser: Focuser move to 22505 - 400 Backlash compensation 2018-03-24T22:48:29.121 ASCOM.USB_Focus.Focuser: Focuser move to 22505 2018-03-24T22:48:35.423 Autofocus running, hfd=32.3 peak:4727.3 snr:64.7 2018-03-24T22:48:35.439 ASCOM.USB_Focus.Focuser: Focuser move to 23105 2018-03-24T22:48:41.913 Autofocus running, hfd=29.1 peak:5166.6 snr:67.9 2018-03-24T22:48:41.928 ASCOM.USB_Focus.Focuser: Focuser move to 23705 2018-03-24T22:48:48.433 Autofocus running, hfd=25.6 peak:5628.2 snr:71.2 2018-03-24T22:48:48.465 ASCOM.USB_Focus.Focuser: Focuser move to 24305 2018-03-24T22:48:55.204 Autofocus running, hfd=22.4 peak:6156.4 snr:74.8 2018-03-24T22:48:55.235 ASCOM.USB_Focus.Focuser: Focuser move to 24905 2018-03-24T22:49:02.083 Autofocus running, hfd=19.2 peak:7341.1 snr:82.3 2018-03-24T22:49:02.115 ASCOM.USB_Focus.Focuser: Focuser move to 25505 2018-03-24T22:49:08.713 Autofocus running, hfd=15.8 peak:8936.3 snr:91.4 2018-03-24T22:49:09.213 ASCOM.USB_Focus.Focuser: Focuser move to 26105 2018-03-24T22:49:16.701 Autofocus running, hfd=13.7 peak:11509.4 snr:104.5 2018-03-24T22:49:17.215 ASCOM.USB_Focus.Focuser: Focuser move to 26705 2018-03-24T22:49:24.329 Autofocus running, hfd=12.8 peak:13036.2 snr:111.6 2018-03-24T22:49:24.345 ASCOM.USB_Focus.Focuser: Focuser move to 27305 2018-03-24T22:49:30.928 Autofocus running, hfd=13.1 peak:13432.0 snr:113.3 2018-03-24T22:49:31.271 ASCOM.USB_Focus.Focuser: Focuser move to 27905 2018-03-24T22:49:38.229 Autofocus running, hfd=14.5 peak:11161.1 snr:102.9 2018-03-24T22:49:38.260 ASCOM.USB_Focus.Focuser: Focuser move to 28505 2018-03-24T22:49:45.093 Autofocus running, hfd=16.7 peak:7992.8 snr:86.2 2018-03-24T22:49:45.311 ASCOM.USB_Focus.Focuser: Focuser move to 29105 2018-03-24T22:49:52.003 Autofocus running, hfd=19.7 peak:5889.3 snr:73.0 2018-03-24T22:49:52.019 ASCOM.USB_Focus.Focuser: Focuser move to 29705 2018-03-24T22:49:58.134 Autofocus running, hfd=23.1 peak:4921.4 snr:66.2 2018-03-24T22:49:58.150 ASCOM.USB_Focus.Focuser: Focuser move to 30305 2018-03-24T22:50:05.529 Autofocus running, hfd=26.4 peak:3813.6 snr:57.3 2018-03-24T22:50:05.544 ASCOM.USB_Focus.Focuser: Focuser move to 30905 2018-03-24T22:50:12.237 Autofocus running, hfd=30.3 peak:2813.2 snr:48.1 2018-03-24T22:50:12.377 HYPERBOLA curve fitting focus at 8.350, remaining curve fits error 0.0420, iteration cycles 3 2018-03-24T22:50:12.408 ASCOM.USB_Focus.Focuser: Focuser move to 26315 - 400 Backlash compensation 2018-03-24T22:50:43.561 ASCOM.USB_Focus.Focuser: Focuser move to 26315 2018-03-24T22:50:47.446 ASCOM.USB_Focus.Focuser: Focuser move to 26915 2018-03-24T22:50:54.403 Autofocus finished, POS=26915 HFD=12.7 PEAK:13485.5 SNR:113.6 TEMP:8.9 2018-03-24T22:50:54.637 Camera frame x=0 y=0 width=2328 height=1760 2018-03-24T22:50:55.449 AutoFocus successful |
|
|
In the message : Autofocus running, hfd=0.7 peak:659.7 snr:18.5 The values for multiple stars are as follow: the HFD is the median, the peak is the highest value, the SNR is the lowest value. With a HFD=0.7 this look like a faint hot pixel with peak intensity between 600 and 700. What is the bg level on this image? do you save the image that produce this result? But I not understand how this can get to the focus star list as in GetStarList at line 1903 we check for HFD>0.8 before to add it to the list. And the same check is done in MeasureStarList. |
|
|
Sorry, just realize this is with a single star. I look this not accept this bad detection. |
|
|
With this change it also reject HFD<0.8 for a single star measurement: https://github.com/pchev/ccdciel/commit/bfa48071aa92abaf0161ad27a831e4ae1fcc6f68 The SNR itself is not the problem, it indicate the detection of the hot pixel. If there is really only noise the HFD is not measured because no pixels have a value > 5* bgdev. |
|
|
I apply the asymmetry code change. But the release build is compiled with -O3 so the compiler will mix that anyway. |
|
|
I will test again when it is dark here and see if it was a hot pixel. I also tried yesterday "stay local" equals focus on "multiple stars". I could strangely not find the bright artificial star but only a small artifact. I will create a multiply artificial stars by making more holes in the aluminium foil. Making very small holes is not so simple. The current hole produce an lowest HFD of 12 at maybe 10 meter distance but that can be compensated by making large steps. A longer garden would help but I don't have that. Binning of 4x4 will also help but it didn't work last night. |
|
|
It is also possible that it is detecting cosmic radiation and not a hot pixel. Then a new bad pixel map will not help. Testing will demonstrate this. |
|
|
Inspecting my flat darks of 4.99 seconds, I found one dark containing a double dot of about 46 times the standard deviation (36) above the background (300) and an other dark containing a single dot 61 times the standard deviation above the background. So this cosmic radiation or noise is real. It looks like in most cases it is a single pixel. A minimum required HFD or multiple stars focusing could filter it out. It's a once off, so it can't explain the problem for the SNR where several images indicate more or less the same peak value. That should be caused by an hot pixel. But the pixel triggered by cosmic radiation could start the event, so a minimum HFD will be beneficial to prevent this happening. |
|
|
A cosmic ray must be rejected as the hot pixels by accepting only HFD>0.8 I commit another change to reduce the asymmetry factor to 0.9 and accept a measurement windows up to the configured size in the Focus tab. Otherwise the attached image failed. This is typical of my newtonian at the beginning of the night when the tube is still not cooled. bad_shape.png (243,087 bytes) |
|
|
star_detection.png (743,501 bytes) |
|
|
I had some clear sky this night and did some successfull testing Both V-curve and Dynamic focusing worked fine. Focus was reached without any mayor problem. So current setup works However there is still some improvement possible: 1) I'm not fully happy with the detection. Still some large star fields are detected as an focus star. See attached. If a human can distinguish between a donut and a few stars, a program should be able to do the same. Maybe only asymmetry testing is not the ideal solution. While reducing rs, maybe the program should not only look to assymetry but also if 40 or 50% of the box pixels are illuminated. Will test that tommorow. This should be valid for the newton donut but not for two or three stars in the box. 2) V-learn. The detection box doesn't move after a second V-learn run. If the telescope has moved a little after the first V-lean, it seems not to adapt and misses the star. 3) Final best focus position of a V-learn run is not used as the next focus position |
|
|
Hello Han, Nice you can make this testing, I am still under clouds here :( About 1), in the previous version I have a test to check the hfd/fwhm ratio to reject this false group detection. This use the fact that fwhm is actually a count of the illuminated pixels. But it was wrongly placed in GetStarList() and I remove it when I introduce your GetHFD2. I add it again but at the right place in GetHFD2: https://github.com/pchev/ccdciel/commit/f935a0a1e3ac595364132b640a0ab61a68a415ee As you can see in the attached screenshot this is very efficient. On the left is the version without this change, almost no real stars are measured because of very big group that take the space. On the right with this simple test it find the real stars. I will also look at the problem with the Vcurve star centering. For 3) you meant to update the value in "Focus position" at the top of Vcurve learning? hfd-fwhm.jpg (154,791 bytes) |
|
|
Routine below seems to work well to filter out star groups. It checks if in the final box (2rix2ri) 35% of the pixels are illuminated. See variable pixels_illuminated. I also tried something in the repeat .. until loop but that was no success last night. Will try to test it this night. procedure TFits.GetHFD2(x,y,s: integer; out xc,yc,bg,bg_standard_deviation,hfd,star_fwhm,valmax,snr: double); // x,y, s, test location x,y and box size s x s // xc,yc, center of gravity // bg, background value // bf_standard_deviation, standard deviation of background // hfd, Half Flux Diameter of star disk // star_fwhm, Full Width Half Maximum of star disk // valmax, maximum value of brightest pixel in final test box. // SNR, signal noise ratio const max_ri=100; var i,j,rs,distance,counter,ri, distance_top_value, illuminated_pixels: integer; SumVal,SumValX,SumValY,SumvalR,val,xg,yg,bg_average, pixel_counter,r, val_00,val_01,val_10,val_11,af :double; distance_histogram : array [0..max_ri] of integer; HistStart,asymmetry : boolean; begin valmax:=0; bg:=0; snr:=0; valmax:=0; hfd:=-1; star_fwhm:=-1; rs:=s div 2; if (x-s)<1+3 then x:=s+1+3; if (x+s)>(Fwidth-1-3) then x:=Fwidth-s-1-3; if (y-s)<1+3 then y:=s+1+3; if (y+s)>(Fheight-1-3) then y:=Fheight-s-1-3; try // average background counter:=0; bg_average:=0; for i:=-rs-3 to rs+3 do {calculate mean at square boundaries of detection box} for j:=-rs-3 to rs+3 do begin if ( (abs(i)>=rs) and (abs(j)>=rs) ) then begin bg_average:=bg_average+Fimage[0,y+i,x+j]; inc(counter) end; end; bg_average:=bg_average/counter; {mean value background} bg:=bg_average; counter:=0; bg_standard_deviation:=0; for i:=-rs-3 to rs+3 do {calculate standard deviation background at the square boundaries of detection box} for j:=-rs-3 to rs+3 do begin if ( (abs(i)>=rs) and (abs(j)>=rs) ) then begin bg_standard_deviation:=bg_standard_deviation+sqr(bg_average-Fimage[0,y+i,x+j]); inc(counter) end; end; bg_standard_deviation:=sqrt(0.0001+bg_standard_deviation/(counter)); {standard deviation in background} bg:=bg_average; repeat {## reduce box size till symmetry to remove stars} // Get center of gravity whithin star detection box and count signal pixels SumVal:=0; SumValX:=0; SumValY:=0; valmax:=0; for i:=-rs to rs do for j:=-rs to rs do begin val:=Fimage[0,y+j,x+i]-bg; if val>5*bg_standard_deviation then begin if val>valmax then valmax:=val; SumVal:=SumVal+val; SumValX:=SumValX+val*(i); SumValY:=SumValY+val*(j); illuminated_pixels:=illuminated_pixels+1; end; end; if sumval<=10*bg_standard_deviation then exit; {no star found, too noisy} Xg:=SumValX/SumVal; Yg:=SumValY/SumVal; xc:=(x+Xg); yc:=(y+Yg); {center of gravity found} if ((xc-rs<=1) or (xc+rs>=Fwidth-2) or (yc-rs<=1) or (yc+rs>=Fheight-2) ) then begin exit;end;{prevent runtime errors near sides of images} // Check for asymmetry. Are we testing a group of stars or a defocused star? val_00:=0;val_01:=0;val_10:=0;val_11:=0; for i:=0 to max_ri do distance_histogram[i]:=0;{clear histogram} for i:=-rs to 0 do begin for j:=-rs to 0 do begin val_00:=val_00+ value_subpixel(xc+i,yc+j)-bg; {value top left} val_01:=val_01+ value_subpixel(xc+i,yc-j)-bg; {value bottom left} val_10:=val_10+ value_subpixel(xc-i,yc+j)-bg; {value top right} val_11:=val_11+ value_subpixel(xc-i,yc-j)-bg; {value bottom right} end; end; af:=min(0.90,rs/10); {## variable asymmetry factor. 1=is allow only prefect symmetrical, 0.000001=off} {more critital detection if rs is large} asymmetry:=( (val_00<af*val_11) or (val_00>val_11/af) or {diagonal asymmetry} {has shape large asymmetry?} (val_01<af*val_10) or (val_01>val_10/af) or {diagonal asymmetry} ((val_00+val_01)<af*(val_10+val_11)) or ((val_00+val_01)>(val_10+val_11)/af) or {east west asymmetry} ((val_00+val_10)<af*(val_01+val_11)) or ((val_00+val_10)>(val_01+val_11)/af) ); {north south asymmetry} if asymmetry then dec(rs,2); {try a smaller window to exclude nearby stars} if rs<4 then exit; {try to reduce box up to rs=4 equals 8x8 box else exit} until asymmetry=false ;{loop and reduce box size until asymmetry is gone or exit if box is too small} // Get diameter of signal shape above the noise level. Find maximum distance of pixel with signal from the center of gravity. This works for donut shapes. for i:=0 to max_ri do distance_histogram[i]:=0;{clear histogram of pixel distances} for i:=-rs to rs do begin for j:=-rs to rs do begin Val:=value_subpixel(xc+i,yc+j)-bg;{##} if val>((5*bg_standard_deviation)) then {5 * sd should be signal } begin distance:=round((sqrt(1+ i*i + j*j )));{distance from gravity center } if distance<=max_ri then distance_histogram[distance]:=distance_histogram[distance]+1;{build distance histogram} end; end; end; ri:=0; distance_top_value:=0; HistStart:=false; illuminated_pixels:=0; repeat inc(ri); illuminated_pixels:=illuminated_pixels+distance_histogram[ri]; if distance_histogram[ri]>0 then HistStart:=true;{continue until we found a value>0, center of defocused star image can be black having a central obstruction in the telescope} if distance_top_value<distance_histogram[ri] then distance_top_value:=distance_histogram[ri]; {this should be 2*pi*ri if it is nice defocused star disk} until ((ri>=max_ri) or (ri>=rs){##} or (HistStart and (distance_histogram[ri]<=0.1*distance_top_value {##drop-off detection})));{find a distance where there is no pixel illuminated, so the border of the star image of interest} if ri>=rs then {star is larger then box, abort} exit; {hfd:=-1} if illuminated_pixels<0.35*sqr(ri+ri){35% surface} then {not a star disk but stars, abort} exit; {hfd:=-1} if ri<3 then ri:=3; {Minimum 3x3 box} // Get HFD SumVal:=0; SumValR:=0; pixel_counter:=0; if valmax>1 then snr:=valmax/sqrt(valmax+2*bg) else snr:=valmax*MAXWORD/sqrt(valmax*MAXWORD+2*bg*MAXWORD); if snr>3 then begin for i:=-ri to ri do for j:=-ri to ri do begin Val:=value_subpixel(xc+i,yc+j)-bg; if val>5*bg_standard_deviation then {5 * sd should be signal } begin r:=sqrt(i*i+j*j); SumVal:=SumVal+Val; SumValR:=SumValR+Val*r; if val>=valmax*0.5 then pixel_counter:=pixel_counter+1;{how many pixels are above half maximum for FWHM} end; end; Sumval:=Sumval+0.00001;{prevent divide by zero} hfd:=2*SumValR/SumVal; hfd:=max(0.7,hfd); // minimum value for a star size of 1 pixel star_fwhm:=2*sqrt(pixel_counter/pi);{The surface is calculated by counting pixels above half max. The diameter of that surface called FWHM is then 2*sqrt(surface/pi) } end; except on E: Exception do begin hfd:=-1; star_fwhm:=-1; end; end; end;{gethfd2} |
|
|
Hello Patrick, Just mist your last addition. Yes we are on the same track. Some hazy but clear sky is expected here this night, but I'm very confident in the last modification. :) |
|
|
2) and 3) must be fixed by: https://github.com/pchev/ccdciel/commit/7adf9b6b60158a0670b8c7c62e42b2f038555e45 2) was because I selected the star at the end of vcurve learning to show the hfd in the star profile tool. But this selection remain when you start the next learning and it wrongly thing you selected this star for the measurement. 3) will update the position if the quality is > 0.9 |
|
|
Han, Your solution is good, it make a good filter and work with all my test images. I like the idea to make the rejection more early in the function if possible. |
|
|
>>For 3) you meant to update the value in "Focus position" at the top of Vcurve learning? No, My idea is that if the curve is saved, the focuser should move to the best focus position. The user is happy with the curve so best focus position should set. I would not do it on quality Adapting the position in the top of the Vcurve learning menu after a V learning run is probably less helpful and probably counter productive if the curve is bad. I would leave that as it is. |
|
|
OK, this is for the case the quality is initially not >0.9 but good enough for saving and better than the initial position. This change move the focuser when the curve is saved and remove the update of the position on the top: https://github.com/pchev/ccdciel/commit/0b8094ae439eac7057ae5d129d14c7fe24494498 |
|
|
I make one more correction for the autofocus. MeasureStarList() was not using GetHFD2 but the old FindStarPos() + GetHFD(). This make the recent improvement to be taken into account only for the star detection but not for the measurement during autofocus. This is now corrected: https://github.com/pchev/ccdciel/commit/d8ed9f30a86c48398a62805a561e314b0f428af0 |
|
|
I do comparison between your method with illuminated_pixels and mine on many images and with simulated autofocus on M38 using the INDI simulator. Both work and give similar result, but I prefer your method because it is more direct and it is more easy to understand we accept a ratio of 35% of the pixels. I commit this change so it is more easy for you to test if you get some clear sky: https://github.com/pchev/ccdciel/commit/f667a9e7a958bb78a1b7b363f15fe57cb03ebbfd |
|
|
One small correction is required. The 35% illumination filtering doesn't allow small stars of 2, 3 pixels. Rather then ri, ri-1 should be used. Attached a test image with some artificial pixels. - if illuminated_pixels<0.35*sqr(ri+ri){35% surface} then {not a star disk but stars, abort} exit; {hfd:=-1} + if illuminated_pixels<0.35*sqr(ri+ri-2){35% surface} then {not a star disk but stars, abort} exit; {hfd:=-1} Later, I have thought about what to do with one pixels stars. That could happen at the bottom of the curve for people with very good seeing and under sampled setup. Still it is probably better to ignore them since they could be hot pixel or cosmic radiation. |
|
|
The weather is very poor here but I had a short moment of clear sky. Both V-curve and dynamic focusing with multiple stars worked well. Both reached a good HFD. There was not much time for testing. I would like to have tested a factor of 0.40 rather then 0.35 due to the -2 in above formula. |
|
|
I apply the change for (ri+ri-2), this effectively improve the detection of small stars below hfd=2. I try to change 0.35 by 0.4 but this not make any difference on any of my test images. Maybe we must add a configuration setting for the minimum HFD to accept and replace every test we have for HFD<0.8. Because if you image with a KAF9000 on a FS60 you are likely to have stars with HFD<1 and the only solution is to set this limit to 0. But with a IMX183 on a SCT rejecting anything smaller than 2 or 3 make sens. |
|
|
Yes I'm thinking of something similar. If one pixel is illuminated you could just set HFD=0.7. The current setup works fine up to two pixels but HFD=0 is never reached. Line hfd:=max(0.7,hfd); isn't doing anything. The problem with one pixel stars is that the program will react on hot pixels and cosmic radiation. CMOS sensors like my 1600 are suffering more from that then a classic CCD sensor. Maybe a menu option accept one pixel star is the way to go Ignoring one pixel stars will be working well for most users and prevent many false detections. For the undersampled systems an option "undersampled" to activate one pixel star detection. I will play once more with the 0.35 factor. Then this should be finished. |
|
|
While testing, I notice that the due to the subpixel routine the number of illuminated pixels is larger then expected. So a star of 3 pixels has maybe 11 illuminated pixels due to the center gravity. So calculating the number of pixels is depending on the center of gravity which is a little unpredictable for small star images I assume now that your original HFD/FWHM ratio to eliminate star clusters is a better solution. With the illuminated_pixes line removed, the routine worsk fine up to 1 pixel illuminated so the hfd:=max(0.7,hfd); should stay. I still think an undersampled option in the menu is a good idea. That option activated could allow stars below HFD<0.8 Later: The factor of 3 seems too soft. A factor 0.8 works fine for my images: hfd1/(star_fwhm+0.00001)>0.8 |
|
|
Hello Han, The hfd/fwhm ratio can be even more tricky to tune. You say about a factor 0.8. But normally for a well sampled and focused star the hfd must be greater than the fwhm, so the ratio is alway >1. I initially use this factor of 3 to accept some deformation in non-focused star, but with this value the rejection of group is not as good than the illuminated_pixels test. I added the Undersampled option in the Focus tab. This allow to make some test with 1 pixel stars and I finally stay with the illuminated_pixels test, but only if ri>2. if ri<=2 there is no need to check for group of stars anyway. This change is in: https://github.com/pchev/ccdciel/commit/f131a04a7812dcfe4f75ea3587de72c137f5d0ee |
|
|
Routine works good in simulation. All stars are detected from HFD 0.7 and larger. The (Undersampled or (hfd1>0.8) option can make everybody happy and in off position will prevent false detections. I will test it this evening and then for me this topic could be closed. |
|
|
Hello Patrick, Worked fine this night. The dynamic focusing is just better for faint stars. For maximum HFD=3 or twice minimum it performs good. The V-curve doesn't reach well focus for near focus below HFD=10. So requires stars two to three magnitude brighter. With H-alpha small band filter the stars (stay in place) are often just too faint unless you expose 15 seconds. I think this can be described in a table, maybe something for the manual: p.s one spelling error in Log "Hyperbola curve fits error" should be "Hyperbola curve fit error" focusing.png (9,343 bytes) |
|
|
Hello Han, Good this work better! I added your table in the documentation. I also add a message to suggest to use "slew to focus star" with Vcurve. I make a new version with this changes now. |
- Anonymous
