View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0002952 | CCdciel | General | public | 26-09-19 22:06 | 26-10-07 14:21 |
| Reporter | Han | Assigned To | |||
| Priority | normal | Severity | minor | Reproducibility | have not tried |
| Status | new | Resolution | open | ||
| Summary | 0002952: Code review | ||||
| Description | I had some spare tokens for Claude.ai and used them to analyse the CCDCiel code. Attached the result. | ||||
| Tags | No tags attached. | ||||
| Attached Files | CCDciel_bugs.txt (30,892 bytes)
================================================================================
CCDciel - BUG AND BAD-PRACTICE REPORT
================================================================================
Source reviewed : ccdciel-master.zip, src/ only (143 units, ~114,700 lines)
Forms (.lfm) : not included, so no form/component-level checks were possible
Method : static reading + pattern scanning. The code was NOT compiled
(no FPC in the analysis environment), so nothing below is a
compiler diagnostic; every item was verified by reading the
source at the quoted line numbers.
Line numbers : refer to the files as shipped in the archive.
Severity key
[CRIT] memory corruption, data corruption, or crash
[HIGH] wrong results, resource exhaustion, security exposure
[MED] wrong behaviour under specific but realistic conditions
[LOW] robustness, maintainability, latent risk
================================================================================
1. [CRIT] Double free + corrupted list entry in the polar alignment dialog
================================================================================
File : pu_polaralign2.pas
Lines: 181-184, 215-232, 233-236
Tf_polaralign2.FormShow builds the "GotoPosition" combo box. At line 181 it
creates the first TAltAzPosition, sets the four sentinel values to -1
(lines 182-183) and adds it as the "custom position" entry at index 0.
In the "near equator, NORTHERN hemisphere" branch the next statement is:
215 if ObsLatitude>0 then begin
216 {North, preference for south horizon}
217 GetAE(-4,0,1,altaz); <-- altaz is STILL the object from 181
218 GetAE(-0.5,-40,2,altaz);
219 GotoPosition.items.AddObject(rsMeasureES, altaz);
220 altaz:=TAltAzPosition.Create; <-- the Create belongs BEFORE line 217
There is no TAltAzPosition.Create before line 217. Two things follow:
a) GetAE writes through azp.az1/alt1/az2/alt2, so the -1 sentinels of the
"custom position" entry are overwritten with real alt/az values. The
custom-position logic (PanelCustomPosition.Visible, the -1 defaults) no
longer behaves as designed.
b) The SAME pointer is now stored at index 0 and at index 1 of
GotoPosition.Items.
The cleanup loop that runs at the top of every FormShow:
178 for i:=0 to GotoPosition.items.Count-1 do
179 GotoPosition.items.Objects[i].free;
then calls Free twice on one object. TObject.Free only checks Self<>nil and the
reference is never nil'd, so the second call operates on freed memory: heap
corruption or an access violation the second time the dialog is opened.
Proof that this is an editing accident, not intent: the mirrored SOUTHERN branch
has the Create twice in a row -
233 else begin
234 {South, preference for north horizon}
235 altaz:=TAltAzPosition.Create;
236 altaz:=TAltAzPosition.Create; <-- object from 235 leaks immediately
237 GetAE(-4,0,1,altaz);
The Create was moved out of the north branch and pasted twice into the south one.
Affected users: any observer with 0 < latitude <= 30 (north branch, the crash)
and -30 <= latitude < 0 (south branch, the leak).
Fix: delete line 236, and insert "altaz:=TAltAzPosition.Create;" before line 217.
Consider also giving the list ownership of its objects, or nil'ing
Objects[i] after Free, so a repeat of this mistake fails loudly.
================================================================================
2. [CRIT] Wrong buffer flushed when writing 32-bit float FITS
================================================================================
File : cu_fits.pas
Lines: 2603-2623 (TFits.WriteFitsImage)
The BITPIX=-32 branch fills and flushes "dm32" throughout the pixel loop:
2612 if not first then FStream.Write(dm32,sizeof(dm32));
2613 FillByte(dm32,sizeof(dm32),0);
...
2619 dm32[npix]:=single(NtoBE(longword(xs)));
but the final partial-block flush writes the wrong array:
2623 if npix>0 then FStream.Write(d32,sizeof(d32)); <-- should be dm32
Compare the three integer branches, which all flush their own buffer:
line 2566 Write(d8,...) after filling d8
line 2585 Write(d16,...) after filling d16
line 2604 Write(d32,...) after filling d32
Because d32 and dm32 are both 2880 bytes there is no range error and no crash.
The effect is silent data corruption: the LAST 2880-byte data record (up to 720
pixels) of EVERY 32-bit-float FITS file written by CCDciel contains whatever is
in d32 - stale big-endian integer data from a previous save, or zeros. npix is
always in 1..720 at that point, so the bug fires on every float save, regardless
of image dimensions.
Fix: change line 2623 to Write(dm32,sizeof(dm32)).
Related, same branch, line 2619:
dm32[npix]:=single(NtoBE(longword(xs)));
This relies on FPC treating longword(<single expression>) and
single(<longword expression>) as bit reinterpretation rather than value
conversion. That holds for same-size variable typecasts but is fragile on an
expression and is not guaranteed across compiler versions or modes. Prefer an
explicit variant record or an "absolute" alias:
var fv: single; iv: longword absolute fv;
... fv := (Fimage[k,ii,j]-bzero)/bscale;
dm32[npix] := single(NtoBE(iv)); {or store iv into a longword array}
================================================================================
3. [HIGH] InvertF64 corrupts roughly half of all double-precision pixels
================================================================================
File : u_utils.pas
Lines: 220-231 (InvertF64), used by cu_fits.pas:1130 for BITPIX=-64 images
temp:=4294967296 * ((P[0] shl 24) or (P[1] shl 16) or (P[2] shl 8) or (P[3]))
+ ((P[4] shl 24) or (P[5] shl 16) or (P[6] shl 8) or (P[7]));
P[] is PByteArray, so each element is a Byte. In FPC, "Byte shl 24" is promoted
to a SIGNED 32-bit LongInt. Therefore:
- The low group ((P[4] shl 24) or ...) is NEGATIVE whenever P[4] >= $80.
- The high group ((P[0] shl 24) or ...) is NEGATIVE whenever P[0] >= $80.
Intended value : (UInt32(hi) shl 32) or UInt32(lo)
Actual value : Int64(hi) * 2^32 + Int64(lo)
The high group happens to survive: a negative hi is (hi_unsigned - 2^32), and
multiplying by 2^32 wraps modulo 2^64 back to the correct bits.
The LOW group does not. When lo is negative the result is
(hi_unsigned - 1)*2^32 + lo_unsigned - i.e. the high 32 bits of the double are
decremented by one. That flips bit 32 of the 52-bit mantissa, a relative error
of about 2^-20 (~1e-6).
P[4] >= $80 for roughly half of arbitrary pixel data, so about half the pixels
of every BITPIX=-64 FITS image read by CCDciel carry ~1e-6 relative noise. The
image looks correct; photometry and statistics are quietly degraded.
Fix: replace the whole hand-rolled swap with the RTL primitive, which the same
file already uses elsewhere (cu_fits.pas:1257, 1287, 2379):
function InvertF64(X: Int64): Double;
var i: Int64;
begin
i := BEtoN(X);
Move(i, Result, 8);
end;
Same for InvertF32 with BEtoN(LongWord).
--------------------------------------------------------------------------------
3b. [MED] InvertF32/InvertF64 zero out legitimate large finite values
--------------------------------------------------------------------------------
File : u_utils.pas, lines 213 and 225
if (P[0]=$7F)or(P[0]=$FF) then result:=0 // IEEE-754 NaN
P[0] is the most significant big-endian byte. For a single, top byte $7F covers
exponents $FE-$FF: everything from about 1.7e38 upward, of which only
$7F80_0000 and above is Inf/NaN. Values in $7F00_0000..$7F7F_FFFF are perfectly
valid finite floats and are silently replaced by zero. Same for $FF (negative).
For doubles, top byte $7F covers everything from about 9e307 up.
A correct NaN/Inf test is (P[0] and $7F)=$7F and ((P[1] and $80)=$80) for
singles, or simply IsNaN/IsInfinite on the decoded value.
================================================================================
4. [HIGH] Use-after-free on the TCP server objects
================================================================================
Files: cu_tcpserver.pas (lines 126, 270), pu_main.pas (4440, 5521, 8219,
Tf_main.StopServer, Tf_main.MenuStatusClick)
cu_tcpserver.pas:122 constructor TTCPDaemon.Create;
cu_tcpserver.pas:126 FreeOnTerminate := True;
cu_tcpserver.pas:267 constructor TTCPThrd.Create(Hsock: TSocket);
cu_tcpserver.pas:270 FreeOnTerminate := True;
Both thread classes destroy themselves when they finish, but the owner keeps the
pointers and never nils them:
* pu_main keeps TCPDaemon (and TCPDaemon4) as fields and dereferences them:
StopServer: if (TCPDaemon = nil) or TCPDaemon.Finished then exit;
TCPDaemon.TCPThrd[i].sock ...
TCPDaemon.stoping := True;
MenuStatusClick: if (TCPDaemon<>nil)and(not TCPDaemon.Finished)
line 4440: if (TCPDaemon<>nil) then StopServer;
Reading TCPDaemon.Finished IS the use-after-free. The nil guard cannot help
because the variable is never set to nil.
* TTCPDaemon.TCPThrd[1..Maxclient] holds client threads that free themselves.
The array slots are never cleared, and both the accept loop
(cu_tcpserver.pas:214-221) and StopServer read .Fsock / .sock / .terminated
through them.
The accept loop is usually saved by short-circuit evaluation - ThrdActive[i] is
cleared in the worker's finally block before the object dies - but that is luck,
not design, and the ordering is not guaranteed.
Fix: set FreeOnTerminate := False for both classes and free them explicitly from
the owner after WaitFor; or keep FreeOnTerminate and nil the reference from
TThread.OnTerminate (synchronised to the owning thread). Do not mix the two.
Related, same area:
* ThrdActive[] is written from the worker thread (via FTerminate(id), called
from TTCPThrd.Execute's finally) and read from the daemon thread with no
synchronisation.
* RestartServer (pu_main) does StopServer; wait(2); StartServer. The old
daemon may still hold the listening socket, so the new bind can fail while
the old daemon object is orphaned.
================================================================================
5. [HIGH] Integer overflow bypasses the FITS image-size guard
================================================================================
File : cu_fits.pas
Lines: 39 (declaration), 2412 (the check)
39 bitpix,naxis,naxis1,naxis2,naxis3 : integer;
...
2412 if (FFitsInfo.naxis1*FFitsInfo.naxis2)>(maxl*maxl) then
2413 raise exception.Create('Image too big! ...');
naxis1 and naxis2 are 32-bit Integers and the multiplication is done in 32 bits.
A header claiming NAXIS1=100000, NAXIS2=100000 gives 1e10, which wraps to a small
(possibly negative) value, passes the guard, and falls through to
SetLength(Fimage, n_plane, Fheight, Fwidth) with the unvalidated dimensions.
Header values arrive from an untrusted file and are parsed with bare strtoint
(cu_fits.pas:2187-2191), so a corrupt or hostile FITS reaches this path directly.
Fix: if (Int64(naxis1)*Int64(naxis2)) > (Int64(maxl)*maxl) then ...
Also reject negative or zero NAXISn explicitly.
--------------------------------------------------------------------------------
5b. [MED] 32-bit stream offsets limit FITS files to 2 GB
--------------------------------------------------------------------------------
File : cu_fits.pas, TReadFits.Execute
1081 streamstart, startline, endline, xs, ys: integer;
1103 streamstart:=fits.fhdr_end+round(xs*startline*abs(fits.FFitsInfo.bitpix/8));
1121 fits.FStream.Position:=streamstart+streaminc*sizeof(d64);
streamstart is Integer, and streaminc*sizeof(d64) is Integer*Integer. Both
overflow past 2^31. A 20000x20000 float64 image - which the maxl=20000 guard
explicitly permits - is 3.2 GB, so the guard allows files this code cannot
address. The result is a truncated offset and garbage pixels, not an error.
Fix: declare streamstart (and the offset expression) as Int64.
================================================================================
6. [HIGH] Unsafe thread join in TFits.ReadFitsImage
================================================================================
File : cu_fits.pas, lines 2449-2477
2450 timeout:=60;
2451 timelimit := now + timeout / secperday;
2452 repeat
2453 sleep(100);
2454 working := False;
2455 for i := 0 to tc - 1 do
2456 working := working or thread[i].working;
2457 timingout := (now > timelimit);
2458 until (not working) or timingout;
If the 60-second timeout fires while workers are still running, execution falls
straight through to code that assumes they have stopped:
* lines 2460-2468 read thread[i].dmin/dmax/sum/sum2/ni/hist while the workers
are still writing them - a data race producing wrong statistics.
* line 2471 DoneCriticalSection(ReadFitsCS) destroys a critical section that a
worker may be inside (TReadFits.Execute enters it every 360/720/2880 pixels).
That is undefined behaviour.
* lines 2472-2474 Free the still-running threads.
60 s is not generous: a multi-gigabyte float image on a slow or network disk
reaches it routinely.
Fix: join properly. With FreeOnTerminate=False (which is already the case),
call thread[i].WaitFor before touching any result field, and drop the timeout -
or keep a watchdog that aborts the whole operation rather than continuing.
Also in the same routine:
* line 2408 checks "if FFitsInfo.naxis1=0 then exit" but never checks naxis2.
With NAXIS2=0, Fheight=0, no worker loop body runs, ni stays 0, and
line 2479 Fmean:=sum/ni divides by zero.
* line 2480 Fsigma:=sqrt((sum2/ni)-(Fmean*Fmean)) uses the cancellation-prone
form. On a near-uniform frame (bias, flat, saturated) the argument can go
slightly negative through rounding and sqrt raises EInvalidOp or returns NaN.
Use max(0, ...) at minimum, or accumulate with Welford's algorithm.
* InitCriticalSection at 2441 and DoneCriticalSection at 2471 are not
exception-safe; any raise in between leaks the critical section.
================================================================================
7. [HIGH] Memory leaks on error paths
================================================================================
7a. cu_fits.pas:1891 TFits.LoadFromFile
1898 mem:=TMemoryStream.Create;
1901 i:=UnpackFits(fn,mem,rmsg);
1902 if i<>0 then begin
1903 ClearImage;
1904 msg('FITS decompression error ...');
1905 exit; <-- mem never freed
...
1909 mem.LoadFromFile(fn); <-- can raise; mem never freed
Note that on the success path ownership passes to SetStream (line 1913), so
the leak only occurs on the two failure paths - which is exactly why it has
gone unnoticed. Wrap in try/except and free mem before re-raising or exiting.
7b. cu_fits.pas:1860 TSaveFits.Execute
The thread frees mem on the success path (line 1878) but the except handler
(1880-1886) does not. Every failed save - disk full, read-only directory,
permission error, network share dropped - orphans a full image buffer.
Because FreeOnTerminate is true the thread object disappears and the stream
is unreachable. On a long unattended sequence with a failing disk this grows
without bound.
7c. cu_fits.pas:1804 TFits.SaveToFile, synchronous path
mem.SaveToFile(fn) at lines 1840/1848 can raise; mem.Free at line 1851 is not
in a finally. Same leak, main thread.
7d. pu_main.pas:15554 LoadRawFile and :15602 LoadPictureFile
Both create RawStream/PictStream AND FitsStream, but the finally block frees
only the first one. FitsStream is normally adopted by "fits.Stream:=FitsStream"
(TFits.SetStream takes ownership), so the success path is fine - but:
* the "if FitsStream.size<2880" branch skips the assignment entirely;
* any exception before the assignment skips it too.
Add FitsStream to the finally with a flag, or restructure so ownership
transfer is the last statement in the try.
7e. pu_main.pas:15669 CCDCIELMessageHandler
The first line is "if AppClose then exit;". Messages posted with
PtrInt(strnew(PChar(...))) (cu_astrometry_engine.pas:596, 665, 678, 747, 760,
820, 832, 842, 848; cu_fits.pas:1874, 1885, 3446, 3467) are then never
StrDispose'd. Minor in absolute terms, but it is the only place those
allocations can be released.
================================================================================
8. [MED] Shared fixed temp filename races between threads
================================================================================
File : cu_fits.pas, lines 1839 and 1866
tmpf:=slash(TmpDir)+'tmppack.fits';
The identical hard-coded name is used by the synchronous SaveToFile path and by
TSaveFits.Execute running in a background thread. Two concurrent saves - or an
async save overlapping a sync one, which the Async parameter exists to allow -
write to the same file and then compress whichever version won the race. The
output .fz can contain another exposure's data.
Fix: derive a unique name (GetTempFileName, or append the thread id / a counter),
and delete it when done - the current code never removes it either.
================================================================================
9. [MED] Credential storage defects
================================================================================
File : u_utils.pas (EncryptStr/DecryptStr/hextostr), u_global.pas:377
Used : pu_devicesetup.pas:1687-1739+ for all ASCOM Alpaca/REST user+password
9a. hextostr silently returns the ciphertext as plaintext
u_utils.pas:
k := strtointdef('$' + str[2*i+1] + str[2*i+2], -1);
if k > 0 then
Result := Result + char(k)
else begin
Result := str; // if not numeric default to the input string
break;
end;
The guard is "k > 0", not "k >= 0". A legitimate $00 byte in the ciphertext
- which RotateBits can and does produce - aborts decoding and returns the raw
hex STRING as if it were the decoded value. The user then authenticates with
a hex blob instead of their password and gets an unexplained 401. Change to
k >= 0, and distinguish "invalid hex" (should be an error) from "byte 0".
9b. DecryptStr destroys leading/trailing spaces
EncryptStr pads with 15 spaces when encoding; DecryptStr undoes this with a
blanket trim(). Any password that legitimately begins or ends with a space is
silently corrupted. Use a length prefix or an explicit terminator instead.
9c. The key is a compile-time constant
u_global.pas:377 encryptpwd='m=Nrv"wE+W^RA?$b:]w<!t1v]pcTT>3$B?3";~OG9\7...'
Everyone running CCDciel has the same key, so this is obfuscation against
casual inspection of the config file, not encryption. That may be an accepted
trade-off, but the function names EncryptStr/DecryptStr promise more than
they deliver. At minimum rename them (ObfuscateStr/DeobfuscateStr) and
document the threat model; better, use DPAPI / libsecret / Keychain.
================================================================================
10. [MED] TCP/HTTP server hardening
================================================================================
File : cu_tcpserver.pas
10a. Unbounded Content-Length (line 512 ProcessPost / 306-330 Execute)
if Pos('CONTENT-LENGTH:',UpperCase(buf))=1 then begin
delete(buf,1,15);
cl:=StrToIntDef(trim(buf),0);
end;
...
if cl>0 then Fbody:=RecvBufferStr(cl,500);
No upper bound. "Content-Length: 2000000000" forces a 2 GB allocation from
an unauthenticated client. Clamp to a sane maximum and reject the rest.
10b. Unbounded body read when Content-Length is 0
else if cl=0 then begin
repeat Fbody:=Fbody+RecvPacket(500); until LastError<>0;
end;
Reads until the peer errors out, with O(n^2) string concatenation. Both a
memory and a CPU DoS. Use a size cap and a TStringStream / TMemoryStream.
10c. No authentication anywhere
The raw command channel and the HTTP GET/POST channel both dispatch into
TCPcmd / TCPjsoncmd, which drive the mount, camera and sequencer. There is
no token, no password, no origin check. If the server is bound to a routable
address this is unauthenticated remote control of the observatory.
Recommend: default to loopback, and require a shared secret for anything
that is not read-only. (There is an isLocalIP helper in u_utils.pas that is
not used here.)
10d. Fatal socket errors do not stop the daemon (lines 180-202)
After CreateSocket/bind/listen the code only calls Synchronize(@ShowError)
and then falls through into "repeat ... until False". If bind fails - the
common "port already in use" after a failed RestartServer - the thread spins
forever calling Accept on an unbound socket. Exit the Execute method on a
fatal error instead.
10e. Property name collision (line 78-98)
TTCPThrd exposes "onTerminate" of type TIntProc, which hides the inherited
TThread.OnTerminate: TNotifyEvent. A future reader (or a future maintainer
adding standard TThread cleanup) will assume the inherited semantics.
Rename to onSlotFree or similar.
10f. Blanket "except end" at the end of TTCPThrd.Execute (line 374)
Any unexpected failure in a client thread vanishes without a log entry.
================================================================================
11. [MED] 171 silently swallowed exceptions
================================================================================
A scan of the implementation sections found 171 except blocks whose body is
empty or is "on E: Exception do ;". They are concentrated in the device drivers,
e.g.:
cu_ascomcamera.pas:475-484
try FBinX:=V.BinX; except end;
try FBinY:=V.BinY; except end;
cu_ascomrestweather.pas:403-480 (twelve consecutive empty handlers)
cu_ascommount.pas:773, 787, 1217
cu_ascomrestmount.pas:740, 750, 1115
cu_ascomrestcamera.pas:1019, 1069, 1305, 1354
For genuinely optional ASCOM/Alpaca properties this is defensible - the spec
does allow a driver to throw PropertyNotImplemented. But as written:
* the target variable silently keeps its previous value, so a transient comms
failure is indistinguishable from "property not supported" and stale data is
presented as live;
* there is no trace at any log level, so a user reporting "the temperature
stopped updating" leaves nothing to diagnose;
* a real fault (device unplugged, driver crashed) is absorbed along with the
benign case.
Suggested pattern - keep the tolerance, lose the silence:
function TryGetDouble(const name: string; out v: double): boolean;
begin
Result := false;
try
v := ...; Result := true;
except
on E: Exception do begin
if not FNotImplemented[name] then begin
FNotImplemented[name] := true; // log once, not every timer tick
msg(name + ': ' + E.Message, 9);
end;
end;
end;
end;
Then the caller can distinguish "never supported" from "stopped working".
================================================================================
12. [LOW] Angle normalisation loses precision
================================================================================
File : u_utils.pas
function Rmod(x,y:Double):Double;
begin Rmod := x - Int(x/y) * y ; end;
function to360(x:Extended):Extended;
begin result:=rmod(x+3600000000,360); end;
function to180(x:Extended):Extended;
begin result:=rmod(x+3600000000,360);
if result>180 then result:=result-360; end;
The +3,600,000,000 offset exists because Int() truncates toward zero, so Rmod
returns a negative result for negative x. The consequence is that the subtraction
inside Rmod cancels two numbers of magnitude 3.6e9 to produce a result of
magnitude <360: about ten significant digits are lost.
On x86_64 Windows FPC maps Extended to Double (64-bit), so the absolute error is
approximately 3.6e9 * 2.2e-16 ~= 8e-7 degrees ~= 3 milliarcseconds. That is below
most guiding tolerances but is not nothing for polar alignment and plate-solve
residuals, and it is entirely avoidable:
function to360(x: double): double;
begin
Result := FMod(x, 360); // Math unit, sign of x
if Result < 0 then Result := Result + 360;
end;
The magic constant also silently fails for |x| > 3.6e9.
================================================================================
13. [LOW] Unguarded conversions when parsing FITS headers
================================================================================
File : cu_fits.pas, TFits.GetFitsInfo, lines 2187-2205+
if (keyword='BITPIX') then bitpix:=strtoint(buf);
if (keyword='NAXIS') then naxis:=strtoint(buf);
if (keyword='BZERO') then bzero:=strtofloat(buf);
...
Bare strtoint/strtofloat on values taken from a file. A truncated, hand-edited
or non-conforming header raises EConvertError from inside GetFitsInfo. The
caller chain LoadFromFile -> LoadStream -> GetFitsInfo has no handler (SetStream
has one, but LoadStream is called outside it), so the exception surfaces as a
raw error dialog during an unattended sequence.
Use StrToIntDef / StrToFloatDef with sentinel defaults and mark the header
invalid, as the code already does for NAXIS1 at line 593.
(Locale is handled correctly - pu_main.pas:1525 forces
DefaultFormatSettings.DecimalSeparator:='.' and 1540 sets
Application.UpdateFormatSettings:=False - so this is purely about malformed input.)
================================================================================
14. [LOW] SafeFileName does not make a filename safe
================================================================================
File : u_utils.pas
result:=StringReplace(fn,'*','',[rfReplaceAll]);
result:=StringReplace(result,'/','',[rfReplaceAll]);
result:=StringReplace(result,'\','',[rfReplaceAll]);
result:=StringReplace(result,':','',[rfReplaceAll]);
Path traversal is blocked (both separators are removed), but the following still
pass through and cause a failed file creation on Windows:
? " < > | and control characters #0..#31
trailing '.' or ' '
reserved device names: CON PRN AUX NUL COM1..COM9 LPT1..LPT9
an empty result (a name consisting only of stripped characters)
Object names reach this function from user input and from planetarium/SIMBAD
lookups, so odd characters are realistic. Add the missing characters, reject or
rename reserved words, trim trailing dots/spaces, and substitute a default when
the result is empty.
================================================================================
15. [LOW] Assorted smaller items
================================================================================
15a. u_utils.pas IsNumber is not reentrant
var dummy_ext : extended; // unit-level global, line ~198
function IsNumber(n : string): boolean;
begin result:=TextToFloat(PChar(n),Dummy_ext); end;
Two threads calling IsNumber concurrently write the same global. Make it a
local variable; there is no reason for it to be shared.
15b. u_utils.pas quicksort is declared "inline" but is recursive
FPC cannot inline a recursive routine, so the directive is ignored (and may
produce a hint). Worth removing to avoid the misleading claim. The nested
"sort" also has no recursion-depth guard; the median-of-three pivot it uses
makes pathological input unlikely but not impossible.
15c. u_utils.pas SortFilterListInc contains two dead statements
list.Move(j, j-1);
list[j] := tmpname; // no-op
TFilterExp(list.Objects[j]).ExpFact := tmpexp; // no-op
TStringList.Move already moves both the string and its object, so after the
Move both assignments write back exactly the values that are already there.
They are harmless but signal that the author was unsure what Move does; a
future edit based on that misunderstanding would be a real bug. It is also
an O(n^2) bubble sort where TStringList.CustomSort would do.
15d. cu_fits.pas SaveToFile mutates the header as a side effect
Lines 1810-1827 rewrite BITPIX/BSCALE/BZERO and set FStreamValid:=false
when saving a stack as float. A method named SaveToFile permanently
changes the in-memory object. If the user saves and then continues to work
with the same TFits, the header no longer describes the data.
15e. cu_fits.pas Delete-then-Insert idiom depends on undocumented behaviour
i:=FHeader.Indexof('BITPIX');
if i>=0 then FHeader.Delete(i);
FHeader.Insert(i,'BITPIX',FFitsInfo.bitpix,'');
When the keyword is absent, i is -1 and Insert(-1,...) falls into its
"add key at end" branch (cu_fits.pas:937-942), appending the card AFTER the
END record - producing a non-conforming FITS header. Everywhere else the
code correctly writes Insert(FHeader.Indexof('END'), ...). Note also that
Insert already replaces an existing key in place (lines 917-925), so the
preceding Delete is redundant. Same pattern at lines 4176-4183.
15f. cu_fits.pas 8-bit write scaling (line 2562)
x:=max(min(round(((Fimage[k,ii,j]/255)-bzero)/bscale),MAXBYTE),0);
Fimage is scaled 0..65535. Dividing by 255 gives 0..257, so everything above
65025 clips to 255. The correct divisor for 16->8 bit is 257
(65535/257 = 255 exactly).
15g. u_utils.pas GetThreadCount undercounts on large Windows machines
for i := 0 to 31 do ... ProcessAffinityMask and Mask
Only 32 affinity bits and only the current processor group are examined.
On a >32-thread or multi-group machine the count is wrong (and the value
feeds MaxThreadCount, which is capped at 16 anyway, so the impact is small).
15h. u_utils.pas AirMass wraps a pure arithmetic expression in try/except
Since h>0 is already guaranteed by the enclosing if, the denominator
(165 + 47*h**1.1) cannot be zero and the handler is unreachable. Dead code
that suggests an unclear invariant.
================================================================================
END OF BUG REPORT
================================================================================
CCDciel_improvement_suggestions.txt (18,197 bytes)
================================================================================
CCDciel - IMPROVEMENT SUGGESTIONS
================================================================================
Source reviewed : ccdciel-master.zip, src/ only (143 units, ~114,700 lines)
Scope : structural, architectural and maintainability suggestions.
Outright defects are in the companion file
CCDciel_bugs_and_bad_practices.txt.
Nothing here is a correctness problem. These are changes that would make the
existing correctness easier to preserve. They are ordered roughly by
return-on-effort, not by size.
================================================================================
1. Collapse the ASCOM / Alpaca driver duplication
================================================================================
Measured line-level similarity between each COM driver and its REST twin
(whitespace-insensitive, blank lines removed):
device cu_ascom*.pas cu_ascomrest*.pas similarity
cover 331 298 59%
dome 392 355 58%
focuser 523 501 65%
mount 1,223 1,095 55%
rotator 349 308 55%
safety 230 206 56%
switch 317 425 53%
weather 737 694 66%
wheel 351 334 62%
camera 1,751 1,580 60%
----------------------------------------------------------
total 6,204 5,796 ~6,000 duplicated lines
The two families differ only in how a property is fetched:
COM : FBinX := V.BinX;
REST: FBinX := V.Get('binx').AsInt;
Everything around that - status timers, state machines, connect/disconnect
handshakes, unit conversion, event dispatch, the empty exception handlers - is
copied.
Why this matters beyond tidiness: a fix applied to one copy will not reach the
other. The two weather units alone contain twenty-four near-identical empty
exception handlers; a change to how "property not implemented" is reported would
have to be made twenty-four times in two files.
Suggested shape:
T_ascomdevice_base = class(T_basedevice) // all the logic
protected
function GetD(const prop: string): double; virtual; abstract;
function GetI(const prop: string): integer; virtual; abstract;
function GetB(const prop: string): boolean; virtual; abstract;
procedure PutD(const prop: string; v: double); virtual; abstract;
...
end;
T_ascomcom_transport = class(T_ascomdevice_base) // OLE variant calls
T_ascomrest_transport = class(T_ascomdevice_base) // HTTP/JSON calls
Then each device becomes one unit with two thin transport descendants. A staged
migration is realistic: start with the smallest pair (safety, 230/206 lines),
prove the base class, then work up to camera.
Related: cu_indi*.pas is another 8,230 lines with its own copy of much of the
same device-state logic. A common T_basedevice contract across all three
transports (INDI / ASCOM COM / Alpaca) would be the larger prize.
================================================================================
2. Break up pu_main.pas
================================================================================
pu_main.pas 20,275 lines
That is 18% of the codebase in one unit, and it contains the message pump, the
capture loop, the sequence driver, file loading, the image display, the menu
handlers, the TCP command dispatcher, the guider and finder views, and the
astrometry glue.
Practical consequences visible in this review:
* the TCP lifecycle bug (bug report item 4) spans pu_main and cu_tcpserver but
the pu_main half is spread over five call sites hundreds of lines apart;
* LoadRawFile and LoadPictureFile (15554, 15602) are near-identical and could
not easily be shared because both are methods of the 20k-line form;
* Tf_main has enough state that reasoning about re-entrancy (see item 5) is
impractical.
The natural seams already exist in the naming:
* TCP/JSON command dispatch -> its own unit, driven by an interface, testable
without a form
* file open / save / format -> a TImageIO helper
* capture and sequence drive -> already partly in fu_capture / fu_sequence /
cu_targets; move the remaining logic there
* the finder / guider views -> fu_finder and fu_internalguider already exist
None of this requires a redesign - it is moving methods out and replacing direct
field access with a small number of properties or events.
================================================================================
3. Make TFits stream ownership explicit
================================================================================
cu_fits.pas:254 Property Stream : TMemoryStream read GetStream write SetStream;
cu_fits.pas:1701 procedure TFits.SetStream(value:TMemoryStream);
...
if FStream<>nil then FreeAndNil(FStream);
FStream:=value;
Assigning to a property takes ownership of the assigned object and destroys the
previous one. Nothing at the call site says so:
fits.Stream := FitsStream; // pu_main.pas:15580 - looks like a reference
Every leak in section 7 of the bug report traces back to this. Callers correctly
omit FitsStream.Free on the happy path and therefore also omit it on the error
paths, where ownership never transferred.
Suggestion: keep the property read-only and add an explicit method.
procedure TFits.TakeStream(var value: TMemoryStream); // nils the caller's
// reference
With a var parameter the compiler makes the transfer visible and the caller
cannot double-free. Call sites become:
FitsStream := TMemoryStream.Create;
try
...
fits.TakeStream(FitsStream); // FitsStream is nil afterwards
finally
FitsStream.Free; // no-op if it was taken
end;
which is leak-proof on every path.
GetStream (used at cu_fits.pas:1830) has the mirror-image problem: it returns a
stream the caller must free, which SaveToFile does and TSaveFits.Execute forgets
on its error path.
================================================================================
4. Replace hand-rolled endian conversion with the RTL primitives
================================================================================
cu_fits.pas already uses the right tool in three places:
1257 x16:=BEtoN(d16[npix]);
1287 x:=BEtoN(LongInt(d32[npix]));
2379 x16:=BEtoN(d16[j]);
but the float paths route through InvertF32/InvertF64 in u_utils.pas, which are
hand-written byte shuffles (and, per bug report item 3, one of them is wrong).
Standardising on BEtoN / NtoBE plus a Move or a variant record would:
* remove the signed-promotion class of bug entirely;
* remove the ad-hoc NaN handling, which is currently both wrong and
inconsistent between the 32- and 64-bit versions;
* let FPC emit a single BSWAP instruction instead of four shifts and three ORs
per pixel - measurable on a 60-megapixel float image.
A small type-punning helper used consistently everywhere is worth having:
type
TF32 = packed record case boolean of
true: (f: single);
false: (i: longword);
end;
This also resolves the fragile single()/longword() expression casts at
cu_fits.pas:2619.
================================================================================
5. Reduce reliance on Application.ProcessMessages
================================================================================
99 occurrences across the codebase.
Each one allows arbitrary user interaction - including menu clicks that start a
new capture, abort the sequence, or close the form - to run inside a routine that
is mid-way through updating state. The failure mode is not a crash on the
developer's machine; it is a rare, unreproducible "the sequence did something
strange" report from a user who clicked at the wrong moment during a 300-second
exposure.
Two incremental mitigations that do not require rearchitecting:
a) Guard the re-entrant entry points. A simple busy flag on the handlers that
ProcessMessages can reach (Start/Stop capture, sequence control, file open)
costs little and eliminates the worst cases.
b) Where ProcessMessages exists only to keep the UI responsive during a wait,
replace the wait with a timer-driven state machine or move the work to a
thread. u_utils.Wait and cu_waitthread.WaitExecute already provide the
scaffolding for the latter.
Related: 16 direct Sleep() calls. Sleep on the main thread freezes the UI;
Sleep in a polling loop (cu_fits.pas:2453) wastes up to 100 ms per image read.
An event or condition variable is both more responsive and more correct.
================================================================================
6. Speed up and simplify FITS header parsing
================================================================================
cu_fits.pas:2170-2260 TFits.GetFitsInfo
for i:=0 to FHeader.Rows.Count-1 do begin
keyword:=trim(FHeader.Keys[i]);
buf:=trim(FHeader.Values[i]);
if (keyword='SIMPLE') then ...
if (keyword='XTENSION') ...
if (keyword='BITPIX') ...
... approximately 40 more sequential string comparisons ...
end;
Every keyword is compared against all forty literals, and none of the tests is
an "else if", so the comparisons continue after a match. For a header with a few
hundred cards - common for stacked or plate-solved images - that is tens of
thousands of string comparisons per image load.
Options, in increasing order of effort:
* change the chain to "else if" (one-line change, roughly halves the work);
* order the tests by frequency;
* dispatch on a case over a hashed or interned keyword;
* build a keyword -> handler map once at unit initialisation.
The same file also has the Indexof/Delete/Insert triplet repeated for BITPIX,
BSCALE and BZERO in two places (1817-1826 and 4176-4183). A single
FHeader.SetValue(key, value, comment) that handles "replace in place or insert
before END" would remove the duplication and the negative-index edge case noted
in the bug report.
================================================================================
7. Introduce a logging discipline
================================================================================
The codebase has a usable msg(text, level) convention, but it is applied
unevenly:
* 171 exception handlers log nothing at all;
* some paths log at level 0 (user-visible error) for conditions that are
routine, e.g. an ASCOM property a driver does not implement;
* debug output is compiled in via {$ifdef debug_raw} writeln(...) rather than
routed through msg, so it goes to stdout and is unavailable in a normal build
when a user reports a problem.
Suggestions:
* define what each level means, in a comment at the top of the unit that owns
msg;
* log every swallowed exception at a diagnostic level, once per condition
rather than once per status-timer tick (the drivers poll continuously, so an
unconditional log would flood);
* fold the debug_raw writeln calls into msg at level 9, so the information is
obtainable from a released build by raising the log level.
A user who reports "images stopped saving after two hours" currently leaves
nothing behind; with the above, the failed TSaveFits exception would be in the log.
================================================================================
8. Concurrency primitives are used inconsistently
================================================================================
Counted across the codebase:
Synchronize 53
EnterCriticalSection 6
TCriticalSection 8
InitCriticalSection 1
The pattern in cu_fits.pas - InitCriticalSection at the start of ReadFitsImage,
DoneCriticalSection at the end - creates and destroys the lock on every image
read, which is both wasteful and unsafe if an exception occurs in between. The
critical section is a field of TFits and has the same lifetime as the object; it
should be created in the constructor and destroyed in the destructor.
Cross-thread state elsewhere is shared without any protection at all:
* TTCPDaemon.ThrdActive[] is written from worker threads and read from the
daemon loop;
* TReadFits result fields (dmin, dmax, sum, sum2, ni, hist) are read by the
main thread after a timeout that does not guarantee the workers have stopped.
A short document - or even a comment block per unit - stating which fields are
owned by which thread would prevent most of this. Where the data really is
shared, prefer TThread.WaitFor over polling a boolean, and Interlocked* or a
critical section over unprotected reads.
The PostMessage + strnew(PChar(...)) channel used between the astrometry/save
threads and the main form (thirteen allocation sites, three disposal sites) is
another candidate for replacement - a thread-safe queue of strings, drained by
Synchronize or a timer, removes the manual allocation entirely and with it the
leak-on-shutdown described in the bug report.
================================================================================
9. Reconsider the global DefaultFormatSettings override
================================================================================
pu_main.pas:1525 DefaultFormatSettings.DecimalSeparator:='.';
pu_main.pas:1526 DefaultFormatSettings.TimeSeparator:=':';
pu_main.pas:1540 Application.UpdateFormatSettings := False;
This is the right answer for FITS I/O, which mandates '.', and it is why the
locale-dependent strtofloat calls in GetFitsInfo do not misbehave on a Dutch,
German or French system. But it is applied process-wide, so every number and
date shown to the user also ignores their locale.
A cleaner split is to keep the user's locale in DefaultFormatSettings and use a
dedicated, explicitly-constructed TFormatSettings for all file I/O:
var FitsFormat: TFormatSettings; // '.' separator, invariant
...
bzero := StrToFloat(buf, FitsFormat);
unit_ricecomp_fits.pas already does exactly this (lines 205-208, 279-282, 449),
so the pattern exists in the codebase and just needs to be applied consistently.
================================================================================
10. Small consistency items
================================================================================
10a. Naming collisions
TTCPThrd.onTerminate (TIntProc) hides TThread.OnTerminate (TNotifyEvent).
Rename to something that cannot be mistaken for the inherited member.
10b. Threading model per class
TTCPDaemon and TTCPThrd use FreeOnTerminate := True; TReadFits,
TDebayerImage, TMedianFilter, TGetStarList use False; TSaveFits uses True.
Both models are valid, but mixing them in one codebase means every reader
has to check. Pick one as the default and document the exception.
10c. Fixed-size thread arrays
Six places declare "thread: array[0..15] of T..." and clamp with
max(1, min(16, MaxThreadCount)). The bound is consistent and correct today,
but it is expressed as a literal in twelve places. A single constant
(MaxParallelThreads = 16) used in both the declaration and the clamp would
make a future increase a one-line change.
10d. Magic numbers in buffer sizes
d8[1..2880], d16[1..1440], d32[1..720], d64[1..360] are all "one 2880-byte
FITS record". Deriving them from a named constant (FitsRecordSize = 2880)
would make the relationship - and the correctness of the npix mod N tests
that pair with them - self-evident.
10e. TStringList sorting
SortFilterListInc / SortFilterListDec are hand-written bubble sorts.
TStringList.CustomSort with a comparison function is shorter, faster, and
removes the dead-statement confusion noted in the bug report.
10f. Dead code
u_utils.AirMass wraps an expression that cannot raise in a try/except;
the Delete before Insert in the FITS header updates is redundant because
Insert already replaces in place. Removing dead defensive code makes the
remaining defensive code meaningful.
================================================================================
11. Testability
================================================================================
The most valuable single change in this list, if the goal is to stop this class
of bug recurring, is to make the pure computational units testable without a GUI.
The following are already free of LCL dependencies or nearly so:
u_utils.pas (coordinate transforms, JD, sorting, endian helpers)
u_refraction.pas
u_ephem.pas
uDE.pas
unit_ricecomp.pas / unit_ricecomp_fits.pas
cu_fits.pas (mostly - it descends from TComponent, not TForm)
A small console test project over just those would have caught, mechanically:
* InvertF64's sign-promotion error (round-trip a known bit pattern);
* the float FITS write flushing the wrong buffer (write, read back, compare);
* the naxis1*naxis2 overflow (feed a crafted header);
* the to360 precision loss (compare against FMod);
* the hextostr zero-byte abort (round-trip a string containing #0).
FPCUnit ships with Lazarus, so the infrastructure cost is close to zero. Even
twenty assertions over the FITS read/write round trip would be a meaningful
safety net for a program whose entire purpose is to not corrupt astronomical data.
================================================================================
END OF IMPROVEMENT SUGGESTIONS
================================================================================
| ||||
|
|
I have asked Claude to implement the suggestions into the code. See attached. A lot of info is in CHANGES.txt and in all-changes.diff. I have looked to it only very briefly, but previously it proved to be very accurate when I used it on my own code review. |
|
|
Thank you Han. I will look in detail after I return in one week. |
|
|
I apply the suggested change, this look good for me and simulator test show no problem. https://github.com/pchev/ccdciel/commit/634da90871e8c82105f6829340b9c7589f8900e4 |
|
|
The first review was done with Opus 5.0, low effort. Today I have asked Opus 5.5 in high effort mode to do a new review. It has found a long list of points. It still surprises me how thoroughly it goes trough the code. The points found are in the text file ccdciel_bug.txt. It has also created a file ccdciel_improvements.txt with some suggestions for the future. And a text file ccdciel_future_todo.txt for the remaining work for a other thorough review later. Not all source files where fully analysed. I went briefly to the list and asked it to implement it with some exceptions. See the code files in the attached zip file. I have checked only if it compiles and functional testing is outstanding. ccdciel_bugs-2.txt (51,419 bytes)
CCDciel source review - BUGS AND BAD PRACTICES
================================================
Scope and method
----------------
- Static review of the uploaded ccdciel-master/src (143 Pascal units, ~115,000 lines).
The code was NOT compiled or run: no FPC/Lazarus toolchain was available, so every
item below comes from reading the source. Line numbers refer to the uploaded files.
- Reviewed in depth: cu_fits, cu_tcpserver, the TCP/JSON-RPC parts of pu_main,
pu_scriptengine (script launching), u_utils (process, string, time helpers),
cu_ascomrest (Get), unit_ricecomp_fits (unpack and header cards), cu_autoguider,
cu_autoguider_phd, cu_autoguider_internal (star selection, calibration), cu_tiff
(write path), cu_indicamera (image reception, frame), cu_astrometry (engine
lifetime), pu_dark, pu_polaralign2, cu_waitthread.
- Second round (sections E and F): the mount, focuser and dome drivers for all
three interfaces (cu_mount, cu_ascommount, cu_ascomrestmount, cu_indimount,
cu_focuser, cu_ascomfocuser, cu_ascomrestfocuser, cu_indifocuser, cu_dome,
cu_ascomdome, cu_ascomrestdome, cu_indidome), the sequence engine (cu_targets,
cu_plan, cu_sequencefile) and the meridian flip logic in pu_main.
- Added later: C16-C18, found when comparing cu_tiff.pas and unit_ricecomp.pas
with the same units of ASTAP (the Rice core itself is still not reviewed in
depth, only its thread dispatchers and the encoder clamp).
- Covered by automated pattern scans only (empty except blocks, loop bounds,
self-assignments, Format() argument counts, unreleased locals, async calls):
the camera, wheel, rotator, weather, safety, switch and cover drivers,
unit_ricecomp (encoder), the ws_* websocket units, fu_sequence/pu_edittargets
(GUI) and the rest of pu_main.
- The code already contains many "2024 fix (report item N)" comments from an earlier
review. Those fixes were re-checked; section A lists the ones that are wrong or
incomplete.
Status overview (each item below ends with a "Status:" line)
------------------------------------------------------------
implemented (52): A1, A2, A4, A6, A7, B1, B2, B3, B4, B7, B8,
B9, B10, B11, B12, B13, B14, B15, B16, C1, C2,
C3, C5, C6, C7, C8, C9, C13, C14, C15, C16,
C17, C18, D3, D4, D5, D6, E1, E2, E4, E7, E8,
E9, E10, E11, F1, F2, F3, F5, F7, F8, F10
implemented differently ( 6): C4, C11, E3, E5, E6, F4
partially implemented ( 9): A3, A5, B6, C10, C12, D8, E12, E13, F6
not implemented ( 8): B5, D1, D2, D7, D9, D10, D11, F9
Severity: HIGH = data corruption, crash, security or wrong result in normal use;
MEDIUM = edge cases, races, leaks, robustness; LOW = minor or code hygiene.
=====================================================================
A. PROBLEMS IN EARLIER "2024 FIX" CHANGES
=====================================================================
A1. [HIGH] 8-bit FITS values change on every save (regression from "item 15f")
cu_fits.pas:2706 (TFits.WriteFitsImage, 8-bit branch)
The writer now divides by 257, but the reader still stores 8-bit pixels as
x8*255 (cu_fits.pas:1217 and 1254). The fix comment assumes Fimage spans 0..65535,
but for data loaded from an 8-bit FITS it spans 0..65025.
Effect: open + save of an 8-bit file maps 255->253, 128->127, etc. Every
re-save darkens the image a little more.
Fix: use the same constant in both directions (either revert the writer to 255,
or change the reader to *257 and keep 257 in the writer). Add a round-trip test.
Status: IMPLEMENTED.
A2. [MEDIUM] Overflow fix applied in one place only ("item 5")
cu_fits.pas:2456 (TFits.GetFlatLevel)
if (FFitsInfo.naxis1*FFitsInfo.naxis2)>(maxl*maxl) is still 32-bit, exactly the
pattern that was widened to Int64 in ReadFitsImage. A corrupt header can wrap the
product and bypass the size guard.
Also: endline := startline+cropsizey (line 2468) makes the loop read
cropsizey+1 rows while flatlimit assumes cropsizey rows (off by one).
Status: IMPLEMENTED.
A3. [MEDIUM] Temporary file race fixed only for tmppack ("item 8")
UniqueTempFits is used for the pack/unpack temp file, but other fixed shared
names remain: ccdcieltmp.fits, ccdcielsolved.fits, fitstmp.fits, findertmp.fits,
finder.fits, findersolved.fits, guidetmp.fits, guidesolved.fits,
guidereftmp.fits (cu_astrometry.pas 415-947, cu_autoguider_internal.pas:2756),
and tmppack.fits.fz in cu_indicamera.pas:1331.
Examples: Tf_main.LoadPictureFile writes ccdcieltmp.fits while an asynchronous
plate solve of the main image may still be reading the same file; a guide and a
main INDI camera both receiving .fits.fz at the same time share tmppack.fits.fz.
The .fits.fz temp file is also not deleted when UnpackFits fails.
Status: PARTIALLY IMPLEMENTED. Separate names for picture import, ASTAP
menu and INDI .fz; solved-file menu actions refuse while a solve
runs; the astrometry module keeps its own fixed names (serialised
by its Busy flag).
A4. [LOW] Misleading safety claim ("item 1")
pu_polaralign2.pas:154 and :186
The comment says nil-ing each slot after Free turns a duplicated pointer into a
harmless Free(nil). It does not: if the same object is in slots i and j, slot i
is freed and nil'd but slot j still holds the dangling pointer, and freeing it is
still a double free. The actual fix (the missing Create) is correct; only the
comment is wrong and could mislead a later maintainer.
Status: IMPLEMENTED. Correction note added after the original comment.
A5. [LOW] Wrong reasoning in StopServer comment ("item 4")
pu_main.pas:16666-16676
The comment says a plain WaitFor would deadlock because the daemon uses
Synchronize. In current FPC versions TThread.WaitFor called from the main thread
services CheckSynchronize while waiting, so it does not deadlock. More important:
after the 10-second Wait() loop the code calls TCPDaemon.WaitFor unconditionally,
so the "timeout" is not a timeout - if a client never stops, shutdown hangs.
The Wait() loop also runs Application.ProcessMessages, so the user can trigger
StartServer/StopServer again while TCPDaemon is half torn down.
Status: PARTIALLY IMPLEMENTED. Comment corrected, re-entrancy guard and
progress message added; the final WaitFor is kept (freeing a
running thread is unsafe).
A6. [MEDIUM] The worker-thread "join" fixes still ignore thread failures ("item 6")
cu_fits.pas:2579, 2596, 2869, 2924, 3481 (and the other WaitFor sites)
WaitFor is now correct, but an exception inside TReadFits/TDebayerImage/
TMedianFilter/TGetBgraThread.Execute is stored in TThread.FatalException and never
inspected. The caller then uses partially filled image arrays and statistics as
if nothing happened. Check FatalException after WaitFor and raise/report it.
Status: IMPLEMENTED. ReadFitsImage refuses the image; the other workers
report an error message.
A7. [HIGH] Program never closes: TCP server thread blocked in Accept
(found during testing of the corrections)
cu_tcpserver.pas TTCPDaemon.Execute
The accept loop called the blocking Accept without waiting for a pending
connection first, so the stop request was only checked when a client
connected. Since "item 4" made Tf_main.StopServer wait for the thread
(WaitFor), closing the program hung forever when no client connected
(always on Windows, where the socket timeout does not apply to accept).
The review missed this (A5 only noted a hang with a connected client).
Fix: CanRead(500) before Accept, the stop request is checked every 500 ms.
Status: IMPLEMENTED.
=====================================================================
B. HIGH SEVERITY
=====================================================================
B1. JSON flattening counts nodes instead of depth
cu_tcpserver.pas:553 (TTCPThrd.JsonDataToStringlist)
cu_autoguider_phd.pas:178 (T_autoguider_phd.JsonDataToStringlist)
JsonRecurseLevel is incremented on every call and never decremented, so it counts
all visited values. Any message with about 100 values raises
"JSON data recursion > 100" and is dropped, even though it is only 2 levels deep.
Fix: dec(JsonRecurseLevel) on exit (try/finally), or pass depth as a parameter.
Also: in both units GetJSON's result J leaks when the flattening raises
(cu_tcpserver.pas:531, cu_autoguider_phd.pas:~215) - J.Free is not in a finally.
Status: IMPLEMENTED.
B2. HTTP POST with empty body sends two responses
cu_tcpserver.pas:646 (TTCPThrd.ProcessPost)
After sending "HTTP/1.0 405 Invalid request" there is no exit, so the code
continues, runs ProcessJSON and sends a second "HTTP/1.0 200" response.
Status: IMPLEMENTED.
B3. POST body reading logic inverted, 413 does not stop processing
cu_tcpserver.pas:430 (TTCPThrd.Execute)
- The "read until the peer stops" branch runs when cl=0, i.e. when the client
sent "Content-Length: 0" (or an unparsable value, because StrToIntDef(...,0)).
When the header is absent (cl=-1) nothing is read at all.
Intended: cl=0 -> empty body; cl<0 -> read until close (bounded).
- Inside that branch the 413 "Request body too large" uses break, which only
leaves the inner repeat loop. The truncated body is then still passed to
ProcessPost, which sends a second response and executes the request.
- The header lines are accumulated in "hdr" without limit and hdr is never used.
Status: IMPLEMENTED.
B4. HTTP protocol check can never fail
cu_tcpserver.pas:599 else if pos('HTTP/',protocol)<0 then
Pos never returns a negative value. Should be =0 (or <>1).
Status: IMPLEMENTED.
B5. (moved to D11 - remote access without authentication is intended)
Status: NOT IMPLEMENTED. Moved to D11, remote access is intended.
B6. Invalid JSON produced by the JSON-RPC server
pu_main.pas:17897 and cu_tcpserver.pas ProcessJSON (error branch)
- The request id is inserted unquoted. A string id (allowed by JSON-RPC 2.0, e.g.
"id":"abc") produces "id": abc - invalid JSON.
- Exception messages and values from the configuration (driver names, paths with
backslashes, object names with quotes) are concatenated without escaping.
Fix: build replies with fpjson (TJSONObject) instead of string concatenation.
Status: PARTIALLY IMPLEMENTED. Ids, error messages and string results are
escaped; replies are still built by string concatenation.
B7. Use-after-free of asynchronous Python script threads
pu_scriptengine.pas:2422 (RunPythonAsync)
PythonScr[n].FreeOnTerminate:=true, but the pointer stays in PythonScr[n].
ScriptRunning(), StopPython() and the slot search in RunPython/RunPythonAsync
later dereference the freed thread (.Terminated, .FRunning, .PyProcess).
This is the same pattern that "item 4" fixed for the TCP threads.
Fix: keep FreeOnTerminate=false and free finished threads when the slot is
reused, or clear the slot (in ShowOutput, on the main thread) before the thread
frees itself.
Status: IMPLEMENTED.
B8. Successful synchronous script can be reported as failed (race)
pu_scriptengine.pas:2585 (TPythonThread.Execute)
FRunning:=false is set BEFORE Synchronize(@ShowOutput). RunPython polls
ScriptRunning() and can see "not running" and read PythonResult[n] before
ShowOutput has stored the exit code, i.e. it reads the initial -1.
Fix: set FRunning:=false after the Synchronize call.
Status: IMPLEMENTED.
B9. Switch lock left on forever after a failed script start
pu_scriptengine.pas:390 (RunScriptAsync)
LockSwitch is called, but when the script file does not exist, or when
RunPythonAsync fails (all 10 slots in use), the function exits without
UnLockSwitch. Rate-limited switch status updates then stop permanently.
The lock is also not reference-counted: with two scripts running, the first one
to finish unlocks the switches while the second is still running.
Status: IMPLEMENTED.
B10. Forked child keeps running a copy of CCDciel if exec fails (Unix)
u_utils.pas:1309 (ExecFork)
If fpExecVP returns -1 the child process just returns 0 from ExecFork and continues
executing the GUI program as a second instance (duplicate timers, sockets, device
connections). The child must call fpExit(127) (or _exit) after a failed exec.
The parent never calls fpWaitPid (no waitpid anywhere in the code), so every
launched program leaves a zombie process until CCDciel exits.
Status: IMPLEMENTED.
B11. Stack buffer overflow reading stderr
u_utils.pas:1161 and :1196 (ExecProcessMem)
cerr is array[0..1024] of char, but P.Stderr.Read(cerr, i) uses
i := NumBytesAvailable, which can be much larger than 1025. A noisy external
program (astrometry solver, script) overwrites the stack.
The buffer is also never re-terminated: after a short read, bytes left over from
an earlier longer read are appended to "err".
Fix: read into a dynamic buffer of size i, or read min(i, SizeOf(cerr)-1) and
terminate.
Status: IMPLEMENTED.
B12. Buffer overflows and handle leaks in Windows process launchers
u_utils.pas:1134 (ExecNoWait, Windows): StrPCopy into a fixed 1025-char buffer
with no length check; process and thread handles from CreateProcess are never
closed (handle leak per call).
u_utils.pas:1320 (ExecuteFile, Windows): three array[0..255] buffers filled by
StrPCopy - a file path longer than 255 characters overflows the stack.
u_utils.pas:1129 (ExecNoWait, Unix): fpSystem(cmd+' &') runs the string through
/bin/sh; any path or object name containing ; & $ ` or quotes is executed as
shell syntax.
Status: IMPLEMENTED.
B13. Alpaca ImageBytes detection is always true
cu_ascomrest.pas:423
Result.imagebytes:=pos('application/imagebytes',buf)>=0;
Pos returns 0 when not found, so this is true for every Content-Type header.
A JSON image reply (from a server without ImageBytes support) is then parsed as
binary, fails the MetadataVersion check and the image download aborts with
"Unsupported Alpaca ImageBytes MetadataVersion". Should be >0.
Same function:
- errmsg (line 388) is an uninitialised stack array; Read(errmsg,1024) does not
add a terminator, so the error text contains stack garbage.
- the "Not implemented Alpaca image array type" raise does not free Result (leak).
- dimension1*dimension2*dimension3 is a 32-bit product of device-supplied
values and is not validated before SetLength.
Status: IMPLEMENTED.
B14. Internal guider: frame coordinates multiplied by the binning inside the loop
cu_autoguider_internal.pas:794
In the loop that keeps only stars inside the new sub-frame, the INDI-specific
conversion frx:=frx*BinX; fry:=fry*BinX; frw:=frw*BinY; frh:=frh*BinY runs once
per accepted star. With INDI and binning>1, after the first accepted star the
frame bounds used for the following stars and the offsets x1-frx are wrong, and
SetFrame finally receives values scaled by bin^N.
Also X and Y binning are swapped (fry uses BinX, frw uses BinY).
Fix: do the conversion once, after the loop, just before FCamera.SetFrame.
Status: IMPLEMENTED.
B15. Thread-unsafe GUI and string access from the PHD2 client thread
cu_autoguider_phd.pas:128 FonConnect(self) is called directly from the worker
thread. The handler Tf_main.AutoguiderConnect (pu_main.pas:8790) changes button
and menu captions - LCL controls must only be touched from the main thread
(crashes on GTK/Qt).
cu_autoguider_phd.pas:147 StarLostTimer.Enabled is changed from the worker
thread (TTimer is not thread-safe).
cu_autoguider.pas:185 / pu_main.pas:15702 FErrorDesc (an AnsiString) is
appended in the worker thread and read and cleared in the main thread with no
lock. Concurrent reference-count updates on a managed string can corrupt the
heap.
cu_autoguider_phd.pas:152 tcpclient.Free in the thread's finally, but the field
is not set to nil. Send() (called from the main thread) checks tcpclient<>nil
and then uses the freed object.
Status: IMPLEMENTED.
B16. Dark library dialog: double free and stale callbacks
pu_dark.pas:121 (FormClose), :161 (btnStartClick), btnCancelClick
- FormClose frees fstack and fdark but does not set them to nil. The form is
reused (f_dark.ShowModal is called repeatedly from pu_main), so after a
cancelled run the next close frees the same objects again.
- Cancel during a run only aborts the exposure; the pending WaitExecute(1000,
@NextExposure) still fires. If the user closes the dialog within that second,
NextExposure's except branch frees the already-freed fstack/fdark.
- btnStartClick creates new fdark/fstack without freeing earlier ones (leak).
- nloop:=trunc(LogN(sq2,maxexp))+1 (line 146) raises if MaxExposure is 0.
Status: IMPLEMENTED.
=====================================================================
C. MEDIUM SEVERITY
=====================================================================
C1. FITS header reader: uninitialised variables and 32-bit size arithmetic
cu_fits.pas TFitsHeader.ReadHeader / GetExtend (around 560-760)
naxis, naxis1..3 and bitpix are not initialised; if the header lacks a keyword,
stack garbage decides whether extensions are searched. Data-unit sizes are
computed in 32-bit Integer (overflow for large or 3-plane images) and rounded
DOWN with trunc(size/2880)*2880 instead of up. It only works because GetExtend
then scans block by block; a data block that happens to start with "XTENSION="
would be misread as a header.
Status: IMPLEMENTED.
C2. NaN pixels poison image statistics
cu_fits.pas TReadFits.Execute (-32 and -64 branches)
Float FITS files commonly use NaN for blank pixels. NaN is not filtered, so
dmin/dmax/sum/sum2 become NaN and the display stretch and statistics break.
Status: IMPLEMENTED.
C3. Threads that mutate shared object state
cu_fits.pas TReadFits.Execute, 8-bit RGBA branch: each worker writes
fits.n_plane:=4 and later 3. This is only safe because ReadFitsImage forces
1 thread when n_plane>1; a later change to the thread count would create a race.
Pass the plane count to the worker instead of modifying the shared object.
Status: IMPLEMENTED.
C4. Tile-compressed FITS reader (NativeUnpackFits) loses data
unit_ricecomp_fits.pas
- :785 every header keyword starting with "Z" is dropped, not only the
compression keywords. User keywords such as ZD, ZENITH, ZPOINT are lost.
- :770 integer images are always written as BITPIX=16, despite the comment
"keep original ZBITPIX". ZBITPIX=8 or 32 data is truncated.
- :611 ZNAXIS1=0 (or absent) makes ztile1:=0 and tiles_x:=.../ztile1 divides by
zero. ZNAXISn are never range-checked before SetLength(img,...).
- tableBytes := bnaxis1*bnaxis2 is a 32-bit product of header values.
- :776 k := HdrIndex(hx,'') inside the loop is dead code and makes it O(n^2).
- CHECKSUM/DATASUM of the compressed HDU are copied into the uncompressed header,
where they are now wrong.
Status: IMPLEMENTED DIFFERENTLY. ZBITPIX other than 16 and -32 is refused
with a message instead of supporting 8/32-bit output (the decoder
supports only 16 and -32).
C5. FITS header card writers produce invalid cards
unit_ricecomp_fits.pas:283 MakeFloatCard: round(value) raises for
|value|>9.2e18 (and for NaN); exponent notation gets ".0" appended ("1E+30.0").
unit_ricecomp_fits.pas:298 MakeStrCard: single quotes in the value are not
doubled; values longer than 68 characters lose the closing quote.
cu_fits.pas:915 TFitsHeader.Insert: copy(row,1,80) truncates long quoted
strings without the closing quote (invalid card).
cu_fits.pas:798 TFitsHeader.GetJSON: values are not JSON-escaped.
Status: IMPLEMENTED.
C6. Header-derived filename parts not sanitised
cu_fits.pas:5302 (CapturePath) and :5359 (CaptureFilename)
OBJECT goes through SafeFileName, but FILTER and FRAME do not. Filter names come
from the wheel driver or user configuration; a filter called "Ha/OIII" or
"R:G:B" creates an extra sub-directory or makes the save fail on Windows.
DateIso2DateTime (u_utils.pas:1502) raises EConvertError for DATE-OBS values it
does not understand (e.g. a trailing "Z"), which would abort filename creation.
Status: IMPLEMENTED. Also the sequence step name used as directory name.
C7. AstroTIFF export
cu_fits.pas:3589 (TFits.SaveAstroTiff)
Always calls save_tiff_16, which writes only img[0]. An RGB (3-plane) image is
saved as its red plane only. For scaled images (float, 32-bit) it passes
Frawimage, whose values are in physical units; clamp16 then flattens most of
the image to 0/1. save_tiff_new already selects the right writer and is unused.
The "delayed" parameter is unused.
cu_tiff.pas:293: height2=0 makes thread_count 0 and the next line divides by zero.
Strip-compression thread exceptions are not checked; a nil strip stream would
then be written. read_tiff is not called anywhere in CCDciel (dead code).
Status: IMPLEMENTED. Read_tiff (dead code) kept on request.
C8. INDI image reception
cu_indicamera.pas:1311-1324 (NewImageFile)
- When the uncompressed size is unknown, a 200 MB buffer (10000*10000*2) is
allocated for every compressed frame.
- The frame-size estimate assumes 2 bytes per pixel; 32-bit images do not fit and
uncompress fails.
- If uncompress fails (i<>0) nothing is reported; ".z" is still stripped from
the type and the compressed bytes are processed as a FITS file.
cu_indicamera.pas:1696 x:=round(x+0.5) "force even values" only works because
FPC's Round uses banker's rounding. Use x := x and not 1 for clarity.
Status: IMPLEMENTED.
C9. TCP daemon socket freed but not cleared
cu_tcpserver.pas:331 / :205
Sock.Free in Execute's finally does not set Sock:=nil, but GetIPport tests
"if sock=nil". Reading TCPDaemon.IPport after the daemon thread has ended
dereferences freed memory (and GetIPport reads the socket from the main thread
while the daemon thread owns it). StartServer swallows every exception
(empty except).
Status: IMPLEMENTED.
C10. Queued asynchronous calls can run on freed objects
24 calls to Application.QueueAsyncCall (pu_main 11, pu_devicesetup 3,
cu_autoguider_internal 2, pu_polaralign 2, cu_autoguider_phd, cu_camera,
cu_targets, fu_finder, pu_polaralign2, pu_scriptengine) and no call to
Application.RemoveAsyncCalls anywhere.
Concrete case: T_autoguider_phd queues ProcessEventAsync with a TStringStream
(cu_autoguider_phd.pas:231). The autoguider thread is FreeOnTerminate and is
replaced on disconnect, so events still in the queue run on the freed object and
their stream is never freed.
Fix: call Application.RemoveAsyncCalls(Self) in every destructor that queued calls.
Status: PARTIALLY IMPLEMENTED. RemoveAsyncCalls added everywhere; data
passed to a removed call (PHD2 event stream) is not freed.
C11. Self-freeing threads referenced after they may have ended
cu_astrometry.pas:292 engine.Resolver is read after engine.Resolve started a
FreeOnTerminate thread; if the solver fails immediately the object is gone.
cu_astrometry.pas:332 StopAstrometry calls engine.Stop when FBusy is true, but
FBusy is only cleared when the posted M_AstrometryDone message is processed;
in between the engine thread may already have freed itself.
cu_autoguider (FreeOnTerminate=true): when AppClose is set, the disconnect
handler does not replace the global "autoguider" reference, which then
dangles.
cu_waitthread.pas:76 WaitExecute calls the target method after sleeping with no
check that the target object still exists (e.g. form closed meanwhile).
Status: IMPLEMENTED DIFFERENTLY. Engine reference cleared through
OnTerminate; WaitExecute skips calls at program close; the
autoguider reference at close is replaced by an inactive
T_autoguider_none object instead of nil.
C12. Internal guider calibration state machine
cu_autoguider_internal.pas:2326 When measure_drift fails in direction 6 / step 1,
the code switches to direction 7 and recursively calls InternalCalibration, but
then falls through and sets InternalguiderCalibrationStep:=1 or 2 afterwards,
overwriting the state just set by the recursive call. An exit is missing.
cu_autoguider_internal.pas:2344 drift can be 0 (mount did not move) ->
floating-point division by zero. Same for pulsegainEast+pulsegainWest
(CalPixSize1) and pulsegainEast/pulsegainWest, pulsegainNorth/pulsegainSouth.
cu_autoguider_internal.pas:2004 "if Calthecos=0" never triggers because
cos(90 deg) is about 6e-17, not 0. Use abs(Calthecos)<1e-6.
cu_autoguider_internal.pas:2318, 2333 sleep(500) on the main thread freezes the
GUI during the scale measurement (up to 8 times).
Status: PARTIALLY IMPLEMENTED. State machine and divisions fixed; the two
sleep(500) are kept on purpose (Wait() would allow re-entry).
C13. Sequence plan loading leaks configuration objects
cu_targets.pas:1129 (CompatLoadPlan)
pfile:=TCCDconfig.Create(self) is never freed. Because it is owned by the
targets object it is released eventually, but every load of an old-format
sequence keeps another parsed XML file in memory until then.
Status: IMPLEMENTED.
C14. Busy-wait loops
u_utils.pas ExecProcessMem / ExecProcess: "while P.Running" without any sleep.
Off the main thread (ProcessMessages skipped) this spins a core at 100%.
cu_ascomrest.pas:410 and GetImagearrayBase64: sleep(5)+ProcessMessages polling
for every Alpaca request.
pu_scriptengine RunPython: wait(1) polling.
Status: IMPLEMENTED. Process-wait loops; the Alpaca polling loops already
had a sleep.
C15. Websocket frame length (vendored library)
ws_websocket2.pas:1409 The 64-bit payload length is built from signed
multiplications; a first byte >=128 makes "len" negative, and there is no upper
bound on the frame size before allocating. The peer is the INDI server, so this
is low risk, but the library should reject oversized or negative lengths.
Status: IMPLEMENTED.
C16. [MEDIUM] Float images written as 16-bit TIFF (found when comparing with ASTAP)
cu_tiff.pas save_tiff_new
"if bitpix<=16" is also true for the negative FITS BITPIX of float images
(-32, -64), so a caller that passes the FITS BITPIX gets a 16-bit TIFF.
(CCDciel's own SaveAstroTiff passes 32 since C7 and was not affected.)
Fix: abs(bitpix)<=16.
Status: IMPLEMENTED.
C17. [LOW] NaN pixels raise an exception when saving TIFF or .fz
(found when comparing with ASTAP)
cu_tiff.pas fill_tiff_row/clamp16, unit_ricecomp.pas TRiceEncodeThread
round() raises EInvalidOp for NaN (the clamps compare false for NaN), and
in the Rice encoder also for values beyond the Int64 range. Since C2 the
FITS reader replaces NaN by 0, so this is a safety measure.
Fix: NaN written as 0; the Rice encoder clamps to 0..65535 before round().
Status: IMPLEMENTED.
C18. [MEDIUM] Failed Rice worker threads not detected (found when comparing
with ASTAP; unit_ricecomp.pas had not been reviewed before)
unit_ricecomp.pas rice_decode_tiles, rice_encode_rows
Same pattern as A6: TRiceTileThread and TRiceEncodeThread use try/finally
without except and the dispatchers did not check FatalException. Decoding
left the tiles of a failed worker blank without an error; encoding returned
ok=true with nil tile_data[] that the caller then wrote.
Fix: after WaitFor, a FatalException sets err_decode (decoder) or ok:=false
(encoder) with its message.
Status: IMPLEMENTED.
=====================================================================
D. LOW SEVERITY / CODE HYGIENE
=====================================================================
D1. 182 empty "except ... end" blocks silently swallow errors. The largest groups:
cu_ascomweather.pas and cu_ascomrestweather.pas (29 each), pu_main.pas (16),
cu_ascomcamera / cu_ascomrestcamera (9 each), u_utils.pas (8), cu_autoguider_phd (7).
At minimum log the exception message.
Status: NOT IMPLEMENTED. On request.
D2. Application.ProcessMessages is called in about 100 places, including inside
Wait() (u_utils.pas:1343), every Alpaca request and process launch. Each call
allows re-entrant event handling (a button can be clicked while its own handler
is still running). Several bugs above (A5, C11) depend on this.
Status: NOT IMPLEMENTED. On request.
D3. Self-assignments with no effect (likely leftover or unfinished code):
cu_autoguider_internal.pas:790 xy_array[star_counter2].flux:=...flux
fu_starprofile.pas:870 s:=s
pu_pause.pas:127 PauseLabel.Caption:=PauseLabel.Caption
pu_sensoranalysis.pas:611 minstep:=minstep
u_refraction.pas:74, 127 h:=h
Status: IMPLEMENTED.
D4. u_utils.pas:1408/1414 to360 can return exactly 360.0 for tiny negative inputs
(e.g. -1e-14 + 360 rounds to 360).
u_utils.pas:401 Splitarg removes one character of a multi-character separator.
u_utils.pas:3248 hextostr reads past the end of an odd-length string (range
check error in debug builds).
u_utils.pas:1353 GetCdCPort: Reset() with I/O checks on and no try/except.
Status: IMPLEMENTED.
D5. unit_ricecomp_fits.pas MakeLogicalCard computes "s" twice (first assignment is
dead); TFormWidthBytes has an unused isP variable.
Status: IMPLEMENTED.
D6. cu_fits.pas:1301 "ii:=ys-1-i;;" double semicolon; "thread[0]:=nil;" leftovers in
Debayer/MedianFilter.
Status: IMPLEMENTED.
D7. 48 "with" statements. They hide which object a field belongs to and have
already caused confusion in this code (e.g. "with sock do" blocks in
cu_tcpserver mixing thread and socket members).
Status: NOT IMPLEMENTED. On request.
D8. Credentials: EncryptStr is obfuscation with a constant key (already noted by the
earlier review). In addition, TAscomRest.SetBaseUrl embeds user:password in the
URL, which can end up in logs or error messages that print the URL.
Status: PARTIALLY IMPLEMENTED. Credentials sent as HTTP Basic
authentication instead of in the URL; the obfuscation of stored
credentials is unchanged.
D9. pu_main.pas is 20,329 lines. Many unrelated responsibilities (UI, TCP server,
JSON-RPC dispatcher, sequencing glue, image display) live in one form unit.
Status: NOT IMPLEMENTED. On request.
D10. Globally mutating DefaultFormatSettings.DecimalSeparator (pu_main.pas:1525) works
but affects every unit and any library; prefer a dedicated TFormatSettings for
FITS/protocol formatting.
Status: NOT IMPLEMENTED. On request.
D11. Unauthorized access is possible
pu_main.pas StartServer, TCPjsoncmd
The TCP/IP server listens on all network interfaces and has no
authentication, so anyone who can reach the port can slew the mount, open
the dome, run scripts or shut the program down. Remote access is intended
(remote clients, the HTML status page), so this is left as designed;
restrict the port with a firewall to the hosts that need it.
Implemented: SCRIPT_RUN accepts only a plain script name (no path
separators, ':' or '..'), so a client can no longer run .script files from
outside the configuration directory, and double quotes are removed from
script parameters.
Status: NOT IMPLEMENTED. Intended, left as designed; only the SCRIPT_RUN
script name check is implemented.
=====================================================================
E. MOUNT, FOCUSER AND DOME DRIVERS
=====================================================================
Common pattern: the drivers report success without checking the outcome
-----------------------------------------------------------------------
Almost every "do something" method in these units is a procedure, or a function
that returns true as soon as the command was sent. Timeouts and failure states are
not propagated to the caller, so the sequence logic (section F) cannot react.
The individual items below are the concrete cases.
E1. [HIGH] Slew always reports success
cu_ascommount.pas:537, cu_ascomrestmount.pas (Slew), cu_indimount.pas (Slew)
- ASCOM/Alpaca: WaitMountSlewing(SlewDelay) returns false on timeout, but Slew
ignores it and returns true.
- INDI: when WaitBusy times out the code only prints rsTimeout, and an
IPS_ALERT state on EQUATORIAL_EOD_COORD (slew refused, e.g. below the horizon
limit) is not checked at all. Slew still returns true.
- cu_targets.pas:3157 (T_Targets.Slew, non-precision branch) does not even look
at the return value: "Mount.Slew(ra, de); Wait(delay); result:=true;".
Effect: in an unattended sequence a refused or timed-out slew is followed by a
full plan of exposures at the wrong position.
Status: IMPLEMENTED.
E2. [HIGH] Slewing flag stuck after an exception
cu_ascommount.pas Slew, cu_ascomrestmount.pas Slew
FMountSlewing:=true is set inside the try, but the except branch does not reset
it. After one driver exception, GetMountSlewing returns true until the program
is restarted. CheckMeridianFlip skips its checks while the mount is "slewing"
(pu_main.pas:15848), so the meridian flip is never performed.
Fix: reset FMountSlewing in a finally block.
Status: IMPLEMENTED.
E3. [HIGH] Dome shutter "closed" test treats error and opening states as closed
cu_ascomdome.pas:319, cu_ascomrestdome.pas:298 (GetShutter)
GetShutter returns true only for ShutterStatus=0 (open). Status 2 (opening),
3 (closing) and 4 (error), and any exception while reading it, all return
false, i.e. "closed". Consequences in T_mount.SetParkInterface
(cu_mount.pas, dclCloseDome):
- a shutter in the ERROR state, or still opening, is reported as "already
closed" and the close command is never sent;
- a close that fails (ends in status 4) passes the check "if dome.Shutter then
abort", so the park sequence reports success.
For an unattended end of night this is a weather-safety issue.
Fix: expose the full shutter state (open/closed/opening/closing/error/unknown)
and test for "closed" explicitly.
Status: IMPLEMENTED DIFFERENTLY. New ShutterState property used for the
safety-relevant checks; the boolean Shutter property is unchanged
elsewhere.
E4. [MEDIUM] INDI dome without DOME_SHUTTER (roll-off roofs): shutter state is wrong
cu_indidome.pas:419 (GetShutter) / :429 (SetShutter)
SetShutter falls back to SetPark(not value) when there is no shutter property,
which is how INDI roll-off roof drivers work. GetShutter does not use the same
fallback and always returns true ("manual shutter, expect open").
- Open action (dopOpenDome): "if dome.Shutter then already open" is always true,
so the roof is never opened by this action.
- Close action (dclCloseDome): after the roof has closed, "if dome.Shutter"
is still true, so the park sequence prints "dome not closed" and aborts the
remaining steps.
- InitTarget's check "Dome is not ready" (cu_targets.pas, not dome.Shutter)
passes with the roof closed, so the sequence starts imaging the inside of
the roof.
Fix: when DomeShutterProp=nil, GetShutter should return not GetPark.
Status: IMPLEMENTED.
E5. [MEDIUM] ASCOM/Alpaca dome cannot be unparked, which aborts the unpark sequence
cu_ascomdome.pas:303, cu_ascomrestdome.pas:283 (SetPark)
SetPark(false) does nothing (ASCOM has no Unpark). With the "unpark dome"
action configured, T_mount.SetParkInterface sets dome.Park:=false, then checks
"if dome.Park then abort" - AtPark is still true, so the whole unpark
sequence aborts every time. ASCOM clears AtPark on FindHome or SlewToAzimuth;
use one of those, or skip the check for ASCOM domes.
Status: IMPLEMENTED DIFFERENTLY. CanUnpark property; the "unpark dome"
action is skipped for ASCOM/Alpaca domes instead of sending
FindHome.
E6. [MEDIUM] Mount is not parked when the slaved dome is disconnected
cu_mount.pas T_mount.SetParkInterface
With SlaveDome enabled and the dome not connected, the method prints
"dome not connected / abort mount park" and returns without parking the mount.
At the end of an unattended night (RunEndAction, AtEndPark) a lost dome
connection therefore leaves the telescope unparked and tracking.
Parking the mount alone is normally the safe fallback. Because SetParkInterface
is a procedure, the caller does not learn that nothing happened.
Status: IMPLEMENTED DIFFERENTLY. Mount is parked alone; SetParkInterface
stays a procedure, its result is in the new LastParkOK property.
E7. [MEDIUM] INDI park/position checks read the requested state, not the result
cu_indimount.pas SetPark/GetPark, cu_indidome.pas SetPark/SetShutter,
cu_indifocuser.pas SetPosition
The code changes the local switch/number values, sends them, and later reads
the same local copy (swpark.s, DomeShutterOpen.s, FocusAbsolutePosition value).
The property state (IPS_ALERT) is never checked, and WaitBusy's result is
ignored. If the driver rejects the command and does not echo the old values
back (driver-dependent), CCDciel believes the mount is parked, the shutter is
closed, or the focuser is at the new position.
Status: IMPLEMENTED.
E8. [MEDIUM] Hard-coded 60 s wait for dome, park and focuser moves
cu_ascomdome / cu_ascomrestdome WaitShutter/WaitDomePark(60000),
cu_indidome WaitBusy(...,60000), cu_*focuser SetPosition WaitFocuserMoving(60000)
Many roll-off roofs and some focusers over their full range take longer than
60 s. The timeout result is ignored: the focuser reports the requested position
(FonPositionChange(p)) while it is still moving, so autofocus takes its next
image during the move. Make the timeouts configurable and check the result.
Status: IMPLEMENTED. Timeouts configurable in the configuration file only
(/Dome/ActionTimeout, /Focuser/MoveTimeout), no Options dialog
setting.
E9. [MEDIUM] Fail-open safety checks
cu_dome.pas SetParkInterface/SetShutterInterface, cu_mount.pas unpark branch
Opening the dome or unparking is blocked only if the safety monitor is
connected AND reports unsafe. If the safety monitor is disconnected, opening is
allowed. For an unattended setup "no safety information" should block opening.
Status: IMPLEMENTED.
E10. [MEDIUM] Focuser backlash compensation silently skipped near the limits
cu_focuser.pas:127 (SetPositionInt), cu_ascomfocuser.pas:335 (SetPosition)
The overshoot p+FBacklash (or p-FBacklash) is outside MaxStep/0 near the ends of
the range; SetPosition rejects it with only a level-1 message and the final
move then happens without compensation. Relative moves clamp p+FBacklash to the
range, so the compensation amount is also wrong there. Clamp the overshoot to
the range instead of rejecting it.
Status: IMPLEMENTED.
E11. [LOW] Meridian flip success test with unknown pier side
cu_ascommount.pas:690 / cu_indimount.pas FlipMeridian
result:=(pierside2<>pierside1). Mounts that report pierUnknown return false
even when the flip worked. The fallback flip (three slews) ignores the result of
each Slew.
Status: IMPLEMENTED.
E12. [LOW] SlewAsync on a mount without async support blocks the GUI
cu_ascommount.pas:502 Falls back to the synchronous SlewToCoordinates COM call,
which blocks the main thread for the duration of the slew, and does not set
FMountSlewing.
Status: PARTIALLY IMPLEMENTED. Slewing flag set during a synchronous slew;
the GUI is still blocked during that call (unavoidable).
E13. [LOW] Unfinished feature
cu_mount.pas SlewToDomeFlatPosition: DomeFlatPositionHome is a "#todo" and does
nothing, without telling the user.
Status: PARTIALLY IMPLEMENTED. Still not implemented, a message is shown
instead of doing nothing.
=====================================================================
F. SEQUENCE LOGIC
=====================================================================
F1. [HIGH] Wrong pivot time for targets without coordinates
cu_targets.pas:2360 (SetTargetTime), used at :2509 and :1729
SetTargetTime returns the pivot in HOURS when the target has coordinates
(pivot:=rmod(ht+12+24,24)) but as a FRACTION OF A DAY otherwise
(pivot:=frac(middletime)). Both callers divide the pivot by 24.
For a target without coordinates (calibration frames, a script target with a
time window) the pivot becomes about 00:52 instead of the sequence start time.
Example: window 22:00-02:00, sequence started at 21:00. At 01:00
InTimeInterval treats 01:00 as "before 22:00 today", returns -1 and the target
waits until the next evening (or is skipped if "skip" is set).
Fix: pivot:=frac(middletime)*24.
Status: IMPLEMENTED.
F2. [HIGH] Meridian flip retries ignore the elapsed time
pu_main.pas ~15985 (CheckMeridianFlip, retry loop)
When the mount is still on the wrong side after FlipMeridian, the loop retries
up to 5 times, each with a 300-second pause. The safety test
"if MeridianDelay2<=0 then mount.AbortMotion" uses the value computed before the
loop and is never recomputed. If it was positive on entry, the mount keeps
tracking for up to ~25 minutes beyond the configured limit without being
stopped - a pier collision risk on mounts with a tight limit.
Fix: recompute the hour angle (hhmin, MeridianDelay2) in each iteration.
Status: IMPLEMENTED.
F3. [MEDIUM] Mandatory start time not detected after midnight
cu_targets.pas TargetTimerTimer:
InTimeInterval(frac(now), t.starttime, nextt.starttime, t.starttime)=1
The pivot is the CURRENT target's start time. When that target has no start
time or is circumpolar (starttime -1 / -2), the pivot is negative, times after
midnight are not shifted by one day, and a mandatory start of the next target
at e.g. 01:00 is never detected; the current target keeps running.
Status: IMPLEMENTED.
F4. [MEDIUM] Failed guiding start silently skips the light step
cu_plan.pas StartStep (capture branch)
If guiding cannot be started, the step only logs rsFailedToStar and exits. The
plan timer then sees "not running" and moves to the next step, so the light
frames of that step are skipped. When the plan ends, TargetTimerTimer increments
t.repeatdone, so the target is counted as done even if no frame was taken.
Retry, pause, or abort the target instead.
Status: IMPLEMENTED DIFFERENTLY. The target is stopped and the sequence
goes to the next target without counting it; no retry.
F5. [MEDIUM] Script-running flag left set after a failed initialization script
cu_targets.pas ~2640-2655 (InitTarget)
FScriptRunning:=true is set before seq_scriptengine.RunScript; when the script
returns false the code exits without resetting it. GetScriptRunning
(cu_targets.pas ~3440) then reports a running script until another script
resets the flag.
Status: IMPLEMENTED.
F6. [MEDIUM] End-of-night actions are not verified and not ordered for safety
cu_targets.pas RunEndAction
- Mount.Park, Dome.Park and Dome.Shutter are set without checking whether they
succeeded; there is no retry and no e-mail/alert on failure, even though
e-mail notification exists for other events.
- With AtEndCloseDome enabled but AtEndPark disabled, the dome (or roll-off
roof) is closed with the telescope in any position. Closing a roll-off roof
over an unparked telescope can hit the tube. Check mount.Park (or park first)
before closing, or at least warn when the options are combined this way.
Status: PARTIALLY IMPLEMENTED. Park and close verified, retried once,
reported and e-mailed; closing over an unparked telescope only
gives a warning.
F7. [MEDIUM] Slew retry and precision-slew results
cu_targets.pas T_Targets.Slew
- Non-precision slews: see E1 (result always true), so the retry dialog
(rsTelescopeSle2) can only ever appear for precision slews.
pu_main.pas CheckMeridianFlip: astrometry.PrecisionSlew after the flip is not
checked when LastResult is false; imaging resumes uncentered without a message.
Status: IMPLEMENTED.
F8. [LOW] Moon avoidance checked only once per target
cu_targets.pas InitTarget: the moon distance is tested when the target starts.
During a long target the moon can move several degrees closer. The check also
compares the moon's apparent coordinates of date with the target's J2000
coordinates (about 0.35 deg difference in 2026), which is negligible but worth
noting if the limit is small.
Status: IMPLEMENTED. Checked every 5 minutes, and the target position is
converted to apparent coordinates.
F9. [LOW] Blocking waits in the sequence engine
InitTarget's weather loop ("while not FWeather.Clear do Wait(5)"), NextStep's
wait(2), the dome action waits in SetParkInterface and the 300-second pauses in
the meridian flip all run on the main thread through Wait()/ProcessMessages.
Timers and user actions are processed re-entrantly during these waits
(see D2), which makes the sequence state machine hard to reason about.
Status: NOT IMPLEMENTED. Design change needed (state machine with timers).
F10. [LOW] Leaked plan configuration (already listed as C13)
cu_targets.pas:1129 CompatLoadPlan creates a TCCDconfig that is never freed.
Status: IMPLEMENTED. Done as C13.
=====================================================================
IMPLEMENTATION STATUS
=====================================================================
Every change in the source is marked with a comment "2026 fix (review item Xn)"
or "2026 (review item Xn)"; the original comments are kept. The code has not
been compiled here; the A-D changes were compiled by the user, the E-F changes
still need a compile and a test.
NOT IMPLEMENTED
---------------
D1 Empty except blocks - on request
D2 Application.ProcessMessages re-entrancy - on request (design change)
D7 "with" statements - on request
D9 Size of pu_main.pas - on request
D10 Global DecimalSeparator - on request
D11 Remote access without authentication - intended, left as designed
(only the SCRIPT_RUN script name check is implemented)
F9 Blocking waits in the sequence engine - design change (needs a state
machine with timers instead of Wait()); not done
cu_tiff.pas read_tiff (dead code) - kept on request
PARTIALLY IMPLEMENTED OR IMPLEMENTED DIFFERENTLY
------------------------------------------------
Sections A-D
- A7: found while testing (program did not close), implemented.
- A3: separate file names for picture import (ccdcielpicture.fits), the ASTAP
menu (ccdcielastap.fits) and INDI .fz transfers (unique name); the menu
actions that write ccdcielsolved.fits refuse while a solve is running. The
astrometry module keeps its fixed names (its solves are serialised by Busy).
- A5: the 10 second wait is kept but no longer presented as a timeout; a
message is shown when the server is still stopping, and a guard prevents
re-entrant StartServer/StopServer.
- B6: string ids, error messages and string results are escaped; replies are
still built by string concatenation (no full fpjson rewrite).
- C4: ZBITPIX other than 16 and -32 is refused with a message (the decoder only
supports those two) instead of being written as 16-bit.
- C10: RemoveAsyncCalls added everywhere; data passed to a removed call (the
PHD2 event TStringStream) is not freed.
- C11: the autoguider reference at program close is replaced by an inactive
T_autoguider_none object (nil rejected, see Tf_main.AutoguiderDisconnect).
- C12: divisions guarded and the state machine fixed; the two sleep(500) calls
are kept on purpose (Wait() would allow re-entry of the calibration).
- C14: process-wait loops sleep now; the Alpaca polling loops already did.
- D8: credentials sent as HTTP Basic authentication instead of in the URL;
the obfuscation of stored credentials is unchanged.
Sections E-F
- E3: a TShutterState (open, closed, opening, closing, error, unknown) and a
ShutterState property were added to T_dome and implemented for ASCOM, Alpaca
and INDI. It is used where safety matters: the dome actions of the mount
park/unpark sequence, the "dome ready" check in InitTarget and the end of
sequence verification. The boolean Shutter property is unchanged for the
rest of the program (display, scripts).
- E5: instead of an unpark command (ASCOM/Alpaca have none), a CanUnpark
property was added; the "unpark dome" action is skipped for such a dome with
a message (it leaves the park position when it moves).
- E6: SetParkInterface stays a procedure (it is the setter of the Park
property); its result is available in the new property LastParkOK.
- E8: timeouts are configurable only in the configuration file, there is no
setting in the Options dialog yet:
/Dome/ActionTimeout seconds, default 60 (minimum 10)
/Focuser/MoveTimeout seconds, default 60 (minimum 10)
- E9: the safety monitor is "configured" when /Devices/Safety is true; opening,
unparking the dome and unparking the mount are then refused while it is not
connected (unless DomeNoSafetyCheck is set).
- E12: the slewing flag is now set during a synchronous slew; the GUI is still
blocked during that COM/HTTP call, which cannot be avoided for a mount
without asynchronous slew.
- E13: "Home" is still not implemented; a message is shown instead of doing
nothing silently.
- F4: a plan stopped because guiding could not be started goes to the next
target without counting the target as done; it is not retried.
- F6: park and close are verified, retried once and reported (also by e-mail
when end or abort e-mails are enabled). Closing the dome over a telescope
that is not parked only gives a warning, because CCDciel cannot tell a
roll-off roof from a rotating dome.
- F8: the moon distance is checked every 5 minutes while the plan runs; when
the moon is too close the target is stopped and the next one starts.
- F10: was implemented as C13.
ccdciel_improvements.txt (10,423 bytes)
CCDciel source review - IMPROVEMENT SUGGESTIONS
===============================================
These are suggestions, not bugs. They are ordered roughly by benefit for effort.
References like "B7" point to items in ccdciel_bugs_and_bad_practices.txt.
1. Tests that would have caught the current problems
-----------------------------------------------------
- FITS round-trip test: write a synthetic image for each BITPIX (8, 16, 32, -32, -64)
and for 1 and 3 planes, save, reload, compare pixel-exact. This catches A1
immediately and protects WriteFitsImage/ReadFitsImage from future "fixes".
- Rice .fz round trip: NativePackFits -> NativeUnpackFits, compare pixels and the
full keyword list (catches C4).
- Header card tests: long strings, quotes, very large/small floats, NaN (C5).
- JSON-RPC tests against the TCP server: 150-element params, string ids, empty
POST body, missing Content-Length, oversized body (B1-B4, B6).
- These can run headless with FPCUnit in a small console project that uses
cu_fits / unit_ricecomp_fits / u_utils without the LCL forms.
2. Compiler settings
--------------------
- Treat these warnings as errors (or at least review them): 5036/5037 (variable not
initialised - see C1), 5033 (function result not set), 4055/4082 (conversions),
5057 (local not initialised). Use -vw -Sew in a CI build.
- Keep a CI build with -Cr -Co -Ci (range, overflow, I/O checks) and heaptrc (-gh)
and run the tests above with it. heaptrc reports the leaks listed in the bug file.
- Build once with -Criot on a nightly sequence simulation (ASCOM/INDI simulators).
3. One helper for worker threads
--------------------------------
The same create / start / wait / free / sum-results code appears in ReadFitsImage,
Debayer, MedianFilter, GetBGRA, GetExpBitmap, GetStarList and cu_tiff. One generic
helper removes the duplication and fixes A6 everywhere:
procedure RunWorkers(const W: array of TThread);
var i: integer; err: string;
begin
err := '';
for i := 0 to High(W) do W[i].Start;
for i := 0 to High(W) do begin
W[i].WaitFor;
if (W[i].FatalException <> nil) and (err = '') then
err := Exception(W[i].FatalException).Message;
end;
if err <> '' then raise Exception.Create('Worker failed: ' + err);
end;
Callers free the threads in a finally block. Consider TThread.ExecuteInThread or a
small parallel-for (e.g. MTProcs from Lazarus) instead of hand-written thread
classes for the per-row image loops.
4. Thread / GUI boundary
------------------------
- Rule: device and network threads never call event handlers directly. Every
On... callback from a thread goes through Synchronize or TThread.Queue (B15).
- Data shared between a thread and the main thread (FErrorDesc, ThrdActive[],
FRunning flags) is either protected by a TCriticalSection or passed by value
in the queued call.
- Replace FreeOnTerminate threads that are referenced from elsewhere (autoguider,
astrometry engine, async Python scripts) with owned threads that are joined and
freed by their owner, as was already done for the TCP server (B7, C11).
- Every class that calls Application.QueueAsyncCall calls
Application.RemoveAsyncCalls(Self) in its destructor (C10).
5. Fewer ProcessMessages loops
------------------------------
Wait(), the Alpaca request polling and the process launchers all spin with
Application.ProcessMessages. Better options, in order of effort:
- Use TEvent.WaitFor with a timeout in worker threads instead of sleep loops.
- For the Alpaca client, run the request in the THTTPthread and deliver the result
through TThread.Queue to a continuation, instead of blocking with ProcessMessages.
- Where a modal wait is unavoidable, add a re-entrancy guard (a "busy" flag that
disables the relevant actions) so the user cannot start the same operation twice
while it is still running.
6. JSON-RPC / TCP server
------------------------
- Build all responses with fpjson (TJSONObject.Add / AsJSON) - this removes every
escaping problem at once (B6, C5 GetJSON).
- Validate the request id type and echo it back unchanged (number, string or null).
- Bind to 127.0.0.1 by default; offer "allow network access" as an option, with a
shared token checked on each request (B5). Log remote connections.
- Whitelist script names for SCRIPT_RUN: only [A-Za-z0-9_-], and verify that the
resolved path is inside ScriptsDir/ConfigDir.
- Do the HTTP request parsing in the client thread and only marshal the
command execution to the main thread. Currently ProcessHttp/ProcessPost run inside
Synchronize, so socket writes (including JPEG images) block the GUI.
- Consider fphttpserver/fpjsonrpc from FPC instead of the hand-written HTTP parser.
7. External processes
---------------------
- Use TProcess (or RunCommandInDir) everywhere and remove fpSystem, raw
CreateProcess and fpFork/ExecVP. TProcess takes arguments as a list, so no shell
quoting problems (B12), handles pipes safely (B11) and reaps child processes (B10).
- One shared helper "RunAndCapture(exe, args, timeout, out stdout, out stderr)"
can replace ExecProcess, ExecProcessMem and the copy inside TPythonThread.
8. Temporary files
------------------
- Extend UniqueTempFits into a general UniqueTempFile(prefix, ext) and use it for
every temporary FITS/solve/log file (A3), deleting files in finally blocks.
- Better still for plate solving: give each solve request its own temp directory.
9. FITS handling
----------------
- Centralise card formatting in one unit (key, value, comment, type) used by both
TFitsHeader and unit_ricecomp_fits, with correct quoting, CONTINUE for long
strings and exponent formatting (C5).
- Handle NaN/Inf in statistics and display (C2) with IsNan checks or by mapping NaN
to BLANK.
- Validate all header dimensions in one place (NAXISn > 0, product within limits,
computed in Int64) before any allocation, and call that from every reader
(ReadFitsImage, GetFlatLevel, header extension scan, .fz reader, Alpaca image).
- Compute data unit sizes as ceil(bytes/2880)*2880 in Int64 (C1).
- Sanitise every header value used in file or directory names, not only OBJECT (C6).
10. Calibration and guiding math
--------------------------------
- Guard all divisions by measured quantities (drift, gain sums, durations) with a
minimum threshold and a clear error message instead of an FPU exception (C12).
- Replace blocking sleep() calls on the main thread in calibration with a timer
step in the existing state machine.
11. Error handling policy
-------------------------
- Replace empty except blocks with at least a log line
(e.g. except on E: Exception do LogDebug('ascom weather: ' + E.Message); end).
Hidden failures in unattended sequences are hard to diagnose (D1).
- Where an exception is expected (driver does not implement a property), catch the
specific exception class rather than everything.
12. Structure of pu_main
------------------------
pu_main.pas (20k lines) could be split without changing behaviour:
- the JSON-RPC dispatcher (TCPjsoncmd and jsoncmd_*) into its own unit with a
method table (name -> handler) instead of a long if/else chain;
- the TCP server start/stop into a small controller class;
- image display/annotation into its own unit.
This makes the code easier to test and lowers the chance of unrelated regressions.
13. Credentials
---------------
Store Alpaca credentials with the platform secret store (Windows DPAPI via
CryptProtectData, libsecret on Linux, Keychain on macOS), keep the current format as
a read-only fallback for migration, and send them as an HTTP Authorization header
instead of embedding them in the URL (D8).
14. Small clean-ups
-------------------
- Remove dead code: read_tiff (not called), the unused hdr string in the TCP thread,
HdrIndex(hx,'') in NativeUnpackFits, self-assignments (D3), the "delayed"
parameter of SaveAstroTiff.
- Make the even-coordinate rounding explicit (x := x and not 1) instead of relying on
banker's rounding (C8).
- Replace "with" blocks in thread code with explicit references (D7).
15. Device layer: report outcomes (mount, focuser, dome)
--------------------------------------------------------
- Turn the "action" methods into functions that return a result record or enum,
for example TDeviceResult = (drOK, drTimeout, drRefused, drError) plus a message.
Slew, Park, SetShutter, SetPosition and FlipMeridian would all return it, and
T_mount.SetParkInterface / T_dome.SetParkInterface would pass it on.
- Implement each wait once in the base class (WaitUntil(condition, timeout)) and
let the drivers only provide the condition. That removes the five nearly
identical WaitMountSlewing/WaitDomePark/WaitShutter/WaitFocuserMoving loops per
interface and gives one place to make timeouts configurable.
- For INDI, check the property state (IPS_OK / IPS_ALERT) after WaitBusy instead of
the locally modified switch or number values (E7).
- Model the shutter as a state (open, closed, opening, closing, error, unknown)
instead of a boolean, in all three dome drivers (E3, E4).
- Make the safety checks fail-closed: when the safety monitor is configured but
not connected, refuse to open or unpark unless the user explicitly allowed it (E9).
16. Sequence engine
-------------------
- Store all time-of-day values in one unit (fractions of a day, or hours, but not
both). A small TTimeOfDay type or helper functions with the unit in the name
(PivotHours, StartDayFraction) would have prevented F1 and F3. Add tests for
windows that cross midnight, with and without target coordinates.
- Let the end-of-night routine (RunEndAction) verify every step, retry once, and
send an e-mail when a park or close fails. Enforce "park mount before closing a
roll-off roof" in code, not only in documentation (F6).
- Make the meridian flip loop time-aware: recompute the hour angle every iteration
and stop tracking as soon as the hard limit is reached (F2).
- Treat "guiding failed to start" and "slew failed" as step/target failures that
are retried or reported, and do not increment repeatdone for a target that took
no frames (F4).
- Consider moving the sequence state machine off Wait()/ProcessMessages to an
explicit state + timer design; most of the sequence code already uses timers,
so the remaining blocking waits are the exception (F9).
ccdciel_future_todo.txt (7,643 bytes)
CCDciel / ASTAP - THINGS TO DO IN THE FUTURE
=============================================
Status at the end of the first review round (items A to F of
ccdciel_bugs_and_bad_practices.txt implemented, compiled and partly tested).
1. TEST THE CORRECTIONS
-----------------------
- Run the tests in ccdciel_behaviour_changes_to_test.txt. The A-D changes are
compiled and partly tested (T1.1 close program OK on Windows 10); the E-F
changes are compiled but not yet run.
2. CCDCIEL UNITS NOT YET REVIEWED IN DEPTH
------------------------------------------
Of the 143 units about 30 were reviewed in depth, several only partly. The
others were only covered by automatic pattern scans (empty except blocks, loop
bounds, self-assignments, Format() argument counts, unreleased objects,
asynchronous calls), which find typical slips but not logic errors.
Suggested order (most impact first):
1. Cameras
cu_camera, cu_ascomcamera, cu_ascomrestcamera, rest of cu_indicamera
(exposure start/abort, timeouts, image download, cooling).
2. Weather and safety (unattended operation depends on them)
cu_weather, cu_safety, cu_ascomweather, cu_ascomrestweather, cu_indiweather,
cu_ascomsafety, cu_ascomrestsafety, cu_indisafety, fu_weather, fu_safety.
3. Data integrity of compressed images
unit_ricecomp (Rice compressor/decompressor core) and NativePackFits in
unit_ricecomp_fits.
4. Guiding loop
rest of cu_autoguider_internal (guide corrections, dithering, spectroscopy
functions), cu_autoguider_dither.
5. Remaining device drivers
filter wheels (cu_wheel, cu_ascomwheel, cu_ascomrestwheel, cu_indiwheel,
cu_incamerawheel, cu_manualwheel), rotators, switches, covers/calibrators.
6. Remaining program logic
rest of pu_main (capture, autofocus control, device connection),
fu_starprofile (autofocus), fu_capture, fu_sequence, pu_edittargets.
7. Lower risk
planetarium links (cu_planetarium*), Alpaca discovery
(cu_alpacamanagement), websocket units (third-party), u_libraw, astronomy
maths (u_ephem, uDE, u_refraction), remaining GUI forms and dialogs.
Parts of reviewed units that were NOT covered:
cu_fits: star detection, HFD, photometry, stacking (Math), debayer
algorithms.
pu_main: everything except the TCP/JSON-RPC server, the meridian flip,
picture loading and the menu actions changed for A3.
cu_tiff: read_tiff (dead code in CCDciel, but used in ASTAP).
3. ITEMS NOT IMPLEMENTED (see the status lines in the bug report)
-----------------------------------------------------------------
D1 Empty except blocks (182) - log at least the message
D2 Application.ProcessMessages re-entrancy - design change
D7 "with" statements - readability
D9 Size of pu_main.pas - split into units
D10 Global DecimalSeparator - own TFormatSettings
D11 Remote access without authentication - intended; optional token
F9 Blocking waits in the sequence engine - state machine with timers
and the partial items: A3, A5, B6, C10, C12, D8, E12, E13, F6.
4. IMPROVEMENTS
---------------
See ccdciel_improvements.txt (unit tests for FITS/.fz round trips, compiler
warnings as errors, one worker-thread helper, device result type, Options
dialog settings for /Dome/ActionTimeout and /Focuser/MoveTimeout, etc.).
5. ASTAP / ASTAP_CLI UNITS (unit_tiff, unit_ricecomp and the _unthreaded copies)
--------------------------------------------------------------------------------
Checked against the CCDciel findings. The threaded ASTAP units are the same
code as CCDciel's cu_tiff.pas and unit_ricecomp.pas before the corrections
(only the unit name, the CPU count function and one comment differ).
Corrections that apply. STATUS: all five implemented (2026, see
astap_corrections.diff); every change is marked "2026 fix (review item ASn)".
Still to do: compile and test in ASTAP and ASTAP_CLI.
AS1 unit_tiff.pas, compress_image_strips (= CCDciel C7)
height2=0 gives thread_count=0 and a division by zero on the next line.
Fix: refuse an empty image before computing thread_count.
AS2 unit_tiff.pas, compress_image_strips (= CCDciel C7 / A6)
An exception in a strip thread (TStripCompressThread has try/finally,
no except) is stored in FatalException and never checked; the strip
stream stays nil and the writer then uses it. Fix: check FatalException
and nil strips after WaitFor, free the strips, raise; in save_tiff_16/32/
48/96 catch it, free the file stream, delete the partial file and return
false (this is how it was done in CCDciel's cu_tiff.pas).
AS3 unit_ricecomp.pas, rice_decode_tiles and rice_encode_rows (= CCDciel A6)
Same pattern: TRiceTileThread and TRiceEncodeThread use try/finally
without except, and the dispatchers do not check FatalException.
- decoder: the tiles of a failed worker stay blank and no error is
reported (err_decode stays false);
- encoder: ok stays true while tile_data[] of the failed rows is nil, so
the caller writes a nil tile (access violation or corrupt .fz file).
Fix: after WaitFor, if FatalException<>nil set err_decode (decoder) or
ok:=false (encoder) with its message.
Not applicable to unit_ricecomp_unthreaded.pas: without threads the
exception reaches the caller normally.
AS4 save_tiff_new in unit_tiff.pas and unit_tiff_unthreaded.pas
(found while implementing CCDciel C7)
The test "if bitpix<=16 then 16-bit writer" is also true for a negative
FITS BITPIX (-32, -64), so a float image is written as 16-bit TIFF if a
caller passes the FITS BITPIX value. CCDciel now passes 32. Check the
ASTAP / ASTAP_CLI callers; if they can pass -32/-64, use abs(bitpix) or
test for bitpix=8/16 explicitly.
Implemented: the test is now abs(bitpix)<=16 (both TIFF units).
AS5 NaN pixels (related to CCDciel C2)
The TIFF writers (round(dum) after the 0..65535 clamp) and the Rice
encoder (round(img[0,y,x])) raise EInvalidOp for a NaN pixel, because
the comparisons in the clamp are false for NaN. In the threaded Rice
encoder this becomes AS3. Applies only if ASTAP image arrays can contain
NaN (CCDciel replaces NaN by 0 when reading since C2) - check the ASTAP
FITS reader; otherwise add "if IsNan(v) then v:=0" before round.
Implemented in all four units: NaN is written as 0; the Rice encoders
also clamp to 0..65535 before round() (round raised for huge values).
Checked and OK:
- unit_tiff_unthreaded.pas: flip_H works correctly (the bug fixed in the
threaded version, "flip_H previously had no effect", is not present).
- unit_tiff_unthreaded.pas: no strip threads, so AS1/AS2 do not apply.
- unit_ricecomp_unthreaded.pas: rice_encode_rows already returns for
height<=0; rice_decode_tiles returns for ntiles<=0.
Not reviewed (to do):
- read_tiff (all versions): it is used in ASTAP and parses untrusted files
(strip offsets/byte counts read from the file without limits, 32-bit
row size arithmetic). Review together with CCDciel item 2.3 above.
- The Rice decoder/encoder core (fits_rdecomp*, fits_rcomp_short) - same
code as CCDciel unit_ricecomp, see item 2.3.
- If ASTAP has its own copy of the .fz FITS layer (CCDciel
unit_ricecomp_fits: NativeUnpackFits, card writers), CCDciel items C4 and
C5 apply to it as well (keywords starting with Z dropped, ZBITPIX not
checked, unvalidated ZNAXIS, MakeFloatCard/MakeStrCard).
|
|
|
Unfortunately I have to revert all this change. There was problem with the program failing to exit on Mac and Windows because a thread was in infinite wait. And another problem with the program crashing when loading a FITS image in some specific condition. So I prefer to no more apply such large change for potential issue no one rise at at the moment. It is probably better to focus on small change for an issue that can be reproduced and tested after the fix. Jan Dubovsky open many issue and pull request today for such AI detected small problem and I will first look at them. |
|
|
Yes better to check the raised points one by one with no urgency. My intention was never to create a lot of work only to help you to make the code better. |
|
|
Sure, I am not accustomed to work with AI and I go too fast with the change, After I finish with current work, I will look at the point in the bugs and improvement files to address them one by one. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 26-09-19 22:06 | Han | New Issue | |
| 26-09-19 22:06 | Han | File Added: CCDciel_bugs.txt | |
| 26-09-19 22:06 | Han | File Added: CCDciel_improvement_suggestions.txt | |
| 26-09-20 09:44 | Han | Note Added: 0009942 | |
| 26-09-20 09:44 | Han | File Added: ccdciel_files.zip | |
| 26-09-21 09:10 | Patrick Chevalley | Note Added: 0009943 | |
| 26-09-29 18:08 | Patrick Chevalley | Note Added: 0009946 | |
| 26-10-01 22:42 | Han | Note Added: 0009949 | |
| 26-10-01 22:42 | Han | File Added: ccdciel_bugs-2.txt | |
| 26-10-01 22:42 | Han | File Added: ccdciel_improvements.txt | |
| 26-10-01 22:42 | Han | File Added: ccdciel_future_todo.txt | |
| 26-10-01 22:42 | Han | File Added: ccdciel corrections 2026-10-01.zip | |
| 26-10-07 13:31 | Patrick Chevalley | Note Added: 0009969 | |
| 26-10-07 13:55 | Han | Note Added: 0009970 | |
| 26-10-07 14:21 | Patrick Chevalley | Note Added: 0009971 |