================================================================================ CCDciel - IMPROVEMENT SUGGESTIONS ================================================================================ Source reviewed : ccdciel-master.zip, src/ only (143 units, ~114,700 lines) Scope : structural, architectural and maintainability suggestions. Outright defects are in the companion file CCDciel_bugs_and_bad_practices.txt. Nothing here is a correctness problem. These are changes that would make the existing correctness easier to preserve. They are ordered roughly by return-on-effort, not by size. ================================================================================ 1. Collapse the ASCOM / Alpaca driver duplication ================================================================================ Measured line-level similarity between each COM driver and its REST twin (whitespace-insensitive, blank lines removed): device cu_ascom*.pas cu_ascomrest*.pas similarity cover 331 298 59% dome 392 355 58% focuser 523 501 65% mount 1,223 1,095 55% rotator 349 308 55% safety 230 206 56% switch 317 425 53% weather 737 694 66% wheel 351 334 62% camera 1,751 1,580 60% ---------------------------------------------------------- total 6,204 5,796 ~6,000 duplicated lines The two families differ only in how a property is fetched: COM : FBinX := V.BinX; REST: FBinX := V.Get('binx').AsInt; Everything around that - status timers, state machines, connect/disconnect handshakes, unit conversion, event dispatch, the empty exception handlers - is copied. Why this matters beyond tidiness: a fix applied to one copy will not reach the other. The two weather units alone contain twenty-four near-identical empty exception handlers; a change to how "property not implemented" is reported would have to be made twenty-four times in two files. Suggested shape: T_ascomdevice_base = class(T_basedevice) // all the logic protected function GetD(const prop: string): double; virtual; abstract; function GetI(const prop: string): integer; virtual; abstract; function GetB(const prop: string): boolean; virtual; abstract; procedure PutD(const prop: string; v: double); virtual; abstract; ... end; T_ascomcom_transport = class(T_ascomdevice_base) // OLE variant calls T_ascomrest_transport = class(T_ascomdevice_base) // HTTP/JSON calls Then each device becomes one unit with two thin transport descendants. A staged migration is realistic: start with the smallest pair (safety, 230/206 lines), prove the base class, then work up to camera. Related: cu_indi*.pas is another 8,230 lines with its own copy of much of the same device-state logic. A common T_basedevice contract across all three transports (INDI / ASCOM COM / Alpaca) would be the larger prize. ================================================================================ 2. Break up pu_main.pas ================================================================================ pu_main.pas 20,275 lines That is 18% of the codebase in one unit, and it contains the message pump, the capture loop, the sequence driver, file loading, the image display, the menu handlers, the TCP command dispatcher, the guider and finder views, and the astrometry glue. Practical consequences visible in this review: * the TCP lifecycle bug (bug report item 4) spans pu_main and cu_tcpserver but the pu_main half is spread over five call sites hundreds of lines apart; * LoadRawFile and LoadPictureFile (15554, 15602) are near-identical and could not easily be shared because both are methods of the 20k-line form; * Tf_main has enough state that reasoning about re-entrancy (see item 5) is impractical. The natural seams already exist in the naming: * TCP/JSON command dispatch -> its own unit, driven by an interface, testable without a form * file open / save / format -> a TImageIO helper * capture and sequence drive -> already partly in fu_capture / fu_sequence / cu_targets; move the remaining logic there * the finder / guider views -> fu_finder and fu_internalguider already exist None of this requires a redesign - it is moving methods out and replacing direct field access with a small number of properties or events. ================================================================================ 3. Make TFits stream ownership explicit ================================================================================ cu_fits.pas:254 Property Stream : TMemoryStream read GetStream write SetStream; cu_fits.pas:1701 procedure TFits.SetStream(value:TMemoryStream); ... if FStream<>nil then FreeAndNil(FStream); FStream:=value; Assigning to a property takes ownership of the assigned object and destroys the previous one. Nothing at the call site says so: fits.Stream := FitsStream; // pu_main.pas:15580 - looks like a reference Every leak in section 7 of the bug report traces back to this. Callers correctly omit FitsStream.Free on the happy path and therefore also omit it on the error paths, where ownership never transferred. Suggestion: keep the property read-only and add an explicit method. procedure TFits.TakeStream(var value: TMemoryStream); // nils the caller's // reference With a var parameter the compiler makes the transfer visible and the caller cannot double-free. Call sites become: FitsStream := TMemoryStream.Create; try ... fits.TakeStream(FitsStream); // FitsStream is nil afterwards finally FitsStream.Free; // no-op if it was taken end; which is leak-proof on every path. GetStream (used at cu_fits.pas:1830) has the mirror-image problem: it returns a stream the caller must free, which SaveToFile does and TSaveFits.Execute forgets on its error path. ================================================================================ 4. Replace hand-rolled endian conversion with the RTL primitives ================================================================================ cu_fits.pas already uses the right tool in three places: 1257 x16:=BEtoN(d16[npix]); 1287 x:=BEtoN(LongInt(d32[npix])); 2379 x16:=BEtoN(d16[j]); but the float paths route through InvertF32/InvertF64 in u_utils.pas, which are hand-written byte shuffles (and, per bug report item 3, one of them is wrong). Standardising on BEtoN / NtoBE plus a Move or a variant record would: * remove the signed-promotion class of bug entirely; * remove the ad-hoc NaN handling, which is currently both wrong and inconsistent between the 32- and 64-bit versions; * let FPC emit a single BSWAP instruction instead of four shifts and three ORs per pixel - measurable on a 60-megapixel float image. A small type-punning helper used consistently everywhere is worth having: type TF32 = packed record case boolean of true: (f: single); false: (i: longword); end; This also resolves the fragile single()/longword() expression casts at cu_fits.pas:2619. ================================================================================ 5. Reduce reliance on Application.ProcessMessages ================================================================================ 99 occurrences across the codebase. Each one allows arbitrary user interaction - including menu clicks that start a new capture, abort the sequence, or close the form - to run inside a routine that is mid-way through updating state. The failure mode is not a crash on the developer's machine; it is a rare, unreproducible "the sequence did something strange" report from a user who clicked at the wrong moment during a 300-second exposure. Two incremental mitigations that do not require rearchitecting: a) Guard the re-entrant entry points. A simple busy flag on the handlers that ProcessMessages can reach (Start/Stop capture, sequence control, file open) costs little and eliminates the worst cases. b) Where ProcessMessages exists only to keep the UI responsive during a wait, replace the wait with a timer-driven state machine or move the work to a thread. u_utils.Wait and cu_waitthread.WaitExecute already provide the scaffolding for the latter. Related: 16 direct Sleep() calls. Sleep on the main thread freezes the UI; Sleep in a polling loop (cu_fits.pas:2453) wastes up to 100 ms per image read. An event or condition variable is both more responsive and more correct. ================================================================================ 6. Speed up and simplify FITS header parsing ================================================================================ cu_fits.pas:2170-2260 TFits.GetFitsInfo for i:=0 to FHeader.Rows.Count-1 do begin keyword:=trim(FHeader.Keys[i]); buf:=trim(FHeader.Values[i]); if (keyword='SIMPLE') then ... if (keyword='XTENSION') ... if (keyword='BITPIX') ... ... approximately 40 more sequential string comparisons ... end; Every keyword is compared against all forty literals, and none of the tests is an "else if", so the comparisons continue after a match. For a header with a few hundred cards - common for stacked or plate-solved images - that is tens of thousands of string comparisons per image load. Options, in increasing order of effort: * change the chain to "else if" (one-line change, roughly halves the work); * order the tests by frequency; * dispatch on a case over a hashed or interned keyword; * build a keyword -> handler map once at unit initialisation. The same file also has the Indexof/Delete/Insert triplet repeated for BITPIX, BSCALE and BZERO in two places (1817-1826 and 4176-4183). A single FHeader.SetValue(key, value, comment) that handles "replace in place or insert before END" would remove the duplication and the negative-index edge case noted in the bug report. ================================================================================ 7. Introduce a logging discipline ================================================================================ The codebase has a usable msg(text, level) convention, but it is applied unevenly: * 171 exception handlers log nothing at all; * some paths log at level 0 (user-visible error) for conditions that are routine, e.g. an ASCOM property a driver does not implement; * debug output is compiled in via {$ifdef debug_raw} writeln(...) rather than routed through msg, so it goes to stdout and is unavailable in a normal build when a user reports a problem. Suggestions: * define what each level means, in a comment at the top of the unit that owns msg; * log every swallowed exception at a diagnostic level, once per condition rather than once per status-timer tick (the drivers poll continuously, so an unconditional log would flood); * fold the debug_raw writeln calls into msg at level 9, so the information is obtainable from a released build by raising the log level. A user who reports "images stopped saving after two hours" currently leaves nothing behind; with the above, the failed TSaveFits exception would be in the log. ================================================================================ 8. Concurrency primitives are used inconsistently ================================================================================ Counted across the codebase: Synchronize 53 EnterCriticalSection 6 TCriticalSection 8 InitCriticalSection 1 The pattern in cu_fits.pas - InitCriticalSection at the start of ReadFitsImage, DoneCriticalSection at the end - creates and destroys the lock on every image read, which is both wasteful and unsafe if an exception occurs in between. The critical section is a field of TFits and has the same lifetime as the object; it should be created in the constructor and destroyed in the destructor. Cross-thread state elsewhere is shared without any protection at all: * TTCPDaemon.ThrdActive[] is written from worker threads and read from the daemon loop; * TReadFits result fields (dmin, dmax, sum, sum2, ni, hist) are read by the main thread after a timeout that does not guarantee the workers have stopped. A short document - or even a comment block per unit - stating which fields are owned by which thread would prevent most of this. Where the data really is shared, prefer TThread.WaitFor over polling a boolean, and Interlocked* or a critical section over unprotected reads. The PostMessage + strnew(PChar(...)) channel used between the astrometry/save threads and the main form (thirteen allocation sites, three disposal sites) is another candidate for replacement - a thread-safe queue of strings, drained by Synchronize or a timer, removes the manual allocation entirely and with it the leak-on-shutdown described in the bug report. ================================================================================ 9. Reconsider the global DefaultFormatSettings override ================================================================================ pu_main.pas:1525 DefaultFormatSettings.DecimalSeparator:='.'; pu_main.pas:1526 DefaultFormatSettings.TimeSeparator:=':'; pu_main.pas:1540 Application.UpdateFormatSettings := False; This is the right answer for FITS I/O, which mandates '.', and it is why the locale-dependent strtofloat calls in GetFitsInfo do not misbehave on a Dutch, German or French system. But it is applied process-wide, so every number and date shown to the user also ignores their locale. A cleaner split is to keep the user's locale in DefaultFormatSettings and use a dedicated, explicitly-constructed TFormatSettings for all file I/O: var FitsFormat: TFormatSettings; // '.' separator, invariant ... bzero := StrToFloat(buf, FitsFormat); unit_ricecomp_fits.pas already does exactly this (lines 205-208, 279-282, 449), so the pattern exists in the codebase and just needs to be applied consistently. ================================================================================ 10. Small consistency items ================================================================================ 10a. Naming collisions TTCPThrd.onTerminate (TIntProc) hides TThread.OnTerminate (TNotifyEvent). Rename to something that cannot be mistaken for the inherited member. 10b. Threading model per class TTCPDaemon and TTCPThrd use FreeOnTerminate := True; TReadFits, TDebayerImage, TMedianFilter, TGetStarList use False; TSaveFits uses True. Both models are valid, but mixing them in one codebase means every reader has to check. Pick one as the default and document the exception. 10c. Fixed-size thread arrays Six places declare "thread: array[0..15] of T..." and clamp with max(1, min(16, MaxThreadCount)). The bound is consistent and correct today, but it is expressed as a literal in twelve places. A single constant (MaxParallelThreads = 16) used in both the declaration and the clamp would make a future increase a one-line change. 10d. Magic numbers in buffer sizes d8[1..2880], d16[1..1440], d32[1..720], d64[1..360] are all "one 2880-byte FITS record". Deriving them from a named constant (FitsRecordSize = 2880) would make the relationship - and the correctness of the npix mod N tests that pair with them - self-evident. 10e. TStringList sorting SortFilterListInc / SortFilterListDec are hand-written bubble sorts. TStringList.CustomSort with a comparison function is shorter, faster, and removes the dead-statement confusion noted in the bug report. 10f. Dead code u_utils.AirMass wraps an expression that cannot raise in a try/except; the Delete before Insert in the FITS header updates is redundant because Insert already replaces in place. Removing dead defensive code makes the remaining defensive code meaningful. ================================================================================ 11. Testability ================================================================================ The most valuable single change in this list, if the goal is to stop this class of bug recurring, is to make the pure computational units testable without a GUI. The following are already free of LCL dependencies or nearly so: u_utils.pas (coordinate transforms, JD, sorting, endian helpers) u_refraction.pas u_ephem.pas uDE.pas unit_ricecomp.pas / unit_ricecomp_fits.pas cu_fits.pas (mostly - it descends from TComponent, not TForm) A small console test project over just those would have caught, mechanically: * InvertF64's sign-promotion error (round-trip a known bit pattern); * the float FITS write flushing the wrong buffer (write, read back, compare); * the naxis1*naxis2 overflow (feed a crafted header); * the to360 precision loss (compare against FMod); * the hextostr zero-byte abort (round-trip a string containing #0). FPCUnit ships with Lazarus, so the infrastructure cost is close to zero. Even twenty assertions over the FITS read/write round trip would be a meaningful safety net for a program whose entire purpose is to not corrupt astronomical data. ================================================================================ END OF IMPROVEMENT SUGGESTIONS ================================================================================