CCDciel source review - IMPROVEMENT SUGGESTIONS =============================================== These are suggestions, not bugs. They are ordered roughly by benefit for effort. References like "B7" point to items in ccdciel_bugs_and_bad_practices.txt. 1. Tests that would have caught the current problems ----------------------------------------------------- - FITS round-trip test: write a synthetic image for each BITPIX (8, 16, 32, -32, -64) and for 1 and 3 planes, save, reload, compare pixel-exact. This catches A1 immediately and protects WriteFitsImage/ReadFitsImage from future "fixes". - Rice .fz round trip: NativePackFits -> NativeUnpackFits, compare pixels and the full keyword list (catches C4). - Header card tests: long strings, quotes, very large/small floats, NaN (C5). - JSON-RPC tests against the TCP server: 150-element params, string ids, empty POST body, missing Content-Length, oversized body (B1-B4, B6). - These can run headless with FPCUnit in a small console project that uses cu_fits / unit_ricecomp_fits / u_utils without the LCL forms. 2. Compiler settings -------------------- - Treat these warnings as errors (or at least review them): 5036/5037 (variable not initialised - see C1), 5033 (function result not set), 4055/4082 (conversions), 5057 (local not initialised). Use -vw -Sew in a CI build. - Keep a CI build with -Cr -Co -Ci (range, overflow, I/O checks) and heaptrc (-gh) and run the tests above with it. heaptrc reports the leaks listed in the bug file. - Build once with -Criot on a nightly sequence simulation (ASCOM/INDI simulators). 3. One helper for worker threads -------------------------------- The same create / start / wait / free / sum-results code appears in ReadFitsImage, Debayer, MedianFilter, GetBGRA, GetExpBitmap, GetStarList and cu_tiff. One generic helper removes the duplication and fixes A6 everywhere: procedure RunWorkers(const W: array of TThread); var i: integer; err: string; begin err := ''; for i := 0 to High(W) do W[i].Start; for i := 0 to High(W) do begin W[i].WaitFor; if (W[i].FatalException <> nil) and (err = '') then err := Exception(W[i].FatalException).Message; end; if err <> '' then raise Exception.Create('Worker failed: ' + err); end; Callers free the threads in a finally block. Consider TThread.ExecuteInThread or a small parallel-for (e.g. MTProcs from Lazarus) instead of hand-written thread classes for the per-row image loops. 4. Thread / GUI boundary ------------------------ - Rule: device and network threads never call event handlers directly. Every On... callback from a thread goes through Synchronize or TThread.Queue (B15). - Data shared between a thread and the main thread (FErrorDesc, ThrdActive[], FRunning flags) is either protected by a TCriticalSection or passed by value in the queued call. - Replace FreeOnTerminate threads that are referenced from elsewhere (autoguider, astrometry engine, async Python scripts) with owned threads that are joined and freed by their owner, as was already done for the TCP server (B7, C11). - Every class that calls Application.QueueAsyncCall calls Application.RemoveAsyncCalls(Self) in its destructor (C10). 5. Fewer ProcessMessages loops ------------------------------ Wait(), the Alpaca request polling and the process launchers all spin with Application.ProcessMessages. Better options, in order of effort: - Use TEvent.WaitFor with a timeout in worker threads instead of sleep loops. - For the Alpaca client, run the request in the THTTPthread and deliver the result through TThread.Queue to a continuation, instead of blocking with ProcessMessages. - Where a modal wait is unavoidable, add a re-entrancy guard (a "busy" flag that disables the relevant actions) so the user cannot start the same operation twice while it is still running. 6. JSON-RPC / TCP server ------------------------ - Build all responses with fpjson (TJSONObject.Add / AsJSON) - this removes every escaping problem at once (B6, C5 GetJSON). - Validate the request id type and echo it back unchanged (number, string or null). - Bind to 127.0.0.1 by default; offer "allow network access" as an option, with a shared token checked on each request (B5). Log remote connections. - Whitelist script names for SCRIPT_RUN: only [A-Za-z0-9_-], and verify that the resolved path is inside ScriptsDir/ConfigDir. - Do the HTTP request parsing in the client thread and only marshal the command execution to the main thread. Currently ProcessHttp/ProcessPost run inside Synchronize, so socket writes (including JPEG images) block the GUI. - Consider fphttpserver/fpjsonrpc from FPC instead of the hand-written HTTP parser. 7. External processes --------------------- - Use TProcess (or RunCommandInDir) everywhere and remove fpSystem, raw CreateProcess and fpFork/ExecVP. TProcess takes arguments as a list, so no shell quoting problems (B12), handles pipes safely (B11) and reaps child processes (B10). - One shared helper "RunAndCapture(exe, args, timeout, out stdout, out stderr)" can replace ExecProcess, ExecProcessMem and the copy inside TPythonThread. 8. Temporary files ------------------ - Extend UniqueTempFits into a general UniqueTempFile(prefix, ext) and use it for every temporary FITS/solve/log file (A3), deleting files in finally blocks. - Better still for plate solving: give each solve request its own temp directory. 9. FITS handling ---------------- - Centralise card formatting in one unit (key, value, comment, type) used by both TFitsHeader and unit_ricecomp_fits, with correct quoting, CONTINUE for long strings and exponent formatting (C5). - Handle NaN/Inf in statistics and display (C2) with IsNan checks or by mapping NaN to BLANK. - Validate all header dimensions in one place (NAXISn > 0, product within limits, computed in Int64) before any allocation, and call that from every reader (ReadFitsImage, GetFlatLevel, header extension scan, .fz reader, Alpaca image). - Compute data unit sizes as ceil(bytes/2880)*2880 in Int64 (C1). - Sanitise every header value used in file or directory names, not only OBJECT (C6). 10. Calibration and guiding math -------------------------------- - Guard all divisions by measured quantities (drift, gain sums, durations) with a minimum threshold and a clear error message instead of an FPU exception (C12). - Replace blocking sleep() calls on the main thread in calibration with a timer step in the existing state machine. 11. Error handling policy ------------------------- - Replace empty except blocks with at least a log line (e.g. except on E: Exception do LogDebug('ascom weather: ' + E.Message); end). Hidden failures in unattended sequences are hard to diagnose (D1). - Where an exception is expected (driver does not implement a property), catch the specific exception class rather than everything. 12. Structure of pu_main ------------------------ pu_main.pas (20k lines) could be split without changing behaviour: - the JSON-RPC dispatcher (TCPjsoncmd and jsoncmd_*) into its own unit with a method table (name -> handler) instead of a long if/else chain; - the TCP server start/stop into a small controller class; - image display/annotation into its own unit. This makes the code easier to test and lowers the chance of unrelated regressions. 13. Credentials --------------- Store Alpaca credentials with the platform secret store (Windows DPAPI via CryptProtectData, libsecret on Linux, Keychain on macOS), keep the current format as a read-only fallback for migration, and send them as an HTTP Authorization header instead of embedding them in the URL (D8). 14. Small clean-ups ------------------- - Remove dead code: read_tiff (not called), the unused hdr string in the TCP thread, HdrIndex(hx,'') in NativeUnpackFits, self-assignments (D3), the "delayed" parameter of SaveAstroTiff. - Make the even-coordinate rounding explicit (x := x and not 1) instead of relying on banker's rounding (C8). - Replace "with" blocks in thread code with explicit references (D7). 15. Device layer: report outcomes (mount, focuser, dome) -------------------------------------------------------- - Turn the "action" methods into functions that return a result record or enum, for example TDeviceResult = (drOK, drTimeout, drRefused, drError) plus a message. Slew, Park, SetShutter, SetPosition and FlipMeridian would all return it, and T_mount.SetParkInterface / T_dome.SetParkInterface would pass it on. - Implement each wait once in the base class (WaitUntil(condition, timeout)) and let the drivers only provide the condition. That removes the five nearly identical WaitMountSlewing/WaitDomePark/WaitShutter/WaitFocuserMoving loops per interface and gives one place to make timeouts configurable. - For INDI, check the property state (IPS_OK / IPS_ALERT) after WaitBusy instead of the locally modified switch or number values (E7). - Model the shutter as a state (open, closed, opening, closing, error, unknown) instead of a boolean, in all three dome drivers (E3, E4). - Make the safety checks fail-closed: when the safety monitor is configured but not connected, refuse to open or unpark unless the user explicitly allowed it (E9). 16. Sequence engine ------------------- - Store all time-of-day values in one unit (fractions of a day, or hours, but not both). A small TTimeOfDay type or helper functions with the unit in the name (PivotHours, StartDayFraction) would have prevented F1 and F3. Add tests for windows that cross midnight, with and without target coordinates. - Let the end-of-night routine (RunEndAction) verify every step, retry once, and send an e-mail when a park or close fails. Enforce "park mount before closing a roll-off roof" in code, not only in documentation (F6). - Make the meridian flip loop time-aware: recompute the hour angle every iteration and stop tracking as soon as the hard limit is reached (F2). - Treat "guiding failed to start" and "slew failed" as step/target failures that are retried or reported, and do not increment repeatdone for a target that took no frames (F4). - Consider moving the sequence state machine off Wait()/ProcessMessages to an explicit state + timer design; most of the sequence code already uses timers, so the remaining blocking waits are the exception (F9).