View Issue Details

IDProjectCategoryView StatusLast Update
0001669SkyChart1-Softwarepublic17-06-04 11:48
ReporterSasa Assigned ToPatrick Chevalley  
PrioritynormalSeveritytrivialReproducibilityhave not tried
Status resolvedResolutionfixed 
PlatformLinuxOSLubuntu 32-bitOS Version16.10
Product Version3.11 SVN 
Target Version4.2Fixed in Version4.1 SVN 
Summary0001669: Download progress
DescriptionWhen download something from CDC (PDF help, DSS, MPC, NEO or similar data), it will be shown progress of how many bytes is downloaded, which is unclear for user how many bytes still need to be download.

Progress indicator or percentage may be very useful in case of slow broadband internet and large data files to download. Many servers (HTTP) return size of the file on request for download (for FTP request that is trivial), which can be used then to indicate progress download rate.
TagsNo tags attached.

Activities

Sasa

17-05-28 10:06

reporter   ~0003823

Both download requests (HTTP and FTP) are fixed to collects size of the file and show download progress. The file size collection depend on server support.

HTTP download is fully tested, however FTP is not, as MPC site currently do not support anonymous FTP requests.

http://www.minorplanetcenter.net/iau/MPCORB.html


Related with an old issue:
http://www.ap-i.net/mantis/view.php?id=295

Sasa

17-05-28 10:31

reporter  

Sasa

17-05-28 10:35

reporter   ~0003824

Last edited: 17-05-28 10:37

View 2 revisions

It was attached older buggy file (Headers and Document properties must be cleaned properly before new GET call).

Proper file:
downloaddialog.pas-PROPER update.zip

Patrick Chevalley

17-05-28 22:44

administrator   ~0003825

Thank you very much for the patch!

This work fine, I only do one change, moving "FFileSize := 0;" at the start of TDownloadDaemon.Execute to be sure it is initialized.

I commit the change in rev 3593:
https://sourceforge.net/p/skychart/code/3593/

Sasa

17-05-29 18:29

reporter   ~0003828

More robust implementation is attached.

Sasa

17-05-29 18:30

reporter  

DL_progress-update.txt (1,663 bytes)   
function HTTP_FileSize(AHTTP: THTTPSend; AURL: string): int64;
{
  //SZ To get size of the file on HTTP request

  AHTTP - HTTP instance
  AURL  - URL of the file
}

var
  head: string;
  i: integer;
  Size: int64;
begin

  Size := 0;

  try

    AHTTP.Headers.Clear;
    AHTTP.Document.Clear;

    AHTTP.HTTPMethod('HEAD', AURL);
    head := AHTTP.Headers.Text;

    i := pos('content-length', LowerCase(head));

    if i > 0 then
    begin

      inc(i, 14);

      // Skip colon and white space chars
      while i <= length(head) do
      begin

	if not (head[i] in [' ', ':',#9, #10, #13]) then
          break;

        inc(i);

      end;

      while i <= length(head) do
      begin

        if head[i] in ['0'..'9'] then
          Size := Size * 10 + ord(head[i]) - 48
        else
          break;

        inc(i);

      end;

    end;

    AHTTP.Headers.Clear;
    AHTTP.Document.Clear;

  finally
  end;

  Result := Size;

end;


procedure TDownloadDaemon.Execute;
begin
  Fsockreadcount:=0;
  Fsockwritecount:=0;
  LastRead:=0;
  LastWrite:=0;
  FFileSize := 0;

  if protocol=prHttp then
  begin
    phttp^.Sock.OnStatus:=@SockStatus;

    //SZ Added code to retrieve file size for download progress
    FFileSize := HTTP_FileSize(phttp^, Durl);

    ok:=phttp^.HTTPMethod('GET', Durl)
  end
  else
  if protocol=prFtp then
  begin

    pftp^.OnStatus:=@FTPStatus;

    if pftp^.Login then
    begin
      pftp^.ChangeWorkingDir(Dftpdir);

      //SZ
      FFileSize := pftp^.FileSize(Dftpfile);

      ok:=pftp^.RetrieveFile(Dftpfile,false);
    end;

  end;

  if Assigned(FonDownloadComplete) then
     Synchronize(FonDownloadComplete);

end;
DL_progress-update.txt (1,663 bytes)   

Sasa

17-05-31 20:37

reporter   ~0003831

Last edited: 17-05-31 20:52

View 6 revisions

I have revised entire code, as there was a problem in slow download. As well, FTP download was fully tested.

1. By side of onStatus, it is added onMonitor event hook to read block of data. This will be fired only on body transfer, not all download (which include the header data) as on onStatus for HTTP.
2. FTP is fully tested. DSock is the proper sock to listen DL data transfer.
3. When transfer is finished, downloaded stream size is updated to match with actual size.
4. Instead fixed value to refresh progress, one second is used.

This way, all seems to work as intended.
Attached: downloaddialog.pas - FULL code revision.zip

Sasa

17-05-31 20:38

reporter  

Patrick Chevalley

17-06-04 11:48

administrator   ~0003836

Thank you!

I apply the change in revision 3595:
https://sourceforge.net/p/skychart/code/3595

Issue History

Date Modified Username Field Change
17-02-28 21:33 Sasa New Issue
17-03-05 10:57 Patrick Chevalley Assigned To => Patrick Chevalley
17-03-05 10:57 Patrick Chevalley Status new => acknowledged
17-03-05 10:57 Patrick Chevalley Target Version => 4.2
17-05-28 10:06 Sasa Note Added: 0003823
17-05-28 10:07 Sasa File Added: downloaddialog.pas-update.zip
17-05-28 10:31 Sasa File Added: downloaddialog.pas-PROPER update.zip
17-05-28 10:35 Sasa Note Added: 0003824
17-05-28 10:37 Sasa Note Edited: 0003824 View Revisions
17-05-28 22:44 Patrick Chevalley Status acknowledged => resolved
17-05-28 22:44 Patrick Chevalley Resolution open => fixed
17-05-28 22:44 Patrick Chevalley Fixed in Version => 4.1 SVN
17-05-28 22:44 Patrick Chevalley Note Added: 0003825
17-05-28 22:44 Patrick Chevalley File Deleted: downloaddialog.pas-update.zip
17-05-29 18:29 Sasa Status resolved => new
17-05-29 18:29 Sasa Resolution fixed => reopened
17-05-29 18:29 Sasa Note Added: 0003828
17-05-29 18:30 Sasa File Added: DL_progress-update.txt
17-05-31 20:37 Sasa Note Added: 0003831
17-05-31 20:38 Sasa File Added: downloaddialog.pas - FULL code revision.zip
17-05-31 20:43 Sasa Note Edited: 0003831 View Revisions
17-05-31 20:44 Sasa Note Edited: 0003831 View Revisions
17-05-31 20:48 Sasa Note Edited: 0003831 View Revisions
17-05-31 20:49 Sasa Note Edited: 0003831 View Revisions
17-05-31 20:52 Sasa Note Edited: 0003831 View Revisions
17-06-04 11:48 Patrick Chevalley Status new => resolved
17-06-04 11:48 Patrick Chevalley Resolution reopened => fixed
17-06-04 11:48 Patrick Chevalley Note Added: 0003836