Downloading files

This article explains how you can download transfers and individual files using the Filemail API.

In order to download files first you need to fetch the transfer details. This is done using the Transfer Details endpoint. This endpoint requires a unique transfer identifier or transfer tracking ID. Optionally it also requires a password, in case the sender password-protected the transfer. Please refer to the linked article for more details.

Example transfer

This is an example of a typical transfer response (fields which are not relevant to this article are removed)

GET /transfer/gaaihvyksuejqub
"transfer": {
    "id": "gaaihvyksuejqub",
    "url": "https://app.filemail.com/d/gaaihvyksuejqub",
    "compressedfileurl": "https://8888.filemail.com/api/file/get?compressedfilekey=GIhUF8z7...",
    "files": [
      {
        "filename": "path/to/file.jpg",
        "filesize": 93073,
        "downloadurl": "https://8888.filemail.com/api/file/get?filekey=n94A8Bc5...",
        "md5": "247797b2eb3e8e4ce398f66dbd9e7879"
      },
      {
        "filename": "..."
        ...
      }
    ]
}

Note that there are 2 different possible approaches to downloading files in Filemail:

  1. files[n].downloadurl - for downloading individual files in their original form (not compressed).

  2. transfer.compressedfileurl - for downloading all files within the transfer as a ZIP archive.

Download endpoints - HTTP layer recommendations

When using these endpoints please note the following:

  • Always use HTTPS to ensure the download is encrypted and protected from tampering or interception by malicious actors. Filemail API will always provide HTTPS URLs in response to transfer details requests.

  • Use GET HTTP method.

  • It is recommended to include your API Key in the request.

  • It is recommended to provide custom User-Agent header, which briefly describes the calling service or app. E.g. ACME File Downloader v.2.5.1

  • It is strongly recommended to apply chunking while downloading. This is achieved with the Range header.

Use chunk size equal to 50 * 1024 * 1024 bytes when downloading files and verifying their integrity. This provides maximum performance for file integrity checks. Also it is not guaranteed that the verification endpoints will work with a different chunk size in future releases.

  • Both types of URLs (ZIP and individual files) support range headers however only one range within that header is supported:

    • Range: bytes=100-200 ✅

    • Range: bytes=200-999, 2000-2499, 9500- â›”

  • When using Range header bytes are zero-indexed.

  • Provide chunk boundaries that both are within the file length limits. When downloading last chunk you need to calculate the end position taking into account the total file size - use the files[n].filesize property - value is in bytes.

    • E.g. if you chose a chunk size of 50*1024*1024 (recommended) then a 70MB file (70*10^6 bytes) will have 2 chunks with the following boundaries:

      • chunk 1: Range: bytes=0-52428799 (end position = 50*1024*1024-1)

      • chunk 2: Range: bytes=52428800-69999999

  • When using Range header expect 206 Partial Content response status code.

  • Without Range header expect 200 OK response status code.

  • Response body contains raw file bytes.

Download strategies

There are 2 approaches available:

  1. Download all files as ZIP:

    1. Use the compressedfileurl property.

    2. It is recommended to use Range header in order to perform chunked downloads. It is possible to download multiple chunks in parallel. Recommended maximum parallelism level is up to 4 chunks at a time.

    3. This method does not allow MD5 integrity check of the chunks or the final ZIP archive. If you need to perform an integrity check - you'll need to unzip the archive and check every file individually. Read more on verification.

    4. Filemail will register single download tracking event, at the end of the archive.

  2. Download files one-by-one:

    1. Use the files[n].downloadurl property.

    2. Make sure that you request all files from the Filemail API first - see filesLimit parameter in the GET /transfer/{transferid}endpoint.

    3. Loop over all returned files. It is possible to download multiple files in parallel. Recommended maximum parallelism level is up to 4 files at a time.

    4. While download single file - it is also possible to introduce parallelism on the file chunk level using the Range header. Recommended maximum parallelism level is up to 4 chunks at a time.

    5. Recommended chunk size is exactly 50*1024*1024 bytes. This gives you maximum performance when verifying file integrity. It is also a reasonable size in case a chunk should be re-downloaded (e.g. due to loss of network connectivity or because of a failed integrity check).

    6. In this approach Filemail will register download tracking multiple times, at the end of each file.

Verifying integrity of individual files and chunks

The following techniques apply only to individual files. They are not suitable for verifying entire ZIP archives or their chunks.

In order to verify integrity of an individual file:

  • Calculate MD5 hash of the downloaded file (see LocalChunkMD5 method in the Code Snippet section. Use start=0 and end=fileSize-1 .

  • Compare the resulting hash with the value in transfer details response files[n].md5 .

In order to verify integrity of a specific file chunk (downloaded using Range header):

  • Take the host from the URL you used to download the file. In the example response above it will be 8888.filemail.com .

  • Issue a GET request to https://[host]/GetRangeHash.ashx endpoint.

  • Make sure to always use HTTPS protocol.

  • Include your API Key in the request.

  • Provide request parameters in query string:

Query parameter
Parameter value

transferid

Use the value from transfer details transfer.id

thefilename

Use the URL-encoded value from transfer details files[n].filename

position

This should be the start byte 0-based index of the chunk. Use the same value as in the download request Range header.

length

Number of bytes equal to the chunk length. Note that last chunk of a file is usually shorter than the default 50 * 1024 * 1024 bytes.

Full request URL matching the example transfer and the returned response:

GET https://8888.filemail.com/GetRangeHash.ashx?
  transferid=gaaihvyksuejqub&
  thefilename=path%2Fto%2Ffile.jpg&
  position=0&
  length=93073
---
{ 
  "responsestatus": "OK",
  "hash": "d47d82f98f02b204e9200a98ecf8417e"
}

Compare the received hash with an MD5 hash calculated locally on raw bytes of a downloaded chunk. MD5 hashes returned from Filemail API are formatted as lowercase hex-strings e.g. d47d82f98f02b204e9200a98ecf8417e

It is strongly recommended to use chunk size of 50 * 1024 * 1024 bytes when downloading files and verifying their integrity. This provides maximum performance for file integrity checks. Also it is not guaranteed that the verification endpoints will work with a different chunk size in future releases.

In case hash verification fails, this means there was a data corruption either when reading data from Filemail servers or during nwetwork transport. In such case re-download the entire chunk once again and repeat verification.

Error handling

Prepare your download flow to handle unexpected errors:

  • Retry a chunk when it fails (HTTP status other than 200 or 206).

  • Use exponential backoff when issuing retries.

  • Apply maximum number of retries, then issue a notification to the calling app/service about file download error. Also cancel downloads of any other pending or queued chunks.

  • When a retry is successful - make sure to write the re-downloaded bytes to the same area of the file on the local file system.

  • When using .NET and HttpClient class - handle at least HttpRequestException and OperationCanceledException (when using a CancellationToken ).

.NET code snippet - chunked download

Below you can find a simplified downloader implementation including remote MD5 verification written in C#/.NET 9. For clarity parallelism and error handling are not included.

This code is not production ready and should not be used in real-world applications. Please adjust the implementation by applying error handling. Optionally add parallelism and download entire transfer as ZIP archive instead of individual files.

var transferId = "gaaihvyksuejqub";
var fileName = "path/to/file.jpg";
var downloadUrl = "https://8888.filemail.com/api/file/get?filekey=n94A8Bc5...";
var fileSize = 93073;

using var outputFile = new FileStream("file.jpg", FileMode.Create, FileAccess.ReadWrite);
var httpClient = new HttpClient();
var start = 0L;
var chunkSize = 50000L * 3;

while (start < fileSize)
{
    var end = start + chunkSize - 1;
    if (end >= fileSize)
    {
        end = fileSize - 1;
    }

    var httpRequest = new HttpRequestMessage(HttpMethod.Get, new Uri(downloadUrl));
    httpRequest.Headers.Add("x-api-key", "mJE5v1zHs....");
    httpRequest.Headers.Range = new System.Net.Http.Headers.RangeHeaderValue(start, end);
    httpRequest.Headers.UserAgent.ParseAdd("FilemailDemoClient/2.5.1");

    var responseStream = await httpClient.SendAsync(httpRequest, cancel);

    outputFile.Seek(start, SeekOrigin.Begin);
    await responseStream.Content.CopyToAsync(outputFile);

    Console.WriteLine($"chunk {start}-{end} complete");
    Console.WriteLine($"verifying md5");

    var localMD5 = await LocalChunkMD5(outputFile, start, end);
    var remoteMD5 = await GetRangeHash(downloadUrl, transferId, fileName, start, end - start + 1);

    if (false == string.Equals(localMD5, remoteMD5, StringComparison.OrdinalIgnoreCase))
    {
        throw new ApplicationException($"{fileName} MD5 mismatch {start}-{end} / {localMD5} / {remoteMD5}");
    }
    
    Console.WriteLine($"md5 OK");
    start = end + 1;
}

Console.WriteLine($"file {fileName} complete");
Console.ReadLine();

Here are the helper methods LocalChunkMD5 (calculated from a fragment of the local file) and GetRangeHash (requested from Filemail server):

private static async Task<string?> GetRangeHash(string downloadUrl, string? transferid, string? thefilename, long position, long length)
{
    var builder = new UriBuilder("https", new Uri(downloadUrl).Host);
    builder.Path = "GetRangeHash.ashx";
    var query = HttpUtility.ParseQueryString(string.Empty);
    query["transferid"] = transferid;
    query["thefilename"] = thefilename;
    query["position"] = position.ToString();
    query["length"] = length.ToString();
    builder.Query = query.ToString();

    var rangeHashUrl = builder.Uri.ToString();
    var httpClient = new HttpClient();
    var httpRequest = new HttpRequestMessage(HttpMethod.Get, rangeHashUrl);
    httpRequest.Headers.Add("x-api-key", "mJE5v1zHs....");
    httpRequest.Headers.UserAgent.ParseAdd("FilemailDemoClient/2.5.1");

    var response = await httpClient.SendAsync(httpRequest);
    var json = await response.Content.ReadAsStringAsync();
    var hash = JObject.Parse(json)!["hash"]!.Value<string>();
    
    return hash;
}

private static async Task<string> LocalChunkMD5(Stream stream, long start, long end)
{
    stream.Seek(start, SeekOrigin.Begin);

    var buffer = new byte[1024 * 1024]; // 1MB memory buffer
    var remaining = end - start + 1;

    using var md5 = MD5.Create();
    while (remaining > 0)
    {
        int read = await stream.ReadAsync(buffer, 0, (int)Math.Min(buffer.Length, remaining));
        if (read <= 0)
        {
            break;
        }
        md5.TransformBlock(buffer, 0, read, null, 0);
        remaining -= read;
    }

    md5.TransformFinalBlock(Array.Empty<byte>(), 0, 0);

    return Convert.ToHexString(md5.Hash!).ToLowerInvariant();
}
PreviousDeleting a file in a transferNextHandling contacts

Last updated

Was this helpful?