Contents

About

NScurl is a NSIS (Nullsoft Scriptable Install System) plugin with advanced HTTP/HTTPS capabilities.
It's implemented on top of libcurl with OpenSSL as SSL backend.

Official project page: https://github.com/negrutiu/nsis-nscurl
Dependencies: https://github.com/negrutiu/libcurl-devel

Features


NScurl::http <METHOD> <URL> <DEST> <params> /END

Description:

This function creates a new HTTP request.
Requests are added to the internal transfer queue and wait there until a worker thread becomes available.
Completed transfers remain in the queue and can be queried at any time.
By default the function waits (synchronously) for the new request to complete, unless /BACKGROUND parameter is used.

Return value:

The return value is pushed to the stack.
By default, the function returns the transfer status (aka "@ERROR@" query keyword). "OK" for success.
If /BACKGROUND is specified, the function returns an unique transfer ID.
If /RETURN is specified, the function returns a custom value depending on the specified query keyword.

Examples:

# Quick transfer (with Cancel and resume)
NScurl::http GET "http://live.sysinternals.com/Files/SysinternalsSuite.zip" "$TEMP\SysinternalsSuite.zip" /CANCEL /RESUME /END
Pop $0 ; Status text ("OK" for success)

# Quick transfer (with custom GET parameters and custom request headers)
NScurl::http GET "https://httpbin.org/get?param1=value1&param2=value2" "$TEMP\httpbin_get.json" /HEADER "Header1: Value1" /HEADER "Header2: Value2" /END
Pop $0 ; Status text ("OK" for success)

# POST json file
NScurl::http POST "https://httpbin.org/post" Memory /HEADER "Content-Type: application/json" /DATA '{ "number_of_the_beast" : 666 }' /END
Pop $0 ; Status text ("OK" for success)

# POST json file (as multi-part form)
NScurl::http POST "https://httpbin.org/post" Memory /POST "User" "My user name" /POST "Password" "My password" /POST FILENAME=maiden.json TYPE=application/json "Details" '{ "number_of_the_beast" : 666 }' /END
Pop $0 ; Status text ("OK" for success)

Parameters:

GET | POST | HEAD | PUT | ... HTTP method.
NOTE: This parameter is mandatory.
URL Full URI, including query parameters.
NOTE: Query parameters must be escaped by the caller. NScurl::escape is available for that.
NOTE: This parameter is mandatory.
filename|MEMORY Transfer destination.
If "MEMORY" is specified the remote content is downloaded to RAM. It can be retrieved later using the @RECVDATA@ query keyword.
NOTE: The data downloaded to MEMORY will be truncated to NSIS' maximum string length (1KB, 4KB, 8KB, depending on the NSIS build). If you expect larger data, it's recommended to download it to a file on disk.
NOTE: This parameter is mandatory.
/RETURN query_string Request a custom return value. Default is "@ERROR@" (aka transfer status)
Any combination of query keywords is accepted.
(e.g. "@ERRORCODE@ - @ELAPSEDTIME@")
/PROXY proxy Connect through a proxy server.
Syntax: scheme://address.domain[:port].
Supported schemes: http, https, socks4, socks4a, socks5, socks5a
For more information visit libcurl's CURLOPT_PROXY documentation.
/DOH <url> Specify a DNS over HTTPS server to resolve DNS requests securely.
NOTE: This DoH server is used for the current transfer only. It's not a global setting.
EXAMPLE: [...] /DOH "https://cloudflare-dns.com/dns-query"
EXAMPLE: [...] /DOH "https://doh.opendns.com/dns-query"
EXAMPLE: [...] /DOH "https://dns.google/dns-query"
/TIMEOUT time
/CONNECTTIMEOUT time
Connect timeout (default: 5m)
This value applies to a single connection attempt.
Unless you also specify /INSIST, NScurl will only try to connect once.

NOTE: The time value represents the timeout period in milliseconds. The "s", "m" or "h" suffixes are accepted.
EXAMPLES: /TIMEOUT 5000, /TIMEOUT 5s, /TIMEOUT 15m, /TIMEOUT 24h
/COMPLETETIMEOUT time Transfer timeout (default: infinite)
This value sets a maximum time limit that a transfer is allowed to run. When this timeout is reached the transfer is cancelled.
See /TIMEOUT for time examples.
/LOWSPEEDLIMIT bps time Low speed limit (default: none)
Aborts the transfer if the transfer speed falls below bps for a period of time.
See /TIMEOUT for time examples.

EXAMPLE: [...] /LOWSPEEDLIMIT 204800 10s [...]
/SPEEDCAP bps Speed cap (default: none)
The transfer speed will not be allowed to exceed this value.
/INSIST NScurl will try more aggressively to connect to the webserver.
It will keep trying to connect until a timeout is reached, even if critical errors occur (e.g. no network connection, server down, etc.)
In addition NScurl will try to re-establish dropped HTTP GET transfers and resume them.
By default any error will abort the transfer.
/RESUME Resume the transfer if (part of) the destination file already exists.
By default the destination is overwritten and the transfer starts over.
/NOREDIRECT Don't follow HTTP redirections. They are followed by default
/USERAGENT agent Overwrite the default user agent.
Default is nscurl/<version>
/REFERER url Optional referrer URL.
/DEBUG [nodata] debugfile Write transfer HTTP/SSL details to a debugging file.
nodata prevents the remote content from being actually written to the debugging file. Only its size gets written.
Disabled by default.
/AUTH [TYPE=basic|digest|digestie] user pass HTTP authentication.
Type is determined automatically if not specified. However, an explicit type must be specified when connecting to servers with "hidden" authentication.
Username and password must be cleartext, unescaped.
For more information visit libcurl's CURLOPT_HTTPAUTH documentation.
/AUTH TYPE=bearer token HTTP bearer authentication.
The OAuth 2.0 token is mandatory.
For more information visit libcurl's CURLOPT_HTTPAUTH documentation.
/TLSAUTH user pass TLS-SRP (Secure Remote Password) authentication.
Username and password must be cleartext, unescaped.
For more information visit libcurl's CURLOPT_TLSAUTH_TYPE documentation.
/HEADER hdr(s) Additional HTTP request header(s).
NOTE: Multiple headers can be separated by CRLF ($\r$\n)
NOTE: Multiple /HEADER parameters are accepted.
/DATA [-string|file|memory] <data> Upload local data to the server.

The data is read either from a file or directly from memory.
The optional -string, -file, -memory hints can be used to indicate data's origin.
If unspecified, the function tries to guess whether data represents a file name or a generic string.

EXAMPLES:
NScurl::http PUT ${url} ${file} /DATA "Send generic string" /END
NScurl::http PUT ${url} ${file} /DATA "C:\dir\file.dat" /END
NScurl::http PUT ${url} ${file} /DATA -string "C:\dir\file.dat" /END /* Upload the file path itself as string */
NScurl::http PUT ${url} ${file} /DATA -memory 0xdeadbeef 256 /END

NOTE: The -memory hint must be followed by two parameters: address and size.
NOTE: Although optional, it's recommended that you always specify the -string|file|memory hint.
NOTE: Requires POST or PUT methods. Ignored otherwise.
/POST

[FILENAME=remote_filename]
[TYPE=mime_type]
<name>
[-string|file|memory] <data>

Upload a multipart form to the server.

FILENAME: Optional remote file name
TYPE: Optional MIME type
name: Form part name
data: Form part data

EXAMPLE:
NScurl::http POST ${url} ${file} \

/POST "User" "My User Name" \
/POST "Pass" "My Password" \
/POST "InfoFile" -string "$TEMP\MyDetail.json" /* Upload the file path itself as string */ \
/POST "InfoData" -file "$TEMP\MyDetail.json" /* Upload the file content */ \
/POST "Image" FILENAME=MyImage.jpg TYPE=image/jpeg "$TEMP\MyImage.jpg" \
/POST "Details" FILENAME=MyDetails.json TYPE=application/json '{ "number_of_the_beast" : 666 }' \
/END


NOTE: See /DATA to learn how data is interpreted.
NOTE: Multiple /POST parameters are accepted. All form parts will be sent as one multipart form.
NOTE: Requires POST method. Ignored otherwise.
/Zone.Identifier
/MARKOFTHEWEB
Marks the output file with the "Mark of the Web".
An alternate NTFS data stream named "Zone.Identifier" is attached to the output file.
/CACERT "path\cacert.pem" Validate webserver identity using a custom cacert.pem certificate database.
By default a built-in cacert.pem is extracted and used at runtime.
SSL validation can be turned off by specifying an empty ("") path.

NOTE: The embedded cacert.pem can rapidly become obsolete. That would lead to legitimate websites being treated as untrusted.
NOTE: The libcurl project maintains a cacert.pem database that is considered trusted. You can download it from their website and feed it to NScurl as custom certificate store.
/CERT thumbprint Specify an additional trusted certificate SHA1 thumbprint (e.g. /CERT 917e732d330f9a12404f73d8bea36948b929dffc).
Trusted thumbprints are used for SSL validation in addition to the cacert.pem database.

NOTE: Multiple /CERT parameters are accepted. All thumbprints are aggregated into a big certificate whitelist.
NOTE: cacert.pem database can be turned off by specifying an empty path (/CACERT ""), leaving the thumbprint list the only certificate store used for validation.
/DEPEND id Makes this HTTP request dependent on another request.
The new request will wait in queue until its dependency gets completed.
Useful to establish a precise HTTP request order in a multi-threaded environment.
/TAG tag Assign a tag to the new HTTP request.
Multiple transfers can be grouped together under the same tag.
NOTE: Tags are custom strings with no character restrictions.

EXAMPLE:
NScurl::http GET ${URL1} ${File1} /TAG "FirstTag" /BACKGROUND /END
NScurl::http GET ${URL2} ${File2} /TAG "SecondTag" /BACKGROUND /END
NScurl::http GET ${URL3} ${File3} /TAG "FirstTag" /BACKGROUND /END
NScurl::wait /TAG "FirstTag" /END
NScurl::cancel /TAG "FirstTag" /REMOVE
/BACKGROUND By default, NScurl::http creates a new HTTP request and waits for it to complete.
With /BACKGROUND you mark the new request as a background transfer and place it in the queue for later processing.
The call returns immediately, no visual progress is displayed.
NOTE: NScurl::http will return a transfer ID instead of the transfer status. The ID can be used later for querying.
/PAGE Wait for the transfer to complete in Page-mode.
When waiting from an NSIS section (while on the InstFiles page), the function will create a dedicated progress bar to visually display the progress.
NOTE: /PAGE is the default wait mode if nothing else is specified!
NOTE: /PAGE is incompatible with /BACKGROUND.
/POPUP Wait for the transfer to complete in Popup-mode.
Progress is displayed on a pop-up window.
NOTE: /POPUP is incompatible with /BACKGROUND.
/SILENT Wait silently for the transfer to complete.
No visual progress is displayed.
NOTE: This is the default wait mode for silent installers.
NOTE: /SILENT is incompatible with /BACKGROUND.
/CANCEL Enable Cancel button when waiting in Page-mode or Popup-mode.
By default transfers can not be cancelled.
/TITLEWND hwnd
/TEXTWND hwnd
/PROGRESSWND hwnd
/CANCELWND hwnd
Optional Title window handle.
Optional Text/Status window handle.
Optional progress bar window handle.
Optional Cancel button handle.
NOTE: These windows are automatically detected/created by default.
/STRING TITLE string
/STRING TITLE_NOSIZE string
/STRING TITLE_MULTI string
/STRING TEXT string
/STRING TEXT_NOSIZE string
/STRING TEXT_MULTI string
Overwrite the default (English) UI strings.
Useful to create localized installers.
/END Must always conclude the list of parameters.
NOTE: This parameter is mandatory.


NScurl::query [/ID id] [/TAG tag] "query_string"

Description:

Query information.
This function will replace query keywords inside query_string with real data.
Transfer-specific keywords are only available when a single transfer is matched (See /ID and /TAG).
Global keywords are always available.

Return value:

query_string with expanded keywords is returned on the stack.

Examples:

; Query information about a specific HTTP request
NScurl::http GET "http://live.sysinternals.com/Files/SysinternalsSuite.zip" "$TEMP\SysinternalsSuite.zip" /RETURN "@ID@" /END
Pop $0 ; Transfer ID
NScurl::query /ID $0 "Status: @ERROR@, Headers: @RECVHEADERS@"
Pop $1 ; Server response headers

; Query global information
NScurl::query "@TOTALSIZE@ - @TOTALSPEED@"
Pop $0

Parameters:

/ID id Query information about a specific transfer.
The transfer ID is returned by NScurl::http.
/TAG tag Query information about multiple transfers tagged with the same tag.
See NScurl::http.
query_string The input string.
The function will replace query keywords with real data and return the result.

Query Keywords:

Transfer-specific keywords Comments
@ID@ Unique transfer ID
Value: (Integer) >= 1
Useful for NScurl::http /return "@ID@"
@STATUS@ Transfer status.
Value: (String) One of "Waiting", "Running" or "Complete"
@METHOD@ HTTP method (e.g. GET, POST, PUT, etc.)
@URL@ Full HTTP request URI.
@FINALURL@ The final HTTP request URI, after all redirections had been followed.
@OUT@ Transfer destination.
Can be a file path, or the special term "Memory".
@OUTFILE@ Destination file name extracted from @OUT@ (e.g. "SysinternalsSuite.zip")
@OUTDIR@ Destination directory extracted from @OUT@ (e.g. "C:\Downloads")
@SERVERIP@ HTTP server IP address.
@SERVERPORT@ HTTP server IP port number (usually 80 or 443)
@FILESIZE@
@FILESIZE_B@
Remote file size extracted from Content-Length HTTP header.
Value: (String) Nicely formatted file size (e.g. "100 bytes", "250 KB", "10 MB", "1.2 GB", etc.)
NOTE: It may be unknown during the transfer. Some servers don't report this value.
NOTE: @FILESIZE_B@ returns the same value in bytes.
@XFERSIZE@
@XFERSIZE_B@
The amount of data actually transferred.
NOTE: It's usually the same value as @FILESIZE@, but it can be smaller if the transfer got cancelled.
NOTE: @XFERSIZE_B@ returns the same value in bytes.
@PERCENT@ Transfer progress.
Value: (Integer) 0-100, or -1 if the percent is unknown.
NOTE: The percent might be unknown if the webserver doesn't send Content-Length header.
@SPEED@
@SPEED_B@
Transfer speed, nicely formatted (e.g. "100 KB/s", "1.2 MB/s", etc.)
NOTE: @SPEED_B@ returns the same value in bytes/s.
@TIMEELAPSED@
@TIMEELAPSED_MS@
Transfer elapsed time. Formatted as [d.][hh:]mm:ss (e.g. "05:02" for 5m and 2s)
It doesn't include the time this request has waited in the queue.
NOTE: @TIMEELAPSED_MS@ returns the same value in milliseconds.
@TIMEREMAINING@
@TIMEREMAINING_MS@
The estimated time required by this transfer to complete. Formatted as [d.][hh:]mm:ss
NOTE: @TIMEREMAINING_MS@ returns the same value in milliseconds.
@SENTHEADERS@
@SENTHEADERS:Header-Name@
@SENTHEADERS_RAW@
HTTP request headers.
Some special characters such as \t, \r, \n are replaced by their string representation "\t", "\r", "\n"
NOTE: @SENTHEADERS:Header-Name@ extracts the value of a specific header.
NOTE: @SENTHEADERS_RAW@ returns the same value with no characters replaced.
@RECVHEADERS@
@RECVHEADERS:Header-Name@
@RECVHEADERS_RAW@
HTTP response headers.
Some special characters such as \t, \r, \n are replaced by their string representation "\t", "\r", "\n"
NOTE: @RECVHEADERS:Header-Name@ extracts the value of a specific header.
NOTE: @RECVHEADERS_RAW@ returns the same value with no characters replaced.
@RECVDATA@
@RECVDATA_RAW@
A sample of the received data.
Non-printable characters are replaced with "."
NOTE: Can retrieve remote content downloaded to MEMORY.
NOTE: This value is truncated to NSIS' maximum string length (1KB, 4KB, 8KB, depending on the NSIS build).
NOTE: @RECVDATA_RAW@ returns the same value with no characters replaced.
@TAG@ Transfer tag, empty by default.
Multiple transfers can be tagged with the same tag.
@ERROR@ The final transfer status.
Values: "OK" for success, '0x2a "Callback aborted"', etc.
@ERRORCODE@ The numeric status code.
It can be either an HTTP status code, a libcurl error code, or a Win32 error code.
Value: (Integer) 200, 206, 404, 0x2a, etc.
@ERRORTYPE@ Returns @ERRORCODE@ error type.
Value: (String) "win32", "curl" or "http"
@CANCELLED@ Indicates whether the transfer was cancelled by the user.
Value: (Boolean) 0 or 1.
Global keywords Comments
@PLUGINNAME@ "NScurl"
@PLUGINVERSION@ Plugin's version.
Returns the "FileVersion" value from Version Information resource block.
@PLUGINAUTHOR@ Author's name.
Returns the "CompanyName" value from Version Information resource block.
@PLUGINWEB@ Project's website.
Returns the "LegalTrademarks" value from Version Information resource block.
@CURLVERSION@ libcurl version (e.g. "7.68.0")
@CURLSSLVERSION@ SSL backend version (e.g. "7.68.0")
@CURLPROTOCOLS@ libcurl built-in protocols (e.g. "http https")
@CURLFEATURES@ libcurl built-in features (e.g. "SSL NTLM Debug AsynchDNS Largefile TLS-SRP UnixSockets")
@USERAGENT@ The default user agent (e.g. "nscurl/1.2020.3.1")
@TOTALCOUNT@ The number of HTTP requests in the queue.
Includes all "Waiting", "Running" and "Complete" requests.
@TOTALWAITING@ The number of "Waiting" requests in the queue.
@TOTALRUNNING@ The number of "Running" requests in the queue.
@TOTALCOMPLETE@ The number of "Complete" requests in the queue.
@TOTALACTIVE@ The number of "Waiting" + "Running" requests in the queue.
@TOTALSTARTED@ The number of "Running" + "Completed" requests in the queue.
@TOTALERRORS@ The number of failed requests in the queue.
@TOTALSPEED@
@TOTALSPEED_B@
The aggregated speed of "Running" transfers (e.g. "120 KB/s", "1.2 MB/s", etc.)
NOTE: @TOTALSPEED_B@ returns the same value in bytes/s.
@TOTALSIZE@
@TOTALSIZE_B@
The aggregated amount of Downloaded + Uploaded data (e.g. "100 MB", "5 GB", etc.)
NOTE: @TOTALSIZE_B@ returns the same value in bytes.
@TOTALSIZEUP@
@TOTALSIZEUP_B@
The aggregated amount of Uploaded data (e.g. "100 MB", "5 GB", etc.)
NOTE: @TOTALSIZEUP_B@ returns the same value in bytes.
@TOTALSIZEDOWN@
@TOTALSIZEDOWN_B@
The aggregated amount of Downloaded data (e.g. "100 MB", "5 GB", etc.)
NOTE: @TOTALSIZEDOWN_B@ returns the same value in bytes.
@THREADS@ Current number of worker threads.
@MAXTHREADS@ Maximum number of worker threads.


NScurl::wait [/ID id] [/TAG tag] <params> /END

Description:

Wait synchronously for background transfer(s) to complete.
Depending on parameters visual progress may or may not be displayed.
You can either wait for a specific transfer, or wait for multiple transfers in one go.

Return value:

None

Example:

# Start a background transfer
NScurl::http GET "${URL}" "${FILE}" /BACKGROUND /END
Pop $0 ; Transfer ID

# ...do some useful stuff...

# Wait...
NScurl::wait /ID $0 /CANCEL /END

Parameters:

/ID id Wait for a specific transfer.
The transfer ID is returned by NScurl::http.
/TAG tag Wait for multiple transfers tagged with the same tag.
See NScurl::http
/PAGE See NScurl::http
/POPUP See NScurl::http
/SILENT See NScurl::http
/CANCEL See NScurl::http
/TITLEWND hwnd
/TEXTWND hwnd
/PROGRESSWND hwnd
/CANCELWND hwnd
See NScurl::http
/STRING TITLE string
/STRING TITLE_NOSIZE string
/STRING TITLE_MULTI string
/STRING TEXT string
/STRING TEXT_NOSIZE string
/STRING TEXT_MULTI string
See NScurl::http
/END Must always conclude the list of parameters.
NOTE: This parameter is mandatory.


NScurl::enumerate [/TAG tag] [/STATUS status] /END

Description:

Enumerate HTTP transfers from the internal transfer queue.

Return value:

Transfer ID's are pushed one by one to the stack.
An empty string ("") is pushed to the stack to mark the end of the enumeration.

Examples:

NScurl::enumerate /END
_enum_loop:

Pop $0
StrCmp $0 "" _enum_end
DetailPrint 'TODO: Transfer ID $0'
Goto _enum_loop

_enum_end:

Parameters:

/TAG tag Enumerate transfers tagged with the same tag.
/STATUS Waiting|Running|Complete Enumerate transfer with specific status:
  • Waiting: transfers that are still waiting in the queue
  • Running: transfers currently in progress.
  • Complete: complete/aborted/failed transfers.

    NOTE: Multiple /STATUS parameters are accepted.
  • /END Must always conclude the list of parameters.
    NOTE: This parameter is mandatory.


    NScurl::cancel [/ID id] [/TAG tag] [/REMOVE]

    Description:

    Cancel (background) transfers and optionally remove them from the queue.

    Return value:

    None.

    Parameters:

    /ID id Cancel a specific transfer.
    The transfer ID is returned by NScurl::http /BACKGROUND
    /TAG tag Cancel multiple transfers tagged with the same tag.
    See NScurl::http /TAG
    /REMOVE In addition to cancelling, the transfer(s) are also permanently removed from the queue.
    Further attempts to NScurl::query them will fail.


    NScurl::escape <string>
    NScurl::unescape <string>

    Description:

    Un/Escape URL strings (converts all illegal URLs characters to/from their %XX format).

    Return value:

    Un/Escaped string is pushed to the stack.

    Examples:

    NScurl::escape "aaa bbb ccc=ddd&eee"
    Pop $0 ; Returns "aaa%20bbb%20ccc%3Dddd%26eee"

    NScurl::unescape $0
    Pop $0 ; Returns the original string


    NScurl::md5 [-string|file|memory] <data>
    NScurl::sha1 [-string|file|memory] <data>
    NScurl::sha256 [-string|file|memory] <data>

    Description:

    Compute MD5 / SHA1 / SHA256 hashes.
    The data can be read either from a file or directly from memory.
    NOTE: See /DATA to learn how data is interpreted.

    Return value:

    The hash (formatted as hex string) is pushed to the stack.

    Examples:

    NScurl::md5 "Hash this string"
    Pop $0 ; e.g. "376630459092d7682c2a2e745d74aa6b"

    NScurl::md5 $EXEPATH
    Pop $0 ; e.g. "93a52d04f7b56bc267d45bd95c6de49b"

    NScurl::sha1 -file $EXEPATH
    Pop $0 ; e.g. "faff487654d4dfa1deb5e5d462c8cf51b00a4acd"

    NScurl::sha1 -string $EXEPATH /* Hash the file path as regular string */
    Pop $0

    NScurl::sha256 $EXEPATH
    Pop $0 ; e.g. "e6fababe9530b1d5c4395ce0a1379c201ebb017997e4671a442a8410d1e2e6ac"