/home/crealab/cntxt.brainware.com.co/curlssl.tar
usr/share/man/man1/curl.1 0000644 00000401407 15116223777 0011136 0 ustar 00 .\" **************************************************************************
.\" * _ _ ____ _
.\" * Project ___| | | | _ \| |
.\" * / __| | | | |_) | |
.\" * | (__| |_| | _ <| |___
.\" * \___|\___/|_| \_\_____|
.\" *
.\" * Copyright (C) 1998 - 2018, Daniel Stenberg, <daniel@haxx.se>, et al.
.\" *
.\" * This software is licensed as described in the file COPYING, which
.\" * you should have received as part of this distribution. The terms
.\" * are also available at https://curl.haxx.se/docs/copyright.html.
.\" *
.\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
.\" * copies of the Software, and permit persons to whom the Software is
.\" * furnished to do so, under the terms of the COPYING file.
.\" *
.\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
.\" * KIND, either express or implied.
.\" *
.\" **************************************************************************
.\"
.\" DO NOT EDIT. Generated by the curl project gen.pl man page generator.
.\"
.TH curl 1 "November 16, 2016" "Curl 7.64.0" "Curl Manual"
.SH NAME
curl \- transfer a URL
.SH SYNOPSIS
.B curl [options / URLs]
.SH DESCRIPTION
.B curl
is a tool to transfer data from or to a server, using one of the supported
protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP,
LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET
and TFTP). The command is designed to work without user interaction.
curl offers a busload of useful tricks like proxy support, user
authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer
resume, Metalink, and more. As you will see below, the number of features will
make your head spin!
curl is powered by libcurl for all transfer-related features. See
\fIlibcurl(3)\fP for details.
.SH URL
The URL syntax is protocol-dependent. You'll find a detailed description in
RFC 3986.
You can specify multiple URLs or parts of URLs by writing part sets within
braces as in:
http://site.{one,two,three}.com
or you can get sequences of alphanumeric series by using [] as in:
ftp://ftp.example.com/file[1-100].txt
ftp://ftp.example.com/file[001-100].txt (with leading zeros)
ftp://ftp.example.com/file[a-z].txt
Nested sequences are not supported, but you can use several ones next to each
other:
http://example.com/archive[1996-1999]/vol[1-4]/part{a,b,c}.html
You can specify any amount of URLs on the command line. They will be fetched
in a sequential manner in the specified order. You can specify command line
options and URLs mixed and in any order on the command line.
You can specify a step counter for the ranges to get every Nth number or
letter:
http://example.com/file[1-100:10].txt
http://example.com/file[a-z:2].txt
When using [] or {} sequences when invoked from a command line prompt, you
probably have to put the full URL within double quotes to avoid the shell from
interfering with it. This also goes for other characters treated special, like
for example '&', '?' and '*'.
Provide the IPv6 zone index in the URL with an escaped percentage sign and the
interface name. Like in
http://[fe80::3%25eth0]/
If you specify URL without protocol:// prefix, curl will attempt to guess what
protocol you might want. It will then default to HTTP but try other protocols
based on often-used host name prefixes. For example, for host names starting
with "ftp." curl will assume you want to speak FTP.
curl will do its best to use what you pass to it as a URL. It is not trying to
validate it as a syntactically correct URL by any means but is instead
\fBvery\fP liberal with what it accepts.
curl will attempt to re-use connections for multiple file transfers, so that
getting many files from the same server will not do multiple connects /
handshakes. This improves speed. Of course this is only done on files
specified on a single command line and cannot be used between separate curl
invokes.
.SH "PROGRESS METER"
curl normally displays a progress meter during operations, indicating the
amount of transferred data, transfer speeds and estimated time left, etc. The
progress meter displays number of bytes and the speeds are in bytes per
second. The suffixes (k, M, G, T, P) are 1024 based. For example 1k is 1024
bytes. 1M is 1048576 bytes.
curl displays this data to the terminal by default, so if you invoke curl to
do an operation and it is about to write data to the terminal, it
\fIdisables\fP the progress meter as otherwise it would mess up the output
mixing progress meter and response data.
If you want a progress meter for HTTP POST or PUT requests, you need to
redirect the response output to a file, using shell redirect (>), \fI-o, --output\fP or
similar.
It is not the same case for FTP upload as that operation does not spit out
any response data to the terminal.
If you prefer a progress "bar" instead of the regular meter, \fI-#, --progress-bar\fP is
your friend. You can also disable the progress meter completely with the
\fI-s, --silent\fP option.
.SH OPTIONS
Options start with one or two dashes. Many of the options require an
additional value next to them.
The short "single-dash" form of the options, -d for example, may be used with
or without a space between it and its value, although a space is a recommended
separator. The long "double-dash" form, \fI-d, --data\fP for example, requires a space
between it and its value.
Short version options that don't need any additional values can be used
immediately next to each other, like for example you can specify all the
options -O, -L and -v at once as -OLv.
In general, all boolean options are enabled with --\fBoption\fP and yet again
disabled with --\fBno-\fPoption. That is, you use the exact same option name
but prefix it with "no-". However, in this list we mostly only list and show
the --option version of them. (This concept with --no options was added in
7.19.0. Previously most options were toggled on/off on repeated use of the
same command line option.)
.IP "--abstract-unix-socket <path>"
(HTTP) Connect through an abstract Unix domain socket, instead of using the network.
Note: netstat shows the path of an abstract socket prefixed with '@', however
the <path> argument should not have this leading character.
Added in 7.53.0.
.IP "--anyauth"
(HTTP) Tells curl to figure out authentication method by itself, and use the most
secure one the remote site claims to support. This is done by first doing a
request and checking the response-headers, thus possibly inducing an extra
network round-trip. This is used instead of setting a specific authentication
method, which you can do with \fI--basic\fP, \fI--digest\fP, \fI--ntlm\fP, and \fI--negotiate\fP.
Using \fI--anyauth\fP is not recommended if you do uploads from stdin, since it may
require data to be sent twice and then the client must be able to rewind. If
the need should arise when uploading from stdin, the upload operation will
fail.
Used together with \fI-u, --user\fP.
See also \fI--proxy-anyauth\fP and \fI--basic\fP and \fI--digest\fP.
.IP "-a, --append"
(FTP SFTP) When used in an upload, this makes curl append to the target file instead of
overwriting it. If the remote file doesn't exist, it will be created. Note
that this flag is ignored by some SFTP servers (including OpenSSH).
.IP "--basic"
(HTTP) Tells curl to use HTTP Basic authentication with the remote host. This is the
default and this option is usually pointless, unless you use it to override a
previously set option that sets a different authentication method (such as
\fI--ntlm\fP, \fI--digest\fP, or \fI--negotiate\fP).
Used together with \fI-u, --user\fP.
See also \fI--proxy-basic\fP.
.IP "--cacert <file>"
(TLS) Tells curl to use the specified certificate file to verify the peer. The file
may contain multiple CA certificates. The certificate(s) must be in PEM
format. Normally curl is built to use a default file for this, so this option
is typically used to alter that default file.
curl recognizes the environment variable named 'CURL_CA_BUNDLE' if it is
set, and uses the given path as a path to a CA cert bundle. This option
overrides that variable.
The windows version of curl will automatically look for a CA certs file named
\'curl-ca-bundle.crt\', either in the same directory as curl.exe, or in the
Current Working Directory, or in any folder along your PATH.
If curl is built against the NSS SSL library, the NSS PEM PKCS#11 module
(libnsspem.so) needs to be available for this option to work properly.
(iOS and macOS only) If curl is built against Secure Transport, then this
option is supported for backward compatibility with other SSL engines, but it
should not be set. If the option is not set, then curl will use the
certificates in the system and user Keychain to verify the peer, which is the
preferred method of verifying the peer's certificate chain.
(Schannel only) This option is supported for Schannel in Windows 7 or later with
libcurl 7.60 or later. This option is supported for backward compatibility
with other SSL engines; instead it is recommended to use Windows' store of
root certificates (the default for Schannel).
If this option is used several times, the last one will be used.
.IP "--capath <dir>"
(TLS) Tells curl to use the specified certificate directory to verify the
peer. Multiple paths can be provided by separating them with ":" (e.g.
\&"path1:path2:path3"). The certificates must be in PEM format, and if curl is
built against OpenSSL, the directory must have been processed using the
c_rehash utility supplied with OpenSSL. Using \fI--capath\fP can allow
OpenSSL-powered curl to make SSL-connections much more efficiently than using
\fI--cacert\fP if the --cacert file contains many CA certificates.
If this option is set, the default capath value will be ignored, and if it is
used several times, the last one will be used.
.IP "--cert-status"
(TLS) Tells curl to verify the status of the server certificate by using the
Certificate Status Request (aka. OCSP stapling) TLS extension.
If this option is enabled and the server sends an invalid (e.g. expired)
response, if the response suggests that the server certificate has been revoked,
or no response at all is received, the verification fails.
This is currently only implemented in the OpenSSL, GnuTLS and NSS backends.
Added in 7.41.0.
.IP "--cert-type <type>"
(TLS) Tells curl what type the provided client certificate is using. PEM, DER, ENG
and P12 are recognized types. If not specified, PEM is assumed.
If this option is used several times, the last one will be used.
See also \fI-E, --cert\fP and \fI--key\fP and \fI--key-type\fP.
.IP "-E, --cert <certificate[:password]>"
(TLS) Tells curl to use the specified client certificate file when getting a file
with HTTPS, FTPS or another SSL-based protocol. The certificate must be in
PKCS#12 format if using Secure Transport, or PEM format if using any other
engine. If the optional password isn't specified, it will be queried for on
the terminal. Note that this option assumes a \&"certificate" file that is the
private key and the client certificate concatenated! See \fI-E, --cert\fP and \fI--key\fP to
specify them independently.
If curl is built against the NSS SSL library then this option can tell
curl the nickname of the certificate to use within the NSS database defined
by the environment variable SSL_DIR (or by default /etc/pki/nssdb). If the
NSS PEM PKCS#11 module (libnsspem.so) is available then PEM files may be
loaded. If you want to use a file from the current directory, please precede
it with "./" prefix, in order to avoid confusion with a nickname. If the
nickname contains ":", it needs to be preceded by "\\" so that it is not
recognized as password delimiter. If the nickname contains "\\", it needs to
be escaped as "\\\\" so that it is not recognized as an escape character.
If curl is built against OpenSSL library, and the engine pkcs11 is available,
then a PKCS#11 URI (RFC 7512) can be used to specify a certificate located in
a PKCS#11 device. A string beginning with "pkcs11:" will be interpreted as a
PKCS#11 URI. If a PKCS#11 URI is provided, then the \fI--engine\fP option will be set
as "pkcs11" if none was provided and the \fI--cert-type\fP option will be set as
"ENG" if none was provided.
(iOS and macOS only) If curl is built against Secure Transport, then the
certificate string can either be the name of a certificate/private key in the
system or user keychain, or the path to a PKCS#12-encoded certificate and
private key. If you want to use a file from the current directory, please
precede it with "./" prefix, in order to avoid confusion with a nickname.
(Schannel only) Client certificates must be specified by a path
expression to a certificate store. (Loading PFX is not supported; you can
import it to a store first). You can use
"<store location>\\<store name>\\<thumbprint>" to refer to a certificate
in the system certificates store, for example,
"CurrentUser\\MY\\934a7ac6f8a5d579285a74fa61e19f23ddfe8d7a". Thumbprint is
usually a SHA-1 hex string which you can see in certificate details. Following
store locations are supported: CurrentUser, LocalMachine, CurrentService,
Services, CurrentUserGroupPolicy, LocalMachineGroupPolicy,
LocalMachineEnterprise.
If this option is used several times, the last one will be used.
See also \fI--cert-type\fP and \fI--key\fP and \fI--key-type\fP.
.IP "--ciphers <list of ciphers>"
(TLS) Specifies which ciphers to use in the connection. The list of ciphers must
specify valid ciphers. Read up on SSL cipher list details on this URL:
https://curl.haxx.se/docs/ssl-ciphers.html
If this option is used several times, the last one will be used.
.IP "--compressed-ssh"
(SCP SFTP) Enables built-in SSH compression.
This is a request, not an order; the server may or may not do it.
Added in 7.56.0.
.IP "--compressed"
(HTTP) Request a compressed response using one of the algorithms curl supports, and
save the uncompressed document. If this option is used and the server sends
an unsupported encoding, curl will report an error.
.IP "-K, --config <file>"
Specify a text file to read curl arguments from. The command line arguments
found in the text file will be used as if they were provided on the command
line.
Options and their parameters must be specified on the same line in the file,
separated by whitespace, colon, or the equals sign. Long option names can
optionally be given in the config file without the initial double dashes and
if so, the colon or equals characters can be used as separators. If the option
is specified with one or two dashes, there can be no colon or equals character
between the option and its parameter.
If the parameter is to contain whitespace, the parameter must be enclosed
within quotes. Within double quotes, the following escape sequences are
available: \\\\, \\", \\t, \\n, \\r and \\v. A backslash preceding any other
letter is ignored. If the first column of a config line is a '#' character,
the rest of the line will be treated as a comment. Only write one option per
physical line in the config file.
Specify the filename to \fI-K, --config\fP as '-' to make curl read the file from stdin.
Note that to be able to specify a URL in the config file, you need to specify
it using the \fI--url\fP option, and not by simply writing the URL on its own
line. So, it could look similar to this:
url = "https://curl.haxx.se/docs/"
When curl is invoked, it (unless \fI-q, --disable\fP is used) checks for a default
config file and uses it if found. The default config file is checked for in
the following places in this order:
1) curl tries to find the "home dir": It first checks for the CURL_HOME and
then the HOME environment variables. Failing that, it uses getpwuid() on
Unix-like systems (which returns the home dir given the current user in your
system). On Windows, it then checks for the APPDATA variable, or as a last
resort the '%USERPROFILE%\\Application Data'.
2) On windows, if there is no _curlrc file in the home dir, it checks for one
in the same dir the curl executable is placed. On Unix-like systems, it will
simply try to load .curlrc from the determined home dir.
.nf
# --- Example file ---
# this is a comment
url = "example.com"
output = "curlhere.html"
user-agent = "superagent/1.0"
# and fetch another URL too
url = "example.com/docs/manpage.html"
-O
referer = "http://nowhereatall.example.com/"
# --- End of example file ---
.fi
This option can be used multiple times to load multiple config files.
.IP "--connect-timeout <seconds>"
Maximum time in seconds that you allow curl's connection to take. This only
limits the connection phase, so if curl connects within the given period it
will continue - if not it will exit. Since version 7.32.0, this option
accepts decimal values.
If this option is used several times, the last one will be used.
See also \fI-m, --max-time\fP.
.IP "--connect-to <HOST1:PORT1:HOST2:PORT2>"
For a request to the given HOST1:PORT1 pair, connect to HOST2:PORT2 instead.
This option is suitable to direct requests at a specific server, e.g. at a
specific cluster node in a cluster of servers. This option is only used to
establish the network connection. It does NOT affect the hostname/port that is
used for TLS/SSL (e.g. SNI, certificate verification) or for the application
protocols. "HOST1" and "PORT1" may be the empty string, meaning "any
host/port". "HOST2" and "PORT2" may also be the empty string, meaning "use the
request's original host/port".
A "host" specified to this option is compared as a string, so it needs to
match the name used in request URL. It can be either numerical such as
"127.0.0.1" or the full host name such as "example.org".
This option can be used many times to add many connect rules.
See also \fI--resolve\fP and \fI-H, --header\fP. Added in 7.49.0.
.IP "-C, --continue-at <offset>"
Continue/Resume a previous file transfer at the given offset. The given offset
is the exact number of bytes that will be skipped, counting from the beginning
of the source file before it is transferred to the destination. If used with
uploads, the FTP server command SIZE will not be used by curl.
Use "-C -" to tell curl to automatically find out where/how to resume the
transfer. It then uses the given output/input files to figure that out.
If this option is used several times, the last one will be used.
See also \fI-r, --range\fP.
.IP "-c, --cookie-jar <filename>"
(HTTP) Specify to which file you want curl to write all cookies after a completed
operation. Curl writes all cookies from its in-memory cookie storage to the
given file at the end of operations. If no cookies are known, no data will be
written. The file will be written using the Netscape cookie file format. If
you set the file name to a single dash, "-", the cookies will be written to
stdout.
This command line option will activate the cookie engine that makes curl
record and use cookies. Another way to activate it is to use the \fI-b, --cookie\fP
option.
If the cookie jar can't be created or written to, the whole curl operation
won't fail or even report an error clearly. Using \fI-v, --verbose\fP will get a warning
displayed, but that is the only visible feedback you get about this possibly
lethal situation.
If this option is used several times, the last specified file name will be
used.
.IP "-b, --cookie <data>"
(HTTP) Pass the data to the HTTP server in the Cookie header. It is supposedly
the data previously received from the server in a "Set-Cookie:" line. The
data should be in the format "NAME1=VALUE1; NAME2=VALUE2".
If no '=' symbol is used in the argument, it is instead treated as a filename
to read previously stored cookie from. This option also activates the cookie
engine which will make curl record incoming cookies, which may be handy if
you're using this in combination with the \fI-L, --location\fP option or do multiple URL
transfers on the same invoke. If the file name is exactly a minus ("-"), curl
will instead the contents from stdin.
The file format of the file to read cookies from should be plain HTTP headers
(Set-Cookie style) or the Netscape/Mozilla cookie file format.
The file specified with \fI-b, --cookie\fP is only used as input. No cookies will be
written to the file. To store cookies, use the \fI-c, --cookie-jar\fP option.
Exercise caution if you are using this option and multiple transfers may
occur. If you use the NAME1=VALUE1; format, or in a file use the Set-Cookie
format and don't specify a domain, then the cookie is sent for any domain
(even after redirects are followed) and cannot be modified by a server-set
cookie. If the cookie engine is enabled and a server sets a cookie of the same
name then both will be sent on a future transfer to that server, likely not
what you intended. To address these issues set a domain in Set-Cookie (doing
that will include sub domains) or use the Netscape format.
If this option is used several times, the last one will be used.
Users very often want to both read cookies from a file and write updated
cookies back to a file, so using both \fI-b, --cookie\fP and \fI-c, --cookie-jar\fP in the same
command line is common.
.IP "--create-dirs"
When used in conjunction with the \fI-o, --output\fP option, curl will create the
necessary local directory hierarchy as needed. This option creates the dirs
mentioned with the \fI-o, --output\fP option, nothing else. If the --output file name
uses no dir or if the dirs it mentions already exist, no dir will be created.
To create remote directories when using FTP or SFTP, try \fI--ftp-create-dirs\fP.
.IP "--crlf"
(FTP SMTP) Convert LF to CRLF in upload. Useful for MVS (OS/390).
(SMTP added in 7.40.0)
.IP "--crlfile <file>"
(TLS) Provide a file using PEM format with a Certificate Revocation List that may
specify peer certificates that are to be considered revoked.
If this option is used several times, the last one will be used.
Added in 7.19.7.
.IP "--data-ascii <data>"
(HTTP) This is just an alias for \fI-d, --data\fP.
.IP "--data-binary <data>"
(HTTP) This posts data exactly as specified with no extra processing whatsoever.
If you start the data with the letter @, the rest should be a filename. Data
is posted in a similar manner as \fI-d, --data\fP does, except that newlines and
carriage returns are preserved and conversions are never done.
Like \fI-d, --data\fP the default content-type sent to the server is
application/x-www-form-urlencoded. If you want the data to be treated as
arbitrary binary data by the server then set the content-type to octet-stream:
-H "Content-Type: application/octet-stream".
If this option is used several times, the ones following the first will append
data as described in \fI-d, --data\fP.
.IP "--data-raw <data>"
(HTTP) This posts data similarly to \fI-d, --data\fP but without the special
interpretation of the @ character.
See also \fI-d, --data\fP. Added in 7.43.0.
.IP "--data-urlencode <data>"
(HTTP) This posts data, similar to the other \fI-d, --data\fP options with the exception
that this performs URL-encoding.
To be CGI-compliant, the <data> part should begin with a \fIname\fP followed
by a separator and a content specification. The <data> part can be passed to
curl using one of the following syntaxes:
.RS
.IP "content"
This will make curl URL-encode the content and pass that on. Just be careful
so that the content doesn't contain any = or @ symbols, as that will then make
the syntax match one of the other cases below!
.IP "=content"
This will make curl URL-encode the content and pass that on. The preceding =
symbol is not included in the data.
.IP "name=content"
This will make curl URL-encode the content part and pass that on. Note that
the name part is expected to be URL-encoded already.
.IP "@filename"
This will make curl load data from the given file (including any newlines),
URL-encode that data and pass it on in the POST.
.IP "name@filename"
This will make curl load data from the given file (including any newlines),
URL-encode that data and pass it on in the POST. The name part gets an equal
sign appended, resulting in \fIname=urlencoded-file-content\fP. Note that the
name is expected to be URL-encoded already.
.RE
See also \fI-d, --data\fP and \fI--data-raw\fP. Added in 7.18.0.
.IP "-d, --data <data>"
(HTTP) Sends the specified data in a POST request to the HTTP server, in the same way
that a browser does when a user has filled in an HTML form and presses the
submit button. This will cause curl to pass the data to the server using the
content-type application/x-www-form-urlencoded. Compare to \fI-F, --form\fP.
\fI--data-raw\fP is almost the same but does not have a special interpretation of
the @ character. To post data purely binary, you should instead use the
\fI--data-binary\fP option. To URL-encode the value of a form field you may use
\fI--data-urlencode\fP.
If any of these options is used more than once on the same command line, the
data pieces specified will be merged together with a separating
&-symbol. Thus, using '-d name=daniel -d skill=lousy' would generate a post
chunk that looks like \&'name=daniel&skill=lousy'.
If you start the data with the letter @, the rest should be a file name to
read the data from, or - if you want curl to read the data from
stdin. Multiple files can also be specified. Posting data from a file named
'foobar' would thus be done with \fI-d, --data\fP @foobar. When --data is told to read
from a file like that, carriage returns and newlines will be stripped out. If
you don't want the @ character to have a special interpretation use \fI--data-raw\fP
instead.
See also \fI--data-binary\fP and \fI--data-urlencode\fP and \fI--data-raw\fP. This option overrides \fI-F, --form\fP and \fI-I, --head\fP and \fI-T, --upload-file\fP.
.IP "--delegation <LEVEL>"
(GSS/kerberos) Set LEVEL to tell the server what it is allowed to delegate when it
comes to user credentials.
.RS
.IP "none"
Don't allow any delegation.
.IP "policy"
Delegates if and only if the OK-AS-DELEGATE flag is set in the Kerberos
service ticket, which is a matter of realm policy.
.IP "always"
Unconditionally allow the server to delegate.
.RE
.IP "--digest"
(HTTP) Enables HTTP Digest authentication. This is an authentication scheme that
prevents the password from being sent over the wire in clear text. Use this in
combination with the normal \fI-u, --user\fP option to set user name and password.
If this option is used several times, only the first one is used.
See also \fI-u, --user\fP and \fI--proxy-digest\fP and \fI--anyauth\fP. This option overrides \fI--basic\fP and \fI--ntlm\fP and \fI--negotiate\fP.
.IP "--disable-eprt"
(FTP) Tell curl to disable the use of the EPRT and LPRT commands when doing active
FTP transfers. Curl will normally always first attempt to use EPRT, then LPRT
before using PORT, but with this option, it will use PORT right away. EPRT and
LPRT are extensions to the original FTP protocol, and may not work on all
servers, but they enable more functionality in a better way than the
traditional PORT command.
--eprt can be used to explicitly enable EPRT again and --no-eprt is an alias
for \fI--disable-eprt\fP.
If the server is accessed using IPv6, this option will have no effect as EPRT
is necessary then.
Disabling EPRT only changes the active behavior. If you want to switch to
passive mode you need to not use \fI-P, --ftp-port\fP or force it with \fI--ftp-pasv\fP.
.IP "--disable-epsv"
(FTP) (FTP) Tell curl to disable the use of the EPSV command when doing passive FTP
transfers. Curl will normally always first attempt to use EPSV before PASV,
but with this option, it will not try using EPSV.
--epsv can be used to explicitly enable EPSV again and --no-epsv is an alias
for \fI--disable-epsv\fP.
If the server is an IPv6 host, this option will have no effect as EPSV is
necessary then.
Disabling EPSV only changes the passive behavior. If you want to switch to
active mode you need to use \fI-P, --ftp-port\fP.
.IP "-q, --disable"
If used as the first parameter on the command line, the \fIcurlrc\fP config
file will not be read and used. See the \fI-K, --config\fP for details on the default
config file search path.
.IP "--disallow-username-in-url"
(HTTP) This tells curl to exit if passed a url containing a username.
See also \fI--proto\fP. Added in 7.61.0.
.IP "--dns-interface <interface>"
(DNS) Tell curl to send outgoing DNS requests through <interface>. This option is a
counterpart to \fI--interface\fP (which does not affect DNS). The supplied string
must be an interface name (not an address).
See also \fI--dns-ipv4-addr\fP and \fI--dns-ipv6-addr\fP. \fI--dns-interface\fP requires that the underlying libcurl was built to support c-ares. Added in 7.33.0.
.IP "--dns-ipv4-addr <address>"
(DNS) Tell curl to bind to <ip-address> when making IPv4 DNS requests, so that
the DNS requests originate from this address. The argument should be a
single IPv4 address.
See also \fI--dns-interface\fP and \fI--dns-ipv6-addr\fP. \fI--dns-ipv4-addr\fP requires that the underlying libcurl was built to support c-ares. Added in 7.33.0.
.IP "--dns-ipv6-addr <address>"
(DNS) Tell curl to bind to <ip-address> when making IPv6 DNS requests, so that
the DNS requests originate from this address. The argument should be a
single IPv6 address.
See also \fI--dns-interface\fP and \fI--dns-ipv4-addr\fP. \fI--dns-ipv6-addr\fP requires that the underlying libcurl was built to support c-ares. Added in 7.33.0.
.IP "--dns-servers <addresses>"
Set the list of DNS servers to be used instead of the system default.
The list of IP addresses should be separated with commas. Port numbers
may also optionally be given as \fI:<port-number>\fP after each IP
address.
\fI--dns-servers\fP requires that the underlying libcurl was built to support c-ares. Added in 7.33.0.
.IP "--doh-url <URL>"
(all) Specifies which DNS-over-HTTPS (DOH) server to use to resolve hostnames,
instead of using the default name resolver mechanism. The URL must be HTTPS.
If this option is used several times, the last one will be used.
.IP "-D, --dump-header <filename>"
(HTTP FTP) Write the received protocol headers to the specified file.
This option is handy to use when you want to store the headers that an HTTP
site sends to you. Cookies from the headers could then be read in a second
curl invocation by using the \fI-b, --cookie\fP option! The \fI-c, --cookie-jar\fP option is a
better way to store cookies.
When used in FTP, the FTP server response lines are considered being "headers"
and thus are saved there.
If this option is used several times, the last one will be used.
See also \fI-o, --output\fP.
.IP "--egd-file <file>"
(TLS) Specify the path name to the Entropy Gathering Daemon socket. The socket is
used to seed the random engine for SSL connections.
See also \fI--random-file\fP.
.IP "--engine <name>"
(TLS) Select the OpenSSL crypto engine to use for cipher operations. Use \fI--engine\fP
list to print a list of build-time supported engines. Note that not all (or
none) of the engines may be available at run-time.
.IP "--expect100-timeout <seconds>"
(HTTP) Maximum time in seconds that you allow curl to wait for a 100-continue
response when curl emits an Expects: 100-continue header in its request. By
default curl will wait one second. This option accepts decimal values! When
curl stops waiting, it will continue as if the response has been received.
See also \fI--connect-timeout\fP. Added in 7.47.0.
.IP "--fail-early"
Fail and exit on the first detected transfer error.
When curl is used to do multiple transfers on the command line, it will
attempt to operate on each given URL, one by one. By default, it will ignore
errors if there are more URLs given and the last URL's success will determine
the error code curl returns. So early failures will be "hidden" by subsequent
successful transfers.
Using this option, curl will instead return an error on the first transfer
that fails, independent of the amount of URLs that are given on the command
line. This way, no transfer failures go undetected by scripts and similar.
This option is global and does not need to be specified for each use of \fI-:, --next\fP.
This option does not imply \fI-f, --fail\fP, which causes transfers to fail due to the
server's HTTP status code. You can combine the two options, however note \fI-f, --fail\fP
is not global and is therefore contained by \fI-:, --next\fP.
Added in 7.52.0.
.IP "-f, --fail"
(HTTP) Fail silently (no output at all) on server errors. This is mostly done to
better enable scripts etc to better deal with failed attempts. In normal cases
when an HTTP server fails to deliver a document, it returns an HTML document
stating so (which often also describes why and more). This flag will prevent
curl from outputting that and return error 22.
This method is not fail-safe and there are occasions where non-successful
response codes will slip through, especially when authentication is involved
(response codes 401 and 407).
.IP "--false-start"
(TLS) Tells curl to use false start during the TLS handshake. False start is a mode
where a TLS client will start sending application data before verifying the
server's Finished message, thus saving a round trip when performing a full
handshake.
This is currently only implemented in the NSS and Secure Transport (on iOS 7.0
or later, or OS X 10.9 or later) backends.
Added in 7.42.0.
.IP "--form-string <name=string>"
(HTTP SMTP IMAP) Similar to \fI-F, --form\fP except that the value string for the named parameter is used
literally. Leading \&'@' and \&'<' characters, and the \&';type=' string in
the value have no special meaning. Use this in preference to \fI-F, --form\fP if
there's any possibility that the string value may accidentally trigger the
\&'@' or \&'<' features of \fI-F, --form\fP.
See also \fI-F, --form\fP.
.IP "-F, --form <name=content>"
(HTTP SMTP IMAP) For HTTP protocol family, this lets curl emulate a filled-in form in which a
user has pressed the submit button. This causes curl to POST data using the
Content-Type multipart/form-data according to RFC 2388.
For SMTP and IMAP protocols, this is the mean to compose a multipart mail
message to transmit.
This enables uploading of binary files etc. To force the 'content' part to be
a file, prefix the file name with an @ sign. To just get the content part from
a file, prefix the file name with the symbol <. The difference between @ and <
is then that @ makes a file get attached in the post as a file upload, while
the < makes a text field and just get the contents for that text field from a
file.
Tell curl to read content from stdin instead of a file by using - as
filename. This goes for both @ and < constructs. When stdin is used, the
contents is buffered in memory first by curl to determine its size and allow a
possible resend. Defining a part's data from a named non-regular file (such
as a named pipe or similar) is unfortunately not subject to buffering and will
be effectively read at transmission time; since the full size is unknown
before the transfer starts, such data is sent as chunks by HTTP and rejected
by IMAP.
Example: send an image to an HTTP server, where \&'profile' is the name of the
form-field to which the file portrait.jpg will be the input:
curl -F profile=@portrait.jpg https://example.com/upload.cgi
Example: send a your name and shoe size in two text fields to the server:
curl -F name=John -F shoesize=11 https://example.com/
Example: send a your essay in a text field to the server. Send it as a plain
text field, but get the contents for it from a local file:
curl -F "story=<hugefile.txt" https://example.com/
You can also tell curl what Content-Type to use by using 'type=', in a manner
similar to:
curl -F "web=@index.html;type=text/html" example.com
or
curl -F "name=daniel;type=text/foo" example.com
You can also explicitly change the name field of a file upload part by setting
filename=, like this:
curl -F "file=@localfile;filename=nameinpost" example.com
If filename/path contains ',' or ';', it must be quoted by double-quotes like:
curl -F "file=@\\"localfile\\";filename=\\"nameinpost\\"" example.com
or
curl -F 'file=@"localfile";filename="nameinpost"' example.com
Note that if a filename/path is quoted by double-quotes, any double-quote
or backslash within the filename must be escaped by backslash.
Quoting must also be applied to non-file data if it contains semicolons,
leading/trailing spaces or leading double quotes:
curl -F 'colors="red; green; blue";type=text/x-myapp' example.com
You can add custom headers to the field by setting headers=, like
curl -F "submit=OK;headers=\\"X-submit-type: OK\\"" example.com
or
curl -F "submit=OK;headers=@headerfile" example.com
The headers= keyword may appear more that once and above notes about quoting
apply. When headers are read from a file, Empty lines and lines starting
with '#' are comments and ignored; each header can be folded by splitting
between two words and starting the continuation line with a space; embedded
carriage-returns and trailing spaces are stripped.
Here is an example of a header file contents:
# This file contain two headers.
.br
X-header-1: this is a header
# The following header is folded.
.br
X-header-2: this is
.br
another header
To support sending multipart mail messages, the syntax is extended as follows:
.br
- name can be omitted: the equal sign is the first character of the argument,
.br
- if data starts with '(', this signals to start a new multipart: it can be
followed by a content type specification.
.br
- a multipart can be terminated with a '=)' argument.
Example: the following command sends an SMTP mime e-mail consisting in an
inline part in two alternative formats: plain text and HTML. It attaches a
text file:
curl -F '=(;type=multipart/alternative' \\
.br
-F '=plain text message' \\
.br
-F '= <body>HTML message</body>;type=text/html' \\
.br
-F '=)' -F '=@textfile.txt' ... smtp://example.com
Data can be encoded for transfer using encoder=. Available encodings are
\fIbinary\fP and \fI8bit\fP that do nothing else than adding the corresponding
Content-Transfer-Encoding header, \fI7bit\fP that only rejects 8-bit characters
with a transfer error, \fIquoted-printable\fP and \fIbase64\fP that encodes
data according to the corresponding schemes, limiting lines length to
76 characters.
Example: send multipart mail with a quoted-printable text message and a
base64 attached file:
curl -F '=text message;encoder=quoted-printable' \\
.br
-F '=@localfile;encoder=base64' ... smtp://example.com
See further examples and details in the MANUAL.
This option can be used multiple times.
This option overrides \fI-d, --data\fP and \fI-I, --head\fP and \fI-T, --upload-file\fP.
.IP "--ftp-account <data>"
(FTP) When an FTP server asks for "account data" after user name and password has
been provided, this data is sent off using the ACCT command.
If this option is used several times, the last one will be used.
Added in 7.13.0.
.IP "--ftp-alternative-to-user <command>"
(FTP) If authenticating with the USER and PASS commands fails, send this command.
When connecting to Tumbleweed's Secure Transport server over FTPS using a
client certificate, using "SITE AUTH" will tell the server to retrieve the
username from the certificate.
Added in 7.15.5.
.IP "--ftp-create-dirs"
(FTP SFTP) When an FTP or SFTP URL/operation uses a path that doesn't currently exist on
the server, the standard behavior of curl is to fail. Using this option, curl
will instead attempt to create missing directories.
See also \fI--create-dirs\fP.
.IP "--ftp-method <method>"
(FTP) Control what method curl should use to reach a file on an FTP(S)
server. The method argument should be one of the following alternatives:
.RS
.IP multicwd
curl does a single CWD operation for each path part in the given URL. For deep
hierarchies this means very many commands. This is how RFC 1738 says it should
be done. This is the default but the slowest behavior.
.IP nocwd
curl does no CWD at all. curl will do SIZE, RETR, STOR etc and give a full
path to the server for all these commands. This is the fastest behavior.
.IP singlecwd
curl does one CWD with the full target directory and then operates on the file
\&"normally" (like in the multicwd case). This is somewhat more standards
compliant than 'nocwd' but without the full penalty of 'multicwd'.
.RE
Added in 7.15.1.
.IP "--ftp-pasv"
(FTP) Use passive mode for the data connection. Passive is the internal default
behavior, but using this option can be used to override a previous \fI-P, --ftp-port\fP
option.
If this option is used several times, only the first one is used. Undoing an
enforced passive really isn't doable but you must then instead enforce the
correct \fI-P, --ftp-port\fP again.
Passive mode means that curl will try the EPSV command first and then PASV,
unless \fI--disable-epsv\fP is used.
See also \fI--disable-epsv\fP. Added in 7.11.0.
.IP "-P, --ftp-port <address>"
(FTP) Reverses the default initiator/listener roles when connecting with FTP. This
option makes curl use active mode. curl then tells the server to connect back
to the client's specified address and port, while passive mode asks the server
to setup an IP address and port for it to connect to. <address> should be one
of:
.RS
.IP interface
e.g. "eth0" to specify which interface's IP address you want to use (Unix only)
.IP "IP address"
e.g. "192.168.10.1" to specify the exact IP address
.IP "host name"
e.g. "my.host.domain" to specify the machine
.IP "-"
make curl pick the same IP address that is already used for the control
connection
.RE
If this option is used several times, the last one will be used. Disable the
use of PORT with \fI--ftp-pasv\fP. Disable the attempt to use the EPRT command
instead of PORT by using \fI--disable-eprt\fP. EPRT is really PORT++.
Since 7.19.5, you can append \&":[start]-[end]\&" to the right of the address,
to tell curl what TCP port range to use. That means you specify a port range,
from a lower to a higher number. A single number works as well, but do note
that it increases the risk of failure since the port may not be available.
See also \fI--ftp-pasv\fP and \fI--disable-eprt\fP.
.IP "--ftp-pret"
(FTP) Tell curl to send a PRET command before PASV (and EPSV). Certain FTP servers,
mainly drftpd, require this non-standard command for directory listings as
well as up and downloads in PASV mode.
Added in 7.20.0.
.IP "--ftp-skip-pasv-ip"
(FTP) Tell curl to not use the IP address the server suggests in its response
to curl's PASV command when curl connects the data connection. Instead curl
will re-use the same IP address it already uses for the control
connection.
This option has no effect if PORT, EPRT or EPSV is used instead of PASV.
See also \fI--ftp-pasv\fP. Added in 7.14.2.
.IP "--ftp-ssl-ccc-mode <active/passive>"
(FTP) Sets the CCC mode. The passive mode will not initiate the shutdown, but
instead wait for the server to do it, and will not reply to the shutdown from
the server. The active mode initiates the shutdown and waits for a reply from
the server.
See also \fI--ftp-ssl-ccc\fP. Added in 7.16.2.
.IP "--ftp-ssl-ccc"
(FTP) Use CCC (Clear Command Channel) Shuts down the SSL/TLS layer after
authenticating. The rest of the control channel communication will be
unencrypted. This allows NAT routers to follow the FTP transaction. The
default mode is passive.
See also \fI--ssl\fP and \fI--ftp-ssl-ccc-mode\fP. Added in 7.16.1.
.IP "--ftp-ssl-control"
(FTP) Require SSL/TLS for the FTP login, clear for transfer. Allows secure
authentication, but non-encrypted data transfers for efficiency. Fails the
transfer if the server doesn't support SSL/TLS.
Added in 7.16.0.
.IP "-G, --get"
When used, this option will make all data specified with \fI-d, --data\fP, \fI--data-binary\fP
or \fI--data-urlencode\fP to be used in an HTTP GET request instead of the POST
request that otherwise would be used. The data will be appended to the URL
with a '?' separator.
If used in combination with \fI-I, --head\fP, the POST data will instead be appended to
the URL with a HEAD request.
If this option is used several times, only the first one is used. This is
because undoing a GET doesn't make sense, but you should then instead enforce
the alternative method you prefer.
.IP "-g, --globoff"
This option switches off the "URL globbing parser". When you set this option,
you can specify URLs that contain the letters {}[] without having them being
interpreted by curl itself. Note that these letters are not normal legal URL
contents but they should be encoded according to the URI standard.
.IP "--happy-eyeballs-timeout-ms <milliseconds>"
Happy eyeballs is an algorithm that attempts to connect to both IPv4 and IPv6
addresses for dual-stack hosts, preferring IPv6 first for the number of
milliseconds. If the IPv6 address cannot be connected to within that time then
a connection attempt is made to the IPv4 address in parallel. The first
connection to be established is the one that is used.
The range of suggested useful values is limited. Happy Eyeballs RFC 6555 says
"It is RECOMMENDED that connection attempts be paced 150-250 ms apart to
balance human factors against network load." libcurl currently defaults to
200 ms. Firefox and Chrome currently default to 300 ms.
If this option is used several times, the last one will be used.
Added in 7.59.0.
.IP "--haproxy-protocol"
(HTTP) Send a HAProxy PROXY protocol v1 header at the beginning of the connection. This
is used by some load balancers and reverse proxies to indicate the client's
true IP address and port.
This option is primarily useful when sending test requests to a service that
expects this header.
Added in 7.60.0.
.IP "-I, --head"
(HTTP FTP FILE) Fetch the headers only! HTTP-servers feature the command HEAD which this uses
to get nothing but the header of a document. When used on an FTP or FILE file,
curl displays the file size and last modification time only.
.IP "-H, --header <header/@file>"
(HTTP) Extra header to include in the request when sending HTTP to a server. You may
specify any number of extra headers. Note that if you should add a custom
header that has the same name as one of the internal ones curl would use, your
externally set header will be used instead of the internal one. This allows
you to make even trickier stuff than curl would normally do. You should not
replace internally set headers without knowing perfectly well what you're
doing. Remove an internal header by giving a replacement without content on
the right side of the colon, as in: -H \&"Host:". If you send the custom
header with no-value then its header must be terminated with a semicolon, such
as \-H \&"X-Custom-Header;" to send "X-Custom-Header:".
curl will make sure that each header you add/replace is sent with the proper
end-of-line marker, you should thus \fBnot\fP add that as a part of the header
content: do not add newlines or carriage returns, they will only mess things up
for you.
Starting in 7.55.0, this option can take an argument in @filename style, which
then adds a header for each line in the input file. Using @- will make curl
read the header file from stdin.
See also the \fI-A, --user-agent\fP and \fI-e, --referer\fP options.
Starting in 7.37.0, you need \fI--proxy-header\fP to send custom headers intended
for a proxy.
Example:
curl -H "X-First-Name: Joe" http://example.com/
\fBWARNING\fP: headers set with this option will be set in all requests - even
after redirects are followed, like when told with \fI-L, --location\fP. This can lead to
the header being sent to other hosts than the original host, so sensitive
headers should be used with caution combined with following redirects.
This option can be used multiple times to add/replace/remove multiple headers.
.IP "-h, --help"
Usage help. This lists all current command line options with a short
description.
.IP "--hostpubmd5 <md5>"
(SFTP SCP) Pass a string containing 32 hexadecimal digits. The string should
be the 128 bit MD5 checksum of the remote host's public key, curl will refuse
the connection with the host unless the md5sums match.
Added in 7.17.1.
.IP "--http0.9"
(HTTP) Tells curl to be fine with HTTP version 0.9 response.
HTTP/0.9 is a completely headerless response and therefore you can also
connect with this to non-HTTP servers and still get a response since curl will
simply transparently downgrade - if allowed.
A future curl version will deny continuing if the response isn't at least
HTTP/1.0 unless this option is used.
.IP "-0, --http1.0"
(HTTP) Tells curl to use HTTP version 1.0 instead of using its internally preferred
HTTP version.
This option overrides \fI--http1.1\fP and \fI--http2\fP.
.IP "--http1.1"
(HTTP) Tells curl to use HTTP version 1.1.
This option overrides \fI-0, --http1.0\fP and \fI--http2\fP. Added in 7.33.0.
.IP "--http2-prior-knowledge"
(HTTP) Tells curl to issue its non-TLS HTTP requests using HTTP/2 without HTTP/1.1
Upgrade. It requires prior knowledge that the server supports HTTP/2 straight
away. HTTPS requests will still do HTTP/2 the standard way with negotiated
protocol version in the TLS handshake.
\fI--http2-prior-knowledge\fP requires that the underlying libcurl was built to support HTTP/2. This option overrides \fI--http1.1\fP and \fI-0, --http1.0\fP and \fI--http2\fP. Added in 7.49.0.
.IP "--http2"
(HTTP) Tells curl to use HTTP version 2.
See also \fI--no-alpn\fP. \fI--http2\fP requires that the underlying libcurl was built to support HTTP/2. This option overrides \fI--http1.1\fP and \fI-0, --http1.0\fP and \fI--http2-prior-knowledge\fP. Added in 7.33.0.
.IP "--ignore-content-length"
(FTP HTTP) For HTTP, Ignore the Content-Length header. This is particularly useful for
servers running Apache 1.x, which will report incorrect Content-Length for
files larger than 2 gigabytes.
For FTP (since 7.46.0), skip the RETR command to figure out the size before
downloading a file.
.IP "-i, --include"
Include the HTTP response headers in the output. The HTTP response headers can
include things like server name, cookies, date of the document, HTTP version
and more...
To view the request headers, consider the \fI-v, --verbose\fP option.
See also \fI-v, --verbose\fP.
.IP "-k, --insecure"
(TLS)
By default, every SSL connection curl makes is verified to be secure. This
option allows curl to proceed and operate even for server connections
otherwise considered insecure.
The server connection is verified by making sure the server's certificate
contains the right name and verifies successfully using the cert store.
See this online resource for further details:
https://curl.haxx.se/docs/sslcerts.html
See also \fI--proxy-insecure\fP and \fI--cacert\fP.
.IP "--interface <name>"
Perform an operation using a specified interface. You can enter interface
name, IP address or host name. An example could look like:
curl --interface eth0:1 https://www.example.com/
If this option is used several times, the last one will be used.
On Linux it can be used to specify a VRF, but the binary needs to either
have CAP_NET_RAW or to be run as root. More information about Linux VRF:
https://www.kernel.org/doc/Documentation/networking/vrf.txt
See also \fI--dns-interface\fP.
.IP "-4, --ipv4"
This option tells curl to resolve names to IPv4 addresses only, and not for
example try IPv6.
See also \fI--http1.1\fP and \fI--http2\fP. This option overrides \fI-6, --ipv6\fP.
.IP "-6, --ipv6"
This option tells curl to resolve names to IPv6 addresses only, and not for
example try IPv4.
See also \fI--http1.1\fP and \fI--http2\fP. This option overrides \fI-4, --ipv4\fP.
.IP "-j, --junk-session-cookies"
(HTTP) When curl is told to read cookies from a given file, this option will make it
discard all "session cookies". This will basically have the same effect as if
a new session is started. Typical browsers always discard session cookies when
they're closed down.
See also \fI-b, --cookie\fP and \fI-c, --cookie-jar\fP.
.IP "--keepalive-time <seconds>"
This option sets the time a connection needs to remain idle before sending
keepalive probes and the time between individual keepalive probes. It is
currently effective on operating systems offering the TCP_KEEPIDLE and
TCP_KEEPINTVL socket options (meaning Linux, recent AIX, HP-UX and more). This
option has no effect if \fI--no-keepalive\fP is used.
If this option is used several times, the last one will be used. If
unspecified, the option defaults to 60 seconds.
Added in 7.18.0.
.IP "--key-type <type>"
(TLS) Private key file type. Specify which type your \fI--key\fP provided private key
is. DER, PEM, and ENG are supported. If not specified, PEM is assumed.
If this option is used several times, the last one will be used.
.IP "--key <key>"
(TLS SSH) Private key file name. Allows you to provide your private key in this separate
file. For SSH, if not specified, curl tries the following candidates in order:
'~/.ssh/id_rsa', '~/.ssh/id_dsa', './id_rsa', './id_dsa'.
If curl is built against OpenSSL library, and the engine pkcs11 is available,
then a PKCS#11 URI (RFC 7512) can be used to specify a private key located in a
PKCS#11 device. A string beginning with "pkcs11:" will be interpreted as a
PKCS#11 URI. If a PKCS#11 URI is provided, then the \fI--engine\fP option will be set
as "pkcs11" if none was provided and the \fI--key-type\fP option will be set as
"ENG" if none was provided.
If this option is used several times, the last one will be used.
.IP "--krb <level>"
(FTP) Enable Kerberos authentication and use. The level must be entered and should
be one of 'clear', 'safe', 'confidential', or 'private'. Should you use a
level that is not one of these, 'private' will instead be used.
If this option is used several times, the last one will be used.
\fI--krb\fP requires that the underlying libcurl was built to support Kerberos.
.IP "--libcurl <file>"
Append this option to any ordinary curl command line, and you will get a
libcurl-using C source code written to the file that does the equivalent
of what your command-line operation does!
If this option is used several times, the last given file name will be
used.
Added in 7.16.1.
.IP "--limit-rate <speed>"
Specify the maximum transfer rate you want curl to use - for both downloads
and uploads. This feature is useful if you have a limited pipe and you'd like
your transfer not to use your entire bandwidth. To make it slower than it
otherwise would be.
The given speed is measured in bytes/second, unless a suffix is appended.
Appending 'k' or 'K' will count the number as kilobytes, 'm' or 'M' makes it
megabytes, while 'g' or 'G' makes it gigabytes. Examples: 200K, 3m and 1G.
If you also use the \fI-Y, --speed-limit\fP option, that option will take precedence and
might cripple the rate-limiting slightly, to help keeping the speed-limit
logic working.
If this option is used several times, the last one will be used.
.IP "-l, --list-only"
(FTP POP3) (FTP)
When listing an FTP directory, this switch forces a name-only view. This is
especially useful if the user wants to machine-parse the contents of an FTP
directory since the normal directory view doesn't use a standard look or
format. When used like this, the option causes a NLST command to be sent to
the server instead of LIST.
Note: Some FTP servers list only files in their response to NLST; they do not
include sub-directories and symbolic links.
(POP3)
When retrieving a specific email from POP3, this switch forces a LIST command
to be performed instead of RETR. This is particularly useful if the user wants
to see if a specific message id exists on the server and what size it is.
Note: When combined with \fI-X, --request\fP, this option can be used to send an UIDL
command instead, so the user may use the email's unique identifier rather than
it's message id to make the request.
Added in 7.21.5.
.IP "--local-port <num/range>"
Set a preferred single number or range (FROM-TO) of local port numbers to use
for the connection(s). Note that port numbers by nature are a scarce resource
that will be busy at times so setting this range to something too narrow might
cause unnecessary connection setup failures.
Added in 7.15.2.
.IP "--location-trusted"
(HTTP) Like \fI-L, --location\fP, but will allow sending the name + password to all hosts that
the site may redirect to. This may or may not introduce a security breach if
the site redirects you to a site to which you'll send your authentication info
(which is plaintext in the case of HTTP Basic authentication).
See also \fI-u, --user\fP.
.IP "-L, --location"
(HTTP) If the server reports that the requested page has moved to a different
location (indicated with a Location: header and a 3XX response code), this
option will make curl redo the request on the new place. If used together with
\fI-i, --include\fP or \fI-I, --head\fP, headers from all requested pages will be shown. When
authentication is used, curl only sends its credentials to the initial
host. If a redirect takes curl to a different host, it won't be able to
intercept the user+password. See also \fI--location-trusted\fP on how to change
this. You can limit the amount of redirects to follow by using the
\fI--max-redirs\fP option.
When curl follows a redirect and the request is not a plain GET (for example
POST or PUT), it will do the following request with a GET if the HTTP response
was 301, 302, or 303. If the response code was any other 3xx code, curl will
re-send the following request using the same unmodified method.
You can tell curl to not change the non-GET request method to GET after a 30x
response by using the dedicated options for that: \fI--post301\fP, \fI--post302\fP and
\fI--post303\fP.
.IP "--login-options <options>"
(IMAP POP3 SMTP) Specify the login options to use during server authentication.
You can use the login options to specify protocol specific options that may
be used during authentication. At present only IMAP, POP3 and SMTP support
login options. For more information about the login options please see
RFC 2384, RFC 5092 and IETF draft draft-earhart-url-smtp-00.txt
If this option is used several times, the last one will be used.
Added in 7.34.0.
.IP "--mail-auth <address>"
(SMTP) Specify a single address. This will be used to specify the authentication
address (identity) of a submitted message that is being relayed to another
server.
See also \fI--mail-rcpt\fP and \fI--mail-from\fP. Added in 7.25.0.
.IP "--mail-from <address>"
(SMTP) Specify a single address that the given mail should get sent from.
See also \fI--mail-rcpt\fP and \fI--mail-auth\fP. Added in 7.20.0.
.IP "--mail-rcpt <address>"
(SMTP) Specify a single address, user name or mailing list name. Repeat this
option several times to send to multiple recipients.
When performing a mail transfer, the recipient should specify a valid email
address to send the mail to.
When performing an address verification (VRFY command), the recipient should be
specified as the user name or user name and domain (as per Section 3.5 of
RFC5321). (Added in 7.34.0)
When performing a mailing list expand (EXPN command), the recipient should be
specified using the mailing list name, such as "Friends" or "London-Office".
(Added in 7.34.0)
Added in 7.20.0.
.IP "-M, --manual"
Manual. Display the huge help text.
.IP "--max-filesize <bytes>"
Specify the maximum size (in bytes) of a file to download. If the file
requested is larger than this value, the transfer will not start and curl will
return with exit code 63.
A size modifier may be used. For example, Appending 'k' or 'K' will count the
number as kilobytes, 'm' or 'M' makes it megabytes, while 'g' or 'G' makes it
gigabytes. Examples: 200K, 3m and 1G. (Added in 7.58.0)
\fBNOTE:\fP The file size is not always known prior to download, and for such
files this option has no effect even if the file transfer ends up being larger
than this given limit. This concerns both FTP and HTTP transfers.
See also \fI--limit-rate\fP.
.IP "--max-redirs <num>"
(HTTP) Set maximum number of redirection-followings allowed. When \fI-L, --location\fP is used,
is used to prevent curl from following redirections \&"in absurdum". By
default, the limit is set to 50 redirections. Set this option to -1 to make it
unlimited.
If this option is used several times, the last one will be used.
.IP "-m, --max-time <seconds>"
Maximum time in seconds that you allow the whole operation to take. This is
useful for preventing your batch jobs from hanging for hours due to slow
networks or links going down. Since 7.32.0, this option accepts decimal
values, but the actual timeout will decrease in accuracy as the specified
timeout increases in decimal precision.
If this option is used several times, the last one will be used.
See also \fI--connect-timeout\fP.
.IP "--metalink"
This option can tell curl to parse and process a given URI as Metalink file
(both version 3 and 4 (RFC 5854) are supported) and make use of the mirrors
listed within for failover if there are errors (such as the file or server not
being available). It will also verify the hash of the file after the download
completes. The Metalink file itself is downloaded and processed in memory and
not stored in the local file system.
Example to use a remote Metalink file:
curl --metalink http://www.example.com/example.metalink
To use a Metalink file in the local file system, use FILE protocol (file://):
curl --metalink file://example.metalink
Please note that if FILE protocol is disabled, there is no way to use a local
Metalink file at the time of this writing. Also note that if \fI--metalink\fP and
\fI-i, --include\fP are used together, --include will be ignored. This is because
including headers in the response will break Metalink parser and if the
headers are included in the file described in Metalink file, hash check will
fail.
\fI--metalink\fP requires that the underlying libcurl was built to support metalink. Added in 7.27.0.
.IP "--negotiate"
(HTTP) Enables Negotiate (SPNEGO) authentication.
This option requires a library built with GSS-API or SSPI support. Use
\fI-V, --version\fP to see if your curl supports GSS-API/SSPI or SPNEGO.
When using this option, you must also provide a fake \fI-u, --user\fP option to activate
the authentication code properly. Sending a '-u :' is enough as the user name
and password from the \fI-u, --user\fP option aren't actually used.
If this option is used several times, only the first one is used.
See also \fI--basic\fP and \fI--ntlm\fP and \fI--anyauth\fP and \fI--proxy-negotiate\fP.
.IP "--netrc-file <filename>"
This option is similar to \fI-n, --netrc\fP, except that you provide the path (absolute
or relative) to the netrc file that curl should use. You can only specify one
netrc file per invocation. If several \fI--netrc-file\fP options are provided,
the last one will be used.
It will abide by \fI--netrc-optional\fP if specified.
This option overrides \fI-n, --netrc\fP. Added in 7.21.5.
.IP "--netrc-optional"
Very similar to \fI-n, --netrc\fP, but this option makes the .netrc usage \fBoptional\fP
and not mandatory as the \fI-n, --netrc\fP option does.
See also \fI--netrc-file\fP. This option overrides \fI-n, --netrc\fP.
.IP "-n, --netrc"
Makes curl scan the \fI.netrc\fP (\fI_netrc\fP on Windows) file in the user's
home directory for login name and password. This is typically used for FTP on
Unix. If used with HTTP, curl will enable user authentication. See
\fInetrc(5)\fP \fIftp(1)\fP for details on the file format. Curl will not
complain if that file doesn't have the right permissions (it should not be
either world- or group-readable). The environment variable "HOME" is used to
find the home directory.
A quick and very simple example of how to setup a \fI.netrc\fP to allow curl
to FTP to the machine host.domain.com with user name \&'myself' and password
\&'secret' should look similar to:
.B "machine host.domain.com login myself password secret"
.IP "-:, --next"
Tells curl to use a separate operation for the following URL and associated
options. This allows you to send several URL requests, each with their own
specific options, for example, such as different user names or custom requests
for each.
\fI-:, --next\fP will reset all local options and only global ones will have their
values survive over to the operation following the \fI-:, --next\fP instruction. Global
options include \fI-v, --verbose\fP, \fI--trace\fP, \fI--trace-ascii\fP and \fI--fail-early\fP.
For example, you can do both a GET and a POST in a single command line:
curl www1.example.com --next -d postthis www2.example.com
Added in 7.36.0.
.IP "--no-alpn"
(HTTPS) Disable the ALPN TLS extension. ALPN is enabled by default if libcurl was built
with an SSL library that supports ALPN. ALPN is used by a libcurl that supports
HTTP/2 to negotiate HTTP/2 support with the server during https sessions.
See also \fI--no-npn\fP and \fI--http2\fP. \fI--no-alpn\fP requires that the underlying libcurl was built to support TLS. Added in 7.36.0.
.IP "-N, --no-buffer"
Disables the buffering of the output stream. In normal work situations, curl
will use a standard buffered output stream that will have the effect that it
will output the data in chunks, not necessarily exactly when the data arrives.
Using this option will disable that buffering.
Note that this is the negated option name documented. You can thus use
--buffer to enforce the buffering.
.IP "--no-keepalive"
Disables the use of keepalive messages on the TCP connection. curl otherwise
enables them by default.
Note that this is the negated option name documented. You can thus use
--keepalive to enforce keepalive.
.IP "--no-npn"
(HTTPS) Disable the NPN TLS extension. NPN is enabled by default if libcurl was built
with an SSL library that supports NPN. NPN is used by a libcurl that supports
HTTP/2 to negotiate HTTP/2 support with the server during https sessions.
See also \fI--no-alpn\fP and \fI--http2\fP. \fI--no-npn\fP requires that the underlying libcurl was built to support TLS. Added in 7.36.0.
.IP "--no-sessionid"
(TLS) Disable curl's use of SSL session-ID caching. By default all transfers are
done using the cache. Note that while nothing should ever get hurt by
attempting to reuse SSL session-IDs, there seem to be broken SSL
implementations in the wild that may require you to disable this in order for
you to succeed.
Note that this is the negated option name documented. You can thus use
--sessionid to enforce session-ID caching.
Added in 7.16.0.
.IP "--noproxy <no-proxy-list>"
Comma-separated list of hosts which do not use a proxy, if one is specified.
The only wildcard is a single * character, which matches all hosts, and
effectively disables the proxy. Each name in this list is matched as either
a domain which contains the hostname, or the hostname itself. For example,
local.com would match local.com, local.com:80, and www.local.com, but not
www.notlocal.com.
Since 7.53.0, This option overrides the environment variables that disable the
proxy. If there's an environment variable disabling a proxy, you can set
noproxy list to \&"" to override it.
Added in 7.19.4.
.IP "--ntlm-wb"
(HTTP) Enables NTLM much in the style \fI--ntlm\fP does, but hand over the authentication
to the separate binary ntlmauth application that is executed when needed.
See also \fI--ntlm\fP and \fI--proxy-ntlm\fP.
.IP "--ntlm"
(HTTP) Enables NTLM authentication. The NTLM authentication method was designed by
Microsoft and is used by IIS web servers. It is a proprietary protocol,
reverse-engineered by clever people and implemented in curl based on their
efforts. This kind of behavior should not be endorsed, you should encourage
everyone who uses NTLM to switch to a public and documented authentication
method instead, such as Digest.
If you want to enable NTLM for your proxy authentication, then use
\fI--proxy-ntlm\fP.
If this option is used several times, only the first one is used.
See also \fI--proxy-ntlm\fP. \fI--ntlm\fP requires that the underlying libcurl was built to support TLS. This option overrides \fI--basic\fP and \fI--negotiate\fP and \fI--digest\fP and \fI--anyauth\fP.
.IP "--oauth2-bearer <token>"
(IMAP POP3 SMTP) Specify the Bearer Token for OAUTH 2.0 server authentication. The Bearer Token
is used in conjunction with the user name which can be specified as part of
the \fI--url\fP or \fI-u, --user\fP options.
The Bearer Token and user name are formatted according to RFC 6750.
If this option is used several times, the last one will be used.
.IP "-o, --output <file>"
Write output to <file> instead of stdout. If you are using {} or [] to fetch
multiple documents, you can use '#' followed by a number in the <file>
specifier. That variable will be replaced with the current string for the URL
being fetched. Like in:
curl http://{one,two}.example.com -o "file_#1.txt"
or use several variables like:
curl http://{site,host}.host[1-5].com -o "#1_#2"
You may use this option as many times as the number of URLs you have. For
example, if you specify two URLs on the same command line, you can use it like
this:
curl -o aa example.com -o bb example.net
and the order of the -o options and the URLs doesn't matter, just that the
first -o is for the first URL and so on, so the above command line can also be
written as
curl example.com example.net -o aa -o bb
See also the \fI--create-dirs\fP option to create the local directories
dynamically. Specifying the output as '-' (a single dash) will force the
output to be done to stdout.
See also \fI-O, --remote-name\fP and \fI--remote-name-all\fP and \fI-J, --remote-header-name\fP.
.IP "--pass <phrase>"
(SSH TLS) Passphrase for the private key
If this option is used several times, the last one will be used.
.IP "--path-as-is"
Tell curl to not handle sequences of /../ or /./ in the given URL
path. Normally curl will squash or merge them according to standards but with
this option set you tell it not to do that.
Added in 7.42.0.
.IP "--pinnedpubkey <hashes>"
(TLS) Tells curl to use the specified public key file (or hashes) to verify the
peer. This can be a path to a file which contains a single public key in PEM
or DER format, or any number of base64 encoded sha256 hashes preceded by
\'sha256//\' and separated by \';\'
When negotiating a TLS or SSL connection, the server sends a certificate
indicating its identity. A public key is extracted from this certificate and
if it does not exactly match the public key provided to this option, curl will
abort the connection before sending or receiving any data.
PEM/DER support:
7.39.0: OpenSSL, GnuTLS and GSKit
7.43.0: NSS and wolfSSL/CyaSSL
7.47.0: mbedtls
7.49.0: PolarSSL
sha256 support:
7.44.0: OpenSSL, GnuTLS, NSS and wolfSSL/CyaSSL.
7.47.0: mbedtls
7.49.0: PolarSSL
Other SSL backends not supported.
If this option is used several times, the last one will be used.
.IP "--post301"
(HTTP) Tells curl to respect RFC 7231/6.4.2 and not convert POST requests into GET
requests when following a 301 redirection. The non-RFC behaviour is ubiquitous
in web browsers, so curl does the conversion by default to maintain
consistency. However, a server may require a POST to remain a POST after such
a redirection. This option is meaningful only when using \fI-L, --location\fP.
See also \fI--post302\fP and \fI--post303\fP and \fI-L, --location\fP. Added in 7.17.1.
.IP "--post302"
(HTTP) Tells curl to respect RFC 7231/6.4.3 and not convert POST requests into GET
requests when following a 302 redirection. The non-RFC behaviour is ubiquitous
in web browsers, so curl does the conversion by default to maintain
consistency. However, a server may require a POST to remain a POST after such
a redirection. This option is meaningful only when using \fI-L, --location\fP.
See also \fI--post301\fP and \fI--post303\fP and \fI-L, --location\fP. Added in 7.19.1.
.IP "--post303"
(HTTP) Tells curl to violate RFC 7231/6.4.4 and not convert POST requests into GET
requests when following 303 redirections. A server may require a POST to
remain a POST after a 303 redirection. This option is meaningful only when
using \fI-L, --location\fP.
See also \fI--post302\fP and \fI--post301\fP and \fI-L, --location\fP. Added in 7.26.0.
.IP "--preproxy [protocol://]host[:port]"
Use the specified SOCKS proxy before connecting to an HTTP or HTTPS \fI-x, --proxy\fP. In
such a case curl first connects to the SOCKS proxy and then connects (through
SOCKS) to the HTTP or HTTPS proxy. Hence pre proxy.
The pre proxy string should be specified with a protocol:// prefix to specify
alternative proxy protocols. Use socks4://, socks4a://, socks5:// or
socks5h:// to request the specific SOCKS version to be used. No protocol
specified will make curl default to SOCKS4.
If the port number is not specified in the proxy string, it is assumed to be
1080.
User and password that might be provided in the proxy string are URL decoded
by curl. This allows you to pass in special characters such as @ by using %40
or pass in a colon with %3a.
If this option is used several times, the last one will be used.
Added in 7.52.0.
.IP "-#, --progress-bar"
Make curl display transfer progress as a simple progress bar instead of the
standard, more informational, meter.
This progress bar draws a single line of '#' characters across the screen and
shows a percentage if the transfer size is known. For transfers without a
known size, there will be space ship (-=o=-) that moves back and forth but
only while data is being transferred, with a set of flying hash sign symbols on
top.
.IP "--proto-default <protocol>"
Tells curl to use \fIprotocol\fP for any URL missing a scheme name.
Example:
curl --proto-default https ftp.mozilla.org
An unknown or unsupported protocol causes error
\fICURLE_UNSUPPORTED_PROTOCOL\fP (1).
This option does not change the default proxy protocol (http).
Without this option curl would make a guess based on the host, see \fI--url\fP for
details.
Added in 7.45.0.
.IP "--proto-redir <protocols>"
Tells curl to limit what protocols it may use on redirect. Protocols denied by
\fI--proto\fP are not overridden by this option. See --proto for how protocols are
represented.
Example, allow only HTTP and HTTPS on redirect:
curl --proto-redir -all,http,https http://example.com
By default curl will allow all protocols on redirect except several disabled
for security reasons: Since 7.19.4 FILE and SCP are disabled, and since 7.40.0
SMB and SMBS are also disabled. Specifying \fIall\fP or \fI+all\fP enables all
protocols on redirect, including those disabled for security.
Added in 7.20.2.
.IP "--proto <protocols>"
Tells curl to limit what protocols it may use in the transfer. Protocols are
evaluated left to right, are comma separated, and are each a protocol name or
'all', optionally prefixed by zero or more modifiers. Available modifiers are:
.RS
.TP 3
.B +
Permit this protocol in addition to protocols already permitted (this is
the default if no modifier is used).
.TP
.B -
Deny this protocol, removing it from the list of protocols already permitted.
.TP
.B =
Permit only this protocol (ignoring the list already permitted), though
subject to later modification by subsequent entries in the comma separated
list.
.RE
.IP
For example:
.RS
.TP 15
.B \fI--proto\fP -ftps
uses the default protocols, but disables ftps
.TP
.B \fI--proto\fP -all,https,+http
only enables http and https
.TP
.B \fI--proto\fP =http,https
also only enables http and https
.RE
Unknown protocols produce a warning. This allows scripts to safely rely on
being able to disable potentially dangerous protocols, without relying upon
support for that protocol being built into curl to avoid an error.
This option can be used multiple times, in which case the effect is the same
as concatenating the protocols into one instance of the option.
See also \fI--proto-redir\fP and \fI--proto-default\fP. Added in 7.20.2.
.IP "--proxy-anyauth"
Tells curl to pick a suitable authentication method when communicating with
the given HTTP proxy. This might cause an extra request/response round-trip.
See also \fI-x, --proxy\fP and \fI--proxy-basic\fP and \fI--proxy-digest\fP. Added in 7.13.2.
.IP "--proxy-basic"
Tells curl to use HTTP Basic authentication when communicating with the given
proxy. Use \fI--basic\fP for enabling HTTP Basic with a remote host. Basic is the
default authentication method curl uses with proxies.
See also \fI-x, --proxy\fP and \fI--proxy-anyauth\fP and \fI--proxy-digest\fP.
.IP "--proxy-cacert <file>"
Same as \fI--cacert\fP but used in HTTPS proxy context.
See also \fI--proxy-capath\fP and \fI--cacert\fP and \fI--capath\fP and \fI-x, --proxy\fP. Added in 7.52.0.
.IP "--proxy-capath <dir>"
Same as \fI--capath\fP but used in HTTPS proxy context.
See also \fI--proxy-cacert\fP and \fI-x, --proxy\fP and \fI--capath\fP. Added in 7.52.0.
.IP "--proxy-cert-type <type>"
Same as \fI--cert-type\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-cert <cert[:passwd]>"
Same as \fI-E, --cert\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-ciphers <list>"
Same as \fI--ciphers\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-crlfile <file>"
Same as \fI--crlfile\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-digest"
Tells curl to use HTTP Digest authentication when communicating with the given
proxy. Use \fI--digest\fP for enabling HTTP Digest with a remote host.
See also \fI-x, --proxy\fP and \fI--proxy-anyauth\fP and \fI--proxy-basic\fP.
.IP "--proxy-header <header/@file>"
(HTTP) Extra header to include in the request when sending HTTP to a proxy. You may
specify any number of extra headers. This is the equivalent option to \fI-H, --header\fP
but is for proxy communication only like in CONNECT requests when you want a
separate header sent to the proxy to what is sent to the actual remote host.
curl will make sure that each header you add/replace is sent with the proper
end-of-line marker, you should thus \fBnot\fP add that as a part of the header
content: do not add newlines or carriage returns, they will only mess things
up for you.
Headers specified with this option will not be included in requests that curl
knows will not be sent to a proxy.
Starting in 7.55.0, this option can take an argument in @filename style, which
then adds a header for each line in the input file. Using @- will make curl
read the header file from stdin.
This option can be used multiple times to add/replace/remove multiple headers.
Added in 7.37.0.
.IP "--proxy-insecure"
Same as \fI-k, --insecure\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-key-type <type>"
Same as \fI--key-type\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-key <key>"
Same as \fI--key\fP but used in HTTPS proxy context.
.IP "--proxy-negotiate"
Tells curl to use HTTP Negotiate (SPNEGO) authentication when communicating
with the given proxy. Use \fI--negotiate\fP for enabling HTTP Negotiate (SPNEGO)
with a remote host.
See also \fI--proxy-anyauth\fP and \fI--proxy-basic\fP. Added in 7.17.1.
.IP "--proxy-ntlm"
Tells curl to use HTTP NTLM authentication when communicating with the given
proxy. Use \fI--ntlm\fP for enabling NTLM with a remote host.
See also \fI--proxy-negotiate\fP and \fI--proxy-anyauth\fP.
.IP "--proxy-pass <phrase>"
Same as \fI--pass\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-pinnedpubkey <hashes>"
(TLS) Tells curl to use the specified public key file (or hashes) to verify the
proxy. This can be a path to a file which contains a single public key in PEM
or DER format, or any number of base64 encoded sha256 hashes preceded by
\'sha256//\' and separated by \';\'
When negotiating a TLS or SSL connection, the server sends a certificate
indicating its identity. A public key is extracted from this certificate and
if it does not exactly match the public key provided to this option, curl will
abort the connection before sending or receiving any data.
If this option is used several times, the last one will be used.
.IP "--proxy-service-name <name>"
This option allows you to change the service name for proxy negotiation.
Added in 7.43.0.
.IP "--proxy-ssl-allow-beast"
Same as \fI--ssl-allow-beast\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-tls13-ciphers <ciphersuite list>"
(TLS) Specifies which cipher suites to use in the connection to your HTTPS proxy
when it negotiates TLS 1.3. The list of ciphers suites must specify valid
ciphers. Read up on TLS 1.3 cipher suite details on this URL:
https://curl.haxx.se/docs/ssl-ciphers.html
If this option is used several times, the last one will be used.
.IP "--proxy-tlsauthtype <type>"
Same as \fI--tlsauthtype\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-tlspassword <string>"
Same as \fI--tlspassword\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-tlsuser <name>"
Same as \fI--tlsuser\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "--proxy-tlsv1"
Same as \fI-1, --tlsv1\fP but used in HTTPS proxy context.
Added in 7.52.0.
.IP "-U, --proxy-user <user:password>"
Specify the user name and password to use for proxy authentication.
If you use a Windows SSPI-enabled curl binary and do either Negotiate or NTLM
authentication then you can tell curl to select the user name and password
from your environment by specifying a single colon with this option: "-U :".
If this option is used several times, the last one will be used.
.IP "-x, --proxy [protocol://]host[:port]"
Use the specified proxy.
The proxy string can be specified with a protocol:// prefix. No protocol
specified or http:// will be treated as HTTP proxy. Use socks4://, socks4a://,
socks5:// or socks5h:// to request a specific SOCKS version to be used.
(The protocol support was added in curl 7.21.7)
HTTPS proxy support via https:// protocol prefix was added in 7.52.0 for
OpenSSL, GnuTLS and NSS.
Unrecognized and unsupported proxy protocols cause an error since 7.52.0.
Prior versions may ignore the protocol and use http:// instead.
If the port number is not specified in the proxy string, it is assumed to be
1080.
This option overrides existing environment variables that set the proxy to
use. If there's an environment variable setting a proxy, you can set proxy to
\&"" to override it.
All operations that are performed over an HTTP proxy will transparently be
converted to HTTP. It means that certain protocol specific operations might
not be available. This is not the case if you can tunnel through the proxy, as
one with the \fI-p, --proxytunnel\fP option.
User and password that might be provided in the proxy string are URL decoded
by curl. This allows you to pass in special characters such as @ by using %40
or pass in a colon with %3a.
The proxy host can be specified the exact same way as the proxy environment
variables, including the protocol prefix (http://) and the embedded user +
password.
If this option is used several times, the last one will be used.
.IP "--proxy1.0 <host[:port]>"
Use the specified HTTP 1.0 proxy. If the port number is not specified, it is
assumed at port 1080.
The only difference between this and the HTTP proxy option \fI-x, --proxy\fP, is that
attempts to use CONNECT through the proxy will specify an HTTP 1.0 protocol
instead of the default HTTP 1.1.
.IP "-p, --proxytunnel"
When an HTTP proxy is used \fI-x, --proxy\fP, this option will cause non-HTTP protocols
to attempt to tunnel through the proxy instead of merely using it to do
HTTP-like operations. The tunnel approach is made with the HTTP proxy CONNECT
request and requires that the proxy allows direct connect to the remote port
number curl wants to tunnel through to.
To suppress proxy CONNECT response headers when curl is set to output headers
use \fI--suppress-connect-headers\fP.
See also \fI-x, --proxy\fP.
.IP "--pubkey <key>"
(SFTP SCP) Public key file name. Allows you to provide your public key in this separate
file.
If this option is used several times, the last one will be used.
(As of 7.39.0, curl attempts to automatically extract the public key from the
private key file, so passing this option is generally not required. Note that
this public key extraction requires libcurl to be linked against a copy of
libssh2 1.2.8 or higher that is itself linked against OpenSSL.)
.IP "-Q, --quote"
(FTP SFTP)
Send an arbitrary command to the remote FTP or SFTP server. Quote commands are
sent BEFORE the transfer takes place (just after the initial PWD command in an
FTP transfer, to be exact). To make commands take place after a successful
transfer, prefix them with a dash '-'. To make commands be sent after curl
has changed the working directory, just before the transfer command(s), prefix
the command with a '+' (this is only supported for FTP). You may specify any
number of commands.
If the server returns failure for one of the commands, the entire operation
will be aborted. You must send syntactically correct FTP commands as RFC 959
defines to FTP servers, or one of the commands listed below to SFTP servers.
This option can be used multiple times. When speaking to an FTP server, prefix
the command with an asterisk (*) to make curl continue even if the command
fails as by default curl will stop at first failure.
SFTP is a binary protocol. Unlike for FTP, curl interprets SFTP quote commands
itself before sending them to the server. File names may be quoted
shell-style to embed spaces or special characters. Following is the list of
all supported SFTP quote commands:
.RS
.IP "chgrp group file"
The chgrp command sets the group ID of the file named by the file operand to
the group ID specified by the group operand. The group operand is a decimal
integer group ID.
.IP "chmod mode file"
The chmod command modifies the file mode bits of the specified file. The
mode operand is an octal integer mode number.
.IP "chown user file"
The chown command sets the owner of the file named by the file operand to the
user ID specified by the user operand. The user operand is a decimal
integer user ID.
.IP "ln source_file target_file"
The ln and symlink commands create a symbolic link at the target_file location
pointing to the source_file location.
.IP "mkdir directory_name"
The mkdir command creates the directory named by the directory_name operand.
.IP "pwd"
The pwd command returns the absolute pathname of the current working directory.
.IP "rename source target"
The rename command renames the file or directory named by the source
operand to the destination path named by the target operand.
.IP "rm file"
The rm command removes the file specified by the file operand.
.IP "rmdir directory"
The rmdir command removes the directory entry specified by the directory
operand, provided it is empty.
.IP "symlink source_file target_file"
See ln.
.RE
.IP "--random-file <file>"
Specify the path name to file containing what will be considered as random
data. The data may be used to seed the random engine for SSL connections. See
also the \fI--egd-file\fP option.
.IP "-r, --range <range>"
(HTTP FTP SFTP FILE) Retrieve a byte range (i.e. a partial document) from an HTTP/1.1, FTP or SFTP
server or a local FILE. Ranges can be specified in a number of ways.
.RS
.TP 10
.B 0-499
specifies the first 500 bytes
.TP
.B 500-999
specifies the second 500 bytes
.TP
.B -500
specifies the last 500 bytes
.TP
.B 9500-
specifies the bytes from offset 9500 and forward
.TP
.B 0-0,-1
specifies the first and last byte only(*)(HTTP)
.TP
.B 100-199,500-599
specifies two separate 100-byte ranges(*) (HTTP)
.RE
.IP
(*) = NOTE that this will cause the server to reply with a multipart
response!
Only digit characters (0-9) are valid in the 'start' and 'stop' fields of the
\&'start-stop' range syntax. If a non-digit character is given in the range,
the server's response will be unspecified, depending on the server's
configuration.
You should also be aware that many HTTP/1.1 servers do not have this feature
enabled, so that when you attempt to get a range, you'll instead get the whole
document.
FTP and SFTP range downloads only support the simple 'start-stop' syntax
(optionally with one of the numbers omitted). FTP use depends on the extended
FTP command SIZE.
If this option is used several times, the last one will be used.
.IP "--raw"
(HTTP) When used, it disables all internal HTTP decoding of content or transfer
encodings and instead makes them passed on unaltered, raw.
Added in 7.16.2.
.IP "-e, --referer <URL>"
(HTTP) Sends the "Referrer Page" information to the HTTP server. This can also be set
with the \fI-H, --header\fP flag of course. When used with \fI-L, --location\fP you can append
";auto" to the \fI-e, --referer\fP URL to make curl automatically set the previous URL
when it follows a Location: header. The \&";auto" string can be used alone,
even if you don't set an initial \fI-e, --referer\fP.
If this option is used several times, the last one will be used.
See also \fI-A, --user-agent\fP and \fI-H, --header\fP.
.IP "-J, --remote-header-name"
(HTTP) This option tells the \fI-O, --remote-name\fP option to use the server-specified
Content-Disposition filename instead of extracting a filename from the URL.
If the server specifies a file name and a file with that name already exists
in the current working directory it will not be overwritten and an error will
occur. If the server doesn't specify a file name then this option has no
effect.
There's no attempt to decode %-sequences (yet) in the provided file name, so
this option may provide you with rather unexpected file names.
\fBWARNING\fP: Exercise judicious use of this option, especially on Windows. A
rogue server could send you the name of a DLL or other file that could possibly
be loaded automatically by Windows or some third party software.
.IP "--remote-name-all"
This option changes the default action for all given URLs to be dealt with as
if \fI-O, --remote-name\fP were used for each one. So if you want to disable that for a
specific URL after \fI--remote-name-all\fP has been used, you must use "-o -" or
--no-remote-name.
Added in 7.19.0.
.IP "-O, --remote-name"
Write output to a local file named like the remote file we get. (Only the file
part of the remote file is used, the path is cut off.)
The file will be saved in the current working directory. If you want the file
saved in a different directory, make sure you change the current working
directory before invoking curl with this option.
The remote file name to use for saving is extracted from the given URL,
nothing else, and if it already exists it will be overwritten. If you want the
server to be able to choose the file name refer to \fI-J, --remote-header-name\fP which
can be used in addition to this option. If the server chooses a file name and
that name already exists it will not be overwritten.
There is no URL decoding done on the file name. If it has %20 or other URL
encoded parts of the name, they will end up as-is as file name.
You may use this option as many times as the number of URLs you have.
.IP "-R, --remote-time"
When used, this will make curl attempt to figure out the timestamp of the
remote file, and if that is available make the local file get that same
timestamp.
.IP "--request-target"
(HTTP) Tells curl to use an alternative "target" (path) instead of using the path as
provided in the URL. Particularly useful when wanting to issue HTTP requests
without leading slash or other data that doesn't follow the regular URL
pattern, like "OPTIONS *".
Added in 7.55.0.
.IP "-X, --request <command>"
(HTTP) Specifies a custom request method to use when communicating with the
HTTP server. The specified request method will be used instead of the method
otherwise used (which defaults to GET). Read the HTTP 1.1 specification for
details and explanations. Common additional HTTP requests include PUT and
DELETE, but related technologies like WebDAV offers PROPFIND, COPY, MOVE and
more.
Normally you don't need this option. All sorts of GET, HEAD, POST and PUT
requests are rather invoked by using dedicated command line options.
This option only changes the actual word used in the HTTP request, it does not
alter the way curl behaves. So for example if you want to make a proper HEAD
request, using -X HEAD will not suffice. You need to use the \fI-I, --head\fP option.
The method string you set with \fI-X, --request\fP will be used for all requests, which
if you for example use \fI-L, --location\fP may cause unintended side-effects when curl
doesn't change request method according to the HTTP 30x response codes - and
similar.
(FTP)
Specifies a custom FTP command to use instead of LIST when doing file lists
with FTP.
(POP3)
Specifies a custom POP3 command to use instead of LIST or RETR. (Added in
7.26.0)
(IMAP)
Specifies a custom IMAP command to use instead of LIST. (Added in 7.30.0)
(SMTP)
Specifies a custom SMTP command to use instead of HELP or VRFY. (Added in 7.34.0)
If this option is used several times, the last one will be used.
.IP "--resolve <host:port:address[,address]...>"
Provide a custom address for a specific host and port pair. Using this, you
can make the curl requests(s) use a specified address and prevent the
otherwise normally resolved address to be used. Consider it a sort of
/etc/hosts alternative provided on the command line. The port number should be
the number used for the specific protocol the host will be used for. It means
you need several entries if you want to provide address for the same host but
different ports.
By specifying '*' as host you can tell curl to resolve any host and specific
port pair to the specified address. Wildcard is resolved last so any \fI--resolve\fP
with a specific host and port will be used first.
The provided address set by this option will be used even if \fI-4, --ipv4\fP or \fI-6, --ipv6\fP
is set to make curl use another IP version.
Support for providing the IP address within [brackets] was added in 7.57.0.
Support for providing multiple IP addresses per entry was added in 7.59.0.
Support for resolving with wildcard was added in 7.64.0.
This option can be used many times to add many host names to resolve.
Added in 7.21.3.
.IP "--retry-connrefused"
In addition to the other conditions, consider ECONNREFUSED as a transient
error too for \fI--retry\fP. This option is used together with --retry.
Added in 7.52.0.
.IP "--retry-delay <seconds>"
Make curl sleep this amount of time before each retry when a transfer has
failed with a transient error (it changes the default backoff time algorithm
between retries). This option is only interesting if \fI--retry\fP is also
used. Setting this delay to zero will make curl use the default backoff time.
If this option is used several times, the last one will be used.
Added in 7.12.3.
.IP "--retry-max-time <seconds>"
The retry timer is reset before the first transfer attempt. Retries will be
done as usual (see \fI--retry\fP) as long as the timer hasn't reached this given
limit. Notice that if the timer hasn't reached the limit, the request will be
made and while performing, it may take longer than this given time period. To
limit a single request\'s maximum time, use \fI-m, --max-time\fP. Set this option to
zero to not timeout retries.
If this option is used several times, the last one will be used.
Added in 7.12.3.
.IP "--retry <num>"
If a transient error is returned when curl tries to perform a transfer, it
will retry this number of times before giving up. Setting the number to 0
makes curl do no retries (which is the default). Transient error means either:
a timeout, an FTP 4xx response code or an HTTP 408 or 5xx response code.
When curl is about to retry a transfer, it will first wait one second and then
for all forthcoming retries it will double the waiting time until it reaches
10 minutes which then will be the delay between the rest of the retries. By
using \fI--retry-delay\fP you disable this exponential backoff algorithm. See also
\fI--retry-max-time\fP to limit the total time allowed for retries.
If this option is used several times, the last one will be used.
Added in 7.12.3.
.IP "--sasl-ir"
Enable initial response in SASL authentication.
Added in 7.31.0.
.IP "--service-name <name>"
This option allows you to change the service name for SPNEGO.
Examples: \fI--negotiate\fP \fI--service-name\fP sockd would use sockd/server-name.
Added in 7.43.0.
.IP "-S, --show-error"
When used with \fI-s, --silent\fP, it makes curl show an error message if it fails.
.IP "-s, --silent"
Silent or quiet mode. Don't show progress meter or error messages. Makes Curl
mute. It will still output the data you ask for, potentially even to the
terminal/stdout unless you redirect it.
Use \fI-S, --show-error\fP in addition to this option to disable progress meter but
still show error messages.
See also \fI-v, --verbose\fP and \fI--stderr\fP.
.IP "--socks4 <host[:port]>"
Use the specified SOCKS4 proxy. If the port number is not specified, it is
assumed at port 1080.
This option overrides any previous use of \fI-x, --proxy\fP, as they are mutually
exclusive.
Since 7.21.7, this option is superfluous since you can specify a socks4 proxy
with \fI-x, --proxy\fP using a socks4:// protocol prefix.
Since 7.52.0, \fI--preproxy\fP can be used to specify a SOCKS proxy at the same time
\fI-x, --proxy\fP is used with an HTTP/HTTPS proxy. In such a case curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS proxy.
If this option is used several times, the last one will be used.
Added in 7.15.2.
.IP "--socks4a <host[:port]>"
Use the specified SOCKS4a proxy. If the port number is not specified, it is
assumed at port 1080.
This option overrides any previous use of \fI-x, --proxy\fP, as they are mutually
exclusive.
Since 7.21.7, this option is superfluous since you can specify a socks4a proxy
with \fI-x, --proxy\fP using a socks4a:// protocol prefix.
Since 7.52.0, \fI--preproxy\fP can be used to specify a SOCKS proxy at the same time
\fI-x, --proxy\fP is used with an HTTP/HTTPS proxy. In such a case curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS proxy.
If this option is used several times, the last one will be used.
Added in 7.18.0.
.IP "--socks5-basic"
Tells curl to use username/password authentication when connecting to a SOCKS5
proxy. The username/password authentication is enabled by default. Use
\fI--socks5-gssapi\fP to force GSS-API authentication to SOCKS5 proxies.
Added in 7.55.0.
.IP "--socks5-gssapi-nec"
As part of the GSS-API negotiation a protection mode is negotiated. RFC 1961
says in section 4.3/4.4 it should be protected, but the NEC reference
implementation does not. The option \fI--socks5-gssapi-nec\fP allows the
unprotected exchange of the protection mode negotiation.
Added in 7.19.4.
.IP "--socks5-gssapi-service <name>"
The default service name for a socks server is rcmd/server-fqdn. This option
allows you to change it.
Examples: \fI--socks5\fP proxy-name \fI--socks5-gssapi-service\fP sockd would use
sockd/proxy-name \fI--socks5\fP proxy-name \fI--socks5-gssapi-service\fP sockd/real-name
would use sockd/real-name for cases where the proxy-name does not match the
principal name.
Added in 7.19.4.
.IP "--socks5-gssapi"
Tells curl to use GSS-API authentication when connecting to a SOCKS5 proxy.
The GSS-API authentication is enabled by default (if curl is compiled with
GSS-API support). Use \fI--socks5-basic\fP to force username/password authentication
to SOCKS5 proxies.
Added in 7.55.0.
.IP "--socks5-hostname <host[:port]>"
Use the specified SOCKS5 proxy (and let the proxy resolve the host name). If
the port number is not specified, it is assumed at port 1080.
This option overrides any previous use of \fI-x, --proxy\fP, as they are mutually
exclusive.
Since 7.21.7, this option is superfluous since you can specify a socks5
hostname proxy with \fI-x, --proxy\fP using a socks5h:// protocol prefix.
Since 7.52.0, \fI--preproxy\fP can be used to specify a SOCKS proxy at the same time
\fI-x, --proxy\fP is used with an HTTP/HTTPS proxy. In such a case curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS proxy.
If this option is used several times, the last one will be used.
Added in 7.18.0.
.IP "--socks5 <host[:port]>"
Use the specified SOCKS5 proxy - but resolve the host name locally. If the
port number is not specified, it is assumed at port 1080.
This option overrides any previous use of \fI-x, --proxy\fP, as they are mutually
exclusive.
Since 7.21.7, this option is superfluous since you can specify a socks5 proxy
with \fI-x, --proxy\fP using a socks5:// protocol prefix.
Since 7.52.0, \fI--preproxy\fP can be used to specify a SOCKS proxy at the same time
\fI-x, --proxy\fP is used with an HTTP/HTTPS proxy. In such a case curl first connects to
the SOCKS proxy and then connects (through SOCKS) to the HTTP or HTTPS proxy.
If this option is used several times, the last one will be used.
This option (as well as \fI--socks4\fP) does not work with IPV6, FTPS or LDAP.
Added in 7.18.0.
.IP "-Y, --speed-limit <speed>"
If a download is slower than this given speed (in bytes per second) for
speed-time seconds it gets aborted. speed-time is set with \fI-y, --speed-time\fP and is
30 if not set.
If this option is used several times, the last one will be used.
.IP "-y, --speed-time <seconds>"
If a download is slower than speed-limit bytes per second during a speed-time
period, the download gets aborted. If speed-time is used, the default
speed-limit will be 1 unless set with \fI-Y, --speed-limit\fP.
This option controls transfers and thus will not affect slow connects etc. If
this is a concern for you, try the \fI--connect-timeout\fP option.
If this option is used several times, the last one will be used.
.IP "--ssl-allow-beast"
This option tells curl to not work around a security flaw in the SSL3 and
TLS1.0 protocols known as BEAST. If this option isn't used, the SSL layer may
use workarounds known to cause interoperability problems with some older SSL
implementations. WARNING: this option loosens the SSL security, and by using
this flag you ask for exactly that.
Added in 7.25.0.
.IP "--ssl-no-revoke"
(Schannel) This option tells curl to disable certificate revocation checks.
WARNING: this option loosens the SSL security, and by using this flag you ask
for exactly that.
Added in 7.44.0.
.IP "--ssl-reqd"
(FTP IMAP POP3 SMTP) Require SSL/TLS for the connection. Terminates the connection if the server
doesn't support SSL/TLS.
This option was formerly known as --ftp-ssl-reqd.
Added in 7.20.0.
.IP "--ssl"
(FTP IMAP POP3 SMTP)
Try to use SSL/TLS for the connection. Reverts to a non-secure connection if
the server doesn't support SSL/TLS. See also \fI--ftp-ssl-control\fP and \fI--ssl-reqd\fP
for different levels of encryption required.
This option was formerly known as --ftp-ssl (Added in 7.11.0). That option
name can still be used but will be removed in a future version.
Added in 7.20.0.
.IP "-2, --sslv2"
(SSL) Forces curl to use SSL version 2 when negotiating with a remote SSL
server. Sometimes curl is built without SSLv2 support. SSLv2 is widely
considered insecure (see RFC 6176).
See also \fI--http1.1\fP and \fI--http2\fP. \fI-2, --sslv2\fP requires that the underlying libcurl was built to support TLS. This option overrides \fI-3, --sslv3\fP and \fI-1, --tlsv1\fP and \fI--tlsv1.1\fP and \fI--tlsv1.2\fP.
.IP "-3, --sslv3"
(SSL) Forces curl to use SSL version 3 when negotiating with a remote SSL
server. Sometimes curl is built without SSLv3 support. SSLv3 is widely
considered insecure (see RFC 7568).
See also \fI--http1.1\fP and \fI--http2\fP. \fI-3, --sslv3\fP requires that the underlying libcurl was built to support TLS. This option overrides \fI-2, --sslv2\fP and \fI-1, --tlsv1\fP and \fI--tlsv1.1\fP and \fI--tlsv1.2\fP.
.IP "--stderr"
Redirect all writes to stderr to the specified file instead. If the file name
is a plain '-', it is instead written to stdout.
If this option is used several times, the last one will be used.
See also \fI-v, --verbose\fP and \fI-s, --silent\fP.
.IP "--styled-output"
Enables the automatic use of bold font styles when writing HTTP headers to the
terminal. Use --no-styled-output to switch them off.
Added in 7.61.0.
.IP "--suppress-connect-headers"
When \fI-p, --proxytunnel\fP is used and a CONNECT request is made don't output proxy
CONNECT response headers. This option is meant to be used with \fI-D, --dump-header\fP or
\fI-i, --include\fP which are used to show protocol headers in the output. It has no
effect on debug options such as \fI-v, --verbose\fP or \fI--trace\fP, or any statistics.
See also \fI-D, --dump-header\fP and \fI-i, --include\fP and \fI-p, --proxytunnel\fP.
.IP "--tcp-fastopen"
Enable use of TCP Fast Open (RFC7413).
Added in 7.49.0.
.IP "--tcp-nodelay"
Turn on the TCP_NODELAY option. See the \fIcurl_easy_setopt(3)\fP man page for
details about this option.
Since 7.50.2, curl sets this option by default and you need to explicitly
switch it off if you don't want it on.
Added in 7.11.2.
.IP "-t, --telnet-option <opt=val>"
Pass options to the telnet protocol. Supported options are:
TTYPE=<term> Sets the terminal type.
XDISPLOC=<X display> Sets the X display location.
NEW_ENV=<var,val> Sets an environment variable.
.IP "--tftp-blksize <value>"
(TFTP) Set TFTP BLKSIZE option (must be >512). This is the block size that curl will
try to use when transferring data to or from a TFTP server. By default 512
bytes will be used.
If this option is used several times, the last one will be used.
Added in 7.20.0.
.IP "--tftp-no-options"
(TFTP) Tells curl not to send TFTP options requests.
This option improves interop with some legacy servers that do not acknowledge
or properly implement TFTP options. When this option is used \fI--tftp-blksize\fP is
ignored.
Added in 7.48.0.
.IP "-z, --time-cond <time>"
(HTTP FTP) Request a file that has been modified later than the given time and date, or
one that has been modified before that time. The <date expression> can be all
sorts of date strings or if it doesn't match any internal ones, it is taken as
a filename and tries to get the modification date (mtime) from <file>
instead. See the \fIcurl_getdate(3)\fP man pages for date expression details.
Start the date expression with a dash (-) to make it request for a document
that is older than the given date/time, default is a document that is newer
than the specified date/time.
If this option is used several times, the last one will be used.
.IP "--tls-max <VERSION>"
(SSL) VERSION defines maximum supported TLS version. The minimum acceptable version
is set by tlsv1.0, tlsv1.1, tlsv1.2 or tlsv1.3.
.RS
.IP "default"
Use up to recommended TLS version.
.IP "1.0"
Use up to TLSv1.0.
.IP "1.1"
Use up to TLSv1.1.
.IP "1.2"
Use up to TLSv1.2.
.IP "1.3"
Use up to TLSv1.3.
.RE
See also \fI--tlsv1.0\fP and \fI--tlsv1.1\fP and \fI--tlsv1.2\fP and \fI--tlsv1.3\fP. \fI--tls-max\fP requires that the underlying libcurl was built to support TLS. Added in 7.54.0.
.IP "--tls13-ciphers <list of TLS 1.3 ciphersuites>"
(TLS) Specifies which cipher suites to use in the connection if it negotiates TLS
1.3. The list of ciphers suites must specify valid ciphers. Read up on TLS 1.3
cipher suite details on this URL:
https://curl.haxx.se/docs/ssl-ciphers.html
If this option is used several times, the last one will be used.
.IP "--tlsauthtype <type>"
Set TLS authentication type. Currently, the only supported option is "SRP",
for TLS-SRP (RFC 5054). If \fI--tlsuser\fP and \fI--tlspassword\fP are specified but
\fI--tlsauthtype\fP is not, then this option defaults to "SRP". This option works
only if the underlying libcurl is built with TLS-SRP support, which requires
OpenSSL or GnuTLS with TLS-SRP support.
Added in 7.21.4.
.IP "--tlspassword"
Set password for use with the TLS authentication method specified with
\fI--tlsauthtype\fP. Requires that \fI--tlsuser\fP also be set.
Added in 7.21.4.
.IP "--tlsuser <name>"
Set username for use with the TLS authentication method specified with
\fI--tlsauthtype\fP. Requires that \fI--tlspassword\fP also is set.
Added in 7.21.4.
.IP "--tlsv1.0"
(TLS) Forces curl to use TLS version 1.0 or later when connecting to a remote TLS server.
Added in 7.34.0.
.IP "--tlsv1.1"
(TLS) Forces curl to use TLS version 1.1 or later when connecting to a remote TLS server.
Added in 7.34.0.
.IP "--tlsv1.2"
(TLS) Forces curl to use TLS version 1.2 or later when connecting to a remote TLS server.
Added in 7.34.0.
.IP "--tlsv1.3"
(TLS) Forces curl to use TLS version 1.3 or later when connecting to a remote TLS server.
Note that TLS 1.3 is only supported by a subset of TLS backends. At the time
of this writing, they are BoringSSL, NSS, and Secure Transport (on iOS 11 or
later, and macOS 10.13 or later).
Added in 7.52.0.
.IP "-1, --tlsv1"
(SSL) Tells curl to use at least TLS version 1.x when negotiating with a remote TLS
server. That means TLS version 1.0 or higher
See also \fI--http1.1\fP and \fI--http2\fP. \fI-1, --tlsv1\fP requires that the underlying libcurl was built to support TLS. This option overrides \fI--tlsv1.1\fP and \fI--tlsv1.2\fP and \fI--tlsv1.3\fP.
.IP "--tr-encoding"
(HTTP) Request a compressed Transfer-Encoding response using one of the algorithms
curl supports, and uncompress the data while receiving it.
Added in 7.21.6.
.IP "--trace-ascii <file>"
Enables a full trace dump of all incoming and outgoing data, including
descriptive information, to the given output file. Use "-" as filename to have
the output sent to stdout.
This is very similar to \fI--trace\fP, but leaves out the hex part and only shows
the ASCII part of the dump. It makes smaller output that might be easier to
read for untrained humans.
If this option is used several times, the last one will be used.
This option overrides \fI--trace\fP and \fI-v, --verbose\fP.
.IP "--trace-time"
Prepends a time stamp to each trace or verbose line that curl displays.
Added in 7.14.0.
.IP "--trace <file>"
Enables a full trace dump of all incoming and outgoing data, including
descriptive information, to the given output file. Use "-" as filename to have
the output sent to stdout. Use "%" as filename to have the output sent to
stderr.
If this option is used several times, the last one will be used.
This option overrides \fI-v, --verbose\fP and \fI--trace-ascii\fP.
.IP "--unix-socket <path>"
(HTTP) Connect through this Unix domain socket, instead of using the network.
Added in 7.40.0.
.IP "-T, --upload-file <file>"
This transfers the specified local file to the remote URL. If there is no file
part in the specified URL, curl will append the local file name. NOTE that you
must use a trailing / on the last directory to really prove to Curl that there
is no file name or curl will think that your last directory name is the remote
file name to use. That will most likely cause the upload operation to fail. If
this is used on an HTTP(S) server, the PUT command will be used.
Use the file name "-" (a single dash) to use stdin instead of a given file.
Alternately, the file name "." (a single period) may be specified instead
of "-" to use stdin in non-blocking mode to allow reading server output
while stdin is being uploaded.
You can specify one \fI-T, --upload-file\fP for each URL on the command line. Each
\fI-T, --upload-file\fP + URL pair specifies what to upload and to where. curl also
supports "globbing" of the \fI-T, --upload-file\fP argument, meaning that you can upload
multiple files to a single URL by using the same URL globbing style supported
in the URL, like this:
curl --upload-file "{file1,file2}" http://www.example.com
or even
curl -T "img[1-1000].png" ftp://ftp.example.com/upload/
When uploading to an SMTP server: the uploaded data is assumed to be RFC 5322
formatted. It has to feature the necessary set of headers and mail body
formatted correctly by the user as curl will not transcode nor encode it
further in any way.
.IP "--url <url>"
Specify a URL to fetch. This option is mostly handy when you want to specify
URL(s) in a config file.
If the given URL is missing a scheme name (such as "http://" or "ftp://" etc)
then curl will make a guess based on the host. If the outermost sub-domain
name matches DICT, FTP, IMAP, LDAP, POP3 or SMTP then that protocol will be
used, otherwise HTTP will be used. Since 7.45.0 guessing can be disabled by
setting a default protocol, see \fI--proto-default\fP for details.
This option may be used any number of times. To control where this URL is
written, use the \fI-o, --output\fP or the \fI-O, --remote-name\fP options.
.IP "-B, --use-ascii"
(FTP LDAP) Enable ASCII transfer. For FTP, this can also be enforced by using a URL that
ends with ";type=A". This option causes data sent to stdout to be in text mode
for win32 systems.
.IP "-A, --user-agent <name>"
(HTTP)
Specify the User-Agent string to send to the HTTP server. To encode blanks in
the string, surround the string with single quote marks. This header can also
be set with the \fI-H, --header\fP or the \fI--proxy-header\fP options.
If this option is used several times, the last one will be used.
.IP "-u, --user <user:password>"
Specify the user name and password to use for server authentication. Overrides
\fI-n, --netrc\fP and \fI--netrc-optional\fP.
If you simply specify the user name, curl will prompt for a password.
The user name and passwords are split up on the first colon, which makes it
impossible to use a colon in the user name with this option. The password can,
still.
When using Kerberos V5 with a Windows based server you should include the
Windows domain name in the user name, in order for the server to successfully
obtain a Kerberos Ticket. If you don't then the initial authentication
handshake may fail.
When using NTLM, the user name can be specified simply as the user name,
without the domain, if there is a single domain and forest in your setup
for example.
To specify the domain name use either Down-Level Logon Name or UPN (User
Principal Name) formats. For example, EXAMPLE\\user and user@example.com
respectively.
If you use a Windows SSPI-enabled curl binary and perform Kerberos V5,
Negotiate, NTLM or Digest authentication then you can tell curl to select
the user name and password from your environment by specifying a single colon
with this option: "-u :".
If this option is used several times, the last one will be used.
.IP "-v, --verbose"
Makes curl verbose during the operation. Useful for debugging and seeing
what's going on "under the hood". A line starting with '>' means "header data"
sent by curl, '<' means "header data" received by curl that is hidden in
normal cases, and a line starting with '*' means additional info provided by
curl.
If you only want HTTP headers in the output, \fI-i, --include\fP might be the option
you're looking for.
If you think this option still doesn't give you enough details, consider using
\fI--trace\fP or \fI--trace-ascii\fP instead.
Use \fI-s, --silent\fP to make curl really quiet.
See also \fI-i, --include\fP. This option overrides \fI--trace\fP and \fI--trace-ascii\fP.
.IP "-V, --version"
Displays information about curl and the libcurl version it uses.
The first line includes the full version of curl, libcurl and other 3rd party
libraries linked with the executable.
The second line (starts with "Protocols:") shows all protocols that libcurl
reports to support.
The third line (starts with "Features:") shows specific features libcurl
reports to offer. Available features include:
.RS
.IP "IPv6"
You can use IPv6 with this.
.IP "krb4"
Krb4 for FTP is supported.
.IP "SSL"
SSL versions of various protocols are supported, such as HTTPS, FTPS, POP3S
and so on.
.IP "libz"
Automatic decompression of compressed files over HTTP is supported.
.IP "NTLM"
NTLM authentication is supported.
.IP "Debug"
This curl uses a libcurl built with Debug. This enables more error-tracking
and memory debugging etc. For curl-developers only!
.IP "AsynchDNS"
This curl uses asynchronous name resolves. Asynchronous name resolves can be
done using either the c-ares or the threaded resolver backends.
.IP "SPNEGO"
SPNEGO authentication is supported.
.IP "Largefile"
This curl supports transfers of large files, files larger than 2GB.
.IP "IDN"
This curl supports IDN - international domain names.
.IP "GSS-API"
GSS-API is supported.
.IP "SSPI"
SSPI is supported.
.IP "TLS-SRP"
SRP (Secure Remote Password) authentication is supported for TLS.
.IP "HTTP2"
HTTP/2 support has been built-in.
.IP "UnixSockets"
Unix sockets support is provided.
.IP "HTTPS-proxy"
This curl is built to support HTTPS proxy.
.IP "Metalink"
This curl supports Metalink (both version 3 and 4 (RFC 5854)), which
describes mirrors and hashes. curl will use mirrors for failover if
there are errors (such as the file or server not being available).
.IP "PSL"
PSL is short for Public Suffix List and means that this curl has been built
with knowledge about "public suffixes".
.IP "MultiSSL"
This curl supports multiple TLS backends.
.RE
.IP "-w, --write-out <format>"
Make curl display information on stdout after a completed transfer. The format
is a string that may contain plain text mixed with any number of
variables. The format can be specified as a literal "string", or you can have
curl read the format from a file with "@filename" and to tell curl to read the
format from stdin you write "@-".
The variables present in the output format will be substituted by the value or
text that curl thinks fit, as described below. All variables are specified as
%{variable_name} and to output a normal % you just write them as %%. You can
output a newline by using \\n, a carriage return with \\r and a tab space with
\\t.
The output will be written to standard output, but this can be switched to
standard error by using %{stderr}.
.B NOTE:
The %-symbol is a special symbol in the win32-environment, where all
occurrences of % must be doubled when using this option.
The variables available are:
.RS
.TP 15
.B content_type
The Content-Type of the requested document, if there was any.
.TP
.B filename_effective
The ultimate filename that curl writes out to. This is only meaningful if curl
is told to write to a file with the \fI-O, --remote-name\fP or \fI-o, --output\fP
option. It's most useful in combination with the \fI-J, --remote-header-name\fP
option. (Added in 7.26.0)
.TP
.B ftp_entry_path
The initial path curl ended up in when logging on to the remote FTP
server. (Added in 7.15.4)
.TP
.B http_code
The numerical response code that was found in the last retrieved HTTP(S) or
FTP(s) transfer. In 7.18.2 the alias \fBresponse_code\fP was added to show the
same info.
.TP
.B http_connect
The numerical code that was found in the last response (from a proxy) to a
curl CONNECT request. (Added in 7.12.4)
.TP
.B http_version
The http version that was effectively used. (Added in 7.50.0)
.TP
.B local_ip
The IP address of the local end of the most recently done connection - can be
either IPv4 or IPv6 (Added in 7.29.0)
.TP
.B local_port
The local port number of the most recently done connection (Added in 7.29.0)
.TP
.B num_connects
Number of new connects made in the recent transfer. (Added in 7.12.3)
.TP
.B num_redirects
Number of redirects that were followed in the request. (Added in 7.12.3)
.TP
.B proxy_ssl_verify_result
The result of the HTTPS proxy's SSL peer certificate verification that was
requested. 0 means the verification was successful. (Added in 7.52.0)
.TP
.B redirect_url
When an HTTP request was made without \fI-L, --location\fP to follow redirects (or when
--max-redir is met), this variable will show the actual URL a redirect
\fIwould\fP have gone to. (Added in 7.18.2)
.TP
.B remote_ip
The remote IP address of the most recently done connection - can be either
IPv4 or IPv6 (Added in 7.29.0)
.TP
.B remote_port
The remote port number of the most recently done connection (Added in 7.29.0)
.TP
.B scheme
The URL scheme (sometimes called protocol) that was effectively used (Added in 7.52.0)
.TP
.B size_download
The total amount of bytes that were downloaded.
.TP
.B size_header
The total amount of bytes of the downloaded headers.
.TP
.B size_request
The total amount of bytes that were sent in the HTTP request.
.TP
.B size_upload
The total amount of bytes that were uploaded.
.TP
.B speed_download
The average download speed that curl measured for the complete download. Bytes
per second.
.TP
.B speed_upload
The average upload speed that curl measured for the complete upload. Bytes per
second.
.TP
.B ssl_verify_result
The result of the SSL peer certificate verification that was requested. 0
means the verification was successful. (Added in 7.19.0)
.TP
.B stderr
From this point on, the \fI-w, --write-out\fP output will be written to standard
error. (Added in 7.63.0)
.TP
.B stdout
From this point on, the \fI-w, --write-out\fP output will be written to standard output.
This is the default, but can be used to switch back after switching to stderr.
(Added in 7.63.0)
.TP
.B time_appconnect
The time, in seconds, it took from the start until the SSL/SSH/etc
connect/handshake to the remote host was completed. (Added in 7.19.0)
.TP
.B time_connect
The time, in seconds, it took from the start until the TCP connect to the
remote host (or proxy) was completed.
.TP
.B time_namelookup
The time, in seconds, it took from the start until the name resolving was
completed.
.TP
.B time_pretransfer
The time, in seconds, it took from the start until the file transfer was just
about to begin. This includes all pre-transfer commands and negotiations that
are specific to the particular protocol(s) involved.
.TP
.B time_redirect
The time, in seconds, it took for all redirection steps including name lookup,
connect, pretransfer and transfer before the final transaction was
started. time_redirect shows the complete execution time for multiple
redirections. (Added in 7.12.3)
.TP
.B time_starttransfer
The time, in seconds, it took from the start until the first byte was just
about to be transferred. This includes time_pretransfer and also the time the
server needed to calculate the result.
.TP
.B time_total
The total time, in seconds, that the full operation lasted.
.TP
.B url_effective
The URL that was fetched last. This is most meaningful if you've told curl
to follow location: headers.
.RE
.IP
If this option is used several times, the last one will be used.
.IP "--xattr"
When saving output to a file, this option tells curl to store certain file
metadata in extended file attributes. Currently, the URL is stored in the
xdg.origin.url attribute and, for HTTP, the content type is stored in
the mime_type attribute. If the file system does not support extended
attributes, a warning is issued.
.SH FILES
.I ~/.curlrc
.RS
Default config file, see \fI-K, --config\fP for details.
.SH ENVIRONMENT
The environment variables can be specified in lower case or upper case. The
lower case version has precedence. http_proxy is an exception as it is only
available in lower case.
Using an environment variable to set the proxy has the same effect as using
the \fI-x, --proxy\fP option.
.IP "http_proxy [protocol://]<host>[:port]"
Sets the proxy server to use for HTTP.
.IP "HTTPS_PROXY [protocol://]<host>[:port]"
Sets the proxy server to use for HTTPS.
.IP "[url-protocol]_PROXY [protocol://]<host>[:port]"
Sets the proxy server to use for [url-protocol], where the protocol is a
protocol that curl supports and as specified in a URL. FTP, FTPS, POP3, IMAP,
SMTP, LDAP etc.
.IP "ALL_PROXY [protocol://]<host>[:port]"
Sets the proxy server to use if no protocol-specific proxy is set.
.IP "NO_PROXY <comma-separated list of hosts/domains>"
list of host names that shouldn't go through any proxy. If set to an asterisk
\&'*' only, it matches all hosts. Each name in this list is matched as either
a domain name which contains the hostname, or the hostname itself.
This environment variable disables use of the proxy even when specified with
the \fI-x, --proxy\fP option. That is
.B NO_PROXY=direct.example.com curl -x http://proxy.example.com
.B http://direct.example.com
accesses the target URL directly, and
.B NO_PROXY=direct.example.com curl -x http://proxy.example.com
.B http://somewhere.example.com
accesses the target URL through the proxy.
The list of host names can also be include numerical IP addresses, and IPv6
versions should then be given without enclosing brackets.
.SH "PROXY PROTOCOL PREFIXES"
Since curl version 7.21.7, the proxy string may be specified with a
protocol:// prefix to specify alternative proxy protocols.
If no protocol is specified in the proxy string or if the string doesn't match
a supported one, the proxy will be treated as an HTTP proxy.
The supported proxy protocol prefixes are as follows:
.IP "http://"
Makes it use it as an HTTP proxy. The default if no scheme prefix is used.
.IP "https://"
Makes it treated as an \fBHTTPS\fP proxy.
.IP "socks4://"
Makes it the equivalent of \fI--socks4\fP
.IP "socks4a://"
Makes it the equivalent of \fI--socks4a\fP
.IP "socks5://"
Makes it the equivalent of \fI--socks5\fP
.IP "socks5h://"
Makes it the equivalent of \fI--socks5-hostname\fP
.SH EXIT CODES
There are a bunch of different error codes and their corresponding error
messages that may appear during bad conditions. At the time of this writing,
the exit codes are:
.IP 1
Unsupported protocol. This build of curl has no support for this protocol.
.IP 2
Failed to initialize.
.IP 3
URL malformed. The syntax was not correct.
.IP 4
A feature or option that was needed to perform the desired request was not
enabled or was explicitly disabled at build-time. To make curl able to do
this, you probably need another build of libcurl!
.IP 5
Couldn't resolve proxy. The given proxy host could not be resolved.
.IP 6
Couldn't resolve host. The given remote host was not resolved.
.IP 7
Failed to connect to host.
.IP 8
Weird server reply. The server sent data curl couldn't parse.
.IP 9
FTP access denied. The server denied login or denied access to the particular
resource or directory you wanted to reach. Most often you tried to change to a
directory that doesn't exist on the server.
.IP 10
FTP accept failed. While waiting for the server to connect back when an active
FTP session is used, an error code was sent over the control connection or
similar.
.IP 11
FTP weird PASS reply. Curl couldn't parse the reply sent to the PASS request.
.IP 12
During an active FTP session while waiting for the server to connect back to
curl, the timeout expired.
.IP 13
FTP weird PASV reply, Curl couldn't parse the reply sent to the PASV request.
.IP 14
FTP weird 227 format. Curl couldn't parse the 227-line the server sent.
.IP 15
FTP can't get host. Couldn't resolve the host IP we got in the 227-line.
.IP 16
HTTP/2 error. A problem was detected in the HTTP2 framing layer. This is
somewhat generic and can be one out of several problems, see the error message
for details.
.IP 17
FTP couldn't set binary. Couldn't change transfer method to binary.
.IP 18
Partial file. Only a part of the file was transferred.
.IP 19
FTP couldn't download/access the given file, the RETR (or similar) command
failed.
.IP 21
FTP quote error. A quote command returned error from the server.
.IP 22
HTTP page not retrieved. The requested url was not found or returned another
error with the HTTP error code being 400 or above. This return code only
appears if \fI-f, --fail\fP is used.
.IP 23
Write error. Curl couldn't write data to a local filesystem or similar.
.IP 25
FTP couldn't STOR file. The server denied the STOR operation, used for FTP
uploading.
.IP 26
Read error. Various reading problems.
.IP 27
Out of memory. A memory allocation request failed.
.IP 28
Operation timeout. The specified time-out period was reached according to the
conditions.
.IP 30
FTP PORT failed. The PORT command failed. Not all FTP servers support the PORT
command, try doing a transfer using PASV instead!
.IP 31
FTP couldn't use REST. The REST command failed. This command is used for
resumed FTP transfers.
.IP 33
HTTP range error. The range "command" didn't work.
.IP 34
HTTP post error. Internal post-request generation error.
.IP 35
SSL connect error. The SSL handshaking failed.
.IP 36
Bad download resume. Couldn't continue an earlier aborted download.
.IP 37
FILE couldn't read file. Failed to open the file. Permissions?
.IP 38
LDAP cannot bind. LDAP bind operation failed.
.IP 39
LDAP search failed.
.IP 41
Function not found. A required LDAP function was not found.
.IP 42
Aborted by callback. An application told curl to abort the operation.
.IP 43
Internal error. A function was called with a bad parameter.
.IP 45
Interface error. A specified outgoing interface could not be used.
.IP 47
Too many redirects. When following redirects, curl hit the maximum amount.
.IP 48
Unknown option specified to libcurl. This indicates that you passed a weird
option to curl that was passed on to libcurl and rejected. Read up in the
manual!
.IP 49
Malformed telnet option.
.IP 51
The peer's SSL certificate or SSH MD5 fingerprint was not OK.
.IP 52
The server didn't reply anything, which here is considered an error.
.IP 53
SSL crypto engine not found.
.IP 54
Cannot set SSL crypto engine as default.
.IP 55
Failed sending network data.
.IP 56
Failure in receiving network data.
.IP 58
Problem with the local certificate.
.IP 59
Couldn't use specified SSL cipher.
.IP 60
Peer certificate cannot be authenticated with known CA certificates.
.IP 61
Unrecognized transfer encoding.
.IP 62
Invalid LDAP URL.
.IP 63
Maximum file size exceeded.
.IP 64
Requested FTP SSL level failed.
.IP 65
Sending the data requires a rewind that failed.
.IP 66
Failed to initialise SSL Engine.
.IP 67
The user name, password, or similar was not accepted and curl failed to log in.
.IP 68
File not found on TFTP server.
.IP 69
Permission problem on TFTP server.
.IP 70
Out of disk space on TFTP server.
.IP 71
Illegal TFTP operation.
.IP 72
Unknown TFTP transfer ID.
.IP 73
File already exists (TFTP).
.IP 74
No such user (TFTP).
.IP 75
Character conversion failed.
.IP 76
Character conversion functions required.
.IP 77
Problem with reading the SSL CA cert (path? access rights?).
.IP 78
The resource referenced in the URL does not exist.
.IP 79
An unspecified error occurred during the SSH session.
.IP 80
Failed to shut down the SSL connection.
.IP 82
Could not load CRL file, missing or wrong format (added in 7.19.0).
.IP 83
Issuer check failed (added in 7.19.0).
.IP 84
The FTP PRET command failed
.IP 85
RTSP: mismatch of CSeq numbers
.IP 86
RTSP: mismatch of Session Identifiers
.IP 87
unable to parse FTP file list
.IP 88
FTP chunk callback reported error
.IP 89
No connection available, the session will be queued
.IP 90
SSL public key does not matched pinned public key
.IP 91
Invalid SSL certificate status.
.IP 92
Stream error in HTTP/2 framing layer.
.IP XX
More error codes will appear here in future releases. The existing ones
are meant to never change.
.SH AUTHORS / CONTRIBUTORS
Daniel Stenberg is the main author, but the whole list of contributors is
found in the separate THANKS file.
.SH WWW
https://curl.haxx.se
.SH "SEE ALSO"
.BR ftp (1),
.BR wget (1)
usr/share/doc/alt-curlssl/FAQ 0000644 00000202713 15116224004 0012023 0 ustar 00 _ _ ____ _
___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
FAQ
1. Philosophy
1.1 What is cURL?
1.2 What is libcurl?
1.3 What is curl not?
1.4 When will you make curl do XXXX ?
1.5 Who makes curl?
1.6 What do you get for making curl?
1.7 What about CURL from curl.com?
1.8 I have a problem who do I mail?
1.9 Where do I buy commercial support for curl?
1.10 How many are using curl?
1.11 Why don't you update ca-bundle.crt
1.12 I have a problem who can I chat with?
1.13 curl's ECCN number?
1.14 How do I submit my patch?
1.15 How do I port libcurl to my OS?
2. Install Related Problems
2.1 configure doesn't find OpenSSL even when it is installed
2.1.1 native linker doesn't find OpenSSL
2.1.2 only the libssl lib is missing
2.2 Does curl work/build with other SSL libraries?
2.3 Where can I find a copy of LIBEAY32.DLL?
2.4 Does curl support SOCKS (RFC 1928) ?
3. Usage Problems
3.1 curl: (1) SSL is disabled, https: not supported
3.2 How do I tell curl to resume a transfer?
3.3 Why doesn't my posting using -F work?
3.4 How do I tell curl to run custom FTP commands?
3.5 How can I disable the Accept: */* header?
3.6 Does curl support ASP, XML, XHTML or HTML version Y?
3.7 Can I use curl to delete/rename a file through FTP?
3.8 How do I tell curl to follow HTTP redirects?
3.9 How do I use curl in my favorite programming language?
3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
3.11 How do I POST with a different Content-Type?
3.12 Why do FTP specific features over HTTP proxy fail?
3.13 Why does my single/double quotes fail?
3.14 Does curl support Javascript or PAC (automated proxy config)?
3.15 Can I do recursive fetches with curl?
3.16 What certificates do I need when I use SSL?
3.17 How do I list the root dir of an FTP server?
3.18 Can I use curl to send a POST/PUT and not wait for a response?
3.19 How do I get HTTP from a host using a specific IP address?
3.20 How to SFTP from my user's home directory?
3.21 Protocol xxx not supported or disabled in libcurl
3.22 curl -X gives me HTTP problems
4. Running Problems
4.1 Problems connecting to SSL servers.
4.2 Why do I get problems when I use & or % in the URL?
4.3 How can I use {, }, [ or ] to specify multiple URLs?
4.4 Why do I get downloaded data even though the web page doesn't exist?
4.5 Why do I get return code XXX from a HTTP server?
4.5.1 "400 Bad Request"
4.5.2 "401 Unauthorized"
4.5.3 "403 Forbidden"
4.5.4 "404 Not Found"
4.5.5 "405 Method Not Allowed"
4.5.6 "301 Moved Permanently"
4.6 Can you tell me what error code 142 means?
4.7 How do I keep user names and passwords secret in Curl command lines?
4.8 I found a bug!
4.9 Curl can't authenticate to the server that requires NTLM?
4.10 My HTTP request using HEAD, PUT or DELETE doesn't work!
4.11 Why does my HTTP range requests return the full document?
4.12 Why do I get "certificate verify failed" ?
4.13 Why is curl -R on Windows one hour off?
4.14 Redirects work in browser but not with curl!
4.15 FTPS doesn't work
4.16 My HTTP POST or PUT requests are slow!
4.17 Non-functional connect timeouts on Windows
4.18 file:// URLs containing drive letters (Windows, NetWare)
4.19 Why doesn't curl return an error when the network cable is unplugged?
4.20 curl doesn't return error for HTTP non-200 responses!
4.21 Why is there a HTTP/1.1 in my HTTP/2 request?
5. libcurl Issues
5.1 Is libcurl thread-safe?
5.2 How can I receive all data into a large memory chunk?
5.3 How do I fetch multiple files with libcurl?
5.4 Does libcurl do Winsock initing on win32 systems?
5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
5.6 What about Keep-Alive or persistent connections?
5.7 Link errors when building libcurl on Windows!
5.8 libcurl.so.X: open failed: No such file or directory
5.9 How does libcurl resolve host names?
5.10 How do I prevent libcurl from writing the response to stdout?
5.11 How do I make libcurl not receive the whole HTTP response?
5.12 Can I make libcurl fake or hide my real IP address?
5.13 How do I stop an ongoing transfer?
5.14 Using C++ non-static functions for callbacks?
5.15 How do I get an FTP directory listing?
5.16 I want a different time-out!
5.17 Can I write a server with libcurl?
5.18 Does libcurl use threads?
6. License Issues
6.1 I have a GPL program, can I use the libcurl library?
6.2 I have a closed-source program, can I use the libcurl library?
6.3 I have a BSD licensed program, can I use the libcurl library?
6.4 I have a program that uses LGPL libraries, can I use libcurl?
6.5 Can I modify curl/libcurl for my program and keep the changes secret?
6.6 Can you please change the curl/libcurl license to XXXX?
6.7 What are my obligations when using libcurl in my commercial apps?
7. PHP/CURL Issues
7.1 What is PHP/CURL?
7.2 Who wrote PHP/CURL?
7.3 Can I perform multiple requests using the same handle?
7.4 Does PHP/CURL have dependencies?
==============================================================================
1. Philosophy
1.1 What is cURL?
cURL is the name of the project. The name is a play on 'Client for URLs',
originally with URL spelled in uppercase to make it obvious it deals with
URLs. The fact it can also be pronounced 'see URL' also helped, it works as
an abbreviation for "Client URL Request Library" or why not the recursive
version: "Curl URL Request Library".
The cURL project produces two products:
libcurl
A free and easy-to-use client-side URL transfer library, supporting DICT,
FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3,
POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET and TFTP.
libcurl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading,
Kerberos, SPNEGO, HTTP form based upload, proxies, cookies, user+password
authentication, file transfer resume, http proxy tunneling and more!
libcurl is highly portable, it builds and works identically on numerous
platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HP-UX,
IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, BeOS, Mac
OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS, Symbian, OSF,
Android, Minix, IBM TPF and more...
libcurl is free, thread-safe, IPv6 compatible, feature rich, well
supported and fast.
curl
A command line tool for getting or sending files using URL syntax.
Since curl uses libcurl, curl supports the same wide range of common
Internet protocols that libcurl does.
We pronounce curl with an initial k sound. It rhymes with words like girl
and earl. This is a short WAV file to help you:
https://media.merriam-webster.com/soundc11/c/curl0001.wav
There are numerous sub-projects and related projects that also use the word
curl in the project names in various combinations, but you should take
notice that this FAQ is directed at the command-line tool named curl (and
libcurl the library), and may therefore not be valid for other curl-related
projects. (There is however a small section for the PHP/CURL in this FAQ.)
1.2 What is libcurl?
libcurl is a reliable and portable library which provides you with an easy
interface to a range of common Internet protocols.
You can use libcurl for free in your application, be it open source,
commercial or closed-source.
libcurl is most probably the most portable, most powerful and most often
used C-based multi-platform file transfer library on this planet - be it
open source or commercial.
1.3 What is curl not?
Curl is not a wget clone. That is a common misconception. Never, during
curl's development, have we intended curl to replace wget or compete on its
market. Curl is targeted at single-shot file transfers.
Curl is not a web site mirroring program. If you want to use curl to mirror
something: fine, go ahead and write a script that wraps around curl to make
it reality (like curlmirror.pl does).
Curl is not an FTP site mirroring program. Sure, get and send FTP with curl
but if you want systematic and sequential behavior you should write a
script (or write a new program that interfaces libcurl) and do it.
Curl is not a PHP tool, even though it works perfectly well when used from
or with PHP (when using the PHP/CURL module).
Curl is not a program for a single operating system. Curl exists, compiles,
builds and runs under a wide range of operating systems, including all
modern Unixes (and a bunch of older ones too), Windows, Amiga, BeOS, OS/2,
OS X, QNX etc.
1.4 When will you make curl do XXXX ?
We love suggestions of what to change in order to make curl and libcurl
better. We do however believe in a few rules when it comes to the future of
curl:
Curl -- the command line tool -- is to remain a non-graphical command line
tool. If you want GUIs or fancy scripting capabilities, you should look for
another tool that uses libcurl.
We do not add things to curl that other small and available tools already do
very well at the side. Curl's output can be piped into another program or
redirected to another file for the next program to interpret.
We focus on protocol related issues and improvements. If you want to do more
magic with the supported protocols than curl currently does, chances are good
we will agree. If you want to add more protocols, we may very well agree.
If you want someone else to do all the work while you wait for us to
implement it for you, that is not a very friendly attitude. We spend a
considerable time already on maintaining and developing curl. In order to
get more out of us, you should consider trading in some of your time and
effort in return. Simply go to the GitHub repo which resides at
https://github.com/curl/curl, fork the project, and create pull requests
with your proposed changes.
If you write the code, chances are better that it will get into curl faster.
1.5 Who makes curl?
curl and libcurl are not made by any single individual. Daniel Stenberg is
project leader and main developer, but other persons' submissions are
important and crucial. Anyone can contribute and post their changes and
improvements and have them inserted in the main sources (of course on the
condition that developers agree that the fixes are good).
The full list of all contributors is found in the docs/THANKS file.
curl is developed by a community, with Daniel at the wheel.
1.6 What do you get for making curl?
Project cURL is entirely free and open. No person gets paid for developing
curl full time. We do this voluntarily, mostly in our spare time.
Occasionally companies pay individual developers to work on curl, but that's
up to each company and developer. This is not controlled by nor supervised in
any way by the project.
We still get help from companies. Haxx provides web site, bandwidth, mailing
lists etc, GitHub hosts the primary git repository and other services like
the bug tracker at https://github.com/curl/curl. Also again, some companies
have sponsored certain parts of the development in the past and I hope some
will continue to do so in the future.
If you want to support our project, consider a donation or a banner-program
or even better: by helping us with coding, documenting or testing etc.
1.7 What about CURL from curl.com?
During the summer of 2001, curl.com was busy advertising their client-side
programming language for the web, named CURL.
We are in no way associated with curl.com or their CURL programming
language.
Our project name curl has been in effective use since 1998. We were not the
first computer related project to use the name "curl" and do not claim any
rights to the name.
We recognize that we will be living in parallel with curl.com and wish them
every success.
1.8 I have a problem whom do I mail?
Please do not mail any single individual unless you really need to. Keep
curl-related questions on a suitable mailing list. All available mailing
lists are listed in the MANUAL document and online at
https://curl.haxx.se/mail/
Keeping curl-related questions and discussions on mailing lists allows
others to join in and help, to share their ideas, to contribute their
suggestions and to spread their wisdom. Keeping discussions on public mailing
lists also allows for others to learn from this (both current and future
users thanks to the web based archives of the mailing lists), thus saving us
from having to repeat ourselves even more. Thanks for respecting this.
If you have found or simply suspect a security problem in curl or libcurl,
mail curl-security at haxx.se (closed list of receivers, mails are not
disclosed) and tell. Then we can produce a fix in a timely manner before the
flaw is announced to the world, thus lessen the impact the problem will have
on existing users.
1.9 Where do I buy commercial support for curl?
curl is fully open source. It means you can hire any skilled engineer to fix
your curl-related problems.
We list available alternatives on the curl web site:
https://curl.haxx.se/support.html
1.10 How many are using curl?
It is impossible to tell.
We don't know how many users that knowingly have installed and use curl.
We don't know how many users that use curl without knowing that they are in
fact using it.
We don't know how many users that downloaded or installed curl and then
never use it.
In May 2012 Daniel did a counting game and came up with a number that may
be completely wrong or somewhat accurate. Over 500 million!
See https://daniel.haxx.se/blog/2012/05/16/300m-users/
1.11 Why don't you update ca-bundle.crt
The ca cert bundle that used to be shipped with curl was very outdated and
must be replaced with an up-to-date version by anyone who wants to verify
peers. It is no longer provided by curl. The last curl release that ever
shipped a ca cert bundle was curl 7.18.0.
In the cURL project we've decided not to attempt to keep this file updated
(or even present anymore) since deciding what to add to a ca cert bundle is
an undertaking we've not been ready to accept, and the one we can get from
Mozilla is perfectly fine so there's no need to duplicate that work.
Today, with many services performed over HTTPS, every operating system
should come with a default ca cert bundle that can be deemed somewhat
trustworthy and that collection (if reasonably updated) should be deemed to
be a lot better than a private curl version.
If you want the most recent collection of ca certs that Mozilla Firefox
uses, we recommend that you extract the collection yourself from Mozilla
Firefox (by running 'make ca-bundle), or by using our online service setup
for this purpose: https://curl.haxx.se/docs/caextract.html
1.12 I have a problem who can I chat with?
There's a bunch of friendly people hanging out in the #curl channel on the
IRC network irc.freenode.net. If you're polite and nice, chances are good
that you can get -- or provide -- help instantly.
1.13 curl's ECCN number?
The US government restricts exports of software that contains or uses
cryptography. When doing so, the Export Control Classification Number (ECCN)
is used to identify the level of export control etc.
Apache Software Foundation gives a good explanation of ECCNs at
https://www.apache.org/dev/crypto.html
We believe curl's number might be ECCN 5D002, another possibility is
5D992. It seems necessary to write them (the authority that administers ECCN
numbers), asking to confirm.
Comprehensible explanations of the meaning of such numbers and how to obtain
them (resp.) are here
https://www.bis.doc.gov/licensing/exportingbasics.htm
https://www.bis.doc.gov/licensing/do_i_needaneccn.html
An incomprehensible description of the two numbers above is here
https://www.bis.doc.gov/index.php/documents/new-encryption/1653-ccl5-pt2-3
1.14 How do I submit my patch?
When you have made a patch or a change of whatever sort, and want to submit
that to the project, there are a few different ways we prefer:
o send a patch to the curl-library mailing list. We're many subscribers
there and there are lots of people who can review patches, comment on them
and "receive" them properly.
o if your patch changes or fixes a bug, you can also opt to submit a bug
report in the bug tracker and attach your patch there. There are less
people involved there.
Lots of more details are found in the CONTRIBUTE and INTERNALS docs.
1.15 How do I port libcurl to my OS?
Here's a rough step-by-step:
1. copy a suitable lib/config-*.h file as a start to lib/config-[youros].h
2. edit lib/config-[youros].h to match your OS and setup
3. edit lib/curl_setup.h to include config-[youros].h when your OS is
detected by the preprocessor, in the style others already exist
4. compile lib/*.c and make them into a library
2. Install Related Problems
2.1 configure doesn't find OpenSSL even when it is installed
This may be because of several reasons.
2.1.1 native linker doesn't find openssl
Affected platforms:
Solaris (native cc compiler)
HPUX (native cc compiler)
SGI IRIX (native cc compiler)
SCO UNIX (native cc compiler)
When configuring curl, I specify --with-ssl. OpenSSL is installed in
/usr/local/ssl Configure reports SSL in /usr/local/ssl, but fails to find
CRYPTO_lock in -lcrypto
Cause: The cc for this test places the -L/usr/local/ssl/lib AFTER
-lcrypto, so ld can't find the library. This is due to a bug in the GNU
autoconf tool.
Workaround: Specifying "LDFLAGS=-L/usr/local/ssl/lib" in front of
./configure places the -L/usr/local/ssl/lib early enough in the command
line to make things work
2.1.2 only the libssl lib is missing
If all include files and the libcrypto lib is present, with only the
libssl being missing according to configure, this is most likely because
a few functions are left out from the libssl.
If the function names missing include RSA or RSAREF you can be certain
that this is because libssl requires the RSA and RSAREF libs to build.
See the INSTALL file section that explains how to add those libs to
configure. Make sure that you remove the config.cache file before you
rerun configure with the new flags.
2.2 Does curl work/build with other SSL libraries?
Curl has been written to use a generic SSL function layer internally, and
that SSL functionality can then be provided by one out of many different SSL
backends.
curl can be built to use one of the following SSL alternatives: OpenSSL,
GnuTLS, yassl, NSS, PolarSSL, MesaLink, Secure Transport (native iOS/OS X),
WinSSL (native Windows) or GSKit (native IBM i). They all have their pros
and cons, and we try to maintain a comparison of them here:
https://curl.haxx.se/docs/ssl-compared.html
2.3 Where can I find a copy of LIBEAY32.DLL?
That is an OpenSSL binary built for Windows.
Curl can be built with OpenSSL to do the SSL stuff. The LIBEAY32.DLL is then
what curl needs on a windows machine to do https:// etc. Check out the curl
web site to find accurate and up-to-date pointers to recent OpenSSL DLLs and
other binary packages.
2.4 Does curl support SOCKS (RFC 1928) ?
Yes, SOCKS 4 and 5 are supported.
3. Usage problems
3.1 curl: (1) SSL is disabled, https: not supported
If you get this output when trying to get anything from a https:// server,
it means that the instance of curl/libcurl that you're using was built
without support for this protocol.
This could've happened if the configure script that was run at build time
couldn't find all libs and include files curl requires for SSL to work. If
the configure script fails to find them, curl is simply built without SSL
support.
To get the https:// support into a curl that was previously built but that
reports that https:// is not supported, you should dig through the document
and logs and check out why the configure script doesn't find the SSL libs
and/or include files.
Also, check out the other paragraph in this FAQ labelled "configure doesn't
find OpenSSL even when it is installed".
3.2 How do I tell curl to resume a transfer?
Curl supports resumed transfers both ways on both FTP and HTTP.
Try the -C option.
3.3 Why doesn't my posting using -F work?
You can't arbitrarily use -F or -d, the choice between -F or -d depends on the
HTTP operation you need curl to do and what the web server that will receive
your post expects.
If the form you're trying to submit uses the type 'multipart/form-data', then
and only then you must use the -F type. In all the most common cases, you
should use -d which then causes a posting with the type
'application/x-www-form-urlencoded'.
This is described in some detail in the MANUAL and TheArtOfHttpScripting
documents, and if you don't understand it the first time, read it again
before you post questions about this to the mailing list. Also, try reading
through the mailing list archives for old postings and questions regarding
this.
3.4 How do I tell curl to run custom FTP commands?
You can tell curl to perform optional commands both before and/or after a
file transfer. Study the -Q/--quote option.
Since curl is used for file transfers, you don't normally use curl to
perform FTP commands without transferring anything. Therefore you must
always specify a URL to transfer to/from even when doing custom FTP
commands, or use -I which implies the "no body" option sent to libcurl.
3.5 How can I disable the Accept: */* header?
You can change all internally generated headers by adding a replacement with
the -H/--header option. By adding a header with empty contents you safely
disable that one. Use -H "Accept:" to disable that specific header.
3.6 Does curl support ASP, XML, XHTML or HTML version Y?
To curl, all contents are alike. It doesn't matter how the page was
generated. It may be ASP, PHP, Perl, shell-script, SSI or plain HTML
files. There's no difference to curl and it doesn't even know what kind of
language that generated the page.
See also item 3.14 regarding javascript.
3.7 Can I use curl to delete/rename a file through FTP?
Yes. You specify custom FTP commands with -Q/--quote.
One example would be to delete a file after you have downloaded it:
curl -O ftp://download.com/coolfile -Q '-DELE coolfile'
or rename a file after upload:
curl -T infile ftp://upload.com/dir/ -Q "-RNFR infile" -Q "-RNTO newname"
3.8 How do I tell curl to follow HTTP redirects?
Curl does not follow so-called redirects by default. The Location: header
that informs the client about this is only interpreted if you're using the
-L/--location option. As in:
curl -L http://redirector.com
Not all redirects are HTTP ones, see 4.14
3.9 How do I use curl in my favorite programming language?
There exist many language interfaces/bindings for curl that integrates it
better with various languages. If you are fluid in a script language, you
may very well opt to use such an interface instead of using the command line
tool.
Find out more about which languages that support curl directly, and how to
install and use them, in the libcurl section of the curl web site:
https://curl.haxx.se/libcurl/
All the various bindings to libcurl are made by other projects and people,
outside of the cURL project. The cURL project itself only produces libcurl
with its plain C API. If you don't find anywhere else to ask you can ask
about bindings on the curl-library list too, but be prepared that people on
that list may not know anything about bindings.
In October 2009, there were interfaces available for the following
languages: Ada95, Basic, C, C++, Ch, Cocoa, D, Dylan, Eiffel, Euphoria,
Ferite, Gambas, glib/GTK+, Haskell, ILE/RPG, Java, Lisp, Lua, Mono, .NET,
Object-Pascal, OCaml, Pascal, Perl, PHP, PostgreSQL, Python, R, Rexx, Ruby,
Scheme, S-Lang, Smalltalk, SP-Forth, SPL, Tcl, Visual Basic, Visual FoxPro,
Q, wxwidgets and XBLite. By the time you read this, additional ones may have
appeared!
3.10 What about SOAP, WebDAV, XML-RPC or similar protocols over HTTP?
Curl adheres to the HTTP spec, which basically means you can play with *any*
protocol that is built on top of HTTP. Protocols such as SOAP, WEBDAV and
XML-RPC are all such ones. You can use -X to set custom requests and -H to
set custom headers (or replace internally generated ones).
Using libcurl is of course just as good and you'd just use the proper
library options to do the same.
3.11 How do I POST with a different Content-Type?
You can always replace the internally generated headers with -H/--header.
To make a simple HTTP POST with text/xml as content-type, do something like:
curl -d "datatopost" -H "Content-Type: text/xml" [URL]
3.12 Why do FTP specific features over HTTP proxy fail?
Because when you use a HTTP proxy, the protocol spoken on the network will
be HTTP, even if you specify a FTP URL. This effectively means that you
normally can't use FTP specific features such as FTP upload and FTP quote
etc.
There is one exception to this rule, and that is if you can "tunnel through"
the given HTTP proxy. Proxy tunneling is enabled with a special option (-p)
and is generally not available as proxy admins usually disable tunneling to
ports other than 443 (which is used for HTTPS access through proxies).
3.13 Why does my single/double quotes fail?
To specify a command line option that includes spaces, you might need to
put the entire option within quotes. Like in:
curl -d " with spaces " url.com
or perhaps
curl -d ' with spaces ' url.com
Exactly what kind of quotes and how to do this is entirely up to the shell
or command line interpreter that you are using. For most unix shells, you
can more or less pick either single (') or double (") quotes. For
Windows/DOS prompts I believe you're forced to use double (") quotes.
Please study the documentation for your particular environment. Examples in
the curl docs will use a mix of both of these as shown above. You must
adjust them to work in your environment.
Remember that curl works and runs on more operating systems than most single
individuals have ever tried.
3.14 Does curl support Javascript or PAC (automated proxy config)?
Many web pages do magic stuff using embedded Javascript. Curl and libcurl
have no built-in support for that, so it will be treated just like any other
contents.
.pac files are a netscape invention and are sometimes used by organizations
to allow them to differentiate which proxies to use. The .pac contents is
just a Javascript program that gets invoked by the browser and that returns
the name of the proxy to connect to. Since curl doesn't support Javascript,
it can't support .pac proxy configuration either.
Some workarounds usually suggested to overcome this Javascript dependency:
Depending on the Javascript complexity, write up a script that translates it
to another language and execute that.
Read the Javascript code and rewrite the same logic in another language.
Implement a Javascript interpreter, people have successfully used the
Mozilla Javascript engine in the past.
Ask your admins to stop this, for a static proxy setup or similar.
3.15 Can I do recursive fetches with curl?
No. curl itself has no code that performs recursive operations, such as
those performed by wget and similar tools.
There exists wrapper scripts with that functionality (for example the
curlmirror perl script), and you can write programs based on libcurl to do
it, but the command line tool curl itself cannot.
3.16 What certificates do I need when I use SSL?
There are three different kinds of "certificates" to keep track of when we
talk about using SSL-based protocols (HTTPS or FTPS) using curl or libcurl.
CLIENT CERTIFICATE
The server you communicate with may require that you can provide this in
order to prove that you actually are who you claim to be. If the server
doesn't require this, you don't need a client certificate.
A client certificate is always used together with a private key, and the
private key has a pass phrase that protects it.
SERVER CERTIFICATE
The server you communicate with has a server certificate. You can and should
verify this certificate to make sure that you are truly talking to the real
server and not a server impersonating it.
CERTIFICATE AUTHORITY CERTIFICATE ("CA cert")
You often have several CA certs in a CA cert bundle that can be used to
verify a server certificate that was signed by one of the authorities in the
bundle. curl does not come with a CA cert bundle but most curl installs
provide one. You can also override the default.
The server certificate verification process is made by using a Certificate
Authority certificate ("CA cert") that was used to sign the server
certificate. Server certificate verification is enabled by default in curl
and libcurl and is often the reason for problems as explained in FAQ entry
4.12 and the SSLCERTS document
(https://curl.haxx.se/docs/sslcerts.html). Server certificates that are
"self-signed" or otherwise signed by a CA that you do not have a CA cert
for, cannot be verified. If the verification during a connect fails, you are
refused access. You then need to explicitly disable the verification to
connect to the server.
3.17 How do I list the root dir of an FTP server?
There are two ways. The way defined in the RFC is to use an encoded slash
in the first path part. List the "/tmp" dir like this:
curl ftp://ftp.sunet.se/%2ftmp/
or the not-quite-kosher-but-more-readable way, by simply starting the path
section of the URL with a slash:
curl ftp://ftp.sunet.se//tmp/
3.18 Can I use curl to send a POST/PUT and not wait for a response?
No.
But you could easily write your own program using libcurl to do such stunts.
3.19 How do I get HTTP from a host using a specific IP address?
For example, you may be trying out a web site installation that isn't yet in
the DNS. Or you have a site using multiple IP addresses for a given host
name and you want to address a specific one out of the set.
Set a custom Host: header that identifies the server name you want to reach
but use the target IP address in the URL:
curl --header "Host: www.example.com" http://127.0.0.1/
You can also opt to add faked host name entries to curl with the --resolve
option. That has the added benefit that things like redirects will also work
properly. The above operation would instead be done as:
curl --resolve www.example.com:80:127.0.0.1 http://www.example.com/
3.20 How to SFTP from my user's home directory?
Contrary to how FTP works, SFTP and SCP URLs specify the exact directory to
work with. It means that if you don't specify that you want the user's home
directory, you get the actual root directory.
To specify a file in your user's home directory, you need to use the correct
URL syntax which for sftp might look similar to:
curl -O -u user:password sftp://example.com/~/file.txt
and for SCP it is just a different protocol prefix:
curl -O -u user:password scp://example.com/~/file.txt
3.21 Protocol xxx not supported or disabled in libcurl
When passing on a URL to curl to use, it may respond that the particular
protocol is not supported or disabled. The particular way this error message
is phrased is because curl doesn't make a distinction internally of whether
a particular protocol is not supported (i.e. never got any code added that
knows how to speak that protocol) or if it was explicitly disabled. curl can
be built to only support a given set of protocols, and the rest would then
be disabled or not supported.
Note that this error will also occur if you pass a wrongly spelled protocol
part as in "htpt://example.com" or as in the less evident case if you prefix
the protocol part with a space as in " http://example.com/".
3.22 curl -X gives me HTTP problems
In normal circumstances, -X should hardly ever be used.
By default you use curl without explicitly saying which request method to
use when the URL identifies a HTTP transfer. If you just pass in a URL like
"curl http://example.com" it will use GET. If you use -d or -F curl will use
POST, -I will cause a HEAD and -T will make it a PUT.
If for whatever reason you're not happy with these default choices that curl
does for you, you can override those request methods by specifying -X
[WHATEVER]. This way you can for example send a DELETE by doing "curl -X
DELETE [URL]".
It is thus pointless to do "curl -XGET [URL]" as GET would be used
anyway. In the same vein it is pointless to do "curl -X POST -d data
[URL]"... But you can make a fun and somewhat rare request that sends a
request-body in a GET request with something like "curl -X GET -d data
[URL]"
Note that -X doesn't actually change curl's behavior as it only modifies the
actual string sent in the request, but that may of course trigger a
different set of events.
Accordingly, by using -XPOST on a command line that for example would follow
a 303 redirect, you will effectively prevent curl from behaving
correctly. Be aware.
4. Running Problems
4.1 Problems connecting to SSL servers.
It took a very long time before we could sort out why curl had problems to
connect to certain SSL servers when using SSLeay or OpenSSL v0.9+. The
error sometimes showed up similar to:
16570:error:1407D071:SSL routines:SSL2_READ:bad mac decode:s2_pkt.c:233:
It turned out to be because many older SSL servers don't deal with SSLv3
requests properly. To correct this problem, tell curl to select SSLv2 from
the command line (-2/--sslv2).
There have also been examples where the remote server didn't like the SSLv2
request and instead you had to force curl to use SSLv3 with -3/--sslv3.
4.2 Why do I get problems when I use & or % in the URL?
In general unix shells, the & symbol is treated specially and when used, it
runs the specified command in the background. To safely send the & as a part
of a URL, you should quote the entire URL by using single (') or double (")
quotes around it. Similar problems can also occur on some shells with other
characters, including ?*!$~(){}<>\|;`. When in doubt, quote the URL.
An example that would invoke a remote CGI that uses &-symbols could be:
curl 'http://www.altavista.com/cgi-bin/query?text=yes&q=curl'
In Windows, the standard DOS shell treats the percent sign specially and you
need to use TWO percent signs for each single one you want to use in the
URL.
If you want a literal percent sign to be part of the data you pass in a POST
using -d/--data you must encode it as '%25' (which then also needs the
percent sign doubled on Windows machines).
4.3 How can I use {, }, [ or ] to specify multiple URLs?
Because those letters have a special meaning to the shell, to be used in
a URL specified to curl you must quote them.
An example that downloads two URLs (sequentially) would be:
curl '{curl,www}.haxx.se'
To be able to use those characters as actual parts of the URL (without using
them for the curl URL "globbing" system), use the -g/--globoff option:
curl -g 'www.site.com/weirdname[].html'
4.4 Why do I get downloaded data even though the web page doesn't exist?
Curl asks remote servers for the page you specify. If the page doesn't exist
at the server, the HTTP protocol defines how the server should respond and
that means that headers and a "page" will be returned. That's simply how
HTTP works.
By using the --fail option you can tell curl explicitly to not get any data
if the HTTP return code doesn't say success.
4.5 Why do I get return code XXX from a HTTP server?
RFC2616 clearly explains the return codes. This is a short transcript. Go
read the RFC for exact details:
4.5.1 "400 Bad Request"
The request could not be understood by the server due to malformed
syntax. The client SHOULD NOT repeat the request without modifications.
4.5.2 "401 Unauthorized"
The request requires user authentication.
4.5.3 "403 Forbidden"
The server understood the request, but is refusing to fulfill it.
Authorization will not help and the request SHOULD NOT be repeated.
4.5.4 "404 Not Found"
The server has not found anything matching the Request-URI. No indication
is given of whether the condition is temporary or permanent.
4.5.5 "405 Method Not Allowed"
The method specified in the Request-Line is not allowed for the resource
identified by the Request-URI. The response MUST include an Allow header
containing a list of valid methods for the requested resource.
4.5.6 "301 Moved Permanently"
If you get this return code and an HTML output similar to this:
<H1>Moved Permanently</H1> The document has moved <A
HREF="http://same_url_now_with_a_trailing_slash/">here</A>.
it might be because you request a directory URL but without the trailing
slash. Try the same operation again _with_ the trailing URL, or use the
-L/--location option to follow the redirection.
4.6 Can you tell me what error code 142 means?
All curl error codes are described at the end of the man page, in the
section called "EXIT CODES".
Error codes that are larger than the highest documented error code means
that curl has exited due to a crash. This is a serious error, and we
appreciate a detailed bug report from you that describes how we could go
ahead and repeat this!
4.7 How do I keep user names and passwords secret in Curl command lines?
This problem has two sides:
The first part is to avoid having clear-text passwords in the command line
so that they don't appear in 'ps' outputs and similar. That is easily
avoided by using the "-K" option to tell curl to read parameters from a file
or stdin to which you can pass the secret info. curl itself will also
attempt to "hide" the given password by blanking out the option - this
doesn't work on all platforms.
To keep the passwords in your account secret from the rest of the world is
not a task that curl addresses. You could of course encrypt them somehow to
at least hide them from being read by human eyes, but that is not what
anyone would call security.
Also note that regular HTTP (using Basic authentication) and FTP passwords
are sent in clear across the network. All it takes for anyone to fetch them
is to listen on the network. Eavesdropping is very easy. Use more secure
authentication methods (like Digest, Negotiate or even NTLM) or consider the
SSL-based alternatives HTTPS and FTPS.
4.8 I found a bug!
It is not a bug if the behavior is documented. Read the docs first.
Especially check out the KNOWN_BUGS file, it may be a documented bug!
If it is a problem with a binary you've downloaded or a package for your
particular platform, try contacting the person who built the package/archive
you have.
If there is a bug, read the BUGS document first. Then report it as described
in there.
4.9 Curl can't authenticate to the server that requires NTLM?
NTLM support requires OpenSSL, GnuTLS, mbedTLS, NSS, Secure Transport, or
Microsoft Windows libraries at build-time to provide this functionality.
NTLM is a Microsoft proprietary protocol. Proprietary formats are evil. You
should not use such ones.
4.10 My HTTP request using HEAD, PUT or DELETE doesn't work!
Many web servers allow or demand that the administrator configures the
server properly for these requests to work on the web server.
Some servers seem to support HEAD only on certain kinds of URLs.
To fully grasp this, try the documentation for the particular server
software you're trying to interact with. This is not anything curl can do
anything about.
4.11 Why does my HTTP range requests return the full document?
Because the range may not be supported by the server, or the server may
choose to ignore it and return the full document anyway.
4.12 Why do I get "certificate verify failed" ?
You invoke curl 7.10 or later to communicate on a https:// URL and get an
error back looking something similar to this:
curl: (35) SSL: error:14090086:SSL routines:
SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Then it means that curl couldn't verify that the server's certificate was
good. Curl verifies the certificate using the CA cert bundle that comes with
the curl installation.
To disable the verification (which makes it act like curl did before 7.10),
use -k. This does however enable man-in-the-middle attacks.
If you get this failure but are having a CA cert bundle installed and used,
the server's certificate is not signed by one of the CA's in the bundle. It
might for example be self-signed. You then correct this problem by obtaining
a valid CA cert for the server. Or again, decrease the security by disabling
this check.
Details are also in the SSLCERTS file in the release archives, found online
here: https://curl.haxx.se/docs/sslcerts.html
4.13 Why is curl -R on Windows one hour off?
Since curl 7.53.0 this issue should be fixed as long as curl was built with
any modern compiler that allows for a 64-bit curl_off_t type. For older
compilers or prior curl versions it may set a time that appears one hour off.
This happens due to a flaw in how Windows stores and uses file modification
times and it is not easily worked around. For more details read this:
https://www.codeproject.com/Articles/1144/Beating-the-Daylight-Savings-Time-bug-and-getting
4.14 Redirects work in browser but not with curl!
curl supports HTTP redirects well (see item 3.8). Browsers generally support
at least two other ways to perform redirects that curl does not:
Meta tags. You can write a HTML tag that will cause the browser to redirect
to another given URL after a certain time.
Javascript. You can write a Javascript program embedded in a HTML page that
redirects the browser to another given URL.
There is no way to make curl follow these redirects. You must either
manually figure out what the page is set to do, or you write a script that
parses the results and fetches the new URL.
4.15 FTPS doesn't work
curl supports FTPS (sometimes known as FTP-SSL) both implicit and explicit
mode.
When a URL is used that starts with FTPS://, curl assumes implicit SSL on
the control connection and will therefore immediately connect and try to
speak SSL. FTPS:// connections default to port 990.
To use explicit FTPS, you use a FTP:// URL and the --ftp-ssl option (or one
of its related flavours). This is the most common method, and the one
mandated by RFC4217. This kind of connection will then of course use the
standard FTP port 21 by default.
4.16 My HTTP POST or PUT requests are slow!
libcurl makes all POST and PUT requests (except for POST requests with a
very tiny request body) use the "Expect: 100-continue" header. This header
allows the server to deny the operation early so that libcurl can bail out
before having to send any data. This is useful in authentication
cases and others.
However, many servers don't implement the Expect: stuff properly and if the
server doesn't respond (positively) within 1 second libcurl will continue
and send off the data anyway.
You can disable libcurl's use of the Expect: header the same way you disable
any header, using -H / CURLOPT_HTTPHEADER, or by forcing it to use HTTP 1.0.
4.17 Non-functional connect timeouts
In most Windows setups having a timeout longer than 21 seconds make no
difference, as it will only send 3 TCP SYN packets and no more. The second
packet sent three seconds after the first and the third six seconds after
the second. No more than three packets are sent, no matter how long the
timeout is set.
See option TcpMaxConnectRetransmissions on this page:
https://support.microsoft.com/en-us/kb/175523/en-us
Also, even on non-Windows systems there may run a firewall or anti-virus
software or similar that accepts the connection but does not actually do
anything else. This will make (lib)curl to consider the connection connected
and thus the connect timeout won't trigger.
4.18 file:// URLs containing drive letters (Windows, NetWare)
When using curl to try to download a local file, one might use a URL
in this format:
file://D:/blah.txt
You'll find that even if D:\blah.txt does exist, curl returns a 'file
not found' error.
According to RFC 1738 (https://www.ietf.org/rfc/rfc1738.txt),
file:// URLs must contain a host component, but it is ignored by
most implementations. In the above example, 'D:' is treated as the
host component, and is taken away. Thus, curl tries to open '/blah.txt'.
If your system is installed to drive C:, that will resolve to 'C:\blah.txt',
and if that doesn't exist you will get the not found error.
To fix this problem, use file:// URLs with *three* leading slashes:
file:///D:/blah.txt
Alternatively, if it makes more sense, specify 'localhost' as the host
component:
file://localhost/D:/blah.txt
In either case, curl should now be looking for the correct file.
4.19 Why doesn't curl return an error when the network cable is unplugged?
Unplugging a cable is not an error situation. The TCP/IP protocol stack
was designed to be fault tolerant, so even though there may be a physical
break somewhere the connection shouldn't be affected, just possibly
delayed. Eventually, the physical break will be fixed or the data will be
re-routed around the physical problem through another path.
In such cases, the TCP/IP stack is responsible for detecting when the
network connection is irrevocably lost. Since with some protocols it is
perfectly legal for the client to wait indefinitely for data, the stack may
never report a problem, and even when it does, it can take up to 20 minutes
for it to detect an issue. The curl option --keepalive-time enables
keep-alive support in the TCP/IP stack which makes it periodically probe the
connection to make sure it is still available to send data. That should
reliably detect any TCP/IP network failure.
But even that won't detect the network going down before the TCP/IP
connection is established (e.g. during a DNS lookup) or using protocols that
don't use TCP. To handle those situations, curl offers a number of timeouts
on its own. --speed-limit/--speed-time will abort if the data transfer rate
falls too low, and --connect-timeout and --max-time can be used to put an
overall timeout on the connection phase or the entire transfer.
A libcurl-using application running in a known physical environment (e.g.
an embedded device with only a single network connection) may want to act
immediately if its lone network connection goes down. That can be achieved
by having the application monitor the network connection on its own using an
OS-specific mechanism, then signalling libcurl to abort (see also item 5.13).
4.20 curl doesn't return error for HTTP non-200 responses!
Correct. Unless you use -f (--fail).
When doing HTTP transfers, curl will perform exactly what you're asking it
to do and if successful it will not return an error. You can use curl to
test your web server's "file not found" page (that gets 404 back), you can
use it to check your authentication protected web pages (that gets a 401
back) and so on.
The specific HTTP response code does not constitute a problem or error for
curl. It simply sends and delivers HTTP as you asked and if that worked,
everything is fine and dandy. The response code is generally providing more
higher level error information that curl doesn't care about. The error was
not in the HTTP transfer.
If you want your command line to treat error codes in the 400 and up range
as errors and thus return a non-zero value and possibly show an error
message, curl has a dedicated option for that: -f (CURLOPT_FAILONERROR in
libcurl speak).
You can also use the -w option and the variable %{response_code} to extract
the exact response code that was returned in the response.
4.21 Why is there a HTTP/1.1 in my HTTP/2 request?
If you use verbose to see the HTTP request when you send off a HTTP/2
request, it will still say 1.1.
The reason for this is that we first generate the request to send using the
old 1.1 style and show that request in the verbose output, and then we
convert it over to the binary header-compressed HTTP/2 style. The actual
"1.1" part from that request is then not actually used in the transfer.
The binary HTTP/2 headers are not human readable.
5. libcurl Issues
5.1 Is libcurl thread-safe?
Yes.
We have written the libcurl code specifically adjusted for multi-threaded
programs. libcurl will use thread-safe functions instead of non-safe ones if
your system has such. Note that you must never share the same handle in
multiple threads.
There may be some exceptions to thread safety depending on how libcurl was
built. Please review the guidelines for thread safety to learn more:
https://curl.haxx.se/libcurl/c/threadsafe.html
5.2 How can I receive all data into a large memory chunk?
[ See also the examples/getinmemory.c source ]
You are in full control of the callback function that gets called every time
there is data received from the remote server. You can make that callback do
whatever you want. You do not have to write the received data to a file.
One solution to this problem could be to have a pointer to a struct that you
pass to the callback function. You set the pointer using the
CURLOPT_WRITEDATA option. Then that pointer will be passed to the callback
instead of a FILE * to a file:
/* imaginary struct */
struct MemoryStruct {
char *memory;
size_t size;
};
/* imaginary callback function */
size_t
WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data)
{
size_t realsize = size * nmemb;
struct MemoryStruct *mem = (struct MemoryStruct *)data;
mem->memory = (char *)realloc(mem->memory, mem->size + realsize + 1);
if (mem->memory) {
memcpy(&(mem->memory[mem->size]), ptr, realsize);
mem->size += realsize;
mem->memory[mem->size] = 0;
}
return realsize;
}
5.3 How do I fetch multiple files with libcurl?
libcurl has excellent support for transferring multiple files. You should
just repeatedly set new URLs with curl_easy_setopt() and then transfer it
with curl_easy_perform(). The handle you get from curl_easy_init() is not
only reusable, but you're even encouraged to reuse it if you can, as that
will enable libcurl to use persistent connections.
5.4 Does libcurl do Winsock initialization on win32 systems?
Yes, if told to in the curl_global_init() call.
5.5 Does CURLOPT_WRITEDATA and CURLOPT_READDATA work on win32 ?
Yes, but you cannot open a FILE * and pass the pointer to a DLL and have
that DLL use the FILE * (as the DLL and the client application cannot access
each others' variable memory areas). If you set CURLOPT_WRITEDATA you must
also use CURLOPT_WRITEFUNCTION as well to set a function that writes the
file, even if that simply writes the data to the specified FILE *.
Similarly, if you use CURLOPT_READDATA you must also specify
CURLOPT_READFUNCTION.
5.6 What about Keep-Alive or persistent connections?
curl and libcurl have excellent support for persistent connections when
transferring several files from the same server. Curl will attempt to reuse
connections for all URLs specified on the same command line/config file, and
libcurl will reuse connections for all transfers that are made using the
same libcurl handle.
When you use the easy interface the connection cache is kept within the easy
handle. If you instead use the multi interface, the connection cache will be
kept within the multi handle and will be shared among all the easy handles
that are used within the same multi handle.
5.7 Link errors when building libcurl on Windows!
You need to make sure that your project, and all the libraries (both static
and dynamic) that it links against, are compiled/linked against the same run
time library.
This is determined by the /MD, /ML, /MT (and their corresponding /M?d)
options to the command line compiler. /MD (linking against MSVCRT dll) seems
to be the most commonly used option.
When building an application that uses the static libcurl library, you must
add -DCURL_STATICLIB to your CFLAGS. Otherwise the linker will look for
dynamic import symbols. If you're using Visual Studio, you need to instead
add CURL_STATICLIB in the "Preprocessor Definitions" section.
If you get linker error like "unknown symbol __imp__curl_easy_init ..." you
have linked against the wrong (static) library. If you want to use the
libcurl.dll and import lib, you don't need any extra CFLAGS, but use one of
the import libraries below. These are the libraries produced by the various
lib/Makefile.* files:
Target: static lib. import lib for libcurl*.dll.
-----------------------------------------------------------
MingW: libcurl.a libcurldll.a
MSVC (release): libcurl.lib libcurl_imp.lib
MSVC (debug): libcurld.lib libcurld_imp.lib
Borland: libcurl.lib libcurl_imp.lib
5.8 libcurl.so.X: open failed: No such file or directory
This is an error message you might get when you try to run a program linked
with a shared version of libcurl and your run-time linker (ld.so) couldn't
find the shared library named libcurl.so.X. (Where X is the number of the
current libcurl ABI, typically 3 or 4).
You need to make sure that ld.so finds libcurl.so.X. You can do that
multiple ways, and it differs somewhat between different operating systems,
but they are usually:
* Add an option to the linker command line that specify the hard-coded path
the run-time linker should check for the lib (usually -R)
* Set an environment variable (LD_LIBRARY_PATH for example) where ld.so
should check for libs
* Adjust the system's config to check for libs in the directory where you've
put the dir (like Linux's /etc/ld.so.conf)
'man ld.so' and 'man ld' will tell you more details
5.9 How does libcurl resolve host names?
libcurl supports a large a number of different name resolve functions. One
of them is picked at build-time and will be used unconditionally. Thus, if
you want to change name resolver function you must rebuild libcurl and tell
it to use a different function.
- The non-IPv6 resolver that can use one of four different host name resolve
calls (depending on what your system supports):
A - gethostbyname()
B - gethostbyname_r() with 3 arguments
C - gethostbyname_r() with 5 arguments
D - gethostbyname_r() with 6 arguments
- The IPv6-resolver that uses getaddrinfo()
- The c-ares based name resolver that uses the c-ares library for resolves.
Using this offers asynchronous name resolves.
- The threaded resolver (default option on Windows). It uses:
A - gethostbyname() on plain IPv4 hosts
B - getaddrinfo() on IPv6 enabled hosts
Also note that libcurl never resolves or reverse-lookups addresses given as
pure numbers, such as 127.0.0.1 or ::1.
5.10 How do I prevent libcurl from writing the response to stdout?
libcurl provides a default built-in write function that writes received data
to stdout. Set the CURLOPT_WRITEFUNCTION to receive the data, or possibly
set CURLOPT_WRITEDATA to a different FILE * handle.
5.11 How do I make libcurl not receive the whole HTTP response?
You make the write callback (or progress callback) return an error and
libcurl will then abort the transfer.
5.12 Can I make libcurl fake or hide my real IP address?
No. libcurl operates on a higher level. Besides, faking IP address would
imply sending IP packets with a made-up source address, and then you normally
get a problem with receiving the packet sent back as they would then not be
routed to you!
If you use a proxy to access remote sites, the sites will not see your local
IP address but instead the address of the proxy.
Also note that on many networks NATs or other IP-munging techniques are used
that makes you see and use a different IP address locally than what the
remote server will see you coming from. You may also consider using
https://www.torproject.org/ .
5.13 How do I stop an ongoing transfer?
With the easy interface you make sure to return the correct error code from
one of the callbacks, but none of them are instant. There is no function you
can call from another thread or similar that will stop it immediately.
Instead, you need to make sure that one of the callbacks you use returns an
appropriate value that will stop the transfer. Suitable callbacks that you
can do this with include the progress callback, the read callback and the
write callback.
If you're using the multi interface, you can also stop a transfer by
removing the particular easy handle from the multi stack at any moment you
think the transfer is done or when you wish to abort the transfer.
5.14 Using C++ non-static functions for callbacks?
libcurl is a C library, it doesn't know anything about C++ member functions.
You can overcome this "limitation" with relative ease using a static
member function that is passed a pointer to the class:
// f is the pointer to your object.
static size_t YourClass::func(void *buffer, size_t sz, size_t n, void *f)
{
// Call non-static member function.
static_cast<YourClass*>(f)->nonStaticFunction();
}
// This is how you pass pointer to the static function:
curl_easy_setopt(hcurl, CURLOPT_WRITEFUNCTION, YourClass::func);
curl_easy_setopt(hcurl, CURLOPT_WRITEDATA, this);
5.15 How do I get an FTP directory listing?
If you end the FTP URL you request with a slash, libcurl will provide you
with a directory listing of that given directory. You can also set
CURLOPT_CUSTOMREQUEST to alter what exact listing command libcurl would use
to list the files.
The follow-up question tends to be how is a program supposed to parse the
directory listing. How does it know what's a file and what's a dir and what's
a symlink etc. If the FTP server supports the MLSD command then it will
return data in a machine-readable format that can be parsed for type. The
types are specified by RFC3659 section 7.5.1. If MLSD is not supported then
you have to work with what you're given. The LIST output format is entirely
at the server's own liking and the NLST output doesn't reveal any types and
in many cases doesn't even include all the directory entries. Also, both LIST
and NLST tend to hide unix-style hidden files (those that start with a dot)
by default so you need to do "LIST -a" or similar to see them.
Example - List only directories.
ftp.funet.fi supports MLSD and ftp.kernel.org does not:
curl -s ftp.funet.fi/pub/ -X MLSD | \
perl -lne 'print if s/(?:^|;)type=dir;[^ ]+ (.+)$/$1/'
curl -s ftp.kernel.org/pub/linux/kernel/ | \
perl -lne 'print if s/^d[-rwx]{9}(?: +[^ ]+){7} (.+)$/$1/'
If you need to parse LIST output in libcurl one such existing
list parser is available at https://cr.yp.to/ftpparse.html Versions of
libcurl since 7.21.0 also provide the ability to specify a wildcard to
download multiple files from one FTP directory.
5.16 I want a different time-out!
Time and time again users realize that CURLOPT_TIMEOUT and
CURLOPT_CONNECTIMEOUT are not sufficiently advanced or flexible to cover all
the various use cases and scenarios applications end up with.
libcurl offers many more ways to time-out operations. A common alternative
is to use the CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME options to
specify the lowest possible speed to accept before to consider the transfer
timed out.
The most flexible way is by writing your own time-out logic and using
CURLOPT_XFERINFOFUNCTION (perhaps in combination with other callbacks) and
use that to figure out exactly when the right condition is met when the
transfer should get stopped.
5.17 Can I write a server with libcurl?
No. libcurl offers no functions or building blocks to build any kind of
internet protocol server. libcurl is only a client-side library. For server
libraries, you need to continue your search elsewhere but there exist many
good open source ones out there for most protocols you could possibly want a
server for. And there are really good stand-alone ones that have been tested
and proven for many years. There's no need for you to reinvent them!
5.18 Does libcurl use threads?
Put simply: no, libcurl will execute in the same thread you call it in. All
callbacks will be called in the same thread as the one you call libcurl in.
If you want to avoid your thread to be blocked by the libcurl call, you make
sure you use the non-blocking API which will do transfers asynchronously -
but still in the same single thread.
libcurl will potentially internally use threads for name resolving, if it
was built to work like that, but in those cases it'll create the child
threads by itself and they will only be used and then killed internally by
libcurl and never exposed to the outside.
6. License Issues
Curl and libcurl are released under a MIT/X derivate license. The license is
very liberal and should not impose a problem for your project. This section
is just a brief summary for the cases we get the most questions. (Parts of
this section was much enhanced by Bjorn Reese.)
We are not lawyers and this is not legal advice. You should probably consult
one if you want true and accurate legal insights without our prejudice. Note
especially that this section concerns the libcurl license only; compiling in
features of libcurl that depend on other libraries (e.g. OpenSSL) may affect
the licensing obligations of your application.
6.1 I have a GPL program, can I use the libcurl library?
Yes!
Since libcurl may be distributed under the MIT/X derivate license, it can be
used together with GPL in any software.
6.2 I have a closed-source program, can I use the libcurl library?
Yes!
libcurl does not put any restrictions on the program that uses the library.
6.3 I have a BSD licensed program, can I use the libcurl library?
Yes!
libcurl does not put any restrictions on the program that uses the library.
6.4 I have a program that uses LGPL libraries, can I use libcurl?
Yes!
The LGPL license doesn't clash with other licenses.
6.5 Can I modify curl/libcurl for my program and keep the changes secret?
Yes!
The MIT/X derivate license practically allows you to do almost anything with
the sources, on the condition that the copyright texts in the sources are
left intact.
6.6 Can you please change the curl/libcurl license to XXXX?
No.
We have carefully picked this license after years of development and
discussions and a large amount of people have contributed with source code
knowing that this is the license we use. This license puts the restrictions
we want on curl/libcurl and it does not spread to other programs or
libraries that use it. It should be possible for everyone to use libcurl or
curl in their projects, no matter what license they already have in use.
6.7 What are my obligations when using libcurl in my commercial apps?
Next to none. All you need to adhere to is the MIT-style license (stated in
the COPYING file) which basically says you have to include the copyright
notice in "all copies" and that you may not use the copyright holder's name
when promoting your software.
You do not have to release any of your source code.
You do not have to reveal or make public any changes to the libcurl source
code.
You do not have to broadcast to the world that you are using libcurl within
your app.
All we ask is that you disclose "the copyright notice and this permission
notice" somewhere. Most probably like in the documentation or in the section
where other third party dependencies already are mentioned and acknowledged.
As can be seen here: https://curl.haxx.se/docs/companies.html and elsewhere,
more and more companies are discovering the power of libcurl and take
advantage of it even in commercial environments.
7. PHP/CURL Issues
7.1 What is PHP/CURL?
The module for PHP that makes it possible for PHP programs to access curl-
functions from within PHP.
In the cURL project we call this module PHP/CURL to differentiate it from
curl the command line tool and libcurl the library. The PHP team however
does not refer to it like this (for unknown reasons). They call it plain
CURL (often using all caps) or sometimes ext/curl, but both cause much
confusion to users which in turn gives us a higher question load.
7.2 Who wrote PHP/CURL?
PHP/CURL was initially written by Sterling Hughes.
7.3 Can I perform multiple requests using the same handle?
Yes - at least in PHP version 4.3.8 and later (this has been known to not
work in earlier versions, but the exact version when it started to work is
unknown to me).
After a transfer, you just set new options in the handle and make another
transfer. This will make libcurl re-use the same connection if it can.
7.4 Does PHP/CURL have dependencies?
PHP/CURL is a module that comes with the regular PHP package. It depends on
and uses libcurl, so you need to have libcurl installed properly before
PHP/CURL can be used.
usr/share/doc/alt-curlssl/TheArtOfHttpScripting 0000644 00000071034 15116224011 0015611 0 ustar 00 _ _ ____ _
___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
The Art Of Scripting HTTP Requests Using Curl
1. HTTP Scripting
1.1 Background
1.2 The HTTP Protocol
1.3 See the Protocol
1.4 See the Timing
1.5 See the Response
2. URL
2.1 Spec
2.2 Host
2.3 Port number
2.4 User name and password
2.5 Path part
3. Fetch a page
3.1 GET
3.2 HEAD
3.3 Multiple URLs in a single command line
3.4 Multiple HTTP methods in a single command line
4. HTML forms
4.1 Forms explained
4.2 GET
4.3 POST
4.4 File Upload POST
4.5 Hidden Fields
4.6 Figure Out What A POST Looks Like
5. HTTP upload
5.1 PUT
6. HTTP Authentication
6.1 Basic Authentication
6.2 Other Authentication
6.3 Proxy Authentication
6.4 Hiding credentials
7. More HTTP Headers
7.1 Referer
7.2 User Agent
8. Redirects
8.1 Location header
8.2 Other redirects
9. Cookies
9.1 Cookie Basics
9.2 Cookie options
10. HTTPS
10.1 HTTPS is HTTP secure
10.2 Certificates
11. Custom Request Elements
11.1 Modify method and headers
11.2 More on changed methods
12. Web Login
12.1 Some login tricks
13. Debug
13.1 Some debug tricks
14. References
14.1 Standards
14.2 Sites
==============================================================================
1. HTTP Scripting
1.1 Background
This document assumes that you're familiar with HTML and general networking.
The increasing amount of applications moving to the web has made "HTTP
Scripting" more frequently requested and wanted. To be able to automatically
extract information from the web, to fake users, to post or upload data to
web servers are all important tasks today.
Curl is a command line tool for doing all sorts of URL manipulations and
transfers, but this particular document will focus on how to use it when
doing HTTP requests for fun and profit. I'll assume that you know how to
invoke 'curl --help' or 'curl --manual' to get basic information about it.
Curl is not written to do everything for you. It makes the requests, it gets
the data, it sends data and it retrieves the information. You probably need
to glue everything together using some kind of script language or repeated
manual invokes.
1.2 The HTTP Protocol
HTTP is the protocol used to fetch data from web servers. It is a very simple
protocol that is built upon TCP/IP. The protocol also allows information to
get sent to the server from the client using a few different methods, as will
be shown here.
HTTP is plain ASCII text lines being sent by the client to a server to
request a particular action, and then the server replies a few text lines
before the actual requested content is sent to the client.
The client, curl, sends a HTTP request. The request contains a method (like
GET, POST, HEAD etc), a number of request headers and sometimes a request
body. The HTTP server responds with a status line (indicating if things went
well), response headers and most often also a response body. The "body" part
is the plain data you requested, like the actual HTML or the image etc.
1.3 See the Protocol
Using curl's option --verbose (-v as a short option) will display what kind
of commands curl sends to the server, as well as a few other informational
texts.
--verbose is the single most useful option when it comes to debug or even
understand the curl<->server interaction.
Sometimes even --verbose is not enough. Then --trace and --trace-ascii offer
even more details as they show EVERYTHING curl sends and receives. Use it
like this:
curl --trace-ascii debugdump.txt http://www.example.com/
1.4 See the Timing
Many times you may wonder what exactly is taking all the time, or you just
want to know the amount of milliseconds between two points in a
transfer. For those, and other similar situations, the --trace-time option
is what you need. It'll prepend the time to each trace output line:
curl --trace-ascii d.txt --trace-time http://example.com/
1.5 See the Response
By default curl sends the response to stdout. You need to redirect it
somewhere to avoid that, most often that is done with -o or -O.
2. URL
2.1 Spec
The Uniform Resource Locator format is how you specify the address of a
particular resource on the Internet. You know these, you've seen URLs like
https://curl.haxx.se or https://yourbank.com a million times. RFC 3986 is the
canonical spec. And yeah, the formal name is not URL, it is URI.
2.2 Host
The host name is usually resolved using DNS or your /etc/hosts file to an IP
address and that's what curl will communicate with. Alternatively you specify
the IP address directly in the URL instead of a name.
For development and other trying out situations, you can point to a different
IP address for a host name than what would otherwise be used, by using curl's
--resolve option:
curl --resolve www.example.org:80:127.0.0.1 http://www.example.org/
2.3 Port number
Each protocol curl supports operates on a default port number, be it over TCP
or in some cases UDP. Normally you don't have to take that into
consideration, but at times you run test servers on other ports or
similar. Then you can specify the port number in the URL with a colon and a
number immediately following the host name. Like when doing HTTP to port
1234:
curl http://www.example.org:1234/
The port number you specify in the URL is the number that the server uses to
offer its services. Sometimes you may use a local proxy, and then you may
need to specify that proxy's port number separately for what curl needs to
connect to locally. Like when using a HTTP proxy on port 4321:
curl --proxy http://proxy.example.org:4321 http://remote.example.org/
2.4 User name and password
Some services are setup to require HTTP authentication and then you need to
provide name and password which is then transferred to the remote site in
various ways depending on the exact authentication protocol used.
You can opt to either insert the user and password in the URL or you can
provide them separately:
curl http://user:password@example.org/
or
curl -u user:password http://example.org/
You need to pay attention that this kind of HTTP authentication is not what
is usually done and requested by user-oriented web sites these days. They
tend to use forms and cookies instead.
2.5 Path part
The path part is just sent off to the server to request that it sends back
the associated response. The path is what is to the right side of the slash
that follows the host name and possibly port number.
3. Fetch a page
3.1 GET
The simplest and most common request/operation made using HTTP is to GET a
URL. The URL could itself refer to a web page, an image or a file. The client
issues a GET request to the server and receives the document it asked for.
If you issue the command line
curl https://curl.haxx.se
you get a web page returned in your terminal window. The entire HTML document
that that URL holds.
All HTTP replies contain a set of response headers that are normally hidden,
use curl's --include (-i) option to display them as well as the rest of the
document.
3.2 HEAD
You can ask the remote server for ONLY the headers by using the --head (-I)
option which will make curl issue a HEAD request. In some special cases
servers deny the HEAD method while others still work, which is a particular
kind of annoyance.
The HEAD method is defined and made so that the server returns the headers
exactly the way it would do for a GET, but without a body. It means that you
may see a Content-Length: in the response headers, but there must not be an
actual body in the HEAD response.
3.3 Multiple URLs in a single command line
A single curl command line may involve one or many URLs. The most common case
is probably to just use one, but you can specify any amount of URLs. Yes
any. No limits. You'll then get requests repeated over and over for all the
given URLs.
Example, send two GETs:
curl http://url1.example.com http://url2.example.com
If you use --data to POST to the URL, using multiple URLs means that you send
that same POST to all the given URLs.
Example, send two POSTs:
curl --data name=curl http://url1.example.com http://url2.example.com
3.4 Multiple HTTP methods in a single command line
Sometimes you need to operate on several URLs in a single command line and do
different HTTP methods on each. For this, you'll enjoy the --next option. It
is basically a separator that separates a bunch of options from the next. All
the URLs before --next will get the same method and will get all the POST
data merged into one.
When curl reaches the --next on the command line, it'll sort of reset the
method and the POST data and allow a new set.
Perhaps this is best shown with a few examples. To send first a HEAD and then
a GET:
curl -I http://example.com --next http://example.com
To first send a POST and then a GET:
curl -d score=10 http://example.com/post.cgi --next http://example.com/results.html
4. HTML forms
4.1 Forms explained
Forms are the general way a web site can present a HTML page with fields for
the user to enter data in, and then press some kind of 'OK' or 'Submit'
button to get that data sent to the server. The server then typically uses
the posted data to decide how to act. Like using the entered words to search
in a database, or to add the info in a bug tracking system, display the entered
address on a map or using the info as a login-prompt verifying that the user
is allowed to see what it is about to see.
Of course there has to be some kind of program on the server end to receive
the data you send. You cannot just invent something out of the air.
4.2 GET
A GET-form uses the method GET, as specified in HTML like:
<form method="GET" action="junk.cgi">
<input type=text name="birthyear">
<input type=submit name=press value="OK">
</form>
In your favorite browser, this form will appear with a text box to fill in
and a press-button labeled "OK". If you fill in '1905' and press the OK
button, your browser will then create a new URL to get for you. The URL will
get "junk.cgi?birthyear=1905&press=OK" appended to the path part of the
previous URL.
If the original form was seen on the page "www.hotmail.com/when/birth.html",
the second page you'll get will become
"www.hotmail.com/when/junk.cgi?birthyear=1905&press=OK".
Most search engines work this way.
To make curl do the GET form post for you, just enter the expected created
URL:
curl "http://www.hotmail.com/when/junk.cgi?birthyear=1905&press=OK"
4.3 POST
The GET method makes all input field names get displayed in the URL field of
your browser. That's generally a good thing when you want to be able to
bookmark that page with your given data, but it is an obvious disadvantage
if you entered secret information in one of the fields or if there are a
large amount of fields creating a very long and unreadable URL.
The HTTP protocol then offers the POST method. This way the client sends the
data separated from the URL and thus you won't see any of it in the URL
address field.
The form would look very similar to the previous one:
<form method="POST" action="junk.cgi">
<input type=text name="birthyear">
<input type=submit name=press value=" OK ">
</form>
And to use curl to post this form with the same data filled in as before, we
could do it like:
curl --data "birthyear=1905&press=%20OK%20" \
http://www.example.com/when.cgi
This kind of POST will use the Content-Type
application/x-www-form-urlencoded and is the most widely used POST kind.
The data you send to the server MUST already be properly encoded, curl will
not do that for you. For example, if you want the data to contain a space,
you need to replace that space with %20 etc. Failing to comply with this
will most likely cause your data to be received wrongly and messed up.
Recent curl versions can in fact url-encode POST data for you, like this:
curl --data-urlencode "name=I am Daniel" http://www.example.com
If you repeat --data several times on the command line, curl will
concatenate all the given data pieces - and put a '&' symbol between each
data segment.
4.4 File Upload POST
Back in late 1995 they defined an additional way to post data over HTTP. It
is documented in the RFC 1867, why this method sometimes is referred to as
RFC1867-posting.
This method is mainly designed to better support file uploads. A form that
allows a user to upload a file could be written like this in HTML:
<form method="POST" enctype='multipart/form-data' action="upload.cgi">
<input type=file name=upload>
<input type=submit name=press value="OK">
</form>
This clearly shows that the Content-Type about to be sent is
multipart/form-data.
To post to a form like this with curl, you enter a command line like:
curl --form upload=@localfilename --form press=OK [URL]
4.5 Hidden Fields
A very common way for HTML based applications to pass state information
between pages is to add hidden fields to the forms. Hidden fields are
already filled in, they aren't displayed to the user and they get passed
along just as all the other fields.
A similar example form with one visible field, one hidden field and one
submit button could look like:
<form method="POST" action="foobar.cgi">
<input type=text name="birthyear">
<input type=hidden name="person" value="daniel">
<input type=submit name="press" value="OK">
</form>
To POST this with curl, you won't have to think about if the fields are
hidden or not. To curl they're all the same:
curl --data "birthyear=1905&press=OK&person=daniel" [URL]
4.6 Figure Out What A POST Looks Like
When you're about fill in a form and send to a server by using curl instead
of a browser, you're of course very interested in sending a POST exactly the
way your browser does.
An easy way to get to see this, is to save the HTML page with the form on
your local disk, modify the 'method' to a GET, and press the submit button
(you could also change the action URL if you want to).
You will then clearly see the data get appended to the URL, separated with a
'?'-letter as GET forms are supposed to.
5. HTTP upload
5.1 PUT
Perhaps the best way to upload data to a HTTP server is to use PUT. Then
again, this of course requires that someone put a program or script on the
server end that knows how to receive a HTTP PUT stream.
Put a file to a HTTP server with curl:
curl --upload-file uploadfile http://www.example.com/receive.cgi
6. HTTP Authentication
6.1 Basic Authentication
HTTP Authentication is the ability to tell the server your username and
password so that it can verify that you're allowed to do the request you're
doing. The Basic authentication used in HTTP (which is the type curl uses by
default) is *plain* *text* based, which means it sends username and password
only slightly obfuscated, but still fully readable by anyone that sniffs on
the network between you and the remote server.
To tell curl to use a user and password for authentication:
curl --user name:password http://www.example.com
6.2 Other Authentication
The site might require a different authentication method (check the headers
returned by the server), and then --ntlm, --digest, --negotiate or even
--anyauth might be options that suit you.
6.3 Proxy Authentication
Sometimes your HTTP access is only available through the use of a HTTP
proxy. This seems to be especially common at various companies. A HTTP proxy
may require its own user and password to allow the client to get through to
the Internet. To specify those with curl, run something like:
curl --proxy-user proxyuser:proxypassword curl.haxx.se
If your proxy requires the authentication to be done using the NTLM method,
use --proxy-ntlm, if it requires Digest use --proxy-digest.
If you use any one of these user+password options but leave out the password
part, curl will prompt for the password interactively.
6.4 Hiding credentials
Do note that when a program is run, its parameters might be possible to see
when listing the running processes of the system. Thus, other users may be
able to watch your passwords if you pass them as plain command line
options. There are ways to circumvent this.
It is worth noting that while this is how HTTP Authentication works, very
many web sites will not use this concept when they provide logins etc. See
the Web Login chapter further below for more details on that.
7. More HTTP Headers
7.1 Referer
A HTTP request may include a 'referer' field (yes it is misspelled), which
can be used to tell from which URL the client got to this particular
resource. Some programs/scripts check the referer field of requests to verify
that this wasn't arriving from an external site or an unknown page. While
this is a stupid way to check something so easily forged, many scripts still
do it. Using curl, you can put anything you want in the referer-field and
thus more easily be able to fool the server into serving your request.
Use curl to set the referer field with:
curl --referer http://www.example.come http://www.example.com
7.2 User Agent
Very similar to the referer field, all HTTP requests may set the User-Agent
field. It names what user agent (client) that is being used. Many
applications use this information to decide how to display pages. Silly web
programmers try to make different pages for users of different browsers to
make them look the best possible for their particular browsers. They usually
also do different kinds of javascript, vbscript etc.
At times, you will see that getting a page with curl will not return the same
page that you see when getting the page with your browser. Then you know it
is time to set the User Agent field to fool the server into thinking you're
one of those browsers.
To make curl look like Internet Explorer 5 on a Windows 2000 box:
curl --user-agent "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" [URL]
Or why not look like you're using Netscape 4.73 on an old Linux box:
curl --user-agent "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" [URL]
8. Redirects
8.1 Location header
When a resource is requested from a server, the reply from the server may
include a hint about where the browser should go next to find this page, or a
new page keeping newly generated output. The header that tells the browser
to redirect is Location:.
Curl does not follow Location: headers by default, but will simply display
such pages in the same manner it displays all HTTP replies. It does however
feature an option that will make it attempt to follow the Location: pointers.
To tell curl to follow a Location:
curl --location http://www.example.com
If you use curl to POST to a site that immediately redirects you to another
page, you can safely use --location (-L) and --data/--form together. Curl will
only use POST in the first request, and then revert to GET in the following
operations.
8.2 Other redirects
Browser typically support at least two other ways of redirects that curl
doesn't: first the html may contain a meta refresh tag that asks the browser
to load a specific URL after a set number of seconds, or it may use
javascript to do it.
9. Cookies
9.1 Cookie Basics
The way the web browsers do "client side state control" is by using
cookies. Cookies are just names with associated contents. The cookies are
sent to the client by the server. The server tells the client for what path
and host name it wants the cookie sent back, and it also sends an expiration
date and a few more properties.
When a client communicates with a server with a name and path as previously
specified in a received cookie, the client sends back the cookies and their
contents to the server, unless of course they are expired.
Many applications and servers use this method to connect a series of requests
into a single logical session. To be able to use curl in such occasions, we
must be able to record and send back cookies the way the web application
expects them. The same way browsers deal with them.
9.2 Cookie options
The simplest way to send a few cookies to the server when getting a page with
curl is to add them on the command line like:
curl --cookie "name=Daniel" http://www.example.com
Cookies are sent as common HTTP headers. This is practical as it allows curl
to record cookies simply by recording headers. Record cookies with curl by
using the --dump-header (-D) option like:
curl --dump-header headers_and_cookies http://www.example.com
(Take note that the --cookie-jar option described below is a better way to
store cookies.)
Curl has a full blown cookie parsing engine built-in that comes in use if you
want to reconnect to a server and use cookies that were stored from a
previous connection (or hand-crafted manually to fool the server into
believing you had a previous connection). To use previously stored cookies,
you run curl like:
curl --cookie stored_cookies_in_file http://www.example.com
Curl's "cookie engine" gets enabled when you use the --cookie option. If you
only want curl to understand received cookies, use --cookie with a file that
doesn't exist. Example, if you want to let curl understand cookies from a
page and follow a location (and thus possibly send back cookies it received),
you can invoke it like:
curl --cookie nada --location http://www.example.com
Curl has the ability to read and write cookie files that use the same file
format that Netscape and Mozilla once used. It is a convenient way to share
cookies between scripts or invokes. The --cookie (-b) switch automatically
detects if a given file is such a cookie file and parses it, and by using the
--cookie-jar (-c) option you'll make curl write a new cookie file at the end
of an operation:
curl --cookie cookies.txt --cookie-jar newcookies.txt \
http://www.example.com
10. HTTPS
10.1 HTTPS is HTTP secure
There are a few ways to do secure HTTP transfers. By far the most common
protocol for doing this is what is generally known as HTTPS, HTTP over
SSL. SSL encrypts all the data that is sent and received over the network and
thus makes it harder for attackers to spy on sensitive information.
SSL (or TLS as the latest version of the standard is called) offers a
truckload of advanced features to allow all those encryptions and key
infrastructure mechanisms encrypted HTTP requires.
Curl supports encrypted fetches when built to use a TLS library and it can be
built to use one out of a fairly large set of libraries - "curl -V" will show
which one your curl was built to use (if any!). To get a page from a HTTPS
server, simply run curl like:
curl https://secure.example.com
10.2 Certificates
In the HTTPS world, you use certificates to validate that you are the one
you claim to be, as an addition to normal passwords. Curl supports client-
side certificates. All certificates are locked with a pass phrase, which you
need to enter before the certificate can be used by curl. The pass phrase
can be specified on the command line or if not, entered interactively when
curl queries for it. Use a certificate with curl on a HTTPS server like:
curl --cert mycert.pem https://secure.example.com
curl also tries to verify that the server is who it claims to be, by
verifying the server's certificate against a locally stored CA cert
bundle. Failing the verification will cause curl to deny the connection. You
must then use --insecure (-k) in case you want to tell curl to ignore that
the server can't be verified.
More about server certificate verification and ca cert bundles can be read
in the SSLCERTS document, available online here:
https://curl.haxx.se/docs/sslcerts.html
At times you may end up with your own CA cert store and then you can tell
curl to use that to verify the server's certificate:
curl --cacert ca-bundle.pem https://example.com/
11. Custom Request Elements
11.1 Modify method and headers
Doing fancy stuff, you may need to add or change elements of a single curl
request.
For example, you can change the POST request to a PROPFIND and send the data
as "Content-Type: text/xml" (instead of the default Content-Type) like this:
curl --data "<xml>" --header "Content-Type: text/xml" \
--request PROPFIND url.com
You can delete a default header by providing one without content. Like you
can ruin the request by chopping off the Host: header:
curl --header "Host:" http://www.example.com
You can add headers the same way. Your server may want a "Destination:"
header, and you can add it:
curl --header "Destination: http://nowhere" http://example.com
11.2 More on changed methods
It should be noted that curl selects which methods to use on its own
depending on what action to ask for. -d will do POST, -I will do HEAD and so
on. If you use the --request / -X option you can change the method keyword
curl selects, but you will not modify curl's behavior. This means that if you
for example use -d "data" to do a POST, you can modify the method to a
PROPFIND with -X and curl will still think it sends a POST. You can change
the normal GET to a POST method by simply adding -X POST in a command line
like:
curl -X POST http://example.org/
... but curl will still think and act as if it sent a GET so it won't send any
request body etc.
12. Web Login
12.1 Some login tricks
While not strictly just HTTP related, it still causes a lot of people problems
so here's the executive run-down of how the vast majority of all login forms
work and how to login to them using curl.
It can also be noted that to do this properly in an automated fashion, you
will most certainly need to script things and do multiple curl invokes etc.
First, servers mostly use cookies to track the logged-in status of the
client, so you will need to capture the cookies you receive in the
responses. Then, many sites also set a special cookie on the login page (to
make sure you got there through their login page) so you should make a habit
of first getting the login-form page to capture the cookies set there.
Some web-based login systems feature various amounts of javascript, and
sometimes they use such code to set or modify cookie contents. Possibly they
do that to prevent programmed logins, like this manual describes how to...
Anyway, if reading the code isn't enough to let you repeat the behavior
manually, capturing the HTTP requests done by your browsers and analyzing the
sent cookies is usually a working method to work out how to shortcut the
javascript need.
In the actual <form> tag for the login, lots of sites fill-in random/session
or otherwise secretly generated hidden tags and you may need to first capture
the HTML code for the login form and extract all the hidden fields to be able
to do a proper login POST. Remember that the contents need to be URL encoded
when sent in a normal POST.
13. Debug
13.1 Some debug tricks
Many times when you run curl on a site, you'll notice that the site doesn't
seem to respond the same way to your curl requests as it does to your
browser's.
Then you need to start making your curl requests more similar to your
browser's requests:
* Use the --trace-ascii option to store fully detailed logs of the requests
for easier analyzing and better understanding
* Make sure you check for and use cookies when needed (both reading with
--cookie and writing with --cookie-jar)
* Set user-agent to one like a recent popular browser does
* Set referer like it is set by the browser
* If you use POST, make sure you send all the fields and in the same order as
the browser does it.
A very good helper to make sure you do this right, is the LiveHTTPHeader tool
that lets you view all headers you send and receive with Mozilla/Firefox
(even when using HTTPS). Chrome features similar functionality out of the box
among the developer's tools.
A more raw approach is to capture the HTTP traffic on the network with tools
such as ethereal or tcpdump and check what headers that were sent and
received by the browser. (HTTPS makes this technique inefficient.)
14. References
14.1 Standards
RFC 7230 is a must to read if you want in-depth understanding of the HTTP
protocol
RFC 3986 explains the URL syntax
RFC 1867 defines the HTTP post upload format
RFC 6525 defines how HTTP cookies work
14.2 Sites
https://curl.haxx.se is the home of the curl project
usr/share/doc/alt-curlssl/RESOURCES 0000644 00000004477 15116224017 0013001 0 ustar 00 _ _ ____ _
Project ___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
This document lists documents and standards used by curl.
RFC 959 - FTP Protocol
RFC 1635 - How to Use Anonymous FTP
RFC 1738 - Uniform Resource Locators
RFC 1777 - Lightweight Directory Access Protocol (LDAP)
RFC 1808 - Relative Uniform Resource Locators
RFC 1867 - Form-based File Upload in HTML
RFC 1950 - ZLIB Compressed Data Format Specification
RFC 1951 - DEFLATE Compressed Data Format Specification
RFC 1952 - GZIP File Format Specification
RFC 1959 - LDAP URL Syntax
RFC 2045-2049 - Everything you need to know about MIME! (needed for form
based upload)
RFC 2068 - HTTP 1.1 (obsoleted by RFC 2616)
RFC 2104 - Keyed-Hashing for Message Authentication
RFC 2109 - HTTP State Management Mechanism (cookie stuff)
- Also, read Netscape's specification at
https://curl.haxx.se/rfc/cookie_spec.html
RFC 2183 - The Content-Disposition Header Field
RFC 2195 - CRAM-MD5 Authentication
RFC 2229 - A Dictionary Server Protocol
RFC 2255 - Newer LDAP URL Format
RFC 2231 - MIME Parameter Value and Encoded Word Extensions:
Character Sets, Languages, and Continuations
RFC 2388 - "Returning Values from Forms: multipart/form-data"
Use this as an addition to the RFC1867
RFC 2396 - "Uniform Resource Identifiers: Generic Syntax and Semantics" This
one obsoletes RFC 1738, but since RFC 1738 is often mentioned
I've left it in this list.
RFC 2428 - FTP Extensions for IPv6 and NATs
RFC 2577 - FTP Security Considerations
RFC 2616 - HTTP 1.1, the latest
RFC 2617 - HTTP Authentication
RFC 2718 - Guidelines for new URL Schemes
RFC 2732 - Format for Literal IPv6 Addresses in URL's
RFC 2818 - HTTP Over TLS (TLS is the successor to SSL)
RFC 2821 - Simple Mail Transfer Protocol (SMTP)
RFC 2964 - Use of HTTP State Management
RFC 2965 - HTTP State Management Mechanism. Cookies. Obsoletes RFC2109
RFC 3207 - SMTP Over TLS
RFC 4616 - PLAIN Authentication
RFC 4954 - SMTP Authentication
RFC 7932 - Brotli Compressed Data Format
usr/share/doc/alt-curlssl/CHANGES 0000644 00000661772 15116224024 0012504 0 ustar 00 _ _ ____ _
___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
Changelog
Version 7.64.0 (6 Feb 2019)
Daniel Stenberg (6 Feb 2019)
- RELEASE-NOTES: 7.64.0
- RELEASE-PROCEDURE: update the release calendar
- THANKS: 7.64.0 status
Daniel Gustafsson (5 Feb 2019)
- ROADMAP: remove already performed item
Commit 7a09b52c98ac8d840a8a9907b1a1d9a9e684bcf5 introduced support
for the draft-ietf-httpbis-cookie-alone-01 cookie draft, and while
the entry was removed from the TODO it was mistakenly left here.
Fix by removing and rewording the entry slightly.
Closes #3530
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- [Etienne Simard brought this change]
CONTRIBUTE.md: Fix grammatical errors
Fix grammatical errors making the document read better. Also fixes
a typo.
Closes #3525
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Daniel Stenberg (4 Feb 2019)
- [Julian Z brought this change]
docs: use $(INSTALL_DATA) to install man page
Fixes #3518
Closes #3522
Jay Satiro (4 Feb 2019)
- [Ladar Levison brought this change]
runtests.pl: Fix perl call to include srcdir
- Use explicit include opt for perl calls.
Prior to this change some scripts couldn't find their dependencies.
At the top, perl is called using with the "-Isrcdir" option, and it
works:
https://github.com/curl/curl/blob/curl-7_63_0/tests/runtests.pl#L183
But on line 3868, that option is omitted. This caused problems for me,
as the symbol-scan.pl script in particular couldn't find its
dependencies properly:
https://github.com/curl/curl/blob/curl-7_63_0/tests/runtests.pl#L3868
This patch fixes that oversight by making calls to perl sub-shells
uniform.
Closes https://github.com/curl/curl/pull/3496
Daniel Stenberg (4 Feb 2019)
- [Daniel Gustafsson brought this change]
smtp: avoid risk of buffer overflow in strtol
If the incoming len 5, but the buffer does not have a termination
after 5 bytes, the strtol() call may keep reading through the line
buffer until is exceeds its boundary. Fix by ensuring that we are
using a bounded read with a temporary buffer on the stack.
Bug: https://curl.haxx.se/docs/CVE-2019-3823.html
Reported-by: Brian Carpenter (Geeknik Labs)
CVE-2019-3823
- ntlm: fix *_type3_message size check to avoid buffer overflow
Bug: https://curl.haxx.se/docs/CVE-2019-3822.html
Reported-by: Wenxiang Qian
CVE-2019-3822
- NTLM: fix size check condition for type2 received data
Bug: https://curl.haxx.se/docs/CVE-2018-16890.html
Reported-by: Wenxiang Qian
CVE-2018-16890
Marcel Raad (1 Feb 2019)
- [georgeok brought this change]
spnego_sspi: add support for channel binding
Attempt to add support for Secure Channel binding when negotiate
authentication is used. The problem to solve is that by default IIS
accepts channel binding and curl doesn't utilise them. The result was a
401 response. Scope affects only the Schannel(winssl)-SSPI combination.
Fixes https://github.com/curl/curl/issues/3503
Closes https://github.com/curl/curl/pull/3509
Daniel Stenberg (1 Feb 2019)
- RELEASE-NOTES: synced
- schannel: stop calling it "winssl"
Stick to "Schannel" everywhere. The configure option --with-winssl is
kept to allow existing builds to work but --with-schannel is added as an
alias.
Closes #3504
- multi: set the EXPIRE_*TIMEOUT timers at TIMER_STARTSINGLE time
To make sure Curl_timeleft() also thinks the timeout has been reached
when one of the EXPIRE_*TIMEOUTs expires.
Bug: https://curl.haxx.se/mail/lib-2019-01/0073.html
Reported-by: Zhao Yisha
Closes #3501
- [John Marshall brought this change]
doc: use meaningless port number in CURLOPT_LOCALPORT example
Use an ephemeral port number here; previously the example had 8080
which could be confusing as the common web server port number might
be misinterpreted as suggesting this option affects the remote port.
URL: https://curl.haxx.se/mail/lib-2019-01/0084.html
Closes #3513
GitHub (29 Jan 2019)
- [Gisle Vanem brought this change]
Escape the '\'
A backslash should be escaped in Roff / Troff.
Jay Satiro (29 Jan 2019)
- TODO: WinSSL: 'Add option to disable client cert auto-send'
By default WinSSL selects and send a client certificate automatically,
but for privacy and consistency we should offer an option to disable the
default auto-send behavior.
Reported-by: Jeroen Ooms
Closes https://github.com/curl/curl/issues/2262
Daniel Stenberg (28 Jan 2019)
- [Jeremie Rapin brought this change]
sigpipe: if mbedTLS is used, ignore SIGPIPE
mbedTLS doesn't have a sigpipe management. If a write/read occurs when
the remote closes the socket, the signal is raised and kills the
application. Use the curl mecanisms fix this behavior.
Signed-off-by: Jeremie Rapin <j.rapin@overkiz.com>
Closes #3502
- unit1653: make it survive torture tests
Jay Satiro (28 Jan 2019)
- [Michael Kujawa brought this change]
timeval: Disable MSVC Analyzer GetTickCount warning
Compiling with msvc /analyze and a recent Windows SDK warns against
using GetTickCount (Suggests to use GetTickCount64 instead.)
Since GetTickCount is only being used when GetTickCount64 isn't
available, I am disabling that warning.
Fixes https://github.com/curl/curl/issues/3437
Closes https://github.com/curl/curl/pull/3440
Daniel Stenberg (26 Jan 2019)
- configure: rewrite --enable-code-coverage
The previously used ax_code_coverage.m4 is not license compatible and
must not be used.
Reported-by: William A. Rowe Jr
Fixes #3497
Closes #3499
- [Felix Hädicke brought this change]
setopt: enable CURLOPT_SSH_KNOWNHOSTS and CURLOPT_SSH_KEYFUNCTION for libssh
CURLOPT_SSH_KNOWNHOSTS and CURLOPT_SSH_KEYFUNCTION are supported for
libssh as well. So accepting these options only when compiling with
libssh2 is wrong here.
Fixes #3493
Closes #3494
- [Felix Hädicke brought this change]
libssh: do not let libssh create socket
By default, libssh creates a new socket, instead of using the socket
created by curl for SSH connections.
Pass the socket created by curl to libssh using ssh_options_set() with
SSH_OPTIONS_FD directly after ssh_new(). So libssh uses our socket
instead of creating a new one.
This approach is very similar to what is done in the libssh2 code, where
the socket created by curl is passed to libssh2 when
libssh2_session_startup() is called.
Fixes #3491
Closes #3495
- RELEASE-NOTES: synced
- [Archangel_SDY brought this change]
schannel: preserve original certificate path parameter
Fixes #3480
Closes #3487
- KNOWN_BUGS: tests not compatible with python3
Closes #3289
[skip ci]
Daniel Gustafsson (20 Jan 2019)
- memcmp: avoid doing single char memcmp
There is no real gain in performing memcmp() comparisons on single
characters, so change these to array subscript inspections which
saves a call and makes the code clearer.
Closes #3486
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
Daniel Stenberg (19 Jan 2019)
- COPYING: it's 2019
[skip ci]
- [hhb brought this change]
configure: fix recv/send/select detection on Android
This reverts commit d4f25201fb7da03fc88f90d51101beb3d0026db9.
The overloadable attribute is removed again starting from
NDK17. Actually they only exist in two NDK versions (15 and 16). With
overloadable, the first condition tried will succeed. Results in wrong
detection result.
Closes #3484
Marcel Raad (19 Jan 2019)
- [georgeok brought this change]
ntlm_sspi: add support for channel binding
Windows extended potection (aka ssl channel binding) is required
to login to ntlm IIS endpoint, otherwise the server returns 401
responses.
Fixes #3280
Closes #3321
Daniel Stenberg (18 Jan 2019)
- schannel: on connection close there might not be a transfer
Reported-by: Marcel Raad
Fixes #3412
Closes #3483
- [Joel Depooter brought this change]
ssh: log the libssh2 error message when ssh session startup fails
When a ssh session startup fails, it is useful to know why it has
failed. This commit changes the message from:
"Failure establishing ssh session"
to something like this, for example:
"Failure establishing ssh session: -5, Unable to exchange encryption keys"
Closes #3481
Alessandro Ghedini (16 Jan 2019)
- Fix typo in manpage
Daniel Stenberg (16 Jan 2019)
- RELEASE-NOTES: synced
Sergei Nikulov (16 Jan 2019)
- cmake: updated check for HAVE_POLL_FINE to match autotools
Daniel Stenberg (16 Jan 2019)
- curl-compilers.m4: check for __ibmxl__ to detect xlclang
Follow-up to 2fa0d57e2e3. The __xlc__ symbol is only defined there if a
particular flag is used for legacy macros.
Fixes #3474
Closes #3479
- openssl: fix the SSL_get_tlsext_status_ocsp_resp call
.... to not pass in a const in the second argument as that's not how it
is supposed to be used and might cause compiler warnings.
Reported-by: Pavel Pavlov
Fixes #3477
Closes #3478
- curl-compilers.m4: detect xlclang
Since it isn't totally clang compatible, we detect this IBM clang
front-end and if detected, avoids some clang specific magic.
Reported-by: Kees Dekker
Fixes #3474
Closes #3476
- README: add codacy code quality badge
[skip ci]
- extract_if_dead: follow-up to 54b201b48c90a
extract_if_dead() dead is called from two functions, and only one of
them should get conn->data updated and now neither call path clears it.
scan-build found a case where conn->data would be NULL dereferenced in
ConnectionExists() otherwise.
Closes #3473
- multi: remove "Dead assignment"
Found by scan-build. Follow-up to 4c35574bb785ce.
Closes #3471
- tests: move objnames-* from lib into tests
Since they're used purely for testing purposes, I think they should
rather be stored there.
Closes #3470
Sergei Nikulov (15 Jan 2019)
- travis: added cmake build for osx
Daniel Stenberg (14 Jan 2019)
- [Frank Gevaerts brought this change]
cookie: fix comment typo (url_path_len -> uri_path_len)
Closes #3469
Marcel Raad (14 Jan 2019)
- winbuild: conditionally use /DZLIB_WINAPI
zlibwapi.lib (dynamic library) and zlibstat.lib (static library) have
the ZLIB_WINAPI define set by default. Using them requires that define
too.
Ref: https://zlib.net/DLL_FAQ.txt
Fixes https://github.com/curl/curl/issues/3133
Closes https://github.com/curl/curl/pull/3460
Daniel Stenberg (14 Jan 2019)
- src/Makefile: make 'tidy' target work for metalink builds
- extract_if_dead: use a known working transfer when checking connections
Make sure that this function sets a proper "live" transfer for the
connection before calling the protocol-specific connection check
function, and then clear it again afterward as a non-used connection has
no current transfer.
Reported-by: Jeroen Ooms
Reviewed-by: Marcel Raad
Reviewed-by: Daniel Gustafsson
Fixes #3463
Closes #3464
- openssl: adapt to 3.0.0, OpenSSL_version_num() is deprecated
OpenSSL_version() replaces OpenSSL_version_num()
Closes #3462
Sergei Nikulov (11 Jan 2019)
- cmake: added checks for HAVE_VARIADIC_MACROS_C99 and HAVE_VARIADIC_MACROS_GCC
Daniel Stenberg (11 Jan 2019)
- urldata: rename easy_conn to just conn
We use "conn" everywhere to be a pointer to the connection.
Introduces two functions that "attaches" and "detaches" the connection
to and from the transfer.
Going forward, we should favour using "data->conn" (since a transfer
always only has a single connection or none at all) to "conn->data"
(since a connection can have none, one or many transfers associated with
it and updating conn->data to be correct is error prone and a frequent
reason for internal issues).
Closes #3442
- tool_cb_prg: avoid integer overflow
When calculating the progress bar width.
Reported-by: Peng Li
Fixes #3456
Closes #3458
Daniel Gustafsson (11 Jan 2019)
- travis: turn off copyright year checks in checksrc
Invoking the maintainer intended COPYRIGHTYEAR check for everyone
in the PR pipeline is too invasive, especially at the turn of the
year when many files get affected. Remove and leave it as a tool
for maintainers to verify patches before commits.
This reverts f7bdf4b2e1d81b2652b81b9b3029927589273b41.
After discussion with: Daniel Stenberg
Daniel Stenberg (10 Jan 2019)
- KNOWN_BUGS: cmake makes unusable tool_hugehelp.c with MinGW
Closes #3125
- KNOWN_BUGS: Improve --data-urlencode space encoding
Closes #3229
Patrick Monnerat (10 Jan 2019)
- os400: add a missing closing bracket
See https://github.com/curl/curl/issues/3453#issuecomment-453054458
Reported-by: jonrumsey on github
- os400: fix extra parameter syntax error.
Reported-by: jonrumsey on github
Closes #3453
Daniel Stenberg (10 Jan 2019)
- test1558: verify CURLINFO_PROTOCOL on file:// transfer
Attempt to reproduce issue #3444.
Closes #3447
- RELEASE-NOTES: synced
- xattr: strip credentials from any URL that is stored
Both user and password are cleared uncondtitionally.
Added unit test 1621 to verify.
Fixes #3423
Closes #3433
- cookies: allow secure override when done over HTTPS
Added test 1562 to verify.
Reported-by: Jeroen Ooms
Fixes #3445
Closes #3450
- multi: multiplexing improvements
Fixes #3436
Closes #3448
Problem 1
After LOTS of scratching my head, I eventually realized that even when doing
10 uploads in parallel, sometimes the socket callback to the application that
tells it what to wait for on the socket, looked like it would reflect the
status of just the single transfer that just changed state.
Digging into the code revealed that this was indeed the truth. When multiple
transfers are using the same connection, the application did not correctly get
the *combined* flags for all transfers which then could make it switch to READ
(only) when in fact most transfers wanted to get told when the socket was
WRITEABLE.
Problem 1b
A separate but related regression had also been introduced by me when I
cleared connection/transfer association better a while ago, as now the logic
couldn't find the connection and see if that was marked as used by more
transfers and then it would also prematurely remove the socket from the socket
hash table even in times other transfers were still using it!
Fix 1
Make sure that each socket stored in the socket hash has a "combined" action
field of what to ask the application to wait for, that is potentially the ORed
action of multiple parallel transfers. And remove that socket hash entry only
if there are no transfers left using it.
Problem 2
The socket hash entry stored an association to a single transfer using that
socket - and when curl_multi_socket_action() was called to tell libcurl about
activities on that specific socket only that transfer was "handled".
This was WRONG, as a single socket/connection can be used by numerous parallel
transfers and not necessarily a single one.
Fix 2
We now store a list of handles in the socket hashtable entry and when libcurl
is told there's traffic for a particular socket, it now iterates over all
known transfers using that single socket.
- test1561: improve test name
[skip ci]
- [Katsuhiko YOSHIDA brought this change]
cookies: skip custom cookies when redirecting cross-site
Closes #3417
- THANKS: fixups and a dedupe
[skip ci]
- timediff: fix math for unsigned time_t
Bug: https://curl.haxx.se/mail/lib-2018-12/0088.html
Closes #3449
- [Bernhard M. Wiedemann brought this change]
tests: allow tests to pass by 2037-02-12
similar to commit f508d29f3902104018
Closes #3443
- RELEASE-NOTES: synced
- [Brad Spencer brought this change]
curl_multi_remove_handle() don't block terminating c-ares requests
Added Curl_resolver_kill() for all three resolver modes, which only
blocks when necessary, along with test 1592 to confirm
curl_multi_remove_handle() doesn't block unless it must.
Closes #3428
Fixes #3371
- Revert "http_negotiate: do not close connection until negotiation is completed"
This reverts commit 07ebaf837843124ee670e5b8c218b80b92e06e47.
This also reopens PR #3275 which brought the change now reverted.
Fixes #3384
Closes #3439
- curl/urlapi.h: include "curl.h" first
This allows programs to include curl/urlapi.h directly.
Reviewed-by: Daniel Gustafsson
Reported-by: Ben Kohler
Fixes #3438
Closes #3441
Marcel Raad (6 Jan 2019)
- VS projects: fix build warning
Starting with Visual Studio 2017 Update 9, Visual Studio doesn't like
the MinimalRebuild option anymore and warns:
cl : Command line warning D9035: option 'Gm' has been deprecated and
will be removed in a future release
The option can be safely removed so that the default is used.
Closes https://github.com/curl/curl/pull/3425
- schannel: fix compiler warning
When building with Unicode on MSVC, the compiler warns about freeing a
pointer to const in Curl_unicodefree. Fix this by declaring it as
non-const and casting the argument to Curl_convert_UTF8_to_tchar to
non-const too, like we do in all other places.
Closes https://github.com/curl/curl/pull/3435
Daniel Stenberg (4 Jan 2019)
- [Rikard Falkeborn brought this change]
printf: introduce CURL_FORMAT_TIMEDIFF_T
- [Rikard Falkeborn brought this change]
printf: fix format specifiers
Closes #3426
- libtest/stub_gssapi: use "real" snprintf
... since it doesn't link with libcurl.
Reverts the commit dcd6f81025 changes from this file.
Bug: https://curl.haxx.se/mail/lib-2019-01/0000.html
Reported-by: Shlomi Fish
Reviewed-by: Daniel Gustafsson
Reviewed-by: Kamil Dudka
Closes #3434
- INTERNALS: correct some outdated function names
Closes #3431
- docs/version.d: mention MultiSSL
Reviewed-by: Daniel Gustafsson
Closes #3432
Daniel Gustafsson (2 Jan 2019)
- [Rikard Falkeborn brought this change]
examples: Update .gitignore
Add a few missing examples to make `make examples` not leave the
workspace in a dirty state.
Closes #3427
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
- THANKS: add more missing names
Add Adrian Burcea who made the artwork for the curl://up 2018 event
which was held in Stockholm, Sweden.
- docs: mention potential leak in curl_slist_append
When a non-empty list is appended to, and used as the returnvalue,
the list pointer can leak in case of an allocation failure in the
curl_slist_append() call. This is correctly handled in curl code
usage but we weren't explicitly pointing it out in the API call
documentation. Fix by extending the RETURNVALUE manpage section
and example code.
Closes #3424
Reported-by: dnivras on github
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Marcel Raad (1 Jan 2019)
- tvnow: silence conversion warnings
MinGW-w64 defaults to targeting Windows 7 now, so GetTickCount64 is
used and the milliseconds are represented as unsigned long long,
leading to a compiler warning when implicitly converting them to long.
Daniel Stenberg (1 Jan 2019)
- THANKS: dedupe more names
Researched-by: Tae Wong
Marcel Raad (1 Jan 2019)
- [Markus Moeller brought this change]
ntlm: update selection of type 3 response
NTLM2 did not work i.e. no NTLMv2 response was created. Changing the
check seems to work.
Ref: https://winprotocoldoc.blob.core.windows.net/productionwindowsarchives/MS-NLMP/[MS-NLMP].pdf
Fixes https://github.com/curl/curl/issues/3286
Closes https://github.com/curl/curl/pull/3287
Closes https://github.com/curl/curl/pull/3415
Daniel Stenberg (31 Dec 2018)
- THANKS: added missing names from year <= 2000
Due to a report of a missing name in THANKS I manually went through an
old CHANGES.0 file and added many previously missing names here.
Daniel Gustafsson (30 Dec 2018)
- urlapi: fix parsing ipv6 with zone index
The previous fix for parsing IPv6 URLs with a zone index was a paddle
short for URLs without an explicit port. This patch fixes that case
and adds a unit test case.
This bug was highlighted by issue #3408, and while it's not the full
fix for the problem there it is an isolated bug that should be fixed
regardless.
Closes #3411
Reported-by: GitYuanQu on github
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (30 Dec 2018)
- THANKS: dedupe Guenter Knauf
Reported-by: Tae Wong
- THANKS: missing name from the 6.3.1 release!
Daniel Gustafsson (27 Dec 2018)
- RELEASE-NOTES: synced
- [Claes Jakobsson brought this change]
hostip: support wildcard hosts
This adds support for wildcard hosts in CURLOPT_RESOLVE. These are
try-last so any non-wildcard entry is resolved first. If specified,
any host not matched by another CURLOPT_RESOLVE config will use this
as fallback.
Example send a.com to 10.0.0.1 and everything else to 10.0.0.2:
curl --resolve *:443:10.0.0.2 --resolve a.com:443:10.0.0.1 \
https://a.com https://b.com
This is probably quite similar to using:
--connect-to a.com:443:10.0.0.1:443 --connect-to :443:10.0.0.2:443
Closes #3406
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- url: fix incorrect indentation
Patrick Monnerat (26 Dec 2018)
- os400: upgrade ILE/RPG binding.
- Trailer function support.
- http 0.9 option.
- curl_easy_upkeep.
Daniel Gustafsson (25 Dec 2018)
- FAQ: remove mention of sourceforge for github
The project bug tracker is no longer hosted at sourceforge but is now
hosted on the curl Github page. Update the FAQ to reflect.
Closes #3410
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- openvms: fix typos in documentation
- openvms: fix OpenSSL discovery on VAX
The DCL code had a typo in one of the commands which would make the
OpenSSL discovery on VAX fail. The correct syntax is F$ENVIRONMENT.
Closes #3407
Reviewed-by: Viktor Szakats <commit@vszakats.net>
Daniel Stenberg (24 Dec 2018)
- [Ruslan Baratov brought this change]
cmake: use lowercase for function name like the rest of the code
Reviewed-by: Sergei Nikulov
closes #3196
- Revert "libssh: no data pointer == nothing to do"
This reverts commit c98ee5f67f497195c9 since commit f3ce38739fa fixed the
problem in a more generic way.
- disconnect: set conn->data for protocol disconnect
Follow-up to fb445a1e18d: Set conn->data explicitly to point out the
current transfer when invoking the protocol-specific disconnect function
so that it can work correctly.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=12173
Jay Satiro (23 Dec 2018)
- [Pavel Pavlov brought this change]
timeval: Use high resolution timestamps on Windows
- Use QueryPerformanceCounter on Windows Vista+
There is confusing info floating around that QueryPerformanceCounter
can leap etc, which might have been true long time ago, but no longer
the case nowadays (perhaps starting from WinXP?). Also, boost and
std::chrono::steady_clock use QueryPerformanceCounter in a similar way.
Prior to this change GetTickCount or GetTickCount64 was used, which has
lower resolution. That is still the case for <= XP.
Fixes https://github.com/curl/curl/issues/3309
Closes https://github.com/curl/curl/pull/3318
Daniel Stenberg (22 Dec 2018)
- libssh: no data pointer == nothing to do
- conncache_unlock: avoid indirection by changing input argument type
- disconnect: separate connections and easy handles better
Do not assume/store assocation between a given easy handle and the
connection if it can be avoided.
Long-term, the 'conn->data' pointer should probably be removed as it is a
little too error-prone. Still used very widely though.
Reported-by: masbug on github
Fixes #3391
Closes #3400
- libssh: free sftp_canonicalize_path() data correctly
Assisted-by: Harry Sintonen
Fixes #3402
Closes #3403
- RELEASE-NOTES: synced
- http: added options for allowing HTTP/0.9 responses
Added CURLOPT_HTTP09_ALLOWED and --http0.9 for this purpose.
For now, both the tool and library allow HTTP/0.9 by default.
docs/DEPRECATE.md lays out the plan for when to reverse that default: 6
months after the 7.64.0 release. The options are added already now so
that applications/scripts can start using them already now.
Fixes #2873
Closes #3383
- if2ip: remove unused function Curl_if_is_interface_name
Closes #3401
- http2: clear pause stream id if it gets closed
Reported-by: Florian Pritz
Fixes #3392
Closes #3399
Daniel Gustafsson (20 Dec 2018)
- [David Garske brought this change]
wolfssl: Perform cleanup
This adds a cleanup callback for cyassl. Resolves possible memory leak
when using ECC fixed point cache.
Closes #3395
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Daniel Stenberg (20 Dec 2018)
- mbedtls: follow-up VERIFYHOST fix from f097669248
Fix-by: Eric Rosenquist
Fixes #3376
Closes #3390
- curlver: bump to 7.64.0 for next release
Daniel Gustafsson (19 Dec 2018)
- cookies: extend domain checks to non psl builds
Ensure to perform the checks we have to enforce a sane domain in
the cookie request. The check for non-PSL enabled builds is quite
basic but it's better than nothing.
Closes #2964
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (19 Dec 2018)
- [Matus Uzak brought this change]
smb: fix incorrect path in request if connection reused
Follow-up to 09e401e01bf9. If connection gets reused, then data member
will be copied, but not the proto member. As a result, in smb_do(),
path has been set from the original proto.share data.
Closes #3388
- curl -J: do not append to the destination file
Reported-by: Kamil Dudka
Fixes #3380
Closes #3381
- mbedtls: use VERIFYHOST
Previously, VERIFYPEER would enable/disable all checks.
Reported-by: Eric Rosenquist
Fixes #3376
Closes #3380
- pingpong: change default response timeout to 120 seconds
Previously it was 30 minutes
- pingpong: ignore regular timeout in disconnect phase
The timeout set with CURLOPT_TIMEOUT is no longer used when
disconnecting from one of the pingpong protocols (FTP, IMAP, SMTP,
POP3).
Reported-by: jasal82 on github
Fixes #3264
Closes #3374
- TODO: Windows: set attribute 'archive' for completed downloads
Closes #3354
- RELEASE-NOTES: synced
- http: minor whitespace cleanup from f464535b
- [Ayoub Boudhar brought this change]
http: Implement trailing headers for chunked transfers
This adds the CURLOPT_TRAILERDATA and CURLOPT_TRAILERFUNCTION
options that allow a callback based approach to sending trailing headers
with chunked transfers.
The test server (sws) was updated to take into account the detection of the
end of transfer in the case of trailing headers presence.
Test 1591 checks that trailing headers can be sent using libcurl.
Closes #3350
- darwinssl: accept setting max-tls with default min-tls
Reported-by: Andrei Neculau
Fixes #3367
Closes #3373
- gopher: fix memory leak from 9026083ddb2a9
- [Leonardo Taccari brought this change]
test1201: Add a trailing `?' to the selector
This verify that the `?' in the selector is kept as is.
Verifies the fix in #3370
- [Leonardo Taccari brought this change]
gopher: always include the entire gopher-path in request
After the migration to URL API all octets in the selector after the
first `?' were interpreted as query and accidentally discarded and not
passed to the server.
Add a gopherpath to always concatenate possible path and query URL
pieces.
Fixes #3369
Closes #3370
- [Leonardo Taccari brought this change]
urlapi: distinguish possibly empty query
If just a `?' to indicate the query is passed always store a zero length
query instead of having a NULL query.
This permits to distinguish URL with trailing `?'.
Fixes #3369
Closes #3370
Daniel Gustafsson (13 Dec 2018)
- OS400: handle memory error in list conversion
Curl_slist_append_nodup() returns NULL when it fails to create a new
item for the specified list, and since the coding here reassigned the
new list on top of the old list it would result in a dangling pointer
and lost memory. Also, in case we hit an allocation failure at some
point during the conversion, with allocation succeeding again on the
subsequent call(s) we will return a truncated list around the malloc
failure point. Fix by assigning to a temporary list pointer, which can
be checked (which is the common pattern for slist appending), and free
all the resources on allocation failure.
Closes #3372
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- cookies: leave secure cookies alone
Only allow secure origins to be able to write cookies with the
'secure' flag set. This reduces the risk of non-secure origins
to influence the state of secure origins. This implements IETF
Internet-Draft draft-ietf-httpbis-cookie-alone-01 which updates
RFC6265.
Closes #2956
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (13 Dec 2018)
- docs: fix the --tls-max description
Reported-by: Tobias Lindgren
Pointed out in #3367
Closes #3368
Daniel Gustafsson (12 Dec 2018)
- urlapi: Fix port parsing of eol colon
A URL with a single colon without a portnumber should use the default
port, discarding the colon. Fix, add a testcase and also do little bit
of comment wordsmithing.
Closes #3365
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Version 7.63.0 (12 Dec 2018)
Daniel Stenberg (12 Dec 2018)
- RELEASE-NOTES: 7.63.0
- THANKS: from the curl 7.62.0 cycle
- test1519: use lib1518 and test CURLINFO_REDIRECT_URL more
- Curl_follow: extract the Location: header field unvalidated
... when not actually following the redirect. Otherwise we return error
for this and an application can't extract the value.
Test 1518 added to verify.
Reported-by: Pavel Pavlov
Fixes #3340
Closes #3364
- multi: convert two timeout variables to timediff_t
The time_t type is unsigned on some systems and these variables are used
to hold return values from functions that return timediff_t
already. timediff_t is always a signed type.
Closes #3363
- delta: use --diff-filter on the git diff-tree invokes
Suggested-by: Dave Reisner
Patrick Monnerat (11 Dec 2018)
- documentation: curl_formadd field and file names are now escaped
Prior to 7.56.0, fieldnames and filenames were set in Content-Disposition
header without special processing: this may lead to invalid RFC 822
quoted-strings.
7.56.0 introduces escaping of backslashes and double quotes in these names:
mention it in the documentation.
Reported-by: daboul on github
Closes #3361
Daniel Stenberg (11 Dec 2018)
- scripts/delta: show repo delta info from last release
... where "last release" should be the git tag in the repo.
Daniel Gustafsson (11 Dec 2018)
- tests: add urlapi unittest
This adds a new unittest intended to cover the internal functions in
the urlapi code, starting with parse_port(). In order to avoid name
collisions in debug builds, parse_port() is renamed Curl_parse_port()
since it will be exported.
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
- urlapi: fix portnumber parsing for ipv6 zone index
An IPv6 URL which contains a zone index includes a '%%25<zode id>'
string before the ending ']' bracket. The parsing logic wasn't set
up to cope with the zone index however, resulting in a malformed url
error being returned. Fix by breaking the parsing into two stages
to correctly handle the zone index.
Closes #3355
Closes #3319
Reported-by: tonystz on Github
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Daniel Stenberg (11 Dec 2018)
- [Jay Satiro brought this change]
http: fix HTTP auth to include query in URI
- Include query in the path passed to generate HTTP auth.
Recent changes to use the URL API internally (46e1640, 7.62.0)
inadvertently broke authentication URIs by omitting the query.
Fixes https://github.com/curl/curl/issues/3353
Closes #3356
- [Michael Kaufmann brought this change]
http: don't set CURLINFO_CONDITION_UNMET for http status code 204
The http status code 204 (No Content) should not change the "condition
unmet" flag. Only the http status code 304 (Not Modified) should do
this.
Closes #359
- [Samuel Surtees brought this change]
ldap: fix LDAP URL parsing regressions
- Match URL scheme with LDAP and LDAPS
- Retrieve attributes, scope and filter from URL query instead
Regression brought in 46e164069d1a5230 (7.62.0)
Closes #3362
- RELEASE-NOTES: synced
- [Stefan Kanthak brought this change]
(lib)curl.rc: fixup for minor bugs
All resources defined in lib/libcurl.rc and curl.rc are language
neutral.
winbuild/MakefileBuild.vc ALWAYS defines the macro DEBUGBUILD, so the
ifdef's in line 33 of lib/libcurl.rc and src/curl.rc are wrong.
Replace the hard-coded constants in both *.rc files with #define'd
values.
Thumbs-uped-by: Rod Widdowson, Johannes Schindelin
URL: https://curl.haxx.se/mail/lib-2018-11/0000.html
Closes #3348
- test329: verify cookie max-age=0 immediate expiry
- cookies: expire "Max-Age=0" immediately
Reported-by: Jeroen Ooms
Fixes #3351
Closes #3352
- [Johannes Schindelin brought this change]
Upon HTTP_1_1_REQUIRED, retry the request with HTTP/1.1
This is a companion patch to cbea2fd2c (NTLM: force the connection to
HTTP/1.1, 2018-12-06): with NTLM, we can switch to HTTP/1.1
preemptively. However, with other (Negotiate) authentication it is not
clear to this developer whether there is a way to make it work with
HTTP/2, so let's try HTTP/2 first and fall back in case we encounter the
error HTTP_1_1_REQUIRED.
Note: we will still keep the NTLM workaround, as it avoids an extra
round trip.
Daniel Stenberg helped a lot with this patch, in particular by
suggesting to introduce the Curl_h2_http_1_1_error() function.
Closes #3349
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- [Ben Greear brought this change]
openssl: fix unused variable compiler warning with old openssl
URL: https://curl.haxx.se/mail/lib-2018-11/0055.html
Closes #3347
- [Johannes Schindelin brought this change]
NTLM: force the connection to HTTP/1.1
Since v7.62.0, cURL tries to use HTTP/2 whenever the server announces
the capability. However, NTLM authentication only works with HTTP/1.1,
and will likely remain in that boat (for details, see
https://docs.microsoft.com/en-us/iis/get-started/whats-new-in-iis-10/http2-on-iis#when-is-http2-not-supported).
When we just found out that we want to use NTLM, and when the current
connection runs in HTTP/2 mode, let's force the connection to be closed
and to be re-opened using HTTP/1.1.
Fixes https://github.com/curl/curl/issues/3341.
Closes #3345
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- [Johannes Schindelin brought this change]
curl_global_sslset(): id == -1 is not necessarily an error
It is allowed to call that function with id set to -1, specifying the
backend by the name instead. We should imitate what is done further down
in that function to allow for that.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes #3346
Johannes Schindelin (6 Dec 2018)
- .gitattributes: make tabs in indentation a visible error
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Daniel Stenberg (6 Dec 2018)
- RELEASE-NOTES: synced
- doh: fix memory leak in OOM situation
Reviewed-by: Daniel Gustafsson
Closes #3342
- doh: make it work for h2-disabled builds too
Reported-by: dtmsecurity at github
Fixes #3325
Closes #3336
- packages: remove old leftover files and dirs
This subdir has mostly become an attic of never-used cruft from the
past.
Closes #3331
- [Gergely Nagy brought this change]
openssl: do not use file BIOs if not requested
Moves the file handling BIO calls to the branch of the code where they
are actually used.
Closes #3339
- [Paul Howarth brought this change]
nss: Fix compatibility with nss versions 3.14 to 3.15
- [Paul Howarth brought this change]
nss: Improve info message when falling back SSL protocol
Use descriptive text strings rather than decimal numbers.
- [Paul Howarth brought this change]
nss: Fall back to latest supported SSL version
NSS may be built without support for the latest SSL/TLS versions,
leading to "SSL version range is not valid" errors when the library
code supports a recent version (e.g. TLS v1.3) but it has explicitly
been disabled.
This change adjusts the maximum SSL version requested by libcurl to
be the maximum supported version at runtime, as long as that version
is at least as high as the minimum version required by libcurl.
Fixes #3261
Daniel Gustafsson (3 Dec 2018)
- travis: enable COPYRIGHTYEAR extended warning
The extended warning for checking incorrect COPYRIGHTYEAR is quite
expensive to run, so rather than expecting every developer to do it
we ensure it's turned on locally for Travis.
- checksrc: add COPYRIGHTYEAR check
Forgetting to bump the year in the copyright clause when hacking has
been quite common among curl developers, but a traditional checksrc
check isn't a good fit as it would penalize anyone hacking on January
1st (among other things). This adds a more selective COPYRIGHTYEAR
check which intends to only cover the currently hacked on changeset.
The check for updated copyright year is currently not enforced on all
files but only on files edited and/or committed locally. This is due to
the amount of files which aren't updated with their correct copyright
year at the time of their respective commit.
To further avoid running this expensive check for every developer, it
adds a new local override mode for checksrc where a .checksrc file can
be used to turn on extended warnings locally.
Closes #3303
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (3 Dec 2018)
- CHECKSRC.md: document more warnings
Closes #3335
[ci skip]
- RELEASE-NOTES: synced
- SECURITY-PROCESS: bountygraph shuts down
This backpedals back the documents to the state before bountygraph.
Closes #3311
- curl: fix memory leak reading --writeout from file
If another string had been set first, the writout function for reading
the syntax from file would leak the previously allocated memory.
Reported-by: Brian Carpenter
Fixes #3322
Closes #3330
- tool_main: rename function to make it unique and better
... there's already another function in the curl tool named
free_config_fields!
Daniel Gustafsson (29 Nov 2018)
- TODO: remove CURLOPT_DNS_USE_GLOBAL_CACHE entry
Commit 7c5837e79280e6abb3ae143dfc49bca5e74cdd11 deprecated the option
making it a manual code-edit operation to turn it back on. The removal
process has thus started and is now documented in docs/DEPRECATE.md so
remove from the TODO to avoid anyone looking for something to pick up
spend cycles on an already in-progress entry.
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Jay Satiro (29 Nov 2018)
- [Sevan Janiyan brought this change]
connect: fix building for recent versions of Minix
EBADIOCTL doesn't exist on more recent Minix.
There have also been substantial changes to the network stack.
Fixes build on Minix 3.4rc
Closes https://github.com/curl/curl/pull/3323
- [Konstantin Kushnir brought this change]
CMake: fix MIT/Heimdal Kerberos detection
- fix syntax error in FindGSS.cmake
- correct krb5 include directory. FindGSS exports
"GSS_INCLUDE_DIR" variable.
Closes https://github.com/curl/curl/pull/3316
Daniel Stenberg (28 Nov 2018)
- test328: verify Content-Encoding: none
Because of issue #3315
Closes #3317
- [James Knight brought this change]
configure: include all libraries in ssl-libs fetch
When compiling a collection of SSL libraries to link against (SSL_LIBS),
ensure all libraries are included. The call `--libs-only-l` can produce
only a subset of found in a `--libs` call (e.x. pthread may be excluded).
Adding `--libs-only-other` ensures other libraries are also included in
the list. This corrects select build environments compiling against a
static version of OpenSSL. Before the change, the following could be
observed:
checking for openssl options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lssl -lz -ldl -lcrypto -lz -ldl "
configure: pkg-config: SSL_LDFLAGS: "-L/home/jdknight/<workdir>/staging/usr/lib -L/home/jdknight/<workdir>/staging/usr/lib "
configure: pkg-config: SSL_CPPFLAGS: "-I/home/jdknight/<workdir>/staging/usr/include "
checking for HMAC_Update in -lcrypto... no
checking for HMAC_Init_ex in -lcrypto... no
checking OpenSSL linking with -ldl... no
checking OpenSSL linking with -ldl and -lpthread... no
configure: WARNING: SSL disabled, you will not be able to use HTTPS, FTPS, NTLM and more.
configure: WARNING: Use --with-ssl, --with-gnutls, --with-polarssl, --with-cyassl, --with-nss, --with-axtls, --with-winssl, or --with-darwinssl to address this.
...
SSL support: no (--with-{ssl,gnutls,nss,polarssl,mbedtls,cyassl,axtls,winssl,darwinssl} )
...
And include the other libraries when compiling SSL_LIBS succeeds with:
checking for openssl options with pkg-config... found
configure: pkg-config: SSL_LIBS: "-lssl -lz -ldl -pthread -lcrypto -lz -ldl -pthread "
configure: pkg-config: SSL_LDFLAGS: "-L/home/jdknight/<workdir>/staging/usr/lib -L/home/jdknight/<workdir>/staging/usr/lib "
configure: pkg-config: SSL_CPPFLAGS: "-I/home/jdknight/<workdir>/staging/usr/include "
checking for HMAC_Update in -lcrypto... yes
checking for SSL_connect in -lssl... yes
...
SSL support: enabled (OpenSSL)
...
Signed-off-by: James Knight <james.d.knight@live.com>
Closes #3193
Daniel Gustafsson (26 Nov 2018)
- doh: fix typo in infof call
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- cmdline-opts/gen.pl: define the correct varname
The variable definition had a small typo making it declare another
variable then the intended.
Closes #3304
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (25 Nov 2018)
- RELEASE-NOTES: synced
- curl_easy_perform: fix timeout handling
curl_multi_wait() was erroneously used from within
curl_easy_perform(). It could lead to it believing there was no socket
to wait for and then instead sleep for a while instead of monitoring the
socket and then miss acting on that activity as swiftly as it should
(causing an up to 1000 ms delay).
Reported-by: Antoni Villalonga
Fixes #3305
Closes #3306
Closes #3308
- CURLOPT_WRITEFUNCTION.3: spell out that it gets called many times
- cookies: create the cookiejar even if no cookies to save
Important for when the file is going to be read again and thus must not
contain old contents!
Adds test 327 to verify.
Reported-by: daboul on github
Fixes #3299
Closes #3300
- checksrc: ban snprintf use, add command line flag to override warns
- snprintf: renamed and we now only use msnprintf()
The function does not return the same value as snprintf() normally does,
so readers may be mislead into thinking the code works differently than
it actually does. A different function name makes this easier to detect.
Reported-by: Tomas Hoger
Assisted-by: Daniel Gustafsson
Fixes #3296
Closes #3297
- [Tobias Hintze brought this change]
test: update test20/1322 for eglibc bug workaround
The tests 20 and 1322 are using getaddrinfo of libc for resolving. In
eglibc-2.19 there is a memory leakage and invalid free bug which
surfaces in some special circumstances (PF_UNSPEC hint with invalid or
non-existent names). The valgrind runs in testing fail in these
situations.
As the tests 20/1322 are not specific on either protocol (IPv4/IPv6)
this commit changes the hints to IPv4 protocol by passing `--ipv4` flag
on the tests' command line. This prevents the valgrind failures.
- [Tobias Hintze brought this change]
host names: allow trailing dot in name resolve, then strip it
Delays stripping of trailing dots to after resolving the hostname.
Fixes #3022
Closes #3222
- [UnknownShadow200 brought this change]
CURLOPT_HEADERFUNCTION.3: match 'nitems' name in synopsis and description
Closes #3295
Daniel Gustafsson (21 Nov 2018)
- configure: Fix typo in comment
Michael Kaufmann (21 Nov 2018)
- openssl: support session resume with TLS 1.3
Session resumption information is not available immediately after a TLS 1.3
handshake. The client must wait until the server has sent a session ticket.
Use OpenSSL's "new session" callback to get the session information and put it
into curl's session cache. For TLS 1.3 sessions, this callback will be invoked
after the server has sent a session ticket.
The "new session" callback is invoked only if OpenSSL's session cache is
enabled, so enable it and use the "external storage" mode which lets curl manage
the contents of the session cache.
A pointer to the connection data and the sockindex are now saved as "SSL extra
data" to make them available to the callback.
This approach also works for old SSL/TLS versions and old OpenSSL versions.
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Fixes #3202
Closes #3271
- ssl: fix compilation with OpenSSL 0.9.7
- ENGINE_cleanup() was used without including "openssl/engine.h"
- enable engine support for OpenSSL 0.9.7
Closes #3266
Daniel Stenberg (21 Nov 2018)
- openssl: disable TLS renegotiation with BoringSSL
Since we're close to feature freeze, this change disables this feature
with an #ifdef. Define ALLOW_RENEG at build-time to enable.
This could be converted to a bit for CURLOPT_SSL_OPTIONS to let
applications opt-in this.
Concern-raised-by: David Benjamin
Fixes #3283
Closes #3293
- [Romain Fliedel brought this change]
ares: remove fd from multi fd set when ares is about to close the fd
When using c-ares for asyn dns, the dns socket fd was silently closed
by c-ares without curl being aware. curl would then 'realize' the fd
has been removed at next call of Curl_resolver_getsock, and only then
notify the CURLMOPT_SOCKETFUNCTION to remove fd from its poll set with
CURL_POLL_REMOVE. At this point the fd is already closed.
By using ares socket state callback (ARES_OPT_SOCK_STATE_CB), this
patch allows curl to be notified that the fd is not longer needed
for neither for write nor read. At this point by calling
Curl_multi_closed we are able to notify multi with CURL_POLL_REMOVE
before the fd is actually closed by ares.
In asyn-ares.c Curl_resolver_duphandle we can't use ares_dup anymore
since it does not allow passing a different sock_state_cb_data
Closes #3238
- [Romain Fliedel brought this change]
examples/ephiperfifo: report error when epoll_ctl fails
Daniel Gustafsson (20 Nov 2018)
- [pkubaj brought this change]
ntlm: Remove redundant ifdef USE_OPENSSL
lib/curl_ntlm.c had code that read as follows:
#ifdef USE_OPENSSL
# ifdef USE_OPENSSL
# else
# ..
# endif
#endif
Remove the redundant USE_OPENSSL along with #else (it's not possible to
reach it anyway). The removed construction is a leftover from when the
SSLeay support was removed.
Closes #3269
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (20 Nov 2018)
- [Han Han brought this change]
ssl: replace all internal uses of CURLE_SSL_CACERT
Closes #3291
Han Han (19 Nov 2018)
- docs: add more description to unified ssl error codes
- curle: move deprecated error code to ifndef block
Patrick Monnerat (19 Nov 2018)
- os400: add CURLOPT_CURLU to ILE/RPG binding.
- os400: Add curl_easy_conn_upkeep() to ILE/RPG binding.
- os400: fix return type of curl_easy_pause() in ILE/RPG binding.
Daniel Stenberg (19 Nov 2018)
- RELEASE-NOTES: synced
- impacket: add LICENSE
The license for the impacket package was not in our tree.
Imported now from upstream's
https://github.com/SecureAuthCorp/impacket/blob/master/LICENSE
Reported-by: infinnovation-dev on github
Fixes #3276
Closes #3277
Daniel Gustafsson (18 Nov 2018)
- tool_doswin: Fix uninitialized field warning
The partial struct initialization in 397664a065abffb7c3445ca9 caused
a warning on uninitialized MODULEENTRY32 struct members:
/src/tool_doswin.c:681:3: warning: missing initializer for field
'th32ModuleID' of 'MODULEENTRY32 {aka struct tagMODULEENTRY32}'
[-Wmissing-field-initializers]
This is sort of a bogus warning as the remaining members will be set
to zero by the compiler, as all omitted members are. Nevertheless,
remove the warning by omitting all members and setting the dwSize
members explicitly.
Closes #3254
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
- openssl: Remove SSLEAY leftovers
Commit 709cf76f6bb7dbac deprecated USE_SSLEAY, as curl since long isn't
compatible with the SSLeay library. This removes the few leftovers that
were omitted in the less frequently used platform targets.
Closes #3270
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (16 Nov 2018)
- [Elia Tufarolo brought this change]
http_negotiate: do not close connection until negotiation is completed
Fix HTTP POST using CURLAUTH_NEGOTIATE.
Closes #3275
- pop3: only do APOP with a valid timestamp
Brought-by: bobmitchell1956 on github
Fixes #3278
Closes #3279
Jay Satiro (16 Nov 2018)
- [Peter Wu brought this change]
openssl: do not log excess "TLS app data" lines for TLS 1.3
The SSL_CTX_set_msg_callback callback is not just called for the
Handshake or Alert protocols, but also for the raw record header
(SSL3_RT_HEADER) and the decrypted inner record type
(SSL3_RT_INNER_CONTENT_TYPE). Be sure to ignore the latter to avoid
excess debug spam when using `curl -v` against a TLSv1.3-enabled server:
* TLSv1.3 (IN), TLS app data, [no content] (0):
(Following this message, another callback for the decrypted
handshake/alert messages will be be present anyway.)
Closes https://github.com/curl/curl/pull/3281
Marc Hoersken (15 Nov 2018)
- tests: disable SO_EXCLUSIVEADDRUSE for stunnel on Windows
SO_EXCLUSIVEADDRUSE is on by default on Vista or newer,
but does not work together with SO_REUSEADDR being on.
The default changes were made with stunnel 5.34 and 5.35.
Daniel Stenberg (13 Nov 2018)
- [Kamil Dudka brought this change]
nss: remove version selecting dead code
Closes #3262
- nss: set default max-tls to 1.3/1.2
Fixes #3261
Daniel Gustafsson (13 Nov 2018)
- tool_cb_wrt: Silence function cast compiler warning
Commit 5bfaa86ceb3c2a9ac474a928e748c4a86a703b33 introduced a new
compiler warning on Windows cross compilation with GCC. See below
for an example of the warning from the autobuild logs (whitespace
edited to fit):
/src/tool_cb_wrt.c:175:9: warning: cast from function call of type
'intptr_t {aka long long int}' to non-matching type 'void *'
[-Wbad-function-cast]
(HANDLE) _get_osfhandle(fileno(outs->stream)),
^
Store the return value from _get_osfhandle() in an intermediate
variable and cast the variable in WriteConsoleW() rather than the
function call directly to avoid a compiler warning.
In passing, also add inspection of the MultiByteToWideChar() return
value and return failure in case an error is reported.
Closes #3263
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Reviewed-by: Viktor Szakats <commit@vszakats.net>
Daniel Stenberg (12 Nov 2018)
- nss: fix fallthrough comment to fix picky compiler warning
- docs: expanded on some CURLU details
- [Tim Rühsen brought this change]
ftp: avoid two unsigned int overflows in FTP listing parser
Curl_ftp_parselist: avoid unsigned integer overflows
The overflow has no real world impact, just avoid it for "best
practice".
Closes #3225
- curl: --local-port range was not "including"
The end port number in a given range was not included in the range used,
as it is documented to be.
Reported-by: infinnovation-dev on github
Fixes #3251
Closes #3255
- [Jérémy Rocher brought this change]
openssl: support BoringSSL TLS renegotiation
As per BoringSSL porting documentation [1], BoringSSL rejects peer
renegotiations by default.
curl fails when trying to authenticate to server through client
certificate if it is requested by server after the initial TLS
handshake.
Enable renegotiation by default with BoringSSL to get same behavior as
with OpenSSL. This is done by calling SSL_set_renegotiate_mode [2]
which was introduced in commit 1d5ef3bb1eb9 [3].
1 - https://boringssl.googlesource.com/boringssl/+/HEAD/PORTING.md#tls-renegotiation
2 - https://boringssl.googlesource.com/boringssl/+/master/include/openssl/ssl.h#3482
3 - https://boringssl.googlesource.com/boringssl/+/1d5ef3bb1eb97848617db5e7d633d735a401df86
Signed-off-by: Jérémy Rocher <rocher.jeremy@gmail.com>
Fixes #3258
Closes #3259
- HISTORY: add some milestones
Added a few of the more notable milestones in curl history that were
missing. Primarily more recent ones but I also noted some older that
could be worth mentioning.
[ci skip]
Closes #3257
Daniel Gustafsson (9 Nov 2018)
- KNOWN_BUGS: add --proxy-any connection issue
Add the identified issue with --proxy-any and proxy servers which
advertise authentication schemes other than the supported one.
Closes #876
Closes #3250
Reported-by: NTMan on Github
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (9 Nov 2018)
- [Jim Fuller brought this change]
setopt: add CURLOPT_CURLU
Allows an application to pass in a pre-parsed URL via a URL handle.
Closes #3227
- [Gisle Vanem brought this change]
docs: ESCape "\n" codes
Groff / Troff will display a:
printaf("Errno: %ld\n", error);
as:
printf("Errno: %ld0, error);
when a "\n" is not escaped. Use "\\n" instead.
Closes #3246
- curl: --local-port fix followup
Regression by 52db54869e6.
Reported-by: infinnovation-dev on github
Fixes #3248
Closes #3249
GitHub (7 Nov 2018)
- [Gisle Vanem brought this change]
More "\n" ESCaping
Daniel Stenberg (7 Nov 2018)
- RELEASE-NOTES: synced
- curl: fix --local-port integer overflow
The tool's local port command line range parser didn't check for integer
overflows and could pass "weird" data to libcurl for this option.
libcurl however, has a strict range check for the values so it rejects
anything outside of the accepted range.
Reported-by: Brian Carpenter
Closes #3242
- curl: correct the switch() logic in ourWriteOut
Follow-up to e431daf013, as I did the wrong correction for a compiler
warning. It should be a break and not a fall-through.
Pointed-out-by: Frank Gevaerts
- [Frank Gevaerts brought this change]
curl: add %{stderr} and %{stdout} for --write-out
Closes #3115
Daniel Gustafsson (7 Nov 2018)
- winssl: be consistent in Schannel capitalization
The productname from Microsoft is "Schannel", but in infof/failf
reporting we use "schannel". This removes different versions.
Closes #3243
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (7 Nov 2018)
- TODO: Have the URL API offer IDN decoding
Similar to how URL decoding/encoding is done, we could have URL
functions to convert IDN host names to punycode.
Suggested-by: Alexey Melnichuk
Closes #3232
- urlapi: only skip encoding the first '=' with APPENDQUERY set
APPENDQUERY + URLENCODE would skip all equals signs but now it only skip
encoding the first to better allow "name=content" for any content.
Reported-by: Alexey Melnichuk
Fixes #3231
Closes #3231
- url: a short host name + port is not a scheme
The function identifying a leading "scheme" part of the URL considered a
few letters ending with a colon to be a scheme, making something like
"short:80" to become an unknown scheme instead of a short host name and
a port number.
Extended test 1560 to verify.
Also fixed test203 to use file_pwd to make it get the correct path on
windows. Removed test 2070 since it was a duplicate of 203.
Assisted-by: Marcel Raad
Reported-by: Hagai Auro
Fixes #3220
Fixes #3233
Closes #3223
Closes #3235
- [Sangamkar brought this change]
libcurl: stop reading from paused transfers
In the transfer loop it would previously not acknwledge the pause bit
and continue until drained or loop ended.
Closes #3240
Jay Satiro (6 Nov 2018)
- tool: add undocumented option --dump-module-paths for win32
- Add an undocumented diagnostic option for Windows to show the full
paths of all loaded modules regardless of whether or not libcurl
initialization succeeds.
This is needed so that in the CI we can get a list of all DLL
dependencies after initialization (when they're most likely to have
finished loading) and then package them as artifacts so that a
functioning build can be downloaded. Also I imagine it may have some use
as a diagnostic for help requests.
Ref: https://github.com/curl/curl/pull/3103
Closes https://github.com/curl/curl/pull/3208
- curl_multibyte: fix a malloc overcalculation
Prior to this change twice as many bytes as necessary were malloc'd when
converting wchar to UTF8. To allay confusion in the future I also
changed the variable name for the amount of bytes from len to bytes.
Closes https://github.com/curl/curl/pull/3209
Michael Kaufmann (5 Nov 2018)
- netrc: don't ignore the login name specified with "--user"
- for "--netrc", don't ignore the login/password specified with "--user",
only ignore the login/password in the URL.
This restores the netrc behaviour of curl 7.61.1 and earlier.
- fix the documentation of CURL_NETRC_REQUIRED
- improve the detection of login/password changes when reading .netrc
- don't read .netrc if both login and password are already set
Fixes #3213
Closes #3224
Patrick Monnerat (5 Nov 2018)
- OS400: add URL API ccsid wrappers and sync ILE/RPG bindings
Daniel Stenberg (5 Nov 2018)
- [Yasuhiro Matsumoto brought this change]
curl: fixed UTF-8 in current console code page (Windows)
Fixes #3211
Fixes #3175
Closes #3212
- TODO: 2.6 multi upkeep
Closes #3199
Daniel Gustafsson (5 Nov 2018)
- unittest: make 1652 stable across collations
The previous coding used a format string whose output depended on the
current locale of the environment running the test. Since the gist of
the test is to have a format string, with the actual formatting being
less important, switch to a more stable formatstring with decimals.
Reported-by: Marcel Raad
Closes #3234
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Daniel Stenberg (5 Nov 2018)
- Revert "url: a short host name + port is not a scheme"
This reverts commit 226cfa8264cd979eff3fd52c0f3585ef095e7cf2.
This commit caused test failures on appveyor/windows. Work on fixing them is
in #3235.
- symbols-in-versions: add missing CURLU_ symbols
...and fix symbol-scan.pl to also scan urlapi.h
Reported-by: Alexey Melnichuk
Fixes #3226
Closes #3230
Daniel Gustafsson (3 Nov 2018)
- infof: clearly indicate truncation
The internal buffer in infof() is limited to 2048 bytes of payload plus
an additional byte for NULL termination. Servers with very long error
messages can however cause truncation of the string, which currently
isn't very clear, and leads to badly formatted output.
This appends a "...\n" (or just "..." in case the format didn't with a
newline char) marker to the end of the string to clearly show
that it has been truncated.
Also include a unittest covering infof() to try and catch any bugs
introduced in this quite important function.
Closes #3216
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Michael Kaufmann (3 Nov 2018)
- tool_getparam: fix some comments
Daniel Stenberg (3 Nov 2018)
- url: a short host name + port is not a scheme
The function identifying a leading "scheme" part of the URL considered a few
letters ending with a colon to be a scheme, making something like "short:80"
to become an unknown scheme instead of a short host name and a port number.
Extended test 1560 to verify.
Reported-by: Hagai Auro
Fixes #3220
Closes #3223
- URL: fix IPv6 numeral address parser
Regression from 46e164069d1a52. Extended test 1560 to verify.
Reported-by: tpaukrt on github
Fixes #3218
Closes #3219
- travis: remove curl before a normal build
on Linux. To make sure the test suite runs with its newly build tool and
doesn't require an external one present.
Bug: #3198
Closes #3200
- [Tim Rühsen brought this change]
mprintf: avoid unsigned integer overflow warning
The overflow has no real world impact.
Just avoid it for "best practice".
Code change suggested by "The Infinnovation Team" and Daniel Stenberg.
Closes #3184
- Curl_follow: accept non-supported schemes for "fake" redirects
When not actually following the redirect and the target URL is only
stored for later retrieval, curl always accepted "non-supported"
schemes. This was a regression from 46e164069d1a5230.
Reported-by: Brad King
Fixes #3210
Closes #3215
Daniel Gustafsson (2 Nov 2018)
- openvms: fix example name
Commit efc696a2e09225bfeab4 renamed persistant.c to persistent.c to
fix the typo in the name, but missed to update the OpenVMS package
files which still looked for the old name.
Closes #3217
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Viktor Szakats <commit@vszakats.net>
Daniel Stenberg (1 Nov 2018)
- configure: show CFLAGS, LDFLAGS etc in summary
To make it easier to understand other people's and remote builds etc.
Closes #3207
- version: bump for next cycle
- axtls: removed
As has been outlined in the DEPRECATE.md document, the axTLS code has
been disabled for 6 months and is hereby removed.
Use a better supported TLS library!
Assisted-by: Daniel Gustafsson
Closes #3194
- [marcosdiazr brought this change]
schannel: make CURLOPT_CERTINFO support using Issuer chain
Closes #3197
- travis: build with sanitize=address,undefined,signed-integer-overflow
... using clang
Closes #3190
- schannel: use Curl_ prefix for global private symbols
Curl_verify_certificate() must use the Curl_ prefix since it is globally
available in the lib and otherwise steps outside of our namespace!
Closes #3201
Kamil Dudka (1 Nov 2018)
- tests: drop http_pipe.py script no longer used
It is unused since commit f7208df7d9d5cd5e15e2d89237e828f32b63f135.
Closes #3204
Daniel Stenberg (31 Oct 2018)
- runtests: use the local curl for verifying
... revert the mistaken change brought in commit 8440616f53.
Reported-by: Alessandro Ghedini
Bug: https://curl.haxx.se/mail/lib-2018-10/0118.html
Closes #3198
Version 7.62.0 (30 Oct 2018)
Daniel Stenberg (30 Oct 2018)
- RELEASE-NOTES: 7.62.0
- THANKS: 7.62.0 status
Daniel Gustafsson (30 Oct 2018)
- vtls: add MesaLink to curl_sslbackend enum
MesaLink support was added in commit 57348eb97d1b8fc3742e02c but the
backend was never added to the curl_sslbackend enum in curl/curl.h.
This adds the new backend to the enum and updates the relevant docs.
Closes #3195
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (30 Oct 2018)
- [Ruslan Baratov brought this change]
cmake: Remove unused CURL_CONFIG_HAS_BEEN_RUN_BEFORE variable
Closes #3191
- test2080: verify the fix for CVE-2018-16842
- voutf: fix bad arethmetic when outputting warnings to stderr
CVE-2018-16842
Reported-by: Brian Carpenter
Bug: https://curl.haxx.se/docs/CVE-2018-16842.html
- [Tuomo Rinne brought this change]
cmake: uniform ZLIB to use USE_ variable and clean curl-config.cmake.in
Closes #3123
- [Tuomo Rinne brought this change]
cmake: add find_dependency call for ZLIB to CMake config file
- [Tuomo Rinne brought this change]
cmake: add support for transitive ZLIB target
- unit1650: fix "null pointer passed as argument 1 to memcmp"
Detected by UndefinedBehaviorSanitizer
Closes #3187
- travis: add a "make tidy" build that runs clang-tidy
Closes #3182
- unit1300: fix stack-use-after-scope AddressSanitizer warning
Closes #3186
- Curl_auth_create_plain_message: fix too-large-input-check
CVE-2018-16839
Reported-by: Harry Sintonen
Bug: https://curl.haxx.se/docs/CVE-2018-16839.html
- Curl_close: clear data->multi_easy on free to avoid use-after-free
Regression from b46cfbc068 (7.59.0)
CVE-2018-16840
Reported-by: Brian Carpenter (Geeknik Labs)
Bug: https://curl.haxx.se/docs/CVE-2018-16840.html
- [randomswdev brought this change]
system.h: use proper setting with Sun C++ as well
system.h selects the proper Sun settings when __SUNPRO_C is defined. The
Sun compiler does not define it when compiling C++ files. I'm adding a
check also on __SUNPRO_CC to allow curl to work properly also when used
in a C++ project on Sun Solaris.
Closes #3181
- rand: add comment to skip a clang-tidy false positive
- test1651: unit test Curl_extract_certinfo()
The version used for Gskit, NSS, GnuTLS, WolfSSL and schannel.
- x509asn1: always check return code from getASN1Element()
- Makefile: add 'tidy' target that runs clang-tidy
Available in the root, src and lib dirs.
Closes #3163
- RELEASE-PROCEDURE: adjust the release dates
See: https://curl.haxx.se/mail/lib-2018-10/0107.html
Patrick Monnerat (27 Oct 2018)
- x509asn1: suppress left shift on signed value
Use an unsigned variable: as the signed operation behavior is undefined,
this change silents clang-tidy about it.
Ref: https://github.com/curl/curl/pull/3163
Reported-By: Daniel Stenberg
Michael Kaufmann (27 Oct 2018)
- multi: Fix error handling in the SENDPROTOCONNECT state
If Curl_protocol_connect() returns an error code,
handle the error instead of switching to the next state.
Closes #3170
Daniel Stenberg (27 Oct 2018)
- RELEASE-NOTES: synced
- openssl: output the correct cipher list on TLS 1.3 error
When failing to set the 1.3 cipher suite, the wrong string pointer would
be used in the error message. Most often saying "(nil)".
Reported-by: Ricky-Tigg on github
Fixes #3178
Closes #3180
- docs/CIPHERS: fix the TLS 1.3 cipher names
... picked straight from the OpenSSL man page:
https://www.openssl.org/docs/manmaster/man3/SSL_CTX_set_ciphersuites.html
Reported-by: Ricky-Tigg on github
Bug: #3178
Marcel Raad (27 Oct 2018)
- travis: install gnutls-bin package
This is required for gnutls-serv, which enables a few more tests.
Closes https://github.com/curl/curl/pull/2958
Daniel Gustafsson (26 Oct 2018)
- ssh: free the session on init failures
Ensure to clear the session object in case the libssh2 initialization
fails.
It could be argued that the libssh2 error function should be called to
get a proper error message in this case. But since the only error path
in libssh2_knownhost_init() is memory a allocation failure it's safest
to avoid since the libssh2 error handling allocates memory.
Closes #3179
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (26 Oct 2018)
- docs/RELEASE-PROCEDURE: remove old entries, modify the Dec 2018 date
... I'm moving it up one week due to travels. The rest stays.
- [Daniel Gustafsson brought this change]
openssl: make 'done' a proper boolean
Closes #3176
- gtls: Values stored to but never read
Detected by clang-tidy
Closes #3176
- [Alexey Eremikhin brought this change]
curl.1: --ipv6 mutexes ipv4 (fixed typo)
Fixes #3171
Closes #3172
- tool_main: make TerminalSettings static
Reported-by: Gisle Vanem
Bug: https://github.com/curl/curl/commit/becfe1233ff2b6b0c3e1b6a10048b55b68c2539f#commitcomment-31008819
Closes #3161
- curl-config.in: remove dependency on bc
Reported-by: Dima Pasechnik
Fixes #3143
Closes #3174
- [Gisle Vanem brought this change]
rtmp: fix for compiling with lwIP
Compiling on _WIN32 and with USE_LWIPSOCK, causes this error:
curl_rtmp.c(223,3): error: use of undeclared identifier 'setsockopt'
setsockopt(r->m_sb.sb_socket, SOL_SOCKET, SO_RCVTIMEO,
^
curl_rtmp.c(41,32): note: expanded from macro 'setsockopt'
#define setsockopt(a,b,c,d,e) (setsockopt)(a,b,c,(const char *)d,(int)e)
^
Closes #3155
- configure: remove CURL_CONFIGURE_CURL_SOCKLEN_T
Follow-up to #3166 which did the cmake part of this. This type/define is
not used.
Closes #3168
- [Ruslan Baratov brought this change]
cmake: remove unused variables
Remove variables:
* HAVE_SOCKLEN_T
* CURL_SIZEOF_CURL_SOCKLEN_T
* CURL_TYPEOF_CURL_SOCKLEN_T
Closes #3166
Michael Kaufmann (25 Oct 2018)
- urldata: Fix comment in header
The "connecting" function is used by multiple protocols, not only FTP
- netrc: free temporary strings if memory allocation fails
- Change the inout parameters after all needed memory has been
allocated. Do not change them if something goes wrong.
- Free the allocated temporary strings if strdup() fails.
Closes #3122
Daniel Stenberg (24 Oct 2018)
- [Ruslan Baratov brought this change]
config: Remove unused SIZEOF_VOIDP
Closes #3162
- RELEASE-NOTES: synced
GitHub (23 Oct 2018)
- [Gisle Vanem brought this change]
Fix for compiling with lwIP (3)
lwIP on Windows does not have a WSAIoctl() function.
But it do have a SO_SNDBUF option to lwip_setsockopt(). But it currently does nothing.
Daniel Stenberg (23 Oct 2018)
- Curl_follow: return better errors on URL problems
... by making the converter function global and accessible.
Closes #3153
- Curl_follow: remove remaining free(newurl)
Follow-up to 05564e750e8f0c. This function no longer frees the passed-in
URL.
Reported-by: Michael Kaufmann
Bug: https://github.com/curl/curl/commit/05564e750e8f0c79016c680f301ce251e6e86155#commitcomm
ent-30985666
Daniel Gustafsson (23 Oct 2018)
- headers: end all headers with guard comment
Most headerfiles end with a /* <headerguard> */ comment, but it was
missing from some. The comment isn't the most important part of our
code documentation but consistency has an intrinsic value in itself.
This adds header guard comments to the files that were lacking it.
Closes #3158
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Jay Satiro (23 Oct 2018)
- CIPHERS.md: Mention the options used to set TLS 1.3 ciphers
Closes https://github.com/curl/curl/pull/3159
Daniel Stenberg (20 Oct 2018)
- docs/BUG-BOUNTY: the sponsors actually decide the amount
Retract the previous approach as the sponsors will be the ones to set the
final amounts.
Closes #3152
[ci skip]
- multi: avoid double-free
Curl_follow() no longer frees the string. Make sure it happens in the
caller function, like we normally handle allocations.
This bug was introduced with the use of the URL API internally, it has
never been in a release version
Reported-by: Dario Weißer
Closes #3149
- multi: make the closure handle "inherit" CURLOPT_NOSIGNAL
Otherwise, closing that handle can still cause surprises!
Reported-by: Martin Ankerl
Fixes #3138
Closes #3147
Marcel Raad (19 Oct 2018)
- VS projects: add USE_IPV6
The Visual Studio builds didn't use IPv6. Add it to all projects since
Visual Studio 2008, which is verified to build via AppVeyor.
Closes https://github.com/curl/curl/pull/3137
- config_win32: enable LDAPS
As done in the autotools and CMake builds by default.
Closes https://github.com/curl/curl/pull/3137
Daniel Stenberg (18 Oct 2018)
- travis: add build for "configure --disable-verbose"
Closes #3144
Kamil Dudka (17 Oct 2018)
- tool_cb_hdr: handle failure of rename()
Detected by Coverity.
Closes #3140
Reviewed-by: Jay Satiro
Daniel Stenberg (17 Oct 2018)
- RELEASE-NOTES: synced
- docs/SECURITY-PROCESS: the hackerone IBB program drops curl
... now there's only BountyGraph.
Jay Satiro (16 Oct 2018)
- [Matthew Whitehead brought this change]
x509asn1: Fix SAN IP address verification
For IP addresses in the subject alternative name field, the length
of the IP address (and hence the number of bytes to perform a
memcmp on) is incorrectly calculated to be zero. The code previously
subtracted q from name.end. where in a successful case q = name.end
and therefore addrlen equalled 0. The change modifies the code to
subtract name.beg from name.end to calculate the length correctly.
The issue only affects libcurl with GSKit SSL, not other SSL backends.
The issue is not a security issue as IP verification would always fail.
Fixes #3102
Closes #3141
Daniel Gustafsson (15 Oct 2018)
- INSTALL: mention mesalink in TLS section
Commit 57348eb97d1b8fc3742e02c6587d2d02ff592da5 added support for the
MesaLink vtls backend, but missed updating the TLS section containing
supported backends in the docs.
Closes #3134
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Marcel Raad (14 Oct 2018)
- nonblock: fix unused parameter warning
If USE_BLOCKING_SOCKETS is defined, curlx_nonblock's arguments are not
used.
Michael Kaufmann (13 Oct 2018)
- Curl_follow: Always free the passed new URL
Closes #3124
Viktor Szakats (12 Oct 2018)
- replace rawgit links [ci skip]
Ref: https://rawgit.com/ "RawGit has reached the end of its useful life"
Ref: https://news.ycombinator.com/item?id=18202481
Closes https://github.com/curl/curl/pull/3131
Daniel Stenberg (12 Oct 2018)
- docs/BUG-BOUNTY.md: for vulns published since Aug 1st 2018
[ci skip]
- travis: make distcheck scan for BOM markers
and remove BOM from projects/wolfssl_override.props
Closes #3126
Marcel Raad (11 Oct 2018)
- CMake: remove BOM
Accidentally aded in commit 1bb86057ff07083deeb0b00f8ad35879ec4d03ea.
Reported-by: Viktor Szakats
Ref: https://github.com/curl/curl/pull/3120#issuecomment-428673136
Daniel Gustafsson (10 Oct 2018)
- transfer: fix typo in comment
Michael Kaufmann (10 Oct 2018)
- docs: add "see also" links for SSL options
- link TLS 1.2 and TLS 1.3 options
- link proxy and non-proxy options
Closes #3121
Marcel Raad (10 Oct 2018)
- AppVeyor: remove BDIR variable that sneaked in again
Removed in ae762e1abebe3a5fe75658583c85059a0957ef6e, accidentally added
again in 9f3be5672dc4dda30ab43e0152e13d714a84d762.
- CMake: disable -Wpedantic-ms-format
As done in the autotools build. This is required for MinGW, which
supports only %I64 for printing 64-bit values, but warns about it.
Closes https://github.com/curl/curl/pull/3120
Viktor Szakats (9 Oct 2018)
- ldap: show precise LDAP call in error message on Windows
Also add a unique but common text ('bind via') to make it
easy to grep this specific failure regardless of platform.
Ref: https://github.com/curl/curl/pull/878/files#diff-7a636f08047c4edb53a240f540b4ecf6R468
Closes https://github.com/curl/curl/pull/3118
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Daniel Stenberg (9 Oct 2018)
- docs/DEPRECATE: minor reformat to render nicer on web
Daniel Gustafsson (9 Oct 2018)
- CURLOPT_SSL_VERIFYSTATUS: Fix typo
Changes s/OSCP/OCSP/ and bumps the copyright year due to the change.
Marcel Raad (9 Oct 2018)
- curl_setup: define NOGDI on Windows
This avoids an ERROR macro clash between <wingdi.h> and <arpa/tftp.h>
on MinGW.
Closes https://github.com/curl/curl/pull/3113
- Windows: fixes for MinGW targeting Windows Vista
Classic MinGW has neither InitializeCriticalSectionEx nor
GetTickCount64, independent of the target Windows version.
Closes https://github.com/curl/curl/pull/3113
Daniel Stenberg (8 Oct 2018)
- TODO: fixed 'API for URL parsing/splitting'
Daniel Gustafsson (8 Oct 2018)
- KNOWN_BUGS: Fix various typos
Closes #3112
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Viktor Szakats (8 Oct 2018)
- spelling fixes [ci skip]
as detected by codespell 1.14.0
Closes https://github.com/curl/curl/pull/3114
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Daniel Stenberg (8 Oct 2018)
- RELEASE-NOTES: synced
- curl_ntlm_wb: check aprintf() return codes
... when they return NULL we're out of memory and MUST return failure.
closes #3111
- docs/BUG-BOUNTY: proposed additional docs
Bug bounty explainer. See https://bountygraph.com/programs/curl
Closes #3067
- [Rick Deist brought this change]
hostip: fix check on Curl_shuffle_addr return value
Closes #3110
- FILE: fix CURLOPT_NOBODY and CURLOPT_HEADER output
Now FILE transfers send headers to the header callback like HTTP and
other protocols. Also made curl_easy_getinfo(...CURLINFO_PROTOCOL...)
work for FILE in the callbacks.
Makes "curl -i file://.." and "curl -I file://.." work like before
again. Applied the bold header logic to them too.
Regression from c1c2762 (7.61.0)
Reported-by: Shaun Jackman
Fixes #3083
Closes #3101
Daniel Gustafsson (7 Oct 2018)
- gskit: make sure to terminate version string
In case a very small buffer was passed to the version function, it could
result in the buffer not being NULL-terminated since strncpy() doesn't
guarantee a terminator on an overflowed buffer. Rather than adding code
to terminate (and handle zero-sized buffers), move to using snprintf()
instead like all the other vtls backends.
Closes #3105
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Viktor Szakats <commit@vszakats.net>
- TODO: add LD_PRELOAD support on macOS
Add DYLD_INSERT_LIBRARIES support to the TODO list. Reported in #2394.
- runtests: skip ld_preload tests on macOS
The LD_PRELOAD functionality doesn't exist on macOS, so skip any tests
requiring it.
Fixes #2394
Closes #3106
Reported-by: Github user @jakirkham
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Marcel Raad (7 Oct 2018)
- AppVeyor: use Debug builds to run tests
This enables more tests.
Closes https://github.com/curl/curl/pull/3104
- AppVeyor: add HTTP_ONLY build
Closes https://github.com/curl/curl/pull/3104
- AppVeyor: add WinSSL builds
Use the oldest and latest Windows SDKs for them.
Also, remove all but one OpenSSL build.
Closes https://github.com/curl/curl/pull/3104
- AppVeyor: add remaining Visual Studio versions
This adds Visual Studio 9 and 10 builds.
There's no 64-bit VC9 compiler on AppVeyor, so use it as the Win32
build. Also, VC9 cannot be used for running the test suite.
Closes https://github.com/curl/curl/pull/3104
- AppVeyor: break long line
Closes https://github.com/curl/curl/pull/3104
- AppVeyor: remove unused BDIR variable
Closes https://github.com/curl/curl/pull/3104
Daniel Stenberg (6 Oct 2018)
- test2100: test DoH using IPv4-only
To make it only send one DoH request and avoid the race condition that
could lead to the requests getting sent in reversed order and thus
making it hard to compare in the test case.
Fixes #3107
Closes #3108
- tests/FILEFORMAT: mention how to use <fileN> and <stripfileN> too
[ci skip]
- RELEASE-NOTES: synced
- [Dmitry Kostjuchenko brought this change]
timeval: fix use of weak symbol clock_gettime() on Apple platforms
Closes #3048
- doh: keep the IPv4 address in (original) network byte order
Ideally this will fix the reversed order shown in SPARC tests:
resp 8: Expected 127.0.0.1 got 1.0.0.127
Closes #3091
Jay Satiro (5 Oct 2018)
- INTERNALS.md: wrap lines longer than 79
Daniel Gustafsson (5 Oct 2018)
- INTERNALS: escape reference to parameter
The parameter reference <string> was causing rendering issues in the
generated HTML page, as <string> isn't a valid HTML tag. Fix by back-
tick escaping it.
Closes #3099
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- checksrc: handle zero scoped ignore commands
If a !checksrc! disable command specified to ignore zero errors, it was
still added to the ignore block even though nothing was ignored. While
there were no blocks ignored that shouldn't be ignored, the processing
ended with with a warning:
<filename>:<line>:<col>: warning: Unused ignore: LONGLINE (UNUSEDIGNORE)
/* !checksrc! disable LONGLINE 0 */
^
Fix by instead treating a zero ignore as a a badcommand and throw a
warning for that one.
Closes #3096
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- checksrc: enable strict mode and warnings
Enable strict and warnings mode for checksrc to ensure we aren't missing
anything due to bugs in the checking code. This uncovered a few things
which are all fixed in this commit:
* several variables were used uninitialized
* several variables were not defined in the correct scope
* the whitelist filehandle was read even if the file didn't exist
* the enable_warn() call when a disable counter had expired was passing
incorrect variables, but since the checkwarn() call is unlikely to hit
(the counter is only decremented to zero on actual ignores) it didn't
manifest a problem.
Closes #3090
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Marcel Raad (5 Oct 2018)
- CMake: suppress MSVC warning C4127 for libtest
It's issued by older Windows SDKs (prior to version 8.0).
Sergei Nikulov (5 Oct 2018)
- Merge branch 'dmitrykos-fix_missing_CMake_defines'
- [Dmitry Kostjuchenko brought this change]
cmake: test and set missed defines during configuration
Added configuration checks for HAVE_BUILTIN_AVAILABLE and HAVE_CLOCK_GETTIME_MONOTONIC.
Closes #3097
Marcel Raad (5 Oct 2018)
- AppVeyor: disable test 500
It almost always results in
"starttransfer vs total: 0.000001 0.000000".
I cannot reproduce this locally, so disable it for now.
Closes https://github.com/curl/curl/pull/3100
- AppVeyor: set custom install prefix
CMake's default has spaces and in 32-bit mode parentheses, which result
in syntax errors in curl-config.
Closes https://github.com/curl/curl/pull/3100
- AppVeyor: Remove non-SSL non-test builds
They don't add much value.
Closes https://github.com/curl/curl/pull/3100
- AppVeyor: run test suite
Use the preinstalled MSYS2 bash for that.
Disable test 1139 as the CMake build doesn't generate curl.1.
Ref: https://github.com/curl/curl/issues/3070#issuecomment-425922224
Closes https://github.com/curl/curl/pull/3100
- AppVeyor: use in-tree build
Required to run the tests.
Closes https://github.com/curl/curl/pull/3100
Daniel Stenberg (4 Oct 2018)
- doh: make sure TTL isn't re-inited by second (discarded?) response
Closes #3092
- test320: strip out more HTML when comparing
To make the test case work with different gnutls-serv versions better.
Reported-by: Kamil Dudka
Fixes #3093
Closes #3094
Marcel Raad (4 Oct 2018)
- runtests: use Windows paths for Windows curl
curl generated by CMake's Visual Studio generator has "Windows" in the
version number.
Daniel Stenberg (4 Oct 2018)
- [Colin Hogben brought this change]
tests/negtelnetserver.py: fix Python2-ism in neg TELNET server
Fix problems caused by differences in treatment of bytes objects between
python2 and python3.
Fixes #2929
Closes #3080
Daniel Gustafsson (3 Oct 2018)
- memory: ensure to check allocation results
The result of a memory allocation should always be checked, as we may
run under memory pressure where even a small allocation can fail. This
adds checking and error handling to a few cases where the allocation
wasn't checked for success. In the ftp case, the freeing of the path
variable is moved ahead of the allocation since there is little point
in keeping it around across the strdup, and the separation makes for
more readable code. In nwlib, the lock is aslo freed in the error path.
Also bumps the copyright years on affected files.
Closes #3084
Reviewed-by: Jay Satiro <raysatiro@yahoo.com>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- comment: Fix multiple typos in function parameters
Ensure that the parameters in the comment match the actual names in the
prototype.
Closes #3079
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- CURLOPT_SSLVERSION.3: fix typos and consistent spelling
Use TLS vX.Y throughout the document, instead of TLS X.Y, as that was
already done in all but a few cases. Also fix a few typos.
Closes #3076
Reviewed-by: Marcel Raad <Marcel.Raad@teamviewer.com>
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- SECURITY-PROCESS: make links into hyperlinks
Use proper Markdown hyperlink format for the Bountygraph links in order
for the generated website page to be more user friendly. Also link to
the sponsors to give them a little extra credit.
Closes #3082
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Jay Satiro (3 Oct 2018)
- CURLOPT_HEADER.3: fix typo
- nss: fix nssckbi module loading on Windows
- Use .DLL extension instead of .so to load modules on Windows.
Bug: https://curl.haxx.se/mail/lib-2018-09/0077.html
Reported-by: Maxime Legros
Ref: https://github.com/curl/curl/pull/3016/#issuecomment-423069442
Closes https://github.com/curl/curl/pull/3086
- data-binary.d: clarify default content-type is x-www-form-urlencoded
- Advise user that --data-binary sends a default content type of
x-www-form-urlencoded, and to have the data treated as arbitrary
binary data by the server set the content-type header to octet-stream.
Ref: https://github.com/curl/curl/pull/2852#issuecomment-426465094
Closes https://github.com/curl/curl/pull/3085
Marcel Raad (2 Oct 2018)
- test1299: use single quotes around asterisk
Ref: https://github.com/curl/curl/issues/1751#issuecomment-321522580
Daniel Stenberg (2 Oct 2018)
- docs/CIPHERS: mention the colon separation for OpenSSL
Bug: #3077
- runtests: ignore disabled even when ranges are given
runtests.pl support running a range of tests, like "44 to 127". Starting
now, the code makes sure that even such given ranges will ignore tests
that are marked as disabled.
Disabled tests can still be run by explictly specifying that test
number.
Closes #3075
- urlapi: starting with a drive letter on win32 is not an abs url
... and libcurl doesn't support any single-letter URL schemes (if there
even exist any) so it should be fairly risk-free.
Reported-by: Marcel Raad
Fixes #3070
Closes #3071
Marcel Raad (2 Oct 2018)
- doh: fix curl_easy_setopt argument type
CURLOPT_POSTFIELDSIZE is long. Fixes a compiler warning on 64-bit
MinGW.
Daniel Stenberg (2 Oct 2018)
- RELEASE-NOTES: synced
Jay Satiro (1 Oct 2018)
- [Ruslan Baratov brought this change]
CMake: Improve config installation
Use 'GNUInstallDirs' standard module to set destinations of installed
files.
Use uppercase "CURL" names instead of lowercase "curl" to match standard
'FindCURL.cmake' CMake module:
* https://cmake.org/cmake/help/latest/module/FindCURL.html
Meaning:
* Install 'CURLConfig.cmake' instead of 'curl-config.cmake'
* User should call 'find_package(CURL)' instead of 'find_package(curl)'
Use 'configure_package_config_file' function to generate
'CURLConfig.cmake' file. This will make 'curl-config.cmake.in' template
file smaller and handle components better. E.g. current configuration
report no error if user specified unknown components (note: new
configuration expects no components, report error if user will try to
specify any).
Closes https://github.com/curl/curl/pull/2849
Daniel Stenberg (1 Oct 2018)
- test1650: make it depend on http/2
Follow-up to 570008c99da0ccbb as it gets link errors.
Reported-by: Michael Kaufmann
Closes #3068
- [Nate Prewitt brought this change]
MANUAL: minor grammar fix
Noticed a typo reading through the docs.
Closes #3069
- doh: only build if h2 enabled
The DoH spec says "HTTP/2 [RFC7540] is the minimum RECOMMENDED version
of HTTP for use with DoH".
Reported-by: Marcel Raad
Closes #3066
- test2100: require http2 to run
Reported-by: Marcel Raad
Fixes #3064
Closes #3065
- multi: fix memory leak in content encoding related error path
... a missing multi_done() call.
Credit to OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10728
Closes #3063
- travis: bump the Secure Transport build to use xcode 10
Due to an issue with travis
(https://github.com/travis-ci/travis-ci/issues/9956) we've been using
Xcode 9.2 for darwinssl builds for a while. Now xcode 10 is offered as
an alternative and as it builds curl+darwinssl fine that seems like a
better choice.
Closes #3062
- [Rich Turner brought this change]
curl: enabled Windows VT Support and UTF-8 output
Enabled Console VT support (if running OS supports VT) in tool_main.c.
Fixes #3008
Closes #3011
- multi: fix location URL memleak in error path
Follow-up to #3044 - fix a leak OSS-Fuzz detected
Closes #3057
Sergei Nikulov (28 Sep 2018)
- cmake: fixed path used in generation of docs/tests during curl build through add_subdicectory(...)
- [Brad King brought this change]
cmake: Backport to work with CMake 3.0 again
Changes in commit 7867aaa9a0 (cmake: link curl to the OpenSSL targets
instead of lib absolute paths, 2018-07-17) and commit f826b4ce98 (cmake:
bumped minimum version to 3.4, 2018-07-19) required CMake 3.4 to fix
issue #2746. This broke support for users on older versions of CMake
even if they just want to build curl and do not care whether transitive
dependencies work.
Backport the logic to work with CMake 3.0 again by implementing the
fix only when the version of CMake is at least 3.4.
Marcel Raad (27 Sep 2018)
- curl_threads: fix classic MinGW compile break
Classic MinGW still has _beginthreadex's return type as unsigned long
instead of uintptr_t [0]. uintptr_t is not even defined because of [1].
[0] https://sourceforge.net/p/mingw/mingw-org-wsl/ci/wsl-5.1-release/tree/mingwrt/include/process.h#l167
[1] https://sourceforge.net/p/mingw/mingw-org-wsl/ci/wsl-5.1-release/tree/mingwrt/include/process.h#l90
Bug: https://github.com/curl/curl/issues/2924#issuecomment-424334807
Closes https://github.com/curl/curl/pull/3051
Daniel Stenberg (26 Sep 2018)
- configure: s/AC_RUN_IFELSE/CURL_RUN_IFELSE
fix a few leftovers
Fixes #3006
Closes #3049
- [Doron Behar brought this change]
example/htmltidy: fix include paths of tidy libraries
Closes #3050
- RELEASE-NOTES: synced
- Curl_http2_done: fix memleak in error path
Free 'header_recvbuf' unconditionally even if 'h2' isn't (yet) set, for
early failures.
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10669
Closes #3046
- http: fix memleak in rewind error path
If the rewind would fail, a strdup() would not get freed.
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10665
Closes #3044
Viktor Szakats (24 Sep 2018)
- test320: fix regression in [ci skip]
The value in question is coming directly from `gnutls-serv`, so it cannot
be modified freely.
Reported-by: Marcel Raad
Ref: https://github.com/curl/curl/commit/6ae6b2a533e8630afbb21f570305bd4ceece6348#commitcomment-30621004
Daniel Stenberg (24 Sep 2018)
- Curl_retry_request: fix memory leak
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10648
Closes #3042
- openssl: load built-in engines too
Regression since 38203f1
Reported-by: Jean Fabrice
Fixes #3023
Closes #3040
- [Christian Heimes brought this change]
OpenSSL: enable TLS 1.3 post-handshake auth
OpenSSL 1.1.1 requires clients to opt-in for post-handshake
authentication.
Fixes: https://github.com/curl/curl/issues/3026
Signed-off-by: Christian Heimes <christian@python.org>
Closes https://github.com/curl/curl/pull/3027
- [Even Rouault brought this change]
Curl_dedotdotify(): always nul terminate returned string.
This fixes potential out-of-buffer access on "file:./" URL
$ valgrind curl "file:./"
==24516== Memcheck, a memory error detector
==24516== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==24516== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==24516== Command: /home/even/install-curl-git/bin/curl file:./
==24516==
==24516== Conditional jump or move depends on uninitialised value(s)
==24516== at 0x4C31F9C: strcmp (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==24516== by 0x4EBB315: seturl (urlapi.c:801)
==24516== by 0x4EBB568: parseurl (urlapi.c:861)
==24516== by 0x4EBC509: curl_url_set (urlapi.c:1199)
==24516== by 0x4E644C6: parseurlandfillconn (url.c:2044)
==24516== by 0x4E67AEF: create_conn (url.c:3613)
==24516== by 0x4E68A4F: Curl_connect (url.c:4119)
==24516== by 0x4E7F0A4: multi_runsingle (multi.c:1440)
==24516== by 0x4E808E5: curl_multi_perform (multi.c:2173)
==24516== by 0x4E7558C: easy_transfer (easy.c:686)
==24516== by 0x4E75801: easy_perform (easy.c:779)
==24516== by 0x4E75868: curl_easy_perform (easy.c:798)
Was originally spotted by
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10637
Credit to OSS-Fuzz
Closes #3039
Viktor Szakats (23 Sep 2018)
- update URLs in tests
- and one in docs/MANUAL as well
Closes https://github.com/curl/curl/pull/3038
- whitespace fixes
- replace tabs with spaces where possible
- remove line ending spaces
- remove double/triple newlines at EOF
- fix a non-UTF-8 character
- cleanup a few indentations/line continuations
in manual examples
Closes https://github.com/curl/curl/pull/3037
Daniel Stenberg (23 Sep 2018)
- http: add missing return code check
Detected by Coverity. CID 1439610.
Follow-up from 46e164069d1a523
Closes #3034
- ftp: don't access pointer before NULL check
Detected by Coverity. CID 1439611.
Follow-up from 46e164069d1a523
- unit1650: fix out of boundary access
Fixes #2987
Closes #3035
Viktor Szakats (23 Sep 2018)
- docs/examples: URL updates
- also update two URLs outside of docs/examples
- fix spelling of filename persistant.c
- fix three long lines that started failing checksrc.pl
Closes https://github.com/curl/curl/pull/3036
- examples/Makefile.m32: sync with core [ci skip]
also:
- fix two warnings in synctime.c (one of them Windows-specific)
- upgrade URLs in synctime.c and remove a broken one
Closes https://github.com/curl/curl/pull/3033
Daniel Stenberg (22 Sep 2018)
- examples/parseurl.c: show off the URL API a bit
Closes #3030
- SECURITY-PROCESS: mention the bountygraph program [ci skip]
Closes #3032
- url: use the URL API internally as well
... to make it a truly unified URL parser.
Closes #3017
Viktor Szakats (22 Sep 2018)
- URL and mailmap updates, remove an obsolete directory [ci skip]
Closes https://github.com/curl/curl/pull/3031
Daniel Stenberg (22 Sep 2018)
- RELEASE-NOTES: synced
- configure: force-use -lpthreads on HPUX
When trying to detect pthreads use on HPUX the checks will succeed
without the correct -l option but then end up failing at run-time.
Reported-by: Eason-Yu on github
Fixes #2697
Closes #3025
- [Erik Minekus brought this change]
Curl_saferealloc: Fixed typo in docblock
Closes #3029
- urlapi: fix support for address scope in IPv6 numerical addresses
Closes #3024
- [Loganaden Velvindron brought this change]
GnutTLS: TLS 1.3 support
Closes #2971
- TODO: c-ares and CURLOPT_OPENSOCKETFUNCTION
Removed DoH.
Closes #2734
Jay Satiro (20 Sep 2018)
- vtls: fix ssl version "or later" behavior change for many backends
- Treat CURL_SSLVERSION_MAX_NONE the same as
CURL_SSLVERSION_MAX_DEFAULT. Prior to this change NONE would mean use
the minimum version also as the maximum.
This is a follow-up to 6015cef which changed the behavior of setting
the SSL version so that the requested version would only be the minimum
and not the maximum. It appears it was (mostly) implemented in OpenSSL
but not other backends. In other words CURL_SSLVERSION_TLSv1_0 used to
mean use just TLS v1.0 and now it means use TLS v1.0 *or later*.
- Fix CURL_SSLVERSION_MAX_DEFAULT for OpenSSL.
Prior to this change CURL_SSLVERSION_MAX_DEFAULT with OpenSSL was
erroneously treated as always TLS 1.3, and would cause an error if
OpenSSL was built without TLS 1.3 support.
Co-authored-by: Daniel Gustafsson
Fixes https://github.com/curl/curl/issues/2969
Closes https://github.com/curl/curl/pull/3012
Daniel Stenberg (20 Sep 2018)
- certs: generate tests certs with sha256 digest algorithm
As OpenSSL 1.1.1 starts to complain and fail on sha1 CAs:
"SSL certificate problem: CA signature digest algorithm too weak"
Closes #3014
- urlapi: document the error codes, remove two unused ones
Assisted-by: Daniel Gustafsson
Closes #3019
- urlapi: add CURLU_GUESS_SCHEME and fix hostname acceptance
In order for this API to fully work for libcurl itself, it now offers a
CURLU_GUESS_SCHEME flag that makes it "guess" scheme based on the host
name prefix just like libcurl always did. If there's no known prefix, it
will guess "http://".
Separately, it relaxes the check of the host name so that IDN host names
can be passed in as well.
Both these changes are necessary for libcurl itself to use this API.
Assisted-by: Daniel Gustafsson
Closes #3018
Kamil Dudka (19 Sep 2018)
- nss: try to connect even if libnssckbi.so fails to load
One can still use CA certificates stored in NSS database.
Reported-by: Maxime Legros
Bug: https://curl.haxx.se/mail/lib-2018-09/0077.html
Closes #3016
Daniel Gustafsson (19 Sep 2018)
- urlapi: don't set value which is never read
In the CURLUPART_URL case, there is no codepath which invokes url
decoding so remove the assignment of the urldecode variable. This
fixes the deadstore bug-report from clang static analysis.
Closes #3015
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- todo: Update reference to already done item
TODO item 1.1 was implemented in commit 946ce5b61f, update reference
to it with instead referencing the implemented option.
Closes #3013
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (18 Sep 2018)
- RELEASE-NOTES: synced
- [slodki brought this change]
cmake: don't require OpenSSL if USE_OPENSSL=OFF
User must have OpenSSL installed even if not used by libcurl at all
since 7.61.1 release. Broken at
7867aaa9a01decf93711428462335be8cef70212
Reviewed-by: Sergei Nikulov
Closes #3001
- curl_multi_wait: call getsock before figuring out timeout
.... since getsock may update the expiry timer.
Fixes #2996
Closes #3000
- examples/http2-pushinmemory: receive HTTP/2 pushed files in memory
Closes #3004
Daniel Gustafsson (18 Sep 2018)
- darwinssl: Fix realloc memleak
The reallocation was using the input pointer for the return value, which
leads to a memory leak on reallication failure. Fix by instead use the
safe internal API call Curl_saferealloc().
Closes #3005
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Reviewed-by: Nick Zitzmann <nickzman@gmail.com>
- [Kruzya brought this change]
examples: Fix memory leaks from realloc errors
Make sure to not overwrite the reallocated pointer in realloc() calls
to avoid a memleak on memory errors.
- memory: add missing curl_printf header
ftp_send_command() was using vsnprintf() without including the libcurl
*rintf() replacement header. Fix by including curl_printf.h and also
add curl_memory.h while at it since memdebug.h depends on it.
Closes #2999
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (16 Sep 2018)
- [Si brought this change]
curl: update --tlsv* descriptions in --help output
Closes #2994
- http: made Curl_add_buffer functions take a pointer-pointer
... so that they can clear the original pointer on failure, which makes
the error-paths and their cleanups easier.
Closes #2992
- http2: fix memory leaks on error-path
- [Rikard Falkeborn brought this change]
libtest: Add chkdecimalpoint to .gitignore
Closes #2998
Viktor Szakats (14 Sep 2018)
- secure Openwall URLs
Daniel Stenberg (14 Sep 2018)
- openssl: show "proper" version number for libressl builds
Closes #2989
- [Rainer Jung brought this change]
openssl: assume engine support in 0.9.8 or later
Fixes #2983
Closes #2988
Daniel Gustafsson (13 Sep 2018)
- sendf: use failf() rather than Curl_failf()
The failf() macro is the name used for invoking Curl_failf(). While
there isn't a way to turn off failf like there is for infof, but it's
still a good idea to use the macro.
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- sendf: Fix whitespace in infof/failf concatenation
Strings broken on multiple rows in the .c file need to have appropriate
whitespace padding on either side of the concatenation point to render
a correct amalgamated string. Fix by adding a space at the occurrences
found.
Closes #2986
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- krb5: fix memory leak in krb_auth
The FTP command allocated by aprintf() must be freed after usage.
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- ftp: include command in Curl_ftpsend sendbuffer
Commit 8238ba9c5f10414a88f502bf3f5d5a42d632984c inadvertently removed
the actual command to be sent from the send buffer in a refactoring.
Add back copying the command into the buffer. Also add more guards
against malformed input while at it.
Closes #2985
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
- ntlm_wb: Fix memory leaks in ntlm_wb_response
When erroring out on a request being too large, the existing buffer was
leaked. Fix by explicitly freeing on the way out.
Closes #2966
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Daniel Stenberg (13 Sep 2018)
- [Yiming Jing brought this change]
travis: build the MesaLink vtls backend with MesaLink 0.7.1
- [Yiming Jing brought this change]
runtests.pl: run tests against the MesaLink vtls backend
- [Yiming Jing brought this change]
vtls: add a MesaLink vtls backend
Closes #2984
- [Yiming Jing brought this change]
configure.ac: add a MesaLink vtls backend
- [Dave Reisner brought this change]
curl_url_set.3: properly escape \n in example code
This yields
"the scheme is %s\n"
instead of
"the scheme is %s0
Closes #2970
- [Dave Reisner brought this change]
curl_url_set.3: fix typo in reference to CURLU_APPENDQUERY
- urlglob: improve error message
to help user understand what the problem is
Reported-by: Daniel Shahaf
Fixes #2763
Closes #2977
- [Yiming Jing brought this change]
tests/certs: rebuild certs with 2048-bit RSA keys
The previous test certificates contained RSA keys of only 1024 bits.
However, RSA claims that 1024-bit RSA keys are likely to become
crackable some time before 2010. The NIST recommends at least 2048-bit
keys for RSA for now.
Better use full 2048 also for testing.
Closes #2973
Daniel Gustafsson (12 Sep 2018)
- TODO: fix typo in item
Closes #2968
Reviewed-by: Daniel Stenberg <daniel@haxx.se>
Marcel Raad (12 Sep 2018)
- anyauthput: fix compiler warning on 64-bit Windows
On Windows, the read function from <io.h> is used, which has its byte
count parameter as unsigned int instead of size_t.
Closes https://github.com/curl/curl/pull/2972
Viktor Szakats (12 Sep 2018)
- lib: fix gcc8 warning on Windows
Closes https://github.com/curl/curl/pull/2979
Jay Satiro (12 Sep 2018)
- openssl: fix gcc8 warning
- Use memcpy instead of strncpy to copy a string without termination,
since gcc8 warns about using strncpy to copy as many bytes from a
string as its length.
Suggested-by: Viktor Szakats
Closes https://github.com/curl/curl/issues/2980
Daniel Stenberg (10 Sep 2018)
- libcurl-url.3: overview man page for the URL API
Closes #2967
- example/asiohiper: insert warning comment about its status
This example is simply not working correctly but there's nobody around
with the skills and energy to fix it.
Closes #2407
Kamil Dudka (10 Sep 2018)
- docs/cmdline-opts: update the documentation of --tlsv1.0
... to reflect the changes in 6015cefb1b2cfde4b4850121c42405275e5e77d9
Closes #2955
- docs/examples: do not wait when no transfers are running
Closes #2948
Daniel Stenberg (10 Sep 2018)
- [Daniel Gustafsson brought this change]
cookies: Move failure case label to end of function
Rather than jumping backwards to where failure cleanup happens
to be performed, move the failure case to end of the function
where it is expected per existing coding convention.
Closes #2965
- [Daniel Gustafsson brought this change]
misc: fix typos in comments
Closes #2963
- [Daniel Gustafsson brought this change]
cookies: fix leak when writing cookies to file
If the formatting fails, we error out on a fatal error and
clean up on the way out. The array was however freed within
the wrong scope and was thus never freed in case the cookies
were written to a file instead of STDOUT.
Closes #2957
- [Daniel Gustafsson brought this change]
cookies: Remove redundant expired check
Expired cookies have already been purged at a later expiration time
before this check, so remove the redundant check.
closes #2962
- ntlm_wb: bail out if the response gets overly large
Exit the realloc() loop if the response turns out ridiculously large to
avoid worse problems.
Reported-by: Harry Sintonen
Closes #2959
- [Daniel Gustafsson brought this change]
url.c: fix comment typo and indentation
Closes #2960
- urlapi: avoid derefencing a possible NULL pointer
Coverity CID 1439134
- RELEASE-NOTES: synced
Marcel Raad (8 Sep 2018)
- test324: fix after 3f3b26d6feb0667714902e836af608094235fca2
The expected error code is now 60. 51 is dead.
Daniel Stenberg (8 Sep 2018)
- curl_url_set.3: correct description
- curl_url-docs: fix AVAILABILITY as Added in curl 7.62.0
- URL-API
See header file and man pages for API. All documented API details work
and are tested in the 1560 test case.
Closes #2842
- curl_easy_upkeep: removed 'conn' from the name
... including the associated option.
Fixes #2951
Closes #2952
- [Max Dymond brought this change]
upkeep: add a connection upkeep API: curl_easy_conn_upkeep()
Add functionality so that protocols can do custom keepalive on their
connections, when an external API function is called.
Add docs for the new options in 7.62.0
Closes #1641
- [Philipp Waehnert brought this change]
configure: add option to disable automatic OpenSSL config loading
Sometimes it may be considered a security risk to load an external
OpenSSL configuration automatically inside curl_global_init(). The
configuration option --disable-ssl-auto-load-config disables this
automatism. The Windows build scripts winbuild/Makefile.vs provide a
corresponding option ENABLE_SSL_AUTO_LOAD_CONFIG accepting a boolean
value.
Setting neither of these options corresponds to the previous behavior
loading the external OpenSSL configuration automatically.
Fixes #2724
Closes #2791
- doh: minor edits to please Coverity
The gcc typecheck macros and coverity combined made it warn on the 2nd
argument for ERROR_CHECK_SETOPT(). Here's minor rearrange to please it.
Coverity CID 1439115 and CID 1439114.
- schannel: avoid switch-cases that go to default anyway
SEC_E_APPLICATION_PROTOCOL_MISMATCH isn't defined in some versions of
mingw and would require an ifdef otherwise.
Reported-by: Thomas Glanzmann
Approved-by: Marc Hörsken
Bug: https://curl.haxx.se/mail/lib-2018-09/0020.html
Closes #2950
- [Nicklas Avén brought this change]
imap: change from "FETCH" to "UID FETCH"
... and add "MAILINDEX".
As described in #2789, this is a suggested solution. Changing UID=xx to
actually get mail with UID xx and add "MAILINDEX" to get a mail with a
special index in the mail box (old behavior). So MAILINDEX=1 gives the
first non deleted mail in the mail box.
Fixes #2789
Closes #2815
- CURLOPT_UPLOAD_BUFFERSIZE: set upload buffer size
This is step 3 of #2888.
Fixes #2888
Closes #2896
- travis: add the DOH tests to the torture testing
- DOH: add test case 1650 and 2100
- curl: --doh-url added
- setopt: add CURLOPT_DOH_URL
Closes #2668
- [Han Han brought this change]
ssl: deprecate CURLE_SSL_CACERT in favour of a unified error code
Long live CURLE_PEER_FAILED_VERIFICATION
- [Han Han brought this change]
x509asn1: return CURLE_PEER_FAILED_VERIFICATION on failure to parse cert
CURLE_PEER_FAILED_VERIFICATION makes more sense because Curl_parseX509
does not allocate memory internally as its first argument is a pointer
to the certificate structure. The same error code is also returned by
Curl_verifyhost when its call to Curl_parseX509 fails so the change
makes error handling more consistent.
- [Han Han brought this change]
openssl: return CURLE_PEER_FAILED_VERIFICATION on failure to parse issuer
Failure to extract the issuer name from the server certificate should
return a more specific error code like on other TLS backends.
- [Han Han brought this change]
schannel: unified error code handling
Closes #2901
- [Han Han brought this change]
darwinssl: more specific and unified error codes
Closes #2901
- CURLOPT_DNS_USE_GLOBAL_CACHE: deprecated
Disable the CURLOPT_DNS_USE_GLOBAL_CACHE option and mark it for
deprecation and complete removal in six months.
Bug: https://curl.haxx.se/mail/lib-2018-09/0010.html
Closes #2942
- url: default to CURL_HTTP_VERSION_2TLS if built h2-enabled
Closes #2709
- multiplex: enable by default
Starting 7.62.0, multiplexing is enabled by default in multi handles.
- [Jim Fuller brought this change]
tests: add unit tests for url.c
Approved-by: Daniel Gustafsson
Closes #2937
- test1452: mark as flaky
makes it not run in the CI builds
Closes #2941
- pipelining: deprecated
Transparently. The related curl_multi_setopt() options all still returns
OK when pipelining is selected.
To re-enable the support, the single line change in lib/multi.c needs to
be reverted.
See docs/DEPRECATE.md
Closes #2705
- RELEASE-NOTES: start working on 7.62.0
Version 7.61.1 (4 Sep 2018)
Daniel Stenberg (4 Sep 2018)
- THANKS: 7.61.1 status
- RELEASE-NOTES: 7.61.1
- Curl_getoff_all_pipelines: ignore unused return values
Since scan-build would warn on the dead "Dead store/Dead increment"
Viktor Szakats (4 Sep 2018)
- sftp: fix indentation
Daniel Stenberg (4 Sep 2018)
- [Przemysław Tomaszewski brought this change]
sftp: don't send post-qoute sequence when retrying a connection
Fixes #2939
Closes #2940
Kamil Dudka (3 Sep 2018)
- url, vtls: make CURLOPT{,_PROXY}_TLS13_CIPHERS work
This is a follow-up to PR #2607 and PR #2926.
Closes #2936
Daniel Stenberg (3 Sep 2018)
- [Jay Satiro brought this change]
tool_operate: Add http code 408 to transient list for --retry
- Treat 408 request timeout as transient so that curl will retry the
request if --retry was used.
Closes #2925
- [Jay Satiro brought this change]
openssl: Fix setting TLS 1.3 cipher suites
The flag indicating TLS 1.3 cipher support in the OpenSSL backend was
missing.
Bug: https://github.com/curl/curl/pull/2607#issuecomment-417283187
Reported-by: Kamil Dudka
Closes #2926
- Curl_ntlm_core_mk_nt_hash: return error on too long password
... since it would cause an integer overflow if longer than (max size_t
/ 2).
This is CVE-2018-14618
Bug: https://curl.haxx.se/docs/CVE-2018-14618.html
Closes #2756
Reported-by: Zhaoyang Wu
- [Rikard Falkeborn brought this change]
http2: Use correct format identifier for stream_id
Closes #2928
Marcel Raad (2 Sep 2018)
- test1148: fix precheck output
"precheck command error" is not very helpful.
Daniel Stenberg (1 Sep 2018)
- all: s/int/size_t cleanup
Assisted-by: Rikard Falkeborn
Closes #2922
- ssh-libssh: use FALLTHROUGH to silence gcc8
Jay Satiro (31 Aug 2018)
- tool_operate: Fix setting proxy TLS 1.3 ciphers
Daniel Stenberg (31 Aug 2018)
- [Daniel Gustafsson brought this change]
cookies: support creation-time attribute for cookies
According to RFC6265 section 5.4, cookies with equal path lengths
SHOULD be sorted by creation-time (earlier first). This adds a
creation-time record to the cookie struct in order to make cookie
sorting more deterministic. The creation-time is defined as the
order of the cookies in the jar, the first cookie read fro the
jar being the oldest. The creation-time is thus not serialized
into the jar. Also remove the strcmp() matching in the sorting as
there is no lexicographic ordering in RFC6265. Existing tests are
updated to match.
Closes #2524
Marcel Raad (31 Aug 2018)
- Don't use Windows path %PWD for SSH tests
All these tests failed on Windows because something like
sftp://%HOSTIP:%SSHPORT%PWD/
expanded to
sftp://127.0.0.1:1234c:/msys64/home/bla/curl
and then curl complained about the port number ending with a letter.
Use the original POSIX path instead of the Windows path created in
checksystem to fix this.
Closes https://github.com/curl/curl/pull/2920
Jay Satiro (29 Aug 2018)
- CURLOPT_SSL_CTX_FUNCTION.3: clarify connection reuse warning
Reported-by: Daniel Stenberg
Closes https://github.com/curl/curl/issues/2916
Daniel Stenberg (28 Aug 2018)
- THANKS-filter: dedup Daniel Jeliński
- RELEASE-NOTES: synced
- CURLOPT_ACCEPT_ENCODING.3: list them comma-separated [ci skip]
- CURLOPT_SSL_CTX_FUNCTION.3: might cause unintended connection reuse [ci skip]
Added a warning!
Closes #2915
- curl: fix time-of-check, time-of-use race in dir creation
Patch-by: Jay Satiro
Detected by Coverity
Fixes #2739
Closes #2912
- cmdline-opts/page-footer: fix edit mistake
There was a missing newline.
follow-up to a7ba60bb7250
- docs: clarify NO_PROXY env variable functionality
Reported-by: Kirill Marchuk
Fixes #2773
Closes #2911
Marcel Raad (24 Aug 2018)
- lib1522: fix curl_easy_setopt argument type
CURLOPT_POSTFIELDSIZE is a long option.
- curl_threads: silence bad-function-cast warning
As uintptr_t and HANDLE are always the same size, this warning is
harmless. Just silence it using an intermediate uintptr_t variable.
Closes https://github.com/curl/curl/pull/2908
Daniel Stenberg (24 Aug 2018)
- README: add appveyor build badge [ci skip]
Closes #2913
- [Ihor Karpenko brought this change]
schannel: client certificate store opening fix
1) Using CERT_STORE_OPEN_EXISTING_FLAG ( or CERT_STORE_READONLY_FLAG )
while opening certificate store would be sufficient in this scenario and
less-demanding in sense of required user credentials ( for example,
IIS_IUSRS will get "Access Denied" 0x05 error for existing CertOpenStore
call without any of flags mentioned above ),
2) as 'cert_store_name' is a DWORD, attempt to format its value like a
string ( in "Failed to open cert store" error message ) will throw null
pointer exception
3) adding GetLastError(), in my opinion, will make error message more
useful.
Bug: https://curl.haxx.se/mail/lib-2018-08/0198.html
Closes #2909
- [Leonardo Taccari brought this change]
gopher: Do not translate `?' to `%09'
Since GOPHER support was added in curl `?' character was automatically
translated to `%09' (`\t').
However, this behaviour does not seems documented in RFC 4266 and for
search selectors it is documented to directly use `%09' in the URL.
Apart that several gopher servers in the current gopherspace have CGI
support where `?' is used as part of the selector and translating it to
`%09' often leads to surprising results.
Closes #2910
Marcel Raad (23 Aug 2018)
- cookie tests: treat files as text
Fixes test failures because of wrong line endings on Windows.
Daniel Stenberg (23 Aug 2018)
- libcurl-thread.3: expand somewhat on the NO_SIGNAL motivation
Multi-threaded applictions basically MUST set CURLOPT_NO_SIGNAL to 1L to
avoid the risk of getting a SIGPIPE.
Either way, a multi-threaded application that uses libcurl/openssl needs
to have a signhandler for or ignore SIGPIPE on its own.
Based on discussions in #2800
Closes #2904
- RELEASE-NOTES: synced
Marcel Raad (22 Aug 2018)
- Tests: fixes for Windows
- test 1268 requires unix sockets
- test 2072 must be disabled also for MSYS/MinGW
Daniel Stenberg (22 Aug 2018)
- http2: abort the send_callback if not setup yet
When Curl_http2_done() gets called before the http2 data is setup all
the way, we cannot send anything and this should just return an error.
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=10012
- http2: remove four unused nghttp2 callbacks
Closes #2903
- x509asn1: use FALLTHROUGH
... as no other comments are accepted since 014ed7c22f51463
Marcel Raad (21 Aug 2018)
- test1148: disable if decimal separator is not point
Modifying the locale with environment variables doesn't work for native
Windows applications. Just disable the test in this case if the decimal
separator is something different than a point. Use a precheck with a
small C program to achieve that.
Closes https://github.com/curl/curl/pull/2786
- Enable more GCC warnings
This enables the following additional warnings:
-Wold-style-definition
-Warray-bounds=2 instead of the default 1
-Wformat=2, but only for GCC 4.8+ as Wno-format-nonliteral is not
respected for older versions
-Wunused-const-variable, which enables level 2 instead of the default 1
-Warray-bounds also in debug mode through -ftree-vrp
-Wnull-dereference also in debug mode through
-fdelete-null-pointer-checks
Closes https://github.com/curl/curl/pull/2747
- curl-compilers: enable -Wimplicit-fallthrough=4 for GCC
This enables level 4 instead of the default level 3, which of the
currently used comments only allows /* FALLTHROUGH */ to silence the
warning.
Closes https://github.com/curl/curl/pull/2747
- curl-compilers: enable -Wbad-function-cast on GCC
This warning used to be enabled only for clang as it's a bit stricter
on GCC. Silence the remaining occurrences and enable it on GCC too.
Closes https://github.com/curl/curl/pull/2747
- configure: conditionally enable pedantic-errors
Enable pedantic-errors for GCC >= 5 with --enable-werror. Before GCC 5,
pedantic-errors was synonymous to -Werror=pedantic [0], which is still
the case for clang [1]. With GCC 5, it became complementary [2].
Also fix a resulting error in acinclude.m4 as main's return type was
missing, which is illegal in C99.
[0] https://gcc.gnu.org/onlinedocs/gcc-4.9.0/gcc/Warning-Options.html
[1] https://clang.llvm.org/docs/UsersManual.html#options-to-control-error-and-warning-messages
[2] https://gcc.gnu.org/onlinedocs/gcc-5.1.0/gcc/Warning-Options.html
Closes https://github.com/curl/curl/pull/2747
- Remove unused definitions
Closes https://github.com/curl/curl/pull/2747
Daniel Stenberg (21 Aug 2018)
- x509asn1: make several functions static
and remove the private SIZE_T_MAX define and use the generic one.
Closes #2902
- INTERNALS: require GnuTLS >= 2.11.3
Since the public pinning support was brought in e644866caf4. GnuTLS
2.11.3 was released in October 2010.
Figured out in #2890
- http2: avoid set_stream_user_data() before stream is assigned
... before the stream is started, we have it set to -1.
Fixes #2894
Closes #2898
- SSLCERTS: improve the openssl command line
... for extracting certs from a live HTTPS server to make a cacerts.pem
from them.
- docs/SECURITY-PROCESS: now we name the files after the CVE id
- RELEASE-NOTES: synced
- upload: change default UPLOAD_BUFSIZE to 64KB
To make uploads significantly faster in some circumstances.
Part 2 of #2888
Closes #2892
- upload: allocate upload buffer on-demand
Saves 16KB on the easy handle for operations that don't need that
buffer.
Part 1 of #2888
- [Laurent Bonnans brought this change]
vtls: reinstantiate engine on duplicated handles
Handles created with curl_easy_duphandle do not use the SSL engine set
up in the original handle. This fixes the issue by storing the engine
name in the internal url state and setting the engine from its name
inside curl_easy_duphandle.
Reported-by: Anton Gerasimov
Signed-of-by: Laurent Bonnans
Fixes #2829
Closes #2833
- http2: make sure to send after RST_STREAM
If this is the last stream on this connection, the RST_STREAM might not
get pushed to the wire otherwise.
Fixes #2882
Closes #2887
Researched-by: Michael Kaufmann
- test1268: check the stderr output as "text"
Follow-up to 099f37e9c57
Pointed-out-by: Marcel Raad
- urldata: remove unused pipe_broke struct field
This struct field is never set TRUE in any existing code path. This
change removes the field completely.
Closes #2871
- curl: warn the user if a given file name looks like an option
... simply because this is usually a sign of the user having omitted the
file name and the next option is instead "eaten" by the parser as a file
name.
Add test1268 to verify
Closes #2885
- http2: check nghttp2_session_set_stream_user_data return code
Might help bug #2688 debugging
Closes #2880
- travis: revert back to gcc-7 for coverage builds
... since the gcc-8 ones seem to fail frequently.
Follow-up from b85207199544ca
Closes #2886
- RELEASE-NOTES: synced
... and now listed in alphabetical order!
- [Adrien brought this change]
CMake: CMake config files are defining CURL_STATICLIB for static builds
This change allows to use the CMake config files generated by Curl's
CMake scripts for static builds of the library.
The symbol CURL_STATIC lib must be defined to compile downstream,
thus the config package is the perfect place to do so.
Fixes #2817
Closes #2823
Reported-by: adnn on github
Reviewed-by: Sergei Nikulov
- TODO: host name sections in config files
Kamil Dudka (14 Aug 2018)
- ssh-libssh: fix infinite connect loop on invalid private key
Added test 656 (based on test 604) to verify the fix.
Bug: https://bugzilla.redhat.com/1595135
Closes #2879
- ssh-libssh: reduce excessive verbose output about pubkey auth
The verbose message "Authentication using SSH public key file" was
printed each time the ssh_userauth_publickey_auto() was called, which
meant each time a packet was transferred over network because the API
operates in non-blocking mode.
This patch makes sure that the verbose message is printed just once
(when the authentication state is entered by the SSH state machine).
Daniel Stenberg (14 Aug 2018)
- travis: disable h2 torture tests for "coverage"
Since they started to fail almost 100% since a few days.
Closes #2876
Marcel Raad (14 Aug 2018)
- travis: update to GCC 8
Closes https://github.com/curl/curl/pull/2869
Daniel Stenberg (13 Aug 2018)
- http: fix for tiny "HTTP/0.9" response
Deal with tiny "HTTP/0.9" (header-less) responses by checking the
status-line early, even before a full "HTTP/" is received to allow
detecting 0.9 properly.
Test 1266 and 1267 added to verify.
Fixes #2420
Closes #2872
Kamil Dudka (13 Aug 2018)
- docs: add disallow-username-in-url.d and haproxy-protocol.d on the list
... to make make the files appear in distribution tarballs
Closes #2856
- .travis.yml: verify that man pages can be regenerated
... when curl is built from distribution tarball
Closes #2856
Marcel Raad (11 Aug 2018)
- Split non-portable part off test 1133
Split off testing file names with double quotes into new test 1158.
Disable it for MSYS using a precheck as it doesn't support file names
with double quotes (but Cygwin does, for example).
Fixes https://github.com/curl/curl/issues/2796
Closes https://github.com/curl/curl/pull/2854
Jay Satiro (11 Aug 2018)
- projects: Improve Windows perl detection in batch scripts
- Determine if perl is in the user's PATH by running perl.exe.
Prior to this change detection was done by checking the PATH for perl/
but that did not work in all cases (eg git install includes perl but
not in perl/ path).
Bug: https://github.com/curl/curl/pull/2865
Reported-by: Daniel Jeliński
- [Michael Kaufmann brought this change]
docs: Improve the manual pages of some callbacks
- CURLOPT_HEADERFUNCTION: add newlines
- CURLOPT_INTERLEAVEFUNCTION: fix the description of 'userdata'
- CURLOPT_READDATA: mention crashes, same as in CURLOPT_WRITEDATA
- CURLOPT_READFUNCTION: rename 'instream' to 'userdata' and explain
how to set it
Closes https://github.com/curl/curl/pull/2868
Marcel Raad (11 Aug 2018)
- GCC: silence -Wcast-function-type uniformly
Pointed-out-by: Rikard Falkeborn
Closes https://github.com/curl/curl/pull/2860
- Silence GCC 8 cast-function-type warnings
On Windows, casting between unrelated function types is fine and
sometimes even necessary, so just use an intermediate cast to
(void (*) (void)) to silence the warning as described in [0].
[0] https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/Warning-Options.html
Closes https://github.com/curl/curl/pull/2860
Daniel Stenberg (11 Aug 2018)
- CURLINFO_SIZE_UPLOAD: fix missing counter update
Adds test 1522 for verification.
Reported-by: cjmsoregan
Fixes #2847
Closes #2864
- [Daniel Jelinski brought this change]
Documentation: fix CURLOPT_SSH_COMPRESSION copy/paste bug
Closes #2867
- RELEASE-NOTES: synced
- openssl: fix potential NULL pointer deref in is_pkcs11_uri
Follow-up to 298d2565e
Coverity CID 1438387
Marcel Raad (10 Aug 2018)
- travis: execute "set -eo pipefail" for coverage build
Follow-up to 2de63ab179eb78630ee039ad94fb2a5423df522d and
0b87c963252d3504552ee0c8cf4402bd65a80af5.
Closes https://github.com/curl/curl/pull/2862
Daniel Stenberg (10 Aug 2018)
- lib1502: fix memory leak in torture test
Reported-by: Marcel Raad
Fixes #2861
Closes #2863
- docs: mention NULL is fine input to several functions
Fixes #2837
Closes #2858
Reported-by: Markus Elfring
- [Bas van Schaik brought this change]
README.md: add LGTM.com code quality grade for C/C++
Closes #2857
- [Rikard Falkeborn brought this change]
test1531: Add timeout
Previously, the macro TEST_HANG_TIMEOUT was unused, but since there is
looping going on, we might as well add timing instead of removing it.
Closes #2853
- [Rikard Falkeborn brought this change]
test1540: Remove unused macro TEST_HANG_TIMEOUT
The macro has never been used, and it there is not really any place
where it would make sense to add timing checks.
Closes #2852
- [Rikard Falkeborn brought this change]
asyn-thread: Remove unused macro
The macro seems to never have been used.
Closes #2852
- [Rikard Falkeborn brought this change]
http_proxy: Remove unused macro SELECT_TIMEOUT
Usage was removed in 5113ad0424044458ac497fa1458ebe0101356b22.
Closes #2852
- [Rikard Falkeborn brought this change]
formdata: Remove unused macro HTTPPOST_CONTENTTYPE_DEFAULT
Its usage was removed in
84ad1fd3047815f9c6e78728bb351b828eac10b1.
Closes #2852
- [Rikard Falkeborn brought this change]
telnet: Remove unused macros TELOPTS and TELCMDS
Their usage was removed in 3a145180cc754a5959ca971ef3cd243c5c83fc51.
Closes #2852
- [Daniel Jelinski brought this change]
openssl: fix debug messages
Fixes #2806
Closes #2843
- configure: fix for -lpthread detection with OpenSSL and pkg-config
... by making sure it uses the -I provided by pkg-config!
Reported-by: pszemus on github
Fixes #2848
Closes #2850
- RELEASE-NOTES: synced
- windows: follow up to the buffer-tuning 1ba1dba7
Somehow I didn't include the amended version of the previous fix. This
is the missing piece.
Pointed-out-by: Viktor Szakats
- [Daniel Jelinski brought this change]
windows: implement send buffer tuning
Significantly enhances upload performance on modern Windows versions.
Bug: https://curl.haxx.se/mail/lib-2018-07/0080.html
Closes #2762
Fixes #2224
- [Anderson Toshiyuki Sasaki brought this change]
ssl: set engine implicitly when a PKCS#11 URI is provided
This allows the use of PKCS#11 URI for certificates and keys without
setting the corresponding type as "ENG" and the engine as "pkcs11"
explicitly. If a PKCS#11 URI is provided for certificate, key,
proxy_certificate or proxy_key, the corresponding type is set as "ENG"
if not provided and the engine is set to "pkcs11" if not provided.
Acked-by: Nikos Mavrogiannopoulos
Closes #2333
- [Ruslan Baratov brought this change]
CMake: Respect BUILD_SHARED_LIBS
Use standard CMake variable BUILD_SHARED_LIBS instead of introducing
custom option CURL_STATICLIB.
Use '-DBUILD_SHARED_LIBS=%SHARED%' in appveyor.yml.
Reviewed-by: Sergei Nikulov
Closes #2755
- [John Butterfield brought this change]
cmake: bumped minimum version to 3.4
Closes #2753
- [John Butterfield brought this change]
cmake: link curl to the OpenSSL targets instead of lib absolute paths
Reviewed-by: Jakub Zakrzewski
Reviewed-by: Sergei Nikulov
Closes #2753
- travis: build darwinssl on macos 10.12
... as building on 10.13.x before 10.13.4 leads to link errors.
Assisted-by: Nick Zitzmann
Fixes #2835
Closes #2845
- DEPRECATE: remove release date from 7.62.0
Since it will slip and the version is the important part there, not the
date.
- lib/Makefile: only do symbol hiding if told to
This restores the ability to build a static lib with
--disable-symbol-hiding to keep non-curl_ symbols.
Researched-by: Dan Fandrich
Reported-by: Ran Mozes
Fixes #2830
Closes #2831
Marcel Raad (2 Aug 2018)
- hostip: fix unused variable warning
addresses is only used in an infof call, which is a macro expanding to
nothing if CURL_DISABLE_VERBOSE_STRINGS is set.
Daniel Stenberg (2 Aug 2018)
- test1307: disabled
Turns out that since we're using the native fnmatch function now when
available, and they simply disagree on a huge number of test patterns
that make it hard to test this function like this...
Fixes #2825
- smb: don't mark it done in smb_do
Follow-up to 09e401e01bf9. The SMB protocol handler needs to use its
doing function too, which requires smb_do() to not mark itself as
done...
Closes #2822
- [Rikard Falkeborn brought this change]
general: fix printf specifiers
Closes #2818
- RELEASE-NOTES: synced
- mailmap: Daniel Jelinski
- [Harry Sintonen brought this change]
HTTP: Don't attempt to needlessly decompress redirect body
This change fixes a regression where redirect body would needlessly be
decompressed even though it was to be ignored anyway. As it happens this
causes secondary issues since there appears to be a bug in apache2 that
it in certain conditions generates a corrupt zlib response. The
regression was created by commit:
dbcced8e32b50c068ac297106f0502ee200a1ebd
Discovered-by: Harry Sintonen
Closes #2798
- curl: use Content-Disposition before the "URL end" for -OJ
Regression introduced in 7.61.0
Reported-by: Thomas Klausner
Fixes #2783
Closes #2813
- [Daniel Jelinski brought this change]
retry: return error if rewind was necessary but didn't happen
Fixes #2801
Closes #2812
- http2: clear the drain counter in Curl_http2_done
Reported-by: Andrei Virtosu
Fixes #2800
Closes #2809
- smb: fix memory leak on early failure
... by making sure connection related data (->share) is stored in the
connection and not in the easy handle.
Detected by OSS-fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9369
Fixes #2769
Closes #2810
- travis: run a 'make checksrc' too
... to make sure the examples are all checked.
Closes #2811
Jay Satiro (29 Jul 2018)
- examples/ephiperfifo: checksrc compliance
- [Michael Kaufmann brought this change]
sws: handle EINTR when calling select()
Closes https://github.com/curl/curl/pull/2808
Daniel Stenberg (29 Jul 2018)
- test1157: follow-up to 35ecffb9
Ignore the user-agent line.
Pointed-out-by: Marcel Raad
Michael Kaufmann (29 Jul 2018)
- tests/http_pipe.py: Use /usr/bin/env to find python
Daniel Stenberg (28 Jul 2018)
- TODO: Support Authority Information Access certificate extension (AIA)
Closes #2793
- conn_free: updated comment to clarify
Let's call it disassociate instead of disconnect since the latter term
is used so much for (TCP) connections already.
- test1157: test -H from empty file
Verifies bugfix #2797
- [Tobias Blomberg brought this change]
curl: Fix segfault when -H @headerfile is empty
The curl binary would crash if the -H command line option was given a
filename to read using the @filename syntax but that file was empty.
Closes #2797
- mime: check Curl_rand_hex's return code
Bug: https://curl.haxx.se/mail/archive-2018-07/0015.html
Reported-by: Jeffrey Walton
Closes #2795
- [Josh Bialkowski brought this change]
docs/examples: add hiperfifo example using linux epoll/timerfd
Closes #2804
- [Darío Hereñú brought this change]
docs/INSTALL.md: minor formatting fixes
Closes #2794
- [Christopher Head brought this change]
docs/CURLOPT_URL: fix indentation
The statement, “The application does not have to keep the string around
after setting this option,” appears to be indented under the RTMP
paragraph. It actually applies to all protocols, not just RTMP.
Eliminate the extra indentation.
Closes #2788
- [Christopher Head brought this change]
docs/CURLOPT_WRITEFUNCTION: size is always 1
For compatibility with `fwrite`, the `CURLOPT_WRITEFUNCTION` callback is
passed two `size_t` parameters which, when multiplied, designate the
number of bytes of data passed in. In practice, CURL always sets the
first parameter (`size`) to 1.
This practice is also enshrined in documentation and cannot be changed
in future. The documentation states that the default callback is
`fwrite`, which means `fwrite` must be a suitable function for this
purpose. However, the documentation also states that the callback must
return the number of *bytes* it successfully handled, whereas ISO C
`fwrite` returns the number of items (each of size `size`) which it
wrote. The only way these numbers can be equal is if `size` is 1.
Since `size` is 1 and can never be changed in future anyway, document
that fact explicitly and let users rely on it.
Closes #2787
- [Carie Pointer brought this change]
wolfSSL/CyaSSL: Fix memory leak in Curl_cyassl_random
RNG structure must be freed by call to FreeRng after its use in
Curl_cyassl_random. This call fixes Valgrind failures when running the
test suite with wolfSSL.
Closes #2784
- [Even Rouault brought this change]
reuse_conn(): free old_conn->options
This fixes a memory leak when CURLOPT_LOGIN_OPTIONS is used, together with
connection reuse.
I found this with oss-fuzz on GDAL and curl master:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9582
I couldn't reproduce with the oss-fuzz original test case, but looking
at curl source code pointed to this well reproducable leak.
Closes #2790
Marcel Raad (25 Jul 2018)
- [Daniel Jelinski brought this change]
system_win32: fix version checking
In the current version, VERSION_GREATER_THAN_EQUAL 6.3 will return false
when run on windows 10.0. This patch addresses that error.
Closes https://github.com/curl/curl/pull/2792
Daniel Stenberg (24 Jul 2018)
- [Johannes Schindelin brought this change]
auth: pick Bearer authentication whenever a token is available
So far, the code tries to pick an authentication method only if
user/password credentials are available, which is not the case for
Bearer authentictation...
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes #2754
- [Johannes Schindelin brought this change]
auth: only ever pick CURLAUTH_BEARER if we *have* a Bearer token
The Bearer authentication was added to cURL 7.61.0, but there is a
problem: if CURLAUTH_ANY is selected, and the server supports multiple
authentication methods including the Bearer method, we strongly prefer
that latter method (only CURLAUTH_NEGOTIATE beats it), and if the Bearer
authentication fails, we will never even try to attempt any other
method.
This is particularly unfortunate when we already know that we do not
have any Bearer token to work with.
Such a scenario happens e.g. when using Git to push to Visual Studio
Team Services (which supports Basic and Bearer authentication among
other methods) and specifying the Personal Access Token directly in the
URL (this aproach is frequently taken by automated builds).
Let's make sure that we have a Bearer token to work with before we
select the Bearer authentication among the available authentication
methods.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes #2754
Marcel Raad (22 Jul 2018)
- test320: treat curl320.out file as binary
Otherwise, LF line endings are converted to CRLF on Windows,
but no conversion is done for the reply, so the test case fails.
Closes https://github.com/curl/curl/pull/2776
Daniel Stenberg (22 Jul 2018)
- vtls: set conn->data when closing TLS
Follow-up to 1b76c38904f0. The VTLS backends that close down the TLS
layer for a connection still needs a Curl_easy handle for the session_id
cache etc.
Fixes #2764
Closes #2771
Marcel Raad (21 Jul 2018)
- tests: fixes for Windows line endlings
Set mode="text" when line endings depend on the system representation.
Closes https://github.com/curl/curl/pull/2772
- test214: disable MSYS2's POSIX path conversion for URL
By default, the MSYS2 bash converts all backslashes to forward slashes
in URLs. Disable this with MSYS2_ARG_CONV_EXCL for the test to pass.
Ref https://github.com/msys2/msys2/wiki/Porting#filesystem-namespaces
Daniel Stenberg (20 Jul 2018)
- http2: several cleanups
- separate easy handle from connections better
- added asserts on a number of places
- added sanity check of pipelines for debug builds
Closes #2751
- smb_getsock: always wait for write socket too
... the protocol is doing read/write a lot, so it needs to write often
even when downloading. A more proper fix could check for eactly when it
wants to write and only ask for it then.
Without this fix, an SMB download could easily get stuck when the event-driven
API was used.
Closes #2768
Marcel Raad (20 Jul 2018)
- test1143: disable MSYS2's POSIX path conversion
By default, the MSYS2 bash interprets http:/%HOSTIP:%HTTPPORT/want/1143
as a POSIX file list and converts it to a Windows file list.
Disable this with MSYS2_ARG_CONV_EXCL for the test to pass.
Ref https://github.com/msys2/msys2/wiki/Porting#filesystem-namespaces
Closes https://github.com/curl/curl/pull/2765
Daniel Stenberg (18 Jul 2018)
- RELEASE-NOTES: sync
... and work toward 7.61.1
- [Ruslan Baratov brought this change]
CMake: Update scripts to use consistent style
Closes #2727
Reviewed-by: Sergei Nikulov
- header output: switch off all styles, not just unbold
... the "unbold" sequence doesn't work on the mac Terminal.
Reported-by: Zero King
Fixes #2736
Closes #2738
Nick Zitzmann (14 Jul 2018)
- [Rodger Combs brought this change]
darwinssl: add support for ALPN negotiation
Marcel Raad (14 Jul 2018)
- test1422: add required file feature
curl configured with --enable-debug --disable-file currently complains
on test1422:
Info: Protocol "file" not supported or disabled in libcurl
Make test1422 dependend on enabled FILE protocol to fix this.
Fixes https://github.com/curl/curl/issues/2741
Closes https://github.com/curl/curl/pull/2742
Patrick Monnerat (12 Jul 2018)
- content_encoding: accept up to 4 unknown trailer bytes after raw deflate data
Some servers issue raw deflate data that may be followed by an undocumented
trailer. This commit makes curl tolerate such a trailer of up to 4 bytes
before considering the data is in error.
Reported-by: clbr on github
Fixes #2719
Daniel Stenberg (12 Jul 2018)
- smb: fix memory-leak in URL parse error path
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9369
Closes #2740
Marcel Raad (12 Jul 2018)
- schannel: enable CALG_TLS1PRF for w32api >= 5.1
The definition of CALG_TLS1PRF has been fixed in the 5.1 branch:
https://osdn.net/projects/mingw/scm/git/mingw-org-wsl/commits/73aedcc0f2e6ba370de0d86ab878ad76a0dda7b5
Daniel Stenberg (12 Jul 2018)
- docs/SECURITY-PROCESS: mention bounty, drop pre-notify
+ The hackerone bounty and its process
- We don't and can't handle pre-notification
- multi: always do the COMPLETED procedure/state
It was previously erroneously skipped in some situations.
libtest/libntlmconnect.c wrongly depended on wrong behavior (that it
would get a zero timeout) when no handles are "running" in a multi
handle. That behavior is no longer present with this fix. Now libcurl
will always return a -1 timeout when all handles are completed.
Closes #2733
- Curl_getoff_all_pipelines: improved for multiplexed
On multiplexed connections, transfers can be removed from anywhere not
just at the head as for pipelines.
- ares: check for NULL in completed-callback
- conn: remove the boolean 'inuse' field
... as the usage needs to be counted.
- [Paul Howarth brought this change]
openssl: assume engine support in 1.0.0 or later
Commit 38203f1585da changed engine detection to be version-based,
with a baseline of openssl 1.0.1. This does in fact break builds
with openssl 1.0.0, which has engine support - the configure script
detects that ENGINE_cleanup() is available - but <openssl/engine.h>
doesn't get included to declare it.
According to upstream documentation, engine support was added to
mainstream openssl builds as of version 0.9.7:
https://github.com/openssl/openssl/blob/master/README.ENGINE
This commit drops the version test down to 1.0.0 as version 1.0.0d
is the oldest version I have to test with.
Closes #2732
Marcel Raad (11 Jul 2018)
- schannel: fix MinGW compile break
Original MinGW's w32api has a sytax error in its definition of
CALG_TLS1PRF [0]. Don't use original MinGW w32api's CALG_TLS1PRF
until this bug [1] is fixed.
[0] https://osdn.net/projects/mingw/scm/git/mingw-org-wsl/blobs/d1d4a17e51a2b78e252ef0147d483267d56c90cc/w32api/include/wincrypt.h
[1] https://osdn.net/projects/mingw/ticket/38391
Fixes https://github.com/curl/curl/pull/2721#issuecomment-403636043
Closes https://github.com/curl/curl/pull/2728
Daniel Stenberg (11 Jul 2018)
- examples/crawler.c: move #ifdef to column 0
Apparently the C => HTML converter on the web site doesn't quite like it
otherwise.
Reported-by: Jeroen Ooms
Version 7.61.0 (11 Jul 2018)
Daniel Stenberg (11 Jul 2018)
- release: 7.61.0
- TODO: Configurable loading of OpenSSL configuration file
Closes #2724
- post303.d: clarify that this is an RFC violation
... and not the other way around, which this previously said.
Reported-by: Vasiliy Faronov
Fixes #2723
Closes #2726
- [Ruslan Baratov brought this change]
CMake: remove redundant and old end-of-block syntax
Reviewed-by: Jakub Zakrzewski
Closes #2715
Jay Satiro (9 Jul 2018)
- lib/curl_setup.h: remove unicode character
Follow-up to 82ce416.
Ref: https://github.com/curl/curl/commit/8272ec5#commitcomment-29646818
Daniel Stenberg (9 Jul 2018)
- lib/curl_setup.h: remove unicode bom from 8272ec50f02
Marcel Raad (9 Jul 2018)
- schannel: fix -Wsign-compare warning
MinGW warns:
/lib/vtls/schannel.c:219:64: warning: signed and unsigned type in
conditional expression [-Wsign-compare]
Fix this by casting the ptrdiff_t to size_t as we know it's positive.
Closes https://github.com/curl/curl/pull/2721
- schannel: workaround for wrong function signature in w32api
Original MinGW's w32api has CryptHashData's second parameter as BYTE *
instead of const BYTE *.
Closes https://github.com/curl/curl/pull/2721
- schannel: make more cipher options conditional
They are not defined in the original MinGW's <wincrypt.h>.
Closes https://github.com/curl/curl/pull/2721
- curl_setup: include <winerror.h> before <windows.h>
Otherwise, only part of it gets pulled in through <windows.h> on
original MinGW.
Fixes https://github.com/curl/curl/issues/2361
Closes https://github.com/curl/curl/pull/2721
- examples: fix -Wformat warnings
When size_t is not a typedef for unsigned long (as usually the case on
Windows), GCC emits -Wformat warnings when using lu and lx format
specifiers with size_t. Silence them with explicit casts to
unsigned long.
Closes https://github.com/curl/curl/pull/2721
Daniel Stenberg (9 Jul 2018)
- smtp: use the upload buffer size for scratch buffer malloc
... not the read buffer size, as that can be set smaller and thus cause
a buffer overflow! CVE-2018-0500
Reported-by: Peter Wu
Bug: https://curl.haxx.se/docs/adv_2018-70a2.html
- [Dave Reisner brought this change]
scripts: include _curl as part of CLEANFILES
Closes #2718
- [Nick Zitzmann brought this change]
darwinssl: allow High Sierra users to build the code using GCC
...but GCC users lose out on TLS 1.3 support, since we can't weak-link
enumeration constants.
Fixes #2656
Closes #2703
- [Ruslan Baratov brought this change]
CMake: Remove unused 'output_var' from 'collect_true'
Variable 'output_var' is not used and can be removed.
Function 'collect_true' renamed to 'count_true'.
- [Ruslan Baratov brought this change]
CMake: Remove unused functions
Closes #2711
- KNOWN_BUGS: Stick to same family over SOCKS proxy
- libssh: goto DISCONNECT state on error, not SSH_SESSION_FREE
... because otherwise not everything get closed down correctly.
Fixes #2708
Closes #2712
- libssh: include line number in state change debug messages
Closes #2713
- KNOWN_BUGS: Borland support is dropped, AIX problem is too old
- [Jeroen Ooms brought this change]
example/crawler.c: simple crawler based on libxml2
Closes #2706
- RELEASE-NOTES: synced
- DEPRECATE: include year when specifying date
- DEPRECATE: linkified
- DEPRECATE: mention the PR that disabled axTLS
- docs/DEPRECATE.md: spelling and minor formatting
- DEPRECATE: new doc describing planned item removals
Closes #2704
- [Gisle Vanem brought this change]
telnet: fix clang warnings
telnet.c(1401,28): warning: cast from function call of type 'int' to
non-matching type 'HANDLE' (aka 'void *') [-Wbad-function-cast]
Fixes #2696
Closes #2700
- docs: fix missed option name markups
- [Gaurav Malhotra brought this change]
openssl: Remove some dead code
Closes #2698
- openssl: make the requested TLS version the *minimum* wanted
The code treated the set version as the *exact* version to require in
the TLS handshake, which is not what other TLS backends do and probably
not what most people expect either.
Reported-by: Andreas Olsson
Assisted-by: Gaurav Malhotra
Fixes #2691
Closes #2694
- RELEASE-NOTES: synced
- openssl: allow TLS 1.3 by default
Reported-by: Andreas Olsson
Fixes #2692
Closes #2693
- [Adrian Peniak brought this change]
CURLINFO_TLS_SSL_PTR.3: improve the example
The previous example was a little bit confusing, because SSL* structure
(or other "in use" SSL connection pointer) is not accessible after the
transfer is completed, therefore working with the raw TLS library
specific pointer needs to be done during transfer.
Closes #2690
- travis: add a build using the synchronous name resolver
... since default uses the threaded one and we test the c-ares build
already.
Closes #2689
- configure: remove CURL_CHECK_NI_WITHSCOPEID too
Since it isn't used either and requires the getnameinfo check
Follow-up to 0aeca41702d2
- getnameinfo: not used
Closes #2687
- easy_perform: use *multi_timeout() to get wait times
... and trim the threaded Curl_resolver_getsock() to return zero
millisecond wait times during the first three milliseconds so that
localhost or names in the OS resolver cache gets detected and used
faster.
Closes #2685
Max Dymond (27 Jun 2018)
- configure: Add dependent libraries after crypto
The linker is pretty dumb and processes things left to right, keeping a
tally of symbols it hasn't resolved yet. So, we need -ldl to appear
after -lcrypto otherwise the linker won't find the dl functions.
Closes #2684
Daniel Stenberg (27 Jun 2018)
- GOVERNANCE: linkify, changed some titles
- GOVERNANCE: add maintainer details/duties
- url: check Curl_conncache_add_conn return code
... it was previously unchecked in two places and thus errors could
remain undetected and cause trouble.
Closes #2681
- include/README: remove "hacking" advice, not the right place
- RELEASE-NOTES: synced
- CURLOPT_SSL_VERIFYPEER.3: fix syntax mistake
Follow-up to b6a16afa0aa5
- netrc: use a larger buffer
... to work with longer passwords etc. Grow it from a 256 to a 4096
bytes buffer.
Reported-by: Dario Nieuwenhuis
Fixes #2676
Closes #2680
- [Patrick Schlangen brought this change]
CURLOPT_SSL_VERIFYPEER.3: Add performance note
Closes #2673
- [Javier Blazquez brought this change]
multi: fix crash due to dangling entry in connect-pending list
Fixes #2677
Closes #2679
- ConnectionExists: make sure conn->data is set when "taking" a connection
Follow-up to 2c15693.
Bug #2674
Closes #2675
- [Kevin R. Bulgrien brought this change]
system.h: fix for gcc on 32 bit OpenServer
Bug: https://curl.haxx.se/mail/lib-2018-06/0100.html
- [Raphael Gozzo brought this change]
cmake: allow multiple SSL backends
This will make possible to select the SSL backend (using
curl_global_sslset()) even when the libcurl is built using CMake
Closes #2665
- url: fix dangling conn->data pointer
By masking sure to use the *current* easy handle with extracted
connections from the cache, and make sure to NULLify the ->data pointer
when the connection is put into the cache to make this mistake easier to
detect in the future.
Reported-by: Will Dietz
Fixes #2669
Closes #2672
- CURLOPT_INTERFACE.3: interface names not supported on Windows
- travis: run more tests for coverage check
... run a few more tortured based and run all tests event-based.
Closes #2664
- multi: fix memory leak when stopped during name resolve
When the application just started the transfer and then stops it while
the name resolve in the background thread hasn't completed, we need to
wait for the resolve to complete and then cleanup data accordingly.
Enabled test 1553 again and added test 1590 to also check when the host
name resolves successfully.
Detected by OSS-fuzz.
Closes #1968
Viktor Szakats (15 Jun 2018)
- maketgz: delete .bak files, fix indentation
Ref: https://github.com/curl/curl/pull/2660
Closes https://github.com/curl/curl/pull/2662
Daniel Stenberg (15 Jun 2018)
- runtests.pl: remove debug leftover from bb9a340c73f3
- curl-confopts.m4: fix typo from ed224f23d5beb
Fixes my local configure to detect a custom installed c-ares without
pkgconfig.
- docs/RELEASE-PROCEDURE.md: renamed to use .md extension
Closes #2663
- RELEASE-PROCEDURE: gpg sign the tags
- RELEASE-NOTES: synced
- CURLOPT_HTTPAUTH.3: CURLAUTH_BEARER was added in 7.61.0
- [Mamta Upadhyay brought this change]
maketgz: fix sed issues on OSX
maketgz creates release tarballs and removes the -DEV string in curl
version (e.g. 7.58.0-DEV), else -DEV shows up on command line when curl
is run. maketgz works fine on linux but fails on OSX. Problem is with
the sed commands that use option -i without an extension. Maketgz
expects GNU sed instead of BSD and this simply won't work on OSX. Adding
a backup extension .bak after -i fixes this issue
Running the script as if on OSX gives this error:
sed: -e: No such file or directory
Adding a .bak extension resolves it
Closes #2660
- configure: enhance ability to detect/build with static openssl
Fix the -ldl and -ldl + -lpthread checks for OpenSSL, necessary for
building with static libs without pkg-config.
Reported-by: Marcel Raad
Fixes #2199
Closes #2659
- configure: use pkg-config for c-ares detection
First check if there's c-ares information given as pkg-config info and use
that as first preference.
Reported-by: pszemus on github
Fixes #2203
Closes #2658
- GOVERNANCE.md: explains how this project is run
Closes #2657
- KNOWN_BUGS: NTLM doen't support password with § character
Closes #2120
- KNOWN_BUGS: slow connect to localhost on Windows
Closes #2281
- [Matteo Bignotti brought this change]
mk-ca-bundle.pl: make -u delete certdata.txt if found not changed
certdata.txt should be deleted also when the process is interrupted by
"same certificate downloaded, exiting"
The certdata.txt is currently kept on disk even if you give the -u
option
Closes #2655
- progress: remove a set of unused defines
Reported-by: Peter Wu
Closes #2654
- TODO: "Option to refuse usernames in URLs" done
Implemented by Björn in 946ce5b61f
- [Lyman Epp brought this change]
Curl_init_do: handle NULL connection pointer passed in
Closes #2653
- runtests: support variables in <strippart>
... and make use of that to make 1455 work better without using a fixed
local port number.
Fixes #2649
Closes #2650
- Curl_debug: remove dead printhost code
The struct field is never set (since 5e0d9aea3) so remove the use of it
and remove the connectdata pointer from the prototype.
Reported-by: Tejas
Bug: https://curl.haxx.se/mail/lib-2018-06/0054.html
Closes #2647
Viktor Szakats (12 Jun 2018)
- schannel: avoid incompatible pointer warning
with clang-6.0:
```
vtls/schannel_verify.c: In function 'add_certs_to_store':
vtls/schannel_verify.c:212:30: warning: passing argument 11 of 'CryptQueryObject' from incompatible pointer type [-Wincompatible-pointer-types]
&cert_context)) {
^
In file included from /usr/share/mingw-w64/include/schannel.h:10:0,
from /usr/share/mingw-w64/include/schnlsp.h:9,
from vtls/schannel.h:29,
from vtls/schannel_verify.c:40:
/usr/share/mingw-w64/include/wincrypt.h:4437:26: note: expected 'const void **' but argument is of type 'CERT_CONTEXT ** {aka struct _CERT_CONTEXT **}'
WINIMPM WINBOOL WINAPI CryptQueryObject (DWORD dwObjectType, const void *pvObject, DWORD dwExpectedContentTypeFlags, DWORD dwExpectedFormatTypeFlags, DWORD dwFlags,
^~~~~~~~~~~~~~~~
```
Ref: https://msdn.microsoft.com/library/windows/desktop/aa380264
Closes https://github.com/curl/curl/pull/2648
Daniel Stenberg (12 Jun 2018)
- [Robert Prag brought this change]
schannel: support selecting ciphers
Given the contstraints of SChannel, I'm exposing these as the algorithms
themselves instead; while replicating the ciphersuite as specified by
OpenSSL would have been preferable, I found no way in the SChannel API
to do so.
To use this from the commandline, you need to pass the names of contants
defining the desired algorithms. For example, curl --ciphers
"CALG_SHA1:CALG_RSA_SIGN:CALG_RSA_KEYX:CALG_AES_128:CALG_DH_EPHEM"
https://github.com The specific names come from wincrypt.h
Closes #2630
- [Bernhard M. Wiedemann brought this change]
test 46: make test pass after 2025
shifting the expiry date to 2037 for now
to be before the possibly problematic year 2038
similar in spirit to commit e6293cf8764e9eecb
Closes #2646
- [Marian Klymov brought this change]
cppcheck: fix warnings
- Get rid of variable that was generating false positive warning
(unitialized)
- Fix issues in tests
- Reduce scope of several variables all over
etc
Closes #2631
- openssl: assume engine support in 1.0.1 or later
Previously it was checked for in configure/cmake, but that would then
leave other build systems built without engine support.
While engine support probably existed prior to 1.0.1, I decided to play
safe. If someone experience a problem with this, we can widen the
version check.
Fixes #2641
Closes #2644
- RELEASE-NOTES: synced
- RELEASE-PROCEDURE: update the release calendar for 2019
- [Gisle Vanem brought this change]
boringssl + schannel: undef X509_NAME in lib/schannel.h
Fixes the build problem when both boringssl and schannel are enabled.
Fixes #2634
Closes #2643
- [Vladimir Kotal brought this change]
mk-ca-bundle.pl: leave certificate name untouched in decode()
Closes #2640
- [Rikard Falkeborn brought this change]
tests/libtests/Makefile.am: Add lib1521.c to CLEANFILES
This removes the generated lib1521.c when running make clean.
Closes #2633
- [Rikard Falkeborn brought this change]
tests/libtest: Add lib1521 to nodist_SOURCES
Since 467da3af0, lib1521.c is generated instead of checked in. According
to the commit message, the intention was to remove it from the tarball
as well. However, it is still present when running make dist. To remove
it, add it to nodist_lib1521_SOURCES. This also means there is no need
for the manually added dist-rule in the Makefile.
Also update CMakelists.txt to handle the fact that we now may have
nodist_SOURCES.
- [Stephan Mühlstrasser brought this change]
system.h: add support for IBM xlc C compiler
Added a section to system.h guarded with __xlc__ for the IBM xml C
compiler. Before this change the section titled 'generic "safe guess" on
old 32 bit style' was used, which resulted in a wrong definition of
CURL_TYPEOF_CURL_SOCKLEN_T, and for 64-bit also CURL_TYPEOF_CURL_OFF_T
was wrong.
Compilation warnings fixed with this change:
CC libcurl_la-ftp.lo
"ftp.c", line 290.55: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
"ftp.c", line 293.48: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
"ftp.c", line 1070.49: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
"ftp.c", line 1154.53: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
"ftp.c", line 1187.51: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
CC libcurl_la-connect.lo
"connect.c", line 448.56: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
"connect.c", line 516.66: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
"connect.c", line 687.55: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
"connect.c", line 696.55: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
CC libcurl_la-tftp.lo
"tftp.c", line 1115.33: 1506-280 (W) Function argument assignment between types "unsigned long* restrict" and "int*" is not allowed.
Closes #2637
- cmdline-opts/cert-type.d: mention "p12" as a recognized type as well
Viktor Szakats (3 Jun 2018)
- spelling fixes
Detected using the `codespell` tool (version 1.13.0).
Also secure and fix an URL.
Daniel Stenberg (2 Jun 2018)
- axtls: follow-up spell fix of comment
- axTLS: not considered fit for use
URL: https://curl.haxx.se/mail/lib-2018-06/0000.html
This is step one. It adds #error statements that require source edits to
make curl build again if asked to use axTLS. At a later stage we might
remove the axTLS specific code completely.
Closes #2628
- build: remove the Borland specific makefiles
According to the user survey 2018, not even one out of 670 users use
them. Nobody on the mailing list spoke up for them either.
Closes #2629
- curl_addrinfo: use same #ifdef conditions in source as header
... for curl_dofreeaddrinfo
- multi: remove a DEBUGF()
... it might call infof() with a NULL first argument that isn't harmful
but makes it not do anything. The infof() line is not very useful
anymore, it has served it purpose. Good riddance!
Fixes #2627
- [Alibek.Jorajev brought this change]
CURLOPT_RESOLVE: always purge old entry first
If there's an existing entry using the selected name.
Closes #2622
- fnmatch: use the system one if available
If configure detects fnmatch to be available, use that instead of our
custom one for FTP wildcard pattern matching. For standard compliance,
to reduce our footprint and to use already well tested and well
exercised code.
A POSIX fnmatch behaves slightly different than the internal function
for a few test patterns currently and the macOS one yet slightly
different. Test case 1307 is adjusted for these differences.
Closes #2626
Patrick Monnerat (31 May 2018)
- os400: add new option in ILE/RPG binding
Follow-up to commit 946ce5b
Daniel Stenberg (31 May 2018)
- tests/libtest/.gitignore: follow-up fix to ignore lib5* too
- KNOWN_BUGS: CURL_GLOBAL_SSL
Closes #2276
- [Bernhard Walle brought this change]
configure: check for declaration of getpwuid_r
On our x86 Android toolchain, getpwuid_r is implemented but the header
is missing:
netrc.c:81:7: error: implicit declaration of function 'getpwuid_r' [-Werror=implicit-function-declaration]
Unfortunately, the function is used in curl_ntlm_wb.c, too, so I moved
the prototype to curl_setup.h.
Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
Closes #2609
- [Rikard Falkeborn brought this change]
tests: update .gitignore for libtests
Closes #2624
- [Rikard Falkeborn brought this change]
strictness: correct {infof, failf} format specifiers
Closes #2623
- [Björn Stenberg brought this change]
option: disallow username in URL
Adds CURLOPT_DISALLOW_USERNAME_IN_URL and --disallow-username-in-url. Makes
libcurl reject URLs with a username in them.
Closes #2340
- libcurl-security.3: improved layout for two rememdy lists
- libcurl-security.3: refer to URL instead of in-source markdown file
Viktor Szakats (30 May 2018)
- curl.rc: embed manifest for correct Windows version detection
* enable it in `src/Makefile.m32`
* enable it in `winbuild/MakefileBuild.vc` if a custom manifest is
_not_ enabled via the existing `EMBED_MANIFEST` option
* enable it for all Windows CMake builds (also disable the built-in
minimal manifest, added by CMake by default.)
For other build systems, add the `-DCURL_EMBED_MANIFEST` option to
the list of RC (Resource Compiler) flags to enable the manifest
included in `src/curl.rc`. This may require to disable whatever
automatic or other means in which way another manifest is added to
`curl.exe`.
Notice that Borland C doesn't support this method due to a
long-pending resource compiler bug. Watcom C may also not handle
it correctly when the `-zm` `wrc` option is used (this option may
be unnecessary though) and regardless of options in certain earlier
revisions of the 2.0 beta version.
Closes https://github.com/curl/curl/pull/1221
Fixes https://github.com/curl/curl/issues/2591
Patrick Monnerat (30 May 2018)
- os400: sync EBCDIC wrappers and ILE/RPG binding with latest options
- os400: implement mime api EBCDIC wrappers
Also sync ILE/RPG binding to define the new functions.
Daniel Stenberg (29 May 2018)
- setopt: add TLS 1.3 ciphersuites
Adds CURLOPT_TLS13_CIPHERS and CURLOPT_PROXY_TLS13_CIPHERS.
curl: added --tls13-ciphers and --proxy-tls13-ciphers
Fixes #2435
Reported-by: zzq1015 on github
Closes #2607
- configure: override AR_FLAGS to silence warning
The automake default ar flags are 'cru', but the 'u' flag in there
causes warnings on many modern Linux distros. Removing 'u' may have a
minor performance impact on older distros but should not cause harm.
Explained on the automake mailing list already back in April 2015:
https://www.mail-archive.com/automake-patches@gnu.org/msg07705.html
Reported-by: elephoenix on github
Fixes #2617
Closes #2619
Sergei Nikulov (29 May 2018)
- cmake: fixed comments in compile checks code
Daniel Stenberg (29 May 2018)
- INSTALL: LDFLAGS=-Wl,-R/usr/local/ssl/lib
... the older description doesn't work
Reported-by: Peter Varga
Fixes #2615
Closes #2616
- [Will Dietz brought this change]
KNOWN_BUGS: restore text regarding #2101.
This was added earlier but appears to have been removed accidentally.
AFAICT this is very much still an issue.
-----
I say "accidentally" because the text seems to have harmlessly snuck
into [1] (which makes no mention of it). [1] was later reverted for
unspecified reasons in [2], presumably because the mentioned issue was
fixed or invalid.
[1] de9fac00c40db321d44fa6fbab6eb62ec4c83998
[2] 16d1f369403cbb04bd7b085eabbeebf159473fc2
Closes #2618
- fnmatch: insist on escaped bracket to match
A non-escaped bracket ([) is for a character group - as documented. It
will *not* match an individual bracket anymore. Test case 1307 updated
accordingly to match.
Problem detected by OSS-Fuzz, although this fix is probably not a final
fix for the notorious timeout issues.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8525
Closes #2614
Patrick Monnerat (28 May 2018)
- psl: use latest psl and refresh it periodically
The latest psl is cached in the multi or share handle. It is refreshed
before use after 72 hours.
New share lock CURL_LOCK_DATA_PSL controls the psl cache sharing.
If the latest psl is not available, the builtin psl is used.
Reported-by: Yaakov Selkowitz
Fixes #2553
Closes #2601
Daniel Stenberg (28 May 2018)
- [Fabrice Fontaine brought this change]
configure: fix ssh2 linking when built with a static mbedtls
The ssh2 pkg-config file could contain the following lines when build
with a static version of mbedtls:
Libs: -L${libdir} -lssh2 /xxx/libmbedcrypto.a
Libs.private: /xxx/libmbedcrypto.a
This static mbedtls library must be used to correctly detect ssh2
support and this library must be copied in libcurl.pc otherwise
compilation of any application (such as upmpdcli) with libcurl will fail
when trying to found mbedtls functions included in libssh2. So, replace
pkg-config --libs-only-l by pkg-config --libs.
Fixes:
- http://autobuild.buildroot.net/results/43e24b22a77f616d6198c10435dcc23cc3b9088a
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Closes #2613
- RELEASE-NOTES: synced
- [Bernhard Walle brought this change]
cmake: check for getpwuid_r
The autotools-based build system does it, so we do it also in CMake.
Bug: #2609
Signed-off-by: Bernhard Walle <bernhard@bwalle.de>
- cmdline-opts/gen.pl: warn if mutexes: or see-also: list non-existing options
- [Frank Gevaerts brought this change]
curl.1: Fix cmdline-opts reference errors.
--data, --form, and --ntlm were declared to be mutually exclusive with
non-existing options. --data and --form referred to --upload (which is
short for --upload-file and therefore did work, so this one was merely
a bit confusing), --ntlm referred to --negotiated instead of --negotiate.
Closes #2612
- [Frank Gevaerts brought this change]
docs: fix cmdline-opts metadata headers case consistency.
Almost all headers start with an uppercase letter, but some didn't.
- mailmap: Max Savenkov
Sergei Nikulov (28 May 2018)
- [Max Savenkov brought this change]
Fix the test for fsetxattr and strerror_r tests in CMake to work without compiling
Daniel Stenberg (27 May 2018)
- mailmap: a Richard Alcock fixup
- [Richard Alcock brought this change]
schannel: add failf calls for client certificate failures
Closes #2604
- [Richard Alcock brought this change]
winbuild: In MakefileBuild.vc fix typo DISTDIR->DIRDIST
Change requirement from $(DISTDIR) to $(DIRDIST)
closes #2603
- [Richard Alcock brought this change]
winbuild: only delete OUTFILE if it exists
This removes the slightly annoying "Could not file LIBCURL_OBJS.inc" and
"Could not find CURL_OBJS.inc.inc" message when building into a clean
folder.
closes #2602
- [Alejandro R. Sedeño brought this change]
content_encoding: handle zlib versions too old for Z_BLOCK
Fallback on Z_SYNC_FLUSH when Z_BLOCK is not available.
Fixes #2606
Closes #2608
- multi: provide a socket to wait for in Curl_protocol_getsock
... even when there's no protocol specific handler setup.
Bug: https://curl.haxx.se/mail/lib-2018-05/0062.html
Reported-by: Sean Miller
Closes #2600
- [Linus Lewandowski brought this change]
httpauth: add support for Bearer tokens
Closes #2102
- TODO: CURLINFO_PAUSE_STATE
Closes #2588
Sergei Nikulov (24 May 2018)
- cmake: set -d postfix for debug builds if not specified
using -DCMAKE_DEBUG_POSTFIX explicitly
fixes #2121, obsoletes #2384
Daniel Stenberg (23 May 2018)
- configure: add basic test of --with-ssl prefix
When given a prefix, the $PREFIX_OPENSSL/lib/openssl.pc or
$PREFIX_OPENSSL/include/openssl/ssl.h files must be present or cause an
error. Helps users detect when giving configure the wrong path.
Reported-by: Oleg Pudeyev
Assisted-by: Per Malmberg
Fixes #2580
Patrick Monnerat (22 May 2018)
- http resume: skip body if http code 416 (range error) is ignored.
This avoids appending error data to already existing good data.
Test 92 is updated to match this change.
New test 1156 checks all combinations of --range/--resume, --fail,
Content-Range header and http status code 200/416.
Fixes #1163
Reported-By: Ithubg on github
Closes #2578
Daniel Stenberg (22 May 2018)
- tftp: make sure error is zero terminated before printfing it
- configure: add missing m4/ax_compile_check_sizeof.m4
follow-up to mistake in 6876ccf90b4
Jay Satiro (22 May 2018)
- [Johannes Schindelin brought this change]
schannel: make CAinfo parsing resilient to CR/LF
OpenSSL has supported --cacert for ages, always accepting LF-only line
endings ("Unix line endings") as well as CR/LF line endings ("Windows
line endings").
When we introduced support for --cacert also with Secure Channel (or in
cURL speak: "WinSSL"), we did not take care to support CR/LF line
endings, too, even if we are much more likely to receive input in that
form when using Windows.
Let's fix that.
Happily, CryptQueryObject(), the function we use to parse the ca-bundle,
accepts CR/LF input already, and the trailing LF before the END
CERTIFICATE marker catches naturally any CR/LF line ending, too. So all
we need to care about is the BEGIN CERTIFICATE marker. We do not
actually need to verify here that the line ending is CR/LF. Just
checking for a CR or an LF is really plenty enough.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Closes https://github.com/curl/curl/pull/2592
Daniel Stenberg (22 May 2018)
- CURLOPT_ACCEPT_ENCODING.3: add brotli and clarify a bit
- RELEASE-NOTES: synced
- KNOWN_BUGS: mention the -O with %-encoded file names
Closes #2573
- checksrc: make sure sizeof() is used *with* parentheses
... and unify the source code to adhere.
Closes #2563
- curl: added --styled-output
It is enabled by default, so --no-styled-output will switch off the
detection/use of bold headers.
Closes #2538
- curl: show headers in bold
The feature is only enabled if the output is believed to be a tty.
-J: There's some minor differences and improvements in -J handling, as
now J should work with -i and it actually creates a file first using the
initial name and then *renames* that to the one found in
Content-Disposition (if any).
-i: only shows headers for HTTP transfers now (as documented).
Previously it would also show for pieces of the transfer that were HTTP
(for example when doing FTP over a HTTP proxy).
-i: now shows trailers as well. Previously they were not shown at all.
--libcurl: the CURLOPT_HEADER is no longer set, as the header output is
now done in the header callback.
- configure: compile-time SIZEOF checks
... instead of exeucting code to get the size. Removes the use of
LD_LIBRARY_PATH for this.
Fixes #2586
Closes #2589
Reported-by: Bernhard Walle
- configure: replace AC_TRY_RUN with CURL_RUN_IFELSE
... and export LD_LIBRARY_PATH properly. This is a follow-up from
2d4c215.
Fixes #2586
Reported-by: Bernhard Walle
- docs: clarify CURLOPT_HTTPGET somewhat
Reported-by: bsammon on github
Fixes #2590
- curl_fnmatch: only allow two asterisks for matching
The previous limit of 5 can still end up in situation that takes a very
long time and consumes a lot of CPU.
If there is still a rare use case for this, a user can provide their own
fnmatch callback for a version that allows a larger set of wildcards.
This commit was triggered by yet another OSS-Fuzz timeout due to this.
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8369
Closes #2587
- checksrc: fix too long line
follow-up to e05ad5d
- [Aleks brought this change]
docs: mention HAproxy protocol "version 1"
...as there's also a version 2.
Closes #2579
- examples/progressfunc: make it build on older libcurls
This example was changed in ce2140a8c1 to use the new microsecond based
getinfo option. This change makes it conditionally keep using the older
option so that the example still builds with older libcurl versions.
Closes #2584
- stub_gssapi: fix numerous 'unused parameter' warnings
follow-up to d9e92fd9fd1d
- [Philip Prindeville brought this change]
getinfo: add microsecond precise timers for various intervals
Provide a set of new timers that return the time intervals using integer
number of microseconds instead of floats.
The new info names are as following:
CURLINFO_APPCONNECT_TIME_T
CURLINFO_CONNECT_TIME_T
CURLINFO_NAMELOOKUP_TIME_T
CURLINFO_PRETRANSFER_TIME_T
CURLINFO_REDIRECT_TIME_T
CURLINFO_STARTTRANSFER_TIME_T
CURLINFO_TOTAL_TIME_T
Closes #2495
- openssl: acknowledge --tls-max for default version too
... previously it only used the max setting if a TLS version was also
explicitly asked for.
Reported-by: byte_bucket
Fixes #2571
Closes #2572
- bump: start working on the pending 7.61.0
- [Dagobert Michelsen brought this change]
tests/libtest/Makefile: Do not unconditionally add gcc-specific flags
The warning flag leads e.g. Sun Studio compiler to bail out.
Closes #2576
- schannel_verify: fix build for non-schannel
Jay Satiro (16 May 2018)
- rand: fix typo
- schannel: disable manual verify if APIs not available
.. because original MinGW and old compilers do not have the Windows API
definitions needed to support manual verification.
- [Archangel_SDY brought this change]
schannel: disable client cert option if APIs not available
Original MinGW targets Windows 2000 by default, which lacks some APIs and
definitions for this feature. Disable it if these APIs are not available.
Closes https://github.com/curl/curl/pull/2522
Version 7.60.0 (15 May 2018)
Daniel Stenberg (15 May 2018)
- RELEASE-NOTES: 7.60.0 release
- THANKS: added people from the curl 7.60.0 release
- docs/libcurl/index.html: removed
The HTML files are long gone from the dist, now remove the last HTML
file pointing to those missing files.
d
- [steini2000 brought this change]
http2: remove unused variable
Closes #2570
- [steini2000 brought this change]
http2: use easy handle of stream for logging
- gcc: disable picky gcc-8 function pointer warnings in two places
Reported-by: Rikard Falkeborn
Bug: #2560
Closes #2569
- http2: use the correct function pointer typedef
Fixes gcc-8 picky compiler warnings
Reported-by: Rikard Falkeborn
Bug: #2560
Closes #2568
- CODE_STYLE: mention return w/o parens, but sizeof with
... and remove the github markdown syntax so that it renders better on
the web site. Also, don't use back-ticks inlined to allow the CSS to
highlight source code better.
- [Rikard Falkeborn brought this change]
examples: Fix format specifiers
Closes #2561
- [Rikard Falkeborn brought this change]
tool: Fix format specifiers
- [Rikard Falkeborn brought this change]
ntlm: Fix format specifiers
- [Rikard Falkeborn brought this change]
tests: Fix format specifiers
- [Rikard Falkeborn brought this change]
lib: Fix format specifiers
- contributors.sh: use "on github", not at
- http2: getsock fix for uploads
When there's an upload in progress, make sure to wait for the socket to
become writable.
Detected-by: steini2000 on github
Bug: #2520
Closes #2567
- pingpong: fix response cache memcpy overflow
Response data for a handle with a large buffer might be cached and then
used with the "closure" handle when it has a smaller buffer and then the
larger cache will be copied and overflow the new smaller heap based
buffer.
Reported-by: Dario Weisser
CVE: CVE-2018-1000300
Bug: https://curl.haxx.se/docs/adv_2018-82c2.html
- http: restore buffer pointer when bad response-line is parsed
... leaving the k->str could lead to buffer over-reads later on.
CVE: CVE-2018-1000301
Assisted-by: Max Dymond
Detected by OSS-Fuzz.
Bug: https://curl.haxx.se/docs/adv_2018-b138.html
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7105
Patrick Monnerat (13 May 2018)
- cookies: do not take cookie name as a parameter
RFC 6265 section 4.2.1 does not set restrictions on cookie names.
This is a follow-up to commit 7f7fcd0.
Also explicitly check proper syntax of cookie name/value pair.
New test 1155 checks that cookie names are not reserved words.
Reported-By: anshnd at github
Fixes #2564
Closes #2566
Daniel Stenberg (12 May 2018)
- smb: reject negative file sizes
Assisted-by: Max Dymond
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8245
- setup_transfer: deal with both sockets being -1
Detected by Coverity; CID 1435559. Follow-up to f8d608f38d00. It would
index the array with -1 if neither index was a socket.
- travis: add build using NSS
Closes #2558
- [Sunny Purushe brought this change]
openssl: change FILE ops to BIO ops
To make builds with VS2015 work. Recent changes in VS2015 _IOB_ENTRIES
handling is causing problems. This fix changes the OpenSSL backend code
to use BIO functions instead of FILE I/O functions to circumvent those
problems.
Closes #2512
- travis: add a build using WolfSSL
Assisted-by: Dan Fandrich
Closes #2528
- RELEASE-NOTES: typo
- RELEASE-NOTES: synced
- [Daniel Gustafsson brought this change]
URLs: fix one more http url
This file wasn't included in commit 4af40b3646d3b09 which updated all
haxx.se http urls to https. The file was committed prior to that update,
but may have been merged after it and hence didn't get updated.
Closes #2550
- github/lock: auto-lock closed issues after 90 days of inactivity
- vtls: fix missing commas
follow-up to e66cca046cef
- vtls: use unified "supports" bitfield member in backends
... instead of previous separate struct fields, to make it easier to
extend and change individual backends without having to modify them all.
closes #2547
- transfer: don't unset writesockfd on setup of multiplexed conns
Curl_setup_transfer() can be called to setup a new individual transfer
over a multiplexed connection so it shouldn't unset writesockfd.
Bug: #2520
Closes #2549
- [Frank Gevaerts brought this change]
configure: put CURLDEBUG and DEBUGBUILD in lib/curl_config.h
They are removed from the compiler flags.
This ensures that make dependency tracking will force a rebuild whenever
configure --enable-debug or --enable-curldebug changes.
Closes #2548
- http: don't set the "rewind" flag when not uploading anything
It triggers an assert.
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8144
Closes #2546
- travis: add an mbedtls build
Closes #2531
- configure: only check for CA bundle for file-using SSL backends
When only building with SSL backends that don't use the CA bundle file
(by default), skip the check.
Fixes #2543
Fixes #2180
Closes #2545
- ssh-libssh.c: fix left shift compiler warning
ssh-libssh.c:2429:21: warning: result of '1 << 31' requires 33 bits to
represent, but 'int' only has 32 bits [-Wshift-overflow=]
'len' will never be that big anyway so I converted the run-time check to
a regular assert.
- [Stephan Mühlstrasser brought this change]
URL: fix ASCII dependency in strcpy_url and strlen_url
Commit 3c630f9b0af097663a64e5c875c580aa9808a92b partially reverted the
changes from commit dd7521bcc1b7a6fcb53c31f9bd1192fcc884bd56 because of
the problem that strcpy_url() was modified unilaterally without also
modifying strlen_url(). As a consequence strcpy_url() was again
depending on ASCII encoding.
This change fixes strlen_url() and strcpy_url() in parallel to use a
common host-encoding independent criterion for deciding whether an URL
character must be %-escaped.
Closes #2535
- [Denis Ollier brought this change]
docs: remove extraneous commas in man pages
Closes #2544
- RELEASE-NOTES: synced
- Revert "TODO: remove configure --disable-pthreads"
This reverts commit d5d683a97f9765bddfd964fe32e137aa6e703ed3.
--disable-pthreads can be used to disable pthreads and get the threaded
resolver to use the windows threading when building with mingw.
- vtls: don't define MD5_DIGEST_LENGTH for wolfssl
... as it defines it (too)
- TODO: remove configure --disable-pthreads
Jay Satiro (2 May 2018)
- [David Garske brought this change]
wolfssl: Fix non-blocking connect
Closes https://github.com/curl/curl/pull/2542
Daniel Stenberg (30 Apr 2018)
- CURLOPT_URL.3: add ENCODING section [ci skip]
Feedback-by: Michael Kilburn
- KNOWN_BUGS: Client cert with Issuer DN differs between backends
Closes #1411
- KNOWN_BUGS: Passive transfer tries only one IP address
Closes #1508
- KNOWN_BUGS: --upload-file . hang if delay in STDIN
Closes #2051
- KNOWN_BUGS: Connection information when using TCP Fast Open
Closes #1332
- travis: enable libssh2 on both macos and Linux
It seems to not be detected by default anymore (which is a bug I
believe)
Closes #2541
- TODO: Support the clienthello extension
Closes #2299
- TODO: CLOEXEC
Closes #2252
- tests: provide 'manual' as a feature to optionally require
... and make test 1026 rely on that feature so that --disable-manual
builds don't cause test failures.
Reported-by: Max Dymond and Anders Roxell
Fixes #2533
Closes #2540
- CURLINFO_PROTOCOL.3: mention the existing defined names
Jay Satiro (27 Apr 2018)
- [Daniel Gustafsson brought this change]
cookies: remove unused macro
Commit 2bc230de63 made the macro MAX_COOKIE_LINE_TXT become unused,
so remove as it's not part of the published API.
Closes https://github.com/curl/curl/pull/2537
Daniel Stenberg (27 Apr 2018)
- [Daniel Gustafsson brought this change]
checksrc: force indentation of lines after an else
This extends the INDENTATION case to also handle 'else' statements
and require proper indentation on the following line. Also fixes the
offending cases found in the codebase.
Closes #2532
- http2: fix null pointer dereference in http2_connisdead
This function can get called on a connection that isn't setup enough to
have the 'recv_underlying' function pointer initialized so it would try
to call the NULL pointer.
Reported-by: Dario Weisser
Follow-up to db1b2c7fe9b093f8 (never shipped in a release)
Closes #2536
- http2: get rid of another strstr()
Follow-up to 1514c44655e12e: replace another strstr() call done on a
buffer that might not be zero terminated - with a memchr() call, even if
we know the substring will be found.
Assisted-by: Max Dymond
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8021
Closes #2534
- cyassl: adapt to libraries without TLS 1.0 support built-in
WolfSSL doesn't enable it by default anymore
- configure: provide --with-wolfssl as an alias for --with-cyassl
- RELEASE-NOTES: synced
- [Daniel Gustafsson brought this change]
os400.c: fix ASSIGNWITHINCONDITION checksrc warnings
All occurrences of assignment within conditional expression in
os400sys.c rewritten into two steps: first assignment and then the check
on the success of the assignment. Also adjust related incorrect brace
positions to match project indentation style.
This was spurred by seeing "if((inp = input_token))", but while in there
all warnings were fixed.
There should be no functional change from these changes.
Closes #2525
- [Daniel Gustafsson brought this change]
cookies: ensure that we have cookies before writing jar
The jar should be written iff there are cookies, so ensure that we still
have cookies after expiration to avoid creating an empty file.
Closes #2529
- strcpy_url: only %-encode values >= 0x80
OSS-Fuzz detected
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=8000
Broke in dd7521bcc1b7
- mime: avoid NULL pointer dereference risk
Coverity detected, CID 1435120
Closes #2527
- [Stephan Mühlstrasser brought this change]
ctype: restore character classification for non-ASCII platforms
With commit 4272a0b0fc49a1ac0ceab5c4a365c9f6ab8bf8e2 curl-speficic
character classification macros and functions were introduced in
curl_ctype.[ch] to avoid dependencies on the locale. This broke curl on
non-ASCII, e.g. EBCDIC platforms. This change restores the previous set
of character classification macros when CURL_DOES_CONVERSIONS is
defined.
Closes #2494
- ftplistparser: keep state between invokes
Fixes FTP wildcard parsing when done over a number of read buffers.
Regression from f786d1f14
Reported-by: wncboy on github
Fixes #2445
Closes #2526
- examples/http2-upload: expand buffer to avoid silly warning
http2-upload.c:135:44: error: ‘%02d’ directive output may be truncated
writing between 2 and 11 bytes into a region of size between 8 and 17
- examples/sftpuploadresume: typecast fseek argument to long
/docs/examples/sftpuploadresume.c:102:12: warning: conversion to 'long
int' from 'curl_off_t {aka long long int}' may alter its value
- Revert "ftplistparser: keep state between invokes"
This reverts commit abbc8457d85aca74b7cfda1d394b0844932b2934.
Caused fuzzer problems on travis not seen when this was a PR!
- Curl_memchr: zero length input can't match
Avoids undefined behavior.
Reported-by: Geeknik Labs
- ftplistparser: keep state between invokes
Fixes FTP wildcard parsing when doing over a number of read buffers.
Regression from f786d1f14
Reported-by: wncboy on github
Fixes #2445
Closes #2519
- ftplistparser: renamed some members and variables
... to make them better spell out what they're for.
- RELEASE-NOTES: synced
- [Christian Schmitz brought this change]
curl_global_sslset: always provide available backends
Closes #2499
- http2: convert an assert to run-time check
Fuzzing has proven we can reach code in on_frame_recv with status_code
not having been set, so let's detect that in run-time (instead of with
assert) and error error accordingly.
(This should no longer happen with the latest nghttp2)
Detected by OSS-Fuzz
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7903
Closes #2514
- curl.1: clarify that options and URLs can be mixed
Fixes #2515
Closes #2517
Jay Satiro (23 Apr 2018)
- [Archangel_SDY brought this change]
CURLOPT_SSLCERT.3: improve WinSSL-specific usage info
Ref: https://github.com/curl/curl/pull/2376#issuecomment-381858780
Closes https://github.com/curl/curl/pull/2504
- [Archangel_SDY brought this change]
schannel: fix build error on targets <= XP
- Use CRYPT_STRING_HEX instead of CRYPT_STRING_HEXRAW since XP doesn't
support the latter.
Ref: https://github.com/curl/curl/pull/2376#issuecomment-382153668
Closes https://github.com/curl/curl/pull/2504
Daniel Stenberg (23 Apr 2018)
- Revert "ftplistparser: keep state between invokes"
This reverts commit 8fb78f9ddc6d858d630600059b8ad84a80892fd9.
Unfortunately this fix introduces memory leaks I've not been able to fix
in several days. Reverting this for now to get the leaks fixed.
Jay Satiro (21 Apr 2018)
- tool_help: clarify --max-time unit of time is seconds
Before:
-m, --max-time <time> Maximum time allowed for the transfer
After:
-m, --max-time <seconds> Maximum time allowed for the transfer
Daniel Stenberg (20 Apr 2018)
- http2: handle GOAWAY properly
When receiving REFUSED_STREAM, mark the connection for close and retry
streams accordingly on another/fresh connection.
Reported-by: Terry Wu
Fixes #2416
Fixes #1618
Closes #2510
- http2: clear the "drain counter" when a stream is closed
This fixes the notorious "httpc->drain_total >= data->state.drain"
assert.
Reported-by: Anders Bakken
Fixes #1680
Closes #2509
- http2: avoid strstr() on data not zero terminated
It's not strictly clear if the API contract allows us to call strstr()
on a string that isn't zero terminated even when we know it will find
the substring, and clang's ASAN check dislikes us for it.
Also added a check of the return code in case it fails, even if I can't
think of a situation how that can trigger.
Detected by OSS-Fuzz
Closes #2513
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7760
- [Stephan Mühlstrasser brought this change]
openssl: fix subjectAltName check on non-ASCII platforms
Curl_cert_hostcheck operates with the host character set, therefore the
ASCII subjectAltName string retrieved with OpenSSL must be converted to
the host encoding before comparison.
Closes #2493
Jay Satiro (20 Apr 2018)
- openssl: Add support for OpenSSL 1.1.1 verbose-mode trace messages
- Support handling verbose-mode trace messages of type
SSL3_RT_INNER_CONTENT_TYPE, SSL3_MT_ENCRYPTED_EXTENSIONS,
SSL3_MT_END_OF_EARLY_DATA, SSL3_MT_KEY_UPDATE, SSL3_MT_NEXT_PROTO,
SSL3_MT_MESSAGE_HASH
Reported-by: iz8mbw@users.noreply.github.com
Fixes https://github.com/curl/curl/issues/2403
Daniel Stenberg (19 Apr 2018)
- ftplistparser: keep state between invokes
Regression from f786d1f14
Reported-by: wncboy on github
Fixes #2445
Closes #2508
- detect_proxy: only show proxy use if it had contents
- http2: handle on_begin_headers() called more than once
This triggered an assert if called more than once in debug mode (and a
memory leak if not debug build). With the right sequence of HTTP/2
headers incoming it can happen.
Detected by OSS-Fuzz
Closes #2507
Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7764
Jay Satiro (18 Apr 2018)
- [Dan McNulty brought this change]
schannel: add support for CURLOPT_CAINFO
- Move verify_certificate functionality in schannel.c into a new
file called schannel_verify.c. Additionally, some structure defintions
from schannel.c have been moved to schannel.h to allow them to be
used in schannel_verify.c.
- Make verify_certificate functionality for Schannel available on
all versions of Windows instead of just Windows CE. verify_certificate
will be invoked on Windows CE or when the user specifies
CURLOPT_CAINFO and CURLOPT_SSL_VERIFYPEER.
- In verify_certificate, create a custom certificate chain engine that
exclusively trusts the certificate store backed by the CURLOPT_CAINFO
file.
- doc updates of --cacert/CAINFO support for schannel
- Use CERT_NAME_SEARCH_ALL_NAMES_FLAG when invoking CertGetNameString
when available. This implements a TODO in schannel.c to improve
handling of multiple SANs in a certificate. In particular, all SANs
will now be searched instead of just the first name.
- Update tool_operate.c to not search for the curl-ca-bundle.crt file
when using Schannel to maintain backward compatibility. Previously,
any curl-ca-bundle.crt file found in that search would have been
ignored by Schannel. But, with CAINFO support, the file found by
that search would have been used as the certificate store and
could cause issues for any users that have curl-ca-bundle.crt in
the search path.
- Update url.c to not set the build time CURL_CA_BUNDLE if the selected
SSL backend is Schannel. We allow setting CA location for schannel
only when explicitly specified by the user via CURLOPT_CAINFO /
--cacert.
- Add new test cases 3000 and 3001. These test cases check that the first
and last SAN, respectively, matches the connection hostname. New test
certificates have been added for these cases. For 3000, the certificate
prefix is Server-localhost-firstSAN and for 3001, the certificate
prefix is Server-localhost-secondSAN.
- Remove TODO 15.2 (Add support for custom server certificate
validation), this commit addresses it.
Closes https://github.com/curl/curl/pull/1325
- schannel: fix warning
- Fix warning 'integer from pointer without a cast' on 3rd arg in
CertOpenStore. The arg type HCRYPTPROV may be a pointer or integer
type of the same size.
Follow-up to e35b025.
Caught by Marc's CI builds.
- [Jakub Wilk brought this change]
docs: fix typos
Closes https://github.com/curl/curl/pull/2503
Daniel Stenberg (17 Apr 2018)
- RELEASE-NOTES: synced
Jay Satiro (17 Apr 2018)
- [Kees Dekker brought this change]
winbuild: Support custom devel paths for each dependency
- Support custom devel paths for c-ares, mbedTLS, nghttp2, libSSH2,
OpenSSL and zlib. Respectively: CARES_PATH, MBEDTLS_PATH,
NGHTTP2_PATH, SSH2_PATH, SSL_PATH and ZLIB_PATH.
- Use lib.exe for making the static library instead of link.exe /lib.
The latter is undocumented and could cause problems as noted in the
comments.
- Remove a dangling URL that no longer worked. (I was not able to find
the IDN download at MSDN/microsoft.com, so it seems to be removed.)
- Remove custom override for release-ssh2-ssl-dll-zlib configuration.
Nobody knows why it was there and as far as we can see is unnecessary.
Closes https://github.com/curl/curl/pull/2474
Daniel Stenberg (17 Apr 2018)
- [Jess brought this change]
README.md: add backers and sponsors
Closes #2484
- [Archangel_SDY brought this change]
schannel: add client certificate authentication
Users can now specify a client certificate in system certificates store
explicitly using expression like `--cert "CurrentUser\MY\<thumbprint>"`
Closes #2376
Marcel Raad (16 Apr 2018)
- [toughengineer brought this change]
ntlm_sspi: fix authentication using Credential Manager
If you pass empty user/pass asking curl to use Windows Credential
Storage (as stated in the docs) and it has valid credentials for the
domain, e.g.
curl -v -u : --ntlm example.com
currently authentication fails.
This change fixes it by providing proper SPN string to the SSPI API
calls.
Fixes https://github.com/curl/curl/issues/1622
Closes https://github.com/curl/curl/pull/1660
Daniel Stenberg (16 Apr 2018)
- configure: keep LD_LIBRARY_PATH changes local
... only set it when we actually have to run tests to reduce its impact
on for example build commands etc.
Fixes #2490
Closes #2492
Reported-by: Dmitry Mikhirev
Marcel Raad (16 Apr 2018)
- urldata: make service names unconditional
The ifdefs have become quite long. Also, the condition for the
definition of CURLOPT_SERVICE_NAME and for setting it from
CURLOPT_SERVICE_NAME have diverged. We will soon also need the two
options for NTLM, at least when using SSPI, for
https://github.com/curl/curl/pull/1660.
Just make the definitions unconditional to make that easier.
Closes https://github.com/curl/curl/pull/2479
Daniel Stenberg (16 Apr 2018)
- test1148: tolerate progress updates better
Fixes #2446
Closes #2488
- [Christian Schmitz brought this change]
ssh: show libSSH2 error code when closing fails
Closes #2500
Jay Satiro (15 Apr 2018)
- [Daniel Gustafsson brought this change]
vauth: Fix typo
Address various spellings of "credentials".
Closes https://github.com/curl/curl/pull/2496
- [Dagobert Michelsen brought this change]
system.h: Add sparcv8plus to oracle/sunpro 32-bit detection
With specific compiler options selecting the arch like -xarch=sparc on
newer compilers like Oracle Studio 12.4 there is no definition of
__sparcv8 but __sparcv8plus which means the V9 ISA, but limited to the
32ÎíÎñbit subset defined by the V8plus ISA specification, without the
Visual Instruction Set (VIS), and without other implementation-specific
ISA extensions. So it should be the same as __sparcv8.
Closes https://github.com/curl/curl/pull/2491
- [Daniel Gustafsson brought this change]
checksrc: Fix typo
Fix typo in "semicolon" spelling and remove stray tab character.
Closes https://github.com/curl/curl/pull/2498
- [Daniel Gustafsson brought this change]
all: Refactor malloc+memset to use calloc
When a zeroed out allocation is required, use calloc() rather than
malloc() followed by an explicit memset(). The result will be the
same, but using calloc() everywhere increases consistency in the
codebase and avoids the risk of subtle bugs when code is injected
between malloc and memset by accident.
Closes https://github.com/curl/curl/pull/2497
Daniel Stenberg (12 Apr 2018)
- duphandle: make sure CURLOPT_RESOLVE is duplicated fine too
Verified in test 1502 now
Fixes #2485
Closes #2486
Reported-by: Ernst Sjöstrand
- mailmap: add a monnerat fixup [ci skip]
- proxy: show getenv proxy use in verbose output
... to aid debugging etc as it sometimes isn't immediately obvious why
curl uses or doesn't use a proxy.
Inspired by #2477
Closes #2480
- travis: build libpsl and make builds use it
closes #2471
- travis: bump to clang 6 and gcc 7
Extra-eye-on-this-by: Marcel Raad
Closes #2478
Marcel Raad (10 Apr 2018)
- travis: use trusty for coverage build
This works now and precise is in the process of being decommissioned.
Closes https://github.com/curl/curl/pull/2476
- lib: silence null-dereference warnings
In debug mode, MingGW-w64's GCC 7.3 issues null-dereference warnings
when dereferencing pointers after DEBUGASSERT-ing that they are not
NULL.
Fix this by removing the DEBUGASSERTs.
Suggested-by: Daniel Stenberg
Ref: https://github.com/curl/curl/pull/2463
- [Kees Dekker brought this change]
winbuild: fix URL
Follow up on https://github.com/curl/curl/pull/2472.
Now using en-us instead of nl-nl as language code in the URL.
Closes https://github.com/curl/curl/pull/2475
Daniel Stenberg (9 Apr 2018)
- [Kees Dekker brought this change]
winbuild: updated the documentation
The setenv command no longer exists and visual studio build prompts got
changed. Used Visual Studio 2015/2017 as reference.
Closes #2472
- test1136: fix cookie order after commit c990eadd1277
- build: cleanup to fix clang warnings/errors
unit1309 and vtls/gtls: error: arithmetic on a null pointer treated as a
cast from integer to pointer is a GNU extension
Reported-by: Rikard Falkeborn
Fixes #2466
Closes #2468
Jay Satiro (7 Apr 2018)
- examples/sftpuploadresmue: Fix Windows large file seek
- Use _fseeki64 instead of fseek (long) to seek curl_off_t in Windows.
- Use CURL_FORMAT_CURL_OFF_T specifier instead of %ld to print
curl_off_t.
Caught by Marc's CI builds.
Daniel Stenberg (7 Apr 2018)
- curl_setup: provide a CURL_SA_FAMILY_T type if none exists
... and use this type instead of 'sa_family_t' in the code since several
platforms don't have it.
Closes #2463
- [Eric Gallager brought this change]
build: add picky compiler warning flags for gcc 6 and 7
- configure: detect sa_family_t
Jay Satiro (7 Apr 2018)
- [Stefan Agner brought this change]
tool_operate: Fix retry on FTP 4xx to ignore other protocols
Only treat response code as FTP response codes in case the
protocol type is FTP.
This fixes an issue where an HTTP download was treated as FTP
in case libcurl returned with 33. This happens when the
download has already finished and the server responses 416:
HTTP/1.1 416 Requested Range Not Satisfiable
This should not be treated as an FTP error.
Fixes #2464
Closes #2465
Daniel Stenberg (6 Apr 2018)
- hash: calculate sizes with size_t instead of longs
... since they return size_t anyway!
closes #2462
- RELEASE-NOTES: synced
- [Jay Satiro brought this change]
build-openssl.bat: Refer to VS2017 as VC14.1 instead of VC15
.. and do the same for build-wolfssl.bat.
Because MS calls it VC14.1.
Closes https://github.com/curl/curl/pull/2189
- [Kees Dekker brought this change]
winbuild: make the clean target work without build-type
Due to the check in Makefile.vc and MakefileBuild.vc, no make call can
be invoked unless a build-type was specified. However, a clean target
only existed when a build type was specified. As a result, the clean
target was unreachable. Made clean target unconditional.
Closes #2455
- [patelvivekv1993 brought this change]
build-openssl.bat: allow custom paths for VS and perl
Fixes #2430
Closes #2457
- [Laurie Clark-Michalek brought this change]
FTP: allow PASV on IPv6 connections when a proxy is being used
In the situation of a client connecting to an FTP server using an IPv6
tunnel proxy, the connection info will indicate that the connection is
IPv6. However, because the server behing the proxy is IPv4, it is
permissable to attempt PSV mode. In the case of the FTP server being
IPv4 only, EPSV will always fail, and with the current logic curl will
be unable to connect to the server, as the IPv6 fwdproxy causes curl to
think that EPSV is impossible.
Closes #2432
- [Jon DeVree brought this change]
file: restore old behavior for file:////foo/bar URLs
curl 7.57.0 and up interpret this according to Appendix E.3.2 of RFC
8089 but then returns an error saying this is unimplemented. This is
actually a regression in behavior on both Windows and Unix.
Before curl 7.57.0 this URL was treated as a path of "//foo/bar" and
then passed to the relevant OS API. This means that the behavior of this
case is actually OS dependent.
The Unix path resolution rules say that the OS must handle swallowing
the extra "/" and so this path is the same as "/foo/bar"
The Windows path resolution rules say that this is a UNC path and
automatically handles the SMB access for the program. So curl on Windows
was already doing Appendix E.3.2 without any special code in curl.
Regression
Closes #2438
- [Gaurav Malhotra brought this change]
Revert "openssl: Don't add verify locations when verifypeer==0"
This reverts commit dc85437736e1fc90e689bb1f6c51c8f1aa9430eb.
libcurl (with the OpenSSL backend) performs server certificate verification
even if verifypeer == 0 and the verification result is available using
CURLINFO_SSL_VERIFYRESULT. The commit that is being reverted caused the
CURLINFO_SSL_VERIFYRESULT to not have useful information for the
verifypeer == 0 use case (it would always have
X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY).
Closes #2451
- [Wyatt O'Day brought this change]
tls: fix mbedTLS 2.7.0 build + handle sha256 failures
(mbedtls 2.70 compiled with MBEDTLS_DEPRECATED_REMOVED)
Closes #2453
- [Lauri Kasanen brought this change]
cookie: case-insensitive hashing for the domains
closes #2458
Patrick Monnerat (4 Apr 2018)
- cookie: fix and optimize 2nd top level domain name extraction
This fixes a segfault occurring when a name of the (invalid) form "domain..tld"
is processed.
test46 updated to cover this case.
Follow-up to commit c990ead.
Ref: https://github.com/curl/curl/pull/2440
Daniel Stenberg (4 Apr 2018)
- openssl: provide defines for argument typecasts to build warning-free
... as OpenSSL >= 1.1.0 and libressl >= 2.7.0 use different argument types.
- [Bernard Spil brought this change]
openssl: fix build with LibreSSL 2.7
- LibreSSL 2.7 implements (most of) OpenSSL 1.1 API
Fixes #2319
Closes #2447
Closes #2448
Signed-off-by: Bernard Spil <brnrd@FreeBSD.org>
- [Lauri Kasanen brought this change]
cookie: store cookies per top-level-domain-specific hash table
This makes libcurl handle thousands of cookies much better and speedier.
Closes #2440
- [Lauri Kasanen brought this change]
cookies: when reading from a file, only remove_expired once
This drops the cookie load time for 8k cookies from 178ms to 15ms.
Closes #2441
- test1148: set a fixed locale for the test
...as otherwise it might use a different decimal sign.
Bug: #2436
Reported-by: Oumph on github
Jay Satiro (31 Mar 2018)
- docs: fix CURLINFO_*_T examples use of CURL_FORMAT_CURL_OFF_T
- Put a percent sign before each CURL_FORMAT_CURL_OFF_T in printf.
For example "%" CURL_FORMAT_CURL_OFF_T becomes %lld or similar.
Bug: https://curl.haxx.se/mail/lib-2018-03/0140.html
Reported-by: David L.
Sergei Nikulov (27 Mar 2018)
- [Michał Janiszewski brought this change]
cmake: Add advapi32 as explicit link library for win32
ARM targets need advapi32 explicitly.
Closes #2363
Daniel Stenberg (27 Mar 2018)
- TODO: connection cache sharing is now supporte
Jay Satiro (26 Mar 2018)
- travis: enable apt retry on fail
This is a workaround for an unsolved travis issue that is causing CI
instances to sporadically fail due to 'unable to connect' issues during
apt stage.
Ref: https://github.com/travis-ci/travis-ci/issues/8507
Ref: https://github.com/travis-ci/travis-ci/issues/9112#issuecomment-376305909
Michael Kaufmann (26 Mar 2018)
- runtests.pl: fix warning 'use of uninitialized value'
follow-up to a9a7b60
Closes #2428
Daniel Stenberg (24 Mar 2018)
- gitignore: ignore more generated files
- threaded resolver: track resolver time and set suitable timeout values
In order to make curl_multi_timeout() return suitable "sleep" times even
when there's no socket to wait for while the name is being resolved in a
helper thread.
It will increases the timeouts as time passes.
Closes #2419
- [Howard Chu brought this change]
openldap: fix for NULL return from ldap_get_attribute_ber()
Closes #2399
GitHub (22 Mar 2018)
- [Sergei Nikulov brought this change]
travis-ci: enable -Werror for CMake builds (#2418)
- [Sergei Nikulov brought this change]
cmake: avoid warn-as-error during config checks (#2411)
- Move the CURL_WERROR option processing after the configuration checks
to avoid failures in case of warnings during the configuration checks.
This is a partial fix for #2358
- [Sergei Nikulov brought this change]
timeval: remove compilation warning by casting (#2417)
This is fixes #2358
Daniel Stenberg (22 Mar 2018)
- http2: read pending frames (including GOAWAY) in connection-check
If a connection has received a GOAWAY frame while not being used, the
function now reads frames off the connection before trying to reuse it
to avoid reusing connections the server has told us not to use.
Reported-by: Alex Baines
Fixes #1967
Closes #2402
- [Bas van Schaik brought this change]
CI: add lgtm.yml for tweaking lgtm.com analysis
Closes #2414
- CURLINFO_SSL_VERIFYRESULT.3: fix the example, add some text
Reported-by: Michal Trybus
Fixes #2400
- TODO: expand ~/ in config files
Closes #2317
- cookie.d: mention that "-" as filename means stdin
Reported-by: Dongliang Mu
Fixes #2410
- CURLINFO_COOKIELIST.3: made the example not leak memory
Reported-by: Muz Dima
- vauth/cleartext: fix integer overflow check
Make the integer overflow check not rely on the undefined behavior that
a size_t wraps around on overflow.
Detected by lgtm.com
Closes #2408
- lib/curl_path.h: add #ifdef header guard
Detected by lgtm.com
- vauth/ntlm.h: fix the #ifdef header guard
Detected by lgtm.com
Jay Satiro (20 Mar 2018)
- examples/hiperfifo: checksrc compliance
Daniel Stenberg (19 Mar 2018)
- [Nikos Tsipinakis brought this change]
parsedate: support UT timezone
RFC822 section 5.2 mentions Universal Time, 'UT', to be synonymous with
GMT.
Closes #2401
- RELEASE-NOTES: synced
- [Don brought this change]
cmake: add support for brotli
Currently CMake cannot detect Brotli support. This adds detection of the
libraries and associated header files. It also adds this to the
generated config.
Closes #2392
- [Chris Araman brought this change]
darwinssl: fix iOS build
Patrick Monnerat (18 Mar 2018)
- ILE/RPG binding: Add CURLOPT_HAPROXYPROTOCOL/Fix CURLOPT_DNS_SHUFFLE_ADDRESSES
Daniel Stenberg (17 Mar 2018)
- [Rick Deist brought this change]
resolve: add CURLOPT_DNS_SHUFFLE_ADDRESSES
This patch adds CURLOPT_DNS_SHUFFLE_ADDRESSES to explicitly request
shuffling of IP addresses returned for a hostname when there is more
than one. This is useful when the application knows that a round robin
approach is appropriate and is willing to accept the consequences of
potentially discarding some preference order returned by the system's
implementation.
Closes #1694
- add_handle/easy_perform: clear errorbuffer on start if set
To offer applications a more defined behavior, we clear the buffer as
early as possible.
Assisted-by: Jay Satiro
Fixes #2190
Closes #2377
- [Lawrence Matthews brought this change]
CURLOPT_HAPROXYPROTOCOL: support the HAProxy PROXY protocol
Add --haproxy-protocol for the command line tool
Closes #2162
- curl_version_info.3: fix ssl_version description
Reported-by: Vincas Razma
Fixes #2364
- multi: improved pending transfers handling => improved performance
When a transfer is requested to get done and it is put in the pending
queue when limited by number of connections, total or per-host, libcurl
would previously very aggressively retry *ALL* pending transfers to get
them transferring. That was very time consuming.
By reducing the aggressiveness in how pending are being retried, we
waste MUCH less time on putting transfers back into pending again.
Some test cases got a factor 30(!) speed improvement with this change.
Reported-by: Cyril B
Fixes #2369
Closes #2383
- pause: when changing pause state, update socket state
Especially unpausing a transfer might have to move the socket back to the
"currently used sockets" hash to get monitored. Otherwise it would never get
any more data and get stuck. Easily triggered with pausing using the
multi_socket API.
Reported-by: Philip Prindeville
Bug: https://curl.haxx.se/mail/lib-2018-03/0048.html
Fixes #2393
Closes #2391
- [Philip Prindeville brought this change]
examples/hiperfifo.c: improved
* use member struct event’s instead of pointers to alloc’d struct
events
* simplify the cases for the mcode_or_die() function via macros;
* make multi_timer_cb() actually do what the block comment says it
should;
* accept a “stop” command on the FIFO to shut down the service;
* use cleaner notation for unused variables than the (void) hack;
* allow following redirections (304’s);
- rate-limit: use three second window to better handle high speeds
Due to very frequent updates of the rate limit "window", it could
attempt to rate limit within the same milliseconds and that then made
the calculations wrong, leading to it not behaving correctly on very
fast transfers.
This new logic updates the rate limit "window" to be no shorter than the
last three seconds and only updating the timestamps for this when
switching between the states TOOFAST/PERFORM.
Reported-by: 刘佩东
Fixes #2386
Closes #2388
- [luz.paz brought this change]
cleanup: misc typos in strings and comments
Found via `codespell`
Closes #2389
- RELEASE-NOTES: toward 7.60.0
- [Kobi Gurkan brought this change]
http2: fixes typo
Closes #2387
- user-agent.d:: mention --proxy-header as well
Bug: https://github.com/curl/curl/issues/2381
- transfer: make HTTP without headers count correct body size
This is what "HTTP/0.9" basically looks like.
Reported on IRC
Closes #2382
- test1208: marked flaky
It fails somewhere between every 3rd to 10th travis-CI run
- SECURITY-PROCESS: mention how we write/add advisories
- [dasimx brought this change]
FTP: fix typo in recursive callback detection for seeking
Fixes #2380
Version 7.59.0 (13 Mar 2018)
Daniel Stenberg (13 Mar 2018)
- release: 7.59.0
Kamil Dudka (13 Mar 2018)
- tests/.../spnego.py: fix identifier typo
Detected by Coverity Analysis:
Error: IDENTIFIER_TYPO:
curl-7.58.0/tests/python_dependencies/impacket/spnego.py:229: identifier_typo: Using "SuportedMech" appears to be a typo:
* Identifier "SuportedMech" is only known to be referenced here, or in copies of this code.
* Identifier "SupportedMech" is referenced elsewhere at least 4 times.
curl-7.58.0/tests/python_dependencies/impacket/smbserver.py:2651: identifier_use: Example 1: Using identifier "SupportedMech".
curl-7.58.0/tests/python_dependencies/impacket/smbserver.py:2308: identifier_use: Example 2: Using identifier "SupportedMech".
curl-7.58.0/tests/python_dependencies/impacket/spnego.py:252: identifier_use: Example 3: Using identifier "SupportedMech" (2 total uses in this function).
curl-7.58.0/tests/python_dependencies/impacket/spnego.py:229: remediation: Should identifier "SuportedMech" be replaced by "SupportedMech"?
Closes #2379
Daniel Stenberg (13 Mar 2018)
- CURLOPT_COOKIEFILE.3: "-" as file name means stdin
Reported-by: Aron Bergman
Bug: https://curl.haxx.se/mail/lib-2018-03/0049.html
[ci skip]
- Revert "hostip: fix compiler warning: 'variable set but not used'"
This reverts commit a577059f92fc65bd6b81717f0737f897a5b34248.
The assignment really needs to be there or we risk working with an
uninitialized pointer.
Michael Kaufmann (12 Mar 2018)
- limit-rate: fix compiler warning
follow-up to 72a0f62
Viktor Szakats (12 Mar 2018)
- checksrc.pl: add -i and -m options
To sync it with changes made for the libssh2 project.
Also cleanup some whitespace.
- curl-openssl.m4: fix spelling [ci skip]
- FAQ: fix a broken URL [ci skip]
Daniel Stenberg (12 Mar 2018)
- http2: mark the connection for close on GOAWAY
... don't consider it an error!
Assisted-by: Jay Satiro
Reported-by: Łukasz Domeradzki
Fixes #2365
Closes #2375
- credits: Viktor prefers without accent
- openldap: white space changes, fixed up the copyright years
- openldap: check ldap_get_attribute_ber() results for NULL before using
CVE-2018-1000121
Reported-by: Dario Weisser
Bug: https://curl.haxx.se/docs/adv_2018-97a2.html
- FTP: reject path components with control codes
Refuse to operate when given path components featuring byte values lower
than 32.
Previously, inserting a %00 sequence early in the directory part when
using the 'singlecwd' ftp method could make curl write a zero byte
outside of the allocated buffer.
Test case 340 verifies.
CVE-2018-1000120
Reported-by: Duy Phan Thanh
Bug: https://curl.haxx.se/docs/adv_2018-9cd6.html
- readwrite: make sure excess reads don't go beyond buffer end
CVE-2018-1000122
Bug: https://curl.haxx.se/docs/adv_2018-b047.html
Detected by OSS-fuzz
- BUGS: updated link to security process
- limit-rate: kick in even before "limit" data has been received
... and make sure to avoid integer overflows with really large values.
Reported-by: 刘佩东
Fixes #2371
Closes #2373
- docs/SECURITY.md -> docs/SECURITY-PROCESS.md
- SECURITY.md: call it the security process
Michael Kaufmann (11 Mar 2018)
- Curl_range: fix FTP-only and FILE-only builds
follow-up to e04417d
- hostip: fix compiler warning: 'variable set but not used'
Daniel Stenberg (11 Mar 2018)
- HTTP: allow "header;" to replace an internal header with a blank one
Reported-by: Michael Kaufmann
Fixes #2357
Closes #2362
- http2: verbose output new MAX_CONCURRENT_STREAMS values
... as it is interesting for many users.
- SECURITY: distros' max embargo time is 14 days now
Patrick Monnerat (8 Mar 2018)
- curl tool: accept --compressed also if Brotli is enabled and zlib is not.
Daniel Stenberg (5 Mar 2018)
- THANKS + mailmap: remove duplicates, fixup full names
- [sergii.kavunenko brought this change]
WolfSSL: adding TLSv1.3
Closes #2349
usr/share/doc/alt-curlssl/FEATURES 0000644 00000013671 15116224031 0012635 0 ustar 00 _ _ ____ _
___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
FEATURES
curl tool
- config file support
- multiple URLs in a single command line
- range "globbing" support: [0-13], {one,two,three}
- multiple file upload on a single command line
- custom maximum transfer rate
- redirectable stderr
- metalink support (*13)
libcurl
- full URL syntax with no length limit
- custom maximum download time
- custom least download speed acceptable
- custom output result after completion
- guesses protocol from host name unless specified
- uses .netrc
- progress bar with time statistics while downloading
- "standard" proxy environment variables support
- compiles on win32 (reported builds on 40+ operating systems)
- selectable network interface for outgoing traffic
- IPv6 support on unix and Windows
- persistent connections
- socks 4 + 5 support, with or without local name resolving
- supports user name and password in proxy environment variables
- operations through proxy "tunnel" (using CONNECT)
- support for large files (>2GB and >4GB) during upload and download
- replaceable memory functions (malloc, free, realloc, etc)
- asynchronous name resolving (*6)
- both a push and a pull style interface
- international domain names (*11)
HTTP
- HTTP/1.1 compliant (optionally uses 1.0)
- GET
- PUT
- HEAD
- POST
- Pipelining
- multipart formpost (RFC1867-style)
- authentication: Basic, Digest, NTLM (*9) and Negotiate (SPNEGO) (*3)
to server and proxy
- resume (both GET and PUT)
- follow redirects
- maximum amount of redirects to follow
- custom HTTP request
- cookie get/send fully parsed
- reads/writes the netscape cookie file format
- custom headers (replace/remove internally generated headers)
- custom user-agent string
- custom referrer string
- range
- proxy authentication
- time conditions
- via http-proxy
- retrieve file modification date
- Content-Encoding support for deflate and gzip
- "Transfer-Encoding: chunked" support in uploads
- data compression (*12)
- HTTP/2 (*5)
HTTPS (*1)
- (all the HTTP features)
- using client certificates
- verify server certificate
- via http-proxy
- select desired encryption
- force usage of a specific SSL version (SSLv2 (*7), SSLv3 (*10) or TLSv1)
FTP
- download
- authentication
- Kerberos 5 (*14)
- active/passive using PORT, EPRT, PASV or EPSV
- single file size information (compare to HTTP HEAD)
- 'type=' URL support
- dir listing
- dir listing names-only
- upload
- upload append
- upload via http-proxy as HTTP PUT
- download resume
- upload resume
- custom ftp commands (before and/or after the transfer)
- simple "range" support
- via http-proxy
- all operations can be tunneled through a http-proxy
- customizable to retrieve file modification date
- no dir depth limit
FTPS (*1)
- implicit ftps:// support that use SSL on both connections
- explicit "AUTH TLS" and "AUTH SSL" usage to "upgrade" plain ftp://
connection to use SSL for both or one of the connections
SCP (*8)
- both password and public key auth
SFTP (*8)
- both password and public key auth
- with custom commands sent before/after the transfer
TFTP
- download
- upload
TELNET
- connection negotiation
- custom telnet options
- stdin/stdout I/O
LDAP (*2)
- full LDAP URL support
DICT
- extended DICT URL support
FILE
- URL support
- upload
- resume
SMB
- SMBv1 over TCP and SSL
- download
- upload
- authentication with NTLMv1
SMTP
- authentication: Plain, Login, CRAM-MD5, Digest-MD5, NTLM (*9), Kerberos 5
(*4) and External.
- send e-mails
- mail from support
- mail size support
- mail auth support for trusted server-to-server relaying
- multiple recipients
- via http-proxy
SMTPS (*1)
- implicit smtps:// support
- explicit "STARTTLS" usage to "upgrade" plain smtp:// connections to use SSL
- via http-proxy
POP3
- authentication: Clear Text, APOP and SASL
- SASL based authentication: Plain, Login, CRAM-MD5, Digest-MD5, NTLM (*9),
Kerberos 5 (*4) and External.
- list e-mails
- retrieve e-mails
- enhanced command support for: CAPA, DELE, TOP, STAT, UIDL and NOOP via
custom requests
- via http-proxy
POP3S (*1)
- implicit pop3s:// support
- explicit "STLS" usage to "upgrade" plain pop3:// connections to use SSL
- via http-proxy
IMAP
- authentication: Clear Text and SASL
- SASL based authentication: Plain, Login, CRAM-MD5, Digest-MD5, NTLM (*9),
Kerberos 5 (*4) and External.
- list the folders of a mailbox
- select a mailbox with support for verifying the UIDVALIDITY
- fetch e-mails with support for specifying the UID and SECTION
- upload e-mails via the append command
- enhanced command support for: EXAMINE, CREATE, DELETE, RENAME, STATUS,
STORE, COPY and UID via custom requests
- via http-proxy
IMAPS (*1)
- implicit imaps:// support
- explicit "STARTTLS" usage to "upgrade" plain imap:// connections to use SSL
- via http-proxy
FOOTNOTES
=========
*1 = requires a TLS library
*2 = requires OpenLDAP or WinLDAP
*3 = requires a GSS-API implementation (such as Heimdal or MIT Kerberos) or
SSPI (native Windows)
*4 = requires a GSS-API implementation, however, only Windows SSPI is
currently supported
*5 = requires nghttp2 and possibly a recent TLS library
*6 = requires c-ares
*7 = requires OpenSSL, NSS, GSKit, WinSSL or Secure Transport; GnuTLS, for
example, only supports SSLv3 and TLSv1
*8 = requires libssh2
*9 = requires OpenSSL, GnuTLS, mbedTLS, NSS, yassl, Secure Transport or SSPI
(native Windows)
*10 = requires an SSL library that supports SSLv3
*11 = requires libidn or Windows
*12 = requires libz
*13 = requires libmetalink, and either an Apple or Microsoft operating
system, or OpenSSL, or GnuTLS, or NSS
*14 = requires a GSS-API implementation (such as Heimdal or MIT Kerberos)
usr/share/doc/alt-curlssl/BUGS 0000644 00000030771 15116224036 0012164 0 ustar 00 _ _ ____ _
___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
BUGS
1. Bugs
1.1 There are still bugs
1.2 Where to report
1.3 Security bugs
1.4 What to report
1.5 libcurl problems
1.6 Who will fix the problems
1.7 How to get a stack trace
1.8 Bugs in libcurl bindings
1.9 Bugs in old versions
2. Bug fixing procedure
2.1 What happens on first filing
2.2 First response
2.3 Not reproducible
2.4 Unresponsive
2.5 Lack of time/interest
2.6 KNOWN_BUGS
2.7 TODO
2.8 Closing off stalled bugs
==============================================================================
1.1 There are still bugs
Curl and libcurl keep being developed. Adding features and changing code
means that bugs will sneak in, no matter how hard we try not to.
Of course there are lots of bugs left. And lots of misfeatures.
To help us make curl the stable and solid product we want it to be, we need
bug reports and bug fixes.
1.2 Where to report
If you can't fix a bug yourself and submit a fix for it, try to report an as
detailed report as possible to a curl mailing list to allow one of us to
have a go at a solution. You can optionally also post your bug/problem at
curl's bug tracking system over at
https://github.com/curl/curl/issues
Please read the rest of this document below first before doing that!
If you feel you need to ask around first, find a suitable mailing list and
post there. The lists are available on https://curl.haxx.se/mail/
1.3 Security bugs
If you find a bug or problem in curl or libcurl that you think has a
security impact, for example a bug that can put users in danger or make them
vulnerable if the bug becomes public knowledge, then please report that bug
using our security development process.
Security related bugs or bugs that are suspected to have a security impact,
should be reported by email to curl-security@haxx.se so that they first can
be dealt with away from the public to minimize the harm and impact it will
have on existing users out there who might be using the vulnerable versions.
The curl project's process for handling security related issues is
documented here:
https://curl.haxx.se/dev/secprocess.html
1.4 What to report
When reporting a bug, you should include all information that will help us
understand what's wrong, what you expected to happen and how to repeat the
bad behavior. You therefore need to tell us:
- your operating system's name and version number
- what version of curl you're using (curl -V is fine)
- versions of the used libraries that libcurl is built to use
- what URL you were working with (if possible), at least which protocol
and anything and everything else you think matters. Tell us what you
expected to happen, tell use what did happen, tell us how you could make it
work another way. Dig around, try out, test. Then include all the tiny bits
and pieces in your report. You will benefit from this yourself, as it will
enable us to help you quicker and more accurately.
Since curl deals with networks, it often helps us if you include a protocol
debug dump with your bug report. The output you get by using the -v or
--trace options.
If curl crashed, causing a core dump (in unix), there is hardly any use to
send that huge file to anyone of us. Unless we have an exact same system
setup as you, we can't do much with it. Instead we ask you to get a stack
trace and send that (much smaller) output to us instead!
The address and how to subscribe to the mailing lists are detailed in the
MANUAL file.
1.5 libcurl problems
When you've written your own application with libcurl to perform transfers,
it is even more important to be specific and detailed when reporting bugs.
Tell us the libcurl version and your operating system. Tell us the name and
version of all relevant sub-components like for example the SSL library
you're using and what name resolving your libcurl uses. If you use SFTP or
SCP, the libssh2 version is relevant etc.
Showing us a real source code example repeating your problem is the best way
to get our attention and it will greatly increase our chances to understand
your problem and to work on a fix (if we agree it truly is a problem).
Lots of problems that appear to be libcurl problems are actually just abuses
of the libcurl API or other malfunctions in your applications. It is advised
that you run your problematic program using a memory debug tool like
valgrind or similar before you post memory-related or "crashing" problems to
us.
1.6 Who will fix the problems
If the problems or bugs you describe are considered to be bugs, we want to
have the problems fixed.
There are no developers in the curl project that are paid to work on bugs.
All developers that take on reported bugs do this on a voluntary basis. We
do it out of an ambition to keep curl and libcurl excellent products and out
of pride.
But please do not assume that you can just lump over something to us and it
will then magically be fixed after some given time. Most often we need
feedback and help to understand what you've experienced and how to repeat a
problem. Then we may only be able to assist YOU to debug the problem and to
track down the proper fix.
We get reports from many people every month and each report can take a
considerable amount of time to really go to the bottom with.
1.7 How to get a stack trace
First, you must make sure that you compile all sources with -g and that you
don't 'strip' the final executable. Try to avoid optimizing the code as
well, remove -O, -O2 etc from the compiler options.
Run the program until it cores.
Run your debugger on the core file, like '<debugger> curl core'. <debugger>
should be replaced with the name of your debugger, in most cases that will
be 'gdb', but 'dbx' and others also occur.
When the debugger has finished loading the core file and presents you a
prompt, enter 'where' (without the quotes) and press return.
The list that is presented is the stack trace. If everything worked, it is
supposed to contain the chain of functions that were called when curl
crashed. Include the stack trace with your detailed bug report. It'll help a
lot.
1.8 Bugs in libcurl bindings
There will of course pop up bugs in libcurl bindings. You should then
primarily approach the team that works on that particular binding and see
what you can do to help them fix the problem.
If you suspect that the problem exists in the underlying libcurl, then
please convert your program over to plain C and follow the steps outlined
above.
1.9 Bugs in old versions
The curl project typically releases new versions every other month, and we
fix several hundred bugs per year. For a huge table of releases, number of
bug fixes and more, see: https://curl.haxx.se/docs/releases.html
The developers in the curl project do not have bandwidth or energy enough to
maintain several branches or to spend much time on hunting down problems in
old versions when chances are we already fixed them or at least that they've
changed nature and appearance in later versions.
When you experience a problem and want to report it, you really SHOULD
include the version number of the curl you're using when you experience the
issue. If that version number shows us that you're using an out-of-date
curl, you should also try out a modern curl version to see if the problem
persists or how/if it has changed in appearance.
Even if you cannot immediately upgrade your application/system to run the
latest curl version, you can most often at least run a test version or
experimental build or similar, to get this confirmed or not.
At times people insist that they cannot upgrade to a modern curl version,
but instead they "just want the bug fixed". That's fine, just don't count on
us spending many cycles on trying to identify which single commit, if that's
even possible, that at some point in the past fixed the problem you're now
experiencing.
Security wise, it is almost always a bad idea to lag behind the current curl
versions by a lot. We keeping discovering and reporting security problems
over time see you can see in this table:
https://curl.haxx.se/docs/vulnerabilities.html
2. Bug fixing procedure
2.1 What happens on first filing
When a new issue is posted in the issue tracker or on the mailing list, the
team of developers first need to see the report. Maybe they took the day
off, maybe they're off in the woods hunting. Have patience. Allow at least a
few days before expecting someone to have responded.
In the issue tracker you can expect that some labels will be set on the
issue to help categorize it.
2.2 First response
If your issue/bug report wasn't perfect at once (and few are), chances are
that someone will ask follow-up questions. Which version did you use? Which
options did you use? How often does the problem occur? How can we reproduce
this problem? Which protocols does it involve? Or perhaps much more specific
and deep diving questions. It all depends on your specific issue.
You should then respond to these follow-up questions and provide more info
about the problem, so that we can help you figure it out. Or maybe you can
help us figure it out. An active back-and-forth communication is important
and the key for finding a cure and landing a fix.
2.3 Not reproducible
For problems that we can't reproduce and can't understand even after having
gotten all the info we need and having studied the source code over again,
are really hard to solve so then we may require further work from you who
actually see or experience the problem.
2.4 Unresponsive
If the problem haven't been understood or reproduced, and there's nobody
responding to follow-up questions or questions asking for clarifications or
for discussing possible ways to move forward with the task, we take that as
a strong suggestion that the bug is not important.
Unimportant issues will be closed as inactive sooner or later as they can't
be fixed. The inactivity period (waiting for responses) should not be
shorter than two weeks but may extend months.
2.5 Lack of time/interest
Bugs that are filed and are understood can unfortunately end up in the
"nobody cares enough about it to work on it" category. Such bugs are
perfectly valid problems that *should* get fixed but apparently aren't. We
try to mark such bugs as "KNOWN_BUGS material" after a time of inactivity
and if no activity is noticed after yet some time those bugs are added to
KNOWN_BUGS and are closed in the issue tracker.
2.6 KNOWN_BUGS
This is a list of known bugs. Bugs we know exist and that have been pointed
out but that haven't yet been fixed. The reasons for why they haven't been
fixed can involve anything really, but the primary reason is that nobody has
considered these problems to be important enough to spend the necessary time
and effort to have them fixed.
The KNOWN_BUGS are always up for grabs and we will always love the ones who
bring one of them back to live and offers solutions to them.
The KNOWN_BUGS document has a sibling document known as TODO.
2.7 TODO
Issues that are filed or reported that aren't really bugs but more missing
features or ideas for future improvements and so on are marked as
'enhancement' or 'feature-request' and will be added to the TODO document
instead and the issue is closed. We don't keep TODO items in the issue
tracker.
The TODO document is full of ideas and suggestions of what we can add or fix
one day. You're always encouraged and free to grab one of those items and
take up a discussion with the curl development team on how that could be
implemented or provided in the project so that you can work on ticking it
odd that document.
If the issue is rather a bug and not a missing feature or functionality, it
is listed in KNOWN_BUGS instead.
2.8 Closing off stalled bugs
The issue and pull request trackers on https://github.com/curl/curl will
only hold "active" entries (using a non-precise definition of what active
actually is, but they're at least not completely dead). Those that are
abandoned or in other ways dormant will be closed and sometimes added to
TODO and KNOWN_BUGS instead.
This way, we only have "active" issues open on github. Irrelevant issues and
pull requests will not distract developers or casual visitors.
usr/share/doc/alt-curlssl/TODO 0000644 00000135657 15116224044 0012201 0 ustar 00 _ _ ____ _
___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
Things that could be nice to do in the future
Things to do in project curl. Please tell us what you think, contribute and
send us patches that improve things!
Be aware that these are things that we could do, or have once been considered
things we could do. If you want to work on any of these areas, please
consider bringing it up for discussions first on the mailing list so that we
all agree it is still a good idea for the project!
All bugs documented in the KNOWN_BUGS document are subject for fixing!
1. libcurl
1.2 More data sharing
1.3 struct lifreq
1.4 signal-based resolver timeouts
1.5 get rid of PATH_MAX
1.6 Modified buffer size approach
1.7 Support HTTP/2 for HTTP(S) proxies
1.8 CURLOPT_RESOLVE for any port number
1.9 Cache negative name resolves
1.10 auto-detect proxy
1.11 minimize dependencies with dynamically loaded modules
1.12 updated DNS server while running
1.13 c-ares and CURLOPT_OPENSOCKETFUNCTION
1.14 Typesafe curl_easy_setopt()
1.15 Monitor connections in the connection pool
1.16 Try to URL encode given URL
1.17 Add support for IRIs
1.18 try next proxy if one doesn't work
1.19 Timeout idle connections from the pool
1.20 SRV and URI DNS records
1.21 Have the URL API offer IDN decoding
1.22 CURLINFO_PAUSE_STATE
1.23 Offer API to flush the connection pool
1.24 TCP Fast Open for windows
1.25 Expose tried IP addresses that failed
1.26 CURL_REFUSE_CLEARTEXT
1.27 hardcode the "localhost" addresses
1.28 FD_CLOEXEC
2. libcurl - multi interface
2.1 More non-blocking
2.2 Better support for same name resolves
2.3 Non-blocking curl_multi_remove_handle()
2.4 Split connect and authentication process
2.5 Edge-triggered sockets should work
2.6 multi upkeep
3. Documentation
3.2 Provide cmake config-file
4. FTP
4.1 HOST
4.2 Alter passive/active on failure and retry
4.3 Earlier bad letter detection
4.4 REST for large files
4.5 ASCII support
4.6 GSSAPI via Windows SSPI
4.7 STAT for LIST without data connection
4.8 Option to ignore private IP addresses in PASV response
5. HTTP
5.1 Better persistency for HTTP 1.0
5.2 support FF3 sqlite cookie files
5.3 Rearrange request header order
5.5 auth= in URLs
5.6 Refuse "downgrade" redirects
5.7 QUIC
6. TELNET
6.1 ditch stdin
6.2 ditch telnet-specific select
6.3 feature negotiation debug data
7. SMTP
7.1 Pipelining
7.2 Enhanced capability support
7.3 Add CURLOPT_MAIL_CLIENT option
8. POP3
8.1 Pipelining
8.2 Enhanced capability support
9. IMAP
9.1 Enhanced capability support
10. LDAP
10.1 SASL based authentication mechanisms
11. SMB
11.1 File listing support
11.2 Honor file timestamps
11.3 Use NTLMv2
11.4 Create remote directories
12. New protocols
12.1 RSYNC
13. SSL
13.1 Disable specific versions
13.2 Provide mutex locking API
13.3 Support in-memory certs/ca certs/keys
13.4 Cache/share OpenSSL contexts
13.5 Export session ids
13.6 Provide callback for cert verification
13.7 improve configure --with-ssl
13.8 Support DANE
13.9 Configurable loading of OpenSSL configuration file
13.10 Support Authority Information Access certificate extension (AIA)
13.11 Support intermediate & root pinning for PINNEDPUBLICKEY
13.12 Support HSTS
13.13 Support HPKP
13.14 Support the clienthello extension
14. GnuTLS
14.1 SSL engine stuff
14.2 check connection
15. WinSSL/SChannel
15.1 Add support for client certificate authentication
15.3 Add support for the --ciphers option
15.4 Add option to disable client certificate auto-send
16. SASL
16.1 Other authentication mechanisms
16.2 Add QOP support to GSSAPI authentication
16.3 Support binary messages (i.e.: non-base64)
17. SSH protocols
17.1 Multiplexing
17.2 SFTP performance
17.3 Support better than MD5 hostkey hash
17.4 Support CURLOPT_PREQUOTE
18. Command line tool
18.1 sync
18.2 glob posts
18.3 prevent file overwriting
18.4 simultaneous parallel transfers
18.5 UTF-8 filenames in Content-Disposition
18.6 warning when setting an option
18.7 warning if curl version is not in sync with libcurl version
18.8 offer color-coded HTTP header output
18.9 Choose the name of file in braces for complex URLs
18.10 improve how curl works in a windows console window
18.11 Windows: set attribute 'archive' for completed downloads
18.12 keep running, read instructions from pipe/socket
18.13 support metalink in http headers
18.14 --fail without --location should treat 3xx as a failure
18.15 --retry should resume
18.16 send only part of --data
18.17 consider file name from the redirected URL with -O ?
18.18 retry on network is unreachable
18.19 expand ~/ in config files
18.20 host name sections in config files
19. Build
19.1 roffit
19.2 Enable PIE and RELRO by default
20. Test suite
20.1 SSL tunnel
20.2 nicer lacking perl message
20.3 more protocols supported
20.4 more platforms supported
20.5 Add support for concurrent connections
20.6 Use the RFC6265 test suite
20.7 Support LD_PRELOAD on macOS
21. Next SONAME bump
21.1 http-style HEAD output for FTP
21.2 combine error codes
21.3 extend CURLOPT_SOCKOPTFUNCTION prototype
22. Next major release
22.1 cleanup return codes
22.2 remove obsolete defines
22.3 size_t
22.4 remove several functions
22.5 remove CURLOPT_FAILONERROR
22.7 remove progress meter from libcurl
22.8 remove 'curl_httppost' from public
==============================================================================
1. libcurl
1.2 More data sharing
curl_share_* functions already exist and work, and they can be extended to
share more. For example, enable sharing of the ares channel.
1.3 struct lifreq
Use 'struct lifreq' and SIOCGLIFADDR instead of 'struct ifreq' and
SIOCGIFADDR on newer Solaris versions as they claim the latter is obsolete.
To support IPv6 interface addresses for network interfaces properly.
1.4 signal-based resolver timeouts
libcurl built without an asynchronous resolver library uses alarm() to time
out DNS lookups. When a timeout occurs, this causes libcurl to jump from the
signal handler back into the library with a sigsetjmp, which effectively
causes libcurl to continue running within the signal handler. This is
non-portable and could cause problems on some platforms. A discussion on the
problem is available at https://curl.haxx.se/mail/lib-2008-09/0197.html
Also, alarm() provides timeout resolution only to the nearest second. alarm
ought to be replaced by setitimer on systems that support it.
1.5 get rid of PATH_MAX
Having code use and rely on PATH_MAX is not nice:
https://insanecoding.blogspot.com/2007/11/pathmax-simply-isnt.html
Currently the SSH based code uses it a bit, but to remove PATH_MAX from there
we need libssh2 to properly tell us when we pass in a too small buffer and
its current API (as of libssh2 1.2.7) doesn't.
1.6 Modified buffer size approach
Current libcurl allocates a fixed 16K size buffer for download and an
additional 16K for upload. They are always unconditionally part of the easy
handle. If CRLF translations are requested, an additional 32K "scratch
buffer" is allocated. A total of 64K transfer buffers in the worst case.
First, while the handles are not actually in use these buffers could be freed
so that lingering handles just kept in queues or whatever waste less memory.
Secondly, SFTP is a protocol that needs to handle many ~30K blocks at once
since each need to be individually acked and therefore libssh2 must be
allowed to send (or receive) many separate ones in parallel to achieve high
transfer speeds. A current libcurl build with a 16K buffer makes that
impossible, but one with a 512K buffer will reach MUCH faster transfers. But
allocating 512K unconditionally for all buffers just in case they would like
to do fast SFTP transfers at some point is not a good solution either.
Dynamically allocate buffer size depending on protocol in use in combination
with freeing it after each individual transfer? Other suggestions?
1.7 Support HTTP/2 for HTTP(S) proxies
Support for doing HTTP/2 to HTTP and HTTPS proxies is still missing.
1.8 CURLOPT_RESOLVE for any port number
This option allows applications to set a replacement IP address for a given
host + port pair. Consider making support for providing a replacement address
for the host name on all port numbers.
See https://github.com/curl/curl/issues/1264
1.9 Cache negative name resolves
A name resolve that has failed is likely to fail when made again within a
short period of time. Currently we only cache positive responses.
1.10 auto-detect proxy
libcurl could be made to detect the system proxy setup automatically and use
that. On Windows, macOS and Linux desktops for example.
The pull-request to use libproxy for this was deferred due to doubts on the
reliability of the dependency and how to use it:
https://github.com/curl/curl/pull/977
libdetectproxy is a (C++) library for detecting the proxy on Windows
https://github.com/paulharris/libdetectproxy
1.11 minimize dependencies with dynamically loaded modules
We can create a system with loadable modules/plug-ins, where these modules
would be the ones that link to 3rd party libs. That would allow us to avoid
having to load ALL dependencies since only the necessary ones for this
app/invoke/used protocols would be necessary to load. See
https://github.com/curl/curl/issues/349
1.12 updated DNS server while running
If /etc/resolv.conf gets updated while a program using libcurl is running, it
is may cause name resolves to fail unless res_init() is called. We should
consider calling res_init() + retry once unconditionally on all name resolve
failures to mitigate against this. Firefox works like that. Note that Windows
doesn't have res_init() or an alternative.
https://github.com/curl/curl/issues/2251
1.13 c-ares and CURLOPT_OPENSOCKETFUNCTION
curl will create most sockets via the CURLOPT_OPENSOCKETFUNCTION callback and
close them with the CURLOPT_CLOSESOCKETFUNCTION callback. However, c-ares
does not use those functions and instead opens and closes the sockets
itself. This means that when curl passes the c-ares socket to the
CURLMOPT_SOCKETFUNCTION it isn't owned by the application like other sockets.
See https://github.com/curl/curl/issues/2734
1.14 Typesafe curl_easy_setopt()
One of the most common problems in libcurl using applications is the lack of
type checks for curl_easy_setopt() which happens because it accepts varargs
and thus can take any type.
One possible solution to this is to introduce a few different versions of the
setopt version for the different kinds of data you can set.
curl_easy_set_num() - sets a long value
curl_easy_set_large() - sets a curl_off_t value
curl_easy_set_ptr() - sets a pointer
curl_easy_set_cb() - sets a callback PLUS its callback data
1.15 Monitor connections in the connection pool
libcurl's connection cache or pool holds a number of open connections for the
purpose of possible subsequent connection reuse. It may contain a few up to a
significant amount of connections. Currently, libcurl leaves all connections
as they are and first when a connection is iterated over for matching or
reuse purpose it is verified that it is still alive.
Those connections may get closed by the server side for idleness or they may
get a HTTP/2 ping from the peer to verify that they're still alive. By adding
monitoring of the connections while in the pool, libcurl can detect dead
connections (and close them) better and earlier, and it can handle HTTP/2
pings to keep such ones alive even when not actively doing transfers on them.
1.16 Try to URL encode given URL
Given a URL that for example contains spaces, libcurl could have an option
that would try somewhat harder than it does now and convert spaces to %20 and
perhaps URL encoded byte values over 128 etc (basically do what the redirect
following code already does).
https://github.com/curl/curl/issues/514
1.17 Add support for IRIs
IRIs (RFC 3987) allow localized, non-ascii, names in the URL. To properly
support this, curl/libcurl would need to translate/encode the given input
from the input string encoding into percent encoded output "over the wire".
To make that work smoothly for curl users even on Windows, curl would
probably need to be able to convert from several input encodings.
1.18 try next proxy if one doesn't work
Allow an application to specify a list of proxies to try, and failing to
connect to the first go on and try the next instead until the list is
exhausted. Browsers support this feature at least when they specify proxies
using PACs.
https://github.com/curl/curl/issues/896
1.19 Timeout idle connections from the pool
libcurl currently keeps connections in its connection pool for an indefinite
period of time, until it either gets reused, gets noticed that it has been
closed by the server or gets pruned to make room for a new connection.
To reduce overhead (especially for when we add monitoring of the connections
in the pool), we should introduce a timeout so that connections that have
been idle for N seconds get closed.
1.20 SRV and URI DNS records
Offer support for resolving SRV and URI DNS records for libcurl to know which
server to connect to for various protocols (including HTTP!).
1.21 Have the URL API offer IDN decoding
Similar to how URL decoding/encoding is done, we could have URL functions to
convert IDN host names to punycode (probably not the reverse).
https://github.com/curl/curl/issues/3232
1.22 CURLINFO_PAUSE_STATE
Return information about the transfer's current pause state, in both
directions. https://github.com/curl/curl/issues/2588
1.23 Offer API to flush the connection pool
Sometimes applications want to flush all the existing connections kept alive.
An API could allow a forced flush or just a forced loop that would properly
close all connections that have been closed by the server already.
1.24 TCP Fast Open for windows
libcurl supports the CURLOPT_TCP_FASTOPEN option since 7.49.0 for Linux and
Mac OS. Windows supports TCP Fast Open starting with Windows 10, version 1607
and we should add support for it.
1.25 Expose tried IP addresses that failed
When libcurl fails to connect to a host, it should be able to offer the
application the list of IP addresses that were used in the attempt.
https://github.com/curl/curl/issues/2126
1.26 CURL_REFUSE_CLEARTEXT
An environment variable that when set will make libcurl refuse to use any
cleartext network protocol. That's all non-encrypted ones (FTP, HTTP, Gopher,
etc). By adding the check to libcurl and not just curl, this environment
variable can then help users to block all libcurl-using programs from
accessing the network using unsafe protocols.
The variable could be given some sort of syntax or different levels and be
used to also allow for example users to refuse libcurl to do transfers with
HTTPS certificate checks disabled.
It could also automatically refuse usernames in URLs when set
(see CURLOPT_DISALLOW_USERNAME_IN_URL)
1.27 hardcode the "localhost" addresses
There's this new spec getting adopted that says "localhost" should always and
unconditionally be a local address and not get resolved by a DNS server. A
fine way for curl to fix this would be to simply hard-code the response to
127.0.0.1 and/or ::1 (depending on what IP versions that are requested). This
is what the browsers probably will do with this hostname.
https://bugzilla.mozilla.org/show_bug.cgi?id=1220810
https://tools.ietf.org/html/draft-ietf-dnsop-let-localhost-be-localhost-02
1.28 FD_CLOEXEC
It sets the close-on-exec flag for the file descriptor, which causes the file
descriptor to be automatically (and atomically) closed when any of the
exec-family functions succeed. Should probably be set by default?
https://github.com/curl/curl/issues/2252
2. libcurl - multi interface
2.1 More non-blocking
Make sure we don't ever loop because of non-blocking sockets returning
EWOULDBLOCK or similar. Blocking cases include:
- Name resolves on non-windows unless c-ares or the threaded resolver is used
- SOCKS proxy handshakes
- file:// transfers
- TELNET transfers
- The "DONE" operation (post transfer protocol-specific actions) for the
protocols SFTP, SMTP, FTP. Fixing Curl_done() for this is a worthy task.
2.2 Better support for same name resolves
If a name resolve has been initiated for name NN and a second easy handle
wants to resolve that name as well, make it wait for the first resolve to end
up in the cache instead of doing a second separate resolve. This is
especially needed when adding many simultaneous handles using the same host
name when the DNS resolver can get flooded.
2.3 Non-blocking curl_multi_remove_handle()
The multi interface has a few API calls that assume a blocking behavior, like
add_handle() and remove_handle() which limits what we can do internally. The
multi API need to be moved even more into a single function that "drives"
everything in a non-blocking manner and signals when something is done. A
remove or add would then only ask for the action to get started and then
multi_perform() etc still be called until the add/remove is completed.
2.4 Split connect and authentication process
The multi interface treats the authentication process as part of the connect
phase. As such any failures during authentication won't trigger the relevant
QUIT or LOGOFF for protocols such as IMAP, POP3 and SMTP.
2.5 Edge-triggered sockets should work
The multi_socket API should work with edge-triggered socket events. One of
the internal actions that need to be improved for this to work perfectly is
the 'maxloops' handling in transfer.c:readwrite_data().
2.6 multi upkeep
In libcurl 7.62.0 we introduced curl_easy_upkeep. It unfortunately only works
on easy handles. We should introduces a version of that for the multi handle,
and also consider doing "upkeep" automatically on connections in the
connection pool when the multi handle is in used.
See https://github.com/curl/curl/issues/3199
3. Documentation
3.2 Provide cmake config-file
A config-file package is a set of files provided by us to allow applications
to write cmake scripts to find and use libcurl easier. See
https://github.com/curl/curl/issues/885
4. FTP
4.1 HOST
HOST is a command for a client to tell which host name to use, to offer FTP
servers named-based virtual hosting:
https://tools.ietf.org/html/rfc7151
4.2 Alter passive/active on failure and retry
When trying to connect passively to a server which only supports active
connections, libcurl returns CURLE_FTP_WEIRD_PASV_REPLY and closes the
connection. There could be a way to fallback to an active connection (and
vice versa). https://curl.haxx.se/bug/feature.cgi?id=1754793
4.3 Earlier bad letter detection
Make the detection of (bad) %0d and %0a codes in FTP URL parts earlier in the
process to avoid doing a resolve and connect in vain.
4.4 REST for large files
REST fix for servers not behaving well on >2GB requests. This should fail if
the server doesn't set the pointer to the requested index. The tricky
(impossible?) part is to figure out if the server did the right thing or not.
4.5 ASCII support
FTP ASCII transfers do not follow RFC959. They don't convert the data
accordingly.
4.6 GSSAPI via Windows SSPI
In addition to currently supporting the SASL GSSAPI mechanism (Kerberos V5)
via third-party GSS-API libraries, such as Heimdal or MIT Kerberos, also add
support for GSSAPI authentication via Windows SSPI.
4.7 STAT for LIST without data connection
Some FTP servers allow STAT for listing directories instead of using LIST,
and the response is then sent over the control connection instead of as the
otherwise usedw data connection: https://www.nsftools.com/tips/RawFTP.htm#STAT
This is not detailed in any FTP specification.
4.8 Option to ignore private IP addresses in PASV response
Some servers respond with and some other FTP client implementations can
ignore private (RFC 1918 style) IP addresses when received in PASV responses.
To consider for libcurl as well. See https://github.com/curl/curl/issues/1455
5. HTTP
5.1 Better persistency for HTTP 1.0
"Better" support for persistent connections over HTTP 1.0
https://curl.haxx.se/bug/feature.cgi?id=1089001
5.2 support FF3 sqlite cookie files
Firefox 3 is changing from its former format to a a sqlite database instead.
We should consider how (lib)curl can/should support this.
https://curl.haxx.se/bug/feature.cgi?id=1871388
5.3 Rearrange request header order
Server implementors often make an effort to detect browser and to reject
clients it can detect to not match. One of the last details we cannot yet
control in libcurl's HTTP requests, which also can be exploited to detect
that libcurl is in fact used even when it tries to impersonate a browser, is
the order of the request headers. I propose that we introduce a new option in
which you give headers a value, and then when the HTTP request is built it
sorts the headers based on that number. We could then have internally created
headers use a default value so only headers that need to be moved have to be
specified.
5.5 auth= in URLs
Add the ability to specify the preferred authentication mechanism to use by
using ;auth=<mech> in the login part of the URL.
For example:
http://test:pass;auth=NTLM@example.com would be equivalent to specifying --user
test:pass;auth=NTLM or --user test:pass --ntlm from the command line.
Additionally this should be implemented for proxy base URLs as well.
5.6 Refuse "downgrade" redirects
See https://github.com/curl/curl/issues/226
Consider a way to tell curl to refuse to "downgrade" protocol with a redirect
and/or possibly a bit that refuses redirect to change protocol completely.
5.7 QUIC
The standardization process of QUIC has been taken to the IETF and can be
followed on the [IETF QUIC Mailing
list](https://www.ietf.org/mailman/listinfo/quic). I'd like us to get on the
bandwagon. Ideally, this would be done with a separate library/project to
handle the binary/framing layer in a similar fashion to how HTTP/2 is
implemented. This, to allow other projects to benefit from the work and to
thus broaden the interest and chance of others to participate.
6. TELNET
6.1 ditch stdin
Reading input (to send to the remote server) on stdin is a crappy solution for
library purposes. We need to invent a good way for the application to be able
to provide the data to send.
6.2 ditch telnet-specific select
Move the telnet support's network select() loop go away and merge the code
into the main transfer loop. Until this is done, the multi interface won't
work for telnet.
6.3 feature negotiation debug data
Add telnet feature negotiation data to the debug callback as header data.
7. SMTP
7.1 Pipelining
Add support for pipelining emails.
7.2 Enhanced capability support
Add the ability, for an application that uses libcurl, to obtain the list of
capabilities returned from the EHLO command.
7.3 Add CURLOPT_MAIL_CLIENT option
Rather than use the URL to specify the mail client string to present in the
HELO and EHLO commands, libcurl should support a new CURLOPT specifically for
specifying this data as the URL is non-standard and to be honest a bit of a
hack ;-)
Please see the following thread for more information:
https://curl.haxx.se/mail/lib-2012-05/0178.html
8. POP3
8.1 Pipelining
Add support for pipelining commands.
8.2 Enhanced capability support
Add the ability, for an application that uses libcurl, to obtain the list of
capabilities returned from the CAPA command.
9. IMAP
9.1 Enhanced capability support
Add the ability, for an application that uses libcurl, to obtain the list of
capabilities returned from the CAPABILITY command.
10. LDAP
10.1 SASL based authentication mechanisms
Currently the LDAP module only supports ldap_simple_bind_s() in order to bind
to an LDAP server. However, this function sends username and password details
using the simple authentication mechanism (as clear text). However, it should
be possible to use ldap_bind_s() instead specifying the security context
information ourselves.
11. SMB
11.1 File listing support
Add support for listing the contents of a SMB share. The output should probably
be the same as/similar to FTP.
11.2 Honor file timestamps
The timestamp of the transferred file should reflect that of the original file.
11.3 Use NTLMv2
Currently the SMB authentication uses NTLMv1.
11.4 Create remote directories
Support for creating remote directories when uploading a file to a directory
that doesn't exist on the server, just like --ftp-create-dirs.
12. New protocols
12.1 RSYNC
There's no RFC for the protocol or an URI/URL format. An implementation
should most probably use an existing rsync library, such as librsync.
13. SSL
13.1 Disable specific versions
Provide an option that allows for disabling specific SSL versions, such as
SSLv2 https://curl.haxx.se/bug/feature.cgi?id=1767276
13.2 Provide mutex locking API
Provide a libcurl API for setting mutex callbacks in the underlying SSL
library, so that the same application code can use mutex-locking
independently of OpenSSL or GnutTLS being used.
13.3 Support in-memory certs/ca certs/keys
You can specify the private and public keys for SSH/SSL as file paths. Some
programs want to avoid using files and instead just pass them as in-memory
data blobs. There's probably a challenge to make this work across the
plethory of different TLS and SSH backends that curl supports.
https://github.com/curl/curl/issues/2310
13.4 Cache/share OpenSSL contexts
"Look at SSL cafile - quick traces look to me like these are done on every
request as well, when they should only be necessary once per SSL context (or
once per handle)". The major improvement we can rather easily do is to make
sure we don't create and kill a new SSL "context" for every request, but
instead make one for every connection and re-use that SSL context in the same
style connections are re-used. It will make us use slightly more memory but
it will libcurl do less creations and deletions of SSL contexts.
Technically, the "caching" is probably best implemented by getting added to
the share interface so that easy handles who want to and can reuse the
context specify that by sharing with the right properties set.
https://github.com/curl/curl/issues/1110
13.5 Export session ids
Add an interface to libcurl that enables "session IDs" to get
exported/imported. Cris Bailiff said: "OpenSSL has functions which can
serialise the current SSL state to a buffer of your choice, and recover/reset
the state from such a buffer at a later date - this is used by mod_ssl for
apache to implement and SSL session ID cache".
13.6 Provide callback for cert verification
OpenSSL supports a callback for customised verification of the peer
certificate, but this doesn't seem to be exposed in the libcurl APIs. Could
it be? There's so much that could be done if it were!
13.7 improve configure --with-ssl
make the configure --with-ssl option first check for OpenSSL, then GnuTLS,
then NSS...
13.8 Support DANE
DNS-Based Authentication of Named Entities (DANE) is a way to provide SSL
keys and certs over DNS using DNSSEC as an alternative to the CA model.
https://www.rfc-editor.org/rfc/rfc6698.txt
An initial patch was posted by Suresh Krishnaswamy on March 7th 2013
(https://curl.haxx.se/mail/lib-2013-03/0075.html) but it was a too simple
approach. See Daniel's comments:
https://curl.haxx.se/mail/lib-2013-03/0103.html . libunbound may be the
correct library to base this development on.
Björn Stenberg wrote a separate initial take on DANE that was never
completed.
13.9 Configurable loading of OpenSSL configuration file
libcurl calls the OpenSSL function CONF_modules_load_file() in openssl.c,
Curl_ossl_init(). "We regard any changes in the OpenSSL configuration as a
security risk or at least as unnecessary."
Please add a configuration switch or something similar to disable the
CONF_modules_load_file() call.
See https://github.com/curl/curl/issues/2724
13.10 Support Authority Information Access certificate extension (AIA)
AIA can provide various things like CRLs but more importantly information
about intermediate CA certificates that can allow validation path to be
fulfilled when the HTTPS server doesn't itself provide them.
Since AIA is about downloading certs on demand to complete a TLS handshake,
it is probably a bit tricky to get done right.
See https://github.com/curl/curl/issues/2793
13.11 Support intermediate & root pinning for PINNEDPUBLICKEY
CURLOPT_PINNEDPUBLICKEY does not consider the hashes of intermediate & root
certificates when comparing the pinned keys. Therefore it is not compatible
with "HTTP Public Key Pinning" as there also intermediate and root certificates
can be pinned. This is very useful as it prevents webadmins from "locking
themself out of their servers".
Adding this feature would make curls pinning 100% compatible to HPKP and allow
more flexible pinning.
13.12 Support HSTS
"HTTP Strict Transport Security" is TOFU (trust on first use), time-based
features indicated by a HTTP header send by the webserver. It is widely used
in browsers and it's purpose is to prevent insecure HTTP connections after
a previous HTTPS connection. It protects against SSLStripping attacks.
Doc: https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security
RFC 6797: https://tools.ietf.org/html/rfc6797
13.13 Support HPKP
"HTTP Public Key Pinning" is TOFU (trust on first use), time-based
features indicated by a HTTP header send by the webserver. It's purpose is
to prevent Man-in-the-middle attacks by trusted CAs by allowing webadmins
to specify which CAs/certificates/public keys to trust when connection to
their websites.
It can be build based on PINNEDPUBLICKEY.
Wikipedia: https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning
OWASP: https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning
Doc: https://developer.mozilla.org/de/docs/Web/Security/Public_Key_Pinning
RFC: https://tools.ietf.org/html/draft-ietf-websec-key-pinning-21
13.14 Support the clienthello extension
Certain stupid networks and middle boxes have a problem with SSL handshake
pakets that are within a certain size range because how that sets some bits
that previously (in older TLS version) were not set. The clienthello
extension adds padding to avoid that size range.
https://tools.ietf.org/html/rfc7685
https://github.com/curl/curl/issues/2299
14. GnuTLS
14.1 SSL engine stuff
Is this even possible?
14.2 check connection
Add a way to check if the connection seems to be alive, to correspond to the
SSL_peak() way we use with OpenSSL.
15. WinSSL/SChannel
15.1 Add support for client certificate authentication
WinSSL/SChannel currently makes use of the OS-level system and user
certificate and private key stores. This does not allow the application
or the user to supply a custom client certificate using curl or libcurl.
Therefore support for the existing -E/--cert and --key options should be
implemented by supplying a custom certificate to the SChannel APIs, see:
- Getting a Certificate for Schannel
https://msdn.microsoft.com/en-us/library/windows/desktop/aa375447.aspx
15.3 Add support for the --ciphers option
The cipher suites used by WinSSL/SChannel are configured on an OS-level
instead of an application-level. This does not allow the application or
the user to customize the configured cipher suites using curl or libcurl.
Therefore support for the existing --ciphers option should be implemented
by mapping the OpenSSL/GnuTLS cipher suites to the SChannel APIs, see
- Specifying Schannel Ciphers and Cipher Strengths
https://msdn.microsoft.com/en-us/library/windows/desktop/aa380161.aspx
15.4 Add option to disable client certificate auto-send
Microsoft says "By default, Schannel will, with no notification to the client,
attempt to locate a client certificate and send it to the server." That could
be considered a privacy violation and unexpected.
Some Windows users have come to expect that default behavior and to change the
default to make it consistent with other SSL backends would be a breaking
change. An option should be added that can be used to disable the default
Schannel auto-send behavior.
https://github.com/curl/curl/issues/2262
16. SASL
16.1 Other authentication mechanisms
Add support for other authentication mechanisms such as OLP,
GSS-SPNEGO and others.
16.2 Add QOP support to GSSAPI authentication
Currently the GSSAPI authentication only supports the default QOP of auth
(Authentication), whilst Kerberos V5 supports both auth-int (Authentication
with integrity protection) and auth-conf (Authentication with integrity and
privacy protection).
16.3 Support binary messages (i.e.: non-base64)
Mandatory to support LDAP SASL authentication.
17. SSH protocols
17.1 Multiplexing
SSH is a perfectly fine multiplexed protocols which would allow libcurl to do
multiple parallel transfers from the same host using the same connection,
much in the same spirit as HTTP/2 does. libcurl however does not take
advantage of that ability but will instead always create a new connection for
new transfers even if an existing connection already exists to the host.
To fix this, libcurl would have to detect an existing connection and "attach"
the new transfer to the existing one.
17.2 SFTP performance
libcurl's SFTP transfer performance is sub par and can be improved, mostly by
the approach mentioned in "1.6 Modified buffer size approach".
17.3 Support better than MD5 hostkey hash
libcurl offers the CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 option for verifying the
server's key. MD5 is generally being deprecated so we should implement
support for stronger hashing algorithms. libssh2 itself is what provides this
underlying functionality and it supports at least SHA-1 as an alternative.
SHA-1 is also being deprecated these days so we should consider working with
libssh2 to instead offer support for SHA-256 or similar.
17.4 Support CURLOPT_PREQUOTE
The two other QUOTE options are supported for SFTP, but this was left out for
unknown reasons!
18. Command line tool
18.1 sync
"curl --sync http://example.com/feed[1-100].rss" or
"curl --sync http://example.net/{index,calendar,history}.html"
Downloads a range or set of URLs using the remote name, but only if the
remote file is newer than the local file. A Last-Modified HTTP date header
should also be used to set the mod date on the downloaded file.
18.2 glob posts
Globbing support for -d and -F, as in 'curl -d "name=foo[0-9]" URL'.
This is easily scripted though.
18.3 prevent file overwriting
Add an option that prevents curl from overwriting existing local files. When
used, and there already is an existing file with the target file name
(either -O or -o), a number should be appended (and increased if already
existing). So that index.html becomes first index.html.1 and then
index.html.2 etc.
18.4 simultaneous parallel transfers
The client could be told to use maximum N simultaneous parallel transfers and
then just make sure that happens. It should of course not make more than one
connection to the same remote host. This would require the client to use the
multi interface. https://curl.haxx.se/bug/feature.cgi?id=1558595
Using the multi interface would also allow properly using parallel transfers
with HTTP/2 and supporting HTTP/2 server push from the command line.
18.5 UTF-8 filenames in Content-Disposition
RFC 6266 documents how UTF-8 names can be passed to a client in the
Content-Disposition header, and curl does not support this.
https://github.com/curl/curl/issues/1888
18.6 warning when setting an option
Display a warning when libcurl returns an error when setting an option.
This can be useful to tell when support for a particular feature hasn't been
compiled into the library.
18.7 warning if curl version is not in sync with libcurl version
This is usually a sign of a funny, weird or unexpected install situations
that aren't always quickly nor easily detected by users. curl and libcurl are
always released in sync and should use the same version numbers unless very
special situations.
18.8 offer color-coded HTTP header output
By offering different color output on the header name and the header
contents, they could be made more readable and thus help users working on
HTTP services.
18.9 Choose the name of file in braces for complex URLs
When using braces to download a list of URLs and you use complicated names
in the list of alternatives, it could be handy to allow curl to use other
names when saving.
Consider a way to offer that. Possibly like
{partURL1:name1,partURL2:name2,partURL3:name3} where the name following the
colon is the output name.
See https://github.com/curl/curl/issues/221
18.10 improve how curl works in a windows console window
If you pull the scrollbar when transferring with curl in a Windows console
window, the transfer is interrupted and can get disconnected. This can
probably be improved. See https://github.com/curl/curl/issues/322
18.11 Windows: set attribute 'archive' for completed downloads
The archive bit (FILE_ATTRIBUTE_ARCHIVE, 0x20) separates files that shall be
backed up from those that are either not ready or have not changed.
Downloads in progress are neither ready to be backed up, nor should they be
opened by a different process. Only after a download has been completed it's
sensible to include it in any integer snapshot or backup of the system.
See https://github.com/curl/curl/issues/3354
18.12 keep running, read instructions from pipe/socket
Provide an option that makes curl not exit after the last URL (or even work
without a given URL), and then make it read instructions passed on a pipe or
over a socket to make further instructions so that a second subsequent curl
invoke can talk to the still running instance and ask for transfers to get
done, and thus maintain its connection pool, DNS cache and more.
18.13 support metalink in http headers
Curl has support for downloading a metalink xml file, processing it, and then
downloading the target of the metalink. This is done via the --metalink option.
It would be nice if metalink also supported downloading via metalink
information that is stored in HTTP headers (RFC 6249). Theoretically this could
also be supported with the --metalink option.
See https://tools.ietf.org/html/rfc6249
See also https://lists.gnu.org/archive/html/bug-wget/2015-06/msg00034.html for
an implematation of this in wget.
18.14 --fail without --location should treat 3xx as a failure
To allow a command line like this to detect a redirect and consider it a
failure:
curl -v --fail -O https://example.com/curl-7.48.0.tar.gz
... --fail must treat 3xx responses as failures too. The least problematic
way to implement this is probably to add that new logic in the command line
tool only and not in the underlying CURLOPT_FAILONERROR logic.
18.15 --retry should resume
When --retry is used and curl actually retries transfer, it should use the
already transferred data and do a resumed transfer for the rest (when
possible) so that it doesn't have to transfer the same data again that was
already transferred before the retry.
See https://github.com/curl/curl/issues/1084
18.16 send only part of --data
When the user only wants to send a small piece of the data provided with
--data or --data-binary, like when that data is a huge file, consider a way
to specify that curl should only send a piece of that. One suggested syntax
would be: "--data-binary @largefile.zip!1073741823-2147483647".
See https://github.com/curl/curl/issues/1200
18.17 consider file name from the redirected URL with -O ?
When a user gives a URL and uses -O, and curl follows a redirect to a new
URL, the file name is not extracted and used from the newly redirected-to URL
even if the new URL may have a much more sensible file name.
This is clearly documented and helps for security since there's no surprise
to users which file name that might get overwritten. But maybe a new option
could allow for this or maybe -J should imply such a treatment as well as -J
already allows for the server to decide what file name to use so it already
provides the "may overwrite any file" risk.
This is extra tricky if the original URL has no file name part at all since
then the current code path will error out with an error message, and we can't
*know* already at that point if curl will be redirected to a URL that has a
file name...
See https://github.com/curl/curl/issues/1241
18.18 retry on network is unreachable
The --retry option retries transfers on "transient failures". We later added
--retry-connrefused to also retry for "connection refused" errors.
Suggestions have been brought to also allow retry on "network is unreachable"
errors and while totally reasonable, maybe we should consider a way to make
this more configurable than to add a new option for every new error people
want to retry for?
https://github.com/curl/curl/issues/1603
18.19 expand ~/ in config files
For example .curlrc could benefit from being able to do this.
See https://github.com/curl/curl/issues/2317
18.20 host name sections in config files
config files would be more powerful if they could set different
configurations depending on used URLs, host name or possibly origin. Then a
default .curlrc could a specific user-agent only when doing requests against
a certain site.
19. Build
19.1 roffit
Consider extending 'roffit' to produce decent ASCII output, and use that
instead of (g)nroff when building src/tool_hugehelp.c
19.2 Enable PIE and RELRO by default
Especially when having programs that execute curl via the command line, PIE
renders the exploitation of memory corruption vulnerabilities a lot more
difficult. This can be attributed to the additional information leaks being
required to conduct a successful attack. RELRO, on the other hand, masks
different binary sections like the GOT as read-only and thus kills a handful
of techniques that come in handy when attackers are able to arbitrarily
overwrite memory. A few tests showed that enabling these features had close
to no impact, neither on the performance nor on the general functionality of
curl.
20. Test suite
20.1 SSL tunnel
Make our own version of stunnel for simple port forwarding to enable HTTPS
and FTP-SSL tests without the stunnel dependency, and it could allow us to
provide test tools built with either OpenSSL or GnuTLS
20.2 nicer lacking perl message
If perl wasn't found by the configure script, don't attempt to run the tests
but explain something nice why it doesn't.
20.3 more protocols supported
Extend the test suite to include more protocols. The telnet could just do FTP
or http operations (for which we have test servers).
20.4 more platforms supported
Make the test suite work on more platforms. OpenBSD and Mac OS. Remove
fork()s and it should become even more portable.
20.5 Add support for concurrent connections
Tests 836, 882 and 938 were designed to verify that separate connections aren't
used when using different login credentials in protocols that shouldn't re-use
a connection under such circumstances.
Unfortunately, ftpserver.pl doesn't appear to support multiple concurrent
connections. The read while() loop seems to loop until it receives a disconnect
from the client, where it then enters the waiting for connections loop. When
the client opens a second connection to the server, the first connection hasn't
been dropped (unless it has been forced - which we shouldn't do in these tests)
and thus the wait for connections loop is never entered to receive the second
connection.
20.6 Use the RFC6265 test suite
A test suite made for HTTP cookies (RFC 6265) by Adam Barth is available at
https://github.com/abarth/http-state/tree/master/tests
It'd be really awesome if someone would write a script/setup that would run
curl with that test suite and detect deviances. Ideally, that would even be
incorporated into our regular test suite.
20.7 Support LD_PRELOAD on macOS
LD_RELOAD doesn't work on macOS, but there are tests which require it to run
properly. Look into making the preload support in runtests.pl portable such
that it uses DYLD_INSERT_LIBRARIES on macOS.
21. Next SONAME bump
21.1 http-style HEAD output for FTP
#undef CURL_FTP_HTTPSTYLE_HEAD in lib/ftp.c to remove the HTTP-style headers
from being output in NOBODY requests over FTP
21.2 combine error codes
Combine some of the error codes to remove duplicates. The original
numbering should not be changed, and the old identifiers would be
macroed to the new ones in an CURL_NO_OLDIES section to help with
backward compatibility.
Candidates for removal and their replacements:
CURLE_FILE_COULDNT_READ_FILE => CURLE_REMOTE_FILE_NOT_FOUND
CURLE_FTP_COULDNT_RETR_FILE => CURLE_REMOTE_FILE_NOT_FOUND
CURLE_FTP_COULDNT_USE_REST => CURLE_RANGE_ERROR
CURLE_FUNCTION_NOT_FOUND => CURLE_FAILED_INIT
CURLE_LDAP_INVALID_URL => CURLE_URL_MALFORMAT
CURLE_TFTP_NOSUCHUSER => CURLE_TFTP_ILLEGAL
CURLE_TFTP_NOTFOUND => CURLE_REMOTE_FILE_NOT_FOUND
CURLE_TFTP_PERM => CURLE_REMOTE_ACCESS_DENIED
21.3 extend CURLOPT_SOCKOPTFUNCTION prototype
The current prototype only provides 'purpose' that tells what the
connection/socket is for, but not any protocol or similar. It makes it hard
for applications to differentiate on TCP vs UDP and even HTTP vs FTP and
similar.
22. Next major release
22.1 cleanup return codes
curl_easy_cleanup() returns void, but curl_multi_cleanup() returns a
CURLMcode. These should be changed to be the same.
22.2 remove obsolete defines
remove obsolete defines from curl/curl.h
22.3 size_t
make several functions use size_t instead of int in their APIs
22.4 remove several functions
remove the following functions from the public API:
curl_getenv
curl_mprintf (and variations)
curl_strequal
curl_strnequal
They will instead become curlx_ - alternatives. That makes the curl app
still capable of using them, by building with them from source.
These functions have no purpose anymore:
curl_multi_socket
curl_multi_socket_all
22.5 remove CURLOPT_FAILONERROR
Remove support for CURLOPT_FAILONERROR, it has gotten too kludgy and weird
internally. Let the app judge success or not for itself.
22.7 remove progress meter from libcurl
The internally provided progress meter output doesn't belong in the library.
Basically no application wants it (apart from curl) but instead applications
can and should do their own progress meters using the progress callback.
The progress callback should then be bumped as well to get proper 64bit
variable types passed to it instead of doubles so that big files work
correctly.
22.8 remove 'curl_httppost' from public
curl_formadd() was made to fill in a public struct, but the fact that the
struct is public is never really used by application for their own advantage
but instead often restricts how the form functions can or can't be modified.
Changing them to return a private handle will benefit the implementation and
allow us much greater freedoms while still maintaining a solid API and ABI.
usr/share/doc/alt-curlssl/COPYING 0000644 00000002100 15116224051 0012512 0 ustar 00 COPYRIGHT AND PERMISSION NOTICE
Copyright (c) 1996 - 2019, Daniel Stenberg, <daniel@haxx.se>, and many
contributors, see the THANKS file.
All rights reserved.
Permission to use, copy, modify, and distribute this software for any purpose
with or without fee is hereby granted, provided that the above copyright
notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN
NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
OR OTHER DEALINGS IN THE SOFTWARE.
Except as contained in this notice, the name of a copyright holder shall not
be used in advertising or otherwise to promote the sale, use or other dealings
in this Software without prior written authorization of the copyright holder.
usr/share/doc/alt-curlssl/README 0000644 00000003112 15116224057 0012351 0 ustar 00 _ _ ____ _
___| | | | _ \| |
/ __| | | | |_) | |
| (__| |_| | _ <| |___
\___|\___/|_| \_\_____|
README
Curl is a command line tool for transferring data specified with URL
syntax. Find out how to use curl by reading the curl.1 man page or the
MANUAL document. Find out how to install Curl by reading the INSTALL
document.
libcurl is the library curl is using to do its job. It is readily
available to be used by your software. Read the libcurl.3 man page to
learn how!
You find answers to the most frequent questions we get in the FAQ document.
Study the COPYING file for distribution terms and similar. If you distribute
curl binaries or other binaries that involve libcurl, you might enjoy the
LICENSE-MIXING document.
CONTACT
If you have problems, questions, ideas or suggestions, please contact us
by posting to a suitable mailing list. See https://curl.haxx.se/mail/
All contributors to the project are listed in the THANKS document.
WEB SITE
Visit the curl web site for the latest news and downloads:
https://curl.haxx.se/
GIT
To download the very latest source off the GIT server do this:
git clone https://github.com/curl/curl.git
(you'll get a directory named curl created, filled with the source code)
NOTICE
Curl contains pieces of source code that is Copyright (c) 1998, 1999
Kungliga Tekniska Högskolan. This notice is included here to comply with the
distribution terms.
usr/share/doc/alt-curlssl/MANUAL 0000644 00000116131 15116224064 0012375 0 ustar 00 LATEST VERSION
You always find news about what's going on as well as the latest versions
from the curl web pages, located at:
https://curl.haxx.se
SIMPLE USAGE
Get the main page from Netscape's web-server:
curl http://www.netscape.com/
Get the README file the user's home directory at funet's ftp-server:
curl ftp://ftp.funet.fi/README
Get a web page from a server using port 8000:
curl http://www.weirdserver.com:8000/
Get a directory listing of an FTP site:
curl ftp://cool.haxx.se/
Get the definition of curl from a dictionary:
curl dict://dict.org/m:curl
Fetch two documents at once:
curl ftp://cool.haxx.se/ http://www.weirdserver.com:8000/
Get a file off an FTPS server:
curl ftps://files.are.secure.com/secrets.txt
or use the more appropriate FTPS way to get the same file:
curl --ftp-ssl ftp://files.are.secure.com/secrets.txt
Get a file from an SSH server using SFTP:
curl -u username sftp://example.com/etc/issue
Get a file from an SSH server using SCP using a private key
(not password-protected) to authenticate:
curl -u username: --key ~/.ssh/id_rsa \
scp://example.com/~/file.txt
Get a file from an SSH server using SCP using a private key
(password-protected) to authenticate:
curl -u username: --key ~/.ssh/id_rsa --pass private_key_password \
scp://example.com/~/file.txt
Get the main page from an IPv6 web server:
curl "http://[2001:1890:1112:1::20]/"
Get a file from an SMB server:
curl -u "domain\username:passwd" smb://server.example.com/share/file.txt
DOWNLOAD TO A FILE
Get a web page and store in a local file with a specific name:
curl -o thatpage.html http://www.netscape.com/
Get a web page and store in a local file, make the local file get the name
of the remote document (if no file name part is specified in the URL, this
will fail):
curl -O http://www.netscape.com/index.html
Fetch two files and store them with their remote names:
curl -O www.haxx.se/index.html -O curl.haxx.se/download.html
USING PASSWORDS
FTP
To ftp files using name+passwd, include them in the URL like:
curl ftp://name:passwd@machine.domain:port/full/path/to/file
or specify them with the -u flag like
curl -u name:passwd ftp://machine.domain:port/full/path/to/file
FTPS
It is just like for FTP, but you may also want to specify and use
SSL-specific options for certificates etc.
Note that using FTPS:// as prefix is the "implicit" way as described in the
standards while the recommended "explicit" way is done by using FTP:// and
the --ftp-ssl option.
SFTP / SCP
This is similar to FTP, but you can use the --key option to specify a
private key to use instead of a password. Note that the private key may
itself be protected by a password that is unrelated to the login password
of the remote system; this password is specified using the --pass option.
Typically, curl will automatically extract the public key from the private
key file, but in cases where curl does not have the proper library support,
a matching public key file must be specified using the --pubkey option.
HTTP
Curl also supports user and password in HTTP URLs, thus you can pick a file
like:
curl http://name:passwd@machine.domain/full/path/to/file
or specify user and password separately like in
curl -u name:passwd http://machine.domain/full/path/to/file
HTTP offers many different methods of authentication and curl supports
several: Basic, Digest, NTLM and Negotiate (SPNEGO). Without telling which
method to use, curl defaults to Basic. You can also ask curl to pick the
most secure ones out of the ones that the server accepts for the given URL,
by using --anyauth.
NOTE! According to the URL specification, HTTP URLs can not contain a user
and password, so that style will not work when using curl via a proxy, even
though curl allows it at other times. When using a proxy, you _must_ use
the -u style for user and password.
HTTPS
Probably most commonly used with private certificates, as explained below.
PROXY
curl supports both HTTP and SOCKS proxy servers, with optional authentication.
It does not have special support for FTP proxy servers since there are no
standards for those, but it can still be made to work with many of them. You
can also use both HTTP and SOCKS proxies to transfer files to and from FTP
servers.
Get an ftp file using an HTTP proxy named my-proxy that uses port 888:
curl -x my-proxy:888 ftp://ftp.leachsite.com/README
Get a file from an HTTP server that requires user and password, using the
same proxy as above:
curl -u user:passwd -x my-proxy:888 http://www.get.this/
Some proxies require special authentication. Specify by using -U as above:
curl -U user:passwd -x my-proxy:888 http://www.get.this/
A comma-separated list of hosts and domains which do not use the proxy can
be specified as:
curl --noproxy localhost,get.this -x my-proxy:888 http://www.get.this/
If the proxy is specified with --proxy1.0 instead of --proxy or -x, then
curl will use HTTP/1.0 instead of HTTP/1.1 for any CONNECT attempts.
curl also supports SOCKS4 and SOCKS5 proxies with --socks4 and --socks5.
See also the environment variables Curl supports that offer further proxy
control.
Most FTP proxy servers are set up to appear as a normal FTP server from the
client's perspective, with special commands to select the remote FTP server.
curl supports the -u, -Q and --ftp-account options that can be used to
set up transfers through many FTP proxies. For example, a file can be
uploaded to a remote FTP server using a Blue Coat FTP proxy with the
options:
curl -u "Remote-FTP-Username@remote.ftp.server Proxy-Username:Remote-Pass" \
--ftp-account Proxy-Password --upload-file local-file \
ftp://my-ftp.proxy.server:21/remote/upload/path/
See the manual for your FTP proxy to determine the form it expects to set up
transfers, and curl's -v option to see exactly what curl is sending.
RANGES
HTTP 1.1 introduced byte-ranges. Using this, a client can request
to get only one or more subparts of a specified document. Curl supports
this with the -r flag.
Get the first 100 bytes of a document:
curl -r 0-99 http://www.get.this/
Get the last 500 bytes of a document:
curl -r -500 http://www.get.this/
Curl also supports simple ranges for FTP files as well. Then you can only
specify start and stop position.
Get the first 100 bytes of a document using FTP:
curl -r 0-99 ftp://www.get.this/README
UPLOADING
FTP / FTPS / SFTP / SCP
Upload all data on stdin to a specified server:
curl -T - ftp://ftp.upload.com/myfile
Upload data from a specified file, login with user and password:
curl -T uploadfile -u user:passwd ftp://ftp.upload.com/myfile
Upload a local file to the remote site, and use the local file name at the remote
site too:
curl -T uploadfile -u user:passwd ftp://ftp.upload.com/
Upload a local file to get appended to the remote file:
curl -T localfile -a ftp://ftp.upload.com/remotefile
Curl also supports ftp upload through a proxy, but only if the proxy is
configured to allow that kind of tunneling. If it does, you can run curl in
a fashion similar to:
curl --proxytunnel -x proxy:port -T localfile ftp.upload.com
SMB / SMBS
curl -T file.txt -u "domain\username:passwd" \
smb://server.example.com/share/
HTTP
Upload all data on stdin to a specified HTTP site:
curl -T - http://www.upload.com/myfile
Note that the HTTP server must have been configured to accept PUT before
this can be done successfully.
For other ways to do HTTP data upload, see the POST section below.
VERBOSE / DEBUG
If curl fails where it isn't supposed to, if the servers don't let you in,
if you can't understand the responses: use the -v flag to get verbose
fetching. Curl will output lots of info and what it sends and receives in
order to let the user see all client-server interaction (but it won't show
you the actual data).
curl -v ftp://ftp.upload.com/
To get even more details and information on what curl does, try using the
--trace or --trace-ascii options with a given file name to log to, like
this:
curl --trace trace.txt www.haxx.se
DETAILED INFORMATION
Different protocols provide different ways of getting detailed information
about specific files/documents. To get curl to show detailed information
about a single file, you should use -I/--head option. It displays all
available info on a single file for HTTP and FTP. The HTTP information is a
lot more extensive.
For HTTP, you can get the header information (the same as -I would show)
shown before the data by using -i/--include. Curl understands the
-D/--dump-header option when getting files from both FTP and HTTP, and it
will then store the headers in the specified file.
Store the HTTP headers in a separate file (headers.txt in the example):
curl --dump-header headers.txt curl.haxx.se
Note that headers stored in a separate file can be very useful at a later
time if you want curl to use cookies sent by the server. More about that in
the cookies section.
POST (HTTP)
It's easy to post data using curl. This is done using the -d <data>
option. The post data must be urlencoded.
Post a simple "name" and "phone" guestbook.
curl -d "name=Rafael%20Sagula&phone=3320780" \
http://www.where.com/guest.cgi
How to post a form with curl, lesson #1:
Dig out all the <input> tags in the form that you want to fill in.
If there's a "normal" post, you use -d to post. -d takes a full "post
string", which is in the format
<variable1>=<data1>&<variable2>=<data2>&...
The 'variable' names are the names set with "name=" in the <input> tags, and
the data is the contents you want to fill in for the inputs. The data *must*
be properly URL encoded. That means you replace space with + and that you
replace weird letters with %XX where XX is the hexadecimal representation of
the letter's ASCII code.
Example:
(page located at http://www.formpost.com/getthis/
<form action="post.cgi" method="post">
<input name=user size=10>
<input name=pass type=password size=10>
<input name=id type=hidden value="blablabla">
<input name=ding value="submit">
</form>
We want to enter user 'foobar' with password '12345'.
To post to this, you enter a curl command line like:
curl -d "user=foobar&pass=12345&id=blablabla&ding=submit" \
http://www.formpost.com/getthis/post.cgi
While -d uses the application/x-www-form-urlencoded mime-type, generally
understood by CGI's and similar, curl also supports the more capable
multipart/form-data type. This latter type supports things like file upload.
-F accepts parameters like -F "name=contents". If you want the contents to
be read from a file, use <@filename> as contents. When specifying a file,
you can also specify the file content type by appending ';type=<mime type>'
to the file name. You can also post the contents of several files in one
field. For example, the field name 'coolfiles' is used to send three files,
with different content types using the following syntax:
curl -F "coolfiles=@fil1.gif;type=image/gif,fil2.txt,fil3.html" \
http://www.post.com/postit.cgi
If the content-type is not specified, curl will try to guess from the file
extension (it only knows a few), or use the previously specified type (from
an earlier file if several files are specified in a list) or else it will
use the default type 'application/octet-stream'.
Emulate a fill-in form with -F. Let's say you fill in three fields in a
form. One field is a file name which to post, one field is your name and one
field is a file description. We want to post the file we have written named
"cooltext.txt". To let curl do the posting of this data instead of your
favourite browser, you have to read the HTML source of the form page and
find the names of the input fields. In our example, the input field names
are 'file', 'yourname' and 'filedescription'.
curl -F "file=@cooltext.txt" -F "yourname=Daniel" \
-F "filedescription=Cool text file with cool text inside" \
http://www.post.com/postit.cgi
To send two files in one post you can do it in two ways:
1. Send multiple files in a single "field" with a single field name:
curl -F "pictures=@dog.gif,cat.gif"
2. Send two fields with two field names:
curl -F "docpicture=@dog.gif" -F "catpicture=@cat.gif"
To send a field value literally without interpreting a leading '@'
or '<', or an embedded ';type=', use --form-string instead of
-F. This is recommended when the value is obtained from a user or
some other unpredictable source. Under these circumstances, using
-F instead of --form-string would allow a user to trick curl into
uploading a file.
REFERRER
An HTTP request has the option to include information about which address
referred it to the actual page. Curl allows you to specify the
referrer to be used on the command line. It is especially useful to
fool or trick stupid servers or CGI scripts that rely on that information
being available or contain certain data.
curl -e www.coolsite.com http://www.showme.com/
NOTE: The Referer: [sic] field is defined in the HTTP spec to be a full URL.
USER AGENT
An HTTP request has the option to include information about the browser
that generated the request. Curl allows it to be specified on the command
line. It is especially useful to fool or trick stupid servers or CGI
scripts that only accept certain browsers.
Example:
curl -A 'Mozilla/3.0 (Win95; I)' http://www.nationsbank.com/
Other common strings:
'Mozilla/3.0 (Win95; I)' Netscape Version 3 for Windows 95
'Mozilla/3.04 (Win95; U)' Netscape Version 3 for Windows 95
'Mozilla/2.02 (OS/2; U)' Netscape Version 2 for OS/2
'Mozilla/4.04 [en] (X11; U; AIX 4.2; Nav)' NS for AIX
'Mozilla/4.05 [en] (X11; U; Linux 2.0.32 i586)' NS for Linux
Note that Internet Explorer tries hard to be compatible in every way:
'Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)' MSIE for W95
Mozilla is not the only possible User-Agent name:
'Konqueror/1.0' KDE File Manager desktop client
'Lynx/2.7.1 libwww-FM/2.14' Lynx command line browser
COOKIES
Cookies are generally used by web servers to keep state information at the
client's side. The server sets cookies by sending a response line in the
headers that looks like 'Set-Cookie: <data>' where the data part then
typically contains a set of NAME=VALUE pairs (separated by semicolons ';'
like "NAME1=VALUE1; NAME2=VALUE2;"). The server can also specify for what
path the "cookie" should be used for (by specifying "path=value"), when the
cookie should expire ("expire=DATE"), for what domain to use it
("domain=NAME") and if it should be used on secure connections only
("secure").
If you've received a page from a server that contains a header like:
Set-Cookie: sessionid=boo123; path="/foo";
it means the server wants that first pair passed on when we get anything in
a path beginning with "/foo".
Example, get a page that wants my name passed in a cookie:
curl -b "name=Daniel" www.sillypage.com
Curl also has the ability to use previously received cookies in following
sessions. If you get cookies from a server and store them in a file in a
manner similar to:
curl --dump-header headers www.example.com
... you can then in a second connect to that (or another) site, use the
cookies from the 'headers' file like:
curl -b headers www.example.com
While saving headers to a file is a working way to store cookies, it is
however error-prone and not the preferred way to do this. Instead, make curl
save the incoming cookies using the well-known netscape cookie format like
this:
curl -c cookies.txt www.example.com
Note that by specifying -b you enable the "cookie awareness" and with -L
you can make curl follow a location: (which often is used in combination
with cookies). So that if a site sends cookies and a location, you can
use a non-existing file to trigger the cookie awareness like:
curl -L -b empty.txt www.example.com
The file to read cookies from must be formatted using plain HTTP headers OR
as netscape's cookie file. Curl will determine what kind it is based on the
file contents. In the above command, curl will parse the header and store
the cookies received from www.example.com. curl will send to the server the
stored cookies which match the request as it follows the location. The
file "empty.txt" may be a nonexistent file.
To read and write cookies from a netscape cookie file, you can set both -b
and -c to use the same file:
curl -b cookies.txt -c cookies.txt www.example.com
PROGRESS METER
The progress meter exists to show a user that something actually is
happening. The different fields in the output have the following meaning:
% Total % Received % Xferd Average Speed Time Curr.
Dload Upload Total Current Left Speed
0 151M 0 38608 0 0 9406 0 4:41:43 0:00:04 4:41:39 9287
From left-to-right:
% - percentage completed of the whole transfer
Total - total size of the whole expected transfer
% - percentage completed of the download
Received - currently downloaded amount of bytes
% - percentage completed of the upload
Xferd - currently uploaded amount of bytes
Average Speed
Dload - the average transfer speed of the download
Average Speed
Upload - the average transfer speed of the upload
Time Total - expected time to complete the operation
Time Current - time passed since the invoke
Time Left - expected time left to completion
Curr.Speed - the average transfer speed the last 5 seconds (the first
5 seconds of a transfer is based on less time of course.)
The -# option will display a totally different progress bar that doesn't
need much explanation!
SPEED LIMIT
Curl allows the user to set the transfer speed conditions that must be met
to let the transfer keep going. By using the switch -y and -Y you
can make curl abort transfers if the transfer speed is below the specified
lowest limit for a specified time.
To have curl abort the download if the speed is slower than 3000 bytes per
second for 1 minute, run:
curl -Y 3000 -y 60 www.far-away-site.com
This can very well be used in combination with the overall time limit, so
that the above operation must be completed in whole within 30 minutes:
curl -m 1800 -Y 3000 -y 60 www.far-away-site.com
Forcing curl not to transfer data faster than a given rate is also possible,
which might be useful if you're using a limited bandwidth connection and you
don't want your transfer to use all of it (sometimes referred to as
"bandwidth throttle").
Make curl transfer data no faster than 10 kilobytes per second:
curl --limit-rate 10K www.far-away-site.com
or
curl --limit-rate 10240 www.far-away-site.com
Or prevent curl from uploading data faster than 1 megabyte per second:
curl -T upload --limit-rate 1M ftp://uploadshereplease.com
When using the --limit-rate option, the transfer rate is regulated on a
per-second basis, which will cause the total transfer speed to become lower
than the given number. Sometimes of course substantially lower, if your
transfer stalls during periods.
CONFIG FILE
Curl automatically tries to read the .curlrc file (or _curlrc file on win32
systems) from the user's home dir on startup.
The config file could be made up with normal command line switches, but you
can also specify the long options without the dashes to make it more
readable. You can separate the options and the parameter with spaces, or
with = or :. Comments can be used within the file. If the first letter on a
line is a '#'-symbol the rest of the line is treated as a comment.
If you want the parameter to contain spaces, you must enclose the entire
parameter within double quotes ("). Within those quotes, you specify a
quote as \".
NOTE: You must specify options and their arguments on the same line.
Example, set default time out and proxy in a config file:
# We want a 30 minute timeout:
-m 1800
# ... and we use a proxy for all accesses:
proxy = proxy.our.domain.com:8080
White spaces ARE significant at the end of lines, but all white spaces
leading up to the first characters of each line are ignored.
Prevent curl from reading the default file by using -q as the first command
line parameter, like:
curl -q www.thatsite.com
Force curl to get and display a local help page in case it is invoked
without URL by making a config file similar to:
# default url to get
url = "http://help.with.curl.com/curlhelp.html"
You can specify another config file to be read by using the -K/--config
flag. If you set config file name to "-" it'll read the config from stdin,
which can be handy if you want to hide options from being visible in process
tables etc:
echo "user = user:passwd" | curl -K - http://that.secret.site.com
EXTRA HEADERS
When using curl in your own very special programs, you may end up needing
to pass on your own custom headers when getting a web page. You can do
this by using the -H flag.
Example, send the header "X-you-and-me: yes" to the server when getting a
page:
curl -H "X-you-and-me: yes" www.love.com
This can also be useful in case you want curl to send a different text in a
header than it normally does. The -H header you specify then replaces the
header curl would normally send. If you replace an internal header with an
empty one, you prevent that header from being sent. To prevent the Host:
header from being used:
curl -H "Host:" www.server.com
FTP and PATH NAMES
Do note that when getting files with the ftp:// URL, the given path is
relative the directory you enter. To get the file 'README' from your home
directory at your ftp site, do:
curl ftp://user:passwd@my.site.com/README
But if you want the README file from the root directory of that very same
site, you need to specify the absolute file name:
curl ftp://user:passwd@my.site.com//README
(I.e with an extra slash in front of the file name.)
SFTP and SCP and PATH NAMES
With sftp: and scp: URLs, the path name given is the absolute name on the
server. To access a file relative to the remote user's home directory,
prefix the file with /~/ , such as:
curl -u $USER sftp://home.example.com/~/.bashrc
FTP and firewalls
The FTP protocol requires one of the involved parties to open a second
connection as soon as data is about to get transferred. There are two ways to
do this.
The default way for curl is to issue the PASV command which causes the
server to open another port and await another connection performed by the
client. This is good if the client is behind a firewall that doesn't allow
incoming connections.
curl ftp.download.com
If the server, for example, is behind a firewall that doesn't allow connections
on ports other than 21 (or if it just doesn't support the PASV command), the
other way to do it is to use the PORT command and instruct the server to
connect to the client on the given IP number and port (as parameters to the
PORT command).
The -P flag to curl supports a few different options. Your machine may have
several IP-addresses and/or network interfaces and curl allows you to select
which of them to use. Default address can also be used:
curl -P - ftp.download.com
Download with PORT but use the IP address of our 'le0' interface (this does
not work on windows):
curl -P le0 ftp.download.com
Download with PORT but use 192.168.0.10 as our IP address to use:
curl -P 192.168.0.10 ftp.download.com
NETWORK INTERFACE
Get a web page from a server using a specified port for the interface:
curl --interface eth0:1 http://www.netscape.com/
or
curl --interface 192.168.1.10 http://www.netscape.com/
HTTPS
Secure HTTP requires SSL libraries to be installed and used when curl is
built. If that is done, curl is capable of retrieving and posting documents
using the HTTPS protocol.
Example:
curl https://www.secure-site.com
Curl is also capable of using your personal certificates to get/post files
from sites that require valid certificates. The only drawback is that the
certificate needs to be in PEM-format. PEM is a standard and open format to
store certificates with, but it is not used by the most commonly used
browsers (Netscape and MSIE both use the so called PKCS#12 format). If you
want curl to use the certificates you use with your (favourite) browser, you
may need to download/compile a converter that can convert your browser's
formatted certificates to PEM formatted ones. This kind of converter is
included in recent versions of OpenSSL, and for older versions Dr Stephen
N. Henson has written a patch for SSLeay that adds this functionality. You
can get his patch (that requires an SSLeay installation) from his site at:
https://web.archive.org/web/20170715155512/www.drh-consultancy.demon.co.uk/
Example on how to automatically retrieve a document using a certificate with
a personal password:
curl -E /path/to/cert.pem:password https://secure.site.com/
If you neglect to specify the password on the command line, you will be
prompted for the correct password before any data can be received.
Many older SSL-servers have problems with SSLv3 or TLS, which newer versions
of OpenSSL etc use, therefore it is sometimes useful to specify what
SSL-version curl should use. Use -3, -2 or -1 to specify that exact SSL
version to use (for SSLv3, SSLv2 or TLSv1 respectively):
curl -2 https://secure.site.com/
Otherwise, curl will first attempt to use v3 and then v2.
To use OpenSSL to convert your favourite browser's certificate into a PEM
formatted one that curl can use, do something like this:
In Netscape, you start with hitting the 'Security' menu button.
Select 'certificates->yours' and then pick a certificate in the list
Press the 'Export' button
enter your PIN code for the certs
select a proper place to save it
Run the 'openssl' application to convert the certificate. If you cd to the
openssl installation, you can do it like:
# ./apps/openssl pkcs12 -in [file you saved] -clcerts -out [PEMfile]
In Firefox, select Options, then Advanced, then the Encryption tab,
View Certificates. This opens the Certificate Manager, where you can
Export. Be sure to select PEM for the Save as type.
In Internet Explorer, select Internet Options, then the Content tab, then
Certificates. Then you can Export, and depending on the format you may
need to convert to PEM.
In Chrome, select Settings, then Show Advanced Settings. Under HTTPS/SSL
select Manage Certificates.
RESUMING FILE TRANSFERS
To continue a file transfer where it was previously aborted, curl supports
resume on HTTP(S) downloads as well as FTP uploads and downloads.
Continue downloading a document:
curl -C - -o file ftp://ftp.server.com/path/file
Continue uploading a document(*1):
curl -C - -T file ftp://ftp.server.com/path/file
Continue downloading a document from a web server(*2):
curl -C - -o file http://www.server.com/
(*1) = This requires that the FTP server supports the non-standard command
SIZE. If it doesn't, curl will say so.
(*2) = This requires that the web server supports at least HTTP/1.1. If it
doesn't, curl will say so.
TIME CONDITIONS
HTTP allows a client to specify a time condition for the document it
requests. It is If-Modified-Since or If-Unmodified-Since. Curl allows you to
specify them with the -z/--time-cond flag.
For example, you can easily make a download that only gets performed if the
remote file is newer than a local copy. It would be made like:
curl -z local.html http://remote.server.com/remote.html
Or you can download a file only if the local file is newer than the remote
one. Do this by prepending the date string with a '-', as in:
curl -z -local.html http://remote.server.com/remote.html
You can specify a "free text" date as condition. Tell curl to only download
the file if it was updated since January 12, 2012:
curl -z "Jan 12 2012" http://remote.server.com/remote.html
Curl will then accept a wide range of date formats. You always make the date
check the other way around by prepending it with a dash '-'.
DICT
For fun try
curl dict://dict.org/m:curl
curl dict://dict.org/d:heisenbug:jargon
curl dict://dict.org/d:daniel:web1913
Aliases for 'm' are 'match' and 'find', and aliases for 'd' are 'define'
and 'lookup'. For example,
curl dict://dict.org/find:curl
Commands that break the URL description of the RFC (but not the DICT
protocol) are
curl dict://dict.org/show:db
curl dict://dict.org/show:strat
Authentication is still missing (but this is not required by the RFC)
LDAP
If you have installed the OpenLDAP library, curl can take advantage of it
and offer ldap:// support.
On Windows, curl will use WinLDAP from Platform SDK by default.
Default protocol version used by curl is LDAPv3. LDAPv2 will be used as
fallback mechanism in case if LDAPv3 will fail to connect.
LDAP is a complex thing and writing an LDAP query is not an easy task. I do
advise you to dig up the syntax description for that elsewhere. One such
place might be:
RFC 2255, "The LDAP URL Format" https://curl.haxx.se/rfc/rfc2255.txt
To show you an example, this is how I can get all people from my local LDAP
server that has a certain sub-domain in their email address:
curl -B "ldap://ldap.frontec.se/o=frontec??sub?mail=*sth.frontec.se"
If I want the same info in HTML format, I can get it by not using the -B
(enforce ASCII) flag.
You also can use authentication when accessing LDAP catalog:
curl -u user:passwd "ldap://ldap.frontec.se/o=frontec??sub?mail=*"
curl "ldap://user:passwd@ldap.frontec.se/o=frontec??sub?mail=*"
By default, if user and password provided, OpenLDAP/WinLDAP will use basic
authentication. On Windows you can control this behavior by providing
one of --basic, --ntlm or --digest option in curl command line
curl --ntlm "ldap://user:passwd@ldap.frontec.se/o=frontec??sub?mail=*"
On Windows, if no user/password specified, auto-negotiation mechanism will
be used with current logon credentials (SSPI/SPNEGO).
ENVIRONMENT VARIABLES
Curl reads and understands the following environment variables:
http_proxy, HTTPS_PROXY, FTP_PROXY
They should be set for protocol-specific proxies. General proxy should be
set with
ALL_PROXY
A comma-separated list of host names that shouldn't go through any proxy is
set in (only an asterisk, '*' matches all hosts)
NO_PROXY
If the host name matches one of these strings, or the host is within the
domain of one of these strings, transactions with that node will not be
proxied. When a domain is used, it needs to start with a period. A user can
specify that both www.example.com and foo.example.com should not use a
proxy by setting NO_PROXY to ".example.com". By including the full name you
can exclude specific host names, so to make www.example.com not use a proxy
but still have foo.example.com do it, set NO_PROXY to "www.example.com"
The usage of the -x/--proxy flag overrides the environment variables.
NETRC
Unix introduced the .netrc concept a long time ago. It is a way for a user
to specify name and password for commonly visited FTP sites in a file so
that you don't have to type them in each time you visit those sites. You
realize this is a big security risk if someone else gets hold of your
passwords, so therefore most unix programs won't read this file unless it is
only readable by yourself (curl doesn't care though).
Curl supports .netrc files if told to (using the -n/--netrc and
--netrc-optional options). This is not restricted to just FTP,
so curl can use it for all protocols where authentication is used.
A very simple .netrc file could look something like:
machine curl.haxx.se login iamdaniel password mysecret
CUSTOM OUTPUT
To better allow script programmers to get to know about the progress of
curl, the -w/--write-out option was introduced. Using this, you can specify
what information from the previous transfer you want to extract.
To display the amount of bytes downloaded together with some text and an
ending newline:
curl -w 'We downloaded %{size_download} bytes\n' www.download.com
KERBEROS FTP TRANSFER
Curl supports kerberos4 and kerberos5/GSSAPI for FTP transfers. You need
the kerberos package installed and used at curl build time for it to be
available.
First, get the krb-ticket the normal way, like with the kinit/kauth tool.
Then use curl in way similar to:
curl --krb private ftp://krb4site.com -u username:fakepwd
There's no use for a password on the -u switch, but a blank one will make
curl ask for one and you already entered the real password to kinit/kauth.
TELNET
The curl telnet support is basic and very easy to use. Curl passes all data
passed to it on stdin to the remote server. Connect to a remote telnet
server using a command line similar to:
curl telnet://remote.server.com
And enter the data to pass to the server on stdin. The result will be sent
to stdout or to the file you specify with -o.
You might want the -N/--no-buffer option to switch off the buffered output
for slow connections or similar.
Pass options to the telnet protocol negotiation, by using the -t option. To
tell the server we use a vt100 terminal, try something like:
curl -tTTYPE=vt100 telnet://remote.server.com
Other interesting options for it -t include:
- XDISPLOC=<X display> Sets the X display location.
- NEW_ENV=<var,val> Sets an environment variable.
NOTE: The telnet protocol does not specify any way to login with a specified
user and password so curl can't do that automatically. To do that, you need
to track when the login prompt is received and send the username and
password accordingly.
PERSISTENT CONNECTIONS
Specifying multiple files on a single command line will make curl transfer
all of them, one after the other in the specified order.
libcurl will attempt to use persistent connections for the transfers so that
the second transfer to the same host can use the same connection that was
already initiated and was left open in the previous transfer. This greatly
decreases connection time for all but the first transfer and it makes a far
better use of the network.
Note that curl cannot use persistent connections for transfers that are used
in subsequence curl invokes. Try to stuff as many URLs as possible on the
same command line if they are using the same host, as that'll make the
transfers faster. If you use an HTTP proxy for file transfers, practically
all transfers will be persistent.
MULTIPLE TRANSFERS WITH A SINGLE COMMAND LINE
As is mentioned above, you can download multiple files with one command line
by simply adding more URLs. If you want those to get saved to a local file
instead of just printed to stdout, you need to add one save option for each
URL you specify. Note that this also goes for the -O option (but not
--remote-name-all).
For example: get two files and use -O for the first and a custom file
name for the second:
curl -O http://url.com/file.txt ftp://ftp.com/moo.exe -o moo.jpg
You can also upload multiple files in a similar fashion:
curl -T local1 ftp://ftp.com/moo.exe -T local2 ftp://ftp.com/moo2.txt
IPv6
curl will connect to a server with IPv6 when a host lookup returns an IPv6
address and fall back to IPv4 if the connection fails. The --ipv4 and --ipv6
options can specify which address to use when both are available. IPv6
addresses can also be specified directly in URLs using the syntax:
http://[2001:1890:1112:1::20]/overview.html
When this style is used, the -g option must be given to stop curl from
interpreting the square brackets as special globbing characters. Link local
and site local addresses including a scope identifier, such as fe80::1234%1,
may also be used, but the scope portion must be numeric or match an existing
network interface on Linux and the percent character must be URL escaped. The
previous example in an SFTP URL might look like:
sftp://[fe80::1234%251]/
IPv6 addresses provided other than in URLs (e.g. to the --proxy, --interface
or --ftp-port options) should not be URL encoded.
METALINK
Curl supports Metalink (both version 3 and 4 (RFC 5854) are supported), a way
to list multiple URIs and hashes for a file. Curl will make use of the mirrors
listed within for failover if there are errors (such as the file or server not
being available). It will also verify the hash of the file after the download
completes. The Metalink file itself is downloaded and processed in memory and
not stored in the local file system.
Example to use a remote Metalink file:
curl --metalink http://www.example.com/example.metalink
To use a Metalink file in the local file system, use FILE protocol (file://):
curl --metalink file://example.metalink
Please note that if FILE protocol is disabled, there is no way to use a local
Metalink file at the time of this writing. Also note that if --metalink and
--include are used together, --include will be ignored. This is because including
headers in the response will break Metalink parser and if the headers are included
in the file described in Metalink file, hash check will fail.
MAILING LISTS
For your convenience, we have several open mailing lists to discuss curl,
its development and things relevant to this. Get all info at
https://curl.haxx.se/mail/. Some of the lists available are:
curl-users
Users of the command line tool. How to use it, what doesn't work, new
features, related tools, questions, news, installations, compilations,
running, porting etc.
curl-library
Developers using or developing libcurl. Bugs, extensions, improvements.
curl-announce
Low-traffic. Only receives announcements of new public versions. At worst,
that makes something like one or two mails per month, but usually only one
mail every second month.
curl-and-php
Using the curl functions in PHP. Everything curl with a PHP angle. Or PHP
with a curl angle.
curl-and-python
Python hackers using curl with or without the python binding pycurl.
Please direct curl questions, feature requests and trouble reports to one of
these mailing lists instead of mailing any individual.
usr/lib64/libcurl.so.4.5.0 0000755 00002403410 15116224071 0010745 0 ustar 00 ELF >