Skip to content

cURL

"command line tool and library for transferring data with URLs" - https://curl.haxx.se

curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.

Examples

Follow location redirects

curl -L http://whatever

Show the HTTP code for a given request

This downloads 1 byte from the remote URI and shows the HTTP code, so it's a quick way to discover HTTP codes.

curl --max-filesize 1 -s -w "%{http_code}\n" -o /dev/null http://www.example.com/

Request a specific vhost from a server

This is useful for testing production code on non-production multi-tennant name based virtual hosts.

curl -H 'Host: www.domain.com' http://example.com

Get the length of the file to be downloaded

curl -qI  https://www.google.com/index.php 2>/dev/null | awk '/Length/ {print $2}'

Fetch only HTTP headers, not content

curl -I http://www.example.com/some_huge_file.iso

Send POST variables

curl --data "user=foo&pass=bar" http://example.com/login.php

Scrape URLs from a page

This appears to have problems with some strings. For instance, this doesn't catch the full https://accounts.google.com string. The regex is correct according to http://regexpal.com, but egrep is apparently not handling it correctly.

curl -s http://www.google.com | egrep -o '(((https?|ftp|gopher)://|(mailto|file|news):)[^’ <>\n"]+|(www|web|w3)\.[-a-z0-9.]+)[^’ .,;<>":]'

Use curl to fetch the current RFC 2822 time

If you don't have NTP you can use this to manually feed the current time into date -s to set your system clock to within a few seconds of accuracy.

curl -sIH 'Cache-Control: no-cache' example.org | grep '^Date'

Using GNU cut and date (IE: on linux, like on a Raspberry Pi image that does not have NTP properly set up) you can set your time using this command:

sudo date -s "$(curl -sIH 'Cache-Control: no-cache' example.org | grep '^Date:' | cut -f 1 -d ' ' --complement)"

See Also