45. Using wget and curl for HTTP Requests

wget and curl are essential utilities for transferring data, downloading files, and interacting with web APIs directly from the command line.

1. wget (Web Get)

wget is mainly used for non-interactive downloading of files from the web. It is good at recursive downloads and resuming interrupted transfers.

Installation (usually pre-installed):

bash $ pkg install wget

Example 1: Simple Download

bash

Downloads the file to the current directory

$ wget https://example.com/data.zip

Example 2: Downloading an entire website (Recursive)

bash

Caution: Use carefully, can download large amounts of data

$ wget -r -l 1 -k -p http://example.com/

2. curl (Client URL)

curl is more versatile than wget, supporting a wider range of protocols (FTP, SCP, TELNET) and being excellent for testing APIs (POST, GET, headers).

Installation (usually pre-installed):

bash $ pkg install curl

Example 1: Fetching Content and Displaying Headers

bash $ curl -I https://google.com

Displays only the HTTP response headers

Example 2: Making a POST Request (API Testing)

bash $ curl -X POST -H "Content-Type: application/json" -d '{"key":"value"}' https://api.example.com/resource