GNU/Linux Desktop Survival Guide by Graham Williams
Desktop Survival Project Home Preface Advocacy History Distributions Installing GNU/Linux Basic Survival Wajig Applications AI, Machine Learning, Data Science Android - GNU/Linux Devices Audio Backup Booting CD Chinese ChRoot Clock Command Line Databases DIA Directories Disks EcoSysL Emacs Email Evolution Files File Systems Firewalls Floppy Disks FreedomBox Glade GNOME Graphics, Images, Photos Hardware Initialisations on Booting Installation Examples Internet IOT Java KDE Kernels Keyboard Konqueror KVM Switch LATEX Log System Login Magellan MATE Mathematics Memory Modems Mounting Devices MS/Windows Music MySQL Nautilus Networks Nextcloud Network File System NFS NT File System Oracle Packages Partitions Passwords PDF Power Management PPP Presentations Printing Python R Remote Desktops Rsync Samba Scanning Science Screens Security Spreadsheet SSH Swap Teradata Data Warehouse Themes TV Unity USB Version Control Video Virtualization Voice over IP WebCam WordPress Web Sites Word Processing and Printing X XML Troubleshooting Index

CLICK HERE TO VISIT THE UPDATED SURVIVAL GUIDE

Wget Mirror Websites

20200526 A popular use case for wget is to make a complete copy of a website, perhaps for local perusal or local archival. For example, we might backup a conference website for archival and historical purposes:

$ wget --mirror --convert-links --adjust-extension --page-requisites \
  --no-parent https://ausdm18.ausdm.org/

This will create a directory called ausdm18.ausdm.org in the current working directory. Browsing to this directory within a browser using a URL like file:///home/kayon/ausdm18.ausdm.org will interact with the local copy of the web site.

Another use case is to download all of the available Debian packages that start with r as available from a particular Debian mirror.

  $ wget --mirror --accept '.deb' --no-directories \
    http://archive.ubuntu.com/ubuntu/ubuntu/pool/main/r/

Useful comman line options include -r (--recursive) which indicates that we want to recurse through the given URL link. The --mirror option includes --recursive as well as some other options (see the manual page for details). The -l 1 (--level=1) option specifies how many levels we should dive into at the web site. Here we recurse only a single level. The -A .deb (--accept) resticts the download to just those files the have a deb extension. The extenstions can be a comma separated list. The -nd (--no-directories) requests wget to not create any directories locally—the files are downloaded to the current directory.

For a website that no longer exists, the wayback machine is useful. To copy a website from there, install the wayback machine downloader and then:

$ wayback_machine_downloader http://ausdm17.azurewebsites.net/

Unlike wget, fixed links are not updated to be internally consistent. That will need to be done by hand.