jigdo - Jigsaw Download

Jigsaw Download, or short jigdo, is a tool designed to ease the distribution of very large files over the internet, for example CD or DVD images. Its aim is to make downloading the images as easy for users as a click on a direct download link in a browser, while avoiding all the problems that server administrators have with hosting such large files.

Article about jigdo on heise.de (c't-Magazin; German)

This software is now in "maintenance mode", development has stopped. Unfortunately, this means that various parts (in particular the GUI download tool) will remain unfinished.


Download


Help

  • Documentation: jigdo FAQ (frequently asked questions)
  • Documentation: The FAQ is part of the Debian jigdo mini-HOWTO. Look here if you have trouble using jigdo-lite.
  • jigdo-user mailing list: General discussion about jigdo.
    Subscribe to jigdo-user, browse the archive or send mail directly to jigdo-user [at] lists [dot] berlios [dot] de.
    The mailing list is public - your mails will be archived, and it is not guaranteed that someone won't abuse the archive and send you spam.
  • Documentation: The Debian CD FAQ is not directly related to jigdo, but worth checking if you have questions regarding the images you downloaded using jigdo.
  • Debian-CD mailing list: Discussion about problems with the Debian CD images.
    Subscribe to debian-cd, browse the archive or send mail directly to debian-cd [at] lists [dot] debian [dot] org. This mailing list is also public.

Why Is It Needed?

jigdo was inspired by the problems that Debian Linux had with the distribution of its CD images. Debian is a 100% volunteer-driven organization which depends on donations to finance its infrastructure. For this reason, it also doesn't own any of the 300 servers around the world on which the distribution is mirrored.

For some time, the size of Debian has been a problem - Debian supports 11 different processor architectures, more than any other Linux distribution, and for every single one of these architectures, there are more than 10000 software packages - again more than any other distro. Today, a full Debian mirror needs roughly 100 GB of disc space!

Debian also offers the distribution in the form of CD images (and more recently DVD images). If these images were distributed as full .iso images on the mirrors, the size of a Debian mirror would double to 200 GB (or even triple to 300 GB if there were also DVD images) - nobody is prepared to mirror such a huge amount of data!

jigdo solves this problem by not requiring that the full image be stored on the servers. Instead, it can download the individual files from a normal Debian mirror and only assemble the image on the user's machine. Additionally, it can also do other useful things like helping with the mirror selection and upgrading images.


How Does It Work?

When someone has produced a large file they wish to distribute (for example, a CD image), they process this file with the jigdo-file tool. Among other things, jigdo-file creates a file with a .jigdo extension. To download the original file, a user only needs to tell the jigdo-lite download tool the URL of the jigdo file, it will extract all the necessary information and perform the download, fetching the data in many small pieces, possibly from several different servers.

So far, this is not exciting; there are already several other programs which allow you to split and reassemble large files, or which make their download easier. Also, at the moment only the shell script jigdo-lite can be used for downloading. Unfortunately, the jigdo GUI download manager shown below never reached a usable state.

[Screenshot]
The jigdo download tool (unfinished, not usable)

However, jigdo's approach is unique due to the way it identifies the small pieces that the large file consists of: For instance, if the file is a CD image, the individual files on the CD are contained somewhere within the image file. jigdo-file is capable of finding these files, so if copies of the individual files on the CD are stored separately on the server, jigdo-lite can be made to download these files one by one, and assemble the CD image on the fly on the user's computer. This scheme has several advantages:

  • The large image does not need to be stored on the server, instead only the many small files contained in the image. This works with CD images, DVD images (both ISO9660 and UDF format), uncompressed zip files, tar archives... jigdo-file is format independent.
  • In spite of the above, jigdo creates a bit-exact copy of the original image on the user's machine. (To achieve this, the directory data, boot block etc. of the image is stored in a special .template file which is distributed alongside the .jigdo file.)
  • There is full control over where jigdo-lite will download the individual parts. It is possible to define mirrors so users can choose their nearest mirror.
  • jigdo relies on standard HTTP/FTP, no support for special protocols like rsync, and no installation of special software is required on the mirrors.
  • jigdo-lite supports resuming aborted downloads, or continuing the download with another mirror if the current one proves slow.
  • It is possible to "upgrade" a CD image: If a new version of an image is released, jigdo-lite can download only the data that has changed, the remaining data is read from the outdated version.
  • It is possible to release modified versions: If someone distributes a CD image and you have made a few small modifications, e.g. added some files, then you only need to upload the data for your modifications to your webspace. All the data from the original CD image is still fetched from the original site.

If you have read this far, you may be confused now, asking yourself what jigdo does and doesn't do. :-) It seems that the jigdo concept is a bit difficult to grasp at first. If you're still interested, have a look at the respective section of the HOWTO and at the Examples section of the jigdo-file manual.


errata for the mini-howto

I'm downloading the last Debian testing and all is well, thanks for the great job !

I just noticed that the URL directing to your website is outdated in the mini-howto, it redirects to atterer.net instead of .org . Have a nice day !

jigdo download tool

please!! make this work on debian or windows, maybe i'll star using Jigdo....
(send me email with news)

thanks

re:jigdo download tool

I'm using this in a Gentoo-based resuce usb thumbdrive to reinstall Debian, and it works _VERY_ well. It should work in Debian, too; just follow the directions and don't worry about the lack of a GUI - the command-line version is easy enough to follow.

Please!! read the first three paragraphs on the page...

This software is now in "maintenance mode", development has stopped.

Already done

To my knowledge it already works on Debian and Windows.

Nice program!,looks like a

Nice program!,looks like a good solution to reduce serverload.
As for me unless the folks over at debian.org can explain to me in plain dutch(as opposed to some sort of linux-dutch dialect)how to actually get a hold of complete url's for the images i'm afraid it's going to be a regular download for me(i am a linux noob!).

got it working!

I finally succeeded with instructions from third-party website,again great app but documentation on debian.org leaves something to be desired especially for n00bs.