As noted in Paragraph 32, supra, the OpenAI Books2 dataset can be estimated to contain about 294,000 titles. The only “internet-based books corpora” that have ever offered that much material are notorious “shadow library” websites like Library Genesis (aka LibGen), Z-Library (aka B-4ok), Sci-Hub, and Bibliotik. The books aggregated by these websites have also been available in bulk via torrent systems. These flagrantly illegal shadow libraries have long been of interest to the AI-training community: for instance, an AI training dataset published in December 2020 by EleutherAI called “Books3” includes a recreation of the Bibliotik collection and contains nearly 200,000 books. On information and belief, the OpenAI Books2 dataset includes books copied from these “shadow libraries,” because those are the most sources of trainable books most similar in nature and size to OpenAI’s description of Books2.
Meta
Bibliotik is one of a number of notorious “shadow library” websites that also includes Library Genesis (aka LibGen), Z-Library (aka B-ok), and Sci-Hub. The books and other materials aggregated by these websites have also been available in bulk via torrent systems. These shadow libraries have long been of interest to the AI-training community because of the large quantity of copyrighted material they host. For that reason, these shadow libraries are also flagrantly illegal.
Absolutely peeved that according to laws: Libraries in a digital format literally cannot exist without being illegal. Archive.org only managed to exist as a Library because they enforced DRM which limited available rentals to the books they “bought” and had copies of.
This is because physical Libraries allow you to borrow their own copies, thus you can even read copyrighted material without asking for permission from the rights holder. So they could argue in court that the DRM only emulated the real thing.
Come COVID and they decide to be nice to people by temporarily stripping the rental bullocks. Their reward for a good deed is a sledgehammer to the stomach.
It matters not, books shall be, and remain forever free (For those that need them). One way or another. All I know is that I’ll never buy a book if I’m treated as a criminal.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]
⚓ A community devoted to in-depth debate on topics concerning digital piracy, ethical problems, and legal advancements.
Absolutely peeved that according to laws: Libraries in a digital format literally cannot exist without being illegal. Archive.org only managed to exist as a Library because they enforced DRM which limited available rentals to the books they “bought” and had copies of.
This is because physical Libraries allow you to borrow their own copies, thus you can even read copyrighted material without asking for permission from the rights holder. So they could argue in court that the DRM only emulated the real thing.
Come COVID and they decide to be nice to people by temporarily stripping the rental bullocks. Their reward for a good deed is a sledgehammer to the stomach.
It matters not, books shall be, and remain forever free (For those that need them). One way or another. All I know is that I’ll never buy a book if I’m treated as a criminal.