I have a digital subscription to a textbook, but it’s super annoying to have to use the website to access the book. I’d like to scrape the ebook and dump the contents into a pdf. I have downloaded proprietary pdfs from websites before using downloader browser plugins and predictable urls, but this site is pretty locked down, with randomly generated url tokens and a combination of xml and image data.
Has anyone managed to scrape a digital textbook like this? Any ideas where I should begin?
I had a few books like that that were directly on a scummy academic editors website. No pdf or usable files. I’m currently far from home, so I can’t tell you exacly what program i used. But i noticed that every page was downloaded in my temporary files as image data (cached version on page). So i had to manually flip a few pages, download them 1 by 1 and naming them correctly. I’ll look ok my pc to try to find the program that did that when I’m back
I can print pages to PDF without a VM. The problem with printing is that these books are over 1000 pages, so I need to automate a good chunk of the process. Ideally, I’d like to capture the XML text for the pdf as well as it will look much better and I will not have to manually crop 1000 PDFs with annoying borders.
Yeah, I believe you can do that by printing to a non-existent printer and then finding the file image waiting in the print queue. I don’t know if it works on Windows 11 but it used to work pretty well.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: [email protected]
⚓ A community devoted to in-depth debate on topics concerning digital piracy, ethical problems, and legal advancements.
I had a few books like that that were directly on a scummy academic editors website. No pdf or usable files. I’m currently far from home, so I can’t tell you exacly what program i used. But i noticed that every page was downloaded in my temporary files as image data (cached version on page). So i had to manually flip a few pages, download them 1 by 1 and naming them correctly. I’ll look ok my pc to try to find the program that did that when I’m back
Sounds like you could also use a image downloader browser extension for that
Sounds promising! Please let me know what you find.
It was MZCacheview but the same autor made one for chrome and a general one. But theoware is probable right, a brower extension could also do it!
Looks like this particular publisher has anticipated cache sniffing. No dice.
You can try printing the page
I’m looking for something a bit more detailed. I’d like to auto-scrape the entire book.
Then this method probably won’t work for you
Why don’t you simply open the book in a virtual machine like VMware and hit print? It can print to a PDF.
I can print pages to PDF without a VM. The problem with printing is that these books are over 1000 pages, so I need to automate a good chunk of the process. Ideally, I’d like to capture the XML text for the pdf as well as it will look much better and I will not have to manually crop 1000 PDFs with annoying borders.
Yeah, I believe you can do that by printing to a non-existent printer and then finding the file image waiting in the print queue. I don’t know if it works on Windows 11 but it used to work pretty well.