feat: Add Bookys scraper alongside EBoek with startup mode chooser
Adds a second scraper for bookys-ebooks.com that harvests direct 1fichier download URLs into a .txt file. The existing EBoek scraper core is unchanged; only the GUI gained a Mode menu entry to switch between the two. Bookys specifics: - Visible (never headless) browser, since the site is behind Cloudflare. The challenge is auto-detected: the scraper polls for real listing items and continues on its own once it clears, no button needed. - Persistent Chrome profile in ~/.eboek_scraper/bookys_chrome_profile so the Cloudflare clearance cookie survives between runs. Removed the hardcoded user-agent override, which claimed Chrome/120 against a real Chrome/150 and invalidated that cookie on every launch. - Pop-up ads are avoided by never clicking host links: hrefs are read and navigated to directly, with window.open neutered via CDP as a backstop. - Host links are read with textContent rather than Selenium's .text, which returns empty for these elements because they sit in a collapsed container. - Only 1fichier hosts are followed; other hosts on a page are ignored. Build configs list the new modules as hidden imports in all four places. gui_main imports the Bookys window lazily, so PyInstaller's static analysis would otherwise miss it and the .exe would crash on switching modes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVJc2XuTvqBHSuqotetseS
This commit is contained in:
58
gui/startup_dialog.py
Normal file
58
gui/startup_dialog.py
Normal file
@@ -0,0 +1,58 @@
|
||||
"""
|
||||
Startup chooser: pick which site to scrape when the app boots.
|
||||
"""
|
||||
|
||||
from PyQt5.QtWidgets import (
|
||||
QDialog, QVBoxLayout, QHBoxLayout, QPushButton, QLabel
|
||||
)
|
||||
from PyQt5.QtCore import Qt
|
||||
|
||||
|
||||
class StartupDialog(QDialog):
|
||||
"""Small modal shown on boot to choose EBoek or Bookys."""
|
||||
|
||||
EBOEK = "eboek"
|
||||
BOOKYS = "bookys"
|
||||
|
||||
def __init__(self, parent=None):
|
||||
super().__init__(parent)
|
||||
self.choice = None
|
||||
self._init_ui()
|
||||
|
||||
def _init_ui(self):
|
||||
self.setWindowTitle("Choose Scraper")
|
||||
self.setModal(True)
|
||||
self.setMinimumWidth(420)
|
||||
|
||||
layout = QVBoxLayout(self)
|
||||
layout.setSpacing(18)
|
||||
layout.setContentsMargins(28, 28, 28, 28)
|
||||
|
||||
title = QLabel("Which site do you want to scrape?")
|
||||
title.setStyleSheet("font-size: 16px; font-weight: bold;")
|
||||
title.setAlignment(Qt.AlignCenter)
|
||||
layout.addWidget(title)
|
||||
|
||||
subtitle = QLabel("You can switch anytime from the menu bar.")
|
||||
subtitle.setAlignment(Qt.AlignCenter)
|
||||
subtitle.setStyleSheet("color: #666;")
|
||||
layout.addWidget(subtitle)
|
||||
|
||||
buttons = QHBoxLayout()
|
||||
buttons.setSpacing(14)
|
||||
|
||||
eboek_btn = QPushButton("EBoek.info\n\nDownload comic files")
|
||||
eboek_btn.setMinimumHeight(90)
|
||||
eboek_btn.clicked.connect(lambda: self._choose(self.EBOEK))
|
||||
buttons.addWidget(eboek_btn)
|
||||
|
||||
bookys_btn = QPushButton("Bookys\n\nHarvest 1fichier links → .txt")
|
||||
bookys_btn.setMinimumHeight(90)
|
||||
bookys_btn.clicked.connect(lambda: self._choose(self.BOOKYS))
|
||||
buttons.addWidget(bookys_btn)
|
||||
|
||||
layout.addLayout(buttons)
|
||||
|
||||
def _choose(self, choice):
|
||||
self.choice = choice
|
||||
self.accept()
|
||||
Reference in New Issue
Block a user