feat: Add Bookys scraper alongside EBoek with startup mode chooser
Adds a second scraper for bookys-ebooks.com that harvests direct 1fichier download URLs into a .txt file. The existing EBoek scraper core is unchanged; only the GUI gained a Mode menu entry to switch between the two. Bookys specifics: - Visible (never headless) browser, since the site is behind Cloudflare. The challenge is auto-detected: the scraper polls for real listing items and continues on its own once it clears, no button needed. - Persistent Chrome profile in ~/.eboek_scraper/bookys_chrome_profile so the Cloudflare clearance cookie survives between runs. Removed the hardcoded user-agent override, which claimed Chrome/120 against a real Chrome/150 and invalidated that cookie on every launch. - Pop-up ads are avoided by never clicking host links: hrefs are read and navigated to directly, with window.open neutered via CDP as a backstop. - Host links are read with textContent rather than Selenium's .text, which returns empty for these elements because they sit in a collapsed container. - Only 1fichier hosts are followed; other hosts on a page are ignored. Build configs list the new modules as hidden imports in all four places. gui_main imports the Bookys window lazily, so PyInstaller's static analysis would otherwise miss it and the .exe would crash on switching modes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NVJc2XuTvqBHSuqotetseS
This commit is contained in:
37
gui_main.py
37
gui_main.py
@@ -59,7 +59,8 @@ class EBoekScraperApp(QApplication):
|
||||
# Handle exceptions
|
||||
sys.excepthook = self.handle_exception
|
||||
|
||||
self.main_window = None
|
||||
self.main_window = None # EBoek window
|
||||
self.bookys_window = None # Bookys window
|
||||
|
||||
def set_application_icon(self):
|
||||
"""Set the application icon if available."""
|
||||
@@ -101,9 +102,16 @@ class EBoekScraperApp(QApplication):
|
||||
# Check system requirements
|
||||
self.check_requirements()
|
||||
|
||||
# Create and show main window
|
||||
self.main_window = MainWindow()
|
||||
self.main_window.show()
|
||||
# Ask which site to scrape, then show the matching window.
|
||||
from gui.startup_dialog import StartupDialog
|
||||
chooser = StartupDialog()
|
||||
chooser.exec_()
|
||||
|
||||
if chooser.choice == StartupDialog.BOOKYS:
|
||||
self.show_bookys()
|
||||
else:
|
||||
# Default to EBoek (also covers the dialog being closed).
|
||||
self.show_eboek()
|
||||
|
||||
return True
|
||||
|
||||
@@ -124,6 +132,27 @@ class EBoekScraperApp(QApplication):
|
||||
|
||||
return False
|
||||
|
||||
def show_eboek(self):
|
||||
"""Show the EBoek window, hiding the Bookys one if present."""
|
||||
if self.main_window is None:
|
||||
self.main_window = MainWindow()
|
||||
if self.bookys_window is not None:
|
||||
self.bookys_window.hide()
|
||||
self.main_window.show()
|
||||
self.main_window.raise_()
|
||||
self.main_window.activateWindow()
|
||||
|
||||
def show_bookys(self):
|
||||
"""Show the Bookys window, hiding the EBoek one if present."""
|
||||
from gui.bookys_window import BookysWindow
|
||||
if self.bookys_window is None:
|
||||
self.bookys_window = BookysWindow()
|
||||
if self.main_window is not None:
|
||||
self.main_window.hide()
|
||||
self.bookys_window.show()
|
||||
self.bookys_window.raise_()
|
||||
self.bookys_window.activateWindow()
|
||||
|
||||
def check_requirements(self):
|
||||
"""Check system requirements and dependencies."""
|
||||
errors = []
|
||||
|
||||
Reference in New Issue
Block a user