Skip to content

Fix Cardmarket scraping #456

@mazz3rr

Description

@mazz3rr

The URL works in a browser and contains decklists.

2025-10-25 02:42:32,179 [mtg.deck.scrapers] INFO: Gathered 2 deck tag(s) from a Cardmarket article at: 'https://www.cardmarket.com/en/Insight/Articles/on-video-this-week-2025-43'
2025-10-25 02:42:32,180 [mtg.yt] ERROR: Scraping of channel 'UCwatLpmoIeoL9egqVW2Cxbw' failed with: KeyError('name'). Skipping...
2025-10-25 02:42:32,182 [mtg.yt] ERROR: Traceback (most recent call last):
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/yt/__init__.py", line 748, in 
  scrape_channels
    ch.scrape(videos_limit, only_newer_than_last_scraped, soft_limit)
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/utils/__init__.py", line 57, in 
  wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/yt/__init__.py", line 674, in scrape
    self._scrape_videos(*video_ids)
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/yt/__init__.py", line 611, in 
  _scrape_videos
    scraper.scrape()
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/utils/__init__.py", line 57, in 
  wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/utils/scrape/__init__.py", line 218, 
  in
   
  wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/yt/__init__.py", line 397, in scrape
    self._scrape_decks()
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/yt/__init__.py", line 383, in 
  _scrape_decks
    self._decks = self._collect(links, lines)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/yt/__init__.py", line 370, in 
  _collect
    decks.update(scraper.scrape_decks())
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/utils/__init__.py", line 57, in 
  wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/_venv_3_12/lib/python3
  .12/site-packages/backoff/_sync.py", line 105, in retry
    ret = target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/deck/scrapers/__init__.py", line 747,
   
  in
   scrape_decks
    decks += self._process_deck_tags()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/deck/scrapers/__init__.py", line 517,
   
  in
   _process_deck_tags
    deck = self.TAG_BASED_DECK_PARSER(deck_tag, self._metadata).parse()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/deck/__init__.py", line 1144, in 
  parse
    self._parse_deck()
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/deck/scrapers/cardmarket.py", line 
  124,
   
  in _parse_deck
    self._parse_ul_tags(ul_tags, has_sideboard)
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/deck/scrapers/cardmarket.py", line 
  95,
   
  in _parse_ul_tags
    board += self._parse_li_tag(li_tag)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/Documents/Code/PyCharm/web-scraping/mtg/mtg/deck/scrapers/cardmarket.py", line 
  88, in _parse_li_tag
    name = li_tag.find("hoverable-card").attrs["name"]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'name'

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions