Rom Scraper
  • mholgatemmholgatem
    Hey guys and gals,

    As many of you know, we have been working hard rebuilding the rom scraper. Well it's just about ready, and I just wanted to give everyone a heads up. While we have worked to include as many crc values as possible, to easily identify wrongly named files; the best way to get proper matches with the scraper is to have properly named roms. So you may want to run your roms through something like clrmamepro if they didn't all come from the same source.

    A few bits about this scraper:
    • FBA, MAME, and NeoGeo roms have to be named corresponding to the mame naming conventions (example - 'Cadillacs and Dinosaurs' is 'dinos.zip')
    • All of the other systems run each rom file name through an algorithm. The algorithm runs like this:
      Original Filename: 'Adventures of Lolo III (U) [!p].nes'
      First pass, strip rom tags + extension: 'Adventures of Lolo III'
      Second pass, return keywords: 'adventures lolo iii'
      Third pass, convert to search terms: 'adventures lolo 3'
      Finally: compare all titles with these search terms

       So, as you can see: You don't need perfectly named files, but they should contain at least the main search terms that you want the program to compare.

    • The scraper is completely locally based, which means you don't need an internet connection to run it.
    • All images need to be located in: /path/to/rom/folder/images/ (example: /home/pi/pimame/roms/nes/images/)
    • The images must be located there PRIOR to scraping
    • Images are found in a 2 tier process:
      • First, it will look for an image named exactly the same as the file name
      • Second, it will look through it's image database for a match

    • You can download the database images from the links at the bottom and include them in your /path/to/rom/folder/images
    • If there is a change to the number of files in a rom folder, you will be prompted to run the scraper.
      • If you choose 'yes', it will only try to update the new entries (and games that previously had no match)
      • If you choose 'no', it will ask if you want to add the new games to the database without scraping.

    • Run the scraper from the command line like this: python /home/pi/pimame/pimame-menu/scraper/scrape_script.py
      • command line options:
      • --platform #, run specified platform only [--platform 13]
      • --crc True/False, compare crc values to get rom name [--crc True]
      • --match_rate .85, set minimum ratio to be considered a match [--match_rate .85]

      • --verbose True/False/'FULL'/'SEMI', display progress/prompt matches [--verbose 'FULL']
      • --clean_slate True/False, re-scrape ALL roms in folder, not just new/unmatched [--clean_slate True]
      • --dont_match True/False, True=just add files to database, don't find match [--dont_match False]



    Box cover images
    Cartridge .png images
  • mholgatemmholgatem
    Hey guys, I just noticed that I screwed up and the scraper is only finding images that are exactly named. To fix this, you just need to edit 1 line of code in
    '/home/pi/pimame/pimame-menu/scraper/scrape_script.py'


    line 434 needs to be changed from:
    ...best_match_game[0], best_match_game[3]...
    to:
    ...best_match_game[3], best_match_game[0]...

    FIXED IN VERSION 8BETA9
Add a Comment

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In Apply for Membership

Categories

Recommended Accessories