Welcome to rpscrape_dataframes Discussions!

[Caprice63]

👋 Welcome!

This code has been duplicated from https://github.com/joenano/rpscrape.
My aim is to amend the code so that it creates a dataframe of the results/racecards that can be easily saved to xls / csv files.

EDIT+++ I've discovered that the results are in csv so I just need to find a way to convert the json racecards to csv.

I'm learning python as I go so please feel free to comment on my errors. I'm also new to Github so bear with me as I learn how to use it.

[Caprice63]

@tastethewhip : I noticed that you have managed to create html pages of the race results in your repo - https://github.com/tastethewhip/tastethewhip.github.io (It looks very impressive and I am grateful for sharing on GH.
I assume you use the https://github.com/joenano/rpscrape scripts to and then convert the json files to html - I'm a relative newbie so please forgive me if my understanding does not make sense.
I hope you don't mind me asking - I am trying to learn python and I've been trying to convert the @joenano json output file into a flat csv/xls file like the one that your html creates - so I was wondering if you had a script that you use to convert from json to html as I may be able to replicate it in python to enable me to concert json to csv/xls.

I know that I could use your files to create the csv but I'd like to be able to do them myself on a daily basis.

Thanks in advance for any pointers you can give me.

[tastethewhip]

Hi @Caprice63 , hope you are well, and welcome to Github :)

• are you using Linux , Windows or 'something else' ? I use linux and when I run 'rpscrape.py' from the command line [inside the 'scripts' folder] any scraping I do ALWAYS saves a {.csv file}. (The 'racecards.py' option is a different issue; I'll come back to that in a moment. ) I tested the rep on your own page and it works the same.
• When I have the {.csv file} I use a program called 'csvtotable' which (if you use Linux) you can download by 'pip install --upgrade csvtotablepip install --upgrade csvtotable' ... on WIndows system, I'm not sure but you can clone the repo from https://github.com/vividvilla/csvtotable
• coming back to the 'racecards.py', it ALWAYS downloads as a json. Best way to deal with them is by 'jq' in linux [there will be a windows equiv; I don't know what it is!]
  I hope that was at least SOME help; feel free to ask anything more.
  Good luck in your Python journey; it is a VERY steep learning curve :)
  Regards

Derek {@tastethewhip}

[tastethewhip]

ps ; the best scrape that works for me (I do this x2 every week) is (for example) '-d 2022/6/28-2022/6/1 all'

Cheers :)

[Caprice63]

Thanks for your reply @tastethewhip . I'm on win 11 pro. I only tried the racecards.py and it saved as json so I assumed the rpscrape.py would be the same - I'll have to give it a try. I did try to search for some way to convert json to csv but could not see anything that looked to be doable with my level of knowledge. I'll try searching an equivalent to 'jq'.
I've downloaded your csv files which is a great help as it will no doubt save me several hours of downloading historical data.
And thanks for the scrape settings - as soon as I get my head round everything I'll probably do a weekly scrape.
Thanks again,
Chris

[Caprice63]

p.s. I just tested the rpscrape.py - fantastic! I get a csv. Thanks @tastethewhip and thanks @joenano for a great tool.
I will let you know how I get on converting the json to csv.

[tastethewhip]

Hi Chris! , I should mention: if you go into settings.. copy the 'default.....toml' to user_settings.toml and THEN play with it. there'e a lot of of stuff you will/wont want. This is the best advice you will ever get lol :)