Accessing and downloading (xml, pdf ) files from handelsregister API

[timtensor]

Hi @wirthual ,
I am trying the following workflow

• Using the advanced search query : https://www.handelsregister.de/rp_web/erweitertesuche.xhtml i do the following
  a) Choose Federal States - Berlin / Bavaria
  b) Company / search words - "Hallo"
  c) Choose the option "contain all words"
  d ) Type of registar "HRA"
  e ) And choose 100 hits per list .

• This will give us a search list as shown in the screen shot

• I want to then downlaod SI or DK content of all the hits .
  I am wondering if that is possible or not ?
  Screenshot :
  

Please let me know what could be the best way forward

[melihsunbul]

Hello, did you able to figure a way out?

[timtensor]

Hi unfortunately not , since it has to come up from the team it is currently kind of blocked

[melihsunbul]

Hi unfortunately not , since it has to come up from the team it is currently kind of blocked

Thanks

[monkeygopro]

interested in other solutions? or do you want to use this one specifically?

[melihsunbul]

interested in other solutions? or do you want to use this one specifically?

I am open to hear other alternatives if there is any.

[monkeygopro]

I don't know what your project is about or what you are gonna do with the data, but Selenium looks like a very good option. Contact me if you want further information.

[timtensor]

As I am using a colab notebook , I think an API would be the best right ? I think the colab notebook doesn't support selenium .

The project basically collects data to analyze different companies .

[melihsunbul]

Yeah, I agree. I already tried an alternative to selenium which is playwright, but it has limitations that come together with scraping the website.

[monkeygopro]

what limitations do you mean?

[melihsunbul]

IP blocks, for example.

[monkeygopro]

Even if you are using this API your IP will be blocked if you send more than 60 Requests per hour. for avoiding getting blocked, you have to rotate your proxies. How many requests are you planing to use in your project?

[melihsunbul]

I already tried to use proxies by rotating, but in that case, the website is not being loaded in a reasonable time. I am planning to send above 200 requests periodically.

[monkeygopro]

hmm, you can modify the code handelsregister.py and send post requests within the form "ergebnisseform" in .../ergebnisse.xhtml !but you have also to deal with javax.faces.ViewState its readonly and you can't control it.

[timtensor]

Oh ok , I think the API just makes it a bit easier and is there for a reason , I guess but I think it is not maintained

[muhmtayyab]

I tried and found the solution, I found their API to get the document but that API required a cookie (SESSIONID). Which is not accessible using javascript. Do you have any solution to get the cookie?