python – Using BeautifulSoup to call a JAVA function

python – Using BeautifulSoup to call a JAVA function

The code that the page execute is JavaScript more specific AJAX, I recommend you use Selenium to make this work, mainly because this up a browser and with this you can make a program to make a click in this link and load the AJAX call and then scrap the content. This is the more accurate solution. Selenium is available for a lot of languages like JAVA, C#, Python, etc.

If you dont want to use Selenium instead you can see the XHTML request browser do and obtain directly the CSV, I think. You can see this using Chrome pressing F12 for view the developer tool or installing Firebug for Firefox, all in the tag network.

I am not familiar with BeautifulSoup and know very little Python, but I have dabbled in trying to scrape profootball reference in java and JSoup and then later HtmlUnit…

JSoup, and likely BeautifulSoup (as they are similar according to my recent google search), are not designed to invoke javascript functions.

Additionally, the page does not invoke a network request when the CSV link is invoked. Therefore, there is no known url that can be invoked to obtain the data in CSV format. The table2csv function in javascript creates the csv data from the html table data.

Your best option is to do as the javascript table2csv function does. Take the table data, obtainable via BeautifulSoup, and parse that directly.

python – Using BeautifulSoup to call a JAVA function

Leave a Reply

Your email address will not be published. Required fields are marked *