Search code examples
jsonpython-2.7pandaspython-requestsurllib2

Recieve JSON data from link in whattomine without scraping HTML


Explanation

This link is where you are sent to after entering in your hardware stats (hashrate, power, power cost, etc.). On the top bar (below the blue Twitter follow button) is a link to a JSON file created after the page loads with the hardware stats information entered; clicking on that JSON link redirects you to another URL (https://whattomine.com/asic.json).

Goal

My goal is to access that JSON file directly after manipulating the values in the URL string via the terminal. For example, if I would like to change hashrate from 100 to 150 in this portion of the URL:

[sha256_hr]=100& ---> [sha256_hr]=150&

After the URL manipulations (like above, but not limited to), I would like to receive the JSON output so that I can pick-out the desired data.

My Code

Advisory: I started Python programming ~June 2017, please forgive.

import json
import pandas as pd
import urllib2
import requests


hashrate_ghs = float(raw_input('Hash Rate (TH/s): '))
power_W = float(raw_input('Power of Miner (W): '))
electric_cost = float(raw_input('Cost of Power ($/kWh): '))
hashrate_ths = hashrate_ghs * 1000

initial_request = ('https://whattomine.com/asic?utf8=%E2%9C%93&sha256f=true&factor[sha256_hr]={0}&factor[sha256_p]={1}&factor[cost]={2}&sort=Profitability24&volume=0&revenue=24h&factor[exchanges][]=&factor[exchanges][]=bittrex&dataset=Main&commit=Calculate'.format(hashrate_ths, power_W, electric_cost))
data_stream_mine = urllib2.Request(initial_request)

json_data = requests.get('https://whattomine.com/asic.json')
print json_data

Error from My Code

I am getting an HTTPS handshake error. This is where my Python freshness is second most blatantly visible:

Traceback (most recent call last):
  File "calc_1.py", line 16, in <module>
    s.get('https://whattomine.com/asic.json')
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='whattomine.com', port=443): Max retries exceeded with url: /asic.json (Caused by SSLError(SSLError(1, u'[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)'),))

Thank you for your help and time!

Please advise me of any changes or for more information concerning this question.


Solution

  • This is just a comment. The following approach would suffice (Python 3).

    import requests
    
    initial_request = 'http://whattomine.com/asic.json?utf8=1&dataset=Main&commit=Calculate'
    
    json_data = requests.get(initial_request)
    print(json_data.json())
    

    The key point here this part - put .json in your initial_request and it will be enough. You may add all you parameters as you did in the query part after ? sign