I'm working on a personal project utilizing data from the NEO Earth Close Approaches database. I'm cleaning/prepping the dataset with pandas to be used within a Tableau Viz, but I am very curious to hear how others would approach one particular issue.
When viewing the database in browser, the table includes the column "View CA", which links out to an Orbit Viewer showing each close approach. Downloading the CSV or Excel file does not provide these links. I would like to have these links in my dataframe.
I have figured out the format of the link through inspect element; for example, the View CA link for Object (2023 LD):
At the end of the link, you can see that ...desig=2023%20LD&cajd=2460104.544743703& uses the Object/designation (desig=2023%20LD... from 2023 LD) and cajd=... uses the object's close approach date in Julian Date format, which I have created a conversion function to generate.
Firstly, is my outlined solution a sane way to go about this? Secondly, how would you approach this challenge yourself? My main curiosity as I stated is whether there are other, possibly simpler ways to do this as I took way too much time to figure out my solution where it currently is.
Thanks!
So far my solution has been to use regex functions and the cajd conversion function to generate the links based on the data available from the downloadable format.
Looking at the source code of the page, the URLs are constructed on the fly via JavaScript. The Js function that constructs the links looks like this (you can find it through the Web Developer Tools):
$scope.data.onRenderView = function (desig, jd, dist) {
// console.log('DEBUG: enter onRenderView()...'); // TESTING
//
// Set the viewer scale based on the approach distance:
var scale = 0.1;
if ( dist < 0.02 ) { scale = 1.0; }
else if ( dist < 0.05 ) { scale = 0.5; }
else if ( dist < 0.1 ) { scale = 0.2; }
//
// Set the orbit-viewer query-parameters string:
var ov_par = '';
ov_par += '#load=&orientation=0,0,0,1';
// ov_par += '&viewfrom=above';
ov_par += '&lookat=Earth';
ov_par += '&interval=2';
ov_par += '&eclipticaxis=false';
ov_par += '&eclipticgrid=false';
ov_par += '&largeFont=' + isSafari;
ov_par += '&distance=29919.57414';
ov_par += '&pitch=0&roll=0&yaw=0';
ov_par += '&scale=' + scale.toFixed(3);
ov_par += '&rotateX=-30.20870289631195';
ov_par += '&rotateY=38.134339235185024';
ov_par += '&desig=DESIG';
ov_par += '&cajd=JD&';
ov_par = '/ca/ov/' + encodeURI(ov_par);
ov_par = ov_par.replace('DESIG', desig);
ov_par = ov_par.replace('JD', jd);
$sce.trustAsResourceUrl(ov_par);
//
// Remove any prior viewer then add the new viewer to the DOM:
$('#orbit-viewer-iframe').remove();
var iframe = '<iframe id="orbit-viewer-iframe" style="width: 1000px; height: 600px;" src="';
iframe += ov_par;
iframe += '"></iframe>';
$('#orbit-viewer-container').append(iframe);
//
// Set a flag to show the new viewer:
$scope.data.orbitViewerShow = true;
};
So to get all links you need 3 parameters desig, jd, dist. The page gets them via Ajax call. So to construct the links you can use next example:
import urllib
import requests
import pandas as pd
def get_link(desig, jd, dist):
scale = 0.1
if dist < 0.02:
scale = 1.0
elif dist < 0.05:
scale = 0.5
elif dist < 0.1:
scale = 0.2
ov_par = ''
ov_par += '#load=&orientation=0,0,0,1'
# ov_par += '&viewfrom=above'
ov_par += '&lookat=Earth'
ov_par += '&interval=2'
ov_par += '&eclipticaxis=false'
ov_par += '&eclipticgrid=false'
ov_par += '&largeFont=false' #+ str(isSafari)
ov_par += '&distance=29919.57414'
ov_par += '&pitch=0&roll=0&yaw=0'
ov_par += '&scale=' + format(scale, '.3f')
ov_par += '&rotateX=-30.20870289631195'
ov_par += '&rotateY=38.134339235185024'
ov_par += '&desig=DESIG'
ov_par += '&cajd=JD&'
ov_par = '/ca/ov/' + ov_par
ov_par = ov_par.replace('DESIG', urllib.parse.quote(desig))
ov_par = ov_par.replace('JD', str(jd))
return 'https://cneos.jpl.nasa.gov' + ov_par
api_url= 'https://ssd-api.jpl.nasa.gov/cad.api?diameter=1&dist-max=0.05&fullname=1&nea-comet=1&rating=1&www=1'
data = requests.get(api_url).json()
df = pd.DataFrame(data['data'], columns=data['fields'])
df['jd'] = pd.to_numeric(df['jd'])
df['dist'] = pd.to_numeric(df['dist'])
for des, jd, dist in zip(df['des'], df['jd'], df['dist']):
print(get_link(des.split('|')[-1], jd, dist))
Prints:
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=1.000&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=2023%20LD&cajd=2460104.544743703&
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=0.500&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=2023%20JB3&cajd=2460105.360445169&
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=1.000&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=2023%20LA&cajd=2460107.524286449&
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=0.500&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=488453&cajd=2460107.536744452&
...
The df looks like this:
des orbit_id jd cd dist dist_min dist_max v_rel v_inf t_sigma_f h diameter diameter_sigma fullname rating
0 bK23L00D|2023 LD 2 2460104.544743703 2023-Jun-09 01:04 0.00635081714223801 0.0063401113338132 0.00636152260566893 9.10079205473851 9.05457441910954 < 00:01 26.371 None None (2023 LD) 0
1 bK23J03B|2023 JB3 13 2460105.360445169 2023-Jun-09 20:39 0.0362163376263994 0.0361880148384531 0.0362446602808189 6.94939728103983 6.93880250289472 < 00:01 24.174 None None (2023 JB3) 0
2 bK23L00A|2023 LA 3 2460107.524286449 2023-Jun-12 00:35 0.0044093539762013 0.00439127221463214 0.00442743413650688 10.4456580761954 10.3876472288082 < 00:01 25.208 None None (2023 LA) 1
3 a0488453|488453 93 2460107.536744452 2023-Jun-12 00:53 0.0211423059659372 0.0211421868604232 0.0211424250714549 21.4724651802065 21.4665951888416 < 00:01 19.28 None None 488453 (1994 XD) 2
4 bK22W04N|2022 WN4 21 2460108.897696208 2023-Jun-13 09:33 0.0276624541212163 0.0275584911929838 0.027766703533987 15.0756063294164 15.0692157693588 00:08 21.76 None None (2022 WN4) 1
...