Search code examples
javascriptarraysd3.jsbubble-chart

Is there a way to fetch data from CSV where radius is formed from json file so that Bubble can be resized with CSV data as per button click?

I have created a World Bubble Map where the bubbles are formed according to geolocation of the countries and bubble radius (size) should change according to selected parameters from radio button.

For instance, if population is selected, the bubble size should be formed according to population size of every nation and by selecting the next button it should change as per selected parameter.

So far I have managed to form a Bubble Map which reads the data from JSON file to plot the first bubbles, but I am stuck on how to make it read my other parameters from my CSV file, as all the necessary data that needs to be visualized is within a separate CSV file to the JSON file that the geolocation data comes from.

Is there a way, that I can link my CSV file to create the bubbles as per the countries geolocation and parameters from CSV file. All my coding has been done in this Observable Notebook

Solution

  • Yes there is, you have already demonstrated code that actively re-renders when the selected value from a radio button list is changed. The problem with your current code has nothing to do with bubbles or radius, it is a simple data referencing issue, importantly, how to merge arrays of data from multiple sources in javascript.

    NOTE: A post in this format doesn't work very well on SO, a link to a fiddle or observable is a great secondary resource, but your post on SO needs to be focused on the specific issue. Going to that effort usually helps you identify the solution as part of the process. In this solution I'll take you through my debugging effort.

    First, lets have a look at your current attempt, to show the value of the currently selected property from the radio buttons. We start here because it shows your current understanding of the code and how the data is loaded into memory. Before worrying about the bubbles, that you have already demonstrated how to render lets solve the data issue first.

    Tooltip Screenshot

    Your current tooltip content is not displaying the value for the selected property (in yellow), it is however displaying the label of the selected option (blue underline). lets dig a bit closer, this is the current content logic:

    <div>
  <h3>${hover.NAME}</h3>
  <div style="margin: 10px">
    <h4>CountryCode : ${hover.ISO_A2}  </h4> 
    <h4> ContinentCode : ${hover.CONTINENT}  </h>
    <h4>${parameter } : ${d3.format(",")(hover[p])}</h4>
  </div>
</div>

    in this case hover is an object that has properties ISO_A2 and CONTINENT which are not properties from your referenced CSV file. This is properties from the countries.json

    To confirm this and to quickly explore the properties that are available we can make this simple change to the tooltip content, we want to verify the value of p so we can make sure it exists in the expected dataset, then we want to explore the structure of hover. When I need to quickly explore the structure of objects at a point in time in the code and I can't easily step through and debug the script, using JSON.stringify() to include content in the output is a quick and dirty solution:

    <div>
  <h3>${hover.NAME}</h3>
  <div style="margin: 10px">
    <h4>CountryCode : ${hover.ISO_A2}  </h4> 
    <h4> ContinentCode : ${hover.CONTINENT}  </h>
    <h4>${parameter } : ${d3.format(",")(hover[p])}</h4>
    <h4>Expected Property: ${p}</h4>
    <smaller>${JSON.stringify(hover)}</smaller>
  </div>
</div>

    console.log() is a better option if you need to capture the data rather than simply dumping it into the screen as content to visually inspect it.

    Tooltip with hover stringified

    It is clear that the original intent was to have the data available as part of the data feed available at that point, it is also clear that nothing has been done to merge these two datasets, so lets do that now.

    My preference in this case is to use forEach to iterate over countries and modify each entry to include the entire matching record from the csv file. You could use map but in this case we are already loading heaps of data into memory, we don't need to retain the unchanged version of the countries JSON as well as the new output that map would have produced.

    The other reason that I recommend injecting the data from the CSV into the array of countries is that we can avoid the overhead of a lookup when we need to access the data. Arranging the csv data and the country data into arrays of the same length and sorted so that the same indexes correlate to the same country in each file incurs the same over head as injecting the data and makes it more deterministic. It is also easier to support scenarios where one array has more values that the other.

    All we need to know is the property mapping to identity the unique record in each file. We can see that the csv file is using 2 letter country codes:

    country_id CountryName CountryCode ContinentCode CenterLongitude CenterLatitude areasqkm population airports gdpgrowthrate inflationrate unemploymentrate popnbelowpoverty medianage
    12 Australia AU OC 135.0 -25.0 7741220 26141369 418 1.84 1.60 5.16 0.00 37.50
    13 Austria AT EU 13.33 47.33 83871 8913088 50 1.42 1.50 7.35 13.30 44.50
    114 India IN AS 77.0 20.0 3287263 1389637446 346 4.86 3.70 8.50 21.90 28.70
    115 Indonesia ID AS 120.0 -5.0 1904569 277329163 673 5.03 2.80 5.31 9.40 31.10

    I have picked two pairs Australia, Austria and India, Indonesia for comparison as they cover common matching issues, Most countries have a 2 letter postal code, and it is common for people to pick this first, however India's postal code is 3 letters, Indonesia's is 4 and Austria have 1. I find this dataset particularly interesting for how close the name terms are lexicographically yet how different their unique codes are, so I always check with these specific cases first before running a test or proof across an entire dataset.

    The feature property metadata in the json file has a lot of information, importantly, it also has the ISO_A2 value, which is the ISO standard 2 letter country code.

    featurecla scalerank LABELRANK SOVEREIGNT SOV_A3 ADM0_DIF LEVEL TYPE TLC ADMIN ADM0_A3 GEOU_DIF GEOUNIT GU_A3 SU_DIF SUBUNIT SU_A3 BRK_DIFF NAME NAME_LONG BRK_A3 BRK_NAME BRK_GROUP ABBREV POSTAL FORMAL_EN FORMAL_FR NAME_CIAWF NOTE_ADM0 NOTE_BRK NAME_SORT NAME_ALT MAPCOLOR7 MAPCOLOR8 MAPCOLOR9 MAPCOLOR13 POP_EST POP_RANK POP_YEAR GDP_MD GDP_YEAR ECONOMY INCOME_GRP FIPS_10 ISO_A2 ISO_A2_EH ISO_A3 ISO_A3_EH ISO_N3 ISO_N3_EH UN_A3 WB_A2 WB_A3 WOE_ID WOE_ID_EH WOE_NOTE ADM0_ISO ADM0_DIFF ADM0_TLC ADM0_A3_US ADM0_A3_FR ADM0_A3_RU ADM0_A3_ES ADM0_A3_CN ADM0_A3_TW ADM0_A3_IN ADM0_A3_NP ADM0_A3_PK ADM0_A3_DE ADM0_A3_GB ADM0_A3_BR ADM0_A3_IL ADM0_A3_PS ADM0_A3_SA ADM0_A3_EG ADM0_A3_MA ADM0_A3_PT ADM0_A3_AR ADM0_A3_JP ADM0_A3_KO ADM0_A3_VN ADM0_A3_TR ADM0_A3_ID ADM0_A3_PL ADM0_A3_GR ADM0_A3_IT ADM0_A3_NL ADM0_A3_SE ADM0_A3_BD ADM0_A3_UA ADM0_A3_UN ADM0_A3_WB CONTINENT REGION_UN SUBREGION REGION_WB NAME_LEN LONG_LEN ABBREV_LEN TINY HOMEPART MIN_ZOOM MIN_LABEL MAX_LABEL LABEL_X LABEL_Y NE_ID WIKIDATAID NAME_AR NAME_BN NAME_DE NAME_EN NAME_ES NAME_FA NAME_FR NAME_EL NAME_HE NAME_HI NAME_HU NAME_ID NAME_IT NAME_JA NAME_KO NAME_NL NAME_PL NAME_PT NAME_RU NAME_SV NAME_TR NAME_UK NAME_UR NAME_VI NAME_ZH NAME_ZHT FCLASS_ISO TLC_DIFF FCLASS_TLC FCLASS_US FCLASS_FR FCLASS_RU FCLASS_ES FCLASS_CN FCLASS_TW FCLASS_IN FCLASS_NP FCLASS_PK FCLASS_DE FCLASS_GB FCLASS_BR FCLASS_IL FCLASS_PS FCLASS_SA FCLASS_EG FCLASS_MA FCLASS_PT FCLASS_AR FCLASS_JP FCLASS_KO FCLASS_VN FCLASS_TR FCLASS_ID FCLASS_PL FCLASS_GR FCLASS_IT FCLASS_NL FCLASS_SE FCLASS_BD FCLASS_UA
    Admin-0 country 1 2 Australia AU1 1 2 Country 1 Australia AUS 0 Australia AUS 0 Australia AUS 0 Australia Australia AUS Australia Auz. AU Commonwealth of Australia Australia Australia 1 2 2 7 25364307 15 2019 1396567 2019 2. Developed region: nonG7 1. High income: OECD AS AU AU AUS AUS 36 36 36 AU AUS -90 23424748 Includes Ashmore and Cartier Islands (23424749) and Coral Sea Islands (23424790). AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS AUS -99 -99 Oceania Oceania Australia and New Zealand East Asia & Pacific 9 9 4 -99 1 0 1.7 5.7 134.04972 -24.129522 1159320355 Q408 أستراليا অস্ট্রেলিয়া Australien Australia Australia استرالیا Australie Αυστραλία אוסטרליה ऑस्ट्रेलिया Ausztrália Australia Australia オーストラリア 오스트레일리아 Australië Australia Austrália Австралия Australien Avustralya Австралія آسٹریلیا Úc 澳大利亚 澳大利亞 Admin-0 country Admin-0 country
    Admin-0 country 1 4 Austria AUT 0 2 Sovereign country 1 Austria AUT 0 Austria AUT 0 Austria AUT 0 Austria Austria AUT Austria Aust. A Republic of Austria Austria Austria 3 1 3 4 8877067 13 2019 445075 2019 2. Developed region: nonG7 1. High income: OECD AU AT AT AUT AUT 40 40 40 AT AUT 23424750 23424750 Exact WOE match as country AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT AUT -99 -99 Europe Europe Western Europe Europe & Central Asia 7 7 5 -99 1 0 3 8 14.130515 47.518859 1159320379 Q40 النمسا অস্ট্রিয়া Österreich Austria Austria اتریش Autriche Αυστρία אוסטריה ऑस्ट्रिया Ausztria Austria Austria オーストリア 오스트리아 Oostenrijk Austria Áustria Австрия Österrike Avusturya Австрія آسٹریا Áo 奥地利 奧地利 Admin-0 country Admin-0 country
    Admin-0 country 1 2 India IND 0 2 Sovereign country 1 India IND 0 India IND 0 India IND 0 India India IND India India IND Republic of India India India 1 3 2 2 1366417754 18 2019 2868929 2019 3. Emerging region: BRIC 4. Lower middle income IN IN IN IND IND 356 356 356 IN IND 23424848 23424848 Exact WOE match as country IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND IND -99 -99 Asia Asia Southern Asia South Asia 5 5 5 -99 1 0 1.7 6.7 79.358105 22.686852 1159320847 Q668 الهند ভারত Indien India India هند Inde Ινδία הודו भारत India India India インド 인도 India Indie Índia Индия Indien Hindistan Індія بھارت Ấn Độ 印度 印度 Admin-0 country Admin-0 country
    Admin-0 country 3 2 Indonesia IDN 0 2 Sovereign country 1 Indonesia IDN 0 Indonesia IDN 0 Indonesia IDN 0 Indonesia Indonesia IDN Indonesia Indo. INDO Republic of Indonesia Indonesia Indonesia 6 6 6 11 270625568 17 2019 1119190 2019 4. Emerging region: MIKT 4. Lower middle income ID ID ID IDN IDN 360 360 360 ID IDN 23424846 23424846 Exact WOE match as country IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN IDN -99 -99 Asia Asia South-Eastern Asia East Asia & Pacific 9 9 5 -99 1 0 1.7 6.7 101.892949 -0.954404 1159320845 Q252 إندونيسيا ইন্দোনেশিয়া Indonesien Indonesia Indonesia اندونزی Indonésie Ινδονησία אינדונזיה इंडोनेशिया Indonézia Indonesia Indonesia インドネシア 인도네시아 Indonesië Indonezja Indonésia Индонезия Indonesien Endonezya Індонезія انڈونیشیا Indonesia 印度尼西亚 印度尼西亞 Admin-0 country Admin-0 country

    there are lots of different ways to do this, I'm just going to find the matching record and stuff it into a new property called csv using a verbose method to make it easy to follow:

    // Convert the TopoJSON to GeoJSON
countries = {
  const geo = topojson.feature(map_layer, map_layer.objects.countries);
  geo.features.forEach(feature => {
    if(feature.geometry) {
      feature.centroid = centroid(feature);
    }
    let code = feature.properties.ISO_A2;
    let cData = country_data.find(x => x.CountryCode === code);
    feature.properties.csv = cData ?? {
  areasqkm: "",
  population: "",
  airports: "",
  gdpgrowthrate: "",
  inflationrate: "",
  unemploymentrate: "",
  popnbelowpoverty: "",
  medianage: ""
};
    return feature;
  });
  return geo;
}

    Now the data is available, including a null record to simplify error handling later. Lets add this to the tooltip content first, to verify that it works, i'll dump all the extended data just so we can explore the different values:

    <div>
  <h3>${hover.NAME}</h3>
  <div style="margin: 10px">
    <h4>CountryCode : ${hover.ISO_A2}  </h4> 
    <h4> ContinentCode : ${hover.CONTINENT}  </h>
    <h4>${parameter } : ${d3.format(",")(hover.csv[p])}</h4>
    <hr style="padding:0px"/>
    <smaller>
     Area Sqkm: ${d3.format(",")(hover.csv['areasqkm'])}<br/>
     Population: ${d3.format(",")(hover.csv['population'])}<br/>
     Airports: ${hover.csv['airports']}<br/>
     GDP Growth Rate: ${d3.format(",")(hover.csv['gdpgrowthrate'])}<br/>
     Inflation Rate: ${d3.format(",")(hover.csv['inflationrate'])}<br/>
     Unemployment Rate: ${d3.format(",")(hover.csv['unemploymentrate'])}<br/>
     Pop. Below Poverty: ${d3.format(",")(hover.csv['popnbelowpoverty'])}<br/>
     Median Age: ${hover.csv['medianage']}<br/>
    <smaller>
  </div>
</div>

    Extended Tooltips

    Now you just need to apply those property mapping paths to your bubble logic, I'm actually going to leave that logic out here, the main issue was how to make the data available, there are practicality issues to using the direct value to render proportional shapes. Very quickly you end up with some items so small you can't click or see them, other shapes will have such large discrepancies that functions like forceCollide() will just make a mess... have a look at zimbabwe's inflation rate in this bubble map:

    https://observablehq.com/@chrisschaller/worldbubblemap

    enter image description here

    It is often more practical to map values to specific size domains, a scale of 1-5 is usually enough but you can go to 10, where 1 is the smallest on the map, and 10 the largest, then trim the data range into standard deviations so that outliers below all get 1 and outliers above all get the same max.

    The techniques and reasoning are well outside of this discussion space.