In R/rvest, there are 50 products in the web page, but below code only get top 30 information. How to get handle ? Thanks!
library(rvest)
url <- "https://www.amazon.com/Best-Sellers-Industrial-Scientific-3D-Printers/zgbs/industrial/6066127011/ref=zg_bs_pg_1?_encoding=UTF8&pg=1"
web <- rvest::read_html(url)
web %>% html_node("div.p13n-desktop-grid") %>% html_text() %>% strsplit("#")
I have been able to get the 50 with the following code :
library(RSelenium)
library(rvest)
url <- "https://www.amazon.com/Best-Sellers-Industrial-Scientific-3D-Printers/zgbs/industrial/6066127011/ref=zg_bs_pg_1?_encoding=UTF8&pg=1"
shell('docker run -d -p 4445:4444 selenium/standalone-firefox')
remDr <- remoteDriver(remoteServerAddr = "localhost", port = 4445L, browserName = "firefox")
remDr$open()
remDr$navigate(url)
# go to the bottom of the page to load all the page
for(i in 1 : 30)
{
print(i)
for(j in 1 : 30)
{
js_Script <- paste0("scroll(", (i - 1) * 300, ",", (j - 1) * 300, ");")
remDr$executeScript(js_Script)
}
}
Sys.sleep(5)
htmltxt <- remDr$getPageSource()[[1]]
read_html(htmltxt) %>% html_node("div.p13n-desktop-grid") %>% html_text() %>% strsplit("#")
[[1]]
[1] ""
[2] "1Official Creality Ender 3 V2 Upgraded 3D Printer with Silent Motherboard Meanwell Power Supply Carborundum Glass Platform Resume Printing Function, DIY FDM 3D Printers Build Size 8.66x8.66x9.84 inch4.4 out of 5 stars 5,954-13% $279.00List: $319.99"
[3] "2Official Creality Ender 3 3D Printer Fully Open Source with Resume Printing Function DIY 3D Printers Printing Size 8.66x8.66x9.84 inch4.4 out of 5 stars 7,978-24% $189.00List: $249.99"
[4] "3ANYCUBIC Photon Mono 4K, Resin 3D Printer with 6.23\" Monochrome Screen, Upgraded UV LCD 3D Printer and Fast & Precise Printing, 5.19\" x 3.14\" x 6.49\" Printing Size4.4 out of 5 stars 2,505-30% $196.99Was: $279.99"
[5] "4Official Creality Ender 3 Pro 3D Printer with Removable Build Surface Plate and UL Certified Meanwell Power Supply, FDM 3D Printers for DIY Home and School Printing Size 8.66x8.66x9.84 inch4.5 out of 5 stars 8,459-31% $236.00List: $339.99"
[6] "5ANYCUBIC Wash and Cure Station, Newest Upgraded 2 in 1 Wash and Cure 2.0 Machine for Mars Anycubic Photon S Photon Mono LCD SLA DLP 3D Printer Models UV Rotary Curing Resin Box4.7 out of 5 stars 2,366-30% $139.99List: $199.99"
[7] "6ELEGOO Resin 3D Printer Mars 2 Pro Mono MSLA 3D Printer UV Photocuring LCD Resin 3D Printer with 6.08 inch 2K Monochrome LCD, Printing Size 129x80x160mm/5.1x3.1x6.3inch4.7 out of 5 stars 2,338-20% $171.99List: $213.99"
[8] "7ANYCUBIC Kobra Max 3D Printer, Smart Auto Leveling with Self-Developed ANYCUBIC LeviQ Leveling and Filament Run-Out Detection, Large Build Size 17.7\" x 15.7\" x 15.7\"4.1 out of 5 stars 1,756-24% $569.99$749.99"
[9] "8ELEGOO 5PCs FEP Release Film Mars LCD 3D Printer 140x200 MM 0.15mm Thickness4.7 out of 5 stars 1,335-38% $15.99List: $25.99"
[10] "9ELEGOO Neptune 3 FDM 3D Printer with Auto Leveling, Removable HD Touch Screen, Dual Gear Metal Extruder, Ultra-Quiet Printing, Ideal for Beginners, 220x220x280mm Printing Size4.3 out of 5 stars 528$259.99"
[11] "10ANYCUBIC Photon M3 Resin 3D Printer, 7.6'' LCD SLA UV 3D Resin Printer with 4K+ Monochrome Screen, Protective Film, Fast Printing, Max Printing Size 7.08\" × 6.45\" × 4.03\"4.4 out of 5 stars 1,302-20% $319.99List: $399.99"
[12] "11Official Creality Ender 3 V2 Neo 3D Printer with CR Touch Auto Leveling Kit, PC Steel Printing Platform, Metal Bowden Extruder, 95% Pre-Install 3D Printers for Beginner and Pro(220*220*250mm)4.4 out of 5 stars 53$319.00"
[13] "12ANYCUBIC Vyper, Upgrade Intelligent Auto Leveling 3D Printer with TMC2209 32-bit Silent Mainboard, Removable Magnetic Platform, Large 3D Printers with 9.6\" x 9.6\" x 10.2\" Printing Size4.3 out of 5 stars 809-20% $343.99$429.99"
[14] "13ANYCUBIC Vyper 3D Printer, Auto Leveling Upgrade Fast FDM Printer Integrated Structure Design with TMC2209 32-bit Silent Mainboard, Removable Magnetic Platform, 9.6\" x 9.6\" x 10.2\" Printing Size4.2 out of 5 stars 355-20% $343.99Was: $429.99"
[15] "14WEEDO TINA2 Mini 3D Printer for Kids and Beginners, Full Auto Leveling, Fully Assembled, Removable Build Plate, Small Enclosed FDM 3D Printers for Home Use, PLA/TPU Filament Supported4.0 out of 5 stars 199-40% $119.00List: $199.00"
[16] "15ELEGOO Saturn 2 MSLA 3D Printer, UV Resin Photocuring Printer with 10-inch 8K Monochrome LCD, 219x123x250mm / 8.62x4.84x9.84 Inch Larger Printing Size4.6 out of 5 stars 1,228$670.99"
[17] "16Official Creality Ender 3 V2 Neo 3D Printer with CR Touch Auto Leveling Kit PC Spring Steel Platform Full-Metal Extruder, 95% Pre-Installed 3D Printers with Resume Printing and Model Preview Function4.4 out of 5 stars 593-20% $319.00List: $399.00"
[18] "17ANYCUBIC Photon Mono 4K 3D Printer, 6.23'' Monochrome Screen Upgraded LCD SLA UV Resin 3D Printers with Fast & Precise Printing and Large Printing Size 5.20''X3.14''X6.50''4.3 out of 5 stars 989$249.99"
[19] "18ELEGOO Mercury X Bundle with Separate Washing Station and Curing Station for Large Resin 3D Prints, Compatible with Saturn, Mars MSLA 3D Printers4.6 out of 5 stars 499-14% $213.99List: $249.99"
[20] "19Mintion Beagle Camera for 3D Printer, Plug&Play, Remote Monitoring, WiFi Connection, Support PC/APP, Auto Generate Time-Lapse Video, with 32G Micro SD Card4.2 out of 5 stars 76$79.99"
[21] "20Official Creality Ender 3 S1 3D Printer with Direct Drive Extruder CR Touch Auto Leveling High Precision Double Z-axis Screw Silent Board Printing Size 8.6X8.6X10.6in, Upgrade Ender 3 V2 for Beginners4.5 out of 5 stars 722$399.00"
[22] "21ANYCUBIC Water Washable Resin, 3D Printer Resin with Low Viscosity and Fast Printing, 405nm High Precision UV-Curing 3D Resin, Photopolymer Resin for 8K Capable LCD DLP 3D Printing (Grey, 1kg)4.8 out of 5 stars 29$39.99"
[23] "22Voxelab Aquila C2 3D Printer with Improved Alloy Frame Structure,UL Certified Power Supply,Removable Build Surface Plate, Fully Open Source and Resume Printing Function,Printing Size 8.66x8.66x9.84in4.2 out of 5 stars 1,762$169.99"
[24] "23Official Creality Ender 3 3D Printer Fully Open Source with Resume Printing All Metal Frame FDM DIY Printers with Resume Printing Function 220x220x250mm4.5 out of 5 stars 1,850$189.00"
[25] "24ELEGOO 6.08 Inches Monochrome 2K LCD Compatible with ELEGOO Mars 2 and ELEGOO Mars 2 Pro Resin 3D Printers with 1620x2560 Resolution, Active Area 3.25 x 5.14 inches4.5 out of 5 stars 246$34.99"
[26] "25Creality Fireproof and Dustproof 3D Printer Warm Enclosure Mini 3D Printer Tent for Ender 3/Ender 3 pro/Ender 3 V2/Ender 5 Pro, Constant Temperature Protective Cover Room4.6 out of 5 stars 1,183$75.00"
[27] "26Siraya Tech Build 3D Printer Resin High Resolution Non-Brittle Tappable Engineering Resin 405nm UV-Curing Standard Photopolymer Rapid Resin for LCD DLP 3D Printing 8K Capable (Smoky Black, 1kg)4.7 out of 5 stars 270-18% $41.99List: $50.99"
[28] "27ELEGOO Metal Resin Tank for ELEGOO Saturn 3D Printer, FEP Pre-Installed, with Lid and 3 Allen Wrench, Resin Vat for ELEGOO Saturn and Saturn S4.8 out of 5 stars 220$43.99"
[29] "28Creality Ender 3 V2 3D Printer Upgraded Silent Motherboard MeanWell Power Supply and Carborundum Glass Platform Resume Print, Comes with 1.75mm Filament Print Size 8.66x8.66x9.84 inch4.6 out of 5 stars 65$279.00"
[30] "29Flashforge 3D Printer Adventurer 3 Pro with 2 Removable Nozzle, Glass Bed and Leveling-Free, Fully Assembled, High Precision Printing with PLA/ABS/PETG/PLA-CF/PETG-CF4.3 out of 5 stars 687$399.00"
[31] "30WEEFUN Mini 3D Printer, FDM 3D Printer with Resume Printing Function Quick Guide Fully Assembled Fully Open Source, Auto Leveling, Removable Flexible Magnetic Build Plate, Build Volume 100x120x100mm3.8 out of 5 stars 86-6% $149.00List: $159.00"
[32] "31SUNLU Resin UV Curing Box, 405nm UV LED Lights, UV Light Curing Machine with Timer, 360° Spinning Turntable and Acrylic Reflector, Big Size Rotary Resin Dryer4.3 out of 5 stars 935-11% $49.99List: $55.99"
[33] "32ELEGOO Mercury Plus 2.0 Large Wash and Cure Machine for LCD/SLA/DLP 3D Printing Models Cure Box with Rotary Curing Turntable and Washing Bucket4.7 out of 5 stars 193$111.99"
[34] "33ELEGOO 8.9 Inches 4K Monochrome LCD with an HD Resolution of 3840 x 2400 for Saturn Resin 3D Printe4.1 out of 5 stars 107$79.99"
[35] "34ELEGOO Mercury Curing Machine 405nm Resin UV Curing Box with Light Driven Turntable for LCD DLP SLA 3D Printed Model4.6 out of 5 stars 1,2331 offer from $49.99"
[36] "35FUNGDO Solar Turntable 360° Rotating Stand Display Plate for Better Curing UV Resin Printed Jewelry or Small Items Model from LCD/SLA/DLP 3D Printer Solar or AA Battery Power (Not Included) (Silver)4.4 out of 5 stars 605-36% $8.99List: $13.99"
[37] "36ELEGOO Mercury X Bundle Wash and Cure Station Set, Newly Design Separate Washing Machine and UV Resin Curing Machine for LCD/DLP/SLA Saturn Mars Photon S Photon Mono 3D Printed Models4.6 out of 5 stars 143-7% $204.99List: $219.99"
[38] "37Upgraded Creality Ender 3 V2 3D Printer with 1KG White and 1KG Black PLA Filaments, Integrated Structure Design with Silent Motherboard and Meanwell Power Supply, Printing Size 8.66x8.66x9.84 inch4.4 out of 5 stars 136$289.99"
[39] "38Official Creality Ender 3 Neo 3D Printer with CR Touch Auto Bed Leveling Kit Full-Metal Extruder Carborundum Glass Printing Platform with Resume Printing Function Silent Mainboard 8.66x8.66x9.84 inch4.5 out of 5 stars 310$239.00"
[40] "39ELEGOO Mars 3 Pro Resin 3D Printer with 6.66 inch 4K Monochrome LCD Screen Odor Reduction Function Fast Printing and High Accuracy 143mm x 89mm x 175mm Large Printing Size4.6 out of 5 stars 492$335.99"
[41] "40Phrozen Sonic Mini 4K LCD Resin 3D Printer, Monochrome/Mono LCD Screen, Matrix LED UV Light Tech, Longer Working Hours, for Jewelry-Making and Miniatures, L5.2 x W2.9 x H5.1 Printing Volume4.4 out of 5 stars 396-28% $299.99List: $415.99"
[42] "41Mini 3D Printer for Kids with Removable Magnetic Build Plate 1.75mm Free Test PLA Filament DIY 3D Printers Printing Size 100x100x100mm3.4 out of 5 stars 139-25% $89.99List: $119.99"
[43] "42Official Creality Ender 3 S1 3D Printer, Upgraded Direct Drive FDM 3D Printers, Quick Assembly, with Auto Leveling Bed, Dual Z-axis Screws, Spring Steel Print Bed, Build Size 220x220x270 mm4.5 out of 5 stars 100$399.00"
[44] "43FlashForge Adventurer 3 Lite FDM 3D Printer4.4 out of 5 stars 564$299.99"
[45] "44PHROZEN Sonic Mini LCD Resin 3D Printer UV Photocuring 5.5\" Monochrome LCD Screen, ParaLED Tech, with 2.8'' Smart Touch Screen Offline Printing, Print Volume: 4.7 x 2.6 x 5.1 in.4.3 out of 5 stars 403-52% $129.99List: $269.99"
[46] "45Upgrade 3D Printer Tina 2, Mini 3D Printers with Wi-Fi Cloud Printing Function, Fully Open Source with Resume Printing 3D Printers, Removable Magnetic Build Plate, Work with PLA/PLA Pro/TPU4.1 out of 5 stars 220$179.00"
[47] "46Official Creality Ender 3 S1 Pro 3D Printer with High-Temp Nozzle, All Metal Direct Drive Extruder, Flexible PEI Bed, CR Touch Auto Leveling, Upgraded Ender 3 3D Printers for Home Use, 220×220×270mm3.9 out of 5 stars 43$499.00"
[48] "47Phrozen Sonic Mini 8K LCD Resin 3D Printer, Highest Resolution of 22 µm and 1152 ppi, 7.1” Mono LCD Screen, Prints Beyond Handcrafted Models, L16.5 x W7.2 x H18 cm Printing Volume4.3 out of 5 stars 121 offer from $599.99"
[49] "48Creality Ender 3 Max Neo FDM 3D Printer 300 x 300 x 320mm Large Build Volume CR Touch Automatic Leveling Dual Z-axis Upgrade Red Metal Extruder4.1 out of 5 stars 20-7% $389.00List: $419.00"
[50] "49Official Creality Ender 3 V2 3D Printer, Upgraded Ender 3 3D Printer with Carborundum Glass Bed, Silent Motherboard and MeanWell Power Supply, Build Volume 220 x 220 x 250 mm4.5 out of 5 stars 1,257$279.00"
[51] "50Longer LK5 Pro 3D Printer, FDM Large Size 3D Printer, 11.8x11.8x15.7 Inch Big Print Size, 90% Pre-Assembled, Dual Blower Kit, Silent Motherboard for DIY Home School Printing4.3 out of 5 stars 484$369.99"