Search code examples
neo4jcypherload-csv

Some way to create 1 million relationships with a neo4j query


With this query I am importing 75000 nodes from my csv file. (Category)

USING PERIODIC COMMIT 
LOAD CSV WITH HEADERS 
FROM "file:///prodcategory.csv" AS row 
CREATE (:Category {id: row.idProdCategory, name: row.name, idRestaurant: row.idRestaurant});

And with this query I am also importing 1 million nodes from my csv file (Product)

USING PERIODIC COMMIT 
LOAD CSV WITH HEADERS 
FROM "file:///products.csv" AS row 
CREATE (:Product {id: row.idProduct, idProductCategory: row.idProductCategory,name: row.name,idRestaurant:row.idRestaurant ,description: row.description, price: row.price, shipping_price: row.shippingPrice});

I am using this query to create the relationship between id -> category and idProductCategory -> products.

MATCH (category:Category {id: category.id})
MATCH (Product:Product {idProductCategory: Product.idProductCategory})
WHERE Product.idProductCategory=category.id
MERGE (category)-[:OF_CATEGORY]->(Product);

This query only creates 2999 relationships and I do not believe the 1 million relationships I should create, please if there is a method or configuration to be able to create more than 1 million relationships please help me I would be very grateful.


Solution

  • Ensure you have indexes on Product.idProductCategory.

    I assume that the category id is unique across categories.

    CREATE CONSTRAINT ON (category:Category) ASSERT category.id IS UNIQUE;
    

    I assume that there are multiple products with the same category ID.

    CREATE INDEX ON :Product(idProductCategory);
    

    Then you can simply match each category and then for each category find the appropriate products and create the relationships.

    // match all of your categories
    MATCH (category:Category)
    
    // then with each category find all the products
    WITH category 
    MATCH (Product:Product {idProductCategory: category.id })
    
    // and then create the 
    MERGE (category)-[:OF_CATEGORY]->(Product);
    

    If you are running into memory constraints you could use the APOC periodic commit to wrap your query...

    call apoc.periodic.commit("
      MATCH (category:Category)
      WITH category 
      MATCH (Product:Product {idProductCategory: category.id })
      MERGE (category)-[:OF_CATEGORY]->(Product)
    ",{limit:10000})