Skip to content

improve performance of spatial.import() for large data sets #385

@jamesdhope

Description

@jamesdhope

I ran an import of the London OSM data from Planet on a 2 Core machine with 4GIG RAM. OSM Nodes and Ways imported within 3-4 hours. After 30 hours Relations were still being imported. In the end I had to kill the process.

It was reassuring that the import ran without memory overflow on such a small machine (possibly using SWAP i'm not sure) but the speed at which relations were being imported was the main issue for us and would likely be too slow to maintain a large OSM dataset. Indeed, one option would be to run this on a larger machine and further testing would be required to see if this would be performant enough for our needs.

It would be good to see if further optimisation of the import routine could be made to improve performance particularly with regard to OSM Relations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions