Added buchhandlung.de importer #74

When we add the changes, there will be also conflicts with the Rupprecht stores. Rupprecht uses the Buchhandlung.de service for its online shop.

The schoeningh book store in Würzburg do also have two online shop: https://www.schoeningh-buch.de/ and https://ebooksinfranken.buchhandlung.de/shop/

Awesome. Thank you.

I would say we should add a denylist in this filter and remove the stores that are already in the data through other sources. The schoeningh for example shows store availability in their own store. Which I would prefer...

Also I will add a system tomorrow to not load all cities on load of the page, but only the 15 closest and if people start searching, ask the backend for more.

Another thing: there are not that many openstreetmap ids in the data...did the importer fail to find any? Will check tomorrow.

I think the API calls run into a rate limit problems

https://dev.overpass-api.de/overpass-doc/en/preface/commons.html

Ah yeah, I limited it to one request/second. might need to go slower.

I build a cache into it so that for locations we alreaady have the osm id, we never query for that again.

Okay, the rate limit handling in the osm part is now a bit smarter

OK, I'll rerun it tomorrow, so we should get more ids

closes #87

@Aziz-JH do you think I could merge this one?

Btw: i now have this branch running locally and use that to modify the view layer to be able to handle so many entries

Ah and the healthcheck finds duplicates with the existing data. I think we should remove the existing data * in favour of this importer, excluding the rupprecht importer and the buchbox case. I want those still grouped together. Or we find a smarter way to group them and using this data?

And we then add some tags (e.g. Queer) into the generated file and rescue them in each run so that we don't loose the manual added data (tags, note, lat/lng, osmid) with a helper method in common.js

Okay, after playing around with this, we need a "addToGroup": "Rupprecht" that we manually add? Not sure how to automate that.

Only question: How do you group this:

https://www.buchhandlung-ludwig.de/shop/magazine/146155/standorte_und_oeffnungszeiten.html

"Ludwig & Eckert"?! What a strange shop that is.

LUDWIG (Bhf. Lichtenberg) Weitlingstr. 22, 10317 Berlin has the wrong osm id. It should be way/430955095 . Maybe we could do one pass checking if the osmId lat/lng is more than 300m off?

@bitboxer wrote in #74 (comment):

Only question: How do you group this:

https://www.buchhandlung-ludwig.de/shop/magazine/146155/standorte_und_oeffnungszeiten.html

"Ludwig & Eckert"?! What a strange shop that is.

I think I made somthing so we can group them under a "chain". It will be a patch function that can be used to do changes after the import

@bitboxer wrote in #74 (comment):

LUDWIG (Bhf. Lichtenberg) Weitlingstr. 22, 10317 Berlin has the wrong osm id. It should be way/430955095 . Maybe we could do one pass checking if the osmId lat/lng is more than 300m off?

The Problem is also it is as "newsagent" stored https://www.openstreetmap.org/way/430955095#map=19/52.509476/13.496774

Ah, damn. I think we need to also search for "newsagent" then...because i can assume that this happens often and newagent also carry books in a lot of cases.

@Aziz-JH wrote in #74 (comment):

I think I made somthing so we can group them under a "chain". It will be a patch function that can be used to do changes after the import

Yes, let's do this. This function can then be used by all importers. We just need to have a chain.json that defines some filters how to group the stuff.

Yes the newsagent stores are sometimes also selling books. I will later rerun the script and push it. Should we also filter out stores that do not have a osm ID?

No, let us keep them. My plan is to add a UI that says: we don't know the OpenStreetmap for this, please add it. And we should try to figure out the lat/lng without openstreetmap osmid and use that in the map if possible. If not possible, we change the text to: "sorry, we don't know where the store is, please help us (link to form where people can send us more details)"

This is amazing. Loving the new patching approach <3

At first, I wasn't sure whether we should use a JSON or a TS file for patching, as that's how it's currently done.

If we need to do more patching, we should use JSON instead. What do you think?

We also have some problems with duplicate OSM IDs, which are causing issues. Some interesting things are happening. For example, the OSM search is showing results for a completely different city.

https://www.openstreetmap.org/search?query=Am+Markt+23%2C+39288+Burg&zoom=16&minlon=11.181185245513918&minlat=54.43500780204213&maxlon=11.214873790740969&maxlat=54.44568958121143#map=16/54.43931/11.19751

Thats why the store a OSM ID from a other store: node/1370584944

@Aziz-JH wrote in #74 (comment):

We also have some problems with duplicate OSM IDs, which are causing issues. Some interesting things are happening. For example, the OSM search is showing results for a completely different city.

https://www.openstreetmap.org/search?query=Am+Markt+23%2C+39288+Burg&zoom=16&minlon=11.181185245513918&minlat=54.43500780204213&maxlon=11.214873790740969&maxlat=54.44568958121143#map=16/54.43931/11.19751

Thats why the store a OSM ID from a other store: node/1370584944

The problem is that the correct address is "Markt 23, 39288 Burg", the "Am Markt" is confusing nominatim and then it finds another "Am Markt" in a city starting with "Burg".

@Aziz-JH wrote in #74 (comment):

At first, I wasn't sure whether we should use a JSON or a TS file for patching, as that's how it's currently done.

If we need to do more patching, we should use JSON instead. What do you think?

Yeah, go for it. Easy solution first and if we see something else fits better we can switch.