Qobuz barcodes are sometimes missing #40

derat commented 2024年01月15日 16:29:36 +01:00

I added support for fetching Qobuz barcodes for #36, but they seem to be missing sometimes as described in this comment.

I was able to see the problem myself a few times when using https://yambs.erat.org/ to seed https://www.qobuz.com/fr-fr/album/revolution-little-steven/wwmjrcrz1n09b. Oddly, I don't see anything in the logs about the API call failing:

2024年01月15日T15:15:50.965676Z Handling 498-byte "online" request from ...
2024年01月15日T15:15:50.965887Z Fetching https://www.qobuz.com/fr-fr/album/revolution-little-steven/wwmjrcrz1n09b
2024年01月15日T15:15:50.966018Z Fetching https://www.qobuz.com/api.json/0.2/album/get?album_id=wwmjrcrz1n09b
2024年01月15日T15:15:52.150389Z Parsing https://www.qobuz.com/fr-fr/album/revolution-little-steven/wwmjrcrz1n09b
2024年01月15日T15:15:52.159698Z Returning cached language for "Revolution\nWhere Do We Go from Here?\nRe..." (147 bytes)
2024年01月15日T15:15:52.159757Z Returning 2 edit(s) to ...

I seeded the same URL using a local yambsd instance and couldn't repro the problem there. I wonder if the API is returning bad data (e.g. an empty JSON object?) to the App Engine instance sometimes.

I guess the next step is adding some more logging and redeploying the App Engine instance to try to figure out what's going on.

I added support for fetching Qobuz barcodes for #36, but they seem to be missing sometimes as described in [this comment](https://community.metabrainz.org/t/announcing-yambs-yet-another-musicbrainz-seeder/608630/15). I was able to see the problem myself a few times when using https://yambs.erat.org/ to seed <https://www.qobuz.com/fr-fr/album/revolution-little-steven/wwmjrcrz1n09b>. Oddly, I don't see anything in the logs about the API call failing: ``` 2024年01月15日T15:15:50.965676Z Handling 498-byte "online" request from ... 2024年01月15日T15:15:50.965887Z Fetching https://www.qobuz.com/fr-fr/album/revolution-little-steven/wwmjrcrz1n09b 2024年01月15日T15:15:50.966018Z Fetching https://www.qobuz.com/api.json/0.2/album/get?album_id=wwmjrcrz1n09b 2024年01月15日T15:15:52.150389Z Parsing https://www.qobuz.com/fr-fr/album/revolution-little-steven/wwmjrcrz1n09b 2024年01月15日T15:15:52.159698Z Returning cached language for "Revolution\nWhere Do We Go from Here?\nRe..." (147 bytes) 2024年01月15日T15:15:52.159757Z Returning 2 edit(s) to ... ``` I seeded the same URL using a local `yambsd` instance and couldn't repro the problem there. I wonder if the API is returning bad data (e.g. an empty JSON object?) to the App Engine instance sometimes. I guess the next step is adding some more logging and redeploying the App Engine instance to try to figure out what's going on.

derat commented 2024年01月15日 17:19:52 +01:00

Looks like an API issue. Here's what I see on the App Engine instance:

2024年01月15日T16:15:20.847900Z Fetching https://www.qobuz.com/api.json/0.2/album/get?album_id=wwmjrcrz1n09b
2024年01月15日T16:15:21.030514Z Decoded 75-byte object
2024年01月15日T16:15:21.030534Z Got UPC "" from API

Compare that with what I get when running locally:

2024年01月15日 11:59:53 Fetching https://www.qobuz.com/api.json/0.2/album/get?album_id=wwmjrcrz1n09b 
2024年01月15日 11:59:53 Decoded 9930-byte object 
2024年01月15日 11:59:53 Got UPC "0886448228950" from API 

I haven't checked what the 75 bytes that they're sending back contain, but probably it's something along the lines of {"error_code": "Go away, we don't want your spammy requests"}. :-/

Looks like an API issue. Here's what I see on the App Engine instance: ``` 2024年01月15日T16:15:20.847900Z Fetching https://www.qobuz.com/api.json/0.2/album/get?album_id=wwmjrcrz1n09b 2024年01月15日T16:15:21.030514Z Decoded 75-byte object 2024年01月15日T16:15:21.030534Z Got UPC "" from API ``` Compare that with what I get when running locally: ``` 2024年01月15日 11:59:53 Fetching https://www.qobuz.com/api.json/0.2/album/get?album_id=wwmjrcrz1n09b 2024年01月15日 11:59:53 Decoded 9930-byte object 2024年01月15日 11:59:53 Got UPC "0886448228950" from API ``` I haven't checked what the 75 bytes that they're sending back contain, but probably it's something along the lines of `{"error_code": "Go away, we don't want your spammy requests"}`. :-/

derat commented 2024年01月15日 18:26:22 +01:00

They're actually returning a "not found" error:

API returned short response: "{\"status\":\"error\",\"code\":404,\"message\":\"No result matching given argument\"}"

And I just now noticed that I forgot to check the HTTP status code when fetching JSON objects.

So yeah, this looks like it's probably the same geographical restriction issue mentioned in https://community.metabrainz.org/t/qobuz-importer-script/569614/9.

They're actually returning a "not found" error: ``` API returned short response: "{\"status\":\"error\",\"code\":404,\"message\":\"No result matching given argument\"}" ``` And I just now noticed that I forgot to check the HTTP status code when fetching JSON objects. So yeah, this looks like it's probably the same geographical restriction issue mentioned in https://community.metabrainz.org/t/qobuz-importer-script/569614/9.

derat commented 2024年01月15日 20:24:47 +01:00

Quoting myself from the forum thread just so I don't forget about it:

It’d be a huge hack, but it might be possible for me to make the user’s browser query the API and then incorporate the barcode from there into the rest of the seeded data from the server. I’d need to think about that a bit more.

Quoting myself from the forum thread just so I don't forget about it: > It’d be a huge hack, but it might be possible for me to make the user’s browser query the API and then incorporate the barcode from there into the rest of the seeded data from the server. I’d need to think about that a bit more.