I have to extract rasters from a large TIFF file (about 8MB). For this purpose I use GeoTIFF v.2.0.7 readRasters, but performance is unsatisfying:
reading rasters took: 395143.48280000035 ms
As a solution for my problem I found something like Pools in geotiff documentation, however just after declaring Pool
let pool = new Pool()
I am running into an error.
First I thought it could be a backend fault, but this comment reassured me that Pools are not only supporting WebWorkers but also worker-threads.
Should I do any workaround instead of just declaring Pool? May it be a GeoTIFF library problem?
Here is my code and the occurring error. (For testing - I saved TIFF converted to array buffer to JSON file):
import { GeoTIFFImage, fromArrayBuffer, Pool } from "geotiff";
import type { NextApiRequest, NextApiResponse } from "next";
import rawInput from "~/rawInput.json";
const handler = async (req: NextApiRequest, res: NextApiResponse) => {
const arrayBuffer = new Uint8Array(rawInput.buffer.data).buffer;
const geotiff = await fromArrayBuffer(arrayBuffer);
const image = await geotiff.getImage();
const start = performance.now();
let pool = new Pool();
const rasters = (await image.readRasters({ pool, interleave: true })) as any as number[];
const end = performance.now();
console.log(`reading rasters took: ${end - start} ms`);
res.status(200).json({ name: "test" });
};
The error:
TypeError [ERR_INVALID_URL_SCHEME]: The URL must be of scheme file
at new NodeError (node:internal/errors:400:5)
at Object.fileURLToPath (node:internal/url:1492:11)
at new Worker (D:\WORK\mock-project\node_modules\web-worker\cjs\node.js:108:19)
at create (file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/worker/decoder.js:5:18)
at file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/pool.js:57:39
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
code: 'ERR_INVALID_URL_SCHEME'
}
- error Error [TypeError]: Cannot set properties of undefined (setting 'idle')
at file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/pool.js:77:21
at new Promise (<anonymous>)
at Pool.decode (file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/pool.js:74:9)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/geotiffimage.js:391:20
at async GeoTIFFImage.getTileOrStrip (file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/geotiffimage.js:418:34)
at async Promise.all (index 0)
at async GeoTIFFImage._readRaster (file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/geotiffimage.js:517:5)
at async GeoTIFFImage.readRasters (file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/geotiffimage.js:611:20)
at async handler (webpack-internal:///./src/pages/api/index.ts:32:21) {
digest: undefined
}
- error unhandledRejection: TypeError [ERR_INVALID_URL_SCHEME]: The URL must be of scheme file
at new NodeError (node:internal/errors:400:5)
at Object.fileURLToPath (node:internal/url:1492:11)
at new Worker (D:\WORK\mock-project\node_modules\web-worker\cjs\node.js:108:19)
at create (file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/worker/decoder.js:5:18)
at file:///D:/WORK/mock-project/node_modules/geotiff/dist-module/pool.js:57:39
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
digest: undefined
}
-
If reading a 8 MB TIFF takes more that 6 minutes then something is fundamentally wrong. I am pretty sure that by fiddling with Pools you are trying to optimize something that is not the real bottle neck.user30184– user301842023年08月30日 18:48:59 +00:00Commented Aug 30, 2023 at 18:48
-
@user30184 thanks for reply, here is an example of the tiff which needs to be converted to rasters using geotiff library. Processing this file(without pools) took me 462403ms, and the final array of rasters weights 18.87MB. Do you have any idea what am I doing wrong or what may be the real bottle neck?Michał Dubrowski– Michał Dubrowski2023年09月01日 12:54:18 +00:00Commented Sep 1, 2023 at 12:54