COG: add support for INTERLEAVE=BAND and TILE

[rouault]

Fixes #10859

Up to now the COG driver only produced INTERLEAVE=PIXEL / PlanarConfiguration=Contiguous files where values for all bands are in the same tile. This PR add supports for INTERLEAVE=BAND where the tiles of band 1 are placed first, followed by the ones of band 2, etc. It also introduces a INTERLEAVE=TILE mode, which is similar to the BIL (Band Interleave by Line), but generalized to tiles. That is you put first tile (x,y)=(0,0) of band 1, then tile (0, 0) of band 2, ... tile (0, 0) of band N, tile (1, 0) of band 1, ... tile (1, 0) of band N, etc. Both modes can be useful for hyper-spectral datasets for example.

I'm not totally sure about the naming for INTERLEAVE=TILE. Open to suggestions

[coveralls]

coverage: 70.094% (+0.001%) from 70.093%
when pulling a3d090e on rouault:gtiff_interleave_tile
into 4cc3cd9 on OSGeo:master.

[rouault]

Just FYI @joanma747 (this doesn't violate anything in the OGC COG spec which allow PlanarConfiguration=Separate as a potential option)

[rouault]

@jakimowb @tbonfort Any feedback on this PR ? (in particular for the naming of the "TILE" interleaving)

[sgillies]

@rouault I'm confused about "INTERLEAVE=TILE; COG". Do you intend that there be a semi-colon in the option value or is this just a shorthand way of naming the feature in discussion?

[rouault]

or is this just a shorthand way of naming the feature in discussion?

yes, should read:

• GTIFF: add INTERLEAVE=TILE
• COG: add INTERLEAVE=BAND
• COG: add INTERLEAVE=TILE

Or said differently, with this PR, both GTIFF and COG drivers support INTERLEAVE=PIXEL/BAND/TILE

[tbonfort]

TILE looks good to me for COG. Just to make sure I understand correctly: TILE and BAND are identical for GTIFF, they only differ for COG, right? (the actual layout /may/ differ between TILE/BAND for GTIFF, but said layout is not specified/enforced in this case)

[rouault]

TILE and BAND are identical for GTIFF, they only differ for COG, right? (the actual layout /may/ differ between TILE/BAND for GTIFF, but said layout is not specified/enforced in this case)

Good question. Actually that's a bit subtle. TILE is identical for GTIFF and COG, but for GTIFF, it can only be used in CreateCopy() mode (an error will be emitted at runtime if trying to use it in Create() mode. Also documented at https://github.com/OSGeo/gdal/pull/11541/files#diff-c98fddcda04a8dab184552a28d2b386d941862584daeb6a5b39e9cabdeaaac5dR484). Because in Create() "random" update mode, we can only guarantee the PlanarConfiguration=Separate TIFF setting, and not that the TILE interleaving (which would be dependent on the order users fill blocks, and would require the block cache to honour to be aware of the appropriate order in which to evict dirty blocks).

[tbonfort]

I'm not sure it's a good idea to start introducing layout options for GTIFF : won't we end up implementing a second COG driver? For me TILE should either be unsupported for GTiff, or an exact alias of BAND. This is a nitpick however, not a hill I will die for.

[rouault]

I'm not sure it's a good idea to start introducing layout options for GTIFF : won't we end up implementing a second COG driver?

actually, the COG driver is mostly an empty shell that forwards all the heavy work to the GTiff one, so this has to be supported in some way by the GTiff driver. But I see your point. I could possibly make INTERLEAVE=TILE an undocumented option of the GTiff driver, that only the COG driver would be supposed to make use of it.

[jakimowb]

@rouault thank you for that implementation. It all looks very good to me, but unfortunately I don't have the possibility to compile and test it locally at the moment.

I like the descriptions that were added to doc/source/drivers/raster/cog.rst and doc/source/drivers/raster/gtiff.rst
To me, they clearly show how the INTERLEAVE=[PIXEL/BAND/TILE] options are meant. Especially the pseudo-code examples are great! They help users like me, with background in hyperspectral remote sensing and more familiar with the terms used by the ENVI driver (INTERLEAVE=[BSQ​/​BIP​/​BIL]), to understand how the COGs are structured.

[rouault]

I could possibly make INTERLEAVE=TILE an undocumented option of the GTiff driver, that only the COG driver would be supposed to make use of it.

I've just done that