ENH: Add MZ3 format #1066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

effigies wants to merge 7 commits into nipy:master

from effigies:enh/mz3

Open

ENH: Add MZ3 format #1066

effigies wants to merge 7 commits into nipy:master from effigies:enh/mz3

Conversation

effigies

Copy link

Member

@effigies effigies commented Nov 19, 2021

Currently implements reading the header and "scalar" array. Haven't tested if we need to switch ordering to handle multiple scalar arrays. Does not yet implement reading vertices, faces or rgba.

Two significant wrinkles to this format:

Gzipped and uncompressed versions have the same extension.
The data array size must be calculated from the file size. For the compressed case, I first try to use the member-size mod 2**32 field in the gzip footer to detect the size. If we hit 20GB we bet that it's a weird file and just seek to the end of the gzip stream.

Loading face/header data will be a good test for BIAP 0009 (#1056), but don't want to clog up that PR at the moment.

effigies added 2 commits

November 19, 2021 08:36

@effigies


 ENH: Add ability for compression to be set explicitly in ImageOpener/...

204a48c

...ArrayProxy

@effigies


 ENH: First pass at MZ3 image

ef9b1b3

@pep8speaks

Copy link

pep8speaks commented Nov 19, 2021 •

edited

Loading

Hello @effigies, Thank you for updating!

In the file nibabel/surfice.py:

Line 175:1: W293 blank line contains whitespace

To test for issues locally, pip install flake8 and then run flake8 nibabel.

Comment last updated at 2021年11月19日 13:45:48 UTC

@effigies


 Typo

fdd94d1

@codecov

Copy link

codecov bot commented Nov 19, 2021 •

edited

Loading

Codecov Report

❌ Patch coverage is 29.53020% with 105 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.91%. Comparing base (db4c859) to head (cb9a77a).
⚠️ Report is 10 commits behind head on master.

Files with missing lines	Patch %	Lines
nibabel/surfice.py	25.92%	100 Missing ⚠️
nibabel/openers.py	42.85%	3 Missing and 1 partial ⚠️
nibabel/arrayproxy.py	85.71%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@
## master #1066 +/- ##
==========================================
- Coverage 95.34% 94.91% -0.43% 
==========================================
 Files 209 210 +1 
 Lines 29777 29917 +140 
 Branches 3357 3380 +23 
==========================================
+ Hits 28390 28396 +6 
- Misses 948 1073 +125 
- Partials 439 448 +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@neurolabusc

Copy link

neurolabusc commented Jan 16, 2022

@effigies thanks for supporting this. The mz3 specification is here.

It is true that a single extension is used for both compressed and uncompressed data. You can discriminate these by looking at the first two bytes of the file. Specifically, all GZip files begin with hex = 0x1F8B, all uncompressed MZ3 files begin with hex = 0x4D5A
You do not need to know the file size to compute the array size. The header NFACE and NVERT explicitly describe the number of indexed triangles (faces) and spatial points (vertices).

Happy to work directly with you to tidy this up.