MDF4.3

What's New in v4.2

New MDF 4.x Features

• MDF 4.3 support: new compression algorithms (ZstdSD, LZMA, LZ4) introduced in MDF 4.3
• VLSC (Variable Length Signal Channel): initial support for variable-length signal data
• New block types: CL, DS, CU, CV block parsing
• BOM reading and VD (Value Definition) block support
• SD block reading fixes

MFX / Compressed File Support

• MFXZ decompression no longer requires a temporary file on disk

File Finalization

• Finalization feature: cycle count and block size recalculation for unfinished recordings
• DL (Data List) block updates for finalization

Bug Fixes

• Unsorted data groups: bit masking now applied for non-byte-aligned channels
• Signed integer initialization: fixed stale high-byte bug for 24-bit signed integers in dataRead
• Integrity checks: multiple fixes for compressed (DZ) and empty data blocks with wrong size calculations
• Unfinalized DZ blocks: fixed wrong block length calculation
• Complex data types: fixes for f16 complex, general complex type bugs, and BOM+complex value handling
• MDF 2: bug fix for MDF 2 file reading
• Export: various export bug fixes
• resample(): discard intermediate arrays to reduce memory usage
• Null timestamp (1970-01-01T00:00:00) no longer raises an exception
• DG/CG keys changed from sequential integer to file position
• channel class attribute casing fixed (camelCase)
• Improved channel extraction in masterChannelList
• Native endianness check added
• Byte order handling fixed when returning a DataFrame
• copy() bug fix
• KeyError and overflow warning fixes

Python & NumPy Compatibility

• Python 3.13: Cython dataRead extension updated
• NumPy 2.x: backwards-compatible fromstring/fromarrays imports
• Minimum Python version raised to 3.9
• distutils dependency removed

Resample

• resample() now accepts interpolation options from scipy.interp1d

Performance

• Various read-path optimisations
• Memory usage reduced during large file reads

Packaging & CI

• GitHub Actions workflows for automated PyPI publishing (Linux x86_64/aarch64, Windows x86_64, macOS x86_64/arm64)
• pyproject.toml added for PEP 517 build support

Community Contributions

• Byte order fix when returning DataFrame (#220 — LaurentBlanc73)
• Data storage fix (#212 — LaurentBlanc73)
• Improved masterChannelList channel extraction (#213 — LaurentBlanc73)
• NumPy 2.x import compatibility + camelCase attribute fix (#215 — Pzdrs)
• Native endianness check and null timestamp fix (cyclichertz)

time handling and merge/concatenation changes

• Fixed bug swapping bytes order for signed int 8bytes with dataRead module.
• Time handling improvement: fileMetadata['time'] epoch floating in seconds is commonised for all mdf versions, no more dates as found in mdf version 3. More versatile this way and python datetime module can still be used to extract date if needed.
• Reworked merge_mdf to cope with heterogeneous data and renamed into concat_mdf
• Created merge_mdf

Python3.x only

• Mainly dropping python2.7 support
• Switched from scipy to hdf5storage requirement for Matlab export with more recent format version 7.3
• Manipulation methods made compatible to mdf4
  • Adjusted cut method to only cut data groups having same master type
  • Rewrote resample in order to only resample data groups with same master type, added resample_group to resample only one specific data group.
• Fixed bug for attachment attribute that was empty for attachments and events in case of normal reading (noDataLoading=False)
• Improved parsing performance for VLSD and unsorted data by off loading heavy computations to cython module

bug fixes

Added possibility to write and read channels in column potentially for faster read.
Added half floating and complex types
Last release both compatible python 2.7 and 3.4+
Improved reading speed for SD Blocks (for variable length channels) and unsorted data

Fixed following bugs:
Several issues with VLSD_CG reading
bug when combining reading with no_data_loading and extracting data with raw_data, issue #168 , thanks bbto

bug fixes

fixed bug with dataRead having not aligned number of bytes.

Pandas related improvement and bug fixes

Added parquet export
Added pandas dataframe export per data group
fixed a bug in dataRead with big endian channels

Refactoring

• /!\ Major refactoring for more pep8 compliance --> names of classes, methods and arguments have changed
• Added new method getChannelName4() for mdf4.x to only find mdf channel name (unique key within mdf class inherited from dict) based on name and source path or name (same name can be used in mdf file file but source paths or names shall be different, making it unique). Can be reused then with getChannelData4()
• Improved xml metadata parsing speed.

Fixed bugs related to
-nBytes calculation
-HL Block in VLSD data
-setup.py and cythonize
-channel by channel array reading implementation in python (fallback of dataRead)
-SD Block read of byte array type

bug fixes

• fixed bugs in dataRead, missing return in some cases
• write3 bug with unicode handling of metadata
• bug in plot with only one raster, wrong parsing of masterChannelList
• bug in function converting bits into bytes, wrong calculation for small string channel below length 8bytes

Bug fix

• fixed bit sign issue when reading byte aligned data
• Made possible install without cython (not advised) while keeping dependency resolution

Bug fixes

• introduced subplots
• Improved setup.py robustness when cython not available
• made available compression writing argument from write
• resample() robustness improvement : added function to clean data group having master non evenly increasing
• corrected channel name appending when doublon and SIBlock available
• added possibility to write arrays in mdf 4
• fixed bug with CEBlock and mdf3
• fixed bug reading mdf3 data unaligned
• fixed bug with channel structures or bus logging and duplicated names