Skip to content

unexpected behavior when handling np.ma.core.MaskedArray #199

@jdduprey

Description

@jdduprey

I was trying out the wav_write function and noticed some behavior I don't quite understand.

start_time = datetime(2019,1,12,3)
end_time = datetime(2019,1,12,3,1)
node = 'LJ01C'

hdata = ooipy.get_acoustic_data(start_time, end_time, node, verbose=True)

Fetching URLs...
Sorting valid URLs for Time Window...
Downloading mseed files...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:05<00:00, 1.41s/it]
file fragmented but timestamps are unique. Segment kept
Merging 4 Traces...
Data has Gaps

The logs warn me the data have gaps. This is becuase hdata.data is a masked array.

masked_array(data=[27445.0, 27944.0, 28181.0, ..., 28364.0, 28849.0,
                   29322.0],
             mask=[False, False, False, ..., False, False, False],
       fill_value=1e+20)

But when I call the data directly:
np.unique(hdata.data.mask)
array([False])
So it appears there arn't actually any gaps. Is this due to using obspy merge method zero by default?

Finally - I think the masked array may be causing problems with wav_write. Unless wav_write should be deprecated and save is the method to use?

hdata.wav_write("hyd_output/test2.wav", norm=False)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File [~/miniconda3/envs/ooipy/lib/python3.11/site-packages/numpy/ma/core.py:3251](http://localhost:8890/lab/workspaces/auto-l/tree/HYD/~/miniconda3/envs/ooipy/lib/python3.11/site-packages/numpy/ma/core.py#line=3250), in MaskedArray.view(self, dtype, type, fill_value)
   3250 try:
-> 3251     if issubclass(dtype, ndarray):
   3252         output = ndarray.view(self, dtype)

TypeError: issubclass() arg 1 must be a class

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[13], line 1
----> 1 hdata.wav_write("hyd_output[/test2.wav](http://localhost:8890/test2.wav)", norm=False)

File [~/repos/ooipy/src/ooipy/hydrophone/basic.py:527](http://localhost:8890/lab/workspaces/auto-l/tree/HYD/~/repos/ooipy/src/ooipy/hydrophone/basic.py#line=526), in HydrophoneData.wav_write(self, filename, norm, new_sample_rate)
    524             data = signal.decimate(data, int(self.stats.sampling_rate [/](http://localhost:8890/) new_sample_rate))
    525             sampling_rate = new_sample_rate
--> 527 wavfile.write(filename, int(sampling_rate), data)

File [~/miniconda3/envs/ooipy/lib/python3.11/site-packages/scipy/io/wavfile.py:870](http://localhost:8890/lab/workspaces/auto-l/tree/HYD/~/miniconda3/envs/ooipy/lib/python3.11/site-packages/scipy/io/wavfile.py#line=869), in write(filename, rate, data)
    867 if data.dtype.byteorder == '>' or (data.dtype.byteorder == '=' and
    868                                    sys.byteorder == 'big'):
    869     data = data.byteswap()
--> 870 _array_tofile(fid, data)
    872 # Determine file size and place it in correct
    873 # position at start of the file or the data chunk.
    874 size = fid.tell()

File [~/miniconda3/envs/ooipy/lib/python3.11/site-packages/scipy/io/wavfile.py:891](http://localhost:8890/lab/workspaces/auto-l/tree/HYD/~/miniconda3/envs/ooipy/lib/python3.11/site-packages/scipy/io/wavfile.py#line=890), in _array_tofile(fid, data)
    889 def _array_tofile(fid, data):
    890     # ravel gives a c-contiguous buffer
--> 891     fid.write(data.ravel().view('b').data)

File [~/miniconda3/envs/ooipy/lib/python3.11/site-packages/numpy/ma/core.py:3257](http://localhost:8890/lab/workspaces/auto-l/tree/HYD/~/miniconda3/envs/ooipy/lib/python3.11/site-packages/numpy/ma/core.py#line=3256), in MaskedArray.view(self, dtype, type, fill_value)
   3255             output = ndarray.view(self, dtype)
   3256     except TypeError:
-> 3257         output = ndarray.view(self, dtype)
   3258 else:
   3259     output = ndarray.view(self, dtype, type)

File [~/miniconda3/envs/ooipy/lib/python3.11/site-packages/numpy/ma/core.py:3490](http://localhost:8890/lab/workspaces/auto-l/tree/HYD/~/miniconda3/envs/ooipy/lib/python3.11/site-packages/numpy/ma/core.py#line=3489), in MaskedArray.dtype(self, dtype)
   3487 # Try to reset the shape of the mask (if we don't have a void).
   3488 # This raises a ValueError if the dtype change won't work.
   3489 try:
-> 3490     self._mask.shape = self.shape
   3491 except (AttributeError, TypeError):
   3492     pass

ValueError: cannot reshape array of size 3840001 into shape (30720008,)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingquestionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions