HDF5 library version mismatched error

created at 01-03-2022 views: 169

error message

Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.10.4, library is 1.10.5
        SUMMARY OF THE HDF5 CONFIGURATION
        =================================

General Information:
-------------------
                   HDF5 Version: 1.10.5
                  Configured on: 2019-03-04
                  Configured by: Visual Studio 14 2015 Win64
                    Host system: Windows-10.0.17763
              Uname information: Windows
                       Byte sex: little-endian
             Installation point: C:/Program Files/HDF5

Compiling Options:
------------------
                     Build Mode:
              Debugging Symbols:
                        Asserts:
                      Profiling:
             Optimization Level:

Linking Options:
----------------
                      Libraries:
  Statically Linked Executables: OFF
                        LDFLAGS: /machine:x64
                     H5_LDFLAGS:
                     AM_LDFLAGS:
                Extra libraries:
                       Archiver:
                         Ranlib:

Languages:
----------
                              C: yes
                     C Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe 19.0.24218.1
                       CPPFLAGS:
                    H5_CPPFLAGS:
                    AM_CPPFLAGS:
                         CFLAGS:  /DWIN32 /D_WINDOWS /W3
                      H5_CFLAGS:
                      AM_CFLAGS:
               Shared C Library: YES
               Static C Library: YES

                        Fortran: OFF
               Fortran Compiler:
                  Fortran Flags:
               H5 Fortran Flags:
               AM Fortran Flags:
         Shared Fortran Library: YES
         Static Fortran Library: YES

                            C++: ON
                   C++ Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe 19.0.24218.1
                      C++ Flags: /DWIN32 /D_WINDOWS /W3 /GR /EHsc
                   H5 C++ Flags:
                   AM C++ Flags:
             Shared C++ Library: YES
             Static C++ Library: YES

                            JAVA: OFF
                   JAVA Compiler:

Features:
---------
                   Parallel HDF5: OFF
Parallel Filtered Dataset Writes:
              Large Parallel I/O:
              High-level library: ON
                    Threadsafety: OFF
             Default API mapping: v110
  With deprecated public symbols: ON
          I/O filters (external):  DEFLATE DECODE ENCODE
                             MPE:
                      Direct VFD:
                         dmalloc:
  Packages w/ extra debug output:
                     API Tracing: OFF
            Using memory checker: OFF
 Memory allocation sanity checks: OFF
          Function Stack Tracing: OFF
       Strict File Format Checks: OFF
    Optimization Instrumentation:
Bye...
[I 00:18:57.112 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 80311e8f-283a-4f90-9ab8-896c3ddfa615 restarted
[I 00:19:35.205 NotebookApp] Interrupted...
[I 00:19:35.205 NotebookApp] Shutting down 1 kernel
[I 00:19:35.427 NotebookApp] Kernel shutdown: 80311e8f-283a-4f90-9ab8-896c3ddfa615

Error analysis

The HDF5 header file version does not match the HDF5 library version (Headers are 1.10.4, library is 1.10.5)

The error message mentions the solution: 1. "HDF5_DISABLE_VERSION_CHECK" = "1" --> Disable this warning 2. "HDF5_DISABLE_VERSION_CHECK"="1" --> Ignore the warning message

For the method 1 in the error message: Since the system used is win10, but the path of LD_LIBRARY_PATH cannot be found, search later When it was discovered, it was Linux, so this method is invalid.

For method 2: Before import tensorflow, add the following code to the code: import os; os.environ[‘HDF5_DISABLE_VERSION_CHECK’] = ‘2’

Introduction to HDF5

Hierarchical Data Format HDF (Hierarchical Data Format) refers to a file designed for storing and processing large-capacity scientific data.

HDF5 has a series of excellent features, making it particularly suitable for the storage and operation of large amounts of scientific data, such as it supports a lot of

Data types, flexible, universal, cross-platform, scalable, efficient I/O performance, support almost unlimited (up to EB) single file storage

Please refer to its official introduction: https://support.hdfgroup.org/HDF5/ for details.

There are a series of tools in Python that can manipulate and use HDF5 data, among which the most commonly used are h5py and PyTables.

File format. HDF was created by the National Supercomputing Application Center NCSA Graphic Fundamental Task Team (GFTF) in 1987.

An architecture-independent software library and file format, dedicated to satisfying the need to move scientific data between the various computing platforms used by NCSA

According to the needs, currently supported by the non-profit team HDF Group, its task is to ensure the continuous development and storage of HDF5 technology in HDF

Continuous accessibility of data.

h5py

The HDF5 file is a container for storing two types of objects. The two types of objects are:

  • dataset: a data collection similar to an array;
  • gropp: A container similar to a directory, which can contain one or more datasets and other groups.

The HDF5 file starts from a group named /, all datasets and other groups are included in this group, when

When operating HDF5 files, if you do not specify the group explicitly, the dataset refers to the dataset under "/" by default, and other types

Group names that seem to be relative file paths are relative to /.

Both dataset and group of HDF5 files can have descriptive metadata called attribute.

To manipulate HDF5 files with h5py, you can use group like a directory, and use dataset like a numpy array.

Using attributes like a dictionary is very convenient and easy to use.

solution

debugging processes:

  1. Directly enter conda install -c anaconda hdf5=1.10.4 in the CMD window, and the error is still reported after re-run, and the error message is Headers are 1.10.4, library is 1.10.5
  2. Activate the corresponding tensorflow environment, enter conda install -c anaconda hdf5=1.10.4, still report an error
  3. Enter pip uninstall h5py command, and then enter pip install h5py, this problem is solved
(tf_2.1_py_3.6) C:\Users\hbwhx>pip install h5py
Collecting h5py
  Downloading h5py-3.1.0-cp36-cp36m-win_amd64.whl (2.7 MB)
     |████████████████████████████████| 2.7 MB 66 kB/s
Collecting cached-property
  Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Requirement already satisfied: numpy>=1.12 in c:\develop\anaconda3\envs\tf_2.1_py_3.6\lib\site-packages (from h5py) (1.19.5)
Installing collected packages: cached-property, h5py
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
deepctr 0.9.0 requires h5py==2.10.0, but you have h5py 3.1.0 which is incompatible.
Successfully installed cached-property-1.5.2 h5py-3.1.0

Summary

# Uninstall h5py, then reinstall h5py
pip uninstall h5py
pip install h5py
created at:01-03-2022
edited at: 01-03-2022: