https://github.com/insarlab/MintPy

sort by:
Revision Author Date Message Commit Date
edebebd version: add version 1.3.3 tag (#761) 14 April 2022, 21:14:12 UTC
fb88aa4 add iono ramp calc from GIM TEC products + add simulation/iono.py sub-module: - add iono mapping functions from Yunjun et al. (2022) - add JPL high resolution GIM I/O - add GIS low resolution GIM downloading, I/O and plotting + add iono_tec.py for high level usage - auto download IGS TEC files, similar to tropo_pyaps3.py - call simulation/iono.py to read and calculate iono ramps + add data/top_tec_perc_s1.py from Yunjun et al.. (2022) + update docs accordingly - docs/api/module_hierarchy.md: add simulation/iono - setup.py: add entry point for iono_tec.py 23 March 2022, 19:52:52 UTC
8edc973 plot very long TEC TS file + coord.geo2radar(): support file longitude exceeding 180 in both end, e.g. the GIM IONEX file + plot.auto_multilook_num(): turn OFF multilooking for small spatial size, even if the over size is large due to the large time dimension, to speed up the reading. Otherwise, it will do the epoch-by-epoch reading with indexing, which is slow. + tsview: for very long TS (> 1000 acquisitions), do not print out displacement TS in the terminal, reduce the default marker size by 10X. 23 March 2022, 19:52:52 UTC
c0d6c3a asc_desc2horz_vert: speedup pixel-wise inversion via win-by-win (#756) + asc_desc2horz_vert.py: - use a window-by-window strategy to speedup the pixel-wised inversion - add get_design_matrix4east_north_up() for future 3D decomposition - bug fix for --oo option during file writing + gps.GPS.read_displacement(): add self.date_list member variable, for easy use + stack.timeseries: remove get_design_matrix4time_func, as it's not used anywhere in mintpy anymore, to avoid circular imports 21 March 2022, 03:53:09 UTC
84421d5 readfile.read_snap_dim(): bugfix on WAVELENGTH unit (#755) + utils.readfile.read_snap_dim(): bugfix on WAVELENGTH unit + asc_desc2horz_vert: pass angles in deg instead of rad + utils.plot.auto_adjust_xaxis_date(): fix a bug introduced in https://github.com/insarlab/MintPy/pull/734 + local_oscilator_drift.get_relative_range_distance(): typo fix + docs/README: update aux links in Yunjun et al. (2019) 19 March 2022, 22:50:47 UTC
44dfae4 get_design_matrix4time_func(): bugfix for `poly_order = 0` (#754) + utils.time_func.get_design_matrix4time_func(): resolves a bug that creates design matrices with columns of zeros, when the polynomial order is zero. 19 March 2022, 03:44:32 UTC
bbbaf2b image_stitch - bugfix for bool / integer matrix (#751) Currently `image_stitch.py` can't correctly merge mask files. It puts the merged arrays into an array initialized as `np.nan`. Nans evaluates to `True`, meaning that we are left with blocks outside the data region that are set to `True`. + Adding a simple check to see if we're merging non-float arrays, then set the default values of `fill_value` to `0` instead of `np.nan` fixes the issue. + use vprint() to shorten the print() if check 18 March 2022, 05:44:57 UTC
bd8795f closure_phase_bias.py - mask for areas susceptible to closure phase bias (#740) + add a new script `closure_phase_bias.py`, to create a mask for areas susceptible to closure phase bias, based on Zheng et al. (2022, TGRS, under review). + setup.py: add entry point for the new script 10 March 2022, 04:57:34 UTC
6e39b67 ifg_inv: bugfix for design matrix w/ date1 > date2 & release memory occupied by covariance matrix (#750) + cluster.DaskCluster.collect_result(): support None in the output list, as an alternative to to numpy matrix, to be more flexible + stack.ifgramStack.get_design_matrix4timeseries(): - support non-sequential network (where the first date is not always the earlier date) - use the same reference date by default for a single reference network + ifgram_inversion.py: - estimate_timeseries(): comment out the B matrix inverrtability check, after correcting the B matrix bug above for non-sequential network - ifgram_inversion_patch(): set ts_cov = None if not calc_cov, to release memory from the huge 4D ts_cov matrix Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 08 March 2022, 20:50:54 UTC
575f32b docs update for python 3.9 and 3.10 + remove "python<3.9" constraint from installation-related documents: - setup.py - docs/environment.yml - docs/requirements.txt + docs/installation.md: - add note on strict conda channel priority activation - remove the strict channel priority note, as it's currently not default in conda yet and NOT working all the time, especially when install mintpy together with isce2=2.6 + .circleci/config.yml: - activate the strict channel priority to avoid installation packages with conflicts versions - remove unnecessary cd command while runing testing on example datasets as it's done inside test scripts already + with the above changes, $MINTPY_HOME/tests/test_smallbaselineApp.py passed without errors on both python 3.9 and 3.10 environments + add mintpy.utils.constants module [preliminary] 22 February 2022, 02:09:40 UTC
8b15058 transection: support distance unit in pixel + plot_transection: - bugfix for the pip installed version - support transection with distance unit in pixel. + utils.utils.transect_yx(): add "distance_unit" to the output dict object, with default value of "m". In case there is not enough metadata to calculate the ground pixel size in meters, such as the raw isce-2 files, set "distance_unit = pixel" and continue. + utils.readfile.py: - read_snap_dim(): force int dtype for A/RLOOKS - remove TRACK_NUMBER from standardMetadata as it has not been tracked before, and has already been saved in the original snap metadata info. + simulation/variance.sample_data(): use np.random.default_rng() 22 February 2022, 02:09:40 UTC
ef3d271 gps.get_gps_los_obs(): use meta / obs_type for args w/ example usage + utils/utils0.four_corners(): switch return order from WESN to SNWE, for consistency. Adjust its usage in the following scripts: - asc_desc2horz_vert.py - save_kmz.py - gps.py + gps.get_gps_los_obs(): - use "meta" and "obs_type" to replace "insar_file" for input arguments, for more explicit and straightforward usage. - add example function usage in the comments. - GPS.get_gps_los_velocity(): bugfix for more robust performance + utils.plot.plot_gps(): bugfix when no GPS found. 22 February 2022, 02:09:40 UTC
9d6d410 add test_asc_desc2horz_vert.py 22 February 2022, 02:09:40 UTC
fe09fc3 asc_desc2horz_vert.py: add --geom-file option Support input geometry files for precise pixel-wised decomposition, which is important for large area analysis. + add -g / --geom-file option to specify the corresponding geometry files. + cmd_line_parse(): fix a bug while checking the reference point when two input files does not has the same spatial coverage, by using only Y/X_FIRST/STEP, without converting to REF_Y/X. + get_overlap_lalo(): support a list of attributes dicts >= 2 + get_design_matrix(): refactor to take matrix as inputs + asc_desc2horz_vert(): - refactor to take multi-dimension matrices as inputs directly. - for pixel-wise inversion, use a for loop + merge main(), asc_desc_files2horz_vert() and write_to_one_file() all into one function: run_asc_desc2horz_vert(). 22 February 2022, 02:09:40 UTC
23550ca image_stitch: support multi-dataset files + support input files with multiple datasets, e.g. geometry files. + If input files do not have the exact same list of datasets, only the common elements of datasets will be used. + For geometry files, ignore zero incidenceAngle pixels in the geometry file, so that one could feed them without masking in advance. + show all figures at the end, for a smoother experience 22 February 2022, 02:09:40 UTC
006f1f9 bugfix: timeseries2velocity `--save-res` excluded dates (#746) Save timeseries_res: instantiate the output hdf5 file accounting for the excluded dates Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 14 February 2022, 19:37:13 UTC
e9b0104 refactor prep_snap.py / add readfile.read_snap_dim() + rename prep_snap.get_ellpsoid_local_radius() to utils.utils0.xyz_to_local_radius() + utils.readfile: - add read_snap_dim() to read SNAP *.dim file with XML modules, instead of searching/parsing in text file in the old prep_snap.py, for simpler, more robust and faster performance. - read_isce_xml(): 1) add if check for coordinate1/2.delta/startingvalue; 2) force lower() case key names, to produce consistent output + significantly simplify prep_snap.py 13 February 2022, 07:41:42 UTC
c316d32 improve *.unw file handling in mask/ref*_point/tropo_pyaps3.py + two 2-band *.unw file IO in the following scripts, instead of single band for phase only, to preserve the native *.unw file format: - mask.py - reference_point.py - tropo_pyaps3.py +ptime.(yy)yymmdd_date12(): support both str and list type inputs 13 February 2022, 07:41:42 UTC
7e321b6 gen*_mask: add --vx/vy/vroipoly options + generate_mask.py: add --vs / vy / vroipoly options to specify the AOI for thresholding operations, the area outside will be untouched. This is usef to create an mask, without pixels on the fault, for phase unwrapping of co-seismic deformation. 13 February 2022, 07:41:42 UTC
592f186 view: add --no-data-value to manually specify the value to ignore + view.py: add --no-data-value option to manually sppecify the value to ignore, for plotting only + plot.plot_colorbar(): a more robust cbar extend style checking, to ignore the numerical precision level value leakage. 13 February 2022, 07:41:42 UTC
2499eeb prep_fringe: write ifgramStack for connected components (#735) + add `--geom-only` option for only creating geometry (useful for making masks) + add `-r / --range` and `-a / --azimuth` option to specify looks (this is only for tracking metadata), if extra multilooking is applied after fringe processing. + create `inputs/ifgramStack.h5` file if *.unw.conncomp files exist. The non-first-date reference date is supported when writing ifgramStack to be more agnostic against the potential change in fringe output in the future. 10 February 2022, 08:02:28 UTC
80df178 fix bool array index bug for `h5py>=3` (#743) + docs/requirements: remove `h5py<3` constrain + fix bool array index bug for `h5py>3` in the following scripts: - objects/stack.py - utils/readfile.py + timeseries2velocity: correct typo in example string 10 February 2022, 06:38:15 UTC
2acac3f save_kite.py - convert to meters for UNIT in radians (#738) 10 February 2022, 05:37:54 UTC
569dd96 resample: bugfix for auto_lat_lon_step_size() via merged src/lut_meta dict (#737) + merge `lut_meta` and `src_meta` and pass to `utils0.auto_lat_lon_step_size()` for a better chance of having enough metadata to calculate the pixel area constrained new lat/lon step. + fix the mis-functioned `try/except` fallback logic bug by using `except KeyError:` instead of `except AttributeError:`. 06 February 2022, 03:59:20 UTC
b33fefc generic multi-dataset file (e.g. velocity) support in `add/diff` (#732) + `add.py` & `diff.py`: generic multi-dataset file support, by searching the common dataset list among all the input files, and doing a for loop for them. Ignore dataset names if all input files have one dataset, to support single dataset files with different dataset names, e.g. mask, mask and waterMask. In the later case, the 1st file dataset name will be used as the reference. + `utils1.run_deramp()`: when operating on velocity files, operate on velocity dset as default Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 03 February 2022, 06:28:29 UTC
70d66d1 initial NO_DATA_VALUE support + add a new standard metadata: NO_DATA_VALUE + readfile.py: - read_attribute(): grab NO_DATA_VALUE with 'none' as default - add get_no_data_value() to get the value of the input file - add SPECIAL_STR2NUM() to facilitate no_data_value handling - read_isce_xml() / read_gdal_vrt(): grab NoDataValue if exists. + view.py: mask out pixels with no_data_value for single subplot + tests/config/*.txt: speedup testing by decrease plot.maxMemory to 2GB to use more parallel processes for plotting 31 January 2022, 19:15:24 UTC
7008581 refactor load2hdf5.py + rename `legacy/load_hdf5.py` to `legacy/load2hdf5.py` + refactor to be more flexible on the input/output file structure + add --sub-x/y + add --meta to add custom metadata + support --dname timeseries with "date" dataset added automatically + add DATA_TYPE_STR2OBJ to support most common numpy data types + check if the output file exists with --force option 31 January 2022, 19:15:24 UTC
04e4661 support plotting of SLC stack and GRACE time-series + identify SLC stack as "FILE_TYPE==timeseries" and "DATA_TYPE==complex(64/128)" + stack.timeseries.open): change CENTER_LINE_UTC from required to optional as SLC stack might not have it + readfile.read_attribute(): check SLC stack and reset UNIT to 1 + coord.geo2radar(): ensure the consistency between point longitude and source file, regardless of the source file convention in (-180, 180] or [0, 360) conventions. For geocoded files only. + plot: - auto_colormap_name(): use gray as default cmap for slc stack - auto_adjust_xaxis_date(): smart every_year/month for extra long time series, e.g. GRACE products. + view: - read_data4figure(): take abs() if data type is complex - move code to form check_map_projection() for easy re-use, e.g. in tsview.py + tsview: - pass `disp_unit` directly for the model estimation result display, for more simple and robust performance against input files that are not in the unit of meters, e.g. giant product or GRACE products. - grab WAVELENGTH if and only if wrap-range is 2 pi. This allows applying --wrap-range option to files w/o WAVELENGTH attributes, e.g. the GRACE products. - support cartopy map projection and its related features, e.g. --coastline, via view.check_map_projection() - take abs() if data is complex - bugfix for --flip-lr/ud options 31 January 2022, 19:15:24 UTC
18539d2 ref_point: add ref_lat value check + reference_point: add a validity checking for input `ref_lat` value + info: --num will turn ON --date - --num will turn ON --date, so that one can run info.py timeseries.h5 --num to print date list and number, without adding --date option, for conveniency. - remove --slice --num combo, as it's not used much, to simplify the code. + sensor: use rs1/rs2/rcm for radarsat series 31 January 2022, 19:15:24 UTC
d219448 bugfix for `timeseries2velocity.py --bootstrap --save-res` (#733) When running `timeseries2velocity.py` with bootstrapping, we use the function `time_func.estimate_time_func` to repeatedly fit the time series data. This function returns the design matrix, `G`, however when bootstrapping we don't assign the design matrix to a variable, meaning that if we try to save the residual file we get the error `UnboundLocalError: local variable 'G' referenced before assignment` at line 702. If we want to output the residual when bootstrapping, we need to get the full design matrix. I've added a line of code to do this. 28 January 2022, 05:55:28 UTC
cfa7991 image_stitch: bugfix & update plotting (#730) + fix the `AttributeError: 'AxesSubplot' error, because `fig` can not be used as return for subplot() anymore. + image_stitch: improved auto plot settings Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 26 January 2022, 05:41:51 UTC
7dd7e1d bugfix in load_data.py & plot_network.py (#729) + load_data.py: fix a runtime bug when re-run after removing the ISCE stack processing folders + plot_network: fix the output file names bug + docs/install: a little bit more notes on dask 25 January 2022, 00:03:56 UTC
f8a3fbe installation: add dask config to post-install setup (#727) 21 January 2022, 19:35:05 UTC
1727c7d plot_network: color-code network/matrix plot based on temp/perp baselines (#693) + `plot_network`: support color-coding temporal and perpendicular baseline via `-d tbase/pbase` and range control still via `-v` option. + utils1.get_residual_std(): add `std_file` variable to the return Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 21 January 2022, 02:50:23 UTC
5c4c54b fix typo in dask documentation (#726) * Minor typo in dask documentation 21 January 2022, 00:25:59 UTC
0cc5665 Remove GUNW reference from HyP3 directory structure (#724) 17 January 2022, 22:24:34 UTC
492719e bugfix for hyp3 reading, circle CI and pip/conda install (#723) + readfile.read_attribute: move GDAL for .tif/.grd file up on the priority than non-rsc metadata files, to fix reading issue related to downloaded hyp3 *.tif file + docs: add notes on preparing hyp3 interferograms via Vertex website + circleci/config: switch circle CI resource class from medium to large + tests/config/*.txt: increase the max plot memory from 2G to 4G + smallbaselineApp: bugfix for --plot when installed via pip/conda + add_attribute: bugfix for input argv checking when installed via pip/conda 17 January 2022, 20:32:56 UTC
70d606e improved auto lat/lon step for geocoding Add utils0.auto_lat_lon_step_size() to calculate the default output lat/lon steps for geocoding. By default, the current version ensure the same number of rows and columns between input and output files; while this new version ensures the same pixel area between the input and output files and with proper dimension ratio using the bounding box of an rotated rectangle. Thus, this is a better version, especially when the azimuth angle is far from the pure north-south, such as the UAVSAR flight path, or when the space-borne SAR track is long. 17 January 2022, 05:55:10 UTC
7131276 use numpy as DATA_TYPE value standard + readfile/writefile: use numpy data type string as standard for mintpy, and use it for the convention translation during data I/O. - DATA_TYPE_ENVI2NUMPY - DATA_TYPE_NUMPY2ENVI - DATA_TYPE_GDAL2NUMPY - DATA_TYPE_NUMPY2GDAL - DATA_TYPE_ISCE2NUMPY - DATA_TYPE_NUMPY2ISCE + ifg_inv: use diff name for numInvOffRg/Az.h5 files 17 January 2022, 05:55:10 UTC
598ac8a support UAVSAR read / prep / SET correction + readfile: support reading UAVSAR .ann/lkv/llh files - add read_uavsar_ann() to read the *.ann file into an dict object - read_attribute(): add metadata preparation for *.lkv and *.llh file - get_slice_list(): provide explicit list of dataset names for *.lkv and *.llh files - read_binary_file(): support multiple bands for non-traditional-InSAR processors. + prep_isce: add generate_random_baseline_timeseries() to support `-b random` input to generate a bperp time series with random values as place holders for mintpy processing. + solid_earth_tides: support T in time [for UAVSAR] 17 January 2022, 05:55:10 UTC
c556817 4D matrix IO for TS full covariance calc/propagation + utils.readfile.read(): read 4D hdf5 file reading + utils.writefile.layout_hdf5/write_hdf5_block(): support 4D matrix wirting to HDF5 file + objects/cluster.collect_results(): support 4D matrix + ifgram_inversion: calc TS full covariance matrix and save as a 4D matrix in HDF5 file - rename --calc-std to --calc-cov - rename estimate_timeseries_std() to estimate_timeseries_cov() - use the linear propagation of a unweighted estimator for the error propagation - use np.linalg.multi_dot() to speedup chain dot operations + timeseries2velocity.py: add --ts-cov-file option to support full cov matrix for error propagation - rename --ts-std-file to --ts-cov-file - support both 4D full covar and 3D var time series file as input - apply the linear propagation for ordinary least squares, to be consistent with the time func estimation. Previously, the weighted least squares version is used in this step, which is not appropriate. - update notes on the math equations in the comments. 17 January 2022, 05:16:43 UTC
8e9f8d6 ifg_inv: keep high SNR pixels for offset + ifgram_inversion.mask_stack_obs(): - distinguish between msk and msk_data for more flexibility - keep high SNR pixels for offset when masking based on offsetSTD, with an hardwired threshold for now. - calc offset SNR via (off - off_med) / off_std 17 January 2022, 05:16:43 UTC
72df5b6 setup: add license 17 January 2022, 05:16:43 UTC
9b66885 add `mintpy.unwrapError.connCompMinArea` option (#719) + add `mintpy.unwrapError.connCompMinArea` option with default value of 2.5e3 to replace the previously hardwired value, for small area high resolution dataset. + pass the above option value to `unwrap_error_bridging.py` and `unwrap_error_phase_closure.py`. + `solid_earth_tides.py`: calculate adaptive step size based on the input file metadata, to replace the hardwired 3e3 value. + `readfile.py`: add support for reading slice list of `slcStack` 13 January 2022, 01:02:29 UTC
4357b8c API to read GMT lonlat file from KML to plot faults (#717) This PR adds a function to read GMT lon/lat file so that one can plot the faults dataset (created by GMT from KML file) on top of the view.py plot, in Python level ONLY. + add `utils.plot.read_gmt_lonlat_file()` to read and plot faults on top of `view.plot_slice()` output - add notes and examples on how to prepare the GMT lonlat file from KML file, and use the function in python + `objects/gps`: - add horz angle to the filename if it's horz and print out file name before the long calc - print warning instead of raise error if no GPS found - more print out num while comparing with GPS + `tsview`: support TS file w/o REF_Y/X + `legacy/2to3_timeseries`: change `P_BASELINE_TIMESERIES` to optional 05 January 2022, 03:20:51 UTC
7fb4108 fix argv bug in view.py for pip installed version (#714) + utils0: add round_up_to_odd() + github bot: fix typo + view: fix argv bug for pip installed version. Catch the manually specified arguments via `sys.argv` for cmd or via `iargs` through python call, and save it to `inps.argv` for later-on checking. Fixed the same issue in save_gbis.py as well. + view: add `--math rad2deg/deg2rad` 23 December 2021, 00:33:09 UTC
7b81971 ignore pyresample warning for xarray/zarr (#707) + objects.resample: Use warnings.catch_warnings() to ignore the potential UserWarning while import pyresample without xarray/zarr installed. + objects.coord.geo2radar(): support lon > 180. + README: use shields API instead of hardwired value to auto grab the version number in the badge 30 November 2021, 02:53:19 UTC
ce7a27c env: revert matplotlib<3.5 & rm pykdtree/xarray/zarr (#706) * revert "set matplotlib<3.5 temporarily due to upstream repo bug". This reverts commit d76edb928ba40e32ef8a7935bffb1644b613f099. * rm `pykdtree/xarray/zarr` for `pyresample` as they are now properly installed by `conda install pyresample`, thus, no manual setup is needed anymore. 27 November 2021, 03:28:39 UTC
d76edb9 set matplotlib<3.5 temporarily due to upstream repo bug (#704) which leads error while saving figures to PDF files. 23 November 2021, 01:54:43 UTC
ca29361 wrap up version 1.3.2 (#703) + version.py: add release_history as a list of Tags - use Tag from the collection module to save the release history explicitly - distinguish between release version number/date and version number/date - update version_number/date/description usage in __inti__.py, view.py and smallbaselineApp.py + bump the release version number to 1.3.2 22 November 2021, 03:04:47 UTC
2fb0a7d update installation note, env setup and circle-ci testing (#700) + simplify the installation since now `pyaps3` and `pysolid` are available via conda-forge, so they are no longer need to be manually cloned and installed anymore. This simplifies the environment setup files and installation notes. + since #659, there's no longer a need to manipulate PATH or PYTHONPATH for mintpy anymore, as a "pip install" will correctly place the mintpy package and all executable scripts to be available in the environment. Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 21 November 2021, 07:47:21 UTC
eb3bd63 tropo_pyaps3: update account check for pyaps3-v0.3 21 November 2021, 07:40:49 UTC
ce67891 gps: add --ex-gps / --gps-ms / check time overlap percentage + view/plot: add --ex-gps option to manually exclude a list of GNSS sites from plotting + plot_gps(): for sites with nan value, do not plot dots, instead of plotting a transparent one previously, to be more clear; do not plot ref GNSS site label for a cleaner figure for publication + gps.get_gps_los_velocity(): ignore sites if its observation time overlap with InSAR time for less than 25%. 21 November 2021, 07:40:49 UTC
f600ad8 ts2vel: add --ref-lalo option to support changing reference point in lat/lon for estimation. 21 November 2021, 07:40:49 UTC
b10d2fb sim: add cross_correlation_std() + decorrelation: add cross_correlation_std() based on Balmer and Eineder (2005) and De Zan (2014) for the coherent and incoherent speckle tracking. + network: add simulate_coherence_v2() for more pythonic usage 21 November 2021, 07:40:49 UTC
adac2b1 bugfix for info.py --slice (#701) + info.py: fix bug introduced in #674. Thank you @falkamelung for reporting it. + tropo_pyaps3.snwe2str(): simplify syntax + README/badge: use native color for circle CI test result 19 November 2021, 19:23:54 UTC
59819bb coord.lalo2yx(): replace np.float128 with np.longdouble for Windows (#699) * Replace np.float128 object with np.longdouble for a platform-neutral syntax 18 November 2021, 07:21:39 UTC
d2727e0 dask: support numWorker input in percentage with improved stability + cluster.DaskCluster: - run(): set the num of scaled workers to the num of boxes, to avoid unnecessary resource initiation - format_num_worker(): revert "divide num core by num of threads per core" (#691) as it's not right. - format_num_worker(): support num of workers in percentage, e.g. 80% to use 80% of all cores + ifgram_inversion: use ceil instead of rint to prevent the occurrence of extra small patches. + cluster.split_box2sub_boxes: - add a min step size constrain - trim the final number of boxes, to avoid negative dimension 13 November 2021, 23:24:38 UTC
e14a53e sensor/ERS: add pulse_length and chirp_slope + sensor: add pulse_length and chirp_slope for ERS + prep_hyp3: replace inc_map with lv_theta, to be consistent with readthedoc + save_kmz: more doc on radar-coded file support + .circleci/config.yml: add resource_class 13 November 2021, 23:24:38 UTC
928db34 prep_hyp3: ignore whitespace in keys (#690) ignore the whitespace in all keys while reading the HyP3 metadata file, to be immune to the potential whitespace adding/removing in the upstream HyP3 metadata, such as the UTC time key change from `UTCtime` to `UTC time`. 13 November 2021, 21:54:47 UTC
92e00ad parallel: divide num_core by num_thread if available (#691) + cluster.format_num_worker(): divide the number of cores by the number of threads if found [for Linux only] using lscpu command. Note again that this only affects the translation of numWorker = all. + smallbaselineApp.plot_result(): translate numWorker = all + update API hierarchy of cluster.py as it now depends on utils0.py 09 November 2021, 08:41:01 UTC
fe32d26 spatial_filter: refactor double difference (#687) * spatial_filter: refactor double difference - combine the local and regional filter into one filter to reduce number of convolutional operations - change regional filter shape from a disk to a doughnut * spatial_filter: update double_difference docs 04 November 2021, 01:29:22 UTC
8472709 Dockerfile: rewrite using Micromamba Image (#685) * Dockerfile: rewrite using Micromamba Image - Rewrites the MintPy dockerfile using the mambaorg/micromamba image - Forces Python=3.6 for faster image build - Sets the user as micromamba, not root - Optionally allows the image to start with a Jupyter Lab instance * Dockerfile: fix codacy issues - tag micromamba version - remove apt-get lists * Update docker/Dockerfile Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> * Dockerfile: reconfigure user * Dockerfile: reconfigure user 2 * Dockerfile: pull-request updates - hard code MINTPY_HOME and PYAPS_HOME - install wget - update installation.md and docker.md * Update docker.md * Update installation.md * Update Dockerfile * Update docker/Dockerfile Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> * Update docker/Dockerfile Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 04 November 2021, 01:27:49 UTC
6b2ce05 sensor: add seasat PRF and NISAR rg pixel size (#684) + sensor: - add seasat PRF - add NISAR range_pixel_size + ERA5: consistent naming with better error msg - use ERA5 thoughout mintpy without hyphen, to avoid confusion. - smallbaselineApp.py: raise error for un-recognized GAM names + timeseries_rms: simplify plot + utils/plot: omit "Time [year]" xaxis label to save space as it's not informative + prep_snap: fix typo 23 October 2021, 06:00:48 UTC
2ed0313 writefile.layout_hdf5(): bugfix when using binary file as the reference (#682) + utils.writefile.layout_hdf5(): support using binary file as structure reference to create empty HDF5 files, using the metadata and slice(s) info. + simplify the code with vprint() 18 October 2021, 02:51:43 UTC
11ae0e5 sensor: add NISAR_L and altitude (#677) + sensor: - add NISAR L-band SAR: only a few parameters from the handbook - add altitude value for all sensors - add range pixel size for ALOS-1 stripmap + utils.utils0.py: add is_number() + view: YYYY-MM-DD style subplot title for py2-mintpy TS file 10 October 2021, 00:16:29 UTC
65c856b view: bugfix for DEM masking (#673) * fix bug related to `--mask-dem` option when DEM shades and contours do not exist at the same time. 02 October 2021, 23:04:57 UTC
bdb4669 codacy style cleanup (#674) 27 September 2021, 20:04:20 UTC
eb30167 add generic writefile.write_binary() + writefile: add write_binary() for generic binary file writing, similar to the generic binary reading in readfile.read_binary(). + writefile.write(): use write_binary() for simpler and more powerful writing support 27 September 2021, 03:26:26 UTC
c4e73ee attr: add BANDS and INTERLEAVE + docs/api/attributes: add BANDS and INTERLEAVE to the standard mintpy self-generated metadata list + use "BANDS" instead of "number_bands" across mintpy + use "INTERLEAVE" instead of "scheme" across mintpy + readfile: grabbing BANDS and INTERLEAVE through standard metadata dict whenever they are available + writefile: - write(): update BANDS according to the input data list - write_isce_xml(): refactor to take mintpy attributes dict as input, for user friendly usage + add/update write_isce_xml() usage in geocode.py, subset.py and multilook.py + utils.plot.auto_figure_title: keep suffix for single-subplot from multi-dset file 27 September 2021, 03:26:26 UTC
9c9b2b6 plot: --mask-gps/dem, --gps-comp horz/vert and American display unit (#670) This PR adds the following new features for more flexible visualization: 1. add `--mask-dem` option to support masking out DEM pixels to be consistent with the masked data matrix 2. add `--mask-gps` option to mask out GNSS sites out of the displayed data matrix. 3. support `--gps-comp horz / vert` to plotting horizontal and vertical GPS components. The horizontal azimuth angle can be adjusted using `--hz-az` option. 4. support American length unit for display. Detailed changes: + asc_desc2horz_vert: set FILE_TYPE to displacement for ifgramStack, timeseries and HDFEOS file types, instead of the previous unique but meaningless name. + objects/gps.py: - get_gps_los_obs(): always write both displacement and velocity to CSV file for simplicity - use az_angle for LOS projection, instead of head_angle, to be consistent with the rest of mintpy convention. - support `horz_az_angle` to project GPS horizontal displacement/velocity to a custom azimuth angle + view.py: - save `self.msk` as the final flag matrix for masked out pixels - turn OFF auto mask file detection when --zero-mask is specified. + utils/plot.py: - prepare_dem_background(): support masking of DEM shade and contour for --mask-gps - plot_gps(): pruning GNSS list based on self.msk for --mask-gps - scale_data2disp_unit(): support American unit: inch, foot, yard and mile. Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 27 September 2021, 00:04:37 UTC
884434f read/write/plot: improved flexibility for h5 file (#671) + `read/plot`: support `geometry` h5 file with other undefined datasets, e.g. "displacement", which is useful for single-pair displacement products for generic sharing purposes. + `read.read_attribute()`: ignore Std while grabbing `UNIT` + `writefile.write()`: add optional `ds_unit_dict` arg to specify different data unit for each dataset, same as `layout_hdf5()` + `geocode`: same h5 compression bwt. input/output 24 September 2021, 05:19:37 UTC
6c7ad9e ifgram_inversion: bugfix for the ionoPhase inversion (#667) * ifgram_inversion: bugfix for ionoPhase inversion + string identification ('ion') for ionoPhase inversion + string identification ('ion') for converting iono time-series unit to meter 22 September 2021, 19:18:04 UTC
670c6a9 sbApp: add mintpy.plot.dpi option (#663) + smallbaselineApp: add mintpy.plot.dpi template option to customize the output figure resolution + sensor: add range pixel size for different ALOS-2 imaging modes + coord/network/ptime: support isinstance() for int/float mixed with numpy dtype 18 September 2021, 03:49:04 UTC
8e285b2 demo: add RidgecrestSenDT71 for hyp3 + add RidgecrestSenDT71 as example dataset for HyP3 - docs/demo*.md: add RidgecrestSenDT71 dataset produced by ASF HyP3 service by Jiang Zhu. - mintpy/data/input_files: add RidgecrestSenDT71.txt + add RidgecrestSenDT71 as test dataset for hyp3 - tests/test_smallbaselineApp: add url for RidgecrestSenDT71 prepared using ASF HyP3 by Jiang Zhu. - tests/configs/RidgecrestSenDT71.cfg: update setups for faster testing + readfile.read_gdal(): use numpy indexing instead of gdal python binding for more robust performance + plot.auto_figure_title(): use custom ds name for generic h5 file 14 September 2021, 19:02:22 UTC
213115b time_func: supp. time of the day in THH(:)MM style This commit adds the capability to feed accurate hour/minute/second info of step/exp/log functions to the design matrix, to support scenarios when the event and SAR acquisition are in the same day, such as The M6.4 Ridgecrest EQ in 4 July 2019. + ptime.yyyymmdd2years(): add seconds arg to add the time of the day info to the output list of years in float + time_func: add seconds arg to pass to ptime.yyyymmdd2years() to support time of the day info to all functions inside + read/prep seconds info to feed to time_func.py in the following scripts: - dem_error.py - timeseries2velocity.py - tsview.py + update example usages in smallbaselineApp.cfg file and arg_group.py 14 September 2021, 19:02:22 UTC
e30dd1d utils: add time_func.py and s1_utils.py + add utils.time_func.py: - move get_design_matrix4time_func() from objects.stack.py. - switch module import/usage in all related scripts: dem_error, gps, simulation, tsview - replace the old func definition with a pointer to the utils.time_func.py for backward compatibility. - move estimate_time_func() from timeseries2velocity.py - switch module import/usage in all related scripts: timeseries2velocity, tsview - set all get_design_matrix4*_func() as independent functions for more flexibility + utils.s1_utils: - add estimate_S1AB_bias() - add get_subswath_masks() + objects/gps.py: check site name during initialization for more informative error msg against typo in "view.py --ref-gps" + docs/api/module_hierarchy.md: add time_func and s1_utils. 14 September 2021, 19:02:22 UTC
6070ab0 ifg_inv: --calc-std for phase/offset and WLS for offset + add weighted LS estimation for offset via offset STD from ampcor [turned off by default, as initial testing does not show improvement over OLS] + add estimate_timeseries_std() for TS STD calculation via linear propagation from the stack of interferograrms or offset pairs. - this corresponds to the uncertainty of decorrelation noise for phase or of cross-correlation estimation for offset - TS STD now can be estimated independent from TS est. regardless of WLS or OLS, via stack_std variable - add get_design_matrix4std() to calc STD with a user chosen reference date, instead of the default 1st date, since TS STD can not be easily re-referenced in time + add skip_invalid_obs() to skip invalid data and update matrice for both phase and offset + adapt calc_inv_quality() for both phase and offset, in order to simplify estimate_timeseries() + use A X = y naming conventions for TS estimation part of the code. + rename some vars for a neutral meaning, e.g.: - pha/off --> obs - ifgram --> stack - num_ifgram --> num_pair + remove obsolete write2hdf5_file() + add print_msg to estimate_timeseries() and calc_inv_quality() 12 September 2021, 04:53:10 UTC
1eb2e57 ifg_inv: mask offset based on STD + sbApp.cfg: set range/azimuthOffsetStd as recommend mask dataset for offset + ifg_inv: - mask out pixel with offset std. dev. LARGER than threshohld, whereas for phase, mask out pixel with coherence SMALLER than threshhold - for offset, skip reference is no REF_Y/X found, otherwise apply referencing [for flexibility] - set inv quality value to 0 for offset - do not convert obs value from nan to zero since the previous skip_zero for phase has been changed to the more generic skip nan for phase and offset. + reference_point: support ifgramStack of offset + ts2vel: improved default output filename for offset TS 12 September 2021, 04:53:10 UTC
cff6e9e load_data: support offset std. dev. + support loading offset std. dev. from ampcor offset_cov.bip file via mintpy.load.rg/azOffStdFile option. + Hardwire no-data value of 99 for offset variance and convert it to NaN during loading process + convert offset variance to std during loading process 12 September 2021, 04:53:10 UTC
7166ef6 load_data: support loading rg off w/o az off supporting loading range offset along without azimuth offset, by adding range offset checking as required datasets during data loading process. 12 September 2021, 04:53:10 UTC
65fe5eb setup: add all executable modules as `entry_points/console_scripts` (#659) + setup.py: - Add all currently executable modules as `entry_points/console_scripts` to replace `scripts` - Drop depreciated `depedency_links` for pysolid as it's now available from conda-forge + Add `main(argv=None)` in add_attribute.py and multi_transect.py to be used in the new setup above + Graceful --help or missing args exit for save_gbis.py and tsview.py. 11 September 2021, 23:41:31 UTC
db15001 speedup testing via *.plot.maxMemory and smaller Fernandina (#658) + smallbaselineApp.plot_results(): - add mintpy.plot.maxMemory option to further reduce memory usage while calling view.py inside smallbaselineApp.py - remove mask_file option as it's auto found by view.py and single mask_file does not apply to both radar and geocoded TS files - add missing --wrap option, otherwise --wrap-range won't work. + tests/configs/*.txt: set mintpy.plot.maxMemory = 2 to increase number of parallel processes by 2X and to speedup regular testing by 25% (2X on the reading part) + tests/test_smallbaselineApp.py: update to the new FernandinaSenDT128.tar.xz file DOI for smaller file size and faster testing (2X speedup from 6 mins to 3 mins) + utils.read_timeseries_yx/lalo(): add zero_first arg so that one could turn it off to read the absolute displacement from ERA5.h5 file. The default value is set to True for backward compatibility. 09 September 2021, 21:11:45 UTC
7757a2e view: add WARNING msg for un-support option scenarios (#657) + view: print warning msg for geo-only options if input file is in radar-coord + view: print warning msg for single-sub plot options if it has multi-subplots + circleci/config.yml: update path for the test_smallbaselineApp.py + objects/coord: more robust debug_mode plotting for _get_lookup_row_col() 08 September 2021, 04:32:57 UTC
0824c57 rename test to tests 07 September 2021, 22:12:11 UTC
185247e install: pykml min version constraint + add pykml >=0.2 to all requirement files + remove pip install pykml from circleci, as it's now included in the mamba install call 07 September 2021, 22:12:11 UTC
90abc35 switch pykml install from pip+git to conda (#655) + install: Switch pykml installation to conda via conda-forge channel as it is now updated to the python2/3 version on both conda-forge channel and PyPI, instead of using pip git+https to the GitHub repo. + tropo_pyaps(3).py: remove usage of subprocess + info/versoin: typo fix 06 September 2021, 19:28:55 UTC
8bda09b ts2vel: save residual displacement TS via --save-res + add --save-res option to turn ON saving residual displacement time-series [OFF by default] + add --res-file option to specify the residual file name [timeseriesResidual.h5 by default, same as dem_error.py] 05 September 2021, 18:41:10 UTC
9b1ede5 plot: handle very long TS file with YYYY-MM-DDTHH:MM format Add the following changes to support the reading and plotting of TS files: + with >8000 dates and + with dates in isoformat, e.g. YYYY-MM-DD[*HH[:MM[:SS[.fff[fff]]]]. Detailed changes includes: + readfile.read_hdf5_file(): support YYYY-MM-DDTHH:MM format in the date dataset with more relaxed but accurate detection of date and "-" + info.py: do not list dates and dates in years, because the former is covered in --date option and the later is not used, to be more compact on TS output + utils.plot.scale_data2disp_unit(): set disp_unit to data unit if it's un-scalable data + view/draw_scalebar(): do not plot scalebar if the longitude span > 30 deg + ptime.py: - get_date_str_format(): support the isoformat date and datetimes - get_date_range(): extend dmin/dmax to any format supported by get_date_str_format() - naming conflict fix for `dt` 05 September 2021, 18:41:10 UTC
cf83ccc init test config file for hyp3 05 September 2021, 18:41:10 UTC
b5e6e5b __init__: rm `sys.path.insert()` and force internal imports (#653) + `mintpy/__inti__.py`: - remove try/except for print and MINTPY_HOME as it's not really used anywhere, and their corresponding module imports. - remove `sys.path.insert()` calls. These were replicates from the isce-2 style (isceobj, iscesys, etc.), but they are not really applicable to and used in mintpy, nor had we encouraged in any other files besides this one. **NOTE**: however, this might still potentially impact downstream workflows, thus, should be noted in the release notes/changelog. + test_smallbaselineApp.py: - use the relative path to locate config file, instead of using mintpy module - remove the occurrence of MINTPY_HOME - remove print view.py cmd as it's now included inside view.py if --noverbose is turned ON - use `pathlib.Path().resolve()` instead of `os.path.dirname()` 01 September 2021, 06:39:50 UTC
c7c12b0 ptime.get_date_range(): add dunit and out_fmt (#651) + ptime.get_date_range(): add dunit and out_fmt - add `dunit` arg with default value as before - add `out_fmt` arg with default value as before + smallbaselineApp.plot_result(): use floor() instead of rint() as a more conservative num calc for parallel processes + unwrap_error_phase_closure: print out # of ifg/triplets 29 August 2021, 06:56:00 UTC
ab01556 Add doc on install MintPy on Windows (#647) + add documentation on installing in windows + fix a few bugs that prevent running through the example + add an independent link to installation on read the docs. Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> 29 August 2021, 05:21:27 UTC
f7b2b4f Triplet design matrix speedup by 10X (#650) + simplify the triplet matrix building by using `itertools.combinations(date_list, 3)`, instead of the previous `for` loop search and check + create inverse mapping objects to speed up the indexing. Testing shows more than 10X speedup for large matrix. 28 August 2021, 23:52:57 UTC
08ae0e0 sbApp --plot: limit num_cores based on max memory (#646) + smallbaselineApp.py: limit the number of parallel processes for plotting based on the max memory from the template file, to avoid exceeding the memory. Circle CI kills processes because of this. Note that matplotlib reserved much larger memory than it actually uses. For users who want to use a lot of parallel processes for plotting, please increase your `mintpy.compute.maxMemory` setting. + view.py: add --memory option. Note this is NOT used in the smallbaselineApp.py + plot.auto_multilook_num(): - more detailed multilook num list, for a more detailed memory manegement. - add max_memory arg to adjust according. Details are in the code comments. + add quotes to the version control for conda/mamba install in command line in docs/installation.md and .circleci/config.yml files + circleci: use pip from conda to avoid warning msg + docs/api/attributes.md: add EPSG 25 August 2021, 01:48:10 UTC
0c5d79d save_gdal.py: support UTM projection (#645) + save_gdal.py: Fix bug when dealing with UTM projections, such as provided by hyp3, by adding additional arguments for EPSG code. + readfile.read_gdal_vrt(): add "EPSG" to file attributes + utils0.py: add new function `utm_zone2epsg_code()` 23 August 2021, 20:30:56 UTC
ab1276e docs: fix typo in installation.md (#643) 19 August 2021, 19:38:41 UTC
949437d simplify generic I/O logic for cpx binary files (#642) This PR modifies readfile/writefile to read and write complex binary files (from isce-2) directly in complex data type by default. For plotting in view.py, which requires real data type, I add readfile.get_slice_list(no_complex) option to translate complex into mag/phase for plotting. This option is turn off by default for generic I/O. + readfile: - read_binary_file(): for "slc", set cpx_band to "complex" unless mag/phase is clearly specified. - get_slice_list(): 1) ignore meaningless geo/rdr file extension; 2) add "no_complex" arg to switch between complex (for I/O) and mag/phase (for plotting) + writefile: - write(): for .slc/.int, write complex directly - write_isce_xml(): add "image_type" arg to initiate different ISCE-2 image objects for better compatiability - write_isce_file(): support file_type == 'isce_slc' - comment out obsolete write_complex64() as write_complex_float32() is equivalent but simpler and faster. + subset/multilook: update usage of writefile.write_isce_xml() + view: update usage of readfile.get_slice_list() Other minor changes: + subset.py: add --sub-x/y/lat/lon and --subset-x/y/lat/lon to be consistent with subset option names in other scripts. + view.py: add --nmli / --num-multilook for --multilook-num + simulation.simulation.py: typo fix and add missing wvl arg while calling timeseries2ifgram() + objects.sensor: typo fix for doppler_bandwidth 19 August 2021, 01:28:14 UTC
2ff5a78 bugfix in tropo_pyaps3.check_pyaps_account_config() (#641) * readfile.read(): add data_type to customized output data type * tropo_pyaps3.py.check_pyaps_account_config(): typo fix 16 August 2021, 18:30:32 UTC
9cbc317 hyp3: fix incAngle/slantRangeDist + add more attributes (pass full test) (#640) + prep_hyp3: - ensure negative HEADING value - grab CENTER_LINE_UTC from UTCtime directly - put P_BASELINE_* to the ifgram specific metadata - add STARTING_RANGE from "Slant range near" - add RANGE/AZIMUTH_PIXEL_SIZE from hardwired Sentinel-1 info - adjust to the new incidence angle file name 'lv_theta' + stackDict: - convert Gamma inc angle to isce/mintpy inc angle convention while loading the data for hyp3 - consider the above conversion while calc slantRangeDistance from incidence angle for hyp3 + docs/dir_struct*/hyp3: add inc angle and water mask path - remove warning for inc angle as the latest HyP3 is producing the correct one now - add geometry file path in the template for incidence angle and water mask - use Ridgecrest SenT71 dataset for the example in the directory structure 14 August 2021, 23:28:47 UTC
back to top