https://github.com/insarlab/MintPy

sort by:
Revision Author Date Message Commit Date
7fb4108 fix argv bug in view.py for pip installed version (#714) + utils0: add round_up_to_odd() + github bot: fix typo + view: fix argv bug for pip installed version. Catch the manually specified arguments via `sys.argv` for cmd or via `iargs` through python call, and save it to `inps.argv` for later-on checking. Fixed the same issue in save_gbis.py as well. + view: add `--math rad2deg/deg2rad` 23 December 2021, 00:33:09 UTC
7b81971 ignore pyresample warning for xarray/zarr (#707) + objects.resample: Use warnings.catch_warnings() to ignore the potential UserWarning while import pyresample without xarray/zarr installed. + objects.coord.geo2radar(): support lon > 180. + README: use shields API instead of hardwired value to auto grab the version number in the badge 30 November 2021, 02:53:19 UTC
ce7a27c env: revert matplotlib<3.5 & rm pykdtree/xarray/zarr (#706) * revert "set matplotlib<3.5 temporarily due to upstream repo bug". This reverts commit d76edb928ba40e32ef8a7935bffb1644b613f099. * rm `pykdtree/xarray/zarr` for `pyresample` as they are now properly installed by `conda install pyresample`, thus, no manual setup is needed anymore. 27 November 2021, 03:28:39 UTC
d76edb9 set matplotlib<3.5 temporarily due to upstream repo bug (#704) which leads error while saving figures to PDF files. 23 November 2021, 01:54:43 UTC
ca29361 wrap up version 1.3.2 (#703) + version.py: add release_history as a list of Tags - use Tag from the collection module to save the release history explicitly - distinguish between release version number/date and version number/date - update version_number/date/description usage in __inti__.py, view.py and smallbaselineApp.py + bump the release version number to 1.3.2 22 November 2021, 03:04:47 UTC
2fb0a7d update installation note, env setup and circle-ci testing (#700) + simplify the installation since now `pyaps3` and `pysolid` are available via conda-forge, so they are no longer need to be manually cloned and installed anymore. This simplifies the environment setup files and installation notes. + since #659, there's no longer a need to manipulate PATH or PYTHONPATH for mintpy anymore, as a "pip install" will correctly place the mintpy package and all executable scripts to be available in the environment. Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 21 November 2021, 07:47:21 UTC
eb3bd63 tropo_pyaps3: update account check for pyaps3-v0.3 21 November 2021, 07:40:49 UTC
ce67891 gps: add --ex-gps / --gps-ms / check time overlap percentage + view/plot: add --ex-gps option to manually exclude a list of GNSS sites from plotting + plot_gps(): for sites with nan value, do not plot dots, instead of plotting a transparent one previously, to be more clear; do not plot ref GNSS site label for a cleaner figure for publication + gps.get_gps_los_velocity(): ignore sites if its observation time overlap with InSAR time for less than 25%. 21 November 2021, 07:40:49 UTC
f600ad8 ts2vel: add --ref-lalo option to support changing reference point in lat/lon for estimation. 21 November 2021, 07:40:49 UTC
b10d2fb sim: add cross_correlation_std() + decorrelation: add cross_correlation_std() based on Balmer and Eineder (2005) and De Zan (2014) for the coherent and incoherent speckle tracking. + network: add simulate_coherence_v2() for more pythonic usage 21 November 2021, 07:40:49 UTC
adac2b1 bugfix for info.py --slice (#701) + info.py: fix bug introduced in #674. Thank you @falkamelung for reporting it. + tropo_pyaps3.snwe2str(): simplify syntax + README/badge: use native color for circle CI test result 19 November 2021, 19:23:54 UTC
59819bb coord.lalo2yx(): replace np.float128 with np.longdouble for Windows (#699) * Replace np.float128 object with np.longdouble for a platform-neutral syntax 18 November 2021, 07:21:39 UTC
d2727e0 dask: support numWorker input in percentage with improved stability + cluster.DaskCluster: - run(): set the num of scaled workers to the num of boxes, to avoid unnecessary resource initiation - format_num_worker(): revert "divide num core by num of threads per core" (#691) as it's not right. - format_num_worker(): support num of workers in percentage, e.g. 80% to use 80% of all cores + ifgram_inversion: use ceil instead of rint to prevent the occurrence of extra small patches. + cluster.split_box2sub_boxes: - add a min step size constrain - trim the final number of boxes, to avoid negative dimension 13 November 2021, 23:24:38 UTC
e14a53e sensor/ERS: add pulse_length and chirp_slope + sensor: add pulse_length and chirp_slope for ERS + prep_hyp3: replace inc_map with lv_theta, to be consistent with readthedoc + save_kmz: more doc on radar-coded file support + .circleci/config.yml: add resource_class 13 November 2021, 23:24:38 UTC
928db34 prep_hyp3: ignore whitespace in keys (#690) ignore the whitespace in all keys while reading the HyP3 metadata file, to be immune to the potential whitespace adding/removing in the upstream HyP3 metadata, such as the UTC time key change from `UTCtime` to `UTC time`. 13 November 2021, 21:54:47 UTC
92e00ad parallel: divide num_core by num_thread if available (#691) + cluster.format_num_worker(): divide the number of cores by the number of threads if found [for Linux only] using lscpu command. Note again that this only affects the translation of numWorker = all. + smallbaselineApp.plot_result(): translate numWorker = all + update API hierarchy of cluster.py as it now depends on utils0.py 09 November 2021, 08:41:01 UTC
fe32d26 spatial_filter: refactor double difference (#687) * spatial_filter: refactor double difference - combine the local and regional filter into one filter to reduce number of convolutional operations - change regional filter shape from a disk to a doughnut * spatial_filter: update double_difference docs 04 November 2021, 01:29:22 UTC
8472709 Dockerfile: rewrite using Micromamba Image (#685) * Dockerfile: rewrite using Micromamba Image - Rewrites the MintPy dockerfile using the mambaorg/micromamba image - Forces Python=3.6 for faster image build - Sets the user as micromamba, not root - Optionally allows the image to start with a Jupyter Lab instance * Dockerfile: fix codacy issues - tag micromamba version - remove apt-get lists * Update docker/Dockerfile Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> * Dockerfile: reconfigure user * Dockerfile: reconfigure user 2 * Dockerfile: pull-request updates - hard code MINTPY_HOME and PYAPS_HOME - install wget - update installation.md and docker.md * Update docker.md * Update installation.md * Update Dockerfile * Update docker/Dockerfile Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> * Update docker/Dockerfile Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 04 November 2021, 01:27:49 UTC
6b2ce05 sensor: add seasat PRF and NISAR rg pixel size (#684) + sensor: - add seasat PRF - add NISAR range_pixel_size + ERA5: consistent naming with better error msg - use ERA5 thoughout mintpy without hyphen, to avoid confusion. - smallbaselineApp.py: raise error for un-recognized GAM names + timeseries_rms: simplify plot + utils/plot: omit "Time [year]" xaxis label to save space as it's not informative + prep_snap: fix typo 23 October 2021, 06:00:48 UTC
2ed0313 writefile.layout_hdf5(): bugfix when using binary file as the reference (#682) + utils.writefile.layout_hdf5(): support using binary file as structure reference to create empty HDF5 files, using the metadata and slice(s) info. + simplify the code with vprint() 18 October 2021, 02:51:43 UTC
11ae0e5 sensor: add NISAR_L and altitude (#677) + sensor: - add NISAR L-band SAR: only a few parameters from the handbook - add altitude value for all sensors - add range pixel size for ALOS-1 stripmap + utils.utils0.py: add is_number() + view: YYYY-MM-DD style subplot title for py2-mintpy TS file 10 October 2021, 00:16:29 UTC
65c856b view: bugfix for DEM masking (#673) * fix bug related to `--mask-dem` option when DEM shades and contours do not exist at the same time. 02 October 2021, 23:04:57 UTC
bdb4669 codacy style cleanup (#674) 27 September 2021, 20:04:20 UTC
eb30167 add generic writefile.write_binary() + writefile: add write_binary() for generic binary file writing, similar to the generic binary reading in readfile.read_binary(). + writefile.write(): use write_binary() for simpler and more powerful writing support 27 September 2021, 03:26:26 UTC
c4e73ee attr: add BANDS and INTERLEAVE + docs/api/attributes: add BANDS and INTERLEAVE to the standard mintpy self-generated metadata list + use "BANDS" instead of "number_bands" across mintpy + use "INTERLEAVE" instead of "scheme" across mintpy + readfile: grabbing BANDS and INTERLEAVE through standard metadata dict whenever they are available + writefile: - write(): update BANDS according to the input data list - write_isce_xml(): refactor to take mintpy attributes dict as input, for user friendly usage + add/update write_isce_xml() usage in geocode.py, subset.py and multilook.py + utils.plot.auto_figure_title: keep suffix for single-subplot from multi-dset file 27 September 2021, 03:26:26 UTC
9c9b2b6 plot: --mask-gps/dem, --gps-comp horz/vert and American display unit (#670) This PR adds the following new features for more flexible visualization: 1. add `--mask-dem` option to support masking out DEM pixels to be consistent with the masked data matrix 2. add `--mask-gps` option to mask out GNSS sites out of the displayed data matrix. 3. support `--gps-comp horz / vert` to plotting horizontal and vertical GPS components. The horizontal azimuth angle can be adjusted using `--hz-az` option. 4. support American length unit for display. Detailed changes: + asc_desc2horz_vert: set FILE_TYPE to displacement for ifgramStack, timeseries and HDFEOS file types, instead of the previous unique but meaningless name. + objects/gps.py: - get_gps_los_obs(): always write both displacement and velocity to CSV file for simplicity - use az_angle for LOS projection, instead of head_angle, to be consistent with the rest of mintpy convention. - support `horz_az_angle` to project GPS horizontal displacement/velocity to a custom azimuth angle + view.py: - save `self.msk` as the final flag matrix for masked out pixels - turn OFF auto mask file detection when --zero-mask is specified. + utils/plot.py: - prepare_dem_background(): support masking of DEM shade and contour for --mask-gps - plot_gps(): pruning GNSS list based on self.msk for --mask-gps - scale_data2disp_unit(): support American unit: inch, foot, yard and mile. Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 27 September 2021, 00:04:37 UTC
884434f read/write/plot: improved flexibility for h5 file (#671) + `read/plot`: support `geometry` h5 file with other undefined datasets, e.g. "displacement", which is useful for single-pair displacement products for generic sharing purposes. + `read.read_attribute()`: ignore Std while grabbing `UNIT` + `writefile.write()`: add optional `ds_unit_dict` arg to specify different data unit for each dataset, same as `layout_hdf5()` + `geocode`: same h5 compression bwt. input/output 24 September 2021, 05:19:37 UTC
6c7ad9e ifgram_inversion: bugfix for the ionoPhase inversion (#667) * ifgram_inversion: bugfix for ionoPhase inversion + string identification ('ion') for ionoPhase inversion + string identification ('ion') for converting iono time-series unit to meter 22 September 2021, 19:18:04 UTC
670c6a9 sbApp: add mintpy.plot.dpi option (#663) + smallbaselineApp: add mintpy.plot.dpi template option to customize the output figure resolution + sensor: add range pixel size for different ALOS-2 imaging modes + coord/network/ptime: support isinstance() for int/float mixed with numpy dtype 18 September 2021, 03:49:04 UTC
8e285b2 demo: add RidgecrestSenDT71 for hyp3 + add RidgecrestSenDT71 as example dataset for HyP3 - docs/demo*.md: add RidgecrestSenDT71 dataset produced by ASF HyP3 service by Jiang Zhu. - mintpy/data/input_files: add RidgecrestSenDT71.txt + add RidgecrestSenDT71 as test dataset for hyp3 - tests/test_smallbaselineApp: add url for RidgecrestSenDT71 prepared using ASF HyP3 by Jiang Zhu. - tests/configs/RidgecrestSenDT71.cfg: update setups for faster testing + readfile.read_gdal(): use numpy indexing instead of gdal python binding for more robust performance + plot.auto_figure_title(): use custom ds name for generic h5 file 14 September 2021, 19:02:22 UTC
213115b time_func: supp. time of the day in THH(:)MM style This commit adds the capability to feed accurate hour/minute/second info of step/exp/log functions to the design matrix, to support scenarios when the event and SAR acquisition are in the same day, such as The M6.4 Ridgecrest EQ in 4 July 2019. + ptime.yyyymmdd2years(): add seconds arg to add the time of the day info to the output list of years in float + time_func: add seconds arg to pass to ptime.yyyymmdd2years() to support time of the day info to all functions inside + read/prep seconds info to feed to time_func.py in the following scripts: - dem_error.py - timeseries2velocity.py - tsview.py + update example usages in smallbaselineApp.cfg file and arg_group.py 14 September 2021, 19:02:22 UTC
e30dd1d utils: add time_func.py and s1_utils.py + add utils.time_func.py: - move get_design_matrix4time_func() from objects.stack.py. - switch module import/usage in all related scripts: dem_error, gps, simulation, tsview - replace the old func definition with a pointer to the utils.time_func.py for backward compatibility. - move estimate_time_func() from timeseries2velocity.py - switch module import/usage in all related scripts: timeseries2velocity, tsview - set all get_design_matrix4*_func() as independent functions for more flexibility + utils.s1_utils: - add estimate_S1AB_bias() - add get_subswath_masks() + objects/gps.py: check site name during initialization for more informative error msg against typo in "view.py --ref-gps" + docs/api/module_hierarchy.md: add time_func and s1_utils. 14 September 2021, 19:02:22 UTC
6070ab0 ifg_inv: --calc-std for phase/offset and WLS for offset + add weighted LS estimation for offset via offset STD from ampcor [turned off by default, as initial testing does not show improvement over OLS] + add estimate_timeseries_std() for TS STD calculation via linear propagation from the stack of interferograrms or offset pairs. - this corresponds to the uncertainty of decorrelation noise for phase or of cross-correlation estimation for offset - TS STD now can be estimated independent from TS est. regardless of WLS or OLS, via stack_std variable - add get_design_matrix4std() to calc STD with a user chosen reference date, instead of the default 1st date, since TS STD can not be easily re-referenced in time + add skip_invalid_obs() to skip invalid data and update matrice for both phase and offset + adapt calc_inv_quality() for both phase and offset, in order to simplify estimate_timeseries() + use A X = y naming conventions for TS estimation part of the code. + rename some vars for a neutral meaning, e.g.: - pha/off --> obs - ifgram --> stack - num_ifgram --> num_pair + remove obsolete write2hdf5_file() + add print_msg to estimate_timeseries() and calc_inv_quality() 12 September 2021, 04:53:10 UTC
1eb2e57 ifg_inv: mask offset based on STD + sbApp.cfg: set range/azimuthOffsetStd as recommend mask dataset for offset + ifg_inv: - mask out pixel with offset std. dev. LARGER than threshohld, whereas for phase, mask out pixel with coherence SMALLER than threshhold - for offset, skip reference is no REF_Y/X found, otherwise apply referencing [for flexibility] - set inv quality value to 0 for offset - do not convert obs value from nan to zero since the previous skip_zero for phase has been changed to the more generic skip nan for phase and offset. + reference_point: support ifgramStack of offset + ts2vel: improved default output filename for offset TS 12 September 2021, 04:53:10 UTC
cff6e9e load_data: support offset std. dev. + support loading offset std. dev. from ampcor offset_cov.bip file via mintpy.load.rg/azOffStdFile option. + Hardwire no-data value of 99 for offset variance and convert it to NaN during loading process + convert offset variance to std during loading process 12 September 2021, 04:53:10 UTC
7166ef6 load_data: support loading rg off w/o az off supporting loading range offset along without azimuth offset, by adding range offset checking as required datasets during data loading process. 12 September 2021, 04:53:10 UTC
65fe5eb setup: add all executable modules as `entry_points/console_scripts` (#659) + setup.py: - Add all currently executable modules as `entry_points/console_scripts` to replace `scripts` - Drop depreciated `depedency_links` for pysolid as it's now available from conda-forge + Add `main(argv=None)` in add_attribute.py and multi_transect.py to be used in the new setup above + Graceful --help or missing args exit for save_gbis.py and tsview.py. 11 September 2021, 23:41:31 UTC
db15001 speedup testing via *.plot.maxMemory and smaller Fernandina (#658) + smallbaselineApp.plot_results(): - add mintpy.plot.maxMemory option to further reduce memory usage while calling view.py inside smallbaselineApp.py - remove mask_file option as it's auto found by view.py and single mask_file does not apply to both radar and geocoded TS files - add missing --wrap option, otherwise --wrap-range won't work. + tests/configs/*.txt: set mintpy.plot.maxMemory = 2 to increase number of parallel processes by 2X and to speedup regular testing by 25% (2X on the reading part) + tests/test_smallbaselineApp.py: update to the new FernandinaSenDT128.tar.xz file DOI for smaller file size and faster testing (2X speedup from 6 mins to 3 mins) + utils.read_timeseries_yx/lalo(): add zero_first arg so that one could turn it off to read the absolute displacement from ERA5.h5 file. The default value is set to True for backward compatibility. 09 September 2021, 21:11:45 UTC
7757a2e view: add WARNING msg for un-support option scenarios (#657) + view: print warning msg for geo-only options if input file is in radar-coord + view: print warning msg for single-sub plot options if it has multi-subplots + circleci/config.yml: update path for the test_smallbaselineApp.py + objects/coord: more robust debug_mode plotting for _get_lookup_row_col() 08 September 2021, 04:32:57 UTC
0824c57 rename test to tests 07 September 2021, 22:12:11 UTC
185247e install: pykml min version constraint + add pykml >=0.2 to all requirement files + remove pip install pykml from circleci, as it's now included in the mamba install call 07 September 2021, 22:12:11 UTC
90abc35 switch pykml install from pip+git to conda (#655) + install: Switch pykml installation to conda via conda-forge channel as it is now updated to the python2/3 version on both conda-forge channel and PyPI, instead of using pip git+https to the GitHub repo. + tropo_pyaps(3).py: remove usage of subprocess + info/versoin: typo fix 06 September 2021, 19:28:55 UTC
8bda09b ts2vel: save residual displacement TS via --save-res + add --save-res option to turn ON saving residual displacement time-series [OFF by default] + add --res-file option to specify the residual file name [timeseriesResidual.h5 by default, same as dem_error.py] 05 September 2021, 18:41:10 UTC
9b1ede5 plot: handle very long TS file with YYYY-MM-DDTHH:MM format Add the following changes to support the reading and plotting of TS files: + with >8000 dates and + with dates in isoformat, e.g. YYYY-MM-DD[*HH[:MM[:SS[.fff[fff]]]]. Detailed changes includes: + readfile.read_hdf5_file(): support YYYY-MM-DDTHH:MM format in the date dataset with more relaxed but accurate detection of date and "-" + info.py: do not list dates and dates in years, because the former is covered in --date option and the later is not used, to be more compact on TS output + utils.plot.scale_data2disp_unit(): set disp_unit to data unit if it's un-scalable data + view/draw_scalebar(): do not plot scalebar if the longitude span > 30 deg + ptime.py: - get_date_str_format(): support the isoformat date and datetimes - get_date_range(): extend dmin/dmax to any format supported by get_date_str_format() - naming conflict fix for `dt` 05 September 2021, 18:41:10 UTC
cf83ccc init test config file for hyp3 05 September 2021, 18:41:10 UTC
b5e6e5b __init__: rm `sys.path.insert()` and force internal imports (#653) + `mintpy/__inti__.py`: - remove try/except for print and MINTPY_HOME as it's not really used anywhere, and their corresponding module imports. - remove `sys.path.insert()` calls. These were replicates from the isce-2 style (isceobj, iscesys, etc.), but they are not really applicable to and used in mintpy, nor had we encouraged in any other files besides this one. **NOTE**: however, this might still potentially impact downstream workflows, thus, should be noted in the release notes/changelog. + test_smallbaselineApp.py: - use the relative path to locate config file, instead of using mintpy module - remove the occurrence of MINTPY_HOME - remove print view.py cmd as it's now included inside view.py if --noverbose is turned ON - use `pathlib.Path().resolve()` instead of `os.path.dirname()` 01 September 2021, 06:39:50 UTC
c7c12b0 ptime.get_date_range(): add dunit and out_fmt (#651) + ptime.get_date_range(): add dunit and out_fmt - add `dunit` arg with default value as before - add `out_fmt` arg with default value as before + smallbaselineApp.plot_result(): use floor() instead of rint() as a more conservative num calc for parallel processes + unwrap_error_phase_closure: print out # of ifg/triplets 29 August 2021, 06:56:00 UTC
ab01556 Add doc on install MintPy on Windows (#647) + add documentation on installing in windows + fix a few bugs that prevent running through the example + add an independent link to installation on read the docs. Co-authored-by: Joseph H Kennedy <me@jhkennedy.org> 29 August 2021, 05:21:27 UTC
f7b2b4f Triplet design matrix speedup by 10X (#650) + simplify the triplet matrix building by using `itertools.combinations(date_list, 3)`, instead of the previous `for` loop search and check + create inverse mapping objects to speed up the indexing. Testing shows more than 10X speedup for large matrix. 28 August 2021, 23:52:57 UTC
08ae0e0 sbApp --plot: limit num_cores based on max memory (#646) + smallbaselineApp.py: limit the number of parallel processes for plotting based on the max memory from the template file, to avoid exceeding the memory. Circle CI kills processes because of this. Note that matplotlib reserved much larger memory than it actually uses. For users who want to use a lot of parallel processes for plotting, please increase your `mintpy.compute.maxMemory` setting. + view.py: add --memory option. Note this is NOT used in the smallbaselineApp.py + plot.auto_multilook_num(): - more detailed multilook num list, for a more detailed memory manegement. - add max_memory arg to adjust according. Details are in the code comments. + add quotes to the version control for conda/mamba install in command line in docs/installation.md and .circleci/config.yml files + circleci: use pip from conda to avoid warning msg + docs/api/attributes.md: add EPSG 25 August 2021, 01:48:10 UTC
0c5d79d save_gdal.py: support UTM projection (#645) + save_gdal.py: Fix bug when dealing with UTM projections, such as provided by hyp3, by adding additional arguments for EPSG code. + readfile.read_gdal_vrt(): add "EPSG" to file attributes + utils0.py: add new function `utm_zone2epsg_code()` 23 August 2021, 20:30:56 UTC
ab1276e docs: fix typo in installation.md (#643) 19 August 2021, 19:38:41 UTC
949437d simplify generic I/O logic for cpx binary files (#642) This PR modifies readfile/writefile to read and write complex binary files (from isce-2) directly in complex data type by default. For plotting in view.py, which requires real data type, I add readfile.get_slice_list(no_complex) option to translate complex into mag/phase for plotting. This option is turn off by default for generic I/O. + readfile: - read_binary_file(): for "slc", set cpx_band to "complex" unless mag/phase is clearly specified. - get_slice_list(): 1) ignore meaningless geo/rdr file extension; 2) add "no_complex" arg to switch between complex (for I/O) and mag/phase (for plotting) + writefile: - write(): for .slc/.int, write complex directly - write_isce_xml(): add "image_type" arg to initiate different ISCE-2 image objects for better compatiability - write_isce_file(): support file_type == 'isce_slc' - comment out obsolete write_complex64() as write_complex_float32() is equivalent but simpler and faster. + subset/multilook: update usage of writefile.write_isce_xml() + view: update usage of readfile.get_slice_list() Other minor changes: + subset.py: add --sub-x/y/lat/lon and --subset-x/y/lat/lon to be consistent with subset option names in other scripts. + view.py: add --nmli / --num-multilook for --multilook-num + simulation.simulation.py: typo fix and add missing wvl arg while calling timeseries2ifgram() + objects.sensor: typo fix for doppler_bandwidth 19 August 2021, 01:28:14 UTC
2ff5a78 bugfix in tropo_pyaps3.check_pyaps_account_config() (#641) * readfile.read(): add data_type to customized output data type * tropo_pyaps3.py.check_pyaps_account_config(): typo fix 16 August 2021, 18:30:32 UTC
9cbc317 hyp3: fix incAngle/slantRangeDist + add more attributes (pass full test) (#640) + prep_hyp3: - ensure negative HEADING value - grab CENTER_LINE_UTC from UTCtime directly - put P_BASELINE_* to the ifgram specific metadata - add STARTING_RANGE from "Slant range near" - add RANGE/AZIMUTH_PIXEL_SIZE from hardwired Sentinel-1 info - adjust to the new incidence angle file name 'lv_theta' + stackDict: - convert Gamma inc angle to isce/mintpy inc angle convention while loading the data for hyp3 - consider the above conversion while calc slantRangeDistance from incidence angle for hyp3 + docs/dir_struct*/hyp3: add inc angle and water mask path - remove warning for inc angle as the latest HyP3 is producing the correct one now - add geometry file path in the template for incidence angle and water mask - use Ridgecrest SenT71 dataset for the example in the directory structure 14 August 2021, 23:28:47 UTC
a6d5d7c tropo_pyaps3: check GAM account info in pyaps3/model.cfg file (#639) This PR addresses the error of missing API config for PyAPS. If the user did not input their API credentials in `model.cfg` in the pyaps3 directory it raised a vague ZeroDivisionError: float division by zero error. This revision adds `check_pyaps_account_config()`, which will now raise a ValueError with a more appropriate error message telling users that the API config is missing. Note that: + the function includes checking for ERA5/ERAI/NARR/MERRA for completeness, although tropo_pyaps3.py only accepts ERA5 for now. + the function checks both default(demo/invalid) options values as well as empty values, but it can not really check the validity of the user account. 14 August 2021, 22:27:47 UTC
83a2fb1 asc_desc2horz_vert: bugfix on custom azimuth and refactor (#636) + Properly pass the azimuth variable: The azimuth variable is supplied as part of the command line input, however, it's not actually passed to the `get_design_matrix` function where it is needed, meaning the command line variable does nothing and the azimuth angle defaults to 90 degrees. I've added code to pass it through to the design matrix function, so now the azimuth angle can be modified via the command line. + use `los_az_angle` for the direction of LOS vector throughout the script, and to replace `head_angle` to be consistent with the rest of mintpy + ALERT: use `azimuth` in anti-clockwise for consistency. The version before is using a clock-wise direction. Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 10 August 2021, 20:32:57 UTC
79810f6 Avoid overwriting modified attributes (#635) Currently, the file attributes are modified in the `asc_desc_files2horz_vert` function to take account of the fact that we're looking at the overlap between two tracks. However, the `write_to_one_file` as currently written overwrites the input `atr` variable when looping over the list of attributes for the input files. The least to the attributes of the newly created file being incorrect. 10 August 2021, 20:28:44 UTC
988a097 asc_desc2horz_vert --oo: grab sensor name from PROJECT_NAME (#634) instead of from FILE_PATH, as it's a more reliable and flexible source. 10 August 2021, 04:07:55 UTC
25f79aa add.py: handle TS in the same way as diff.py (#633) + add.py: - check num of files == 2 for TS file only - support two TS file with different reference point, different reference date, different date list; to be consistent with diff.py. + diff.py: `_check_reference()` -> `check_reference()` with more comments + view: expand --math to multi-subplots + save_gdal: use float32 as default to save disk + sbApp: plot geo_avgSpatialCoh.h5 file + utils.readfile: more relaxed wildcard filename for offset file from ampcor + utils.ptime.get_date_range(): add 'dstep' option for coarser list to speedup matrix operation laterward 09 August 2021, 21:41:33 UTC
590ecc0 read_attr(): UNIT bugfix for dsName with date info (#630) + utils.readfile.read_attribute(): fix a bug of grabbing the UNIT for HDF5 dataset, if `datasetName` is specified with date info as suffix. This only affects the unit display in the colorbar for plotting. + utils.utils1.get_residual_std/rms(): use np.float32 to replace np.float to avoid numpy deprecation warning + view: print WARNING msg regardless of --verbose setting + tsview: more friendly figsize_img API for the interactive tsview.ipynb in MintPy-tutorial repo + plot_network: fix --nodisplay bug while calling from jupyter notebook + install: use bash *.sh to replace chmod +x cmd in docs/installation.md and .circleci/config.yml files 05 August 2021, 18:05:18 UTC
dd53df9 timeseries2velocity: replace sklearn with numpy for bootstrap resampling (#629) use np.random.Generator.choice() as recommended by numpy (https://numpy.org/doc/stable/reference/random/index.html#random-quick-start) 05 August 2021, 05:32:44 UTC
0e8d7cb Enable processing of cosicorr pixel offset data (#626) + add `prep_cosicorr.py` to prepare metadata of COSI-Corr offset products for MintPy, with example usages + `load_data.py`: add `cosicorr` as a processor and call prep_cosicorr.py + `utils.readfile.py`: read GeoTIFF file produced by `cosicorr` via GDAL + `ifgram_inversion.py`: disable unit conversion from pixel to meter for cosicorr, since it is in meters natively 04 August 2021, 03:09:17 UTC
dd9512b ts2vel: skip writing for empty blocks (#624) Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 02 August 2021, 19:36:58 UTC
ab5793e wrap up for version 1.3.1 02 August 2021, 18:11:14 UTC
528b68f multilook/subset: write VRT/XML file for isce products + multilook/subset: write GDAL VRT & ISCE XML file if the input file is generated by ISCE-2 directly + utils.readfile: add NUMPY2GDAL_DATATYPE global var + [WIP] utils.writefile: add write_gdal_vrt() given the mintpy metadata dict. [work in progress]. !DO NOT USE IT! + view.prep_slice(): bugfix for the missing argv, which affects plot_coherence_matrix.py + prep_aria: update ariaTSsetup.py example usage in the help msg 02 August 2021, 18:11:14 UTC
a47a710 geocode: support isce lat/lon.rdr file + resample: - auto split for non-h5 file - constrain lat/lon dataset in float32, to save memory usage + writefile: - write(): keep magnitude info for .unw/.int file if fed - add write_complex_float32() to properly write complex float32 matrix + geocode: add --lat/lon-file option to grab isce lat/lon.rdr file path 02 August 2021, 18:11:14 UTC
9957b8f docs/dask: typo fix (#622) 30 July 2021, 21:54:58 UTC
2462b73 num_threads: better handle non-exist env vars (#621) For non-exist env variables, set to "1" and remove them afterward. This makes sure that even if one does not set any NUM_THREADS related env variables, the NUM_THREADS limit would still work. 30 July 2021, 00:56:27 UTC
6161d95 limit NUM_THREADS=1 to save CPU usage Testing shows that numpy/scipy with OMP_NUM_THREADS > 1 does not help much on the computing time but uses significantly more CPU. The true fast way is "Dask with multiple workers + OMP_NUM_THREADS = 1". Thus, I am setting all the relevant NUM_THREADS env variables to 1, in ifgram_inversion.py and dem_error.py, before running the big matrix inversion and roll back to the original values afterwards. + objects/cluster: - add set_num_threads() to save original num of threads info and set all relevant env var to the given value - add roll_back_num_threads() to roll back to the original setting + ifgram_inversion/dem_error: call the above two functions in the begining and the end of the process, to ignore the *_NUM_THREADS setting during the numpy/scipy matrix computation. 29 July 2021, 20:31:56 UTC
bf0c176 read_gdal*(): support file w/o SRS info + utils.readfile.read_gdal_vrt(): support reading metadata from file without SRS info by setting defalt X/Y_UNIT in degrees + defaults.smallbaselineApp.cfg: add comments on SNAPHU as the only unwrapper that provides connected components info. + dask: use explicit cluster name instead of non-local to avoid confusion in the smallbaselineApp.cfg and dask.md files. + view: printout multilook-num even if not calculated from auto_*(); plot water/shadowMask in gray by default 29 July 2021, 20:31:56 UTC
d77267e generate_mask: mask velocity based on velocityStd (#618) * generate_mask: mask velocity based on velocityStd Generate mask based on formula |velocity| > cutoff * velocitStd - adds --vstd and --vstd-num options - adds mask generation functionality in create_threshold_mask() 27 July 2021, 03:33:02 UTC
972b927 tropo_pyaps3: add default value of degrees for Y_UNIT attribute (#619) + prep_snap: hardwire attributes X/Y_UNIT = 'degrees' + tropo_pyaps3: add default value of degrees for Y_UNIT attribute + save_kmz: fix typo in the examplage usagge + docs/api/colormaps: update to the latest link of matplotlib colormaps 26 July 2021, 19:01:14 UTC
11bed8d read_attr: check existing FILE_TYPE for HDF5 file (#615) utils.readfile.read_attribute(): use existing 'FILT_TYPE' from the HDF5 file if it's not pre-defined known datasets, to fix the issue: When I run info.py, the "FILE_TYPE" is guessed based on dataset name and if a stack with different naming is called, some options like --date does not work. 25 July 2021, 04:59:24 UTC
9e828af view: bugfix in search_dataset_input() for save_roipac (#612) + view: bugfix in search_dataset_input() by making a local copy of the input inNumList arg, to avoid weird initiation behavior during multiple calls from save_roipac.py + utils.read_timeseries_lalo/yx(): - add method arg to switch between mean and median - return the dispersity time-series if win_size > 1, STD for mean, MAD for median. + test/config: turn on dask parallel for SanFran & WCape datasets to speed up testing + save_gbis: update reference of Yunjun et al. (2021) 21 July 2021, 00:43:12 UTC
4217a04 generate_mask: mask pixel clusters by size (#610) * generate_masks: add -p / --mp / --minpixels option to mask out clusters with size below the input threshold `minpixels`. The `mask` is used as input/output directly in remove_small_objects() instead of the `data` variable because the func takes only int/bool type and data.astype(bool) will convert nan value to true, interfering the connectivity calculation. 19 July 2021, 04:22:42 UTC
eef31b4 simple parallel plotting support in sbApp.py via joblib (#609) + smallbaselineApp: implemented simple parallel plotting support using joblib, based on `mintpy.compute.*` options. + view: print cmd if --noverbose + docs: add joblib to the requirement 16 July 2021, 05:47:04 UTC
eeb2290 modify_net: fix bug in aligning ifgs with coh for MST network (#608) * fix a bug in modify_network.py introduced in #563 on May 22, 2021 for aligning ifgs with coherence while calculating the MST network. Note that this bug should affect the network modification result by not getting the exact network as intended from the configuration of mintpy.network.coherenceBased = yes (no by default) AND mintpy.network.keepMinSpanTree = yes (yes by default). However, this does not affect the correctness of the processed result. 15 July 2021, 03:17:58 UTC
e658521 ts2vel: merge two model2hdf5_*() into one + timeseries2velocity: merge model2hdf5_structure() into model2hdf5_dataset() to reduce the duplicated code + tsview: do not use ref_date info for the time func design matrix, because it affects the value of the poly >= 2 although the fitting looks the same, this is because poly>=2 is similar to exp/log which depends on the onset_time. We might need a better solution to clear this up, so let's keep it the old way for now for consistency. 10 July 2021, 22:11:29 UTC
f390551 docs: standardize requirement files + rename the following files: - readthedocs.yml -> .readthedocs.yaml - docs/requirements.txt -> docs/requirements4rtd.txt - docs/conda.txt -> docs/requirements.txt - docs/conda_env.yml -> docs/environment.yml + update installation notes in mintpy installation page, circle CI and docker file. 09 July 2021, 20:00:06 UTC
ec6cd18 docs/FAQs: simplication with typo fix 09 July 2021, 20:00:06 UTC
d58acf5 bugfix for plotting in image_stitch.py (#600) 09 July 2021, 02:28:25 UTC
e306264 stackDict: boolean conversion of waterBody to waterMask (#599) * stackDict: boolean conversion of waterBody to waterMask the `data` read from a waterBody file is a boolean array with water=True; land=False we only need to flip it to make a waterMask where water=False; land=True 08 July 2021, 21:17:48 UTC
5807229 view: add --show-kept(-ifgram) option + view: add --show-kept / --show-kept-ifgram option to be able to not plot the dropped interferograms + view: more robust dataset index info showing in the title 08 July 2021, 20:09:37 UTC
4392e9d readfile: UNIT as "degrees" for geocoded isce xml When reading files in geo-coordinates, the unit of the coordinates should be read from the XML file directly. For XML files from ISCE without this information, we set the default UNIT as "degrees". 07 July 2021, 23:44:55 UTC
a2b102f save_kmz: point style for radar-coded file This commit adds the capability of saving KMZ point/placemark elements for files in radar-coordinates, thus, it's more precise and PS-like, yeh! + add -g/--geom option to feed geometry file with latitude and longitude data. call ut.get_geometry_file() to auto grab if not input in cmd. + add --step option to reduce the output file size, otherwise Google Earth might freeze + add -sub-x/y/lat/lon options and save row/col number in the original full coverage, so that multi subsets are comparable + save_kmz(_timeseries): add --keep-kml-file option so that one could easily modify the KML file manually. + rename write_kmz_file() to write_kmz_overlay() + add write_kmz_placemark() - use existing style of shaded dot from save_kmz_timeseries.py - leverage ExtendedData feature to show lat/lon/x/y/value in the form style in Google Earthh + kmz: add write_kmz_file() to reduce duplicated code + add create_placemark_element() + move get_hex_color() from save_kmz_timeseries.py to save_kmz.py 07 July 2021, 01:12:15 UTC
204e79a view: add --mask-vmin/vmax option + view.py: add --mask-vmin/vmax to adjust threshold for masking on the fly, useful to check different thresholds using temporal coherence. + reference_point: bugfix when mask file not exists + arg_group: add more aliases for --subset-x/y/lat/lon 07 July 2021, 01:12:15 UTC
bebc2ae prep_fringe: fix sign issue & add --sub-x/y --ps-mask options (#594) + prep_fringe: - add --sub-x/y option to load AOI from fringe to mintpy - add --ps-mask option to load PS mask from fringe to mintpy - fix sign issue, as confirmed by Yujie. + subset: change the default prefix of the output file name from 'subset_' to 'sub_' + utils.ptime: add utc2solar_time() to convert UTC time to the local solar time. + docs/conda.txt: comment out `pyhdf` because it's only used for MERRA, which is currently not supported, thus not needed. 05 July 2021, 00:35:34 UTC
6745249 [preliminary] tsview: plot model conf. interval + ts2vel.estimate_time_func: add ref_date arg to pass to the design matrix construction + tsview: supp. plotting of the confidence interval of the model prediction, following the note from Hanssen et al. (2017) EdX online course [preliminary implementation and NOT throughly check!] - rename show_model to plot_model, as model est. is always printout in the msg and this opt only turns off the plotting of the pred. - add --plot-model-conf-int to plot the confidence interval of the model prediction - adjust fit_time_func() for the conf. interval formula [preliminary] 04 July 2021, 23:48:02 UTC
5e017a3 memory efficient tsview.py for big-data + tsview: memory efficient version via auto multilooking - call pp.auto_multilook_num() for the auto value - multilook ts data and mask as needed - add subset_and_multilook_yx() to convert YX coord for local 3D cube operations, e.g. indexing and referencing - fit_time_func(): support point with nan value / masked out area + utils.readfile.read_hdf5_file(): add print_msg arg to call sys.stdout.write/flush() to print progress, useful for large files + utils.plot: - add auto_multilook_num() to calculate the default / auto multilook number - read_mask(): add x/ystep arg, similar to readfile.read() + view: use plot.auto_multilook_num() to replace the previous check_multilook_input() 04 July 2021, 23:48:02 UTC
22bd98a Effective Coherence Ratio Interferogram Selection (#580) + modify_network: add area-based (ECR) interferogram selection from Kang et al. (2021). - This method excludes/includes interferograms based on the proportional coverage of cells that meet a minimum coherence threshold (defined automatically as the average coherence within the `mintpy.network.maskFile`) - allow for parsing of network.areaBased and network.minArea options from template + utils.utils1: - within spatial_average function, allow inversion of mask to allow for calculation of spatial averages within masked regions - when threshold is specified, values above threshold are set to 1 and values below are set to 0. Thus when the spatial average is calculated, it will represent the proportional area greater than threshold - updates stack function calls - updates output filename and contents + objects.stack: - within ifgramStack and timeseries spatial_average methods, allow inversion of mask to allow for calculation of spatial averages within masked regions - within ifgramStack and timeseries spatial_average methods, when threshold is specified, values above threshold are set to 1 and values below are set to 0. Thus when the spatial average is calculated, it will represent the proportional area greater than threshold + defaults/smallbaselineApp(_auto).cfg: - add network.areaBased and network.minArea options Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 04 July 2021, 07:44:13 UTC
a6e69df tsview: reconstruct and plot temporal model (#589) + tsview: reconstruct and plot temporal model - reconstruct and plot the temporal model (time func predictions) by default - add `plot_ts_fit()` to plot the temp model and print out the time func param results as legend to take advantage of the loc=best option, to auto avoid overlap with data in the plot - update `plot_point_timeseries()` to handle both TS obs and predictions. - this new default TS fit plotting can be turned off using `--nomodel/nofit` option - support complex time func fitting: - handle input cmd via `arg_group.add_timefunc_argument()` - add `fit_time_func()` for OLS estimation, leveraging existing functions in `ts2vel.py` - add `get_model_param_str()` to prepare human-friendly summary of est. results., leveraging `ts2vel.py` for the dataset name/unit dict objects. - rename `save_ts_plot()` to `save_ts_data_and_plot()` and write full time func est results to the header of the txt file. - remove the obsolete `estimate_slope()` + timeseries2velocity.py: - add init_exp_log_dicts() to translate inps.exp/log into inps.expDict/logDict. - fix bug in onset time checking for log/exp in read_inps2model() - estimate_time_func(): 1) check empty e2 due to rank-deficient design matrix with useful printout info; 2) remove unused dis_ts_std arg - add model2hdf5_structure() to get the ds_name/unit_dict from model dict, in order to 1) use the generic writefile.layout_hdf5(), to replace the previous ts2vel-unique layout_hdf5(); 2) re-use in tsview.py - add model2hdf5_dataset() to organize the m/m_std into the HFD5 ready dataset dict, in order to 1) use the generic writefile.write_hdf5_block(), to replace the previous ts2vel-unique write_hdf5_block(); 2) re-use in tsview.py + utils.ptime.py: add get_date_range(): to produce a dense and continuous range of dates between dmin and dmax + utils.arg_group: add add_timefunc_argument() for easy re-use in ts2vel.py and tsview.py. + utils.writefile.layout_hdf5(): - add `ds_unit_dict` arg to support HDF5 dataset attributes - complete arg call for `metadata` of writefile.layout_hdf5() in ifgram_inversion.py, prep_aria.py and prep_fringe.py + view: bugfix in format_coord() for gamma products + docs.demo_dataset: add USGS link and fix typo for the 2008 Wells EQ Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 28 June 2021, 00:22:45 UTC
8cfce73 enu2los+docs: clarify az_angle definition + utils.utils0.enu2los(): support az_angle besides head_angle and use az_angle in the actual calculation because: 1. az_angle is simpler to explain together with incidence angle 2. az_angle can be pixel-wise, thus, more precise than the usually single value of head_angle + utils.utils0: add heading2azimuth_angle() + docs/api/data_structure: add angle definition and vel file structure - Add definitions for geometry datasets, including incidence / azimuth angle, and masks. - Add file structure for the velocity file from the latest timeseries2velocity.py + utils.isce_utils.extract*meta(): expose satelliteSpeed + solid_earth_tides: increase the step size to 3km for speedup 26 June 2021, 04:40:35 UTC
570f417 small changes: dem_error, ifgram_inversion, plot_network (#587) + add a cpt file: romanian.cpt; update the colormaps.md doc + dem_error.py: - allow for periodic temporal function fitting - add defo_model arg group + ifgram_inversion.py: allow for inverting ionosphere timeseries + plot_network.py: baseline plot with zero-mean perp baseline 23 June 2021, 18:57:04 UTC
2029b8c install: add notes for gdal and isce2 + docs/installation: add notes of gdal and isce2 for users with different insar processors + docs/conda_env.yml: add gdal since this install a new env from scratch, thus, won't force un-wanted constraint to existing env. 21 June 2021, 05:35:28 UTC
379c4f9 ts2vel: add --ts-std-file to propagate STD from TS to VEL 20 June 2021, 20:42:56 UTC
20e74a7 docs/dir_structure: add notebook link for hyp3 (#581) + docs/dir_struct: add notebook link for hyp3 Co-authored-by: Andrew Johnston <asjohnston@alaska.edu> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 20 June 2021, 01:42:54 UTC
5fe1cba Set theme jekyll-theme-cayman 19 June 2021, 23:23:30 UTC
2913101 Set theme jekyll-theme-minimal 19 June 2021, 23:21:46 UTC
b4b33b4 Set theme jekyll-theme-cayman 19 June 2021, 23:20:53 UTC
back to top