https://github.com/insarlab/MintPy

sort by:
Revision Author Date Message Commit Date
dd9512b ts2vel: skip writing for empty blocks (#624) Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 02 August 2021, 19:36:58 UTC
ab5793e wrap up for version 1.3.1 02 August 2021, 18:11:14 UTC
528b68f multilook/subset: write VRT/XML file for isce products + multilook/subset: write GDAL VRT & ISCE XML file if the input file is generated by ISCE-2 directly + utils.readfile: add NUMPY2GDAL_DATATYPE global var + [WIP] utils.writefile: add write_gdal_vrt() given the mintpy metadata dict. [work in progress]. !DO NOT USE IT! + view.prep_slice(): bugfix for the missing argv, which affects plot_coherence_matrix.py + prep_aria: update ariaTSsetup.py example usage in the help msg 02 August 2021, 18:11:14 UTC
a47a710 geocode: support isce lat/lon.rdr file + resample: - auto split for non-h5 file - constrain lat/lon dataset in float32, to save memory usage + writefile: - write(): keep magnitude info for .unw/.int file if fed - add write_complex_float32() to properly write complex float32 matrix + geocode: add --lat/lon-file option to grab isce lat/lon.rdr file path 02 August 2021, 18:11:14 UTC
9957b8f docs/dask: typo fix (#622) 30 July 2021, 21:54:58 UTC
2462b73 num_threads: better handle non-exist env vars (#621) For non-exist env variables, set to "1" and remove them afterward. This makes sure that even if one does not set any NUM_THREADS related env variables, the NUM_THREADS limit would still work. 30 July 2021, 00:56:27 UTC
6161d95 limit NUM_THREADS=1 to save CPU usage Testing shows that numpy/scipy with OMP_NUM_THREADS > 1 does not help much on the computing time but uses significantly more CPU. The true fast way is "Dask with multiple workers + OMP_NUM_THREADS = 1". Thus, I am setting all the relevant NUM_THREADS env variables to 1, in ifgram_inversion.py and dem_error.py, before running the big matrix inversion and roll back to the original values afterwards. + objects/cluster: - add set_num_threads() to save original num of threads info and set all relevant env var to the given value - add roll_back_num_threads() to roll back to the original setting + ifgram_inversion/dem_error: call the above two functions in the begining and the end of the process, to ignore the *_NUM_THREADS setting during the numpy/scipy matrix computation. 29 July 2021, 20:31:56 UTC
bf0c176 read_gdal*(): support file w/o SRS info + utils.readfile.read_gdal_vrt(): support reading metadata from file without SRS info by setting defalt X/Y_UNIT in degrees + defaults.smallbaselineApp.cfg: add comments on SNAPHU as the only unwrapper that provides connected components info. + dask: use explicit cluster name instead of non-local to avoid confusion in the smallbaselineApp.cfg and dask.md files. + view: printout multilook-num even if not calculated from auto_*(); plot water/shadowMask in gray by default 29 July 2021, 20:31:56 UTC
d77267e generate_mask: mask velocity based on velocityStd (#618) * generate_mask: mask velocity based on velocityStd Generate mask based on formula |velocity| > cutoff * velocitStd - adds --vstd and --vstd-num options - adds mask generation functionality in create_threshold_mask() 27 July 2021, 03:33:02 UTC
972b927 tropo_pyaps3: add default value of degrees for Y_UNIT attribute (#619) + prep_snap: hardwire attributes X/Y_UNIT = 'degrees' + tropo_pyaps3: add default value of degrees for Y_UNIT attribute + save_kmz: fix typo in the examplage usagge + docs/api/colormaps: update to the latest link of matplotlib colormaps 26 July 2021, 19:01:14 UTC
11bed8d read_attr: check existing FILE_TYPE for HDF5 file (#615) utils.readfile.read_attribute(): use existing 'FILT_TYPE' from the HDF5 file if it's not pre-defined known datasets, to fix the issue: When I run info.py, the "FILE_TYPE" is guessed based on dataset name and if a stack with different naming is called, some options like --date does not work. 25 July 2021, 04:59:24 UTC
9e828af view: bugfix in search_dataset_input() for save_roipac (#612) + view: bugfix in search_dataset_input() by making a local copy of the input inNumList arg, to avoid weird initiation behavior during multiple calls from save_roipac.py + utils.read_timeseries_lalo/yx(): - add method arg to switch between mean and median - return the dispersity time-series if win_size > 1, STD for mean, MAD for median. + test/config: turn on dask parallel for SanFran & WCape datasets to speed up testing + save_gbis: update reference of Yunjun et al. (2021) 21 July 2021, 00:43:12 UTC
4217a04 generate_mask: mask pixel clusters by size (#610) * generate_masks: add -p / --mp / --minpixels option to mask out clusters with size below the input threshold `minpixels`. The `mask` is used as input/output directly in remove_small_objects() instead of the `data` variable because the func takes only int/bool type and data.astype(bool) will convert nan value to true, interfering the connectivity calculation. 19 July 2021, 04:22:42 UTC
eef31b4 simple parallel plotting support in sbApp.py via joblib (#609) + smallbaselineApp: implemented simple parallel plotting support using joblib, based on `mintpy.compute.*` options. + view: print cmd if --noverbose + docs: add joblib to the requirement 16 July 2021, 05:47:04 UTC
eeb2290 modify_net: fix bug in aligning ifgs with coh for MST network (#608) * fix a bug in modify_network.py introduced in #563 on May 22, 2021 for aligning ifgs with coherence while calculating the MST network. Note that this bug should affect the network modification result by not getting the exact network as intended from the configuration of mintpy.network.coherenceBased = yes (no by default) AND mintpy.network.keepMinSpanTree = yes (yes by default). However, this does not affect the correctness of the processed result. 15 July 2021, 03:17:58 UTC
e658521 ts2vel: merge two model2hdf5_*() into one + timeseries2velocity: merge model2hdf5_structure() into model2hdf5_dataset() to reduce the duplicated code + tsview: do not use ref_date info for the time func design matrix, because it affects the value of the poly >= 2 although the fitting looks the same, this is because poly>=2 is similar to exp/log which depends on the onset_time. We might need a better solution to clear this up, so let's keep it the old way for now for consistency. 10 July 2021, 22:11:29 UTC
f390551 docs: standardize requirement files + rename the following files: - readthedocs.yml -> .readthedocs.yaml - docs/requirements.txt -> docs/requirements4rtd.txt - docs/conda.txt -> docs/requirements.txt - docs/conda_env.yml -> docs/environment.yml + update installation notes in mintpy installation page, circle CI and docker file. 09 July 2021, 20:00:06 UTC
ec6cd18 docs/FAQs: simplication with typo fix 09 July 2021, 20:00:06 UTC
d58acf5 bugfix for plotting in image_stitch.py (#600) 09 July 2021, 02:28:25 UTC
e306264 stackDict: boolean conversion of waterBody to waterMask (#599) * stackDict: boolean conversion of waterBody to waterMask the `data` read from a waterBody file is a boolean array with water=True; land=False we only need to flip it to make a waterMask where water=False; land=True 08 July 2021, 21:17:48 UTC
5807229 view: add --show-kept(-ifgram) option + view: add --show-kept / --show-kept-ifgram option to be able to not plot the dropped interferograms + view: more robust dataset index info showing in the title 08 July 2021, 20:09:37 UTC
4392e9d readfile: UNIT as "degrees" for geocoded isce xml When reading files in geo-coordinates, the unit of the coordinates should be read from the XML file directly. For XML files from ISCE without this information, we set the default UNIT as "degrees". 07 July 2021, 23:44:55 UTC
a2b102f save_kmz: point style for radar-coded file This commit adds the capability of saving KMZ point/placemark elements for files in radar-coordinates, thus, it's more precise and PS-like, yeh! + add -g/--geom option to feed geometry file with latitude and longitude data. call ut.get_geometry_file() to auto grab if not input in cmd. + add --step option to reduce the output file size, otherwise Google Earth might freeze + add -sub-x/y/lat/lon options and save row/col number in the original full coverage, so that multi subsets are comparable + save_kmz(_timeseries): add --keep-kml-file option so that one could easily modify the KML file manually. + rename write_kmz_file() to write_kmz_overlay() + add write_kmz_placemark() - use existing style of shaded dot from save_kmz_timeseries.py - leverage ExtendedData feature to show lat/lon/x/y/value in the form style in Google Earthh + kmz: add write_kmz_file() to reduce duplicated code + add create_placemark_element() + move get_hex_color() from save_kmz_timeseries.py to save_kmz.py 07 July 2021, 01:12:15 UTC
204e79a view: add --mask-vmin/vmax option + view.py: add --mask-vmin/vmax to adjust threshold for masking on the fly, useful to check different thresholds using temporal coherence. + reference_point: bugfix when mask file not exists + arg_group: add more aliases for --subset-x/y/lat/lon 07 July 2021, 01:12:15 UTC
bebc2ae prep_fringe: fix sign issue & add --sub-x/y --ps-mask options (#594) + prep_fringe: - add --sub-x/y option to load AOI from fringe to mintpy - add --ps-mask option to load PS mask from fringe to mintpy - fix sign issue, as confirmed by Yujie. + subset: change the default prefix of the output file name from 'subset_' to 'sub_' + utils.ptime: add utc2solar_time() to convert UTC time to the local solar time. + docs/conda.txt: comment out `pyhdf` because it's only used for MERRA, which is currently not supported, thus not needed. 05 July 2021, 00:35:34 UTC
6745249 [preliminary] tsview: plot model conf. interval + ts2vel.estimate_time_func: add ref_date arg to pass to the design matrix construction + tsview: supp. plotting of the confidence interval of the model prediction, following the note from Hanssen et al. (2017) EdX online course [preliminary implementation and NOT throughly check!] - rename show_model to plot_model, as model est. is always printout in the msg and this opt only turns off the plotting of the pred. - add --plot-model-conf-int to plot the confidence interval of the model prediction - adjust fit_time_func() for the conf. interval formula [preliminary] 04 July 2021, 23:48:02 UTC
5e017a3 memory efficient tsview.py for big-data + tsview: memory efficient version via auto multilooking - call pp.auto_multilook_num() for the auto value - multilook ts data and mask as needed - add subset_and_multilook_yx() to convert YX coord for local 3D cube operations, e.g. indexing and referencing - fit_time_func(): support point with nan value / masked out area + utils.readfile.read_hdf5_file(): add print_msg arg to call sys.stdout.write/flush() to print progress, useful for large files + utils.plot: - add auto_multilook_num() to calculate the default / auto multilook number - read_mask(): add x/ystep arg, similar to readfile.read() + view: use plot.auto_multilook_num() to replace the previous check_multilook_input() 04 July 2021, 23:48:02 UTC
22bd98a Effective Coherence Ratio Interferogram Selection (#580) + modify_network: add area-based (ECR) interferogram selection from Kang et al. (2021). - This method excludes/includes interferograms based on the proportional coverage of cells that meet a minimum coherence threshold (defined automatically as the average coherence within the `mintpy.network.maskFile`) - allow for parsing of network.areaBased and network.minArea options from template + utils.utils1: - within spatial_average function, allow inversion of mask to allow for calculation of spatial averages within masked regions - when threshold is specified, values above threshold are set to 1 and values below are set to 0. Thus when the spatial average is calculated, it will represent the proportional area greater than threshold - updates stack function calls - updates output filename and contents + objects.stack: - within ifgramStack and timeseries spatial_average methods, allow inversion of mask to allow for calculation of spatial averages within masked regions - within ifgramStack and timeseries spatial_average methods, when threshold is specified, values above threshold are set to 1 and values below are set to 0. Thus when the spatial average is calculated, it will represent the proportional area greater than threshold + defaults/smallbaselineApp(_auto).cfg: - add network.areaBased and network.minArea options Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 04 July 2021, 07:44:13 UTC
a6e69df tsview: reconstruct and plot temporal model (#589) + tsview: reconstruct and plot temporal model - reconstruct and plot the temporal model (time func predictions) by default - add `plot_ts_fit()` to plot the temp model and print out the time func param results as legend to take advantage of the loc=best option, to auto avoid overlap with data in the plot - update `plot_point_timeseries()` to handle both TS obs and predictions. - this new default TS fit plotting can be turned off using `--nomodel/nofit` option - support complex time func fitting: - handle input cmd via `arg_group.add_timefunc_argument()` - add `fit_time_func()` for OLS estimation, leveraging existing functions in `ts2vel.py` - add `get_model_param_str()` to prepare human-friendly summary of est. results., leveraging `ts2vel.py` for the dataset name/unit dict objects. - rename `save_ts_plot()` to `save_ts_data_and_plot()` and write full time func est results to the header of the txt file. - remove the obsolete `estimate_slope()` + timeseries2velocity.py: - add init_exp_log_dicts() to translate inps.exp/log into inps.expDict/logDict. - fix bug in onset time checking for log/exp in read_inps2model() - estimate_time_func(): 1) check empty e2 due to rank-deficient design matrix with useful printout info; 2) remove unused dis_ts_std arg - add model2hdf5_structure() to get the ds_name/unit_dict from model dict, in order to 1) use the generic writefile.layout_hdf5(), to replace the previous ts2vel-unique layout_hdf5(); 2) re-use in tsview.py - add model2hdf5_dataset() to organize the m/m_std into the HFD5 ready dataset dict, in order to 1) use the generic writefile.write_hdf5_block(), to replace the previous ts2vel-unique write_hdf5_block(); 2) re-use in tsview.py + utils.ptime.py: add get_date_range(): to produce a dense and continuous range of dates between dmin and dmax + utils.arg_group: add add_timefunc_argument() for easy re-use in ts2vel.py and tsview.py. + utils.writefile.layout_hdf5(): - add `ds_unit_dict` arg to support HDF5 dataset attributes - complete arg call for `metadata` of writefile.layout_hdf5() in ifgram_inversion.py, prep_aria.py and prep_fringe.py + view: bugfix in format_coord() for gamma products + docs.demo_dataset: add USGS link and fix typo for the 2008 Wells EQ Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 28 June 2021, 00:22:45 UTC
8cfce73 enu2los+docs: clarify az_angle definition + utils.utils0.enu2los(): support az_angle besides head_angle and use az_angle in the actual calculation because: 1. az_angle is simpler to explain together with incidence angle 2. az_angle can be pixel-wise, thus, more precise than the usually single value of head_angle + utils.utils0: add heading2azimuth_angle() + docs/api/data_structure: add angle definition and vel file structure - Add definitions for geometry datasets, including incidence / azimuth angle, and masks. - Add file structure for the velocity file from the latest timeseries2velocity.py + utils.isce_utils.extract*meta(): expose satelliteSpeed + solid_earth_tides: increase the step size to 3km for speedup 26 June 2021, 04:40:35 UTC
570f417 small changes: dem_error, ifgram_inversion, plot_network (#587) + add a cpt file: romanian.cpt; update the colormaps.md doc + dem_error.py: - allow for periodic temporal function fitting - add defo_model arg group + ifgram_inversion.py: allow for inverting ionosphere timeseries + plot_network.py: baseline plot with zero-mean perp baseline 23 June 2021, 18:57:04 UTC
2029b8c install: add notes for gdal and isce2 + docs/installation: add notes of gdal and isce2 for users with different insar processors + docs/conda_env.yml: add gdal since this install a new env from scratch, thus, won't force un-wanted constraint to existing env. 21 June 2021, 05:35:28 UTC
379c4f9 ts2vel: add --ts-std-file to propagate STD from TS to VEL 20 June 2021, 20:42:56 UTC
20e74a7 docs/dir_structure: add notebook link for hyp3 (#581) + docs/dir_struct: add notebook link for hyp3 Co-authored-by: Andrew Johnston <asjohnston@alaska.edu> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 20 June 2021, 01:42:54 UTC
5fe1cba Set theme jekyll-theme-cayman 19 June 2021, 23:23:30 UTC
2913101 Set theme jekyll-theme-minimal 19 June 2021, 23:21:46 UTC
b4b33b4 Set theme jekyll-theme-cayman 19 June 2021, 23:20:53 UTC
4884694 Set theme jekyll-theme-minimal 19 June 2021, 23:20:33 UTC
1f32f2a resample: support using isce-2 lat/lon.rdr file + objects.resample: Support geocoding using ISCE-2 lat/lon.rdr file directly, as an alternative of using mintpy geometry file in HDF5 format, to facilitate post-processing of single interferogram from isce. An example usage is shown in the function comment. + objects.ramp: add ignore_zero_value argument, to better handle offset field because zero value is common as no-data for phase but valid for offset. + generate_mask: add --view-cmd option for --roipoly to customize view.py display to facilitate the interactive polygon selection + ifgram_inversion: in the comment, replace pinv2 with pinv since scipy v1.7.0 has merged pinv2 into pinv and will deprecate pinv2, for notes in the future + spatial_filter: fix typo in the comments of filter_par with more tolerant handling of input filter_par var type: string or (list of) int/float + plot_network & utils.plot: reduce markersize for less crowd view 19 June 2021, 22:38:32 UTC
3802392 hyp3: convert inc angle unit from rad to deg and fix SET (#578) + prep_hyp3: add UNIT=radian for incidence angle file + stackDict.write2hdf5(): convert the unit of incidence angle from radian to degree if it's hyp3 processor AND unit == radian + solid_earth_tides.prepare_los_geometry(): convert Y/X_FIRST/STEP/UNIT from meters to degrees if needed, similar to the previous changes in tropo_pyaps3.py 14 June 2021, 04:29:35 UTC
3813901 SET: use HEADING if azimuthAngle is not available 12 June 2021, 22:46:44 UTC
7539b16 sbApp.plot_result: bugfix for ifgramStack file + dir_struct*: break line for hyp3 description 12 June 2021, 21:35:47 UTC
2e3dcea prep_hyp3: update to be compatible with current hyp3 products (#573) - removes hard coding of EARTH_RADIUS and HEIGHT - enables use of HyP3 DEM and incidence angle products - reorganizes main() and add_hyp3_metadata() functions - merge add_sentinel1_metadata() into add_hyp3_metadata() - updates help information - update hyp3 related docs 11 June 2021, 03:36:24 UTC
b1f636f timeseries2velocity: add --exp and --log time funcs velocity fitting (#564) This PR adds two more time functions to timeseries2velocity.py: + exp: A series of exponential function(s). Modeling the amplitude of post-seismic relaxation with input: - list of str, dates of the onset of exp function(s), e.g., '20161231', '20190125'. - list of float, exp characteristic time(s) in days, e.g., 80.5, 200. + log: A series of logarithmic function(s). Modeling the amplitude of post-seismic relaxation with input: - list of str, dates of the onset of log function(s), e.g., '20181231', '20200125'. - list of float, log characteristic time(s) in days, e.g., 60.2, 120. + obects.stack.get_design_matrix4time_func(): - add get_design_matrix4exp_func() - add get_design_matrix4log_func() + timeseries2velocity: - add --exp onset_time char_time option to ingest exponential function parameters - add --log onset_time char_time option to ingestt logarithmic function parameters - write periodic phase data as the following dataset names in the unit of radian to file: - annualPase - semiAnnualPhase - periodY{period}Phase - rename the periodic amplitude dataset name from using "Amp" as suffix to using "Amplitude" as suffix Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 08 June 2021, 15:50:28 UTC
a57017e view: better handle multi-subplots with diff units + utils.readfile.read_attribute: grabbing HDF5 dataset attributes if datasetName arg is specified, to get the dataset specific attributes, e.g. unit, no-data value, fill value, etc. + view: better handle multi-subplots with different units - ignore "Std" suffix while grabbing the dataset family list, to better check whether dset list shares the same unit - remove "velocity" from the special file type list while checking `same_unit4all_subplot` because of the recent updates to velocity file with complex time funcs. - if not `same_unit4all_subplot`, printout warning message and set disp_unit to None for more informative msg - add data unit whenever it's possible if showing colorbar for each subplot. + tropo_pyaps3: bugfix for potential path conflict while calling findall to grab date/hour info by use the grib file basename 08 June 2021, 04:07:09 UTC
540b563 circleci: use generic fortran compiler syntax + use mamba install fortran-compiler as the generic name/syntax to replace gfortran_linux-64 + do not use detailed path because mamba in the non-default env is located in an alias in miniconda3/condabin/ and not the default path. 04 June 2021, 01:11:25 UTC
51196d5 plot_insar_vs_gps_scatter(): update to the new GPS CSV file format * objects.insar_vs_gps.plot_insar_vs_gps_scatter(): + update to the new GPS CSV file format + add msk_file option to exclude GPS sites that do not have valid SAR measures + add progress bar + remove nan values + add vlim to 1) prune SAR and GPS obs and 2) custom display range + printout stats info, including min/max/rmse/r2 * objects.gps.get_los_geometry(): check GPS site coordinates against SAR image boundary to avoid errors due to the GPS sites on the edge * utils.plot.plot_gps(): plot the label of reference GPS site * plot_transection.py: add --vlim option to be able to customize the display color range of maps, to facilitate the profile selection. 04 June 2021, 00:32:38 UTC
3b423f8 circleci: speedup dependency installation + circleci: - use mamba to replace conda for dependency solving and installation - increase the no output timeout from 10m to 30m + tropo_pyaps3: bugfix of date info grabbing while downloading GRIB files 04 June 2021, 00:32:38 UTC
27f5c76 view: add --math option for single subplot 04 June 2021, 00:32:38 UTC
521d17b view: support --ref-lalo for radar-coded file if lookup table file exists + view: Use coordinate.geo2radar() to support view.py --ref-lalo on the fly for file in radar coordinates, if the default lookup table file can be found and used, otherwise ignore (same as before), print out warning msg and continue. + utils.plot.plot_colorbar(): improved tick position for numInvIfgram.h5 where color step num is not common + utils.plot.plot_network(): add xlabel and cbar_size for improved flexibility, with default value unchanged. + tropo_pyaps3: add Herbach et al. (2020) to the recommended reference for ERA-5 + utils0.check_parallel(): add link to joblib example + circleci: use full path for pip to avoid warning + save_kmz: leverage default mpl cbar bin num to replace the previous default 7, which does not work nicely as the mpl default. 04 June 2021, 00:32:38 UTC
8212fe1 colormap: add differences.cpt from cpt-city 04 June 2021, 00:32:38 UTC
5dbdde9 solid_earth_tides: bugfix of missing CENTER_LINE_UTC (#568) I've proposed to use the attributes from the time-series file instead of the geometry file. This implementation is made due to the possible incomplete geometryRadar.h5 attributes on the alosStack. This edit is also consistent with the workflow used by tropo_pyaps3.py which uses the time-series file instead of the geometry file for the attributes. 04 June 2021, 00:07:34 UTC
4ee12c6 hyp3: tropo_pyaps support via incidence angle (#567) * modify `tropo_pyaps3.py` and `tsview.py` to handle no-lat/lon coordinates, such as UTM for ASF HyP3. * support loading incidence angle file in prep_hyp3.py * circleci: output text line by line to avoid too long with no output error 03 June 2021, 04:20:54 UTC
0a530d7 get MST from remaining net after other thresholds for coh-based net modify (#563) + generate MST network from the remaining network after applying all the other criteria, instead of from the original network. + adjust the order of template options accordingly, to be visually consistent with the code execution order. Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 22 May 2021, 23:50:21 UTC
a45bba2 gps: re-use LOS vel/dis from CSV file in view.py This commit adds two new functionalities: 1. it save the GPS LOS displacement (and velocity) to the CSV file. The file is named after the --gps-comp option to facilitate the update mode checking. 2. view.py will now check the existing CSV file for plotting whenever it's possible to avoid re-calculating, which can be long for dense GPS regions. This skipping can be overwrite via the --gps-redo option. The resulting new function gps.get_gps_los_obs() can be used for more generic applications. + docs/attr: HEADING angle definition in details + view: do not show the original reference point if --ref-gps is ON 16 May 2021, 02:50:30 UTC
8aa9d49 bring back view.py --update & hardwire offset covar band info in readfile.py + view.py: bring back the --update option, whose functionality is lost during the refractoring of viewer object. + plot.auto_figure_title() for view.py: show dset name for multi-band binary files + readfile: - get_slice_list(): add hardwired band/slice list for offset covariance file from ampcor - get_dataset_list(): call get_slice_list() for binary file type to remove duplicated code body - read_binary_file(): call get_slice_list() for ISCE files for informative and flexible band data reading 16 May 2021, 02:50:30 UTC
afa5f1e stackDict/readfile: follow-up edit of passing geom from tmpl (#559) * stackDict.py,readfile.py: follow-up edit of passing geom baselines from tmpl + stackDict.py - Generate 2D slant range distance if missing from the input template file: For dataset in geo-coordinates, use: 1) incidenceAngle matrix if available OR 2) constant value from SLANT_RANGE_DISTANCE. + readfile.py - read_binary_file(): for los.rdr, allow more flexible filenames, a.k.a. start with 'los' - read_attribute(): force to use the specified metadata file extension if specified 08 May 2021, 21:41:07 UTC
03909fd prep_isce: pass exact geom basenames from tmpl Add prep_isce.py --geom-files to use the exact geometry basenames from input template, for more flexibility, to facilitate topsApp support. + prep_isce.py: - add GEOMETRY_PREFIXS as a global variable - add --geom-files options, to be able to use the exact basenames, with default value for tops/stripmapStack results + load_data: - pass geometry basenames from template file to prep_isce.py - pass ifg metadata in geo-coord to geometryDict if available + docs/CONTRIBUTING: fix typo 08 May 2021, 02:59:46 UTC
67b898a save_kite.py: Added option to subset the data and minor fixes (#557) + add --subset-x/y/lat/lon option via mintpy.utils.arg_group and mintpy.subset. + fix an output file name var bug + split DATE12 regardless of the convention. It's not consistent in the code (which should be corrected for consistency sometime in the future) and can be YYMMDD-YYMMDD or YYYYMMDD_YYYYMMDD. 05 May 2021, 19:02:28 UTC
16b0bed spatial_filter: add double difference filter option (#552) * spatial_filter: add `double_difference` filter option The double difference filter as used in Bekaert et al. (2020, RSE) has been shown to effectively filter out non-local modes of deformation. This particularly when using MintPy to monitor local forms of deformation, such as landslides. This commit implements the double_difference filter in spatial_filter.py and reformats the filter_par parameter so that it accepts multiple arguments, as required by double_difference. This PR also fixes missing the missing dataset bug when datasets were specified, ONLY the datasets specified were written to the output file. This patch ensures that all datasets present in the input file are present in the output file, even if they are unfiltered. 02 May 2021, 04:59:01 UTC
ad3dcde insar_vs_gps: set insar time to zero to be consistent with the GPS datetime 01 May 2021, 21:40:37 UTC
c52ea85 s*b*App: ignore [] in the template opts Ignore the "[" and "]" in all template options that have potentially more than 1 values, to be more tolerant to the input syntax error. 01 May 2021, 19:05:12 UTC
6d5dcf8 docs/dir_struct*: shorten alosStack + docs/dir_structure: remove file structures in alosStack example that are not relevant to mintpy to simplify the structure + docs/dir_structure: improve overall tree structure diagram + view: move tight_layout() after set_size_inches() for a nicer separation between subplots when there are multiple color bars 01 May 2021, 19:05:12 UTC
3d16eae add mintpy.unwrapError.numSample option + add mintpy.unwrapError.numSample option for unwrapping error correction with phase closure, to speedup the LASSO estimation during circle CI testing. The default value is the same as the previously hardwired value (100), so no impact is expected. + add comments on the region-based inversion strategy for phase closure unwrapping error correction. + test/configs/Fernandina*.txt: add mintpy.unwrapError.numSample = 20 to speedup circle CI testing 24 April 2021, 00:00:43 UTC
90aee2b view: share x/y axis for multiple subplots + view: - turn on the sharex/y for multiple subplots using plt.subplots() and fig.delaxes() to replace the previous plt.figure() and fig.add_subplot(), to remove the unnecesary ticklabels between subplots. - plot_figure(): update set_window_title() usage to the recommend usage to avoid deprecation warnning. + utils.plot: set nan value pixel in the mask file as false + utils.readfile.get_slice_list: more tolerant file name pattern for offset bip file + objects.sensor: Add an incomplete set of sensor parameters for SAOCOM from Giudici et al. (2017) + asc_desc2horz_vert: fix typo in the print out msg for distance unit + save_kite: move kite import into sub-func so that one can check the help msg, without kite/pyrocko installed. + docs/dir_struct*: format cleanup 24 April 2021, 00:00:43 UTC
b5cae26 add save_kite.py for pyrocko suite (#549) + add save_kite.py to save 1) linear velocity, 2) step, or 3) timeseries displacement into KITE format (yaml, npz) - this needs kite and pyrocko module to work.: https://pyrocko.org/ - prepared data can be used for inversions in GROND or BEAT + asc_desc2horz_vert: converted azimuth angle from degree to radians in the design matrix Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 23 April 2021, 21:52:48 UTC
7adb3a1 view: support file in UTM projection + readfile.read_gdal_vrt(): - add X/Y_FIRST/STEP for all coordinate types - add UTM_ZONE for file in UTM projection - add X/Y_UNIT in meters/degrees + prep_hyp3: use readfile.read_gdal_vrt() to replace add_geospatial_metadata() for less rerdundancy and the new UTM_ZONE attribute + prep_gmtsar: codacy suggestion + view: support file in UTM projection by default - support native matplotlib display in geo-coord via setting extent arg for files in UTM projection - remove the redundant and obsolete code block in plot_slice() for coord unit of 'm' - untie --coastline and --lalo-label option because cartopy seems to solve the previous default blank ticks issue - raise exception for --lalo-label for file in UTM projection - add code to auto determine the map projection for files with coord unit of deg and meter. + utils/plot.draw_scalebar(): add unit arg to support coordinate unit of meters + utils/arg_group.add_map_arg*(): comment out --proj option because we can not really plot the raster image in a map projection that is different from the native image data projection/transform using cartopy, at least not in the non-global scale 17 April 2021, 22:57:18 UTC
ce69347 view: support ref-point out of the subset box for time-series + view.update_data_with_plot_inps(): - support spatial referencing on the fly with the reference point out of the subset box, for time-series file only. - check ref_val before re-referencing and ignore maskedout or nan ref_val + (ts)view: pass print_msg to auto_adjust_colormap_lut_and_disp_limit() + tsview: remove _ in default outfile base + utils0: add polygon2mask() to convert a polygon into a binary mask file. + deramp: change mask_in from required to optional with default as None + smallbaseApp.plot_results(): move opt common to the front for better alignment in the printout msg + defaults/smallbaselineApp.cfg: fix typo for recommended + ts2vel: linalg.lstsq add default rcond to avoid warning msg 17 April 2021, 22:57:18 UTC
6c9c2f3 ref_date: support -r HDF5 file + reference_date: support HDF5 file as input for --ref-date option, in addition to existing input types + utils1.update_template_file(): bug fix when the exiting key value contains *, by treating it as a normal character, instead of wildcard 17 April 2021, 22:57:18 UTC
c4553c7 network.thres*_baseline(): support both date12_list format + utils/network.thres*_baseline(): support both date12_list format + plot_network: correct labels for offset SNR 17 April 2021, 22:57:18 UTC
7b56e28 load_gbis: add hgt to output HDF5 + load_gbis: - add hgt from .mat file to output HDF5 file since this height could be corrected from ellipsoid to geoid already, which is more suitable for geophysical interpretation. - do not use "height" to avoid confusion because it's pre-occupied by mintpy geometry file for FILE_TYPE by readfile.read_attribute() + asc_desc2horz_vert: API supp for matrix 17 April 2021, 22:57:18 UTC
6d2e234 load_data: supp. gmtsar using test data from Becca + prep_gmtsar.py: - calc/write SLANT_RANGE_DISTANCE/INCIDENCE_ANGLE for dataset in geo-coord since the pixel-wise geometry data is missing - add prepare_geometry() to write rsc file for each geometry file - use upper case for ORBIT_DIR* to be consistent with other processors + load_data.prepare_metadata(): add call of prep_gmtsar.py + load_data.read_inps_dict2ifgram_stack_dict_object(): - support search/match data file using the modern Julian date12 for gmtsar - more comments explaining ifgramPathDict and pairsDict + docs/dir_structure: add example for GMTSAR: - add example dir structure - add template file options - add a link to a GMTSAR stack recipe + utils.ptime.yyyymmdd(): bugfix for date str with "THHMM" suffix, such as the data for UAVSAR + sh/plot_sbApp.sh: - fix the bug of previously missing timeseries.h5 - apply the same generic find and plot syntax to geo_timeseries_* files + sbApp.plot_results(): move output option to the front for visual alignment 17 April 2021, 20:13:43 UTC
176170c add prep_gmtsar.py + utils.readfile: - add attribute names from GMTSAR *.PRM file to standardMetadataKeys - read_binary_file: read GMTSAR files via read_gdal() - add read_gmtsar_prm() and attribute_gmtsar2roipac() to read attributes from GMTSAR *.PRM file + add prep_gmtsar.py based on the discussion/notes from Xiaohua Xu 17 April 2021, 20:13:43 UTC
2b63765 fix links for ASF HyP3 (#545) Links in docs/README.md and docs/dir_structure.md point at https://hyp3.asf.alaska.edu which is the front page of our beta hyp3 system. for information on the latest and greatest HyP3 which you guys implemented these links should point to https://hyp3-docs.asf.alaska.edu. 09 April 2021, 00:09:02 UTC
f171b9f spatial_filter: allow for filtering of specific datasets (#544) If filtering interferograms prior to inversion, it is often desirable to filter only the unwrapPhase dataset and not the coherence dataset. These changes to spatial_filter.py enable this functionality. Changes the command-line interface slightly, which may break older workflows that use spatial_filter.py. 07 April 2021, 04:33:17 UTC
e3440cd save_gbis: maskout pixels w zero phase value (#543) + mask out pixels with zero phase values, for more appropriate masking even without feeding a mask file manually. Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 30 March 2021, 20:08:36 UTC
4d08d5a ingest ASF HyP3 (#542) + create `prep_hyp3.py` to prepare rsc metadata files for ASF HyP3 data, which is an on-demand service for producing Sentinel-1 interferograms. + load_data.py: add hyp3 to the processor list, and sets up a reference to prep_hyp3 when hyp3 is the processor. + utils.readfile: read HyP3 products as GDAL TIFF files + docs/dir_structure: add example directory structure and template options for HyP3 + prep_hyp3/aria: fix bug of LAT/LON_REF1/2/3/4, considering their differences between asc /desc geometries. 27 March 2021, 00:47:02 UTC
569be39 fix minor print typos (#541) Various spelling corrections. 24 March 2021, 03:04:53 UTC
2fbbf8d merge plot_sbApp.sh to sbApp.plot_result() for platform neutral method (#538) + translate `mintpy/sh/plot_smallbaselineApp.sh` to pure python and merge them into `mintpy/smallbaseApp.TimeSeriesAnalysis.plot_result()` to support result plotting on Windows OS. + smallbaselineApp.py: - add `--plot` argument. If only --plot in the command line, plot the result without run the processing steps. - remove the now redundant "plot_smallbaselineApp.sh" related code in startup() - remove unused `re` and `subprocess` module at the top - rename startup() to open() - add close() - use from mintpy import tropo_pyaps for pyaps in python2, as planned. - codacy suggestions + `workflow/__init__.py`: add `view` + utils.check_loaded_dataset(): add relpath arg + view.read_dataset_input(): fix potential bug when view.main() is called multiple times for geometry files Co-authored-by: Panji Brotoisworo <pbrotoisworo@stamina4space.upd.edu.ph> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> 22 March 2021, 02:11:06 UTC
01db97d plot_sbApp.sh: add mising time-series files + utils.ptime.yyyymmdd(): bugfix for date str with "THHMM" suffix, such as the data for UAVSAR + sh/plot_sbApp.sh: - fix the bug of previously missing timeseries.h5 - apply the same generic find and plot syntax to geo_timeseries_* files 19 March 2021, 20:59:54 UTC
bc7d4ff plot_dem_background: move contour line to the front if shade is not shown 19 March 2021, 20:59:54 UTC
078e5e4 circleci: separate dset testing for easy navigation 19 March 2021, 20:59:54 UTC
b760bcc read_attribute: support GSI DEHM file utils.readfile.read_attribute(): add pre-defined attributes for GSI DEHM file so that it can be read and display directly. 19 March 2021, 20:59:54 UTC
77d32b3 readfile: refactor standardize_metadata() + utils.readfile: - more generic and organized format of global var standardMetadataKeys - update standardize_metadata() to use the new standardMetadataKeys - remove unused arguments in read_*(standardize) - read_gamma_par(): keep the order of 'attribute_gamma2roipac()' first and `standardize_metadata()` second - attribute_gamma2roipac(): keep its independent from 'standardize_metadata()' + objects.coord: use explicit import to replace "*" import 19 March 2021, 20:59:54 UTC
89e7a32 ifgramStack.temporal_average: cast ds_size to int64 to avoid overflow 32-bit OS (#537) 19 March 2021, 03:25:26 UTC
debb699 deramp: add --save-ramp-coeff to write est. coefficients to text file + objects.ramp.deramp(): add coeff_file argument write estiated coefficients into coeff_file is it's not None. + utils.utils1.run_deramp(): add save_ramp_coff to write estimated ramp coefficients into "rampCoeff_{}.txt" file by default and prepend dataset info for each 2D matrix: - date for time-series - date12 for ifgramStack - atr['FILE_TYPE'] for the others + remove_ramp.py: add --save-ramp-coeff option to turn on the saving/writing of estimated ramp coefficients into text file. The default is turned off so that the behavior stays the same as before. 14 March 2021, 23:29:07 UTC
d81a9a2 tropo_pyaps3: bugfix for large invalid area in the geometry Fix the bug of SNWE calculation when there is a large invalid area in the geometry file via: + feed the mask of valid pixels to PyAPS + use latitude/longitude dataset directly to calc SNWE if avaialble 14 March 2021, 07:16:45 UTC
a1b53e8 wrap up for version 1.3.0 + update version number and release date for v1.3 + remove solid_f from .gitignore 07 March 2021, 02:07:08 UTC
5535e91 view: cbar for subplots with diff units + view: show colorbar for multiple subplots with different units by default, for more informative display, e.g. geometryRadar.h5 file + sh/plot_smallbaselineApp.sh: one liner to find & grab all time-series files and plot them. Shorter code and more compatibility, e.g. GACOS and SET surfix. + load_data: ignore meta_file is not found, so that sbApp.py can still run with inputs folder and without meta files for isce products + loca_oscilator_drift.py: update default output filename, to be consistent with sbApp.py + save_kmz_ts: skip generating symbols for the 3rd level of step size if <= 0. The whole region concepts need finer tuning, reference link: https://developers.google.com/kml/documentation/regions + utils0: add root_mean_sq_error() for easy call. 02 March 2021, 05:40:07 UTC
814d594 docs/install: add notes on pysolid for manual compilation + docs/install: add notes on pysolid for manual compilation + docs/api/attribute: use LENGTH to replace FILE_LENGTH to be consistent with the actual code usage. + docs/hdfeos5: bold attribute name for easy navigation 02 March 2021, 05:40:07 UTC
b21712c SET: add --date-wise-acq-time option + solid_earth_tides: update usage to the latest PySolid version 0.1.2 + solid_earth_tides: support grabbing the exact UTC second for each acquisition - but negligible impact on SET calculation from testing, thus, turned off by default. - add get_datetime_list(ts_file) to get the list of datetime.datetime objects for the exact time of each acquisition, which has <1 mins difference for Sentinel1(2015-2021) and >30secs difference btw. S1A/B - add plot_sensingMid_variation() - update calc_solid_earth_tides_timeseries() accordingly and write `sensingMid` to SET.h5 file. - add --date-wise-acq-time option for easy turn on/off + solid_earth_tides: minor updates - do not add "geo_" prefix to the calculated SET HDF5 files, to be consistent with tropo correction. - add processor/coordinates checking, same as tropo_gacos.py - update function comments + utils.ptime: add round_seconds() from stack overflow example. + isce_utils: - add safe_list_file2sensor_list() for Sentinel-1 to grab S1A/B - add get_sensing_datetime_list() to grab the exact datetime for all acquisitions from topsStack processing results 25 February 2021, 06:11:45 UTC
0b7edc4 ifg_inv: calc offset estimation residual + ifg_inv: adjust code offset TS estimation: - calculate LS residual instead of temporal coherence for offset and write them to residualInvRg/Az.h5 file - use np.nan as default residual value - rename variables from temp_coh to inv_quality to be nuetral btw. offset and phase. + readfile.get_slice_list(): support offset.bip bands + utils: - plot.read_mask: add maskResInv to the default mask file - utils.transect_yx(): clean up - utils0: move TEC related funcs to the private simulation tools for better independency + tsview: grab label from file name instead of index 25 February 2021, 06:11:45 UTC
b22a156 ts2vel/tsview: supp time-series with empty dates ts2vel: support time-series with empty dates by identifying dates with all zero/nan values and exclude them (reference date is all zero and still kept) tsview: + support time-series with empty dates by using a more tolerant mask + do not show the "Time [year]" label on the X-axis as it's obvious, to save space 25 February 2021, 06:11:45 UTC
0aed97d ref_date: add --force option + reference_date: add --force option to not check existing REF_DATE attribute and to update the matrix value by force. + docs/FAQs: break lines for readthedocs rendering 25 February 2021, 06:11:45 UTC
7a56936 docs: add custom preparation guide for un-supportted InSAR softwares. + docs/FAQs: add guide for custom input preparation + utils.utils0.median_abs_deviation(): more comments on the comparison against the version in scipy 24 February 2021, 19:46:07 UTC
e1a5c3b prep_aria: use bnd.ReadAsArray() for consistency + prep_aria: - write_geometry(): use bnd.ReadAsArray() to replace ds.ReadAsArray() for consistency to write_ifgram_stack() and readfile.read_gdal() - use `with h5py.File() as f` syntax so that the HDF5 is always properly closed, if program is killed during the writing process. + use FileNotFoundError to replace improper FileExistsError usages in info/view/tropo_gacos/pyaps3.py 24 February 2021, 19:46:07 UTC
6c85534 support GACOS .tif format via readfile.read_gdal() + utils.readfile: add read_gdal() - add read_gdal() to read files that contains both data and attributes inside, e.g. *.tif/grd, using GDAL - add "gdal" as an independent processor, for files that 1) are not recognized by existing insar processor AND 2) have GDAL supported file extensions, defined in readfile.GDAL_FILE_EXTS + tropo_gacos: - support the new *.tif format - add more input files checking + docs/references: add literature for GACOS 22 February 2021, 19:36:36 UTC
91bc780 ref_date: bugfix for block arg while calling write_hdf5_block() + bugfix for block arg while calling write_hdf5_block() + use dict.get() for default value check for input file without REF_DATE attribute 22 February 2021, 19:36:36 UTC
c6a32b6 load_data: bugfix for update mode checking when x/ystep > 1 (#517) + load_data: bugfix for update mode with x/ystep > 1 + isce_utils.read_baseline_timeseries(): ignore empty bperp text files for tops/stripmapStack + view: copy argv to inps for robustness as `sys.argv` behaves differently between in command line and in jupyter notebook. 11 February 2021, 03:36:40 UTC
22e82f4 load_data: support attr from template file Support add missing metadata from template files, if they are specified manually with key option name in MACRO_CASE/UPPER_CASE. + sbApp.cfg: add comments regarding manually add attributes in the template file + load_data.get_extra_metadata(): grab metadata with key name in MACRO_CASE 06 February 2021, 03:56:44 UTC
back to top