https://github.com/insarlab/MintPy

sort by:
Revision Author Date Message Commit Date
a1b53e8 wrap up for version 1.3.0 + update version number and release date for v1.3 + remove solid_f from .gitignore 07 March 2021, 02:07:08 UTC
5535e91 view: cbar for subplots with diff units + view: show colorbar for multiple subplots with different units by default, for more informative display, e.g. geometryRadar.h5 file + sh/plot_smallbaselineApp.sh: one liner to find & grab all time-series files and plot them. Shorter code and more compatibility, e.g. GACOS and SET surfix. + load_data: ignore meta_file is not found, so that sbApp.py can still run with inputs folder and without meta files for isce products + loca_oscilator_drift.py: update default output filename, to be consistent with sbApp.py + save_kmz_ts: skip generating symbols for the 3rd level of step size if <= 0. The whole region concepts need finer tuning, reference link: https://developers.google.com/kml/documentation/regions + utils0: add root_mean_sq_error() for easy call. 02 March 2021, 05:40:07 UTC
814d594 docs/install: add notes on pysolid for manual compilation + docs/install: add notes on pysolid for manual compilation + docs/api/attribute: use LENGTH to replace FILE_LENGTH to be consistent with the actual code usage. + docs/hdfeos5: bold attribute name for easy navigation 02 March 2021, 05:40:07 UTC
b21712c SET: add --date-wise-acq-time option + solid_earth_tides: update usage to the latest PySolid version 0.1.2 + solid_earth_tides: support grabbing the exact UTC second for each acquisition - but negligible impact on SET calculation from testing, thus, turned off by default. - add get_datetime_list(ts_file) to get the list of datetime.datetime objects for the exact time of each acquisition, which has <1 mins difference for Sentinel1(2015-2021) and >30secs difference btw. S1A/B - add plot_sensingMid_variation() - update calc_solid_earth_tides_timeseries() accordingly and write `sensingMid` to SET.h5 file. - add --date-wise-acq-time option for easy turn on/off + solid_earth_tides: minor updates - do not add "geo_" prefix to the calculated SET HDF5 files, to be consistent with tropo correction. - add processor/coordinates checking, same as tropo_gacos.py - update function comments + utils.ptime: add round_seconds() from stack overflow example. + isce_utils: - add safe_list_file2sensor_list() for Sentinel-1 to grab S1A/B - add get_sensing_datetime_list() to grab the exact datetime for all acquisitions from topsStack processing results 25 February 2021, 06:11:45 UTC
0b7edc4 ifg_inv: calc offset estimation residual + ifg_inv: adjust code offset TS estimation: - calculate LS residual instead of temporal coherence for offset and write them to residualInvRg/Az.h5 file - use np.nan as default residual value - rename variables from temp_coh to inv_quality to be nuetral btw. offset and phase. + readfile.get_slice_list(): support offset.bip bands + utils: - plot.read_mask: add maskResInv to the default mask file - utils.transect_yx(): clean up - utils0: move TEC related funcs to the private simulation tools for better independency + tsview: grab label from file name instead of index 25 February 2021, 06:11:45 UTC
b22a156 ts2vel/tsview: supp time-series with empty dates ts2vel: support time-series with empty dates by identifying dates with all zero/nan values and exclude them (reference date is all zero and still kept) tsview: + support time-series with empty dates by using a more tolerant mask + do not show the "Time [year]" label on the X-axis as it's obvious, to save space 25 February 2021, 06:11:45 UTC
0aed97d ref_date: add --force option + reference_date: add --force option to not check existing REF_DATE attribute and to update the matrix value by force. + docs/FAQs: break lines for readthedocs rendering 25 February 2021, 06:11:45 UTC
7a56936 docs: add custom preparation guide for un-supportted InSAR softwares. + docs/FAQs: add guide for custom input preparation + utils.utils0.median_abs_deviation(): more comments on the comparison against the version in scipy 24 February 2021, 19:46:07 UTC
e1a5c3b prep_aria: use bnd.ReadAsArray() for consistency + prep_aria: - write_geometry(): use bnd.ReadAsArray() to replace ds.ReadAsArray() for consistency to write_ifgram_stack() and readfile.read_gdal() - use `with h5py.File() as f` syntax so that the HDF5 is always properly closed, if program is killed during the writing process. + use FileNotFoundError to replace improper FileExistsError usages in info/view/tropo_gacos/pyaps3.py 24 February 2021, 19:46:07 UTC
6c85534 support GACOS .tif format via readfile.read_gdal() + utils.readfile: add read_gdal() - add read_gdal() to read files that contains both data and attributes inside, e.g. *.tif/grd, using GDAL - add "gdal" as an independent processor, for files that 1) are not recognized by existing insar processor AND 2) have GDAL supported file extensions, defined in readfile.GDAL_FILE_EXTS + tropo_gacos: - support the new *.tif format - add more input files checking + docs/references: add literature for GACOS 22 February 2021, 19:36:36 UTC
91bc780 ref_date: bugfix for block arg while calling write_hdf5_block() + bugfix for block arg while calling write_hdf5_block() + use dict.get() for default value check for input file without REF_DATE attribute 22 February 2021, 19:36:36 UTC
c6a32b6 load_data: bugfix for update mode checking when x/ystep > 1 (#517) + load_data: bugfix for update mode with x/ystep > 1 + isce_utils.read_baseline_timeseries(): ignore empty bperp text files for tops/stripmapStack + view: copy argv to inps for robustness as `sys.argv` behaves differently between in command line and in jupyter notebook. 11 February 2021, 03:36:40 UTC
22e82f4 load_data: support attr from template file Support add missing metadata from template files, if they are specified manually with key option name in MACRO_CASE/UPPER_CASE. + sbApp.cfg: add comments regarding manually add attributes in the template file + load_data.get_extra_metadata(): grab metadata with key name in MACRO_CASE 06 February 2021, 03:56:44 UTC
072c762 hdfeos5: support attr from template file + auto grab aux filenames based on the input ts_file + read metadata from template files where key does not starts with mintpy or isce, to easily add missing metadata + add create_hdf5_dataset() to simplify write_hdf5_file() + docs/hdfeos5: add number to titles for easy navigation 06 February 2021, 03:56:44 UTC
f34c351 view: shorter fig title/name + view/plot: - do not show subplot number for <=6 subplots - remove ".0" for "_wrap6.0" suffix in the output filename + fix typo: environmental variable -> environment variable + docs: simplify alosStack dir structure 06 February 2021, 03:56:44 UTC
f73768e sensor.standardize_sensor_name(): str/bytes conversion + objects.sensor.standardize_sensor_name(): add str/byte conversion for input sensor name var + test/test_sbApp: add --test-isce for prep_isce 02 February 2021, 20:52:15 UTC
87cd285 save_kmz: add --zero-mask option + save_kmz: add --zero-mask option same as view.py + readfile.read_template(): clean up for better organize code for insarmaps. They should be moved in the future. + ifgram_inversiono.run_or_skip(): fix bug when input is offset + tsview: support --ref-yx if input file don't have REF_Y/X + plot.auto_adjust_lut_and_disp_limit(): support data with inf values + save_gbis: add reference 02 February 2021, 20:52:15 UTC
d42658e asc_desc2hz_up: check ref point using REF_Y/X with --max-ref-yx-diff asc_desc2horz_vert.py: + check the reference point consistency using REF_Y/X (derived from REF_LAT/LON) instead of REF_LAT/LON to better handle various spatial resolution. + add --max-ref-yx-diff option to be able to customize/relax the REF_Y/X difference if needed, with default value of 3 pixels. utls.plot.plot_dem_background(): revert zorder of DEM contour back to 1, because there was no issue there. 27 January 2021, 01:07:32 UTC
8f7c82d load_gbis: support after location changge + support input file reading from *.inp file if the data path saved in the result .mat file does not exist. This happens when one run GBIS in one machine and plot/do other stuff in another machine. 27 January 2021, 01:07:32 UTC
cc6cc04 ts2vel: fix typo in reference pixel print out msg (#510) 25 January 2021, 07:20:48 UTC
dea435c binary file IO: ignore file ext .geo/rdr/full/wgs84 + ignore in the following funcs the file extensions of .geo/rdr/full/wgs84 which does not provide data structure info, for improved smart file auto IO - readfile.read_attribute() - writefile.write() - save_roipac.read_data() + multilook.multilook_file: typo fix + view.format_coord: fix warning msg of masked array + utils.plot: remove deprecated --lat-label vertical/horzontal, which was added while using basemap and is not supported via current implementation of cartopy anymore. + utils.plot.plot_dem_background(): fix coutour zorder issue. + README: update links to google groups and github discussions + update comments on smallbaselineApp.cfg 23 January 2021, 23:46:55 UTC
d426309 bugfix of x/ystep in load_data for gamma/roipac + load_data: apply the same number of looks to geo geometry file if obs is in radar coord (for GAMMA / ROI_PAC), for simplicity. + stackDict.geometryDict.write2hdf5(): update value of range/azimuthCoord if multilooking is applied, which fix the geocoding bug. + tropo_pyaps3: use LAT/LON_REF1/2/3/4 from displacement file if they are required (for GAMMA products in radar-coord) because they are missing in the geometryRadar.h5 file (at least sometimes) + plot.auto_adjust_colormap_lut_and_disp_limit(): fix potential bug when input data has the same value everywhere. + test/WellsEnvD2T399 for GAMMA - test x/ystep - turn off tropo correction to avoid downloading from ECWMF 21 January 2021, 07:20:09 UTC
d22c5e4 integrate SET correction to sbApp + add `mintpy.solidEarthTides` option [turned OFF by default] + sbApp: integrate solid Earth tides correction via `correct_SET` step: - support `correct_SET` step in get_timeseries_filename() and run() - add run_solid_earth_tides_correction() to call solid_earth_tides.py + test solid Earth tides in radar/geo coords via Fernandina/SanFran example datasets 17 January 2021, 07:51:49 UTC
78cc09a install: include PySolid + setup.py & docs/conda_env.yml: add pip install PySolid + add note to install PySolid for conda/port install + .circleci/config.yml: - dload/setup/compile PySolid - remote $PYAPS_HOME env 17 January 2021, 07:51:49 UTC
770c017 add solid_earth_tides.py + add solid_earth_tides.py to support solid Earth tides calculation and correction for time-series file. + speedup the calculation by >30 times via the step_size argument, by feeding the fortran code the coarse grid for calculation and text file writing, then resize the output into the same shape as the input size. This is due to the fact that the SET varies very slowly in space. Comparison with and without step_size shows a difference in vertical SET component up to 5e-8 m, which is negligible. + support input ts-file in radar coord, via: - prepare_los_geometry() to prepare the incidence angle and heading angle from geometry file in geo/radar coord into geo-coordinates, ready to be used by mintpy.objects.solid module - mintpy.geocode and mintpy.objects.resample module to geocode the geometry and radar-code the calculated tide 17 January 2021, 07:51:49 UTC
2e21a4e update PyAPS install + docs/install: update PyAPS installation notes against the latest yunjunz/PyAPS repo. + smallbaselineApp.cfg: update pyaps3 to PyAPS for consistency + tsview: support file w/o REF_Y/X + LOD: printout template for -h/--help 17 January 2021, 07:51:49 UTC
7936907 tsview: no ref point / auto figsize / linewidth + tsview: - bugfix for --ref-yx option if input file is geocoded: 1) do not overwrite --ref-yx in cmd; 2) update ref_lalo - do not plot reference point if --ref-yx not set in cmd and the native default one is out of the subset box - use default figure size for the image from pp.auto_figure_size() same as view.py - larger time slider without text value on the right - add --marker/--linewidth option for custom point time-series plot - plot() instead of scatter() plot to support --linewidth option. - use "(default: %(default)s)" in the argparse help msg. + utils.plot: - auto_figure_size(): add disp_slider/slider_ratio for tsview. Use it in view.py and update usage in plot_coherence_matrix.py - remove set_shared_y/xlabel() because the used method of `bboxes.inverse_transformed(f.transFigure)` is gonna be deprecated by matplotlib very soon. These two functions are not used anywhere by other mintpy scripts, so no impact is expected. Below is a simpler method from StackOverflow (https://stackoverflow.com/questions/16150819/common-xlabel-ylabel-for-matplotlib-subplots): ```python import matplotlib.pyplot as plt fig, ax = plt.subplots(nrows=3, ncols=3, sharex=True, sharey=True, figsize=(6, 6)) fig.text(0.5, 0.04, 'common X', ha='center') fig.text(0.04, 0.5, 'common Y', va='center', rotation='vertical') ``` + buxgix for utils.utils0.ceil_to_1(). This was not used anywhere in mintpy, thus no impact is expected. + image_stitch.stitch_files(): add no_data_value argument. 12 January 2021, 01:18:52 UTC
360507b fractal: set k1 lower bound to 4*res to prevent all 0s (#503) When regime[0] is small enough, the only `k` item selected is 0, this cascades into all of fraction being 0, and the output being white noise. This PR set k1 lower bound to 4*res in fractal.py to prevent all 0s in fractal_surface_atmos. 07 January 2021, 19:33:15 UTC
a2609cb sbApp abspath and with h5py.File() syntax + use absolute path for all file names/paths in smallbaselineApp.py + use "with h5py.File() as f" syntax to replace "f = h5py.File(); f.close()" for all occurrence in mintpy + remove some unused functions: - timeseries.open_hdf5() - timeseries.layout_hdf5() - singleDataset.close() - unwrap_error_phase_closure.write_hdf5_file_patch() + objects.colors: add GMT_CPT_DIR for customizable GMT cpt file locations - add MINTPY/GMT_CPT_DIR in objects/colors.py for simplicity and flexibility - fix broken display of scientific colormaps in readthedocs + tsview: bugfix 03 January 2021, 07:17:03 UTC
d410e9d setup: comment out openmp/gfortran and add sbApp/(ts)view.py to scripts for bin + comment out openmp and gfortran since they are not available via pip + add smallbaselineApp.py, view.py and tsview.py to scripts so that they will be copied to bin folder for executables available in command line 31 December 2020, 05:52:41 UTC
ba1c42d locate data file via mintpy.__file__ instead of __file__ + use os.path.dirname(mintpy.__file__) to locate data files to replace the previous relative location using __file__ of the python script itself. The old version causes trouble during profiling via scalene (`scalene --reduced-profile ~/tools/MintPy/test/test_smallbaselineApp.py`) + move select_network.py to legacy folder since it's not used anywhere within mintpy workflow. + load_data: make a local copy of global variable datasetName2templateKey so that it can be modified according to different insar processors. + test_smallbaselineApp: - use full path for all files to pass profiling with scalene - return back to execute script instead of call main() for a complete clean run - use native argparse list argument for --dset option for simplicity + utils.readfile.read_envi_hdr(): fix a bug when there is no "WGS84,units=Degrees" string after the last number in th .hdr file 31 December 2020, 05:52:41 UTC
74d712a move progressBar to objects/progress.py + move progressBar class definition from utils/ptime.py to objects/progress.py for simplicity. Old usage of ptime.progressBar stays the same. + smallbaselineApp.py -H now printout syntax highlighed text if rich module is installed. 31 December 2020, 05:52:41 UTC
334d984 view: auto discretize colormap if input data has <=20 unique values + utils.plot: add auto_adjust_colormap_lut_and_disp_limit() to calc 'cmap_lut' and 'vlim' given a 2/3D matrix + utils.plot.plot_colorbar(): - do not show the default unit of 1 in the colorbar label - use custom tick number if discrete colormap has <= 5 levels + view/tsview/save_kmz: update cmap_lut/vlim while updating dlim/vlim + view: move the ColormapExt() call into plot_slice/figure() from pp.check_colormap_input() so that it can use the updated cmap_lut for plotting. 28 December 2020, 09:08:26 UTC
faebd3f multilook: add method='median' with bugfix * multilook.multilook_data(): + add method='median' support + bugfix for 3D mat with method='nearest' * tsview support of unwrap error time-series display + utils.plot.scale_data2disp_unit(): support input data matrix in integer, besides float32 + view.plot_slice(): use inps.cmap_lut for more controlled cmap for unwrap error matrix with limited number of unique values + tsview: support unwrap error time-series display - allow ref_idx as None to support input file without REF_DATE - colormap with limited number of colors for better display - skip re-referencing in space if not ref_yx is specified in command line - keep all-zero values if UNIT == 'cycle' * ts2vel: add --ref-yx/date for ERA5.h5 + timeseries2velocity: add --ref-yx/date options to be able to customize reference info, which can be useful for input TS file without reference info, e.g. ERA5.h5 + smallbaselineApp: add --ref-yx/date in the timeseries2velocity.main() call for tropo time-series file, for a meaningful assessment of tropo impact on velocity 28 December 2020, 09:08:26 UTC
3935fb4 cpu_count(): multiprocessing --> os + replace multiprocessing.cpu_count() with os.cpu_count() to eliminate the import of multiprocessing module + defaults/sbApp.cfg: update *.lalo options comments + simulations/decorrelation: move cal_coh4phase_pdf_bias to unwrap_error for simple management + lookup_geo2radar: fix import syntax 28 December 2020, 09:08:26 UTC
a2032f3 delete unused mintpy/dev folder + move save_ifg_list4giant.py from dev to legacy folder + move run_isce_stack.py, split_job.py and unwrap_error_hybrid.py out of mintpy repo (to private dev/bin repo) for simplicity + delete obsolete process_isce_stack*.py 28 December 2020, 09:08:26 UTC
28b84ca tsview.py to handle case of three data points in the stack 28 December 2020, 05:05:38 UTC
bb2ab3b prep_aria: load amplitude with --amp-stack-name as optional + add --amp-stack-name option to be able to load interferogram amplitude data into ifgramStack file as a 3D dataset named "magnitude", in the same level as "unwrapPhase". + add multilook support for amp stack, same as other datasets + add more printout msg writefile.layout_hdf5: copy input compression arg to ds_comp so that it can be changed temporarily for connectComponent without affecting datasets coming afterwards. 17 December 2020, 01:52:58 UTC
28635d1 resample: separate open/prepare() and set max_memory=0 + separate open/prepare() to support matrix/dict IO without src file on the disk for a more flexible API. + set max_memory=0 by default to disable block-by-block IO + add comoments for explicit example usage of resample object + bugs fix + utils/attribute: support both explicit list of simple arguments and one complex res_obj argument for update_attribute4geo2radar/radar2geo(). update geocode.py accordingly. 14 December 2020, 03:13:04 UTC
6938754 stackDict: relax dataset name constraint for HDF5 file to support reading waterMask from waterMask.h5 file with /mask dataset 14 December 2020, 03:13:04 UTC
2f3e6d8 circle CI: add SanFranSenDT42 v0.1 for testing + circle ci: - install gdal to be able to run prep_aria.py - add SanFranSenDT42 for testing for aria + test: - configs/SanFranSenDT42: add mintpy.load.x/ystep for testing - test_sbApp: switch SanFranSenDT42 from v1 to v0 for smaller size and faster download / uncompression 13 December 2020, 21:16:04 UTC
8cda913 add mintpy.load.x/ystep for multilooking while loading data + add `mintpy.load.x/ystep` with default value of 1 for smallbaselineApp.py + multilook.multilook_data(): - add default lks_y/x value of 1 - return directly if no multilook number is specified + load_data.py - use iDict to replace inpsDict for simplicity - read mintpy.load.x/ystep and pass them to ifgramStackDict/geometryDict object + objects/stackDict.py: support x/ystep in all write2hdf5() + prep_aria: support multilook via mintpy.load.x/ystep 13 December 2020, 20:10:15 UTC
353450a add utils/attribute.py for update_attribute4*() + move the following sub-functions into a new sub-module utils.attribute: - utils.utils0.subset_attribute() --> update_attribute4subset() - multilook.multilook_attribute() --> update_attribute4multilook() - geocode.metadata_radar2geo() --> update_attribute4radar2geo() - geocode.metadata_geo2radar() --> update_attribute4geo2radar() + update docs/api/module_hierarchy.md for utils.attribute/arg_group 13 December 2020, 20:10:15 UTC
537f726 docs/hdfeos5: add metadata section + docs/hdfeos5.md: add metadata session in require / recommend / auto-grab sub-section, to facilitate manual specification + save_hdfeos5: change the followingg metadataa - remove "frame" from UNAVCO definition completely and use first/last_frame only, for simplicity - uncomment processing_software - hardwire processing_type = LOS_TIMESERIES. This can be changed in the future if velocity/interferogram capabiligy is added. - hardwire post_processing_method = MintPy. This can be changed in the future if the script supports products from other softwares. + save_hdfeos5: add date-by-date IO to save memory / handle big data 13 December 2020, 20:10:15 UTC
11bc397 dem_error: add dask support for parallel computing + change default mintpy.compute.cluster value from no to none, to be consistent with utils.arg_group.add_parallel_argument() + dem_error: add parallel computing support via dask 13 December 2020, 20:10:15 UTC
41fe446 adjust block size calc + use max_memory to calc block size in temp_avg/pha_closure/ifg_inv + round block step to the nearest 10 + add used time info for load_data and plot_sbApp + Update mkdocs.yml 13 December 2020, 20:10:15 UTC
60a08fe block-by-block IO for geocode.py * geocode.py + add --ram option from utils.arg_group.py + merge -y/x into --lalo-step for consistency with resample obj + more explicit checking / error message for --lalo-step option, since it's only customizable if radar2geo AND lut in radar-coord + block-by-block IO for both HDF5 and binary file, the latter is bbb in read only * objects/resample.py + move all configurations into __init__() to simplify the run_resample() + consistent member variables across all scenarios (radar2geo/geo2radar, geo/radar-coord lookup table, scipy/pyresmaple), including: - lalo_step - SNWE - length/width - src/dest_box_list - src/dest_def_list (for pyresample) - src/dest_pts and interp_mask (for scipy) + add get_num_box() + prepare_geometry_definition_radar(): - add block-by-block geometry preparation for radar2geo - add custom SNWE support for geo2radar + prepare_geometry_definition_geo() - add custom SNWE support for radar2geo 13 December 2020, 20:10:15 UTC
7a0d18f tmpl opt: memorySize --> maxMemory + rename mintpy.compute.memorySize to mintpy.compute.maxMemory for a more intuitive name + change default max memory from 2 GB to 4 GB + dem_error: add dask parallel option in prepare for dask support + sbApp(_auto).cfg: merge mintpy.geocode.latStep and mintpy.geocode.lonStep into one as mintpy.geocode.laloStep for consistency with mintpy.objects.resample object 13 December 2020, 20:10:15 UTC
6b5bbd0 geocode: pyresample as default for geo/radar lookup table files + use pyresample as the default softwre for geocoding dataset produced by isce (lut in radar-coord) and gamma (lut in geo-coord) + comparison of geocodeing results between pyresample / scipy on Wells EQ dataset gives identical results on 99.1% of all valid pixels, thus change the default geocoding software from scipy to pyresample for: 1. consistency with config for other processors 2. flexibility, i.e. customized SNWE and lat/lon step. 3. efficiency, pyresample supports 3D matrix, thus, is more efficient. + consistent internal definition + clean up the following concepts/variables in resample objects: - always to coordinates at pixel center for interpolation - SNWE indicates bounding box at pixel outer boundary, consistent with Y/X_FIRST definition, unless noted in the adjacent comments. 13 December 2020, 20:10:15 UTC
e23cf67 consistent main(sys.argv[1:]) usage For executable scripts, use ``` if __name == '__main__': main(sys.argv[1:]) ``` instead of ``` if __name == '__main__': main() ``` because the latter return error in interactive python when cmd_line_parsee() is not called in the main body of main(), such as the case in tsview.py, therefore the former is more generic and useful. + defaults/auto_path: use watermask.msk for ARIA 13 December 2020, 20:10:15 UTC
a96d1d4 readfile.read(x/ystep): fix size discrepency & save memory via loop for 3D dset + utils.readfile: - read_hdf5_file/binary(): fix the size discrepency when x/ystep > 1 to be consistent with multilooking output size - read_hdf5_file(): use for loop when ystep * xstep > 1 for 3D dataset to save memory + multilook.multilook_dataset(): add method arg to support/swtich btw. average and nearest + view: expand multilook_num to all multiple subplots scenarios to save memory because the readfile.read(x/ystep) now won't distort data (due to nearest sampling instead of previous averaging) 13 December 2020, 20:10:15 UTC
6014874 dem_error: do not write step_model for simplicity + dem_error: do not write step_model into file/disk because: 1. the step function estimation is now supported via timeseries2velocity.h5 and the latter has more powerful functionality 2. the step model HFD5 file from dem_error.py is different from the one from ts2vel.h5 and the latter is preferred for its simplicity. + drop the common operations support in the following scripts of timeseriesStepModel.h5 file, which sometimes has one date in the time dimension: - geocode.py - mask.py - multilook.py - subset.py + dem_error: replace split2boxes with cluster.split_box2sub_boxes() test_sbApp: plot velocity alone for snap and aria also 13 December 2020, 20:10:15 UTC
dac984c block-by-block IO for ts2vel + add --ram/--memory option for max memory usage setup + split run_timeseries2time_func() into: - read_inps2model() to get model dict and print key model info - layout_hdf5() to create HDF5 file with time func structure - write_hdf5_block() to write the block of time func + objects.cluster.split_box2sub_boxes: refactor 13 December 2020, 20:10:15 UTC
21adc44 ts2vel: integrate cpx time func with bootstrap + fully integrate the bootstrap method with complex time func support 13 December 2020, 20:10:15 UTC
a872e5b block-by-block IO for reference_date.py + add --ram/--memory option for custom memory usage with default value of 2GB and template reading support + import cluster.split_box2sub_boxes() for patch spliting + block-by-block processing using writefile.layout_hdf5/write_hdf5_block() 13 December 2020, 20:10:15 UTC
0b42c90 block-by-block IO for tropo_pyaps3.py + block-by-block IO for tropo_pyaps3 using writefile.layout_hdf5() and writefile.write_hdf5_block() + add run_or_skip() within calculate_delay_timeseries() for auto-skip 13 December 2020, 20:10:15 UTC
177d993 bugfix in los.rdr FILE_TYPE identificatioon since both isce and snap could produce .hdr metadata file. This is exposed by the order change of metadata files in previous commit. 13 December 2020, 02:14:56 UTC
9995617 readfile: use unique meta filenames instead of basename + extension comb + utils.readfile.read_attribute(): given the data file, provide the potential metadata filenames in more explicit form directly, instead of using basesname and extension combination, for less error prone meta file detectioon, such as grabbing filt_fine.unw.rsc for filt_fine.unw.conncomp + update prep_snap and SNAP example dataset: - prep_snap: use *.img.rsc instead of *.rsc as the output metadata file, to be consistent with the above change in utils.readfile.read_attribute(). - .circleci/config.yml: add SNAP example dataset for routine testing via test_smallbaselineApp.py - use WCapeSenAT29 version 0.2 from zenodo, which has 1) the latest template file from mintpy repo and 2) new subset/reference point and 3) updated ERA5.h5 for the new subset to avoid downloading ERA5 data on the fly during testing. 12 December 2020, 04:15:21 UTC
b29415f gps: add plot_insar_vs_gps_scatter() and bugfix + objects/gps.get_gps_los_velocity(): bugfix due to the change of linear velocity column number in the design matrix from 0 to 1 in #425 + objects/insar_vs_gps: add plot_insar_vs_gps_scatter() from notebook example from unavco 2020 short course for easy re-use 12 December 2020, 04:15:21 UTC
bede036 ifgramStack.update_drop_ifgram(): add update check besides the update check in modify_network.py 12 December 2020, 04:15:21 UTC
e531361 move unused scripts to legacy/dev folder + move the following unused script to the legacy folder because their funcationality is ready to use as API functions already: - correlation_with_dem.py - incidence_angle.py - load_hdf5.py - range_distance.py + move save_ifg_list4giant.py into dev folder because it not needed for most users. + more comments on isce_utils.extract_alosStack_metadata() + update docs/api/attribute.md 12 December 2020, 04:15:21 UTC
a7e5eaa bring back tropo_gacos.py for geo/radar coord (#483) + re-write `tropo_gacos.py` to work with the latest mintpy version with the following changes: - add block-by-block IO for big data friendly - split get_delay() to get_delay_geo() and get_delay_radar() - improved auto_skip checking, path/file exsitence and documentations + smallbaselineApp: add GACOS support via `mintpy.troposphericDelay.method = gacos` and `mintpy.troposphericDelay.gacosDir` + diff.py: fix a bug on reference point/date when splitting into boxes + subset: fix a bug in subset_dataset() for binary file input + utils0.get_lat_lon(): add dimension arg to support RGI coordinate preparation + delete obsolete legacy/tropo_gacos.py 02 December 2020, 04:08:48 UTC
dfc7556 read_attribute: move .aux.xml to the least priority because it does not contain the geo coordinates info, such as the SRTM_3arcsec.dem file provided by ariaTSsetup.py. This led to a bug while calling view.py with DEM background plot. 27 November 2020, 18:38:10 UTC
032a6fd fix link for input_files/README + Update load_data.py 27 November 2020, 18:38:10 UTC
438a478 load magnitude via mintpy.load.magFile 25 November 2020, 05:08:22 UTC
28ec3a0 conncomp: split get_large_label() into label_conn_comp/boundary() objects/conncomp: + split connectComponent.get_large_label() into label_conn_comp() and label_boundary() for simpler IO interface + move the above two function out of the connectComponent() objects for easy import unwrap_error_phase_closure: update conncomp usage accordingly test/test_sbApp: test unwrap_error_phase_closure in Fernandina dataset 25 November 2020, 04:02:19 UTC
b422c55 rename match.py to image_stich.py with refactoring + refactor the stitching function to stitch all files together without the need of writing the intermediate files, when there are >2 input files, via the following changes: - add stitch_two_matrices() for developer friendly numpy matrices IO - add stitch_files() for user friendly self contained stitching - set --output as required for more restrict but cleaner code - add plot_stitch() and run if apply_offset is ON only. + disable --manual for now, as it's not used much, as far as I know, for easy maintainmence. + remove unused nearest() 25 November 2020, 04:02:19 UTC
3154405 writefile.write: consider diff. format for .cor/hgt between isce and roipac 25 November 2020, 04:02:19 UTC
8096888 speedup h5 dset IO by disabling large list indexing in ifgramStack.temp_avg/read() and use numpy indexing instead 25 November 2020, 03:25:55 UTC
d9d55a2 block-by-block IO for multilook.py + add -m / --method option to choose among average / nearest + add lks_x/y == 1 check + multilook_data(): ensure in/out matrix in the same data type + multilook_file(): - support nearest downsampling using readfile.read(x/ystep) option - add block-by-block IO for HDF5 files + apply block-by-block IO for multilook.py for efficient memory usage + plot.read_pts2inps(): bug fix when no lookup file is availble + utils1.get_center_lat_lon(): support file in geo coordinates + writefile.write_isce_xml(): add example usage in the comment 25 November 2020, 03:25:55 UTC
9025ed8 ref_point: move update mode check before valid value check to avoid un-necessary temporal average computation. 25 November 2020, 03:25:55 UTC
e65811a downsample big dataset for deramp If the number of valid pixels > 1m, apply a uniform sampling for a faster ramp estimation and for less memory usage 25 November 2020, 03:25:55 UTC
c8925a5 block-by-block IO for diff.py + objects/cluster: move split_box2sub_boxes() out of the DaskCluster object for easy import; add print_msg. + diff: add block-by-block IO for time-series files 25 November 2020, 03:25:55 UTC
4069281 block-by-block IO for subset.py + utils/writefile.layout_hdf5(): support output file in different size in space from the reference file + subset.py: - add subset_dataset() to get the subseted data from input file - block-by-block IO for HDF5 file with 3D dataset, to significantly reduce the memory usage. Testing on laptop (16GB memory) for ifgramStack.h5 file with 140GB works very smooth. 25 November 2020, 03:25:55 UTC
5b13103 block-by-block for dem_error.py + add/modify split2boxes() from ifgram_inversion.py + move key msg and design_matrix into get_design_matrix4defo() for cleaner code + read_geometry(): explicit input arguments + box + add correct_dem_error_patch() from the code of correct_dem_error() + use writefile.layout_hdf() and writefile.wirte_hdf5_block() for block-by-block IO, like ifg_inv.py + add --memory option with support of mintpy.compute.memorySize + run_or_skip(): check file size to detect partially written file 25 November 2020, 03:25:55 UTC
866f91b add utils/arg_group.py for the common args + move the commonly shared groups of arguments parsing functions from utils/plot.py into a new sub-module utils/arg_group.py + move the parallel computing related arguments into utils/arg_group.add_parallel_argument() + adjust usages across the whole mintpy package 25 November 2020, 03:25:55 UTC
013f05d Added --no-offset option to match.py (#477) * Added --no-offset option to match.py for files that do not need value adjustment, i.e. coherence. 17 November 2020, 06:17:04 UTC
271bbae Updated mask.py to work with coherence files and others 17 November 2020, 06:12:07 UTC
89c3dcf rollback to h5py native fancy indexing + docs/conda: - add h5py<3 constrain to avoid the mix indexing bug from h5py v3+ - add python<3.9 constrain to avoid dask imcompatiability - add xarray and zarr for the new pyresample. - re-organize with notes for pyaps and pyresample separated from mintpy navtive code + update docs/conda_env.yml, docs/requirements.txt and setup.py to be consistent with the latest conda.txt + obj/stack.ifgramStack.temporal_average(): auto row_step - auto calc for row_step if input value is -1 (default) - fix ref_yx out of coverage bug - rollback to h5py native fancy indexing - check dataset shape before calling np.squeeze() + utils/readfile: - rollback to h5py native fancy indexing - check dataset shape before calling np.squeeze() + ts2vel: - remove sklearn from conda.txt and add ImportError for sklearn - del ts_data right after the last usage to save memory + test_sbApp: update for geocoded velocity png file 15 November 2020, 02:27:47 UTC
d05ad1e aria support in load_data with autoPath + load_data: - move the call of prep_aria.main() from smallbaselineApp.py into load_data.py, so that prep_aria could appears to be the same as other prep_*.py in the user side, a.k.a. "load_data.py -t template file" works for ARIA as well. - support 'mintpy.load.autoPath' for aria - ignore failed prep_aria and continue load_data, becuase 1) the ARIA product could have been loaded into mintpy and deleted already and 2) the potential missing loaded HDF5 files will be reported laterward in smallbaselineApp.py, so it's okay to ignore it here. + utils1.update_template_file(): fix a bug when the option name contains the option value, i.e. mintpy.load.autoPath = auto. + add SanFranSenDT42.txt to the test_smallbaselinApp.py + mintpy/sh/plot_smallbaselineApp: plot ts_demErr.h5 13 November 2020, 23:57:40 UTC
dcf278e example dataset for ARIA on San Francisco + update mintpy/data/input_files/SanFranSenDT42.txt: enable unwrapping error correction with bridging and disable network modification via interferogram index. + docs/demo_dataset: add ARIA section with zenodo link using the San Francisco dataset prepared by Heresh Fattahi using ARIA-tools + view.prep_slice(): update example comments regarding the cartopy projection 13 November 2020, 23:57:40 UTC
2c7bbbe add setup.py for pip install and pypi distribution + add pyproject.toml to use setuptools and wheel for build system + add MANIFEST.in + add setup.py with the following updates: - auto grabbed version, (long_)description - dependencies info - data files + move /sh to /mintpy/sh for easy registration with setup.py + move /docs/examples/input_files into /mintpy/data/input_files + move /mintpy/resources into /mintpy/data + version: add release_description to replace the previous description; and add new description. + bump python min version in doc/setup from 3.5 to 3.6 to be consistent with the docs/conda.txt 09 November 2020, 23:39:26 UTC
e702c67 example dataset & template file for SNAP + add an example input dataset processed with SNAP provided by Andre Theron + add mintpy/data/input_files/WCapSenAT29.txt + add SNAP dataset to test_smallbaselineApp.py and test/configs/WCapSenAT29.txt + view: do not update disp_min/max if --wrap is enabled for multiple subplots, to be consistent with the one for single subplot. 09 November 2020, 04:55:31 UTC
7ab3732 bugfix for loading SNAP DEM with subset enabled + load_data: fix a bug of subsetting geometry file in geo-coord if input dataset is in geo-coord as well. + stackDict: acknowledge the -32768 no data for DEM 09 November 2020, 04:55:31 UTC
c2145b0 fix bool array index bug for h5 dataset + readfile & stack: fix a bug of fancy indexing with numpy boolean array in h5py dataset (https://docs.h5py.org/en/stable/high/dataset.html#fancy-indexing), which is used in utils/readfile.py and objects/stack.py. This seems to be a bug in h5py side because it was working before. + prep_aria: fix a bug in gdal*.ReadAsArray() from numpy array int64 type to native python int16 type 08 November 2020, 07:54:02 UTC
1caf1ac ifg_recon: bugfix + stack.ifgramStack.get_design_matrix4timeseries(): add refDate='no' to disable the column redunction due to temporal referencing. + ifgram_reconstruction.py: - fix the bug of design matrix due to the reference date - update example usage 03 November 2020, 02:06:04 UTC
1352962 ifg_inv for offset: consider A/RLOOKS for az/rg_pixel_size + ifgram_inversion: update az/rg_pixel_size to the resolution before multilooking for meaningful value + add utils.utils0.vtec2range_delay() to convert the zenith TEC to the predicted range delay in SAR, based on Chen and Zebker (2012) + add utils.isce_utils.get_IPF() for Sentinel-1 data + view: show lat/lon in the status bar for file in radar coord, if the geometryRadar.h5 file is in the default location. + tsview: make REF_Y/X optional to support off time-series + update HDF-EOS5 usage in save_hdfeos5.py and docs/hdfeos5.md + remove ECMWF from smallbaselineApp.cfg since it's not been updated anymore and is replaced by ERA-5; and update comments in tropo_pyaps(3).py 03 November 2020, 02:06:04 UTC
139793b dir structure of ISCE/alosStack (#465) * docs/dir_structure: directory structure of ISCE/alosStack added * docs/examples/input_files: Rename template file of ISCE/alosStack 23 October 2020, 23:44:42 UTC
5840680 support for alosStack in ISCE (#464) Adds support for ALOS-2 (and ALOS-4 in the future) stack processed with contrib/stack/alosStack in ISCE-2. The data loading part is similar to the product from topsStack and stripmapStack, whose sub-type is identified automatically based on the input metadata file via mintpy.load.metaFile. Detailed changes are as follows: + utils/isce_utils.py: - modify get_processor() to detect alosStack based on f1_*/*.frame.xml file. - add the following new functions to extract metadata for alosStack: - extract_alosStack_metadata() - alos2_acquisition_mode() - extract_image_size_alosStack() - load_track() - read_alosStack_baseline() + prep_isce.py: - adjust prepare_stack() and prepare_geometry() for alosStack. - cmd_line_parse(): 1) use argparse required argument to replace the checking in cmd_line_parse(); 2) translate wildcard in input metaFile + add docs/examples/input_files/NCalAlos2Scan.txt as the example template file for loading alosStack products + objects/stackDict.geometry.write2hdf5(): convert waterBody to waterMask while loading data + utils/readfile.read_binary_file(): use band2 by default for *.cor file with 2 bands to support reading complex coherence files generated by isce-2. 22 October 2020, 05:24:48 UTC
b164b1b ifg_inv: NaN as no-data value for offset/phase + ifgram_inversion; - use np.nan value as the no-data value for observations (phase or offset) to replace the previous version of using both (zero and nan), by: - setting masked out pixel values as np.nan instead of 0 and - setting zero phase (not offset) value to nan. - The above changes results in the following improvements: - simplified checking of masks and pixels with no-data values in some (not all) interferograms. - remove `skip_zero_value` from estimate_timeseries() due to the above change - skip pixels with zero avgSpatialSnr values for offset - more comments + objects/stack.ifgramStack.temporal_average(): - split the big 3D matrix in row direction instead of ifgram/time dimension. - the above change allows to use nanmean to replace manual mean calculation, to better handle the nan value, for improved avgSpatialSnr.h5 calculation. + adjust printout msg in writefile.layout_hdf5() and view.py 21 October 2020, 20:47:50 UTC
211bfce view: break date1_date2 into date1\ndate2 for number subplots in (20, 50] for clearer display 21 October 2020, 20:47:50 UTC
cda80db prep_aria: update metadata after subseting 16 October 2020, 03:05:21 UTC
daf64cf prep_gamma: support *-* folder style while searching for *.par files to grab LAT/LONO_REF1/2/3/4 metadata 14 October 2020, 04:24:28 UTC
8da3c86 add pip to docs/conda.txt + add pip to conda.txt since it's used in the installation of pykml + remove ${CONDA_PREFIX} while calling conda since it's now removed in the custom environment from the latest version of conda distribution 14 October 2020, 04:24:28 UTC
004a1b5 prep_aria: bugfix when no template file input + bugfix when there is no template file input + update example usage 12 October 2020, 04:33:58 UTC
1dfadb1 prep_aria: support subset / update mode / compression + writefile.layout_hdf(): add compression argument + smallbaselineApp: do not forcely pass --update to prep_aria.py + prep_aria: - add read_subset_box() to support loading part of ARIA products in 'load_data' step using `mintpy.subset.yx/lalo` options - add --compression option - more robust run_or_skip() considering the corrupted h5 files 12 October 2020, 00:31:34 UTC
0cd5689 move resources from docs to mintpy + move resources folder from docs to mintpy as they are used in the code, to make mintpy folder independent from docs folder + rename docs/resources/colormaps/README.md to docs/api/colormaps.md and update its usage in mkdocs.yml + update the location of resources files in save_kmz_timeseries.py, objects/colors.py + remove unused test/unwrap_error_bridging in .gitignore 12 October 2020, 00:31:34 UTC
6d7bf60 net.select_pairs_*(): use date_format instead of date12_format for simplicity 12 October 2020, 00:31:34 UTC
5437feb asc_desc2horz_vert: add --dset option add --dset option to be able to customize the dataset to read/use to asc/desc 2 horz/vert conversion, for input file with more than one dataset. 07 October 2020, 16:34:20 UTC
47d00d1 bugfix on the prep_fringe.py for the updated isce_utils.extract_geometry_metadata() args (#455) There is a bug on line 140 of the prep_fringe.py causing the code to stop working during the extraction of the metadata on the reference xml files. 06 October 2020, 18:15:36 UTC
back to top