| ea7fc46 | Heresh Fattahi | 03 June 2017, 21:41:41 UTC | Merge pull request #4 from yunjunz/master PySAR v1.2 | 03 June 2017, 21:41:41 UTC |
| 473315c | Zhang Yunjun | 03 June 2017, 21:40:16 UTC | Update README.md | 03 June 2017, 21:40:16 UTC |
| b53a287 | Zhang Yunjun | 03 June 2017, 21:02:39 UTC | Update README.md | 03 June 2017, 21:02:39 UTC |
| f545524 | Yunjun Zhang | 02 June 2017, 05:11:37 UTC | bug fixed for loading | 02 June 2017, 05:11:37 UTC |
| c1460f6 | Yunjun Zhang | 02 June 2017, 04:38:53 UTC | bug fixed | 02 June 2017, 04:38:53 UTC |
| fcae7e1 | Yunjun Zhang | 01 June 2017, 18:11:57 UTC | use one bperp by default, for faster processing | 01 June 2017, 18:11:57 UTC |
| 2e363f3 | yunjunz | 01 June 2017, 18:09:44 UTC | add DEM in radar coord loading for Gamma product | 01 June 2017, 18:09:44 UTC |
| 4a6d245 | Yunjun Zhang | 01 June 2017, 05:43:05 UTC | load Gamma unw/cor product into PySAR 1. add new script prep_gamma.py to prepare a .rsc file for Gamma product. It contains the following functions: a. def attribute_gamma2roipac() to convert exsiting GAMMA attribute to ROIPAC attribute b. def get_perp_baseline() to calculate perpendicular baseline based on master/slave par file and interferogram off file, using Gamma command base_orbit and base_perp c. def get_lalo_ref() to calculate lat/lon of interferogram's four corners based on master par file, using Gamma command SLC_corners d. def extract_attribute() to extract the whole attribute based on interferogram file name 2. add new template optioin pysar.insarProcessor into full template list, for pysarApp.py and load_data.py 3. _readfile.py: add byteorder option into read_real_float32() to support Gamma product 4. load_data.py: adjust to be able to load Gamma unw and cor files into PySAR | 01 June 2017, 05:43:05 UTC |
| ee66e89 | Yunjun Zhang | 31 May 2017, 22:32:30 UTC | add function to read Gamma interferogram product for PySAR | 31 May 2017, 22:32:30 UTC |
| 1d85d32 | Falk Amelung | 30 May 2017, 21:47:41 UTC | test test for falk | 30 May 2017, 21:47:41 UTC |
| 1f15551 | Yunjun Zhang | 30 May 2017, 20:50:25 UTC | more plot option for network _network.py: 1. fix bug with MST date12 list exchanged master/slave date 2. add coh_thres to specify color jump/cut in network plot with coherence 3. bug fixed with coloring in plot_network() 4. add disp_title option diff.py: support differencing of two timeseries files with different reference date and reference pixel plot_network.py: enable --figsize option save_kml.py: do not read 'X/Y_UNIT', make it more robust | 30 May 2017, 20:50:25 UTC |
| 7095751 | Yunjun Zhang | 29 May 2017, 04:06:20 UTC | abspath for unavco_attribute.txt file in pysarApp.py | 29 May 2017, 04:06:20 UTC |
| f84d3d6 | Yunjun Zhang | 29 May 2017, 00:27:32 UTC | dem_error: add step function and exclude date option 1. add pysar.topoError.stepFuncDate option to specify the date of step jump, i.e. date of earthquake or volcanic eruption, to the temporal deformation model; more robust for complicated deformation dataset 2. add pysar.topoError.excludeDate option to exclude date(s) for error estimation, which usually are streamely strong ionosphere 3. add --template option to dem_error.py for input argument, and adjust corresponding code in pysarApp.py | 29 May 2017, 00:27:32 UTC |
| 39efbd4 | Yunjun Zhang | 28 May 2017, 05:24:01 UTC | bug fixed when using pysar.subset option | 28 May 2017, 05:24:01 UTC |
| 142026d | Yunjun Zhang | 28 May 2017, 04:07:12 UTC | bug fixed for auto loading for Miami user fix bug in load_data.py introduced by the full template txt file in pysarApp.py do not tight subset DEM in geo coord if it's not exists in pysarApp.py | 28 May 2017, 04:07:12 UTC |
| 2c34e31 | Yunjun Zhang | 28 May 2017, 02:07:56 UTC | adjust to date12_list info and exclude date option for tsviewer.py add list_ifgram2date12() in _datetime.py to convert ifgram list into date12_list, and adjust corresponding script, mainly the one use this info to shown suffix in progress bar consider difference reference date for timeseries file input of diff.py check corrupted hdf5 file in load_data.py more robust tight croping in pysarApp.py add --exclude option in tsviewer.py to show excluded date as gray dot bug fixed for view.py when input exclude date for timeseries file | 28 May 2017, 02:07:56 UTC |
| 3a17162 | Yunjun Zhang | 25 May 2017, 18:34:06 UTC | add pysarApp.py --load-data/--subset-data/--modify-network options add pysarApp.py --subset-data option to create a subset dataset of all required files easily, so that the newly created SUBSET_X*Y* folder could be shared and re-processed. add ROIPAC file support for add_attribute.py, and if new attribute's value is None, drop this item from attribute dictionary reverse back the multilooking for geomap*.trans file, do not divide its value with number of multilooks | 25 May 2017, 18:34:06 UTC |
| 248ff14 | Yunjun Zhang | 23 May 2017, 22:23:06 UTC | use residualRms instead of residualStd to assess timeseries phase error correction performance 1. rename timeseries_std.py to timeseries_rms.py and calculate timeseries_ECMWF_demErrInvResid_quadratic.h5's RMS instead of STD. Test shows that both of them are the same for deramped timeseries, but RMS are more sentative than STD if timeseries is not deramped, and RMS is more clear in definition. Code is updated in pysarApp.py, _pysar_utilities.py, timeseries_rms.py 2. bug fixed for seed_data.py while using pysar.reference.lalo option with template input 3. use default maskTempCoh.h5 mask in view.py for velocity, timeseries, rmse and interferograms. | 23 May 2017, 22:23:06 UTC |
| 80b27f0 | Yunjun Zhang | 23 May 2017, 18:37:16 UTC | bug fixed while plotting network info without coherene input 1. file permission adjustment for _gmt.py and insarmaps_query.py 2. do not plot CoherenceMatrix and CoherenceHistory if no coherence info input 3. tropcor_pyaps.py: get ride of --timeseries option name for timeseries input | 23 May 2017, 18:37:16 UTC |
| 29a4e97 | Yunjun Zhang | 22 May 2017, 23:40:12 UTC | add MST constrain to coherence-based network modification _network.py 1. add coherence_matrix() to calculate 2D coherence matrix given date12_list and coherence_list 2. add threshold_coherence_based_mst() to return a minimum spanning tree of network of interferograms with inverse of coherence as weight, given date12_list and coherence_list 3. add plot_coherence_matrix() and plot_coherence_history() for plotting 4. modify plot_network() to find color jump based on input date12_list to keep and drop, so that all dropped interferograms shown as red dash line and all kept interferograms shown as solid blue line; change min coherence to display from 0.4 to 0.2 modify_network.py: add MST constrain if using coherence-based network modification, so that network will not become non-fully connected, because of coherence thresholding. plot_network.py: plot two new figures - CoherenceMatrix.pdf and CoherenceHistory.pdf pysarApp.py: add comment about coherence-based network with MST constrain. timeseries_std.py: adjust bar width using common mode of sampling length in time | 22 May 2017, 23:40:12 UTC |
| 9cc3b39 | Yunjun Zhang | 21 May 2017, 22:51:02 UTC | bug fixed for multilooking mapping transformation file _pysar_utilities.py: add multiple input files - list - support for update_file(), and adjust corresponding function usage in _pysar_utilities.py and pysarApp.py diff.py: do not exit if all groups of file1 exists within file2 for timeseries file type multilook.py: 1. fix bug while multi looking mapping transformation file 2. add print_message option for multilook_attribute(), so that no message print while multi looking multi_group files | 21 May 2017, 22:51:02 UTC |
| fad711c | Yunjun Zhang | 21 May 2017, 19:48:47 UTC | add full template for pysarApp.py Introduce a default, full template file - pysarApp_template.txt - to pysarApp.py. With parameter setting for all steps run within pysarApp.py. For the first time running, pysarApp.py will generate this txt file with all default settings unless it already exists. If custom template, like KujuAlosAT422F650.template is input, script will read the value from it and update it to the default template file. use pysarApp.py -g to generate/check default template and exit, similar to run -g for Doris. _pysar_utilities.py 1. add function update_template_file() to write template new value into an organized existing template file 2. add function get_spatial_average() to get spatial average list of input file. And change output file extension from .list to .txt _readfile.py: modify read_template() to not read option with empty value load_data.py: adjust script for full template list and the auto value modify_network.py: 1. adjust script for full template list and auto value 2. add —reset option to select all group within input file - drop_ifgram=no 3. support maskAoi.yx/lalo to change area of pixels for average coherence calculation pysarApp.py add full template list with -g option to stop after template reading reference_epoch.py: do not write date to reference_date.txt file and support full template list with auto value seed_data.py: support full template list | 21 May 2017, 19:48:47 UTC |
| 1dc5c76 | Yunjun Zhang | 17 May 2017, 16:15:25 UTC | support pyaps download with bl_list.txt tropcor_pyaps.py: add support to read bl_list.txt file to get list of date for weather re-analysis data downloading use —exclude by default in select_network.py and view.py | 17 May 2017, 16:15:25 UTC |
| ec4ec35 | Yunjun Zhang | 16 May 2017, 18:48:38 UTC | introduce residual standard deviation __init__.py: add figsize_single_max variable for max size _datetime.py: 1. move progress_bar class from _pysar_utilities.py to _datetime.py; and adjust corresponding scripts 2. return start and end date in auto_adjust_xaxis_date() _network.py: add ymin/max argument in auto_adjust_yaxis() _pysar_utilities.py 1. add function timeseries_std() and timeseries_coherence() to calculate standard deviation and phase coherence in space for each epoch of timeseries 2. add function get_residual_std() to calculate deramped phase residual standard deviation and write it to text file 3. check is inFile exists in update_file() dem_error.py: output phase inversion residual to a new timeseries file pysarApp.py: 1. constrain input subset box in radar coord within the data coverage 2. add new pysar options in the template pysar.residualStd.maskFile - mask file used for residual std calculation pysar.residualStd.ramp - ramp type pysar.residualStd.minStd - threshold for turbulence detection pysar.velocity.excludeDate - customized text file for excluded dates in linear velocity estimation 3. plot all command line even they are not ran 4. use tropospheric delay file if exists instead of call pyaps again 5. calculate linear velocity from tropospheric delay file 6. add new step: calculate timeseries residual standard deviation 7. add new step: automatic referencing in time using min residual std reference_epoch.py 1. use argparse to replace get opt for input argument parsing 2. add —ref-date auto option to find reference date automatically based on residual standard deviation new script - timeseries_std.py to calculate standard deviation of deramped phase residual for each epoch of timeseries timeseries2velocity.py 1. rename -m/M option to —start-date and —end-date option, add —template option to use pysar.velocity.excludeDate option 2. remove obsolete function yyyymmdd2years(), call pysar._datetime module instead 3. add function read_template2inps() to read template options into Namespace object 4. add get_exclude_date() to read all date related options 5. add get_velocity_filename() to get output velocity filename based on input initial name, and with/without exclude date | 16 May 2017, 18:48:38 UTC |
| 8ee21d4 | Yunjun Zhang | 12 May 2017, 19:47:16 UTC | rename example to examples | 12 May 2017, 19:47:16 UTC |
| 2a38169 | stackTom | 10 May 2017, 18:30:51 UTC | fix ogr2ogr showing warning saying it's ignoring layer creation option | 10 May 2017, 18:30:51 UTC |
| 08d94be | stackTom | 10 May 2017, 00:26:04 UTC | fix typo | 10 May 2017, 00:26:04 UTC |
| ce74424 | Yunjun Zhang | 08 May 2017, 23:35:08 UTC | rename select_network.py template option to select.network. modify_network.py 1. use exclude_date and exclude_ifg_index instead of ‘drop_’ prefix 2. add pysar.network.minCoherence and pysar.network.maskFile in template to support customized coherence based network modification from pysarApp.py; modify pysarApp.py help for this change. select_network.py 1. use new option prefix ‘select.network.’ 2. add start_date and end_date option 3. change default maxTempBase from 365 days to 1800 days | 08 May 2017, 23:35:08 UTC |
| 6cff0f9 | Yunjun Zhang | 08 May 2017, 22:27:56 UTC | check corrupted grb file while pyaps downloading __init__.py: adjust utilities script order, based on their dependencies _pysar_utilities.py: 1. add normalize_timeseries() function to normalize 3D matrix in time 2. adjust progress_bar style, to make percentage in the center of bar 3. add mode() function from load_data.py use progress_bar in the following scripts: _remove_surface.py multilook.py add ‘maskTempCoh.h5’ as default mask input for view.py and remove_plane.py tropcor_pyaps.py: check corrupted .grb file | 08 May 2017, 22:27:56 UTC |
| 01d9b7f | yunjunz | 07 May 2017, 00:58:08 UTC | use tightly subset dem and geomap*.trans files while re-run pysarApp.py | 07 May 2017, 00:58:08 UTC |
| 2f0b99f | Yunjun Zhang | 06 May 2017, 22:33:17 UTC | bug fix multilook.py: set default output filename the same as input filename, if current directory is different from input file directory bug fixed for diff.py for timeseries file | 06 May 2017, 22:33:17 UTC |
| 05b158a | Yunjun Zhang | 05 May 2017, 22:41:20 UTC | use current directory as weather directory if no timeseries nor dem file input | 05 May 2017, 22:41:20 UTC |
| 6d1352d | Yunjun Zhang | 05 May 2017, 17:38:42 UTC | bug fixed | 05 May 2017, 17:38:42 UTC |
| e05c905 | Yunjun Zhang | 03 May 2017, 23:11:07 UTC | pyaps download without DEM input | 03 May 2017, 23:11:07 UTC |
| 1a397be | Yunjun Zhang | 03 May 2017, 21:48:55 UTC | add --download option for tropcor_pyaps.py tropcor_pyaps.py: 1. add —download option to download weather re-analysis data only, without calculating tropospheric delay and correcting timeseries 2. add —date-list option to download data on date listed in a txt file 3. renamed ‘-t’ option back to ‘-h’ 4. change timeseries_file into optional argument, and adjust pysarApp.py to this change. info.py: add —date option for timeseries file, to print all date existed in the file, use the command below to output all date of timeseries.h5 into a txt file, that could be used in tropcor_pyaps.py —download option: info.py timeseries.h5 —date > date_list.txt | 03 May 2017, 21:48:55 UTC |
| a6e1d9d | Yunjun Zhang | 03 May 2017, 20:57:56 UTC | reverse sign of calculated tropospheric delay from PYAPS tropcor_pyaps.py: reverse the sign of calculated tropospheric delay from PyAPS, make it consistent with tropcor_phase_elevation.py. Now all corrected phase have the same sign, tropo_pyaps, tropo_height-correlation, DEM error, ramp; And use original time series minors them to get the corrected time series. select_network.py: apply threshold filtering of pairs for sequential method | 03 May 2017, 20:57:56 UTC |
| 6ae1af3 | stackTom | 03 May 2017, 19:56:54 UTC | change -r to --remove | 03 May 2017, 19:56:54 UTC |
| e870289 | yunjunz | 02 May 2017, 23:48:16 UTC | Merge branch 'master' of https://github.com/yunjunz/PySAR | 02 May 2017, 23:48:16 UTC |
| 3485c1c | yunjunz | 02 May 2017, 23:45:19 UTC | adjust file structure to UNAVCO format according to Scott's email 1. create hdf5 group with data stored in /HDFEOS/GRIDS/timeseries level 2. write all attributes, including UNAVCO required metadata to the root level | 02 May 2017, 23:45:19 UTC |
| 18d804d | Yunjun Zhang | 02 May 2017, 22:46:50 UTC | always plot Network.pdf in pysarApp.py pysarApp.py 1. always plot Network.pdf and BperpHistory.pdf in pysarApp.py if Network.pdf is not exists or old. 2. do not use Modified_ prefix anymore, overwrite mask.h5 and spatialAverageCoherence.h5 when network is updated. 3. hide message of LOD correction for mission is not Envisat plot_network.py: adjust message output modify_network.py: change default mask file and spatial average coherence file figure name _pysar_utilities.py 1. add check_readable option in update_file() to check timing of output file for non data file, like Network.pdf 2. check all group attributes with input new attribute for multiple group hdd5 file, like unwrapIfgram.h5. With this, user could use the command below to reset all dropped mark for dropped interferograms. add_attribute.py unwrapIfgram.h5 drop_ifgram=no | 02 May 2017, 22:46:50 UTC |
| b9c929d | Yunjun Zhang | 02 May 2017, 21:25:26 UTC | bug fixed | 02 May 2017, 21:25:26 UTC |
| 81a021f | Yunjun Zhang | 02 May 2017, 17:04:24 UTC | add --start-date and --end-date option for modify_network.py modify_network.py: 1. add —start-date and —end-date option for case of subsets with gap in the middle in low coherence 2. use same template option name for perp and temp baseline max as select_network.py plot_network.py: renamed output figure name from BperpHist.pdf to BperpHistory.pdf _datetime.py: bug fixed | 02 May 2017, 17:04:24 UTC |
| 70b74a2 | Yunjun Zhang | 01 May 2017, 18:30:38 UTC | bug fixed with manual modify network 1. bug fixed for modify_network.py —manual method 2. use existed Modified_unwrapIfgram.h5 in pysarApp.py if existed, to support old PySAR product. | 01 May 2017, 18:30:38 UTC |
| cd7fced | Yunjun Zhang | 01 May 2017, 18:15:17 UTC | use ut.progress_bar class instead of ut.print_progress to print progress bar for all scripts. select_network.py: 1. bug fixed for reading pysar.networ.excludeDate option 2. change —keep-seasonal option default value to False remove temporary dem in tropcor_pyaps.py | 01 May 2017, 18:15:17 UTC |
| 5d1c29f | stackTom | 01 May 2017, 05:25:56 UTC | improvements to insarmaps scripts 1) fix json_mbtiles2insarmaps.py to support latest scripts 2) add InsarDatasetController class and move uploading/deleting of mbtiles files to this class 3) remove option now removes mbtiles as well as removing the dataset from the db | 01 May 2017, 05:25:56 UTC |
| 0529d7b | Yunjun Zhang | 30 April 2017, 16:45:14 UTC | new tsviewer.py add new interactive time-series viewer script, new things includes: 1. use argparse for input argument parsing 2. add slider to show cumulative displacement at different epoch by dragging 3. now it works on Mac OS 4. output point displacement to txt file instead of mat file | 30 April 2017, 16:45:14 UTC |
| b50688f | Yunjun Zhang | 30 April 2017, 02:05:51 UTC | use mask in geo coord instead of radar coord | 30 April 2017, 02:05:51 UTC |
| 36df5e0 | Yunjun Zhang | 28 April 2017, 22:07:18 UTC | bug fixed: add mask file into UNAVCO file | 28 April 2017, 22:07:18 UTC |
| 13c99b0 | insaradmin | 28 April 2017, 20:50:51 UTC | improvements to command line opts and scripts make h5file and output dir be positional arguments in unavco2json_mbtiles.py add positional arguments to json_mbtiles2insarmaps.py use pickle area key as the unavco name, instead of relying in the supplied output dir which might not always be the unavco name print errors to stderr | 28 April 2017, 20:50:51 UTC |
| 2a98064 | Yunjun Zhang | 27 April 2017, 17:09:23 UTC | add log to select_network select_network.py: add log() function from rsmas_insar code pysarApp.py: 1. copy bl_list.txt file into PYSAR folder 2. check dropped interferogram list before updating unwrapIfgram.h5 | 27 April 2017, 17:09:23 UTC |
| f7f6a15 | yunjunz | 27 April 2017, 16:47:30 UTC | Merge branch 'master' of https://github.com/yunjunz/PySAR | 27 April 2017, 16:47:30 UTC |
| 3b0a8fd | yunjunz | 27 April 2017, 16:46:12 UTC | use last element while loading DEM or *.trans file if multiple candidates exsit. | 27 April 2017, 16:46:12 UTC |
| accdb2a | Yunjun Zhang | 27 April 2017, 01:35:30 UTC | bug fixed for subset while masking | 27 April 2017, 01:35:30 UTC |
| a497b46 | yunjunz | 25 April 2017, 22:35:27 UTC | bugs fixed | 25 April 2017, 22:35:27 UTC |
| 201d2a0 | Yunjun Zhang | 25 April 2017, 21:03:54 UTC | add progress_bar class _pysar_utilities.py add new class progress_bar, modified from PyAPS release 1.0, with code originally from http://code.activestate.com showing prefix, suffix, running time and estimated remaining time used now only in dem_error.py, will use this widely later after more testing dem_error.py use progress_bar class instead of print_progress(), iterate less by setting up argument ‘every’, reduce file size of output message file. | 25 April 2017, 21:03:57 UTC |
| a0456cd | stackTom | 25 April 2017, 18:05:24 UTC | prevent crashing on strange keys | 25 April 2017, 18:05:24 UTC |
| fa580bf | yunjunz | 24 April 2017, 22:46:34 UTC | bugs fixed _pysar_utilities.py: bug fixed for check_drop_ifgram() for old pysar file load_data.py: output message adjustment save_unavco.py: handle 'unavco.' prefix while converting pysar attributes into unavco attributes, better handling of recommended attributes | 24 April 2017, 22:46:34 UTC |
| f59558f | Yunjun Zhang | 24 April 2017, 22:05:03 UTC | add save to UNAVCO InSAR archive option into pysarApp.py _pysar_utilities.py compare input new attributes with existing file, update and touch file (will change file modification date) only when there are new attribute(s) or existing attribute(s) with new value save_unavco.py add function get_unavco_filename() to get automatic UNAVCO file name from timeseries file attributes pysarApp.py 1. add new template option pysar.save.unavco 2. copy unavco_attributes.txt from PROCESS directory if existed, for Miami user only 3. save to UNAVCO InSAR Archive format at the end of PYSAR processing if pysar.save.unavco is set to yes (default/auto is no) | 24 April 2017, 22:05:03 UTC |
| d505edc | Yunjun Zhang | 24 April 2017, 20:05:58 UTC | update temporal_coherence move main() code into temporal_coherence() for easy call update usage() | 24 April 2017, 20:05:58 UTC |
| 6e3f399 | Yunjun Zhang | 24 April 2017, 19:08:03 UTC | adjust output message print “number of acquisitions/interferograms” instead of “number of epochs” update “ref_x/y” based on updated ‘ref_lat/lon’ during geocoding | 24 April 2017, 19:08:03 UTC |
| d294e8f | Yunjun Zhang | 24 April 2017, 15:48:52 UTC | add --exclude option same as --ex option for timeseries2velocity.py | 24 April 2017, 15:48:52 UTC |
| fad7a4d | Yunjun Zhang | 24 April 2017, 02:25:59 UTC | mark attribute instead of writing new file during network modification add new pysar attribute - ‘drop_ifgram’ with value ‘yes’ or ‘no’ - indicating use or drop this interferogram for SBAS time series analysis. _datetime.py 1. delete duplicate function get_date_list() since _datetime.ifgram_date_list() has the same function 2. update plot_network() and plot_perp_baseline_hist(), including: a. add optional argument date12_list_drop and date8_list_drop b. plot dropped date in gray dot and plot dropped interferogram in dashed line c. truncate input colormap and combine its lower and upper part into a new colormap, so emphasis the color/coherence difference between dropped and kept interferograms _pysar_utilities.py: 1. add function check_drop_ifgram() to remove those marked as dropped group from group list for multi_group_hdf5_file 2. consider drop_ifgram during nonzero_mask(), temporal_average(), timeseries_inversion() 3. add new argument date12_list to function design_matrix(), and use date12_list as the only variable that is needed to get the whole design matrix 4. update perp_baseline_ifgram2timeseries() for updated ifgram_list after drop_ifgram checking load_data.py: set default ‘drop_ifgram’ of every interferogram to ‘no’ during loading modify_network.py 1. add new option ‘—mask-attribute’ to mark the attribute ‘drop_ifgram’ to ‘yes’ for all the interferograms that will be dropped, but do not write into new file 2. compare calculated date12 drop list with already existed date12 drop list, and skip if they are the same 3. change default name of modified mask and average spatial coherence file: Modified_mask.h5 —> maskModified.h5 Modified_averageSpatialCoherence.h5 —> averageSpatialCoherenceModified.h5 plot_network.py plot dropped interferogram information if exists, and do not use ‘Modified_’ in output figure name pysarApp.py: use —mark-attribute option while calling modify_network.py use —mark-attribute option while calling seed_data.py do not generate incidenceAngle.h5 file since it’s won’t be used in default routine processing (precious incidence angle now can be calculated easily whenever it’s needed) temporal_coherence.h5 1. considering drop_ifgram while calculating temporal coherence 2. delete obsolete function date_list() | 24 April 2017, 02:25:59 UTC |
| e537d15 | Yunjun Zhang | 24 April 2017, 02:07:31 UTC | renamed seed_data.py --no-update-data option to --mask-attribute seed_data.py: renamed —no-update-data option into —mark-attribute option delete obsolete usage() _datetime.py: renamed igram_date_list() into ifgram_date_list() | 24 April 2017, 02:07:31 UTC |
| e0283a2 | Yunjun Zhang | 22 April 2017, 22:07:48 UTC | referencing unwrapped interferogram during SBAS network inversion 1. _network.py: add function get_date_list() to get date_list from interferograms hdf5 file 2. _pysar_utilities.py: a. bug fixed for update_file() when output file exists but not readable b. add function add_attribute() to add/update attribute to hdf5 file, and use it in add_attribute.py c. remove obsolete function YYYYMMDD2years() d. update timeseries_inversion() to reference interferogram using ref_x/y; get rid of global variables within the function seed_data.py: a. add option “—no-update-data” to find reference pixel in x/y and update attribute of input file, but do not update the matrix data value. b. add print_message argument in random_select_reference_yx() to random select point but do not print message c. add coh_thr argument in select_max_coherence_yx() to select reference point with coherence larger than input threshold (0.85 by default) in random. Hope this could be better than selecting ‘the highest coherent pixel’, which sometime could be in the scene boundary. pysarApp.py: use’—no-update-data’ option during “reference in space” step instead of writing a new Seeded_*.h5 file temporal_coherence.py: re-referencing unwrapped interferograms while loading. | 22 April 2017, 22:07:51 UTC |
| 58baee3 | stackTom | 21 April 2017, 18:08:33 UTC | move pysar2insarmaps to unavc2oinsarmaps essentially also remove the old unavco2insarmaps | 21 April 2017, 18:08:33 UTC |
| 57a311f | Yunjun Zhang | 20 April 2017, 17:53:33 UTC | bug fixed with masking using threshold | 20 April 2017, 17:53:33 UTC |
| 237f581 | Yunjun Zhang | 20 April 2017, 02:12:19 UTC | print warning if load_data.py can not find master interferogram | 20 April 2017, 02:12:19 UTC |
| 0ef1c59 | Yunjun Zhang | 20 April 2017, 02:10:12 UTC | bug fixed while loading snaphu connect component files | 20 April 2017, 02:10:12 UTC |
| 43df6c6 | Yunjun Zhang | 19 April 2017, 21:00:25 UTC | bug fixed for pysarApp.py 1. fix bug with one instance of RuntimeError 2. subset with —tight option only when it could save > 20% of space 3. update new name rule for modify_network.py | 19 April 2017, 21:00:25 UTC |
| f48ce91 | Yunjun Zhang | 19 April 2017, 20:35:55 UTC | major update for pysarApp.py pysarApp.py 1. use warnings module to print out warning message and RuntimeError for system exit 2. use ut.update_file instead of check_isfile() to check if file exists and readable, or if output file is newer than input file or not. If it’s newer, skip that step. This makes re-run easy without worrying about delete existed files, similar to ROIPAC 3. update default file names: if output file type is different from input file type, use name without underscore; otherwise, use underscore with prefix/suffix to describe the change. detailed file name change includes: Mask.h5 —> mask.h5 average_spatial_coherence.h5 —> averageSpatialCoherence.h5 Mask_tempCoh.h5 —> maskTempCoh.h5 temporal_coherence.h5 —> temporalCoherence.h5 incidence_angle.h5 —> incidenceAngle.h5 std_velocity.h5 —> velocityStd.h5 rmse_velocity.h5 —> velocityRmse.h5 default names change in timeseries2velocity.h5 following this same rule add support old file names, including LoadedData.h5, Coherence.h5 for input, so that old PySAR product could still run 3. move mask.5h and averageSpatialCoherence.h5 generation after subset 4. generate new top level folder “WEATHER” to store all weather related data, including ECWMF data downloaded from PyAPS | 19 April 2017, 20:35:55 UTC |
| 6a09676 | Yunjun Zhang | 19 April 2017, 20:13:59 UTC | major update for load_data _pysar_utilities.py: 1. add update_file() to check whether to write output file or not. It works similar to the old/new file comparison in ROIPAC. It contains the following two scenarios: a. if there is no input file, return False if output file exists and readable; otherwise, return True. It works as the opposite as file_exist() b. if there is input file, return True if overwrite is True, or output file do not exists, or not readable (corrupt file), or older than input file. 2. change default output file name of nonzero_mask() from ‘Mask.h5’ to ‘mask.h5’ 3. get_file_list(): a. add more handling of input: return [] if input is empty; if input is string, convert it to list with one element using that string b. add sorting to each new searching result load_data.py 1. auto_path_miami(): add use of ‘ifgram_list.txt’ file in PROCESS folder to locate master interferogram, to find auto filename for geomap*.trans and radar coord DEM files 2. auto_path_miami(): add PROJECT_NAME/DEM folder to find auto filename for DEM in geo coord 3. update mode() to handle empty input or input with length of 1 4. renamed load_roipac2multi_group_h5() into roipac2multi_group_hdf5(), pass even if no ‘PROJECT_NAME’ attribute found. 5. roipac_nonzero_mask(): change default output file name starts new file every time 6. copy_file(): rename copy_roipac_file() to copy_file() and use shutil to copy instead of calling cp command. 7. add function roipac2single_dataset_hdf5() to convert input single dataset file into HDF5 file, e.g. DEM 8. add function load_file() to load any type of input file(s) into HDF5, including mutli_group_hdf5 and single_dataset_hdf5, or just copy. It’s easy to call from other scripts 9. move previous main function into a new function load_data_from_template() to load the whole dataset for pysarApp.py 10. do not generate mask.h5 and average_spatial_coherence.h5 anymore, for simple code 11. With all the changes above, load_data.py could be used both for single type of file(s) conversion into h5, and to load whole dataset for time series analysis like before. | 19 April 2017, 20:13:59 UTC |
| 3cdf8d6 | Yunjun Zhang | 19 April 2017, 19:50:49 UTC | bug fixed for get_date12_list() in _network.py 1. fix bug for get_date12_list() in _network.py when input txt file only have one line 2. update auto figure size calculation for single dataset plotting in view.py | 19 April 2017, 19:50:49 UTC |
| 2b183c5 | Yunjun Zhang | 19 April 2017, 17:56:44 UTC | change default output directory to projectName/PROCESS for miami user for non-miami user, default output directory is still the same as input reference file or baseline file, if output file is not specified. | 19 April 2017, 17:56:44 UTC |
| dda908e | Yunjun Zhang | 18 April 2017, 13:55:40 UTC | select_network: bug fixed for hierarchical method 1. fix bug in hierarchical method of select network 2. fix bug in output figure path when customized output file is specified without directory info | 18 April 2017, 13:55:40 UTC |
| 661a3d2 | Yunjun Zhang | 17 April 2017, 18:14:02 UTC | custom dir for downloaded grib file 1. add —weather-dir option to be able to choose customized directory for downloaded weather reanalysis data. To support new /ProjectName/WEATHER file structure in Miami 2. bug fixed for calling Pyaps in geo coord 3. if input DEM is h5 file, convert it to ROIPAC format and remove it at the end of the script | 17 April 2017, 18:14:02 UTC |
| 6690f8a | Yunjun Zhang | 14 April 2017, 21:02:23 UTC | use variable P_BASELINE in azimuth direction in DEM error correction dem_error.py: use variable perp baseline if input timeseries is in radar coord and has attribute ‘P_BASELINE_TOP/BOTTOM_TIMESERIES’, which are newly added. It causes difference in result in similar level as various incidence angle, for long track dataset, such as AlosDT73F2950_2990 add new script perp_baseline.py to write perpendicular baseline in 2D image for specific acquisition in timeseries. | 14 April 2017, 21:02:23 UTC |
| 5b0be37 | Yunjun Zhang | 14 April 2017, 20:52:45 UTC | add new attributes P_BASELINE_TOP/BOTTOM_TIMESERIES _pysar_utilities.py 1. rename Baseline_timeseries() to perp_baseline_ifgram2timeseries(); add calculation of pbase_top and pbase_bottom; update its usage in modify_network.py, plot_network.py, 2. timeseries_inversion(): add new attributes P_BASELINE_TOP_TIMESERIES and P_BASELINE_BOTTOM_TIMESERIES into timeseries file during SBAS network inversion 3. add perp_baseline_timeseries() to calculate perpendicular baseline timeseries given attribute dict: return linear variable P_BASELINE in azimuth direction if input file has P_BASELINE_TOP/BOTTOM_TIMESERIES attributes and is in radar coord. 4. add usage message for range_distance() 5. change design_matrix() input from HDF5 file object to interferogram file name; use _datetime.py and _network.py module for less code; add usage message. 6. update corresponding design_matrix() usage in _pysar_utilities.py, reconstruct_igrams.py, temporal_coherence.py, unwrap_error.py | 14 April 2017, 20:52:45 UTC |
| 3060e2f | Yunjun Zhang | 14 April 2017, 20:39:06 UTC | update _datetime module _datetime.py: 1. add fmt option in igram_date_list() to support date_list output in “YYMMDD” format 2. support both YYYYMMDD and YYMMDD input for date_list2tbase() and date_list2vector() save_roipac.py and save_kml.py: use 1st date in date_list of timeseries file instead of “DATE” attribute, to reduce dependence of ROIPAC attributes. Now, PySAR do not use ‘DATE’ attribute from ROI_PAC | 14 April 2017, 20:39:06 UTC |
| c148842 | Yunjun Zhang | 13 April 2017, 18:33:35 UTC | write roipac file without scientific notation 1. write_roipac_rsc() in _writefile.py: write X/Y_STEP/FIRST attribute without scientific notation, so that GeodMod works smoother 2. auto_lalo_sequence() in view.py: add step_candidate 3 for more proper step calculation 3. __init__.py: change default single plot figure size to 6.0 inch so that smaller screen still looks fine without extra manual adjustment of the figure window 4. _readfile.py: temporary turn off the conversion of attribute gamma to roipac for now, will turn on when it’s ready | 13 April 2017, 18:33:35 UTC |
| 8171d1f | Yunjun Zhang | 13 April 2017, 16:32:16 UTC | rename save_unw.py to save_roipac.py 1. rename save_unw.py to save_roipac.py 2. add support for dem format output | 13 April 2017, 16:32:16 UTC |
| 5690e40 | Yunjun Zhang | 12 April 2017, 21:37:49 UTC | bug fixed for module import in multilook.py | 12 April 2017, 21:37:49 UTC |
| 7ef104c | Yunjun Zhang | 12 April 2017, 17:43:38 UTC | add diff_file() in diff.py | 12 April 2017, 17:43:38 UTC |
| 4ddeedd | Yunjun Zhang | 12 April 2017, 17:08:36 UTC | dem_error: support more temporal deformation model and geocoded file dem_error.py: 1. use argparse for input argument parsing 2. add -i and -r option to incidence angle file and range distance file input, supporting precious parameter input for geocoded file; if not input, use 1D precise parameter for radar coord file and average value for geocoded file 3. add —poly-order option to change the order of polynomial temporal deformation model 4. add separate code to inverse dem error pixel-wise, column-wise or area-wise, depending on the dimension of input incidence angle and range distance _pysar_utilities.py: minor adjust for output message | 12 April 2017, 17:08:36 UTC |
| 3a78d34 | Yunjun Zhang | 10 April 2017, 22:19:42 UTC | add script range_distance.py to calculate range distance and write to file, similar to incidence_angle.h5 | 10 April 2017, 22:19:42 UTC |
| 341bd97 | Yunjun Zhang | 10 April 2017, 21:40:47 UTC | support range and incidence angle calculation for geo coded file 1. return center value for incidence_angle() if input file is geocoded 2. add range_distance() to calculate 0-2 D range distance using input attribute dictionary | 10 April 2017, 21:40:47 UTC |
| 5d2a99a | Yunjun Zhang | 10 April 2017, 21:37:42 UTC | update starting_range during subset | 10 April 2017, 21:37:42 UTC |
| 5cebbf0 | stackTom | 10 April 2017, 00:22:51 UTC | Make mbtiles one layer Converter now inserts databases into db as if they only had one chunk. Turns out that multiple chunks weren't necessary to get proper density, and one chunk significantly improves performance when first loading a dataset. TODO: maybe change numchunks, etc to num layers. In reality, these can be taken out altogether, but they are maintained for now to support old datasets. In the future, either take these attributes out altogether, or rename to num_layers | 10 April 2017, 00:22:51 UTC |
| da1d8bb | Yunjun Zhang | 07 April 2017, 22:48:54 UTC | make pysarApp.py run even if only unwrapIfgram.h5 is given pysarApp.py: a. fixed bugs in mask.h5 and average_spatial_coherence.h5 check b. adjust network modification part and seed_data part so it still runs if coherence do not exist. __init__.py: add two global variables: figsize_single_min and figsize_multi for default figure size used in view.py. Thus, user can change it here once for all, if they don’t like the default value _pysar_utilities.py: add abspath option in get_file_list() to return absolute path of files view.py: use pysar.figsize_single_min and pysar.figsize_multi for default figure size, and add code to calculate screen size in pixel and inch, for future use. | 07 April 2017, 22:48:54 UTC |
| 0a8a342 | yunjunz | 05 April 2017, 22:15:10 UTC | use geocode.pl directly for ROIPAC product, turn off default parallel optioin because of some wierd problem in geocode.pl, don't want to dig deep into it. | 05 April 2017, 22:15:10 UTC |
| dbc6403 | yunjunz | 05 April 2017, 19:07:58 UTC | bugs fixed for epoch-num input in view.py and --coherence plot in plot_network() within _network.py | 05 April 2017, 19:07:58 UTC |
| 3fcf7f4 | Yunjun Zhang | 04 April 2017, 22:20:06 UTC | check parallel from pysar.ut use check_parallel() in _pysar_utilities.py to check all the parallel settings, simplify code in individual scripts, and let pysar run even job lib is not installed. | 04 April 2017, 22:20:06 UTC |
| 1a6de63 | yunjunz | 04 April 2017, 21:57:22 UTC | add check_parallel() in _pysar_utilities.py check parallel only in _pysar_utilities.py, so that other scripts don't need to repeat the code, and scripts should be able to run without joblib installed | 04 April 2017, 21:57:22 UTC |
| b28f016 | yunjunz | 04 April 2017, 20:59:28 UTC | bug fixed while geocoding roipac product using geocode.py add parallel_num variable in __init__.py to limit the number of cores used in pysar parallel processing | 04 April 2017, 20:59:28 UTC |
| aaaa442 | yunjunz | 04 April 2017, 02:55:29 UTC | bug fixed for select_master_interferogram() in _network.py | 04 April 2017, 02:55:29 UTC |
| 9a5d743 | yunjunz | 04 April 2017, 02:46:35 UTC | bug fixed while testing in ccs 1. bug fixed for select_master_date() in _network.py 2. bug fixed for template reading in select_network.py 3. do not apply threshold filtering for initial network for method star/sequential/hierarchical and mst | 04 April 2017, 02:46:35 UTC |
| 6962279 | Yunjun Zhang | 03 April 2017, 23:38:43 UTC | major update on network 1. add new script select_network.py, modified from selectPairs.py in rsmas code, to generate network based on different method and output pair info - date12 - into text file add project_name2sensor() to extract sensor name from project name from input template file name add read_template2inps() to read network option from template into Namespace variable all template option could also be input in command line, but template file input is still required for now output ifgram_list.txt file with each line in YYMMDD-YYMMDD format to indicate the interferograms. plot and save network info and its baseline histogram into image file. 2. _network.py: a. read_baseline_file() now read doppler frequency that already takes prf into consideration, since rsmas code divide while writing bl_list.txt file, so that they can be used later without prf input. b. use date12 in YYMMDD-YYMMDD format for pairs selection input/output, instead of pairs_index_list, for more visual reason and reliability, in functions select_pairs_all(), select_pairs_sequential(), select_pairs_hierarchical(), select_pairs_delaunay(), select_pairs_mst(), select_pairs_star() and threshold*() and plot_*() c. move legacy sub functions from selectPairs.py in here, including: azimuth_bandwidth() range_bandwidth() wavelength() incidence_angle() signal2noise_ratio() critical_perp_baseline() calculate_doppler_overlap() threshold_doppler_overlap() threshold_perp_baseline() threshold_temporal_baseline() d. add select_master_date() to select “optimal” master date based on minimum perp/temp baseline distance e. add select_master_ifgram() to select reference interferogram for ROIPAC stacking in rsmas code plot_network.py: adjustment to use date12_list instead of pairs_index_list _datetime.py: return None instead of sys.exit() for yymmdd() for more robust and meaningful output | 03 April 2017, 23:38:43 UTC |
| a874f50 | Yunjun Zhang | 31 March 2017, 20:39:21 UTC | view.py: auto lat/lon label 1. add auto_lalo_sequence() to calculate appropriate lat/lon and its step automatically 2. use ls.shade instead of ls.hillshade to plot shaded relief DEM; change default vert_exag value, vmin/max value and interpolation for lighter and smoother topo background display | 31 March 2017, 20:39:21 UTC |
| 2f85f28 | Zhang Yunjun | 31 March 2017, 20:26:26 UTC | demo data result - Kuju AlosA | 31 March 2017, 20:26:26 UTC |
| 6c875d0 | Yunjun Zhang | 30 March 2017, 20:28:53 UTC | update --scalebar option 1. update --scalebar option to use 999 as indicator to auto value, because argparse required 3 inputs always. 2. display m instead of km when scale bar is less than 1km, and round distance up to integer when it’s more than 1km 3. add metavar for help to reduce visual noise in help 4. bug fixed while re-wrapping velocity file of file 5. increase limit from 0.5 deg to 1 deg, to check whether the area in lat/lon is too large to not to show topographic contour 6. enable fontsize control for auto tick 7. calculate auto fig size with precision to 0.5 inch instead of 1 inch | 30 March 2017, 20:28:53 UTC |
| 8066296 | Yunjun Zhang | 28 March 2017, 15:28:06 UTC | add .travis.yml file for building test | 28 March 2017, 15:28:06 UTC |
