Skip to main content
https://codeberg.org/gnuastro/papers
06 December 2023, 14:06:29 UTC
Revision 43186705c89e99fd4dbf5549ad031d27d77dfe6f authored by Mohammad Akhlaghi on 25 August 2022, 15:02:29 UTC, committed by Mohammad Akhlaghi on 02 September 2022, 17:45:31 UTC
Added server authentication and FITS DATASUM for verficiation
SUMMARY: Nothing special is necessary for your existing projects. This
commit just addds two new features (read the commit description for more):
  1. To provide a user and password to servers that need authentication
     before they allow downloading of proprietary data,
  2. To use the FITS Standard's DATASUM for file verification (for cases
     where the file is not static on the server, and is generated upon
     receiving your download request).

Until now, Maneage didn't have any infrastructure for databases that
require authentication (through a user or password, when calling
'wget'). Furthermore, when the downloaded file is automatically generated
by the server upon request, the server usually adds metadata (like file
date, or query number and etc) in the header. Therefore the simple SHA256
checksum of the file would differ on every download! This made it very hard
to verify if the data (not headers) are unchanged.

With this commit, both these problems have been addressed:

 - Server authentication: the 'reproduce/software/config/LOCAL.conf' now
   contains three new variables for this purpose. With them, you can give
   your username and password, along with the authentication method of the
   server. The comments on top of these three variables give a full
   description of their usage.

 - Verifying only the data in a file (ignoring the headers): The
   'reproduce/analysis/config/INPUTS.conf' now accepts two new optional
   variables for each input file using the FITS standard's DATASUM
   convention: 'INPUT-%-fitsdatasum' and 'INPUT-%-fitshdu'. If the SHA256
   isn't specified for a file, Maneage will use these to verify the
   file. With the latter, you specify the HDU of the data you want to
   verify and with the former you give the DATASUM value for that HDU. As
   the name suggests, this is only valid for FITS files. If we find other
   formats that support a similar behavior, we can add this feature for
   those formats also. This is also thoroughly discussed in the comments of
   'reproduce/analysis/config/INPUTS.conf'.

This commit was done with the help of Pedram Ashofte Ardakani, Sepideh
Eskandarlou and Mohammadreza Khellat.
1 parent 79ee52e
Raw File Download Save again
Tip revision: 43186705c89e99fd4dbf5549ad031d27d77dfe6f authored by Mohammad Akhlaghi on 25 August 2022, 15:02:29 UTC
Added server authentication and FITS DATASUM for verficiation
Tip revision: 4318670
.file-metadata 
MeTaSt00r3./reproduce/software/bibtex/missfits.texraulinfantesainzstaffP¥ê^Y;¤./texraulinfantesainzstaffLü^Aƒ!íA./tex/src/delete-me-image-histogram.texraulinfantesainzstaffv¥ê^ú¼ö¤./reproduce/software/bibtex/fftw.texraulinfantesainzstaff ‡^óõó2¤./reproduce/software/bibtex/sympy.texraulinfantesainzstaffP¥ê^þÛS¤./reproduce/software/bibtex/corner.texraulinfantesainzstaffP¥ê^þ)¤./reproduce/analysis/make/top-prepare.mkraulinfantesainzstaffP¥ê^Õº¤./reproduce/software/bibtex/astropy.texraulinfantesainzstaff ‡^Eóä2¤./reproduce/software/config/software_acknowledge_context.shraulinfantesainzstaffsªê^óÐ4í./reproduce/software/make/atlas-multiple.mkraulinfantesainzstaffÂû^ûÔJ8¤./reproduce/analysis/bash/download-multi-tryraulinfantesainzstaffP¥ê^w¶í./reproduce/analysis/configraulinfantesainzstaffv¥ê^~^ÆíA./reproduce/software/make/python.mkraulinfantesainzstaffÂû^à*y8¤./reproduce/software/make/build-rules.mkraulinfantesainzstaffÂû^D¡`8¤./reproduce/software/config/gnuastroraulinfantesainzstaffúþ˜^QJ£ÿA./README.mdraulinfantesainzstaffv¥ê^·¾±¤./reproduce/analysis/make/prepare.mkraulinfantesainzstaffP¥ê^N%꤁./.dir-locals.elraulinfantesainzstaff ‡^d˜ü1¤./tex/tikzraulinfantesainzstaffLü^:ƒ!í¡./reproduce/software/bibtex/healpix.texraulinfantesainzstaffP¥ê^ž.¤./reproduce/analysis/config/gnuastro/aststatistics.confraulinfantesainzstaffúþ˜^Ú/Ô¤./reproduce/software/shell/git-pre-commitraulinfantesainzstaff ‡^˜´4í./reproduce/software/config/.DS_Storeraulinfantesainzstaff÷¾\Aeà3¤./reproduce/software/bibtexraulinfantesainzstaffK%û^÷J.íA./.gitignoreraulinfantesainzstaff ‡^}S2¤./reproduce/software/config/servers-backup.confraulinfantesainzstaffv¥ê^U¥å¤./paper.pdfraulinfantesainzstaffû}ô^ç·¤./.gnuastroraulinfantesainzstaffLü^ªÂÐ!í¡./reproduce/software/make/basic.mkraulinfantesainzstaffÂû^áD[8¤./reproduce/software/config/installation/LOCAL.confraulinfantesainzstaffg%^*†j,¤./tex/src/references.texraulinfantesainzstaffv¥ê^)O¤./MakefileraulinfantesainzstaffE¦-]8·
7í¡./reproduce/software/bibtex/sip_tpv.texraulinfantesainzstaff ‡^qùB3¤./reproduce/software/shellraulinfantesainzstaff‡~ü^»9ƒíA./reproduce/software/bibtex/galsim.texraulinfantesainzstaffP¥ê^.Û'¤./reproduce/software/make/.basic.mk.swpraulinfantesainzstaff½Lü^µ®¤./cmake-crash.tar.gzraulinfantesainzstaffaú^¢Á¤./reproduce/software/config/checksums.confraulinfantesainzstaffÂû^íi58¤./projectraulinfantesainzstaffÂû^͇#8í./COPYINGraulinfantesainzstaffKȽ\¤./reproduce/software/makeraulinfantesainzstaffþ~ü^ìÐíA./reproduce/software/bibtex/scamp.texraulinfantesainzstaff ‡^ù`43¤./reproduce/software/bibtex/matplotlib.texraulinfantesainzstaffP¥ê^5¤./reproduce/analysis/config/gnuastro/astbuildprog.confraulinfantesainzstaffúþ˜^	Í¤./.localraulinfantesainzstaffLü^
Òü í¡./reproduce/analysis/makeraulinfantesainzstaff‡~ü^p£MíA./reproduce/analysis/make/initialize.mkraulinfantesainzstaff‡~ü^¬V¤./reproduce/software/shell/git-post-checkoutraulinfantesainzstaff ‡^\ç4í./reproduce/software/config/LOCAL.conf.inraulinfantesainzstaffúþ˜^]üö¤./.buildraulinfantesainzstaffLü^kÙÞ í¡./reproduce/software/make/high-level.mkraulinfantesainzstaffþ~ü^*Œ¤./reproduce/software/config/texlive-packages.confraulinfantesainzstaffúþ˜^»
¤./reproduce/software/config/TARGETS.confraulinfantesainzstaffå÷^	*¤./reproduce/analysis/make/download.mkraulinfantesainzstaffv¥ê^aéÓ¤./reproduce/software/config/gnuastro/gnuastro-local.confraulinfantesainzstaffr%^3¤./tex/src/preamble-pgfplots.texraulinfantesainzstaffv¥ê^0ý¤./reproduce/analysis/config/metadata.confraulinfantesainzstaffv¥ê^-Ǥ./reproduce/software/bibtex/scipy.texraulinfantesainzstaff ‡^5Ï93¤.raulinfantesainzstaffLü^üÐ!íA./reproduce/software/bibtex/astrometrynet.texraulinfantesainzstaff ‡^Â3à2¤./reproduce/software/configraulinfantesainzstaff‡~ü^Æû]íA./tex/src/preamble-header.texraulinfantesainzstaffP¥ê^w`¤./reproduce/analysis/config/delete-me-squared-num.confraulinfantesainzstaffv¥ê^$DŤ./reproduce/software/patchesraulinfantesainzstaffP¥ê^RÑíA./reproduce/software/bibtex/swarp.texraulinfantesainzstaff ‡^¶ÙG3¤./paper.texraulinfantesainzstaff“¥ê^DZJ¤./reproduce/software/patches/valgrind-3.15.0-mpi-fix1.patchraulinfantesainzstaffP¥ê^§|Ϥ./reproduce/software/make/atlas-single.mkraulinfantesainzstaffÂû^TÂP8¤./reproduce/analysis/bashraulinfantesainzstaffP¥ê^¶³íA./reproduce/software/config/urls.confraulinfantesainzstaff‡~ü^~äe¤./reproduce/software/bibtex/tides.texraulinfantesainzstaff ‡^S3¤./reproduce/analysis/config/pdf-build.confraulinfantesainzstaff ‡^5õ~2¤./reproduce/softwareraulinfantesainzstaffP¥ê^âÞÉíA./cmake-crash.txtraulinfantesainzstaffaú^¤./tex/buildraulinfantesainzstaffLü^¢v8!í¡./reproduce/software/config/installationraulinfantesainzstaffúþ˜^뮫íA./reproduceraulinfantesainzstaffú]íA./reproduce/software/make/README.mdraulinfantesainzstaffP¥ê^Â<„¤./tex/README.mdraulinfantesainzstaffP¥ê^B
ü¤./reproduce/software/bibtex/gnuastro.texraulinfantesainzstaff ‡^;3¤./reproduce/analysis/config/gnuastro/gnuastro-local.confraulinfantesainzstaffaô^(œ§¤./reproduce/software/shell/bashrc.shraulinfantesainzstaffP¥ê^žÝí./reproduce/.DS_StoreraulinfantesainzstaffÛ]~&…¤./reproduce/analysis/make/verify.mkraulinfantesainzstaffv¥ê^Öߤ./reproduce/software/config/LOCAL.confraulinfantesainzstaffÿÿ˜^¡º¤./reproduce/analysisraulinfantesainzstaff ‡^¹Ÿg2íA./reproduce/analysis/config/gnuastroraulinfantesainzstaffaô^@”íA./reproduce/software/patches/valgrind-3.15.0-mpi-fix2.patchraulinfantesainzstaffP¥ê^;0Ô¤./reproduce/analysis/make/top-make.mkraulinfantesainzstaffP¥ê^Íô÷¤./reproduce/software/.DS_StoreraulinfantesainzstaffÛ])s¤./reproduce/analysis/config/gnuastro/astconvertt.confraulinfantesainzstaffúþ˜^ÖÐ¤./reproduce/software/config/versions.confraulinfantesainzstaffÂû^ÌçA8¤./reproduce/analysis/config/INPUTS.confraulinfantesainzstaffv¥ê^¯À¤./tex/src/delete-me-squared.texraulinfantesainzstaffv¥ê^›'ù¤./reproduce/analysis/config/verify-outputs.confraulinfantesainzstaffP¥ê^Ÿ×Ƥ./reproduce/analysis/make/paper.mkraulinfantesainzstaffsªê^lž4¤./.file-metadataraulinfantesainzstaffÂû^•Ê8¤./tex/src/preamble-style.texraulinfantesainzstaffP¥ê^¬t-¤./reproduce/software/bibtex/mpi4py.texraulinfantesainzstaffP¥ê^ô;H¤./reproduce/software/bibtex/sextractor.texraulinfantesainzstaff ‡^µ4>3¤./reproduce/software/bibtex/imfit.texraulinfantesainzstaff ‡^^˜3¤./reproduce/software/shell/run-parts.inraulinfantesainzstaffÂû^<Œ´8í./reproduce/software/make/xorg.mkraulinfantesainzstaff÷~ü^¥	U¤./reproduce/software/config/texlive.confraulinfantesainzstaffúþ˜^L¥

¤./reproduce/analysis/make/delete-me.mkraulinfantesainzstaffv¥ê^PÈͤ./tex/src/preamble-biblatex.texraulinfantesainzstaffP¥ê^ò¤./reproduce/software/bibtex/numpy.texraulinfantesainzstaff ‡^õÁ-3¤./reproduce/analysis/config/gnuastro/gnuastro.confraulinfantesainzstaffúþ˜^i\×¤./reproduce/software/shell/configure.shraulinfantesainzstaff‡~ü^ðhí./tex/srcraulinfantesainzstaff“¥ê^3UpíA./reproduce/software/bibtex/cython.texraulinfantesainzstaff ‡^Bì2¤./reproduce/software/config/numpy-scipy.cfgraulinfantesainzstaffúþ˜^ü
¤./README-hacking.mdraulinfantesainzstaffv¥ê^èĪ¤./reproduce/software/shell/pre-make-build.shraulinfantesainzstaffÂû^Àé®8í./.DS_Storeraulinfantesainzstaffêuú^æµË(¤./tex/src/preamble-necessary.texraulinfantesainzstaff“¥ê^NÜq¤