| f7e7b46 | Boud Roukema | 11 January 2021, 11:59:59 UTC | Remove template archive today link from ref | 11 January 2021, 11:59:59 UTC |
| da756b2 | Boud Roukema | 11 January 2021, 11:54:21 UTC | Update Zenodo ID to 4432080 | 11 January 2021, 11:54:21 UTC |
| b8505ee | Boud Roukema | 11 January 2021, 11:46:43 UTC | Add ref to open gov data paper This commit adds a reference to an open government data (OGD) research paper: SARS-CoV-2 daily counts are, in principle, open government data, so a gateway article into studies of the way that governmental agencies really handle OGD should help the reader. | 11 January 2021, 11:46:43 UTC |
| 9c13f35 | Boud Roukema | 10 January 2021, 20:31:49 UTC | Merge branch 'maneage' into tandf This is a merge of upstream maneage commit d9a6855 plus a few minor fixes. I check the instructions in the commit description, and did these steps: * cd .build/ * mv -iv * analysis * mv -iv analysis/software . * mv -iv analysis/locks software/ * ./project configure --existing-conf * ./project make clean-paper # or make texclean for the upstream name After this, ./project make seems to work correctly. The 'verify' step passes successfully. | 10 January 2021, 20:31:49 UTC |
| 0e2659f | Boud Roukema | 10 January 2021, 19:57:05 UTC | Minor updates in re-finalising paper Add machine properties to text of paper; check/tidy rules for producing archival files for Zenodo; clarify that the two new papers are not about a GDP/fraud anti-correlation. | 10 January 2021, 19:57:05 UTC |
| cbe3030 | Mohammad Akhlaghi | 10 January 2021, 03:32:54 UTC | make dist: removing temp files moved after project-specific files Until now, when you ran './project make dist', first it would delete the temporary files (like files ending in '~' or '.swp' created by some editors), then it had a place to add project-specific operations for the distribution. However, in the process of cleaning the temporary files, it would 'cd' into the directory that would later be packaged. So project-specific operations would first have to 'cd' back into the top source directory. This was prone to hard-to-find bugs. With this commit, to avoid the problem the project-specific operations are now placed before the cleaning phase. This is also technically good because in the project-specific operations there may also be temporary files that shouldn't go into the distribution tarball. | 10 January 2021, 03:32:54 UTC |
| fcf9ec9 | Boud Roukema | 09 January 2021, 14:40:19 UTC | Finish maneage update; change md5sums for some results This commit should hopefully finish off the Maneage update. Some fixes of files in conflict that were edited but apparently not committed are now done. In this commit, formal verification, using md5sums, of the key results files in .tex format failed. Inspecting the numerical differences shows that these appear in the uncertainties estimated using a pseudo-random number generator by python libraries. The random seed is unchanged, but there have been several updates in python libraries: -python-version = 3.7.4 +python-version = 3.8.5 -h5py-version = 2.9.0 +h5py-version = 2.10.0 -mpi4py-version = 3.0.2 +mpi4py-version = 3.0.3 -numpy-version = 1.17.2 +numpy-version = 1.19.1 -scipy-version = 1.3.1 +scipy-version = 1.5.2 Results obtained with the old libraries (e.g. commit a59436b) successfully pass the verification, remaining bytewise reproducible. Since the more recent libraries can reasonably expected to be improved (e.g. with a better choice of default pseudorandom number generator), and since the results do not appear to be significantly different, the updated version is now considered to be correct, and the md5sums for the results files are updated in this commit. LaTeX trivia: the \input command does not appear to strictly insert the characters from the input file into the outer file. A bug with 'misplaced noalign' for an hline required putting the \hline in the input file instead of the outer file (paper.tex), despite the apparent resulting byte content being identical. | 09 January 2021, 14:40:19 UTC |
| d9a6855 | Mohammad Akhlaghi | 09 January 2021, 01:34:15 UTC | IMPORTANT: analysis outputs written in BDIR/analysis Until now, the build directory contained a 'software/' directory (that hosted all the built software), a 'tex/' subdirectory for the final building of the paper, and many other directories containing intermediate/final data of the specific project. But this mixing of built software and data is against our modularity and minimal complexity principles: built software and built data are separate things and keeping them separate will enable many optimizations. With this commit, the build directory of the core Maneage branch will only contain two sub-directories: 'software/' and 'analysis/'. The 'software/' directory has the same contents as before and is not touched in this commit. However, the 'analysis/' directory is new and everything created in the './project make' phase of the project will be created inside of this directory. To facilitate easy access to these top-level built directories, two new variables are defined at the top of 'initialize.mk': 'badir', which is short for "built-analysis directory" and 'bsdir', which is short for "built-software directory". HOW TO IMPLEMENT THIS CHANGE IN YOUR PROJECT. It is easy: simply replace all occurances of '$(BDIR)' in your project's subMakefiles (except the ones below) to '$(badir)'. To confirm if everything is fine before building your project from scratch after merging, you can run the following command to see where 'BDIR' is used and confirm the only remaning cases. $ grep -r BDIR reproduce/analysis/* --> make/verify.mk: innobdir=$$(echo $$infile | sed -e's|$(BDIR)/||g'); \ --> make/initialize.mk:badir=$(BDIR)/analysis --> make/initialize.mk:bsdir=$(BDIR)/software --> make/initialize.mk: $$sys_rm -rf $(BDIR) --> make/top-prepare.mk:all: $(BDIR)/software/preparation-done.mk 'BDIR' should only be present in lines of the files above. If you see '$(BDIR)' used anywhere else, simply change it to '$(badir)'. Ofcourse, if your project assumes BDIR in other contexts, feel free to keep it, it will not conflict. If anything un-expected happens, please post a comment on the link below (you need to be registered on Savannah to post a comment): https://savannah.nongnu.org/task/?15855 One consequence of this change is that the 'analysis/' subdirectory can be optionally mounted on a separate partition. The need for this actually came up for some new users of Maneage in a Docker image. Docker can fix portability problems on systems that we haven't yet supported (even Windows!), or had a chance to fix low-level issues on. However, Docker doesn't have a GUI interface. So to see the built PDF or intermediate data, it was necessary to copy the built data to the host system after every change, which is annoying during working on a project. It would also need two copies of the source: one in the host, one in the container. All these frustrations can be fixed with this new feature. To describe this scenario, README.md now has a new section titled "Only software environment in the Docker image". It explains step-by-step how you can make a Docker image to only host the built software environment. While your project's source, software tarballs and 'BDIR/analysis' directories are on your host operating system. It has been tested before this commit and works very nicely. | 09 January 2021, 03:00:15 UTC |
| 22aa8a2 | Boud Roukema | 08 January 2021, 19:58:06 UTC | Merge branch 'maneage' into tandf This commit aims to update the subpoisson project to a more recent version of Maneage, which should make it more likely to work 'out of the box' for people on a wider variety of systems, including Darwin/Xnu (macOS, iOS, ...) systems. This commit has not yet been tested. | 08 January 2021, 19:58:06 UTC |
| 39d3e7e | Boud Roukema | 08 January 2021, 17:46:48 UTC | Fix table EOLs; add two COVID-19 1BL papers This commit adds '\\' before \hline in tables where LaTeX is now a bit fussier than before; add brief citations to two COVID-19 1BL papers; updates the bst file and and removes the ^Ms from it. | 08 January 2021, 17:46:48 UTC |
| a59436b | Boud Roukema | 08 January 2021, 04:16:19 UTC | Shorten abstract; country full names; acknowledgments Reduce abstract word length to around 200 words; replace a few of the iso 2-letter country codes in the abstract by the full country names; acknowledge helpful comments by Taha Rouabah. | 08 January 2021, 04:16:19 UTC |
| 3845c89 | Boud Roukema | 08 January 2021, 02:03:31 UTC | Switch to tandf style: first try These commits should enable switching to the style of TandF journals. | 08 January 2021, 02:03:31 UTC |
| b413260 | Boud Roukema | 07 January 2021, 18:15:47 UTC | Merge branch 'subpoisson' of codeberg:boud/subpoisson into subpoisson | 07 January 2021, 18:15:47 UTC |
| 499281a | Boud Roukema | 07 January 2021, 18:14:37 UTC | Minor: merge branch 'wiley' into subpoisson - one keyword change | 07 January 2021, 18:14:37 UTC |
| b91af98 | Mohammad Akhlaghi | 05 January 2021, 18:01:19 UTC | Configuration: GNU Binutils linking bug on some systems fixed Until now, when building GNU Binutils on GNU Linux operating systems, we would simply put a link to the host's core C library components (the '*crt*' files). However, the symbolic link wasn't "forced"! So if it already existed in the build directory, it would crash. With this commit a '-f' option has been added to the 'ln' command and this fixed the problem. This bug was reported by Zahra Sharbaf. | 05 January 2021, 18:01:19 UTC |
| a1a966a | Mohammad Akhlaghi | 04 January 2021, 03:32:38 UTC | Building of Less program now uses patchelf to ensure good linking After correctly setting Less to depend on 'ncurses', I noticed its still not linking to Maneage's 'ncurses', but pointing to my host system's 'ncurses' (that happens to have the same version! So it would crash on a system with a different version). This shows that like some other software, we need to manually correct the RPATH inside Less. With this command, the necessary call to 'patchelf' has been added and with it, the installed 'less' command properly linked to Maneage's internal build of 'ncurses'. | 04 January 2021, 03:32:38 UTC |
| dc4aa8c | Mohammad Akhlaghi | 04 January 2021, 03:21:03 UTC | README-hacking.md: edits and improvements to publication checklist After going through the publication checklist, some edits were made to make things more clear. Also, an item was added to remind the project author that the commit hashes on the uploaded data files should be the same. | 04 January 2021, 03:21:03 UTC |
| 02e53b9 | Mohammad Akhlaghi | 04 January 2021, 02:58:05 UTC | README.md: summary Dockerfile with all necessary lines in one step Until now, the description in 'README.md' to build the Dockerfile in 'README.md' had one item per line, thoroughly describing the reason behind that line. But in many cases, the user is already familiar with Docker (or has already read through the items) and just wants to have the Dockerfile ready fast. In these cases, all those extra explanations are annoying. With this commit, an item '0' has been added at the start of the item list for summary. It only contains the necessary Dockerfile contents with no extra explanation. | 04 January 2021, 02:58:05 UTC |
| 31f4ea3 | Mohammad Akhlaghi | 04 January 2021, 01:52:25 UTC | Building of less software depends on ncurses Until now, the 'less' software package (used to view large files easily on the command-line and used by Git for things like 'git diff' or 'git log') only depended on 'patchelf' (which is a very low-level software). However, as Boud reported in bug #59811 [1], building less would crash with an error saying "Cannot find terminal libraries" in some systems (including the proposed Docker image of 'README.md' which I confirmed afterwards). Looking into the 'configure' script of 'less', I noticed that 'less' is actually just checking for some functions provided by the ncurses library! With this commit, 'less' depends on 'ncurses'. I was able to confirm that with this change, 'less' successfully builds within the Docker image. [1] https://savannah.nongnu.org/bugs/?59811 | 04 January 2021, 01:52:25 UTC |
| b1bd282 | Mohammad Akhlaghi | 02 January 2021, 15:59:49 UTC | ./project make: new texclean target Until now there was only a 'clean' (to delete all files created during the 'make' phase) and the 'distclean' (to delete all files during configuration and make). But sometimes we don't want to delete all the files created during the full 'make' phase, we only want to delete the files that were created by LaTeX for building the paper. Witht this commit, a new target has been added for this job. You can now run the following command for this job: ./project make texclean Only the files in '$(BDIR)/tex/build' will be deleted (and the 'tikz' directory under that location is recreated, ready for a future build). | 02 January 2021, 15:59:49 UTC |
| ff43476 | Mohammad Akhlaghi | 02 January 2021, 15:52:31 UTC | Copyright year updated in all source files Having entered 2021, it was necessary to update the copyright years at the top of the source files. We recommend that you do this for all your project-specific source files also. | 02 January 2021, 15:52:31 UTC |
| b910b2e | Mohammad Akhlaghi | 14 December 2020, 02:14:34 UTC | Better warnings when maneage branch not present and PDF not built Until now, there was no warning when the 'maneage' branch didn't exist in the Git history. This can happen when you forget to push the 'maneage' branch to a remote for your project, and you later clone your project from that remote (for example on another computer). We use the 'maneage' branch to report the latest commit hash and date in the final paper (which can greatly help future readers). Since we check the 'maneage' branch on every run of './project make' (in 'initialize.mk') this would result in a printed statement like this: fatal: Not a valid object name maneage Also until now, the description of what to do when TeXLive wasn't installed properly wasn't complete: it didn't mention that it is necessary to delete the TeXLive target files. This could confuse users (they would re-run './project configure -e', but with no effect). With this commit, for the 'maneage' branch issue a complete warning will be printed. Telling the user what to do to get the 'maneage' branch (and thus fix this warning). Also, the LaTeX macros that go in the paper are now red when the 'maneage' branch doesn't exist, telling the user to see the printed warning (thus encouraging the user to get the branch). For the TeXLive issue, the necessary commands to run are now also printed in the warning. | 14 December 2020, 02:28:14 UTC |
| cce4016 | Raul Infante-Sainz | 09 December 2020, 20:09:43 UTC | Configuration: not settting C_INCLUDE_PATH on macOS Until now, when building the high-level (optional) software, we would give both 'CPPFLAGS' and 'C_INCLUDE_PATH' the same value/directory in 'high-level.mk'. But we recently found that on macOS's C compiler ('clang'), if a directory is included in both 'CPPFLAGS' and 'C_INCLUDE_PATH', then that directory is ignored in 'CPPFLAGS' (which has higher priority). This caused linking problems when the version of a software on the host was different from the Maneage version. With this commit, 'C_INCLUDE_PATH' is not set on macOS any more and this fixed the problem on the reported systems. This bug was fixed with the help of Mohammad Akhlaghi and Mahdieh Navabi. | 09 December 2020, 22:44:13 UTC |
| 021ff34 | Mohammad Akhlaghi | 02 December 2020, 01:01:45 UTC | Less is now built as a basic software Less is rarely used in non-interactive mode and is primarily intended for interactively viewing large files. So its need within Maneage (for batch processing) wasn't often felt until now. However, when running './project shell' (which completely closes-off the outside environment), or building a Maneage'd project within a minimal container that doesn't have less, it becomes hard to use Git (and in particular its 'diff' output which depends on 'less'). With this commit, Less has been added as a dependency of Git in 'basic.mk'. In total its built product is roughly 800KB and builds within a second or two. So it isn't a burden on any project. But it can be very useful when the projects are being developed within the Maneage environment itself. | 02 December 2020, 01:01:45 UTC |
| 1ee1d66 | Mohammad Akhlaghi | 01 December 2020, 17:32:44 UTC | Installation: m4 no longer depends on Texinfo In a recent build on a macOS, we recognized that Texinfo needs the 'libintl.h' headers of Gettext. However, Gettext depends on M4, and until now we had set M4 to depend on Texinfo. Therefore adding Gettext as a dependency of Texinfo would cause a circular dependency. On the macOS, we temporarily disabled M4's Texinfo dependency, and the build went through. I also checked on my GNU/Linux system: temporarily renamed all Texinfo built files from my system and done a clean build of M4 and it succeeded. To be further safe, I built Maneage from this commit (where M4 doesn't depend on Texinfo) in a Docker container, and it went through with no problems. So the current M4 version indeed doesn't need Texinfo. I think adding Texinfo as a dependency of M4 was a historic issue from the early days. In the process, I also cleaned 'basic.mk' a little: - A "# Level N" comment was added on top of each group of software that can be built in parallel (generally). - GNU Nano was moved to the end of the file (to be "Level 6"). - Some comments were edited in some places. | 01 December 2020, 18:53:22 UTC |
| e796e6f | Mohammad Akhlaghi | 01 December 2020, 16:42:50 UTC | README-hacking.md: recommended to push maneage after merging Until now at the end of the updating process, we hadn't explicity talked about pushing the branches. So people would usually only push their 'master' branch to their remote. While the merged 'master' branch does contain the commits from the core Maneage branch, having a no-updated 'maneage' branch reference on their remote can be confusing. With this commit, at the end of the process to merge with the 'maneage' branch we explicitly recommend to push both the 'master' and 'maneage' branches. | 01 December 2020, 16:47:31 UTC |
| ac8eee9 | Mohammad Akhlaghi | 01 December 2020, 11:43:04 UTC | Default paper: macros available for date of commits cited Until now, Maneage only provided the commit hashes (of the project and Maneage) as LaTeX macros to use in your paper. However, they are too cryptic and not really human friendly (unless you have access to the Git history on a computer). With this commit, to make things easier for the readers, the date of both commits are also available as LaTeX macros for use in the paper. The date of the Maneage commit is also included in the acknowledgements. Also, the paragraph above the acknowledgements has been updated with better explanation on why adding this acknowledgement in the science papers is good/necessary. | 01 December 2020, 11:43:04 UTC |
| 6ca2fbf | Mohammad Akhlaghi | 30 November 2020, 23:50:38 UTC | IMPORTANT: organizational improvements in Maneage TeX sources This only concerns the TeX sources in the default branch. In case you don't use them, there should only be a clean conflict in 'paper.tex' (that is obvious and easy to fix). Conflicts may only happen in some of the 'tex/src/preamble-*.tex' files if you have actually changed them for your project. But generally any conflict that does arise by this commit with your project branch should be very clear and easy to fix and test. In short, from now on things will even be easier: any LaTeX configuration that you want to do for your project can be done in 'tex/src/preamble-project.tex', so you don't have to worry about any other LaTeX preamble file. They are either templates (like the ones for PGFPlots and BibLaTeX) or low-level things directly related to Maneage. Until now, this distinction wasn't too clear. Here is a summary of the improvements: - Two new options to './project make': with '--highlight-new' and '--highlight-notes' it is now possible to activate highlighting on the command-line. Until now, there was a LaTeX macro for this at the start of 'paper.tex' (\highlightchanges). But changing that line would change the Git commit hash, making it hard for the readers to trust that this is the same PDF. With these two new run-time options, the printed commit hash will not changed. - paper.tex: the sentences are formatted as one sentence per line (and one line per sentence). This helps in version controlling narrative and following the changes per sentence. A description of this format (and its advantages) is also included in the default text. - The internal Maneage preambles have been modified: - 'tex/src/preamble-header.tex' and 'tex/src/preamble-style.tex' have been merged into one preamble file called 'tex/src/preamble-maneage-default-style.tex'. This helps a lot in simply removing it when you use a journal style file for example. - Things like the options to highlight parts of the text are now put in a special 'tex/src/preamble-maneage.tex'. This helps highlight that these are Maneage-specific features that are independent of the style used in the paper. - There is a new 'tex/src/preamble-project.tex' that is the place you can add your project-specific customizations. | 01 December 2020, 00:12:11 UTC |
| c312f4e | Mohammad Akhlaghi | 28 November 2020, 22:18:36 UTC | README-hacking.md: updated paper to cite for using Maneage Until now, we were asking the users of Maneage to cite the first paper that used its primoridal version (arXiv:1505:01664). But there is now a paper that fully describes the concept (arXiv:2006.03018). With this commit, in the 'citation' section of 'README-hacking.md' we now ask to cite the new paper. | 28 November 2020, 22:18:36 UTC |
| fcc77a7 | Mohammad Akhlaghi | 15 November 2020, 13:28:57 UTC | Building final PDF: pdf-build-final has to be given an explicit yes Until now, when the 'pdf-build-final' configuration variable (defined in 'reproduce/analysis/config/pdf-build.conf') was given any string a PDF would be built. This was very confusing, because people could put a 'no' and the PDF would still be built! With this commit, only when this variable has a value of 'yes' will the PDF be built. If given any other string (or no string at all), it will not produce a PDF. This issue was reported by Zahra Sharbaf. | 15 November 2020, 13:32:57 UTC |
| 9cfb8d3 | Mohammad Akhlaghi | 13 November 2020, 19:11:18 UTC | README.md: added commands to delete all Docker images Until now we had described the basic commands on how to create and use Docker images, but we hadn't mentioned how you can delete them. With this commit the commands necessary for deleting Docker images have also been added at the bottom of the section on Docker. | 13 November 2020, 19:11:18 UTC |
| 2b39a67 | Mohammad Akhlaghi | 18 October 2020, 13:37:16 UTC | Recipes for final initialize and verify targets not on stdout The LaTeX macro files for these two subMakefiles are created on every run of './project make'. So their commands are also printed every time and hardly ever will a normal user want to modify or change these. So to avoid populating the standard output of a Maneaged project with all these extra lines every time (possibly getting mixed with the important analysis or LaTeX outputs), an '@' has been placed at the start of the recipes. With an '@' at the start of the recipe, Make is instructed to not print the commands it wants to run in the standard output. | 18 October 2020, 13:37:16 UTC |
| 04bda5a | Boud Roukema | 09 October 2020, 10:33:58 UTC | Update README-hacking.md with elaphrocentre ArXiv:2010.03742 This commit updates README-hacking.md with the URIs for the 'elaphrocentre' galaxy formation pipeline paper arXiv:2010.03742. This makes three papers currently in the peer review pipeline: arXiv:2006.03018, arXiv:2007.11779, and arXiv:2010.03742, each chronologically corresponding to various stages of the review process. | 09 October 2020, 13:28:21 UTC |
| b677efb | Mohammad Akhlaghi | 02 October 2020, 00:12:35 UTC | TexLive's xstring package is now necessary After a fresh build of Maneage with a newly downloaded TeXLive, I noticed that it is complaining about not finding 'xstring.sty', apparently some package that depeneded on it is no longer including it itself! It is thus now added to the packages that are built by Maneage's TeXLive. | 02 October 2020, 00:12:35 UTC |
| ee8c6be | Mohammad Akhlaghi | 24 September 2020, 01:15:40 UTC | Gnuastro's analysis configuration files removed Until now, the core Maneage branch included some configuration files for Gnuastro's programs. This was actually a remnant of the distant past when Maneage didn't actually build its own software and we had to rely on the host's software versions. This file contained the configuration files specific to Gnuastro for this project and also had a feature to avoid checking the host's own configuration files. However, we now build all our software ourselves with fixed configuration files (for the version that is being installed and its version is stored). So those extra configuration files were just extra and caused confusion and problems in some scenarios. With this commit, those extra files are now removed. Also, two small issues are also addressed in parallel with this commit: - When running './project make clean', the 'hardware-parameters.tex' macro file (which is created by './project configure' is not deleted. - The project title is now written into the default output's PDF's properties (through 'hypersetup' in 'tex/src/preamble-header.tex') through the LaTeX macro. All these issues were found and fixed with the help of Samane Raji. | 24 September 2020, 01:15:40 UTC |
| 0c882a4 | Raul Infante-Sainz | 14 September 2020, 14:49:24 UTC | Checking Xcode installation for macOS systems Until now, during the configure step it was checked if the host Operative System were GNU/Linux, and if not, we assumed it is macOS. However, it can be any other different OS! With this commit, now we explicity check if the system is GNU/Linux or Darwin (macOS). If it is not any of them, a warning message says to the user that the host system is different from which we have checked so far (and invite to contact us if there is any problem). In addition to this, if the system is macOS, now it checks if Xcode is already installed in the host system. If it is not installed, a warning message informs the user to do that in case a problem/crash in the configure step occurs. We have found that it is convenient to have Xcode installed in order to avoid some problems. | 15 September 2020, 17:49:42 UTC |
| 11b11e9 | Boud Roukema | 09 September 2020, 08:08:20 UTC | R is built without tcl/tk (for GUI) dependence Tcl/Tk are a set of tools to provide Graphic User Interface (GUI) support in some software. But they are not yet natively built within Maneage, primarily because we have higher-priority work right now. GUI tools in general aren't high on our priority list right now because GUI tools are generally good for human interaction (which is contrary to the reproducible philosophy), not automatic analysis (a core concept in reproducibility). So even later, when we do include Tcl/Tk in Maneage, their direct usage will be discouraged. Until this commit, because we don't yet build Tcl/Tk, the default maneage install of the statistical package R failed on a Debian Stretch, with 6227 repeats of the line: '/usr/lib//tcl8.5/tclConfig.sh: line 2: dpkg-architecture: command not found' To fix this problem (atleast until Tcl/Tk is installed within Maneage), R is now configured with the '--without-tcltk' option which fixed the problem. Please see the description above the R installation instructions in 'reproduce/software/make/high-level.mk' for more. | 09 September 2020, 11:46:15 UTC |
| 24d01f5 | Mohammad Akhlaghi | 08 September 2020, 00:25:29 UTC | Removed all occurances of IFS in low-level scripts Following the previous commit, we recognized that the 'IFS' terms are not necessary and can be even cause problems. So all their occurances in the scripts of Maneage have been removed with this commit. | 08 September 2020, 00:25:29 UTC |
| 6727f84 | Mohammad Akhlaghi | 07 September 2020, 18:42:03 UTC | Software installation: removed IFS statements in pre-make-build.sh Until a recent commit, the IFS='"' was added at the start of the variables in this shell script and as a result, the SPACE character wasn't being used as a delimiter. This caused a major problem when downloading the tarballs (all the backup servers were considered as the top link). With this commit we removed these 'IFS' statements). Because we now check for the existance of meta-characters in the build directory name, there is no more problem, and also generally both the calling command and internally, we have double-qutations around the variable names. So removal of IFS will not affect the result in this scenario. This bug was found by Mohammadreza Khellat. | 07 September 2020, 18:42:03 UTC |
| f5fe8cf | Boud Roukema | 31 August 2020, 18:48:39 UTC | Clarify which graph the data came from; fix absent dates This commit adds the name of the graph from which the data came; this will not help much in archival terms, since the URL is unarchivable, but for a short period may be useful. A minor fix is made to the early dates in March, for which CDTA does not provide data. | 31 August 2020, 18:48:39 UTC |
| 57b1a90 | Boud Roukema | 31 August 2020, 18:31:13 UTC | Verify that the WP C19CCTF data match the CDTA DZ data This commit adds the file `reproduce/inputs/Wikipedia_SARSCoV2_charts_DZ_cdta_dz.dat` which includes a fresh download of WP C19CCTF data for Algeria, and a manually verified check on the official Algerian data provided by the Center for Development of Advanced Technologies (CDTA) at https://covid19.cdta.dz/dashboard/production/index.php . The fifth column indicates the CDTA data at all dates provided, and 0 in dates not provided. The cdta.dz website provides an http response that requires javascript mouse events for extraction of the data. These data were extracted manually, although in principle a script could be written for automatic extraction. Almost no differences were found, except where the CDTA data appears to disagree with the WP C19CCTF version of the official data: * on 2020-04-08, CDTA has 1578 instead of 1572; * on 2020-04-19, CDTA has 2534 (identical to 2020-04-18) instead of 2629; * on 2020-06-23, CDTA has 12077 instead of 12077; * seven dates in early July are missing in the CDTA data. | 31 August 2020, 18:31:13 UTC |
| 79aa59a | Boud Roukema | 30 August 2020, 20:31:30 UTC | Fix superscripts in pdf | 30 August 2020, 20:31:30 UTC |
| 79a3c67 | Boud Roukema | 30 August 2020, 20:02:52 UTC | Minor: better keyword - data validation | 30 August 2020, 20:02:52 UTC |
| 80e3202 | Boud Roukema | 30 August 2020, 19:12:47 UTC | Changes to shift journal style to Wiley; algeria.sh This commit makes a whole bunch of style changes to shift to the Wiley journal style. A script to compare the WP C19CCTF and JHU Algerian daily SARS-CoV-2 counts with the Rouabah+2020 counts is added for running by hand. | 30 August 2020, 19:12:47 UTC |
| 4254c80 | Boud Roukema | 30 August 2020, 19:06:34 UTC | Add a sentence on JHU to first para of method section Since some readers convinced by the JHU name might not read far enough to notice that the JHU data are analysed in an appendix and give almost identical results (apart from India), a sentence is added earlier to increase the chance that the reader knows that JHU data is considered. | 30 August 2020, 19:06:34 UTC |
| 290b80d | Boud Roukema | 30 August 2020, 01:15:42 UTC | Minor reference fix | 30 August 2020, 01:15:42 UTC |
| b4be72e | Boud Roukema | 30 August 2020, 01:09:14 UTC | Remove eps hack needed for old hyperref; include .sty files This commit removes the eps file hack from `reproduce/analysis/make/paper.mk` since the hack does not seem to be needed any more with a recent hyperref/texlive setup. In this commit, style files (*.sty) are added to 'package-contents' in `reproduce/analysis/make/initialize.mk`. Hint: if compiling in an old texlive system, remove `NJDnatbib.sty` by hand from the distribution in order to avoid superscript links that have almost zero height. | 30 August 2020, 01:09:14 UTC |
| f1c88e4 | Boud Roukema | 30 August 2020, 00:10:55 UTC | Fix doubled word Email | 30 August 2020, 00:10:55 UTC |
| ca8d5ae | Boud Roukema | 30 August 2020, 00:07:53 UTC | Do a proper LaTeX n-accent-aigu in Torun | 30 August 2020, 00:07:53 UTC |
| 21dd1f1 | Boud Roukema | 29 August 2020, 23:59:17 UTC | Do journal patches properly; package-contents: eps files With this commit, the journal patches are put into their correct places and applied correctly. The 'package-contents' rule in `reproduce/analysis/make/initialize.mk` is changed so that only the .eps files used in the paper are included in the package. While these figures may be interesting to people wanting more info, beyond the formally published paper, they will be annoying and confusing to journal typesetting staff, who for some journals say that figures have to be numbered from 1 to N. Giving them N+M figures with only N to be really used could lead to errors that waste the authors' time to clean up later, or errors that the authors don't notice in the proofs. | 29 August 2020, 23:59:17 UTC |
| 1c16987 | Boud Roukema | 29 August 2020, 04:06:04 UTC | Fix unremoved old Wiley .log file | 29 August 2020, 04:06:04 UTC |
| 59ba0b1 | Boud Roukema | 29 August 2020, 03:52:21 UTC | Fix confusing percent symbol; full run This commit fixes a '%' symbol that was creating problems in the latexpand inclusion of the .bbl file into paper-full.tex (the % was protected, creating \%, which was unexpected in a list of \bibitem's). This commit disables the dev mode. | 29 August 2020, 03:52:21 UTC |
| 5c775f9 | Boud Roukema | 29 August 2020, 03:29:46 UTC | Update to Wiley LaTeX format This commit should satisfy the Wiley LaTeX style. It is ready for testing. | 29 August 2020, 03:29:46 UTC |
| 6d18576 | Mohammad Akhlaghi | 28 August 2020, 15:10:27 UTC | Edited README.md to remove installation of a text editor With the previous commit, we now build Nano by default within Maneage, and project authors can ask to install Emacs and Vim within 'TARGETS.conf'. So in the instructions to build within a Docker image have been removed. | 28 August 2020, 15:10:27 UTC |
| 8794810 | Mohammad Akhlaghi | 27 August 2020, 15:47:26 UTC | Plain text editors: nano in basic, emacs and vim in high-level While a project is under development, the raw analysis software are not the only necessary software in a project. We also need tools to all the edit plain-text files within the Maneaged project. Usually people use their operating system's plain-text editor. However, when working on the project on a new computer, or in a container, the plain-text editors will have different versions, or may not be present at all! This can be very annoying and frustrating! With this commit, Maneage now installs GNU Nano as part of the basic tools. GNU Nano is a very simple and small plain text editor (the installed size is only ~3.5MB, and it is friendly to new users). Therefore, any Maneaged project can assume atleast Nano will be present (in particular when no editor is available on the running system!). GNU Emacs and VIM (both without extra dependencies, in particular without GUI support) are also optionally available in 'high-level.mk' (by adding them to 'TARGETS.conf'). The basic idea for the more advanced editors (Emacs and VIM) is that project authors can add their favorite editor while they are working on the project, but upon publication they can remove them from 'TARGETS.conf'. A few other minor things came up during this work and are now also fixed: - The 'file' program and its libraries like 'libmagic' were linking to system's 'libseccomp'! This dependency then leaked into Nano (which depends on 'libmagic'). But this is just an extra feature of 'file', only for the Linux kernel. Also, we have no dependency on it so far. So 'file' is not configured to not build with 'libseccomp'. - A typo was fixed in the line where the physical core information is being read on macOS. - The top-level directories when running './project shell' are now quoted (in case they have special characters). | 28 August 2020, 14:02:30 UTC |
| 253b72c | Boud Roukema | 28 August 2020, 03:12:31 UTC | Incomplete fixes for Wiley style Changes to journal style for Wiley. TODO = * ArXiv is still TODO (Jiang+ probably has a double entry); * some wording near the superscript refs needs to be tidied up; * (author?) in Code availability needs fixing; * the style of the "ethics" sections needs to be compared to common journal style. | 28 August 2020, 03:12:31 UTC |
| c6a4aaa | Mohammadreza Khellat | 26 August 2020, 10:53:10 UTC | Machine architecture and byte-order available as LaTeX macro Until now, no machine-related specifications were being documented in the workflow. This information can become helpful when observing differences in the outcome of both software and analysis segments of the workflow by others (some software may behave differently based on host machine). With this commit, the host machine's 'hardware class' and 'byte-order' are collected and now available as LaTeX macros for the authors to use in the paper. Currently it is placed in the acknowledgments, right after mentioning the Maneage commit. Furthermore, the project and configuration scripts are now capable of dealing with input directory names that have SPACE (and other special characters) by putting them inside double-quotes. However, having spaces and metacharacters in the address of the build directory could cause build/install failure for some software source files which are beyond the control of Maneage. So we now check the user's given build directory string, and if the string has any '@', '#', '$', '%', '^', '&', '*', '(', ')', '+', ';', and ' ' (SPACE), it will ask the user to provide a different directory. | 27 August 2020, 11:40:16 UTC |
| 9a0d968 | Mohammad Akhlaghi | 25 August 2020, 22:03:01 UTC | README.md: added explanation on copying files from Docker image When building Maneage inside a Docker container, in the end the users want to extract the final outputs from the container into their host operating system to inspect more comfortably. So with this commit, a short examplanation has been added on how to do this. We also noticed that it is much better if the 'Dockerfile' is stored and run in an empty directory, otherwise, it will start parsing the full directory and its subdirectories as the docker image's environment. | 25 August 2020, 22:03:01 UTC |
| 7d03a2a | Boud Roukema | 23 August 2020, 21:28:27 UTC | TODO: brief reference to 2005.13516 Since Algerian data turn out to be especially interesting, a reference to ArXiv:2005.13516 would be useful, including the published bibliometry parameters if available by the time of finalising this article in response to the referee(s). | 23 August 2020, 21:28:27 UTC |
| 2ba8d31 | Boud Roukema | 19 August 2020, 02:43:16 UTC | TODO: refer to Balashov+2020 2007.14841 Add note to TODO file to refer to Balashov+2020 2007.14841 which does a Newcomb--Benford law on the initial (exponential) phase of the pandemic. | 19 August 2020, 02:43:16 UTC |
| 02548e9 | Boud Roukema | 18 August 2020, 23:43:33 UTC | Add Keegan Tan reference for Wikipedia | 18 August 2020, 23:43:33 UTC |
| 3e9d308 | Boud Roukema | 18 August 2020, 22:26:05 UTC | Merge branch 'jhu' into subpoisson | 18 August 2020, 22:26:05 UTC |
| 6e35ac0 | Boud Roukema | 18 August 2020, 22:24:31 UTC | Update zenodo ID for v2 The main difference between v1 and v2 is that JHU CSSE data is analysed in an appendix; the results have some minor differences with those for the Wikipedia COVID-19 Case Count Task Force data. | 18 August 2020, 22:24:31 UTC |
| 0d3f4f6 | Boud Roukema | 18 August 2020, 18:51:51 UTC | JHU appendix - minor wording improvement Minor improvement in the comment in the absence of India from the 7-day most un-noisy table in the case of the JHU data. | 18 August 2020, 18:51:51 UTC |
| 364a563 | Boud Roukema | 17 August 2020, 17:43:45 UTC | Swap calculation order of WP and JHU To simplify the question of naming output files, it is easier to calculate the JHU data first, and then recreate the equivalent files for the WP data, so that main results are for the WP data, with the JHU data only included in the appendix of the paper. This commit does this by swapping the order of analysing the two datasets in `reproduce/analysis/make/poisson.mk`. | 17 August 2020, 17:43:45 UTC |
| 9d9e9c3 | Boud Roukema | 16 August 2020, 23:11:51 UTC | Disable dev override - JHU branch | 16 August 2020, 23:11:51 UTC |
| b2910af | Boud Roukema | 16 August 2020, 22:35:48 UTC | Updates for JHU, MathBiol This commit does several updates, including an Appendix and brief comments in the introduction and at the end of the discussion section to point to the appendix. This still has to be run with the devmode disabled to check the full resolution results, but they appear to be largely compatible with the WPC19CCTF results. Section numbers are reintroduced. Some of the verification md5sums are updated. | 16 August 2020, 22:36:54 UTC |
| 56ecbb0 | Boud Roukema | 16 August 2020, 00:40:21 UTC | Try summing JHU data over sub-national divisions The JHU CSSE GIS data was not used in the main analysis because many of its entries are for sub-national divisions. Since the primary question of interest is the validity of data provided by national authorities, the JHU dataset is inhomogeneous. Nevertheless, by summing over the data for province/states in countries for which these sub-national data are available, a dataset that is as close as possible to the national counts can be reconstructed. This branch/commit adds the JHU data, replaces the CCTF19 main dataset for analysis by the JHU data, and disables the `verify` config parameter. The pdf produced with this commit will be mostly quite misleading, since it will incorrectly claim that data is from CCTF19 when in reality it is JHU data; this pdf should be interpreted keeping this in mind. Probably the most useful role for the JHU analysis would be to add four tables as an appendix, equivalent to the four tables with the CCTF19 analyses, since the results only have small numerical differences for most of the low phi_i countries, and the lists of low phi_i countries are more or less the same as for the CCTF19 data. | 16 August 2020, 00:40:21 UTC |
| 9573d4e | Mohammad Akhlaghi | 13 August 2020, 16:33:00 UTC | --check-config option now prints names of last 5 installed Until now, './project --check-config' would only print the names of the software that were being built. Besides that, it is also useful to know which packages have most recently finished. With this commit, we now print the last 5 built software packages with '--check-config' also, and the output has also been placed in a row of '='s to help separate it in each round. Also some more sanity checks have been added so it doesn't print error messages. | 13 August 2020, 16:33:00 UTC |
| 0984087 | Mohammad Akhlaghi | 08 August 2020, 21:48:53 UTC | Software tarballs saved as symlinks if already in filesystem Until now, if the software source tarballs already existed on the system they would be copied inside the project. However, the software source tarballs are sometimes/mostly larger than their actual product and can consume significant space (~375 MB in the core branch!). With this commit, when the software are present on the system, their symbolic link will be placed in 'BDIR/software/tarballs', not a full copy. Also, because the tarballs in software tarball directory may themselves be links, we use 'realpath' to find the final place of the actual file and link to that location. Therefore if 'realpath' can't be found (prior to installing Coreutils in Maneage), we will copy the tarballs from the given software tarball directory. After Maneage has installed Coreutils, the project's own 'realpath' will be used. Of course, if the software are downloaded, their full downloaded copy will be kept in 'BDIR/software/tarballs', nothing has changed in the downloading scenario. | 08 August 2020, 21:56:07 UTC |
| b3b4479 | Mohammad Akhlaghi | 18 June 2020, 17:01:33 UTC | IMPORTANT: New software versions (17 basic, 16 high-level and 7 Python) It was a long time that the Maneage software versions hadn't been updated. With this commit, the versions of all basic software were checked and 17 of that had newer versions were updated. Also, 16 high-level programs and libraries were updated as well as 7 Python modules. The full list is available below. Basic Software (affecting all projects) --------------------------------------- bash 5.0.11 -> 5.0.18 binutils 2.32 -> 2.35 coreutils 8.31 -> 8.32 curl 7.65.3 -> 7.71.1 file 5.36 -> 5.39 gawk 5.0.1 -> 5.1.0 gcc 9.2.0 -> 10.2.0 gettext 0.20.2 -> 0.21 git 2.26.2 -> 2.28.0 gmp 6.1.2 -> 6.2.0 grep 3.3 -> 3.4 libbsd 0.9.1 -> 0.10.0 ncurses 6.1 -> 6.2 perl 5.30.0 -> 5.32.0 sed 4.7 -> 4.8 texinfo 6.6 -> 6.7 xz 5.2.4 -> 5.2.5 Custom programs/libraries ------------------------- astrometrynet 0.77 -> 0.80 automake 0.16.1 -> 0.16.2 bison 3.6 -> 3.7 cfitsio 3.47 -> 3.48 cmake 3.17.0 -> 3.18.1 freetype 2.9 -> 2.10.2 gdb 8.3 -> 9.2 ghostscript 9.50 -> 9.52 gnuastro 0.11 -> 0.12 libgit2 0.28.2 -> 1.0.1 libidn 1.35 -> 1.36 openmpi 4.0.1 -> 4.0.4 R 3.6.2 -> 4.0.2 python 3.7.4 -> 3.8.5 wcslib 6.4 -> 7.3 yaml 0.2.2 -> 0.2.5 Python modules -------------- cython 0.29.6 -> 0.29.21 h5py 2.9.0 -> 2.10.0 matplotlib 3.1.1 -> 3.3.0 mpi4py 3.0.2 -> 3.0.3 numpy 1.17.2 -> 1.19.1 pybind11 2.4.3 -> 2.5.0 scipy 1.3.1 -> 1.5.2 | 08 August 2020, 21:44:23 UTC |
| d373993 | Boud Roukema | 08 August 2020, 15:58:03 UTC | Configuration fail if gfortran necessary, but not built or available When the host C compiler is used (either by calling '--host-cc' or on OSs that we can't build the GNU C Compiler), Maneage will also not build the Fortran compiler 'gfortran'. Until now, the './project configure' script would give a big warning about the need for 'gfortran' and the fact that it is missing, and would for 5 seconds, but it would continue anyway. For projects that don't need 'gfortran', this can be confusing to the users and for those that need 'gfortran', it means that a lot of time and cpu cycles are wasted compiling non-fortran software that are unusable in the end. With this commit, the 'need_gfortarn' variable has been added 'reproduce/software/shell/configure.sh', in a new part that is devoted to project-specific features. If it equals '0', then the 'gfortran' test (and message!) isn't done at all, but if it is set to '1', then the configure stage will halt immediately gfortran is not found and not built. The default operations of the core Maneage branch don't need 'gfortran', so by default it is set to 0. But 'gfortran' is necessary for all projects that use Numpy (Python's numeric library) for example. So if your project needs 'gfortran', please set this new variable to 1. As mentioned in the comments of 'configure.sh', ideally we should detect this automatically, but we haven't had the time to implement it yet. | 08 August 2020, 17:40:38 UTC |
| 32f3ba1 | Boud Roukema | 01 August 2020, 23:48:06 UTC | initialize.mk: accounting for no maneage branch One of the LaTeX macros reported by 'initialize.mk' is the git commit hash of the most recent 'maneage' branch that the project has been branched from. However, not all projects will retain the maneage reference. This can happen for example when people don't push the 'maneage' reference to their repository and then clone from their own repository to a second computer. Therefore, until now, in such situations, Maneage would break with an error. With this commit, in such scenarios, a place holder string is used instead, clearly highlighting that there is no 'maneage' reference. | 01 August 2020, 23:51:16 UTC |
| cbd4a41 | Boud Roukema | 31 July 2020, 17:04:08 UTC | OpenMPI build with slurm compatibility Prior to this commit, compilation of OpenMPI used the default OpenMPI choices of deciding which libraries should be used in relating to a job scheduler [1] (such as Slurm [2]). Given that the user on a multi-user cluster has to accept the sysadmin's choice of a job scheduler, the question of whether to (1) link with OpenMPI's own libraries (and increase the reproducibility of the science project) or rather (2) link with the sysadmin managed libraries (more likely to be compatible with the host's job scheduler), is an open question of which the best strategy for reproducibility needs to be debated and studied. In this commit, strategy (1) is adopted. The options '--withpmix=internal' and '--with-hwloc=internal' are added to the configure command. The working assumption is that the Maneage version of OpenMPI is likely to be modern enough to be compatible with the native job scheduler such as Slurm. Compilation without any 'pmix' option gave a fail in at least one case; it appears that an external pmix library was sought by the configure script. As of OpenMPI 4.0.1, the internal libevent library is used by default, so there appears to be no option to force it to be chosen internally. This commit also includes the option '--without-verbs'. This option removes a library related to "infiniband", "verbs", "openib" and "BTL"; this library appears to be deprecated. See [3], [4] for discussion. Please add feedback and discussion to the Maneage task about openmpi linking strategies (1) (internal) and (2) (external) at Savannah [5]. [1] https://en.wikipedia.org/wiki/Job_scheduler#Batch_queuing_for_HPC_clusters [2] https://en.wikipedia.org/wiki/Slurm_Workload_Manager - To avoid a name clash, 'slurm-wlm' is the metapackage in Debian for the client commands, the compute node daemon, and the central node daemon. An unrelated package 'slurm' also exists. [3] https://www-lb.open-mpi.org/faq/?category=openfabrics#ofa-device-error [4] https://www-lb.open-mpi.org/faq/?category=building [5] https://savannah.nongnu.org/task/index.php?15737 | 01 August 2020, 23:43:09 UTC |
| f9b5117 | Boud Roukema | 29 July 2020, 01:06:47 UTC | Clarify that Example 3 is not Poisson The previous version of 'README-popular-science.md' sounded like 'Example 3' was the usual expectation of noise, when we know nothing else about a random counting process - the Poisson point process. But that's not the case. This commit is intended to clarify the difference, at the cost of introducing words that may frighten some readers ("Poisson distribution"). | 29 July 2020, 01:06:47 UTC |
| 67a5e31 | Boud Roukema | 28 July 2020, 18:21:24 UTC | Insert ArXiv metadata ID: 2007.11779 Insert ArXiv ID in `reproduce/analysis/config/metadata.conf` . | 28 July 2020, 18:21:24 UTC |
| b3d8fea | Boud Roukema | 25 July 2020, 06:17:43 UTC | README-hacking.md: added new paper using Maneage (arXiv:2007.11779) Roukema+2020 (arXiv:2007.11779) is a newly published (as preprint) paper that uses Maneage, so it is being added to the list of published or submitted papers in 'README-hacking.md'. The Software Heritage URL sticks out way beyond the standard number of columns in the plain text form of the updated 'README-hacking.md' file, when rendered using markdown, it shouldn't look so bad. Also, see the related task https://savannah.nongnu.org/task/index.php?15736 (Raul+2020 should be Infante-Sainz+2020) for a suggestion of a more standard machine-readable format. It should be mentioned and emphasised to the reader that one should very carefully and obediently note and pay attention to the noteworthy fact that a few distracting words [1] such as "Note that" are removed in this commit. ;) [1] https://en.wiktionary.org/wiki/pontification | 27 July 2020, 23:11:54 UTC |
| d73c958 | Boud Roukema | 26 July 2020, 15:54:10 UTC | Fix up markdown for README headers Git repositories typically show README*.md files in markdown format [1]. This commit is intended to tidy the header sections of 'README-popular-science.md' and 'README.md'. [1] https://www.markdownguide.org | 26 July 2020, 15:54:10 UTC |
| 7ad5dec | Boud Roukema | 25 July 2020, 12:20:09 UTC | Add popular-level explanation - README-popular-science.md In this commit, the file 'README-popular-science.md' is added to give people a highly simplified explanation of the main idea of the paper. | 25 July 2020, 12:20:09 UTC |
| 8cca47d | Boud Roukema | 24 July 2020, 15:58:46 UTC | TODO: rm things done; v2 notes In this commit, some things on the TODO list that have alread been done are removed, and a list of items to do for v2 is started. | 24 July 2020, 15:58:46 UTC |
| 84e4345 | Boud Roukema | 23 July 2020, 09:48:53 UTC | Update poisson.tex md5sum and reenable verify-outputs A 3-significant-digit version of a phi_i value was added recently to the tex outputs into the build tex macro file 'poisson.tex'; this is needed for the Russia phi value of 10.35 or 10.4, which looks odd as 10 \times (10^{log10 uncertainty}). The updated m5sum is inserted into the default version of 'reproduce/analysis/make/verify.mk' (not yet into the developers' version), and verify-outputs is re-enabled in 'reproduce/analysis/config/verify-outputs.conf'. | 23 July 2020, 09:48:53 UTC |
| 71ae2a0 | Boud Roukema | 23 July 2020, 02:24:31 UTC | Add missing word to abstract | 23 July 2020, 02:24:31 UTC |
| 252cf1c | Boud Roukema | 23 July 2020, 00:53:04 UTC | Three significant figures for RU phi_i | 23 July 2020, 00:53:04 UTC |
| 1c0ae47 | Boud Roukema | 23 July 2020, 00:42:31 UTC | This should be the submitted version Mostly minor language edits of the text. The abstract length according to pdf cut/paste + wc is 250 words. | 23 July 2020, 00:42:31 UTC |
| c719f30 | Boud Roukema | 22 July 2020, 21:28:03 UTC | IN: +few more details; README.md overall software description | 22 July 2020, 21:28:03 UTC |
| 2fa9d5c | Boud Roukema | 22 July 2020, 17:36:56 UTC | Printable width error bars; PL values into paper | 22 July 2020, 17:36:56 UTC |
| ee1e371 | Boud Roukema | 22 July 2020, 15:52:38 UTC | Revert accidental commit of default 50 cpus | 22 July 2020, 15:52:38 UTC |
| 75ecd83 | Boud Roukema | 22 July 2020, 15:51:39 UTC | Fix checksum for WHO_vs_WP.dat for 2020-07-15 | 22 July 2020, 15:51:39 UTC |
| 5e061a2 | Boud Roukema | 22 July 2020, 15:40:47 UTC | Try to fix numthreads syntax | 22 July 2020, 15:40:47 UTC |
| 26588b2 | Boud Roukema | 22 July 2020, 15:36:27 UTC | Checksum update for WHO 2020-07-15; not yet checked | 22 July 2020, 15:36:27 UTC |
| 3712c41 | Boud Roukema | 22 July 2020, 15:31:59 UTC | Start checksums for WHO 2020-07-15; numthreads As of today 2020-07-22, more recent Wayback snapshots of the WHO data than 2020-07-15 are not available - probably there's some sort of long-term storage issue that involves a delay. This commit restores the 2020-07-15 archived URL and a checksum; other checksums will have to be fixed. The numthreads rule in 'poisson.mk' is adjusted (hopefully fixed) here. A user-level (subpoisson.conf) value should override the default value. | 22 July 2020, 15:31:59 UTC |
| b08f737 | Boud Roukema | 22 July 2020, 09:26:41 UTC | Merge branch 'subpoisson' of codeberg:boud/subpoisson into subpoisson | 22 July 2020, 09:26:41 UTC |
| 30c4e96 | Boud Roukema | 22 July 2020, 09:24:52 UTC | Update WHO Wayback archive to 20200722024239 The Wayback machine archive now seems to be stable for the 20200722024239 snapshot of the WHO dataset. This commit updates to this snapshot and sha512sum. | 22 July 2020, 09:24:52 UTC |
| 565454b | Boud Roukema | 22 July 2020, 09:06:16 UTC | Remove obsolete python import Subpoisson.py tries to import 'replace_pairs.py', which was used as a partial fix for the WHO data jumps/drops. This is obsolete since the higher quality data - the WP C19CCTF data - is now used. | 22 July 2020, 09:06:16 UTC |
| 0ab3169 | Boud Roukema | 22 July 2020, 03:14:23 UTC | Downdate two checksums for WHO 20200715 Since Wayback (and the WHO website too, I think) doesn't want to give the 20200721 version of the data file, here we give the checksums for the 20200715 version of their file. | 22 July 2020, 03:14:23 UTC |
| 76b49d2 | Boud Roukema | 22 July 2020, 02:49:21 UTC | TODO: git-less archive bug; Wayback volatility TODO list: Some of the scripts in the present set of *.mk and bash scripts use git; these will file in a pure snapshot that has no git information. So the 'git archive' snapshot risks leading to more user errors than the git bundle and is better not distributed by default. The Wayback machine is sometimes a bit volatile in terms of snapshots - probably something to do with managing huge amounts of files and backing them up and mirroring them properly. Right not the 20200721 snapshot of the WHO data set redirects to a 20200715 snapshot. Probably waiting a few hours may be enough, but meanwhile, here's the 20200715 snapshot - which may be safer to use since it seems to be stored properly and available. The verify checksums would have to be updated for this 20200715 snapshot. | 22 July 2020, 02:49:21 UTC |
| 91f12d2 | Boud Roukema | 21 July 2020, 21:50:16 UTC | Preparing for distribution: arXiv version; snapshot This commit adds a command for creating a git snapshot: the user should be able to reproduce the full project from the state of the current git commit. Suffixes are added to some of the .tar.gz output files to reduce ambiguity in what packet is intended for what purpose. The TODO list for submission guidelines is updated. | 21 July 2020, 21:50:16 UTC |
| 6bb7f36 | Boud Roukema | 21 July 2020, 21:36:44 UTC | Trivial fix to printf lines in paper.mk | 21 July 2020, 21:36:44 UTC |
| 3ae050d | Boud Roukema | 21 July 2020, 21:11:54 UTC | Update md5sum; EJE/Springer Fig/Table trivia The high-resolution calculation verification checksum is updated in this commit to take into account the cutoff N_plot_N_lim_lowest_phi passing through the pipeline. Trivia to get "Fig. number" and "Table number" closer to Springer/EJE style are implemented. | 21 July 2020, 21:11:54 UTC |
| b2af741 | Boud Roukema | 21 July 2020, 19:12:22 UTC | Text: subsubsections in discussion; N_i cutoff for cases curves The discussion section is reorganised in this commit to discuss the Finland case, which has low \phi_i and \psi_i but also low N_i, and to clarify the different subgroups of the general low \psi_i group. The selection of counts curves to display has a criterion added to exclude low-total-count countries, since these typically have closer to Poissonian counts. The summary line in the abstract and corresponding parameters in the pipeline have the "strongly sub-Poissonian" preferred models switched to a cutoff of \phi_i^{28} = 0.5, since 1.0 is not "strongly sub-". | 21 July 2020, 19:12:22 UTC |
