swh:1:snp:97876ef728a0cbdeeccc1febc25d1c11a87346b9

sort by:
Revision Author Date Message Commit Date
31e981b Merge pull request #4333 from aiidateam/release/1.3.1 Release `v1.3.1` 27 August 2020, 20:24:38 UTC
bed2014 Release `v1.3.1` 27 August 2020, 18:36:13 UTC
f9f6c9c `Runner`: close loop when runner stops if runner created it (#4307) If the loop is not closed, the file handles it managed during its lifetime might not be properly cleaned up, leading to file leaks. Therefore, if the loop is created by the `Runner` upon construction, it should also close it when the runner closes. It cannot be done for loops passed into the constructor, because they migth actually still be in use by other parts of the code. 27 August 2020, 18:32:21 UTC
856fc06 `ArithmeticAddParser`: attach output before checking for negative value (#4267) For the recent documentation revamp, the `ArithmeticAddCalculation` and `ArithmeticAddParser` were simplified significantly, by getting rid off as much as the unnecessary code, because they are being literally included as example for the basic how-to create a code plugin. A part that was removed was the `settings` input node that allowed to change the behavior of the parser and allow negative sums instead of it returning an exit code. This, however, in turn cause the Reverse Polish Notation tests on Jenkins to fail. Since these tests are not required to pass, the changes were merged without a fix. In the scope of the RPN tests, negative sums are fine, which anyway is just a mechanism to introduce some kind of failure mode for demonstration purposes. To fix this, without making the logic of the parser more complex, we simply change the order of attaching the output node and performing the final check. Since the RPN workchains only check if the output node is there and do not care about the exit status of the calculation, they will happily continue and the code of the parser keeps the same complexity. 27 August 2020, 16:59:14 UTC
52bd5ad Merge pull request #4211 from aiidateam/release/1.3.0 Release `v1.3.0` 01 July 2020, 15:33:06 UTC
c20c868 Release `v1.3.0` 01 July 2020, 15:09:17 UTC
87f7fe2 Update `AUTHORS.txt` and add copyright to new files Moved Chris J. Sewell from contributors to the current active team and added new member Marnik Bercx. 01 July 2020, 11:22:39 UTC
862fa20 `BaseRestartWorkChain`: do not assume `metadata` exists in inputs (#4210) The contract of the base restart work chain is that the `setup` will define the `self.ctx.inputs` attribute, however, it is not defined what its contents should be. The `run_process` method was unjustly assuming that the `metadata` key would be set to a dictionary, which is not always the case. In this fix, we simply check for its existence and create the dictionary if it doesn't yet exist. 01 July 2020, 11:10:31 UTC
d582b0f Updates to the setup procedure for building and installing the package (#4204) Overview of the changes in this commit: * Limit `find_packages` in `setup.py` to only the `aiida` package * Unvendor the `fastentrypoints` dependency * Remove files from the `MANIFEST.in` The `setup.py` uses the `setuptools.find_packages` function to determine all packages that need to be included automatically. Since we don't specify any rules, it finds all packages in the current directory. This means that `docs`, `tests`, and `utils` are currently also installed. To avoid this, we use the `include` keyword to only find `aiida` package and all of its sub-packages. We have been using the `fastentrypoints` dependency as a hack to ensure that console scripts that are installed through entry points, such as `verdi` in our case, have a fast start up time. This is critical for tab-completion. The reason this hack is necessary is because setuptools, which creates the wrapper script, uses `pkg_resources` by default, which is really slow and adds 200 ms of overhead. The `fastentrypoints` monkey patches `setuptools` to generate a faster wrapper script. Note that none of this applies if the package is installed through a wheel. For this reason we have been vendoring `fastentrypoints` but we now move it to a build requirement in `pyproject.toml`, which ensures that it can be imported whenever the package is installed through `pip`. This incurs a slight overhead of having to install `fastentrypoints` even when the package is installed as a wheel, but we no longer have to vendor it. Finally, we remove the following files from the distribution by removing them from the manifest: * The `*.aiida` export files which had already been moved to `tests/` * The `script.py.mako` which is only needed for devs to create migration files for the SqlAlchemy backend * The `*.png` files from the `docs` directory * The files in `utils` which are only necessary for developers In addition to removing the images from the `docs/` we change the readme to use a permalink for the AiiDA logo from the website instead of the relative path to the docs, such that the logo also displays correctly on, among others, PyPI. 30 June 2020, 08:00:57 UTC
9a32cec Docs: update ReadTheDocs configuration and build process (#4203) The `.readthedocs.yml` configuration file is updated to ask RTD to also install the `aiida-core` package. This is necessary for the entry points to be properly installed, which is turn is necessary for the automated API documentation to be successfully built. This was always failing but went unnoticed because the nitpick option was not enabled. This is also changed such that failures like these will fail the build from now on. Due to the package now being installed through `pip` it is no longer necessary, nor preferable, to hack it into the `sys.path` in the source configuration file. It also allows to remote the change of the previous commit that manualy called `reentry scan`, since this was not the actual reason for the entry points not being available. Since the package is now installed directly through `pip` the separate `docs/requirements_for_rtd.txt` file is now obsolete and it is removed along with all the pre-commit hooks and scripts that automatically updated it. Finally, delete folder `aiida/sphinxext/tests` which became obsolete when all tests were moved to the `tests` folder. 28 June 2020, 11:59:06 UTC
bb2098e Add reentry scan to Sphinx build (#4201) This is necessary for the API docs to be correctly built. 26 June 2020, 15:19:25 UTC
92f2dc6 Update dependency requirement `kiwipy~=0.5.5` (#4198) Note that `kiwipy==0.5.4` will break existing installations. 24 June 2020, 15:10:16 UTC
a97f725 `CalcJob`: only use MPI procs per machine if scheduler supports it The `CalcJob.presubmit` method was always adding the resource keyword `num_mpiprocs_per_machine` if it was not already (implicitly) specified and the computer defined a default. However, this keyword is not valid for all schedulers, and for example, the SGE and LSF scheduler will raise if it is passed. Here we add the `Scheduler.preprocess_resources` method, which can be called and will properly take the type of job resources into account when deciding whether `num_mpiprocs_per_machine` should be defined. 23 June 2020, 13:26:12 UTC
aa9a2cb Refactor the scheduler base class and data structure classes The `Scheduler.get_valid_schedulers` method is deprecated as the `aiida.plugins` module is reserved to inquire about installed plugins. Furthermore, it is properly marked as an abstract class which allows to remove the explicit `raise NotImplementedError` from methods that are abstract and have to be implemented by subclasses. Finally, there is some minor cleaning up of the code and docstrings. The implementation of the `validate_resources` method of the subclasses of `JobResource` has been improved and significantly simplified in the case of `NodeNumberJobResource` making the logic a lot more readable. Tests are added for the `ParEnvJobResource` job resource class which was completely untested. 23 June 2020, 13:26:12 UTC
00c38dd `CalcJob`: move job resource validation to the `Scheduler` class This is necessary because the resource validation is scheduler dependent. Up till now, the `CalcJob` defined a validator on the entire input namespace that validates the `metadata.options.resources` input. Specifically, it demanded that the `num_machines` keyword was defined, which is indeed a requirement for the `NodeNumberJobResource`, used for schedulers like SLURM, however, this field doesn't even make sense for schedulers like LSF and SGE that use the `ParEnvJobResource` type of job resources. The solution is to delegate the validation of the resources to the scheduler which in turn delegates it to the `JobResource` class that it uses. The validation used to happend on construction of the job resource instance, but is factored out to the `validate_resources` classmethod. This allows it to be called without having to construct an instance which is more efficient. Finally, the signature of the `CalcJob` input validators is changed and no longer raise an `InputValidationError` but instead return an error message as the interface of `Port.validator` requires. The port itself will detect if an error message is returned and raise a `ValueError` with all the relevant error messages. 23 June 2020, 13:26:12 UTC
f8be9de Only color directories in `verdi node repo ls --color` (#4195) The help string says that the `--color` flag can be used to color directories different from files to easily distinguish them, except the implementation colored all files, regardless of type. 23 June 2020, 10:19:23 UTC
af153c9 `CalcJobResultManager`: fix bug that broke tab completion (#4187) For tab-completion to work properly, the `__getattr__` method need to properly return an `AttributeError` if the corresponding attribute does not exist, however, it was accidentally raising a `KeyError` since the implementation caught an `AttributeError` but the dereferencing was using `__getitem__` so would never raise an `AttributeError`. Likewise, the `__getitem__` implementation was incorrectly returning an `AttributeError` instead of a `KeyError` making it behave different from any other mapping-like data structure. 23 June 2020, 09:42:06 UTC
3339fd5 Apply the pre-commit hooks with new configuration 21 June 2020, 17:20:02 UTC
cb5bd63 Update the pre-commit configuration * Drop `prospector`: it was only being used to run `pylint` but in that case it is much simpler to just run it directly as `prospector` has often given issues with incompatible dependencies * Add more hooks from `pre-commit` itself: - end-of-file-fixer automatically add new lines to end of files - fix-encoding-pragma automatically add the utf-8 encoding pragma - mixed-line-ending automatically deal with line break characters * Run `pylint` and `yapf` directly from the remote repository Since `pylint` still needs to be run with `system` such that it can see all dependencies and not throw false import errors, we still need to keep it in the `pre-commit` extras requirements. * Renamed extra requires categories to match other projects dev_precommit -> pre-commit testing -> tests 21 June 2020, 17:20:02 UTC
e9d3153 Add minimal `mypy` run to the pre-commit hooks. (#4176) Currently, only two files are checked: `aiida/tools/groups/paths.py` and `aiida/engine/processes/calcjobs/calcjob.py`. The option `follow_imports` is set to `skip`, because there are a lot of issues in other files, mostly because these lack type hints. From here, we can continue expanding the type hints one file at a time: * Add the file to `files` in the `mypy` pre-commit hook * Fix all issues that `mypy` reports Co-authored-by: Chris Sewell <chrisj_sewell@hotmail.com> 20 June 2020, 10:58:55 UTC
41f5d5d Restore deleted images for `README.md` and update acknowledgements (#4186) Commit `94a40ae4768e7bcefce8409bed50ec9487b0e4c2` accidentally removed the AiiDA logo because it was replaced with a logo specifically designed for the docs. However, the logo was also being used by the `README.md`. The logo is restored and renamed for consistency. In addition, the acknowledgments are simplified as they are in the docs, by removing the images of logo's of the supporting agencies and referring to the AiiDA website for a complete list. 19 June 2020, 14:22:15 UTC
112f4b8 `NodeTranslator`: do not assume `get_export_formats` exists (#4188) The `get_all_download_formats` will try to call `get_export_formats` on all the known plugin types. However, it is not guaranteed that all plugins implement this method. This was properly considered in the case that `full_type` is not None, however, in the other case the `AttributeError` was not being caught causing the REST API to except. 19 June 2020, 14:01:07 UTC
3a57c03 Raise when calling `Node.objects.delete` for node with incoming links (#4168) This guard was already in place for a node that has outgoing links but if a node had only incoming links, the method would happily delete it. This means that one could delete output nodes for example as long as they had not been used as inputs. Note also that this would leave the link in place as the deletion is not cascaded automatically. Deletion through the objects collection currently cannot deal with automated cascading rules on links and nodes. These rules are implemented in the ORM itself through the graph traversal rules for deletion. Nodes that are not isolated should therefore only be deleted through the utility function that makes sure to leave the resulting provenance graph in a coherent state. 19 June 2020, 09:00:53 UTC
3a5853e `verdi status`: add `--no-rmq` flag to skip the RabbitMQ check (#4181) This makes it possible to use the command to check whether the system is up to at least query the database, for which RabbitMQ is not necessary. The behavior of `verdi status` is also adapted to no longer return a non-zero exit code if the daemon is not running. The daemon not running is not the same as not being able to connect to the Postgres database or the RabbitMQ message broker. So it shouldn't return a non-zero exit code either. If the user wants to poll this, the command `verdi daemon status` should be used instead. Co-authored-by: Sebastiaan Huber <mail@sphuber.net> 18 June 2020, 21:46:41 UTC
94a40ae Docs: improved landing page, custom docs logo and updated acknowledgments (#4179) * Change the documentation logo from the normal AiiDA logo to one that is designed specifically for the documentation. It includes under it the word documentation in the same Futura font. This makes it clearer that this logo just references the docs homepage and not the AiiDA web page, which could be the expected behavior with the normal logo. * Updating the description of AiiDA (homepage + about page) to be in sync with the recent updates AiiDA's website * Remove acknowledgement logos and only explicitly mention MARVEL and MaX, with a link pointing to the AiiDA website for a complete list. * Fix typo in the word Kerberos Co-Authored-By: Andrius Merkys <andrius.merkys@gmail.com> 18 June 2020, 18:56:13 UTC
b807740 Increase the default for `runner.poll.interval` config option to 60 (#4150) This option is used to define the `poll_interval` argument of all `Runner` instances that are created to run all processes. It determines the interval with which the state of a subprocess, that a `WorkChain` is waiting on, is checked whether it is terminated. If that is the case, a callback is called which will signal to the `WorkChain` that it can continue. This polling is a backup mechanism in case the broadcast by the process when it terminates is missed by the caller, which would cause it to wait indefinitely. The original default of 1 was causing unnecessary load on the CPUs as well as the database that each time had to query for the process state. When running many calculations this would spin the CPUs noticeably. Since this is supposed to be a fail-safe mechanism and it should only be required rarely, it is fine to increase the time significantly to reduce the load. 18 June 2020, 09:24:00 UTC
f61dd19 Docs: update `sphinx-copybutton` dep (#4182) The new version allows to specify a regex for the configuration option `copybutton_prompt_text`, which will allow to target multiple markers. This way the copy button will now copy snippets that start with a dollar sign, corresponding to shell commands, or those starting with `In [5]:` and ` :` corresponding to iPython inputs. 18 June 2020, 08:05:59 UTC
14c868a `WorkChain`: guarantee to maintain order of appended awaitables (#4156) Intuitively, one would expect the order of awaitables in the context once they have been resolved, to maintain the same order with which they had been inserted using the `append_` construct in the `to_context` call. However, this was not the case, as the resolved awaitables were just appended to the target container in the order that they resolved. To correct this behavior, once the awaitable is inserted, it is already inserted in the context, where its resolved value should be placed in the future. This allows the `resolve_awaitable` method to find the correct place and simply swap out the awaitable, that has been serving as a placeholder, with the resolved value. 18 June 2020, 07:40:17 UTC
1af57cc Reinstate broadcast subscription of workchain for child processes When a `WorkChain` step submits sub processes, awaitables will be created for them. Only when these awaitables have been resolved, meaning the subprocesses, have terminated, can the workchain continue to the next step. The original concept was, for each awaitable, to schedule a callback once the process had reached a terminal state. The callback was supposed to be triggered by having the runner add a broadcast subscriber that would listen for state changes of the sub process. As a fail-safe, a polling mechanism would also check periodically just in case the broadcast message would be missed and prevent the caller from waiting indefinitely. However, the broadcast subscriber was never added and so the system relied solely on the polling mechanism. This completely undermines the benefits of having an event-based mechanism, so in this commit the `Runner.call_on_process_finish` now also registers the broadcast subscriber. Note that the affected code had some references to `calculation` which has been generalized to `process`, since this also applies to workflows that might waited upon. The `CalculationFuture` has been renamed to `ProcessFuture` in similar vein. It is currently not used, but it could have been used for the problem that this commit solves, so it has been decided to leave it in for now and not remove it entirely. 17 June 2020, 15:01:24 UTC
56a4cc8 Wrap communicator of `Runner` in `plumpy.LoopCommunicator` This is necessary because otherwise subscribers that use the runner's communicator and receive a message will execute the callback on the communication thread. This should be prevented for two reasons. Firstly, it will cause the business logic to be executed on the communication thread which can cause it to fail to respond in time to heartbeat requests from RabbitMQ when it is under heavy load. Secondly, the communication thread does not share the same session as the main thread. This can cause problems when the callbacks query the database as it can be out of sync with the session of the main thread. For Django this will most likely go by unnoticed, but SqlAlchemy will except. Since the communicator of the runner is now wrapped, the process launcher of the daemon runner no longer has to be added as a task subscriber through a callback, but the `add_task_subscriber` can be called directly and the `LoopCommunicator` will take of everything. 17 June 2020, 15:01:24 UTC
f9d66b4 `SshTransport.gettree`: allow non-existing nested target directories (#4175) The `gettree` method would raise an `OSError` for the `SshTransport` if the target local path contains intermediate subdirectories that are not created beforehand. The same would work without problem for the `LocalTransport` plugin. It makes sense to just create non-existing directories along the way, so the `os.mkdir` command that was used originally to create the target directory is replaced with the method `os.makedirs` setting `exists_ok=True`. 17 June 2020, 13:36:33 UTC
216b4ec Docs: remove `readthedocs-sphinx-search` plugin (#4180) There seem to be unresolved UX issues, at least with the sphinx template we are now using. We may consider reintroducing the extension once these have been resolved. 17 June 2020, 12:11:33 UTC
ee822ff Docs: various small improvements (#4164) * Customize footer to link to supporting projects * Fix logo layout * Shorten descriptions of blocks * Add AiiDA 1.0 arXiv publication reference 16 June 2020, 11:08:45 UTC
84e49fe Update tutorial and how-to guides to literal-include for code snippets The basic tutorial and how-to guides on implementing `CalcJob` and `Parser` plugins, as well as how-to write work functions and work chains have been updated to `literal-include` the implementation of the actual plugins that ship with `aiida-core`. This has the advantage that the code that is shown in these crucial parts of the documentation is guaranteed to work with each release as the plugins themselves are tested in the unit test suite. The down side, however, is that we do not always want to show everything from the entire implementation at any point, so we have to resort to specifying line numbers. This is fine, except that it runs the risk that the line numbers run out of sync if the implementation is changed and automatically detecting this is difficult. Nonetheless, the other solution of just duplicating the code in the documentation has the same problem, and in addition the code is not easily tested. Tests *could* be added, but given that testing these plugins is not trivial, the required additional code and infrastructure would be more of a burden than the current solution. Finally, by using the `:pyobject:`, `:start-after:` and `:end-before:` options as much as possible, depending on explicit line numbers is also reduced. Currently, only a snippet in `howto/codes.rst` on exit codes in parsers relies on explicit line numbers. That is why the implementation in `aiida.parsers.plugins.arithmetic.add` has a comment with a warning that if edited, the snippet in the documentation should be checked for consistency. 16 June 2020, 07:14:13 UTC
9943ec4 Simplify `ArithmeticAddCalculation/ArithmeticAddParser` implementation The purpose of these basic `CalcJob` and `Parser` plugins is for easy integration testing, but also to serve as an example for the documentation and tutorials on how to implement these plugin types. To prevent code duplication and make sure that the code that is presented in the documentation is actually tested, we want to use this actual implementation there by literal-including it. This means that the code has to be as simple and minimal as is possible, within reason and by maintaining correctness for testing purposes. That is why we get rid of unnecessary utility methods and try to implement the code as straightforward as possible. Since the code is simplified it fits well into a single method anyway. 16 June 2020, 07:14:13 UTC
1f79416 Docs: add troubleshooting section for RabbitMQ (#4146) 16 June 2020, 06:01:22 UTC
5f486e3 add improved search using rtd-sphinx-search (#4111) includes "search as you type" https://github.com/readthedocs/readthedocs-sphinx-search Co-authored-by: Sebastiaan Huber <mail@sphuber.net> 11 June 2020, 08:15:35 UTC
c817021 Docs: update the MaX acknowledgment (#4161) Add the grant reference for the new 2018 - 2021 program in the README.md and the index page of the documentation. 09 June 2020, 16:51:58 UTC
4b0a7ee Add repository issue templates (#4148) 09 June 2020, 16:22:10 UTC
5b89d6f Fix bug in `BaseRestartWorkChain.inspect_process` (#4166) If at the end of the method, `last_report` is defined, meaning at least one handler returned a report, it should return the corresponding exit code, except it was erroneously accessing that of the `report` variable, which could very well be `None` if the last called handler returned nothing. 09 June 2020, 12:13:08 UTC
6f029df Fix the unhandled failure mechanism of the `BaseRestartWorkChain` (#4155) Any `BaseRestartWorkChain` will fail if the sub process fails twice in a row without being handled by a registered process handler. To monitor this the `unhandled_failure` context variable is used, which is set to `True` once a failed process was not handled. It should be unset as soon as the next process finishes successfully, or the failed process is handled. The reset was there, but was incorrectly setting the context var to `True` instead of `False`, which went unnoticed due to a lack of tests. Tests are now added, as well as for the case where the sub process excepted or was killed. 09 June 2020, 10:01:13 UTC
a0e39c6 Docs: Add release version pinning to conda install instructions (#4147) Automatically adds the current release version to the conda install instructions. Adding the version explicitly significantly improves the performance of the conda environment solver. It might be possible to remove the explicit version in the future pending improvements of the solver implementation. 05 June 2020, 15:42:43 UTC
f112cd9 `QueryBuilder`: fix type string filter generation for `Group` subclasses (#4144) The `aiida.orm.querybuilder.get_group_type_filter` function that is responsible for generating the correct filters for the `type_string` when a `Group` or subclass of it is appended, contained a bug. To generate the correct type string, it should strip the `group.` prefix from the provided ORM classifier. This was incorrectly done with the string method `lstrip`, which does not strip an entire prefix as is, but will strip any characters that are matched starting from the left. This would cause extra characters to be erroneously stripped if the rest of the type string started with any letters in the sequence `group.`. For example `group.pseudo.family` would be stripped to `seudo.family`. The solution is to use the first N characters where N is the length of the prefix. 03 June 2020, 16:17:52 UTC
690025e Highlight nodes in provenance graph (#4081) The provenance graph produced by verdi node generate can often be difficult to navigate, so this acommand line option is introduced to provide the users with an easy way to quickly identify nodes of specific classes. * The origin node is back to being highlighted in red by default. * The classes to be highlighted are identified by their labels in the provenance graph. * The highlighting is performed by "muting" of all other classes (showing them without colour). Co-authored-by: Chris Sewell <chrisj_sewell@hotmail.com> 03 June 2020, 12:34:07 UTC
2a0d11e Add support for process functions in `verdi plugin list` (#4117) So far, only actual `Process` subclasses were supported, however, process functions can be turned into a `Process` subclass just as well. The wrapped function gets a new attribute `spec`, which when called returns the `ProcessSpec` that is dynamically built based on the function signature of the decorated function. 03 June 2020, 09:31:25 UTC
e9a3e95 Enable loading config.yml files from URL for ceratin `verdi` commands (#3977) In the view of the plans of starting online repositories that maintain yaml configuration files for computers and codes, it becomes very useful to be able to directly import a configuration file from a URL. Version 0.6 of the `config-file-option` now respects the `type` parameter, and file handles can be passed directly to the configuration file provider. This allows handling opening files locally and from URLs on the same footing. 02 June 2020, 18:05:35 UTC
a427699 Merge pull request #4141 from aiidateam/docs/re-enable-ci Docs revamp 02 June 2020, 14:23:28 UTC
8e69062 Docs: final changes before merging the revamp * Update autogenerated `verdi` docs. * Fix `docs_path` in `docs/source/conf.py`. * Enable `warnings turn into errors` for docs-build. * Remove obsolete page `configuration`. * Move some unresolved references into todo-items. Sections `how-to:codes:transport` and `how-to:codes:scheduler` are not yet written. * Replace `sphinx.ext.imgmath` with `sphinx.ext.mathjax`. This way, latex is no longer required for documentation builds. * Add temporary fix for `pydata-sphinx-theme` issue #193. * Add AiiDA team twitter account to docs conf `html_theme_options` Co-authored-by: Chris Sewell <chrisj_sewell@hotmail.com> 02 June 2020, 13:54:05 UTC
8c25cfe Convert all stubs of missing sections into todo items. (#4137) Using the 'todo::' directive. 02 June 2020, 12:11:15 UTC
f472dba Docs: final touches (#4135) * Exclude not yet migrated documentation source files. * Add pandas intersphinx link. * Remove empty verdi_user_guide.rst. * Fix sphinx references by either updating or removing them. * Move downloadable files into include subfolder. * Inline unreferenced file about moving PostgreSQL in "Tuning performance", * Fix various broken links and references. 29 May 2020, 19:19:12 UTC
39c8671 Docs: Move section about log-level to new troubleshooting page. (#4131) 29 May 2020, 19:19:11 UTC
908e169 Docs: add "How To - Visualising graphs" section (#4138) Mostly moved content but some additional changes: * Small hotfix to the code that was broken accidentally in 47cfe34 * Copied the example export file to the docs since it was moved from the main package to the test folder making it unreachable for the documentation 29 May 2020, 19:19:11 UTC
6c093e4 DOCS: Adopt pydata-sphinx-theme (#4129) * Add top level indexes. * Adopt pydata-sphinx-theme. * Fix CSS of top bar link icons. 29 May 2020, 19:19:11 UTC
bb71f83 Docs: Add advanced querying section to database topics (#4132) Docs: Add advanced querying section to database topics * Add the more advanced elements of the QueryBuilder to the topics section on databases. * Add the reference tables for the operators, relationships and cast types. * Add the explanation of the queryhelp. 29 May 2020, 19:19:11 UTC
a35392d Docs: add "How To - Interfacing external codes" section (#4093) It is now divided into how-to interface codes and how-to parse outputs. The parse output section also contains a sub-section on how to handle errors during parsing. Co-authored-by: Sebastiaan Huber <mail@sphuber.net> 29 May 2020, 19:19:11 UTC
0d6756d Docs: add "How To - Writing workflows" section (#4112) Co-authored-by: Sebastiaan Huber <mail@sphuber.net> Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:19:11 UTC
c8ccc9f Docs: add a Frequently Asked Questions page (#3966) Add a Frequently Asked Questions page. Currently, the new FAQ page mostly focuses on common problems that users encounter and direct either towards the mailing list or create issues for. This FAQ page should become the central place to redirect to in case of these common problems. Besides the FAQ page itself, there are some other minor changes: * Improve `update installation` instructions to include the necessity of running `reentry` scan, and to be even more clear that the daemon should absolutely be stopped before upgrading, after having finished all running processes first. * Include link to FAQ page in the README * Remove outdated section from the Troublehooting section, since it was about Postgres versions that are no longer supported. 29 May 2020, 19:19:11 UTC
8363d99 Docs: add "How To - Grouping nodes" section (#4113) This moves in existing data largely as it was, just adapting it to the new style guidelines, and rephrasing a few parts. 29 May 2020, 19:19:11 UTC
bffaebd Docs: add brief intro text to "About" page and link from lading page. (#4127) 29 May 2020, 19:19:11 UTC
e06ebbf Docs: How to organize data - Group path (#4126) Add documentation for 'GroupPath'. Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:19:10 UTC
281a694 Docs: Revamp Intro/Install (#4060) Revision of all sections within the first chapter. Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:19:10 UTC
a675946 Docs: add the "How To - Finding and querying for data" section (#4080) Move in the sections on the `QueryBuilder` to the How-to section called "Finding and querying for data" The original text was too extensive for the purposes of the How-to and so it has been split in two. The detailed information on advanced queries will be added in another PR in the section "Topic - Database". The text for this How-to has been adapted from the original text with significant changes in the structure and simplifying the examples. Here wherever possible they use the `ArithmeticAddCalculation` for consistency with other sections and tutorials. The new structure is as follows: * Selecting entities * Retrieving results * Filters * Relationships * Projections 29 May 2020, 19:18:29 UTC
b746398 Docs: add the "How To - Running external codes" section (#4114) Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> Co-authored-by: Sebastiaan Huber <mail@sphuber.net> 29 May 2020, 19:18:29 UTC
9942b14 Docs: add the "How To - Launching a predefined workflow" section (#4087) Co-authored-by: Sebastiaan Huber <mail@sphuber.net> Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:18:29 UTC
704d49a Docs: Add basic tutorial (#4070) Add a tutorial to teach new users basic AiiDA concepts. Co-authored-by: Chris Sewell <chrisj_sewell@hotmail.com> Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:18:29 UTC
9e998d4 Docs: How to manage multiple users (#4109) Currently, we state that multiple user setups for single AiiDA instances are not officially supported, and until we do, we do not describe the ways that people have used multiple users. 29 May 2020, 19:18:28 UTC
04bf65b Docs: Add How-Tos to "Managing installation" (#4074) Add how-to sections for: how-to:tune performance (fixes #4002) how-to:update installation (fixes #4003) how-to:backup (fixes #4004) how-to:installation:running-on-supercomputers Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> Co-authored-by: Nicola Spallanzani <nicola.spallanzani@max-centre.eu> 29 May 2020, 19:18:28 UTC
eebde41 Docs: Adding How-to delete data (#4110) Adding How-to delete data. Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:18:28 UTC
48106ec Docs: Move plugin system docs (#4108) Move (& update) documentation on internal AiiDA plugin system to the right place. Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:18:28 UTC
0c95468 Add the `sphinx-panels` and `sphinx-copybutton` plugins (#4107) Remove the internal custom `accordion` directive that was added for the `docs-revamp` branch and replace it with the `sphinx-panels` plugin. The `sphinx-copybutton` extension is added to allow users to easily copy code snippets to the clipboard. 29 May 2020, 19:18:28 UTC
a589108 Move to CircleCI for CI Documentation builds (#4099) Until Github actions also allow these types of build artifacts. 29 May 2020, 19:18:28 UTC
2ea1f76 Docs: add "How to on plugins" (#4078) Add "How to package plugins": * add "Creating a plugin package" * add "Adding entry points" * add "Testing a plugin package" * add "Documenting a plugin package" * add "Publishing a plugin package" Also start adding the "plugins" topical guide. Co-authored-by: Carl Simon Adorf <carl.simon.adorf@gmail.com> 29 May 2020, 19:18:28 UTC
14ee892 Docs: add "How to - Configuring your installation" (#4068) This section describes how multiple instances of AiiDA can be isolated on the same machine, how tab-completion can be activated and how the `verdi config` command can be used to configure profile options. Except for the `verdi config` section, which is completely new, the other content has been adapted from existing documentation. 29 May 2020, 19:18:28 UTC
1f2c48f Docs: add "How to manage your installation - Managing profiles" (#4066) This is a new section that is intended to describe the function of profiles and the various commands that can be used to work with them. Note that the explanation on the purpose and details of profiles is kept to a minimum as this is a How-to guide and is merely intended to help users with goal oriented problems. The section quickly explains the various profile commands: * verdi profile list * verdi profile setdefault * verdi profile show * verdi profile delete Information on how further configuration through profiles is reserved for the next how-to section "Configuring your installation". 29 May 2020, 19:18:28 UTC
e530368 Remove sphinx_cheatsheet and visual_aids (#4069) Removal of the SPHINX cheat sheet and visual aids page. Instructions for creating graphs has been moved to: https://github.com/aiidateam/aiida-core/wiki/Graph-Easy 29 May 2020, 19:18:28 UTC
ff11ad8 Docs: move in section "Topics - Provenance" (#4058) The content is largely untouched and mostly moved in to its new place in the scaffolding as is. There were slight changes in link labels and header characters to adhere to new style guide rules. Also fix various incorrect label references. 29 May 2020, 19:18:28 UTC
fec8406 Docs: move in section "Topics - Processes" (#4048) The content that was envisioned for this section was originally divided over the "Concepts" and "Working" sections, which were itself roughly divided in information on "Processes", "Calculations" and "Workflows". The original idea was to nest the material on calculations and workflows within the global processes topic, since in the code the former are also sub classes of the latter. However, since there are also documentation on processes in general, I was not able to make this hierarchy work. Instead, "Calculations" and "Workflows" are now individul topics on the same level as "Processes". Each of these topics are roughly subdivided in a "Concepts" and "Usage" section, where the content from the latter came from the original "Working" directory. The concepts explains the concepts of the topic, where the usage really exposes and explains the API with all the nitty gritty details. 29 May 2020, 19:18:27 UTC
0560faf Docs: add the "How to add support for custom data types" section (#4049) 29 May 2020, 19:18:26 UTC
13e34fa Docs: How to work with data - Importing data (#4045) 29 May 2020, 19:18:25 UTC
7062bc1 Docs: add the "Topics - Command line interface" section (#4044) This content already existed and has been largely moved as is. Just the style was adapted to match that of the new documentation, such as link labels, indentation and header styles. 29 May 2020, 19:18:25 UTC
8e9ae68 Docs: move auto-generated CLI docs and REST API reference The REST API reference is not yet auto-generated, like the command line reference section, but is moved to the "Reference" section nonetheless. Co-Authored-By: ramirezfranciscof <ramirezfranciscof@users.noreply.github.com> 29 May 2020, 19:16:57 UTC
92551e7 Docs: add scaffolding for the "Reference" and "Internal Architecture" section Co-Authored-By: ramirezfranciscof <ramirezfranciscof@users.noreply.github.com> 29 May 2020, 19:16:28 UTC
a12575c Docs: add scaffolding for the "Tutorials" section (#4025) With reference to the GitHub issues Co-authored-by: Sebastiaan Huber <mail@sphuber.net> 29 May 2020, 19:16:28 UTC
a427743 Docs: scaffolding for the "Intro" section (#4011) The intro section contains the landing page with information about motivation and scope of the AiiDA software, as well as an overview of the documentation structure with links to other sections. This intro section also contains the instructions for how to install and setup AiiDA. 29 May 2020, 19:16:28 UTC
de2f8e3 Docs: add scaffolding for the "Topics" section (#4027) The Topics section will host documents that go in more-depth about certain concepts and will give detailed information about certain parts of the API. 29 May 2020, 19:16:28 UTC
535b335 Docs: add links to referenced GH issues (#4012) 29 May 2020, 19:16:28 UTC
3672e06 Docs: flesh out skeleton of "How-to Guides" This section is subdivided in 5 main categories: * How to run external codes * How to run multi-step workflows * How to work with data * How to manage your installation * How to package plugins The naming has been purposefully kept free of AiiDA specific jargon as these guides need to be quickly findable by relatively new users that after having run through a tutorial want to quickly know how to accomplish certain specific goals. Each section has its own document that will be presented as its own page with a number of subsections that present guides for specific goals. The sections and subsections are all created with the corresponding link labels for cross-referencing. 29 May 2020, 19:16:28 UTC
f4a3c29 Remove -W flag from docs build CI step This flag turns warnings into errors, but since we are reorganizing the docs, we temporarily remove this. Once the work is completed, we can enable it once more. 29 May 2020, 19:16:28 UTC
091d920 Setup very basic skeleton for docs revamp The new skeleton contains the following top level sections * Getting started (into) * Tutorial (tutorial) * How-To Guides (howto) * Topics (topics) * Reference (reference) * Plugins (plugins) * Development (development) The value between parentheses is the name of the directory in which the relevent documents should be placed. Currently these new folders only contain a single `placeholder.rst` index file. From here, we will slowly start to move existing documentation from their old files into the new locations, while correcting and improving the text. The Plugins section may eventually be split up and absorbed in the other sections if that makes more sense. 29 May 2020, 19:16:28 UTC
10a5cdb Add tests for new `MultiplyAddWorkChain` and `add_multiply` plugins Add also tests for the `ArithmeticAddCalculation` which is not new but was up till now untested. Now that this will start to be used heavily in tutorials and documentation for teaching purposes, we should make sure that it is well tested. Here we add an integration test and a unit test for the `prepare_for_submission` method. 28 May 2020, 15:53:05 UTC
62a108a Add arithmetic workflows and restructure calculation plugins Refactor the calculations package to remove the plugin directory. The calculation job plugins that ship with `aiida-core` were being stored in `aiida.calculations.plugins` but the `plugins` submodule is unnecessary so it is removed. The old import is kept for now and will emit a deprecation warning. The `ArithemeticAddCalculation` is updated to simplify its usage as well as to make it easier to understand the implementation, which is important as it is used consistenly as an example in tutorials and the documentation. The following changes are applied: * `metadata.options.resources` has a default * Unnecessary utility methods are removed to reduce amount of code * `calc_info.cmdline_params` are replaced with `calc_info.stdin_name` * Code will now work directly with bash instead of custom `add.sh` script Finally, two workflows implementations are added for testing and demonstration purposes. The `add_multiply` work function and the `MultiplyAddWorkChain`. These will be used extensively in the how-to guides of the documentation and the tutorials. 28 May 2020, 15:53:05 UTC
5e02e16 Ensure `verdi group show --limit` respects limit even in raw mode (#4092) The code is affected by the `--limit` flag both where the `--raw` flag is enabled and if it is disabled. However, in the disabled case the limit was not respected. This was not detected because the unit test only tested the code path when `--raw` is enabled. 20 May 2020, 19:17:19 UTC
0f23507 Increase the efficiency of the `SqlaGroup.nodes` iterator (#4094) The iterator was calling dbnodes.__iter__() which was violting the lazy loading of the nodes as they are needed during iteration leading to a huge load time for big groups even when only a slice is demanded. For reference, on a group with roughly half a million nodes, getting a single one through group.nodes[0] went from taking a minute to a few milliseconds. 20 May 2020, 18:46:14 UTC
f57a0b9 Fix bug in `Code.get_full_text_info` (#4083) This method was not tested and so it went unnoticed that it was still using the `_get_folder_pathsubfolder` repository method that has been removed in `aiida-core==1.0.0`. Direct access to the folder underlying the node repository is no longer allowed and instead the API should be used to inspect the available objects, in this case `list_objects`. 20 May 2020, 13:36:44 UTC
b86425f Add support for binary files to the various `verdi` cat commands (#4077) The commands `verdi calcjob inputcat`, `verdi calcjob outputcat` and `verdi node repo cat` were implemented such to only be able to deal with non-binary files. They are changed to simply copy the content of the target file directly to the stdout buffer. This fixes two problems: the command no longer assumes anything about the file content and just copies the bytes as is, and the copy is performed in chunks preventing from the whole content being loaded in memory. 20 May 2020, 06:52:14 UTC
51ea47d Increase the minimal required version of setuptools to 42.0.0 [py35] (#4085) This is necessary in order for the requirements of Python to be properly enforced by `setuptools`. A common problem that occurs when this is not respected is where a new `numpy` pre-release is released and the setup of `pymatgen` installs it and when not compatible with the Python version the build fails. The newer version of setuptools should respect the requirements that we impose. 19 May 2020, 16:19:10 UTC
9d9d095 Allow profile selection ipython magic %aiida (#4071) The ipython %aiida magic always selected the default profile. This PR adds a profile name argument allowing to select a different profile. 14 May 2020, 11:42:47 UTC
9d918e0 Add a progress bar for export and import related functionality (#3599) When provenance graphs become large, exporting and importing the resulting archives can become time consuming operations. To give the user more feedback of how the operation is progressing, a progress bar is added. The implementation is provided by the `tqdm` library which is therefore added as a new dependency. Since the export and import code is still not written in a modular way, to allow the progress bar to get access to the inner parts and provide information that is granular enough, it is implemented as a global singleton that the export and import functions fetch. This is to prevent having to pass the progress bar instance around in method calls. This is still not an ideal solution and in the future, this should be replaced with hooks that methods can call in order to update their status. A similar problem is faced with the logging of textual progress in the export and import functions that are done haphazardly with print statements, even though it concerns module functions and not CLI facing code. This has been changed to go through logging instead. The log level is temporarily updated based on the `silent` argument that the export and import functions take. Finally, the function signature of `export` and `export_tree` have been changed. The `what` and `outfile` arguments have been deprecated and replaced by `entities` and `filename`, respectively. A deprecation message is printed if they are used and they should be removed with the release of `aiida-core==2.0.0`. 14 May 2020, 08:57:32 UTC
b9d4bbe Close SQLA session after every REST API request This is needed due to the server running in threaded mode, i.e., creating a new thread for each incoming request. This concept is great for handling many requests, but crashes when used together with AiiDA's global singleton SQLA session used, no matter the backend of the profile by the `QueryBuilder`. Specifically, this leads to issues with the SQLA QueuePool, since the connections are not properly released when a thread is closed. This leads to unintended QueuePool overflow. This fix wraps all HTTP method requests and makes sure to close the current thread's SQLA session after the request as been completely handled. Use Flask-RESTful's integrated `Resource` attribute `method_decorators` to apply `close_session` wrapper to all and any HTTP request that may be requested of AiiDA's `BaseResource` (and its sub-classes). Additionally, remove the `__init__` function overwritten in `Node(BaseResource)`, since it is redundant, and the attributes `tclass` is not relevant with v4 (AiiDA v1.0.0 and above), but was never removed. It should have been removed when moving to v4 in 4ff2829. Concerning the added tests: the timeout needs to be set for Python 3.5 in order to stop the http socket and properly raise (and escape out of an infinite loop). The `capfd` fixture must be used, otherwise the exception cannot be properly captured. The tests were simplified into the pytest scheme with ideas from @sphuber and @greschd. 11 May 2020, 10:11:34 UTC
924f02e Add possibility to pass keyword arguments to `create_engine` The function `aiida.backends.utils.create_sqlalchemy_engine` now takes keyword arguments that are passed straight to the SQLAlchemy function `create_engine` which creates the engine that connects to the database. This utility function is used by the SQLAlchemy and by both backends for the `QueryBuilder`. By allowing these keyword arguments to be passed and plumbing them all the way in to the `Backend` class, the parameters of the SQLAlchemy engine and the associated queue pool can be customized. This is not meant to be used by end users, but can be important for important use, for testing and performance reasons. 11 May 2020, 10:11:34 UTC
back to top