e3b0c3d | Iraklis Psaroudakis | 19 July 2023, 08:58:31 UTC | Mute some PkiAuthDelegationIntegTests (#97774) (#97779) Relates #97772 | 19 July 2023, 08:58:31 UTC |
ca0aa27 | Mark Vieira | 19 July 2023, 00:22:23 UTC | Upgrade bundle JDK to Java 20.0.2 (#97758) (#97762) # Conflicts: # gradle/verification-metadata.xml Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> | 19 July 2023, 00:22:23 UTC |
df7528e | Mark Vieira | 18 July 2023, 23:27:47 UTC | Fix import | 18 July 2023, 23:27:47 UTC |
b526892 | Ryan Ernst | 18 July 2023, 15:19:01 UTC | Muting pki tests until expired test certs are regenerated (#97757) see https://github.com/elastic/elasticsearch/issues/97756 | 18 July 2023, 23:08:27 UTC |
0c1c6ff | Mark Vieira | 18 July 2023, 17:34:48 UTC | Mute SecurityDocumentationIT.testDelegatePkiAuthentication | 18 July 2023, 17:34:48 UTC |
a5433e3 | Ignacio Vera | 14 July 2023, 01:59:53 UTC | [7.17] Support for Byte and Short as vector tiles value tag (#97619) (#97673) * Port lucene fix github-12352 to Elasticsearch 7.17 * Update docs/changelog/96721.yaml * Support for Byte and Short as vector tiles value tag (#97619) Byte and short data types are not supported by the vector tiles specification. If a user tries to add one of those, it actually gets ignored. This add s support to those values by casting them as integers. In addition it will throw an exception if the value added is a type it cannot understand so it is not silently ignored. # Conflicts: # x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/vector-tile/10_basic.yml | 14 July 2023, 01:59:53 UTC |
d4069ef | David Turner | 10 July 2023, 11:09:36 UTC | Add note on troubleshooting laggy cancellations (#97485) (#97499) Today we document that tasks may not react to cancellations immediately, but in practice it's surprising to users and kind of a bug if they run for too long after being cancelled. This commit adds a little extra detail about the information to collect to troubleshoot such a situation. | 10 July 2023, 11:09:36 UTC |
7d11e41 | Artem Prigoda | 05 July 2023, 09:00:52 UTC | [7.17] Preserve context in ResultDeduplicator (#84038) (#96868) Today the `ResultDeduplicator` may complete a collection of listeners in contexts different from the ones in which they were submitted. This commit makes sure that the context is preserved in the listener. Co-authored-by: David Turner <david.turner@elastic.co> | 05 July 2023, 09:00:52 UTC |
9bb69a2 | Nhat Nguyen | 03 July 2023, 21:40:31 UTC | Fix delay timeout in MockTransportService (#97336) (#97338) We can pass an invalid duration to TimeValue, which accepts either -1 or an non-negative duration. | 03 July 2023, 21:40:31 UTC |
4ccdcce | David Turner | 03 July 2023, 17:31:41 UTC | [7.17] Handle failure in TransportUpdateAction#handleUpdateFailureWithRetry (#97290) (#97326) * Handle failure in TransportUpdateAction#handleUpdateFailureWithRetry (#97290) Here executor(request.getShardId()) may throw, but we're already handling a failure so we cannot simply let this exception bubble up. This commit adjusts things to catch the exception, using it to fail the listener. Closes #97286 * Fix --------- Co-authored-by: Iraklis Psaroudakis <kingherc@gmail.com> | 03 July 2023, 17:31:41 UTC |
3bdd3fc | Craig Taverner | 03 July 2023, 15:22:37 UTC | Incorrect name for sort field (#97328) (#97330) | 03 July 2023, 15:22:37 UTC |
29ff32e | Mark Tozzi | 03 July 2023, 13:57:14 UTC | Less jank in after-key parsing for unmapped fields (#86359) (#97282) Resolves #85928 The after-key parsing is pretty weird, and there are probably more bugs there. I did not take the opportunity to refactor the whole thing, but we should. This fixes the immediate problem by treating after keys as bytes refs when we don't have a field but think we want a keyword. We were already doing that if the user asked for a missing bucket, this just extends the behavior in the case that we don't. Long term, the terms Composite source (and probably other Composite sources) should have specializations for unmapped fields. That's the direction we want to take aggs in general. | 03 July 2023, 13:57:14 UTC |
22ca339 | Slobodan Adamović | 30 June 2023, 08:03:59 UTC | [7.17][DOC] Consistently reference indices parameter in API key examples (#97268) Backports the following commits to 7.17: - [DOC] Consistently reference indices parameter in API key examples (#97174) | 30 June 2023, 08:03:59 UTC |
3ab0425 | Lee Hinman | 29 June 2023, 21:01:21 UTC | Bump Guava from 30.1-jre to 32.0.1-jre (#97211) (#97255) This bumps Watcher's Guava dependency to 32.0.1. | 29 June 2023, 21:01:21 UTC |
0410133 | Mark Vieira | 29 June 2023, 20:31:44 UTC | Prune changelogs after 7.17.11 release | 29 June 2023, 20:31:44 UTC |
6a55b87 | Mark Vieira | 29 June 2023, 20:29:47 UTC | Bump versions after 7.17.11 release | 29 June 2023, 20:30:37 UTC |
23703ca | Joe Gallo | 29 June 2023, 07:31:34 UTC | Add release notes for 7.17.11 release (#97057) * Add release notes for 7.17.11 release * Add release notes for 7.17.11 release * Revert "Add release notes for 7.17.11 release" This reverts commit 924cc1619be6da6c3adf3ccb2c9a70b3bc4fd14d. | 29 June 2023, 07:31:34 UTC |
e1c90fa | David Turner | 29 June 2023, 07:17:36 UTC | Remove linebreak from #75598 known-issue docs (#97216) Linebreaks in `preformatted text` are preserved in the rendered docs, which is not what we want here. | 29 June 2023, 07:17:36 UTC |
32deaf9 | Volodymyr Krasnikov | 27 June 2023, 18:09:50 UTC | Update Guava dependency version to 32.0.1-jre (#97060) * Update Guava dependency version to 32.0.1-jre * Fix dependencies * Remove direct dependencies on guava ver.31 * Revert "Remove direct dependencies on guava ver.31" This reverts commit 1e90a271c2fdec8ec5c00b27c75cc465f23bc9df. | 27 June 2023, 18:09:50 UTC |
537a1c9 | Ryan Ernst | 27 June 2023, 16:47:29 UTC | Capture max processors in static init (#97119) (#97153) The number of processors available to the jvm can change over time. However, most of Elasticsearch assumes this value is constant. Although we could rework all code relying on the number of processors to dynamically support updates and poll the jvm, doing so has little value since the processors changing is an edge case. Instead, this commit fixes validation of the node.processors setting (our internal number of processors) to validate based on the max processors available at launch. closes #97088 | 27 June 2023, 16:47:29 UTC |
62579b0 | Joe Gallo | 27 June 2023, 14:04:22 UTC | Fix unhandled exception when blobstore repository contains unexpected file (#93914) (#97113) If there's any file with the `index-` prefix but not a number after that at the repo root we must not throw here. If we do, we will end up throwing an unexpected exception that is not properly handled by `org.elasticsearch.snapshots.SnapshotsService#failAllListenersOnMasterFailOver`, leading to the repository generation not getting correctly set in the cluster state down the line. | 27 June 2023, 14:04:22 UTC |
6cd68fc | Joe Gallo | 27 June 2023, 13:29:56 UTC | [7.17] Clarify index template settings precedence (#87374) (#97121) [docs] Clarify that index template settings take precedence over component templates. Co-authored-by: Toby Sutor <55087308+toby-sutor@users.noreply.github.com> Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> Co-authored-by: Adam Locke <adam.locke@chronosphere.io> | 27 June 2023, 13:29:56 UTC |
2a3d71b | Iraklis Psaroudakis | 26 June 2023, 16:05:24 UTC | Upgrade Netty to 4.1.94.Final (#97114) Backport of https://github.com/elastic/elasticsearch/pull/97112 | 26 June 2023, 16:05:24 UTC |
68092d0 | Przemysław Witek | 26 June 2023, 14:39:25 UTC | [7.17] [Transform] Adding null check to fix potential NPE (#96785) (#97109) | 26 June 2023, 14:39:25 UTC |
4e3cc9b | Tanguy Leroux | 26 June 2023, 12:46:57 UTC | [7.17] Fix ESCCRRestTestCase.verifyAutoFollowMonitoring (#97054) (#97105) The method `verifyAutoFollowMonitoring` searches for `ccr_auto_follow_stats` documents in `monitoring-es-*` indices to see if one document has the field `number_of_successful_follow_indices` greater than 0. If such document is found, it means that x-pack monitoring successfully indexed documents about CCR stats and that CCR at that time had at least 1 following index successfully running. But the current `verifyAutoFollowMonitoring` method only check the first 10 docs returned by the search requests. Recent failures of `AutoFollowIT.testAutoFollowPatterns` on 7.17 branch indicates that more than 10 docs were found, so I suspect one of them has `number_of_successful_follow_indices` > 0 but is not returned in the first 10 top docs. This pull request changes the method to only count documents with `number_of_successful_follow_indices` > 0. Closes #91984 | 26 June 2023, 12:46:57 UTC |
eeedb98 | David Turner | 22 June 2023, 07:05:03 UTC | Make cluster health API cancellable (#96990) This API can be quite heavy in large clusters, and might spam the `MANAGEMENT` threadpool queue with work for clients that have long-since given up. This commit adds some basic cancellability checks to reduce the problem. Backport of #96551 to 7.17 | 22 June 2023, 07:05:03 UTC |
e1995a7 | Nhat Nguyen | 20 June 2023, 18:43:56 UTC | Increase concurrent request of opening point-in-time (#96959) * Increase concurrent request of opening point-in-time (#96782) (#96957) Today, we mistakenly throttle the opening point-in-time API to 1 request per node. As a result, when attempting to open a point-in-time across large clusters, it can take a significant amount of time and eventually fails due to relocated target shards or deleted target indices managed by ILM. Ideally, we should batch the requests per node and eliminate this throttle completely. However, this requires all clusters to be on the latest version. This PR increases the number of concurrent requests from 1 to 5, which is the default of search. * Fix tests * Fix tests | 20 June 2023, 18:43:56 UTC |
1fe30f5 | Nikolaj Volgushev | 19 June 2023, 12:17:24 UTC | [7.17] Update OIDC and JWT dependencies (#96862) (#96917) * Update OIDC and JWT dependencies (#96862) Updates internal dependencies for OIDC and JWT realms. * Order --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> | 19 June 2023, 12:17:24 UTC |
edb34ad | Craig Taverner | 18 June 2023, 20:08:27 UTC | Correct rare-terms default precision in docs (#96887) (#96907) | 18 June 2023, 20:08:27 UTC |
2b42390 | Ignacio Vera | 14 June 2023, 08:42:13 UTC | Port lucene tessellator fix github-12352 to Elasticsearch 7.17 (#96721) | 14 June 2023, 08:42:13 UTC |
969770f | David Turner | 12 June 2023, 14:58:19 UTC | Include targetNodeName in shutdown metadata toString() (#96765) Reporting the `targetNodeName` was added to `main` in #78727 but omitted from the backport in #78865. This commit adds the missing field to the `toString()` response. | 12 June 2023, 14:58:19 UTC |
b326926 | Ievgen Degtiarenko | 09 June 2023, 07:16:07 UTC | Debug ccr connection failure in docs build (#96699) (#96719) Set DEBUG log level to be able to troubleshoot failures to connect to local CCR cluster. | 09 June 2023, 07:16:07 UTC |
3fc6eb6 | Ievgen Degtiarenko | 08 June 2023, 09:32:18 UTC | Fix AsyncShardFetchTests#testTwoNodesRemoveOne (#93734) (#96695) This test was using the wrong `DiscoveryNodes`, but that mistake was hidden by other leniency elsewhere in this test suite. This commit fixes the test bug and also makes the test suite stricter. Closes #93729 (cherry picked from commit 774e396ed5a904789e8aca5cf5f9f0f2d252ca2a) Co-authored-by: David Turner <david.turner@elastic.co> | 08 June 2023, 09:32:18 UTC |
77b1499 | Ryan Ernst | 06 June 2023, 21:27:49 UTC | Retry windows file deletes in packaging tests (#96630) (#96634) We continue to have CI failures for open files when trying to cleanup on Windows. This commit tries to account for one of those cases, where the out/err redirects are cleaned up, opting to retry once after a delay. | 06 June 2023, 21:27:49 UTC |
7c5d82d | David Turner | 05 June 2023, 09:51:23 UTC | Add one more consistency check in AsyncShardFetch (#96553) (#96557) Relates #93632 | 05 June 2023, 09:51:23 UTC |
9a1ee48 | David Turner | 05 June 2023, 08:14:21 UTC | Fork TransportClusterHealthAction to MANAGEMENT (#96546) This action can become fairly expensive for large states. Plus it is called at high rates on e.g. Cloud which is blocking transport threads needlessly in large deployments. Lets fork it to MANAGEMENT like we do for similar CPU bound actions. Backport of #90621 to 7.17 Co-authored-by: Armin Braun <me@obrown.io> | 05 June 2023, 08:14:21 UTC |
6a755e3 | David Turner | 05 June 2023, 08:13:09 UTC | Streamline AsyncShardFetch#getNumberOfInFlightFetches (#96545) Avoids an O(#nodes) iteration by tracking the number of fetches directly. Backport of #93632 to 7.17 Co-authored-by: luyuncheng <luyuncheng@bytedance.com> | 05 June 2023, 08:13:09 UTC |
72afc2e | David Turner | 01 June 2023, 07:39:00 UTC | Bring-your-own JDK docs improvements (#96404) - Bury these docs a little more, there's no need for them to be on the landing page for "Set up Elasticsearch". - Clarify the responsibilities for JDK updates. | 01 June 2023, 07:46:33 UTC |
e991438 | Pooya Salehi | 26 May 2023, 11:59:33 UTC | Remove EncryptedAzureBlobStoreRepositoryIntegTests (#96377) Closes https://github.com/elastic/elasticsearch/issues/96246 | 26 May 2023, 11:59:33 UTC |
3445e5b | Ievgen Degtiarenko | 23 May 2023, 07:10:41 UTC | Fix testFollowerBehaviour (#96257) (#96269) The test was failing when responseDelay == leaderCheckTimeoutMillis. This resulted in scheduling both handling the response and timeout at the same mills and executing them in random order. The fix makes it impossible to reply the same time as request is timeout as the behavior is not deterministic in such case. | 23 May 2023, 07:10:41 UTC |
fb243a9 | Keith Massey | 22 May 2023, 17:02:42 UTC | Avoiding running IO on scheduler thread in ResourceWatcherService (#96251) (#96261) | 22 May 2023, 17:02:42 UTC |
59c1b19 | Mark Vieira | 17 May 2023, 21:18:56 UTC | Use correct target compatibility for third party audit tasks (#94648) (#94652) # Conflicts: # build-tools-internal/src/main/groovy/elasticsearch.runtime-jdk-provision.gradle Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> | 17 May 2023, 21:18:56 UTC |
bf324b8 | Iraklis Psaroudakis | 16 May 2023, 13:16:31 UTC | Mute testAliasFields (#96153) (#96158) Relates #93228 | 16 May 2023, 13:16:31 UTC |
19d6efa | Tim Vernum | 16 May 2023, 04:58:33 UTC | [DOCS] Recommend API service tokens (over file) (#96091) (#96121) File based service tokens were added to support orchestration requirements in environments such as ECE and ECK. Outside of these environments we recommend that API based tokens are used instead. Resolves: #83491 | 16 May 2023, 04:58:33 UTC |
eb9e452 | Francisco Fernández Castaño | 12 May 2023, 16:36:32 UTC | [7.17] Fix Azure InputStream#read method (#96034) (#96045) | 12 May 2023, 16:36:32 UTC |
724c7d1 | Mark Vieira | 11 May 2023, 17:00:25 UTC | Add SLES 15.4 to the docker exclusion list (#96037) (#96043) Our docker exclusion list is sensitive to minor versions. With SLES 15.4 we need to explicitly add it to the exclusion list since it doesn't play nicely with Docker in CI. Closes https://github.com/elastic/elasticsearch/issues/93898 # Conflicts: # .ci/dockerOnLinuxExclusions | 11 May 2023, 17:00:25 UTC |
868373e | Tanguy Leroux | 11 May 2023, 11:50:56 UTC | [7.17] Cancel cold cache prewarming tasks if store is closing (#95891) (#96024) Cold cache prewarming tasks are not stopped immediately when the shard is closed, causing excessive use of disk for nothing. This change adds a Supplier<Boolean> to prewarming logic that can be checked before executing any consuming operation to know if the Store is closing. I'm not super happy with my test changes but the logic for checking if prewarming works correctly is tricky. Closes #95504 | 11 May 2023, 11:50:56 UTC |
fc5e40b | Andrei Dan | 09 May 2023, 18:13:47 UTC | [ILM] Fix the migrate to tiers service and migrate action tiers configuration (#95934) (#95966) The migrate action (although no allowed in the frozen phase) would seem to convert `frozen` to `data_frozen,data_cold,data_warm,data_hot` tier configuration. As the migrate action is not allowed in the frozen phase this would never happen, however the code is confusing as it seems like it could. The migrate to data tiers routing service shared the code used by the `migrate` action that converted `frozen` to `data_frozen,data_cold,data_warm,data_hot` if it would encounter an index without any `_tier_preference` setting but with a custom node attribute configured to `frozen` e.g. `include.data: frozen` As part of https://github.com/elastic/elasticsearch/issues/84758 we have seen frozen indices with the `data_frozen,data_cold,data_warm,data_hot` tier preference however we could never reproduce it. Relates to https://github.com/elastic/elasticsearch/issues/84758 | 09 May 2023, 18:13:47 UTC |
311f90e | Rene Groeschke | 03 May 2023, 20:18:51 UTC | Update snyk ci job to use branch as target reference (#95794) (#95805) | 03 May 2023, 20:18:51 UTC |
dc9c1e1 | Stef Nestor | 03 May 2023, 12:42:48 UTC | [DOC] Move warning & explicate static setting (#81166) * [DOC] Move warning & explicate static setting Howdy! Small edits for user `CMD+F` ability & emphasis: explicates static setting (to explain why get errors if attempt to edit on existing index) & pulls CCR warning to top. As non-code change curiosity, AFAICT all other static/dynamic settings that fit under modules are left to the child pages [from here](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#_static_index_settings) but [History retention](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-history-retention.html) seems to uniquely locate the info on the parent page and then reference it on a child page. Is this intended (asking because it breaks general flow)? --------- Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> | 03 May 2023, 12:42:48 UTC |
a7072e1 | Bogdan Pintea | 03 May 2023, 10:58:03 UTC | Remove 'coming' notification in release notes (#95777) Remove 'coming' notification in release notes. | 03 May 2023, 10:58:03 UTC |
e4ad337 | Michal Junek | 03 May 2023, 09:29:25 UTC | Fixes a typo in 7.0 migration guide (#95765) | 03 May 2023, 09:29:25 UTC |
008ba54 | Mark Vieira | 02 May 2023, 16:46:42 UTC | Prune changelogs after 7.17.10 release | 02 May 2023, 16:46:42 UTC |
9090332 | Mark Vieira | 02 May 2023, 16:44:41 UTC | Bump versions after 7.17.10 release | 02 May 2023, 16:45:33 UTC |
d402f9c | Abdon Pijpelink | 28 April 2023, 12:42:42 UTC | [7.17] [DOCS] Clarify configuring indices/data streams used by stack monitoring (#95285) (#95587) * [DOCS] Clarify configuring indices/data streams used by stack monitoring (#95285) * [DOCS] Clarify indices/data streams using by stack monitoring * Incorporate review feedback * Update docs/reference/monitoring/indices.asciidoc Co-authored-by: Kevin Lacabane <klacabane@gmail.com> --------- Co-authored-by: Kevin Lacabane <klacabane@gmail.com> (cherry picked from commit 2e31832c34fdf5158f6d1aae3a288e624841f43a) * Add Elastic Agent monitoring | 28 April 2023, 12:42:42 UTC |
38b70f6 | Armin Braun | 28 April 2023, 08:26:15 UTC | Upgrade Netty to 4.1.89 (#95559) Upgrading 7.17.x to the version we've been using in 8.x for a while to resolve audit issues. | 28 April 2023, 08:26:15 UTC |
d13bda8 | Bogdan Pintea | 24 April 2023, 14:56:16 UTC | Add release notes for 7.17.10 (#95365) Add release notes for 7.17.10 | 24 April 2023, 14:56:16 UTC |
fecd68e | Tim Vernum | 19 April 2023, 15:11:51 UTC | [7.17] Bump bundled JDK to Java 20.0.1 (#95371) | 19 April 2023, 15:11:51 UTC |
7284dd2 | Armin Braun | 19 April 2023, 13:10:05 UTC | Deduplicate Heavy CCR Repository CS Requests (#91398) (#95372) We run the same request back to back for each put-follower call during the restore. Also, concurrent put-follower calls will all run the same full CS request concurrently. In older versions prior to https://github.com/elastic/elasticsearch/pull/87235 the concurrency was limited by the size of the snapshot pool. With that fix though, they are run at almost arbitry concurrency when many put-follow requests are executed concurrently. -> fixed by using the existing deduplicator to only run a single remote CS request at a time for each CCR repository. Also, this removes the needless forking in the put-follower action that is not necessary any longer now that we have the CCR repository non-blocking (we do the same for normal restores that can safely be started from a transport thread), which should fix some bad-ux situations where the snapshot threads are busy on master, making the put-follower requests not go through in time. | 19 April 2023, 13:10:05 UTC |
2b42144 | Athena Brown | 17 April 2023, 23:23:43 UTC | Correct Guava version used for OWASP Java HTML Sanitizer (#94442) The OWASP Java HTML Sanitizer has a dependency on Guava, specifically on version 30.1. However, prior to this PR, we have been using 27.1. While this does not seem to have caused any issues, we should still correct the problem. (cherry picked from commit f2f45ad1cfa2cc052d76bc896a400d549054fc2c) | 17 April 2023, 23:23:43 UTC |
a68e83a | Joe Gallo | 13 April 2023, 20:34:40 UTC | Fix ignore_missing docs for a couple of Ingest processors (#95244) | 13 April 2023, 20:36:37 UTC |
3588521 | Nikolaj Volgushev | 13 April 2023, 12:55:50 UTC | Bump version on internal OIDC depedency (#95142) (#95179) Version bump for a third-party dependency for OIDC. | 13 April 2023, 12:55:50 UTC |
50880a5 | Pablo Alcantar Morales | 12 April 2023, 12:08:16 UTC | fix typo in `cron.asciidoc` (#95181) (#95184) | 12 April 2023, 12:08:16 UTC |
6a1c7e4 | Keith Massey | 11 April 2023, 14:46:17 UTC | Fixing BulkProcessor2RetryIT (#95091) | 11 April 2023, 14:46:17 UTC |
d43aa81 | Salvatore Campagna | 11 April 2023, 13:27:43 UTC | mute: testScrollWithDatetimeAndTimezoneParam (#95153) | 11 April 2023, 13:27:43 UTC |
68905df | Mark Vieira | 07 April 2023, 01:35:25 UTC | Add mechanism to initialize YAML tests against a subset of test cases (#95095) (#95097) This commit adds the ability to initialize YAML rest test suites against a subset of available test cases. Previously, the only way to do this is via the `tests.rest.suite` system property, but that can only be set at the test _task_ level. Configuring this at the test _class_ level means that we can support having multiple test suite classes that execute subsets of tests within a project. That allows for things like parallelization, or having different test cluster setups for different YAML tests within the same project. For example: ```java @ParametersFactory public static Iterable<Object[]> parameters() throws Exception { return ESClientYamlSuiteTestCase.createParameters(new String[] { "analysis-common", "indices.analyze" }); } ``` The above example would mean that only tests in the `analysis-common` and `indices.analyze` directories would be included in this suite. cc @jdconrad Closes #95089 | 07 April 2023, 01:35:25 UTC |
f00bf45 | Tim Vernum | 06 April 2023, 06:31:13 UTC | Reuse FieldPermissionsCache in Role parsing (#95065) When parsing role descriptors, we ensure that the FieldPermissions (`"field_security":{ "grant":[ ... ], "except":[ ... ] }`) are valid - that is that any patterns compile correctly, and the "except" is a subset of the "grant". However, the previous implementation would not use the FieldPermissionsCache for this, so it would compile (union, intersect & minimize) automatons every time a role was parsed. This was particularly an issue when parsing roles (from the security index) in the GET /_security/role/ endpoint. If there were a large number of roles with field level security the automaton parsing could have significant impact on the performance of this API. Backport of: #94931 | 06 April 2023, 06:31:13 UTC |
21329d4 | Pooya Salehi | 05 April 2023, 08:29:58 UTC | [DOC] Specify upgrade order for tiers (#95022) Outdated info in comparison to https://www.elastic.co/guide/en/elastic-stack/8.7/upgrading-elasticsearch.html#upgrading-elasticsearch | 05 April 2023, 08:29:58 UTC |
8035851 | David Turner | 31 March 2023, 10:41:17 UTC | AwaitsFix for #94461 and #94329 | 31 March 2023, 10:41:36 UTC |
6413b2d | David Turner | 31 March 2023, 09:30:56 UTC | AwaitsFix for #94941 | 31 March 2023, 09:31:20 UTC |
47054bd | Nhat Nguyen | 30 March 2023, 18:46:22 UTC | [7.17] Closing an empty PIT should return 200 (#94708) (#94908) * Closing an empty PIT should return 200 (#94708) * Fix compilation | 30 March 2023, 18:46:22 UTC |
21b4ea2 | Abdon Pijpelink | 29 March 2023, 09:53:45 UTC | [DOCS] clarify v7 file realm configuration (#94778) * [DOCS] clarify v7 file realm configuration * Update x-pack/docs/en/security/authentication/configuring-file-realm.asciidoc Co-authored-by: Yang Wang <ywangd@gmail.com> --------- Co-authored-by: Yang Wang <ywangd@gmail.com> | 29 March 2023, 09:53:45 UTC |
ad68908 | Joe Gallo | 28 March 2023, 22:01:05 UTC | [7.17] Fix _cluster/stats .nodes.fs deduplication (#94843) * Refactor _cluster/stats .nodes.fs deduplication (#94780) * Fix FsInfo device deduplication (#94744) * Fix _cluster/stats .nodes.fs deduplication (#94798) | 28 March 2023, 22:01:05 UTC |
9644135 | James Baiera | 28 March 2023, 18:43:50 UTC | Fix async enrich execution prematurely releases enrich policy lock (#94702) (#94841) This PR ups the timeout on the EnrichExecutor's task API call and adds additional logic in the event that the task await call fails. Without this change, the task API call can timeout and unlock the policy prematurely. Premature unlocking can lead to the index being removed while the policy is executing. (cherry picked from commit f56fc01222208c98563aff789433a3948460b9a4) # Conflicts: # x-pack/plugin/enrich/src/main/java/org/elasticsearch/xpack/enrich/EnrichPolicyExecutor.java # x-pack/plugin/enrich/src/test/java/org/elasticsearch/xpack/enrich/EnrichPolicyExecutorTests.java | 28 March 2023, 18:43:50 UTC |
c40f7af | James Baiera | 28 March 2023, 18:03:17 UTC | [7.17] Refactor enrich maintenance coordination logic (#90931) (#94840) This PR refactors the locking logic for enrich policies so that enrich index names are resolved early so that they may be explicitly protected from maintenance tasks on the master node. The maintenance service has been optimized to allow for concurrent removal of old enrich indices while policies are executing. Further concurrency changes were made to improve the thread safety of the system (such as removing the double check locking in maintenance and the ability to unlock policies from code that does not hold the lock). (cherry picked from commit 998520e111cee9a090127ea8c811bcd40a42c44a) | 28 March 2023, 18:03:17 UTC |
4ba72b4 | Mark Vieira | 28 March 2023, 14:33:29 UTC | Don't check for beats updates in 7.17 branch | 28 March 2023, 14:33:29 UTC |
45e5853 | Andrei Dan | 28 March 2023, 07:08:44 UTC | [7.17] Allow ILM to transition to implicit cached steps (#91779) (#94766) * Allow ILM to transition to implicit cached steps (#91779) ILM tries to honour the cached phase however, some steps are implicit (e.g. injected actions or the terminal policy/phase step) Currently, ILM would throw an exception if the currently cached phase was removed: ``` step [{"phase":"warm","action":"complete","name":"complete"}] for index [index] with policy [my-policy] does not exist ``` ILM would currently also throw an exception if the next step was an implicit one and the phase was removed from the underlying policy e.g. if the index is on the `migrate` step and looking to transition to the `check-migration` step, whilt the `warm` phase doesn't exist in the policy anymore ``` step [{"phase":"warm","action":"migrate","name":"check-migration"}] for index [index] with policy [my-policy] does not exist ``` This fixes these scenarios by enhancing the `PolicyStepsRegistry#parseStepKeysFromPhase` method to compute all the steps in the phase (including all implicit steps) (cherry picked from commit dc726e60a9a247cecee5566497cdbd6f3c274b60) Signed-off-by: Andrei Dan <andrei.dan@elastic.co> | 28 March 2023, 07:08:44 UTC |
a7fbd6a | Mark Vieira | 27 March 2023, 22:12:35 UTC | Skip staging update job on the main branch | 27 March 2023, 22:13:49 UTC |
1185fc5 | Mark Vieira | 27 March 2023, 21:35:35 UTC | Add scheduled CI job to update staging artifact when dependencies change | 27 March 2023, 22:13:44 UTC |
5ae7d40 | Ievgen Degtiarenko | 27 March 2023, 15:14:07 UTC | [7.17] Check no unassigned shards even if the node already left (#94722) (#94774) We are currently checking if there are any unassigned shards on the shutting down node before returning SingleNodeShutdownMetadata.Status.COMPLETE. However this check is only triggered if the node is still running. If the node unexpectedly exists the cluster or dies during the migration then we are at risk of loosing shard data of those unassigned shards. This change performs the check even if the node is no longer part of the cluster. | 27 March 2023, 15:14:07 UTC |
3be6f95 | Christoph Büscher | 24 March 2023, 09:33:56 UTC | Fix `_id` field fetch issue (#94528) (#94685) When queries through the "fields" API, the metadata _id field is not returned consistently due to frequently clearing an internal data cache. This commit fixes this by making copies of those values in the value fetcher so clearing the cache doesn't affect displaying the results in the search hits. Backport of #94528 | 24 March 2023, 09:33:56 UTC |
3b5fcca | Mark Vieira | 23 March 2023, 21:10:08 UTC | Improve efficiency of packer cache script (#94701) (#94705) # Conflicts: # .ci/packer_cache.sh # distribution/bwc/build.gradle # distribution/docker/build.gradle | 23 March 2023, 21:10:08 UTC |
73e0437 | Mark Vieira | 23 March 2023, 20:53:09 UTC | Fail build immediately if there is an error downloading artifacts | 23 March 2023, 20:54:40 UTC |
e9c2151 | David Turner | 23 March 2023, 12:31:02 UTC | Disable ESLoggingHandler by default (#94516) Relates #93133 | 23 March 2023, 12:48:34 UTC |
fbb73d8 | Mark Vieira | 21 March 2023, 18:38:32 UTC | Upgrade bundled JDK to Java 20 (#94600) (#94606) # Conflicts: # gradle/verification-metadata.xml | 21 March 2023, 18:38:32 UTC |
35999ac | Przemysław Witek | 20 March 2023, 18:35:52 UTC | [Transform] Catch deprecations as `Exception` rather than `IOException` (#94553) (#94563) | 20 March 2023, 18:35:52 UTC |
8b1ae76 | Andrei Dan | 20 March 2023, 11:39:02 UTC | Allow ILM step transition to the phase terminal step (#91754) (#94542) ILM tries to honour the cached phase however, some steps are implicit (e.g. injected actions or the terminal policy/phase step) Currently ILM would throw an exception if the currently cached phase was removed: ``` step [{"phase":"warm","action":"complete","name":"complete"}] for index [index] with policy [my-policy] does not exist ``` This fixes this scenario allowing ILM to transition to the phase complete step even if the phase has been removed from the actual policy. (cherry picked from commit 39625567e0fb5acf8d8d5f7a30bd17e4b7e2fab3) Signed-off-by: Andrei Dan <andrei.dan@elastic.co> | 20 March 2023, 11:39:02 UTC |
36a70fe | Tanguy Leroux | 10 March 2023, 17:34:32 UTC | [7.17] Log HTTP response trace when sending is complete (#94472) HttpTracer logs a message when an HTTP response is sent to a HttpChannel but that does not mean the response sending process is completed. This pull request changes the HttpTracer so that the message is now logged when the sending is complete. Backport of #94436 | 10 March 2023, 17:34:32 UTC |
8a6f3a5 | Simon Cooper | 10 March 2023, 16:16:13 UTC | [7.17] Add checks for exception loops through suppressed exceptions (#93944) (#94467) This refactors how ElasticsearchException serializes causal & suppressed exceptions, to maintain the nested level count throughout. Previously, we were losing the nested level through ElasticsearchException.writeTo, which meant that loops through recursively suppressed exceptions were not handled. This also moves the tests for such exceptions to ElasticsearchExceptionTests. (cherry picked from commit db0500ee24c9fed5de6e96eaf626b0d974251998) | 10 March 2023, 16:16:13 UTC |
53c7798 | Ryan Ernst | 09 March 2023, 19:57:16 UTC | Check jdk20 assumption after starting cluster (#94413) (#94439) When assumptions fail, @After rules are still run. For ESIntegTestCase, it does cluster cleanup. But disruption tests manually create the cluster, so if a test does not create the cluster, the after rule will fail with assertions. This commit moves the jdk20 assumption after creating the cluster, so that cleanup can run normally. | 09 March 2023, 19:57:16 UTC |
4946691 | Mark Vieira | 08 March 2023, 19:42:13 UTC | Use Java 8 as system Java home for packaging tests | 08 March 2023, 19:42:13 UTC |
d6756ed | Ryan Ernst | 06 March 2023, 17:32:19 UTC | Mute one more disruption test (#94298) (#94340) In #94207 we missed one disruption test still mimicking GC. | 06 March 2023, 17:32:19 UTC |
1c42f32 | Joe Gallo | 06 March 2023, 14:44:19 UTC | Add an example of dot_expander's path option (#94291) (#94331) | 06 March 2023, 14:44:19 UTC |
7d37a78 | Mark Vieira | 03 March 2023, 00:19:48 UTC | Fix packaging test reliance on SYSTEM_JAVA_HOME (#94293) (#94295) Fixes some fallout from recent CI changes that affect packaging tests. | 03 March 2023, 00:19:48 UTC |
4ac5471 | Mark Vieira | 02 March 2023, 20:06:45 UTC | Use bundled JDK by default when testing in CI (cherry picked from commit 7c2135c6bcfaa10e3b51a7cfac7b85b9824dbbc2) | 02 March 2023, 23:02:03 UTC |
048c4f4 | Mark Vieira | 02 March 2023, 20:01:45 UTC | Update CI job definitions to remove the use of RUNTIME_JAVA_HOME (cherry picked from commit 66aeaf5810180b4a36bab74c1e5e0b48a8df43a3) | 02 March 2023, 22:31:52 UTC |
bc25aeb | Keith Massey | 02 March 2023, 20:54:28 UTC | Avoiding BulkProcessor deadlock in ILMHistoryStore (#91238) (#93424) * backporting 91238 and 86184 * increasing test timeouts (#92771) BulkProcessor2IT can occasionally fail with timeouts like this: ``` java.util.concurrent.TimeoutException: (No message provided) at __randomizedtesting.SeedInfo.seed([164F04355E8E8724:9D44A00946BBB3F3]:0) at java.util.concurrent.Phaser.awaitAdvanceInterruptibly(Phaser.java:795) at org.elasticsearch.action.bulk.Retry2.awaitClose(Retry2.java:129) at org.elasticsearch.action.bulk.BulkProcessor2.awaitClose(BulkProcessor2.java:254) at org.elasticsearch.action.bulk.BulkProcessor2IT.testBulkProcessor2ConcurrentRequestsReadOnlyIndex(BulkProcessor2IT.java:197) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:568) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) at java.lang.Thread.run(Thread.java:833) ``` It looks like we're just cutting it a little too closely using a 1-second timeout to wait for all requests to complete. This PR bumps that timeout to 5 seconds. In the previous version of this test (BulkProcessorIT) the code did not actually wait for all requests to complete, which explains why this behavior is new. Closes #92770 * fixing build problems * reverting accidental change * fixing build problems * fixing a unit test * fixing tests * fixing tests * Not propagating TimeoutException from Retry2::awaitClose (#92773) Logging a message rather than propagating a TimeoutException from Retry2::awaitClose --------- Co-authored-by: Joe Gallo <joe.gallo@elastic.co> | 02 March 2023, 20:54:28 UTC |
8113c92 | Ryan Ernst | 01 March 2023, 21:14:39 UTC | Mute disruption tests for jdk20+ (#94207) (#94254) In JDK 20 Thread suspend/resume is soft removed (they now throw UnsupportedOperationException). Many ES disruption tests simulate GC pauses with suspend/resume. As that strategy will no longer work, this commit mutes those tests for jdk20+. relates #94206 closes #93707 | 01 March 2023, 21:14:39 UTC |
8d40b1a | Mark Vieira | 28 February 2023, 18:44:07 UTC | Avoid building test fixtures when building release artifacts (#94200) (#94203) We don't want to build test fixture projects when building our release artifacts. | 28 February 2023, 18:44:07 UTC |
dabb787 | Mark Vieira | 28 February 2023, 16:48:08 UTC | Add Java 20 to testing matrix | 28 February 2023, 16:48:08 UTC |