b11c15b | Benjamin Trent | 03 August 2021, 17:06:14 UTC | [ML] Adding new trained model allocation service (#75778) Adds a new service for trained model allocation to nodes. Initially, this only supports PyTorch models and simply allocates to nodes with the ML roles. Design is fairly simple: - A master node service runs allowing for new allocations to be created/updated/deleted from cluster state - A node service runs listening to updates referencing the local node + any models it may have allocated and updates accordingly. This type of service sort of splits the difference between the logic of shard allocation and persistent tasks. Neither really fully addressed the need here. | 03 August 2021, 17:06:14 UTC |
10a1d27 | David Roberts | 03 August 2021, 16:22:06 UTC | [ML] Deleting a job now deletes the datafeed if necessary (#76010) Previously attempting to delete a job that had a datafeed would return an exception. However, this was unnecessarily pedantic - the user would always want to delete both job and datafeed together, and would react by deleting the datafeed and then subsequently deleting the job again. This change makes the delete job API automatically delete a datafeed associated with the job. The same level of force is used for this delete datafeed request as was used on the delete job request. This means that it's possible to force-delete an open job with a started datafeed (since force-delete datafeed will automatically stop a started datafeed). It's still not possible to delete an opened job without using force. | 03 August 2021, 16:22:06 UTC |
705a408 | Mark Vieira | 03 August 2021, 16:09:15 UTC | Bump bundled JDK to 16.0.2 (#75981) | 03 August 2021, 16:09:15 UTC |
10261d0 | James Rodewig | 03 August 2021, 15:44:31 UTC | [DOCS] Remove coming tags for release highlights (#76037) Removes coming tag from release highlights. Our 7.x and 8.x docs already have a "preliminary docs" advisory until they're released. This coming tag duplicates that message. Removing it creates an unneeded chore at release. This also replaces an outdated coming tag for component template API redirects. | 03 August 2021, 15:44:31 UTC |
134dab6 | Alan Woodward | 03 August 2021, 15:38:31 UTC | Don't rebuild shadowed field lookup on every document (#76023) #75595 added better checks for fields shadowed by runtime fields, so that we don't index data that would never be searched. However, the shadow lookup was being rebuilt for every document, which has caused a noticeable regression in indexing times. This commit reworks things so that this lookup is built once per mapping update and lives on MappingLookup. | 03 August 2021, 15:38:31 UTC |
453877d | Lyudmila Fokina | 03 August 2021, 15:21:32 UTC | Enroll API to return CA cert using standard Base64 (#75858) * Enroll API to return CA cert using standard Base64 encoding | 03 August 2021, 15:21:32 UTC |
715bbb6 | Przemko Robakowski | 03 August 2021, 09:42:36 UTC | add version 7.14.1 | 03 August 2021, 14:33:22 UTC |
fc0ac19 | James Rodewig | 03 August 2021, 13:55:48 UTC | [DOCS] Correct spelling for geo terms (#76028) Changes: * Use "geopoint" when not referring to the literal field type * Use "geoshape" when not referring to the literal field type or query type * Use "GeoJSON" consistently | 03 August 2021, 13:55:48 UTC |
10ffb93 | Dan Hermann | 03 August 2021, 13:44:05 UTC | Create data stream aliases from component templates (#75956) | 03 August 2021, 13:44:05 UTC |
11a5676 | Tanguy Leroux | 03 August 2021, 13:24:04 UTC | SearchableSnapshotsIT.testCacheStats should wait for cache writes to complete before checking stats (#76011) * Wait for cache writes to terminate before checking stats * remove empty method | 03 August 2021, 13:24:04 UTC |
f339282 | Przemyslaw Gomulka | 03 August 2021, 13:20:34 UTC | [Rest Api Compatibility] Typed query (#75453) Type query support was removed in #47207. This query will throw an exception in v7 rest api compatibility indicating that the support was removed + deprecation warnings. In v8 it will not be available and error about type query being not found will be returned. relates main meta issue #51816 relates types removal meta #54160 | 03 August 2021, 13:20:34 UTC |
d671e3f | a-k-g | 03 August 2021, 12:57:16 UTC | [Docs] Include `index` param in `geo_point` docs (#75798) | 03 August 2021, 12:57:16 UTC |
37c806c | Adam Locke | 03 August 2021, 12:23:29 UTC | [DOCS] Update Elastic Stack security layers graphic (#75990) | 03 August 2021, 12:23:29 UTC |
cc07145 | Hendrik Muhs | 03 August 2021, 12:20:58 UTC | [CI][Transform] fix GroupByOptimizerTests randomization failure part 2 (#76009) avoid clashing field name by using a unique prefix fixes #75957 | 03 August 2021, 12:20:58 UTC |
2b97cc8 | David Turner | 03 August 2021, 11:37:22 UTC | Fix test failure introduced in #75917 In #75917 we introduced an assertion that relied on the order of the shards in the JSON representation of a `SnapshotsInProgress` but in fact we might end up with the shards in either order. This commit weakens the assertion to fix the test. | 03 August 2021, 11:46:23 UTC |
15933c0 | Ed Savage | 03 August 2021, 11:44:36 UTC | [ML] Unmute BWC tests (#76008) Unmute the the BWC tests and alter the BWC version for the model_prune_window field to be 7_15_0 Relates #75741, #76003 | 03 August 2021, 11:44:36 UTC |
7f16939 | David Turner | 03 August 2021, 10:42:40 UTC | Include extra snapshot details in logs/APIs (#75917) Today we do not expose the shard generations in the cluster state API, nor do we indicate which snapshots are being deleted in the cluster state update description; these data would have been useful in a recent debugging session. This commit adds these extra details, and also expands some comments on `ClusterState` describing how it's used as an `XContentFragment`. | 03 August 2021, 10:42:40 UTC |
fb60475 | Ed Savage | 03 August 2021, 10:06:56 UTC | [ML] Temporarily mute BWC tests (#76003) Temporarily mute BWC tests to allow #75999 to be merged Relates #75741 | 03 August 2021, 10:06:56 UTC |
0350307 | Julie Tibshirani | 03 August 2021, 08:38:15 UTC | Make sure to include class name in Query hash codes (#75871) In some Query subclasses, we forgot to include the class name when computing hashCode. This could increase the chance of collision between query hash codes. Hash code collisions shouldn't affect correctness, but could change query caching behavior since UsageTrackingQueryCachingPolicy tracks query frequency based on hash codes. The PR also simplifies some 'equals' methods to use the helper method sameClassAs. | 03 August 2021, 08:38:15 UTC |
d299954 | Bogdan Pintea | 03 August 2021, 08:35:23 UTC | EQL: correct time accounting for an intermediary response (#75804) This corrects the accounting of answer time reported in "took" attribute of an intermediary EQL async response. This has been reported as a negative value, since the delta mixes wall-clock time with VM time. (The confusion is favoured by Task#getStartTime() returning a wall-clock time, while Task#getStartTimeNanos() VM's time.) | 03 August 2021, 08:35:23 UTC |
c8c5d22 | Przemyslaw Gomulka | 03 August 2021, 08:25:21 UTC | [Rest Api Compatibility] Make query registration easier (#75722) Refactoring to NamedXContentRegistry to make it easier to register new query builders. It removes the concept of separate compatibel namedXContentRegistry and adds a second dimension - restApiVersion - to registry in NamedXContentRegistry. This makes the design similar to the solution in ObjectParser where the field parser lookup map also needs has a restApiVersion relates #51816 | 03 August 2021, 08:25:21 UTC |
5651215 | Ed Savage | 03 August 2021, 08:16:43 UTC | [ML] Add 'model_prune_window' field to AD job config (#75741) Add configuration for pruning dead split fields in anomaly detection jobs via the `model_prune_window` field for both the job creation and update APIs. Relates to ml-cpp/#1962 | 03 August 2021, 08:16:43 UTC |
8c09fc8 | Yang Wang | 03 August 2021, 05:21:31 UTC | A new search API for API keys - core search function (#75335) This PR adds a new API for searching API keys. The API supports searching API keys with a controlled list of field names and a subset of Query DSL. It also provides a translation layer between the field names used in the REST layer and those in the index layer. This is to prevent tight coupling between the user facing request and index mappings so that they can evolve separately. Compared to the Get API key API, this new search API automatically applies calling user's security context similar to regular searches, e.g. if the user has only manage_own_api_key privilege, only keys owned by the user are returned in the search response. Relates: #71023 | 03 August 2021, 05:21:31 UTC |
e4f7132 | Yang Wang | 03 August 2021, 03:43:46 UTC | Return file-backed service tokens from all nodes (#75200) The Get service account credentials API now returns file-backed tokens from all nodes instead of only the local node. For each file-backed service token, we list names of the nodes where this token is found. The response for node-local credentials (currently only file-backed tokens) is place inside the "nodes_credentials.file_tokens" field. There is also a nodes_credentials._nodes field containing information about the overall request execution (it works the same way as the _nodes field of Nodes info API, etc.) Detailed response sample can be found in #74530 This PR also removes the beta label from the API's documentation page. Resolves: #74530 | 03 August 2021, 03:43:46 UTC |
c6a90bb | Tim Vernum | 03 August 2021, 03:22:13 UTC | Add support for KeyStore filters to ssl-config (#75407) This commit adds the concept of a KeyStore filter to the SSL configuration library. Such a filter it applied to a KeyStore before it is used to construct a KeyManager, in order to modify the entries in the keystore (typically to remove entries that should not be used as SSL client/server keys). | 03 August 2021, 03:22:13 UTC |
c579664 | Tim Vernum | 03 August 2021, 02:50:43 UTC | Include removal of PKCS#11 in breaking changes doc (#75440) Relates: #75404 | 03 August 2021, 02:50:43 UTC |
61e56e5 | Mark Vieira | 02 August 2021, 22:57:41 UTC | Remove 7.x-only operating systems from pull request checks for now | 02 August 2021, 22:57:49 UTC |
e425e0a | Keith Massey | 02 August 2021, 21:55:16 UTC | Using actual time instead of cached time to avoid the snapshot time and policy action time being identical in tests (#75983) In TimeSeriesLifecycleActionsIT.testWaitForSnapshotSlmExecutedBefore() we create a snapshot, then associate an ILM policy with an index, and then that policy checks if a snapshot has been started at the same millisecond or later than the policy's action's date. Since both the snapshot start time and policy are using ThreadPool.absoluteTimeInMillis(), it is possible that they get the same cached result back (it is kept for about 200 ms). This change configures ThreadPool.absoluteTimeInMillis() to always use System.currentTimeMillis() rather than a cached time. So the policy's action date is always after the snapshot's start. | 02 August 2021, 21:55:16 UTC |
13414e9 | Armin Braun | 02 August 2021, 20:04:25 UTC | Fix Finalizing Failed Shard Snapshots (#75953) We must never write generations extracted out of failed shard snapshot status values to the repository as these can not be trusted in all cases. Instead we must always put a `null` for these into the generations object to write to the repo so that the existing generations are not changed. | 02 August 2021, 20:04:25 UTC |
cf4fde3 | James Rodewig | 02 August 2021, 18:46:43 UTC | [DOCS] Fix typo in mount searchable snapshots API docs (#75786) (#75984) Co-authored-by: Luca Wintergerst <luca.wintergerst@elastic.co> | 02 August 2021, 18:46:43 UTC |
a76ee40 | Lee Hinman | 02 August 2021, 17:15:36 UTC | Flip node shutdown feature flag to default to true on snapshot builds (#75962) * Flip node shutdown feature flag to default to true on snapshot builds It previously defaulted to false. The setting can still only be set to 'true' on a non-release (snapshot) build of Elasticsearch. Relates to #70338 * Handle case where operator privileges are enabled | 02 August 2021, 17:15:36 UTC |
dda172e | Martijn van Groningen | 02 August 2021, 17:06:46 UTC | Enable bwc tests now that #74226 has been to 7.x branch. (#75968) | 02 August 2021, 17:06:46 UTC |
ec588cb | David Pilato | 02 August 2021, 16:50:01 UTC | [DOCS] Fix extracted date (#75918) In the example, we have a date with year `2018` but the extracted date is `1998`. | 02 August 2021, 16:50:01 UTC |
f92ac0e | Armin Braun | 02 August 2021, 16:24:36 UTC | Fix Bug Causing Queued Snapshots of Deleted Indices to Never Finalize (#75942) We have to run the loop checking for completed snapshots if we see an index disappearing. Otherwise, we never get around to finalizing a queued snapshot stuck after a clone if the index is deleted during cloning. | 02 August 2021, 16:24:36 UTC |
ce537a3 | István Zoltán Szabó | 02 August 2021, 16:10:03 UTC | [DOCS] Adds link that points to outlier detection example to GET DFA stats API docs. (#75689) | 02 August 2021, 16:10:03 UTC |
75f7da3 | Adam Locke | 02 August 2021, 15:20:06 UTC | [DOCS] Updating cluser privilege for create API key (#75959) | 02 August 2021, 15:20:06 UTC |
4ea25b3 | David Turner | 02 August 2021, 15:05:03 UTC | Fix pending deletes timeout log message (#75776) In 2f0d1586925a we introduced a message reporting a timeout of 30 seconds when acquiring shard locks in order to delete an index, but in fact the timeout is 30 minutes. This commit fixes the message. | 02 August 2021, 15:05:03 UTC |
35b016c | David Turner | 02 August 2021, 14:57:18 UTC | Log reindex bulk request completion (#75935) Today we log at `DEBUG` when we receive a scroll response and send the bulk request but do not log the completion of the bulk request or the start of the next scroll request. This makes it impossible to tell from the logs how long these things are taking. This commit adds the missing logging. | 02 August 2021, 14:57:18 UTC |
9324c19 | David Turner | 02 August 2021, 14:44:20 UTC | Fix up wire format versions for backport of #75670 | 02 August 2021, 14:48:01 UTC |
4bfead0 | Rory Hunter | 02 August 2021, 14:45:53 UTC | Fix compiler warnings in :server - part 2 (#75792) Part of #40366. Fix a number of javac issues when linting is enforced in `server/`. | 02 August 2021, 14:45:53 UTC |
64855b1 | David Roberts | 02 August 2021, 14:25:37 UTC | [ML] Exceptions about starting native processes now include the node (#75937) * [ML] Exceptions about starting native processes now include the node Previously exceptions about starting native processes did not include the name of the node where the attempt to start the process failed. Although this doesn't matter when looking at the log of the node where the attempt was made, it is crippling when the report of the problem is just the exception received by a client. When the initial report comes from a client exception we need to be able to easily determine which node to ask for the logs for, or to look at operating system level issues on. * Review comments Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> | 02 August 2021, 14:25:37 UTC |
95edc6d | David Turner | 02 August 2021, 14:14:09 UTC | Clarify allocation explain if random shard chosen (#75670) Today we often encounter users that are confused by the behaviour of calling `GET _cluster/allocation/explain` without a body: it _seems_ to work, but it explains a random shard, and if this isn't the shard they're thinking of then it's unclear how to proceed. With this commit we add a note to the response when a shard was randomly chosen indicating that it is possible, and possibly useful, to explain a different shard. We also adjust the exception message in the case when all shards are assigned to indicate why it's an invalid request and what to do to make it valid. | 02 August 2021, 14:14:09 UTC |
2131c44 | Martijn van Groningen | 02 August 2021, 13:46:33 UTC | Mute bwc tests as part of backporting #74226 to 7.x branch. (#75954) | 02 August 2021, 13:46:33 UTC |
3740f67 | Rory Hunter | 02 August 2021, 13:25:40 UTC | Tighten up write permissions in Docker image (#70635) Explicitly set permissions for all files in the Elasticsearch home directory to the minimum required set, and change ownership to `root:root` where possible. | 02 August 2021, 13:25:40 UTC |
3aeac89 | nibbl3 | 02 August 2021, 13:13:38 UTC | Fix comment in default `jvm.options` configuration file (#75783) | 02 August 2021, 13:13:38 UTC |
55c4138 | James Rodewig | 02 August 2021, 13:11:23 UTC | [DOCS] Add docs for rejected requests and high CPU usage (#72640) Adds docs for rejected requests and high CPU usage. Closes #72468. Closes #69868. | 02 August 2021, 13:11:23 UTC |
f21c864 | Francisco Fernández Castaño | 02 August 2021, 12:17:37 UTC | Take into account base path on AzureBlobStoreRepositoryTests#testNotFoundErrorMessageContainsFullKey (#75934) | 02 August 2021, 12:17:37 UTC |
8d4fb3a | István Zoltán Szabó | 02 August 2021, 11:45:37 UTC | [DOCS] Changes link to outlier detection docs in PUTDFA API docs. (#75933) | 02 August 2021, 11:45:37 UTC |
29d5674 | Armin Braun | 02 August 2021, 11:14:55 UTC | Fix Force Merge Action Invoking a Listener Twice (#75897) Just a noisy thing, this isn't causing a bug since the listener is of the notify-once kind here but still. | 02 August 2021, 11:14:55 UTC |
a3ec2ee | Hendrik Muhs | 02 August 2021, 11:01:53 UTC | allow prefixing field names in random object creation. Fix test failure (#75928) caused by clashing field names. fixes #75845 | 02 August 2021, 11:01:53 UTC |
2357f6b | David Roberts | 02 August 2021, 10:20:56 UTC | [ML] Fix serialization of datafeed running state for relocated datafeed (#75923) The datafeed running state was set to null for a datafeed that had recently moved to a different node. This resulted in an exception while serializing between nodes (but would also cause incomplete information in the datafeed stats response). This change makes the datafeed running state return the best information available in this case: - real_time_configured will return the correct value - real_time_running will return false, which is reasonable since the datafeed has yet got started on the new node Fixes #75507 | 02 August 2021, 10:20:56 UTC |
6d99735 | Armin Braun | 02 August 2021, 09:51:49 UTC | Fix Needless Warnings when Restoring over Closed Index (#75912) We're trying to delete each file twice and will always warn on the second non-suppressing delete method when there's files to delete. | 02 August 2021, 09:51:49 UTC |
09a5db3 | Francisco Fernández Castaño | 02 August 2021, 09:38:37 UTC | Include full key in Azure read operations exceptions (#75920) Closes #75916 | 02 August 2021, 09:38:37 UTC |
f355d93 | David Roberts | 02 August 2021, 09:35:49 UTC | [ML] Unmute PyTorchModelIT.testEvaluate on Aarch64 (#75891) It stands a chance of working now due to https://github.com/elastic/ml-cpp/pull/1974 | 02 August 2021, 09:35:49 UTC |
b413554 | Armin Braun | 02 August 2021, 08:31:24 UTC | Fix Spurious Warnings During Snapshot Delete (#75911) We were trying to delete snapshots from index directories that they did not belong to when bulk-deleting multiple snapshots that did not all contain the same set of indices. This did not cause any further issues, except for confusing and needless warnings and needlessly attempting deletes. | 02 August 2021, 08:31:24 UTC |
6099d68 | Francisco Fernández Castaño | 02 August 2021, 08:23:53 UTC | Fix IndexRecoveryIT.testRecoverLocallyUpToGlobalCheckpoint (#75898) In certain scenarios the synced global checkpoint can fall behind due to a slow disk. The test made the assumption that the global checkpoint was stable when the global checkpoint was fetched, this commit adds a new method that waits until the global checkpoint is stable. Closes #75451 | 02 August 2021, 08:23:53 UTC |
b0f68ef | Tim Vernum | 02 August 2021, 08:17:13 UTC | Remove X-Pack specific SSL enums (#75870) This commit removes the X-Pack specific enums SSLClientAuth and VerificationMode and updates places where they were used to instead use the SslClientAuthenticationMode and SslVerificationMode enums from the ssl-config library. Relates: #68719 | 02 August 2021, 08:17:13 UTC |
e93db2a | Ignacio Vera | 02 August 2021, 05:19:18 UTC | vector tiles: Add runtime dependency on slf4j (#75869) When an illegal geometry is sent to the mvt library and fails parsing it, it tries to log an error using slf4j library. | 02 August 2021, 05:19:18 UTC |
6ea0ca4 | MiguelFerreira1998 | 02 August 2021, 04:50:52 UTC | Issue warning in certutil when using long passwords Older versions of OpenSSL (prior to 1.1.0) had a fixed 50 char buffer for password input. This means that keys (etc) encrypted with a password > 50 chars cannot be used by old versions of OpenSSL. This change adds warnings/prompts when creating encrypted files with passwords longer than 50 characters in elasticsearch-certutil. Co-authored-by: Tim Vernum <tim@adjective.org> | 02 August 2021, 04:50:52 UTC |
2dcc337 | Keith Massey | 30 July 2021, 20:55:38 UTC | Updating version for wait-for-snapshot feature after backporting to 7.15 (#75907) After backporting the fix for #73357, updating the version of serialization compatibility for the snapshot start_time field to 7.15.0.. | 30 July 2021, 20:55:38 UTC |
3d6c428 | Keith Massey | 30 July 2021, 18:57:46 UTC | Changing wait_for_snapshot to check start time rather than finish time (#75644) WaitForSnapshotStep now waits until there is a snapshot with a _start_ time after the time when the WaitForSnapshotAction began, rather than using the snapshot _finish_ time. Also, this commit changes WaitForSnapshotStep so that it uses the action start time instead of the phase start time, since WaitForSnapshotAction will likely be used in places other than the beginning of phases in the near future. And finally, this commit changes the way we get dates for ILM policies (the policy and action dates). It now uses ThreadPool.absoluteTimeInMillis(). This is to be consistent with the way that snapshot start dates are generated. Without this change it is possible to generate a snapshot after the WaitForSnapshotAction has been created, where the snapshot has a start date before the action date. Closes #73357 | 30 July 2021, 18:57:46 UTC |
4d8d88b | Howard | 30 July 2021, 17:06:36 UTC | Remove shard multiple data path selections. (#75822) Clean up multiple data path selections for new shard creation. relates #71205 | 30 July 2021, 17:06:36 UTC |
33e9c07 | Jack Conradson | 30 July 2021, 16:01:06 UTC | Remove un-used doc fields phase in Painless (#75851) Originally, a doc fields phase was created to collect information about what fields are accessed using constant values. This was going to be used for detecting cyclical field access in runtime fields, but another approach was taken instead. This change deletes the un-used phase. | 30 July 2021, 16:01:06 UTC |
6592cfe | Armin Braun | 30 July 2021, 15:46:11 UTC | Refactor SnapshotsInProgress to Use RepositoryId for Concurency Logic (#75501) This refactors the snapshots-in-progress logic to work from `RepositoryShardId` when working out what parts of the repository are in-use by writes for snapshot concurrency safety. This change does not go all the way yet on this topic and there are a number of possible follow-up further improvements to simplify the logic that I'd work through over time. But for now this allows fixing the remaining known issues that snapshot stress testing surfaced when combined with the fix in https://github.com/elastic/elasticsearch/pull/75530. These issues all come from the fact that `ShardId` is not a stable key across multiple snapshots if snapshots are partial. The scenarios that are broken are all roughly this: * snapshot-1 for index-A with uuid-A runs and is partial * index-A is deleted and re-created and now has uuid-B * snapshot-2 for index-A is started and we now have it queued up behind snapshot-1 for the index * snapshot-1 finishes and the logic tries to start the next snapshot for the same shard-id * this fails because the shard-id is not the same, we can't compare index uuids, just index name + shard id * this change fixes all these spots by always taking the round trip via `RepositoryShardId` planned follow-ups here are: * dry up logic across cloning and snapshotting more as both now essentially run the same code in many state-machine steps * serialize snapshots-in-progress efficiently instead of re-computing the index and by-repository-shard-id lookups in the constructor every time * refactor the logic in snapshots-in-progress away from maps keyed by shard-id in almost all spots to this end, just keep an index name to `Index` map to work out what exactly is being snapshotted * refactoring snapshots-in-progress to be a map of list of operations keyed by repository shard id instead of a list of maps as it currently is to make the concurrency simpler and more obviously correct closes #75423 relates (#75339 ... should also fix this, but I have to verify by testing with a backport to 7.x) | 30 July 2021, 15:46:11 UTC |
ffeaab8 | Armin Braun | 30 July 2021, 14:09:11 UTC | Fix Issues in Netty4MessageChannelHandler (#75861) Fixes a few rough edges in this class: * we need to always pass a flush call down the pipeline and not just conditionally if they apply to the message handler, otherwise we lose flushes e.g. when a channel becomes not-writable due to a write from off the event-loop that exceeds the outbound buffer size * this is suspected of causing recently observed intermittent and unexplained slow message writes (logged by the outbound slow logger) where a message became stuck until a subsequent message was sent (e.g. during period leader checks or so) * Pass size `0` messages down the pipeline instead of just resolving their promise to avoid unexpected behavior (though we don't make use of `0`-length writes as of today * Avoid unnecessary flushes in queued-writes loop and only flush if the channel stops being writable * Release buffers on queued writes that we fail on channel close (not doing this wasn't causing bugs today because we release the underlying bytes elsewhere but could cause trouble later) Unfortunately, I was not able to reproduce the issue in the first point reliably as the timing is really tricky. I therefore tried to make this PR as short and uncontroversial as possible. I think there's possible further improvements here and this should have been caught by a test but it's not yet clear to me how to design a reliable reproducer here. | 30 July 2021, 14:09:11 UTC |
54bdc09 | Lukas Wegmann | 30 July 2021, 13:28:00 UTC | SQL: Fix groupings on empty results and HAVING on local relations (#74809) Resolves #74311 The goal of this PR is to make implicit and explicit groupings behave consistently on empty result sets no matter whether the query is run on ES or optimized to a local relation. A typical example is a query like `SELECT COUNT(*) FROM test_emp WHERE languages = 2` vs `SELECT COUNT(*) FROM test_emp WHERE 1 = 2` where the later used to produce an empty result set instead of the expected 1 row with a count of 0. The PR also fixes other edge case around contradictions in `HAVING` clauses that used to produce a wrong result: `SELECT COUNT(*) c, 'a' a HAVING COUNT(*) > 1000` | 30 July 2021, 13:28:00 UTC |
292abd9 | Benjamin Trent | 30 July 2021, 13:03:20 UTC | [ML] ensure tasks API executes through ML_ORIGIN (#75880) | 30 July 2021, 13:03:20 UTC |
1eaf1be | James Rodewig | 30 July 2021, 13:01:26 UTC | [DOCS] Reword internal use copy for `dimension` mapping parameter | 30 July 2021, 13:01:46 UTC |
8b9b8a9 | Nhat Nguyen | 30 July 2021, 12:29:58 UTC | Unsigned long should accept value with zero decimal (#75646) Closes #67565 | 30 July 2021, 12:29:58 UTC |
828a36d | Stef Nestor | 30 July 2021, 12:20:26 UTC | Add frequently shared Painless field catch script (#75758) Support frequently shares https://gist.github.com/jclosure/8e103dee2f7e9491845a2c0bb64c6b7a#gistcomment-3780127, adds into docs | 30 July 2021, 12:20:26 UTC |
d86abdc | Pius | 30 July 2021, 12:02:05 UTC | Suggest DEBUG level logging instead for SAML (#74661) * Suggest DEBUG level logging instead for SAML * Update troubleshooting.asciidoc * Incorporate reviewer feedback Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Adam Locke <adam.locke@elastic.co> | 30 July 2021, 12:02:05 UTC |
cbf5009 | Martijn van Groningen | 30 July 2021, 11:47:17 UTC | Add resolve_during_rolling_upgrade field to deprecation issue (#74226) Add an additional boolean field to a deprecation issue to indicate that a deprecation issue can only be resolved during a rolling upgrade when a node is offline (for the upgrade). No deprecation issue has been marked as restart required as part of this change. Closes #73091 | 30 July 2021, 11:47:17 UTC |
c287841 | David Roberts | 30 July 2021, 10:20:00 UTC | [ML] Ignore exceptions while opening job after SIGTERM to JVM (#75850) * [ML] Ignore exceptions while opening job after SIGTERM to JVM We observed that some jobs failed during a rolling upgrade in Elastic Cloud. This happened because steps of the job open sequence failed with exceptions after core Elasticsearch services shut down in response to the SIGTERM. This change makes the persistent task executor for anomaly detection jobs ignore exceptions received after the JVM has received a shutdown signal, for example a SIGTERM. By doing nothing in response to such exceptions the persistent task remains in cluster state and will get assigned to a different node after the current node leaves the cluster. * Address review comment | 30 July 2021, 10:20:00 UTC |
98554b4 | Tim Vernum | 30 July 2021, 02:53:55 UTC | [TEST] Replace reference to blacklist with failed (#75866) This commit replaces use of "blacklist" with "failed" to better reflect the behaviour of the test Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> | 30 July 2021, 02:53:55 UTC |
0b032fa | Mark Vieira | 29 July 2021, 23:07:00 UTC | Add SLES 15 SP3 to docker exclusion list | 29 July 2021, 23:07:12 UTC |
02ea1f9 | Gordon Brown | 29 July 2021, 17:36:04 UTC | Properly apply `system` flag on data streams when restoring a snapshot (#75819) This commit modifies the restore process to ensure that the `system` flag is properly applied to restored data streams. Otherwise, this flag is lost when restoring system data streams, which causes errors and/or assertion failures as the backing indices are properly marked as system indices, but the restored data stream is no longer a system data stream. Also adds a test to ensure this flag survives a round trip through the snapshot/restore process. | 29 July 2021, 17:36:04 UTC |
fd6d966 | Przemysław Witek | 29 July 2021, 16:51:14 UTC | [ML] Adapt wire serialization code and re-enable BWC tests after backport (#75847) | 29 July 2021, 16:51:14 UTC |
7ace25a | Przemysław Witek | 29 July 2021, 15:03:25 UTC | [ML] disable bwc for backporting new Job setting (system_annotations_retention_days) (#75843) | 29 July 2021, 15:03:25 UTC |
1518976 | Jake Landis | 29 July 2021, 14:00:37 UTC | [DOCS] 8.0.0-alpha1 release notes Adds release notes for 8.0.0-alpha1. Co-authored-by: lcawl <lcawley@elastic.co> Co-authored-by: David Roberts <dave.roberts@elastic.co> Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com> Co-authored-by: Lukas Wegmann <wegmaluk@gmail.com> | 29 July 2021, 14:00:37 UTC |
30d9f13 | Przemysław Witek | 29 July 2021, 13:27:03 UTC | [ML] Delete expired annotations (#75617) | 29 July 2021, 13:27:03 UTC |
c4f3b93 | Benjamin Trent | 29 July 2021, 12:56:47 UTC | [ML] throttle job audit msgs if delayed data occurs for consecutive buckets (#75815) In a consistently delayed environment, it is possible that many buckets in a row have delayed data detected. This can flood the audit message page, causing noise that detracts from other useful information This commit reduces the number of audit messages written as the number of consecutive buckets with delayed data increases. closes https://github.com/elastic/elasticsearch/issues/73679 | 29 July 2021, 12:56:47 UTC |
02d0414 | Benjamin Trent | 29 July 2021, 11:29:52 UTC | [ML] fix count ks test aggregator test consistency (#75793) testKsTest_LowerTailedValues is failing due to distribution issues. The random variables in the sparse case are too uniform and causing two-sided alternative tests to not be consistent. closes #74909 | 29 July 2021, 11:29:52 UTC |
27e27e0 | Armin Braun | 29 July 2021, 10:50:00 UTC | Make Authentication/Authorization Stacks Shallower/Simpler (#75662) Same as #75252 pretty much just continuing to make this logic a little simpler for easier profiling and (very) maybe performance through saving some allocations/indirection. | 29 July 2021, 10:50:00 UTC |
2933914 | David Roberts | 29 July 2021, 09:12:20 UTC | [ML] Don't try to respond to shutdown API when disabled (#75828) The ML plugin should not try to respond to shutdown API calls when disabled. | 29 July 2021, 09:12:20 UTC |
57b5a1b | Andrei Dan | 29 July 2021, 07:51:08 UTC | Docs: ILM document behaviour for changing lifecycle setting (#75790) Add xref Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com> | 29 July 2021, 07:51:08 UTC |
d15445e | Adrien Grand | 29 July 2021, 06:36:09 UTC | Remove usage of RAM accounting of segments (#75674) This is a pre-requisite for the upgrade to Lucene 9, which removes the ability to estimate RAM usage of segments. | 29 July 2021, 06:36:09 UTC |
b7fc0ac | Yang Wang | 29 July 2021, 01:57:42 UTC | Fix privileges for GetRollupIndexCapabilities API (#75614) In addition to read, access to this API is now also granted by view_index_metadata and manage. Resolves: #74779 | 29 July 2021, 01:57:42 UTC |
36ee1e1 | Mark Vieira | 28 July 2021, 22:50:09 UTC | Remove Jenkins matrix job combination filter | 28 July 2021, 22:50:09 UTC |
9371273 | Mark Vieira | 28 July 2021, 22:43:49 UTC | Fix Jenkins job configuration syntax | 28 July 2021, 22:43:49 UTC |
be4b09f | Mark Vieira | 28 July 2021, 22:39:54 UTC | Ensure we still test older operating systems where supported | 28 July 2021, 22:40:16 UTC |
73e3b60 | Adam Locke | 28 July 2021, 18:53:49 UTC | [DOCS] Steps for updating TLS certificates (#73781) * [DOCS] Steps for updating TLS certificates * Updates for changing CA * Updates for rotating certs with a new CA * Add instructions for generating HTTP certs with a new CA * Add steps for creating HTTP certs with new CA * Clarify note about cluser restart and other edits * Clarifying scenarios * Apply suggestions from code review Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com> * Incorporating review feedback and making necessary changes * Clarifications and changes regarding restarts * Remove errant --pem in basic security setup * Incorporate suggestions from code review Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com> * Many, many updates. But good ones. * Add languages for snippets * Reorder steps to reference rolling restart throughout for consistency * Add clarifying what's next steps * Add instructions for updating Kibana certificate * Apply suggestions from Ioannis' stellar code review Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com> * Update instructions to use a single keystore, plus other review changes * Incorporating another round of review comments * Minor updates from reviewer feedback * Clarifying examples and fixing numbering * Skip tests that are creating unnecessary noise * Quieting other tests Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com> | 28 July 2021, 18:53:49 UTC |
c1ba949 | Lisa Cawley | 28 July 2021, 18:29:48 UTC | [DOCS] Fixes bulleted list in ML aggregations (#75806) | 28 July 2021, 18:29:48 UTC |
52de9d1 | Benjamin Trent | 28 July 2021, 17:42:14 UTC | [ML] fixing potential network thread lockup on Pytorch model load (#75784) After the model is loaded via the search request, that searching thread MAY be a network thread. Consequently, we should not lock it up while creating the named pipes. This commit addresses by forcing named pipe creation to be done within the ML_UTILITY executor threadpool. | 28 July 2021, 17:42:14 UTC |
f04eed7 | David Kyle | 28 July 2021, 16:59:04 UTC | [ML] Mute PyTorchModelIT in advance of input format changes (#75800) For elastic/ml-cpp#1961 and #75759 | 28 July 2021, 16:59:04 UTC |
f0008d3 | David Roberts | 28 July 2021, 16:08:09 UTC | Removing local abort availability checks (#75785) Now that #74115 is backported to 7.x the code to check whether local abort is supported within a cluster is redundant, as 8.x only supports running in a mixed cluster with 7.last, and 7.last contains the local abort functionality. This change removes the redundant code. Followup to #74115 | 28 July 2021, 16:08:09 UTC |
8211fa6 | Mark Vieira | 28 July 2021, 16:07:08 UTC | Avoid running all EQL BWC tasks when running check (#75743) The bwc-test plugin automatically wires up BWC tasks to the appropriate lifecycle tasks for us. There's no need to do this explicitly. Running 'check' should only run BWC tests against unreleased versions as-per convention. Released version tests are run via periodic jobs in CI. | 28 July 2021, 16:07:08 UTC |
0790107 | Keith Massey | 28 July 2021, 14:43:49 UTC | Re-enabling BWC tests, and updating supported versions for shard count feature after backport (#75789) | 28 July 2021, 14:43:49 UTC |
fb0846a | Hendrik Muhs | 28 July 2021, 14:34:17 UTC | [Transform][Rollup] remove unnecessary list indirection (#75459) Remove an unnecessary indirection and refactor progress tracking. Both rollup and transform process documents as stream, however in the AsyncTwoPhaseIndexer takes a List of index requests. This change removes the unnecessary temporary container and makes upcoming transform enhancements easier. | 28 July 2021, 14:34:17 UTC |
e99160d | Keith Massey | 28 July 2021, 14:01:05 UTC | Disabling BWC tests for _nodes/stats shard count backport (#75787) | 28 July 2021, 14:01:05 UTC |
c96139d | Przemyslaw Gomulka | 28 July 2021, 12:17:49 UTC | [Rest Api Compatibility] Deprecate the use of synced flush (#75372) synced flush is going to be replaced by flush. This commit allows to synced_flush api only in v7 compatibility mode. Worth noting - sync_id is gone and won't be available in v7 responses from indices.stats relates removal pr #50882 relates #51816 | 28 July 2021, 12:17:49 UTC |