Skip to main content

Browse the archive

https://github.com/elastic/elasticsearch
05 April 2024, 04:14:31 UTC
sort by:
Revision Author Date Message Commit Date
31df968 debadair 29 March 2022, 00:05:58 UTC [DOCS] Reuse data tier content in node role docs (#84346) (#85426) (#85429) * [DOCS] Reuse data tier content in node role docs (#84346) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> * [DOCS] Fix duplicate anchor (#85424) * [DOCS] Fix duplicate anchor * Removed misplaced end tag Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> 29 March 2022, 00:05:58 UTC
be4576b Mark Vieira 28 March 2022, 22:35:16 UTC Fall back to naive parallel forks calculation in non-standard cases (#85410) (#85422) Some architectures provide non-standard output in /proc/cpuinfo. Specifically, AWS graviton. In case we can't properly calculate physical cpu cores via the standard method just fallback to using the JDK-backed implementation. 28 March 2022, 22:35:16 UTC
16d4d07 Elasticsearch addict 28 March 2022, 19:26:52 UTC Update histogram-aggregation.asciidoc (#85356) Fix small grammatical mistake. Closes #85355 28 March 2022, 19:26:52 UTC
16850bc Mark Vieira 28 March 2022, 18:31:58 UTC Fix calculation of max parallel forks for test execution (#85360) (#85403) 28 March 2022, 18:31:58 UTC
fe97af9 Benjamin Trent 28 March 2022, 15:15:33 UTC [ML] fix race condition with job snapshot upgrade and feature reset (#85121) (#85396) This addresses a race condition in feature reset && model snapshot upgrade. It is possible that feature reset is called soon after the task is assigned, meaning the persistent task is cancelled before the model snapshot upgrader is set on the task. This means that the task could continue executing even after being cancelled as nothing else checks if the feature is being reset or if the task has been cancelled. closes #84709 28 March 2022, 15:15:33 UTC
a2ae13f Joe Gallo 28 March 2022, 14:31:10 UTC Replace failing PolicyStepsRegistry sanity check assert with a test (#85346) 28 March 2022, 14:32:14 UTC
f012269 Polar Knight 28 March 2022, 09:37:55 UTC Fix so curls the correct location of cert (#85364) 28 March 2022, 09:37:55 UTC
bc9de12 Nhat Nguyen 26 March 2022, 19:54:58 UTC Limit concurrent shard requests in disk usage API (#84900) (#85367) The disk_usage API fails on indices that have more than 16 shards. The reason is that we broadcast all shard requests simultaneously, and the handling thread pool - the analyzer thread pool - whose queue size is only 16. That means some shard requests will be rejected. This PR limits the number of concurrent shard requests sent to each node when we broadcast them. Closes #84779 26 March 2022, 19:54:58 UTC
9576c09 Nhat Nguyen 17 March 2022, 17:10:38 UTC TransportBroadcastAction should always set response for each shard (#84926) The TransportBroadcastAction can leave the responses of some group unset if all shards of that group hit shard_unavailable_exception. 26 March 2022, 18:39:48 UTC
53606ba Nhat Nguyen 08 March 2022, 21:22:46 UTC Relax error handling in transport disk usage API (#84774) Relax the failure handling to cover other exceptions such as BroadcastShardOperationFailedException. 26 March 2022, 18:33:04 UTC
fbe7cf7 Ryan Ernst 25 March 2022, 05:48:23 UTC Redirect testclusters ES process stdout/stderr to log file (#85349) (#85354) Although most of the time the stdout and stderr of the Java process is not needed since ES has its own log file, there are some cases where it is very important to have the output before ES has initialized its logging system. This commit changes the stdout/stderr of the ES process created by testclusters to use the same log file that ES will write to. There should not be contention with ProcessBuilder and ES trying to write at the same time because ES swaps out the stream handles to write to its logging system, so after that point ProcessBuilder will never see anymore output. relates #68333 25 March 2022, 05:48:23 UTC
b7aa27a David Roberts 24 March 2022, 19:58:48 UTC [ML] Do not fetch source when finding index of last state docs (#85334) (#85344) There are two places where the ML code needs to find out which index in the rolling sequence of state indices the previous state document is in. Previously the search to do this was needlessly retrieving _source, which wastes memory. The change tells the two searches not to fetch _source, only the metadata. 24 March 2022, 19:58:48 UTC
70d719e Keith Massey 24 March 2022, 19:49:00 UTC Allowing _cat/allocation to return more than 0 shards in test (#84539) (#85342) We delete all indices in a cluster before running ClientYamlTestSuiteIT. However some system indices can regenerate. The cat.allocation, cat.indices, and cat.shards tests were failing because they assume that no indices not created by the tests exist. They can now all handle system indices being created during the test. Closes #83719 Closes #84765 Closes #84972 Closes #82151 Closes #82660 24 March 2022, 19:49:00 UTC
83d3eb9 Lee Hinman 24 March 2022, 17:42:04 UTC Clarify where force merge will occur in ILM (#85187) (#85337) This commit clarifies the force merge behavior for the `forcemerge` action either as a standalone ILM action, or as part of the searchable snapshot action. When using a searchable snapshot it can be confusing especially because the force merge can occur in a phase prior to the phase configured in the ILM policy (since ILM doesn't migrate the index prior to the searchable snapshot mounting). 24 March 2022, 17:42:04 UTC
e255e40 David Kyle 24 March 2022, 14:12:45 UTC [ML] Reallocate model deployments on node shutdown events. (#85310) (#85329) Trigger reallocation on Node Shutdown changes first then node change events if the Node Shutdown API has not been used. 24 March 2022, 14:12:45 UTC
882cd57 Rene Groeschke 24 March 2022, 13:53:13 UTC Update nebula plugins improving configuration cache support (#85316) (#85327) 24 March 2022, 13:53:13 UTC
885a97d Rory Hunter 24 March 2022, 11:34:29 UTC Tweak generate release notes version filtering 24 March 2022, 11:35:28 UTC
9fbfc05 Tanguy Leroux 24 March 2022, 08:40:31 UTC Fix leaking listeners bug on frozen tier (#85239) (#85301) We identified a bug in searchable snapshot code that could prevent some search queries to be completed. This bug results in search tasks lingering on a data_frozen node for hours or days, filling the search thread pool and blocking other search requests to be processed. This bug can be reproduced on a data_frozen node with: multiple search queries requesting the same region from the shared cache the region corresponds to cached data not yet available in cache the searchable snapshot's thread pool that fetches cached data has no active workers the thread that will write the cached data (ie, the thread that picks up the Gap to fill) has restrictive permissions that prevents new thread to be created When such a situation occurs, the thread pool in charge of executing the AbstractRunnable that fills the cache throws an exception (in our case, an AccessControlException) that is not caught and not logged. In consequence the Gap that should have been written in cache is never filled with data and the listeners waiting for the cached data to be available won't be called back, blocking the execution of the searches waiting for the data. This pull request introduces an integration test that reproduce this situation: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "modifyThreadGroup") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:485) ~[?:?] at java.security.AccessController.checkPermission(AccessController.java:1068) ~[?:?] at java.lang.SecurityManager.checkPermission(SecurityManager.java:416) ~[?:?] at org.elasticsearch.secure_sm.SecureSM.checkThreadGroupAccess(SecureSM.java:186) ~[?:8.2.0-SNAPSHOT] at org.elasticsearch.secure_sm.SecureSM.checkAccess(SecureSM.java:135) ~[?:8.2.0-SNAPSHOT] at java.lang.ThreadGroup.checkAccess(ThreadGroup.java:335) ~[?:?] at java.lang.Thread.<init>(Thread.java:422) ~[?:?] at java.lang.Thread.<init>(Thread.java:708) ~[?:?] at org.elasticsearch.common.util.concurrent.EsExecutors$EsThreadFactory.newThread(EsExecutors.java:274) ~[?:8.2.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:630) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:920) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364) ~[?:?] at org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor.execute(EsThreadPoolExecutor.java:66) ~[?:8.2.0-SNAPSHOT] at org.elasticsearch.xpack.searchablesnapshots.cache.shared.FrozenCacheService$CacheFileRegion.populateAndRead(FrozenCacheService.java:769) [?:8.2.0-SNAPSHOT] at org.elasticsearch.xpack.searchablesnapshots.cache.shared.FrozenCacheService$FrozenCacheFile.populateAndRead(FrozenCacheService.java:896) [?:8.2.0-SNAPSHOT] at org.elasticsearch.xpack.searchablesnapshots.store.input.FrozenIndexInput.readWithoutBlobCache(FrozenIndexInput.java:143) [?:8.2.0-SNAPSHOT] at org.elasticsearch.xpack.searchablesnapshots.store.input.MetadataCachingIndexInput.doReadInternal(MetadataCachingIndexInput.java:104) [?:8.2.0-SNAPSHOT] at org.elasticsearch.xpack.searchablesnapshots.store.input.BaseSearchableSnapshotIndexInput.readInternal(BaseSearchableSnapshotIndexInput.java:112) [?:8.2.0-SNAPSHOT] at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:291) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.apache.lucene.store.BufferedIndexInput.readShort(BufferedIndexInput.java:243) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.apache.lucene.util.packed.DirectReader$DirectPackedReader16.get(DirectReader.java:302) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$20.ordValue(Lucene90DocValuesProducer.java:852) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.apache.lucene.index.SingletonSortedSetDocValues.advanceExact(SingletonSortedSetDocValues.java:81) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.elasticsearch.index.fielddata.FieldData$10.advanceExact(FieldData.java:336) [?:8.2.0-SNAPSHOT] at org.elasticsearch.script.field.AbstractKeywordDocValuesField.setNextDocId(AbstractKeywordDocValuesField.java:44) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.lookup.LeafDocLookup.getScriptFieldFactory(LeafDocLookup.java:74) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:89) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:27) [?:8.2.0-SNAPSHOT] at org.elasticsearch.painless.PainlessScript$Script.execute(doc['text.raw'].value.toString().length() > 0:5) [?:?] at org.elasticsearch.index.query.ScriptQueryBuilder$ScriptQuery$1$1.matches(ScriptQueryBuilder.java:193) [?:8.2.0-SNAPSHOT] at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:281) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:254) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:45) [?:8.2.0-SNAPSHOT] at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.elasticsearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:194) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:167) [?:8.2.0-SNAPSHOT] at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:555) [?:9.1.0 5b522487ba8e0f1002b50a136817ca037aec9686 - jtibs - 2022-03-16 10:32:40] at org.elasticsearch.search.query.QueryPhase.searchWithCollector(QueryPhase.java:232) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.query.QueryPhase.executeInternal(QueryPhase.java:187) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:88) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:462) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:625) [?:8.2.0-SNAPSHOT] at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$2(SearchService.java:487) [?:8.2.0-SNAPSHOT] at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:47) [?:8.2.0-SNAPSHOT] at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62) [?:8.2.0-SNAPSHOT] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [?:8.2.0-SNAPSHOT] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) [?:8.2.0-SNAPSHOT] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [?:8.2.0-SNAPSHOT] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [?:8.2.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:833) [?:?] This permission issue has been fixed yesterday by #85180 but I still think it is valuable to have an integration test that combin scripts and searchable snapshot. In addition to this, this pull request changes the searchable snapshot code to always fail any Gap that was not correctly submitted for execution: this way the current search (and any other searches that are waiting for the cached data to be available) returns a search failure and not block indefinitely. In order to catch any future similar issue we also log and assert any exception that could be thrown when submitting the task to fill the required gaps. 24 March 2022, 08:40:31 UTC
99a6092 Rene Groeschke 24 March 2022, 08:13:44 UTC Update backportrc after 8.1.1 release 24 March 2022, 08:13:44 UTC
c0dcd0c Henning Andersen 24 March 2022, 06:52:15 UTC Fix restart with auto-expand replicas (#85277) No longer contract `number_of_replicas` when a node is marked restarting, since this poses a risk of the shard/index becoming unavailable. 24 March 2022, 07:42:12 UTC
ba84ec5 Jake Landis 23 March 2022, 21:55:06 UTC No longer require master node to install Watcher templates (#85287) (#85294) The master node was required to install Watcher index templates in earlier versions of 7.x to ensure that the master node was aware of composable index templates. When upgrading to 8.x this is no longer an issue since all upgrade paths require elder version of Elasticsearch to be on a version that is aware of composable index templates. This should fix https://github.com/elastic/elasticsearch/issues/85043 by ensuring that the watcher history index template is always in place prior to it's intended usage irregardless of the order of which nodes are upgraded. 23 March 2022, 21:55:06 UTC
e3a5d9a Rory Hunter 23 March 2022, 20:38:47 UTC Formatting 23 March 2022, 20:38:47 UTC
ff9b481 Rory Hunter 23 March 2022, 20:29:33 UTC Split release notes pages to be per-patch version Follow up to #85279 / 4ec75537cdd. 23 March 2022, 20:33:33 UTC
c693f8c Rory Hunter 23 March 2022, 20:08:14 UTC Generate a release notes page per patch version (#85279) Closes #85250. The approach of generating a single asciidoc page for all releases in a minor series makes it harder to preserve any manual edits that we have to make. Instead, generate a page per-version. It should make little difference to the final documentation that users see. 23 March 2022, 20:08:14 UTC
6563e01 William Brafford 23 March 2022, 19:12:41 UTC Validate index format agreement for system index descriptors (#85173) (#85289) * Validate index format for system indices * Add bwc test for watcher meta version issue * Update docs/changelog/85173.yaml 23 March 2022, 19:12:41 UTC
882f789 Tanguy Leroux 23 March 2022, 17:17:51 UTC Wait longer for CCR auto-follow stats to appear in monitoring index (#85278) (#85285) The test `AutoFollowIT.testAutoFollowPatterns()` failed multiple times when waiting for CCR's auto-follow stats to be collected and indexed in the `.monitoring-es-*` index. This is possibly because monitoring data are collected every 10 seconds and some of them can take more time to be collected, exceeding the 30s timeout in the test. This pull request re-uses the `assertLongBusy()` so that the test will wait more time for the data to show up and in case the data are not here it will prints out the current CCR's auto-follow stats for debugging purpose. Closes #84403 23 March 2022, 17:17:51 UTC
19e1e43 Rory Hunter 23 March 2022, 14:23:50 UTC Fix version files when generating release notes (#85264) Closes #85227. The git label version filter didn't take into account snapshot versions. Now it does. 23 March 2022, 14:25:55 UTC
f7138ac Keith Massey 23 March 2022, 13:59:13 UTC Return empty version instead of blowing up if we cannot find it (#85244) (#85272) SystemIndexManager.readMappingVersion() throws an exception in a few cases if we cannot find a valid version. For example in some really old versions of certain system indices we see version numbers like 7090399. This commit changes that method to return an empty version instead. 23 March 2022, 13:59:13 UTC
85435a1 Przemyslaw Gomulka 23 March 2022, 11:28:51 UTC Improve DeprecationHttpIT robustness (#85235) (#85262) Backports the following commits to 8.1: Improve DeprecationHttpIT robustness (Improve DeprecationHttpIT robustness #85235) 23 March 2022, 11:28:51 UTC
fff4c22 Przemyslaw Gomulka 23 March 2022, 10:22:13 UTC [8.1] Cleanup SystemIndexMigration tests (#84281) (#85153) backports * Cleanup SystemIndexMigration tests (#84281) * Fix randomness in SystemIndexMigrationIT (#85224) 23 March 2022, 10:22:13 UTC
772cd96 Gordon Brown 22 March 2022, 22:57:27 UTC Re-add content from removed Security-specific snapshot pages (#85084) (#85252) This commit adds back some notes which were lost when we consolidated the snapshot/restore documentation into a single location. The notes in question are that: 1) If a Snapshot repository contains Security's feature state, then that repository contains security-sensitive information. This may be obvious to some, but is good to state explicitly. 2) Some files, such as the keystore and TLS keys, are not included in snapshots and are important to back up via other means. 22 March 2022, 22:57:27 UTC
26bffda David Turner 22 March 2022, 16:35:48 UTC Fix auto-bootstrap docs (#85215) Today it's no longer true that by default nodes will auto-discover other nodes on the same host and bootstrap them all into a cluster. This commit fixes the docs on auto-bootstrapping to recognise this. 22 March 2022, 16:36:26 UTC
7affcd0 Nhat Nguyen 22 March 2022, 15:01:21 UTC Bump versions for 8.1.1 release 22 March 2022, 15:02:45 UTC
e58ea6e Henning Andersen 22 March 2022, 12:14:04 UTC [DOCS] Searchable snapshot data persistence (#85218) (#85219) Clarified that fully mounted indices persists on-disk data across restarts whereas partially mounted indices are cleared upon restarts. 22 March 2022, 12:14:04 UTC
af70f3b David Roberts 22 March 2022, 12:13:46 UTC Add back ML 8.1.0 release notes (#85205) These were accidentally deleted in #85190 22 March 2022, 12:13:46 UTC
55b6a01 Andrew Wilkins 22 March 2022, 08:01:09 UTC [8.1] Add delete privilege to kibana_system for APM (#85085) (#85199) * Add delete privilege to kibana_system for APM (#85085) * Add delete privilege to kibana_system for APM * Update docs/changelog/85085.yaml * Loosen APM data stream wildcards (To ensure we match future APM data streams as well.) * Fix trailing comma * Fix linting * Make data stream patterns more specific * Update docs/changelog/85199.yaml * Delete 85199.yaml 22 March 2022, 08:01:09 UTC
cd3e042 Nhat Nguyen 22 March 2022, 00:37:04 UTC Add 8.1.1 release notes (#85190) Add 8.1.1 release notes 22 March 2022, 00:37:04 UTC
e7b44bc Ryan Ernst 21 March 2022, 19:35:25 UTC Wrap thread creation in doPrivileged call (#85180) (#85183) EsExecutors has a thread factory for thread construction, and both creates a thread in a given thread group and sets it as a daemon thread. Currently that thread creation happens in the access control context of the calling code, but this could happen from anywhere inside Elasticsearch. Since the point of EsExecutors is be the one place handling thread creation (for the most part), this should happen in the context of server, without caring about the whatever code triggered the thread pool to expand. 21 March 2022, 19:35:25 UTC
00439e5 Nik Everett 21 March 2022, 15:25:45 UTC REST tests for variable width historam (#84836) (#85117) This expands on the super basic REST tests for variable width histogram with some ever so slightly less basic tests. They are still pretty basic, but they do a good job of making sure that we don't break backwards compatibility with the API. Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> 21 March 2022, 15:25:45 UTC
da91af7 Przemyslaw Gomulka 21 March 2022, 15:13:24 UTC Disable deprecation log indexing in docker test (#85150) (#85155) Backports the following commits to 8.1: Disable deprecation log indexing in docker test (Disable deprecation log indexing in docker test #85150) 21 March 2022, 15:13:24 UTC
99ca15a Mark Vieira 18 March 2022, 20:00:46 UTC Add isolated test coverage for example plugins 18 March 2022, 20:17:35 UTC
cda166a David Turner 18 March 2022, 16:42:38 UTC AwaitsFix for #84403 18 March 2022, 16:51:11 UTC
298d625 Nhat Nguyen 18 March 2022, 16:08:08 UTC Increase store ref before snapshotting index commit (#84776) (#85112) Snapshotted commits should also hold a reference to the store, so they are always usable; otherwise, callers need to manage the store's references manually. This change applies only to InternalEngine as we already do this in ReadOnlyEngine. 18 March 2022, 16:08:08 UTC
0d63c5a Ignacio Vera 18 March 2022, 15:10:11 UTC Make sure the random generated shape can be indexed in GeoTilerTests (#84986) (#85103) Make sure the random generated shape can be indexed 18 March 2022, 15:10:11 UTC
488cf48 David Roberts 18 March 2022, 14:27:10 UTC [ML] Fixes for multi-line start patterns in text structure endpoint (#85066) (#85100) This PR contains 3 fixes for the way multi-line start patterns are created in the text structure endpoint: 1. For delimited files the multi-line start pattern used to be based on the first field that was a timestamp, boolean or number. This PR adds the option of a low cardinality keyword field as an alternative (i.e. an enum field effectively). It means there is more chance of a field early in each record being chosen as the mechanism for determining whether a line is the first line of a record. 2. The multi-line start pattern for delimited files now only permits the delimiter character between fields, not within quoted fields. Previously it was possible for the multi-line start pattern to match continuation lines. Unfortunately this may mean we can no longer determine a multi-line start pattern for files whose only suitable field is to the right of fields that sometimes contain commas, and the only solution in this case will be to reorder the columns before importing the data. Hopefully this problem will be very rare. 3. For semi-structured text log files there is now a cap on the complexity of the multi-line start pattern. It has been observed that the patterns generated for slightly malformed CSV files could run for days against the malformed lines of those files - the classic problem of a regex that doesn't match but nearly does doing lots of backtracking. We now throw an error in this situation and suggest overriding the format to delimited. Relates #79708 Fixes elastic/kibana#121966 18 March 2022, 14:27:10 UTC
d0925dd James Rodewig 17 March 2022, 20:27:39 UTC [DOCS] Fix Vector tiles search docs for features.id (#85067) (#85076) Removes the `features.id` property from the response body. This property was actually generated by the tool used to decode the mvt file to JSON. (cherry picked from commit a780558e4c1812d324c759d1f230576f6413e3bf) # Conflicts: # docs/reference/search/search-vector-tile-api.asciidoc Co-authored-by: Ignacio Vera <ivera@apache.org> 17 March 2022, 20:27:39 UTC
5f02f41 Mayya Sharipova 17 March 2022, 10:32:46 UTC Clarify max number of dims for indexed vectors (#85002) 17 March 2022, 10:43:24 UTC
f0e2216 Luca Cavanna 16 March 2022, 19:41:48 UTC DotExpandingXContentParser to expose the original token location (#84970) With #79922 we have introduced a parser that expands dots in fields names on the fly, so that the expansion no longer needs to be handled by consumers. The token location exposed by such parser can be confusing to interpret: consumers are parsing the expanded version which requires jumping ahead reading tokens and exposing additional field names and start objects, while users have sent the unexpanded version and would like errors to refer to the original content. This commit adds a test for this scenario and tweaks the DotExpandingXContentParser to cache the token location before jumping ahead to expand dots in field names. 16 March 2022, 19:41:48 UTC
0bb4c2b Martijn van Groningen 16 March 2022, 16:13:06 UTC Forward port #85000 and #84838 to 8.1 branch (#85040) Forward port #85000 (Revert enrich cache lookup optimisation ) and #84838 (CompoundProcessor should also catch exceptions when executing a processor) to 8.1 branch. Revert enrich cache lookup optimisation (#85028) Forwardporting #85000 to master branch. This PR reverts the optimisation that was added via #77259. This optimisation cleverly ensures no duplicate searches happen if multiple threads concurrently execute the same search. However there are issues with the implementation that cause issues like #84781. The optimisation make use of CompletableFuture and in this case we don't check whether the result has completed exceptionally. Which causes the callback not being invoked and this leads to bulk request not being completed and hanging around. The ingest framework due to its asynchronous nature is already complex and adding CompletableFuture into the mix makes debugging these issues very time consuming. This is the main reason why we like to revert this commit. * CompoundProcessor should also catch exceptions when executing a processor (#84838) (#85035) Currently, CompoundProcessor does not catch Exception and if a processor throws an error and a method higher in the call stack doesn't catch the exception then pipeline execution stalls and bulk requests may not complete. Usually these exceptions are caught by IngestService#executePipelines(...) method, but when a processor executes async (for example: enrich processor) and the thread that executes enrich is no longer the original write thread then there is no logic that deals with failing pipeline execution and cleaning resources up. This then leads to memory leaks. Closes #84781 Also change how 'pipeline doesn't exist' error is thrown in TrackingResultProcessor. With the change to CompoundProcessor thrown exceptions are caught and delegated to handler. SimulateExecutionService in verbose mode ignores exceptions delegated to its handler, since it assumes that processorResultList contains the result (successful or not successful) of every processor in the pipeline. In case TrackingResultProcessor for PipelineProcessor couldn't find the mentioned pipeline then it just throws an error without updating the processorResultList. This commit addresses that. 16 March 2022, 16:13:06 UTC
d6cdf13 Lee Hinman 16 March 2022, 14:54:59 UTC Add list of allowed domains for Watcher email action (#84894) (#85029) This adds the `xpack.notification.email.account.domain_allowlist` dynamic cluster setting that allows an administrator to specify a list of domains to which emails are allowed to be sent. The default value for this setting is `["*"]` which means all domains are allowed. It supports rudimentary globbing (`*`) in the domain name, so `*.company.com` will work as a valid option. Resolves https://github.com/elastic/elasticsearch/issues/84739 16 March 2022, 14:54:59 UTC
375fd80 Mary Gouseti 16 March 2022, 11:59:36 UTC Mark GeoIpDownloaderTask as completed after cancellation #84028 (#85014) Persistent task framework requires tasks to be marked either completed or failed after cancellation. This was missing in GeoIpDownloader. 16 March 2022, 11:59:36 UTC
6e3a6ea Rene Groeschke 16 March 2022, 11:34:06 UTC Avoid overhead of zip / unzip modules when packaging distribution (#84660) (#84971) This removes the overhead of zipping up modules that are immediately unzipped again when packaging the elasticsearch distribution. We also move some logic for packaging the elasticsearch distribution into a plugin and remove some outdated overhead dealing with 'meta plugins' when copying modules into a distribution. Another follow up and related optimization out of scope of this PR is, to also not zip unzip modules declared for usage in our test cluster setups. This partially addresses #76726. 16 March 2022, 11:34:06 UTC
2c54c8d Ievgen Degtiarenko 16 March 2022, 10:53:42 UTC Increment version on system index settings change (#84994) (#85017) The test was failing on assertion around index settings version change in case actual settings were changed. This change adds missing version increment and readable messages to the related assertions. (cherry picked from commit 351a410ebfdbaf97d15971ad0a438f17d1126573) 16 March 2022, 10:53:42 UTC
9859e1a Nik Everett 15 March 2022, 19:46:41 UTC Role descriptor link (#83753) (#85006) Add a reference to the role descriptor Co-authored-by: EdwynPe <78605050+EdwynPe@users.noreply.github.com> 15 March 2022, 19:46:41 UTC
4d8a17b Lukas Wegmann 15 March 2022, 18:03:54 UTC [8.1] SQL: forward warning headers to JDBC driver (#84499) (#84999) * SQL: forward warning headers to JDBC driver (#84499) Follow up on https://github.com/elastic/elasticsearch/pull/83943 to ensure that users of the JDBC drivers also can act on the warnings. The change ensures that warnings in the response headers from the `_sql` API are available in `JdbcResultSet.getWarnings()`. * fix version 15 March 2022, 18:03:54 UTC
42c8300 Nhat Nguyen 15 March 2022, 15:01:49 UTC Add BWC test for field-caps (#84455) (#84989) Relates #83494 15 March 2022, 15:01:49 UTC
107e68b Ioannis Kakavas 15 March 2022, 14:11:31 UTC Update start with security enabled docs (#84936) (#84983) In #84340 we adjusted the docs to show instructions on how to retrieve TLS keystores passwords from the elasticsearch keystore but the security-files-reference.asciidoc was not referenced in all places where the instructions should be used. This commit changes that and also removes references for beta versions. (cherry picked from commit 7c33a770adc91ec69261023c73a7848cddab8d52) 15 March 2022, 14:11:31 UTC
140834a Ioannis Kakavas 15 March 2022, 12:49:45 UTC Instructions to retrieve keystore pwd (#84340) (#84934) In 8.0.0. we introduce TLS autoconfiguration. We store the key and certificate materials in password protected PKCS#12 keystores and we store these passwords in the elasticsearch keystore. This commit adds instructions on how to get hold of the passwords for users to inspect or alter the PKCS#12 keystores. (cherry picked from commit 8cebba57086ee5300d7db92fb35901a33614d25c) 15 March 2022, 12:49:45 UTC
8190962 David Roberts 15 March 2022, 11:54:57 UTC [ML] Fix Kibana date format and similar overrides in text structure endpoint (#84967) (#84973) This PR fixes two problems. The first caused the second to be noticed: 1. The timestamp format pattern for Kibana's default date format was wrong - it was designed to match double digit day of month only, when Kibana uses a single digit for 1-9. 2. Trying to use a timestamp format override to work around the first problem didn't work. This exposed a second, more general problem. When a timestamp format override is supplied we try to work out if a built in timestamp format would always include it, and use that in preference. (This allows friendly format names like "iso8601" to replace complex patterns.) However, the way this ability to substitute was being determined was flawed, leading to incorrect substitutions. In the case of the Kibana timestamp format it resulted in the correct format supplied as an override being replaced with the buggy built in timestamp format. To fix this, substitutions are now only performed if the correct number of digits are present as well as the Grok pattern matching. 15 March 2022, 11:54:57 UTC
dc3d3b5 Mark Vieira 14 March 2022, 22:43:47 UTC [8.1] Remove usages of elasticsearch.build plugin in non-production projects (#84961) 14 March 2022, 22:43:47 UTC
7486462 Andrei Dan 14 March 2022, 15:02:48 UTC GeoIP update logging (#84495) (#84951) Update logging in `DatabaseNodeService` to reflect we are indeed NOT downloading the geo IP database (in the "fetch it from the configured endpoint sense"), but rather using the `.geoip_databases` index. 14 March 2022, 15:02:48 UTC
56326ef Albert Zaharovits 14 March 2022, 14:29:45 UTC Require and preserve content type for filtered rest requests (#84950) RestRequest#getXContentType cannot normally be null for a request with a non-empty request body, or the rest controll will reject the request, see eg RestControllerTests#testUnknownContentWithContentStream. This PR fixes sloppy behavior where the RestRequestFilter cleared the xContentType field of the filtered request. Fixes #84784 14 March 2022, 14:29:45 UTC
ce0edd9 Artem Prigoda 14 March 2022, 13:19:35 UTC [s3-repository] Lookup AWS Region for STS Client from STS endpoint (#84585) (#84946) If we don't instruct to look up the region from the Endpoint URL, AWSSecurityTokenServiceClient tries to look it up implicitly in a custom way which requires reading the /.aws/config file for which we don't have a file permission. The same approach is used for the general AmazonS3ClientBuilder Resolves #83826, #52625 14 March 2022, 13:19:35 UTC
1f6d32e Rory Hunter 14 March 2022, 10:15:34 UTC Lower severity of IDE Checkstyle rules for JavaDoc (#84877) Lower the severity from "warning" to "info", and raise the length of methods that we ignore from 2 lines to 5. 14 March 2022, 10:15:56 UTC
cce5e6d Artem Prigoda 14 March 2022, 09:40:04 UTC [8.1] Correctly return _type field for documents in V7 compatiblity mode (#84873) (#84937) Backports #84873 to 8.1 * Currently, it's not returned for bulk indexing, because BulkItemResponse calls DocWriteResponse.innerToXContent instead of DocWriteResponse.toXContent. 14 March 2022, 09:40:04 UTC
55e1767 edh-oss 14 March 2022, 08:33:27 UTC Stop checking non-asciidoc files for doc snippets (#84390) When SnippetsTask looks for doc snippets, the list of files it checks includes roughly 350 files that aren't asciidoc files. Image files (both png and jpg), yaml files, and so on. Set an explicit include pattern so that Gradle skips these files instead of trying to read them. 14 March 2022, 08:34:22 UTC
90025dd Nhat Nguyen 13 March 2022, 16:59:29 UTC Fix point visitor in DiskUsage API (#84909) (#84916) * Fix point visitor in DiskUsage API (#84909) We should use numIndexDimensions instead of numDimensions when comparing packedValues of PointValues. Relates #84816 * Fix tests 13 March 2022, 16:59:29 UTC
83349fd William Brafford 10 March 2022, 22:51:42 UTC Bump timeout for feature migration in system index migration test (#84828) (#84891) * Bump timeout on assertBusy loop 10 March 2022, 22:51:42 UTC
c24f585 Mark Vieira 10 March 2022, 18:25:23 UTC Update Gradle wrapper to 7.4.1 (#84841) (#84883) 10 March 2022, 18:25:23 UTC
b9f68a4 Mark Vieira 10 March 2022, 17:42:29 UTC Publish transport-netty4 module since it's needed for the test framework (#84848) (#84881) 10 March 2022, 17:42:29 UTC
59b62ac Nikola Grcevski 10 March 2022, 16:41:53 UTC Revert test workaround for #82837 (#84843) (#84875) 10 March 2022, 16:41:53 UTC
a242f9d James Rodewig 10 March 2022, 15:45:14 UTC [DOCS] Fix a typo: "bult-in" ~> "built-in" (#84861) (#84871) (cherry picked from commit 94f7d739d814285e11fc077f03e40a34452e41e9) # Conflicts: # docs/reference/ilm/example-index-lifecycle-policy.asciidoc Co-authored-by: Alex Chan <alex@alexwlchan.net> 10 March 2022, 15:45:14 UTC
cff91e2 Ioannis Kakavas 10 March 2022, 15:26:47 UTC Fix check-running docs (#84856) (#84867) The way this specific doc was structured meant that the path for DEB/RPM packages were wrong (an extra `config` was added to the dir path). This commit changes the structure to be relevant to the ES_CONF_PATH env variable that should work uniformly for all of our installation means. 10 March 2022, 15:26:47 UTC
e9cb448 Tim Vernum 10 March 2022, 02:24:51 UTC [TEST] Ensure copied file has updated timestamp (#84749) (#84853) This test relies on the resource watcher service detecting a change in the configured CA path. However the old and the new file both have the same size (1200 bytes) so the change is only detected if the file's modification time changes. This commit forces that modification time to be different after the file is replaced, so that the resource watcher will trigger a reload. Resolves: #83560 10 March 2022, 02:24:51 UTC
01ad19c Yang Wang 10 March 2022, 01:12:12 UTC Add known issue for audit logging with fitered fields (#84786) (#84851) Users can encounter 401 HTTP error with NPE as the underlying cause when using audit logging with `emit_request_body` and requests with filtered fields. This PR adds this issue to known issues of the 8.1 release notes. Relates: #84784 Co-authored-by: Tim Vernum <tim@adjective.org> Co-authored-by: James Rodewig <james.rodewig@elastic.co> Co-authored-by: Tim Vernum <tim@adjective.org> Co-authored-by: James Rodewig <james.rodewig@elastic.co> 10 March 2022, 01:12:12 UTC
cd7da80 Artem Prigoda 09 March 2022, 19:49:49 UTC [s3-repository] Don't fail if there no symlink for AWS Web Identity Token (#84697) (#84825) Make sure users can use the static credentials even if there is a service account with IAM roles configured on the system. See https://github.com/elastic/elasticsearch/issues/52625#issuecomment-1060154207 09 March 2022, 19:49:49 UTC
861bd0e William Brafford 09 March 2022, 19:25:49 UTC Fix NullPointerException in SystemIndexMetadataUpgradeService hidden alias handling (#84780) (#84833) * Fix an NPE in hidden alias logic * Update docs/changelog/84780.yaml * Simplify conditional 09 March 2022, 19:25:49 UTC
d8d7880 James Rodewig 09 March 2022, 16:29:17 UTC [DOCS] Fix intro sentence for Docker install instructions (#83939) (#84823) Fixes an intro sentence for the Docker install instructions. Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com> (cherry picked from commit 472a7d8e914702e4d77e6c7c5b99d8148c9b602c) Co-authored-by: Alexander Reelsen <alexander@reelsen.net> 09 March 2022, 16:29:17 UTC
2b56626 Andrei Stefan 09 March 2022, 15:34:13 UTC EQL: Clean any used memory by the sequence matcher and circuit breaker used bytes in case of exception (#84451) (#84803) (cherry picked from commit 4c243d69500bb983230434b828bfd798b68a202b) 09 March 2022, 15:34:13 UTC
9d527c5 Alan Woodward 09 March 2022, 13:32:47 UTC Don't run the runtime field YAML tests over TSDB aggs (#84791) Runtime fields don't support dimension parameters so we can't shadow keyword fields in the normal way for TSDB yaml tests. 09 March 2022, 14:43:37 UTC
d6d89c2 James Rodewig 09 March 2022, 13:40:46 UTC add data tier configuration information for Cloud (#84518) (#84806) * add data tier configuration information for Cloud * Move configuration docs and add headings * update ILM tutorial image * add save changes step Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com> (cherry picked from commit 52a1653e3f27fa8c594f972e25a26627c63272b5) Co-authored-by: Dan Roscigno <dan@roscigno.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> 09 March 2022, 13:40:46 UTC
efe4cd0 James Rodewig 09 March 2022, 13:28:55 UTC [8.1] [DOCS] Fix documentation snippet tests (#84808) Co-authored-by: Mark Vieira <portugee@gmail.com> 09 March 2022, 13:28:55 UTC
1b8a3e8 James Rodewig 09 March 2022, 13:11:57 UTC Remove 8.1 coming tag 09 March 2022, 13:11:57 UTC
bf2e611 David Turner 09 March 2022, 12:28:17 UTC Note that S3 compat includes performance (#84798) Today the note in the docs about S3-compatible repositories notes that the repo must behave correctly, but it's also important that it has the same performance profile. This commit extends the docs to include this info. 09 March 2022, 12:34:04 UTC
6242118 Alan Woodward 09 March 2022, 09:31:08 UTC Add tests for min_hash configuration and fix settings names (#84753) Fixes #84578 09 March 2022, 09:43:54 UTC
987a83b ievgen.degtiarenko 08 March 2022, 16:18:24 UTC Bump versions for 8.1.0 release 08 March 2022, 16:45:00 UTC
c59fabc David Kyle 08 March 2022, 16:36:01 UTC [ML] Return all Datafeeds with GET Anomaly Detector (#84759) (#84770) Fixes a bug where only the first 10 datafeeds were included Backport of #84759 08 March 2022, 16:36:01 UTC
79c1cfe Bogdan Pintea 08 March 2022, 14:57:37 UTC Pin specific pip py modules used by connector SDK (#84757) (#84761) This pins certain Python modules used by Tableau's connector SDK to certain versions. This will ensure build success consistency. 08 March 2022, 14:57:37 UTC
813fca0 James Rodewig 08 March 2022, 14:24:17 UTC [DOCS] Remove 8.1.0 coming tag (#84169) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> 08 March 2022, 14:24:17 UTC
a21825a Lee Hinman 08 March 2022, 03:56:17 UTC Remove existing indices/datastreams/aliases before simulating index template (#84675) (#84746) 08 March 2022, 03:56:17 UTC
96e989b Nikola Grcevski 07 March 2022, 20:40:01 UTC [TEST] Fix SystemIndexMigrationIT (#84723) (#84729) Ensure the task has a state, and it's truly started before restarting the master node. 07 March 2022, 20:40:01 UTC
2251250 Mark Vieira 07 March 2022, 20:14:32 UTC Remove Eclipse IDE contributor documentation (#84724) (#84728) 07 March 2022, 20:14:32 UTC
dd99dbe Lisa Cawley 07 March 2022, 19:39:22 UTC [DOCS] Add ML PRs to 8.1 release notes (#84720) (#84725) 07 March 2022, 19:39:22 UTC
e6e8d28 Nik Everett 07 March 2022, 19:30:48 UTC fixed typo (#84694) (#84727) Co-authored-by: Mustafa Balila (rootsofnull) <hitsugayatoshiro899@gmail.com> 07 March 2022, 19:30:48 UTC
d8cf8bd Nik Everett 07 March 2022, 17:42:19 UTC Fix some typos in plugins & reference docs (#84667) (#84718) This pull request removes a few instances of duplicate words or punctuation and erroneous spelling from the docs. Co-authored-by: Abele Mălan <6689720+AbeleMM@users.noreply.github.com> 07 March 2022, 17:42:19 UTC
4582ead Nik Everett 07 March 2022, 17:36:27 UTC [DOCS] Update install instructions for Debian/Ubuntu (#84645) (#84715) The use of `apt-key` is deprecated and will no longer be available after Debian 11 and Ubuntu 22.04. This updates the installation instructions for Debian-based distributions. Closes #84644 Co-authored-by: er0k <er0k@users.noreply.github.com> 07 March 2022, 17:36:27 UTC
ad9dac1 Nik Everett 07 March 2022, 17:28:28 UTC Update painless-reindex-context.asciidoc (#84444) (#84713) ctx['op'] should be set to 'noop', not 'none' when specifying no operation. Elasticsearch error when using 'none': ```json { "error" : { "root_cause" : [ { "type" : "illegal_argument_exception", "reason" : "Operation type [none] not allowed, only [noop, index, delete] are allowed" } ], "type" : "illegal_argument_exception", "reason" : "Operation type [none] not allowed, only [noop, index, delete] are allowed" }, "status" : 400 } ``` Co-authored-by: jalvar08 <jeovanny.alvarez@gmail.com> 07 March 2022, 17:28:28 UTC
aaaca1b Rory Hunter 07 March 2022, 10:48:39 UTC Remove use of Cloudflare zlib (#84680) (#84695) Relates to #81662. This library isn't ready for public consumption. Remove it from the Docker build. 07 March 2022, 10:48:39 UTC
ee951c4 Rory Hunter 07 March 2022, 10:35:28 UTC Configure required IDEA plugins (#84674) Closes #53812. The CheckStyle-IDEA and EcliseCodeFormatter IDEA plugins are to all intents and purposes required when using IntelliJ to work on Elasticsearch, so add configuration that marks then as required. 07 March 2022, 10:36:05 UTC
7d10f59 Ievgen Degtiarenko 07 March 2022, 07:47:21 UTC [DOCS] Update v8.1.0 release notes (#84651) 07 March 2022, 07:47:21 UTC
back to top